chore(release): v0.16.0

Calibrated on the 117-frame TC260-producer cohort (vendor_cohort_harvest +
vendor_mark_calibrate, both committed here): per-mark 2-rung ladder
(0.78, 1.27) for the two measured size modes, fitted locate box (the mark
sits ~0.025 of the short side off the edge; doubao's box clipped the first
glyph), measured template aspect 0.26, gate 0.45 (clean p99 0.301).
Strict-only (the sub-gate band is non-Qwen banners), no rival margin
(0 cross-fires on 400 doubao / 298 jimeng / 286 clean frames).
83/83 real marks detector-clean after cv2 fill.

TextMarkConfig gains a per-mark ladder field; the shipped 3-rung default
is unchanged for every other mark.

.env.example

@@ -1,3 +1,3 @@
-# HuggingFace token (required for invisible watermark removal)
+# HuggingFace token (optional; only needed for gated/private models)
 # Get yours at: https://huggingface.co/settings/tokens
 HF_TOKEN=

.github/workflows/distribute.yml

@@ -0,0 +1,106 @@
+name: Distribute on release
+
+# Fans a published GitHub Release out to the channels that need a nudge.
+# PyPI is handled by publish.yml; conda-forge is handled by its autotick bot.
+# This workflow event-drives the two channels that would otherwise be manual:
+#   - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
+#   - HF Space: factory-rebuild so it reinstalls the latest sdist from PyPI.
+# Both wait for the freshly published sdist to appear on PyPI first, since the
+# Release event fires in parallel with publish.yml's upload.
+
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+    inputs:
+      version:
+        description: "Version to distribute (no leading v), e.g. 0.10.3. Blank = latest on PyPI."
+        required: false
+
+jobs:
+  resolve:
+    runs-on: ubuntu-latest
+    outputs:
+      version: ${{ steps.v.outputs.version }}
+      url: ${{ steps.sdist.outputs.url }}
+      sha: ${{ steps.sdist.outputs.sha }}
+    steps:
+      - name: Resolve target version
+        id: v
+        run: |
+          set -euo pipefail
+          if [ "${{ github.event_name }}" = "release" ]; then
+            v="${GITHUB_REF_NAME#v}"
+          elif [ -n "${{ github.event.inputs.version }}" ]; then
+            v="${{ github.event.inputs.version }}"
+          else
+            v=$(curl -fsSL https://pypi.org/pypi/remove-ai-watermarks/json \
+              | python3 -c "import sys,json;print(json.load(sys.stdin)['info']['version'])")
+          fi
+          echo "version=$v" >> "$GITHUB_OUTPUT"
+          echo "Target version: $v"
+
+      - name: Wait for the sdist on PyPI, capture url + sha256
+        id: sdist
+        run: |
+          set -euo pipefail
+          v="${{ steps.v.outputs.version }}"
+          for i in $(seq 1 30); do
+            if rel=$(curl -fsSL "https://pypi.org/pypi/remove-ai-watermarks/$v/json" 2>/dev/null); then
+              url=$(echo "$rel" | python3 -c "import sys,json;d=json.load(sys.stdin);print(next((u['url'] for u in d['urls'] if u['packagetype']=='sdist'),''))")
+              sha=$(echo "$rel" | python3 -c "import sys,json;d=json.load(sys.stdin);print(next((u['digests']['sha256'] for u in d['urls'] if u['packagetype']=='sdist'),''))")
+              if [ -n "$url" ] && [ -n "$sha" ]; then
+                echo "url=$url" >> "$GITHUB_OUTPUT"
+                echo "sha=$sha" >> "$GITHUB_OUTPUT"
+                echo "Found sdist for $v"
+                exit 0
+              fi
+            fi
+            echo "sdist for $v not on PyPI yet (attempt $i), waiting..."
+            sleep 20
+          done
+          echo "::error::sdist for $v did not appear on PyPI in time"
+          exit 1
+
+  homebrew:
+    needs: resolve
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout the tap
+        uses: actions/checkout@v7
+        with:
+          repository: wiltodelta/homebrew-tap
+          token: ${{ secrets.HOMEBREW_TAP_TOKEN }}
+      - name: Bump the formula
+        env:
+          URL: ${{ needs.resolve.outputs.url }}
+          SHA: ${{ needs.resolve.outputs.sha }}
+          VERSION: ${{ needs.resolve.outputs.version }}
+        run: |
+          set -euo pipefail
+          F=Formula/remove-ai-watermarks.rb
+          sed -i -E "s|url \"[^\"]*\"|url \"$URL\"|" "$F"
+          sed -i -E "s|sha256 \"[^\"]*\"|sha256 \"$SHA\"|" "$F"
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          git add "$F"
+          git commit -m "remove-ai-watermarks $VERSION" || { echo "Formula already current"; exit 0; }
+          git push
+
+  hf-space:
+    needs: resolve
+    runs-on: ubuntu-latest
+    steps:
+      - name: Factory-rebuild the HuggingFace Space
+        env:
+          HF_TOKEN: ${{ secrets.HF_TOKEN }}
+        run: |
+          python3 -m pip install --quiet huggingface_hub
+          python3 - <<'PY'
+          import os
+          from huggingface_hub import HfApi
+          HfApi(token=os.environ["HF_TOKEN"]).restart_space(
+              "wiltodelta/remove-ai-watermarks", factory_reboot=True
+          )
+          print("HF Space factory rebuild triggered")
+          PY

.github/workflows/publish.yml

@@ -13,12 +13,29 @@ jobs:
     runs-on: ubuntu-latest
     environment: pypi
     steps:
-      - uses: actions/checkout@v6
+      - uses: actions/checkout@v7
 
       - uses: astral-sh/setup-uv@v7
 
+      - name: Verify release tag matches package version
+        run: |
+          tag="${{ github.event.release.tag_name }}"
+          version="$(grep -m1 '^version = ' pyproject.toml | sed -E 's/^version = "([^"]+)"/\1/')"
+          if [ "${tag#v}" != "$version" ]; then
+            echo "Release tag '$tag' does not match pyproject.toml version '$version'" >&2
+            exit 1
+          fi
+          echo "Release tag '$tag' matches package version '$version'"
+
       - name: Build package
         run: uv build
 
+      # Publish with `uv publish` (its own uploader, not the twine bundled in the
+      # old pypa action). Trusted publishing is automatic in GitHub Actions: the
+      # `id-token: write` permission + the `pypi` environment supply the OIDC token,
+      # and PyPI's trusted-publisher entry matches on repo + workflow filename +
+      # environment name (all unchanged from the pypa-action setup), so no API token
+      # is needed. uv's uploader also accepts Metadata-Version 2.5 -- the permanent
+      # fix that lets the hatchling pin be relaxed (see pyproject [build-system]).
       - name: Publish to PyPI
-        uses: pypa/gh-action-pypi-publish@release/v1
+        run: uv publish

.github/workflows/test.yml

@@ -13,7 +13,7 @@ jobs:
   lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v6
+      - uses: actions/checkout@v7
       - uses: astral-sh/setup-uv@v7
       - name: Sync dev environment
         run: uv sync --frozen --extra dev
@@ -36,7 +36,7 @@ jobs:
         os: [ubuntu-latest, macos-latest, windows-latest]
         python-version: ["3.10", "3.12"]
     steps:
-      - uses: actions/checkout@v6
+      - uses: actions/checkout@v7
       - uses: astral-sh/setup-uv@v7
         with:
           python-version: ${{ matrix.python-version }}

.gitignore

@@ -18,9 +18,6 @@ Thumbs.db
 *.swp
 *.swo
 
-# Test results
-data/results/
-
 # SynthID corpus reference fills (synthetic black/white calibration tiles,
 # regenerable; the labeled pos/neg/cleaned images ARE tracked, see README)
 data/synthid_corpus/refs/
@@ -36,9 +33,26 @@ yolov8n.pt
 .claude/settings.local.json
 .claude/scheduled_tasks.lock
 
-# Doubao watermark calibration (local only; ship only the derived alpha-map asset).
-# Synthetic seeds + raw Doubao captures are regenerable and not committed.
-# Non-ours reference artifacts go in any _refs/ dir (already ignored above): usable
-# locally for bootstrap/validation, never redistributed in the repo.
+# Visible-watermark alpha calibration. The solid black/gray/white CAPTURES are
+# committed (content-free: a solid colour + the watermark; the source for
+# scripts/visible_alpha_solve.py so the alpha assets are reproducible). The
+# synthetic seeds (regenerable) and any real-content validation download (a real
+# generated scene, kept local for privacy) are NOT committed.
 data/doubao_capture/seeds/
-data/doubao_capture/captures/
+data/jimeng_capture/seeds/
+data/jimeng_capture/captures/jimeng_content_*.png
+data/gemini_capture/seeds/
+data/gemini_capture/captures/gemini_content_*.png
+data/samsung_capture/seeds/
+data/samsung_capture/captures/samsung_content_*
+
+# Leftover GFPGAN weights dir from the retired face-restore experiments
+# (GFPGAN wrote RetinaFace/parsing weights to a CWD ./gfpgan/weights/ working
+# dir on first use). Runtime artifact, never committed.
+gfpgan/
+
+# Qwen ControlNet experiment outputs (throwaway eval; never the committed corpus)
+scripts/_qwen_exp_out/
+
+# Local-only working data for analysis (not a committed corpus; never tracked)
+data/spaces/

.python-version

@@ -0,0 +1 @@
+3.12

CLAUDE.md

@@ -2,82 +2,95 @@
 
 You are a **principal Python engineer** maintaining a CLI tool and library for removing visible and invisible AI watermarks from images.
 
+## Scope and non-goals
+
+The mission is removing **AI-provenance watermarks** that a platform stamps onto content the user generated themselves — SynthID, the Gemini / Nano Banana sparkle, the Doubao / Jimeng / Qwen / Samsung visible AI labels, the Chinese TC260 "由…AI生成" label, and C2PA / IPTC / EXIF "Made with AI" metadata. The point is user autonomy over their own generated output.
+
+It deliberately does **not** remove watermarks that protect someone else's paid or copyrighted content — stock-agency overlays (Shutterstock, Getty, iStock, Adobe Stock), classifieds-site marks, or any tiled / diagonal "preview" watermark whose job is to gate a purchase. Stripping those makes a paid resource free off someone else's work; out of scope **by principle, not by technical difficulty**. The line: a visible mark is in scope when it labels the user's **own** AI generation, and out of scope when it protects a **third party's paid asset**.
+
+Consequences for contributors (do not drift back into the stock niche just because it is technically feasible):
+- Do not add stock / agency / classifieds watermark removal to `watermark_registry.py` or the eraser, and do not build tiled-overlay or multi-image watermark-estimation features aimed at them.
+- `erase --region` stays a generic **user-driven** tool (the user points at their own object); do not ship an *automatic* stock-watermark detector/remover on top of it.
+- New visible-mark templates are for **AI-generation labels only**.
+
+(Established 2026-06-13 by user instruction: "Я пытаюсь сделать платные ресурсы бесплатными — это не то, против чего мы боремся.")
+
 ## How to run
 
-- `uv run remove-ai-watermarks all <image.png> -o <output.png>`
-- `uv run remove-ai-watermarks visible <image.png> -o <out.png>` — known-visible-mark removal, CPU, no GPU. **Reverse-alpha only**: every mark is removed by inverting its captured alpha map (exact pixel recovery, no inpaint). `--mark auto` (default) picks the strongest detected of the Gemini sparkle and the Doubao "豆包AI生成" text strip; `--mark gemini` / `--mark doubao` force one. For arbitrary logos/objects use `erase`.
-- `uv run remove-ai-watermarks erase <image.png> --region x,y,w,h -o <out.png>` — universal region eraser (any logo/object, any position). `--backend cv2` (default, no deps) or `--backend lama` (big-LaMa via onnxruntime, extra `lama`); `--region` is repeatable.
+Per-command exit-code semantics (the no-signal / GPU-missing skip branches), test traps, and regression-guard paths live in `docs/module-internals.md` (section "CLI commands (`cli.py`)") — read it before changing any command's skip/exit behavior. Every single-image command's `source` argument declares `dir_okay=False`: without it `click.Path(exists=True)` accepts a directory, which then reached `open()` and raised `IsADirectoryError` (Tier E, 2026-07-20; `batch`'s `directory` already declared `file_okay=False`). Regression: `tests/test_cli_robustness.py::TestDirectoryInputIsRejected`.
+
+- `uv run remove-ai-watermarks all <image.png> -o <output.png>` — full pipeline (visible + invisible + metadata). Same diffusion knobs as `invisible`, plus the visible-pass `--backend auto|cv2|migan|lama` (default `auto`) and `--sensitivity auto|strict` (default `auto`) for the localize -> fill visible removal (see the `visible` bullet). Skips step 2 (invisible/SynthID) when the `[gpu]` extra is absent or no invisible signal is detectable; see the module doc for the distinct exit codes.
+- `uv run remove-ai-watermarks invisible <image.png> -o <out.png>` — diffusion SynthID removal. **Full knob set** (kept identical across `invisible`/`all`/`batch`): `--strength` (vendor-adaptive default), `--steps` (**interacts with `--strength`**: diffusers derives its timesteps as `int(steps * strength)`, so a low `--steps` used to crash inside torch with `cannot reshape tensor of 0 elements` -- at the default 0.15 that was every value below 7. `noai/watermark_profiles.viable_steps` now raises the count to the minimum that denoises and logs the adjustment; keep the guard where it is, above `_generate_one`, so all three pipelines and the tiled path inherit it), `--guidance-scale` (CFG, default 7.5), `--pipeline sdxl|controlnet|qwen` (default `controlnet`; `qwen` is a manual opt-in only — see the qwen note in the module map), `--controlnet-scale`, `--model` (HF model id, default SDXL base), `--device`, `--seed`, `--hf-token`, `--max-resolution`/`--min-resolution`, `--upscaler lanczos|esrgan`, `--humanize` (Analog Humanizer grain), `--unsharp` (final sharpen), `--adaptive-polish/--no-adaptive-polish` (**ON by default**), `--tile/--no-tile` + `--tile-size`/`--tile-overlap` (**OFF by default**), `--force/--no-force` (default skip = ON, runs the scrub even with no detected signal). `--auto` is deprecated and a no-op that only warns. Skips the diffusion when no invisible signal is detectable (the no-signal gate); see the module doc.
+- `uv run remove-ai-watermarks visible <image.png> -o <out.png>` — known-visible-mark removal by **localize -> fill**: each detected mark is localized to a binary full-frame footprint mask, then one shared, swappable fill inpaints that mask. `--backend auto|cv2|migan|lama` (default `auto`) picks the fill: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. `--mark auto` (default) removes EVERY detected mark in one pass (a Jimeng-basic image carries the top-left "AI生成" pill AND the bottom-right "★ 即梦AI" wordmark) from: Gemini sparkle, Doubao "豆包AI生成", Jimeng "★ 即梦AI", Qwen "千问AI生成", Samsung Galaxy AI "✦ Contenuti generati dall'AI", and the capture-less Jimeng "AI生成" pill (top-left, metadata-gated); `--mark gemini|doubao|jimeng|qwen|samsung|jimeng_pill` forces one. `--sensitivity auto|strict` (default `auto`) sets how hard a borderline mark is trusted: `auto` relaxes a mark's gate only on same-product evidence (metadata provenance for that vendor, or a confidently detected sibling mark of the same product — clean images stay untouched); `strict` never relaxes. Metadata provenance is read automatically and feeds `auto`. (`assume-ai` was REMOVED in 0.16 — see the registry bullet; a user who can SEE a missed mark should point at it with `erase --region`, or name it with `--mark <text-mark> --no-detect`.) For arbitrary logos/objects use `erase`. When no known mark is detected the command writes no output and exits with the no-visible-mark code instead of re-serving the input; `--no-detect` forces the gemini fallback and proceeds. See the module doc for the routing/exit detail. `--backend` and `--sensitivity` are shared across `visible`/`all`/`batch`.
+- `uv run remove-ai-watermarks erase <image.png> --region x,y,w,h -o <out.png>` — universal region eraser (any logo/object, any position). `--backend cv2` (default, no deps), `--backend migan` (MI-GAN via onnxruntime, extra `migan`; ~28 MB, ~1 GB RAM, near-LaMa), or `--backend lama` (big-LaMa, extra `lama`; best quality but ~4.7 GB RAM); `--region` is repeatable.
 - `uv run remove-ai-watermarks identify <image>` — provenance verdict (platform + watermark inventory + confidence); `--json` for machine output, `--no-visible` to skip the cv2 sparkle detector
 - `uv run remove-ai-watermarks metadata <image.png> --check` — inspect AI metadata (C2PA, EXIF, PNG chunks)
 - `uv run remove-ai-watermarks metadata <image.png> --remove -o <out.png>` — strip all AI metadata
+- `uv run remove-ai-watermarks batch <directory>` — process every supported image in a directory (output defaults to `<directory>_clean/`, set with `-o`). `--mode visible|invisible|metadata|all` (default `visible`); the invisible/all path reuses the full `invisible` knob set above, plus `--backend` and `--sensitivity` for the visible localize -> fill pass. Applies the same no-signal skip per image; see the module doc. **Exit code:** non-zero when any image errored OR (mirroring single `all`) a `--mode invisible`/`all` image carried an invisible signal but the GPU extra was absent, so its SynthID scrub was skipped — it emits a loud warning and copies the input through (invisible mode) so the output dir stays complete; a wrapping service can then detect the incomplete run instead of trusting a silent exit 0.
 
 ## Test and lint
 
-- **CI** (`.github/workflows/test.yml`): runs on push to `main` + every PR. A `lint` job (ubuntu: `ruff check` + `ruff format --check`) plus a `test` matrix (ubuntu/macos/windows x py3.10/3.12) that does `uv sync --frozen --extra dev` then `pytest`. The matrix installs only core + dev (no `gpu` extra), so the GPU/model-running tests skip there and it exercises the metadata/identify/visible/cv2-eraser surface on all three OSes. Keep `uv.lock` valid (don't break `--frozen`) when editing `pyproject.toml`. `publish.yml` stays release-only.
-- `bash maintain.sh` — uv-outdated, uv-secure, ruff check/fix, ruff format, pyright, pytest -n auto
-- **Strict pyright is clean across `src/` (0 errors).** The cv2/torch/diffusers boundary files (`gemini_engine`, `region_eraser`, `doubao_engine`, `face_protector`, `humanizer`, `invisible_engine`, `noai/watermark_remover`, and the whole `noai/ctrlregen/` subpackage) carry a documented per-file `# pyright:` relax pragma (or, for `ctrlregen`, a `tool.pyright.executionEnvironments` entry) that turns off only the unknown-type / untyped-third-party rules — those libs ship no usable types, so strict typing there fights the ecosystem. Pure-logic files stay fully strict; `typings/piexif/__init__.pyi` is a local stub so `metadata.py`/`extractor.py` resolve piexif. Public ndarray-returning signatures on the relaxed engines are still annotated `NDArray[Any]` so strict consumers (`cli.py`) stay clean. When touching a relaxed file, prefer fixing real issues over widening the pragma; keep the pragma scoped to genuinely-untyped boundaries. (`uv-secure` is clean since idna was bumped 3.11 -> 3.16, fixing GHSA-65pc-fj4g-8rjx.)
+- **CI** (`.github/workflows/test.yml`): runs on push to `main` + every PR. A `lint` job (ubuntu: `ruff check` + `ruff format --check`) plus a `test` matrix (ubuntu/macos/windows x py3.10/3.12) that does `uv sync --frozen --extra dev` then `pytest`. The matrix installs only core + dev (no `gpu` extra), so the GPU/model-running tests skip there and it exercises the metadata/identify/visible/cv2-eraser surface on all three OSes. Keep `uv.lock` valid (don't break `--frozen`) when editing `pyproject.toml`.
+- **Release flow + distribution channels** (PyPI publish via `publish.yml`/`uv publish`, the automated Homebrew-tap + HF-Space bumps in `distribute.yml`, conda-forge, ComfyUI Registry, the sdist `data/` exclusion, hatchling pin history): see `docs/release-and-distribution.md` before cutting a release.
+- `bash maintain.sh` — uv-outdated, uv-secure, ruff check/fix, ruff format, pyright (scoped `src/`, see the OOM note below), pytest -n auto. The helper tools live in the `dev` extra (`pytest-xdist`, plus `uv-outdated`/`uv-secure` marker-gated to py3.12+ so the py3.10 resolution stays solvable) — a bare env without `--extra dev` does not have them.
+- **Strict pyright is clean across `src/` (0 errors).** The cv2/torch/diffusers boundary files (`gemini_engine`, `region_eraser`, `doubao_engine`, `humanizer`, `invisible_engine`, `noai/watermark_remover`) carry a documented per-file `# pyright:` relax pragma that turns off only the unknown-type / untyped-third-party rules — those libs ship no usable types, so strict typing there fights the ecosystem. Pure-logic files stay fully strict; `typings/piexif/__init__.pyi` is a local stub so `metadata.py`/`extractor.py` resolve piexif. Public ndarray-returning signatures on the relaxed engines are still annotated `NDArray[Any]` so strict consumers (`cli.py`) stay clean. When touching a relaxed file, prefer fixing real issues over widening the pragma; keep the pragma scoped to genuinely-untyped boundaries. The `uv-secure` CVE-resolution history (idna/aiohttp bumps, retired basicsr, the dismissed torch `GHSA-rrmf-rvhw-rf47`) lives in `docs/release-and-distribution.md` — read it before re-triaging a dependency alert.
 - **Full-project `uv run pyright` (no path) OOMs/crashes node on this ML-heavy repo** (emits a `libnode` stack frame, no summary) — a known environment limit, not a code error. Gate with `uv run --extra dev --extra gpu pyright src/` (completes, authoritative) or scope to changed files; also run `uv run ruff check` and `uv run pytest` directly.
 - Run `uv run` from the repo root — from another cwd it falls back to a bare env without numpy/cv2/torch.
-- To add a dev tool (pytest/ruff/pyright) into the env, use `uv sync --frozen --extra dev --extra gpu`, **never `uv pip install`** — `uv pip install` re-resolves and rewrites `uv.lock`, which silently bumped `transformers` to a build incompatible with the pinned `diffusers` (`cannot import name 'Qwen3VLForConditionalGeneration'`) and broke every `identify`/metadata import. Recovery: `git checkout uv.lock && uv sync --frozen --extra gpu --extra dev`. The `gpu` extra holds `diffusers`/`transformers`/`torch`, so a bare `uv sync` (no extras) removes them and `noai/__init__` (eager pipeline import) then fails. `maintain.sh`'s `uv sync --all-extras` also pulls the heavy `trustmark`/`lama` wheels (pytorch-lightning, onnxruntime) — fine on a good connection, but on flaky DNS sync only `--extra gpu --extra dev` and run the lint/test steps by hand.
-- Metadata/C2PA tests assert against real committed fixtures in `data/samples/` (`chatgpt-*.png` = OpenAI C2PA, `firefly-1.png` = Adobe, `mj-*` = Midjourney IPTC, `doubao-1.png` = ByteDance Doubao with the China TC260 `<TC260:AIGC>` XMP label **and** a visible "豆包AI生成" text mark bottom-right; `grok-1.jpg` = xAI Grok with its EXIF-only `Signature:` blob + UUID `Artist` and no C2PA/SynthID/IPTC); synthetic byte blobs cover the JPEG/ISOBMFF format paths. The "non-AI / clean photo" control is no longer in `data/samples/` -- the `clean_photo` conftest fixture serves a verified-negative image from the corpus `neg/` set (skips if the corpus is absent).
-- SynthID reference corpus: `scripts/synthid_corpus.py` ingests labeled images into `data/synthid_corpus/`. The labeled `images/` (`pos/` `neg/` `cleaned/`) are **committed** (public repo -- review every image for private content before adding; `manifest.csv` is kept in sync with the files on disk, one row per tracked image); only the synthetic `refs/` calibration fills are gitignored. See its README for the collection protocol and verification oracles.
+- **Stale `trustmark` remnant in site-packages after an extras change:** the `trustmark` package downloads model weights INTO its own package dir, so when a narrower `uv sync` prunes the package, a `trustmark/models/` directory survives as an empty namespace package. Symptom: pyright `"TrustMark" is unknown import symbol` on `trustmark_detector.py` and `find_spec("trustmark")` returning a loader-less spec (so `is_available()` lies True). Fix: `rm -rf .venv/lib/python3.12/site-packages/trustmark` (regenerable weights cache).
+- To add a dev tool (pytest/ruff/pyright) into the env, use `uv sync --frozen --extra dev --extra gpu`, **never `uv pip install`** — `uv pip install` re-resolves and rewrites `uv.lock`, which silently bumped `transformers` to a build incompatible with the pinned `diffusers` (`cannot import name 'Qwen3VLForConditionalGeneration'`) and broke every `identify`/metadata import. Recovery: `git checkout uv.lock && uv sync --frozen --extra gpu --extra dev`. The `gpu` extra holds `diffusers`/`transformers`/`torch`, so a bare `uv sync` (no extras) removes them; `noai/__init__` is now **lazy** (PEP 562 `__getattr__`, so importing `identify`/`metadata` no longer pulls `watermark_remover`/torch), so a bare env breaks only when the removal pipeline is actually invoked, not on import. `maintain.sh`'s `uv sync --all-extras` also pulls the heavy `trustmark`/`lama` wheels (pytorch-lightning, onnxruntime) — fine on a good connection, but on flaky DNS sync only `--extra gpu --extra dev` and run the lint/test steps by hand.
+- Metadata/C2PA tests assert against real committed fixtures in `data/samples/` (`chatgpt-*.png` = OpenAI C2PA, `firefly-1.png` = Adobe, `mj-*` = Midjourney IPTC, `doubao-1.png` = ByteDance Doubao with the China TC260 `<TC260:AIGC>` XMP label **and** a visible "豆包AI生成" text mark bottom-right; `grok-1.jpg` = xAI Grok with its EXIF-only `Signature:` blob + UUID `Artist` and no C2PA/SynthID/IPTC; `flux-1.png` / `flux-1.jpg` = real Black Forest Labs FLUX.2 Playground output, signed C2PA (issuer "Black Forest Labs" + `trainedAlgorithmicMedia`) -- `flux-1.jpg` is the first committed **JPEG-with-C2PA** fixture, exercising the c2pa-python non-PNG reader path end to end; whether BFL hosted output also embeds the open DWT-DCT pixel watermark is UNRESOLVED -- our detector returns None on these fox samples, but they are high-texture carriers where even a known-embedded watermark fails the round-trip, see the content-fragility caveat in `docs/watermarking-landscape.md`); synthetic byte blobs cover the remaining JPEG/ISOBMFF format paths. The "non-AI / clean photo" control is no longer in `data/samples/` -- the `clean_photo` conftest fixture serves a verified-negative image from the corpus `neg/` set (skips if the corpus is absent).
+- SynthID reference corpus: `scripts/synthid_corpus.py` ingests labeled images into `data/synthid_corpus/`. The labeled `images/` (`pos/` `neg/` `cleaned/`) are **committed** (public repo -- review every image for private content before adding; `manifest.csv` is kept in sync with the files on disk, one row per tracked image); only the synthetic `refs/` calibration fills are gitignored. See its README for the collection protocol and verification oracles. **`cleaned/` examples must be produced by a CURRENT shipped removal method** -- the default SDXL img2img pass (optionally `--max-resolution`). Do NOT archive cleaned outputs from methods that are no longer in the pipeline (ctrlregen, the old text/face-protection, IP-Adapter FaceID, CodeFormer) or from the experimental opt-in paths (controlnet, face restore) as corpus examples; a cleaned reference should represent the canonical removal, and a removed method's output is not a reproducible example. Keep those experiment outputs in a local working dir, never in the committed corpus.
 
 ## Configuration
 
-- GPU/ML modules (invisible_engine, ctrlregen, watermark_remover) are optional — guard imports with `is_available()` checks
+- GPU/ML modules (invisible_engine, watermark_remover) are optional — guard imports with `is_available()` checks
 - Optional detection extras: `detect` (imwatermark — open SD/SDXL/FLUX watermark) and `trustmark` (Adobe TrustMark decoder; pulls torch + downloads weights). Both are guarded by `is_available()` and skipped by `identify` when absent.
-- Tests for the *model-running* paths are limited to availability checks (multi-GB downloads). But the **pure helpers inside ML-adjacent modules are unit-tested without any download** and must stay that way: `_target_size` (native-vs-downscale, `test_invisible_engine.py`), the MPS->CPU fallback control flow via mocked pipelines (`test_img2img_runner.py`, 100% cover), and the tiling math `tile_positions`/`make_blend_weight`/`resize_center_crop` (`test_tiling.py`; `pytest.importorskip("torch")` since `tiling.py` imports torch at module top). Don't skip these as "ML, needs a model" — only `run_tiled`/`remove_watermark`/the diffusion bodies do.
+- Optional `esrgan` extra (spandrel only): Real-ESRGAN pre-diffusion super-resolution for small inputs (`upscaler.py`, CLI `--upscaler esrgan` on `invisible`/`all`/`batch`). Guarded by `upscaler.is_available()`; the default upscaler stays Lanczos (cv2, no deps) and the engine falls back to Lanczos when the extra is absent or the model errors. spandrel is MIT and pulls NO basicsr (only torch/torchvision/safetensors/numpy/einops); Real-ESRGAN weights are BSD-3-Clause and download on first use via `torch.hub` (never bundled). Kept OUT of `all` (heavy + model download).
+- Tests for the *model-running* paths are limited to availability checks (multi-GB downloads). But the **pure helpers inside ML-adjacent modules are unit-tested without any download** and must stay that way: `_target_size` (native-vs-downscale-cap-vs-upscale-floor, `test_invisible_engine.py`), `humanizer.unsharp_mask`/`adaptive_polish` (`test_humanizer.py`), and the MPS->CPU fallback control flow via mocked pipelines (`test_img2img_runner.py`, 100% cover). Don't skip these as "ML, needs a model" — only `remove_watermark`/the diffusion bodies do.
 
 ## Key modules
 
-- `noai/c2pa.py` — PNG chunk parser; use `extract_c2pa_chunk(path)` to get raw caBX payload, `has_c2pa_metadata(path)` to detect. Do not reimplement chunk parsing. `extract_c2pa_info(path)` sets `synthid_watermark`/`synthid_vendors` when the manifest is signed by a SynthID-using vendor, and `soft_binding`/`soft_binding_vendors` when a `c2pa.soft-binding` `alg` names a forensic-watermark vendor (`soft_binding_vendors_in(buffer)` is the shared byte-scan, used by both the PNG parser and the non-PNG binary path).
-- `noai/constants.py` — PNG_SIGNATURE, C2PA_CHUNK_TYPE, C2PA_SIGNATURES, C2PA_ISSUERS, `SYNTHID_C2PA_ISSUERS` (issuers that pair SynthID with C2PA: Google, OpenAI), and `C2PA_SOFT_BINDINGS` (soft-binding `alg` prefix → forensic-watermark vendor: Adobe TrustMark, Digimarc, Imatag, Steg.AI, Microsoft, ...). Add a new issuer/binding here, not inline.
-- `metadata.py` — `scan_head(path, size=1MB)` is the shared input for every C2PA/AIGC/IPTC byte scan: first `size` bytes plus the payloads of any provenance metadata found beyond that window — for ISOBMFF, the late provenance boxes from `isobmff.scan_c2pa_region` (catches a manifest after a large `mdat`); for **PNG**, the late `tEXt`/`iTXt`/`zTXt`/`eXIf`/`iCCP` chunks from `_png_late_metadata` (catches an XMP/EXIF packet appended after a large `IDAT`, e.g. a TC260 AIGC label at ~2.7 MB). Behavior-neutral (`f.read(size)`) for non-ISOBMFF inputs and for any file that fits within `size`. Use it instead of `open().read(1MB)` for any new marker scan. `synthid_source(path)` returns the vendor name(s) if the C2PA manifest implies a SynthID pixel watermark, else None. Format-agnostic: PNG via the caBX parser, JPEG/WebP/AVIF/HEIF/JXL via a binary scan (C2PA marker + SynthID issuer + AI-source marker). `get_ai_metadata` surfaces the verdict, and `metadata --check` prints it as a callout. Both `get_ai_metadata` and `has_ai_metadata` guard the PIL open with `except Exception` (HEIC/unknown formats raise non-OSError) and fall through to the binary scan. `xai_signature(path)` detects xAI/Grok's EXIF-only scheme (`ImageDescription` = `Signature: <base64>` + UUID `Artist`); it feeds `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify`. `iptc_ai_system(path)` detects the IPTC Photo Metadata 2025.1 AI-disclosure XMP properties (`IPTC_AI_FIELD_MARKERS` = `AISystemUsed`/`AISystemVersionUsed`/`AIPromptInformation`/`AIPromptWriterName`) and returns the `AISystemUsed` generator name (or `"fields present"`). `remove_ai_metadata` routes **ISOBMFF video** (`.mp4`/`.mov`/`.m4v`) through the same `isobmff.strip_c2pa_boxes` as AVIF/HEIF (MP4 is ISOBMFF), and `_scrub_ai_exif` removes the xAI signature + AI-generator EXIF tags on JPEG output.
-- `identify.py` — `identify(path)` aggregates every locally-readable signal (C2PA issuer→platform, C2PA soft-binding forensic-watermark vendor, IPTC "Made with AI" + IPTC 2025.1 `AISystemUsed`, embedded SD/ComfyUI params, SynthID proxy, xAI/Grok EXIF signature via `metadata.xai_signature`, the China TC260 AIGC label via `metadata.aigc_label`, the HuggingFace `hf-job-id` job marker via `metadata.huggingface_job`, the Samsung Galaxy AI editing marker via `metadata.samsung_genai`, visible Gemini sparkle, open invisible watermark, Adobe TrustMark via `trustmark_detector`) into one `ProvenanceReport`. `is_ai_generated` is True or None (never asserted False — stripped metadata is not proof of clean origin). The `hf_job`, visible-sparkle, and Samsung `samsung_genai` signals are **medium** confidence: each lifts an otherwise-Unknown verdict to a tentative AI (`hf_only` / `visible_only` / `samsung_only`, parallel branches) but is excluded from the high-confidence `ai_from_metadata` set, so none overrides a hard metadata signal. Visible-sparkle is promoted only at confidence ≥ `_SPARKLE_THRESHOLD` (0.5; corpus-tuned to separate Gemini sparkles ≥0.56 from non-sparkle ≤0.49). The cv2 dependency lives in `gemini_engine.detect_sparkle_confidence`, not here. **C2PA platform attribution is device-token-first, issuer-scan fallback** (`_device_platform` scans manifest bytes for `_DEVICE_C2PA_PLATFORM` tokens, then `_attribute_platform`/`_ISSUER_PLATFORM`). **Why, verified on real signed files 2026-05-26:** the old issuer-only byte-scan matched ANY issuer substring anywhere, so multi-entity manifests mis-attributed -- Leica→"Truepic" (a signing authority in the trust chain), Nikon→"Adobe Firefly" (XMP-toolkit "Adobe" + the sample's "Adobe_MAX" name), Pixel→"Google (Gemini)" ("Google LLC" cert org), Truepic→"Google". A distinctive device token wins instead. **Token distinctiveness is load-bearing:** bare `b"Truepic"` mis-fires (it appears in unrelated trust chains -- it mis-attributed the OpenAI `chatgpt-1.png` fixture), so the token is the specific `b"Truepic_Lens"` from the Lens SDK claim generator; likewise `b"Pixel Camera"` (cert CN) not bare `b"Pixel"`. `_DEVICE_C2PA_PLATFORM` lists ONLY tokens **verified against a real C2PA file**: Leica (`lc_c2pa`/`Leica Camera`), Nikon (`NIKON`), Pixel (`Pixel Camera` -- from a real Pixel 10 Pro file attached to c2pa-rs issue #1609/#1554), Sony (`sony.sig`/`sony.cert` -- Sony's own C2PA assertion namespace, verified on a real Sony PXW-Z300 file; NOT bare "Sony" which is a common EXIF Make), Truepic (`Truepic_Lens`). Canon/Bria have **no public direct-download C2PA sample** (checked exhaustively: GitHub issue/PR attachments, contentcredentials gallery, HF datasets -- all upload-to-verify or token-gated; Canon's only public file was a self-signed hobbyist CR3, not factory), so they stay unmapped until a real file is captured (same fixture discipline as Grok/Doubao). The Sony sample is video (MP4) -- our ISOBMFF C2PA path detects it; Sony Alpha stills likely share the `sony.*` namespace but are not separately verified. **Samsung Galaxy + ASUS Gallery live in a separate `_SIGNER_C2PA_PLATFORM` (scanned after `_device_platform`, before the issuer fallback), NOT in `_DEVICE_C2PA_PLATFORM`** — verified on real signed files 2026-05-29. Reason: a Galaxy phone stamps BOTH its device cert AND a `trainedAlgorithmicMedia`/genAIType AI marker on a Generative-Edit image, so treating it as a "genuine camera capture" would false-fire integrity-clash rule 2 on every Galaxy AI edit. The signer tokens (`b"Samsung Galaxy"` cert org — distinct from the EXIF `SM-xxxx` model string on ordinary Samsung photos; `b"com.asus.gallery"` claim generator) only resolve the platform label; the AI verdict still comes from the source-type / genAIType. ASUS Gallery is a C2PA-signed edit with no AI marker, so it attributes the platform without asserting `is_ai`. **Samsung's `genAIType` (in the proprietary `PhotoEditor_Re_Edit_Data` JSON) is an undocumented Galaxy-AI editing marker** (`metadata.samsung_genai`, gated on the `PhotoEditor_Re_Edit_Data` container; non-zero value = AI tool used, values {1,5} observed): medium-confidence because the field has no public spec (verified 2026-05-29: absent from C2PA spec + Samsung docs), but it co-occurred with `trainedAlgorithmicMedia` in 3/3 verified files that record a source-type and was the SOLE AI marker on a Galaxy S24 file that omits the source type. Camera C2PA marks capture authenticity, not AI (Pixel carries `computationalCapture`, not `trainedAlgorithmicMedia`), so these never set `is_ai` -- that stays driven by digital-source-type. `c2pa.cbor_text_after` (now public) is best-effort for the `generator` detail string only and can be None when the manifest keys it `claim_generator_info` (Pixel). **Issuer→generator mapping is `is_ai`-gated** (`_attribute_platform(issuers, is_ai=c2pa_is_ai)`): a specific AI-generator platform is named only when the Line truncated
-- `watermark_registry.py` — **single catalog of known visible watermarks**, the unified "find known marks in their usual places, recognize, remove" entry. **Reverse-alpha only by policy**: a mark is listed only once a real alpha map has been captured for it, and removal inverts that map (`original = (wm - a*logo)/(1-a)`, exact recovery) — no inpaint/heuristic removal here (arbitrary-region inpainting lives in `region_eraser`/`erase`). Each `KnownMark` ties a key to {usual `location`, `in_auto` flag, `recovery` (="reverse-alpha"), a `detect` adapter → uniform `MarkDetection`, a `remove` adapter}. Entries today: `gemini` (bottom-right sparkle) and `doubao` (bottom-right "豆包AI生成"). `detect_marks` scans all; `best_auto_mark` picks the highest-confidence detection. **Cross-engine confidences aren't directly comparable**, so the gemini adapter applies the corpus-validated 0.5 sparkle threshold (`_GEMINI_AUTO_MIN_CONF`) for its `detected` flag — otherwise the gemini engine's loose internal threshold weakly fires (~0.36) on the Doubao text and hijacks `auto`. `cli.cmd_visible` is registry-driven: `--mark auto` → `best_auto_mark`, `--mark <key>` → that mark; `--mark` choices come from `mark_keys()`. `_doubao_remove` applies reverse-alpha only when the mark is detected AND `reverse_alpha_available` (resolution in the alpha band); outside that, removal is **skipped** (not inpainted). Add a new visible mark = one `KnownMark` entry + its engine (with a captured alpha map); do not re-add per-mark `if` branches in the CLI. **Alpha-on-save policy (issue #30):** `cli._write_bgr_with_alpha` rejoins the input's alpha plane **unchanged** — it must NOT zero alpha in the watermark bbox. Reverse-alpha (and `erase` inpaint) recover real pixels there, so zeroing alpha punched a transparent hole that renders as a solid **white box** on any non-transparent viewer (Gemini app exports are opaque RGBA, so every user hit it; regression-guarded by `test_visible_keeps_alpha_opaque_in_watermark_region`). The registry `remove()` still returns its region (used for `inpaint_residual` positioning), but the CLI no longer uses it to clear alpha.
-- `gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU). `detect_sparkle_confidence(path)` is the file-level entry point used by `identify.py`.
-- `doubao_engine.py` — visible Doubao "豆包AI生成" remover/detector (cv2/numpy, no GPU), **reverse-alpha only**. `DoubaoEngine.locate` anchors a bottom-right box by **geometry** (mark scales with image WIDTH), `extract_mask` pulls the light low-saturation glyphs (the detection candidate). `detect` is **reverse-alpha-consistent**: it matches the bundled alpha glyph silhouette (`assets/doubao_alpha.png`, the exact shape we invert) against the candidate via zero-mean normalized correlation (`_template_match_score`, cv2 `TM_CCOEFF_NORMED`), gated at `DETECT_NCC_THRESHOLD` 0.4 over a small `DETECT_MIN_COVERAGE` floor. Keying on glyph SHAPE (not coverage/structure heuristics) fixed #23: corpus FP fell to 7/1243 (0.6%); old coverage-only fired on ~28%. **Removal is exact reverse-alpha** (`remove_watermark_reverse_alpha`): `original = (wm - a*logo)/(1-a)` from the bundled alpha map + `_ALPHA_LOGO_BGR` (near-white ~253) + `_ALPHA_*_FRAC` geometry. The alpha map + logo were **solved from real black+gray Doubao captures** (`data/doubao_capture/captures/`, gitignored): on black `captured = a*logo`, the black/gray pair solves `a` per-pixel without assuming the logo colour (white capture cross-validates: mark → flat fill). The single captured alpha map (at width 2048) **generalizes to any resolution**: at (near) the captured width (`_ALPHA_NATIVE_BAND` of `_ALPHA_NATIVE_WIDTH`) `_fixed_alpha_map` places it by exact width-relative geometry (pixel-exact recovery, ~0.9 mean error — the whole point of reverse-alpha); off that width it **tries BOTH placements -- fixed geometry AND `_aligned_alpha_map`'s `TM_CCOEFF_NORMED` scale+position search (`_ALPHA_ALIGN_SEARCH`) -- and keeps whichever leaves the least residual mark** (re-`detect` confidence on the bare reverse-alpha). On a faint/busy-background mark the NCC peak wanders a few px and geometry wins; on a clear mark alignment wins -- no magic threshold, it just picks the better removal. Verified **56/56 real detected-Doubao removed clean across all corpus resolutions** (2048 fixed 27/27, 1773 22/22, plus 1185/1187/1535/1672); a single fixed-vs-aligned choice left 2/56 busy-background residuals, try-both fixed them. `reverse_alpha_available` is just "asset present"; the registry still gates removal on `detect` so a clean corner is never touched. **Residual inpaint is off-native-only:** at the captured width the fixed-geometry recovery is exact, so it is returned untouched -- inpainting over exactly-recovered interior pixels only swaps them for a cv2 hallucination (measured worse, native textured-bg error vs true bg **1.6 reverse-alpha-only vs 2.6 with the old always-on full-footprint inpaint**; regression-guarded by `test_native_returns_exact_reverse_alpha_no_inpaint`). Off-native the NCC alignment is only sub-pixel-approximate, so the interior is no longer exact and a residual inpaint over the glyph footprint cleans the seam (costs nothing there and reliably clears the mark). The shipped third-party `_refs/zhengsuanfa_doubao_alpha_120x20.png` is NOT a usable alpha (≈0.85 everywhere → blacks out on inversion; wrong resolution/version), verified 2026-05-29. There is no inpaint-based removal here (removed 2026-05-29; arbitrary-region inpainting is `region_eraser`/`erase`).
-- `region_eraser.py` — universal region eraser (`erase` CLI). `erase(image, boxes=|mask=, backend=)`: `boxes_to_mask` → `cv2.inpaint` (`cv2` backend, default, no deps) or big-LaMa via onnxruntime (`lama` backend, extra `lama`, `Carve/LaMa-ONNX` Apache-2.0 model downloaded on first use, never bundled). `erase_lama` crops a padded region around the mask, runs LaMa at its fixed 512² input, pastes only masked pixels back (untouched areas stay pixel-exact). Lazy `_get_lama_session` singleton; `lama_available()` guards the optional import. **LaMa-ONNX costs ~3.5-4 GB peak RAM and ~5-6 s/call on CPU** (FFC working set, not arena — `enable_cpu_mem_arena=False` does not help), so it does NOT fit a minimal droplet; the cv2 backend (tens of MB, ~30 ms) does. LaMa quality at low RAM = serverless/GPU, mirroring how raiw.cc offloads SDXL to fal.
-- `invisible_watermark.py` — `detect_invisible_watermark(path)` decodes the OPEN DWT-DCT watermarks (public decoder, no key) embedded by Stable Diffusion / SDXL / FLUX via the `imwatermark` library. Known fixed patterns (verified against upstream source) live in `_BITS_48` (SDXL 48-bit, FLUX.2 48-bit) and `_SD1_STRING` ("StableDiffusionV1", SD 1.x/2.x). Optional dep (extra `detect`); returns None when absent. The `detect` extra pulls **torch** transitively (invisible-watermark declares torch a hard dep, and `WatermarkDecoder` eagerly imports `rivaGan` -> `torch` at import time), so detection needs torch present even though dwtDct runs CPU-only on cv2/numpy/pywavelets — no GPU and no separate `gpu` extra required. **Unlike SynthID this is locally detectable**, but the watermark is fragile (does not survive JPEG re-encode/resize — verified gone after JPEG q90), so it confirms origin only on pristine files. Add new known patterns here. The file carries a top-of-module pyright pragma because imwatermark/cv2 ship no type stubs.
-- `trustmark_detector.py` — `detect_trustmark(path)` decodes the OPEN, keyless **Adobe TrustMark** watermark (the soft binding behind Adobe Durable Content Credentials, `alg` `com.adobe.trustmark.P`) via the optional `trustmark` package (extra `trustmark`; pulls torch, downloads model weights on first use). Mirrors `invisible_watermark.py` (lazy singleton, top-of-module pyright pragma, returns None when absent). It detects *provenance*, not AI origin as such (TrustMark also marks human-authored content), so `identify` lists it as a watermark without setting `is_ai_generated`. Other soft-binding vendors (Digimarc/Imatag/Steg.AI/...) have no public decoder — they are only *named* via the `C2PA_SOFT_BINDINGS` scan, not decoded. **False-positive gate (added 2026-05-29):** TrustMark's `wm_present` is a BCH error-correction validity flag that spuriously validates on a content-correlated fraction of un-watermarked images — AI-generated textures trip it far more than camera photos (verified 2026-05-29 on real files: it fires on Gemini/OpenAI/Doubao output that *cannot* carry Adobe's watermark, with a random-bytes decoded secret, while signal-free camera photos did not trip it). A genuine TrustMark is a *durable* soft binding engineered to survive re-encoding, so `detect_trustmark` re-decodes after a mild JPEG round-trip (`_survives_reencode`, `_REENCODE_QUALITY` 95) and requires the same schema both times; every observed false positive collapsed (none survived even q95), so the gate is the durability property the watermark guarantees. The second decode runs only on the rare initial hit, so the cost is negligible. Do NOT remove the gate to "catch more" — a lone TrustMark hit without it is almost always content noise.
-- `text_protector.py` — text-region protection for the `invisible` SDXL img2img pass (issue #21: CJK/small text deforms at watermark-removal strengths). `is_available()` gates on `cv2.dnn.TextDetectionModel_DB`; `TextProtector.detect_text_boxes(bgr)` runs the **PP-OCRv3 DB** ONNX detector (~2.4 MB, Apache-2.0, opencv_zoo, returns rotated quad polygons) — downloaded+cached to `~/.cache/remove-ai-watermarks` on first use via atomic temp-rename, never bundled, **no torch (cv2.dnn only)**. **Detection is script-agnostic** (DB segments text *regions*, not characters), so Latin / Cyrillic / CJK / Hangul / Arabic / digits all detect identically — language was never the recall lever, **resolution was**. `_detection_input_size(h, w)` (pure, unit-tested) detects at the **native long side capped at `_DET_MAX_LONG_SIDE` (1536), never upscaled**: the old fixed 736 downscaled large canvases so small text fell below the detector and was missed (issue #14, e.g. ~16 px text on a 2048 image). `scripts/text_detection_benchmark.py` measures recall across scripts × sizes × canvas: the cap fix lifts overall hit-rate 0.91 → 1.00 (worst cell 2048/16 px: 0.06 → 1.00) at ~100 ms CPU. Very large canvases with tiny text may still need tiling (documented limit, not built). `build_change_map(boxes, h, w, preserve=0.9, feather=15)` paints a Differential-Diffusion change map. **Polarity (verified empirically):** white(1.0)=PRESERVE original pixels, black(0.0)=MAX change; map is black bg + `preserve` inside text polygons, Gaussian-feathered edges, clipped to [0,1]. `preserve` stays below a hard 1.0 freeze by default so text still scrubs lightly (SynthID survives cropping). Wired into `watermark_remover._run_differential` via the community `pipeline_stable_diffusion_xl_differential_img2img` (loaded with `custom_revision="0.38.0"` — HF resolves the **PyPI** version string, not the `v0.38.0` git tag); gated to the SDXL `DEFAULT_MODEL_ID` only (`_can_protect_text`), falls back to plain img2img otherwise. **Autonomous by default** (`protect_text=True` in `invisible_engine`/`watermark_remover`, mirroring `protect_faces`): the detector runs per image and `_run_differential` falls back to plain img2img when **no boxes** are found, so text-free inputs pay only the cheap cv2 detection (no differential-pipeline load). CLI exposes a single off-switch `--no-protect-text` on `invisible`/`all` (passed as `protect_text=not no_protect_text`); the unavailable-model case logs at debug, not warning, since it is now the default path. The diff pipeline upcasts the VAE to fp32 internally, so do **not** add `upcast_vae()`/`enable_attention_slicing` (both produced NaN/black on fp16 MPS). `build_change_map` is unit-tested without any model download (`tests/test_text_protector.py`).
-- `face_protector.py` — YOLO detect + soft-blend pattern; mirror this for any "protect region during diffusion" features
-- `image_io.py` — Unicode-safe cv2 IO (issue #17). `imread(path, flags=None)` / `imwrite(path, img)` wrap `np.fromfile`+`cv2.imdecode` / `cv2.imencode`+`tofile` so non-ASCII paths work on Windows -- bare `cv2.imread`/`cv2.imwrite` use the platform ANSI code-page API there and fail (empty decode + `can't open/read file`) on Chinese/Cyrillic/accented filenames. `imread` keeps `cv2.imread` semantics (defaults to `IMREAD_COLOR`, returns `None` on missing/empty/undecodable). **Every cv2 file read/write in the package routes through here; do not call `cv2.imread`/`cv2.imwrite` directly.** macOS/Linux already accept UTF-8 paths, so it is behavior-neutral there (the bug only reproduces on Windows). cv2/numpy are imported lazily inside the functions, so the module is cheap to import in a bare env.
+Compact map. The full per-module detail (design decisions, tuned thresholds, calibration history, incident records, and the regression-guard map) lives in `docs/module-internals.md` — **read the relevant section there before changing any module below.**
 
-### Doubao clean-reverse-alpha distillation (re-investigated 2026-05-29)
+- `noai/c2pa.py` — C2PA reading. `extract_c2pa_info(path)` uses the official **c2pa-python `Reader`** first (core dep, any container; `read_manifest_store_json` returns the WHOLE store JSON — active + ingredient manifests — so an AI marker on a parent manifest is seen), and falls back to the hand-rolled caBX/CBOR parser (`has_c2pa_metadata` / `extract_c2pa_chunk` / `_extract_c2pa_info_png`) for synthetic/partial blobs the validator rejects or a broken/absent wheel. The registry scan (issuer / source-type / SynthID / soft-binding) is shared by both paths via `_populate_registry_fields`, so the return-dict shape is identical. Do not reimplement chunk parsing; chunk reads are clamped to the remaining file size by design. `extract_c2pa_chunk`/`inject_c2pa_chunk` stay PNG-only (raw caBX bytes, test/extractor use).
+- `noai/constants.py` — the single `C2PA_AI_VENDORS` registry (+ `C2PA_SOFT_BINDINGS`) from which `C2PA_ISSUERS` / `SYNTHID_C2PA_ISSUERS` / `C2PA_IDENTITY_AI_ORGS` / `identify._ISSUER_PLATFORM` are all derived. Add a new vendor as one registry entry; never edit the derived dicts and never add inline. A vendor's `asserts_ai=True` flag means its mere presence asserts AI generation even without a `trainedAlgorithmicMedia` digital-source-type (a pure-generator brand with a distinctive issuer/generator string, e.g. **Dreamina** — ByteDance's international Jimeng brand, signed as "Bytedance Pte. Ltd." with a "Dreamina/x.y" claim generator and no source-type); NEVER set it for common-word issuers (Adobe/Google/OpenAI/Microsoft) that appear incidentally in unrelated bytes — those stay source-type-gated in `identify._attribute_platform`.
+- `metadata.py` — `scan_head(path)` is the shared (memoized) input for every C2PA/AIGC/IPTC byte scan; use it instead of `open().read(1MB)` for any new marker scan. Also home to `synthid_source`, `xai_signature`, `iptc_ai_system`, `aigc_label`, `huggingface_job`, `samsung_genai`, and `remove_ai_metadata` (fail-safe `strip_c2pa_boxes`). **A caller that REPORTS an outcome must use `strip_and_verify`, not `remove_ai_metadata` directly** -- the stripper is deliberately fail-safe (a file PIL cannot decode is copied through UNCHANGED rather than crashing), so its return value cannot distinguish a no-op from a real strip. `metadata --remove` and `batch --mode metadata|all` both re-scan the OUTPUT through it and fail loudly; corpus-observed on real Samsung Galaxy S22 C2PA PNGs, where the command printed "stripped" and exited 0 while the output still read as AI (2026-07-19). **`remove_ai_metadata` is the SINGLE metadata stripper** (the legacy PIL-re-encoding `noai/cleaner` was deleted; the diffusion core and the public `noai.remove_ai_metadata` re-export now point here). It strips **losslessly** per container: ISOBMFF (HEIC/AVIF/MP4) blanks tokens / strips boxes in place; **JPEG uses `_strip_jpeg_metadata_lossless`** — a marker-segment walk that drops the AI-bearing APP segments (C2PA APP11; XMP APP1 carrying C2PA, a China-AIGC token, OR an IPTC `digitalSourceType` / 2025.1 AI-disclosure marker; IPTC-IIM APP13) and scrubs AI EXIF tags via piexif, copying the entropy-coded scan verbatim so **the pixels are bit-identical** (no DCT re-encode). **Detection<->removal parity across every marker placement is load-bearing** — anything a scanner flags, the strip must reach, or a re-served file still reads as AI: (a) the APP1-XMP branch of `_jpeg_app_carries_ai` checks the IPTC marker sets too, not only C2PA/AIGC (the Instagram/MidJourney/Meta "Made with AI" `digitalSourceType` lives in XMP, not the APP13 IIM record); (b) a bare `AIGC{...}` / `{"AIGC":{...}}` block in ANY JPEG APP segment — the specific C2PA(APP11)/XMP(APP1)/IPTC(APP13) checks FALL THROUGH to a generic `_is_aigc_exif_value` drop, so a bare AIGC in APP11 (the common real-corpus placement, NOT a C2PA manifest) is caught, not swallowed by the C2PA-only 0xEB branch — plus the same AIGC block in a STANDARD **PNG text chunk** value (e.g. `Description`, which `_is_ai_key` keeps) is dropped on the value; (c) the China TC260 `{"AIGC":{...}}` block in EXIF `UserComment`/`ImageDescription` is scrubbed by `_scrub_ai_exif` (Doubao producer + Tencent service-provider schemas); (d) the Samsung Galaxy AI `PhotoEditor_Re_Edit_Data` trailer past the JPEG EOI is truncated by `_strip_samsung_trailer` (and `samsung_genai` reads the file tail so a multi-MB photo's trailer past the 512 KB quick-scan window is still DETECTED). Pixels stay bit-identical throughout, so a `--strip-metadata` on a q100 removal output does NOT crush it back to q75; PNG/WebP re-saves are pixel-lossless (WebP written at cv2 lossless mode, quality 101 — quality 1-100 is lossy). **The PIL-fallback save format is chosen by the source's CONTENT, not its file extension** (`_sniff_image_format`, and the JPEG-lossless gate is content-gated too): ~2% of real uploads are misnamed (a PNG served as `.jpg` is the common one), and routing on the extension re-encoded a lossless PNG/WebP into a real JPEG — a silent degradation that broke "work with originals" (corpus-measured ~0.9% of files). A **misnamed** lossless source (source-extension format != content) is preserved in its true format; a **correctly-named** source still honors a deliberate output-extension conversion (e.g. `source.png -> output.jpg`). Not yet handled: a 16-bit PNG is downconverted to 8-bit on the PIL re-save (rare; would need a byte-level PNG chunk stripper). Regression: `tests/test_metadata.py::TestHasAiMetadata::test_strip_preserves_lossless_content_with_mismatched_extension`. **`remove_ai_metadata` is fail-safe on an undecodable image:** a truncated/corrupt file (PIL raises `OSError` decoding it; ~0.2% of real uploads) is copied through UNCHANGED rather than crashing a direct library caller (a web worker would 500 on a partial upload), mirroring `strip_c2pa_boxes` — we cannot strip what we cannot parse, but we never raise. Regression: `tests/test_metadata.py::TestHasAiMetadata::test_remove_ai_metadata_failsafe_on_truncated_png`. Regression: `tests/test_metadata.py::TestHasAiMetadata::{test_jpeg_metadata_strip_is_pixel_lossless, test_jpeg_strip_removes_iptc_marker_in_xmp}`, `TestSamsungGenai::{test_remove_strips_post_eoi_trailer, test_detects_trailer_past_scan_window}`, the AIGC-EXIF/bare-APP removal tests, and `tests/test_noai.py::TestISOBMFF::{test_blank_aigc_block_in_exif, test_blank_xai_signature_pair_in_exif}`. `exif_generator` matches a VALUE against `AI_GENERATOR_TOKENS` across EXIF `Software`/`Make`/`Artist`/`ImageDescription`, XMP `CreatorTool`, AND PNG `tEXt` chunks (`Software`/`Source`/`Title`/`Description` — NovelAI Line truncated
+- `identify.py` — aggregates every locally-readable signal into one `ProvenanceReport`; `is_ai_generated` is True or None, never asserted False. `ProvenanceReport.ai_source_kind` exposes the C2PA digital-source-type split — `"generated"` (trainedAlgorithmicMedia, fully AI) vs `"enhanced"` (compositeWithTrainedAlgorithmicMedia, a real photo with an AI-composited region), else None — so a caller branches full-frame scrub vs region-targeted clean (see `noai/tiling.feather_region_composite` + `WatermarkRemover.remove_watermark(region=...)`). The sparkle provenance threshold is the SHARED `watermark_registry.GEMINI_SPARKLE_TRUST_CONF` (imported, not a private copy) so the provenance "is there a sparkle" verdict and the removal "take the sparkle" decision can never drift. `import identify` is deliberately light (lazy `noai/__init__`, fits a 512 MB host) — keep heavy imports out (the `watermark_registry` constant import stays light: engines are lazy there). Add capture-camera tokens to `_DEVICE_C2PA_PLATFORM` only when verified against a real C2PA file; editing-app/AI-device signer tokens go to `_SIGNER_C2PA_PLATFORM`; generator/issuer platforms to `C2PA_AI_VENDORS` in `constants.py`. The IPTC `digitalSourceType` **`algorithmicMedia`** (bare) is PROCEDURAL (an algorithm not trained on sampled data), NOT AI/ML generation, so it is deliberately absent from `IPTC_AI_MARKERS` — flagging it made `identify` assert AI + `has_invisible_target` True, scrubbing clean procedural content (it is a distinct token from `trainedAlgorithmicMedia`, so real "Made with AI" labels are unaffected; regression `test_metadata.py::...test_bare_algorithmic_media_not_flagged_ai`). Integrity-clash detection is high-precision by design (only hard generator stamps feed it, source-grouped independence). `_vendor_of` normalizes ByteDance/Canva/ElevenLabs/Black Forest Labs (as well as OpenAI/Google/... ) so their C2PA claims participate in the clash check; the generic **China TC260 AIGC label names no specific vendor**, so when a TC260-applying vendor (ByteDance, `_TC260_VENDORS`) is co-attributed the label is attributed to it (a legit Doubao image carrying its own TC260 label must NOT clash), while a NON-TC260 vendor next to a TC260 label still clashes as a laundering tell. Corpus-validated: adding the vendors introduced 0 new clashes on 5000 carriers.
+- `watermark_registry.py` — the single catalog of known visible watermarks (gemini / doubao / jimeng / qwen / samsung / jimeng_pill). **Removal is LOCALIZE -> FILL for every mark:** each mark is localized to a binary full-frame footprint mask (a `Localization`), then ONE shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). Reverse-alpha (the old `original = (wm - a*logo)/(1-a)` inversion of a captured alpha map + thin residual inpaint) is GONE for ALL marks; why it was dropped is recorded in `docs/module-internals.md`. Backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. The captured alpha maps (`scripts/visible_alpha_solve.py`) are still used to DETECT the marks and to shape the mask, but NOT for pixel recovery. **`--mark auto` removes EVERY detected mark in one pass** via `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` (marks coexist -- a Jimeng-basic image has the top-left pill AND the bottom-right wordmark; a single-strongest pick would leave one). **Three orthogonal axes:** `backend` (the fill), `sensitivity` (how hard to trust a borderline mark: `auto`/`strict`, see the `Sensitivity` literal), and `provenance` (vendor keys metadata confirms -- the evidence that drives `auto`). **Perception / decision / action are separated:** `_build_candidates(image)` runs every detector at BOTH trust levels (strict + relaxed) and packages raw verdicts + features into `Candidate`s (no policy); the pure arbiter `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` makes every keep/drop call (per-mark `resolve_trust` + the assumed-trust floor + the pill gate) with no image/IO, so it is unit-testable in isolation; then each winner is localized -> filled. Do NOT put policy back into the engines (the one exception, the Gemini FP gate, stays in `gemini_engine` because `identify` shares that confidence). `detect_marks(..., provenance=frozenset())` stays strict (identify verdict, precision over recall); `KnownMark.remove/detect/localize(..., provenance: bool)` take the already-resolved boolean. **How `auto` decides (this is metadata-INDEPENDENT for recall):** the visual detectors are pixel-based and need no metadata; the recall gain comes from RELAXING the false-positive gate, not from metadata. `strict` never relaxes (clean images untouched); `auto` relaxes a mark only on same-product evidence -- metadata provenance for that vendor OR a confidently detected sibling mark of the SAME product (`_PRODUCT_OF`; Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax). **`resolve_trust` resolves TWO levels:** `confirmed` bypasses the engine's false-positive gate, and only `confirmed` has evidence naming THAT vendor, which is exactly what the bypass is contracted to require (`GeminiEngine.detect_watermark`'s `trust_provenance` docstring: "external metadata already proves this is a Google generation"). **Historical, kept as the reason the third level is gone:** a removed `assumed` level let `assume_ai` bypass the gate on the bare assertion an image is AI. On its first form that left only the raw 0.35 detector threshold and it fired on **59.8% of 256 genuine camera captures**, filling a phantom sparkle on ~6 of every 10 CLEAN photos; a confidence floor made it tolerable, and the mode was removed outright in 0.16. Corpus-measured 2026-07-16 before removal (400 Google-C2PA positives with metadata hidden; 256 camera-capture negatives): recall strict 55.0% / auto 55.2% / assume_ai 62.8%, false fire 0.0% / 0.0% / 2.3% -- the extra recall was never free. A wrong relaxation only fills a small corner near-losslessly (the localize -> fill benign failure mode), which is what made a SMALL false-fire rate arguable; it was never a licence for a 60% one. Metadata provenance mapping (feeds `auto`, read by `cli._visible_provenance`): Google/Gemini C2PA issuer -> gemini; China-AIGC (TC260) label -> doubao/jimeng; `samsung_genai` -> samsung. **The `jimeng_pill` is CAPTURE-LESS** (`pill_engine.py`): the top-left "AI生成" label has no captured alpha map, so it is detect-by-synthetic-silhouette; its footprint is a fixed top-left geometry box. Its weak edge-NCC detector (~7% raw false-fire) is gated in `remove_auto_marks` via **`_keep_pill`** (32k real-upload corpus validation 2026-07): the pill never rides on a **Doubao** or **Qwen** detection (either bottom-right mark names its own product, and neither is Jimeng-basic), and has confirmation arms because metadata/intent confirms the platform, not pill presence. **(1) Bottom-right "★ 即梦AI" wordmark fired** — ~94% precise and survives **metadata-STRIPPED uploads** (screenshots / re-saves, ~61% of pills carry aLine truncated
+- `gemini_engine.py` — visible Gemini-sparkle detector + localizer (cv2/numpy, no GPU): top-K size-weighted fusion candidate selection (`_SELECT_TOPK`), corner-promote, false-positive gate (the provenance prior relaxes the gate + lowers the trust threshold when a Google/Gemini C2PA issuer confirms the vendor). **White-core rescue:** the FP gate demotes a low-gradient match (soft edges), but a real FAINT sparkle also has soft edges -- so the gate keeps a low-grad match that is a strong (conf ≥ `_SPARKLE_KEEP_CONF` 0.52), bright (margin), near-WHITE-core sparkle (`_core_saturation` ≤ `_SPARKLE_WHITE_SAT` 0.20): a real sparkle core is white, a clean bright corner that shape-matches (sky/sun) is colored. This recovers ~14/20 metadata-stripped faint sparkles under the DEFAULT strict/auto (no flag, no metadata) at ~1.25% clean false-fire (baseline 0.55%); the ~0.51-scoring bright-bg FPs stay demoted (below 0.52). A learned classifier on the SAME features was measured WORSE than the tuned gate (2026-07 tier-1: MLP 86.7% recall vs 90.8% at equal FP), so the heuristic stays; a patch-CNN with richer features is the only lever left (roadmapped P2, low expected value -- the wall is fundamental). Detection scores the top-K size-weighted matches by full fusion (spatial+gradient+variance) and keeps the highest — NOT the raw-NCC argmax, which re-admits the tiny-patch FPs the size weight suppresses (the osachub 2026-06-12 sub-0.85 corner-sparkle regression; see `docs/module-internals.md`). Keep the 0.85 corner-promote NCC gate; a margin/chroma-gated lower promote was measured and REJECTED 2026-06-11 (~33% FP on non-Google content). Removal is localize -> fill: `footprint_mask` returns the sparkle footprint (the captured alpha thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin), and the shared `watermark_registry.fill` inpaints it. The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery. `detect_sparkle_confidence` reuses a process-wide `_shared_engine()` singleton (lru_cache) — the engine holds only constant assets (captures, alpha maps, a precomputed 16..118 template ladder) and takes the image as an arg, so do NOT reconstruct `GeminiEngine()` per call: that reloaded assets + recomputed alpha maps + rebuilt the template cache on every one of ~34k `identify` calls (−24% on the sparkle path once made a singleton, output byte-identical). `detect_watermark`/`footprint_mask` guard `image.size == 0` before `to_bgr`, and return an empty (detected=False) result when no template scale fits (short side < 16 px), rather than dereferencing an empty candidate list.
+- `_text_mark_engine.py` — shared base for the text-mark engines (extracted 2026-06-09); the per-engine modules are config-only subclasses. Detection still matches the glyph silhouette (NCC, keys on glyph shape). The removal mask is TEMPLATE-FREE: it is the bounding box of the top-hat glyph blob (`extract_mask`), filled solid + dilated, so the shared fill inpaints the whole wordmark rectangle. This drops the fixed alpha-template placement, so a re-rendered or differently-placed mark is still masked; the captured alpha maps are now used only for the detection silhouette, not for removal. New text mark = a `TextMarkConfig` + a thin subclass + one registry row. Gemini stays a separate engine (different model). The detection scale ladder is per-mark (`TextMarkConfig.ladder`, default `(0.8, 1.0, 1.25)` -- added 2026-07-21 for qwen's two size modes; the shared default is unchanged for every other mark, and densifying the SHARED ladder was measured and rejected, see `docs/verification-plan.md` B2).
+- `pill_engine.py` — the CAPTURE-LESS Jimeng-basic "AI生成" pill (top-left, issue #54). No alpha map: `detect` is edge-NCC of a synthetic font-rendered silhouette (`assets/jimeng_pill.png`, regenerate via `scripts/render_pill_silhouette.py`; committed, data-safe -- corpus stays out of the repo) in the top-left ROI, calibrated on 61 local real positives to threshold 0.22; `footprint_mask` is a generous FIXED top-left geometry box (NOT the NCC match position -- the synthetic silhouette localizes only approximately, the corner is negative space, so a geometry box fills cleanly while a match box leaves outline residue). `footprint_texture`/`footprint_is_flat` (median-Sobel over that box, `_FLAT_TEXTURE_MAX`) back the metadata-only safe-fill gate. Removal is the shared localize -> fill (MI-GAN/cv2). Detector precision is weak (~7% raw false-fire), so it is registry-gated in `remove_auto_marks` via `_keep_pill`: never on Doubao; the bottom-right wordmark removes it unrestricted (~94% precise, survives metadata-STRIPPED uploads); TC260-metadata-only removes it ONLY on a flat footprint (its textured false fires -- ceilings/walls -- are what the fill smears). Do NOT loosen those gates.
+- `doubao_engine.py` / `jimeng_engine.py` / `samsung_engine.py` — thin `TextMarkEngine` subclasses: Doubao "豆包AI生成" (bottom-right), Jimeng "★ 即梦AI" (bottom-right), Samsung Galaxy AI "✦ Contenuti generati dall'AI" (bottom-LEFT, locale-specific — Italian variant calibrated). Detection matches the glyph silhouette (NCC); removal localizes the glyph blob to a solid dilated box (`extract_mask`) and hands it to the shared fill. Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. **Samsung detection is calibrated only for the Italian "Contenuti generati dall'AI" string** (a pre-existing limit, unchanged by the localize -> fill refactor but now surfaced because detection gates removal): non-Italian Samsung locales are not detected, and thus not removed, even though the fill mask itself is locale-independent; other locales need their own detection silhouette (the locale string font-rendered + calibrated on real positives), NOT an app capture.
+- `qwen_engine.py` — the Qwen (Alibaba Tongyi Qianwen) "千问AI生成" text mark (bottom-right), registered 2026-07-21 off the 117-frame TC260-producer cohort (`scripts/vendor_cohort_harvest.py` + `scripts/vendor_mark_calibrate.py`; the full calibration record is in `docs/module-internals.md`). Same `TextMarkEngine` subclass shape, but EVERY tuned number is measured on the cohort, not inherited from doubao: `scale_basis=short`, a **per-mark 2-rung `ladder=(0.78, 1.27)`** (the mark sits in two size modes ~0.124/~0.203 of the short side, ratio 1.64 -- wider than the shared ladder's 1.5625 span, so the best single fraction covers only 74.5% and the small mode falls into the comb's collapse zone; `TextMarkConfig.ladder` was added for this, default `(0.8, 1.0, 1.25)` unchanged for every other mark), a fitted locate box (the real mark sits ~0.025 of short off the right edge and doubao's 0.004-margin box clipped the first glyph, 0.73 -> 0.26 NCC), `alpha_height_frac` from the measured 0.260 aspect (not the silhouette's, not doubao's), gate **0.45** (clean p99 0.301; 83 of ~96 eyeballed visible marks fire = 86% recall of visible marks). **STRICT ONLY** (`provenance_ncc_factor` 1.0, no provenance mapping): the sub-gate band is dominated by non-Qwen banners (夸克 strip 0.274, 造点 mark 0.253), so a relaxed arm would be mostly false fills. **No rival margin** (0 cross-fires on 400 doubao / 298 jimeng / 286 clean frames at the gate; a 0.10 margin would cost ~10% of genuine qwen detections). A confident qwen detection suppresses the jimeng pill like doubao's does. Parity: 83/83 real marks detector-clean after cv2 fill. **Calibration trap for the next vendor:** the 2026-07-18 `present: []` clean labels are in the vocabulary of the REGISTERED marks only -- qwen-cohort frames visibly carrying 千问AI生成 sat in the "clean" arm and were its whole top tail (p99 0.69); `vendor_mark_calibrate.load_sets` now excludes every frame in ANY TC260 cohort.
+- `region_eraser.py` — universal region eraser (`erase` CLI) and the shared fill backend behind `watermark_registry.fill` for the visible localize -> fill removal. Three backends: `cv2` (default, no deps, the floor), `migan` (MI-GAN ONNX, extra `migan`, MIT, ~28 MB / ~0.19 s — the droplet-friendly tier, **the preferred default fill** when the extra is installed), `lama` (big-LaMa ONNX, extra `lama`, ~200 MB / ~4.7 GB peak — best quality, does not fit a minimal droplet, explicit opt-in only). Both `migan` and `lama` **crop a padded region around the mask** before inference and paste only masked pixels back, so peak RAM is bounded by the MARK size, not the image (`migan` ~0.6-0.9 GB regardless of upload size — feeding the whole frame scaled it to ~2.4 GB at 25 MP; `migan` feeds the crop at native resolution, `lama` resizes to its fixed 512²). **Measured end to end 2026-07-20** (`scripts/resource_ceilings.py`, fresh process per cell, 1 MP → 25 MP): `migan` 603 → 775 MB and `lama` 4679 → 4779 MB, both **flat in input size** — the crop-around-the-mask design holds and both documented figures reproduce. **`cv2` is the only backend that GROWS with the input** (74 → 440 MB, 5.9x) because it inpaints the full frame rather than a crop; still the cheapest tier, but size it for the largest upload accepted. Cold wall time 0.02-0.12 s (cv2) / ~0.6 s (migan) / ~3.8 s (lama), model load included. (The harness's own no-op check originally allocated a full-frame temp before reading peak RSS and inflated these by up to 17% at 25 MP; it now compares only the mask box. The conclusion survived re-measurement, the digits moved.) **MI-GAN mask polarity is INVERTED** (0=hole/255=known) vs this package's 255-erase convention; `erase_migan` inverts before feeding the model (feeding 255=hole regenerates the whole frame into stripes — corpus-validated). Both ONNX models download on first use, never bundled. The `erase` command keeps its own `--backend`/`--inpaint-method` (unchanged).
+- `invisible_watermark.py` — decodes the OPEN DWT-DCT watermarks (SD / SDXL / FLUX) via `imwatermark` (extra `detect`, pulls torch). Fragile two ways: (1) does not survive JPEG re-encode/resize; (2) **carrier-fragile on a broad class of pristine images** -- a clean encode->decode round-trip recovers 48/48 on chatgpt/firefly/random but FAILS (28-39/48, below the `_MATCH_48`=44 gate) on the FLUX fox, doubao, a flat FLUX generation, AND a clean synthetic flat fill with no watermark. The failure does NOT track texture; it goes with a degenerate **all-ones decode that is a CARRIER ARTIFACT, not a watermark** (synthetic clean image reproduces it). So `detect_invisible_watermark` is **positive-only**: trust a hit; a `None` is inconclusive unless a same-carrier positive-control embed first recovers >=44. Verified 2026-06-19; full caveat in `docs/watermarking-landscape.md`.
+- `trustmark_detector.py` — Adobe TrustMark open decoder (extra `trustmark`). Do NOT remove the JPEG re-encode false-positive gate — a lone TrustMark hit without it is almost always content noise.
+- `noai/watermark_remover.py` — `WatermarkRemover` with three diffusion pipelines selected by the explicit `pipeline` ctor arg, never inferred from `model_id`: `sdxl` (plain SDXL img2img), `controlnet` (SDXL + canny ControlNet, **the DEFAULT since 2026-06-09**), and `qwen` (Qwen-Image 20B MMDiT img2img, Apache-2.0, CUDA/cloud-class — best **text** preservation (incl. CJK); `_load_qwen_pipeline`/`_run_qwen`, bf16, no MPS fallback; call shape in the pure `_build_qwen_kwargs` using `true_cfg_scale`). Removal comes from the img2img `strength`; ControlNet only preserves text/face STRUCTURE — SynthID CAN survive controlnet on photoreal content at low strength. **Both SDXL loaders (`_load_pipeline`, `_load_controlnet_pipeline`) pass `add_watermarker=False`** — diffusers otherwise embeds an open "Stable Diffusion XL" DWT-DCT invisible watermark on EVERY SDXL output whenever `invisible-watermark` is installed (the `detect` extra), so a watermark REMOVER would re-stamp a detectable AI watermark and the cleaned output re-reads as AI (`identify` → "Open invisible watermark: Stable Diffusion XL"; corpus-observed on the SynthID sample before the fix). Only the pipeline accepts the kwarg, not the `ControlNetModel` sub-model; qwen is not SDXL and has no such watermarker. Regression: `tests/test_platform.py::TestNoReembeddedWatermark`. Qwen CERTIFIED oracle floors (2026-06-20): OpenAI **0.10** (seed-robust, clean on seeds 0-4), Gemini **0.25** (seed 0 verified, pin a seed — Gemini oracle rate-limits volume; higher than the controlnet Gemini floor 0.15). `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`), so `--pipeline qwen` gets the 0.25 Gemini floor automatically (the old manual `--strength 0.25` workaround is retired). `_build_qwen_kwargs` passes an explicit `height`/`width` from the input (floored to /16 via `_qwen_target_size`) — without it the pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (fixed 2026-06-20). **`qwen` is a MANUAL opt-in only — there is NO auto-router.** Measured (`scripts/fidelity_metrics.py`, OCR-CER / ArcFace / LPIPS / Laplacian-var, NOT eyeball): qwen beats controlnet on ONE niche only — **clean body text on a plain background, no faces** (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES (it always has) AND **display/decorative text in a scene** (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes, qwen re-renders and garbles them). So a content `--pipeline auto` router and a faces+text **mixed dual-pass** were prototyped and **DROPPED** (2026-06-20): on the canonical faces+text case controlnet wins every metric incl. text, so mixed loses; and "text→qwen" can't be auto-decided (it is body-vs-display text that matters, undetectable cheaply). qwen stays for callers who KNOW their content is clean-text-heavy and face-free. No face-restore extra ships, by validated decision (every restore approach looked MORE AI-generated). `remove_watermark(region=(x,y,w,h), region_feather=...)` runs the regeneration but feather-composites only the AI box back over the original (via `noai/tiling.feather_region_composite`), preserving the real photo elsewhere — the **AI-enhanced composite** path (`identify` `ai_source_kind == "enhanced"`); the box is supplied by the caller (a C2PA composite manifest carries no reliable machine-readable region, so we do not fabricate one).
+- `noai/tiling.py` — sliding-window tiled diffusion for large inputs (CLI `--tile`). `WatermarkRemover.remove_watermark` branches to `run_tiled` when `tile` is set AND the long side exceeds `tile_size`, refactoring the single-pass `_generate` into a per-tile `_generate_one` (the ControlNet edge map is rebuilt per tile inside it). Pure helpers `plan_tiles` (uniform-size tiles, last one flush to the edge) and `feather_weights` (strictly-positive separable taper -> partition-of-unity blend) are unit-tested without the model. Also home to `feather_region_composite(base, regenerated, box, *, feather)` — the pure region-targeted compositor for **AI-enhanced composites** (`ai_source_kind == "enhanced"`): blends the regenerated AI box back over the original with a feathered seam, leaving the real photo OUTSIDE the box pixel-exact. It backs `WatermarkRemover.remove_watermark(region=...)` (regenerate ONLY the AI region, not the whole frame); the no-model lossless region path stays `region_eraser.erase`. New tile/region-blend tuning goes in these pure helpers; do not inline blend math into the runner.
+- `auto_config.py` + the content-detection layer were REMOVED 2026-06-09; `--auto` is a deprecated no-op (controlnet is the default pipeline and the adaptive polish is ON by default and self-gates to a no-op where there is no detail deficit).
+- `upscaler.py` — optional Real-ESRGAN pre-diffusion super-resolution for small inputs (extra `esrgan`, spandrel only). Manual opt-in; the default `--upscaler` stays `lanczos` and the engine always falls back to Lanczos on absence/error. ESRGAN can degrade faces and thin text.
+- `image_io.py` — Unicode-safe cv2 IO (issue #17). Every cv2 file read/write in the package routes through `imread`/`imwrite`; do not call `cv2.imread`/`cv2.imwrite` directly. `to_bgr(image)` is the shared channel normalizer — use it instead of inlining `cvtColor` branches. `read_bgr_and_alpha`/`write_bgr_with_alpha` (+ `ALPHA_FORMATS`) are the alpha-preserving IO helpers shared by the CLI and the library `api` (moved here from cli so both use ONE implementation; the write MUST NOT zero alpha in the mark bbox — issue #30 white box). **`write_bgr_with_alpha` RETURNS `imwrite`'s success flag and every caller must check it** — `imwrite` is contractually non-raising, so that bool is the ONLY signal the file was not created. The wrapper used to return `None` and swallow it, so a write that a read-only directory silently prevented ran on to `output.stat()` and died with a bare `FileNotFoundError` traceback pointing at the stat rather than the write (Tier E, 2026-07-20). **Three different layers each needed their own handling, and the right one is NOT the same everywhere:** the single-image commands (`visible --mark <name>`, `erase`, `all`) write through the shared `cli._write_output_or_exit`, which exits; `api._write_visible_result` (behind `visible --mark auto`) RAISES `OSError` so a library caller gets an accurate error instead of a confusing `FileNotFoundError` from the downstream metadata strip; and the **batch** sites raise too, never `SystemExit` — the batch loop catches per-image exceptions, counts them and exits non-zero, so aborting there would kill the whole run. Discarding the flag in batch made a read-only output directory produce **zero files and still exit 0** — silent data loss contradicting the documented batch contract. Regression: `tests/test_cli_robustness.py`. cv2/numpy import lazily, so importing `image_io` is cheap. **`imread` has a Pillow fallback (`_pil_read`) for HEIC/AVIF**: cv2 can't decode those containers, so when its decode returns None it opens via Pillow (AVIF native; HEIC via the core `pillow-heif` dep, whose libheif also covers AVIF) and converts to the same BGR/BGRA layout the flags imply — so the pixel/removal path reads iPhone HEIC and AVIF, not just the metadata path. Normal PNG/JPEG/WebP never reach the fallback. Corpus-verified: 54/55 HEIC+AVIF now decode (the 1 miss is a truncated upload). **`imwrite` PRESERVES the input format at max quality** ("work with originals"): the removal only touches the mark footprint (cv2 AND MI-GAN fills composite over the original — untouched pixels are bit-exact), so the container re-encode must not degrade the rest. JPEG is written at quality 100 / 4:4:4 (no chroma subsampling) — PSNR ~55 dB vs the old default-95's ~48; HEIC/AVIF write via Pillow (`_pil_write`) since cv2 has NO encoder for them (writing `.heic` via cv2 RAISES — a HEIC input used to crash on save). `imwrite` never raises (catches `cv2.error`). **`api.remove_visible` copies the original bytes verbatim on a no-op** (nothing removed + same output format) rather than a lossy re-encode, so a clean image round-trips byte-identical. `noai/constants.SUPPORTED_FORMATS` now includes `.heic`/`.heif`/`.avif` alongside png/jpg/jpeg/webp (pillow-heif is core, so read+write both work), so `batch` discovers them and the CLI no longer warns on an iPhone HEIC; JPEG-XL stays OUT (metadata/strip-only, no pixel decoder without pillow-jxl). **The invisible/SynthID path is inherently a full-frame diffusion regeneration (every pixel changes by design — you cannot "work with originals" there), but it no longer piles gratuitous re-encodes on top:** `watermark_remover` saves the regenerated output through `image_io.imwrite` (not raw `PIL.save`, which defaults to JPEG q75), `invisible_engine` writes its pre-diffusion temp as lossless PNG (not a re-compressed copy of a JPEG input), and the output metadata strip goes through the byte-level `metadata.remove_ai_metadata` (see its bullet) which for JPEG does NOT re-encode the DCT at all — pixels stay bit-identical.
+- `api.py` — the high-level convenience API, re-exported lazily at the package top level via `__init__.__getattr__` (PEP 562, so `import remove_ai_watermarks` stays cheap): `remove_visible(source, output=None, *, sensitivity="auto", backend="auto", strip_metadata=True, write_noop=True) -> (result_bgr, [labels])` (source = path OR BGR ndarray; a PATH auto-reads metadata provenance and preserves alpha, an ARRAY does neither; `write_noop=True` writes a clean passthrough copy when nothing is removed, `False` leaves `output` untouched so a "no mark = produce nothing" caller like the CLI `visible` command does not clobber a pre-existing file there) and `visible_provenance(path) -> frozenset[str]` (the single metadata→vendor-keys mapper; `cli._visible_provenance` is a thin None-guarded wrapper over it). **`remove_visible` is the ONE path the CLI and library share** — `cli.cmd_visible`'s `--mark auto` branch delegates entirely to it (read → provenance → `remove_auto_marks` → write → `strip_metadata`), so there is no CLI-vs-library drift; `strip_metadata` defaults True to match `visible --strip-metadata`. This is where a library caller should start — NOT the engines directly (`GeminiEngine`/`TextMarkEngine` have no `remove_watermark` any more; removal is registry `remove_auto_marks`/`KnownMark.remove`; the old single-strongest `best_auto_mark` is gone — removal takes EVERY mark). `identify` is NOT top-level re-exported (it collides with the `identify` submodule); use `from remove_ai_watermarks.identify import identify`.
 
-**RESOLVED 2026-05-29: black+gray Doubao captures were obtained and the exact reverse-alpha is built** (`doubao_engine.remove_watermark_reverse_alpha`, `assets/doubao_alpha.png`; see the `doubao_engine.py` bullet above). The captures (`data/doubao_capture/captures/`, gitignored) confirmed the alpha-composite model: on black `captured = a*logo`, the black/gray pair solves `a` per-pixel (`a_max≈0.65`, logo near-white), and the white capture cross-validates. A single 2048 capture suffices: sub-pixel NCC alignment (`_aligned_alpha_map`) registers the alpha glyph to the real mark, so it works at any resolution (verified on the 1773×2364 3:4 corpus size), not just 2048. The notes below (the failed content-image distillation) are retained as the record of why captures were necessary.
+For the Doubao alpha-distillation history (why content-image reverse-alpha distillation fails by physics and controlled captures were required), see `docs/research-doubao-distillation.md`.
 
-**Conclusion (historical): pure reverse-alpha distilled from content images does NOT work, and the blocker is the WRONG kind of data, not too little of it.** The earlier framing ("need ~5-8 PRISTINE same-resolution originals") is obsolete -- `data/spaces/originals/` holds plenty. Curate them with `DoubaoEngine.detect` + an NCC filter against a clean glyph template, keeping only marks at offset ≈ (0,0): that yields e.g. **15 pixel-aligned 2048² marks** (sub-pixel drift, not the ±50 px the old lossy/mixed-res scrapes had), plus 1086x1448 / 1792x2400 clusters. With those, LaMa-clean `O` + weighted-LS (and per-pixel I-on-O regression) for `α` (+ logo colour) was tried end-to-end and **still leaves a persistent ghost outline.**
+## Watermarking landscape
 
-Diagnosed why, empirically (cached stacks, `/tmp/doubao_distill`): (1) the mark is a clean white overlay with **no dark halo** -- over glyph pixels ~54% are brighter than the clean bg, only ~4% darker -- so the white-logo model `I=(1-α)O+α·255` is correct; (2) but content backgrounds are almost never dark *under* the mark (median darkest available bg over glyph pixels = **58/255**; only ~13% of mark pixels are ever observed on a bg < 40), so on bright backgrounds the equation is ill-conditioned and `α` is unidentifiable; (3) LaMa's `O` is a plausible **hallucination**, not the true pre-mark background, which compounds the error, and per-pixel regression on ~15 obs overfits into colour noise.
-
-**Why Gemini's engine is clean (verified in GeminiWatermarkTool `src/core/watermark_engine.cpp`): its alpha map is the watermark stamped on a PURE-BLACK background**, where `watermarked = α·255 + (1-α)·0 = α·255`, so `alpha = capture/255` exactly -- no estimation. (`gemini_bg_*.png` is literally the sparkle in grey on black.) So the real Doubao unlock is the same controlled capture, **not more content images**. Black/white/gray seeds exist (`data/doubao_capture/seeds/seed_*_1x1_2048x2048.png`); a capture run (feed a black seed through doubao.com edit mode, download the *original*) was requested from the #13 reporter 2026-05-29. With ~2-3 black captures we get `α = capture/255` for free, Gemini-quality.
-
-**Until black captures arrive, the shipped direction is precise canonical glyph mask + inpaint (cv2 default, lama optional), NOT reverse-alpha.** The consensus glyph silhouette across the aligned marks distills cleanly (proto: a tight "豆包AI生成" strip, width ≈ 0.156 × image-width) and is good both as an exact inpaint mask and as an NCC localiser -- the latter also fixes the #23 detector false-positives (match the real glyph shape, not any bright low-saturation corner). Do **not** retry content-image reverse-alpha: it is data-limited by physics (no dark-background observations), not by effort.
-
-## Watermarking landscape (research 2026-05-24)
-
-Who embeds what, and whether it is locally detectable (so we know which gaps are fillable). See `identify.py` for what we read.
-- **Locally detectable (open decoder, no key/API):** Stable Diffusion / SDXL / FLUX via `imwatermark` DWT-DCT (now covered by `invisible_watermark.py`). FLUX uses the same library (`black-forest-labs/flux2` `src/flux2/watermark.py`, 48-bit `0b001010101111111010000111100111001111010100101110`); SDXL is the diffusers `WATERMARK_MESSAGE` (`0b101100111110110010010000011110111011000110011110`). Caveat: fragile to re-encoding.
-- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), and **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`). Still unsampled: Canva (its downloads are re-encoded design *exports* that strip C2PA, so a Canva "positive" is inconclusive — skipped), Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag). **Samsung Galaxy AI** (Generative Edit / Sketch to Image / Portrait Studio on Galaxy S23 FE / S24 / S25, One UI 7+) signs C2PA as "Samsung Galaxy" with the standard `trainedAlgorithmicMedia` source type AND a proprietary `genAIType` marker; verified on real signed files 2026-05-29 (the standard scan catches the source type; `genAIType` additionally catches a Galaxy S24 file that omits it). **ASUS Gallery** also signs edited photos as C2PA (`com.asus.gallery`) but with no AI source type — a signer, not an AI marker. **Black Forest Labs (FLUX)** API output signs C2PA: `claim_generator_info "Black Forest Labs API"` + a `c2pa.ai_generated_content` assertion + `trainedAlgorithmicMedia` (issuer `b"Black Forest Labs"` added to `C2PA_ISSUERS`, platform "Black Forest Labs (FLUX)"). **ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. All three verified on real signed files 2026-05-29.
-- **EXIF/XMP generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal).
-- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).** Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC** — only EXIF `Artist` = a UUID and EXIF `ImageDescription` = `Signature: <base64>` (a crypto signature, unverifiable locally without xAI's public key). This empirically kills the earlier unverified "xAI signs C2PA as xAI" lead — xAI is not even a C2PA member. `exif_generator` misses it (neither field holds an `AI_GENERATOR_TOKENS` token), so a dedicated detector `xai_signature(path)` matches the pair (`ImageDescription ~ ^Signature: [A-Za-z0-9+/=]{64,}` AND UUID `Artist`); wired into `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify` (signal `xai_signature`, platform "xAI (Grok / Aurora)"). **Format confirmed stable across n=3 genuine generations:** exactly three EXIF tags (`Artist`, `ExifOffset`, `ImageDescription`), `Signature:` prefix constant, base64 payload 300-1004 chars. Two capture facts: (a) the `Artist` UUID **equals the public image id** in the asset URL (`https://imagine-public.x.ai/imagine-public/images/<uuid>.jpg`), so it is NOT a private per-user secret — only the `Signature` blob is; (b) the Grok web-UI image is a re-encoded **WebP with no signature** — the EXIF survives only in the *original* JPEG (download button or that public tokenless URL), which is why screenshots / re-encodes are metadata-stripped. A real fixture `data/samples/grok-1.jpg` plus **synthetic** JPEG fixtures (fake UUID + fake `Signature:` blob) cover the detector; never add a real Grok image carrying private content (the repo is public). **Stripped on removal too:** `remove_ai_metadata` now calls `_scrub_ai_exif` on the JPEG EXIF, which deletes the xAI Signature+UUID-Artist pair **and** any `Software`/`Make`/`Artist`/`ImageDescription` tag holding an `AI_GENERATOR_TOKENS` token (so Ideogram's `Make="Ideogram AI"` is scrubbed too), while keeping genuine camera/editor EXIF. The shared `_is_xai_signature_pair` helper (module-level compiled regexes) is the single source of truth for the pattern, used by both `xai_signature` and `_scrub_ai_exif`. (AVIF/HEIF/JXL still strip only C2PA boxes via `isobmff`, not EXIF — unchanged.)
-- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). **Doubao** (ByteDance) uses it (verified on the real #13 sample 2026-05-25; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **three serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on the corpus 2026-05-30) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on the corpus 2026-05-28, `ContentProducer="doubao"`), **and** a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on the corpus 2026-05-30, `ContentProducer="001191440300708461136T1308L"`). Both generic forms (the PNG chunk and the bare `{"AIGC":...}` object) are gated on at least one TC260 field (`_TC260_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
-- **HuggingFace-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** HuggingFace Jobs / Spaces stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID (3 on the corpus 2026-05-28, no other signal). It marks the *hosting job*, not a model — most commonly diffusion output — so it lifts an Unknown verdict to a tentative AI via `hf_only` (parallel to the visible sparkle) but never overrides a hard metadata signal; `_HF_JOB_CAVEAT` states the limit (job, not model; not proof of AI pixels). Stripped on removal (the PNG save whitelist keeps only `STANDARD_METADATA_KEYS`, so `hf-job-id` and the `AIGC` chunk are both dropped). The exact writer is not authoritatively documented (HF Jobs are generic GPU jobs), hence medium not high.
-- **No detectable signal on download (correctly reported `unknown`):** **Recraft** (PNG export is a re-encoded design export — strips everything), **Krea hosting FLUX 2** (no imwatermark despite FLUX — the host omits the encoder, same as Stability's hosted SDXL), and Midjourney (embeds nothing). Lesson: the imwatermark detector only fires on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL), not from re-hosts (Krea/Stability) or re-encoded exports (Recraft/Canva).
-- **Invisible but NOT locally detectable (proprietary, API/oracle only — same wall as SynthID):** Amazon Titan Image Generator + Nova Canvas (Bedrock `DetectGeneratedContent` API), Kakao (new SynthID image adopter, May 2026), NVIDIA Cosmos (SynthID video). No local detector possible; treat like SynthID.
-- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest (Digimarc/Imatag/Steg.AI/...) stay name-only (proprietary decoders).
-- **Built 2026-05-26 (this batch):** soft-binding `alg` vendor detection; IPTC Photo Metadata 2025.1 AI-disclosure fields (`AISystemUsed` etc.); **video C2PA metadata** detect + strip for MP4/MOV/M4V (free — `isobmff.py` is format-agnostic, MP4 is ISOBMFF); Adobe TrustMark open decoder. NOT done (out of cheap reach, per the feasibility review): visible video-logo removal (needs a video frame pipeline) and audio (SynthID/ElevenLabs/Resemble/Suno all oracle-only or unmarked). **Box detection window — now handled (v0.6.8):** detection no longer relies on a fixed first-MB read. `metadata.scan_head(path, size)` reads the first `size` bytes and, for ISOBMFF, appends the payloads of late provenance boxes found by `isobmff.scan_c2pa_region` (a file-seeking top-level box walker that skips past `mdat` by size without reading it), so a C2PA/AIGC/IPTC manifest placed AFTER a large `mdat` in a streaming/non-faststart MP4 is now caught. Every C2PA/marker byte scan (`has_ai_metadata`, `aigc_label`, `iptc_ai_system`, `synthid_source`, `exif_generator` XMP, `get_ai_metadata` soft-binding, and `identify`) goes through `scan_head`; it is behavior-neutral for non-ISOBMFF inputs (exactly `f.read(size)`). **Meta-box XMP removal — now handled (v0.6.9):** an AI-label XMP packet stored as a meta-box `mime` item (HEIF/AVIF; out of reach of the top-level box stripper) is blanked in place by `isobmff.blank_ai_xmp_packets` — it locates the packet by its `<?xpacket begin … end?>` delimiters and, if it carries an AI marker (`_AI_LABEL_MARKERS`), overwrites it with spaces of the SAME length, so box sizes / `iloc` offsets stay valid and the coded image is untouched (selective: plain non-AI XMP is left alone, mirroring the top-level uuid logic). Wired into `remove_ai_metadata`'s ISOBMFF branch after `strip_c2pa_boxes`. The remaining gap is an `Exif` meta-box *item* (rare; the AI labels are XMP) — still needs `iinf`/`iloc` surgery or exiftool.
-- **Regulatory driver (context, not a code change):** AI-content labeling mandates are expanding, which pushes more generators toward exactly the C2PA + watermark signals we read. The full per-jurisdiction table lives in README "## Legal" -- keep it there, not duplicated here. Newly added + primary-source verified 2026-05-26: **EU AI Act Article 50** machine-readable marking applicable **2026-08-02** (verified against the article text); **South Korea AI Framework Act Art. 31(3)** in force since **22 January 2026** (verified via Kim & Chang + FPF/Korea Times; Enforcement Decree accepts an invisible-watermark label); **California AB 853** (amends the CA AI Transparency Act) latent-disclosure duty operative **2026-08-02**, requiring a disclosure "permanent or extraordinarily difficult to remove" (verified against the leginfo bill text -- this is the exact disclosure our tool strips); **India IT Amendment Rules 2026** in force **2026-02-20** (verified via Chambers), which prominently-label + permanent-provenance-id all synthetic media AND **expressly prohibit removing/suppressing the label or metadata** -- the first major all-content removal ban outside China. **Removal liability (README "## Legal" disclaimer):** the tool is lawful general-purpose software; liability sits with the remover and is intent-gated -- downstream acts (fraud/deception/IP), plus US DMCA 17 USC 1202 (removing copyright-management info to conceal infringement), plus the removal-as-such bans in China + India. When extending the README table, verify each date/article against the statute/bill text before committing, not against search summaries.
+Who embeds what (C2PA / IPTC / EXIF / TC260 AIGC / xAI signature / open and proprietary invisible watermarks), whether each is locally detectable, the C2PA 2.4 durable-credentials implications, and the regulatory driver table live in `docs/watermarking-landscape.md` (research 2026-05-24, updated through 2026-06-10). Read it before adding a new `identify` signal, vendor token, or metadata marker. See `identify.py` for what we read today.
 
 ## Known limitations
 
-- `invisible` pipeline processes at **native resolution by default** (`max_resolution=0`), matching the hosted raiw.cc backend (fal fast-sdxl, no pre-downscale). The old forced downscale-to-1024 -> upscale-back round-trip was the main quality loss (issue #10) and is gone; at strength ~0.05 SDXL img2img does not need the ~1024 downscale. `--max-resolution N` re-introduces an opt-in long-side cap purely to bound GPU/MPS memory on very large inputs (it reintroduces the lossy round-trip). For huge images that OOM at native, tile-based diffusion is still the proper long-term fix. **Concrete MPS data point (verified 2026-05-25 on a 1254x1254 gpt-image SDXL run, fp32, 20 GB MPS ceiling):** native res OOMs at the *UNet* step (peak ~17 GiB), not only the VAE decode, and the auto-fallback in `img2img_runner` reloads on CPU and finishes (slow, ~13 min) -- the output is still weight-identical and defeats SynthID, so "looks hung/crashed" on Mac is usually this CPU fallback, not a pipeline error. Adding `enable_vae_tiling()` alone does NOT prevent it (the peak is the UNet, not the VAE). The fast Mac workarounds are fp16 on MPS (roughly halves memory) or `--max-resolution` to cap the long side; neither is wired as the default. The native-vs-downscale decision lives in the pure helper `invisible_engine._target_size(w, h, max_resolution)` (returns `None` for native, a clamped target tuple otherwise) so it is unit-tested (`tests/test_invisible_engine.py::TestTargetSize`, the #10/#15 regression guard) without loading the model -- keep that logic in the helper, don't re-inline it.
+Compact list. Full measurements, incident history, and oracle-validation runs live in `docs/known-limitations.md` — **read the relevant section there before changing the diffusion pipelines, strength defaults, resolution handling, or metadata coverage.**
+
+- **Visible-mark fill quality is background/backend-dependent.** The fill only touches the mark footprint (no outside-box damage) and whether the mark is removed is fill-independent — cv2/MI-GAN/LaMa all strip the shape; only the recovered region's *quality* differs. Flat backgrounds: all clean (cv2 often crispest). Textured/regular-structured (fabric, grid): cv2 smears, MI-GAN can ghost/hallucinate, LaMa best. The old reverse-alpha recovered true pixels so it was sometimes cleaner on structure, but localize -> fill trades that for robustness (moved/re-rendered marks, no per-mark capture); `auto` = LaMa > MI-GAN > cv2 with a one-time cv2-fallback warning. Head-to-head vs v0.12.1 on the full visible set: doubao/jimeng identical (100%/100%), gemini strict coverage a few points lower (the metadata-stripped faint ones now mostly recovered by the default white-core rescue in the gemini FP gate), clearance ~98% both. Detail in `docs/known-limitations.md`.
+- `invisible` processes at native resolution for inputs >= 1024px long side and auto-upscales smaller inputs to a 1024px floor (`--min-resolution 0` disables; `--max-resolution N` is an opt-in cap to bound GPU/MPS memory). MPS OOM is memory-tier dependent, not a hard limit: ~24 GB unified memory falls back to CPU (slow but weight-identical output), 32 GB runs native on MPS. The native-vs-cap-vs-floor decision lives in the pure helper `invisible_engine._target_size` — keep the logic there, unit-tested without the model. For large inputs that OOM, `--tile` is the **lossless** alternative to `--max-resolution`: sliding-window diffusion at native resolution, each tile near SDXL's 1024 training size, feather-blended over the overlap (`noai/tiling.py`). It only engages when the long side exceeds `--tile-size`; the geometry (`plan_tiles`) and the blend window (`feather_weights`) are pure and unit-tested (`tests/test_tiling.py`). Caveat: each tile is an independent low-strength regeneration, so at the certified removal strengths (0.20-0.30) tile drift is minimal but not zero; tiling is a memory workaround, not a quality upgrade over a single native pass.
+- fp16 VAE black-output (issues #29/#41): the fp16-fixed SDXL VAE (`madebyollin/sdxl-vae-fp16-fix`) is swapped in for the default SDXL checkpoint on cuda/xpu fp16, plus a model-agnostic backstop that detects a degenerate (all-black) fp16 output and re-runs once in fp32. cpu/mps run fp32 and never reproduce the bug.
 - Pyright first run is slow (2-3 min) due to ML deps (torch/diffusers/transformers stubs); full-project `uv run pyright` can stall for many minutes — scope it to changed files.
-- `ultralytics` monkey-patches `PIL.Image.open` and tries to autoload `pi_heif`. When `pi_heif` is missing, opening files raises `ModuleNotFoundError`, not `UnidentifiedImageError`. Code that opens user-supplied or unknown-format files should `except Exception`, not just `OSError`/`UnidentifiedImageError`.
-- **rich `console.print` parses `[word]` as a style tag and silently drops unknown ones.** A literal bracketed token in a print string disappears: `pip install 'remove-ai-watermarks[gpu]'` rendered as `...remove-ai-watermarks'` (the `[gpu]` extra eaten), which sent users a broken install command (surfaced via #19). Escape the literal bracket as `\[gpu]` (in a normal Python string that is `"\\[gpu]"`) in any rich string carrying user-facing brackets. Regression-guarded by `tests/test_cli.py::TestGpuHintMarkup`.
-- Metadata detection for AVIF/HEIF/JPEG-XL relies on a binary scan for `C2PA_UUID` + `IPTC_AI_MARKERS`, plus EXIF `Software` / XMP `CreatorTool` generator tags via `metadata.exif_generator` (validated with synthesized AVIF/JPEG fixtures + an XMP raw-scan fixture). C2PA removal in those containers is implemented via `noai/isobmff.py` (top-level ``uuid`` / ``jumb`` box stripper, no re-encoding), which now also drops a top-level XMP ``uuid`` box that carries an AI label (matched by AI-marker content, not by the XMP UUID, so byte-order-robust) and covers MP4/MOV/M4V/M4A by content sniff. **Non-ISOBMFF audio/video removal is via ffmpeg** (`_FFMPEG_STRIP_EXTS` -> `_strip_with_ffmpeg`): WebM/Matroska (EBML), MP3 (ID3), WAV/FLAC/OGG (RIFF/Vorbis) are stripped losslessly with `ffmpeg -map_metadata -1 -map_chapters -1 -c copy` (codec data untouched). Requires ffmpeg on PATH; raises `RuntimeError` if absent or if ffmpeg can't parse the file. Verified end-to-end (a real ffmpeg-made WAV/MP3 with a `title=Suno AI` tag -> tag gone, audio bytes preserved). **Meta-box XMP now handled (`isobmff.blank_ai_xmp_packets`, v0.6.9):** an AI-label XMP packet stored as a meta-box `mime` item (AVIF/HEIF) is blanked in place (overwritten with spaces of the same length, so `iloc` offsets and the coded image stay valid). **Still NOT built:** an `Exif` *item* inside the `meta` box (rare -- AI labels are XMP) needs full `iinf`/`iloc` surgery (offset rewrite) with corruption risk -- exiftool (R/W/C for HEIC/AVIF EXIF+XMP, verified on exiftool.org 2026-05-27) would do it but is a non-installed binary dep, so it stays a documented gap. **Audio watermark DETECTION (Resemble PerTh) was evaluated and NOT built (2026-05-26):** `resemble-perth`'s `PerthImplicitWatermarker.get_watermark()` returns a raw bit-array with **no presence/confidence flag** (clean audio decodes to arbitrary bits too), so reliably distinguishing watermarked-from-clean needs either Resemble's fixed payload or a confidence API -- neither is public, and there's no real Resemble sample to calibrate against. Same wall-class as the SynthID pixel detector: the decode exists, reliable presence-detection does not. (perth's top-level `PerthImplicitWatermarker` is also gated to None unless `librosa` is importable.)
-- **SynthID detection is metadata-only.** There is no reliable *local* detector of the SynthID *pixel* watermark — Google's decoder is proprietary, no public spec or API (only a waitlisted portal). Authoritative confirmation: Google DeepMind's own paper "SynthID-Image: Image watermarking at internet scale" (Gowal et al., arXiv:2510.09263) states the verification service is restricted to "trusted testers" and does not release detector weights or a reproducible algorithm — so a local pixel detector is infeasible by design, not just unbuilt. https://arxiv.org/abs/2510.09263 We detect SynthID by its C2PA companion (`synthid_source` / `SYNTHID_C2PA_ISSUERS`), which is reliable while the manifest is intact but says nothing once C2PA is stripped. **Surface-dependent blind spot (verified 2026-05-24):** the same Google model emits different metadata per surface -- the Gemini *app* wraps outputs in Google C2PA, but the *API/playground* (AI Studio, Nano Banana / gemini-2.5-flash-image) emits the SynthID *pixel* watermark (confirmed via the Gemini-app oracle) + the visible sparkle but **no C2PA/IPTC at all**, so `synthid_source` returns None despite SynthID being present. Only the pixel oracle or the visible-sparkle detector catches those. (Meta AI is another surface mismatch: it writes the IPTC `digitalSourceType=trainedAlgorithmicMedia` marker, not C2PA and not SynthID.) Google→SynthID is long-standing; OpenAI→SynthID is confirmed by OpenAI's Help Center (ChatGPT/Codex/API "include both C2PA metadata and SynthID watermarks", updated 2026-05-21) but time-gated (pre-rollout OpenAI images carry C2PA without SynthID), so the OpenAI verdict is hedged "likely". Oracles: Gemini app "Verify with SynthID" (Google), openai.com/verify (OpenAI). The spectral phase-coherence approach from `github.com/aloshdenny/reverse-SynthID` was evaluated (May 2026) and **does not work for real-content detection**: on its own shipped codebook + validation set, watermarked and cleaned images were indistinguishable (conf within noise, cleaned often higher); it only fires on pure-black 1024x1024 reference images at exact resolution (the controlled case it was calibrated on). The README's "90% / conf=0.91" reproduces only in that lab condition. Do not build a production detector on it; if revisited, it is experimental/diagnostic only and needs a per-resolution, per-model reference corpus. A from-scratch gpt-image pilot (2026-05-24) confirmed this independently: 5 independent solid-black gpt-image outputs share a near-identical fixed signature (pairwise residual correlation **0.92**, avg-template retains 97% energy), so the watermark/carrier IS strongly present and consistent on flat content — but the carrier frequencies extracted from it do NOT discriminate real content (carrier-to-random ratio: cleaned 1.86 > watermarked 1.53; a non-gpt-image image scored highest at 3.67). The signature drowns in content texture. Net: a perfectly consistent solid-color signature still yields no real-content pixel detector with magnitude/carrier methods. A corpus discrimination test (2026-05-24, `scripts/synthid_pixel_probe.py`, raw zero-mean residual NCC) independently re-confirms this: at matched resolution, SynthID positives do NOT cluster apart from negatives (within-Gemini 0.07; at 1024 px pos-vs-neg >= pos-vs-pos). The only high correlations were near-duplicate *content* (5 ChatGPT renders of one prompt at ~0.92, while a distinct ChatGPT image scored ~0 against them) — content, not a carrier. The probe is solid-fills-only and EXPERIMENTAL/DIAGNOSTIC; do not use it on real content. **Correction (deeper re-examination 2026-05-25):** the carrier IS real on solid fills — the earlier "no carrier" was a *method* artifact of using spatial / FFT-magnitude NCC, which can't see it. The carrier is a fixed *phase* at specific low frequencies, so the right metric is **per-bin phase coherence**. On 8 white `gemini-2.5-flash-image` fills (generated via the reverse-SynthID trick: identity-edit prompt "Recreate this image exactly as it is" on a synthetic pure-white PNG — this bypasses the recitation block that rejects text prompts for pure colors), phase coherence at the white carriers `(0,±7..±12,±20..±23)` = **0.86** vs **0.31** random; single-image leave-one-out phase-match **+0.83** vs real photos **-0.24**. (Black `2.5-flash` fills clip to std≈0 — SynthID can't push values below 0, so no carrier in black; the repo's dark carriers come from nano-banana-pro.) **But it does not generalize:** (a) carriers are model-version + resolution + color specific — the repo's v4 codebook (built for `gemini-3.1-flash-image-preview` + `nano-banana-pro-preview`) scores ~0.527 on my 2.5-flash white fills, indistinguishable from negatives (~0.50), i.e. carriers shift across model versions and need a per-model codebook; (b) on real content (30 `2.5-flash` images) the carrier collapses — set phase coherence at carriers 0.37 ≈ random 0.42, and the repo's v4 deteLine truncated
-- **External AI-vs-real classifier models are out of scope (decided 2026-05-24).** Generic HuggingFace detectors (`Organika/sdxl-detector` Swin Transformer, `umm-maybe/AI-image-detector`, and fine-tunes) exist and report ~0.98 on their *own* SDXL-vs-real validation sets, but they are per-generator and the model cards themselves note degraded accuracy off-distribution; they are untested on gpt-image / Gemini Nano Banana (the metadata-stripped surfaces we care about), and our own light SDXL pass would likely defeat them the same way it defeats SynthID. Detection here stays local + signal-based (metadata + visible sparkle); do not add a bundled classifier dependency.
-- **SynthID v2 vs default pipeline:** the SDXL-based default profile (since May 2026) defeats SynthID v2. **Verified end-to-end (May 2026):** local SDXL run on a Gemini 3 Pro output, checked via the Gemini app's "Verify with SynthID" feature, returned "no SynthID watermark detected". Also confirmed against **OpenAI's** SynthID (2026-05-23): a fresh ChatGPT/gpt-image output read "SynthID detected" on openai.com/verify before the local SDXL run and "SynthID not detected" after (corpus regression chain: pos `4ef377bd` -> cleaned `47188e88`). The same configuration is used in raiw-app production (`fal-ai/fast-sdxl/image-to-image`, strength 0.05, steps 50, guidance 7.5, no pre-downscale). fal's own `llms.txt` for `fast-sdxl` names the base checkpoint as `stabilityai/stable-diffusion-xl-base-1.0` (verified 2026-05-25) -- the exact checkpoint the local CLI defaults to (`DEFAULT_MODEL_ID`). So the local `invisible` default is weight-for-weight identical to prod; "fast-sdxl" is fal's optimized serving, not different weights. After the native-resolution fix the local pipeline matches prod on weights + strength + steps + guidance + resolution. SD-1.5 dreamshaper at 768 px was previously the default and does NOT defeat v2 — verified empirically against the same feature (strength 0.04, 0.10, and elastic warp α∈{5,8} all flagged positive). That SD-1.5 path was removed; only `default` (SDXL) and `ctrlregen` profiles remain. **Scope of the claim: defeating the SynthID verifier is NOT the same as forensic invisibility.** "Removing the Watermark Is Not Enough: Forensic Stealth in Generative-AI Watermark Removal" (arXiv:2605.09203, 2026-05) shows that six removal attacks across four families (UnMarker, CtrlRegen+, WatermarkAttacker, etc.) all leave forensic traces: independent detectors flag *removal-processed* images vs genuinely-clean ones at **>98% TPR at 1% FPR**. So our SDXL pass makes the oracle read "SynthID not detected," but the output can still be classifiable as "an image that went through a removal pipeline." Do not over-claim "indistinguishable from a real photo." https://arxiv.org/abs/2605.09203
+- A third-party PIL plugin autoload (e.g. an HEIF/AVIF plugin) can raise a non-OSError (`ModuleNotFoundError`), not `UnidentifiedImageError`, when opening a file. Code that opens user-supplied or unknown-format files should `except Exception`, not just `OSError`/`UnidentifiedImageError`.
+- rich was dropped: the CLI + analysis scripts print plain text (`click.echo` / the `scripts/_plain_console.py` shim). `rich` is NOT a dependency — importing it breaks the core+dev CI sync; new scripts must use the shim. No Unicode glyphs / colors / progress bars in CLI output by design.
+- HEIC/AVIF are decodable on BOTH paths now: the pixel/removal path via the `image_io.imread` Pillow fallback (+ core `pillow-heif`), and metadata detection via a plugin-free binary scan. C2PA removal in those containers (and MP4/MOV/M4V) is `noai/isobmff.py`; JPEG-XL stays metadata/strip-only (Pillow can't decode it without `pillow-jxl`, not a dep). Non-ISOBMFF audio/video (WebM/MP3/WAV/FLAC/OGG) strips losslessly via ffmpeg on PATH. On the ISOBMFF path `remove_ai_metadata` routes to the container branch and never runs the JPEG `_scrub_ai_exif`, so `isobmff.blank_ai_exif_tokens` is the ONLY EXIF scrubber there and must stay in PARITY with it: it blanks **in place** (same-length space overwrite, piexif-validated so a coincidental II/MM run in pixels is ignored — no `iinf`/`iloc` surgery, mirrors `blank_ai_xmp_packets`) an AI-generator token in `Software`/`Make`/`Artist`/`ImageDescription`, the China TC260 `{"AIGC":{...}}` block in `ImageDescription`/`UserComment` (via `_is_aigc_exif_value`), AND the xAI/Grok `Signature:` + UUID-`Artist` pair — leaving camera/editor EXIF intact. Still NOT built: Resemble PerTh audio detection (no presence/confidence flag exists).
+- **SynthID technical reference: `docs/synthid.md`** — primary-source-cited doc covering mechanism (post-hoc encoder/decoder pair, 136-bit payload at 512x512, pixel-space, model weights NOT modified), robustness numbers (arXiv:2510.09263: ~99.98% TPR@0.1%FPR across 30 transforms including JPEG/crop/resize/color/noise), removal attacks and forensic detectability (arXiv:2605.09203: all 6 attacks detectable at >98% TPR@1%FPR), detectability limits (no public decoder, metadata-proxy only), oracle scope, and adoption landscape. Read that doc first before adding notes here.
+- **SynthID detection is metadata-only.** No local pixel detector is possible by design (Google's decoder is proprietary, trusted-testers only); we read the C2PA companion proxy, which goes quiet once metadata is stripped — a quiet proxy is not proof the pixel watermark is gone. Each vendor has its OWN oracle and it detects only that vendor's content: the Gemini app "Verify with SynthID" for Google, `openai.com/verify` for OpenAI. **Validate the OpenAI arm FIRST** — `openai.com/verify` is more accessible (fewer per-check restrictions) and the strongest automation candidate (Playwright / Chrome MCP); the Gemini flow is more manual. Ordering/throughput choice, not a substitution (see `docs/synthid.md`). SynthID survives JPEG re-encode, so GitHub issue attachments remain valid pixel-watermark test subjects. Every spectral/phase detection approach evaluated (reverse-SynthID, our own probes) works only on controlled solid fills, never on real content.
+- **External AI-vs-real classifier models are out of scope** (decided 2026-05-24): per-generator, degrade off-distribution, and our own light SDXL pass would likely defeat them. Detection stays local + signal-based.
+- **Default strength is VENDOR-ADAPTIVE, one ladder for BOTH pipelines** (since 2026-06-09): `resolve_strength(strength, vendor)` picks OpenAI **0.10** / Gemini **0.15** / unknown **0.15** when `--strength` is unset (the 2026-06-14 lowering from the 2026-06-04 cert floors of 0.20/0.30 — the single source of truth is `watermark_profiles.py`, and the full cert/lowering history is in `docs/known-limitations.md`); explicit `--strength` always wins. Removal at low strength is content x pipeline dependent, and near-threshold removal is SEED-NON-DETERMINISTIC — pick a strength with margin and oracle-revalidate per content type.
+- **`controlnet` is the default pipeline**; `--pipeline sdxl` is the lighter opt-down. Neither pipeline clears all content at low strength (photoreal survives controlnet, flat graphics survive sdxl — the lever is higher strength). A removal-priority caller MUST oracle-validate strength across content types; prod recipe: controlnet + per-vendor floor + FIXED seed. Forensic-stealth caveat (arXiv:2605.09203): defeating the SynthID verifier is NOT forensic invisibility — removal-processed images are flaggable at >98% TPR@1%FPR.

LICENSE

@@ -1,21 +1,217 @@
-MIT License
 
-Copyright (c) 2025 wiltodelta
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
 
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
 
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
+   1. Definitions.
 
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+
+
+   Copyright 2025-2026 wiltodelta
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -8,6 +8,10 @@ Strips SynthID, C2PA Content Credentials, EXIF/XMP "Made with AI" labels, and vi
 >
 > No Python, no GPU, no setup. Visible-watermark and metadata removal are **free**. Invisible-watermark removal (SynthID / SDXL regeneration) normally needs a local GPU and ~2 GB of models. On **[raiw.cc](https://raiw.cc)** it runs on cloud GPUs in one click for a small per-image fee.
 
+[![PyPI](https://img.shields.io/pypi/v/remove-ai-watermarks?logo=pypi&logoColor=white)](https://pypi.org/project/remove-ai-watermarks/)
+[![Python](https://img.shields.io/pypi/pyversions/remove-ai-watermarks?logo=python&logoColor=white)](https://pypi.org/project/remove-ai-watermarks/)
+[![Downloads](https://static.pepy.tech/badge/remove-ai-watermarks/month)](https://pepy.tech/project/remove-ai-watermarks)
+[![License](https://img.shields.io/pypi/l/remove-ai-watermarks?color=blue)](LICENSE)
 [![Tests](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml/badge.svg)](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml)
 [![Sponsor](https://img.shields.io/badge/Sponsor-GitHub-db61a2?logo=githubsponsors&logoColor=white)](https://github.com/sponsors/wiltodelta)
 
@@ -15,18 +19,24 @@ If this tool saves you time, consider [sponsoring its development](https://githu
 
 > **Intended for lawful use only.** Publishing and running this software is lawful; responsibility for any downstream use, and for compliance with local law, rests entirely with the user. Some jurisdictions restrict removing an AI label as such (see [Legal](#legal)). The authors do not condone use for deception, fraud, or any unlawful activity.
 
+## Scope
+
+This tool removes **AI-provenance watermarks** that a platform stamps onto content **you generated yourself** — SynthID, the Gemini / Nano Banana sparkle, the Doubao / Jimeng / Qwen / Samsung visible AI labels, the Chinese TC260 "由…AI生成" label, and C2PA / IPTC / EXIF "Made with AI" metadata. The point is your autonomy over your own output.
+
+It does **not** target watermarks that protect someone else's paid or copyrighted content — stock-agency overlays (Shutterstock, Getty, iStock, Adobe Stock), classifieds-site marks, or any tiled "preview" watermark whose job is to gate a purchase. Removing those is out of scope by design. `erase` is a generic, user-driven region tool for your own objects, not an automatic stock-watermark remover.
+
 ## Features
 
-- **Visible watermark removal** — a registry of known marks in their usual places: the Gemini / Nano Banana sparkle and the Doubao "豆包AI生成" text strip. Each is removed by **exact reverse-alpha blending** against a captured alpha map (`original = (wm − α·logo)/(1−α)`), recovering the true pixels rather than inpainting a guess. Fast, offline, no GPU. `visible --mark auto` finds and removes the strongest detected mark. (For arbitrary logos/objects, see `erase`.)
-- **Universal region eraser (`erase`)** — remove any logo / watermark / object inside boxes you specify, regardless of position or colour. Default cv2 inpainting (CPU, instant); optional big-LaMa via onnxruntime (`lama` extra) for higher quality
+- **Visible watermark removal** — a registry of known marks in their usual places: the Gemini / Nano Banana sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, the Qwen "千问AI生成" text strip, and the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-left, locale-specific). Each mark is **localized to a footprint mask, then filled**: the engine finds the mark, builds a binary mask over its footprint, and one shared, swappable fill inpaints that region. Choose the fill with `--backend`: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN, light, the memory-tight pick where LaMa will not fit), or `lama` (big-LaMa, best quality, heavier, auto-preferred when a learned backend is available); the default `auto` uses LaMa > MI-GAN > cv2, best available. The localizer is cheap CPU (cv2/numpy), so it runs anywhere; the heavier MI-GAN/LaMa fill is opt-in. Detection keys on each mark's own shape (NCC against a captured silhouette; the alpha captures rebuilt by `scripts/visible_alpha_solve.py` are used to detect and to shape the mask, not for pixel recovery). The visual detector needs no metadata, but a borderline (faint or moved) mark is only trusted with corroboration: `--sensitivity` (default `auto`) relaxes a mark's gate when local metadata confirms the vendor or a same-product sibling mark is found; `strict` never relaxes. There is deliberately no "assume this is AI" mode: asserting that an image is AI says nothing about *which* vendor made it or *where* the mark is, which is exactly what a gate bypass needs. If you can SEE a mark the detector missed, point at it with `erase --region x,y,w,h`, or force a known text mark by name with `--mark <name> --no-detect` -- both act on what you actually see instead of relaxing every detector at once. `visible --mark auto` finds and removes every detected mark in one pass. Fast, offline, no GPU. (For arbitrary logos/objects, see `erase`.)
+- **Universal region eraser (`erase`)** — remove any logo / watermark / object inside boxes you specify, regardless of position or color. Default cv2 inpainting (CPU, instant); optional big-LaMa via onnxruntime (`lama` extra) for higher quality
 - **Invisible watermark removal** — SynthID, StableSignature, TreeRing via diffusion-based regeneration (needs a local GPU, or run it with no setup on [raiw.cc](https://raiw.cc))
 - **AI metadata stripping** — EXIF, PNG text chunks, C2PA provenance manifests (PNG / JPEG / AVIF / HEIF / JPEG-XL, **MP4 / MOV / M4V / M4A** at the container level, and **WebM / MP3 / WAV / FLAC / OGG** losslessly via ffmpeg), XMP DigitalSourceType
 - **"Made with AI" label removal** — removes the AI-disclosure metadata that platforms read to apply automatic labels (useful for clearing a false-positive label from a human-edited photograph)
 - **Analog Humanizer** — optional film grain and chromatic aberration post-processing
-- **Smart Face Protection** — automatic extraction and blending of human faces to prevent AI distortion
+- **Text and face preservation (default)** — the default pipeline is a canny ControlNet that keeps text and face structure sharp through the removal pass (without copying original pixels, so SynthID is still removed). Use `--pipeline sdxl` for plain SDXL img2img (lighter, no extra model download) on inputs without text or faces. An experimental `--pipeline qwen` runs Qwen-Image (20B, Apache-2.0) img2img, which preserves **text** (including CJK and small text) better than SDXL at equal strength; it is CUDA/cloud-class (does not fit MPS), and its strength floors are not yet certified (pass an explicit `--strength`, especially for Gemini content). Note: measured fidelity (`scripts/fidelity_metrics.py`) shows Qwen wins on text but controlnet preserves **faces** better (Qwen smooths skin more), so Qwen is not a universal upgrade. Canny preserves face *structure*, not *identity* (the regenerated face drifts in likeness). The library does not ship a face-restore extra: every approach evaluated (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned) regenerated the face via SDXL and made the output look more AI-generated than the cleaned image. The cleaned controlnet output is the least-AI face state achievable without re-introducing SynthID.
 - **Batch processing** — process entire directories
 - **Detection** — three-stage NCC watermark detection with confidence scoring
-- **Provenance detection (`identify`)** — aggregate C2PA issuer, the C2PA soft-binding forensic-watermark vendor (Adobe TrustMark, Digimarc, Imatag, ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, embedded SD/ComfyUI params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the China TC260 AIGC label (XMP, PNG chunk, or EXIF), the HuggingFace `hf-job-id` job marker, the SynthID metadata proxy, the visible sparkle, the open SD/SDXL/FLUX invisible watermark, and (with the `trustmark` extra) the open Adobe TrustMark watermark into one origin-platform + watermark-inventory verdict (`--json` for machine output)
+- **Provenance detection (`identify`)** — aggregate C2PA issuer, the C2PA soft-binding forensic-watermark vendor (Adobe TrustMark, Digimarc, Imatag, ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, embedded SD/ComfyUI params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the China TC260 AIGC label (XMP, PNG chunk, EXIF, or JPEG segment), the HuggingFace `hf-job-id` job marker, the SynthID metadata proxy, the C2PA cloud-manifest reference (Adobe Durable Content Credentials, when the embedded manifest is stripped), the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Qwen "千问AI生成" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), the open SD/SDXL/FLUX invisible watermark, and (with the `trustmark` extra) the open Adobe TrustMark watermark into one origin-platform + watermark-inventory verdict (`--json` for machine output)
 
 ## Examples
 
@@ -38,7 +48,7 @@ If this tool saves you time, consider [sponsoring its development](https://githu
 
 | AI model | Visible watermark | Invisible watermark | Metadata | Our approach |
 | --- | --- | --- | --- | --- |
-| **Google Gemini / Nano Banana / Gemini 3 Pro** | ✅ Sparkle logo | ✅ SynthID v1 + v2 (default SDXL pipeline, native resolution) | ✅ C2PA + EXIF | Alpha reversal + diffusion + metadata strip |
+| **Google Gemini / Nano Banana / Gemini 3 Pro** | ✅ Sparkle logo | ✅ SynthID v1 + v2 (default SDXL pipeline, native resolution) | ✅ C2PA + EXIF | Localize + fill + diffusion + metadata strip |
 | **OpenAI DALL-E 3 / ChatGPT** | — | — | ✅ C2PA manifest | Metadata strip |
 | **OpenAI ChatGPT Images 2.0** (gpt-image-2) | — | ✅ SynthID + content-specific pixel watermark (since May 2026; no local decoder, openai.com/verify oracle) | ✅ C2PA manifest (verified) | Diffusion regeneration + metadata strip |
 | **Stable Diffusion / SDXL (AUTOMATIC1111, ComfyUI)** | — | ✅ DWT-DCT (imwatermark — locally detectable) | ✅ PNG text chunks | Diffusion regeneration + metadata strip |
@@ -49,15 +59,17 @@ If this tool saves you time, consider [sponsoring its development](https://githu
 | **xAI Grok (Aurora)** | — | — | ✅ EXIF signature scheme (no C2PA): `Signature:` blob + UUID `Artist` | Detected (`identify`); metadata strip |
 | **Midjourney** | — | — | ✅ EXIF + XMP (prompt, model, seed) | Metadata strip |
 | **Meta AI** | — | — | ✅ IPTC "Made with AI" (digitalSourceType) | Metadata strip (removes the label) |
-| **Doubao** (ByteDance) / China AIGC generators | ✅ "豆包AI生成" text strip (bottom-right) | — | ✅ TC260 AIGC label (`<TC260:AIGC>` XMP, `AIGC` PNG chunk, or EXIF JSON) **+ C2PA** signed by ByteDance Volcano Engine (`volcengine`) | Exact reverse-alpha (captured α map): pixel-exact at native width, NCC-aligned at other resolutions, + metadata strip |
-| **Samsung Galaxy AI** (Generative Edit, Sketch to Image, ...) | — | — | ✅ C2PA (signer "Samsung Galaxy") + `trainedAlgorithmicMedia` / proprietary `genAIType` marker | Detected (`identify`) + metadata strip |
+| **Doubao** (ByteDance) / China AIGC generators | ✅ "豆包AI生成" text strip (bottom-right) | — | ✅ TC260 AIGC label (`<TC260:AIGC>` XMP, `AIGC` PNG chunk, or EXIF JSON) **+ C2PA** signed by ByteDance Volcano Engine (`volcengine`) | Localize glyph footprint + fill + metadata strip |
+| **Jimeng / Dreamina** (即梦AI, ByteDance) | ✅ "★ 即梦AI" wordmark (bottom-right) | — | ✅ TC260 AIGC label + C2PA (Volcano Engine) | Localize glyph footprint + fill + metadata strip |
+| **Qwen / Tongyi Qianwen** (千问, Alibaba) | ✅ "千问AI生成" text strip (bottom-right) | — | ✅ TC260 AIGC label | Localize glyph footprint + fill + metadata strip |
+| **Samsung Galaxy AI** (Generative Edit, Sketch to Image, ...) | ✅ "✦ Contenuti generati dall'AI" strip (bottom-left, Italian-locale detection) | — | ✅ C2PA (signer "Samsung Galaxy") + `trainedAlgorithmicMedia` / proprietary `genAIType` marker | Localize glyph footprint + fill + metadata strip |
 | **Black Forest Labs** (FLUX API) | — | — | ✅ C2PA (`Black Forest Labs API` + `c2pa.ai_generated_content` + `trainedAlgorithmicMedia`) | Metadata strip |
 | **StableSignature** (Meta) | — | ✅ In-model watermark | — | Diffusion regeneration |
 | **TreeRing** | — | ✅ Latent space watermark | — | Diffusion regeneration |
 
-> Visible overlays are used by Google Gemini / Nano Banana (sparkle logo) and by Doubao / China AIGC generators (the mandated "...AI生成" corner text). Both are removed deterministically on CPU. Other services rely on invisible watermarks and/or metadata; our diffusion-based regeneration works against any invisible watermark in pixel or frequency domain. For a visible mark from any other source (any position, any colour), use the universal `erase --region` command.
+> Visible overlays are used by Google Gemini / Nano Banana (sparkle logo), by ByteDance's Doubao ("豆包AI生成" corner text) and Jimeng / Dreamina ("★ 即梦AI" wordmark), by Alibaba's Qwen / Tongyi Qianwen ("千问AI生成" corner text), and by Samsung Galaxy AI ("✦ Contenuti generati dall'AI" strip, bottom-left, locale-specific). All are removed by localizing the mark to a footprint mask and inpainting it with one shared fill (cv2 by default, MI-GAN or big-LaMa via `--backend`); the localizer is CPU-cheap and the heavier fills are opt-in. Other services rely on invisible watermarks and/or metadata; our diffusion-based regeneration works against any invisible watermark in pixel or frequency domain. For a visible mark from any other source (any position, any color), use the universal `erase --region` command.
 
-> **Detection:** `remove-ai-watermarks identify <image>` reports the origin platform and watermark inventory for all the signals above — C2PA issuer, the C2PA soft-binding forensic-watermark vendor (TrustMark / Digimarc / Imatag / ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, the China TC260 AIGC label (XMP, PNG chunk, or EXIF), the HuggingFace `hf-job-id` job marker, embedded generation params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the SynthID metadata proxy, the visible sparkle, and (with the `[detect]` / `[trustmark]` extras) the open SD/SDXL/FLUX and Adobe TrustMark invisible watermarks. SynthID and the proprietary soft-binding watermarks (Digimarc etc.) have no local decoder, so they are reported by metadata proxy / vendor name only.
+> **Detection:** `remove-ai-watermarks identify <image>` reports the origin platform and watermark inventory for all the signals above — C2PA issuer, the C2PA soft-binding forensic-watermark vendor (TrustMark / Digimarc / Imatag / ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, the China TC260 AIGC label (XMP, PNG chunk, EXIF, or JPEG segment), the HuggingFace `hf-job-id` job marker, embedded generation params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the SynthID metadata proxy, the C2PA cloud-manifest reference (Adobe Durable Content Credentials, when the embedded manifest is stripped), the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Qwen "千问AI生成" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), and (with the `[detect]` / `[trustmark]` extras) the open SD/SDXL/FLUX and Adobe TrustMark invisible watermarks. SynthID and the proprietary soft-binding watermarks (Digimarc etc.) have no local decoder, so they are reported by metadata proxy / vendor name only.
 
 ## How it works
 
@@ -69,25 +81,37 @@ Google Gemini (internally codenamed **Nano Banana**) adds a visible sparkle logo
 watermarked = α × logo + (1 − α) × original
 ```
 
-We reverse this with a known alpha map (extracted from Gemini / Nano Banana output on a pure-black background):
+A multi-scale NCC (Normalized Cross-Correlation) detector finds the sparkle position and scale dynamically in the bottom-right region (with a false-positive gate), so it works even if the image was resized or cropped. Removal is **localize then fill**: the sparkle footprint is built from the captured alpha thresholded low (so the faint halo is included) and dilated by a sparkle-relative margin, and that mask is inpainted by the shared fill. The captured alpha maps are used only to detect the sparkle and to shape the mask, not to recover pixels. If local metadata already confirms Google (a Google / Gemini C2PA issuer), the detection trust gate is relaxed so a moved or re-rendered sparkle is still caught. Pick the fill with `--backend` (cv2 default, MI-GAN or big-LaMa opt-in).
 
-```text
-original = (watermarked − α × logo) / (1 − α)
-```
-
-A three-stage NCC (Normalized Cross-Correlation) detector finds the watermark position and scale dynamically, so it works even if the image was resized or cropped. After removal, residual sparkle-edge artifacts are cleaned via gradient-masked inpainting.
-
-**Speed**: ~0.05s per image. No GPU needed.
+**Speed**: fast on CPU with the default cv2 fill. No GPU needed.
 
 ### Removing the Doubao "豆包AI生成" text watermark
 
-Doubao (ByteDance) stamps every output with a light, semi-transparent "豆包AI生成" text strip in the bottom-right corner — the visible AIGC label mandated by China's TC260 standard. It is a fixed semi-transparent white overlay, so — like the Gemini sparkle — it is removed by **exact reverse-alpha blending**: `original = (watermarked - α·logo) / (1 - α)`, recovering the true pixels instead of hallucinating them. The α map and logo colour were solved from controlled black + gray captures (on black, `captured = α·logo`; the black/gray pair solves α per-pixel). At the captured width the placement is exact, so the recovery is returned untouched (inpainting over exactly-recovered pixels only degrades them). The single capture generalizes to any resolution: off the captured width an NCC scale-and-position search registers the α template to the actual mark, and a light residual inpaint cleans the sub-pixel seam there. Detection is consistent with removal: it matches the same alpha glyph silhouette against the corner (normalized correlation), so it keys on the actual "豆包AI生成" shape, not on textured corners.
+Doubao (ByteDance) stamps every output with a light, semi-transparent "豆包AI生成" text strip in the bottom-right corner — the visible AIGC label mandated by China's TC260 standard. Detection matches the glyph silhouette against the corner (normalized correlation), so it keys on the "豆包AI生成" shape, not on textured corners. Removal is **localize then fill**: the light glyph blob is localized to a solid, dilated footprint mask (template-free, so a re-rendered or differently-placed mark is still masked) and the shared fill inpaints it. On corpus images the filled region blends into its surroundings within a few LAB levels, with no color shift and no dark pit. Pick the fill with `--backend` (cv2 default, MI-GAN or big-LaMa opt-in).
 
-**Speed**: ~0.05s, no GPU needed. Reverse-alpha at the captured resolution recovers the true background pixels exactly.
+**Speed**: fast on CPU with the default cv2 fill, no GPU needed.
+
+### Removing the Jimeng "★ 即梦AI" wordmark
+
+Jimeng / Dreamina (即梦AI, also ByteDance, distinct from Doubao) stamps a "★ 即梦AI" wordmark — a four-point sparkle followed by the 即梦AI characters — in the bottom-right corner. Detection keys on the wordmark's glyph shape (NCC against a captured silhouette). `visible --mark auto` detects and removes it (or force it with `--mark jimeng`). Removal is **localize then fill**: the glyph blob is localized to a solid, dilated footprint mask and the shared fill inpaints it, so a re-rendered or differently-placed mark is still cleared. On corpus images the filled region blends into its surroundings within a few LAB levels, no color shift. The two ByteDance marks do not confuse `auto`: detection keys on each mark's own glyph shape (the Jimeng detector scores far below its threshold on a Doubao strip, and vice versa).
+
+```bash
+remove-ai-watermarks visible jimeng.png -o clean.png            # --mark auto picks Jimeng
+remove-ai-watermarks visible jimeng.png --mark jimeng -o clean.png
+```
+
+### Removing the Samsung Galaxy AI "✦ Contenuti generati dall'AI" mark
+
+Samsung's on-device Generative AI edits (Generative Edit, Sketch to Image, Portrait Studio) burn a visible sparkle + "generated with AI" string into the **bottom-left** corner — a faint, low-opacity semi-transparent white overlay. Detection matches the glyph silhouette (NCC), and removal is **localize then fill**: the glyph blob is localized to a solid, dilated footprint mask and the shared fill inpaints it. `visible --mark auto` detects and removes it (or force it with `--mark samsung`); being bottom-left it never confuses the bottom-right Gemini/Doubao/Jimeng marks. **Detection is locale-specific** — this build detects only the Italian "Contenuti generati dall'AI" variant, so other Samsung locales are not detected (and thus not removed) and need their own captured detection template (open a sample on issue #37). The fill mask itself is locale-independent.
+
+```bash
+remove-ai-watermarks visible samsung.jpg -o clean.jpg            # --mark auto picks Samsung
+remove-ai-watermarks visible samsung.jpg --mark samsung -o clean.jpg
+```
 
 ### Universal region eraser
 
-For any visible mark the dedicated engines do not cover — a logo anywhere, any colour — `erase --region x,y,w,h` inpaints the box you specify. The default `cv2` backend is instant and dependency-free; the optional `lama` backend (big-LaMa via onnxruntime, `lama` extra, ~200 MB model downloaded on first use) gives much cleaner fills on textured regions at the cost of ~3-4 GB RAM per call.
+For any visible mark the dedicated engines do not cover — a logo anywhere, any color — `erase --region x,y,w,h` inpaints the box you specify. The default `cv2` backend is instant and dependency-free; the optional `lama` backend (big-LaMa via onnxruntime, `lama` extra, ~200 MB model downloaded on first use) gives much cleaner fills on textured regions at the cost of ~3-4 GB RAM per call.
 
 ### Removing SynthID and other invisible watermarks
 
@@ -98,20 +122,30 @@ The removal pipeline (default profile, SDXL):
 ```text
 image → encode to latent space (VAE) at native resolution
       → add controlled noise (forward diffusion)
-      → denoise (reverse diffusion, ~50 steps at strength 0.05)
+      → denoise (reverse diffusion, ~50 steps; strength is vendor-adaptive:
+        0.20 OpenAI / 0.30 Google / 0.30 unknown, same for both pipelines;
+        override with --strength)
       → decode back to pixels (VAE)
 ```
 
-By default the image is processed at its **native resolution** with no pre-downscale, matching the hosted raiw.cc backend (fal `fast-sdxl`, which is `stabilityai/stable-diffusion-xl-base-1.0` — the same checkpoint the CLI defaults to). At strength ~0.05 SDXL img2img does not need the input shrunk, and the old forced downscale-to-1024 then upscale-back round-trip was the main quality loss. Pass `--max-resolution N` to cap the long side only when a very large image runs out of GPU/MPS memory (it reintroduces that lossy round-trip).
+- Large inputs run at native resolution (no down-then-up round-trip, which was the main quality loss in issue #10); use `--max-resolution N` only to cap GPU/MPS memory on very large inputs. For inputs that run out of GPU/MPS memory at native resolution, `--tile` is the lossless alternative to `--max-resolution`: it regenerates the image in overlapping, feather-blended tiles (each near SDXL's 1024 px size) so there is no downscale and no visible seam. It engages only when the long side exceeds `--tile-size` (default 1024; overlap `--tile-overlap`, default 128); pair it with `--max-resolution 0`. Small inputs (long side under 1024 px) are auto-upscaled to a 1024 px floor before diffusion, because SDXL distorts on a tiny latent, and the result is restored to the original size (a transparent quality boost). Disable the floor with `--min-resolution 0`. The floor upscale uses Lanczos by default; `--upscaler esrgan` (the `esrgan` extra) runs Real-ESRGAN first for sharper detail and falls back to Lanczos if the extra is absent. ESRGAN is a generic photo/texture GAN with no face/glyph prior, so it is best for photo/texture content -- it can degrade faces (the diffusion pass regenerates them, so the final recovers) and thin text; keep Lanczos for text-heavy inputs.
+
+> **Default strength is vendor-adaptive (no flag needed).** The tool reads the C2PA issuer to detect which vendor's SynthID is present and picks the strength accordingly: **OpenAI gpt-image → `0.20`**, **Google Gemini → `0.30`**, **unknown source → `0.30`**. The **same ladder applies to both pipelines** — these are the oracle-certified `controlnet` floors (June 2026 Modal cert, multi-seed). They also cover plain `sdxl`: the two pipelines have opposite hard cases (controlnet leaves SynthID on photoreal, sdxl on flat graphics), but on its own hard case sdxl is the weaker remover, so it needs at least controlnet's strength — using one certified ladder is the safe choice (margin-based for sdxl, not separately certified). The dominant factor is the vendor (Google's SynthID is ~3x more robust). There is no local SynthID detector, so if the oracle still reads SynthID, raise `--strength`; if you care more about preserving fine detail, lower it. (Caveat: Google's `0.30` was validated only at `--max-resolution 1536`; a very large native Gemini image may need ~`0.35`+.)
+>
+> **The default pipeline is `controlnet` — it preserves text and face structure.** It runs the same SDXL img2img scrub but adds a canny ControlNet that conditions the regeneration on the image's edge map, so text and structure stay sharp at the strengths that remove SynthID. The watermark removal still comes from the img2img regeneration (`--strength`); the ControlNet only preserves structure — no original pixels are copied or frozen. The default strength ladder (OpenAI `0.20` / Google `0.30`) is the oracle-certified controlnet floor. `--controlnet-scale` tunes the preservation strength (higher = closer to the original structure). Runs fp32 on mps/cpu (fp16 only on cuda/xpu, where the fp16-fixed SDXL VAE is loaded automatically). Pass `--pipeline sdxl` for plain SDXL img2img (lighter, no extra model download) on inputs without text or faces.
+>
+> **No face-restore extra in the library.** Every ArcFace-based regeneration approach we evaluated (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps, 2026-06-04 - 2026-06-08 Modal cert sweeps) regenerated the face via SDXL diffusion — the output face pixels were diffusion-fresh (SynthID not re-introduced), but the face inherently looked more AI-generated than the cleaned image (SDXL "clean skin" gloss, lost original identity precision). The cleaned image from the main controlnet 0.20 pass is the least-AI face state we can reach without re-introducing SynthID. Empirical conclusion in `docs/synthid-robust-identity-research-2026-06-08.md`.
 
 SDXL is the default since May 2026: empirically defeats SynthID v2 on Gemini 3 Pro outputs, where the older SD-1.5 pipeline at 768 px did not. The SD-1.5 path was removed once it was verified not to handle v2. Note the scope: this defeats the SynthID *verifier*, which is not the same as being forensically indistinguishable from a real photo. Recent work ([arXiv:2605.09203](https://arxiv.org/abs/2605.09203)) shows watermark-removal pipelines leave detectable traces, so a separate "this image was processed" classifier can still flag the output.
 
-**Face Protection**: before diffusion, YOLO detects people in the image and extracts them. After diffusion, the original faces are blended back with a soft elliptical mask to prevent AI distortion of facial features.
+> **Oracle vs `identify` can disagree, and that is expected.** An online verifier reads the actual SynthID *pixel* watermark and detects only its own vendor's content — [openai.com/research/verify](https://openai.com/research/verify/) states "OpenAI generation signals will only be detected if the image was generated with our tools". Our `identify` cannot decode the pixel watermark (no vendor ships a local decoder), so it infers SynthID from the **C2PA metadata** instead. So after the SDXL pass the oracle can read "no SynthID" (pixel watermark gone) while `identify` still reports SynthID from a surviving C2PA manifest. They measure different signals. Run `metadata --remove` (or `all`) to also strip the manifest; note that a quiet metadata proxy is not proof the pixel watermark itself is gone.
+
+> **Technical deep-dive:** see [`docs/synthid.md`](docs/synthid.md) for a primary-source-cited breakdown of how SynthID works mechanically (post-hoc encoder/decoder, 136-bit payload, pixel-space embedding), what it empirically survives (JPEG, crop, resize: ~99.98% TPR at 0.1% FPR from arXiv:2510.09263), what removes it, and the forensic-stealth tradeoff (all known removal attacks are detectable at >98% TPR@1%FPR per arXiv:2605.09203).
+
+**Text and face preservation** (the default pipeline; `--pipeline sdxl` opts down to plain SDXL): a canny ControlNet keeps text and face *structure* sharp through the removal pass, without copying or freezing any original pixels (so SynthID is still removed). Tune the preservation strength with `--controlnet-scale`. Canny preserves structure but not face *identity*: the regenerated face drifts in likeness. The library does not ship a face-restore extra (see the callout above).
 
 **Analog Humanizer**: optional film grain and chromatic aberration injection that mimics a photo of a screen, raising the bar for AI-generated image classifiers. (It frustrates generic classifiers but does not guarantee forensic invisibility — see the [arXiv:2605.09203](https://arxiv.org/abs/2605.09203) note above.)
 
-**Text Protection** (automatic): SDXL img2img regenerates every pixel, so small text and glyphs get deformed at the strengths that defeat SynthID. The SDXL pipeline guards against this by default: a PP-OCRv3 text detector (a 2.4 MB ONNX model run on CPU via OpenCV's DNN module, downloaded and cached on first use) locates text regions, and if any are found the pass switches to Differential Diffusion so a per-pixel change map keeps the text regions largely intact while the background is regenerated normally. Detection is **language-agnostic** (it finds text regions, not characters), so Latin, Cyrillic, CJK, Hangul, Arabic, and digits are all protected, and it runs at the image's native resolution so small text on large images is not missed. Text-free images run the standard pass at no extra cost. Pass `--no-protect-text` to turn it off. SDXL default pipeline only.
-
 ### Stripping C2PA, EXIF, and "Made with AI" metadata
 
 AI tools embed generation metadata that social platforms use to show "Made with AI" labels:
@@ -125,6 +159,34 @@ The cleaner parses each layer, removes AI-related fields, and preserves standard
 
 ## Installation
 
+### Homebrew (macOS / Linux)
+
+```bash
+brew install wiltodelta/tap/remove-ai-watermarks
+```
+
+This installs the core command surface (`identify`, `metadata`, `visible`,
+`erase`) as a self-contained CLI. The diffusion-based `invisible` / `all`
+pipeline needs heavy ML dependencies (torch, diffusers, multi-GB) and is kept
+out of the Homebrew build; add it with the `gpu` extra via pip if you need it:
+
+```bash
+pip install "remove-ai-watermarks[gpu]"
+```
+
+### conda
+
+A conda-forge recipe is under review
+([staged-recipes PR](https://github.com/conda-forge/staged-recipes/pull/33674)).
+Once it merges, the core package installs with:
+
+```bash
+conda install -c conda-forge remove-ai-watermarks
+```
+
+Like the Homebrew build, this is the core command surface; add the diffusion
+`invisible` / `all` pipeline with the pip `gpu` extra.
+
 ### Recommended
 
 Install as an isolated CLI tool — no need to manage virtual environments:
@@ -171,6 +233,10 @@ After installation the `remove-ai-watermarks` command is available system-wide.
 > pip install -e ".[gpu]"   # or: uv pip install -e ".[gpu]"
 > ```
 >
+> Without the `[gpu]` extra, `all` still runs the visible and metadata steps, but
+> it skips the invisible (SynthID) step, prints a clear warning, and exits with a
+> non-zero status so a skipped step is not mistaken for a clean result.
+>
 > To let `identify` decode the open Stable Diffusion / SDXL / FLUX invisible
 > watermarks, install the `detect` extra (adds the `invisible-watermark` decoder):
 >
@@ -185,6 +251,14 @@ After installation the `remove-ai-watermarks` command is available system-wide.
 > ```bash
 > pip install -e ".[trustmark]"   # or: uv pip install -e ".[trustmark]"
 > ```
+>
+> For sharper upscaling of small inputs before diffusion (`--upscaler esrgan`,
+> Real-ESRGAN), install the `esrgan` extra. It loads via spandrel (MIT, no basicsr);
+> the Real-ESRGAN weights (BSD-3-Clause) download on first use:
+>
+> ```bash
+> pip install -e ".[esrgan]"   # or: uv pip install -e ".[esrgan]"
+> ```
 
 #### Invisible watermark removal
 
@@ -215,6 +289,23 @@ pytest
 ./maintain.sh
 ```
 
+## ComfyUI
+
+Custom nodes are available so the watermark tools run inside a ComfyUI graph:
+[ComfyUI-remove-ai-watermarks](https://github.com/wiltodelta/ComfyUI-remove-ai-watermarks)
+([registry](https://registry.comfy.org/nodes/remove-ai-watermarks)).
+
+Install via ComfyUI Manager (search "Remove AI Watermarks") or manually:
+
+```bash
+cd ComfyUI/custom_nodes
+git clone https://github.com/wiltodelta/ComfyUI-remove-ai-watermarks
+pip install -r ComfyUI-remove-ai-watermarks/requirements.txt
+```
+
+Nodes: Remove Visible Watermark, Detect Visible Watermark, Erase Region (by
+mask), and Remove Invisible Watermark / SynthID (needs the `gpu` extra).
+
 ## Usage
 
 ### CLI
@@ -239,20 +330,51 @@ remove-ai-watermarks batch ./images/ --mode all
 # of a clean origin. Add --json for machine-readable output.
 remove-ai-watermarks identify image.png
 
-# Visible watermark only — fast, offline, CPU. --mark auto (default) finds the
-# strongest known mark (Gemini sparkle / Doubao "豆包AI生成" text); force one
-# with --mark gemini / doubao. Removed by exact reverse-alpha (true-pixel recovery).
+# Visible watermark only — fast, offline, CPU. --mark auto (default) removes every
+# detected known mark (Gemini sparkle / Doubao "豆包AI生成" / Jimeng "即梦AI" /
+# Samsung Galaxy AI "Contenuti generati dall'AI"); force one with
+# --mark gemini / doubao / jimeng / qwen / samsung. Removal localizes each mark to a
+# footprint mask and inpaints it with a shared fill; --backend auto|cv2|migan|lama
+# (default auto) picks the fill (auto = LaMa > MI-GAN > cv2, best available).
+# --sensitivity auto|strict (default auto) sets how hard a borderline mark is
+# trusted; there is no "assume this is AI" mode -- to act on a mark you can see
+# but the detector missed, point at it with `erase --region` (below).
+# If no known visible mark is found, it writes no output and exits 2 (not 0),
+# pointing you to `all` (for an invisible/metadata mark) or `erase` (for an
+# arbitrary logo) instead of handing back the unchanged image.
 remove-ai-watermarks visible image.png -o clean.png
 
+# Metadata-stripped screenshot where you can SEE the mark but detection missed it:
+# point at it. This executes what you see instead of relaxing every detector at once.
+remove-ai-watermarks erase screenshot.png --region 812,1180,190,44 -o clean.png
+
 # Erase arbitrary region(s) — universal, any logo/watermark/object, any position.
 # Default cv2 inpainting (CPU). --backend lama uses big-LaMa (extra 'lama').
 remove-ai-watermarks erase image.png --region 1640,1930,400,100 -o clean.png
 
 # Invisible watermark only (SynthID etc.) — requires GPU
-remove-ai-watermarks invisible image.png -o clean.png --humanize 4.0
-# Runs at native resolution by default. On a very large image that OOMs the
-# GPU/MPS, cap the long side: --max-resolution 2048
-# Text / CJK glyphs are preserved automatically; disable with --no-protect-text
+remove-ai-watermarks invisible image.png -o clean.png --humanize 4.0 --unsharp 0.5
+# --humanize adds film grain, --unsharp counters the soft "AI" look (both opt-in).
+# Large images run at native resolution; small ones are upscaled to a 1024 floor
+# first (disable with --min-resolution 0); --upscaler esrgan uses Real-ESRGAN for
+# that floor upscale (needs the 'esrgan' extra). On a very large image that OOMs the
+# GPU/MPS, either cap the long side (--max-resolution 2048, lossy) or pass --tile
+# to regenerate in overlapping feather-blended tiles at native resolution (lossless).
+# Strength is vendor-adaptive by default (OpenAI 0.20 / Google 0.30, same
+# for both pipelines); override with --strength. controlnet (text/face
+# structure preservation) is the default pipeline; --pipeline sdxl opts down
+# to plain SDXL for non-structure inputs. Tune structure preservation with
+# --controlnet-scale, the CFG with --guidance-scale (default 7.5), and the
+# diffusion model with --model (default: SDXL base).
+# --adaptive-polish (ON by default) restores the input's detail level (sparing
+# text) to counter the over-smoothed look; it self-limits to a no-op where
+# there is no detail deficit. Disable with --no-adaptive-polish.
+# By default, if no invisible AI watermark is locally detectable, the diffusion
+# scrub is SKIPPED (regenerating pixels would only degrade a clean image): for
+# `invisible` that writes no output and exits 2, for `all` it skips step 2 but
+# still strips metadata and exits 0. A skip never claims the image is clean
+# (a pixel SynthID is undetectable once its metadata is gone). Pass --force to
+# regenerate regardless when you know the image is AI-generated.
 
 # Check / strip AI metadata (C2PA, EXIF, "Made with AI" labels)
 # --check also flags SynthID-bearing sources: a C2PA manifest signed by
@@ -264,24 +386,62 @@ remove-ai-watermarks metadata image.png --remove
 
 # Batch with a specific mode
 remove-ai-watermarks batch ./images/ --mode visible
+
+# Batch accepts the full invisible knob set (--strength/--guidance-scale/--model/
+# --pipeline/...); --adaptive-polish is on by default (--no-adaptive-polish to disable)
+remove-ai-watermarks batch ./images/ --mode all
 ```
 
 ### Python API
 
+One high-level call removes every detected visible mark (Gemini sparkle, Doubao / Jimeng / Qwen / Samsung text, the Jimeng pill) by localize then fill. For a file it reads metadata provenance automatically and preserves the alpha channel; `import remove_ai_watermarks` stays cheap (the heavy deps load lazily on first use).
+
 ```python
-from remove_ai_watermarks.gemini_engine import GeminiEngine
+import remove_ai_watermarks as raiw
+
+# Clean a file -> writes clean.png, returns the array and what was removed.
+result, removed = raiw.remove_visible("watermarked.png", "clean.png")
+print("removed:", removed)          # e.g. ['Google Gemini sparkle']
+# An empty list means nothing known was found (route to `all` or `erase` instead).
+
+# Array in, array out (BGR numpy). Pick the fill backend: cv2 (default), migan, lama.
 import cv2
+clean, removed = raiw.remove_visible(cv2.imread("in.png"), backend="cv2")
 
-engine = GeminiEngine()
-image = cv2.imread("watermarked.png")
+# Metadata-stripped screenshot you know is AI-generated: relax detection to recover a
+# moved or faint mark (a wrong guess just fills a small corner near-losslessly).
+raiw.remove_visible("screenshot.png", "clean.png", sensitivity="strict")
 
-# Detect
-result = engine.detect_watermark(image)
-print(f"Detected: {result.detected} (confidence: {result.confidence:.1%})")
+# What the file's metadata confirms (drives the default `auto` sensitivity):
+raiw.visible_provenance("in.png")   # e.g. frozenset({'gemini'})
 
-# Remove
-clean = engine.remove_watermark(image)
-cv2.imwrite("clean.png", clean)
+# Full provenance verdict (platform + watermark inventory + confidence):
+from remove_ai_watermarks.identify import identify
+report = identify("in.png")
+```
+
+#### Invisible removal (diffusion)
+
+```python
+from pathlib import Path
+from remove_ai_watermarks.invisible_engine import InvisibleEngine
+
+# pipeline: "controlnet" (default, preserves text/face structure) or "sdxl" (plain).
+# model_id=None uses the SDXL base; controlnet_conditioning_scale tunes preservation.
+engine = InvisibleEngine(pipeline="controlnet")
+
+engine.remove_watermark(
+    Path("watermarked.png"),
+    Path("clean.png"),
+    strength=None,          # None = vendor-adaptive default (OpenAI 0.20 / Google 0.30)
+    num_inference_steps=50,
+    guidance_scale=None,    # None = the library default (7.5)
+    seed=None,              # set for reproducible output
+    adaptive_polish=True,   # detail-targeted polish, self-gating (default on in the CLI)
+    min_resolution=1024,    # upscale tiny inputs to this floor before diffusion
+    max_resolution=0,       # 0 = native; set only to cap GPU/MPS memory
+    upscaler="lanczos",     # or "esrgan" for the floor upscale (needs the 'esrgan' extra)
+)
 ```
 
 ### Metadata stripping
@@ -318,7 +478,7 @@ pip install certifi
 
 - [noai-watermark](https://github.com/mertizci/noai-watermark) by mertizci — invisible watermark removal engine
 - [GeminiWatermarkTool](https://github.com/allenk/GeminiWatermarkTool) by Allen Kuo (MIT) — visible watermark removal algorithm
-- [CtrlRegen](https://github.com/yepengliu/CtrlRegen) by Liu et al. (ICLR 2025) — controllable regeneration pipeline
+- [controlnet-canny-sdxl-1.0](https://huggingface.co/xinsir/controlnet-canny-sdxl-1.0) by xinsir — SDXL canny ControlNet used by the `controlnet` pipeline to preserve text/face structure
 - NeuralBleach (MIT) — analog humanizer technique
 
 ## Roadmap
@@ -326,7 +486,7 @@ pip install certifi
 Tracked but not yet implemented:
 
 - **SynthID-Image v2 automated regression test**. The default SDXL profile defeats v2 per manual checks against the [Gemini app](https://support.google.com/gemini/answer/16722517)'s "Verify with SynthID" feature on a Gemini 3 Pro output (May 2026). An automated end-to-end test would need either programmatic access to the [SynthID Detector portal](https://blog.google/innovation-and-ai/products/google-synthid-ai-content-detector/) (waitlist for media professionals and researchers) or an offline surrogate detector. The spectral phase-coherence surrogate from [reverse-SynthID](https://github.com/aloshdenny/reverse-SynthID) was evaluated and does not separate watermarked from cleaned real-content images (it only fires on controlled solid-color references at exact resolution), so it is not a usable oracle. Open.
-- **Local SynthID *pixel* detector**. Not feasible today: Google's decoder is proprietary, and magnitude/carrier spectral methods do not separate real content (confirmed by three independent evaluations, including a from-scratch gpt-image pilot; see CLAUDE.md). Blocked on either (a) a programmatic generation path (OpenAI / Gemini API) to build a per-(model, resolution) labeled corpus at scale, or (b) a raw watermarked-output dataset. If data arrives, the next approach to try is a learned classifier on diverse content rather than a fixed carrier codebook.
+- **Local SynthID *pixel* detector**. Not feasible today: Google's decoder is proprietary, and magnitude/carrier spectral methods do not separate real content (confirmed by three independent evaluations, including a from-scratch gpt-image pilot; see docs/known-limitations.md). Blocked on either (a) a programmatic generation path (OpenAI / Gemini API) to build a per-(model, resolution) labeled corpus at scale, or (b) a raw watermarked-output dataset. If data arrives, the next approach to try is a learned classifier on diverse content rather than a fixed carrier codebook.
 - **Grow the SynthID reference corpus** (`data/synthid_corpus/`) with oracle-labeled samples per model and resolution (Gemini app for Google, openai.com/verify for OpenAI). Prerequisite for any pixel-detector attempt and for an automated removal-regression set.
 - **Real non-PNG C2PA fixtures**. SynthID-source detection for JPEG / WebP / AVIF is currently covered only by synthetic byte blobs; replace with real vendor-emitted files to ground the binary-scan path.
 - **Maintenance debt**. Strict pyright is now clean across `src/` (0 errors): pure-logic files are fully typed, the cv2 / torch / diffusers boundary files carry a documented per-file relax pragma, and a local `typings/piexif` stub covers piexif. Remaining: full-project `pyright` (no path) still OOMs node on this ML-heavy repo, so it must be scoped to `src/`; narrowing the boundary pragmas back toward full strict (as upstream stubs improve) is the long tail. (`uv-secure` is already clean since `idna` was bumped to 3.16.)
@@ -339,6 +499,13 @@ Won't fix:
 
 - **Nightshade / Glaze / PhotoGuard removal**. These are defensive perturbations used by artists to protect their work from being scraped into AI training sets. Removing them attacks artists, not AI provenance. Out of scope.
 
+## Limitations
+
+- **Visible-mark removal is localized inpainting; metadata removal is lossless.** Each visible mark is localized to a footprint mask and filled by inpainting (cv2, MI-GAN, or big-LaMa), so only the small masked region is reconstructed and it blends into its surroundings within a few LAB levels; a slightly-off localization just fills a small region near-losslessly rather than leaving a color-shifted smear. Metadata stripping never touches image data.
+- **The invisible (SynthID) path is lossy and not guaranteed.** It runs a low-strength SDXL img2img regeneration, so it softens fine detail and is content-dependent. There is no public SynthID decoder, so the tool cannot verify removal locally; confirm with the Gemini app's "Verify with SynthID" oracle and raise `--strength` if it still detects. A vendor can change the scheme at any time, so treat this as an arms race, not a permanent fix.
+- **Large images: native by default, opt-in tiling for OOM.** The SynthID path runs at the diffusion model's native resolution; on a memory-constrained GPU/MPS you can either cap the long side with `--max-resolution` (lossy downscale) or pass `--tile` to regenerate in overlapping, feather-blended tiles at native resolution (lossless, no seam). Tiling is a memory workaround, not a quality upgrade over a single native pass: each tile is an independent low-strength regeneration. (Nano Banana 2 is natively 1024px; GPT Image 2 supports 4K experimentally.)
+- **Out of scope:** defeating trained AI-vs-real classifiers like Hive (see [Threat model](#threat-model)), visible-logo removal from video, and any guarantee that a stripped copy is untraceable server-side.
+
 ## Legal
 
 Watermarking and provenance for AI-generated content is now regulated in several jurisdictions. The table below summarises the May 2026 status. None of this is legal advice.
@@ -356,14 +523,14 @@ Watermarking and provenance for AI-generated content is now regulated in several
 
 ## Threat model
 
-This tool defends already-distributed AI imagery against automatic detection systems (social-platform "Made with AI" labels, third-party classifiers, content-policy filters). It does **not** retroactively anonymise generation.
+This tool removes specific, known signals: the embedded SynthID pixel watermark, the visible vendor marks, and the C2PA / EXIF / IPTC provenance metadata that platforms read to apply automatic "Made with AI" labels. It is **not** a general detector-evasion tool. It does **not** defeat trained statistical AI-vs-real classifiers (for example Hive Moderation), and a light diffusion pass will not reliably fool those, so a clean classifier hit after removal is expected, not a bug. It also does **not** retroactively anonymise generation. And watermarking is a weak trust signal in the first place: a marker that is almost always present yet trivially removable can make a cleaned forgery look more trustworthy, not less, which is why durable provenance more likely comes from signing genuine content than from watermarking synthetic content.
 
 In particular, **SynthID** (Google DeepMind) is embedded across Google's generative media stack — Imagen (images), Veo (video), Lyria (audio) — and Gemini app image outputs (Nano Banana / Gemini 3 Pro, which we verified positive via the Gemini app's SynthID oracle); Google reported over 10 billion items watermarked by December 2025. It carries a **multi-bit payload** — the research paper's SynthID-O variant encodes 136-bit payloads in 512x512 images ([arxiv 2510.09263](https://arxiv.org/abs/2510.09263)). The payload is believed to encode a user / session identifier. If the original watermarked file ever passed through a system controlled by the prompt originator (a saved Gemini account history, a screenshot uploaded to a Google product, a backup), Google retains the ability to link that original to the generating account. Stripping the watermark from a copy you possess does not erase Google's server-side record.
 
 Use cases where the threat model fits:
 - You generated the image yourself, want to publish it as your own work, and accept the consequences if Google ever publishes their detector logs.
 - You are running a security / robustness evaluation.
-- You are preserving art or historical record against false-positive "AI-generated" labels.
+- A real photo of yours was lightly AI-edited (a retouch in Gemini or ChatGPT, say) and now carries a SynthID or C2PA label that overstates how AI-generated it is, and you want to clear that label from your own copy.
 
 Use cases where the threat model **does not** fit:
 - Generating an image, expecting that removing the watermark anonymises you to Google. It doesn't.
@@ -385,4 +552,4 @@ Legitimate uses — publishing your own work, privacy (stripping metadata that l
 
 ## License
 
-MIT
+[Apache 2.0](LICENSE). Copyright 2025-2026 wiltodelta.

data/doubao_capture/README.md

@@ -1,5 +1,12 @@
 # Doubao visible watermark capture
 
+> **Status (captured 2026-05-29; alpha rebuilt 2026-05-31):** the black/gray/white captures were
+> taken and are now **committed** in `captures/` (solid colour + watermark, content-free). The alpha
+> map is rebuilt by `scripts/visible_alpha_solve.py doubao` (the careful gray-self solve shared with
+> Jimeng). The first build claimed "pixel-exact" but left a readable outline on the real sample (issue
+> #13 follow-up); removal now reverse-alphas, NCC-aligns, and applies a thin residual inpaint. See the
+> `doubao_engine.py` notes in the root `CLAUDE.md`. The text below is kept as the historical capture plan.
+
 Goal: capture the Doubao "豆包AI生成" visible watermark over known flat backgrounds so we can
 build a per-pixel alpha map and a reverse-alpha-blend remover, the same way the Gemini sparkle
 engine works (`src/remove_ai_watermarks/gemini_engine.py`).
@@ -16,8 +23,10 @@ engine works (`src/remove_ai_watermarks/gemini_engine.py`).
 - Size **scales with resolution**. Third-party numbers (~90x18 at <=1024, ~180x40 at >1024) are
   approximate and calibrated for ~1024-1280 outputs; at 2048 the strip is much larger. A shipped
   third-party alpha map is only 120x20, too small for our 2K/4K target -> capture fresh.
-- In practice clean inversion leaves residue on textured backgrounds, so the remover pairs the alpha
-  map with inpainting (our Gemini engine already does gradient-masked inpainting for residual edges).
+- The planning assumption was that clean inversion leaves residue on textured backgrounds, so the
+  remover would pair the alpha map with inpainting. After the capture this turned out unnecessary at
+  the native width (recovery is pixel-exact there and inpaint is off); the shipped remover is
+  reverse-alpha only, with a residual inpaint applied off-native only.
 
 ## Use doubao.com specifically
 

data/gemini_capture/README.md

@@ -0,0 +1,45 @@
+# Gemini (Nano Banana) visible sparkle capture
+
+> **Status (captured 2026-05-31):** black/gray/white captures taken and **committed**
+> in `captures/` (solid colour + the sparkle, content-free). The sparkle-on-black assets
+> `gemini_bg_{96,48}.png` are rebuilt by `scripts/visible_alpha_solve.py gemini`.
+
+Google Gemini (Nano Banana) stamps a four-point sparkle icon in the bottom-right corner
+via alpha compositing: `watermarked = a*logo + (1-a)*original`. Unlike the Doubao/Jimeng
+text marks, the sparkle is captured over a **pure-black** background, where
+`watermarked = a*255` (logo is near-white), so the alpha reads directly off the capture
+(`alpha = max(R,G,B)/255`) -- no background fit needed. This is the "golden" capture case.
+
+## What the captures confirmed (2026-05-31)
+
+- The sparkle at a 2048-wide image is **96x96 px** (width_frac ~0.047), bottom-right,
+  margins ~0.031, alpha max ~0.51 -- matching the engine's existing 96px asset.
+- Our own controlled capture matches the previously third-party-sourced
+  `gemini_bg_96.png` to **NCC 0.9998**, so the bundled asset is validated and now
+  reproducible from our own capture.
+
+## How to capture (image-edit path)
+
+For each solid-colour seed (`seeds/seed_{black,gray,white}_2048.png`, gitignored):
+
+1. Open Gemini image generation, image-edit / reference mode, upload the seed.
+2. Prompt: `Recreate this image exactly as it is, keep it identical, do not add or change anything`
+3. Download the ORIGINAL output (not a screenshot). Do not crop / edit / re-save.
+
+Black is the key one (sparkle on black -> exact alpha). Gray/white cross-check.
+
+## Naming, drop into `captures/`
+
+```
+gemini_black_2048.png   # the key capture (sparkle on black)
+gemini_gray_2048.png
+gemini_white_2048.png
+```
+
+The solid captures are **committed** (content-free). The synthetic `seeds/` and any
+real-content `gemini_content_*.png` validation download are gitignored (local-only).
+Rebuild the assets with:
+
+```
+uv run python scripts/visible_alpha_solve.py gemini   # or: all
+```

data/jimeng_capture/README.md

@@ -0,0 +1,75 @@
+# Jimeng (即梦AI) visible watermark capture
+
+> **Status (completed 2026-05-30):** solid black/gray/white Jimeng captures were
+> obtained (issue #13, from @powersee) and the alpha map was solved. Removal is
+> reverse-alpha plus a residual inpaint over the glyph footprint; see the
+> `jimeng_engine.py` notes in the root `CLAUDE.md`. The text below is kept as the
+> capture plan.
+
+Goal: capture the Jimeng / Dreamina "★ 即梦AI" visible wordmark over known flat
+backgrounds so we can build a per-pixel alpha map and a reverse-alpha remover, the
+same way the Gemini sparkle and Doubao strip engines work
+(`src/remove_ai_watermarks/gemini_engine.py`, `doubao_engine.py`).
+
+## What we learned (verified from the captures, 2026-05-30)
+
+- Mark: a four-point sparkle icon followed by the "即梦AI" characters, near-white
+  semi-transparent overlay, bottom-right corner.
+- Blend model: **alpha compositing with a pure-white logo** `watermarked =
+  a*255 + (1-a)*original`, confirmed in sRGB (a linear-light solve made the
+  black/gray cross-residual much worse, so the compositing is plain sRGB). An
+  L-pair-solve (independent of the L assumption) lands at ~254.6, confirming white.
+- **Alpha is solved from the GRAY capture**, not black: `a = (I - B)/(255 - B)`
+  with B a per-capture CUBIC background fit over the non-glyph pixels, averaged
+  over channels, at FULL halo extent (down to a~0.02) and UNBLURRED. Gray (bg ~132)
+  is the best proxy for real content (the mark sits on bright photo areas, not on
+  black). This careful build drops the gray self-residual to ~1.3; an earlier
+  max-channel / quadratic-bg / blurred / halo-truncated build (and a black-dominated
+  least-squares solve) left a visible outline -- the mask quality, not the method,
+  was the limit.
+- Geometry (fraction of image WIDTH, at the captured 2048): asset width ~0.211,
+  height ~0.068, right margin ~0.023, bottom margin ~0.023. The mark scales with
+  width; a real 1440-wide download matched width_frac ~0.21.
+- **Per-image render variation:** the alpha maps solved independently from the
+  black and the gray capture correlate 0.998 but not 1.0 (mean |Δa| ~0.02). Jimeng
+  re-rasterizes the mark per generation AND jitters its position a few px, so a
+  single alpha map does NOT pixel-cancel the mark the way Doubao's deterministic
+  overlay does. Removal therefore: NCC-aligns the alpha to the actual mark (always,
+  not only off-native), reverse-alphas, then clears the residual with a THIN inpaint
+  over the glyph footprint (a wide full-footprint pass smeared the texture/edges).
+
+## How to capture (image-edit path, most reliable)
+
+For each solid-color seed:
+
+1. Open Jimeng image generation, use the image-edit / reference mode, upload the seed.
+2. Prompt (Chinese preferred):
+   `请完全按照原图重新生成这张图片，保持完全一致，不要添加或修改任何内容`
+3. Download the ORIGINAL output file (not a screenshot). Do not crop / edit / re-save.
+
+The black capture is the key one (white logo on black -> `captured ~= a*255`); the
+gray capture refines the alpha at mid-tones; the white capture confirms the logo is
+pure white (the mark is nearly invisible on white, as expected).
+
+## Hygiene
+
+- Original download, never a screenshot. PNG preferred; if Jimeng only gives JPEG, note it.
+- No crop / edit / re-save. Default settings, watermark left ON.
+
+## Naming, drop into `captures/`
+
+```
+jimeng_cap_A.png   # black seed run through Jimeng
+jimeng_cap_B.png   # white seed
+jimeng_cap_C.png   # gray seed
+jimeng_content_1.png  # a normal-content download, for end-to-end validation
+```
+
+The solid `jimeng_cap_{A,B,C}.png` captures are **committed** (content-free: a solid
+colour + the watermark; the source for `scripts/visible_alpha_solve.py jimeng`). The
+synthetic `seeds/` and the real-content `jimeng_content_*.png` validation download are
+gitignored (local-only). Rebuild the alpha asset with:
+
+```
+uv run python scripts/visible_alpha_solve.py jimeng   # or: all
+```

data/qwen_in/README.md

@@ -0,0 +1,35 @@
+# qwen_in — pipeline-fidelity eval set
+
+A small, **stable** set of AI-generated images used to compare the diffusion
+removal pipelines (`controlnet` / `sdxl` / `qwen`) for fidelity with
+`scripts/fidelity_metrics.py`. Fixing the set in the repo keeps comparisons
+reproducible across runs and pipelines.
+
+All four are AI-generated test content (they carry SynthID + C2PA from their
+generator — verify with `remove-ai-watermarks identify`), same class as the
+`data/samples/` fixtures. No real-person photos.
+
+| file | vendor (SynthID) | content | exercises |
+|---|---|---|---|
+| `openai_1_original.png` | OpenAI | typography sheet (EN + RU + ZH) | text (multi-script) |
+| `openai_2_original.png` | OpenAI | Raiw.cc poster | text (EN, small) |
+| `gemini_1_original.png` | Google | landscape + Chinese sign | text (CJK) |
+| `gemini_3_original.png` | Google | 3x3 portrait grid | faces (identity / skin texture) |
+
+## Text ground truth
+
+`ground_truth.json` (`{basename: text}`) is the **hand-verified** OCR of the
+text-bearing originals, seeded by `fidelity_metrics.py ocr` and corrected by
+hand (PaddleOCR mis-reads stylized Cyrillic in particular). It is the reference
+for the text CER metric — much cleaner than OCR-vs-OCR. Regenerate the seed with:
+
+    uv run scripts/fidelity_metrics.py ocr data/qwen_in/openai_1_original.png \
+        data/qwen_in/openai_2_original.png data/qwen_in/gemini_1_original.png \
+        --langs en,ru,ch --out data/qwen_in/ground_truth.json
+    # then re-verify by hand before trusting it.
+
+## Compare
+
+    uv run scripts/fidelity_metrics.py compare \
+        --original data/qwen_in/gemini_3_original.png \
+        --variant controlnet=<out>.png --variant qwen=<out>.png --ocr-langs ""

data/qwen_in/ground_truth.json

@@ -0,0 +1,5 @@
+{
+  "openai_1_original.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行，也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
+  "openai_2_original.png": "raiw.cc\nRaiw.cc – The Platform for the Future\nRaiw.cc is a modern project built for those who value quality, innovation, and reliability.\nSecurity & Reliability\nWe use advanced technologies to protect your data and ensure stable service performance.\nInnovation\nWe continuously explore and implement better solutions to help you stay one step ahead.\nCommunity\nRaiw.cc is an active community of like-minded people who share knowledge, experience, and ideas.\nProgress Together\nWhether you're a developer, creator, or dreamer — raiw.cc is your partner in achieving your goals.\nOur mission is to create opportunities and inspire people to reach new heights.\nRaiw.cc\nVisit raiw.cc today\nExplore more possibilities and start your journey to the future!\nGet Started\nSecure\nInnovative\nCommunity\nForward-Thinking\nINNOVATE · CONNECT · CREATE · GROW",
+  "gemini_1_original.png": "每天都是一个新的机会。\n用微笑开始它。\n世界也会向你微笑。"
+}

data/samsung_capture/README.md

@@ -0,0 +1,66 @@
+# Samsung Galaxy AI visible watermark capture
+
+> **Status (built 2026-06-05):** flat black/gray/white Samsung Galaxy AI captures
+> were obtained (issue #37, from @f-liva) and the alpha map was solved. Removal is
+> reverse-alpha plus a thin residual inpaint over the glyph footprint; see the
+> `samsung_engine.py` notes in the root `CLAUDE.md`. The text below is the capture
+> plan and the open quality follow-up.
+
+Goal: capture the Samsung Galaxy AI "✦ Contenuti generati dall'AI" visible wordmark
+over known flat backgrounds so we can build a per-pixel alpha map and a reverse-alpha
+remover, the same way the Gemini sparkle and the Doubao / Jimeng strips work
+(`src/remove_ai_watermarks/gemini_engine.py`, `doubao_engine.py`, `jimeng_engine.py`).
+
+## What we learned (verified from the captures, 2026-06-05)
+
+- Mark: a sparkle icon followed by the locale string "Contenuti generati dall'AI"
+  (Italian), a light low-opacity (peak alpha ~0.38) semi-transparent **white**
+  overlay, anchored **bottom-LEFT** (Doubao/Jimeng are bottom-right). The string is
+  locale-specific, so the alpha template is per-locale; this build is the Italian
+  variant. Other locales need their own captured template.
+- Blend model: alpha compositing with a pure-white logo, `watermarked =
+  a*255 + (1-a)*original`, solved from the GRAY capture (same careful recipe as
+  Doubao/Jimeng: cubic-background fit, mean over channels, full halo extent,
+  unblurred). The white capture confirms the logo is white; on white the mark is
+  white-on-white and not detectable (no contrast), which is fine -- there is nothing
+  to recover there.
+- Geometry (fraction of image WIDTH): asset width ~0.32, height ~0.038, left margin
+  ~0.011, bottom margin ~0.006. The mark scales with width: a 1086-wide flat capture
+  and a 2958-wide real photo both measure width_frac ~0.31.
+- **Resolution caveat (open quality follow-up):** the flat black/gray/white captures
+  arrived at the phone's flat-edit size (1086 wide and a landscape 1920 set), while
+  the real photos are ~3000 wide, so the captured glyph (~334 px) is ~2.7x smaller
+  than on a real photo (~900 px). The alpha is solved at the capture size and
+  width-scaled + NCC-aligned per image, which removes the mark cleanly (verified on a
+  real 2958-wide photo: re-detect 0.79 -> 0.00, no readable text or outline), but a
+  flat capture taken at the real photo resolution (~3000 wide) would let the alpha be
+  pixel-sharp instead of upscaled. Not a blocker; a quality upgrade if a full-res
+  flat capture is provided.
+
+## Capture protocol (to re-capture or add a locale)
+
+On a Samsung Galaxy AI device (set the UI language to the target locale):
+
+1. Run the AI edit (Generative Edit / Sketch to Image) on a solid black image, so
+   the overlay lands on a flat black background. Download the ORIGINAL output file
+   (not a screenshot, no crop or re-save).
+2. Repeat over solid white and solid gray (those pin the exact glyph color).
+3. Ideally run all three flat edits at the same resolution as real photos (~3000
+   wide) so the alpha map is pixel-sharp rather than upscaled.
+4. Plus 3-5 real outputs with the visible mark over normal content for validation.
+
+## Files
+
+- `captures/samsung_black_1.png`, `samsung_gray_1.png`, `samsung_white_1.png` --
+  portrait flat edits (1086 wide), the primary calibration set.
+- `captures/samsung_black_2.png`, `samsung_gray_2.png`, `samsung_white_2.png` --
+  a second (landscape 1920) set.
+- `captures/samsung_content_*` -- real-photo validation downloads, **gitignored**
+  (user content, repo is public).
+- `seeds/` -- synthetic solid-color inputs, gitignored (regenerable).
+
+Rebuild the alpha asset with:
+
+```
+uv run python scripts/visible_alpha_solve.py samsung
+```

data/synthid_corpus/manifest.csv

@@ -24,3 +24,17 @@ d20d4cc936dbdfe909c52502039a9e84ba93d97b42b24a0acee5b7d6c71930ae,d20d4cc9-Gemini
 c86973424817f62510e2a312b85c52e05adf47ace87a8e717fd442607596f501,c8697342-aistudio_lake.png,pos,Google AI Studio (Nano Banana),gemini-2.5-flash-image,1024,1024,png,,,gemini-app,2026-05-24T21:39:09Z,"API/playground: SynthID pixel CONFIRMED (Gemini-app oracle) + visible sparkle, but NO C2PA/IPTC -> synthid_source blind spot"
 1f81827c06d67cf6f6c7f5d53ec8f9738183942a6d1d2717b161fea0fdcc540a,1f81827c-Designer.png,pos,Microsoft Designer,dall-e (Designer),1024,1024,png,"OpenAI, Microsoft",yes,c2pa-metadata,2026-05-24T22:18:40Z,C2PA issuer OpenAI+Microsoft; synthid_source=OpenAI (DALL-E surface inherits OpenAI SynthID+C2PA)
 f6dd47a5ffd319aea21bf10dcf9877097666420b02c2620080bac12b03976e7e,f6dd47a5-4ef377bd-gpt-image-2-cleaned.png,cleaned,"our pipeline (invisible/SDXL, native-res default)",stabilityai/stable-diffusion-xl-base-1.0,1254,1254,png,,,openai-verify,2026-05-25T20:50:38Z,"cleaned from 4ef377bd via v0.5.3 'all' at native 1254x1254 (prod-equivalent); openai.com/verify: SynthID NOT detected. Re-confirms #10 native-res default defeats OpenAI SynthID (closes #15 root cause). Note: native res OOMs on 20GB MPS, auto-fell back to CPU."
+05b836ecfe40fd689177fda74384ae4fdcc446505bbc4281cd3cbb6523eb669e,05b836ec-openai_1_original.png,pos,ChatGPT,gpt-image,1122,1402,png,OpenAI,yes,openai-verify,2026-06-04T00:07:53Z,June 2026 strength-study subject; openai.com/verify: SynthID detected (docs/synthid.md 2.2)
+794e023ea7ae321267fe5af76f4080c98a84a9865669c0733ebfb9757b8638df,794e023e-openai_2_original.png,pos,ChatGPT,gpt-image,1024,1536,png,OpenAI,yes,openai-verify,2026-06-04T00:07:53Z,June 2026 strength-study subject; openai.com/verify: SynthID detected (docs/synthid.md 2.2)
+28ff8732b037f98a4ef5bc277bbcdaa32e5eb9ccbd00b6c8c616e46ef68ae8a0,28ff8732-openai_3_original.png,pos,ChatGPT,gpt-image,1448,1086,png,OpenAI,yes,openai-verify,2026-06-04T00:07:53Z,June 2026 strength-study subject; openai.com/verify: SynthID detected (docs/synthid.md 2.2)
+4affd7f27767a445db6abf741355743ba8d95108ad922c9fff045feed8492236,4affd7f2-gemini_1_original.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,gemini-app,2026-06-04T00:08:05Z,June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
+8c1a6fb03ef3d45a1f958fb3401e4264e409ff88c2a793061db7f29023454d0e,8c1a6fb0-gemini_2_original.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,gemini-app,2026-06-04T00:08:05Z,June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
+45d79a683134fcba1b147b2aedb669783d474e1fb8a4df329729a0904fd1b46b,45d79a68-gemini_3_original.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,gemini-app,2026-06-04T00:08:05Z,June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
+2c33e75a2db614ce74c83cc0a6ac6c3ac735aca83ab88c9c9345843b124f7856,2c33e75a-gemini_4_original.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,gemini-app,2026-06-04T00:08:05Z,June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
+2e4ce41cfab456c1d9ea0898e47a5a1d434266ba24e88d4cc807a4180a56925f,2e4ce41c-openai_1_clean_s005.png,cleaned,"our pipeline (SDXL img2img, native)",stabilityai/stable-diffusion-xl-base-1.0,1122,1402,png,,,openai-verify,2026-06-04T00:08:05Z,cleaned at strength 0.05 native; openai.com/verify: SynthID NOT detected (docs/synthid.md 2.2)
+f7c52cdfeb14a6be2fff449e89b0181e66e365f36635ee4fcb21567e4cb770ef,f7c52cdf-openai_3_clean_s005.png,cleaned,"our pipeline (SDXL img2img, native)",stabilityai/stable-diffusion-xl-base-1.0,1448,1086,png,,,openai-verify,2026-06-04T00:08:05Z,cleaned at strength 0.05 native; openai.com/verify: SynthID NOT detected (docs/synthid.md 2.2)
+37b34274210888702e56eb74f4aa36578f15bf57157a3ebf394f0b8eaa820e19,37b34274-openai_2_clean_s010.png,cleaned,"our pipeline (SDXL img2img, native)",stabilityai/stable-diffusion-xl-base-1.0,1024,1536,png,,,openai-verify,2026-06-04T00:08:05Z,cleaned at strength 0.10 native (min captured for this subject); openai.com/verify: SynthID NOT detected
+f99bd9a51814265a23de467d14792db903fef99678b7e7c960d0c6813ed9b0fc,f99bd9a5-gemini_1_clean_s015_max1536.png,cleaned,"our pipeline (SDXL img2img, --max-resolution 1536)",stabilityai/stable-diffusion-xl-base-1.0,2816,1536,png,,,gemini-app,2026-06-04T00:08:05Z,cleaned at strength 0.15 --max-resolution 1536; Gemini-app: SynthID NOT detected (docs/synthid.md 2.2)
+4aa5f61c55c1f3fa9bbc49dffff8a404527722637ae694a932245629635b3f2b,4aa5f61c-gemini_2_clean_s015_max1536.png,cleaned,"our pipeline (SDXL img2img, --max-resolution 1536)",stabilityai/stable-diffusion-xl-base-1.0,2816,1536,png,,,gemini-app,2026-06-04T00:08:05Z,cleaned at strength 0.15 --max-resolution 1536; Gemini-app: SynthID NOT detected (docs/synthid.md 2.2)
+356196dd63abf011b30b582a0408ccecb726d746065af20ad3611dde72a88725,356196dd-gemini_3_clean_s015_max1536.png,cleaned,"our pipeline (SDXL img2img, --max-resolution 1536)",stabilityai/stable-diffusion-xl-base-1.0,2816,1536,png,,,gemini-app,2026-06-04T00:08:05Z,cleaned at strength 0.15 --max-resolution 1536; Gemini-app: SynthID NOT detected (docs/synthid.md 2.2)
+9e4160bb8e3e915d2d4593e37c71495ee7cfcec183602541166f622ebfd84403,9e4160bb-gemini_4_clean_s015_max1536.png,cleaned,"our pipeline (SDXL img2img, --max-resolution 1536)",stabilityai/stable-diffusion-xl-base-1.0,2816,1536,png,,,gemini-app,2026-06-04T00:08:05Z,cleaned at strength 0.15 --max-resolution 1536; Gemini-app: SynthID NOT detected (docs/synthid.md 2.2)

docs/controlnet-removal-pipeline-research.md

@@ -0,0 +1,644 @@
+# ControlNet-as-removal-pipeline research: can structure-conditioned regeneration scrub SynthID and keep text?
+
+Date: 2026-06-02. Source: a manual primary-source pass (WebSearch + WebFetch over the
+watermark-removal-attack and SDXL-ControlNet literature). Prompted by issue #35
+(@newideas99 / Jacob): "as we use SDXL even at low strength that kills small text ... Do you
+think ControlNet could be added to preserve and still remove the watermark?" Clarified scope:
+Jacob means **replacing the removal pipeline itself** with a ControlNet-conditioned
+regeneration (structure held by the control signal), NOT a separate text-protection add-on.
+
+A deep-research workflow run was attempted first (`wf_3244411d-ffd`) and failed at the harness
+level (97 agents completed without emitting StructuredOutput; ~4.3 M tokens, no report). This
+note is the hand-run replacement.
+
+## The question, precisely
+
+Can a single full-image ControlNet-conditioned diffusion pass **replace** plain SDXL base 1.0
+img2img as the watermark remover, so that one structure-guided regeneration removes the
+invisible robust pixel watermark (Google SynthID) **everywhere** while keeping fine detail and
+small/CJK **text** legible across the whole image? The hard constraint is unchanged from
+`text-protection-research.md`: the watermark must be scrubbed everywhere including inside text,
+so any path that freezes or composites original text pixels is disqualified.
+
+## Executive summary
+
+The idea is **already academically validated as a watermark remover and is literally what we
+already ship** — CtrlRegen (ICLR 2025) is a canny-ControlNet + DINOv2-semantic pipeline that
+regenerates from clean noise. But the make-or-break gap is exact: **none of the
+watermark-removal papers validate TEXT or fine-detail preservation at all** — CtrlRegen reports
+only FID/PSNR/quality-model scores and explicitly contains no text, fine-detail, or
+hallucination analysis. Our shipped ctrlregen's empirical failure ("destroys real content,
+hallucinates micro-text in smooth regions") is precisely this unstudied failure mode of the
+published method, most likely driven by our **512 px tiling** (text occupies too few pixels per
+tile to regenerate legibly; edge-free smooth regions get DINOv2-semantic hallucination). The
+constructive path is NOT to keep fixing the SD1.5 CtrlRegen, but to port the structure-control
+idea onto an **SDXL-native** ControlNet (xinsir tile-sdxl / ControlNet-Union-SDXL) as a control
+add to our existing SDXL base 1.0 img2img, run it at **1024+, not 512 tiles**, and empirically
+sweep the (denoise strength x conditioning scale x resolution) cube against the SynthID oracle
+AND a text-legibility check. The central tension may be fundamental and must be measured: the
+conditioning strong enough to keep text legible may suppress regeneration enough to let SynthID
+survive; the regeneration strong enough to scrub may deform text regardless of edges.
+
+## Oracle validation 2026-06-04 — measured answer (AUTHORITATIVE; supersedes pre-oracle "scrubs SynthID" claims below)
+
+The central tension the summary predicted is now MEASURED against the live oracles
+(OpenAI `openai.com/verify` for OpenAI content; the Gemini app "Verify with SynthID"
+for Google content — each detects only its own vendor's payload). **Verdict: at the
+shipped low vendor-adaptive strength, controlnet is NOT a drop-in SynthID remover.**
+It preserves structure so well that the watermark survives on exactly the photoreal
+content it protects. Controlnet is the text/structure PRESERVATION pipeline; removal
+is set by STRENGTH, separately calibrated, not by the pipeline choice.
+
+This section is the single consolidated reference for the controlnet pipeline's
+removal behavior. (Mirrored briefly in `docs/synthid.md` §5.5 and the CLAUDE.md
+controlnet / face_restore bullets, which point here.)
+
+### What we measured (real gpt-image + Gemini originals)
+
+**1. Content × pipeline — neither pipeline clears all content at low strength.**
+OpenAI set, strength = vendor-adaptive (0.10 OpenAI / 0.15 no-C2PA), `--max-resolution 1536`:
+
+| content | controlnet (`--auto`) | plain `default` |
+|---|---|---|
+| flat text card | clean | clean |
+| flat graphic (logo/poster, large flat fills) | clean | **SynthID detected** |
+| photoreal (9-face grid) | **SynthID detected** | clean |
+| photoreal (bracelet product photo) | **SynthID detected** | clean |
+
+Mechanism: controlnet's dense edge map keeps the regeneration very close to the original
+on photoreal, so the SynthID-destroying perturbation never happens; but it freely
+repaints large flat fills. Plain img2img at low strength perturbs photoreal texture
+enough yet barely touches flat fills. So the survivors FLIP by content type — pipeline
+choice alone does not guarantee removal.
+
+**2. Seed non-determinism near threshold.** img2img uses a random seed unless `--seed`
+is passed, and there is no local SynthID detector to self-verify. The bracelet survived
+controlnet @0.15 in one run and CLEARED @0.15 in another (same pipeline+strength+res).
+So a single clean run does NOT establish a strength as safe — characterizing a reliable
+floor needs a seed-repeatability sweep (N runs, varied seed), not one pass.
+
+**3. Per-vendor controlnet strength ladder (photoreal, oracle):**
+- **OpenAI:** 0.10 detected → 0.15 borderline/non-deterministic → 0.20 cleared (with margin).
+- **Gemini** (harder vendor; default 0.15 vs OpenAI 0.10): most cleared at 0.15–0.25.
+- **Resolution is NOT the lever:** SynthID is robust to downscaling, and the study's
+  trend says LOWER processing res needs LESS strength, so 1024 was never the wall. A
+  Gemini face that resisted 0.15/0.20/0.25 was blocked by face-restore (#4), not strength.
+
+**4. `--restore-faces` RE-INTRODUCES SynthID (was the "stubborn face" mystery).** A
+Gemini face image stayed SynthID-detected through controlnet 0.15/0.20/0.25 WITH restore,
+but CLEARED at 0.20 with `--no-restore-faces` (clean single-variable A/B). GFPGAN runs on
+the **ORIGINAL watermarked** face at fidelity weight 0.5, blends ~half its pixels with the
+StyleGAN2 prior, and composites that back OVER the diffusion-cleaned face → the watermark
+returns in the face region. Content-dependent (smaller faces can clear with restore). So
+raising strength cannot fix it — the face is re-pasted from the original after diffusion.
+This also corrects the prior "GFPGAN scrubs SynthID / oracle-confirmed clean" claim (it was
+checked on one lucky image).
+
+### Certified controlnet strength floors (Modal GPU sweep + oracle, 2026-06-04)
+
+Run via the isolated `raiw-controlnet-cert` Modal app (`raiw-app/modal_cert.py`):
+controlnet, `restore_faces` OFF (it re-introduces SynthID), `--max-resolution 1536`,
+each image checked on ITS OWN vendor oracle (OpenAI -> openai.com/verify, Gemini -> the
+Gemini app; the two payloads are vendor-specific and never cross-checked):
+
+| vendor | **floor** | evidence (oracle, restore OFF, <= 1536) |
+|---|---|---|
+| **OpenAI** | **0.20** | 2 photoreal images (9-face grid + bracelet) x seed {1,2,3} = **6/6 clean**; the bracelet that FLIPPED at 0.15 is seed-robust at 0.20 |
+| **Gemini** | **0.30** | hardest face (gemini_3): 0.20 detected -> 0.30 clean on **2/2 seeds**; Gemini is the harder vendor (default 0.15 vs OpenAI 0.10) AND resolution-sensitive |
+
+- **OpenAI 0.20 transfers to prod as-is** (OpenAI removal is resolution-independent:
+  the study clears it at 0.05 across 1024-1600).
+- **Gemini 0.30 is the floor at <= 1536 only.** Gemini is resolution-sensitive (study:
+  native 2816 likely needs >= 0.30 even on `default`), and **raiw.cc runs NATIVE**
+  (`max_resolution=0` in `modal_app.py`). So either CAP Gemini to <= 1536 in raiw.cc and
+  use 0.30, or run a native-resolution Gemini cert and expect a higher floor (~0.35+).
+
+### Recommendations for a removal pipeline (raiw.cc)
+
+- **Treat controlnet as PRESERVATION, not removal.** Choose it for text/structure content,
+  `default` for photoreal; removal efficacy comes from STRENGTH in both.
+- **Give controlnet a higher, per-vendor strength than `default`** (today both share
+  `resolve_strength` 0.10/0.15, tuned for plain img2img). **Certified controlnet floors:
+  OpenAI 0.20, Gemini 0.30** (see table above) — add a controlnet-specific per-vendor
+  schedule to `resolve_strength` rather than reusing the `default` ladder.
+- **Fix the seed in prod.** The non-determinism is purely `seed=None` (random); a fixed
+  `--seed` makes every run reproduce the certified-clean result, so you ship a
+  deterministic, re-certifiable config (and the seed sweep collapses to one config).
+- **`--restore-faces` is OFF in prod and stays opt-in.** Two methods ship
+  (`instantid` default, `photomaker`), both NON-COMMERCIAL. They REGENERATE the face
+  from an ArcFace embedding via SDXL diffusion, making the output face look more
+  AI-generated than the cleaned image (gloss, symmetric pores, SDXL "clean skin"
+  aesthetic). For production face preservation the cleaned image from controlnet 0.20
+  is the LEAST-AI state we can reach — any restore on top trades original-look for
+  embedding-driven regeneration. Empirical sweep summary: GFPGAN-on-cleaned polished
+  without identity recovery; PhotoMaker-V2 produced a different person; InstantID
+  txt2img produced studio-portrait patchwork on group photos; InstantID
+  img2img-on-cleaned with three parameter settings integrated scene context cleanly
+  but never recovered original identity precisely — every setting traded one problem
+  for another. See `docs/synthid-robust-identity-research-2026-06-08.md`
+  "Empirical follow-up" for the full sweep.
+- **No local SynthID detector exists** → the service can't self-verify; bake in strength
+  margin and periodic oracle spot-checks.
+- **Lesson:** visual-quality / face-identity recovery does NOT prove removal — only the
+  oracle does, across MULTIPLE content types; never conclude from a partial result (the
+  photoreal-only data first read as "controlnet shields, default removes"; the flat-graphic
+  result reversed it; the face mystery was restore, not strength).
+
+## Findings (with confidence and sources)
+
+### Finding 1 — confidence: high
+
+**Claim.** "ControlNet as the removal pipeline" is exactly CtrlRegen (ICLR 2025), and our
+shipped `ctrlregen` profile is a faithful implementation of it. Its **spatial control is canny
+edges** extracted from the watermarked image; its **semantic control is DINOv2-giant** via a
+trainable projection + decoupled cross-attention. Clean-noise (full-strength) regeneration
+scrubs the watermark from both pixel and latent space while the two control nets hold structure.
+
+**Evidence.** CtrlRegen: spatial control "conditioned on Canny edge images extracted from the
+watermarked image," integrated into the U-Net decoder blocks via a ControlNet structure;
+semantic control on "DINOv2-giant" embeddings. Removal is strong: TPR@1%FPR driven from 1.00 ->
+0.01 (StegaStamp) and 0.99 -> 0.12 (TreeRing). This matches our `ctrlregen/engine.py` exactly
+(canny detector + `facebook/dinov2-giant` + spatial ControlNet from `yepengliu/ctrlregen`).
+
+**Sources.** https://arxiv.org/html/2410.05470v1 · https://github.com/yepengliu/CtrlRegen ·
+https://openreview.net/forum?id=mDKxlfraAn
+
+### Finding 2 — confidence: high
+
+**Claim.** Regeneration provably removes any bounded-perturbation pixel watermark **given enough
+noise** — the operative constraint is the amount of regeneration, which is the same knob that
+trades against fidelity.
+
+**Evidence.** Zhao et al., "Invisible Image Watermarks Are Provably Removable Using Generative
+AI" (NeurIPS 2024): a noise-then-reconstruct regeneration attack "guarantees the removal of any
+invisible watermark" that perturbs the image within a bounded L2 distance. The guarantee is a
+function of injected noise magnitude — low noise preserves detail but leaves the watermark; high
+noise scrubs but discards original signal. This is the knob ControlNet conditioning is meant to
+make survivable (push regeneration high while the control signal holds composition).
+
+**Sources.** https://arxiv.org/abs/2306.01953 · https://github.com/XuandongZhao/WatermarkAttacker
+
+### Finding 3 — confidence: high
+
+**Claim.** The make-or-break gap: **no watermark-removal paper validates text or fine-detail
+preservation.** CtrlRegen's "high perceptual quality" is FID/PSNR/quality-model only and
+explicitly omits text, fine-detail, and hallucination analysis. So the literature does NOT
+support the specific claim Jacob needs (text survives), it is simply unmeasured.
+
+**Evidence.** CtrlRegen reports CLIP-FID, PSNR, Q-Align, LIQE; the fetched analysis confirms
+"the paper contains no discussion of text preservation, fine-detail retention, or hallucination
+artifacts," and "explicitly avoids discussing failure modes." Pixel metrics like PSNR are
+acknowledged not to reflect perception, and text legibility is a different axis than FID.
+
+**Sources.** https://arxiv.org/html/2410.05470v1
+
+### Finding 4 — confidence: medium-high
+
+**Claim.** Resolution is the prime suspect for our shipped ctrlregen's content destruction. We
+tile to **512 px** and run full clean-noise per tile; at 512 px text occupies too few pixels per
+tile to regenerate legibly, and smooth edge-free regions (no canny signal) are filled by the
+DINOv2 semantic prior, which hallucinates texture/micro-text. The paper omits resolution
+entirely, so this is an implementation regime it never characterized.
+
+**Evidence.** Our `ctrlregen/engine.py`: `PROCESS_SIZE = 512`, `TILE_SIZE = 512`, full strength
+on each tile. This mirrors the `_run_region_hires` insight (text needs MORE pixels under
+regeneration so strokes exceed the VAE's ~8 px latent floor), but ctrlregen runs the regeneration
+at LOW res, the opposite. CtrlRegen's paper gives no resolution/tiling spec to contradict this.
+
+**Sources.** internal (`src/remove_ai_watermarks/noai/ctrlregen/engine.py`); resolution-omission
+confirmed against https://arxiv.org/html/2410.05470v1
+
+### Finding 5 — confidence: high
+
+**Claim.** SDXL-native ControlNets exist, so the removal-pipeline upgrade need NOT be the SD1.5
+re-architecture our current ctrlregen is. xinsir `controlnet-tile-sdxl-1.0` and
+`controlnet-union-sdxl-1.0` (ControlNet++) run on SDXL base 1.0. The tile model has a `tile_var`
+image-variation mode purpose-built to regenerate detail while preserving structure, at
+`controlnet_conditioning_scale = 1.0`, optimal 1024 px. This is a drop-in control add to our
+existing SDXL img2img.
+
+**Evidence.** xinsir tile-sdxl model card: use cases = deblur/detail-repaint, **image variation
+(preserving structure)**, super-resolution; `controlnet_conditioning_scale = 1.0`, ~30 steps,
+optimal 1024x1024, works with `madebyollin/sdxl-vae-fp16-fix` (the same VAE our fp16 path
+already swaps in). ControlNet-Union-SDXL / ControlNet++ merges 10+ control types (canny, HED,
+tile, depth, lineart) into one SDXL model.
+
+**Sources.** https://huggingface.co/xinsir/controlnet-tile-sdxl-1.0 ·
+https://huggingface.co/xinsir/controlnet-union-sdxl-1.0 · https://github.com/xinsir6/ControlNetPlus
+
+### Finding 6 — confidence: high
+
+**Claim.** The community tile-ControlNet upscale workflow runs at **LOW denoise (0.3-0.4)** —
+the wrong regime for watermark removal. It preserves detail precisely by regenerating little, so
+a naive tile-upscale preserves text AND preserves the watermark. The open empirical question is
+whether at `conditioning_scale ~1.0` you can push denoise high enough to scrub SynthID while the
+tile conditioning still holds text — the exact cell to test.
+
+**Evidence.** Stable-Diffusion-Art ControlNet-tile upscale: denoise "typically 0.3, max ~0.4 to
+avoid artifacts"; some users push 0.6 with ControlNet strength 0.5. Our own data: SynthID
+survives below the removal-strength threshold (current Gemini needs notably higher denoise than
+the tile-upscale regime). So the detail-preserving regime and the watermark-scrubbing regime are
+on opposite ends of the denoise axis; ControlNet conditioning is the bet that they can meet.
+
+**Sources.** https://stable-diffusion-art.com/controlnet-upscale/ ·
+internal (`docs/synthid.md` strength data)
+
+### Finding 7 — confidence: high
+
+**Claim.** Forensic-stealth caveat: diffusion-based regeneration is among the MOST detectable
+removal families. Even a ControlNet-regeneration that fools the SynthID oracle leaves forensic
+traces flagging the output as "removal-processed" at >98% TPR@1%FPR. This bounds the claim (do
+not over-promise "indistinguishable from an original") but does not block the use case — the
+SynthID oracle still reads negative.
+
+**Evidence.** "Removing the Watermark Is Not Enough: Forensic Stealth in Generative-AI Watermark
+Removal" (arXiv:2605.09203, Goonatilake & Ateniese, GMU): across six removal attacks including
+diffusion-based regeneration, independent forensic detectors separate removal-processed from
+clean content at >98% TPR under a 1% FPR budget.
+
+**Sources.** https://arxiv.org/html/2605.09203v1
+
+### Finding 8 — confidence: low (watch, do not build on yet)
+
+**Claim.** Partial/semantic-guided regeneration is an active sub-direction that explicitly targets
+the removal-vs-fidelity tradeoff, but the specific fidelity-on-text claims were not verifiable
+from the source in this pass.
+
+**Evidence.** "Removing Watermarks with Partial Regeneration using Semantic Information"
+(arXiv:2505.08234) proposes focusing regeneration on watermarked regions with semantic (VLM)
+conditioning to preserve untouched areas; the PDF body did not render cleanly enough to confirm
+its quantitative text/detail results. Treat as a pointer, not evidence.
+
+**Sources.** https://arxiv.org/pdf/2505.08234
+
+## Recommendation / decision
+
+**ControlNet-as-removal-pipeline is worth prototyping — but not by fixing the SD1.5 ctrlregen.**
+Port the structure-control idea onto an SDXL-native ControlNet as a control add to the existing
+SDXL base 1.0 img2img, run it at full resolution (1024+, NOT 512 tiles), and treat the
+text-vs-scrub tension as an empirical question to measure, not assume.
+
+**Prototype (runs locally on 32 GB MPS — no dedicated GPU required):**
+
+Compute is NOT the bottleneck. On a 32 GB Apple-silicon machine (M5 here) native SDXL already
+runs entirely on MPS with no CPU fallback (~155 s at 1122x1402, verified — see `synthid.md` /
+CLAUDE.md). The prototype runs at **1024** (fewer pixels than that) with SDXL base + an SDXL
+ControlNet + activations in **fp32** (MPS fp16 decodes to all-black NaN — issue #29 — confirmed
+on run 1 below; fp32 is the required default on mps/cpu) — fits the 32 GB budget with vae-tiling +
+attention-slicing; ~1-2 min/image, so a coarse sweep is a sub-hour background run. A dedicated GPU
+is needed ONLY for the separate
+native-large-Gemini (2816 px) case, which OOMs even without a ControlNet (that stays a raiw.cc
+GPU task). The genuine external dependency is NOT compute but the **manual SynthID oracle**:
+there is no local SynthID detector, so removal is verified by hand in the Gemini app
+("Verify with SynthID") per image, regardless of where the diffusion runs.
+
+Runner: **`scripts/controlnet_sweep.py`** (built 2026-06-02) implements exactly this sweep —
+SDXL base 1.0 + an SDXL-native ControlNet img2img, one output per (control x strength x scale)
+cell, plus a `sweep_index.csv` with empty `synthid_oracle` / `text_legible` columns to fill by
+hand. It uses the dedicated single-type xinsir models (`controlnet-canny-sdxl-1.0`,
+`controlnet-tile-sdxl-1.0`) rather than the Union model to keep the diffusers API path robust.
+
+    uv run python scripts/controlnet_sweep.py watermarked.png -o sweep_out
+
+1. SDXL base 1.0 img2img + `xinsir/controlnet-canny-sdxl-1.0` / `controlnet-tile-sdxl-1.0`
+   (sweep both `tile` and `canny` control), full image at 1024, `sdxl-vae-fp16-fix`.
+2. Sweep the cube on fresh Gemini + gpt-image inputs that contain small/CJK text:
+   - denoise strength {0.15, 0.3, 0.5, 0.7, 1.0}
+   - `controlnet_conditioning_scale` {0.5, 0.8, 1.0}
+   - control type {tile, canny}
+3. Per cell, measure BOTH axes:
+   - **removal**: Gemini app "Verify with SynthID" oracle (the only valid SynthID oracle; for
+     gpt-image also openai.com/verify for provenance) — must read clean.
+   - **text**: OCR round-trip / visual legibility of the small text.
+   - secondary: SSIM/FID vs original for global fidelity.
+4. Find the Pareto cell where the oracle is clean AND text stays legible.
+
+**The honest fork the prototype resolves:**
+- If such a cell exists -> the answer to Jacob is YES, ship an SDXL-native ControlNet removal
+  profile (replacing the SD1.5 ctrlregen) tuned to that cell.
+- If no cell clears both (the tension is fundamental: scrub-strength always deforms text, or
+  text-preserving conditioning always spares the watermark) -> the canny/tile-ControlNet middle
+  path is dead for text, and the standing answer reverts to `text-protection-research.md`: a full
+  **glyph-conditioned re-render** (EasyText / TextSR on a FLUX-DiT base) is required, which is a
+  base-model migration, not a control add.
+
+**Do not:** keep tuning the 512 px SD1.5 ctrlregen for text (wrong resolution, wrong base model);
+run tile-ControlNet at the community 0.3-0.4 upscale denoise and expect watermark removal (that
+regime preserves the watermark); over-claim forensic invisibility (Finding 7).
+
+## Prototype run 1 — 2026-06-02 (text axis measured; watermark axis pending the oracle)
+
+First sweep on a real, SynthID-positive, text-dense input: the corpus tokyo-street-night
+gpt-image (`88e61a38-chatgpt_tokyo.png`, 1023x1537 -> 680x1024, dense small CJK + Latin neon
+signage; SynthID + C2PA confirmed, so its valid oracle is openai.com/verify). Grid: control
+{canny, tile} x strength {0.3, 0.5, 0.7, 1.0} x `conditioning_scale` 1.0, fp32 on MPS. Outputs +
+`sweep_index.csv` (text verdicts filled by visual inspection; `synthid_oracle` left for the
+manual run) are under `/tmp/cnsweep/` (not committed — derived regenerations of corpus content).
+
+**Measured — PSNR vs input (proxy for how much was regenerated):**
+- canny: 0.3 -> 16.91, 0.5 -> 15.91, 0.7 -> 14.82, 1.0 -> 13.22 (monotonic drop = progressively
+  more regeneration as strength rises; canny only pins edges, so flat regions change).
+- tile: 0.3 -> 17.89, 0.5 -> 17.84, 0.7 -> 17.83, 1.0 -> 17.74 (**flat and high — near-identity
+  even at strength 1.0**; tile@scale1.0 pins the whole image to the input and barely regenerates).
+
+**Measured — text legibility (visual, focused on SMALL text; large high-contrast glyphs survive
+everything because canny/tile hold their edges):**
+- canny: legible at 0.3, softening at 0.5 (partial), garbling at 0.7, hallucinated pseudo-glyphs
+  at 1.0 ("NEC" -> "NWENES"). Same plain-img2img small-text deformation, only big text protected.
+- tile: near-identity through 0.7, only tiny alterations at 1.0 — small text preserved throughout.
+
+**Reading (the make-or-break tension, now visible in the data):**
+- **tile@scale1.0 does not actually regenerate** (flat PSNR), so it preserves all text but almost
+  certainly leaves the watermark intact — it is a near-identity pass, exactly the community
+  "tile-upscale preserves detail by not regenerating" regime (Finding 6), confirmed.
+- **canny@scale1.0 regenerates progressively** (PSNR drops) and so could scrub — but small text
+  breaks at exactly the strength where scrubbing would start to bite. canny saves big edges, not
+  sub-stroke small text.
+- Net on the text axis: neither cell at scale 1.0 cleanly gives "high regeneration + legible small
+  text." This is the literature prior (Findings 3, 6) reproduced empirically. Lowering
+  `conditioning_scale` to force small-text regeneration is the same tradeoff knob, not an escape.
+
+**Still pending (the decisive half, cannot be done locally):** run the 8 cells through the SynthID
+oracle and fill `synthid_oracle`. The most informative cells: canny 1.0 (text dead — does it at
+least scrub? if not, the canny path is dead outright), canny 0.5 (text partial — does it scrub?),
+tile 1.0 (text perfect — predicted to still read present). If no cell is `oracle=clean` AND
+`text=yes`, the fork resolves to the glyph-re-render path (`text-protection-research.md`).
+
+**Incidental bug caught:** the first run used fp16 on MPS (the script's original default) and
+produced **all-black** outputs across every cell (2 KB PNGs, PSNR 9.22 flat) — the issue #29
+fp16-VAE-NaN failure, and the fp16-fix VAE did not save it on MPS. Fixed `scripts/controlnet_sweep.py`
+to default fp32 on mps/cpu (fp16 only on cuda/xpu), matching the production pipeline.
+
+## Tuning ControlNet for text preservation across image types (research 2026-06-03)
+
+Goal: how to configure the canny-ControlNet path to best preserve text (and faces) on diverse
+images. Primary sources: diffusers ControlNet doc, the ControlNet paper (arXiv:2302.05543),
+xinsir model cards, practitioner guides. The **critical reframe**: almost all community ControlNet
+advice optimizes a txt2img *generation* tradeoff (control vs creative freedom). OUR context is
+img2img *watermark removal*, where the objective is the opposite -- maximum faithful preservation
+while regenerating just enough to scrub. So several common recommendations INVERT here.
+
+**Removal is `strength`; everything below is preservation and does not change removal efficacy**
+(only the watermark-shielding risk -- see the caveat). Set `strength` by the oracle/vendor need;
+tune these to keep text/faces intact at that strength.
+
+Knobs, ranked by impact for text:
+
+1. **Canny edge density (the per-image lever, currently hardcoded `_CANNY_LOW=100`/`_CANNY_HIGH=200`).**
+   Lower thresholds capture more/finer edges; higher thresholds keep only major outlines (diffusers
+   doc + practitioner guides; ControlNet paper uses 100/200 as the default). Small-text strokes and
+   fine facial features fall below the default 100/200 and are missed. **For dense small text
+   (infographics, signage) lower the thresholds (~50/120, even 30/100 for facial likeness per
+   practitioner tests); for high-contrast large text 100/200 already suffices.** Denser canny is
+   still a BINARY thresholded edge map, so it does not carry the low-amplitude SynthID pixel pattern
+   -- it passes more shape, not the watermark (still oracle-verify). This is the single highest-value
+   unexplored lever and should become a CLI knob.
+
+2. **`controlnet_conditioning_scale` -> keep at 1.0 (max structure hold).** Community defaults to 0.5
+   for creative balance; we want maximum preservation, so 1.0 (xinsir canny/tile cards also recommend
+   1.0). We measured text on a clean high-contrast image surviving across strength 0.1-0.5 at scale
+   1.0 (PSNR ~26 flat), so scale 1.0 is the right default; only lower it if a specific image needs
+   more regeneration to scrub (raises shielding risk the other way).
+
+3. **`control_guidance_start=0.0`, `control_guidance_end=1.0` (full window) -- KEEP, do not shorten.**
+   The common "end=0.5: establish structure early then let the model render detail freely" is a
+   creative-generation recipe; for text it is HARMFUL -- the late free steps re-render and deform the
+   glyphs. We want the edge control active through ALL denoise steps so text stays pinned. (Our
+   pipeline already uses the 0->1 default; the point is to NOT adopt the shorten-the-window advice.)
+
+4. **Control type, per image type:**
+   - **Text / graphics / high-contrast -> canny** (the literature's reliable choice for defined edges
+     and text; what we ship).
+   - **Faces / smooth tonal content -> soft-edge / HED is a candidate worth testing.** Canny's hard
+     binary threshold fractures smooth skin gradients; HED/soft-edge gives gradual edges that may hold
+     faces better. UNVERIFIED for removal (softer edges may carry slightly more original signal ->
+     oracle-check). A face-heavy image is the test (gemini group photos).
+   - **tile -> NOT for removal.** It is near-identity (detail-enhancement at low denoise); it shields
+     the watermark (measured flat PSNR ~17.8 across strength on the tokyo sweep). Do not use it as the
+     removal control.
+
+5. **Resolution** -- higher long-side = strokes span more VAE latent cells = less softening, while
+   still fully regenerating. Already a knob (`--max-resolution`); for tiny text prefer native/large.
+
+**Multi-ControlNet (canny + soft-edge), list scales e.g. `[1.0, 0.8]`** (diffusers MultiControlNet):
+could hold text edges AND face geometry at once, but doubles ControlNet memory/latency and raises the
+shielding risk; defer to a v2 after the single-canny path is dialed in.
+
+**Image-type playbook (proposed, to validate with the oracle):**
+- Clean high-contrast text (openai_1-style): canny 100/200, scale 1.0, full window -- already optimal.
+- Dense small text / infographics (big_pic3, neon signage): canny **lower thresholds (~50/120)**,
+  scale 1.0, full window, larger resolution.
+- Faces / portraits: try **soft-edge/HED** control, scale 1.0; or multi-ControlNet canny+softedge.
+
+**Hard caveat:** every change that increases preservation (higher scale, denser canny, fuller window,
+softer edges) marginally REDUCES effective regeneration and so raises the chance the watermark
+survives -- exactly the shielding failure mode. There is no local SynthID detector, so each tuning
+change must be re-confirmed on the oracle. These are img2img-context recommendations derived from
+generation-context sources plus our own measurements; treat the playbook as hypotheses to verify, not
+settled defaults.
+
+**Sources.** https://huggingface.co/docs/diffusers/en/using-diffusers/controlnet ·
+https://arxiv.org/pdf/2302.05543 · https://huggingface.co/xinsir/controlnet-canny-sdxl-1.0 ·
+https://huggingface.co/xinsir/controlnet-tile-sdxl-1.0 · https://blog.cephalon.ai/canny-and-softedge/
+
+## FaceID research: identity-preserving face conditioning (research 2026-06-03)
+
+Motivation: canny alone preserves face STRUCTURE/position better than plain SDXL but does NOT hold
+IDENTITY -- verified on a real Gemini group photo (gemini_3, s015): faces drift in expression and
+likeness (the smile/mouth and eyes change), they are "a similar person," not the same one. Canny
+carries edges, not identity, so the regenerated face is identity-drifted. To hold identity WITHOUT
+copying original pixels (the hard constraint -- copied pixels carry SynthID), the conditioning must
+be an identity EMBEDDING, not pixels. Primary sources: diffusers IP-Adapter doc, InstantID
+(arXiv:2401.07519), IP-Adapter (arXiv:2308.06721), practitioner comparisons.
+
+### Findings
+
+**1. IP-Adapter FaceID conditions on an ArcFace identity VECTOR, not pixels (confidence: high).**
+FaceID extracts `insightface` ArcFace `normed_embedding` (a ~512-d identity vector) via
+`FaceAnalysis`, and passes it as `ip_adapter_image_embeds` -- NOT a CLIP image embedding, NOT the
+original pixels. So it is constraint-compatible: the watermark (a pixel-amplitude pattern) is not in
+the identity vector, and the img2img still regenerates the pixels (removal via `strength` unchanged).
+It loads on any SDXL via `load_ip_adapter` (~100 MB), is fast/low-VRAM, but identity fidelity on SDXL
+is ~5-10% lower than the SD1.5 line / dedicated methods.
+
+**2. Multiple distinct faces ARE handled, via regional attention masks (confidence: high -- THE key
+unlock).** This is the make-or-break for group photos (our hardest case). diffusers supports a LIST
+of IP-Adapter face images each with its own binary region mask: `IPAdapterMaskProcessor` builds the
+masks, `set_ip_adapter_scale([[s1, s2, ...]])`, and `cross_attention_kwargs={"ip_adapter_masks":
+masks}`. So you detect each face, extract its own ArcFace embedding, assign it a region mask, and one
+pass preserves N different identities simultaneously. (InstantID, by contrast, is single-subject --
+it averages embeddings for multiple refs, which is wrong for distinct people -- so for group photos
+**IP-Adapter FaceID + masks beats InstantID**.)
+
+**3. IP-Adapter + ControlNet + img2img compose (confidence: high).** The doc shows IP-Adapter +
+ControlNet (depth) in one pipeline and IP-Adapter + img2img (`strength`). Our target stack is the
+union: `StableDiffusionXLControlNetImg2ImgPipeline` (canny = structure) + `load_ip_adapter` (FaceID =
+identity) + `strength` (removal). `set_ip_adapter_scale` (1.0 = image-only, 0.5 = balanced) is the
+identity-hold knob. API friction to verify in implementation: that `ip_adapter_masks` via
+`cross_attention_kwargs` works on the *ControlNet img2img* pipeline (the masking is an attention-
+processor feature, so it should be pipeline-agnostic, but confirm).
+
+**4. InstantID / PuLID positioning (confidence: medium).** InstantID does not train the UNet so it
+composes with canny/depth ControlNets, and gives better single-face fidelity than FaceID -- but it is
+single-subject (needs its own landmark ControlNet + dedicated weights). PuLID has the best identity
+fidelity but is heaviest and Flux-leaning. For our multi-face, constraint-bound, SDXL-canny case,
+IP-Adapter FaceID + masks is the right first build; InstantID/PuLID are single-portrait upgrades.
+
+### Architecture (proposed)
+
+```
+detect faces (insightface) -> per face: ArcFace embed + region mask
+one img2img pass:
+  image=init, control_image=canny(init),                      # structure (existing)
+  ip_adapter_image_embeds=[face_embeds],                       # identity per face
+  cross_attention_kwargs={"ip_adapter_masks": face_masks},     # each face -> its region
+  controlnet_conditioning_scale=1.0, set_ip_adapter_scale(~0.6),
+  strength=vendor-adaptive                                     # removal (unchanged)
+```
+Pixels are regenerated (SynthID removed by `strength`), structure held by canny, each face's identity
+held by its masked ArcFace vector -- no original pixel copied.
+
+### Risks / honest costs
+
+- **Shielding risk (same wall):** FaceID conditioning, like canny, reduces effective regeneration ->
+  higher `set_ip_adapter_scale` raises the chance SynthID survives in the face region (echo of why the
+  old region-hires failed). MUST oracle-verify removal at the chosen FaceID scale; keep `strength` at
+  the vendor threshold.
+- **New heavy dependency:** `insightface` + `onnxruntime` + the `buffalo_l` model (~300 MB, downloaded
+  on first use). Detection + embedding is CPU/ONNX, separate from the diffusion.
+- **Detection floor:** insightface needs faces large enough (det_size ~640); tiny faces in a dense
+  group may not be detected -> not preserved (falls back to canny-only for those).
+- **Identity ceiling:** SDXL FaceID is ~5-10% off true identity -- a meaningful boost over canny-only
+  drift, NOT a perfect face swap. Set expectations; PuLID/InstantID are the higher-fidelity (heavier)
+  paths if needed.
+- **Value scales with strength:** at low strength (OpenAI 0.10) faces barely drift, so FaceID is
+  marginal; at the higher strength a hard vendor (Google 0.30) needs, FaceID earns its keep.
+
+### Build plan (staged)
+
+- v1: optional `--face-id` flag on `--pipeline controlnet`. Detect faces; if any, run the masked
+  FaceID pass (works for 1 or N faces -- masks generalize). If none detected, fall through to plain
+  canny. Oracle-verify SynthID removal is preserved at the default FaceID scale on a face image.
+- v2 (if identity still short): InstantID for single-portrait, or PuLID, as a higher-fidelity opt-in.
+
+**Sources.** https://huggingface.co/docs/diffusers/main/en/using-diffusers/ip_adapter ·
+https://huggingface.co/h94/IP-Adapter-FaceID · https://arxiv.org/pdf/2401.07519 (InstantID) ·
+https://instantid.github.io/ · https://arxiv.org/abs/2308.06721 (IP-Adapter)
+
+### FaceID prototype run 1 -- 2026-06-03 (NEGATIVE on dense small-face groups)
+
+Built and shipped the masked multi-face FaceID layer (`--face-id`, `face_id.py`, `faceid` extra).
+First real run on the gemini_3 group photo (Google, s015, scale 0.6, native 2816 via cap 1536):
+insightface detected **17 faces**, the masked multi-face pass composed and ran end-to-end (non-black
+output), so the API is correct. **At s015 the result is a clear FAILURE: every face corrupted --
+melted/discolored/psychedelic, materially WORSE than canny-only.**
+
+**ROOT CAUSE FOUND (confirmed by ablation, not speculation) -- it is STRENGTH, not scale/masks/faces.**
+Investigated the real data: masks are fine (max overlap depth 2, 33% coverage, only 0.2% of pixels
+double-covered -- NOT an overlap problem), embeddings are fine (`normed_embedding` norm 1.000), the
+FaceID LoRA is not required for SDXL (h94 model card), and faces span 34-181 px (7 medium + 10 tiny).
+None of those is the cause. The decisive test: the SAME image + FaceID at **strength 0.5** produces
+**clean, coherent faces across the whole group** (no psychedelic artifacts). So FaceID needs
+substantial regeneration: the h94 usage is full generation (txt2img, 30 steps); at our removal
+strength (0.10-0.15 = ~7 effective steps) the strong identity cross-attention cannot reconcile with
+a latent that is ~85% the untouched original, so it smears identity-colored noise onto the faces.
+
+**This is a FUNDAMENTAL tension, not a tuning bug:** watermark removal wants LOW strength (minimal
+degradation, just enough to scrub), FaceID wants HIGH strength (regenerate the face to impose
+identity). They are opposed. At strength 0.5 FaceID works AND removes the watermark, but the whole
+image regenerates much more (canny still holds text/edge structure, but texture/detail drifts well
+beyond the 0.15 "minimal degradation" target). So `--face-id` is a HIGH-STRENGTH option: it trades
+whole-image fidelity for face identity, and is a footgun at the low default strength (guaranteed
+garbage). Required follow-up code guard: when `--face-id` is set, floor `strength` at ~0.5 (or refuse
++ warn) -- never run FaceID at the vendor-adaptive removal strength. Open question: whether
+high-strength FaceID's whole-image drift is acceptable for face-centric images, or whether identity
+preservation at LOW strength needs a different mechanism entirely (FaceID structurally cannot do it). (Infra lesson: the `faceid` extra must
+stay numpy<2.0 -- pin `onnx<1.18` + `scipy<1.18`; pinning numpy UP, as the first build did, leaves a
+numpy-1.26 env with a numpy-2-only scipy that crashes the diffusers import via `np.long`.)
+
+## Face preservation, done properly (research 2026-06-03, after the FaceID failure)
+
+The FaceID run failed and I wrongly concluded "faces can't be preserved." Re-research corrected the
+understanding. The hard constraint is unchanged: to remove the watermark FROM a face the face MUST
+be regenerated (freezing it leaves SynthID), so the goal is identity-preserving REGENERATION of the
+face, at minimal overall image degradation. Three things I got wrong and the corrected picture:
+
+**What I got wrong:** (1) I applied FaceID at GLOBAL high strength -- the literature is clear the
+architecture must be REGION-ADAPTIVE (face region handled separately, background stays low-strength);
+(2) I used IP-Adapter FaceID, the WEAKEST identity tool -- InstantID uses an ArcFace encoder and hits
+82-86% face-recognition similarity vs FaceID's weak CLIP-ish signal; (3) I missed the entire
+face-restoration class (CodeFormer / GFPGAN), which is purpose-built for "regenerate a face, keep
+identity."
+
+**The most promising mechanism -- CodeFormer face-restoration post-pass (confidence: high on the
+mechanism, unverified on our watermark).** CodeFormer is a VQ-VAE: a frozen discrete CODEBOOK of HQ
+facial priors + a Transformer that predicts code *tokens* from the input, and a frozen decoder that
+regenerates the face FROM THE CODEBOOK ENTRIES -- "does not depend on feature fusion with low-quality
+cues." So the output face pixels come from a finite learned codebook, NOT from the input pixels:
+**the SynthID pixel-amplitude pattern physically cannot survive a codebook re-synthesis** -- a
+stronger scrub than low-strength img2img (which keeps ~85% of the latent). Fidelity knob `w` in
+[0,1]: higher w preserves identity but fuses MORE low-quality (input) cues (more watermark risk),
+lower w leans on the codebook (cleaner scrub, identity drift) -- the same scrub-vs-fidelity tension,
+settled per-image by the oracle; there is likely a `w` that holds identity AND clears the oracle.
+
+**Constraint-compatible architecture:** run the normal canny low-strength controlnet removal globally
+(minimal degradation everywhere), then detect+align each face, run CodeFormer on the **ORIGINAL** face
+crop (to capture true identity AND re-synthesize from the codebook = scrub), and composite the
+CodeFormer output (codebook-generated, not original pixels -> no copy, no watermark) into the cleaned
+image. Decouples whole-image minimal-degradation from face identity -- no high GLOBAL strength needed.
+
+**Honest costs/caveats:** (a) **License -- CodeFormer is NTU S-Lab 1.0 (non-commercial/research)**, so
+it cannot be bundled in this MIT tool for general use; the license-clean alternative is **GFPGAN
+(Apache-2.0)**, slightly lower quality. (b) Deps (basicsr/facexlib) are heavy and numpy-version-finicky
+(same class of conflict as insightface). (c) CodeFormer is a *restoration* model -- it can subtly
+alter expression/asymmetry; identity is held but not pixel-identical. (d) **The watermark-scrub is
+mechanistically strong but UNVERIFIED -- must oracle-check.** InstantID + region-adaptive strength is
+the alternative if the restoration route disappoints, but it is more complex (differential strength).
+Prototype plan: validate CodeFormer on a real face in a THROWAWAY env (identity held? oracle clean?)
+before any project-env integration or the license/GFPGAN decision.
+
+### CodeFormer prototype -- VALIDATED end-to-end 2026-06-03 (oracle-confirmed)
+
+Prototyped the CodeFormer face-restoration post-pass (codeformer-pip in a throwaway venv, forced CPU
+-- the pip wrapper has an MPS device-mismatch bug) on the gemini_3 group photo (18 faces). Pipeline:
+`all --pipeline controlnet --strength 0.15` (sparkle + SynthID removed from the whole image, minimal
+degradation) -> CodeFormer on the ORIGINAL faces -> feather-composite the CodeFormer faces into the
+all-cleaned image. Oracle results (Gemini app "Verify with SynthID"), isolating each part:
+- pure controlnet-0.15 background (no faces): **clean** -> the background scrub works at 0.15 (no
+  ControlNet-shielding problem for Google on this image).
+- composite with CodeFormer faces at **w0.7**: **SynthID DETECTED** -> high fidelity fuses too much of
+  the original face signal (the watermark) through.
+- composite at **w0.5**: **clean**. composite at **w0.3**: **clean**.
+So the scrub-vs-fidelity threshold is between 0.5 and 0.7; **w=0.5 is the sweet spot** (highest
+fidelity / best identity that still clears the oracle). Identity at w0.3-0.7 all looks like the same
+person (the face is large enough), so the lower w costs little.
+
+**This VALIDATES the corrected face-preservation approach** (and refutes my earlier "faces can't be
+preserved" / FaceID conclusion): controlnet low-strength background scrub + CodeFormer-codebook face
+re-synthesis at w~0.5 + feather composite = oracle-clean SynthID removal everywhere (background AND
+faces), identity preserved, minimal overall degradation, zero original-pixel copying (CodeFormer faces
+are codebook-generated). CodeFormer's discrete-codebook re-synthesis DOES scrub the pixel watermark,
+but only when w is low enough that the decoder leans on the codebook rather than fusing the input
+(watermark-carrying) features -- exactly the predicted fidelity-vs-scrub tension, with an empirical
+clean threshold at w<=0.5.
+
+**Production TODO (not built -- still a throwaway prototype):** (1) license -- CodeFormer is NTU S-Lab
+(non-commercial); decide CodeFormer-as-user-installed-extra vs GFPGAN (Apache-2.0, re-verify it scrubs
+at its fidelity setting); (2) wire a `--restore-faces` post-pass (detect -> restore w~0.5 -> feather
+composite) onto the controlnet pipeline; (3) handle the MPS device bug (force CPU for the face model
+or fix); (4) re-verify the w threshold on more images / vendors (w=0.5 confirmed on one Gemini group
+photo only).
+
+**Sources.** https://arxiv.org/abs/2206.11253 (CodeFormer) · https://github.com/sczhou/CodeFormer ·
+https://arxiv.org/pdf/2401.07519 (InstantID) ·
+https://openaccess.thecvf.com/content/WACV2024/papers/Suin_Diffuse_and_Restore... (region-adaptive) ·
+https://arxiv.org/pdf/2504.12809 (saliency-aware watermark removal)
+
+## Provenance
+
+Hand-run primary-source pass, 2026-06-02. Sources fetched and quoted above; the central
+make-or-break claim (structure-conditioned high-strength regeneration scrubs the watermark while
+keeping text) is **unverified and explicitly flagged as the thing the local prototype must
+measure** (against the manual Gemini SynthID oracle) — the literature supports removal (Findings 1, 2) and supports structure-preserving
+regeneration (Finding 5) but never jointly validated text (Finding 3). No code change implied
+until the prototype validates a Pareto cell on the SynthID oracle.

docs/known-limitations.md

@@ -0,0 +1,177 @@
+# Known limitations: full detail
+
+> Relocated verbatim from `CLAUDE.md` on 2026-06-11 to keep the always-loaded
+> context small. Long single-line entries were reformatted into paragraphs;
+> no content was changed or summarized.
+
+Full detail behind the compact Known-limitations list in `CLAUDE.md`:
+measurements, incident history, oracle runs, and the reasoning behind each
+decision. Read the relevant section here before changing the diffusion
+pipelines, strength defaults, or metadata coverage.
+
+## Visible-mark fill quality is background- and backend-dependent
+
+Visible-mark removal is localize -> fill (the reverse-alpha pixel recovery was
+dropped; see `docs/module-internals.md`). The fill only touches the mark's
+footprint, so there is never collateral damage outside it, and whether the mark
+is *removed* is fill-independent -- cv2, MI-GAN and LaMa all strip the mark's
+shape. What varies is the *quality of the recovered region*, and it depends on
+the background:
+
+- **Flat backgrounds:** every backend is clean; cv2 is often the crispest.
+- **Textured / regular-structured backgrounds** (fabric, foliage, a lattice or
+  grid): an inpaint can only guess the hidden pixels. `cv2` (the classical
+  no-deps floor) visibly smears; `migan` (light, learned) can leave a ghost or
+  hallucinate structure; `lama` (heavy, learned) is the most reliable and
+  recovers structure best.
+
+The old reverse-alpha recovered the *true* pixels under a well-captured, static
+mark, so on structured backgrounds it was sometimes cleaner than any inpaint.
+The trade for localize -> fill is robustness (it also handles moved / re-rendered
+marks and needs no per-mark alpha capture) and a simpler, swappable pipeline.
+`auto` resolves best-first (`LaMa > MI-GAN > cv2`) and warns once when it falls
+back to cv2 because no learned backend is installed; a memory-tight deployment
+that cannot afford LaMa's ~4.7 GB peak pins `--backend migan` explicitly.
+
+## Invisible-pipeline resolution handling (native / 1024 floor / `--max-resolution`; MPS memory tiers)
+
+`invisible` pipeline processes at **native resolution for inputs whose long side is >= 1024px**, and **auto-upscales smaller inputs UP to a 1024px floor** (`min_resolution=1024`, the default; `--min-resolution 0` disables) before diffusion -- SDXL img2img distorts badly on a tiny latent (a 381x512 portrait wrecks at native, the #36 follow-up), and the output is restored to the original input size so the floor is a transparent quality boost (it adds time/memory on small inputs). The floor upscale uses Lanczos by default; **`--upscaler esrgan`** (opt-in, the `esrgan` extra) runs Real-ESRGAN first for better detail before the Lanczos resize to the exact target (`upscaler.py` / `InvisibleEngine._esrgan_upscale`, falls back to Lanczos if the extra is absent). `max_resolution=0` (default) means no downscale cap, matching the hosted raiw.cc backend (fal fast-sdxl, no pre-downscale). The old forced downscale-to-1024 -> upscale-back round-trip for LARGE images was the main quality loss (issue #10) and is gone; at strength ~0.05 SDXL img2img does not need a downscale.
+
+**Final `--unsharp` post-filter (`humanizer.unsharp_mask`, opt-in, default 0):** applied LAST (after the face-restore pass, else it would be smoothed over) to counter the soft/over-smoothed look diffusion + restoration leave (an AI tell); ~0.5-0.8 safe, higher risks halos. Pairs with `--humanize` (grain adds sensor-noise texture, unsharp adds crispness). `--max-resolution N` re-introduces an opt-in long-side cap purely to bound GPU/MPS memory on very large inputs (it reintroduces the lossy round-trip). For huge images that OOM at native, **`--tile` is the lossless alternative** -- see the tiled-diffusion subsection below.
+
+### Tiled diffusion for large inputs (`--tile`, issue #10)
+
+`--tile` (OFF by default; `--tile-size` default 1024, `--tile-overlap` default 128) processes the diffusion pass in overlapping sliding-window tiles instead of one forward pass, so a large image is regenerated at **native resolution** without the OOM and without the lossy `--max-resolution` downscale round-trip. It engages only when the long side exceeds `--tile-size`; a sub-tile image runs a single pass unchanged. `WatermarkRemover.remove_watermark` refactors the single-image `_generate` into a per-tile `_generate_one` (the ControlNet canny edge map is rebuilt per tile, so structure preservation works tile-local) and routes it through `noai.tiling.run_tiled` when tiling is active. The geometry and blend math are pure helpers, unit-tested without the model (`tests/test_tiling.py`):
+
+- `plan_tiles(w, h, tile_size, overlap)` lays out a row-major grid where every tile is exactly `tile_size` (the last tile on each axis is pulled back flush to the far edge, simply overlapping its predecessor more). Uniform tile size keeps each diffusion pass at SDXL's preferred dimension.
+- `feather_weights(w, h, overlap)` is a separable linear taper, ~1 in the interior and ramping toward each edge, kept **strictly positive** so the normalized accumulate-and-divide blend (`accum / weight_sum`) is a partition of unity: a region covered by one feathered edge (an image corner) still divides cleanly. Identical (unchanged) tiles therefore reconstruct the input exactly -- the seam-free guarantee, asserted in `test_identity_generate_reconstructs_image`.
+
+CAVEAT: each tile is an **independent** low-strength regeneration. At the certified removal strengths (0.20-0.30) the per-tile drift is small and the feather blend hides the seams, but tiling is a memory workaround, not a quality upgrade over a single native pass -- a 32 GB MPS box that clears the native UNet peak should prefer no tiling. The MPS->CPU fallback still applies per tile; if the first tile falls back to CPU, the device stays CPU for the rest of the image.
+
+**Concrete MPS data points (the OOM is memory-tier-dependent, NOT a hard MPS limit):** on a ~24 GB unified-memory machine (verified 2026-05-25, 1254x1254 gpt-image SDXL, fp32) native res OOMs at the *UNet* step (peak ~17 GiB), not only the VAE decode, and the auto-fallback in `img2img_runner` reloads on CPU and finishes (slow, ~13 min) -- the output is still weight-identical and defeats SynthID, so "looks hung/crashed" on Mac is usually this CPU fallback, not a pipeline error. On a **32 GB** unified-memory machine the same default SDXL pass runs entirely on MPS with **no CPU fallback** (verified 2026-05-31, 1122x1402 gpt-image, `all`/default, ~155 s end-to-end), so 32 GB clears the native-res UNet peak that 24 GB could not. Adding `enable_vae_tiling()` alone does NOT prevent the 24 GB OOM (the peak is the UNet, not the VAE). The fast Mac workarounds for memory-constrained machines are fp16 on MPS (roughly halves memory) or `--max-resolution` to cap the long side; neither is wired as the default. The `controlnet` pipeline adds the canny ControlNet weights on top of SDXL, so its peak is a bit higher than the plain `default` pass; the same MPS->CPU fallback covers an OOM. The native-vs-cap-vs-floor decision lives in the pure helper `invisible_engine._target_size(w, h, max_resolution, min_resolution)` (returns `None` for native, a target tuple for a downscale cap OR an upscale floor; cap takes precedence, the floor is skipped on a min>max misconfig) so it is unit-tested (`tests/test_invisible_engine.py::TestTargetSize`, the #10/#15/#36 regression guard) without loading the model -- keep that logic in the helper, don't re-inline it.
+
+## fp16 VAE black-output fix (issue #29) + degenerate-output fp32 backstop (issue #41)
+
+**fp16 VAE black-output fix (issue #29, 2026-05-30):** on a **CUDA/XPU fp16** backend the stock SDXL VAE overflows to NaN and the *plain* img2img path decodes to an **all-black** image (reproduced on the raiw.cc result: a 1086x1448 input -> a uniformly black 4.6 KB PNG, mean 0). `watermark_remover._load_pipeline` / `_load_controlnet_pipeline` swap in the fp16-fixed SDXL VAE (`madebyollin/sdxl-vae-fp16-fix` = `_SDXL_FP16_VAE_ID`) when `_needs_fp16_vae_fix(model_id, DEFAULT_MODEL_ID, is_fp16)` is true -- only the default SDXL checkpoint on fp16.
+
+**cpu/mps run fp32** (the stock VAE is fine there, which is why the bug never reproduces on Mac). A custom non-SDXL `model_id` keeps its own VAE (the fp16-fix VAE is SDXL-architecture-specific). The decision is a pure helper, unit-tested without a download (`tests/test_platform.py::TestFp16VaeFix`); the actual black->clean recovery needs a CUDA GPU.
+
+**Confirmed on real CUDA hardware 2026-06-03:** running `all` on a 1086x1448 OpenAI gpt-image (the #29 repro size) at fp16 produced a normal (non-black) output, so the fp16-fix VAE swap resolves the all-black decode. (It was not reproducible on this MPS machine, which runs fp32, so the verification had to happen on an NVIDIA box.)
+
+**Follow-up safety net (issue #41, 2026-06-04):** the swap is gated to `model_id == DEFAULT_MODEL_ID`, so a custom model, a stale pre-fix install, or a fal/custom loader can still hit the black decode -- a new reporter did (gpt-image 1448x1086, the #29 size, with the exact `image_processor.py:142 invalid value encountered in cast` warning the NaN->0 cast emits). `remove_watermark` now adds a model-agnostic backstop: after generation, if the run was fp16 AND the output is degenerate (`_is_degenerate_image`: mean and std both below `_DEGENERATE_THRESHOLD` 1.0 -- a uniform all-black/NaN frame; the variance guard spares a legitimately dark-but-textured photo), it rebuilds the pipeline in fp32 on the SAME device and re-runs once. fp32 is the verified-clean path, so the user never gets a black image regardless of model_id/version. Mirrors the existing MPS->CPU fallback's self-mutation pattern (reset `torch_dtype` + clear `_pipeline`/`_controlnet_pipeline`); `batch` inherits it through `remove_watermark`, and once one image trips it the rest of the batch stays on the safe fp32. The detector is a pure helper, unit-tested without a model (`tests/test_platform.py::TestDegenerateOutputGuard`); the full fp16->detect->fp32-retry chain was verified e2e on this MPS machine by forcing fp16 with the swap disabled (first pass black, guard fired, retry produced a normal image). CAVEAT: the fp32 retry uses ~2x memory, so on a VRAM-constrained GPU it can OOM (a visible error, still better than a silent black frame; the MPS->CPU fallback covers that path). The reporter's "CPU also black" symptom is NOT reproducible here -- fp32 (cpu/mps) decodes clean -- so it points at an old version or a non-fp32 run, pending their version + command.
+
+## rich was dropped (plain-text CLI and scripts)
+
+**rich was dropped (CLI + scripts print plain text via `click.echo`).**
+
+`cli.py` renders through small `_Console`/`_Table`/`_Progress` shims; the analysis scripts (`scripts/synthid_corpus.py`, `synthid_pixel_probe.py`, `text_detection_benchmark.py`, `corpus_gap_scan.py`) import `Console`/`Table` from the shared `scripts/_plain_console.py` shim (markup like `[bold]`/`[/]` is stripped, tables render aligned). Consequences: (1) `rich` is NOT a dependency, so anything that imports it breaks a clean `uv sync --frozen` (CI installs core+dev only) — this exact gap red-failed CI after the refactor when those 4 scripts still imported rich; if you add a script, use the `_plain_console` shim, not rich. (2) The old `[gpu]`-bracket-eaten bug (#19) is gone — plain `click.echo` prints `pip install 'remove-ai-watermarks[gpu]'` verbatim, no escaping needed (regression-guarded by `tests/test_cli.py::TestGpuHintMarkup`). (3) No Unicode glyphs / colors / progress bars in CLI output by design.
+
+## AVIF/HEIF/JPEG-XL metadata, ISOBMFF/ffmpeg removal, audio watermark detection
+
+Metadata detection for AVIF/HEIF/JPEG-XL relies on a binary scan for `C2PA_UUID` + `IPTC_AI_MARKERS`, plus EXIF `Software` / XMP `CreatorTool` generator tags via `metadata.exif_generator` (validated with synthesized AVIF/JPEG fixtures + an XMP raw-scan fixture). C2PA removal in those containers is implemented via `noai/isobmff.py` (top-level ``uuid`` / ``jumb`` box stripper, no re-encoding), which now also drops a top-level XMP ``uuid`` box that carries an AI label (matched by AI-marker content, not by the XMP UUID, so byte-order-robust) and covers MP4/MOV/M4V/M4A by content sniff.
+
+**Non-ISOBMFF audio/video removal is via ffmpeg** (`_FFMPEG_STRIP_EXTS` -> `_strip_with_ffmpeg`): WebM/Matroska (EBML), MP3 (ID3), WAV/FLAC/OGG (RIFF/Vorbis) are stripped losslessly with `ffmpeg -map_metadata -1 -map_chapters -1 -c copy` (codec data untouched). Requires ffmpeg on PATH; raises `RuntimeError` if absent or if ffmpeg can't parse the file. Verified end-to-end (a real ffmpeg-made WAV/MP3 with a `title=Suno AI` tag -> tag gone, audio bytes preserved).
+
+**Meta-box XMP now handled (`isobmff.blank_ai_xmp_packets`, v0.6.9):** an AI-label XMP packet stored as a meta-box `mime` item (AVIF/HEIF) is blanked in place (overwritten with spaces of the same length, so `iloc` offsets and the coded image stay valid).
+
+**`Exif` item inside the `meta` box (AVIF/HEIF), now handled in place (2026-06-19):** an AI-generator token in an EXIF item (its TIFF bytes live in `mdat`/`idat`) is blanked by `isobmff.blank_ai_exif_tokens` — it finds EXIF TIFF blocks by their II/MM byte-order header, validates each with **piexif** (a coincidental II/MM run in pixel data won't parse as a TIFF IFD, so it is ignored), and overwrites any `Software`/`Make`/`Artist`/`ImageDescription` value carrying an `AI_GENERATOR_TOKENS` token with spaces of the **same length**. Same-length means every box size and `iloc` offset stays valid and the coded image is untouched — so it avoids the full `iinf`/`iloc` surgery (offset rewrite) that exiftool would need (exiftool is a non-installed binary dep, deliberately not used). It scrubs only the AI value; camera/editor EXIF is preserved. Wired into `remove_ai_metadata`'s ISOBMFF path after `blank_ai_xmp_packets`. Because the ISOBMFF branch never runs the JPEG `_scrub_ai_exif`, this is the ONLY EXIF scrubber on that path and must stay in PARITY with it: it now also blanks the China TC260 `{"AIGC":{...}}` block in `ImageDescription`/`UserComment` (via `_is_aigc_exif_value` — Doubao producer + Tencent service-provider schemas) and the xAI/Grok `Signature:` + UUID-`Artist` pair, not just `AI_GENERATOR_TOKENS` in `Software`/`Make`/`Artist`/`ImageDescription` (regression `test_noai.py::TestISOBMFF::{test_blank_aigc_block_in_exif, test_blank_xai_signature_pair_in_exif}`). **Still NOT built:** Resemble PerTh audio detection (no presence/confidence flag exists).
+
+**Audio watermark DETECTION (Resemble PerTh) was evaluated and NOT built (2026-05-26):** `resemble-perth`'s `PerthImplicitWatermarker.get_watermark()` returns a raw bit-array with **no presence/confidence flag** (clean audio decodes to arbitrary bits too), so reliably distinguishing watermarked-from-clean needs either Resemble's fixed payload or a confidence API -- neither is public, and there's no real Resemble sample to calibrate against. Same wall-class as the SynthID pixel detector: the decode exists, reliable presence-detection does not. (perth's top-level `PerthImplicitWatermarker` is also gated to None unless `librosa` is importable.)
+
+## SynthID detection is metadata-only (no local pixel detector)
+
+**SynthID detection is metadata-only.**
+
+There is no reliable *local* detector of the SynthID *pixel* watermark — Google's decoder is proprietary, no public spec or API (only a waitlisted portal). Authoritative confirmation: Google DeepMind's own paper "SynthID-Image: Image watermarking at internet scale" (Gowal et al., arXiv:2510.09263) states the verification service is restricted to "trusted testers" and does not release detector weights or a reproducible algorithm — so a local pixel detector is infeasible by design, not just unbuilt. https://arxiv.org/abs/2510.09263 We detect SynthID by its C2PA companion (`synthid_source` / `SYNTHID_C2PA_ISSUERS`), which is reliable while the manifest is intact but says nothing once C2PA is stripped.
+
+**Surface-dependent blind spot (verified 2026-05-24):** the same Google model emits different metadata per surface -- the Gemini *app* wraps outputs in Google C2PA, but the *API/playground* (AI Studio, Nano Banana / gemini-2.5-flash-image) emits the SynthID *pixel* watermark (confirmed via the Gemini-app oracle) + the visible sparkle but **no C2PA/IPTC at all**, so `synthid_source` returns None despite SynthID being present. Only the pixel oracle or the visible-sparkle detector catches those. (Meta AI is another surface mismatch: it writes the IPTC `digitalSourceType=trainedAlgorithmicMedia` marker, not C2PA and not SynthID.) Google→SynthID is long-standing; OpenAI→SynthID is confirmed by OpenAI's Help Center (ChatGPT/Codex/API "include both C2PA metadata and SynthID watermarks", updated 2026-05-21) but time-gated (pre-rollout OpenAI images carry C2PA without SynthID), so the OpenAI verdict is hedged "likely". Oracles: Gemini app "Verify with SynthID" (Google), openai.com/verify (OpenAI).
+
+**Each vendor's oracle detects only its OWN content (verified on the page 2026-05-31):** `openai.com/research/verify` states verbatim "OpenAI generation signals will only be detected if the image was generated with our tools" and "Content could also still be AI-generated by another company's model, which the tool currently does not detect" -- SynthID is shared tech but the verifier is keyed to its own vendor's payload, so a Google-SynthID image reads clean on OpenAI's verifier and vice-versa.
+
+**This explains the recurring "oracle says clean but `identify` still flags SynthID" report (#14):** the oracle reads the *pixel* watermark (gone after our SDXL pass), while `identify` reads the *C2PA-metadata proxy* (still present if the manifest survived). Different signals, not a contradiction -- strip the metadata too (`metadata --remove` / `all`) and the proxy goes quiet, but a quiet proxy is not proof the pixel watermark is gone.
+
+**Consequence for the P0#5 no-signal skip (`has_invisible_target`, 2026-06-22):** `invisible`/`all`/`batch` skip the diffusion scrub by default when no invisible AI signal is *locally* detectable, to avoid degrading a clean image (`--force` overrides). Because SynthID detection is metadata-only, a real AI image whose C2PA was **already stripped** (e.g. a re-encoded download, or the API/playground surfaces above that never emit C2PA) reads as no-signal and is therefore **skipped** — leaving its pixel SynthID in place. This is the deliberate trade: the skip's message never claims the image is clean, and the user re-runs with `--force` when they know it is AI. The blind spot is the same metadata-only ceiling, not a new bug; the visible-sparkle path (`check_visible`) still catches the no-C2PA Gemini-playground case for the *visible* mark, but not the invisible one.
+
+**SynthID is durable to JPEG re-encode by design, so a GitHub-recompressed issue attachment is still a valid SynthID test subject** (verified 2026-06-01 on issue #14's pic3: the GitHub-served JPEG survived re-encoding and openai.com/verify still detected SynthID). Do NOT dismiss issue-attachment JPEGs as "not faithful originals" when reproducing a SynthID-survival report: the recompression strips the **C2PA metadata** (so `identify` reads Unknown on the attachment) but NOT the **pixel watermark** that openai.com/verify reads. A true byte-original only matters for the metadata/C2PA path, not for the pixel-SynthID-removal test. (Contrast the open imwatermark above, which IS fragile to JPEG.) The spectral phase-coherence approach from `github.com/aloshdenny/reverse-SynthID` was evaluated (May 2026) and **does not work for real-content detection**: on its own shipped codebook + validation set, watermarked and cleaned images were indistinguishable (conf within noise, cleaned often higher); it only fires on pure-black 1024x1024 reference images at exact resolution (the controlled case it was calibrated on). The README's "90% / conf=0.91" reproduces only in that lab condition. Do not build a production detector on it; if revisited, it is experimental/diagnostic only and needs a per-resolution, per-model reference corpus. A from-scratch gpt-image pilot (2026-05-24) confirmed this independently: 5 independent solid-black gpt-image outputs share a near-identical fixed signature (pairwise residual correlation **0.92**, avg-template retains 97% energy), so the watermark/carrier IS strongly present and consistent on flat content — but the carrier frequencies extracted from it do NOT discriminate real content (carrier-to-random ratio: cleaned 1.86 > watermarked 1.53; a non-gpt-image image scored highest at 3.67). The signature drowns in content texture. Net: a perfectly consistent solid-color signature still yields no real-content pixel detector with magnitude/carrier methods. A corpus discrimination test (2026-05-24, `scripts/synthid_pixel_probe.py`, raw zero-mean residual NCC) independently re-confirms this: at matched resolution, SynthID positives do NOT cluster apart from negatives (within-Gemini 0.07; at 1024 px pos-vs-neg >= pos-vs-pos). The only high correlations were near-duplicate *content* (5 ChatGPT renders of one prompt at ~0.92, while a distinct ChatGPT image scored ~0 against them) — content, not a carrier. The probe is solid-fills-only and EXPERIMENTAL/DIAGNOSTIC; do not use it on real content.
+
+**Correction (deeper re-examination 2026-05-25):** the carrier IS real on solid fills — the earlier "no carrier" was a *method* artifact of using spatial / FFT-magnitude NCC, which can't see it. The carrier is a fixed *phase* at specific low frequencies, so the right metric is **per-bin phase coherence**. On 8 white `gemini-2.5-flash-image` fills (generated via the reverse-SynthID trick: identity-edit prompt "Recreate this image exactly as it is" on a synthetic pure-white PNG — this bypasses the recitation block that rejects text prompts for pure colors), phase coherence at the white carriers `(0,±7..±12,±20..±23)` = **0.86** vs **0.31** random; single-image leave-one-out phase-match **+0.83** vs real photos **-0.24**. (Black `2.5-flash` fills clip to std≈0 — SynthID can't push values below 0, so no carrier in black; the repo's dark carriers come from nano-banana-pro.)
+
+**But it does not generalize:** (a) carriers are model-version + resolution + color specific — the repo's v4 codebook (built for `gemini-3.1-flash-image-preview` + `nano-banana-pro-preview`) scores ~0.527 on my 2.5-flash white fills, indistinguishable from negatives (~0.50), i.e. carriers shift across model versions and need a per-model codebook; (b) on real content (30 `2.5-flash` images) the carrier collapses — set phase coherence at carriers 0.37 ≈ random 0.42, and the repo's v4 detector gives content 0.518 ≈ negatives 0.504 (no separation; a faint +0.24 single-image lean is likely a brightness confound). Net: the spectral/phase approach is a real *controlled-fill* characterizer, NOT an arbitrary-real-content detector, and is brittle to model version. Metadata proxy + visible sparkle + online oracles remain the ceiling for real content.
+
+## External AI-vs-real classifier models are out of scope
+
+**External AI-vs-real classifier models are out of scope (decided 2026-05-24).**
+
+Generic HuggingFace detectors (`Organika/sdxl-detector` Swin Transformer, `umm-maybe/AI-image-detector`, and fine-tunes) exist and report ~0.98 on their *own* SDXL-vs-real validation sets, but they are per-generator and the model cards themselves note degraded accuracy off-distribution; they are untested on gpt-image / Gemini Nano Banana (the metadata-stripped surfaces we care about), and our own light SDXL pass would likely defeat them the same way it defeats SynthID. Detection here stays local + signal-based (metadata + visible sparkle); do not add a bundled classifier dependency.
+
+## Default strength is vendor-adaptive, one ladder for both pipelines
+
+**DEFAULT STRENGTH IS VENDOR-ADAPTIVE, ONE LADDER FOR BOTH PIPELINES (LOWERED 2026-06-14; raised + unified 2026-06-09; vendor-adaptive since 2026-06-01, SUPERSEDES every fixed-default claim in this bullet and the next).**
+
+`resolve_strength(strength, vendor)` + `vendor_for_strength(path)` (`watermark_profiles.py`) read the C2PA issuer (`metadata.synthid_source`) on the ORIGINAL input and pick `OPENAI_STRENGTH` **0.10** / `GEMINI_STRENGTH` **0.15** / `UNKNOWN_STRENGTH` **0.15** when `--strength` is unset; explicit `--strength` always wins.
+
+**The SAME ladder applies to BOTH pipelines** (`sdxl` and `controlnet`). **2026-06-14: lowered from the 2026-06-04 cert floors (OpenAI 0.20 / Google 0.30) back toward the original 2026-06-01 study (OpenAI ~0.05-0.10 / Google 0.15).** A re-test on the deployed Modal controlnet worker cleared SynthID on the oracle at OpenAI 0.10 (2 photoreal, 1402/1448 px) and Google 0.15 (2 NATIVE 2816x1536 images -- retiring the "native ~2816 likely needs >=0.30" guess), while a pixel sweep showed 0.20/0.30 over-regenerated for no efficacy gain (Google MAE -20% at 0.15). See `watermark_profiles.py` "Data basis". **CAVEATS that stand:** (1) removal near this floor is SEED-NON-DETERMINISTIC (the 2026-06-09 finding below) -- a SERVICE on this ladder must pin a fixed, oracle-verified seed, not rely on a random one; (2) the re-test is n=2 per vendor on photoreal/landscape, NOT flat graphics (the `sdxl` weak spot), so raise `--strength` if an oracle reads SynthID on a flat output.
+
+**Why one ladder (NOT a per-pipeline split):** the cert was run on controlnet and does NOT transfer to `sdxl` by symmetry (opposite hard cases -- controlnet leaves SynthID on photoreal, `sdxl` on flat graphics), BUT on its OWN hard case (flat fills) `sdxl` is the WEAKER remover (plain img2img barely perturbs a flat region at low strength), so it needs AT LEAST controlnet's strength -- hence the certified floor is the right floor for `sdxl` too. It is a MARGIN argument for `sdxl`, not a fresh certification (no local SynthID detector to self-verify); raise `--strength` if an oracle still reads a flat `sdxl` output. The higher strength costs little quality because `controlnet` is now the default pipeline AND the only `--auto` pick, so `sdxl` is reached only via an explicit `--pipeline sdxl` (a deliberate opt-down for inputs without faces/text), where over-regeneration has nothing to damage. (A short-lived per-pipeline split ladder -- `sdxl` 0.15/0.20 vs controlnet 0.20/0.30 -- existed on 2026-06-09 before being unified the same day; the `resolve_strength` `pipeline` param and the `CONTROLNET_*_STRENGTH` constants were removed.) The CLI detects the vendor from the pristine source (before the visible pass / metadata-strip removes C2PA from the temp file) and passes it to display calls so display and execution agree; `cmd_invisible`/`cmd_all`/`batch` thread `vendor`.
+
+**This replaces the single 0.30 default AND the prior "do NOT build a vendor-adaptive default" policy** -- both came from the now-debunked region-rescrub-contaminated study (the per-region re-scrub that contaminated those numbers was removed in the controlnet refactor). Basis: the oracle-verified June 2026 controlled study (clean v0.8.6, protect OFF): OpenAI clears at 0.05 across 1024-1600 (n=4, resolution-independent); Google needs 0.15 on the capped-1536 path (n=4). `docs/synthid.md` §2.2 (data) + §5.2 (the adaptive default) are authoritative.
+
+**CAVEAT (oracle pass 2026-06-04): the OpenAI 0.10 default is content-dependent, NOT universal -- a flat-graphic OpenAI logo/poster still read SynthID-detected after `default` at 0.10, and photoreal images after controlnet at 0.10/0.15 (low-change regions under-perturbed). Removal at 0.10/0.15 is content×pipeline dependent (see the controlnet Known-limitations bullet); the lever is a higher strength, oracle-revalidated per content type. Do NOT assume the vendor-adaptive default clears every image.**
+
+CAVEAT: Google's 0.15 was validated only on `--max-resolution 1536`; native large Gemini (2816) was not locally measurable (OOM on M-series) and is pending GPU validation on raiw.cc -- if it survives 0.15 native, raise `--strength`.
+
+**Everything below in this bullet about a fixed 0.10/0.30 default is HISTORICAL; trust the vendor-adaptive constants + docs/synthid.md.**
+
+## SynthID removal: strength + oracle scope
+
+**SynthID removal: strength + oracle scope.**
+
+Default strength is vendor-adaptive (see the bullet above); `docs/synthid.md` §2.2 is authoritative for the numbers.
+
+**Oracle scope (load-bearing):** the Gemini app "Verify with SynthID" is the ONLY valid SynthID oracle (detects Google's mark on any image); `openai.com/verify` is scoped to OpenAI provenance (its own C2PA), NOT a SynthID oracle -- a negative there is meaningless for SynthID. There is no local SynthID detector, so the tool cannot self-check; if the oracle still reads SynthID, raise `--strength` to the lowest value that verifies clean. Only the `sdxl` (plain SDXL img2img; `default` is a back-compat alias) and `controlnet` (SDXL + canny ControlNet) profiles exist; the local `invisible` default is weight-for-weight identical to raiw.cc prod (`fal-ai/fast-sdxl` = `stabilityai/stable-diffusion-xl-base-1.0`, runtime-downloaded, not bundled).
+
+**Forensic-stealth caveat** (arXiv:2605.09203): defeating the SynthID verifier is NOT forensic invisibility -- independent detectors flag *removal-processed* images vs genuinely-clean ones at >98% TPR@1%FPR, so do not over-claim "indistinguishable from a real photo".
+
+## `controlnet` pipeline: content x pipeline removal, certified floors, no face-restore
+
+**`controlnet` pipeline (text/face STRUCTURE preservation, THE DEFAULT since 2026-06-09; `--pipeline default` opts down to plain SDXL).**
+
+SDXL + the canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` via `StableDiffusionXLControlNetImg2ImgPipeline` (`watermark_remover._run_controlnet` / `_load_controlnet_pipeline`).
+
+**Removal still comes from the img2img regeneration (`strength`); the ControlNet only PRESERVES text and face STRUCTURE by conditioning on the canny edge map** (`cv2.Canny(gray, 100, 200)`, 3-channel). Canny preserves edges, NOT face identity (a regenerated face drifts in likeness). The drifted cleaned face is the LEAST-AI state we can reach without re-introducing SynthID; **the library does NOT ship a face-restore extra** (every approach evaluated 2026-06-04 - 2026-06-08 -- GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps -- regenerated the face via SDXL and made it look MORE AI-generated). Full empirical conclusion in `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up". For production face preservation, ship the cleaned image as-is. No original pixels are copied or frozen, **BUT removal at the low vendor-adaptive strength is CONTENT × PIPELINE dependent and NEITHER pipeline clears all content -- oracle-validated against the OpenAI verifier 2026-06-04 (8 images, strength 0.10/0.15, `--max-resolution 1536`).**
+
+The survivors FLIP by content type: **photoreal** (a 9-face grid, a bracelet product photo) SURVIVES controlnet but CLEARS `default` (controlnet's dense edge map keeps the regen too close to the original, so the SynthID-destroying perturbation never happens; plain img2img perturbs photoreal texture enough); **flat graphic** (a logo/poster with large flat color fills) SURVIVES `default` but CLEARS controlnet (at low strength img2img barely changes flat fills so SynthID persists there, while controlnet repaints them more freely); a flat **text** card cleared under both.
+
+**Root cause is insufficient STRENGTH, not the pipeline: at 0.10 the low-change regions -- dense-edge photoreal under controlnet, large flat fills under `default` -- are not perturbed enough to destroy SynthID. The vendor-adaptive 0.10 from the June study is NOT universally sufficient (that study's content happened to clear at 0.10).**
+
+The robust fix is a HIGHER strength, oracle-revalidated per content type (controlnet can be cranked harder without losing structure; a lower `controlnet_conditioning_scale` also frees the regen on photoreal). So at today's default strength **both pipelines AND `--auto` can LEAVE SynthID on some content** -- a removal-priority caller (raiw.cc) MUST oracle-validate strength across content types before adopting, not pick a pipeline and assume removal.
+
+**Follow-up same day: re-running the two photoreal survivors through controlnet at an explicit `--strength 0.15` cleared BOTH on the oracle -- BUT one of them (the bracelet) had SURVIVED the SAME 0.15 controlnet config in the first pass (only the random, unset seed differed). So removal near the threshold is SEED-NON-DETERMINISTIC: the same image+pipeline+strength+resolution can pass or fail run-to-run (img2img uses `seed=None`/random unless `--seed` is passed, and there is no local SynthID detector to self-verify). 0.15 is the borderline, NOT a robust floor -- pick a strength with MARGIN (controlnet ~>= 0.20) rather than exactly on it; the content×pipeline table's 0.15 data point is near-threshold noise. A confirming run at `--strength 0.20` controlnet cleared BOTH photoreal survivors on the oracle (ladder: 0.10 grid detected → 0.15 borderline/non-deterministic → 0.20 both clean), so **0.20 is the recommended robust controlnet floor for OpenAI photoreal** (one margin run, not an N-run repeatability proof -- a service should add margin or verify repeatability since there is no local SynthID detector to self-check).
+
+**Engineering follow-up DONE 2026-06-09 (three coupled changes):** (1) **strength raised + unified** -- `resolve_strength(strength, vendor)` now applies ONE vendor-adaptive ladder (the certified controlnet floors 0.20/0.30/0.30) to BOTH pipelines; see the DEFAULT STRENGTH bullet above for why one ladder covers `sdxl`. (2) **`controlnet` is now the DEFAULT pipeline** (CLI `--pipeline` default = `controlnet` + both engine ctors). Rationale: with the certified higher ladder it clears BOTH content classes that flipped in the content-x-pipeline table (photoreal AND flat graphic), whereas plain SDXL left SynthID on flat graphics -- so controlnet is the more removal-robust default. Cost: every non-`--auto` run now downloads the canny ControlNet weights + a higher memory peak (MPS->CPU fallback covers OOM). (3) **the plain-SDXL profile was renamed `default` -> `sdxl`** (`watermark_profiles.SDXL_PROFILE`/`normalize_profile`); `default` stays as a back-compat CLI/ctor alias (the `--pipeline` Choice accepts `sdxl`/`controlnet`/`default`, a click callback `_normalize_pipeline` maps `default`->`sdxl` AND warns that `default` is deprecated). (4) **the content-detection layer + `--auto` planner were removed and `--auto` was retired to a deprecated alias for `--adaptive-polish`** -- see the dedicated `auto_config.py`-removal bullet above (controlnet is the default pipeline and the polish self-gates, so detection changed nothing). raiw.cc still needs its own per-vendor/content calibration on the GPU worker for native resolution. The Gemini-native resolution caveat stands: controlnet 0.30 is certified only <=1536.** **CERTIFIED 2026-06-04 via the isolated `raiw-controlnet-cert` Modal app (`raiw-app/modal_cert.py`), restore OFF, ≤1536, each vendor on its own oracle: controlnet floors are OpenAI 0.20 (2 photoreal × 3 seeds = 6/6 clean; the 0.15-flipper is seed-robust at 0.20) and Gemini 0.30 (0.20 detected → 0.30 clean on 2/2 seeds). OpenAI 0.20 transfers to prod (resolution-independent); Gemini 0.30 holds only ≤1536 — Gemini is resolution-sensitive and raiw.cc runs NATIVE (`max_resolution=0`), so cap Gemini ≤1536 + use 0.30, or native-calibrate (~0.35+). Prod recipe: controlnet + per-vendor floor in `resolve_strength` (not the default ladder) + FIXED seed (kills the non-determinism).
+
+**No face-restore in the library:** every approach evaluated (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned, 2026-06-04 - 2026-06-08 cert sweeps) regenerated the face via SDXL diffusion -- the output face inherited SDXL "clean skin" gloss and lost original identity precision, looking MORE AI-generated than the cleaned image, not less. The drifted face from controlnet 0.20 is the least-AI state we can reach; for a paid service that's the prod output. See `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up".**
+
+See `docs/synthid.md` §5.5 + `docs/controlnet-removal-pipeline-research.md` (certified floors table).** **Lesson: visual-quality + face-recovery validation does NOT prove watermark removal -- only the SynthID oracle does, across MULTIPLE content types; never infer removal from sharpness/identity, and never conclude from a partial result (the photoreal-only data first read as "controlnet shields, default removes" -- the flat-graphic result reversed it).**
+
+`controlnet_conditioning_scale` (CLI `--controlnet-scale`, default 1.0) is the structure-preservation knob (higher = closer to the original structure); fp32 on cpu/mps, fp16-fixed VAE on cuda/xpu. The `controlnet` profile is threaded explicitly (`WatermarkRemover(pipeline=...)` / `InvisibleEngine(pipeline=...)`), NOT inferred from `model_id`. This productionizes the `scripts/controlnet_sweep.py` prototype; see `docs/controlnet-removal-pipeline-research.md`.
+
+**Forensic-stealth caveat still applies** (arXiv:2605.09203): defeating the SynthID verifier is not forensic invisibility -- a "this image went through a removal pipeline" classifier can still flag the output.
+
+## `qwen` pipeline (experimental, Qwen-Image 20B, certified floors)
+
+`--pipeline qwen` runs `QwenImageImg2ImgPipeline` on `Qwen/Qwen-Image` (20B MMDiT, Apache-2.0 code AND weights), as an img2img alternative to the SDXL pipelines. Motivation: the controlnet over-regeneration problem above (it plasticizes real photos / loses fine text at the scrub floor). Qwen-Image renders text natively (incl. CJK) and preserves structure markedly better, so at the strength that removes SynthID it damages real content far less.
+
+The scrub still comes from the img2img `strength` (same lever as SDXL); the call shape lives in the pure `_build_qwen_kwargs` (uses Qwen's `true_cfg_scale`, not SDXL's `guidance_scale` — the CLI `--guidance-scale` maps onto it, and ~4.0 is typical vs the SDXL default 7.5). bf16 on CUDA. It is **CUDA/cloud-class — the 20B does not fit MPS — so `_run_qwen` has NO MPS→CPU fallback** (unlike the SDXL paths). Cost on Modal A100-80GB is ~$0.05-0.10/image vs SDXL.
+
+**Certified oracle floors (Modal A100-80GB, 2026-06-20):** on native-resolution OpenAI and Gemini cert inputs (`data/qwen_in/`, both controls SynthID-POSITIVE): **OpenAI 0.10** (0.05 and 0.075 still detected; 0.10 clean and SEED-ROBUST — clean on seeds 0-4, so a random seed is safe) and **Gemini 0.25** (0.20 still detected, 0.25 clean on both images; lowered from the 0.30 first measured). Gemini seed-repeat is single-seed (seed 0): the Gemini oracle rate-limits volume, so PIN a seed in production rather than relying on seed-robustness there.
+
+**Fidelity vs controlnet was MEASURED, not eyeballed (`scripts/fidelity_metrics.py`, text scored against a vision-transcribed ground truth in `data/qwen_in/ground_truth.json` + PaddleOCR on the variants; an initial eyeball read was wrong and overturned by the metrics).** Methodology rule: only compare fidelity at each pipeline's OWN oracle-confirmed scrub floor -- i.e. between outputs where SynthID is actually removed in BOTH (controlnet OpenAI 0.10 / Gemini 0.15; Qwen OpenAI 0.10 / Gemini 0.25). An equal-strength comparison is invalid where it leaves one pipeline un-scrubbed (Qwen at 0.15 does NOT clear Gemini SynthID, so that run was dropped). At those scrub floors:
+- **Text:** Qwen wins on substantial Latin/mixed-script text -- OCR CER, controlnet vs Qwen: openai_1 (EN+RU+ZH, both 0.10) 0.385 vs **0.241**, openai_2 (EN, both 0.10) 0.341 vs **0.290**. On a SHORT CJK sign (gemini_1, cnet 0.15 / Qwen 0.25) it is a TIE (0.037 vs 0.037 -- both near-perfect; the earlier Qwen 0.000 was at the higher 0.30, not the certified floor).
+- **Faces:** controlnet wins -- gemini_3, 18 faces (cnet 0.15 / Qwen 0.25): ArcFace identity 0.546 vs 0.382, Laplacian-variance retention 0.62 vs 0.40, face LPIPS 0.09 vs 0.17 (Qwen smooths faces MORE; the gap narrows vs Qwen 0.30 but controlnet still wins clearly).
+
+**Conclusion: Qwen wins TEXT only for clean body text on a plain background with NO faces; controlnet wins faces AND display/decorative text in a scene. So `qwen` is a MANUAL `--pipeline qwen` opt-in, not a routed lane.** A content `--pipeline auto` router + a faces+text mixed dual-pass were prototyped and DROPPED (2026-06-20): on the canonical faces+text case (the abba poster, faces + display text) controlnet won EVERY metric incl. text (CER 0.114 vs qwen 0.379), so grafting qwen text only hurts; and "text→qwen" is undecidable cheaply (body-vs-display text is what matters). Caveat: `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`, Gemini 0.25), so `--pipeline qwen` gets the 0.25 Gemini floor automatically — the old manual `--strength 0.25` workaround is retired. `_build_qwen_kwargs` now passes an explicit height/width (qwen squished non-square inputs to 1024² without it). Flat-graphic content was not in the sample.
+
+**Improving Qwen (ship vs improve):** the cited research lives in `docs/qwen-improvement-research.md` -- read it before extending the `qwen` pipeline. Verdict: shippable as an opt-in text lane. **The "add a Qwen-Image ControlNet to fix face smoothing" lead was built, measured, and CLOSED (2026-06-20):** a DiffSynth-Studio Qwen + Apache-2.0 blockwise-canny ControlNet at the Gemini floor 0.25 did NOT restore face skin texture (face Laplacian-variance retention flat 0.40 -> 0.40, 13/16 faces within +-0.02; the SDXL+canny target 0.62 was not approached), because canny carries edges not skin grain and Qwen's higher Gemini floor (0.25 vs SDXL+canny 0.15) forces more smoothing -- and a deep-research sweep confirmed NO permissively-licensed Qwen tile/detail/realism/skin ControlNet exists anywhere (every Qwen conditioning is geometry). So **faces stay on SDXL+controlnet; Qwen is the text lane, not a face fix.** The strongest remaining lead is **Z-Image-Turbo** (6B, Apache-2.0, `ZImageImg2ImgPipeline`, scrub mechanism preserved) -- its own SynthID floor and face/text fidelity are UNMEASURED; that is the next experiment. Non-regenerative high-frequency detail re-injection is NOT safe by assumption (the "clean-output high frequencies do not carry the watermark" claim was refuted) -- it must be oracle-gated. Always validate any improvement at the certified floors with `scripts/fidelity_metrics.py` first.
+
+**Seed as a quality lever (measured, openai_1 at 0.10, seeds 0-4):** the seed barely moves whole-image fidelity (img LPIPS 0.062-0.065, SSIM 0.855-0.857, PSNR 28.5-28.7 — flat) but does shift TEXT legibility (OCR CER 0.241-0.290, ~17% spread) -- the seed changes WHICH details get regenerated, not the overall level. So a per-image best-of-N-seed selection is a WEAK, text-only lever (pick the lowest-CER seed that still scrubs; fidelity selection needs no oracle). Not worth the N× cost for general use -- pin one decent seed in prod; reserve best-of-N for text-heavy premium cases.

docs/module-internals.md

@@ -0,0 +1,375 @@
+# Module internals
+
+> Relocated verbatim from `CLAUDE.md` on 2026-06-11 to keep the always-loaded
+> context small. Long single-line entries were reformatted into paragraphs;
+> no content was changed or summarized.
+
+Full per-module detail: design decisions, tuned thresholds, calibration
+history, incident records, and the regression-guard map. The compact module
+list lives in `CLAUDE.md`; read the relevant section here before changing a
+module.
+
+## `noai/c2pa.py`
+
+`noai/c2pa.py` — C2PA reading, **official c2pa-python `Reader` first, hand-rolled parser as fallback** (migrated 2026-06-18; the official lib is a core dep, MIT/Apache, spec-tracking). `read_manifest_store_json(path)` runs `Reader.try_create` with a default `Context` (NO trust enforcement — we report what is in the file, we do not gate on cert trust) and returns the **whole** manifest-store JSON (every manifest plus ingredient manifests); it is memoized per (path, mtime) (`lru_cache(maxsize=8)`) because one `identify`/`get_ai_metadata` call invokes the structured parser ~3x on the same file. `extract_c2pa_info(path)` builds its dict from that store JSON (`_info_from_store_json`: structured `claim_generator` from the active manifest's `claim_generator` / `claim_generator_info[].name`, `timestamp` from `signature_info.time`) and falls back to the legacy caBX parser (`_extract_c2pa_info_png`) when the reader is unavailable (broken/absent wheel, `reader_available()` False) or finds no parseable manifest (synthetic/partial test blobs, the inject round-trip's re-stitched chunk). **Both paths share `_populate_registry_fields(buf, info)`** — the issuer / AI-tool / action / source-type / SynthID / soft-binding registry byte-scan applied to the store JSON (reader path) or the raw caBX bytes (fallback) — so the return-dict shape is identical and the registry stays the single source of truth. Whole-store scanning is load-bearing: a ChatGPT *edit* of a Sora generation keeps `trainedAlgorithmicMedia` + issuer "OpenAI" on the **parent/ingredient** manifest, not the active "opened" one (the active manifest's `signature_info.issuer` is "OpenAI", `common_name` "Truepic Lens CLI in Sora", so the issuer field now reads "OpenAI, Truepic" — first-match-wins platform attribution still resolves OpenAI). `extract_c2pa_info` now also serves non-PNG containers (JPEG/AVIF/MP4) structurally via the reader; the consumers (`identify`, `synthid_source`, `get_ai_metadata`) already merge `info OR byte-scan`, so this strictly upgrades the non-PNG path with no double-counting. `synthid_watermark`/`synthid_vendors` is set when the manifest is signed by a SynthID-using vendor on AI content; `soft_binding`/`soft_binding_vendors` when a `c2pa.soft-binding` `alg` names a forensic-watermark vendor (`soft_binding_vendors_in(buffer)` is the shared byte-scan, used by both paths and the non-PNG binary path). `extract_c2pa_chunk` / `inject_c2pa_chunk` / `has_c2pa_metadata` stay the PNG caBX byte tools (raw-chunk extraction for `extractor.py`, test injection, fallback detection). PNG/caBX chunk reads are clamped to the remaining file size (`safe_length = min(length, remaining)`; skipped chunks use seek) so a malformed huge `length` cannot drive a multi-GB allocation (shared safety discipline matching `isobmff.scan_c2pa_region`). Regression-guarded by `tests/test_noai.py::TestC2PARealSamples::{test_extract_info_uses_reader_store,test_fallback_to_png_parser_when_reader_unavailable}`.
+
+## `noai/constants.py`
+
+`noai/constants.py` — PNG_SIGNATURE, C2PA_CHUNK_TYPE, C2PA_SIGNATURES, and `C2PA_AI_VENDORS` — the single `C2paAiVendor` registry of C2PA-signing vendors (issuer byte, resolved org name, the `identify` platform label, and a `synthid` flag), from which `C2PA_ISSUERS`, `SYNTHID_C2PA_ISSUERS` (issuers that pair SynthID with C2PA: Google, OpenAI), and `identify._ISSUER_PLATFORM` are all **derived** — plus `C2PA_SOFT_BINDINGS` (soft-binding `alg` prefix → forensic-watermark vendor: Adobe TrustMark, Digimarc, Imatag, Steg.AI, Microsoft, ...). Add a new C2PA vendor as one `C2PA_AI_VENDORS` entry (never edit the derived dicts), a new soft-binding to `C2PA_SOFT_BINDINGS`; not inline. A vendor that signs under multiple legal names needs one entry PER distinctive issuer byte string: e.g. ByteDance's Volcano Engine is registered both as latin `volcengine` AND the Chinese legal entity `北京火山引擎科技有限公司` (UTF-8; the latin needle misses the Chinese-named certs entirely) — both normalize to the same "ByteDance" needle/platform. ElevenLabs ("Eleven Labs Inc.", pure generative-AI) is registered as a generator. A vendor may also set **`asserts_ai=True`** — its presence asserts AI generation even without a `trainedAlgorithmicMedia` digital-source-type; the derived `C2PA_IDENTITY_AI_ORGS` frozenset feeds `identify`, which lifts the AI verdict for such an issuer. Set it ONLY for a pure-generator brand with a distinctive issuer/generator string: **Dreamina** (ByteDance's international Jimeng brand, signed as "Bytedance Pte. Ltd." with a `Dreamina/x.y` claim generator and NO source-type — the caBX / store-JSON byte-scan sees the `Dreamina` token across active + ingredient manifests, where the active one is often a plain `c2pa-tool` transcode; verified on the retained corpus 2026-07; normalizes to the shared "ByteDance" needle/platform). Do NOT set `asserts_ai` on common-word issuers (Adobe/Google/OpenAI/Microsoft) — they appear incidentally in unrelated XMP/trust-chain bytes, so they must stay source-type-gated. Deliberately EXCLUDED (mined-corpus candidates 2026-06-20, documented in the file): TikTok Inc. (a content-provenance / AI-labeling signer on uploads, not a generator) and PixelBin.io / "Fynd" (an image transform / CDN signer) — registering either as a generator would mis-label human uploads as AI; the `is_ai` verdict keys off the digitalSourceType, which is already honored.
+
+## `metadata.py`
+
+`metadata.py` — `scan_head(path, size=1MB)` is the shared input for every C2PA/AIGC/IPTC byte scan: first `size` bytes plus the payloads of any provenance metadata found beyond that window — for ISOBMFF, the late provenance boxes from `isobmff.scan_c2pa_region` (catches a manifest after a large `mdat`); for **PNG**, the late `tEXt`/`iTXt`/`zTXt`/`eXIf`/`iCCP` chunks from `_png_late_metadata` (catches an XMP/EXIF packet appended after a large `IDAT`, e.g. a TC260 AIGC label at ~2.7 MB). Behavior-neutral (`f.read(size)`) for non-ISOBMFF inputs and for any file that fits within `size`. Use it instead of `open().read(1MB)` for any new marker scan.
+
+**Memoized per (path, size, mtime)** (added 2026-06-09, `_scan_head_cached` lru_cache, `maxsize=8`): one `identify`/`get_ai_metadata` call fans out to ~8 byte-scan detectors that each re-read the same file head, so the cache turns those into a single read; the mtime key invalidates on change, a stat failure falls back to an uncached read. `synthid_source(path)` returns the vendor name(s) if the C2PA manifest implies a SynthID pixel watermark, else None. Format-agnostic: PNG via the caBX parser, JPEG/WebP/AVIF/HEIF/JXL via a binary scan (C2PA marker + SynthID issuer + AI-source marker). `get_ai_metadata` surfaces the verdict, and `metadata --check` prints it as a callout. Both `get_ai_metadata` and `has_ai_metadata` guard the PIL open with `except Exception` (HEIC/unknown formats raise non-OSError) and fall through to the binary scan. `xai_signature(path)` detects xAI/Grok's EXIF-only scheme (`ImageDescription` = `Signature: <base64>` + UUID `Artist`); it feeds `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify`. `iptc_ai_system(path)` detects the IPTC Photo Metadata 2025.1 AI-disclosure XMP properties (`IPTC_AI_FIELD_MARKERS` = `AISystemUsed`/`AISystemVersionUsed`/`AIPromptInformation`/`AIPromptWriterName`) and returns the `AISystemUsed` generator name (or `"fields present"`). `remove_ai_metadata` routes **ISOBMFF video** (`.mp4`/`.mov`/`.m4v`) through the same `isobmff.strip_c2pa_boxes` as AVIF/HEIF (MP4 is ISOBMFF), and `_scrub_ai_exif` removes the xAI signature + AI-generator EXIF tags on JPEG output. `strip_c2pa_boxes` is **fail-safe** on a malformed box: it returns the original bytes unchanged with a logged warning instead of truncating the tail to EOF (detection-only `scan_c2pa_region` still stops at a malformed box). `_png_late_metadata` clamps each late-chunk read to the remaining file size (`safe_length = min(length, remaining)`) so a malformed `length` cannot drive a multi-GB allocation, AND advances the cursor by `safe_length` (not the raw `length`) so an inflated length cannot jump past EOF and abort the scan, silently skipping a genuine AI-label chunk after it.
+
+## `identify.py`
+
+`identify.py` — the OpenAI rollout caveat is keyed on `_vendor_of(synthid) == "OpenAI"` (not a raw substring over the issuer + verdict blob). `identify(path)` aggregates every locally-readable signal (C2PA issuer→platform, C2PA soft-binding forensic-watermark vendor, **C2PA cloud-manifest reference** via `metadata.c2pa_cloud_manifest` — signal `c2pa_cloud`, **medium**, provenance-only (does NOT set `is_ai`, excluded from `ai_from_metadata` + clash vendors): a C2PA 2.4 Durable-Content-Credentials case where the embedded manifest is stripped but an XMP `dcterms:provenance` pointer to the vendor's cloud manifest store (`_C2PA_MANIFEST_REPOSITORIES`, today `cai-manifests.adobe.com` → "Adobe Content Authenticity") survives, so the credentials stay recoverable server-side; only emitted when no embedded manifest already attributed the file — surfaced on 2 corpus PNGs 2026-06-10 that read fully `unknown` before, IPTC "Made with AI" + IPTC 2025.1 `AISystemUsed`, embedded SD/ComfyUI params, SynthID proxy, xAI/Grok EXIF signature via `metadata.xai_signature`, the China TC260 AIGC label via `metadata.aigc_label`, the HuggingFace `hf-job-id` job marker via `metadata.huggingface_job`, the Samsung Galaxy AI editing marker via `metadata.samsung_genai`, the visible marks — Gemini sparkle plus the ByteDance Doubao 豆包AI生成 / Jimeng 即梦AI / Samsung Galaxy AI "Contenuti generati dall'AI" text marks via the `watermark_registry` — open invisible watermark, Adobe TrustMark via `trustmark_detector`) into one `ProvenanceReport`. `is_ai_generated` is True or None (never asserted False — stripped metadata is not proof of clean origin). The `hf_job`, visible-mark, and Samsung `samsung_genai` signals are **medium** confidence: each lifts an otherwise-Unknown verdict to a tentative AI (`hf_only` / `visible_only` / `samsung_only`, parallel branches; `visible_only` fires on any `visible_*` signal) but is excluded from the high-confidence `ai_from_metadata` set, so none overrides a hard metadata signal.
+
+**AI-generated vs AI-enhanced** (`ProvenanceReport.ai_source_kind`, roadmap item): the C2PA digital-source-type is split into `"generated"` (trainedAlgorithmicMedia, fully synthetic) vs `"enhanced"` (compositeWithTrainedAlgorithmicMedia, a real photo with an AI-composited region) — the two byte strings are unambiguous (`compositeWithTrainedAlgorithmicMedia` capitalizes the inner "Trained", so a lowercase `trainedAlgorithmicMedia` match is standalone full generation; full generation wins when both appear). `ai_source_kind` is set only when the AI verdict actually came from the C2PA source type (a non-C2PA AI signal — IPTC/AIGC/local gen/xAI — leaves it None). It lets a caller branch a full-frame scrub (`generated`) from a region-targeted clean that preserves the real photo (`enhanced`; see `noai/tiling.feather_region_composite`). The CLI verdict line reads "AI-generated (fully synthetic)" vs "AI-enhanced (real content with an AI-composited region)".
+
+**Visible-mark detection** (`check_visible`, signals `visible_sparkle` / `visible_doubao` / `visible_jimeng` / `visible_qwen` / `visible_samsung`): the Gemini sparkle keeps its own file-level path (`_visible_sparkle` → `gemini_engine.detect_sparkle_confidence`, promoted only at confidence ≥ `_SPARKLE_THRESHOLD`, which is the SHARED `watermark_registry.GEMINI_SPARKLE_TRUST_CONF` (0.5) — imported, not a private copy, so the provenance detect threshold and the removal `detect_marks` / `_gemini_detect` arbitration gate can never drift (the detect-vs-remove desync from roadmap P0#7; regression-guarded by `tests/test_identify.py::TestSparkleDetectRemoveAlignment`, which composites the real demo sparkle at borderline opacities and asserts identify and `detect_marks` AGREE on either side of the line). Lowering the gate to recover faint sub-0.5 sparkles was evaluated 2026-06-20 and REJECTED: a real Doubao text mark scores ~0.40-0.42 as a gemini match with a HIGHER core-ring brightness margin than a genuine faint sparkle, so neither confidence nor the brightness gate separates them in the [0.35, 0.5) band — lowering trades a rare miss for false-positive removals on clean images. Corpus-tuned to separate Gemini sparkles ≥0.56 from non-sparkle ≤0.49), while Doubao/Jimeng/Qwen/Samsung reuse the registry detectors (`_visible_text_marks` → `watermark_registry`, iterating `_VISIBLE_MARK_PLATFORM`), each gated by its own engine NCC threshold via `MarkDetection.detected` (Doubao 0.50, Jimeng 0.45, Qwen 0.45, Samsung 0.40). Doubao/Jimeng are normally also caught by the TC260 AIGC metadata label and Samsung by its C2PA + `genAIType` marker, so the visible path is their stripped-metadata fallback. Visible marks set `platform` only when no harder signal already did, and (like the sparkle) are excluded from integrity-clash vendor claims. The cv2 dependency lives in the engines, not here.
+
+**`import identify` is deliberately light** (~26 MB; ~36 MB with cv2 loaded by a visible-mark run, ~106 MB for a full `check_visible` run): it imports the `noai.c2pa`/`noai.constants` submodules, and `noai/__init__` is lazy (see "Test and lint"), so torch/diffusers are NOT pulled at import even in a full `gpu`/`detect` install — fits a 512 MB host. `noai.c2pa` does eagerly import the **c2pa-python** binary (Rust + cryptography, ~+5 MB RSS, no torch) for the primary `Reader` path — light enough to stay on the dependency-light host; a broken/absent wheel degrades to the byte-scan parser (`reader_available()` False). The heavy paths are opt-in: `check_invisible=True` needs the `detect`/`trustmark` extras (each pulls **torch**; TrustMark also **downloads weights**), so on a core-only deploy leave `check_invisible` off (it is a no-op there anyway). Before the lazy `__init__`, the mere presence of torch in the env inflated `import identify` to ~420 MB.
+
+**C2PA platform attribution is device-token-first, issuer-scan fallback** (`_device_platform` scans manifest bytes for `_DEVICE_C2PA_PLATFORM` tokens, then `_attribute_platform`/`_ISSUER_PLATFORM`).
+
+**Why, verified on real signed files 2026-05-26:** the old issuer-only byte-scan matched ANY issuer substring anywhere, so multi-entity manifests mis-attributed -- Leica→"Truepic" (a signing authority in the trust chain), Nikon→"Adobe Firefly" (XMP-toolkit "Adobe" + the sample's "Adobe_MAX" name), Pixel→"Google (Gemini)" ("Google LLC" cert org), Truepic→"Google". A distinctive device token wins instead.
+
+**Token distinctiveness is load-bearing:** bare `b"Truepic"` mis-fires (it appears in unrelated trust chains -- it mis-attributed the OpenAI `chatgpt-1.png` fixture), so the token is the specific `b"Truepic_Lens"` from the Lens SDK claim generator; likewise `b"Pixel Camera"` (cert CN) not bare `b"Pixel"`. `_DEVICE_C2PA_PLATFORM` lists ONLY tokens **verified against a real C2PA file**: Leica (`lc_c2pa`/`Leica Camera`), Nikon (`NIKON`), Pixel (`Pixel Camera` -- from a real Pixel 10 Pro file attached to c2pa-rs issue #1609/#1554), Sony (`sony.sig`/`sony.cert` -- Sony's own C2PA assertion namespace, verified on a real Sony PXW-Z300 file; NOT bare "Sony" which is a common EXIF Make), Truepic (`Truepic_Lens`). Canon/Bria have **no public direct-download C2PA sample** (checked exhaustively: GitHub issue/PR attachments, contentcredentials gallery, HF datasets -- all upload-to-verify or token-gated; Canon's only public file was a self-signed hobbyist CR3, not factory), so they stay unmapped until a real file is captured (same fixture discipline as Grok/Doubao). The Sony sample is video (MP4) -- our ISOBMFF C2PA path detects it; Sony Alpha stills likely share the `sony.*` namespace but are not separately verified.
+
+**Samsung Galaxy + ASUS Gallery live in a separate `_SIGNER_C2PA_PLATFORM` (scanned after `_device_platform`, before the issuer fallback), NOT in `_DEVICE_C2PA_PLATFORM`** — verified on real signed files 2026-05-29. Reason: a Galaxy phone stamps BOTH its device cert AND a `trainedAlgorithmicMedia`/genAIType AI marker on a Generative-Edit image, so treating it as a "genuine camera capture" would false-fire integrity-clash rule 2 on every Galaxy AI edit. The signer tokens (`b"Samsung Galaxy"` cert org — distinct from the EXIF `SM-xxxx` model string on ordinary Samsung photos; `b"com.asus.gallery"` claim generator) only resolve the platform label; the AI verdict still comes from the source-type / genAIType. ASUS Gallery is a C2PA-signed edit with no AI marker, so it attributes the platform without asserting `is_ai`.
+
+**Samsung's `genAIType` (in the proprietary `PhotoEditor_Re_Edit_Data` JSON) is an undocumented Galaxy-AI editing marker** (`metadata.samsung_genai`, gated on the `PhotoEditor_Re_Edit_Data` container; non-zero value = AI tool used, values {1,5} observed; Galaxy AI appends it as a trailer AFTER the JPEG EOI, so `samsung_genai` reads the file TAIL when the 512 KB quick-scan head misses it — else a multi-MB photo's trailer past the window went undetected while removal, which reads the whole file, would still strip it; removal truncates the post-EOI Samsung trailer via `metadata._strip_samsung_trailer`, pixels bit-identical): medium-confidence because the field has no public spec (verified 2026-05-29: absent from C2PA spec + Samsung docs), but it co-occurred with `trainedAlgorithmicMedia` in 3/3 verified files that record a source-type and was the SOLE AI marker on a Galaxy S24 file that omits the source type. Camera C2PA marks capture authenticity, not AI (Pixel carries `computationalCapture`, not `trainedAlgorithmicMedia`), so these never set `is_ai` -- that stays driven by digital-source-type. `c2pa.cbor_text_after` (now public) is best-effort for the `generator` detail string only and can be None when the manifest keys it `claim_generator_info` (Pixel).
+
+**Issuer→generator mapping is `is_ai`-gated** (`_attribute_platform(issuers, is_ai=c2pa_is_ai)`): a specific AI-generator platform is named only when the digital-source-type is `trainedAlgorithmicMedia`; on a non-AI source an issuer substring is treated as incidental (an "Adobe XMP" toolkit string in an *unmapped* Canon/Sony capture would otherwise mislabel it "Adobe Firefly"), so it degrades to the neutral "C2PA signer: X" label. **The one exception is an identity-AI issuer** (`c2pa_is_ai = c2pa_source_kind is not None or c2pa_identity_ai`, where `c2pa_identity_ai` is any resolved issuer org in `C2PA_IDENTITY_AI_ORGS`): a vendor flagged `asserts_ai` (today only Dreamina) sets `c2pa_is_ai` True on its own, so its platform resolves even though the manifest carries no `trainedAlgorithmicMedia`. This is safe precisely because the flag is restricted to distinctive brand strings, not the incidental-mention-prone common words. Real Firefly/OpenAI/Google output carries the AI source-type, so it is unaffected (verified: chatgpt-1.png→OpenAI, firefly-1.png→Adobe Firefly still attribute). `_attribute_platform` defaults `is_ai=True` so the mapping stays unit-testable in isolation. Add capture-camera tokens to `_DEVICE_C2PA_PLATFORM`, editing-app/AI-device signer tokens to `_SIGNER_C2PA_PLATFORM`, generator/issuer platforms to the `C2PA_AI_VENDORS` registry in `constants.py` (which derives `_ISSUER_PLATFORM`), not inline. For non-PNG containers (JPEG/WebP/AVIF/HEIF/JXL) the caBX parser returns nothing, so issuer (`_issuers_in`) and generator (`_ai_tools_in`, reusing `C2PA_AI_TOOLS`) are recovered by binary-scanning the first MB. EXIF `Software` / `Make` / `Artist` / `ImageDescription`, XMP `CreatorTool`, and PNG `tEXt` chunks (`Software`/`Source`/`Title`/`Description` — NovelAI stamps its generator there, not EXIF) are read by `metadata.exif_generator` (PIL+piexif for any format PIL opens incl. AVIF, plus a container-agnostic XMP raw-byte scan that also covers HEIF/JXL), matched against `AI_GENERATOR_TOKENS` so ordinary editors (plain "Adobe Photoshop") and real-camera `Make` ("Apple"/"Canon") are not flagged. Tokens mined from the retained corpus 2026-06-22: `novelai`, `reve.com` (full token, not bare `reve`), `aphrodite ai` — all no-C2PA generator stamps that previously read as no-signal (and under the P0#5 no-signal skip would have skipped the scrub).
+
+**Ideogram tags its output with EXIF `Make="Ideogram AI"`** (verified on a real download 2026-05-24) — that's why `Make` is read.
+
+**Integrity-clash detection** (`_integrity_clashes`, surfaced as `ProvenanceReport.integrity_clashes`, printed in red by `identify` and serialized to `--json`): contradictions between independent generator stamps are a laundering/spoofing tell. Two rules: (1) two or more distinct AI-origin vendors named by **independent** signals (e.g. C2PA OpenAI + EXIF `Make="Ideogram AI"`), and (2) a camera-capture C2PA device (`_DEVICE_C2PA_PLATFORM`) coexisting with an AI-generation marker **from a source INDEPENDENT of the camera's own manifest**.
+
+**Rule 2's independence gate (added 2026-06-11):** a device that both captures and runs on-device generative AI (Google Pixel Magic Editor / Pixel Studio) records the capture AND the AI edit in ONE C2PA manifest — so the AI vendor is named only from that same manifest (`c2pa` issuer + `synthid` proxy, both `c2pa_manifest` source) — a legitimate edit chain, NOT a clash. Rule 2 therefore fires only when some `ai_vendor_claims` family has a source `!= "c2pa_manifest"` (EXIF/XMP generator, IPTC, TC260 AIGC, a second manifest naming AI on a camera capture — the real laundering tell). This killed a false-positive class on the corpus: 2 real Pixel generative-edit PNGs (`computationalCapture` + `trainedAlgorithmicMedia` + "Applied imperceptible SynthID watermark" in one Google manifest) read as camera-vs-AI clashes before the gate. Pure cameras (Leica/Sony/Nikon/Truepic) that do NOT generate AI still clash on any within-manifest AI marker only if it is independent — they never legitimately carry one, so the gate is behavior-neutral for them while fixing Pixel (regression-guarded by `test_identify.py::TestIntegrityClashesHelper::{test_pixel_generative_edit_same_manifest_no_clash,test_camera_plus_independent_ai_marker_still_clashes}` + `TestIntegrityClashEndToEnd::test_pixel_generative_edit_no_clash`).
+
+**Independence is source-grouped (`_CLASH_SOURCE`, added 2026-06-02):** the C2PA issuer attribution (`c2pa`) and the SynthID proxy (`synthid`) are NOT independent — the proxy is inferred from the *same* manifest — so they share one source and two vendors named within a single manifest do not clash. This killed a false-positive class found on the spaces corpus: legitimate multi-actor manifests where a product wraps another vendor's engine (Microsoft Designer on OpenAI → `OpenAI, Microsoft`; Microsoft on Google → `Microsoft, Google LLC, Google C2PA Core Generator Library`) or an edit chain re-signs (Adobe over a Gemini original → Adobe c2pa + Google synthid) — 19 such files across the 2026-06-01/02 batches read as clashes before the fix. Rule 1 still fires when a manifest vendor disagrees with a genuinely independent stamp (EXIF/XMP generator, IPTC `AISystemUsed`, AIGC, xAI); each non-`c2pa`/`synthid` family is its own source (`test_identify.py::TestIntegrityClashes::{test_multi_actor_manifest_no_clash,test_manifest_vendor_vs_independent_signal_clashes}`). Vendor normalization is `_vendor_of` over `_AI_VENDOR_TOKENS` (so a C2PA "Google (Gemini)" issuer and a SynthID-Google proxy agree, while different vendors clash). `_AI_VENDOR_TOKENS` covers ByteDance (all brands: bytedance/doubao/jimeng/dreamina/volcengine), Canva, ElevenLabs, and Black Forest Labs in addition to the OpenAI/Google/Adobe/... set — without them a transplanted ByteDance/Canva/BFL C2PA manifest next to an independent conflicting stamp was silently missed. **The generic `China AIGC (TC260)` label names no SPECIFIC vendor** (any Chinese generator applies it), so it cannot vendor-conflict in the spoofing sense: when a Chinese TC260-applying vendor (`_TC260_VENDORS`, today `{ByteDance}`) is co-attributed, Rule 1 attributes the label to that vendor (a legit Doubao image carries BOTH a ByteDance C2PA manifest and its own TC260 label and must not clash); against a NON-TC260 vendor (OpenAI etc.) the label stays generic and still clashes as a laundering tell (`test_bytedance_c2pa_plus_own_aigc_no_clash`, `test_foreign_vendor_plus_aigc_still_clashes`, `test_bytedance_c2pa_plus_foreign_generator_clashes`). Corpus-validated: 0 new clashes on 5000 ByteDance/AIGC/Canva/FLUX carriers.
+
+**High-precision by design:** only hard generator stamps feed it (C2PA-issuer when source is AI, SynthID, EXIF/XMP generator, IPTC `AISystemUsed`, xAI, AIGC); the fuzzy visible sparkle and the open invisible watermark are **excluded** (both are low-precision/positive-only signals; the open watermark was also historically a by-product of our own SDXL removal pass, until `watermark_remover` was fixed to load the SDXL pipelines with `add_watermarker=False` — it stays excluded as a fuzzy signal regardless). The c2pa vendor is classified from the issuer attribution / generator, NOT the resolved `platform` (a camera label like "Google Pixel" would mis-normalize to "Google"). All real single-origin fixtures (chatgpt/firefly/doubao/grok/mj) verified to produce **zero** clashes (false-positive guard in `test_identify.py::TestRealSamplesHaveNoClash`).
+
+**`ai_from_metadata` field + `has_invisible_target` helper (P0#5, 2026-06-22):** the high-confidence union (everything that sets `confidence == "high"`: C2PA AI-issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, open DWT-DCT / TrustMark — the medium-confidence `hf_only`/`visible_only`/`samsung_only` are excluded) is now surfaced as the public `ProvenanceReport.ai_from_metadata` boolean, so callers gate on intent rather than on the `confidence` string. `has_invisible_target(path)` wraps `identify(path, check_visible=False, check_invisible=True)` and returns that field — it is the decision gate for the diffusion scrub (the CLI `invisible`/`all`/`batch` no-signal skip, `cli._no_invisible_signal_exit`): a visible-only or no-signal image has it False, so regeneration (which would only degrade a clean image) does not run. It fails SAFE — any detector exception returns True so the removal still runs (leaving a watermark on a paid removal is worse than over-regenerating). It does NOT prove a pixel SynthID is absent (SynthID is detectable only via its metadata proxy, gone once stripped), so a False means "no locally-detectable target", never "clean". Guarded by `test_identify.py::{TestIdentifyRealSamples::test_has_invisible_target_*,TestHasInvisibleTargetFailSafe}`.
+
+## `watermark_registry.py`
+
+`watermark_registry.py` — **single catalog of known visible watermarks**, the unified "find known marks in their usual places, recognize, remove" entry.
+
+**Localize -> fill by policy (replaced reverse-alpha):** each mark is localized to a binary full-frame footprint mask (a `Localization`), and one shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). This replaced the old reverse-alpha removal (invert a captured alpha map, `original = (wm - a*logo)/(1-a)`, plus a thin residual inpaint) for ALL marks — gemini, doubao, jimeng, samsung, and jimeng_pill. **Why it changed:** reverse-alpha depended on a fixed captured alpha map at a fixed position, so it broke whenever a vendor moved or re-rendered its mark; and it was not color-lossless even with the right map (it amplifies 8-bit quantization and JPEG-chroma error by `1/(1-a)`), which showed up as "the color just changed, not removed" reports. Localize -> fill has a benign failure mode: a slightly-off localization just inpaints a small region near-losslessly instead of leaving a color-shifted smear. The captured alpha maps are still used to DETECT the marks and to shape the mask (gemini's footprint), but NOT for pixel recovery. Fill backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. Each `KnownMark` ties a key to {usual `location`, `in_auto` flag, a `_detect` callable → uniform `MarkDetection`, a `_mask` callable → full-frame footprint mask}; `KnownMark.remove(image, *, backend="auto", provenance=False, force=False)`. Entries today: `gemini` (bottom-right sparkle), `doubao` (bottom-right "豆包AI生成"), `jimeng` (bottom-right "★ 即梦AI"), `qwen` (bottom-right "千问AI生成", Alibaba Tongyi Qianwen), `samsung` (bottom-**LEFT** "✦ Contenuti generati dall'AI", Samsung Galaxy AI, Italian locale), and the capture-less `jimeng_pill` (top-left "AI生成"). `detect_marks(image, *, provenance=frozenset())` scans all (strict, for the identify verdict); `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` removes every detected mark in one pass. **Sensitivity (`auto`/`strict`/`assume_ai`)** decides how hard a borderline mark is trusted: the visual detectors are pixel-based (no metadata needed) and the recall gain comes from relaxing the false-positive gate, not from metadata. `resolve_trust` turns the policy + evidence into the per-mark trust level the engines consume as `provenance = level != "strict"` — `strict` never relaxes; `auto` relaxes only on same-product evidence (metadata provenance for that vendor, or a confidently strict-detected sibling of the same `_PRODUCT_OF` — Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax); `assume_ai` relaxes every mark (the caller asserts AI, e.g. a metadata-stripped screenshot). **Three levels, not two: `strict` / `assumed` / `confirmed`.** Relaxing bypasses the engine's false-positive gate outright, and that bypass is contracted to mean the vendor is CONFIRMED (`GeminiEngine.detect_watermark`'s `trust_provenance`: "external metadata already proves this is a Google generation"). An `assume_ai` caller asserts the image is AI, which says nothing about WHICH vendor, so a mark relaxed on assumption alone must also clear `_ASSUMED_CONF_FLOOR` (`assumed_floor_ok`; gemini 0.50) — see "Assumed-trust confidence floor" below. **Perception / decision / action are separated three ways** (the removal path only; `identify` keeps calling `KnownMark.detect` directly, so its verdict is untouched): `_build_candidates(image)` is PERCEPTION — it runs each detector at both trust levels and packages raw verdicts + the pill's flatness feature into `Candidate`s, no policy; `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` is the pure DECISION arbiter — all keep/drop policy (`resolve_trust` cross-mark corroboration + the assumed-trust floor + the pill gate) in one image-free, unit-testable function (`tests/test_watermark_registry.py::TestArbiter`); then `remove_auto_marks` does the ACTION, localizing -> filling each winner. The Gemini FP gate deliberately stays inside `gemini_engine` (not the arbiter) because `identify` reads that same gated confidence — pulling it out would drift the provenance verdict. Behavior was byte-identical to the pre-arbiter two-pass when the arbiter landed; `assume_ai` has since gained the assumed-trust confidence floor (see below), which deliberately changes its verdict on weak gate-bypassed matches.
+
+**Head-to-head validation (v0.12.1 reverse-alpha vs the current localize -> fill):** run over the full labelled visible-mark set, with the cv2 / MI-GAN / LaMa fills each compared against the old reverse-alpha. **doubao and jimeng are identical** across every backend -- 100% coverage and 100% clearance either way. **gemini** strict coverage is a few points below reverse-alpha's (the deliberate false-positive tightening), but the metadata-stripped faint ones are now mostly recovered by the DEFAULT white-core rescue in the FP gate (`gemini_engine`: a bright near-WHITE core distinguishes a real faint sparkle from a colored bright corner -- ~14/20 recovered at ~1.25% clean false-fire; a learned classifier on the same features measured worse, 2026-07 tier-1), the residual under `assume_ai`; clearance is equal (~98% both), and neither version touches pixels outside the mark box (outside-box PSNR ~99). **Clearance is fill-independent** -- cv2, MI-GAN and LaMa all strip the mark's shape equally, so the re-detect metric does not separate them; the difference is purely the *visual fill quality* on the recovered region, and it is background-dependent. reverse-alpha recovered textured and especially regular/structured backgrounds (a lattice, a grid) more cleanly than any inpaint; **LaMa closes most of that gap** (the best learned backend), **MI-GAN can ghost or hallucinate structure**, and **cv2 smears** (the last-resort floor). This is why `auto` resolves `LaMa > MI-GAN > cv2` (`preferred_inpaint_backend`) and warns once on the cv2 fallback; on flat backgrounds every backend is clean.
+
+**`assume_ai` removed (2026-07-19).** It relaxed EVERY mark's false-positive gate on the caller's bare assertion that an image is AI. That assertion says nothing about WHICH vendor or WHERE the mark is, which is exactly what a gate bypass is contracted to require -- before it carried a confidence floor it filled a phantom sparkle on 59.8% of genuine camera photos, and even with the floor it was a statistical gamble rather than an instruction. It also had no place in the product's model: detector finds a mark -> remove it; detector finds nothing -> leave the image alone; the USER sees a mark and says so -> act on that.
+
+Removing it collapsed the trust ladder from three levels to two (`strict` / `confirmed`) and took `_ASSUMED_CONF_FLOOR`, `assumed_floor_ok` and the `assumed` level with it. `_keep_pill` lost its `sensitivity` parameter (its assume-arm is gone; the metadata arm and its flatness guard are unchanged). Verified on the 240-image unbiased recall sample: doubao 92%/99%, gemini 96%/80%, jimeng 71%/71%, pill 50%/60% -- identical before and after, so nothing on the default path moved.
+
+**The replacement advice is PER MARK, because the forced paths are not equally reliable** (measured 2026-07-19):
+
+| path | reliability |
+|---|---|
+| `erase --region x,y,w,h` | sound by construction -- the user supplies the coordinates |
+| `--mark <text-mark> --no-detect` | reasonable: the forced mask is the real glyph blob, non-empty on 13/13 missed doubao marks |
+| `--mark gemini --no-detect` | **NOT recommended** -- falls back to a fixed default sparkle slot, which covered the true sparkle on only **31% of 97** sparkles the strict gate missed (median offset 63px up-and-left). The other 69% fill a clean corner AND report a removal that did not happen. |
+
+`cli._no_visible_mark_exit` therefore recommends `erase --region` first and a named text mark second, and never suggests forcing gemini. It previously recommended `--sensitivity assume-ai`, i.e. the product's own hint contradicted its model.
+
+**Migration is LOUD, not silent.** `Sensitivity` is a `Literal` and unenforced at runtime, so a 0.15 caller passing `sensitivity="assume_ai"` would otherwise get `auto` behaviour in silence -- a quiet semantic change on exactly the release where they need telling. `validate_sensitivity` (called by `api.remove_visible` and by `Context.__post_init__`) raises a `ValueError` naming the replacement. Regression: `tests/test_watermark_registry.py::TestNoBlanketRelaxation`.
+
+**Continuous top-hat detection front-end (`detect_frontend` / `tophat_response`, 2026-07-18).** `extract_mask` thresholds the white top-hat into a 0/255 glyph blob and correlates a binary silhouette against it. That is fine for a mark stamped bold and opaque, and destructive for a faint one: a thin translucent overlay shatters into specks under the threshold, and no template can match a blob that is not there (千问 measured 0.170 mean NCC, **0%** over its gate, against doubao's 0.723 / 82% -- same pipeline, each with its own template). The `tophat` front-end never binarizes: the saturation and absolute-luma gates become WEIGHTS instead of hard cuts, so a faint stroke contributes in proportion to its strength, and the response is max-normalized, which makes the score contrast-invariant.
+
+Doubao is switched to it; jimeng and samsung stay `binary` until measured, because a front-end change must be measured per mark before it ships. Corpus effect on the 240-image unbiased recall sample:
+
+| mark | recall before | recall after | precision before | precision after |
+|---|---|---|---|---|
+| doubao | 89% | **92%** | 99% | **99%** |
+| jimeng / gemini / pill | unchanged | unchanged | unchanged | unchanged |
+
+**The gate is FRONT-END SPECIFIC and must be re-calibrated, not ported.** The continuous response scores higher overall (mean 0.809 vs 0.723 on the same 90 positives), so the binary-era 0.40 left the provenance-relaxed gate (x0.7) far too low: at 0.40 the arm ran 96% recall / 91% precision (8 false fires), at 0.50 it runs 92% / 99% (1 false fire). 0.50 was chosen because it beats the binary front-end on recall at IDENTICAL precision -- a front-end that only trades one for the other would not have been worth shipping. A first pass at 0.40 also silently depressed the PILL (recall 50% -> 33%), because `_keep_pill` suppresses the pill whenever doubao fires; a coupling worth remembering when tuning any bottom-right mark.
+
+**The removal MASK must ride the same front-end, and how it does so was fixed twice.** `tophat` detection does not binarize, but `extract_mask` (which bounds the fill) still does, so a mark faint enough to be found only by the continuous response produced an EMPTY binary blob: `localize` returned `mask=None`, `remove()` was a silent no-op, and `identify` reported `visible_doubao` while `visible` said "no visible mark" on the same file (corpus-measured 2026-07-20: 57 of 60 sampled still-detected Doubao marks untouched, ~8% of its detections). The FIRST fallback (2026-07-19) thresholded the continuous response and took the bounding box of everything above the level -- but the level was `0.5` compared against the max-normalized **uint8 0..255** response, so it selected every non-zero pixel and filled ~120% of the corner box on textured frames (a padded whole-ROI box). It passed parity (a mask that fills everything is trivially detector-clean) and its regression test (a FLAT fixture, where the response is non-zero only on the glyph, so every threshold yields the same box). The SECOND fallback (2026-07-20) uses the detector's OWN best-match box instead: `_tophat_score` was split into **`_tophat_best(image, loc) -> (score, box)`**, the single method whose score gates detection and whose argmax box bounds the mask -- so the two cannot drift by construction, which is how the mismatch arose in the first place. Measured over 14 real faint-path frames (cv2 fill, detector re-run after): the match box fills a **58.7%**-median corner box vs the threshold's **120.9%**, both 100% detector-clean. The largest-connected-component alternative was tighter (10.5%) but removed the mark on only 21% of frames, so it does not cover it and was rejected. Regression: `tests/test_text_mark_faint_mask.py`, whose fixture now carries texture (the flatness of the old one is exactly why it could not see the threshold bug -- mutating the constant to 99.0 left it green). **Any future front-end change must move both the detection and the mask path, or re-check this.**
+
+**Vendor ATTRIBUTION for the shared-suffix marks -- solved for 千问 by exact sizing, not by a generic template.** The 2026-07-18 worry was that "千问AI生成" and "豆包AI生成" share the `AI生成` tail (four of six glyph cells), so their templates would cross-fire. That held while Qwen was scored at the WRONG SIZE (AUC 0.41-0.59, a coin flip). Re-measured 2026-07-21 at the fitted geometry on real pools (`scripts/vendor_mark_calibrate.py --crossfire`): Qwen's template scores p50 0.224 / p90 0.242 on 400 Doubao-marked frames against a 0.45 gate -- **0 cross-fires**, and 0 on 298 Jimeng-marked frames and 286 clean frames -- because an exact-size 6-glyph template is specific enough that the 2-glyph prefix mismatch dominates the shared tail. Qwen was therefore registered WITHOUT a rival margin (a 0.10 margin would have suppressed ~10% of genuine Qwen detections, whose margin p10 sits at 0.00). A GENERIC shared-tail template remains a harvesting aid, not a detector (measured: 0.407 on a bold positive vs clean p99 0.298).
+
+**千问 is registered since 2026-07-21; 星绘 is NOT (still one confirmed example).** The 2026-07-18 measurement below is why 千问 was originally held out -- it is kept because it records the failure CLASS that the registration then had to solve. Measured back then on 14 hand-verified corpus positives, same pipeline, each mark scored with its OWN template:
+
+| mark | n | mean NCC | median | above the 0.40 gate |
+|---|---|---|---|---|
+| doubao | 40 | 0.723 | 0.835 | **82%** |
+| qwen | 14 | 0.170 | 0.179 | **0%** |
+
+Three candidate explanations were ruled out in order, each by measurement:
+1. **Not the synthetic render.** A template cut from an ACTUAL Qwen mark scores the same as the font-rendered one (real-vs-real 0.307 vs synthetic 0.308), and real masks do not match EACH OTHER.
+2. **Not the morphology kernel.** `MORPH_OPEN`/`MORPH_CLOSE` use fixed 5px kernels regardless of mark size (~9% of a 57px-tall box, ~2.7% of a 188px one). Scaling them with the box height gained +0.014 mean and moved nothing across the gate.
+3. **Not the appearance thresholds.** Sweeping `tophat_delta` / `logo_min_luma` / kernel size peaked at mean 0.35 with 4/14 over the gate.
+
+The blocker was named as SEGMENTATION on a faint mark -- and the `tophat` front-end (built for Doubao the same week) removed exactly that blocker, yet 千问 still did not register, because the real residual was never segmentation alone: it was **mis-sized geometry** (two size modes the shared 3-rung ladder cannot straddle, plus a locate box that clipped the first glyph). The 2026-07-21 cohort harvest (117 labelled frames) and the full calibration chain are in the `qwen_engine.py` section below and `docs/verification-plan.md`. 星绘 remains where it was: ONE confirmed corpus example, so nothing to calibrate a gate against -- do not register it off a single frame. The synthetic renderer and the evidence chain are kept in `scripts/render_vendor_silhouettes.py`; researched vendor specs are in `docs/watermarking-landscape.md`.
+
+**RECALL, measured at last (unbiased random sample, 2026-07-18).** Every earlier round sampled where detectors FIRED, so recall was structurally unmeasurable. This round draws 240 images at RANDOM within each provenance class (160 TC260, 80 Google-C2PA) and labels them EXHAUSTIVELY -- both corners shown at native scale, so a missed mark is visible as a miss rather than absent from the data. Build it with `scripts/visible_recall_sample.py`; labels live in the gitignored research dir.
+
+| mark | present | recall | 95% CI | precision | 95% CI |
+|---|---|---|---|---|---|
+| doubao | 90 | **89%** | 81-94% | **99%** | 93-100% |
+| gemini | 46 | **96%** | 85-99% | **80%** | 68-88% |
+| jimeng | 14 | 71% | 45-88% | 71% | 45-88% |
+| jimeng_pill | 6 | 50% | 19-81% | 60% | 23-88% |
+
+Effect of the `scale_basis` fix on the same sample (strict verdicts recorded before it): **doubao recall 71% -> 89%**, gemini 91% -> 96%, jimeng 64% -> 71%. Part of the doubao gain is the provenance relaxation rather than the basis alone, since the after-numbers run the full `auto` path.
+
+**Three corrections this forced to earlier numbers:**
+* **Gemini precision is 80% on an unbiased sample, not the 41% the addition-sampled harness reports.** The 41% is precision restricted to relaxation ADDITIONS, which are by construction the marginal cases; production sees mostly strict fires, which are near-perfect. Quote 80% for the product and 41% only when discussing the relaxation arm.
+* Doubao is in excellent shape (89/99) and is no longer the problem it looked like before the basis fix.
+* Landscape is improved but NOT solved: doubao recall by aspect is portrait 92% / square 92% / **landscape 79%**, so a residual geometry gap remains beyond the basis.
+
+**Where the remaining loss actually is:** jimeng and the pill, both at small n with intervals too wide to tune against (a 14-positive and a 6-positive sample), plus **uncovered vendors at 6% of all sampled images** (千问/百度/星绘/抖音-class marks that no registered detector can ever fire on). Adding those vendors is now a larger win than any further tuning of the covered four, and `docs/watermarking-landscape.md` carries their researched specs.
+
+**Per-mark geometry scaling (`scale_basis` / `scale_base`, 2026-07-18) -- the largest single recall defect found so far.** Every tuned fraction in `TextMarkConfig` was calibrated on PORTRAIT captures, where the width and the short side coincide, so the scaling basis was never exercised until landscape inputs were measured. Corpus-measured on 2572 unique TC260 carriers, BEFORE the fix:
+
+| aspect ratio | detected | missed | miss rate |
+|---|---|---|---|
+| tall portrait <0.70 | 323 | 212 | 40% |
+| portrait 0.70-0.95 | 607 | 533 | 47% |
+| square ~1.0 | 190 | 272 | 59% |
+| landscape 1.15-1.6 | 0 | 143 | **100%** |
+| wide >1.6 | 0 | 292 | **100%** |
+
+**Not one landscape image in the corpus ever produced a detection** -- 435 of them, zero. A width-scaled box is inflated by the aspect ratio on a wide image, so the glyph never lands inside it and the blob never gets scored: of the 1452 no-detection TC260 images the median doubao NCC was **0.057**, with 49% at ~zero. This is a LOCALIZATION failure, not a threshold one -- only 2.7% of those images sat in the band a threshold change could reach, which is why a day of threshold tuning could never have found it. Re-running the previously-undetected set with a short-side basis recovers **56% of landscape** (12% square, 4% portrait; 20% overall).
+
+**The basis is PER MARK because the vendors genuinely differ.** The same switch took jimeng's labelled landscape positives from 13/13 to **0/13**: the Jimeng wordmark tracks the WIDTH while the Doubao strip tracks the short side, even though both are ByteDance and share a corner. So doubao is `short`, jimeng is `width`, and samsung stays `width` because there is no corpus evidence either way (1 addition corpus-wide) and an unmeasured change is not an improvement. China's GB 45438-2025 clause 5.2(e) mandates glyph height >= 5% of "the shortest side", which is why short-side is the natural prior -- but jimeng's measured behaviour overrides the prior. Regression: `tests/test_text_mark_engine.py::TestScaleBasis`.
+
+**How this was missed for so long:** precision was measured repeatedly and recall never was. The eval harness now reports a `missed` column for exactly this reason, and it is what caught the jimeng regression the short-side switch introduced.
+
+**Competitive detection among same-corner marks (`rivals` / `_rival_margin_ok`, 2026-07-18).** Detection was purely ABSOLUTE -- every engine scored its own template against its own threshold, so nothing ever asked the discriminative question "does this blob match the NEIGHBOUR's mark better than mine?". Doubao "豆包AI生成" and Jimeng "★ 即梦AI" both sit bottom-right in near-white CJK and survive the top-hat binarization as very similar blobs, so no absolute gate can separate them. Measured on hand-labelled examples, scoring BOTH templates against the SAME glyph blob (n=40 jimeng / 75 doubao / 20 other-vendor labels / 89 clean):
+
+| feature | separability (0.5 = useless, 1.0 = perfect) |
+|---|---|
+| absolute `ncc_jimeng` | 0.96 |
+| `ncc_jimeng` MINUS `ncc_doubao` | **0.99** |
+
+At a 0.10 margin: real Jimeng wordmarks pass **100%**, Doubao strips 8%, other vendors' AI labels (千问/百度/星绘/抖音) 55%, no-mark corners 12%. Corpus effect (`scripts/visible_eval.py`, 741 labelled images): **jimeng precision 38% -> 63%, genuine detections unchanged at 40, false fires 65 -> 23.** Because real marks pass at 100% this is a pure precision gain, unlike raising a threshold -- so the earlier 0.85 relaxation patch was REVERTED to 0.70 and the recall it had sacrificed came back. **The gate is deliberately asymmetric:** doubao declares no rival, because the symmetric gate cost it 7 genuine detections to prevent 5 false ones (1.4:1 against) while jimeng gained 25pp for free -- doubao's absolute detector is already 86% precise and has nothing to buy. A rival's config is looked up lazily by asset name (`_rival_config`) so its template is scored at ITS own geometry; scoring it at the host mark's geometry would compare a correctly-sized template against a mis-sized one and hand the margin a free win. Regression: `tests/test_text_mark_engine.py::TestRivalMargin`.
+
+**Evaluation harness (`scripts/visible_eval.py` + `scripts/visible_groundtruth.py`, 2026-07-18).** Run before AND after any detector change; `--save NAME` snapshots, `--vs NAME` diffs. Ground truth is 741 blind-labelled corpus images (779 cells across two rounds, two-sided control each). Three properties of the harness are load-bearing and were each added after the naive version produced a wrong number:
+
+* **Adjudication scope.** A crop centred on one mark only lets the labeller rule on marks visible IN THAT CROP. Scoring jimeng against a pill-round image (top-left crop) books real bottom-right detections as false fires -- ~61% of pills carry a wordmark. Each image records which marks its crop could rule on; bottom-right marks co-adjudicate each other.
+* **Provenance must come from METADATA, never from the labels.** A relaxation arm only fires when provenance names the vendor, so label-derived provenance hands the detector the answer: it scored gemini at 99% instead of the true 41%.
+* **Recall is NOT reported.** The labelled set was sampled where detectors fired, so images every detector missed are absent by construction; a recall computed here would divide by a denominator that excludes exactly the failures recall exists to expose. The `missed` column catches a change LOSING marks it used to find, nothing more. True recall needs a random corpus sample labelled exhaustively -- not yet done.
+
+Baseline at the time of writing (sensitivity `auto`, provenance from metadata): gemini 41% (321 fires), doubao 86% (77), jimeng 63% (63), jimeng_pill 64% (83), samsung unmeasurable (1 addition corpus-wide).
+
+**Per-mark provenance NCC relaxation + the corroboration gate (2026-07-18).** Two defects, both on the DEFAULT `auto` path (no flag, driven by TC260 metadata), found by blind hand-labelling the ADDITIONS (accepted with provenance, rejected without) over 4417 unique TC260 carriers. Two-sided control: labeller sensitivity 100% (doubao) / 96% (jimeng), specificity 100% / 100% — the controls are what make the low numbers trustworthy, and the "clean" stratum is structural (another vendor's C2PA image, where a ByteDance mark cannot exist) rather than detector-defined, so it is not circular.
+
+(1) **One shared `_PROVENANCE_NCC_FACTOR = 0.7` meant two different things per mark:**
+
+| mark | band | precision | 95% CI | n |
+|---|---|---|---|---|
+| doubao | whole arm | 76% | 61-87% | 42 |
+| | [0.280,0.340) | 58% | 36-77% | 19 |
+| | [0.340,0.400) | 91% | 73-98% | 23 |
+| jimeng | whole arm | 17% | 10-27% | 82 |
+| | [0.315,0.383) | 12% | 6-22% | 68 |
+| | [0.383,0.450) | 43% | 21-67% | 14 |
+
+The factor is now a per-mark `TextMarkConfig.provenance_ncc_factor`. Doubao stays 0.70 — both bands return more true marks than false fills, so tightening would cost 11 genuine recoveries to prevent 8. Jimeng moves to 0.85 (gate 0.3825), dropping the 12% band: −8 genuine recoveries, −60 false fills (7.5:1), arm precision 17% → 43%. **Why jimeng fails is a detector problem, not a threshold one:** of its 68 false additions, 33 were DOUBAO marks and 17 were other vendors' AI labels (千问 / 百度 / 星绘 / 抖音) — relaxed, the silhouette keys on "some text in the bottom-right corner", not on "★ 即梦AI". Damage was scored separately because doubao and jimeng share a corner: 45 of the 68 fill a corner nothing else would touch, the other 23 are harmless (doubao fires strictly there and fills the same box anyway). A better silhouette, not a lower factor, is the real fix.
+
+(2) **A weak detector must not corroborate a sibling (`_CANNOT_CORROBORATE`).** `resolve_trust` grants `confirmed` on a strict-detected sibling of the same `_PRODUCT_OF`, and `confirmed` bypasses the sibling's FP gate outright. The pill (~7% documented raw false-fire; 5.5% on 578 vendor negatives) maps to product "jimeng", so it could hand that bypass to the wordmark — a closed loop: pill false-fires on clean non-ByteDance content → jimeng relaxes 0.45 → 0.3825 and false-fires → `_keep_pill` sees "jimeng" in keys and takes the WORDMARK arm, removing the pill **unrestricted**, skipping the flatness guard written to stop exactly that smear. 3 of 578 negatives ran the full loop, one with `footprint_flat=0`. The fix costs nothing: negatives 3 → 0, TC260 carriers unchanged (jimeng 398 → 398, pill 117 → 117). `_keep_pill` already encoded this distrust for the pill's ACTION; the gap was that its TESTIMONY was ungated.
+
+**Pill arms, re-measured on the same corpus** (149 blind-labelled TC260-arm fires, 35 wordmark, 33 unconfirmed): wordmark **94%** (CI 81-98%, confirming the original claim), TC260-metadata-only **21%** raw (CI 16-29%, consistent with the original ~27%) — **29%** (CI 20-40%) among the flat footprints the guard PASSES vs 14% among those it blocks. The guard works directionally but weakly: the shipped arm still runs at ~2.4 false fills per genuine one. Whether an arm that inaccurate belongs on the default path is a product call, not a tuning one.
+
+**Samsung's relaxation is UNMEASURED and not measurable here:** the corpus holds 14 `samsung_genai` carriers and 3 visible Samsung detections total. Any precision estimate would carry a Wilson interval spanning most of [0,1]. Likely structural — detection is calibrated to the Italian locale string only.
+
+
+**Provenance prior:** when local metadata already confirms the vendor, the mark's detection trust gate is relaxed (a confirmed vendor means the mark is present with high prior, so a mark the conservative detector would demote as a content false positive is trusted). `detect_marks` / `remove_auto_marks` take a `provenance` frozenset and `KnownMark.remove` a `provenance` flag. Mapping: a Google/Gemini C2PA issuer relaxes gemini (skips its false-positive gate and lowers the trust threshold from 0.5 to 0.35); a China-AIGC (TC260) label relaxes doubao/jimeng; `samsung_genai` relaxes samsung. Corpus finding: on Google-C2PA images, Gemini sparkle recall rose from ~46% (plain detector) to ~90% with the provenance prior (recovering marks the vendor moved or re-rendered). That gain is why the bypass exists, and it is conditional on the metadata actually naming the vendor — a caller merely ASSUMING the image is AI does not get it unconditionally (see the assumed-trust confidence floor above). The localizer is cheap CPU (cv2/numpy), so a memory-tight caller runs it anywhere; the heavy MI-GAN/LaMa fill is opt-in and chosen by the caller.
+
+**Cross-engine confidences aren't directly comparable**, so the gemini adapter applies the corpus-validated 0.5 sparkle threshold (`_GEMINI_AUTO_MIN_CONF`) for its `detected` flag (lowered to 0.35 under the Google/Gemini provenance prior) — otherwise the gemini engine's loose internal threshold weakly fires (~0.36) on the Doubao text and hijacks `auto`. The shape-keyed Doubao/Jimeng/Samsung NCC detectors don't cross-fire (jimeng scores ~0.22 on the Doubao strip, well under its 0.45 threshold; Samsung is bottom-left so it shares no corner with the others, and scored 0.0 on Doubao/Jimeng captures and they 0.0 on a real Samsung photo), so `auto` picks the right one. `cli.cmd_visible` is registry-driven: `--mark auto` → `remove_auto_marks` (removes every detected mark), `--mark <key>` → that mark; `--mark` choices come from `mark_keys()`.
+
+**`cli._remove_visible_auto` is the shared visible-removal helper used by `cmd_all`/`cmd_batch` too** (they no longer hardcode `GeminiEngine`), so `all`/`batch` remove Doubao/Jimeng/Qwen/Samsung text marks, not just the Gemini sparkle (regression-guarded by `test_all_visible_step_uses_registry`). The three text-mark adapters were consolidated 2026-06-09: a single `_text_mark(key, label, location)` builds the registry row from one parameterized `_text_mark_detect`/`_text_mark_remove` pair (the remove adapter localizes the glyph footprint and hands it to the shared `fill` only when detected/forced, else skipped); the gemini adapters stay bespoke. Add a new visible mark = one `_text_mark(...)` row + its `TextMarkConfig` (with a captured alpha map for the detection silhouette); do not re-add per-mark `if` branches or copy-paste adapters.
+
+**Alpha-on-save policy (issue #30):** `image_io.write_bgr_with_alpha` (it lives in `image_io`, not `cli` — moved so the CLI and the library `api` share ONE implementation) rejoins the input's alpha plane **unchanged** — it must NOT zero alpha in the watermark bbox. The fill reconstructs real pixels there, so zeroing alpha punched a transparent hole that renders as a solid **white box** on any non-transparent viewer (Gemini app exports are opaque RGBA, so every user hit it; regression-guarded by `test_visible_keeps_alpha_opaque_in_watermark_region`). The registry `remove()` still returns its region, but the CLI no longer uses it to clear alpha. **It returns `imwrite`'s success flag and callers must check it** (2026-07-20): `imwrite` is contractually non-raising, so that bool is the only signal the file was not created. The wrapper previously returned `None` and swallowed it, so every CLI write site ran `output.stat()` to report the size and a read-only destination died with a bare `FileNotFoundError` traceback pointing at the stat instead of the write. The CLI now writes through the shared `cli._write_output_or_exit`. Regression: `tests/test_cli_robustness.py::TestFailedWriteIsReported`.
+
+## `gemini_engine.py`
+
+`gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU). `detect_sparkle_confidence(path)` is the file-level entry point used by `identify.py`. The public entry points normalize a grayscale (2D) or RGBA (4-channel) input to BGR up front so a non-BGR image does not crash the cv2 pipeline.
+
+**Detection localization (issue #36):** `detect_watermark`'s global multi-scale NCC search applies a size weight (`(scale/96)**0.5`) that suppresses tiny-patch false positives but can let a larger, mediocre match (e.g. a bright collar in a portrait) outrank a small, near-perfect sparkle in the corner — so a faint sparkle on a busy background scored below threshold and read as clean (the regression osachub reported from widening the search window 256px->512px between v0.7.2 and v0.8.8). `_corner_promote` adds a bottom-right-corner raw-NCC pass on top of the global search: a match with raw NCC >= `_CORNER_PROMOTE_NCC` 0.85 that beats the global pick overrides it (it only ever replaces a lower-fidelity pick, so it cannot weaken an existing detection), rescuing the buried sparkle without reverting the wider window. The corner side is **relative-clamped** (`_CORNER_PROMOTE_FRAC` 0.20 of the short side, clamped to `[_CORNER_PROMOTE_MIN` 96, `_CORNER_PROMOTE_MAX` 384`]`): a fixed 256px is a true corner on a large image but covers ~70% of a small portrait, where a real photo raw-matches the star at ~0.81 (relative tightening drops that worst case to ~0.69, while the upper clamp stops the corner ballooning on huge images where a real photo reached ~0.83 at 512px). The 0.85 gate sits midway between the worst real-photo corner match (~0.78 across native + downscaled negatives) and a genuine faint sparkle (~0.93), so promotion adds true detections with zero corpus false positives (Gemini's sparkle sits ~60-160px from the corner at fixed margins, covered by the [96, 384] band at every measured size). Regression-guarded by `test_gemini_engine.py::TestCornerPromotion`.
+
+**Top-K fusion selection (osachub follow-up 2026-06-12):** `_corner_promote`'s 0.85 raw-NCC gate still missed a class the 256->512 widening exposed — a genuine MID-scale sparkle whose raw NCC sits *below* 0.85 but is buried by a LARGER, low-fidelity decoy that wins the size weight. The reporter's image (a scale-48 sparkle on light bedding) measured spatial 0.775 / grad 0.960 / fusion 0.676 at the true sparkle, but the size-weighted argmax instead locked onto a decoy at spatial 0.628 / grad 0.036 (fusion 0.325) — so `identify` read `unknown` on v0.8-0.11 where v0.7.2 (256px window) had caught it at 0.676. Fix: `detect_watermark` now keeps the **top-`_SELECT_TOPK` (3)** size-weighted candidates (NMS-deduped by location) plus the corner-promote candidate, scores EACH by the full fusion (spatial+gradient+variance) via the extracted `_grad_var_scores` helper, and selects the highest — the gradient term (the discriminator a contrast-invariant spatial NCC lacks) lifts the true sparkle over the decoy. Critically, selection ranks by the SIZE-WEIGHTED score, NOT raw NCC: a raw-NCC argmax (tried first) re-admitted the exact tiny-patch (scale 16-18) false positives the size weight exists to suppress — it flagged 14/65 doubao + 4/11 jimeng visible-corpus images (non-Gemini content) as Gemini sparkles. Top-K keeps tiny-patch suppression intact: a coincidental 16px match never ranks in the size-weighted top-K, so widening selection added **zero** flips on the doubao/jimeng corpora and left the 495-image Gemini set unchanged (479 detected, both before and after) while recovering the reporter's image. Regression-guarded by `test_gemini_engine.py::TestCornerPromotion::test_low_gradient_decoy_loses_to_high_gradient_corner_sparkle` (mirrors the real spatial/grad signature via a monkeypatched scan) and `test_size_weighted_search_alone_traps_on_the_decoy`.
+
+**Square-image residual misses are NOT fixable by lowering the detector threshold (measured + REJECTED 2026-06-11):** osachub (#36 follow-up) reported the corner-promote still misses Gemini sparkles on Google **square (1:1)** outputs. Reproduced on the spaces corpus: of 330 square Google-C2PA images, 140 score below the identify 0.5 threshold, and visual review confirmed a real class -- faint white sparkles on dark/textured/colored backgrounds (raw NCC 0.46-0.73, below the 0.85 promote gate) landing at fusion conf 0.41-0.47. A margin-gated promote (promote when raw NCC >= 0.50 AND `_core_ring_margin` >= 40) rescued 32/33 confirmed misses at an apparent 0 FP, but that 0 was a **measurement artifact** -- the negative set was the margin<40 misses, which a margin>=40 gate excludes by construction. On an honest 518-image non-Google pool the same gate fired on **~174 (≈33%)**, visually content (screenshots, Chinese "AI生成" Doubao/Jimeng text marks, logos, bright textures), not sparkles. Adding an achromatic-core constraint (`chroma <= 15`) did not separate them either (kept 15/33 POS, 41 NEG still firing). Root cause is the documented contrast-invariant-NCC wall: a faint sparkle on a busy background is indistinguishable from a bright/ornate content corner at the (shape-NCC, brightness-margin, core-chroma) feature level.
+
+**Conclusion: keep the 0.85 corner gate; do NOT add a margin/chroma-gated lower promote.**
+
+The cost (mislabel ~8-33% of non-Gemini content as Gemini) outweighs the benefit -- the visible sparkle is a medium-confidence stripped-metadata fallback, and intact Gemini is caught by C2PA in `identify` regardless. Remaining square misses are an accepted known limitation; a real fix would need a sparkle-specific discriminator (template match on a background-subtracted image, or a hard fixed-margin position prior), which is open research, not a threshold tweak.
+
+**Removal is localize -> fill** (`footprint_mask` → `watermark_registry.fill`): `footprint_mask` returns the sparkle footprint = the captured alpha (computed `alpha = max(R,G,B)/255` from the bundled sparkle-on-black captures `assets/gemini_bg_{96,48}.png`, capture max ~130 for the ~51%-opaque overlay) thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin. That binary mask is inpainted by the shared fill (cv2 / MI-GAN / big-LaMa). The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery. This replaced the old reverse-alpha removal path; because the fill only reconstructs the masked footprint from its surroundings (rather than dividing by `1-a`), the whole reverse-alpha removal tail — the over-subtraction guard (`_reverse_alpha_oversubtracts`, the dark-background black-pit fix), the under-subtraction alpha-gain estimate (`_estimate_alpha_gain`), and the self-verify repair — is GONE, along with the near-white `1/(1-a)` ill-conditioning and the "color changed, not removed" failure mode those guards patched around. A slightly-off localization now just fills a small region near-losslessly instead of leaving a color-shifted smear.
+
+**False-positive gate (added 2026-06-03):** `detect_watermark`'s shape-only NCC (`spatial*0.5 + gradient*0.3 + var*0.2`) fires on ornate/flat content (text strips, banners, hatching) that coincidentally matches the diamond shape — a real Gemini sparkle is a bright WHITE overlay, so its core sits above the local background, but the NCC is contrast-invariant and cannot see that. The fusion now **demotes** (caps confidence to 0.30) any low-confidence (`< _SPARKLE_FP_CONF` 0.65) match that shows NEITHER real-sparkle signature: a bright core (`_core_ring_margin >= _SPARKLE_FP_MARGIN` 5) OR a crisp star silhouette (`gradient_score >= _SPARKLE_FP_GRAD` 0.55). I.e. demote when `low_margin OR low_grad`. Real sparkles escape via high confidence (white-bg sparkles score ≥0.79 despite a low margin — the NCC shape match is strong), high margin (dark/mid backgrounds, incl. the #36 faint-corner case, lift well clear), OR high gradient (a real sparkle is grad ~0.97–1.0). **The gradient condition (added 2026-06-26) closes the bright-background FP class** the margin check alone missed: a snow+sky photo and a white-background product render both scored ~0.51 at `identify`, because a bright background gives the match a HIGH core-ring margin (it genuinely IS brighter than its surroundings), so the brightness gate read it as a real overlay — but a smooth luminance blob that shape-NCC-matches the rough diamond has low gradient fidelity (the two FPs measured grad 0.105 and 0.463 vs ≥0.8 for real sparkles), so the gradient floor demotes them. The OR is **strictly a superset** of the old margin-only demotion (it only ADDS demotions on bright backgrounds, where a real sparkle keeps grad ~0.97), so it cannot regress a dark/mid sparkle (kept by margin) or a white-bg one (kept by confidence ≥ 0.65). The gate is **monotonic** (only ever removes detections, never adds), so it cannot regress the verified-negative corpus (already 0 FPs); the 2026-06-26 corpus re-sweep flipped only OpenAI/ChatGPT content (no Gemini sparkle exists there) and already-`cleaned/` outputs, all sub-0.5 (below the `identify` threshold), so no provenance verdict changed. The original gate demoted 16/495 flagged sparkles on the validation corpus (13 carried no AI metadata = content FPs; the 3 AI-meta were visually FPs / a near-invisible white-on-white sparkle whose AI verdict is held by metadata anyway). `_core_ring_margin` uses the `_core_and_bg` helper (core 75th-pct brightness vs background-ring median). This gate is detection-side and unchanged by the localize -> fill refactor; the provenance prior skips it when a Google/Gemini C2PA issuer confirms the vendor. Regression-guarded by `test_gemini_engine.py::TestSparkleFalsePositiveGate` (incl. `test_bright_background_low_gradient_match_demoted`).
+
+**The reverse-alpha removal tail is retired.** The self-verify repair (`_verify_and_repair`), the offset+scale alignment search, and the near-white `1/(1-a)` ill-conditioning survivors were all artifacts of solving the sparkle by inverting the alpha map. Under localize -> fill the footprint is reconstructed from its surroundings by the shared fill, so those failure classes and their guards no longer exist. The lesson from that era still holds and generalizes: a re-detect-confidence audit metric is gameable by reshaping the residual, so judge a visible removal by physical inspection of the footprint, not the detector alone.
+
+**The bg assets are rebuilt from OUR OWN controlled captures** (`data/gemini_capture/captures/`, committed) by `scripts/visible_alpha_solve.py gemini`, which locates the 96px sparkle on the black capture and crops it to the two logo sizes; our capture matched the previously third-party-sourced `gemini_bg_96.png` to **NCC 0.9998**, validating the asset and making it reproducible. Gemini's multi-size fixed-slot model is genuinely different from the Doubao/Jimeng text-strip engines (so it stays a separate engine, not part of the shared-base refactor).
+
+## `_text_mark_engine.py`
+
+`_text_mark_engine.py` — **shared base for the three text-mark engines (Doubao/Jimeng/Samsung), extracted 2026-06-09** (they were ~90% byte-identical clones). `TextMarkEngine(config: TextMarkConfig)` owns the `locate → extract_mask → detect` detection pipeline plus the removal that localizes the glyph blob to a footprint mask and hands it to the shared `watermark_registry.fill` (+ the asset-keyed `load_alpha_template`/`glyph_silhouette`/`template_match_score` caches). Detection still matches the glyph silhouette (NCC against the captured template); the removal MASK is TEMPLATE-FREE — it is the bounding box of the top-hat glyph blob from `extract_mask`, filled solid + dilated, so a re-rendered or differently-placed mark is still masked. This dropped the fixed alpha-template placement; the captured alpha maps are now used only for the detection silhouette, not for removal. Each engine module is a thin subclass supplying only its `TextMarkConfig` (the tuned constants, the bundled asset, and the bounded structural deltas — `corner` br/bl, `margin_floor` 4/2, `morph_open_size` 5/3, `min_gw` 8/16, and since 2026-07-21 `ladder` — the scale rungs `_tophat_best` sweeps, per-mark because 千问's two size modes do not fit the shared 3-rung comb (default `(0.8, 1.0, 1.25)`, unchanged for every other mark; densifying the SHARED ladder was measured and rejected -- see the verification plan's B2). plus the test-facing module shims (`_alpha_template`/`_glyph_silhouette`/`_template_match_score` + the constants). Gemini stays a SEPARATE engine (its multi-size fixed-slot sparkle model is genuinely different). Add a new text mark = a new `TextMarkConfig` + a thin subclass + one registry `_text_mark(...)` row. The engine bullets below describe each mark's calibration history; the LOGIC lives here. **Small-image detection guard (`_MIN_DETECT_SHORT_SIDE` 200, added 2026-06-26):** `detect` returns not-detected when the image short side is below 200px. Below that the glyph template degrades to the `min_gw` floor (~8px) and `TM_CCOEFF_NORMED` on a few pixels is noise, so an unrelated small geometric shape can spuriously correlate with the CJK silhouette — a 48×48 app-icon chevron scored Doubao 0.41 / Jimeng 0.47 (both above their thresholds), a pure small-size artifact (the same icon upscaled collapses to ~0.06–0.10 NCC at ≥256px). A real AI-generation label is stamped on a full-resolution render (the captured samples are 1086–2048px wide, the smallest positive test image is 1086px), so the floor sits far below any genuine mark while killing the icon/thumbnail band (≤96px); `identify` falls back to "unknown" (the safe default) and removal, gated on detection, is suppressed too. Regression-guarded by `test_{doubao,jimeng,samsung}_engine.py::TestDetect::test_small_image_guarded_from_false_positive`.
+
+**Removal is localize -> fill.** The engine localizes the glyph blob (`extract_mask` over the located box) into a solid, dilated footprint mask and hands it to the shared `watermark_registry.fill` (cv2 / MI-GAN / big-LaMa). The template-free mask (bounding box of the glyph blob, not the fixed alpha template) means a re-rendered or moved mark is still covered, and the fill reconstructs the box from its surroundings. On corpus images doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal.
+
+**The reverse-alpha removal machinery is retired.** The old per-glyph reverse-alpha blend (`_apply_reverse_alpha`), the fixed/aligned alpha-map helpers, the over-subtraction guard (`_reverse_alpha_oversubtracts` → `_inpaint_footprint`, the dark-pit fix on dark/mid-tone backgrounds), and the always-align placement search are all gone — the fill reconstructs the footprint from its surroundings rather than inverting the captured alpha, so the dark-pit and color-shift failure modes those guards patched around no longer arise. `extract_mask` still returns a box-sized (`(loc.h, loc.w)`) mask rather than a full frame, which keeps the memory-tight `identify` detect path cheap.
+
+## `doubao_engine.py`
+
+`doubao_engine.py` — **a thin `_text_mark_engine.TextMarkEngine` subclass (config only) since 2026-06-09.** visible Doubao "豆包AI生成" detector + localizer (cv2/numpy, no GPU). `DoubaoEngine.locate` anchors a bottom-right box by **geometry** (mark scales with image WIDTH), `extract_mask` pulls the light, low-chroma glyphs (the detection candidate) using a per-pixel channel-spread proxy `sat = roi.max(axis=2) - roi.min(axis=2)` (no HSV conversion). `detect` is **shape-consistent**: it matches the bundled glyph silhouette (`assets/doubao_alpha.png`) against the candidate via zero-mean normalized correlation (`_template_match_score`, cv2 `TM_CCOEFF_NORMED`), gated at `DETECT_NCC_THRESHOLD` 0.4 over a small `DETECT_MIN_COVERAGE` floor. Keying on glyph SHAPE (not coverage heuristics) fixed #23 (corpus FP 7/1243).
+
+**Removal is localize -> fill:** the glyph blob is localized to a solid, dilated footprint mask (`extract_mask` over the located box) and the shared `watermark_registry.fill` inpaints it. On corpus images this removes at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit).
+
+**The detection template (`assets/doubao_alpha.png`) is rebuilt by `scripts/visible_alpha_solve.py`** (the careful gray-self solve: cubic background fit, mean over channels, full halo, unblurred), same recipe as Jimeng — the captures are committed in `data/doubao_capture/captures/`. It is used only as the detection silhouette, not for pixel recovery.
+
+**The locate box (`WM_*`) is generous (0.22 wide, margins 0.004) and reaches close to the corner** so a re-rasterized, corner-ward-shifted mark still falls inside the localized box; regression-guarded by `test_recovers_shifted_mark_on_texture` (composes the mark shifted on a known texture). **`extract_mask` guards a degenerate ROI (`bh < 16 or bw < 16` -> empty mask, skips cv2)** — an extremely wide/short image (e.g. 2048x1, `test_wide_short_does_not_raise`) once fed cv2's GaussianBlur a ~1-px-tall ROI and **faulted natively on Windows py3.12**; real images always clear the guard (the `WM_*` box floors are `max(16, …)` height / `max(40, …)` width), so it only short-circuits slivers. The registry gates removal on `detect`. The shipped third-party `_refs/zhengsuanfa_doubao_alpha_120x20.png` is NOT a usable template (verified 2026-05-29). Arbitrary-region inpainting is `region_eraser`/`erase`. **Lesson from the reverse-alpha era (still holds): a detector-only removal test is insufficient; assert visual residual (the textured-shift test).**
+
+## `jimeng_engine.py`
+
+`jimeng_engine.py` — **a thin `TextMarkEngine` subclass (config only) since 2026-06-09.** visible Jimeng / Dreamina "★ 即梦AI" detector + localizer (cv2/numpy, no GPU), built 2026-05-30 from issue #13's solid captures (@powersee). Shares the base with `doubao_engine`: `locate` anchors a bottom-right box by **geometry** (scales with WIDTH), `extract_mask` pulls the light low-chroma glyphs (white top-hat + grayish + min-luma), `detect` matches the bundled "即梦AI" glyph silhouette (`assets/jimeng_alpha.png`) via `TM_CCOEFF_NORMED` over a coverage floor. Threshold `DETECT_NCC_THRESHOLD` **0.45** cleanly separates real Jimeng marks (>=0.81) from the Doubao strip (0.21) and other AI output (0.0), so the two ByteDance marks don't cross-fire in `--mark auto`.
+
+**The detection template (`assets/jimeng_alpha.png`) is rebuilt by `scripts/visible_alpha_solve.py` from the GRAY capture** (`data/jimeng_capture/captures/`, the solid captures committed): `a = (I - B)/(255 - B)`, B a per-capture **cubic** background fit over the non-glyph pixels, **averaged over channels, full halo extent (down to a~0.02), unblurred**. Gray (bg ~132) is the deliberate choice over black: it is the best proxy for real content (the mark sits on bright photo areas, not on black). The captured template is used only as the detection silhouette, not for pixel recovery. Solver geometry at `_ALPHA_NATIVE_WIDTH` 2048: `_ALPHA_WIDTH_FRAC` 0.202, `_ALPHA_HEIGHT_FRAC` 0.058, margins ~0.029.
+
+**Removal is localize -> fill:** the glyph blob is localized to a solid, dilated footprint mask and the shared `watermark_registry.fill` inpaints it; the `WM_*` locate box is generous so a re-rasterized, corner-ward-shifted mark stays inside the localized box (the same widen that fixed Doubao). On corpus images this removes at ~100% with clean footprints (blends within a few LAB levels, no color shift). The registry gates removal on `detect`.
+
+**No committed real sample** (only the solid calibration captures are committed) — `tests/test_jimeng_engine.py` synthesizes a mark from the bundled template, and `test_recovers_shifted_mark_on_texture` guards the localize-on-shift path that the Doubao defect exposed. Jimeng images are independently caught by the China TC260 AIGC label in `metadata`/`identify`, so this engine is the visible-mark *removal* path, not a new `identify` signal.
+
+## `samsung_engine.py`
+
+`samsung_engine.py` — **a thin `TextMarkEngine` subclass (config only) since 2026-06-09.** visible Samsung Galaxy AI "✦ Contenuti generati dall'AI" detector + localizer (cv2/numpy, no GPU), built 2026-06-05 from issue #37's flat captures (@f-liva). Shares the base but anchored **bottom-LEFT** (Doubao/Jimeng are bottom-right): `locate` anchors a bottom-left box by **geometry** (scales with WIDTH), `extract_mask` pulls the light low-chroma glyphs (white top-hat + grayish + min-luma — `LOGO_MIN_LUMA` is lowered to **110** because the mark is faint, peak alpha ~0.38, so on a mid/dark background its glyph luma is lower than Jimeng's), `detect` matches the bundled glyph silhouette (`assets/samsung_alpha.png`) via `TM_CCOEFF_NORMED` over a coverage floor. Threshold `DETECT_NCC_THRESHOLD` **0.40** (real marks ~0.79 on a real photo, ~0.57/0.71 on the black/gray captures; 0.0 on Doubao/Jimeng captures, and Doubao/Jimeng score 0.0 on a real Samsung photo — no cross-fire, also because the corner differs).
+
+**The detection template (`assets/samsung_alpha.png`) is solved by `scripts/visible_alpha_solve.py samsung` from the GRAY capture** (`data/samsung_capture/captures/`, the flat black/gray/white captures committed; the solver gained a `corner="bl"` mode + left-margin logging for this), same careful recipe as Jimeng (cubic background, mean-channel, full halo, unblurred). Geometry emitted at `_ALPHA_NATIVE_WIDTH` **1086** (the flat-edit capture width): `_ALPHA_WIDTH_FRAC` 0.3195, `_ALPHA_HEIGHT_FRAC` 0.0378, `_ALPHA_MARGIN_LEFT_FRAC` 0.0110, `_ALPHA_MARGIN_BOTTOM_FRAC` 0.0064. Used only as the detection silhouette, not for pixel recovery.
+
+**Removal is localize -> fill:** the glyph blob is localized to a solid, dilated footprint mask and the shared `watermark_registry.fill` inpaints it. Verified on a real 2958-wide @f-liva photo: re-detect 0.79→0.00, no readable text or outline on the recovered wooden table — checked **visually**, not just by the detector. The registry gates removal on `detect`.
+
+**Detection is locale-specific** (the string differs per language); this build detects only the Italian "Contenuti generati dall'AI" variant, so non-Italian Samsung locales are not detected — and, because detection gates removal, not removed — even though the fill mask itself is locale-independent. Other locales need their own detection silhouette — the locale string font-rendered and calibrated on real positives (the pill's `scripts/render_pill_silhouette.py` pattern), NOT an app capture (the solid/gray/white capture workflow retired with reverse-alpha). This is a pre-existing limit, unchanged by the localize -> fill refactor.
+
+**No committed real sample** (only the flat calibration captures are committed) — `tests/test_samsung_engine.py` synthesizes a mark from the bundled template (bottom-left geometry), with `test_recovers_shifted_mark_on_texture` guarding the localize-on-shift path. Samsung Galaxy AI edits are independently caught by C2PA + the `genAIType` marker in `metadata`/`identify`, so this engine is the visible-mark *removal* path; it also feeds `identify` as the medium-confidence `visible_samsung` signal via the registry (the stripped-metadata fallback).
+
+## `qwen_engine.py`
+
+`qwen_engine.py` — **a thin `TextMarkEngine` subclass (config only), registered 2026-07-21.** visible Qwen (Alibaba Tongyi Qianwen) "千问AI生成" detector + localizer (cv2/numpy, no GPU), bottom-right, the same GB 45438-2025 6-glyph house style as Doubao (2-glyph vendor prefix + mandated `AI生成` tail; the vendor's tri-lobe logo precedes the text and is deliberately NOT in the silhouette -- logos vary between releases, the CJK run is what discriminates). The detection silhouette `assets/qwen_alpha.png` is font-rendered synthetic (`scripts/render_vendor_silhouettes.py`), never cut from an upload. Feeds `identify` as the medium-confidence `visible_qwen` signal via the registry.
+
+**Why this registration took two attempts:** the 2026-07-18 attempt died at n=1 positives; the unlock was the TC260 label's `ContentProducer` field, whose USCC names the signing entity and partitions carriers into per-vendor cohorts from metadata alone (`scripts/vendor_cohort_harvest.py` -- 117 labelled Qwen frames, owing nothing to any pixel detector). Every constant was then MEASURED on that cohort against 286 hand-labelled clean frames (`scripts/vendor_mark_calibrate.py`), not inherited from Doubao:
+
+* **Basis `short`** (frac_short CV 0.189 vs width 0.273).
+* **Per-mark 2-rung `ladder=(0.78, 1.27)`** against `alpha_width_frac` 0.160: the mark sits in TWO size modes (~0.124 and ~0.203 of the short side, ratio 1.64, both clusters tight) -- wider than the shared 3-rung ladder's 1.5625 span, so the best single fraction covers only 74.5% and the small mode lands in the comb's collapse zone (real marks at the exact rung score ~0.94 vs ~0.67 on the shared ladder). A 4-rung variant scored strictly worse (its big-mode rung sits 4.6% off the mode). The shared default is untouched for every other mark.
+* **Fitted locate box** (`width_frac` 0.231, `height_frac` 0.074, margins ~0.021): the real mark sits ~0.025 of the short side off the right edge, and Doubao's box (0.004 margin) clipped the first glyph -- an exact-size template collapsed 0.73 -> 0.26 on a real frame.
+* **`alpha_height_frac` 0.0416 from the aspect fit** (p50 aspect 0.260 at the winning width) -- not the silhouette's own aspect (0.2219) and not Doubao's ratio.
+* **Gate 0.45**: clean p99 0.301 / max 0.316 (286 frames), and every cohort frame >= 0.45 carries a visible mark (83 of ~96 eyeballed visible marks fire = 86% recall of visible marks; the misses are white-on-near-white contrast losses). 0.45 was picked over 0.32 for margin against unseen clean content at zero measured recall cost.
+* **STRICT ONLY (`provenance_ncc_factor` 1.0):** the score band just below the gate is dominated by non-Qwen banners on same-cohort frames (a 夸克 anti-forgery strip at 0.274, a 造点 mark at 0.253), so a provenance-relaxed arm would be mostly false fills. There is no provenance mapping for qwen and no relaxed arm.
+* **No rival margin:** 0 cross-fires on 400 Doubao-marked / 298 Jimeng-marked / 286 clean frames at the gate, while a 0.10 margin would have cost ~10% of genuine Qwen detections. A confident Qwen detection suppresses the Jimeng pill exactly like Doubao's does (`_keep_pill` -- a Qwen frame is TC260 too but is not Jimeng-basic).
+
+**Parity:** detect -> cv2 fill -> re-detect is clean on **83/83** real cohort marks, no empty masks (the `tophat` faint-mask fallback rides the same `_tophat_best`); the e2e suite drives one live cohort positive through the real CLI (`scripts/real_examples_e2e.py`, qwen bucket = symlinks under the gitignored `_visible_datasets/`). Regression: `tests/test_qwen_engine.py` (pins the ladder, the strict-only factor, the box anchor, and both size modes at a score floor that discriminates the shared-ladder and Doubao-margin mutations).
+
+**The clean-arm contamination trap (load-bearing for any future calibration):** the 2026-07-18 `present: []` labels are in the vocabulary of the REGISTERED marks only, so 146 of the 432 "clean" frames sit in a TC260 cohort -- including Qwen-cohort frames visibly carrying 千问AI生成. They made up the clean arm's entire top tail (clean p99 0.37 -> 0.69). `vendor_mark_calibrate.load_sets` now excludes every frame in ANY TC260 cohort from the clean arm; a gate read off the unguarded arm is meaningless.
+
+## `region_eraser.py`
+
+`region_eraser.py` — universal region eraser (`erase` CLI) AND the shared fill backend behind `watermark_registry.fill` for the visible localize -> fill removal. `erase(image, boxes=|mask=, backend=)` accepts grayscale (2D) and RGBA (4-channel) inputs on **all** backends (each splits off any alpha plane and re-attaches it unchanged, and promotes grayscale to BGR): `boxes_to_mask` → one of three backends.
+- `cv2` (default, no deps): `cv2.inpaint`.
+- `migan` (extra `migan`, `andraniksargsyan/migan` ONNX, MIT, ~28 MB): `erase_migan`. Like `erase_lama`, it crops a padded region around the mask (`pad = max(256, 2*bbox)`), feeds only that crop to the ONNX model, and pastes only masked pixels back — but since MI-GAN accepts arbitrary dims (unlike LaMa's fixed 512² square) the crop is fed at NATIVE resolution (no resize). This **bounds the ONNX working set by the mark size, not the image**: feeding the whole frame made peak RAM scale with the upload (~0.6 GB at 4 MP up to ~2.4 GB at 25 MP, measured 2026-07); cropping holds it roughly constant (~0.6-0.9 GB), so a memory-tight host (a 1-2 GB web worker) can run MI-GAN on a 25 MP upload. The crop does not degrade the fill — a small mark only needs local context, and on real marks the cropped fill is on par with / sometimes cleaner than the full-frame fill (a tighter view gives the GAN less room to hallucinate large background structure; verified by eye on real Gemini/Doubao marks + a ground-truth reconstruction sweep). **Mask polarity is INVERTED** vs this package's 255-erase convention — the shipped ONNX wants 0=hole / 255=known, so `erase_migan` feeds `(crop_mask<=127)*255`; feeding 255=hole regenerates the whole frame into stripes (corpus-validated 2026-07, cost hours to find). ~0.19 s. This is the **preferred default fill** for the visible localize -> fill path.
+- `lama` (extra `lama`, `Carve/LaMa-ONNX` Apache-2.0, ~200 MB): `erase_lama` crops a padded region around the mask, runs at LaMa's fixed 512² input, pastes only masked pixels back. Best quality but ~4.7 GB peak — explicit opt-in only, NOT auto-selected.
+Lazy `_get_{lama,migan}_session` singletons; `{lama,migan}_available()` guard the optional imports (both == onnxruntime present). Note both extras install the same onnxruntime, so the two `*_available()` checks are identical — the fill's `auto` backend therefore resolves to MI-GAN whenever onnxruntime is present, else cv2, and big-LaMa is reachable only by an explicit `lama` backend (`--backend lama` on `erase`, or the shared fill's `backend="lama"`).
+
+**LaMa-ONNX costs ~3.5-4 GB peak RAM and ~5-6 s/call on CPU** (FFC working set, not arena — `enable_cpu_mem_arena=False` does not help), so it does NOT fit a minimal droplet; the cv2 backend (tens of MB, ~30 ms) does. LaMa quality at low RAM = serverless/GPU, mirroring how raiw.cc offloads SDXL to fal.
+
+## `invisible_watermark.py`
+
+`invisible_watermark.py` — `detect_invisible_watermark(path)` decodes the OPEN DWT-DCT watermarks (public decoder, no key) embedded by Stable Diffusion / SDXL / FLUX via the `imwatermark` library. Known fixed patterns (verified against upstream source) live in `_BITS_48` (SDXL 48-bit, FLUX.2 48-bit) and `_SD1_STRING` ("StableDiffusionV1", SD 1.x/2.x). Optional dep (extra `detect`); returns None when absent. The `detect` extra pulls **torch** transitively (invisible-watermark declares torch a hard dep, and `WatermarkDecoder` eagerly imports `rivaGan` -> `torch` at import time), so detection needs torch present even though dwtDct runs CPU-only on cv2/numpy/pywavelets — no GPU and no separate `gpu` extra required.
+
+**Unlike SynthID this is locally detectable**, but the watermark is fragile (does not survive JPEG re-encode/resize — verified gone after JPEG q90), so it confirms origin only on pristine files. Add new known patterns here. The file carries a top-of-module pyright pragma because imwatermark/cv2 ship no type stubs.
+
+## `trustmark_detector.py`
+
+`trustmark_detector.py` — `detect_trustmark(path)` decodes the OPEN, keyless **Adobe TrustMark** watermark (the soft binding behind Adobe Durable Content Credentials, `alg` `com.adobe.trustmark.P`) via the optional `trustmark` package (extra `trustmark`; pulls torch, downloads model weights on first use). Mirrors `invisible_watermark.py` (lazy singleton guarded by a double-checked `threading.Lock` so concurrent callers do not double-download the weights, top-of-module pyright pragma, returns None when absent). It detects *provenance*, not AI origin as such (TrustMark also marks human-authored content), so `identify` lists it as a watermark without setting `is_ai_generated`. Other soft-binding vendors (Digimarc/Imatag/Steg.AI/...) have no public decoder — they are only *named* via the `C2PA_SOFT_BINDINGS` scan, not decoded.
+
+**False-positive gate (added 2026-05-29):** TrustMark's `wm_present` is a BCH error-correction validity flag that spuriously validates on a content-correlated fraction of un-watermarked images — AI-generated textures trip it far more than camera photos (verified 2026-05-29 on real files: it fires on Gemini/OpenAI/Doubao output that *cannot* carry Adobe's watermark, with a random-bytes decoded secret, while signal-free camera photos did not trip it). A genuine TrustMark is a *durable* soft binding engineered to survive re-encoding, so `detect_trustmark` re-decodes after a mild JPEG round-trip (`_survives_reencode`, `_REENCODE_QUALITY` 95) and requires the same schema both times; every observed false positive collapsed (none survived even q95), so the gate is the durability property the watermark guarantees. The second decode runs only on the rare initial hit, so the cost is negligible. Do NOT remove the gate to "catch more" — a lone TrustMark hit without it is almost always content noise.
+
+## `noai/watermark_remover.py`
+
+`noai/watermark_remover.py` — the `WatermarkRemover` class has three diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id`). `sdxl`/`controlnet` share the SDXL base (`DEFAULT_MODEL_ID`); `qwen` is its own base (`QWEN_MODEL_ID`).
+
+**`sdxl`** (renamed from `default` 2026-06-09; `default` kept as a back-compat alias via `normalize_profile`) runs plain SDXL img2img (`_run_img2img`); it is the lighter opt-down alternative (no ControlNet weights).
+
+**`qwen`** (`_run_qwen`, `_load_qwen_pipeline`) runs `QwenImageImg2ImgPipeline` on `Qwen/Qwen-Image` (20B MMDiT, Apache-2.0 code AND weights). The scrub still comes from the img2img `strength`; Qwen's value is **text preservation** (incl. CJK and small text). **Metric-measured nuance (2026-06-19, `scripts/fidelity_metrics.py`, do NOT trust the eyeball here — it misled). Compare ONLY at each pipeline's oracle-confirmed scrub floor (outputs where SynthID is removed in BOTH — an equal-strength compare is invalid where it leaves one un-scrubbed; Qwen at 0.15 does not clear Gemini): Qwen wins TEXT (lower OCR CER across EN/RU/ZH, perfect Chinese) but controlnet wins FACES (higher Laplacian-variance retention and lower LPIPS — Qwen smooths faces MORE; ArcFace identity favors controlnet 0.546 vs 0.331 at the Gemini floors).** So Qwen is the better text-preserving remover, NOT a universal fidelity win — controlnet's canny edge map holds face skin detail better. Specifics: bf16 on CUDA (fp16 risks overflow on the 20B MMDiT — see the dtype branch in `__init__`); loads `QWEN_MODEL_ID` unless `--model` is overridden; the call shape lives in the pure module helper `_build_qwen_kwargs` (unit-tested without torch in `tests/test_platform.py::TestQwenKwargs`), which uses Qwen's `true_cfg_scale` (NOT SDXL's `guidance_scale` — the CLI `--guidance-scale` maps onto it; ~4.0 is typical, the SDXL default 7.5 is high for Qwen) and an explicit `negative_prompt` (`_QWEN_PROMPT`/`_QWEN_NEGATIVE`). It is CUDA/cloud-class (the 20B does not fit MPS), so `_run_qwen` has NO MPS->CPU fallback — an error propagates. `_load_qwen_pipeline` raises a clear ImportError if the installed diffusers lacks `QwenImageImg2ImgPipeline`. **CERTIFIED oracle floors (Modal A100-80GB, 2026-06-20): OpenAI 0.10 (seed-robust — clean on seeds 0-4), Gemini 0.25 (seed 0 verified on 2 images; the Gemini oracle rate-limits volume seed-repeat, so PIN a seed in prod). The Gemini floor (0.25) is HIGHER than the certified controlnet Gemini floor (0.15); `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`), so `--pipeline qwen` gets the 0.25 Gemini floor automatically -- the old manual `--strength 0.25` workaround is retired. `_build_qwen_kwargs` passes an explicit `height`/`width` from the input (floored to /16 via the pure `_qwen_target_size`); WITHOUT it the img2img pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (the abba 2816x1536 case came back 1024x1024, distorting the scene and garbling text — fixed 2026-06-20, tested in `TestQwenKwargs`).** Fidelity vs controlnet was measured at the certified floors (`scripts/fidelity_metrics.py`), NOT eyeballed. **`qwen` is a MANUAL opt-in only — there is NO auto-router (one was prototyped and DROPPED, see below).** It wins ONE niche: clean body text on a plain background, NO faces (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES and **display/decorative text in a scene** (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes; qwen re-renders and garbles them). **`--pipeline auto` + a faces+text mixed dual-pass were built and DROPPED (2026-06-20):** on the canonical faces+text case (abba) controlnet wins EVERY metric incl. text, so grafting qwen text would only hurt; and "text→qwen" is undecidable cheaply (it is body-vs-display text that matters). The router/detector/mixed modules were removed; the geometry fix + the Qwen strength ladder were kept (they make the manual `--pipeline qwen` correct). **Do NOT retry "add a Qwen ControlNet to close the face gap" — it was built, measured, and CLOSED 2026-06-20:** a DiffSynth blockwise-canny Qwen ControlNet did not restore face skin texture (lapvar flat 0.40, canny carries edges not skin grain) and no permissively-licensed Qwen tile/detail/skin ControlNet exists anywhere (all conditioning is geometry). Faces stay on controlnet; the next improvement lead is Z-Image-Turbo (Apache-2.0, unmeasured floor). Full record + the deep-research sweep in `docs/qwen-improvement-research.md`.
+
+**`controlnet`** (**the DEFAULT pipeline since 2026-06-09** for `invisible`/`all`/`batch` and both engine ctors; `_run_controlnet`, `_load_controlnet_pipeline`) runs `StableDiffusionXLControlNetImg2ImgPipeline` with the SDXL-native canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` (`watermark_profiles.CONTROLNET_CANNY_MODEL`): the control image is `cv2.Canny(gray, 100, 200)` stacked to 3 channels (`_CANNY_LOW`/`_CANNY_HIGH`, prompt `_CONTROLNET_PROMPT` / `_CONTROLNET_NEGATIVE`).
+
+**Removal comes from the img2img regeneration (`strength`); the ControlNet only PRESERVES text and face STRUCTURE via the edge map.**
+
+No original pixels are copied or frozen, BUT **validation 2026-06-04 disproved the old "so SynthID does not survive" claim: SynthID CAN survive controlnet on photoreal/high-detail content.**
+
+At the shared low removal strength the canny edge-conditioning keeps the regeneration so close to the original that the pixel perturbation that destroys SynthID does not happen (oracle-confirmed: an OpenAI bracelet photo + a 9-face grid read **SynthID-detected** after controlnet at strength 0.10/0.15, but **SynthID-not-detected** after the `default` pipeline at the SAME strength + resolution -- only the pipeline differed).
+
+**But the reverse also holds: a flat-graphic logo/poster SURVIVED `default` while clearing controlnet** -- removal at the low strength is content×pipeline dependent and neither pipeline is universally safe; the real lever is a higher strength. See the controlnet Known-limitations bullet for the full table + root cause. Canny holds face STRUCTURE but NOT identity (the regenerated face drifts in likeness -- canny carries edges, not identity). The drifted cleaned face is the LEAST-AI state we can reach without re-introducing SynthID; the library does NOT ship a face-restore extra. Every restore approach we evaluated (GFPGAN-on-cleaned, PhotoMaker-V2 txt2img, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps, 2026-06-04 - 2026-06-08 Modal cert sweeps) regenerated the face from an ArcFace embedding via SDXL diffusion -- which makes the output face look MORE AI-generated, not less. Empirical conclusion in `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up". For production face preservation, ship the cleaned image as-is. `controlnet_conditioning_scale` (ctor arg, default 1.0) is the structure-preservation knob. Same dtype rule as `default` (fp32 on cpu/mps, fp16 only on cuda/xpu; the fp16-fixed SDXL VAE `_SDXL_FP16_VAE_ID` is swapped in on fp16 GPUs -- issue #29) and the same MPS->CPU fallback (reload on cpu/fp32, drop a non-cpu generator, retry once).
+
+**Tiled diffusion (`tile`/`tile_size`/`tile_overlap` ctor-path args, CLI `--tile`, issue #10):** for large inputs that OOM at native resolution, `remove_watermark` can process the diffusion pass in overlapping sliding-window tiles instead of one forward pass — the lossless alternative to a `--max-resolution` downscale. The single-image generation closure was refactored into `_generate_one(img)` (dispatches controlnet/img2img, generator shared so the seed advances deterministically across tiles), and `_generate()` routes it through `noai.tiling.run_tiled` when `tile` is set AND `max(init_image.size) > tile_size` (a sub-tile image runs one pass unchanged). The ControlNet canny edge map is rebuilt per tile inside `_generate_one`, so structure preservation is tile-local. See `noai/tiling.py` below and the tiled-diffusion subsection in `docs/known-limitations.md` for the geometry, the partition-of-unity blend, and the quality caveat.
+
+## `noai/tiling.py`
+
+Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL only). `plan_tiles(w, h, tile_size, overlap)` returns a row-major grid of uniform-size `Tile` boxes — every tile is exactly `tile_size` (the SDXL training size), with the last tile on each axis pulled back flush to the far edge (`_axis_positions` clamps a pathological `overlap >= tile` to `tile - 1` so the step stays >= 1). `feather_weights(w, h, overlap)` is a separable linear taper (1 in the interior, ramping toward each edge) floored at `_WEIGHT_EPS` so it is **strictly positive everywhere** — that makes the normalized `accum / weight_sum` blend a partition of unity, so identical/unchanged tiles reconstruct the input exactly (the seam-free guarantee). `run_tiled(generate_tile, image, tile_size, overlap, set_progress)` is the orchestration loop: crop each planned tile, call `generate_tile` (one diffusion pass on a single PIL tile — injected, so this stays decoupled from the pipeline), resize a latent-grid-rounded result back to the exact tile size, and feather-accumulate. All three are unit-tested without the model (`tests/test_tiling.py`: axis math, grid coverage, taper shape/symmetry/positivity, identity reconstruction, per-tile call count, and the resize-back path). New blend tuning belongs in these pure helpers, not inlined into the runner.
+
+`feather_region_composite(base, regenerated, box, *, feather)` is the pure region-targeted compositor for **AI-enhanced composites** (roadmap P1#8; `identify` `ai_source_kind == "enhanced"`, digitalSourceType `compositeWithTrainedAlgorithmicMedia`). It blends `regenerated` over `base` inside `box = (x, y, w, h)` with a separable linear taper of `feather` px at the box edges (the taper anchors to ~0 at the boundary, so unlike `feather_weights` it is NOT floored — the result equals `base` EXACTLY outside the box), preserving dtype and supporting HxW or HxWxC. It backs `WatermarkRemover.remove_watermark(region=..., region_feather=...)`: the remover regenerates the frame (or tiles), then composites only the AI box back over the original input, so the real photo outside the box stays pixel-exact and only the AI region is scrubbed. The box is caller-supplied (a C2PA composite manifest carries no reliable machine-readable region); the no-model lossless region path remains `region_eraser.erase`. Unit-tested in `tests/test_tiling.py::TestFeatherRegionComposite` (outside-box exactness, interior == regenerated, hard-paste at feather 0, monotonic seam ramp, dtype/grayscale/clamp/empty-box/shape-mismatch).
+
+## `auto_config.py` (REMOVED 2026-06-09)
+
+**`auto_config.py` + the content-detection layer were REMOVED 2026-06-09.**
+
+History: `auto_config.plan()` was a content-adaptive planner that detected faces/text/edges (bundled OpenCV YuNet + PP-OCRv3 DBNet models) to route the pipeline and toggle the adaptive polish. Once `controlnet` became the default-and-only auto pipeline (it no longer downgrades a structure-less image to `sdxl`) and the adaptive polish was confirmed to **self-gate by detail level** (`humanizer.adaptive_polish` no-ops when the cleaned image already meets the input's Laplacian variance, so it does real work only on over-smoothed photo/face texture and ~nothing on text/flat), the detection no longer changed any behavior — it only annotated a `reason` string. So the whole layer was deleted: `auto_config.py`, `tests/test_auto_config.py`, and the two detection assets (`assets/face_detection_yunet_2023mar.onnx`, `assets/text_detection_ppocrv3_2023may.onnx`, ~2.6 MB).
+
+**`--auto` is now a DEPRECATED no-op** (`cli._resolve_auto_polish`): controlnet is already the default pipeline AND the adaptive polish is ON by default, so `--auto` has nothing left to do — it only prints a deprecation warning and passes `adaptive_polish` through unchanged (an explicit `--no-adaptive-polish` still wins). (Originally it re-enabled the polish; once the polish default flipped to ON the same day, the parameter-source branch became dead and was dropped.) The **adaptive polish itself lives on** in `humanizer.adaptive_polish` (CLI `--adaptive-polish/--no-adaptive-polish`, **ON by default since 2026-06-09** — it self-gates to a no-op where there is no detail deficit, so default-on is safe; uses the full-res original as the detail reference) — see the `humanizer` test note. `batch` resolves the polish once before the loop (one warning) and caches the invisible engine per pipeline (`ctx.obj["_inv_engines"]`).
+
+## Content `--pipeline auto` router + faces+text mixed dual-pass — PROTOTYPED and DROPPED (2026-06-20)
+
+A `--pipeline auto` content router (`pipeline_router.py` + `content_detect.py`: Haar faces + MSER text → route text→qwen / faces→controlnet / both→mixed) and a faces+text **mixed dual-pass** (`mixed_pipeline.py`: scrub the whole frame on BOTH pipelines, then graft the qwen text regions onto the controlnet base via `tiling.feather_region_composite`) were built, run on Modal (the abba poster: faces + display text), measured, and **removed**. Why it failed:
+- On the canonical faces+text image **controlnet wins EVERY metric, including text** (CER 0.114 vs qwen 0.379; ID 0.64 vs 0.36; lapvar 0.71 vs 0.59) — canny holds the existing letter shapes, qwen re-renders display/decorative text and garbles it. So grafting qwen text onto the controlnet base only HURTS.
+- qwen beats controlnet on text ONLY for clean body text on a plain background with no faces (openai_1/2) — a niche where there are no faces to route around anyway, so `--pipeline qwen` alone covers it. The faces+clean-body-text intersection is near-empty.
+- "text→qwen" is not cheaply decidable: it is body-vs-display text that matters, which face/text detectors can't tell apart. MSER also over-fired (47% of the busy poster, incl. faces).
+
+KEPT from that work (independently valid for the manual `--pipeline qwen`): the qwen **geometry fix** (`_qwen_target_size` + `_build_qwen_kwargs` height/width — qwen squished non-square inputs to 1024² without it) and the **pipeline-aware `resolve_strength`** Qwen ladder (Gemini 0.25). Also kept: the `fidelity_metrics.py` one-to-one face matcher. The throwaway Modal eval scripts were removed after the run (findings recorded here and in `docs/qwen-improvement-research.md`).
+
+## `upscaler.py`
+
+`upscaler.py` — optional Real-ESRGAN pre-diffusion super-resolution for small inputs (spandrel boundary, top-of-file pyright pragma). `is_available()` gates on spandrel+torch (via `importlib.util.find_spec`); `upscale(bgr, device=None)` loads a lazily-built spandrel `ImageModelDescriptor` singleton (double-checked lock) and upscales by the model's native factor (x2), with a non-CPU→CPU device fallback mirroring the diffusion engine's MPS→CPU retry. Weights (`RealESRGAN_x2plus.pth`, BSD-3-Clause) download on first use to the `torch.hub` checkpoints cache; never bundled. Used only when UPscaling to the `min_resolution` floor (a `max_resolution` downscale always uses Lanczos). The wiring is `InvisibleEngine._esrgan_upscale(pil, target)` — Real-ESRGAN at native factor, then a Lanczos resize to the exact target, falling back to a plain Lanczos resize if the extra is absent or the model errors (so an optional upscaler can never break removal). The default `--upscaler` is `lanczos` (cv2, no deps).
+
+**ESRGAN is a generic photo/texture GAN with no face/glyph prior**, so it best fits photo/texture content and can degrade faces (glassy/asymmetric eyes -- the diffusion pass regenerates faces so the full-pipeline final recovers) and thin/small text (the GAN invents wrong strokes, and low-strength diffusion will not fix it). Verified 2026-06-04: isolated upscale lap-var ~5x Lanczos on faces+textures but glassy eyes; end-to-end `invisible` final lap-var 1634 vs Lanczos 663 with natural faces (diffusion cleaned the artifact). Kept a **manual opt-in knob** (the auto plan never selects it) with `lanczos` the default; not content-gated by design (use Lanczos for text-heavy inputs). spandrel is MIT and pulls no basicsr. Unit-tested without the model: `tests/test_upscaler.py` (availability guard + the not-installed RuntimeError) and `tests/test_invisible_engine.py::TestEsrganUpscale` (the three `_esrgan_upscale` branches via a monkeypatched `upscaler`).
+
+## `image_io.py`
+
+`image_io.py` — Unicode-safe cv2 IO (issue #17). `imread(path, flags=None)` / `imwrite(path, img)` wrap `np.fromfile`+`cv2.imdecode` / `cv2.imencode`+`tofile` so non-ASCII paths work on Windows -- bare `cv2.imread`/`cv2.imwrite` use the platform ANSI code-page API there and fail (empty decode + `can't open/read file`) on Chinese/Cyrillic/accented filenames. `imread` keeps `cv2.imread` semantics (defaults to `IMREAD_COLOR`, returns `None` on missing/empty/undecodable).
+
+**Every cv2 file read/write in the package routes through here; do not call `cv2.imread`/`cv2.imwrite` directly.**
+
+`imwrite` returns `False` on an unwritable path (`OSError` caught) instead of raising, matching `cv2.imwrite` semantics. macOS/Linux already accept UTF-8 paths, so it is behavior-neutral there (the bug only reproduces on Windows).
+
+**`to_bgr(image)` (added 2026-06-09)** is the shared channel normalizer: promotes 2D grayscale / (h,w,1) / 4-channel BGRA to 3-channel BGR (a 3-channel input is returned unchanged, no copy). Use it instead of inlining the `cvtColor(GRAY2BGR/BGRA2BGR)` branch — the gemini engine and the `TextMarkEngine` base both route through it so a grayscale/BGRA input (a real Gemini-app export is opaque RGBA) does not crash the `axis=2` channel reductions. cv2/numpy are imported lazily inside the functions, so the module is cheap to import in a bare env.
+
+## CLI commands (`cli.py`)
+
+Full per-command behavior for the skip/exit branches summarized in `CLAUDE.md`'s "How to run". The CLI distinguishes three exit codes: success (0), hard error (1), and a "nothing to do" code (2, `EXIT_NO_VISIBLE_MARK` / `EXIT_NO_INVISIBLE_SIGNAL`) so a wrapping service (raiw.cc) can surface guidance instead of treating an unchanged image as done (the production "it didn't work" / score-0 trap).
+
+**Every single-image command's `source` argument declares `dir_okay=False`** (2026-07-20). `click.Path(exists=True)` accepts a directory unless told otherwise, so `identify <dir>` sailed past argument parsing and raised `IsADirectoryError` out of `metadata.scan_head`'s `open()` — a traceback, not a usage error. Refusing it at the argument layer is the right place: every command gets it, and none needs its own check. (`batch`'s `directory` argument was already correct with `file_okay=False`.) Found by the Tier E adversarial sweep; regression: `tests/test_cli_robustness.py::TestDirectoryInputIsRejected`.
+
+### `all`
+
+Full pipeline (visible + invisible + metadata). Same diffusion knobs as `invisible`, plus the visible-pass `--backend auto|cv2|migan|lama` (default `auto`) that picks the fill for the localize -> fill visible removal. **When the `[gpu]` extra is absent, step 2 (invisible/SynthID) is skipped** — `all` still writes an output (visible mark + metadata stripped) but prints a prominent end-of-run banner ("the invisible (SynthID) watermark was NOT removed") AND exits **non-zero** (1), so a skipped SynthID pass is not mistaken for a clean result (the recurring #14/#47 trap, where the old quiet inline warning was missed). `invisible` already hard-errors without the extra; only `all` continued, hence the loud end-banner. Regression-guarded by `tests/test_cli.py::TestAllCommand::test_all_loud_warning_and_nonzero_exit_when_gpu_missing`. **No-signal skip (P0#5):** step 2 also runs the same `has_invisible_target` gate (see `invisible` below) — when no invisible watermark is detectable and `--force` is not set, step 2 is skipped and the pixels are left intact, but unlike the GPU-missing skip this is a **SUCCESS (exit 0)**: the visible pass + metadata strip still ran and a file is written (the message says so without claiming the image is clean). Distinct exit semantics by design: GPU-missing = couldn't do the work (non-zero); no-signal = nothing to do (zero). Regression-guarded by `test_all_skips_invisible_on_no_signal_but_succeeds`. **Test trap:** any `all` test that exercises the full pipeline MUST `patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True)` — CI installs core+dev only (no `[gpu]`), so an unpatched `all` test takes the skip branch and now hits the non-zero exit. This passed locally (gpu present → `is_available()` True) but red-failed every matrix cell on the v0.11.0 commit (`test_all_basic`/`test_all_visible_step_uses_registry` asserted exit 0); both now patch `is_available` True.
+
+### `invisible`
+
+Diffusion SynthID removal. The `--tile/--no-tile` knob is the *lossless* alternative to a `--max-resolution` downscale for large inputs that OOM on MPS/GPU: it engages only when the long side exceeds `--tile-size` (default 1024); tiles are feather-blended over `--tile-overlap` px (default 128); pair with `--max-resolution 0`. `--adaptive-polish` is a detail-targeted polish that self-gates to a no-op where there is no deficit. `--auto` is deprecated and now a no-op that only warns (the polish it used to enable is ON by default). **No-signal skip (P0#5, roadmap):** before the diffusion runs, the command checks `identify.has_invisible_target(source)` (the `ProvenanceReport.ai_from_metadata` union: C2PA AI-issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, open DWT-DCT / TrustMark — visible marks do NOT count, they are a separate pass). When nothing is locally detectable it does NOT regenerate (that would only degrade a clean image — the dominant paid score-0 cause on no-watermark uploads): it writes NO output, prints guidance that does NOT claim the image is clean (a pixel SynthID is undetectable once its metadata proxy is gone), and exits **`EXIT_NO_INVISIBLE_SIGNAL` (2)** — same value/role as the visible `EXIT_NO_VISIBLE_MARK`. `--force/--no-force` (**default skip = ON**) runs the scrub regardless. The check fails SAFE (a detector exception → run, since leaving a watermark on a paid removal is worse than over-regenerating). Helpers `cli._no_invisible_signal_exit` + `identify.has_invisible_target`; regression-guarded by `tests/test_cli.py::TestInvisibleCommand::{test_invisible_no_signal_skips_and_exits_two,test_invisible_force_runs_scrub_on_no_signal,test_invisible_runs_without_force_when_signal_present}` and `tests/test_identify.py::TestHasInvisibleTargetFailSafe`. **Test trap:** any `invisible`/`all`/`batch` test that exercises the diffusion path on a signal-LESS fixture (e.g. the synthetic `sample_png`) MUST pass `--force`, or the new gate skips step 2 (so `mock_engine.remove_watermark` is never called / `invisible` exits 2).
+
+### `visible`
+
+Known-visible-mark removal by **localize -> fill**: each detected mark is localized to a binary full-frame footprint mask, then one shared, swappable fill inpaints that mask. `--backend auto|cv2|migan|lama` (default `auto`) picks the fill: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available (LaMa is auto-preferred when a learned backend is present; a memory-tight deploy pins migan). `--sensitivity auto|strict|assume-ai` (default `auto`) controls how hard a borderline mark is trusted (see the registry section: the visual detectors are metadata-independent; `auto` relaxes a mark only on same-product evidence, `assume-ai` relaxes every mark on the caller's AI assertion, subject to the assumed-trust confidence floor where the vendor is unconfirmed — the only path to higher recall on a metadata-stripped screenshot). `--backend` and `--sensitivity` are shared across `visible`/`all`/`batch`. Detection keys on each mark's own shape, and under `auto` the trust gate is relaxed when local metadata confirms the vendor (a Google/Gemini C2PA issuer relaxes gemini, a China-AIGC label relaxes doubao/jimeng, `samsung_genai` relaxes samsung), so a moved or re-rendered mark is still caught. `--mark auto` (default) removes EVERY detected mark in one pass (`registry.remove_auto_marks`, not the single strongest -- a Jimeng-basic image carries both the top-left pill and the bottom-right wordmark) from: the Gemini sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, the Qwen "千问AI生成" text strip, the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-LEFT, Italian-locale detection), and the capture-less Jimeng "AI生成" pill (top-left, `pill_engine`). The pill's weak edge-NCC detector is gated in `remove_auto_marks` via `_keep_pill` (32k real-upload corpus validation 2026-07): never on Doubao or Qwen, and two confirmation arms since metadata confirms the platform, not pill presence. (1) The bottom-right wordmark fired — ~94% precise and survives metadata-STRIPPED uploads (screenshots / re-saves) — removes the pill unrestricted. (2) TC260 metadata confirms Jimeng (`"jimeng" in provenance`, from `cli._visible_provenance`) OR the caller asserts AI (`sensitivity == "assume_ai"`), no wordmark — **re-measured 2026-07-18 on 149 blind-labelled pill fires: 21% precise raw (CI 16-29%), 29% (CI 20-40%) among the flat footprints the guard actually PASSES, 14% among those it blocks** — its false fires are textured ceilings/walls that the fill visibly SMEARS — removes the pill ONLY when the top-left footprint is flat enough for an invisible fill (`pill_engine.footprint_is_flat`, median-Sobel ≤ `_FLAT_TEXTURE_MAX`; the flatness guard holds even under `assume_ai`). No confirmation → never removed. `--mark gemini|doubao|jimeng|qwen|samsung|jimeng_pill` forces one (choices come from the registry). Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. For arbitrary logos/objects use `erase`. **When `--mark auto` finds no known mark (the common case — ~74% of real uploads carry no registered visible mark), the command does NOT silently re-serve the input as a finished result.** It runs a cheap metadata-only `identify`, prints actionable guidance (if the image carries an invisible/metadata mark, e.g. an OpenAI/Gemini C2PA image, it points to `all`; otherwise it does NOT imply the image is clean -- it warns that an invisible pixel watermark like SynthID cannot be detected once the metadata proxy is gone and routes to both `all` and `erase --region`), writes NO output file, and exits **`EXIT_NO_VISIBLE_MARK` (2)** — distinct from success (0) and a hard error (1) so a wrapping service (raiw.cc) can surface the message instead of treating the unchanged image as done (the production "it didn't work" / score-0 trap). Same handling for an explicit `--mark <name>` that is not detected. Helper `cli._no_visible_mark_exit`; regression-guarded by `tests/test_cli.py::TestVisibleCommand::test_visible_auto_no_mark_exits_two_with_eraser_hint` and `test_visible_auto_no_mark_routes_to_all_when_metadata`. `--no-detect` still forces the gemini fallback and proceeds (exit 0).
+
+### `batch`
+
+Process every supported image in a directory (output defaults to `<directory>_clean/`, set with `-o`). `--mode visible|invisible|metadata|all` (default `visible`); the invisible/all path reuses the **full `invisible` knob set** (`--strength`/`--steps`/`--guidance-scale`/`--pipeline`/`--controlnet-scale`/`--model`/`--device`/`--max-resolution`/`--min-resolution`/`--upscaler`/`--seed`/`--hf-token`/`--humanize`/`--unsharp`/`--adaptive-polish`/`--tile`/`--tile-size`/`--tile-overlap`/`--force`), plus `--backend` for the visible localize -> fill pass. `--adaptive-polish` is ON by default; `--auto` is deprecated and a no-op that only warns. **No-signal skip (P0#5):** in invisible/all mode each image runs the same `has_invisible_target` gate — a signal-less image is skipped (no diffusion); in `invisible` mode the input is copied through to the output dir so it stays complete, in `all` mode the visible-removed result is kept and metadata is still stripped. `--force` scrubs every image regardless. One engine cached per pipeline; the polish is resolved once before the loop. **Exit code (`batch` used to always exit 0, hiding failures):** `cmd_batch` raises `SystemExit(1)` when any image errored, OR when a `--mode invisible`/`all` image carried an invisible signal but the GPU extra was absent so its SynthID scrub was skipped — mirroring single `all`, it emits a loud "the invisible watermark was NOT removed on N image(s)" warning and (invisible mode) copies the input through so the output dir stays complete, rather than silently dropping the signal-bearing files that most needed processing. `_process_batch_image` returns that skipped-scrub flag; the loop tallies it. Regression-guarded by `tests/test_cli.py::TestBatchCommand::{test_batch_errors_exit_nonzero, test_batch_invisible_gpu_missing_writes_output_and_exits_nonzero}`.

docs/qwen-improvement-research.md

@@ -0,0 +1,220 @@
+# Qwen-Image improvement research (2026-06-20)
+
+Cited research behind the decision **"ship the `qwen` pipeline as-is, or improve it
+first?"** Produced by the multi-source deep-research harness (5 search angles, 22
+sources fetched, 85 claims extracted, 25 verified by a 3-vote adversarial check, 20
+confirmed / 5 killed, 104 agent calls). Findings carry their confidence and vote.
+
+## Context
+
+The `qwen` pipeline runs base Qwen-Image (20B MMDiT, Apache-2.0) as a low-strength
+img2img scrub (removal comes from the denoising `strength`). Certified oracle scrub
+floors: OpenAI 0.10 (seed-robust), Gemini 0.25 (pinned seed). Measured against the
+SDXL + canny-ControlNet pipeline (`scripts/fidelity_metrics.py`): Qwen preserves
+**text** markedly better (incl. CJK and Cyrillic, lower OCR CER) but preserves
+**faces** worse, smoothing skin (Laplacian-variance retention 0.40 vs 0.62, face
+LPIPS 0.17 vs 0.09, ArcFace identity 0.38 vs 0.55 at the scrub floors). The goal of
+the research: keep Qwen's text advantage while fixing the face-smoothing, and judge
+production-readiness.
+
+## Verdict
+
+Base Qwen-Image is **shippable now as an opt-in text-content lane** (Apache-2.0 on
+code and weights, scrub lever confirmed), but it is not a universal upgrade (it loses
+faces). The strongest verified improvement path is to **add structure conditioning**
+(a Qwen-Image ControlNet) to the existing base pass, the direct analog of the SDXL +
+canny conditioning that wins on faces. Separately, **Z-Image / Z-Image-Turbo** (6B,
+Apache-2.0) is the best-verified lighter alternative to evaluate before committing to
+the 20B cost. None of the improvements has measured face-fidelity numbers at our
+scrub floors yet, so each must be validated with `scripts/fidelity_metrics.py` plus
+the oracle before shipping.
+
+## Follow-up: ControlNet experiment + deeper research (2026-06-20)
+
+The verdict's strongest lead -- adding a Qwen-Image ControlNet -- was **built, measured, and
+CLOSED**.
+
+**Experiment** (Modal A100-80GB; DiffSynth-Studio `QwenImagePipeline` + the Apache-2.0
+`DiffSynth-Studio/Qwen-Image-Blockwise-ControlNet-Canny` -- the only framework exposing
+Qwen-Image + canny ControlNet + img2img `denoising_strength` in ONE call; diffusers ships no
+`QwenImageControlNetImg2ImgPipeline`, its three Qwen ControlNet pipelines are txt2img only).
+Measured on `gemini_3` (18 faces) at the Gemini scrub floor 0.25 vs base-Qwen 0.25 with
+`scripts/fidelity_metrics.py`:
+- **The actual failure mode (face skin texture) was NOT restored:** Laplacian-variance
+  retention stayed flat (base 0.40 -> qwen+canny 0.40; per-face 13/16 within +-0.02 after a
+  one-to-one face match, sd 0.016 -- not an averaging artifact). The SDXL+canny target 0.62
+  was not approached.
+- Identity rose modestly and broadly (ArcFace 0.346 -> 0.415, 12/16 faces improved) but the
+  absolute stays ~0.42 ("a different person, slightly closer").
+- Mechanism (verified, not inferred): canny conditioning was applied fully (scale 1.0, full
+  denoise schedule); the canny edge map is clean facial geometry with BLANK skin (4.83% edge
+  density) -- canny carries edges, not skin grain. Root cause: Qwen's Gemini floor (0.25) is
+  higher than SDXL+canny's (0.15), forcing more denoising -> more smoothing; structure
+  conditioning cannot compensate for that.
+
+**Deeper research** (deep-research harness, 103 agents, 3-vote adversarial):
+- **[high, unanimous] No permissively-licensed Qwen-Image tile / detail / realism / skin
+  ControlNet exists anywhere** -- DiffSynth first-party is Canny/Depth/Inpaint only, InstantX
+  Union is canny/soft-edge/depth/pose, the official QwenLM repo ships none. Every Qwen
+  conditioning is GEOMETRY, the same class as the tested canny. **The "add a Qwen ControlNet to
+  fix faces" lead is closed for good.**
+- **[high, unanimous] Z-Image / Z-Image-Turbo (6B, Apache-2.0 on code AND weights, ~1/3 of
+  Qwen 20B)** ships a documented `ZImageImg2ImgPipeline` with standard strength denoising, so
+  it preserves the scrub mechanism. Its own SynthID scrub floor and face/text fidelity are
+  UNMEASURED -- this is the strongest concrete NEXT experiment.
+- **[medium] Lowering Qwen's scrub floor has no off-the-shelf SynthID answer:** the "partial
+  img2img ~0.3 breaks robust watermarks" literature tests open schemes
+  (StegaStamp/TrustMark/VINE), NEVER SynthID (proprietary decoder) -- analogy, not proof. No
+  minimal-strength SynthID attack under a named permissive license was found.
+- **REFUTED [0-3]:** "re-injecting high-frequency detail from a clean diffusion output would
+  not carry the watermark back." So non-regenerative detail transfer is NOT safe by
+  assumption -- the transferred high-frequency band must be gated against the SynthID oracle.
+
+**Net for the pipeline:** **faces stay on SDXL+controlnet**; there is no Qwen face-fix.
+The live frontier is Z-Image-Turbo (next experiment) and oracle-gated non-regenerative detail
+re-injection.
+
+**Follow-up (2026-06-20) — the content-routed lane / mixed dual-pass was tested and DROPPED.**
+A `--pipeline auto` router (Haar+MSER → text→qwen / faces→controlnet / both→mixed) and a
+faces+text mixed dual-pass (scrub the whole frame on both, graft qwen text regions onto the
+controlnet base) were built and run on Modal (the abba poster: faces + display text). On that
+canonical faces+text case **controlnet won EVERY metric, including text** (CER 0.114 vs qwen
+0.379; ID 0.64 vs 0.36) — canny holds existing letter shapes, qwen re-renders display text and
+garbles it, so grafting qwen text only hurts. Qwen beats controlnet on text ONLY for clean body
+text on a plain background with no faces (openai_1/2), a niche `--pipeline qwen` alone covers;
+the faces+clean-body-text intersection is near-empty, and "text→qwen" is undecidable cheaply
+(body-vs-display text is what matters). So the router + mixed modules were removed and **`qwen`
+is a manual `--pipeline qwen` opt-in only.** KEPT (independently valid): the qwen geometry fix
+(it squished non-square inputs to 1024²), the pipeline-aware `resolve_strength` Qwen ladder, and
+the `fidelity_metrics.py` one-to-one face matcher below.
+
+**Tooling fix surfaced by this run:** `scripts/fidelity_metrics.py` face matching was changed
+from per-face nearest-center to a collision-free one-to-one assignment
+(`assign_faces_one_to_one`, gated by face size), after the 18-face `gemini_3` exposed
+collisions (the regenerated variants detected 17 faces, so two originals mapped to the same
+variant face, corrupting the identity metric). lapvar/LPIPS were always anchored to the
+original bbox and stayed collision-immune. Regression-guarded by
+`tests/test_fidelity_matching.py`.
+
+## Findings
+
+1. **[high, 3-0] A permissively-licensed Qwen-Image ControlNet exists today and is
+   CUDA/diffusers-runnable.** InstantX Qwen-Image-ControlNet-Union supports
+   canny/soft-edge/depth/pose; DiffSynth-Studio maintains blockwise Canny/Depth/Inpaint
+   plus an In-Context-Control-Union; diffusers exposes `QwenImageControlNetPipeline`
+   and `QwenImageMultiControlNetModel` with `controlnet_conditioning_scale` (default
+   1.0) and `control_guidance_start`/`end`. This is the direct analog of the certified
+   SDXL+canny structure conditioning that wins on faces. Caveat: canny/depth preserve
+   geometric structure, not face identity per se, and none is a **tile**-ControlNet
+   (the variant most tied to fine-detail/skin retention in the SDXL world).
+   Sources: InstantX/Qwen-Image-ControlNet-Union, InstantX/Qwen-Image-ControlNet-Inpainting,
+   DiffSynth-Studio Qwen-Image docs, diffusers qwenimage pipeline docs.
+
+2. **[high, 3-0] The scrub mechanism is preserved, and the license is clean.**
+   `QwenImageImg2ImgPipeline.strength` (default 0.6, range 0-1; DiffSynth names it
+   `denoising_strength`) keeps the partial-regeneration scrub the project relies on,
+   lower values staying closer to the input. Qwen-Image and Qwen-Image-Edit-2509 are
+   Apache-2.0 on both code and weights.
+
+3. **[medium, mixed 2-1 / 3-0] Qwen-Image-Edit improves identity consistency, but that
+   is not proof it fixes our metric.** The instruction-edit pipeline (2511 better than
+   2509) improves identity/character consistency, but only for identity *through edits*
+   of an input portrait, which is not the same as measured face-skin Laplacian/LPIPS
+   fidelity at a low scrub strength. Architecture: 20B base + Qwen2.5-VL (semantic
+   control) + VAE Encoder (appearance control). Several stronger edit-model face claims
+   were refuted (see below).
+
+4. **[high, 3-0] Z-Image / Z-Image-Turbo is the best-verified lighter alternative.** A
+   6B model (~1/3 of Qwen-Image's 20B), Apache-2.0 on code and weights, strong bilingual
+   (Chinese + English) native text rendering, with an official diffusers
+   `ZImageImg2ImgPipeline` exposing the same 0-1 denoising-strength scrub lever; Turbo
+   runs at ~8 steps (guidance_scale=0.0) vs ~40. A material cost/footprint reduction vs
+   20B/A100-80GB (but see caveat 4 on the refuted consumer-GPU claim).
+
+5. **[high, 3-0] EliGen-V2 is NOT relevant** to the face-smoothing problem. It is an
+   entity-level/regional control model (LoRA + regional attention placing entities via
+   text + mask maps, plus entity-level inpainting); it provides no
+   ControlNet/canny/depth/tile structure conditioning or face-skin-detail retention.
+
+6. **[medium, 2-1] flymy-ai/qwen-image-realism-lora** is Apache-2.0 (code+weights) on
+   base Qwen-Image, so it is permissively usable with the existing base img2img pass,
+   but it is NOT verified to specifically fix the face/skin-smoothing failure mode.
+
+## Caveats
+
+1. The research did NOT surface verified evidence for two things specifically asked:
+   (a) a Qwen-Image **tile**-ControlNet (the variant most tied to fine-detail/skin
+   retention; only canny/soft-edge/depth/pose/inpaint were confirmed), and (b) any
+   **non-regenerative detail-restoration** technique (high-frequency residual transfer,
+   guided filtering) that recovers smoothed faces without re-introducing the watermark.
+   Research angle 4 produced zero surviving claims, so it is unanswered.
+2. No claim provides measured face-fidelity numbers (ArcFace/LPIPS/Laplacian) for ANY
+   recommended intervention at the project's scrub floors. All fidelity evidence is the
+   project's own internal measurement. The improvements are mechanistically sound but
+   unproven for this exact metric, so validate with `scripts/fidelity_metrics.py`
+   before shipping.
+3. Several vendor model cards are marketing-register primary sources (Qwen blog,
+   Z-Image card). Load-bearing facts (license, params, API levers) are independently
+   corroborated, but comparative quality framings are author glosses.
+4. Z-Image's "sub-second" figure is H800-specific and author-benchmarked; consumer-GPU
+   third-party benchmarks are still limited (seconds, not sub-second, though within the
+   <16GB envelope).
+5. Time-sensitivity: Qwen-Image-Edit-2511 and Z-Image are late-2025/2026 releases; the
+   diffusers pipelines cited are on the main/dev branch, so confirm released-version
+   availability before pinning.
+6. Five claims were refuted (below), clustering on over-strong edit-model face-fidelity
+   and one over-strong Z-Image cost claim.
+
+## Open questions
+
+- Does a Qwen-Image **tile-ControlNet** (or equivalent high-resolution detail
+  conditioning) exist under a permissive license?
+- What **non-regenerative detail-restoration** method recovers smoothed faces WITHOUT
+  re-introducing SynthID? Note: residual transfer from the ORIGINAL risks copying back
+  watermark-carrying high frequencies, so it must be verified against the SynthID oracle.
+- Does adding Qwen-Image-ControlNet (canny/depth) at the certified floors (OpenAI 0.10,
+  Gemini 0.25) actually raise face Laplacian/LPIPS toward the SDXL+ControlNet numbers
+  (0.62 / 0.09) WITHOUT re-introducing SynthID, or does the structure constraint
+  preserve the watermark the way ControlNet can on photoreal content (the existing
+  "SynthID CAN survive controlnet at low strength" caveat)?
+- Head-to-head: does Z-Image-Turbo at its scrub floor match Qwen's text advantage
+  (CJK+Cyrillic CER) while not worsening faces, and what are Z-Image's own SynthID
+  scrub floors and seed-robustness (none exist yet)?
+
+## Refuted claims (do NOT rely on these)
+
+- [0-3] "Qwen-Image-Edit-2511 specifically targets/mitigates image drift, the same
+  failure mode as face-detail loss in a low-strength scrub." (qwen.ai/blog, 2511)
+- [0-3] "Qwen-Image-Edit-2509 explicitly improves facial identity preservation and
+  supports portrait styles and pose transformations." (HF Qwen-Image-Edit-2509)
+- [0-3] "Qwen-Image-Edit-2509 has native built-in ControlNet support (depth/edge/
+  keypoint)." (HF Qwen-Image-Edit-2509)
+- [1-2] "flymy realism LoRA specifically targets facial and skin detail, the exact
+  failure mode." (HF flymy-ai/qwen-image-realism-lora)
+- [0-3] "Z-Image-Turbo runs on consumer 16GB-VRAM hardware, far below the A100-80GB of
+  Qwen-Image 20B, materially lowering per-image cost." (HF Tongyi-MAI/Z-Image-Turbo)
+
+## Sources
+
+1. https://qwen.ai/blog?id=qwen-image-edit-2511
+2. https://qwenlm.github.io/blog/qwen-image-edit/
+3. https://docs.comfy.org/tutorials/image/qwen/qwen-image-edit
+4. https://github.com/FurkanGozukara/Stable-Diffusion/wiki/Qwen-Image-Edit-2511-Free-and-Open-Source-Crushes-Qwen-Image-Edit-2509-and-Challenges-Nano-Banana-Pro
+5. https://myaiforce.com/qie-2511/
+6. https://huggingface.co/Qwen/Qwen-Image-Edit-2509
+7. https://huggingface.co/InstantX/Qwen-Image-ControlNet-Union
+8. https://huggingface.co/InstantX/Qwen-Image-ControlNet-Inpainting
+9. https://huggingface.co/DiffSynth-Studio/Qwen-Image-EliGen-V2
+10. https://github.com/modelscope/DiffSynth-Studio/blob/main/docs/en/Model_Details/Qwen-Image.md
+11. https://blog.comfy.org/p/day-1-support-of-qwen-image-instantx
+12. https://learn.thinkdiffusion.com/how-to-use-qwen-image-with-instantx-union-controlnet-in-comfyui-guide-workflow/
+13. https://huggingface.co/flymy-ai/qwen-image-realism-lora
+14. https://huggingface.co/lightx2v/Qwen-Image-Lightning/discussions/4
+15. https://huggingface.co/docs/diffusers/main/en/api/pipelines/qwenimage
+16. https://www.diyphotography.net/skin-retouching-technique-frequency-separation/
+17. https://link.springer.com/content/pdf/10.1007/978-3-642-15549-9_1.pdf
+18. https://github.com/ShieldMnt/invisible-watermark/wiki/Frequency-Methods
+19. https://huggingface.co/Tongyi-MAI/Z-Image-Turbo
+20. https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/z_image/pipeline_z_image_img2img.py
+21. https://arxiv.org/pdf/2511.22699
+22. https://github.com/ModelTC/LightX2V-Qwen-Image-Lightning

docs/release-and-distribution.md

@@ -0,0 +1,39 @@
+# Release and distribution
+
+> Relocated verbatim from `CLAUDE.md` on 2026-06-11 to keep the always-loaded
+> context small. Long single-line entries were reformatted into paragraphs;
+> no content was changed or summarized.
+
+Release flow and every distribution channel (PyPI, Homebrew tap, conda-forge,
+ComfyUI Registry, HF Space), plus sdist/build-backend history. The CI summary
+stays in `CLAUDE.md`; read this before cutting a release.
+
+`publish.yml` stays release-only and now verifies the release tag matches the `pyproject.toml` version (fails the build on a mismatch) before building, then uploads via `uv publish` (PyPI trusted publishing over OIDC, no token — replaced the `pypa/gh-action-pypi-publish` action so the upload no longer depends on that action's bundled twine accepting the Metadata-Version; the `id-token: write` permission + `pypi` environment + workflow filename are unchanged, so PyPI's trusted-publisher entry still matches).
+
+**Release flow:** bump the version in `pyproject.toml` + `src/remove_ai_watermarks/__init__.py` + `uv.lock` (the project's own `[[package]]` entry — find it with `grep -n 'name = "remove-ai-watermarks"' uv.lock`, the `version =` line right below it, ~line 2246), commit `chore(release): vX.Y.Z`, `git tag -a vX.Y.Z -m vX.Y.Z` (annotated — `git tag` without `-m` errors here), push `main` + the tag, then `gh release create vX.Y.Z` — **PyPI publish triggers on the GitHub Release `published` event, NOT on the tag push**, so the tag alone does not publish.
+
+**After the PyPI sdist is live, bump the Homebrew formula** in the separate public tap repo `wiltodelta/homebrew-tap` (`Formula/remove-ai-watermarks.rb`): update `url` to the new sdist URL (from `https://pypi.org/pypi/remove-ai-watermarks/<version>/json`, the `sdist` entry's `url`) and `sha256` to its hash, commit + push there — otherwise `brew install wiltodelta/tap/remove-ai-watermarks` keeps installing the old version. The formula is a core-only venv that pip-installs the sdist (no vendored resources, so pip pulls the binary numpy/opencv wheels per platform at install time); only those two lines change per release.
+
+**This is now AUTOMATED:** the main repo's `.github/workflows/distribute.yml` fires on the GitHub Release `published` event, waits for the sdist to appear on PyPI (poll loop, the Release event races publish.yml's upload), rewrites the formula's `url`+`sha256`, and pushes to the tap using the `HOMEBREW_TAP_TOKEN` repo secret (a fine-grained PAT with Contents:write on `homebrew-tap`). The SAME workflow also factory-rebuilds the HF Space (`HfApi.restart_space(..., factory_reboot=True)`, `HF_TOKEN` secret) so the Space reinstalls the new sdist (it pins `remove-ai-watermarks>=...` and only re-resolves on a rebuild). The manual Homebrew steps above are the fallback / what the workflow automates — a normal release needs no Homebrew or HF action.
+
+**Where the HF Space's own source lives (its demo code, NOT this library):** the private repo **`wiltodelta/raiw-hf-space`** (locally `~/Documents/GitHub/raiw-hf-space`) — `app.py` (CPU-core Gradio demo), `requirements.txt`, `README.md` (the Space card), `assets/`, `examples/`. **Deploying it is a plain `git push` to that repo's `main`:** its `.github/workflows/sync-to-hf.yml` mirrors the files onto the Space via the Hub API (`HfApi.upload_folder`, secret `HF_TOKEN` = a **write**-role token), which adds a commit on top of the Space's own history — deliberately NOT a `git push --force` to the Space, which would clobber it. Do NOT edit the Space through the huggingface.co web UI any more: that was the pre-2026-07-16 workflow (it is why every Space commit before then is authored `@users.noreply.huggingface.co` and why no local write token ever existed), and a web edit now silently diverges from the GitHub source of truth. Note the demo tracks the library's **CPU-core** surface only (`identify` / `visible` / `metadata`); the invisible/SynthID path needs a GPU and stays out. Two distinct automations touch the Space and must not be confused: `sync-to-hf.yml` (in `raiw-hf-space`) ships **demo code changes**; `distribute.yml` (in this repo) factory-rebuilds the Space on a **library release** so its `remove-ai-watermarks>=...` pin re-resolves to the new version.
+
+**If the distribute.yml Homebrew job fails with "Bad credentials" (or the tap push 403s),** the `HOMEBREW_TAP_TOKEN` secret has expired or been revoked — fine-grained PATs expire on a fixed date, so this recurs. Fix: rotate the PAT (a fine-grained token with Contents:write on `wiltodelta/homebrew-tap`), update the `HOMEBREW_TAP_TOKEN` repo secret, then re-run the failed job (`gh run rerun <run-id> --failed`). While the token is being rotated, the manual formula bump above unblocks the release. The same rotate-secret-and-rerun applies to any distribute.yml credential failure (`HF_TOKEN` for the Space rebuild).
+
+**Other distribution channels:** (1) **conda-forge** — recipe source of truth committed at `packaging/conda/recipe.yaml` (v1 `recipe.yaml`, noarch core-only: pillow/piexif/numpy/py-opencv/click/python-dotenv); the initial submission is `conda-forge/staged-recipes` PR #33674 (went green only after **`pip_check: false`** in the python test — rattler-build's `pip check` defaults to ON and fails on the ancient conda-forge `piexif py_2` build's stale metadata with "piexif 1.1.3 is not supported on this platform", though the package installs/imports/works; keep it disabled). Once that merges and the `remove-ai-watermarks-feedstock` exists, the `regro-cf-autotick-bot` auto-opens a version-bump PR on the feedstock when each new PyPI sdist is detected — just review + merge it (hand-edit only if run-deps changed; keep `packaging/conda/recipe.yaml` in sync as the reference copy). (2) **ComfyUI Registry** — the node package is a SEPARATE repo `wiltodelta/ComfyUI-remove-ai-watermarks` with its OWN `pyproject.toml` `version` (independent of the library version). Publish a new node version by bumping that `version` in the node repo's `pyproject.toml` and pushing to `main` — the node repo's `.github/workflows/publish.yml` (`Comfy-Org/publish-node-action@main`, triggered on a push that touches `pyproject.toml`, secret `COMFY_REGISTRY_TOKEN`) **auto-publishes** it; `comfy node publish --token <registry-PAT>` is the manual/local fallback. It is NOT auto-published on a library release (the node has its own version), so only bump it when the node code or its `remove-ai-watermarks>=` dependency floor changes.
+
+**Sdist must exclude `data/`** (`[tool.hatch.build.targets.sdist] exclude = ["/data"]`): hatchling's default sdist bundles all VCS-tracked files, so the committed `data/` test corpora (the multi-hundred-MB synthid_corpus images + the visible-mark captures) pushed the **0.8.0** sdist past PyPI's per-project file-size limit (400 "File too large") — the wheel uploaded but the sdist was rejected, so 0.8.0 shipped wheel-only and 0.8.1 carried the fix. The wheel only ships `src/` (via `[tool.hatch.build.targets.wheel] packages`), so it was never affected.
+
+**A failed PyPI upload of one artifact still leaves the other live and you cannot re-upload the same version** — fix the build and cut the next patch.
+
+**Build backend is unpinned `hatchling`** (`[build-system] requires`) since 2026-06-09. History: it was pinned `<1.31` because hatchling 1.30.0 made Metadata-Version 2.5 (PEP 794) the default and the twine bundled in `pypa/gh-action-pypi-publish@release/v1` rejected it (`"'2.5' is not a valid Metadata-Version"`), which **failed the v0.8.3 PyPI upload on 2026-06-01**; hatchling 1.30.1 reverted the default to 2.4. After the workflow moved to `uv publish` (whose uploader accepts 2.5) the pin was belt-and-suspenders only, and once v0.9.0 + v0.10.0 both published wheel+sdist through that path (verified on PyPI) it was dropped. If a future hatchling flips the default to 2.5 again and some consumer chokes, re-pin with a dated comment.
+
+## Dependency CVE-resolution history (`uv-secure`)
+
+The standing `uv-secure` gate in `maintain.sh` is clean; this is the changelog of how each alert was resolved, so a future alert is not re-triaged from scratch.
+
+- **idna** bumped 3.11 -> 3.16, fixing GHSA-65pc-fj4g-8rjx.
+- **aiohttp** bumped 3.13.5 -> 3.14.0 via `uv lock --upgrade-package aiohttp`, fixing GHSA-hg6j-4rv6-33pg + GHSA-jg22-mg44-37j8.
+- **basicsr** Dependabot alert GHSA-86w8-vhw6-q9qq is resolved by removal: the experimental `restore` extra was retired and basicsr is no longer anywhere in the dependency tree.
+- **torch** Dependabot alert **GHSA-rrmf-rvhw-rf47** (`torch.jit.script` memory corruption, alert range `<= 2.12.1`) was dismissed `not_used` on 2026-06-10 (torch is a transitive dep of the optional `gpu` extra only and the codebase never calls `torch.jit`) and **resolved by upgrade on 2026-07-21**: the lock carries torch **2.13.0**, above the patched floor, so `uv-secure` is clean. If the GitHub alert has not auto-closed on the lock bump, close it manually as fixed.
+- **setuptools** bumped 81.0.0 -> 83.0.0 (2026-07-21), fixing PYSEC-2026-3447.

docs/research-doubao-distillation.md

@@ -0,0 +1,21 @@
+# Doubao clean-reverse-alpha distillation (re-investigated 2026-05-29)
+
+> Relocated verbatim from `CLAUDE.md` on 2026-06-11 to keep the always-loaded
+> context small. Long single-line entries were reformatted into paragraphs;
+> no content was changed or summarized.
+
+**RESOLVED 2026-05-29: black+gray Doubao captures were obtained and a reverse-alpha is built** (`doubao_engine.remove_watermark_reverse_alpha`, `assets/doubao_alpha.png`; see the `doubao_engine.py` section in `docs/module-internals.md`). The captures (`data/doubao_capture/captures/`, now committed) confirmed the alpha-composite model: on black `captured = a*logo`, logo pure white.
+
+**UPDATE 2026-05-31 (issue #13 follow-up): the first build was NOT "exact"** — it left a readable "豆包AI生成" outline on the real sample (the detector was fooled, conf 0.0). The alpha is now rebuilt by `scripts/visible_alpha_solve.py` (the careful gray-self solve shared with Jimeng), removal always-aligns + thin-inpaints, and the locate box was widened; see the `doubao_engine.py` section in `docs/module-internals.md`. The notes below (the failed content-image distillation) are retained as the record of why controlled captures were necessary.
+
+**Conclusion (historical): pure reverse-alpha distilled from content images does NOT work, and the blocker is the WRONG kind of data, not too little of it.**
+
+The earlier framing ("need ~5-8 PRISTINE same-resolution originals") is obsolete -- a local corpus of pristine originals holds plenty. Curate them with `DoubaoEngine.detect` + an NCC filter against a clean glyph template, keeping only marks at offset ≈ (0,0): that yields e.g. **15 pixel-aligned 2048² marks** (sub-pixel drift, not the ±50 px the old lossy/mixed-res scrapes had), plus 1086x1448 / 1792x2400 clusters. With those, LaMa-clean `O` + weighted-LS (and per-pixel I-on-O regression) for `α` (+ logo color) was tried end-to-end and **still leaves a persistent ghost outline.**
+
+Diagnosed why, empirically (cached stacks, `/tmp/doubao_distill`): (1) the mark is a clean white overlay with **no dark halo** -- over glyph pixels ~54% are brighter than the clean bg, only ~4% darker -- so the white-logo model `I=(1-α)O+α·255` is correct; (2) but content backgrounds are almost never dark *under* the mark (median darkest available bg over glyph pixels = **58/255**; only ~13% of mark pixels are ever observed on a bg < 40), so on bright backgrounds the equation is ill-conditioned and `α` is unidentifiable; (3) LaMa's `O` is a plausible **hallucination**, not the true pre-mark background, which compounds the error, and per-pixel regression on ~15 obs overfits into color noise.
+
+**Why Gemini's engine is clean (verified in GeminiWatermarkTool `src/core/watermark_engine.cpp`): its alpha map is the watermark stamped on a PURE-BLACK background**, where `watermarked = α·255 + (1-α)·0 = α·255`, so `alpha = capture/255` exactly -- no estimation. (`gemini_bg_*.png` is literally the sparkle in grey on black.) So the real Doubao unlock is the same controlled capture, **not more content images**. Black/white/gray seeds exist (`data/doubao_capture/seeds/seed_*_1x1_2048x2048.png`); a capture run (feed a black seed through doubao.com edit mode, download the *original*) was requested from the #13 reporter 2026-05-29. With ~2-3 black captures we get `α = capture/255` for free, Gemini-quality.
+
+**Until black captures arrive, the shipped direction is precise canonical glyph mask + inpaint (cv2 default, lama optional), NOT reverse-alpha.**
+
+The consensus glyph silhouette across the aligned marks distills cleanly (proto: a tight "豆包AI生成" strip, width ≈ 0.156 × image-width) and is good both as an exact inpaint mask and as an NCC localiser -- the latter also fixes the #23 detector false-positives (match the real glyph shape, not any bright low-saturation corner). Do **not** retry content-image reverse-alpha: it is data-limited by physics (no dark-background observations), not by effort.

docs/synthid-robust-identity-research-2026-06-08.md

@@ -0,0 +1,185 @@
+# Deep research: SynthID-safe face-identity recovery for SDXL (2026-06-08)
+
+**Stats:** {"angles": 6, "sourcesFetched": 28, "claimsExtracted": 104, "claimsVerified": 25, "confirmed": 19, "killed": 6, "afterSynthesis": 6, "urlDupes": 1, "budgetDropped": 7, "agentCalls": 111}
+
+## Summary
+
+For SDXL-based SynthID-removal pipelines, the entire ArcFace-class identity-adapter ecosystem (PhotoMaker-V2, InstantID, PuLID, IP-Adapter FaceID, Arc2Face) is blocked from commercial use by a single chokepoint: InsightFace's pretrained model packs (buffalo_l, antelopev2, buffalo_s, buffalo_m) are explicitly non-commercial research-only, regardless of the adapter weights' Apache-2.0/MIT license. The InstantX maintainers themselves acknowledged this on HuggingFace ("cannot be Apache 2.0 if it is using Insight Face") and stated intent to retrain on commercially-licensed embedders — work that as of the verified sources has not shipped. The only verified commercial-safe SDXL adapter surfaced is MS-Diffusion (ICLR 2025), which uses CLIP-ViT-bigG-14 instead of ArcFace and runs on SDXL-base-1.0, but it is an IP-Adapter-class subject-similarity method, not a face-identity method — CLIP image embeddings are empirically weaker for face identity than ArcFace (~81% vs ~88% face-ID accuracy), so it solves the license problem but likely not the identity-fidelity problem. Arc2Face is SD1.5-only and so not a drop-in regardless of license; StyleGAN2-based methods (e4e) terminate in a GAN generator with no SDXL wiring. Net: no fully commercial-safe SynthID-robust SDXL identity-preservation stack with ArcFace-grade fidelity exists today — the space is genuinely blocked by InsightFace's grip on commercial ArcFace embeddings, and AdaFace as a permissive ArcFace alternative was REFUTED in verification.
+
+## Findings
+
+### 1. The InsightFace pretrained model packs (buffalo_l, antelopev2, buffalo_s, buffalo_m) are non-commercial research-only and this restriction propagates at runtime to any adapter that calls FaceAnalysis(), regardless of the adapter's own license tag.
+
+**Confidence:** high  
+**Vote:** 3-0 unanimous across 4 supporting claims
+
+
+InsightFace upstream: code is MIT but 'The training data containing the annotation (and the models trained with these data) are available for non-commercial research purposes only.' Licensing page enumerates buffalo_l, antelopev2, buffalo_s, buffalo_m as needing separate commercial licensing. InstantX maintainer on HF: 'cannot be Apache 2.0 if it is using Insight Face... we plan to train on other face encoders that support commercial license.' Mechanical propagation: the runtime FaceAnalysis() call pulls these packs.
+
+- https://github.com/deepinsight/insightface
+- https://www.insightface.ai/solutions/face-recognition-licensing
+- https://huggingface.co/InstantX/InstantID/discussions/2
+
+### 2. InstantID is Apache-2.0 on the adapter weights and architecturally a clean plugin to SDXL-base-1.0 (no UNet training, semantic IdentityNet conditioning + landmark weak-spatial, no pixel copying) — but at runtime it instantiates FaceAnalysis(name='antelopev2'), inheriting the InsightFace non-commercial restriction.
+
+**Confidence:** high  
+**Vote:** 3-0 across 5 claims
+
+
+HF model card: 'License: apache-2.0' AND 'For face encoder, you need to manutally download via this URL to models/antelopev2' with 'from insightface.app import FaceAnalysis' in usage. Diffusers community pipeline shows literal `app = FaceAnalysis(name='antelopev2', ...)` then `face_emb = face_info['embedding']`. Paper: 'IdentityNet by imposing strong semantic and weak spatial conditions, integrating facial and landmark images with textual prompts' + 'seamlessly integrates with SD1.5 and SDXL'. So mechanism is semantic (good for SynthID-no-pixel-leak) but license is blocked.
+
+- https://huggingface.co/InstantX/InstantID
+- https://github.com/huggingface/diffusers/blob/main/examples/community/pipeline_stable_diffusion_xl_instantid.py
+- https://arxiv.org/pdf/2401.07519
+- https://instantid.github.io/
+
+### 3. Arc2Face is SD1.5-only (not SDXL) AND requires InsightFace antelopev2 with arcface.onnx replacing glintr100.onnx — so it is doubly blocked: not a drop-in for the SDXL pipeline AND non-commercial via the model pack.
+
+**Confidence:** high  
+**Vote:** 3-0 across 3 claims
+
+
+Arc2Face README verbatim: 'Arc2Face is built upon SD1.5' with stable-diffusion-v1-5 checkpoint as base. Same README: 'manually download the antelopev2 package and place the checkpoints under models/antelopev2', 'Download arcface.onnx from HuggingFace', 'delete glintr100.onnx (the default backbone from insightface)'. Code is MIT but the runtime antelopev2 dependency carries the non-commercial restriction. No SDXL adaptation exists in the official repo.
+
+- https://github.com/foivospar/Arc2Face
+
+### 4. MS-Diffusion (ICLR 2025) is the one verified candidate that combines SDXL-base-1.0 as its foundation with a non-ArcFace image encoder (CLIP-ViT-bigG-14), so it avoids the InsightFace non-commercial blocker — but it is an IP-Adapter-class multi-subject personalization method, NOT a face-identity-recognition method, so license safety does not equal face-identity fidelity.
+
+**Confidence:** medium  
+**Vote:** 3-0 on the narrow factual claims (SDXL base + CLIP-G encoder)
+
+
+GitHub README explicitly instructs 'Download the pretrained base models from SDXL-base-1.0 and CLIP-G' (CLIP-ViT-bigG-14-laion2B-39B-b160k). Neither README nor arXiv 2406.07209 mention ArcFace/InsightFace/antelopev2/buffalo_l. Architecturally descended from IP-Adapter (CLIP-image-embedding family), not from FaceID/InstantID/PhotoMaker-V2. Verifier caveat (high confidence on the license-narrow claim, medium on suitability): CLIP-image face-ID accuracy ~80.95% vs specialized face recognition ~87.61% — license-safe but probably not identity-grade for portraits. Confidence is medium because the suitability claim for raiw.cc face-identity use case has not been validated empirically.
+
+- https://proceedings.iclr.cc/paper_files/paper/2025/file/ed4df1609bf7d8602435341c9ce2ab5f-Paper-Conference.pdf
+- https://github.com/MS-Diffusion/MS-Diffusion
+
+### 5. ID-Aligner and the StyleGAN2/e4e-based identity method (arXiv 2510.25084) do not solve the problem: the former does not disclose embedder/license in the primary source; the latter terminates identity in a StyleGAN2 generator with no SDXL adapter, so it cannot be wired into the SDXL+canny ControlNet pipeline.
+
+**Confidence:** high  
+**Vote:** 3-0 across 3 claims
+
+
+ID-Aligner paper: no license terms, no weight-release status, no specific face embedder named in the primary source — commercial-safety unresolved. arXiv 2510.25084: 'facial identity features... mapped into the W+ latent space of StyleGAN2 using the e4e encoder' — identity pixels come from StyleGAN2's generator, not from any SDXL-compatible adapter, so architecturally incompatible with our SDXL+canny ControlNet pipeline. Abstract also does not name the face embedder or disclose weights license.
+
+- https://arxiv.org/pdf/2404.15449
+- https://arxiv.org/pdf/2510.25084
+
+### 6. The honest verdict: no fully commercial-safe SynthID-robust ArcFace-grade face-identity stack for SDXL exists today; the space is blocked by InsightFace's grip on ArcFace-class pretrained packs, and the verified permissive alternative (AdaFace as a drop-in replacement) was REFUTED in this verification round.
+
+**Confidence:** high  
+**Vote:** 3-0 on the chokepoint claim, 0-3 against the AdaFace escape hatch
+
+
+Every ArcFace-grade SDXL adapter audited (PhotoMaker-V2, InstantID, PuLID, IP-Adapter FaceID, Arc2Face) instantiates an InsightFace FaceAnalysis pack at runtime. The maintainer-acknowledged commercial-retrain path (InstantX) has not shipped. AdaFace as a permissive ArcFace alternative was refuted 0-3 by the adversarial verifier (MIT-licensed claim and drop-in-replacement claim both failed). Outcome: commercial-safe option for SDXL today is CLIP-image-embedding-based (MS-Diffusion class), which is weaker for face identity than ArcFace. For non-commercial / research-only deployments, InstantID on SDXL is the strongest semantic-only (no-pixel-leak) candidate.
+
+- https://github.com/deepinsight/insightface
+- https://huggingface.co/InstantX/InstantID/discussions/2
+- https://github.com/mk-minchul/AdaFace
+
+## Caveats
+
+Six claims were refuted in adversarial verification, two of them load-bearing: AdaFace as a permissive ArcFace drop-in (both the MIT license and the drop-in characterization failed 0-3) — so the most attractive escape hatch from the InsightFace chokepoint did not survive verification. PuLID's superiority claim and ID-Aligner's embedder claim also failed, leaving those methods uncharacterized at the mechanism level. None of the verified claims directly answered the diffusers-0.38 compat question — InstantID's compat with our shipped diffusers version is unverified by primary source. MS-Diffusion's identity-fidelity for portraits is not empirically validated for the SynthID-removal use case; it is verified as a CLIP-G/SDXL adapter, not as a face-identity method. No 2025-2026 candidate other than MS-Diffusion was both surfaced AND verified as license-safe and SDXL-compatible — the verification round did not produce a confirmed CCSR-V2/ConsistencyID/OmniGen-V2/ConsistentID/MagicID candidate. The multi-face scenario (group photos) was not addressed by any verified claim — MS-Diffusion is the only multi-subject candidate but its face-identity strength is unmeasured. Time-sensitivity: InstantX's stated intent to retrain on commercial embedders ("We agree, we plan to train on other face encoders that support commercial license") is dated and unfulfilled per the verified sources; this could change.
+
+## Open questions
+
+- Does MS-Diffusion (or any CLIP-image-embedding SDXL adapter) achieve usable face-identity fidelity on the raiw.cc input distribution (portraits + group photos), or is the ArcFace gap (~7 pp face-ID accuracy) visually disqualifying — and can a face-specific CLIP fine-tune close it?
+- Has InstantX (or any community fork) actually shipped an InstantID variant retrained on a commercially-licensed face embedder since the maintainer's 2024 commitment, and if so what is its identity-fidelity vs the antelopev2 original?
+- What is the exact diffusers-0.38 compat status of InstantID, MS-Diffusion, and PuLID-FLUX inference scripts — does any need a fork the way PhotoMaker-V1 did, and if so what specifically breaks?
+- Is there a single-pipeline multi-subject identity-preservation method (mask-guided regional ID-adapters, multi-subject InstantID, MS-Diffusion multi-subject mode) that handles group photos without the per-face crop+composite patchwork that PhotoMaker-V2 produced?
+
+## Refuted claims
+
+- **AdaFace code is released under the MIT license, making it permissively licensed for commercial use (in contrast to InsightFace's research-only model packs).** — vote 0-3 — source: https://github.com/mk-minchul/AdaFace
+- **AdaFace produces 512-dim face recognition embeddings comparable to ArcFace, positioning it as a drop-in alternative for ID-conditioning adapters that currently depend on InsightFace ArcFace.** — vote 0-3 — source: https://github.com/mk-minchul/AdaFace
+- **The paper claims PuLID achieves superior performance in both ID fidelity and editability compared to prior ID-customization methods.** — vote 0-3 — source: https://arxiv.org/pdf/2404.16022
+- **Identity consistency in ID-Aligner is enforced by reward feedback from face detection and recognition models, implying dependency on an external face-recognition embedder (typically ArcFace/InsightFace-class) rather than a CLIP-only path.** — vote 1-2 — source: https://arxiv.org/pdf/2404.15449
+- **The pipeline decouples face detection from inference by accepting a pre-computed embedding via image_embeds, so any ArcFace-class embedder could be swapped in if a commercially-licensed equivalent existed.** — vote 1-2 — source: https://github.com/huggingface/diffusers/blob/main/examples/community/pipeline_stable_diffusion_xl_instantid.py
+- **The model card does not explicitly prohibit commercial use, but the license of the required InsightFace antelopev2 embedder is not specified on this page.** — vote 1-2 — source: https://huggingface.co/InstantX/InstantID
+
+## Sources
+
+- [source](https://huggingface.co/InstantX/InstantID/discussions/2)
+- [source](https://www.insightface.ai/solutions/face-recognition-licensing)
+- [source](https://github.com/mk-minchul/AdaFace)
+- [source](https://github.com/foivospar/Arc2Face)
+- [source](https://apatero.com/blog/instantid-vs-pulid-vs-faceid-ultimate-face-swap-comparison-2025)
+- [source](https://arxiv.org/pdf/2404.16022)
+- [source](https://instantid.github.io/)
+- [source](https://arxiv.org/pdf/2404.15449)
+- [source](https://arxiv.org/pdf/2511.11989)
+- [source](https://arxiv.org/pdf/2510.25084)
+- [source](https://github.com/huggingface/diffusers/blob/main/examples/community/pipeline_stable_diffusion_xl_instantid.py)
+- [source](https://huggingface.co/InstantX/InstantID)
+- [source](https://github.com/huggingface/diffusers/issues/9158)
+- [source](https://github.com/huggingface/diffusers/issues/5904)
+- [source](https://github.com/Mikubill/sd-webui-controlnet/discussions/2589)
+- [source](https://arxiv.org/pdf/2401.07519)
+- [source](https://proceedings.iclr.cc/paper_files/paper/2025/file/ed4df1609bf7d8602435341c9ce2ab5f-Paper-Conference.pdf)
+- [source](https://github.com/huggingface/diffusers/issues/8626)
+- [source](https://arxiv.org/pdf/2404.04243)
+- [source](https://huggingface.co/OmniGen2/OmniGen2)
+- [source](https://github.com/VectorSpaceLab/OmniGen)
+- [source](https://github.com/ToTheBeginning/PuLID)
+- [source](https://arc2face.github.io/)
+- [source](https://github.com/mk-minchul/AdaFace/blob/master/LICENSE)
+- [source](https://github.com/IrvingMeng/MagFace/blob/main/LICENSE)
+- [source](https://github.com/askerlee/AdaFace-dev)
+- [source](https://openreview.net/forum?id=Hc2ZwCYgmB)
+- [source](https://github.com/tencent-ailab/IP-Adapter/wiki/IP%E2%80%90Adapter%E2%80%90Face)
+
+## Empirical follow-up (2026-06-08, end of session)
+
+After the research synthesis above, InstantID was integrated end-to-end and cert-swept
+on Modal A100 in two phases:
+
+1. **Phase 1: InstantID txt2img per-face crop + composite.** Per-face InstantID
+   txt2img with the upstream `pipeline_stable_diffusion_xl_instantid`, ArcFace
+   embedding from the original face, landmark stick figure. Three composite
+   iterations:
+   - v1 (rectangular Gaussian alpha on the 2x square_box around each face):
+     visible patchwork on group photos, generated 1024 backgrounds clashing.
+   - v2 (tight crop on YuNet-detected face in the generated 1024 + elliptical
+     alpha 0.45*bw x 0.55*bh + soft feather): ellipse axis exceeded bbox
+     vertically, clipped forehead/chin on single portrait, group still had
+     visible elliptical seams + cool-vs-warm tone clash with scene.
+   - v3 (tighter ellipse 0.32*bw x 0.42*bh + per-channel mean color match to
+     local cleaned canvas + softer feather): patchwork visually softened; faces
+     still read as studio portraits inserted into the scene, not as people
+     shot in the scene. Single portrait identity drifted (tatsunari -> "round
+     Asian male" vs original's thin face).
+2. **Phase 2: InstantID img2img on cleaned crop.** Switched to the upstream
+   `pipeline_stable_diffusion_xl_instantid_img2img` (downloaded at first use
+   from raw.githubusercontent.com; requires `trust_remote_code=True`). Same
+   ArcFace + landmark conditioning but the SDXL diffusion source is the
+   CLEANED face crop, so the diffusion sees scene lighting / shoulders /
+   shadow direction directly. Multi-face composition jumped substantially:
+   faces sit in the bar scene with matching warm tone, no more elliptical
+   seams. Single-portrait identity at the default (`strength=0.55`,
+   `ip_adapter_scale=0.8`, `controlnet_conditioning_scale=0.8`) was "similar
+   person, not exactly the original"; raising to `strength=0.7`,
+   `ip_adapter_scale=1.0`, `controlnet_scale=1.0` brought identity closer to
+   original but introduced more "SDXL gloss / clean skin" aesthetic.
+
+**Net finding for raiw.cc (load-bearing).** The fundamental issue is structural:
+ArcFace encodes "this person's general look" (ethnicity, gender, basic facial
+geometry) at 512 dimensions; SDXL decodes that embedding into pixels with the
+inherent SDXL aesthetic (smooth skin, symmetric pores, AI-photoreal look).
+Stronger identity push (higher strength / IP-Adapter scale) makes the face
+CLOSER to the embedded identity but MORE AI-looking; weaker push leaves
+identity to drift but face looks less AI-generated. There is no parameter
+setting that simultaneously recovers original identity AND looks less AI than
+the cleaned image, because the cleaned image is itself a controlnet-light
+denoise of the original (closer to original pixels) while a restore pass is a
+full SDXL regeneration (further from original pixels).
+
+**Operational conclusion.** Do not ship `--restore-faces` in any monetized
+deployment. The cleaned image from the main controlnet 0.20 pass is the
+LEAST-AI state we can reach without re-introducing SynthID; every restore
+method tested (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img,
+InstantID img2img-on-cleaned at three parameter sweeps) trades original-look
+for embedding-driven regeneration and makes the face read as "AI-generated"
+rather than "the original person". The `instantid` and `photomaker` extras
+stay in the library as opt-in for research / personal use where users
+explicitly want identity regeneration; the CLI flag and module docstrings
+state the trade-off at every entry point.

docs/synthid-robust-identity-research.md

@@ -0,0 +1,282 @@
+# SynthID-robust face identity for an SDXL removal pipeline (research)
+
+> **Status (2026-06-08): retired.** Every approach described below was empirically
+> tested and rejected -- see `docs/synthid-robust-identity-research-2026-06-08.md`
+> "Empirical follow-up" for the final conclusion. The library no longer ships any
+> face-restore extra: the cleaned image from the main controlnet 0.20 pass is the
+> least-AI face state we can reach without re-introducing SynthID. This document
+> is kept as historical record of the exploration.
+
+**Question.** Which face identity-preservation mechanism for an SDXL img2img +
+canny-ControlNet watermark-removal pipeline (denoise 0.20-0.30) is BOTH (a)
+commercial-safe end-to-end and (b) does not re-introduce the SynthID pixel
+watermark the removal pass just destroyed?
+
+**Constraint.** raiw.cc is a paid service, so every component (adapter weights AND
+the face embedder it conditions on AND any base model) must be Apache-2.0 / MIT /
+BSD or otherwise clearly commercial-permitted. Non-commercial is disqualifying.
+
+**One-line verdict.** Today there is **ONE** SDXL identity-conditioning stack that
+is commercial-safe end-to-end: **PhotoMaker-V1** (Apache-2.0, identity encoded as a
+fine-tuned OpenCLIP-ViT-H/14 image embedding -- NO InsightFace). Every other
+candidate -- **including PhotoMaker-V2**, IP-Adapter FaceID, InstantID, PuLID,
+Arc2Face -- inherits InsightFace's non-commercial model-pack license through an
+ArcFace-class embedder and is therefore blocked for paid services, regardless of
+the adapter's own license header. Below is the evidence per component and the
+integration plan.
+
+**Correction notice (2026-06-04).** An earlier version of this doc claimed
+PhotoMaker-V2 was commercial-safe end-to-end. That was WRONG -- the V2 model card
+phrase *"id_encoder includes finetuned OpenCLIP-ViT-H-14 and a few fuse layers"*
+described one of TWO ID branches; the V2 source (model_v2.py) defines
+`PhotoMakerIDEncoder_CLIPInsightfaceExtendtoken` whose forward takes an
+ArcFace `id_embeds` from `insightface.app.FaceAnalysis`, and the upstream package
+`__init__.py` imports InsightFace at module load. A Modal cert sweep caught this
+empirically (`No module named 'insightface'` from `restore_faces_photomaker`). V1
+is the correct commercial-safe target: its `PhotoMakerIDEncoder` (model.py)
+forward takes only `(id_pixel_values, prompt_embeds, class_tokens_mask)` -- no
+ArcFace branch -- so identity is CLIP-only.
+
+**Status notice (2026-06-04, end of session).** Two commercial-safe paths were
+tried and abandoned:
+
+1. **PhotoMaker-V1** (commercial-safe by license but blocked by upstream compat).
+   The cert sweep hit a cascade of upstream compatibility issues with the diffusers
+   version we ship (0.38): missing `einops` declaration, missing `peft` declaration,
+   default `pm_version='v2'` that mis-loads V1 weights into the V2 encoder, custom
+   `id_encoder` left on CPU after `pipe.to(device)`, and a CFG-batch tensor-shape
+   mismatch in the denoising loop (`Expected size 2 but got size 1`). 7 cascading
+   fixes did not get the pipeline running end-to-end. The PhotoMaker `pipeline.py`
+   header notes it was forked from diffusers v0.29.1; SDXL prompt-encoder handling
+   changed significantly between 0.29 and 0.38.
+2. **GFPGAN on the diffusion-CLEANED image** (commercial-safe, but no identity
+   recovery). A one-line change made it SynthID-safe (input pixels are already
+   clean, so the partial blend cannot transport the watermark), but visual
+   inspection of the cert output showed it only polished the already-drifted face
+   without actually restoring identity. Trade-off was real and the value too low
+   to keep.
+
+**The shipped path is PhotoMaker-V2** (`photomaker_restore.py`, the `photomaker`
+extra). V2 uses a DUAL ID encoder (CLIP image features + ArcFace embedding),
+which delivers true identity-from-embedding face regeneration. The cost is that
+the ArcFace embedding comes from InsightFace's `antelopev2`/`buffalo_l` model
+packs, which are released under a non-commercial / research-only license. **So
+the shipped restore path is NON-COMMERCIAL.** raiw.cc and any other monetized
+deployment must NOT install the `photomaker` extra. The CLI flag and module
+docstring both call this out at every entry point.
+
+A future commercial-safe path would need either (a) the PhotoMaker upstream to
+land its diffusers 0.38 compat fix so V1 can run, or (b) an equally good
+ArcFace-class face-recognition model released under a permissive license that
+PhotoMaker-V2 can be retargeted to. Neither is on a near-term horizon as of
+this writing.
+
+## 1. Why identity-by-embedding (not by pixel) is the only SynthID-robust path
+
+The pipeline regenerates pixels to destroy SynthID. Any identity-restoration that
+is "faithful to the input pixels" (GFPGAN, CodeFormer, face-swap-by-blending, our
+previous restore-on-original pass) reproduces the watermark, because SynthID is
+engineered to be robust to fidelity-preserving transforms (resize, JPEG, partial
+blend). Oracle-confirmed on a real Gemini face: controlnet @ 0.20/0.25 WITH the
+GFPGAN restore pass left SynthID detected; the SAME controlnet @ 0.20 with
+`--no-restore-faces` cleared it (clean A/B, see `docs/synthid.md` 5.5 and
+`docs/controlnet-removal-pipeline-research.md`).
+
+The only mechanism that can preserve identity AND not re-introduce SynthID is to
+carry identity in a SEMANTIC EMBEDDING (a vector that encodes "who is in this
+picture") and use it to CONDITION a fresh generation -- the pixels are new, so
+the watermark is not transported. Two embedding families exist in practice:
+
+- **ArcFace-class face-recognition embeddings** (the InsightFace family). Used by
+  IP-Adapter FaceID, InstantID, PuLID, Arc2Face. Highest identity fidelity, but
+  the embedder weights are non-commercial.
+- **CLIP image embeddings of a face crop**. Used by PhotoMaker (and the original
+  IP-Adapter image variant). Lower identity fidelity at small scale than ArcFace,
+  but the encoder (OpenCLIP-ViT-H/14, MIT) is commercial-safe.
+
+## 2. License table (verified against primary sources, 2026-06-04)
+
+| stack | adapter weights | identity encoder | end-to-end commercial-safe? |
+|---|---|---|---|
+| **PhotoMaker-V1** | **Apache-2.0** ([HF][pmhf]) | **OpenCLIP-ViT-H/14 (MIT)** finetuned, identity from `PhotoMakerIDEncoder` (`model.py`); forward takes only ``(id_pixel_values, prompt_embeds, class_tokens_mask)`` -- no ArcFace branch | **YES** |
+| PhotoMaker-V2 | Apache-2.0 (adapter) ([HF][pm2hf]) | DUAL encoder: OpenCLIP-ViT-H/14 AND InsightFace antelopev2/buffalo_l -- `PhotoMakerIDEncoder_CLIPInsightfaceExtendtoken` (`model_v2.py`) forward takes `id_embeds` from `insightface.app.FaceAnalysis`, and `photomaker/__init__.py` imports InsightFace at module load | NO -- InsightFace pack is non-commercial |
+| IP-Adapter FaceID | non-commercial per model card: *"AS InsightFace pretrained models are available for non-commercial research purposes, IP-Adapter-FaceID models are released exclusively for research purposes and is not intended for commercial use"* ([HF][ipafhf]) | InsightFace antelopev2 (non-commercial for the model pack) | NO -- both layers block |
+| InstantID | Apache-2.0 (adapter only) ([HF][insthf]) | requires InsightFace antelopev2 face-analysis at runtime (`FaceAnalysis(name='antelopev2', ...)` per the README usage snippet, [HF][insthf]) | NO -- embedder pack is non-commercial |
+| PuLID | apache-2.0 (HF model metadata, [HF][pulidhf]) | depends on InsightFace face-analysis for ArcFace embedding (per the upstream README; PuLID's own card is sparse and the GitHub README documents the InsightFace install step) | NO -- same embedder issue as IP-Adapter FaceID |
+| Arc2Face | MIT (HF model metadata, [HF][arc2hf]) | uses `insightface.app.FaceAnalysis` to extract the ArcFace embedding ([HF][arc2hf]); also based on SD-v1-5 (NOT SDXL) | NO -- non-commercial embedder + not SDXL |
+
+**The crux is InsightFace.** InsightFace explicitly splits its license: *"Code is
+MIT licensed; models require separate commercial licensing"* and frames the
+pretrained packs as *"Commercial licensing for InsightFace's open-source model
+packages"* requiring users to *"obtain commercial usage rights for model
+packages"* ([insightface.ai][iflic]). antelopev2 and buffalo_l fall under the
+model-pack license, not MIT. So any stack that calls
+`insightface.app.FaceAnalysis(name='antelopev2', ...)` to compute its ArcFace
+embedding is blocked by default, REGARDLESS of the adapter's own Apache header
+above it. This is the same reason IP-Adapter FaceID's card flags itself
+non-commercial.
+
+(Note on PuLID's HF metadata: the model card declares apache-2.0 for the adapter
+weights but the upstream repo's quickstart requires the InsightFace package to
+extract the ID embedding. So PuLID's adapter license is permissive; the BLOCKER
+is the embedder it expects at runtime. This is the same trap as InstantID.)
+
+[pmhf]: https://huggingface.co/TencentARC/PhotoMaker
+[pm2hf]: https://huggingface.co/TencentARC/PhotoMaker-V2
+[ipafhf]: https://huggingface.co/h94/IP-Adapter-FaceID
+[insthf]: https://huggingface.co/InstantX/InstantID
+[pulidhf]: https://huggingface.co/guozinan/PuLID
+[arc2hf]: https://huggingface.co/FoivosPar/Arc2Face
+[iflic]: https://www.insightface.ai/solutions/face-recognition-licensing
+
+## 3. Is there a commercial-safe ArcFace replacement?
+
+Short answer: **no clean drop-in**. The widely deployed pretrained ArcFace packs
+(antelopev2, buffalo_l, glint360k) come from InsightFace and are non-commercial.
+ArcFace as an ARCHITECTURE is published in a paper, so retraining is legally fine,
+but you would need:
+
+- a commercial-licensed training dataset (the big public ones -- MS-Celeb-1M,
+  Glint360K, WebFace -- carry research-only or licensing-uncertain restrictions);
+- compute + time to train an ArcFace-class model on the legal dataset;
+- the result would be a one-off effort, not a maintained dependency.
+
+For a removal service this is a multi-month side project that delivers what
+PhotoMaker already gives us with one pip install. So the practical answer is to
+take the CLIP-embedding path (PhotoMaker-V1; V2 adds InsightFace and is non-commercial), accept the identity-fidelity
+trade-off, and revisit ArcFace later if quality is insufficient.
+
+## 4. Does an identity embedding leak SynthID?
+
+This is the load-bearing assumption of the whole approach. The argument:
+
+- SynthID is a low-amplitude, perceptually-invisible pixel watermark engineered
+  to be robust to "fidelity-preserving" transforms (it survives JPEG, resize,
+  crop, color, noise at >=99% TPR -- see arXiv:2510.09263 referenced in
+  `docs/synthid.md`).
+- A face-recognition / CLIP-image embedding is by design INVARIANT to such low-
+  amplitude pixel changes (compression, brightness, small noise should not change
+  "who is in the photo"). That is the whole training objective.
+- Therefore the embedding extracted from a watermarked face vs. the same face
+  cleaned should be ~identical -- the embedding cannot CARRY the watermark
+  pattern, only the identity, because the watermark sits in exactly the
+  dimensions the embedding learned to discard.
+
+**MEASURED 2026-06-04 — hypothesis confirmed.** Ran a low-amplitude
+perturbation sweep on 31 face crops (3 photoreal originals: gemini_3, gemini_4,
+openai_3 grid), comparing `cos(embedding(orig), embedding(perturbed))` for OpenCLIP-
+ViT-H/14 (laion2B-s32B-b79K, the same OpenCLIP-ViT-H/14 encoder PhotoMaker V1 and V2 both finetune for CLIP-side identity):
+
+| perturbation | mean cos | min | max |
+|---|---|---|---|
+| **synthid_proxy** (±2 LSB low-freq noise, σ=4 px Gaussian carrier — same regime SynthID hides in) | **0.9977** | 0.9937 | 0.9996 |
+| noise3 (Gaussian σ=3, full-spectrum) | 0.9541 | 0.9055 | 0.9825 |
+| jpeg90 (SynthID survives this) | 0.9280 | 0.8806 | 0.9566 |
+| blur1 (Gaussian σ=1) | 0.9139 | 0.8103 | 0.9875 |
+| jpeg70 | 0.8945 | 0.8125 | 0.9603 |
+| (self check: identical crop) | 1.0000 | 1.0000 | 1.0000 |
+
+The SynthID-magnitude perturbation moves the embedding by **0.002** (cosine 0.9977),
+an order of magnitude less than JPEG90 — which SynthID survives at >=99% TPR by
+design. So the embedding cannot carry the watermark pattern: its discriminative
+signal is in dimensions the SynthID payload does not occupy. PhotoMaker-V1
+conditioned on a watermarked face will see ~the same identity vector as if
+conditioned on a clean face of the same person, so the freshly generated face
+inherits the identity, not the watermark.
+
+A first, naive smoke run measured `cos(orig, SDXL-cleaned)` instead — that test is
+about diffusion drift, not watermark invariance (diffusion at strength 0.20-0.30 is a
+much larger perturbation than SynthID), so its 0.56-0.93 spread is the identity
+drift the PhotoMaker pipeline is meant to fix in the first place. The
+synthid_proxy result above is the one that actually answers the load-bearing
+question. Script: `/tmp/identity_smoke/test2_proxy.py` (not committed; reproducible
+from the test set + this doc).
+
+## 5. PhotoMaker-V1 properties for our pipeline
+
+- **SDXL-native.** PhotoMaker v1 and v2 target Stable Diffusion XL; the pipeline
+  is a stacked-ID embedding fused into SDXL's cross-attention via the fuse layers
+  bundled in the released weights.
+- **Identity from a SINGLE reference image works** but the method was designed
+  for "stacked" multi-reference; with one image identity fidelity is lower than
+  with 3-4, and a service has only one (the upload). This is the failure mode to
+  guard.
+- **Compatibility with img2img + canny ControlNet.** PhotoMaker is typically
+  exposed in txt2img workflows in the upstream demo. SDXL img2img + ControlNet
+  is the same denoising backbone, so the cross-attention injection works the same
+  way; community examples on Diffusers and ComfyUI confirm PhotoMaker stacks with
+  ControlNet. Validate this on a representative image before adopting.
+- **Failure modes to expect:**
+  - identity drift on small / multi-face groups (the 9-face grid case);
+  - "plastic" / over-smoothed faces if PhotoMaker's identity weighting is high
+    while the img2img strength is low;
+  - canny ControlNet conditioning can fight the ID embedding (edges of the
+    ORIGINAL face vs identity of the SAME person regenerated) -- expect to tune
+    `controlnet_conditioning_scale` down a notch on photoreal faces;
+  - PhotoMaker was trained on a celebrity-skew distribution; real-user faces
+    (especially non-white, non-Western, elderly, children) may have lower
+    fidelity. Measure on the real upload distribution.
+
+## 6. Integration cost (rough)
+
+- New deps: `diffusers` already in the gpu extra; PhotoMaker ships as a `.bin`
+  loaded via `pipeline.load_photomaker_adapter(...)`. The OpenCLIP encoder is the
+  same one diffusers already pulls. No new heavy pip dep.
+- Weight download: PhotoMaker-V1 weights are ~3 GB. Add to the Modal HF volume
+  alongside SDXL.
+- VRAM: SDXL + canny ControlNet + PhotoMaker-V1 fits comfortably in A100-40GB.
+- Latency: a few extra seconds on cold start (load PhotoMaker), negligible per
+  request after warm-up.
+- No InsightFace install: huge win for `restore` extra's basicsr/numpy hell --
+  this path simply does not touch that ecosystem.
+
+## 7. Recommended path
+
+1. **Embedding-invariance smoke test FIRST** (one afternoon, no codegen):
+   - For ~10 OpenAI / Gemini watermarked faces, compute OpenCLIP-ViT-H/14
+     embeddings; for the same images after our SDXL `default` pass at the
+     certified strength, compute the embeddings again; assert mean cosine
+     similarity > ~0.95.
+   - If yes -> the embedding does not carry SynthID, proceed.
+   - If no -> the assumption is wrong; PhotoMaker would re-introduce the
+     watermark. Stop and reconsider.
+2. **PhotoMaker-V1 prototype** in the existing `controlnet` pipeline:
+   - Mirror the `_load_controlnet_pipeline` path: add a PhotoMaker variant that
+     loads SDXL + canny ControlNet + PhotoMaker adapter on the same engine.
+   - Extract the OpenCLIP face embedding from the watermarked face crops (use
+     OpenCV YuNet, already bundled for `auto`, to find the face boxes).
+   - Pass the embedding as PhotoMaker's `id_embeds` to the SDXL pipeline; run
+     img2img at the certified strength (0.20 OpenAI, 0.30 Gemini-capped-1536)
+     with the canny edge map.
+3. **Oracle validation** on the cert sweep: run the new PhotoMaker variant
+   through `raiw-app/modal_cert.py` over the same 6 image set, certify on the
+   per-vendor oracles. Expected: SynthID cleared (the regeneration is the same)
+   AND identity recovered (the embedding adds it back).
+4. **Honest exit criteria.** Ship only if BOTH oracle reads clean AND a small
+   user-perception test on real uploads says "looks like me". If identity is
+   still too soft on small faces -> add stacked-reference (multiple crops of the
+   same upload at different scales) before reaching for a non-commercial
+   embedder.
+
+## 8. What we are NOT doing, and why
+
+- **No InsightFace.** Non-commercial for model packs (see License table).
+- **No CodeFormer.** Non-commercial.
+- **No GFPGAN on the original image.** It re-introduces SynthID
+  (oracle-confirmed).
+- **No GFPGAN on the cleaned image.** It cannot RECOVER identity that the
+  diffusion pass already drifted -- it can only smooth/sharpen whatever face is
+  already there. Useful as cosmetic polish, not as identity restoration.
+- **No retraining of an in-house ArcFace.** Out of scope for a removal service.
+
+---
+
+## Process note
+
+The deep-research harness was run but its verifier subagents failed to call
+`StructuredOutput` (same harness bug as the prior 2026-05-XX run), so its synthesis
+was unusable. The license claims above were verified by directly fetching the HF
+model cards and the InsightFace licensing page and quoting them; the
+embedding-invariance argument is mechanistic and explicitly flagged as not yet
+measured (it is the first integration step). Do not treat the deep-research
+output as ground truth for this file.

docs/synthid.md

@@ -0,0 +1,628 @@
+# SynthID-Image: technical reference
+
+This document covers how Google SynthID for images works mechanically, what it
+survives, what removes it, and the current deployment landscape. It is written
+for engineers working on watermark detection and removal -- specifically to
+inform decisions about strength settings, test methodology, and what oracle
+results mean.
+
+Primary sources are cited inline. Marketing-only claims are flagged separately
+from independently-verified results.
+
+---
+
+## 1. Mechanism
+
+### 1.1 Post-hoc, model-independent design
+
+SynthID-Image is **not** baked into a diffusion model's weights. It is a
+post-hoc, model-independent system: a separate encoder `f` is applied to an
+already-generated image, and a separate decoder `g` reads it back.
+
+> "We deliberately designed SynthID-Image as a post-hoc, model-independent
+> approach, a choice largely based on deployment considerations."
+> -- Gowal et al., arXiv:2510.09263
+
+The formal definition from the paper:
+
+> "A post-hoc watermarking scheme is a pair f, g consisting of an encoder
+> function f: X -> X, which adds an identification mark, and a decoder
+> function g: X -> {+-1}, which tries to detect if the mark is present."
+
+This is the key architectural fact: **the generative model (Imagen, Gemini's
+image model) is not modified**. The watermark is stamped onto the pixel output
+after generation, by a separate neural network. This means:
+
+- The watermark is in **pixel space**, not in the model's latent activations.
+- Replacing the generative model does not remove the watermarking capability.
+- The encoder/decoder pair can be updated independently of the generative model.
+
+The paper does not disclose the internal architecture of the encoder/decoder
+networks (layer types, capacity). The external variant SynthID-O is available
+to partners; the production internal variant is not published.
+
+### 1.2 How it differs from classical DWT-DCT watermarks
+
+The open watermarks used by Stable Diffusion / SDXL / FLUX (via the
+`imwatermark` library) use classical **DWT-DCT** frequency-domain embedding: a
+fixed bit pattern is added to specific frequency coefficients of the image's
+wavelet transform. This is fast, key-free, and locally detectable with a public
+decoder.
+
+SynthID-Image uses **jointly-trained deep learning models**:
+
+> "SynthID uses two deep learning models -- for watermarking and identifying --
+> that have been trained together on a diverse set of images. The combined model
+> is optimised on a range of objectives, including correctly identifying
+> watermarked content and improving imperceptibility by visually aligning the
+> watermark to the original content."
+> -- Google DeepMind blog, 2023
+
+The practical difference for robustness: the deep learning encoder learns to
+spread the signal across the image in a way that is optimized to survive a
+specific perturbation distribution seen during training. Classical DWT-DCT
+embeds in fixed, predictable frequency bins, making it brittle to any
+operation that hits those bins (e.g., JPEG re-quantization wipes it cleanly at
+quality <= 90).
+
+### 1.3 Payload capacity
+
+SynthID-O (the external/partnership variant) encodes:
+
+- **136 bits** within a **512x512 pixel image**
+
+For comparison (from the same paper):
+
+| Method      | Bits | Resolution |
+|-------------|------|------------|
+| SynthID-O   | 136  | 512x512    |
+| StegaStamp  | 100  | 400x400    |
+| TrustMark   | 100  | 256x256    |
+| WAM         | 32   | 256x256    |
+
+The payload carries an identification mark (not a user-readable secret). The
+paper separates watermark **detection** (is this watermarked?) from payload
+**recovery** (what does the payload say?): the detection path is what oracles
+like the Gemini app's "Verify with SynthID" exercise.
+
+### 1.4 Where in the pipeline it lives
+
+```
+[Diffusion model]
+       |
+  raw pixel output
+       |
+  [SynthID encoder f]   <-- separate neural net, stamps the watermark
+       |
+  watermarked image
+       |
+  [served / downloaded]
+       |
+  [SynthID decoder g]   <-- separate neural net, run by Google's verifier only
+       |
+  present / not present
+```
+
+The VAE decoder of the diffusion model is **not** involved in watermarking.
+Some in-generation watermark approaches (like the research method "Tree Ring")
+inject the signal into the initial noise latent so it propagates through the
+diffusion process and appears in the final image; SynthID-Image does not do
+this -- it is applied after the VAE has already decoded latents to pixels.
+
+---
+
+## 2. Robustness
+
+### 2.1 What the paper claims it survives (primary-source verified)
+
+The SynthID-Image paper (arXiv:2510.09263) evaluates SynthID-O against **30
+image transformations** grouped into 6 categories:
+
+| Category    | Examples                                      |
+|-------------|-----------------------------------------------|
+| Color       | brightness, contrast, saturation, hue shifts  |
+| Combination | combinations of multiple transforms           |
+| Noise       | Gaussian noise, impulse noise, median filter  |
+| Overlay     | text overlays, logos, stickers                |
+| Quality     | JPEG compression, WebP, format conversion     |
+| Spatial     | crop, resize, rotate, flip, padding           |
+
+**TPR at 0.1% FPR -- SynthID-O vs. baselines (resized to 512x512):**
+
+| Category         | SynthID-O | Best baseline (WAM) | Worst baseline (StegaStamp spatial) |
+|------------------|-----------|---------------------|--------------------------------------|
+| Identity (none)  | 100.00%   | 100.00%             | 100.00%                              |
+| Aggregated       | 99.98%    | 90.62%              | ~70%                                 |
+| Color            | 100.00%   | 81.29%              | ~75%                                 |
+| Combination      | 99.96%    | 96.08%              | ~22%                                 |
+| Noise            | 99.98%    | 100.00%             | ~92%                                 |
+| Overlay          | 100.00%   | 100.00%             | 100.00%                              |
+| Quality          | 99.99%    | --                  | ~89%                                 |
+| Spatial (worst)  | 99.97%    | 76.04%              | 15.25%                               |
+
+The "Spatial worst" row is the hardest case (aggressive crop + resize).
+SynthID-O retains 99.97% TPR; StegaStamp collapses to 15.25%. This is where
+the deep-learning approach gains the most over classical methods.
+
+Google's marketing page states the watermark is:
+
+> "designed to stand up to modifications like cropping, adding filters, changing
+> frame rates, or lossy compression."
+> -- deepmind.google/models/synthid/
+
+The marketing claim is broadly consistent with the paper's numbers for these
+specific categories.
+
+**JPEG and format conversion specifically** fall under the "Quality" category,
+where SynthID-O achieves 99.99% TPR. This is the empirical basis for the fact
+that **GitHub-recompressed JPEGs from issue attachments are valid SynthID test
+subjects**: the re-encoding does not remove the pixel watermark.
+
+### 2.2 Stated limits (vendor claim, not independently verified)
+
+> "SynthID isn't foolproof against extreme image manipulations."
+> -- Google DeepMind blog, 2023
+
+This is the only public failure-mode statement Google has made. No specific
+perturbation type, threshold, or quantitative boundary is named. The
+Limitations section of the paper (Section 10) was not recoverable from the
+public HTML version of arXiv:2510.09263v1 due to a rendering failure in the
+conversion (the body text of Section 10 is absent from the HTML).
+
+**What is known empirically from our own oracle-verified testing.**
+
+A controlled study (June 2026, clean v0.8.6 with text/face protection OFF,
+native resolution on this repo's default SDXL pipeline) measured the minimum
+img2img strength that removes the SynthID pixel watermark, verified per image on
+the vendor's own oracle (openai.com/verify for OpenAI, the Gemini app "Verify
+with SynthID" for Google). Each subject is archived in `data/synthid_corpus/` as a
+pos original plus its minimum-clearing cleaned output (manifest `verified_via` =
+`openai-verify` / `gemini-app`), EXCEPT one third-party image from issue #14, which
+was oracle-verified but is not committed (third-party content stays out of the
+public corpus).
+
+**Oracle validation order: start with OpenAI.** When validating removal across
+vendors, run the OpenAI arm first. `openai.com/verify` is more accessible than the
+Gemini app -- fewer per-check restrictions, so it gives the fastest signal and is
+the strongest candidate for automation (Playwright / Chrome MCP driving
+`openai.com/verify`); the Gemini "Verify with SynthID" flow is more manual. This is
+an ordering/throughput choice, not a substitution: each oracle only reads its own
+vendor's SynthID (`openai.com/verify` is OpenAI-scoped), so Google content still
+needs the Gemini app.
+
+| Vendor | Images | Resolution(s) | Pipeline | Removed at |
+|--------|--------|---------------|----------|------------|
+| OpenAI (gpt-image) | n=4 (3 archived + 1 external-only) | 1024x1536 .. 1600x1600 | native | **0.05** |
+| Google (Gemini)    | n=4 | 2816x1536 -> capped 1536 | `--max-resolution 1536` | **0.15** (0.05 and 0.10 do NOT clear) |
+
+**Two findings, both oracle-verified:**
+
+1. **Vendor is the dominant factor, not resolution.** Google's SynthID is
+   roughly 3x more robust than OpenAI's: at a comparable (small) working
+   resolution, OpenAI clears at 0.05 while Google needs 0.15. This matches
+   Google having hardened SynthID more aggressively over time.
+
+2. **OpenAI SynthID removal is resolution-independent in the tested range.**
+   All four OpenAI images (including a 1600x1600) cleared at 0.05.
+
+**CORRECTION (supersedes the earlier "resolution dependence" claim).** A prior
+version of this doc and CLAUDE.md stated that strength 0.30 failed to remove
+SynthID on 1600x1600 gpt-image and that removal was resolution-dependent. That
+was a **measurement artifact of a since-removed per-region re-scrub step** (issue
+#14): on the dense-text infographics tested, that step could reconstitute SynthID
+in text regions. Re-running the *same* 1600x1600 image on the clean current
+pipeline removes SynthID at **0.05**. The "large images resist removal" conclusion
+was false; the resistance was that region-rescrub shielding, since removed.
+
+**Open / not locally testable:**
+
+- **Native large Gemini (2816x1536, ~4.3 MP).** The Gemini floor of 0.15 was
+  measured on the *capped* (`--max-resolution 1536`) path, which is the
+  practical local route on Apple-Silicon (native 2816 OOMs / falls back to slow
+  CPU on a 32 GB M-series). Native large Gemini was not measured here; the
+  vendor and resolution effects would stack, so it plausibly needs >= 0.30 or a
+  discrete GPU. Confirm on a CUDA box if needed.
+- **Heavy JPEG compression** (quality < ~50-60): not oracle-tested; the DL
+  approach is more robust than DWT-DCT but Google acknowledges limits at
+  "extreme" manipulation.
+
+### 2.3 Removal attacks and forensic detectability
+
+The paper arXiv:2605.09203 ("Removing the Watermark Is Not Enough",
+Goonatilake & Ateniese, 2026) evaluates 6 removal attacks against a ResNet-50
+forensic detector. All attacks defeat the watermark verifier but are detected
+by the forensic classifier:
+
+| Attack          | Family           | AUROC  | TPR @ 1% FPR | TPR @ 0.1% FPR |
+|-----------------|------------------|--------|--------------|----------------|
+| UnMarker        | Distortion       | 0.9994 | 99.81%       | 98.28%         |
+| WatermarkAttacker| Regeneration    | 0.9997 | 99.95%       | 99.38%         |
+| CtrlRegen+      | Regeneration     | 0.9999 | 99.97%       | 99.64%         |
+| NFPA            | Inversion/Pert.  | 0.9984 | 99.24%       | 62.10%         |
+| Boundary Leak.  | Inversion/Pert.  | 0.9991 | 99.24%       | 88.34%         |
+| WiTS            | Erosion          | 0.9999 | 99.80%       | 99.55%         |
+
+The forensic detector is a standard ResNet-50 fine-tuned end-to-end; no exotic
+architecture needed. The key finding:
+
+> "These removers do not return images to a clean forensic state. They often
+> trade an explicit watermark for an implicit watermark: a detectable artifact
+> introduced by the removal process itself."
+
+This means: even when our SDXL img2img pass defeats the SynthID pixel
+watermark (oracle reads negative), the output may still be classifiable as
+"an image that went through a removal pipeline" by an independent detector --
+even if that detector is not trained on SynthID specifically. **Defeating the
+verifier does not restore forensic deniability.**
+
+CtrlRegen+ is the most detectable removal method (AUROC 0.9999), which is
+notable because it is also the most powerful removal attack. The paper notes
+that diffusion regeneration "leaves a strong reconstruction signature from the
+diffusion prior."
+
+---
+
+## 3. Detectability and verifier access
+
+### 3.1 No public local detector
+
+The SynthID decoder is proprietary and not released:
+
+> "SynthID-Image has been used to watermark over ten billion images and video
+> frames across Google's services and its corresponding verification service is
+> available to trusted testers."
+> -- Gowal et al., arXiv:2510.09263
+
+There is no public API, no released decoder weights, and no reproducible
+algorithm for local detection. The verification service (SynthID Detector) is:
+
+> "a verification portal" in early testing with "journalists and media
+> professionals" on a waitlist
+> -- deepmind.google/models/synthid/
+
+The external variant SynthID-O is available "through partnerships" only. Our
+tool cannot locally detect SynthID presence or absence -- this is by design,
+not a gap we can fill.
+
+### 3.2 How our tool detects SynthID (metadata proxy)
+
+We detect SynthID indirectly: if the image's C2PA manifest is signed by a
+known SynthID-using issuer (Google, OpenAI), we infer SynthID is present. This
+is a **metadata proxy**, not a pixel watermark decode. It works while the C2PA
+manifest is intact, and is silent once the manifest is stripped or the image
+is re-encoded without C2PA (e.g., a screenshot, a social-media re-upload, or
+after `metadata --remove`).
+
+This is why:
+- `identify` on a GitHub-recompressed issue attachment returns Unknown (C2PA is
+  gone) even though the pixel SynthID is still present and detectable by
+  openai.com/verify.
+- A quiet `identify` output is not proof that SynthID was removed -- it only
+  means the metadata signal is gone.
+
+### 3.3 Oracle scope: each vendor detects only their own
+
+From openai.com/research/verify (verbatim, verified 2026-05-31):
+
+> "OpenAI generation signals will only be detected if the image was generated
+> with our tools."
+> "Content could also still be AI-generated by another company's model, which
+> the tool currently does not detect."
+
+SynthID technology is used by multiple vendors, but each verifier is keyed to
+its own payload:
+
+| Oracle                        | Detects          | Does NOT detect         |
+|-------------------------------|------------------|-------------------------|
+| Gemini app "Verify with SynthID" | Google SynthID | OpenAI SynthID          |
+| openai.com/research/verify    | OpenAI SynthID   | Google SynthID          |
+
+A Google-SynthID image reads clean on openai.com/verify. An OpenAI image reads
+clean in the Gemini oracle. They are different payloads within the same
+framework.
+
+---
+
+## 4. Adoption and current state (as of June 2026)
+
+### 4.1 Google products
+
+Google has watermarked **over 10 billion** images and video frames. The
+deployment split by surface matters for our tool:
+
+| Surface                              | SynthID pixel | C2PA metadata | Visible sparkle |
+|--------------------------------------|---------------|---------------|-----------------|
+| Gemini app (generated images)        | YES           | YES (Google)  | YES             |
+| Gemini API / AI Studio / Nano Banana | YES           | NO            | YES             |
+
+The Gemini API surface is a key blind spot: it embeds the pixel watermark and
+the visible sparkle but **no C2PA or IPTC at all**. Our `identify` returns
+Unknown on API-generated images unless the visible sparkle is detected (via
+`check_visible=True`) or the user runs the Gemini app oracle.
+
+### 4.2 OpenAI
+
+OpenAI confirmed SynthID adoption (Help Center, updated 2026-05-21):
+
+> "ChatGPT images include both C2PA metadata and SynthID watermarks."
+
+This is time-gated: pre-rollout ChatGPT/gpt-image images carry C2PA without
+SynthID. Our C2PA proxy therefore over-reports SynthID presence on old images
+(hence the `_OPENAI_CAVEAT` hedging flag in the codebase).
+
+### 4.3 Other vendors
+
+- **Kakao** (South Korea): SynthID adopter as of May 2026 (Google announcement)
+- **NVIDIA Cosmos**: SynthID for video (not still images; different pipeline)
+- **Meta AI**: does NOT use SynthID; uses IPTC `digitalSourceType` marker instead
+
+### 4.4 Version evolution (v1 vs v2 hardening)
+
+Google has not publicly documented version numbers for the SynthID image
+watermark in a way that maps to our testing observations. What is known
+empirically from oracle tests:
+
+- **Before May 2026 (Gemini)**: strength 0.05 removed the watermark
+- **May 2026 (Gemini)**: strength 0.05 insufficient; 0.10 required
+- **Current (Gemini, June 2026)**: on the capped 1536 path, 0.05 and 0.10 do
+  NOT clear; 0.15 clears (n=4, Gemini app oracle). See section 2.2.
+- **OpenAI (June 2026)**: clears at 0.05 across 1024-1600 (n=4, clean v0.8.6).
+  The earlier "0.30 still detected on 1600x1600" report (issue #14) was the
+  text-protection bug, not a hardening of the watermark -- see the correction in
+  section 2.2.
+
+Google has hardened SynthID relative to OpenAI's (vendor gap measured at ~3x
+strength), but the year-over-year "0.05 -> 0.10 -> 0.30" progression above
+conflates a real hardening trend with the now-debunked region-rescrub artifact;
+treat only the section 2.2 controlled numbers as authoritative.
+
+---
+
+## 5. Practical implications for this tool
+
+### 5.1 Preserving content means regenerating it, never copying it
+
+**Core rule:** SynthID is a pixel-amplitude pattern, so any approach that FREEZES
+or RESTORES original pixels in a region re-introduces the watermark there. Early
+region-based text/face "protection" (since removed) proved this: restoring the
+original face pixels guaranteed SynthID survived in faces, and even a per-region
+high-resolution re-scrub from an upscaled crop could be insufficient to destroy
+the payload, reconstituting SynthID in text. The lesson held and shaped the
+current design: **content is preserved by REGENERATING it under structural
+conditioning, never by copying original pixels.**
+
+- **Text + structure:** `--pipeline controlnet` (SDXL img2img + a canny ControlNet) is
+  **THE DEFAULT pipeline since 2026-06-09** (`--pipeline default` opts down to plain
+  SDXL img2img for inputs without text/faces). It conditions the regeneration on the
+  edge map, so text and structure stay sharp while every pixel is still regenerated. Text legibility is
+  better than plain img2img at the same strength (text stays readable where plain
+  garbles it). **BUT removal efficacy at the low vendor-adaptive strength is CONTENT ×
+  PIPELINE dependent and NEITHER pipeline clears all content -- oracle-validated
+  2026-06-04 (8 OpenAI images, strength 0.10/0.15, max-res 1536).** The survivors FLIP
+  by content type: **photoreal** (a 9-face grid, a bracelet product photo) SURVIVES
+  controlnet but CLEARS `default`; **flat graphic** (a logo/poster with large flat
+  color fills) SURVIVES `default` but CLEARS controlnet; a flat **text** card cleared
+  under both. Why: controlnet's dense edge map keeps the regen too close to the
+  original on photoreal (so SynthID survives) but freely repaints flat fills (so it
+  clears them); plain img2img at low strength perturbs photoreal texture enough but
+  barely touches flat fills. **Root cause = insufficient STRENGTH, not the pipeline:
+  the vendor-adaptive 0.10 is NOT universally sufficient (the June numbers below held
+  for the content they were measured on). The robust fix is a HIGHER strength,
+  oracle-revalidated per content type (controlnet can be cranked harder without losing
+  structure; a lower `controlnet_conditioning_scale` also frees the regen on
+  photoreal).** So neither `--pipeline controlnet` nor plain `default` is a drop-in
+  removal guarantee at today's strength -- pick by what you must PRESERVE (controlnet
+  for text/structure), then raise strength until the oracle reads clean. (The earlier
+  "reads clean on the oracle" claim held only for the one flat/text-background case it
+  was checked on; it does not generalize.) **UPDATE 2026-06-09: the default strengths
+  were raised and made pipeline-aware (controlnet ladder = the certified
+  0.20/0.30/0.30 floors, applied to BOTH pipelines as a single ladder -- see §5.2 for
+  why one ladder covers plain `sdxl` too) and controlnet is now the default pipeline.
+  The plain-SDXL profile was also renamed `default` -> `sdxl` (`default` stays as an
+  alias). The 0.10/0.15 numbers in this analysis are the PRE-raise values it was
+  measured at. See §5.2.**
+- **Face identity:** canny holds face *structure* but not *identity*. Shipped as the
+  optional `--restore-faces` GFPGAN post-pass (`face_restore.py`, the `restore`
+  extra, experimental/opt-in, off by default). It runs GFPGAN on the ORIGINAL
+  faces and feather-composites the restored face REGIONS into the cleaned image.
+  **WARNING (oracle-confirmed 2026-06-04): this pass can RE-INTRODUCE SynthID into
+  the face regions -- the earlier "GFPGAN re-synthesizes from a StyleGAN2 prior ->
+  scrubs SynthID -> oracle-confirmed clean" claim was WRONG.** At the default fidelity
+  weight `0.5` GFPGAN blends ~half the ORIGINAL (watermarked) face pixels with the
+  prior, and SynthID is robust to that partial blend, so the composited face carries
+  the watermark back in -- over the diffusion-cleaned face. Confirmed by a clean A/B:
+  `gemini_3` read SynthID-detected after controlnet @ 0.20/0.25 WITH restore, but
+  NOT-detected after the same controlnet @ 0.20 with `--no-restore-faces` (only
+  restore differed). Content-dependent (a second face image cleared WITH restore),
+  which is why a single-image check earlier read "clean". **Fix directions (not yet
+  done): run GFPGAN on the diffusion-CLEANED image not the original; or drop the
+  weight well below 0.5; or leave restore OFF for removal -- each needs oracle
+  re-validation.** Commercial-
+  safe (GFPGAN Apache-2.0 + RetinaFace MIT); the CodeFormer alternative is
+  NON-COMMERCIAL and is not shipped. (An IP-Adapter FaceID approach was tried and
+  REMOVED -- it needs high denoise strength and corrupts faces at removal strength;
+  see `docs/controlnet-removal-pipeline-research.md`.)
+
+### 5.2 Strength setting
+
+There is no single permanent correct strength, but the controlled June 2026
+study (section 2.2) gives empirical floors:
+
+- **OpenAI**: 0.05 clears across 1024-1600 (n=4) -- **but content-dependent, NOT
+  universal.** The follow-up oracle pass (2026-06-04, 8 images) found a flat-graphic
+  OpenAI logo/poster still SynthID-detected after `default` at 0.10, and photoreal
+  images still detected after controlnet at 0.10/0.15: at low strength the
+  low-change regions (large flat fills under `default`, dense edges under controlnet)
+  are not perturbed enough. So the 0.05 floor held only for the n=4 content it was
+  measured on; treat it as a lower bound, not a guarantee, and raise + oracle-recheck
+  per content type (see §5.1 controlnet bullet).
+- **Google (capped 1536)**: 0.15 (n=4); 0.05 and 0.10 do not clear.
+- **Google native 2816**: not locally measured; likely needs >= 0.30 (vendor +
+  resolution stack). Use a GPU or `--max-resolution 1536`.
+
+The default is **vendor-adaptive** (`watermark_profiles.resolve_strength` +
+`vendor_for_strength`): the tool reads the C2PA issuer on the original input and picks
+`OPENAI_STRENGTH` 0.10 / `GEMINI_STRENGTH` 0.15 / `UNKNOWN_STRENGTH` 0.15 **(LOWERED
+2026-06-14 from the 2026-06-04 cert floors 0.20/0.30/0.30)**. **The SAME ladder applies
+to both pipelines** (`sdxl` and `controlnet`). The 2026-06-14 re-test on the deployed
+Modal controlnet worker (v0.10.0) cleared SynthID on the oracle at OpenAI 0.10 (2
+photoreal) and Google 0.15 (2 NATIVE 2816x1536, contradicting the "native >= 0.30" guess
+on line above), and a pixel sweep showed 0.20/0.30 over-regenerated for no efficacy gain.
+**This re-opens a genuine tension with the 2026-06-04 pass, which found photoreal STILL
+detected after controlnet at 0.10/0.15 (lines above):** either the v0.10.0 controlnet
+default improved the floor, or n=2 landed on the lucky side of the seed-non-determinism
+(§5.5). So a SERVICE on this ladder MUST pin a fixed, oracle-verified seed (not random),
+and flat-graphic hard cases (NOT in the n=2 re-test) still need a per-content oracle
+recheck -- raise `--strength` there. The prior cert floors are the §5.5 record. Why one ladder
+covers plain `sdxl` too: the certification was run on controlnet and does NOT transfer
+by symmetry (the two pipelines have OPPOSITE hard cases -- controlnet leaves SynthID on
+photoreal, `sdxl` on flat graphics, the §5.1 content-x-pipeline table), BUT on its own
+hard case (flat fills) `sdxl` is the WEAKER remover (plain img2img barely perturbs a
+flat region at low strength), so it needs AT LEAST controlnet's strength -- the
+certified floor is therefore the right floor for `sdxl` too. This is a MARGIN argument
+for `sdxl`, not a separate certification (no local SynthID detector to self-verify).
+The higher strength costs little quality where it matters, because `controlnet` is now
+the default pipeline, so `sdxl` is reached only via an explicit `--pipeline sdxl` (a
+deliberate opt-down), where over-regeneration has no faces/text to damage.
+This uses the vendor signal we DO have locally (the C2PA SynthID proxy) to avoid the
+overkill of a single high default on OpenAI images, without needing a local pixel
+detector. An explicit `--strength` always wins. If the watermark still survives (e.g. a
+large native Gemini beyond the capped-1536 validation), raise toward 0.35-0.40 (0.40
+visibly corrupts dense text), using the lowest value that reads clean on the oracle.
+
+### 5.3 Test methodology
+
+- **GitHub-recompressed JPEGs from issue attachments are valid SynthID test
+  subjects.** JPEG re-encoding removes C2PA metadata but does NOT remove the
+  SynthID pixel watermark (verified June 2026 on issue #14 pic3). Do not
+  dismiss these as "not faithful originals" for SynthID-removal tests.
+- **The correct oracle for OpenAI images is openai.com/verify**, not the Gemini
+  app. The two oracles detect different payloads.
+- **A quiet `identify` output after processing is not proof of removal.** It
+  means the metadata proxy is gone. The pixel watermark state is unknown without
+  an oracle check.
+- **After removal, the output may carry forensic artifacts** detectable by an
+  independent classifier even if the vendor oracle reads negative. Defeating the
+  verifier is not the same as being forensically indistinguishable from clean
+  content (arXiv:2605.09203).
+
+### 5.4 Strength vs forensic detectability: the tradeoff
+
+Higher img2img strength removes the watermark but introduces detectable
+regeneration artifacts. The Goonatilake & Ateniese paper shows the strongest
+diffusion-based removers are simultaneously the most forensically detectable
+(AUROC up to 0.9999). The tradeoff is unavoidable with current diffusion-based
+approaches: defeating the vendor's verifier is not the same as being clean.
+
+### 5.5 Oracle validation log -- 2026-06-04 OpenAI pass
+
+Eight OpenAI `gpt-image` originals run through both pipelines and checked on
+openai.com/verify (the OpenAI SynthID oracle). `--max-resolution 1536`; strength
+is the vendor-adaptive default (`vendor_for_strength`): images with an OpenAI C2PA
+manifest get `OPENAI_STRENGTH` 0.10, the one without C2PA falls to
+`UNKNOWN_STRENGTH` 0.15. "detected" = SynthID still found (removal FAILED);
+"clean" = SynthID not detected.
+
+| image | content type | size | strength | `--auto`/controlnet | `default` |
+|---|---|---|---|---|---|
+| typography card | flat text | 1122x1402 | 0.10 | clean | clean |
+| raiw.cc poster | flat graphic (logo + flat fills) | 1024x1536 | 0.10 | clean | **detected** |
+| 9-face grid | photoreal | 1448x1086 | 0.10 | **detected** | clean |
+| bracelet product photo | photoreal | 1600x1600 | 0.15 | **detected** | clean |
+
+(The other four cleared under both and are omitted.) **Reading:** at this strength
+NEITHER pipeline removes SynthID on all content -- the survivors flip by content
+type. Photoreal survives controlnet / clears `default`; flat graphic survives
+`default` / clears controlnet; flat text clears both.
+
+**Follow-up: removal near the threshold is NON-DETERMINISTIC (seed-dependent).**
+Re-running the two photoreal survivors through controlnet at an explicit
+`--strength 0.15` (`--auto`, same `--max-resolution 1536`) cleared BOTH on the
+oracle (SynthID not detected). But the bracelet had SURVIVED controlnet at the
+SAME 0.15 in the first pass (it was the no-C2PA image, so its vendor-adaptive
+strength was already 0.15) -- same pipeline + strength + resolution, only the
+random (unset) seed differed between runs. So **0.15 is the borderline floor for
+controlnet photoreal, not a robust guarantee**: at the threshold the same
+image+settings can pass or fail run-to-run. img2img runs with `seed=None` (random)
+unless `--seed` is passed, so a removal SERVICE gets a coin-flip near threshold and
+has no local SynthID detector to self-verify.
+
+**Controlnet strength ladder on the two photoreal images (oracle, `--auto`,
+`--max-resolution 1536`):**
+
+| controlnet strength | 9-face grid | bracelet photo |
+|---|---|---|
+| 0.10 | detected | (was 0.15) |
+| 0.15 | clean | **non-deterministic** (survived pass 1, clean pass 2) |
+| **0.20** | **clean** | **clean** |
+
+**Recommended robust controlnet strength = 0.20** (0.05 of margin above the 0.15
+non-deterministic borderline); both photoreal survivors cleared at 0.20. Honest
+caveat: 0.20 is one confirming run WITH margin, not an N-run repeatability proof --
+for a removal service, add a little more margin or validate repeatability, since
+there is no local SynthID detector to self-check. **Implications:** (1) the
+content×pipeline table above conflates a borderline/non-deterministic 0.15 result
+with deterministic content behavior -- the photoreal-survives-controlnet effect is
+solid at 0.10 but at 0.15 it is near-threshold noise; (2) for reliable removal pick
+a strength with MARGIN above the borderline (controlnet >= 0.20), not exactly on
+it; (3) **engineering follow-up for raiw.cc: the controlnet pipeline should use a
+HIGHER vendor strength than `default` (it currently shares `resolve_strength`) --
+e.g. controlnet floor 0.20 -- calibrated per vendor/content on the GPU worker where
+batches are cheap. The shared 0.10/0.15 is tuned for `default`, not controlnet.**
+Source images are private (faces / product shots), not committed; reproduce on any
+photoreal + flat-graphic gpt-image pair, varying the seed, and re-checking the
+oracle.
+
+**Gemini pass + the face-restore re-introduction (2026-06-04).** Four Gemini
+originals via `--auto` (controlnet) at `--max-resolution 1024`, checked on the
+Gemini "Verify with SynthID" oracle (Google content needs the Google oracle, not
+openai.com/verify):
+- Most cleared at controlnet 0.15-0.25; `gemini_3` (a large central FACE, +restore)
+  stayed **SynthID-detected at controlnet 0.15, 0.20 AND 0.25** -- raising strength
+  did not crack it.
+- **Root cause was the face-restore pass, not strength/resolution.** `gemini_3` at
+  controlnet 0.20 with `--no-restore-faces` read **SynthID-NOT-detected** (clean
+  A/B, only restore differed). GFPGAN runs on the ORIGINAL watermarked face and at
+  weight 0.5 blends ~half its pixels back, re-introducing SynthID into the
+  composited face over the diffusion-cleaned result (see §5.1 face-identity bullet).
+- (Side note: reducing the processing resolution does NOT weaken SynthID -- it is
+  robust to downscaling by design, and the study's resolution trend says LOWER
+  processing res needs LESS strength, so 1024 was never the wall.)
+
+**Certified controlnet floors (Modal GPU sweep `raiw-app/modal_cert.py` + oracle,
+restore OFF, <= 1536, each vendor on its own oracle):** OpenAI **0.20** (2 photoreal x
+seed {1,2,3} = 6/6 clean; the 0.15-flipper is seed-robust at 0.20) and Gemini **0.30**
+(0.20 detected -> 0.30 clean on 2/2 seeds). OpenAI 0.20 transfers to prod
+(resolution-independent); Gemini 0.30 holds only <= 1536 -- Gemini is
+resolution-sensitive and raiw.cc runs NATIVE, so cap Gemini <= 1536 + use 0.30 or
+native-calibrate (~0.35+). See `docs/controlnet-removal-pipeline-research.md` for the
+table.
+
+**Net for raiw.cc:** (1) controlnet needs a higher, per-vendor strength than
+`default` -- CERTIFIED OpenAI 0.20 / Gemini 0.30 (above); add a controlnet-specific
+schedule to `resolve_strength`, do not reuse the default ladder; (2) the
+`--restore-faces` pass is now SynthID-safe by construction (the GFPGAN-on-original
+path that re-added SynthID was removed 2026-06-04; the shipped restore is
+PhotoMaker-V2, NON-COMMERCIAL, see `photomaker_restore.py`); (3)
+removal near threshold is seed-non-deterministic -> FIX the prod seed (kills the
+coin-flip; ship a deterministic certified config).
+
+---
+
+## References
+
+1. Gowal et al. (2025). **SynthID-Image: Image watermarking at internet scale.**
+   arXiv:2510.09263. https://arxiv.org/abs/2510.09263
+
+2. Google DeepMind. **Identifying AI-generated images with SynthID.** Blog post,
+   2023. https://deepmind.google/blog/identifying-ai-generated-images-with-synthid/
+
+3. Google DeepMind. **SynthID.** Product page.
+   https://deepmind.google/models/synthid/
+
+4. Goonatilake & Ateniese (2026). **Removing the Watermark Is Not Enough:
+   Forensic Stealth in Generative-AI Watermark Removal.** arXiv:2605.09203.
+   https://arxiv.org/abs/2605.09203
+
+5. OpenAI. **Verify tool for AI-generated images.** openai.com/research/verify.
+   Accessed 2026-05-31.

docs/text-protection-research.md

@@ -14,8 +14,10 @@ filter applied to every candidate below.
 
 ## Problem recap
 
-The `invisible` pipeline is SDXL base 1.0 img2img at low strength (~0.05) to defeat
-SynthID with minimal visible change. Text is protected via Differential Diffusion with a
+The `invisible` pipeline is SDXL base 1.0 img2img to defeat SynthID. The default
+strength has risen over time as Google hardens SynthID (0.05 -> 0.10 -> **~0.30**, the
+current threshold for fresh Gemini output); higher strength deforms text more, which is
+exactly why text protection matters. Text is protected via Differential Diffusion with a
 per-pixel change map (`preserve` ~0.9) driven by the PP-OCRv3 DB detector
 (`text_protector.py`). Large text survives; **small text (sub ~8 px strokes) softens or
 garbles** (issue #14, confirmed on real content).

docs/verification-plan.md

@@ -0,0 +1,996 @@
+# Full verification plan
+
+How we convince ourselves the library actually works, across its whole surface, on real data.
+
+This is the pre-release and periodic-audit plan. It is deliberately organized by **oracle
+strength** rather than by module, because the hard part is never "call the function" -- it
+is "know what the right answer was". A sweep with no oracle proves only that nothing threw.
+
+Measured throughput on the local corpus (39,430 images, M-series, 2026-07-19):
+
+| path | per image | full corpus, 8 procs |
+|---|---|---|
+| `identify` | 0.58 s | ~0.8 h |
+| `detect_marks` | 0.28 s | ~0.4 h |
+| visible remove (cv2) | 0.58 s | ~0.8 h |
+| diffusion @512px (MPS) | ~50 s | ~23 days -- sample only |
+
+So every CPU path is affordable at FULL corpus scale; only the diffusion paths need
+sampling. Plan accordingly: never sample where a full sweep costs an hour.
+
+## Data sources
+
+| source | size | committed | role |
+|---|---|---|---|
+| `data/spaces/originals/` | 39,430 imgs, 87.5 GB | no (gitignored) | the real-upload corpus |
+| `data/spaces/identify/` | 39,314 JSON sidecars | no | **recorded `identify` verdict per image** |
+| `data/spaces/_visible_datasets/` | 3,741 imgs, per vendor | no | mark-positive pools |
+| `data/synthid_corpus/` | 39 imgs, labelled | yes | pos/neg/cleaned SynthID references |
+| `data/samples/` | 11 fixtures | yes | deterministic fixtures |
+| `data/*_capture/` | vendor captures | yes | detection silhouettes |
+| synthesized | generated | no | constructed ground truth (tier B) |
+
+**Data safety.** The corpus is user uploads: local analysis only. No run may copy, promote,
+or commit corpus images into a tracked path, and no report may embed them. All harness
+output goes to gitignored paths under `data/spaces/`.
+
+## Tier A -- self-evident oracles (full corpus, unattended)
+
+Properties that are true or false without anyone labelling anything. These are the
+backbone: they scale to 39k images and catch regressions with zero human cost.
+
+### A1. Sidecar regression -- the highest-value check we are not running
+
+`data/spaces/identify/` holds 39,314 recorded `identify` verdicts, keyed by the same uid as
+the image. Re-running `identify` today and diffing against them turns the corpus into a
+**39k-image behavioral regression suite for free**. Any drift in verdict, platform,
+confidence or signal set shows up as a diff, bucketed by cause.
+
+Caveat that makes this honest: a diff is not automatically a bug -- the sidecars were
+written by older versions, so intended improvements also show up. The output is therefore a
+**classified diff** (new detections / lost detections / changed platform / changed
+confidence), reviewed once, then re-baselined. Lost detections are the alarm.
+
+Implemented as `scripts/sidecar_regression.py` (resumable, ~1.5 h at 8 workers).
+
+#### First full run, 2026-07-19, all 39,314 sidecars
+
+| class | n | share |
+|---|---|---|
+| unchanged | 37,326 | 94.9% |
+| lost_signal | 1,302 | 3.3% |
+| platform changed | 922 | 2.3% |
+| confidence changed | 899 | 2.3% |
+| new_signal | 694 | 1.8% |
+| lost_ai | 747 | 1.9% |
+| new_ai | 152 | 0.4% |
+
+Two results worth keeping:
+
+**No metadata signal regressed anywhere.** Lost families were exclusively visual
+(`visible_sparkle` 1,256, `visible_doubao` 45, `visible_jimeng` 4) -- zero c2pa, synthid,
+aigc_tc260, iptc, exif_generator or xai_signature losses across the whole corpus. And
+`identify` raised on **none** of the 39,314 real uploads.
+
+**The sparkle losses are mostly corrected false fires, but not entirely.** Sampling 400 of
+the 1,256 and checking whether the file still carries Google provenance independently of
+the sparkle: 12.5% (95% CI 9.6-16.1%) still do, i.e. ~120-200 corpus-wide are **genuine
+misses**; the other ~1,050-1,135 had nothing backing them. Read the split as a trade the
+FP-gate tightening made, not as a clean win.
+
+Caveat on that split: "no Google provenance" is not proof of a false positive -- a
+metadata-stripped Gemini screenshot also has none while still carrying the pixel sparkle.
+So 87.5% is an **upper bound** on false fires; only the 12.5% genuine-miss figure is solid.
+
+Doubao moved the other way (-45 / +628 net +583), which is the `scale_basis` landscape fix
+showing up at corpus scale. `trustmark` +38 and `open_invisible` +9 are not behavior: those
+extras were simply not installed when the sidecars were written.
+
+### A2. Parity: whatever we detect, we must be able to remove
+
+For every image where a signal fires: remove, re-scan with the same oracle, assert quiet.
+- metadata: `scripts/metadata_removal_audit.py` (exists) -- run full corpus.
+- visible: `scripts/visible_removal_audit.py` (exists) -- run **once per backend**
+  (cv2 / migan / lama). It is single-process, so a full-corpus sweep is ~10 h and three
+  backends ~30 h. Its expensive half is DETECTION, which does not depend on the backend,
+  so run `scripts/visible_positives.py` once (parallel, ~40 min) and feed the result to
+  the audit's `--paths-file` seam: a few thousand images per backend instead of 39k.
+
+#### Metadata parity, first full run, 2026-07-19 (20,153 carriers + 1,500 clean controls)
+
+Zero scan/strip/decode errors. Survival after strip:
+
+| signal | carriers | survived |
+|---|---|---|
+| c2pa_manifest / claim_generator | 15,410 | **3** |
+| synthid_watermark | 14,985 | 0 |
+| aigc_label | 4,414 | 0 |
+| the other 12 signal types | - | 0 |
+
+The no-op control is clean: the strip **added** a signal to 0 of 1,500 clean images. Two
+real defects fell out of the run.
+
+**Defect 1 -- the fail-safe reports success on a file it did not strip.** All 3 parity
+failures are Samsung Galaxy S22 camera PNGs (`Galaxy S22 c2pa-rs/0.37.0`) whose `caBX`
+chunk survives. Cause: PIL raises `UnidentifiedImageError` on them, so
+`remove_ai_metadata`'s fail-safe copies the file through byte-identical -- correct in
+intent (never crash a worker on a partial upload) but it returns an output path
+indistinguishable from a real strip. User-visible: `metadata --remove` prints
+"AI metadata stripped ->", exits 0, and `identify` on the output still reports C2PA. The
+warning is logged but the success line contradicts it. Rare here (3 of 20,153) but the
+mechanism fires on ANY file PIL cannot decode. The fail-safe should stay; what needs
+fixing is that the caller cannot tell a no-op from a strip.
+
+**Defect 2 -- 16-bit PNGs are silently downconverted to 8-bit.** 5 of the 1,500 clean
+controls failed the pixel-identity check; all are 16-bit PNGs, and the PIL re-save halves
+their bit depth (one went 9.2 MB -> 2.5 MB). This is the known limitation recorded in
+CLAUDE.md, now measured: a byte-level IHDR scan over every corpus PNG puts it at
+**42 of 27,018 (0.16%)**.
+
+Method note worth keeping: the first attempt to reproduce Defect 2 said "pixels
+identical" and nearly closed it as a harness bug. That check read both files through
+`image_io.imread`, which returns 8-bit -- **the reader destroyed the very property under
+test**. The audit was right because it reads via `read_bgr_and_alpha`, which preserves
+uint16. When verifying a fidelity property, check that the verification path can still
+represent it.
+
+### A3. Byte-level invariants
+
+- no-op `remove_visible` returns the ORIGINAL bytes (not a re-encode)
+- pixels outside the fill mask are bit-identical to the input
+- JPEG metadata strip is pixel-lossless on the DEFAULT path (`--remove-all` re-encodes by
+  design -- see `metadata.py`; assert the split, not losslessness everywhere)
+- lossless source formats survive a misnamed extension
+
+### A4. Idempotence and order-independence
+
+- `remove_visible(remove_visible(x)) == remove_visible(x)`
+- `strip(remove(x)) == remove(strip(x))` in signal terms
+- a second `identify` on a cleaned output reports no metadata signals
+
+### A5. Contract sweep across every parameter choice
+
+`scripts/smoke_matrix.py` (exists, 68 rows, 0 skipped with `--diffusion`) covers every
+choice-valued flag on fixtures. Extend from fixtures to a stratified corpus slice
+(~500 images spanning format x provenance x aspect ratio), asserting exit-code semantics
+rather than just absence of crash.
+
+Known trap to encode: **exit 2 is triply overloaded** (no-visible-mark, no-invisible-signal,
+Click usage error). A wrapper cannot distinguish them without parsing stderr. Either the
+sweep asserts on stderr, or the codes get split -- the latter is the better fix.
+
+#### Coverage before the extension (measured 2026-07-19, not estimated)
+
+Comparing the flags the matrix actually executed against the flags the CLI declares:
+**18 of 38 had never been executed even once** -- `--pipeline`, `--strength`, `--steps`,
+`--guidance-scale`, `--device`, `--model`, `--upscaler`, `--tile`/`--tile-size`/
+`--tile-overlap`, `--humanize`, `--unsharp`, `--adaptive-polish`, `--controlnet-scale`,
+`--min-resolution`, `--hf-token`, `--auto`, `--verbose`. Plus uncovered VALUES of covered
+flags: `--backend migan|lama` had never been driven through the CLI at all (only at
+library level), `erase --backend` only ever ran `cv2`, `batch --mode all` never ran, and
+`--pipeline` only ever ran its default.
+
+Whole subsystems had unit tests but no real-data run: tiling (27 unit tests, never
+processed a real image through the CLI), the region-targeted composite, the ESRGAN
+upscaler, and the ffmpeg audio/video strip. The gap is not "logic untested" but
+"never executed on real data", which is precisely what this campaign is for.
+
+#### Bug found by the extension: `--steps` below ~7 crashes inside torch
+
+Effective timesteps are `int(steps * strength)`. At the vendor-adaptive default strength
+(0.15, or 0.10 for OpenAI) any `--steps` under 7 rounds to **zero**, and the pipeline dies
+with a raw traceback:
+
+```
+$ remove-ai-watermarks invisible img.png --steps 5
+RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1, 1, 512]
+```
+
+Fully valid CLI arguments, no special flags, no `--force`. The value is accepted, the
+crash is a torch internal, and nothing tells the user that steps and strength interact.
+Fix is either a clamp to >=1 effective step or an up-front validation naming both values.
+
+Method note: the first run of the knob rows failed 12 times with this identical error,
+which read like twelve broken features. It was one bad harness parameter (`--steps 4`)
+sitting on top of one real bug. An error that is IDENTICAL across unrelated rows is
+evidence of a common cause, not of many faults -- check the shared input first.
+
+## Tier B -- constructed ground truth (automatable, no labelling)
+
+Where reality gives no answer key, build one. This is the tier that closes the two biggest
+holes: fill quality has **never** been asserted, and recall was measured once at n=240.
+
+### B1. Fill quality with a true reference
+
+Real marks have no clean counterpart, so quality has only ever been eyeballed. Construct it
+instead: take a clean corpus image, stamp a known mark at a known position (the captured
+alpha maps make this exact), remove it, and compare against the **true original**.
+
+Yields PSNR / SSIM per `--backend` (cv2 / migan / lama), sliced by background class, which
+is exactly the axis where the docs say quality varies but no number exists.
+
+Implemented as `scripts/fill_quality.py`. Two reporting rules are load-bearing: score
+INSIDE the footprint (whole-frame PSNR sits near 60 dB whatever the backend does), and use
+the MEDIAN (a fill that reproduces a flat background exactly scores PSNR=inf, and one inf
+makes a mean inf -- the first run reported "+inf" for every flat bucket).
+
+#### First run, 2026-07-19, 80 verified-clean sources, n=720 measurements
+
+Median dB recovered inside the footprint (filled PSNR minus damaged PSNR):
+
+| mark | bg | cv2 | migan | lama |
+|---|---|---|---|---|
+| doubao | flat | +10.79 | **+15.62** | +13.97 |
+| doubao | mid | +9.25 | **+14.71** | +14.69 |
+| doubao | textured | +2.15 | +0.66 | **+1.64** |
+| jimeng | flat | +5.52 | +8.80 | **+10.30** |
+| jimeng | mid | +10.78 | +11.05 | **+12.33** |
+| jimeng | textured | +2.16 | +1.70 | **+3.03** |
+
+**The `auto` order is CONFIRMED.** Median per-image recovery on the textured tercile:
+
+| mark (textured) | cv2 | migan | lama |
+|---|---|---|---|
+| doubao | +0.06 | +1.53 | +1.38 |
+| gemini | +1.79 | +2.75 | **+5.59** |
+| jimeng | +1.24 | +2.69 | **+3.34** |
+| jimeng_pill | +4.27 | +3.81 | **+5.16** |
+
+LaMa > MI-GAN > cv2 holds on 3 of the 4 marks worth filling, and MI-GAN edges LaMa on
+doubao. Nothing here argues for changing `--backend auto`.
+
+**Correction, and the statistic that caused it.** An earlier version of this section
+claimed the opposite -- that MI-GAN was the WORST on texture, below cv2 -- and it was
+wrong. The report computed recovery as `median(filled) - median(damaged)`: a **difference
+of medians**, not the median of the per-image differences. On skewed data those are
+different statistics and here they disagreed in SIGN. A paired per-mark sign test settled
+it: on the textured tercile cv2 vs MI-GAN is not significant for any mark (p 0.13-1.00;
+pooled n=119, cv2 wins 69, p=0.099), while MI-GAN's medians are higher for 3 of 5.
+
+The wrong statistic nearly shipped a change to `--backend auto`, which is the resolver a
+memory-constrained CPU caller depends on. Two lessons: report the median of the per-image
+DIFFERENCES when the question is paired, and confirm a ranking with a paired test before
+acting on a table of independently-aggregated columns.
+
+**Every backend collapses on texture**: recovery falls from ~+10-15 dB to ~+1-3 dB and SSIM
+from ~0.79-0.95 to ~0.30-0.44. The documented "textured is where fills struggle" is
+confirmed, with numbers, for the first time.
+
+**The invariant held**: 0 violations of "the fill touches nothing outside its mask" across
+all 720 measurements and all three backends.
+
+#### Visible parity, first full run, 2026-07-20 (10,593 image/mark pairs, cv2)
+
+| mark | n | detector-clean after removal |
+|---|---|---|
+| jimeng | 298 | 100% (CI 98.7-100) |
+| gemini | 4,974 | 95.3% (CI 94.7-95.8) |
+| doubao | 2,580 | 91.8% (CI 90.7-92.8) |
+| samsung | 3 | 100% (CI 43.8-100) |
+| jimeng_pill | 2,738 | 32.4% -- **wrong path, see below** |
+| **excluding the pill** | **7,855** | **94.3% (CI 93.8-94.8)** |
+
+Backend-independence was CHECKED, not assumed: the same 309 pairs through cv2 / MI-GAN /
+LaMa agree within one image on doubao (91.9% x3), gemini (91.4 / 92.1 / 91.4) and jimeng
+(100% x3). Only the pill diverges (32.5 / 25.0 / 25.0), and 17 of the 18 disagreements are
+the pill. So one backend suffices for parity; quality is B1's job, not this sweep's.
+
+**The residual is NOT one phenomenon -- it splits by mark.** Checking whether a still-
+detected mark carries vendor provenance independent of the visual detector, against the
+cleaned marks as a control:
+
+| mark | still detected | cleaned | reading |
+|---|---|---|---|
+| doubao | 82.9% corroborated | 80.0% | indistinguishable -- the marks are REAL |
+| gemini | 39.6% | 62.4% | significantly lower -- much of it is false fires that persist |
+
+The doubao half turned out to be the front-end mismatch bug (fixed 2026-07-20, see the
+CLAUDE.md rule): `remove()` was a silent no-op on 57 of 60 sampled cases because the
+binarized mask came back empty. After the fix, 60/60 of those clear, so doubao parity
+should now sit near 99%; **the sweep has not been re-run to confirm that**.
+
+Method note: two intermediate checks were worthless and both looked meaningful.
+`MarkDetection.region` for a text mark is the GEOMETRY box derived from frame dimensions,
+not a match position, so "the detector's box did not move after the fill" was predetermined,
+and "the mask covers 96.5% of the detected region" compared geometry against the same
+geometry. What actually answered it: looking at the pixels, and testing whether `remove()`
+changed the array at all.
+
+#### The Jimeng pill: measure the GATED path or the number is meaningless
+
+The parity audit calls `get_mark(key).remove`, which bypasses `_keep_pill`. On the pill
+that path reports "detector still fires after removal" 68-75% of the time and reads as a
+broken feature. It is not: the product gates the pill hard, and `--mark auto` behaves
+completely differently.
+
+Measured through the product path over **all 2,738 pill positives** (2026-07-20):
+
+| | n | |
+|---|---|---|
+| raw detections | 2,738 | |
+| corroborated real (wordmark or TC260) | 346 | 12.6% |
+| the gate lets through to removal | 125 | 4.6% |
+| **of those, corroborated real** | **125** | **precision 100% (95% CI 97.0-100)** |
+
+Every single pill the product removed was corroborated. Headline recall is 36.1%, but that
+denominator is wrong: **TC260 provenance maps to BOTH ByteDance products**, so a "TC260
+says Jimeng" image may be a Doubao image with no pill at all. Split by evidence strength:
+
+| corroboration | n | removed | recall |
+|---|---|---|---|
+| wordmark (names the product) | 49 | 49 | **100% (CI 92.7-100)** |
+| TC260 only (cannot separate Doubao) | 297 | 76 | 25.6% (CI 21.0-30.8) |
+
+So where the evidence actually names Jimeng, the pill is removed every time. The flatness
+guard suppresses 186 TC260-only cases -- its measured recall cost, paid to avoid the
+smeared textured fills it exists to prevent.
+
+Two harness lessons, both of which produced a wrong number before being caught:
+- **Gated marks must be measured through the gate.** A per-mark audit answers a question
+  the product never asks.
+- **Match mark labels exactly.** A substring test on `AI生成` also matches Doubao's label
+  (`Doubao 豆包AI生成 text`), counting Doubao removals as pill removals and inflating the
+  pill's precision.
+
+#### The finding that was not being looked for: filling a faint mark is net negative
+
+Samsung came out negative in every cell, which looked like a broken mark. It is not -- the
+sign is set by how strongly the mark perturbs the image, not by which mark it is. Per-mark
+recovery by damage band (a HIGH damage-PSNR means a FAINT mark):
+
+| mark | 0-18 dB | 18-22 dB | 22-26 dB | 26+ dB |
+|---|---|---|---|---|
+| doubao | +9.78 (n=160) | -1.52 (36) | -3.37 (18) | -8.38 (23) |
+| jimeng | +7.70 (187) | -2.49 (21) | -2.54 (15) | -2.34 (15) |
+| samsung | - | **+7.23** (48) | -3.78 (102) | -10.78 (90) |
+
+Samsung is POSITIVE where its mark is strong; doubao and jimeng go NEGATIVE where theirs
+are faint. Linear fit over all 720: `recovery = -0.861 * damage_psnr + 19.47`, break-even
+at **~22.6 dB**. Samsung's alpha map peaks at 0.37 against doubao 0.68 and jimeng 0.93, so
+Samsung simply sits on the faint side of that line most of the time.
+
+So: **below ~22.6 dB of mark damage, inpainting costs more fidelity than the mark did.**
+The pipeline currently fills unconditionally once a mark is detected, so this cost is
+invisible today.
+
+Do not read this as "stop removing faint marks". A user who wants the watermark GONE is not
+optimizing PSNR, and a faint mark is still a mark. What it says is that the fill has a real
+cost, it is now measurable, and for faint marks it exceeds the thing it removes -- which
+makes "how faint is too faint" a product decision that can finally be made on evidence.
+
+### B2. Detector response curves -- RUN 2026-07-20
+
+`scripts/detector_response.py`. Stamp marks across a controlled grid (size x contrast,
+crossed with real corpus backgrounds and frame aspects) and measure two things per cell:
+`detected`, and whether the same call path then yields a non-empty removal mask
+(`maskable`). The second column exists because the gap between them is a silent no-op --
+`identify` reports a mark that `visible` skips -- and any detection-only harness scores
+that bug as a success.
+
+**Read the nominal cell, never the aggregate.** The grid deliberately visits sizes and
+opacities no engine was calibrated for, so its overall rate is an adversarial score, not
+recall. Production recall still comes from the unbiased corpus sample.
+
+What it found:
+
+- **The size response is a COMB, not a curve.** `_tophat_score` sweeps exactly three rungs
+  (0.8, 1.0, 1.25). Doubao scores ~0.99 at each rung and collapses to 0.37-0.48 between
+  them, against a 0.50 gate -- so a mark ~10% off a rung is missed at FULL contrast. The
+  `binary` front-end has no ladder at all: jimeng holds one lobe over 0.90-1.20, samsung
+  only 0.95-1.05, i.e. samsung requires a mark at essentially its exact nominal size.
+- **Contrast is nearly irrelevant** on the tophat front-end -- the response is
+  max-normalized, so a mark at 15 luma levels of contrast still scores 0.984.
+- **No detected-but-unmaskable cells** at any grid point, so the front-end/mask parity fix
+  holds across the whole operating range, not just where the corpus happened to look.
+
+And the follow-up that stopped a bad change: dead zones only cost recall if real marks land
+in them. `scripts/ladder_headroom.py` measured that on the corpus (positives = frames whose
+metadata names the vendor, negatives = frames with no metadata signal, deliberately NOT
+filtered on the detector's own verdict, which would make its false-fire rate 0 by
+construction). A 13-rung dense ladder recovers **28 of 368 misses (7.6%)** while false fire
+goes **2.52% -> 3.05%**. So the comb is real and mostly unvisited: the fractions were
+calibrated on real captures and real marks cluster at the rungs. **Do not densify the
+ladder** on this evidence.
+
+The residual looked like a clean lead and, on a full 923-positive / 3546-negative run,
+turned out not to be. A targeted `plus_one` ladder (one extra rung at ~1.116) recovers 36
+of the 39 marks the 13-rung dense ladder recovers -- so the win really is that one rung,
+not density. But **all 36 recoveries AND all 18 of its added false fires are LANDSCAPE at
+that same rung** (2.0:1 overall, 1.7:1 even gated to landscape). The rung is not vendor
+signal; it is a size shift that helps and hurts equally.
+
+And the obvious "fix it at the source" -- bump the landscape width fraction so that
+subpopulation lands on the nominal rung -- is **falsified by the same data**: the 55
+currently-DETECTED landscape positives already peak at scale 1.0, not 1.116. So there are
+two size clusters of landscape doubao marks, one at the calibrated nominal and one ~11%
+larger, and moving the nominal would drop the cluster that works to catch the one that does
+not. The larger cluster is genuinely a different size and is inseparable from landscape
+false fire at the NCC gate -- the same detector-discrimination wall as vendor attribution,
+not a geometry constant anyone forgot to set. **Do not add the rung, and do not move the
+landscape fraction.** The per-rung scores are stored in `_ladder_headroom_doubao.jsonl` so
+this verdict can be re-derived without re-running the sweep.
+
+Why the misses are not a tuning problem: on the sampled misses the dense-ladder score is
+bimodal -- detected marks sit at median 0.938, misses at 0.155, and **the band 0.31-0.52 is
+empty**. There is no near-gate cluster, so no threshold recovers them. Eyeballing 26 miss
+corners explains it: ~6 are jimeng (TC260 names no vendor, so a "doubao positive" is often
+a jimeng frame), ~14 carry no visible mark in that corner at all, 2 belong to **uncovered
+vendors** (`千问AI生成`, `百度 AI生成`), and only ~4 are genuine doubao failures -- on
+saturated or structurally busy corners, with heterogeneous causes (the saturation gate
+explains exactly one of five tested).
+
+### B3. Invisible round-trip, positive-control gated
+
+The open DWT-DCT detector is positive-only and carrier-fragile: "not found" on a fragile
+carrier proves nothing (measured: `chatgpt-1.png` recovers 114/128, below the 118 gate).
+Every invisible assertion must first embed on the SAME carrier and confirm recovery, and
+**degrade to a skip rather than a pass** when the control fails. Already implemented in
+`smoke_matrix.py`; apply the same discipline anywhere else this detector is used.
+
+### B4. Resource ceilings
+
+Peak RSS and wall time per backend x input size, up to 25 MP. The memory-constrained CPU
+tier is a real constraint (MI-GAN must stay ~0.6-0.9 GB by cropping around the mask); a
+regression here is invisible today and would only surface under load.
+
+## Tier C -- human-labelled accuracy (bounded by labelling effort)
+
+The machinery exists: `visible_recall_sample.py` -> `visible_sheets.py` ->
+`visible_groundtruth.py` -> `visible_eval.py`.
+
+- **Recall** is the known weak spot: measured once, unbiased n=240, and that single
+  measurement is what exposed the landscape miss. Expand per mark, especially jimeng
+  (n=14) and jimeng_pill (n=6), whose numbers currently rest on almost nothing.
+- **Precision** re-runs over the existing 779-cell ground truth; benchmark every detector
+  change with `--vs <snapshot>`.
+- **Coverage**, the largest known gap: ~6% of sampled images carry an uncovered vendor's
+  mark (百度 / 星绘 / 抖音-class; 千问 was the first of this class and is registered since
+  2026-07-21) that no registered detector can fire on. This is a
+  coverage problem, not a tuning problem, and no threshold work will move it.
+
+Three harness rules are load-bearing and must not be relaxed: score a mark only within its
+crop's adjudication scope; take provenance from metadata, never from labels; and never
+report recall from the detector-sampled set.
+
+## Tier D -- external oracles (manual, not automatable here)
+
+SynthID removal cannot be verified locally by design -- no public decoder exists. Each
+vendor has its own oracle and it covers only that vendor's content: `openai.com/verify` for
+OpenAI (more accessible, the automation candidate), the Gemini app for Google (manual,
+rate-limited). A quiet metadata proxy is **not** proof the pixel watermark is gone.
+
+Scope honestly: this tier certifies strength floors on a handful of images per vendor, and
+that is all it can do. See `docs/synthid.md`.
+
+### D1. Sampling frame
+
+The sidecars already classify the corpus: **15,000 images whose watermark list mentions
+SynthID**, of which 9,071 carry `verify_oracle=openai` and 5,929 `verify_oracle=google`.
+Stratify on the two axes that actually move removal efficacy -- vendor (the certified
+floors differ: OpenAI 0.10, Gemini 0.15) and content class (photoreal vs flat graphic,
+where the pipelines are documented to diverge).
+
+### D2. The oracle is the bottleneck, not the GPU
+
+Measured on MPS (2026-07-19), single invocation of `invisible`:
+
+| `--max-resolution` | 256 | 384 | 512 | 768 | 1024 |
+|---|---|---|---|---|---|
+| wall time | 37.9 s | 58.9 s | 59.4 s | 65.2 s | 118.3 s |
+
+384/512/768 are indistinguishable, so below ~768 the cost is dominated by **fixed model
+load, not diffusion**. Confirmed by batching: 4 images in one process took 105.5 s
+(26.4 s/image) against 59.4 s/image one at a time -- roughly 40 s fixed overhead per
+invocation and ~15 s marginal per image at 512 (rough: run-to-run variance is large).
+
+Two consequences for the harness:
+- **Amortize the fixed cost**: one long-lived process over many images, never one
+  invocation per image. That is a 2-4x win. Shrinking below 512 is not.
+- The binding constraint is the **external oracle's throughput**, which is manual and rate
+  limited. So do not run a uniform grid; spend each oracle check where the answer is
+  uncertain -- **bisect strength per content class** to certify a floor in ~10 checks
+  instead of ~100.
+
+### D3. Mandatory control before trusting any reduced-size run
+
+`--max-resolution` is downscale -> diffuse -> Lanczos upscale. If the **resize round trip
+alone** damages SynthID, the oracle goes quiet for a reason unrelated to removal and the
+result does not transfer to production at native resolution.
+
+Before any reduced-size sweep, run the resize round trip with **no diffusion** and put the
+result through the vendor oracle. If the watermark survives, the reduced size is a valid
+test bed; if it does not, reduced-size results are measuring the resizer. This is the same
+failure shape as the imwatermark carrier-fragility trap in B3: an oracle that falls silent
+for the wrong reason reads exactly like success.
+
+`docs/synthid.md` cites ~99.98% TPR across 30 transforms including resize, which predicts
+the control passes -- but that is Google's claim about their own decoder, not our
+measurement, so it is a hypothesis to test, not a reason to skip the control.
+
+## Tier E -- robustness and adversarial inputs
+
+Malformed and hostile inputs, since ~0.2% of real uploads are already truncated: truncation
+at many offsets, corrupt headers, 16-bit and CMYK, absurd dimensions, decompression bombs,
+zero-byte files, unicode and RTL filenames, symlinks, read-only output dirs, concurrent runs
+on one file. The bar is never "handles it" but **never raises and never silently degrades**.
+
+## Build order
+
+1. **A1 sidecar regression** -- highest value per hour, unattended, needs no new labels.
+2. **A2/A3/A4 parity and invariants** -- full corpus, reuses existing audit scripts.
+3. **B1 fill quality** -- closes the oldest unmeasured claim in the project.
+4. **B2 detector curves** -- cheap, and directly guards the geometry class of bug.
+5. **A5 contract sweep at corpus scale**.
+6. **B4 resource ceilings**, **E robustness**.
+7. **C recall expansion** -- gated by labelling appetite.
+8. **D oracles** -- manual, per release.
+
+Every tier writes a versioned snapshot so runs are comparable over time; a run that cannot
+be diffed against the last one is a one-off, not a regression suite.
+
+## What the measurements imply for detection work
+
+Recorded here because each item is grounded in a number from this campaign, not because
+it is a prioritized plan (that lives elsewhere -- see the note at the end of this section).
+
+### Metadata absence does not disable the detectors -- it disables the RELAXATION
+
+The detectors are pixel-based and need no metadata. What metadata does is relax the
+false-positive gate (`auto` vs `strict`). So "work better without metadata" means
+strengthening the strict-path detectors themselves; it is not a gating problem.
+
+Per mark, what actually goes away when metadata is stripped:
+
+- **The pill loses an entire arm.** Its TC260 arm is dead without metadata, leaving only
+  the wordmark arm. Measured: where the wordmark corroborates, pill recall is **100%**
+  (49/49); on TC260-only evidence it is 25.6%. So on stripped uploads the pill's fate
+  rests entirely on Jimeng wordmark detection -- whose own recall is **71% on n=14**.
+  This is the weakest link with the most leverage: every point of wordmark recall pulls
+  the pill along with it.
+- **The sparkle already runs on pixels**, and the FP-gate tightening cost ~120-200 genuine
+  detections corpus-wide (12.5% of the 1,256 lost). That headroom exists but the
+  precision trade behind it was deliberate.
+- **The largest gap is metadata-independent by nature**: ~6% of sampled images carry an
+  uncovered vendor's mark (百度 / 星绘 / 抖音-class; 千问 is registered since 2026-07-21)
+  that no registered detector can fire on at all.
+
+### Where the evidence points
+
+1. ~~A generic CJK AI-mark detector~~ **superseded for 千问 (2026-07-21).** The AUC ~0.5
+   non-separability from Doubao was measured at the WRONG size; at the fitted geometry an
+   exact-size 6-glyph template separates 千问 from 400 doubao-marked frames with ZERO
+   cross-fire at the gate, so per-vendor registration won and shipped. The generic
+   class-detector shape may still be right for the long tail of compliant vendors, but
+   its motivating measurement is gone.
+2. **Port the `tophat` front-end to the remaining marks.** It took Doubao from 89% to 92%
+   recall at unchanged 99% precision. But the gate is front-end specific and **must be
+   recalibrated, never ported**: a naive 0.40 produced 8 false fires instead of 1 and
+   silently halved the pill's recall (because `_keep_pill` suppresses the pill whenever
+   Doubao fires).
+3. **The Jimeng wordmark.** Weak on its own (71%/71%) and it gates the pill. Its
+   silhouette is also non-discriminative against Doubao's, which was patched with a 0.85
+   threshold -- a patch on a detector problem, not a fix.
+
+### Measure before improving
+
+Jimeng recall rests on **n=14** and the pill's on **n=6**. Improving what is measured by
+six samples means not knowing whether it improved. Tier B2 (detector response curves on
+stamped marks) is the instrument to build first: recall as a function of size, contrast
+and background texture, with no new hand labelling, and it catches the geometry class of
+bug (`scale_basis`) directly.
+
+This section records what the measurements imply technically. Prioritization is tracked
+separately, outside this repo.
+
+## Real-example end-to-end run (2026-07-20)
+
+The corpus sweeps all call the library IN-PROCESS. This run drives the actual
+`remove-ai-watermarks` entry point as a user would, over real corpus images and the
+committed real fixtures, and checks the OUTCOME rather than the exit code
+(`scripts/real_examples_e2e.py`). **26 of 28 behaviours passed.**
+
+| Command | Real examples | Result |
+|---|---|---|
+| `identify --json` | 6 provenance classes (OpenAI, Adobe, Midjourney, Doubao/TC260, Grok, FLUX) | 6/6 correct verdict + platform |
+| `metadata --check` / `--remove` | the same 5 real AI-metadata files | 10/10 detected, and the output re-scans clean |
+| `visible --mark auto` | a live positive per mark | doubao / jimeng / gemini removed and re-detect clean; pill correctly DECLINED by the gate; samsung partial (below) |
+| `erase --region` | one real image x 3 backends | cv2 / MI-GAN / big-LaMa all wrote output |
+| `batch --mode visible` | a real 5-image directory | 5/5 outputs |
+| `invisible` (MPS, `--max-resolution 512`) | a real Gemini and a real OpenAI carrier | both wrote a genuinely CHANGED image |
+| `all` (MPS) | a real Gemini carrier | one transient exit 1, not reproducible (see below) |
+
+**Samsung is a real, reproducible partial.** It is the faintest registered mark (peak alpha
+~0.38) sitting on a 0.40 gate, so the margin between "detected" and "removed" is razor thin.
+Over the entire corpus population (n=3, all of it) the CLI clears 2 of 3 outright
+(0.446 and 0.440 -> below gate) and on the weakest one reduces 0.431 -> **0.404**, which is
+still fractionally over the gate. The glyph IS filled and the confidence IS reduced; the
+residual re-detects. This is the faint-mark residual class on the `binary` front-end, which
+has no equivalent of the `tophat` faint-mask fallback. **Not fixed:** with n=3 corpus-wide
+there is no way to tell an improvement from noise, which is the same
+improving-what-3-samples-measure trap recorded under Open items.
+
+**The pill "failure" was the harness, not the product.** `detect_marks` fires `jimeng_pill`
+but `remove_auto_marks` returns no label -- `_keep_pill` correctly declines an uncorroborated
+low-confidence pill, so `visible` writes nothing and exits 2. The harness now asks the
+product what it DECIDED and treats a correct decline as a pass.
+
+**The `all` exit 1 did not reproduce** -- the same invocation ran clean standalone twice and
+again in the exact three-run sequence that produced it (all exit 0, step 2 executed, no skip
+banner). It stays recorded as a transient because it could not be diagnosed: the harness
+discarded the command output, and `cmd_all` has three distinct `SystemExit(1)` paths
+(unreadable input, unreadable intermediate, and the deliberate synthid-skipped banner) that
+an exit code alone cannot distinguish. The harness now retains the output tail on any
+failure, so a recurrence is identifiable.
+
+## Tier E: robustness (2026-07-20) -- RUN, and it found two real crashes
+
+`scripts/robustness_suite.py` drives the real CLI over adversarial and degenerate inputs and
+scores GRACEFULNESS, not success: a non-zero exit with a readable message is a pass, an
+unhandled traceback or a hang is a fail. **33/33 graceful after two fixes; 31/33 before.**
+
+Covered: truncated and corrupt files, zero-byte, a text file named `.jpg`, 1x1 and
+1x4000 slivers, a 729 KB decompression bomb declaring 16000x16000, Unicode and RTL
+filenames (read AND write), a mismatched extension, a nonexistent nested output dir, a
+read-only output dir, a directory passed as a file, a missing path, 4x concurrent runs
+against one input, and batch over both an empty and an undecodable directory.
+
+**Both defects were invisible to the 849-test suite**, because unit tests feed well-formed
+fixtures into writable directories. Both are now regression-guarded by
+`tests/test_cli_robustness.py`.
+
+1. **A failed write crashed on the size report.** `image_io.imwrite` is contractually
+   non-raising -- it returns `False` when the path cannot be written. But
+   `write_bgr_with_alpha` discarded that bool and returned `None`, so **no caller could
+   distinguish a failed write from a successful one**, and all five write sites then ran
+   `output.stat()` to print the size. A read-only output directory produced a bare
+   `FileNotFoundError` traceback pointing at the stat rather than at the write. The signal
+   existed the whole way down and was thrown away by one wrapper. Fixed at that wrapper
+   (propagate the flag) plus one shared `cli._write_output_or_exit`, so all sites are
+   covered rather than the one that happened to be caught.
+2. **A directory passed as the image crashed the scanner.** `click.Path(exists=True)`
+   accepts directories unless told otherwise, so `identify <dir>` reached `open()` and
+   raised `IsADirectoryError`. Fixed by `dir_okay=False` on all six `source` arguments --
+   argument parsing now refuses it, which is where it belongs. (`batch` was already correct:
+   it declares `file_okay=False`.)
+
+3. **The worst instance was found by the /simplify review, not by the sweep: `batch` into a
+   read-only directory wrote ZERO files for 2 inputs and exited 0.** No traceback, no
+   error, a success code and an empty output directory -- a wrapping service would treat
+   that as a completed run. `graceful()` structurally cannot see this class, because it
+   scores exit code and traceback markers and this failure has neither. The suite now
+   carries a `batch_readonly_outdir` case that asserts on the ARTIFACTS (how many files
+   exist) rather than on the status. **Any check for a silent no-op has to assert on the
+   output, not the exit code.**
+
+The fix is NOT uniform, and that matters: the single-image commands exit via
+`cli._write_output_or_exit`; `api._write_visible_result` RAISES so a library caller gets an
+accurate error rather than a confusing `FileNotFoundError` from the downstream metadata
+strip; and the batch sites raise too but must never `SystemExit`, because the batch loop
+counts per-image exceptions and aborting would kill the whole run instead of failing one
+image.
+
+The lesson worth keeping: **a non-raising IO contract needs its return value checked at
+every call site, and a wrapper that swallows it silently disables the contract for
+everyone downstream.** Grep for other `-> None` wrappers over non-raising primitives --
+`invisible_engine.py:346` still discards `imwrite`'s flag and is the same shape.
+
+## Tier B4: resource ceilings (2026-07-20) -- RUN
+
+`scripts/resource_ceilings.py`, one FRESH process per cell (peak RSS inside a long-lived
+process is contaminated by whatever ran before it, and the question is what a per-request
+worker needs). The mask is a fixed, small corner region at every input size, so the thing
+under test is whether the learned backends really crop around the MARK rather than
+processing the frame.
+
+| backend | peak RSS 1MP -> 25MP | wall | scaling |
+|---|---|---|---|
+| cv2 | 74 -> 440 MB | 0.02-0.12 s | **grows 5.9x with input size** |
+| migan | 603 -> 775 MB | ~0.6 s | flat |
+| lama | 4679 -> 4779 MB | ~3.8 s | flat |
+
+**The documented claims hold.** CLAUDE.md's "migan ~0.6-0.9 GB regardless of upload size"
+and "lama ~4.7 GB peak" both reproduce to the digit, and the crop-around-the-mask design is
+confirmed: both learned backends are flat in input size. This is also the measurement
+behind the deployment split (free tier `migan` fits a small droplet under 1 GB; paid tier
+`lama` needs ~5 GB).
+
+**New information: cv2 is the only backend that scales with the input.** It inpaints the
+full frame rather than a crop, so at 25 MP it costs 440 MB -- still the cheapest tier, but
+6x its small-image footprint, which is worth knowing when sizing a worker that accepts
+phone-camera uploads. Its wall time stays trivial throughout.
+
+Caveat on the wall times: each is a COLD process including model load, so migan's ~0.6 s
+here is not comparable to the ~0.19 s warm figure quoted elsewhere. Cold is the honest
+number for a per-request worker; warm is the honest number for a long-lived one.
+
+**Two method notes, both cases of the harness corrupting its own measurement.**
+
+1. The first version called `erase()` with a positional `boxes` argument, which the
+   keyword-only signature rejects. All 12 cells failed identically and still reported
+   plausible-looking RSS (60-129 MB) -- the process's numpy footprint with no backend work
+   at all. Twelve identical failures were one bad call, and only the printed error made
+   that visible. The child now asserts the output actually differs, so a no-op cannot be
+   reported as a measurement.
+2. That assertion was then itself the contaminant: `(out != img).any()` allocates a boolean
+   temp the size of the IMAGE before `getrusage` is read -- ~75 MB at 25 MP, and it grows
+   with the input, so the harness partly manufactured the very "cv2 scales with input size"
+   conclusion it was measuring. Now it compares only the mask box. **Re-measured: the
+   conclusion survived, the digits moved.** cv2 6.1x -> 5.9x, migan max 847 -> 775 MB, lama
+   max 4849 -> 4779 MB; the table above is the corrected run. Worth keeping because the
+   contaminated numbers were already committed to CLAUDE.md before the check was
+   questioned -- a verification step is part of the instrument and needs the same scrutiny
+   as the thing it verifies.
+
+## Open items (as of 2026-07-20)
+
+Everything below is known, measured, and deliberately not done yet. Each line says what it
+would take, so none of it has to be rediscovered.
+
+### START HERE next session
+
+Item 1 from the previous session is **DONE (2026-07-21)**: 千问 is registered --
+see "The 千问 harvest (2026-07-21)" below for the decision and the numbers. In
+priority order:
+
+1. **Decide the exit-code split** (open defect 2). It is a deliberate product call, not
+   research: it is breaking for existing wrappers, so it needs a yes/no rather than more
+   measurement.
+2. **The two small correctness items** (open defects 3 and 4) -- both are contained, both
+   have the fix written out below.
+3. **The bonus vendors from the harvest** (元宝 n=50, 可灵 n=30, cat-logo n=19) repeat
+   the 千问 playbook each: font-rendered silhouette, `--fit-geometry`, gate calibration
+   against the contamination-guarded clean arm, crossfire against doubao/jimeng. 可灵
+   additionally stamps a second mark bottom-LEFT, which no current text-mark config
+   expresses (the pill is top-left; a bottom-left CJK mark needs a `corner="bl"` CJK
+   config -- samsung is `bl` but Latin-script and width-based). 星绘/百度 are NOT in the
+   corpus in labelable quantity -- verified, do not hunt them again.
+
+Do NOT restart the sweeps to "check". Their artifacts are on disk and listed under
+"Completed full runs" below; re-running costs hours and answers nothing new. The fast way
+to confirm the whole surface still works after a change is
+`uv run python scripts/real_examples_e2e.py` (~2 min, real corpus examples through the real
+CLI) plus `uv run python scripts/robustness_suite.py` (~3 min, adversarial inputs).
+
+### The 千问 harvest (2026-07-21) -- RESOLVED, registered the same day
+
+**The unlock: the TC260 label is not anonymous.** Its `ContentProducer` field carries the
+producer's Chinese Unified Social Credit Code (`001191110102MACQD9K64010000` -> USCC
+`91110102MACQD9K640`), which names a legal entity. So carriers partition into per-VENDOR
+cohorts from METADATA ALONE, owing nothing to any pixel detector -- which is exactly what
+broke the previous attempt, whose only way to find 千问 frames was to eyeball the misses of
+a detector that cannot see them. A cohort is a LABEL: eyeball one frame, and every frame in
+it is a labelled example. (CLAUDE.md's "the generic TC260 label names no specific vendor" is
+about the label MARKER; the producer FIELD inside the block is a different thing.)
+
+New tools, both lint-clean, both **uncommitted**:
+- `scripts/vendor_cohort_harvest.py` -- full-corpus metadata scan -> cohorts. Joins which
+  detectors fired from the completed `_visible_positives.jsonl` rather than re-running the
+  pixel pass. Artifact `data/spaces/_vendor_cohorts.jsonl` (**4441 carriers, 46 entities**).
+  `--sheets N` writes full-width top/bottom band crops per cohort for eyeballing.
+- `scripts/vendor_mark_calibrate.py` -- scores a cohort against the 432 hand-labelled
+  `present: []` negatives from the 2026-07-18 round, and writes score-SORTED corner crops so
+  mark presence and the gate are read in one visual pass.
+- `src/.../assets/qwen_alpha.png` + `xinghui_alpha.png` regenerated (they were listed in
+  `render_vendor_silhouettes.py` but had never been committed).
+
+**What the corpus actually contains** (this corrects the previous list of targets):
+
+| Cohort USCC | n | quiet | Visible mark | Verdict |
+|---|---|---|---|---|
+| 91440101MA9Y9T4H7A | 117 | 112 | `千问AI生成` bottom-right | **the target, confirmed by eye** |
+| 91340100MAEB4N8H76 | 73 | 70 | mostly none; one `RunningHub AI生成` | metadata-mostly |
+| 913502007378955153 | 113 | 109 | none seen | metadata-only |
+| 91440300708461136T | 50 | 46 | `元宝AI生成` (Tencent Yuanbao) | bonus vendor, bold |
+| 91441900557262083U | 49 | 45 | none seen | metadata-only |
+| 91110108335469089C | 30 | 28 | `可灵AI 3.0` (Kling) + an `AI生成` pill bottom-LEFT | bonus vendor |
+| 91110108562144110X | 19 | 19 | cat-logo + `AI生成`, in **19/19** | bonus vendor, very clean |
+
+**`百度` and the `星绘`/`抖音` class are NOT in this corpus in labelable quantity.** No brand
+token for them appears in any AIGC label field, and every remaining cohort is <= 16 frames.
+Do not spend another session hunting them here; the previous "one confirmed positive each"
+is all there is. The corpus offers 千问 richly plus three DIFFERENT vendors instead.
+
+Also worth knowing: a cohort is a strong grouping key but names the SIGNING ENTITY, not
+always the consumer brand -- the 千问 cohort contains one `造点AI生成` frame. And TC260
+provenance does NOT imply a visible mark, which is why four large cohorts above are
+metadata-only. The cohort is the candidate pool; the eye settles mark presence.
+
+**RESOLVED 2026-07-21: 千问 is registered** (`qwen_engine.py`, strict-only, no rival
+margin). The ladder trade-off that was the open decision is settled in favor of a
+**per-mark ladder**, not the shared one and not a wider shared one -- and the path there
+found two more geometry defects the "single fraction on the shipped ladder" framing had
+missed.
+
+What the final calibration measured, in the order it happened:
+
+1. **The locate box was clipping the mark.** Scoring with the fitted fractions still
+   collapsed the cohort (p50 0.209). Frame-level diff against the wide-ladder fit showed
+   why: the real mark sits ~0.025 of the short side off the right edge, while doubao's
+   inherited box anchors at 0.004 -- the box's left edge cut into the 千 glyph, and an
+   exact-size template scored 0.26 where the fit's wider box scored 0.73. So the locate
+   fractions are as mark-specific as the template size; `--fit-geometry` now records the
+   absolute match rects and fits the box too (margins ~0.021, width 0.231, height 0.074).
+2. **The size distribution is cleanly BIMODAL.** With the box fixed, frac_short clusters
+   at ~0.124 (13 frames) and ~0.203 (38 frames), nothing between -- two stamp sizes,
+   ratio 1.64, just over the shared ladder's 1.5625 span. That is why no single fraction
+   covers the mark.
+3. **A per-mark 2-rung ladder beats both alternatives.** Candidates, both arms scored on
+   the shipped code path: (A) shipped 3-rung @ frac 0.167 -- cohort p50 0.538; (B1)
+   2-rung (0.78, 1.27) @ frac 0.160, one rung centred on each mode -- cohort p50
+   **0.662**; (B2) 4-rung (0.8, 1.0, 1.25, 1.5625) @ frac 0.155 -- p50 0.449, strictly
+   worse (its big-mode rung sits 4.6% off the mode, and the extra rungs cover nothing).
+   B1 also costs one matchTemplate LESS than the shipped 3. `TextMarkConfig.ladder` was
+   added with the default `(0.8, 1.0, 1.25)`, so every other mark's computation is
+   byte-identical (the full 876-test suite plus e2e + robustness confirm); a shared
+   densification was already ruled out by B2's false-fire measurement. The doubao
+   false-fire check the plan asked for reduces to that equivalence-by-construction --
+   doubao's ladder never changed.
+4. **`alpha_height_frac` measured, not inherited:** aspect fit at the winning width, p50
+   0.260 (tight, p10-p90 0.250-0.270) -> 0.0416. The silhouette's own aspect (0.2219)
+   and doubao's ratio were both measurably off.
+5. **The clean arm was contaminated, and fixing it flipped the verdict.** The 2026-07-18
+   `present: []` labels are in the vocabulary of the REGISTERED marks only -- 146 of the
+   432 "clean" frames sit in a TC260 cohort, including 15 qwen-cohort frames VISIBLY
+   carrying 千问AI生成, and they were the clean arm's entire top tail (clean p99 0.37 ->
+   0.69 with the fitted geometry). `load_sets` now drops every frame in ANY cohort.
+   Final arm: 286 frames, clean p99 0.301 / max 0.316.
+6. **Gate 0.45, strict-only, no rival margin.** Every cohort frame >= 0.45 carries a
+   visible mark (83 of ~96 eyeballed visible marks fire = **86% recall of visible
+   marks**; the misses are white-on-near-white contrast losses); 0/286 clean fires;
+   crossfire at the gate: 0/400 on doubao-marked frames, 0/298 on jimeng-marked frames
+   (the shared `AI生成` tail correlates at p50 0.224, far below gate -- the AUC-0.5
+   attribution fear from 2026-07-18 was a mis-sizing artifact). A 0.10 rival margin
+   would have cost ~10% of genuine qwen detections, so `rivals=()`. The band just below
+   the gate is dominated by non-qwen banners (夸克 anti-forgery strip 0.274, 造点 mark
+   0.253), so a provenance-relaxed arm would be mostly false fills:
+   `provenance_ncc_factor` is pinned at 1.0 and qwen has NO provenance mapping.
+7. **Parity confirmed end to end:** detect -> cv2 fill -> re-detect clean on **83/83**
+   real cohort marks, no empty masks; `real_examples_e2e.py` now drives a live qwen
+   positive through the real CLI (qwen bucket = symlinks under the gitignored
+   `_visible_datasets/`); a confident qwen detection suppresses the jimeng pill exactly
+   like doubao's does.
+
+Method note worth keeping: **the wide ladder flattered the clean arm exactly as
+predicted, but the trap that actually bit was the LABEL vocabulary.** "present: []"
+meant "no registered mark", not "no mark" -- and a calibration clean arm has to be
+re-filtered per candidate, or the gate is read off frames that carry the very mark being
+calibrated.
+
+The bonus vendors (元宝, 可灵, cat-logo) need their own font-rendered silhouettes before
+any of this repeats for them; 可灵 additionally puts a second mark bottom-LEFT, which no
+current text-mark config expresses.
+
+### Open defects
+
+| # | Defect | Measured impact | What the fix takes |
+|---|---|---|---|
+| 1 | 16-bit PNGs are downconverted to 8-bit by a metadata strip | 42 of 27,018 corpus PNGs (0.16%); one went 9.2 MB -> 2.5 MB | a byte-level PNG chunk stripper, so the PIL re-save is skipped entirely |
+| 2 | Exit code 2 means three different things (no visible mark / no invisible signal / Click usage error) | any wrapper must parse stderr to tell them apart | split the codes; **breaking for existing wrappers**, so it needs a deliberate call |
+| 3 | `visible_removal_audit.py` measures the UNGATED per-mark path | reports the pill at 32% where the product runs at 100% precision | teach it the product path (`remove_auto_marks`) for gated marks, or at minimum say so loudly in its docstring |
+| 4 | `invisible_engine.py:346` still discards `imwrite`'s success flag | same shape as the crash fixed 2026-07-20, on the diffusion output path; not yet observed failing | check the bool and raise/report, mirroring `cli._write_output_or_exit` |
+| 5 | samsung leaves a residual just over its gate on the weakest of its 3 corpus positives | 0.431 -> 0.404 against a 0.40 gate; the other two clear outright | a faint-mask fallback for the `binary` front-end (samsung has none), but **do not tune on n=3** -- blocked behind item 1 above |
+
+### Closed 2026-07-20 (kept for the reasoning, not for action)
+
+- **The visible-parity re-run** confirmed the doubao front-end fix: **91.8% -> 99.3%**
+  (2562/2580), gemini/jimeng/samsung unchanged. Artifact `_visible_parity_cv2_v2.csv`.
+- **The faint-mask fallback filled the whole corner box** instead of the glyph. Fixed with
+  the detector's own best-match box; full reasoning immediately below, because how it
+  escaped both parity and its own regression test is the instructive part.
+- **Three crash-class defects** found by Tier E and the /simplify review (failed writes
+  crashing on the size report, batch losing data silently, directories crashing the
+  scanner) -- see the Tier E section.
+
+**The faint-mask defect in full, because it is instructive.** The fallback added on
+2026-07-19 reads
+`np.where(resp >= _FAINT_GLYPH_LEVEL)` with the constant at `0.5`, and its comment says
+"thresholded relative to its own peak". But `tophat_response` returns **uint8 0..255**, so
+`>= 0.5` selects every pixel with value >= 1 -- the entire non-zero response, not half the
+peak. Measured against alternatives on 14 real frames (cv2 fill, detector re-run after):
+
+| mask | detector clean after | median filled area (% of corner box) |
+|---|---|---|
+| `thr0.5` (as shipped) | 100% | 120.9% |
+| `thr190` | 100% | 68.5% |
+| largest connected component at 190 | **21%** | 10.5% |
+| **the detector's own best-match box** | 100% | **58.7%** |
+
+The fix is the last row: the correlation already located the mark at a position and scale,
+so thresholding its response was always a weaker proxy for information we had. The
+connected-component variant is rejected outright -- it is the tightest but removes the mark
+on only 21% of frames, i.e. it does not cover it.
+
+Two things about how this was found are worth keeping:
+
+- **Parity could not see it.** Parity asks whether the detector is clean after removal, and
+  a mask that fills everything passes trivially. The defect was in the COST, and nothing
+  measured cost on that path. A green parity run is not evidence about mask size.
+- **The regression test could not see it either, by construction.** Its fixture is a FLAT
+  frame, where the top-hat response is non-zero only on the glyph, so every threshold gives
+  the same bounding box. Mutating the constant to an absurd 99.0 left it green. The fixture
+  now carries texture, which is the condition under which sizing matters and what real
+  corner backgrounds look like -- and it reproduces the corpus number exactly (127% of the
+  corner box, against 120.9% measured).
+
+### Latent, pre-existing, not fixed this pass
+
+The detect-fires / mask-empty silent no-op that fix 4 closed on the `tophat` front-end has a
+**narrower cousin on the `binary` front-end** (jimeng/samsung), surfaced by the /simplify
+altitude review. Binary detection gates on `coverage >= detect_min_coverage` (a FRACTION)
+while the mask gates on `xs.size >= _MIN_GLYPH_PIXELS = 20` (an absolute COUNT), both on the
+same blob. For samsung (`detect_min_coverage = 0.01`) they disagree in a small-image band
+(width ~200-294 px): detection can fire at 10-19 glyph px while the 20-px mask floor returns
+None -> the identical observable. It is NOT introduced by this work (the `else: return None`
+fall-through predates it), it sits well below real mark sizes (captured positives are
+1086-2048 px wide), and fixing it means changing binary detection's gate to match the mask's
+-- which needs its own per-mark measurement. So the "cannot drift by construction" claim is
+scoped to `tophat` (where score and box are one computation); the binary path is coupled but
+by a threshold pair that can still disagree at the edges. Fix only alongside a binary-mark
+detector change, never on its own.
+
+### Dependency alert
+
+RESOLVED 2026-07-21. `GHSA-rrmf-rvhw-rf47` (torch, `torch.jit.script` memory corruption,
+alert range `<= 2.12.1`) is closed by the torch **2.13.0** bump (the lock already carried it;
+`uv-secure` no longer flags torch). The Dependabot alert itself may still need a manual
+close in the GitHub UI if it has not auto-resolved on the lock bump.
+
+### Where detection work should go next
+
+This is the EVIDENCE behind "START HERE" item 1, not a competing list -- it records what was
+measured and ruled out, so the next session does not re-run any of it. In the order the
+evidence supports:
+
+1. **Not the ladder, not the threshold, not the landscape rung.** All three were measured
+   to completion and all three are dead ends: the dense ladder buys 7.6% of misses for a
+   21% relative rise in false fire; the score band below the gate is empty so no threshold
+   recovers the misses; and the one targeted rung that helps (1.116, landscape) adds false
+   fire at 1.7:1 because the recoveries and the false fires are the same landscape size
+   shift. Moving the landscape width fraction is also out -- detected landscape marks
+   already sit at the nominal, so it would break more than it fixes. Do not spend here.
+2. **Coverage of uncovered vendors is the largest lever.** 千问 was the head of this item
+   and is now CLOSED (registered 2026-07-21, see the harvest section above): the blocker
+   turned out to be evidence, and the TC260 producer-USCC cohort trick removed it. The
+   remaining named vendors are 元宝 (n=50), 可灵 (n=30) and cat-logo (n=19) -- each needs
+   a font-rendered silhouette, then the same calibrate-and-crossfire chain. `百度` and the
+   星绘/抖音 class are NOT in the corpus in labelable quantity (verified twice; do not
+   hunt them again). Nothing may be registered off a single frame.
+3. **A generic shared-tail template is not a shortcut.** `AI生成` is guaranteed across
+   compliant vendors by GB 45438-2025, so one template covering all of them is the obvious
+   idea -- and measured on the tophat front-end it separates a bold 千问 positive from clean
+   corners by only 0.407 vs a clean p99 of 0.298. A 4-glyph run is simply less specific
+   than a 6-glyph one. Treat it as a harvesting aid, not a detector.
+
+### Completed full runs -- do not re-run to "check"
+
+Each of these took from tens of minutes to hours and its artifact is on disk (gitignored,
+under `data/spaces/`). A later session asking "did we actually cover X" should read the
+artifact, not relaunch the sweep. Row counts are what the file held when written.
+
+| Run | What it covered | Rows | Artifact |
+|---|---|---|---|
+| A1 sidecar regression | the whole corpus re-run through `identify`, diffed against recorded verdicts | 39,314 | `_sidecar_regression.jsonl` |
+| A2 visible parity (v1 pre-fix, v2 post-fix) | detect -> remove -> re-detect per mark | 10,594 x2 | `_visible_parity_cv2{,_v2}.csv` |
+| metadata removal audit | strip-and-verify, detection/removal parity | 21,654 | `_metadata_removal_audit.csv` |
+| B1 fill quality | stamped ground truth, per backend and background | 1,200 | `_fill_quality.jsonl` |
+| pill gate audit | the pill through the PRODUCT path, not the ungated one | 2,738 | `_pill_gate_audit.jsonl` |
+| B2 / ladder headroom | the scale-ladder and threshold questions, to exhaustion | 4,469 | `_ladder_headroom_doubao.jsonl` |
+| E robustness | 34 adversarial CLI cases | -- | rerun is ~3 min, no artifact needed |
+| B4 resource ceilings | peak RSS per backend, 1 MP -> 25 MP | 12 cells | rerun is ~2 min |
+| real-example E2E | every command over real corpus examples | 28 checks | rerun is ~2 min (+ diffusion) |
+
+### Verification tiers not run
+
+- **C recall expansion** -- gated by labelling appetite. This is now the ONLY unrun tier,
+  and it is blocked on data rather than effort: every remaining detector question needs
+  labelled positives (30+ per uncovered vendor; jimeng still rests on n=14, the pill on
+  n=6, samsung on n=3).
+
+(Tiers E and B4 are now RUN -- see the sections above.)
+
+## Standing gap
+
+None of this is in `maintain.sh`, and it should not all be -- the sweeps take hours. But
+that means **no detector-accuracy or CLI-contract regression is caught automatically**
+today. The endpoint of this plan is a cheap subset (fixtures-only smoke + a sidecar diff on
+a fixed 500-image slice) that CI can run, with the full sweeps staying pre-release.

docs/watermarking-landscape.md

@@ -0,0 +1,81 @@
+# Watermarking landscape (research 2026-05-24)
+
+> Relocated verbatim from `CLAUDE.md` on 2026-06-11 to keep the always-loaded
+> context small. Long single-line entries were reformatted into paragraphs;
+> no content was changed or summarized.
+
+Who embeds what, and whether it is locally detectable (so we know which gaps are fillable). See `identify.py` for what we read.
+- **Locally detectable (open decoder, no key/API):** Stable Diffusion / SDXL / FLUX via `imwatermark` DWT-DCT (now covered by `invisible_watermark.py`). FLUX uses the same library (`black-forest-labs/flux2` `src/flux2/watermark.py`, 48-bit `0b001010101111111010000111100111001111010100101110`); SDXL is the diffusers `WATERMARK_MESSAGE` (`0b101100111110110010010000011110111011000110011110`). **Caveat: the `imwatermark` dwtDct decode is carrier-fragile on a broad class of real images, NOT just re-encode-fragile, and it is a POSITIVE-ONLY signal.** A clean encode->decode round-trip (no re-encode at all) recovers 48/48 bits on some carriers (random noise, chatgpt-1.png 48/48, firefly-1.png 45/48) but FAILS on many others — verified 2026-06-19 that a *known-embedded* watermark only round-trips 28-39/48 (below the safe `_MATCH_48` = 44 gate, random baseline ~24) on the FLUX fox sample (28), doubao-1.png (39), a 1024² minimalist-flat FLUX image (28), AND a **clean synthetic bright-flat fill with NO watermark at all (28)**. The failure does NOT track texture (firefly lapvar ~11 passes; the flat FLUX lapvar ~56 fails); it correlates with a degenerate decode where the raw bits read **all-ones (48/48 ones)** — which a clean synthetic image reproduces, so **all-ones is a CARRIER ARTIFACT, NOT a watermark signal** (a double-embed test also showed a pre-existing embed does not corrupt a second embed — no interference). Net: trust a `detect_invisible_watermark` hit, but treat a `None`/no-match as **inconclusive** whenever a positive-control embed on the same carrier does not first recover >=44/48. The 44 gate is a deliberate precision choice (lowering it would admit false positives).
+
+  **Root cause and external confirmation (deep-research 2026-06-19, adversarially verified).** This is the SCHEME's ceiling, not our usage — there is no better decoder to adopt. The imwatermark maintainers state verbatim (both the ShieldMnt and Stability-AI READMEs) that the algorithm "cannot guarantee to decode the original watermarks 100% accurately even though we don't apply any attack." Independent measurement (WMAdapter, arXiv:2406.08337 Table 2) puts dwtDct at only **~0.79 bit accuracy on CLEAN images (~38/48 bits — already below our 44 gate)**, collapsing to ~0.50 (chance) under crop/JPEG. Two code-verified + locally-reproduced mechanisms drive the content-dependent failures: (1) the decoder reads each bit as the **highest-magnitude DCT coefficient per block**, so any content coefficient exceeding the encoded target flips the bit; (2) the default embed is in the **YUV chroma channel, which 8-bit-clamps on white/bright pixels** (a +36 chroma delta survives a white-fill round-trip as only +4, ~89% loss) — this is the mechanism behind the bright-flat / minimalist failures and the all-ones degenerate decode. No maintained fork or detector decodes this scheme reliably: the WAVES benchmark (arXiv:2401.08573) relegates DWT-DCT to supplementary appendix G.5 and targets Stable Signature / Tree-Ring / StegaStamp instead; learned encoder/decoder schemes reach ~0.98-0.99 clean but are a DIFFERENT watermark class (not what SDXL/FLUX stamp). `dwtDctSvd` does not help (SDXL embeds `dwtDct`; dwtDctSvd cannot decode it, and its clean accuracy ~0.72 is lower). **Authoritative conclusion: the open DWT-DCT mark cannot be turned from positive-only into a reliable real-world detector; keep it positive-only and rely on C2PA.** (Refuted along the way: that the library is unmaintained, and that it is robust to JPEG but only fails on geometric attacks — both did not survive verification.)
+
+  Consequence for the FLUX hosted-output question (BFL Playground, FLUX.2 [pro] + FLUX.1 [dev], 2026-06-19): all samples carry the signed C2PA manifest (issuer "Black Forest Labs"); the open DWT-DCT decode returned `None`, but every available FLUX carrier (textured fox AND a minimalist-flat generation) failed the positive control (28/48), so the detector is blind on them and **whether BFL hosted output embeds the open pixel watermark is UNRESOLVED** (an earlier note here wrongly asserted it absent — overstated; a later note blamed "high texture" — also wrong, flat carriers fail too). What IS established: C2PA is the reliable FLUX identifier; the `_BITS_48` pattern is correct (round-trips on chatgpt/firefly/random). Resolving the hosted question needs a hosted FLUX carrier that first passes a >=44/48 positive control, which neither a textured nor a flat prompt produced — low priority (the open mark is only a stripped-metadata fallback).
+- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; found on real production traffic 2026-06-19, which **disproved the earlier assumption** that Canva downloads are re-encoded exports that always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
+
+**Samsung Galaxy AI** (Generative Edit / Sketch to Image / Portrait Studio on Galaxy S23 FE / S24 / S25, One UI 7+) signs C2PA as "Samsung Galaxy" with the standard `trainedAlgorithmicMedia` source type AND a proprietary `genAIType` marker; verified on real signed files 2026-05-29 (the standard scan catches the source type; `genAIType` additionally catches a Galaxy S24 file that omits it). It ALSO burns a **visible** localized wordmark into the pixels — a sparkle + "generated with AI" string in the bottom-LEFT corner (issue #37; the Italian "✦ Contenuti generati dall'AI" variant is calibrated) — removed by `samsung_engine.py` / `visible --mark samsung` (reverse-alpha, see the engine bullet); detection feeds `identify` as the medium `visible_samsung` signal. The string is locale-specific, so each locale needs its own captured alpha template.
+
+**ASUS Gallery** also signs edited photos as C2PA (`com.asus.gallery`) but with no AI source type — a signer, not an AI marker.
+
+**Black Forest Labs (FLUX)** API output signs C2PA: `claim_generator_info "Black Forest Labs API"` + a `c2pa.ai_generated_content` assertion + `trainedAlgorithmicMedia` (issuer `b"Black Forest Labs"` added to `C2PA_ISSUERS`, platform "Black Forest Labs (FLUX)").
+
+**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. All three verified on real signed files 2026-05-29. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the SAME content as **"Byteplus Pte. Ltd."** — the bare `volcengine` needle missed it, so real BytePlus output was mis-attributed to "Adobe Firefly" (an incidental "Adobe XMP" toolkit string in the file's XMP, picked up by the fallback byte-scan once the clean manifest issuer matched nothing). Added issuer `b"Byteplus"` → org "BytePlus (ByteDance)" (platform resolves to the shared "ByteDance (Doubao / Jimeng / Volcano Engine)" label via the common `ByteDance` needle) so the clean manifest issuer attributes it directly; found on real production traffic 2026-06-19. ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **NO `trainedAlgorithmicMedia`** — the generator name is the only AI signal, and the active manifest is frequently a plain `c2pa-tool` transcode with the real `Dreamina` token on an ingredient manifest. Added issuer `b"Dreamina"` → org "ByteDance (Dreamina)" with **`asserts_ai=True`** (see `constants.py`): the caBX / store-JSON byte-scan sees the token across all manifests, and the identity-AI flag lifts the AI verdict without a source-type. Mined from the retained corpus 2026-07 (7 files read `unknown` before, all now ByteDance). Registering the **issuer** `b"Bytedance Pte"` was deliberately AVOIDED — that same Singapore entity also signs non-AI CapCut edits (`CapCut/x.y` generator, `c2pa.created`, no AI marker), which must stay unattributed per the editor-vs-generator line; keying on the `Dreamina` generator token is precise.
+- **EXIF/XMP/PNG-text generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal). Three more mined from the retained corpus 2026-06-22, all no-C2PA generator stamps that previously read as no-signal: **NovelAI** (anime SD) writes its stamp in PNG `tEXt` chunks `Software="NovelAI"` / `Source="NovelAI Diffusion V4.5 <hash>"` / `Title="NovelAI generated image"` — so `exif_generator` now reads PNG text chunks (`Software`/`Source`/`Title`/`Description`), not just EXIF/XMP; **Reve** (reve.com) writes EXIF `Software` / XMP `CreatorTool` = `reve.com` (token is the full `reve.com`, not bare `reve`, to avoid false-firing on "forever"/"reverie"); **Aphrodite AI** writes EXIF `Make`/`Software` = `Aphrodite AI`.
+- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
+
+Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC** — only EXIF `Artist` = a UUID and EXIF `ImageDescription` = `Signature: <base64>` (a crypto signature, unverifiable locally without xAI's public key). This empirically kills the earlier unverified "xAI signs C2PA as xAI" lead — xAI is not even a C2PA member. `exif_generator` misses it (neither field holds an `AI_GENERATOR_TOKENS` token), so a dedicated detector `xai_signature(path)` matches the pair (`ImageDescription ~ ^Signature: [A-Za-z0-9+/=]{64,}` AND UUID `Artist`); wired into `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify` (signal `xai_signature`, platform "xAI (Grok / Aurora)").
+
+**Format confirmed stable across n=3 genuine generations:** exactly three EXIF tags (`Artist`, `ExifOffset`, `ImageDescription`), `Signature:` prefix constant, base64 payload 300-1004 chars. Two capture facts: (a) the `Artist` UUID **equals the public image id** in the asset URL (`https://imagine-public.x.ai/imagine-public/images/<uuid>.jpg`), so it is NOT a private per-user secret — only the `Signature` blob is; (b) the Grok web-UI image is a re-encoded **WebP with no signature** — the EXIF survives only in the *original* JPEG (download button or that public tokenless URL), which is why screenshots / re-encodes are metadata-stripped. A real fixture `data/samples/grok-1.jpg` plus **synthetic** JPEG fixtures (fake UUID + fake `Signature:` blob) cover the detector; never add a real Grok image carrying private content (the repo is public).
+
+**Stripped on removal too:** `remove_ai_metadata` now calls `_scrub_ai_exif` on the JPEG EXIF, which deletes the xAI Signature+UUID-Artist pair **and** any `Software`/`Make`/`Artist`/`ImageDescription` tag holding an `AI_GENERATOR_TOKENS` token (so Ideogram's `Make="Ideogram AI"` is scrubbed too), while keeping genuine camera/editor EXIF. The shared `_is_xai_signature_pair` helper (module-level compiled regexes) is the single source of truth for the pattern, used by both `xai_signature` and `_scrub_ai_exif`. (AVIF/HEIF/JXL still strip only C2PA boxes via `isobmff`, not EXIF — unchanged.)
+- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`).
+
+**Doubao** (ByteDance) uses it (verified on the real #13 sample 2026-05-25; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on the corpus 2026-05-30) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on the corpus 2026-05-28, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on the corpus 2026-05-30, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on the corpus 2026-06-10 (`ContentProducer="00119144030008867405X210002"`; 3 files read `unknown` before this form was added). The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). All three generic forms (the PNG chunk, the bare `{"AIGC":...}` object, and the bare `AIGC{...}` blob) are gated on at least one TC260 field (`_TC260_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `_TC260_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, mined from the retained corpus 2026-07 (11 files read `unknown` before — the block was found by the raw-JSON scan but rejected because none of its fields were in the producer-only gate; removal already stripped it since it lives in EXIF). In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
+- **HuggingFace-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** HuggingFace Jobs / Spaces stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID (3 on the corpus 2026-05-28, no other signal). It marks the *hosting job*, not a model — most commonly diffusion output — so it lifts an Unknown verdict to a tentative AI via `hf_only` (parallel to the visible sparkle) but never overrides a hard metadata signal; `_HF_JOB_CAVEAT` states the limit (job, not model; not proof of AI pixels). Stripped on removal (the PNG save whitelist keeps only `STANDARD_METADATA_KEYS`, so `hf-job-id` and the `AIGC` chunk are both dropped). The exact writer is not authoritatively documented (HF Jobs are generic GPU jobs), hence medium not high.
+- **No detectable signal on download (correctly reported `unknown`):** **Recraft** (PNG export is a re-encoded design export — strips everything), **Krea hosting FLUX 2** (no imwatermark despite FLUX — the host omits the encoder, same as Stability's hosted SDXL), and Midjourney (embeds nothing). Lesson: the imwatermark detector only fires on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL), not from re-hosts (Krea/Stability) or re-encoded exports (Recraft/Canva).
+- **Invisible but NOT locally detectable (proprietary, API/oracle only — same wall as SynthID):** Amazon Titan Image Generator + Nova Canvas (Bedrock `DetectGeneratedContent` API), Kakao (new SynthID image adopter, May 2026), NVIDIA Cosmos (SynthID video). No local detector possible; treat like SynthID.
+- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest (Digimarc/Imatag/Steg.AI/...) stay name-only (proprietary decoders).
+- **Built 2026-05-26 (this batch):** soft-binding `alg` vendor detection; IPTC Photo Metadata 2025.1 AI-disclosure fields (`AISystemUsed` etc.); **video C2PA metadata** detect + strip for MP4/MOV/M4V (free — `isobmff.py` is format-agnostic, MP4 is ISOBMFF); Adobe TrustMark open decoder. NOT done (out of cheap reach, per the feasibility review): visible video-logo removal (needs a video frame pipeline) and audio (SynthID/ElevenLabs/Resemble/Suno all oracle-only or unmarked).
+
+**Box detection window — now handled (v0.6.8):** detection no longer relies on a fixed first-MB read. `metadata.scan_head(path, size)` reads the first `size` bytes and, for ISOBMFF, appends the payloads of late provenance boxes found by `isobmff.scan_c2pa_region` (a file-seeking top-level box walker that skips past `mdat` by size without reading it), so a C2PA/AIGC/IPTC manifest placed AFTER a large `mdat` in a streaming/non-faststart MP4 is now caught. Every C2PA/marker byte scan (`has_ai_metadata`, `aigc_label`, `iptc_ai_system`, `synthid_source`, `exif_generator` XMP, `get_ai_metadata` soft-binding, and `identify`) goes through `scan_head`; it is behavior-neutral for non-ISOBMFF inputs (exactly `f.read(size)`).
+
+**Meta-box XMP removal — now handled (v0.6.9):** an AI-label XMP packet stored as a meta-box `mime` item (HEIF/AVIF; out of reach of the top-level box stripper) is blanked in place by `isobmff.blank_ai_xmp_packets` — it locates the packet by its `<?xpacket begin … end?>` delimiters and, if it carries an AI marker (`_AI_LABEL_MARKERS`), overwrites it with spaces of the SAME length, so box sizes / `iloc` offsets stay valid and the coded image is untouched (selective: plain non-AI XMP is left alone, mirroring the top-level uuid logic). Wired into `remove_ai_metadata`'s ISOBMFF branch after `strip_c2pa_boxes`. The remaining gap is an `Exif` meta-box *item* (rare; the AI labels are XMP) — still needs `iinf`/`iloc` surgery or exiftool.
+- **Regulatory driver (context, not a code change):** AI-content labeling mandates are expanding, which pushes more generators toward exactly the C2PA + watermark signals we read. The full per-jurisdiction table lives in README "## Legal" -- keep it there, not duplicated here. Newly added + primary-source verified 2026-05-26: **EU AI Act Article 50** machine-readable marking applicable **2026-08-02** (verified against the article text); **South Korea AI Framework Act Art. 31(3)** in force since **22 January 2026** (verified via Kim & Chang + FPF/Korea Times; Enforcement Decree accepts an invisible-watermark label); **California AB 853** (amends the CA AI Transparency Act) latent-disclosure duty operative **2026-08-02**, requiring a disclosure "permanent or extraordinarily difficult to remove" (verified against the leginfo bill text -- this is the exact disclosure our tool strips); **India IT Amendment Rules 2026** in force **2026-02-20** (verified via Chambers), which prominently-label + permanent-provenance-id all synthetic media AND **expressly prohibit removing/suppressing the label or metadata** -- the first major all-content removal ban outside China.
+
+**Removal liability (README "## Legal" disclaimer):** the tool is lawful general-purpose software; liability sits with the remover and is intent-gated -- downstream acts (fraud/deception/IP), plus US DMCA 17 USC 1202 (removing copyright-management info to conceal infringement), plus the removal-as-such bans in China + India. When extending the README table, verify each date/article against the statute/bill text before committing, not against search summaries.
+
+## Visible AI-generation marks + detection methods (deep-research 2026-07-10, adversarially verified)
+
+**Google Gemini visible "sparkle" -- tier-dependent, and spec-undocumented by Google.** Google primary sources (the Nano Banana Pro blog and the gemini.google image-generation page, both WebFetch-verified) confirm Gemini images carry BOTH the invisible SynthID (on ALL Google-AI media) AND a visible sparkle, but the visible mark is **tier-gated**: applied for FREE and Google AI **Pro** users, and **REMOVED** for Google AI **Ultra** subscribers, inside **Google AI Studio**, and on **API / dev** output. So a Google-C2PA image with NO visible sparkle is expected (Ultra / API), not evidence it is clean -- this reinforces the `identify` "no visible mark != clean" rule. The ONLY official verifier is the SynthID flow (upload to the Gemini app, ask if it is AI-generated), which reads the INVISIBLE mark; there is **no official visible-sparkle detector**, and Google publishes **no** glyph geometry / size / opacity / color / locale / placement spec. So our capture-based sparkle template is the only source of truth and cannot be validated against a vendor spec -- keep reverse-engineering from real captures (do not expect a published spec).
+
+**The faint-visible-mark precision/recall wall is fundamental, not a heuristic artifact.** The visible-watermark-detection literature has moved to LEARNED segmentation / object-detection (WDNet WACV'21 arXiv:2012.07616; SLBR ACM MM'21, open code+weights; the PRCV'18 large-scale detector; Su et al. survey 2025), but three verified findings bound what a learned detector actually buys: (1) a claim that a confidence threshold "cleanly separates" true from false matches even with a learned CNN front-end was **REFUTED** in verification (arXiv:1705.08593) -- the precision/recall wall persists even with learned features. (2) Learned detectors need a LARGE, pattern-diverse labeled dataset trained on synthetic composites (PRCV'18: 60k images / 80 watermark classes; CLWD: 60k / 160 marks), and off-distribution degradation is a documented real axis (models trained on limited-pattern LVW transfer worse; diversity of training patterns drives generalization). (3) Inference is cheap (WDNet ~8 ms at 256x256) -- the cost is the data pipeline, not runtime. Net: a learned detector shifts the frontier but does NOT remove the wall; for a SINGLE mark the cheapest next step is a small patch classifier (real-sparkle vs false-positive) on top of the existing NCC localizer, not a full segmentation model. SLBR is a ready baseline. The current NCC + false-positive gate (core-ring brightness margin + gradient-NCC crispness + white-core saturation) is a sound operating point, and the residual miss is the information-theoretic wall the literature confirms.
+
+**Visible-mark landscape beyond the registry.** Meta stamps a visible "Imagined with AI" mark (bottom-LEFT, a small symbol) on its OWN Meta AI / "Imagine" output; for third-party images it relies on C2PA / IPTC, not a visible mark. Samsung Galaxy AI additionally uses a **four-star icon** variant in a corner alongside the localized text wordmark `samsung_engine` calibrates (only the Italian text variant is covered) -- the icon is a distinct, uncovered variant. Every source agrees visible + metadata marks are trivially removable (crop / screenshot, ~2 s), which is the tool's premise.
+
+**Regulatory driver -- China GB 45438-2025 is the strongest VISIBLE-mark mandate.** The CAC / TC260 "Measures for Labeling AI-Generated Synthesized Content" (issued March 2025, **effective 2025-09-01**, technical standard **GB 45438-2025**, building on the TC260 Aug-2023 practice guide) MANDATE a **visible** label for AI images -- a visible textual mark whose height must be **>= 5% of the image's shortest side** -- plus the metadata (implicit) label. So every major Chinese platform now ships visible "AI生成"-style text marks (we cover Doubao / Jimeng; expect more CJK-text marks under this driver). By contrast EU AI Act Article 50 mandates only the MACHINE-READABLE mark (enforceable 2026-08-02, grace to 2026-12-02); a visible label is proposed and modality-specific (visible for images) but is NOT a hard "fixed icon" mandate -- a claim that Art 50 requires a clearly-visible fixed icon for images was refuted in verification. Primary-source dates verified against the article/standard text, not search summaries.
+
+## Uncovered visible marks: implementation specs (deep-research 2026-07-18)
+
+Triggered by a corpus finding: **50% of TC260-labelled uploads (1452 of 2896 unique) produced no detection at all**, and hand-inspection showed the bulk are MISSED Doubao marks (a localization defect, now fixed -- see `scale_basis` in `docs/module-internals.md`) plus a minority of genuinely uncovered vendors. Verification status is labelled per claim; treat (b)/(c) as leads, not ground truth.
+
+**GB 45438-2025 clause 5.2, the binding constraint for every Chinese mark (VERIFIED (a) -- full standard text extracted from the TC260-hosted PDF).** Verbatim requirements for an image's explicit label:
+- 应采用文字提示 (must be a TEXT prompt);
+- must contain BOTH an AI element (`人工智能` or `AI`) AND a generation element (`生成` and/or `合成`);
+- 应位于图片的边或角 -- **edge OR corner**, so bottom-right is NOT mandated (Annex C.2's bottom-right figure is a non-normative example);
+- 字型应清晰可辨 (legible typeface -- no family named);
+- **文字高度不应低于画面最短边长度的 5%** -- glyph HEIGHT >= 5% of the frame's SHORTEST side, with a note defining the shortest side for non-rectangular images.
+
+Two consequences we can exploit: (1) the 5% floor is a **scale prior** -- a compliant CN mark's glyphs are large (>= 51 px on a 1024² image), so a CN silhouette ladder can be anchored at ~5-10% of the short side instead of swept broadly, which should cut false fires; (2) every compliant string shares the tail `AI生成` / `AI合成`, so a shared suffix silhouette plus a per-vendor prefix may beat five independent templates. NOT established: whether 文字高度 means cap height, em box, or rendered bounding box (a ~1.3x spread in template scale). Sources: `https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf`, parent CAC measure `https://www.cac.gov.cn/2025-03/14/c_1743654684782215.htm` (the CAC text itself specifies no size or corner).
+
+**Alibaba Qwen -- two surfaces that differ (API tier VERIFIED (a)).** Model Studio docs state verbatim that the API adds a `Qwen-Image` watermark 在图像右下角 and 默认值为 false -- so API output is **unwatermarked by default**, and when enabled the mark is a LATIN wordmark, not CJK, and not GB-compliant in wording. The consumer app's `千问AI生成` (bottom-right) is (b) secondary only -- no Alibaba primary page states it. So Qwen needs TWO templates, and its absence is never evidence of a clean image. Source: `https://help.aliyun.com/zh/model-studio/qwen-image-api`.
+
+**星绘 is ByteDance (VERIFIED (a): Baidu Baike + App Store listing, now branded 豆包旗下, team folded into Doubao April 2025).** So `星绘AI生成` is very likely the Doubao house style -- same typeface, same corner, possibly the same top-left `AI生成` pill. Starting from the Doubao `TextMarkConfig` and swapping the two lead glyphs is the cheap path. String/position themselves are (c) inferred.
+
+**Baidu: could not establish.** No primary or credible secondary source names the exact string or position; it could be `百度AI生成`, `文心一格AI生成`, or product-specific. Harvest the glyphs from corpus positives, not the web.
+
+**Meta `Imagined with AI` (string VERIFIED (a) from Meta's own newsroom; POSITION NOT VERIFIED).** Sources conflict (bottom-left vs bottom-right) and one claims newer Meta models dropped the visible mark for invisible watermarking; none survived a fetch. Do NOT encode a corner without a corpus sample. Meta also embeds IPTC + invisible watermarks, which `identify` already reads. Source: `https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/`.
+
+**Samsung English/other locales: still not established.** Samsung's own support page says only that "A Galaxy AI watermark will appear on AI-generated images" -- no string, no corner. Every community thread carrying the exact English string returned HTTP 403 to WebFetch, so the search paraphrase (bottom-left) is deliberately NOT recorded as fact. Feature-tier detail (b): the mark is applied by Generative Edit / sketch-to-image but reportedly NOT by Object Eraser, so Samsung absence is feature-dependent. The four-star icon variant: nothing found.
+
+**The one document that would settle ByteDance placement is BLOCKED.** Douyin's 《抖音关于人工智能生成内容标识的水印与元数据规范》 aims to give AI tools a unified watermark style and position, which would cover Doubao / Jimeng / 星绘 at once. Both mirrors return HTTP 403 to WebFetch; a secondary report (b, unconfirmed) says the watermark is `AI生成` + tool name + company name placed **top-left** -- which would explain the Jimeng pill's top-left position but contradicts the GB annex's bottom-right example. Worth one retry through Chrome MCP with a real browser session.
+
+**No vendor publishes typeface, colour, opacity, plate, or margin for ANY of these marks.** The only font-adjacent requirement anywhere is GB's "legible typeface". So each synthetic silhouette's font must be calibrated against corpus positives exactly as the Jimeng pill was; candidate CJK families by platform convention (inference): HarmonyOS Sans / Source Han Sans / Noto Sans CJK SC for Android-origin apps, PingFang SC for iOS-origin.

maintain.sh

@@ -3,9 +3,27 @@
 set -euo pipefail
 
 uv sync --all-extras
-uv run uv-outdated
-uv run uv-secure --ignore-unfixed
+# uv-outdated / uv-secure run via uvx (isolated env), NOT `uv run`: resolving them
+# inside the project env crashes and, with set -e, aborts the whole gate before
+# ruff/pyright/tests (see CLAUDE.md "Test and lint").
+uvx uv-outdated
+# uv-secure prints its verdict but can then crash in an internal teardown with a
+# NON-ZERO exit -- observed as "annotated-doc raised exception" and later "anyio raised
+# exception"; both are bugs in uv-secure's OWN uvx env, not a project vulnerability. With
+# set -e that teardown crash aborts the whole gate before ruff/pyright/tests. So gate on
+# the VERDICT, not the exit code: capture the output, accept the run when uv-secure
+# reported all-safe (even if it then crashed), but still FAIL on a real finding (no
+# all-safe line) so a genuine CVE is never masked, and fail loud if it never got a
+# verdict at all (so a broken run is never silently skipped).
+secure_out="$(uvx uv-secure --ignore-unfixed 2>&1)" || true
+printf '%s\n' "$secure_out"
+if ! grep -qE "No vulnerabilities or maintenance issues detected|All dependencies appear safe" <<<"$secure_out"; then
+    echo "maintain.sh: uv-secure reported a finding or failed before its verdict -- triage before committing." >&2
+    exit 1
+fi
 uv run ruff check --fix
 uv run ruff format
-uv run pyright
+# Scoped to src/: a full-project pyright run OOM-crashes node on this ML-heavy
+# repo (see CLAUDE.md "Test and lint"); src/ is the authoritative strict gate.
+uv run pyright src/
 uv run pytest -n auto

packaging/conda/recipe.yaml

@@ -0,0 +1,63 @@
+schema_version: 1
+
+context:
+  version: "0.10.1"
+  python_min: "3.10"
+
+package:
+  name: remove-ai-watermarks
+  version: ${{ version }}
+
+source:
+  url: https://pypi.org/packages/source/r/remove-ai-watermarks/remove_ai_watermarks-${{ version }}.tar.gz
+  sha256: ec92b450363d947cd897f0b75c18e75c2988ff79314ea247333ba892470b5a77
+
+build:
+  noarch: python
+  number: 0
+  script: python -m pip install . -vv --no-deps --no-build-isolation
+
+requirements:
+  host:
+    - python ${{ python_min }}.*
+    - pip
+    - hatchling
+  run:
+    - python >=${{ python_min }}
+    - pillow >=10.0.0
+    - piexif >=1.1.3
+    - numpy >=1.24.0
+    - py-opencv >=4.8.0
+    - click >=8.0.0
+    - python-dotenv >=1.0.0
+
+tests:
+  - python:
+      imports:
+        - remove_ai_watermarks
+        - remove_ai_watermarks.identify
+        - remove_ai_watermarks.metadata
+      python_version: ${{ python_min }}.*
+      # pip_check defaults to true; disable it explicitly. `pip check` fails on
+      # the conda-forge piexif `py_2` build, whose stale 2019 metadata pip reads
+      # as "piexif 1.1.3 is not supported on this platform" -- even though the
+      # package installs, imports, and works. The imports test verifies loading.
+      pip_check: false
+
+about:
+  homepage: https://github.com/wiltodelta/remove-ai-watermarks
+  summary: Remove visible and invisible AI watermarks from images
+  description: |
+    Detect and remove visible AI watermarks (Gemini / Nano Banana sparkle,
+    ByteDance Doubao and Jimeng, Samsung Galaxy AI) and strip AI-provenance
+    metadata (C2PA, EXIF, IPTC, PNG text chunks) from images. The core package
+    covers the identify / metadata / visible / erase command surface; optional
+    pip extras add SynthID diffusion removal and additional invisible-watermark
+    detectors.
+  license: Apache-2.0
+  license_file: LICENSE
+  repository: https://github.com/wiltodelta/remove-ai-watermarks
+
+extra:
+  recipe-maintainers:
+    - wiltodelta

pyproject.toml

@@ -1,28 +1,69 @@
 [project]
 name = "remove-ai-watermarks"
-version = "0.7.0"
-description = "Remove visible and invisible AI watermarks from images (Gemini / Nano Banana, ChatGPT, Stable Diffusion)"
+version = "0.16.0"
+description = "AI watermark remover: strip visible and invisible AI watermarks (Gemini / Nano Banana sparkle, SynthID) and provenance metadata (C2PA, EXIF) from images"
 readme = "README.md"
 requires-python = ">=3.10"
-license = {text = "MIT"}
+license = {text = "Apache-2.0"}
+keywords = [
+    "ai-watermark",
+    "ai-watermark-remover",
+    "watermark-remover",
+    "watermark-removal",
+    "remove-watermark",
+    "synthid",
+    "c2pa",
+    "content-credentials",
+    "nano-banana",
+    "gemini",
+    "gemini-watermark",
+    "ai-metadata",
+    "metadata-removal",
+    "provenance",
+    "exif",
+    "stable-diffusion",
+    "comfyui",
+    "ai-detection",
+]
 classifiers = [
-    "License :: OSI Approved :: MIT License",
+    "Development Status :: 4 - Beta",
+    "Environment :: Console",
+    "Intended Audience :: Developers",
+    "Intended Audience :: End Users/Desktop",
+    "License :: OSI Approved :: Apache Software License",
     "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3 :: Only",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
     "Programming Language :: Python :: 3.13",
     "Topic :: Multimedia :: Graphics",
+    "Topic :: Multimedia :: Graphics :: Graphics Conversion",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
     "Topic :: Scientific/Engineering :: Image Processing",
+    "Topic :: Security",
+    "Topic :: Utilities",
 ]
 dependencies = [
     "pillow>=10.0.0",
+    # HEIC/AVIF pixel decode for the removal path (iPhone photos, modern exports):
+    # OpenCV cannot decode these containers, so image_io.imread falls back to Pillow
+    # and pillow-heif (bundled libheif, prebuilt wheels) registers the HEIF+AVIF
+    # openers. The metadata path already handles them via a plugin-free binary scan;
+    # this closes the same gap for the pixel path so `visible`/`all` work on them.
+    "pillow-heif>=0.13.0",
     "piexif>=1.1.3",
     "numpy>=1.24.0",
     "opencv-python-headless>=4.8.0",
     "click>=8.0.0",
-    "rich>=13.0.0",
     "python-dotenv>=1.0.0",
+    # Official C2PA reader (Content Authenticity Initiative, MIT/Apache-2.0). The
+    # primary, spec-tracking manifest parser for the identify/metadata path; the
+    # hand-rolled caBX/CBOR scanner in noai/c2pa.py is kept only as a fallback for
+    # synthetic/partial blobs the validator rejects. Binary wheel (Rust), but the
+    # import is light (no torch/numpy) so it fits the dependency-light identify
+    # host. Prebuilt wheels cover the full CI matrix (linux/macos/windows).
+    "c2pa-python>=0.35.0",
 ]
 
 [project.optional-dependencies]
@@ -50,10 +91,7 @@ gpu = [
     "transformers>=5,<6",
     "tokenizers>=0.22,<0.23",
     "accelerate>=0.25.0",
-    "controlnet-aux>=0.0.9",
     "safetensors",
-    "ultralytics>=8.0.0",
-    "color-matcher>=0.5.0",
 ]
 # Open invisible-watermark (imwatermark) decoder for detecting the DWT-DCT
 # watermarks embedded by Stable Diffusion / SDXL / FLUX. Optional because it
@@ -80,21 +118,40 @@ lama = [
     "onnxruntime>=1.16.0",
     "huggingface-hub>=0.20.0",
 ]
+# Lightweight inpaint backend -- MI-GAN via onnxruntime (andraniksargsyan/migan,
+# MIT). CPU, no torch. Model (~28 MB) downloaded on first use and cached by
+# huggingface_hub; never bundled. ~700-950 MB peak RAM / ~0.19 s/call -- the
+# droplet-friendly tier (vs big-LaMa's ~4.7 GB) and the preferred inpaint backend
+# for the visible-mark fallback when installed. Same runtime as `lama`.
+migan = [
+    "onnxruntime>=1.16.0",
+    "huggingface-hub>=0.20.0",
+]
+# Optional pre-diffusion super-resolution for small inputs (Real-ESRGAN). Loaded via
+# spandrel (MIT) -- a pure model-loader with NO basicsr dependency (it pulls only
+# torch / torchvision / safetensors / numpy / einops).
+# The Real-ESRGAN weights (BSD-3-Clause) download on first use and are cached; they
+# are never bundled. CPU works but is slow on large inputs -- it is meant for the
+# pre-diffusion upscale of SMALL inputs (and the GPU worker). Guarded by
+# upscaler.is_available(); the default upscaler stays Lanczos (cv2, no deps). The
+# weights are fetched with torch.hub (bundled with spandrel's torch), so no extra
+# download dependency is needed.
+esrgan = [
+    "spandrel>=0.3.0",
+]
 dev = [
     "pytest>=8.0.0",
     "pytest-cov>=4.1.0",
+    "pytest-xdist>=3.5.0",
     "ruff>=0.4.0",
     "pyright>=1.1.0",
     "invisible-watermark>=0.2.0",
+    # maintain.sh helpers; they only support newer Pythons, so gate them by
+    # marker to keep the py3.10 resolution (and CI matrix) solvable.
+    "uv-outdated>=0.1.0; python_version >= '3.12'",
+    "uv-secure>=0.12.0; python_version >= '3.12'",
 ]
-all = ["remove-ai-watermarks[gpu,detect,trustmark,lama,dev]"]
-
-# diffusers 0.38.0 (security fix for GHSA-98h9-4798-4q5v) declares a dependency
-# on safetensors>=0.8.0rc0 — a pre-release. Allow pre-releases globally so the
-# resolver can satisfy that. Drop once diffusers publishes a release with a
-# stable safetensors pin (or once safetensors 0.8.0 stable is out).
-[tool.uv]
-prerelease = "allow"
+all = ["remove-ai-watermarks[gpu,detect,trustmark,lama,migan,dev]"]
 
 # PyTorch Intel-GPU (XPU) wheel index. ``explicit = true`` keeps it inert for
 # the default CPU/CUDA install: uv consults it only when a torch install
@@ -118,6 +175,13 @@ build-backend = "hatchling.build"
 [tool.hatch.build.targets.wheel]
 packages = ["src/remove_ai_watermarks"]
 
+[tool.hatch.build.targets.sdist]
+# Keep the source distribution small: ship the package + metadata, not the
+# committed test corpora / calibration captures under data/ (tens of MB --
+# synthid_corpus images + the visible-mark captures), which pushed the 0.8.0
+# sdist past PyPI's per-project file-size limit (the wheel ships only src/).
+exclude = ["/data"]
+
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 pythonpath = ["src"]
@@ -136,10 +200,10 @@ ignore = [
 ]
 
 [tool.ruff.lint.per-file-ignores]
+"scripts/*.py" = ["G004", "S108", "S310", "T20"]
 "tests/*.py" = ["ANN", "S101", "S105", "S106", "S108"]
 "src/remove_ai_watermarks/noai/watermark_remover.py" = ["S603", "S606", "S607", "T201"]  # subprocess calls for auto-install/CUDA fix
 "src/remove_ai_watermarks/noai/c2pa.py" = ["S110"]  # try-except-pass for corrupt file handling
-"src/remove_ai_watermarks/noai/ctrlregen/engine.py" = ["S101", "S603"]  # assert for loaded state, subprocess for auto-install
 
 [tool.ruff.format]
 quote-style = "double"
@@ -161,33 +225,3 @@ reportUnknownMemberType = false
 reportUnknownArgumentType = false
 reportUnknownVariableType = false
 reportMissingTypeArgument = false
-
-# CtrlRegen is a torch/diffusers/controlnet-aux boundary: those libs ship no
-# usable types, so strict pyright cannot know the tensor element types. Relax the
-# unknown-type rules for this subpackage only (mirrors the per-file pragmas used
-# for the cv2 engines); the rest of the codebase stays strict.
-[[tool.pyright.executionEnvironments]]
-root = "src/remove_ai_watermarks/noai/ctrlregen"
-reportUnknownMemberType = false
-reportUnknownArgumentType = false
-reportUnknownVariableType = false
-reportUnknownParameterType = false
-reportMissingTypeArgument = false
-reportMissingTypeStubs = false
-reportMissingImports = false
-reportArgumentType = false
-reportAssignmentType = false
-reportReturnType = false
-reportCallIssue = false
-reportIndexIssue = false
-reportOperatorIssue = false
-reportOptionalMemberAccess = false
-reportOptionalCall = false
-reportOptionalSubscript = false
-reportOptionalOperand = false
-reportAttributeAccessIssue = false
-reportPrivateImportUsage = false
-reportPrivateUsage = false
-reportInvalidTypeForm = false
-reportConstantRedefinition = false
-reportUnnecessaryComparison = false

scripts/_plain_console.py

@@ -0,0 +1,61 @@
+"""Minimal plain-text stand-ins for the rich Console/Table API.
+
+rich was dropped as a project dependency (see the CLI plain-text refactor), but
+the analysis scripts still printed through it. These shims keep the scripts
+runnable without rich: ``[bold]``/``[/]``-style markup is stripped and tables
+render as aligned plain text. Output goes through ``click.echo`` to match the
+package CLI (no bare ``print`` in tooling).
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Any
+
+import click
+
+# Matches rich style tags: the bare close ``[/]`` and named open/close tags such
+# as ``[yellow]``, ``[bold yellow]``, ``[/green]``. Anchored to lowercase-letter
+# starts so numeric/data brackets (``[1024]``, ``[file.png]``) are left intact.
+_MARKUP = re.compile(r"\[(?:/|/?[a-z][a-z ]*)\]")
+
+
+def _strip(obj: Any) -> str:
+    return _MARKUP.sub("", str(obj))
+
+
+class Table:
+    """Drop-in for ``rich.table.Table`` covering add_column/add_row + render."""
+
+    def __init__(self, *args: Any, title: str | None = None, **kwargs: Any) -> None:
+        self.title = title
+        self._headers: list[str] = []
+        self._rows: list[list[str]] = []
+
+    def add_column(self, header: str = "", *args: Any, **kwargs: Any) -> None:
+        self._headers.append(_strip(header))
+
+    def add_row(self, *cells: Any) -> None:
+        self._rows.append([_strip(c) for c in cells])
+
+    def render(self) -> str:
+        all_rows = ([self._headers] if self._headers else []) + self._rows
+        cols = max((len(r) for r in all_rows), default=0)
+        widths = [0] * cols
+        for row in all_rows:
+            for i, cell in enumerate(row):
+                widths[i] = max(widths[i], len(cell))
+        lines: list[str] = []
+        if self.title:
+            lines.append(_strip(self.title))
+        if self._headers:
+            lines.append("  ".join(h.ljust(widths[i]) for i, h in enumerate(self._headers)))
+        lines.extend("  ".join(c.ljust(widths[i]) for i, c in enumerate(row)) for row in self._rows)
+        return "\n".join(lines)
+
+
+class Console:
+    """Drop-in for ``rich.console.Console`` covering ``print`` (with Table)."""
+
+    def print(self, *objects: Any, **kwargs: Any) -> None:
+        click.echo(" ".join(o.render() if isinstance(o, Table) else _strip(o) for o in objects))

scripts/cjk_tail_probe.py

@@ -0,0 +1,272 @@
+"""Can one GENERIC template cover the CJK AI labels no per-vendor detector fires on?
+
+THE OPPORTUNITY
+  Corpus inspection of doubao-provenance misses turned up `千问AI生成` (Alibaba Qwen) and
+  `百度 AI生成` (Baidu) sitting in the same bottom-right corner as the marks we do cover,
+  bold and plainly legible, with no detector able to fire on either. `docs/...landscape`
+  puts uncovered vendors at ~6% of sampled images -- larger than any tuning gain left in
+  the covered ones (a dense scale ladder recovers 1 miss in 26; see ladder_headroom.py).
+
+WHY A SHARED TEMPLATE IS EVEN PLAUSIBLE
+  China's GB 45438-2025 mandates the label, and every compliant vendor ends it with the
+  same run: `AI生成`. The vendor PREFIX varies (豆包 / 千问 / 百度 / 星绘) and is what makes
+  per-vendor silhouettes expensive; the TAIL is guaranteed. So the tail is the part worth
+  detecting, and vendor attribution -- which the shared tail cannot provide anyway
+  (measured 千问-vs-doubao AUC ~0.5) -- moves to metadata, where it is reliable.
+
+WHY IT IS WORTH RETRYING NOW
+  `render_vendor_silhouettes.py` records 千问 as ruled out, and its reasoning was sound:
+  the then-current front-end binarized the glyph, and a thin translucent overlay shatters
+  into specks that no template can match. It closes by naming what would be needed --
+  "a detection front-end that does not depend on binarizing the glyph (grayscale/edge
+  correlation on the raw top-hat)". That front-end now EXISTS: `detect_frontend="tophat"`
+  was built for doubao and correlates a soft template against the continuous response.
+  The blocker was removed by unrelated work, so the ruling deserves re-measurement rather
+  than inheritance.
+
+THE MEASUREMENT
+  Three arms, one scorer (the tail template on the tophat response, swept over a dense
+  size ladder because an unregistered vendor's glyph size is genuinely unknown -- the one
+  place a dense ladder earns its cost):
+
+    covered    a registered mark already fires -- a sanity arm; the tail is inside those
+               marks too, so a tail template that cannot score THESE is simply broken
+    uncovered  TC260 provenance but no detector fires -- the target population
+    clean      no metadata signal at all -- the precision arm
+
+  The decision rests on whether `uncovered` carries a high-scoring subpopulation that
+  `clean` does not. It does NOT rest on the raw rate: TC260 provenance does not imply a
+  visible mark, so a modest fire rate on that arm is expected even if the detector is
+  perfect. Any threshold this suggests must then be confirmed by eyeballing the crops it
+  selects -- the script writes a contact sheet for exactly that.
+
+DATA SAFETY
+  Corpus images are user uploads: read-only, local analysis, gitignored output. The
+  template is font-rendered synthetic, never cut from a user upload.
+
+    uv run python scripts/cjk_tail_probe.py --n 6000
+"""
+
+from __future__ import annotations
+
+import argparse
+import glob
+import json
+import os
+import random
+import sys
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from pathlib import Path
+from typing import Any
+
+import cv2
+import numpy as np
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+sys.path.insert(0, str(Path(__file__).parent))
+
+REPO = Path(__file__).resolve().parents[1]
+CORPUS = REPO / "data" / "spaces" / "originals"
+OUT = REPO / "data" / "spaces" / "_cjk_tail_probe.jsonl"
+# Cached under the gitignored data dir, not in scripts/: this is a probe artifact, not a
+# product asset. If the tail mark is ever registered, `render_vendor_silhouettes.py` is
+# what writes the committed silhouette into src/.../assets/.
+TAIL_PNG = REPO / "data" / "spaces" / "_cjk_tail_silhouette.png"
+
+# The tail is a fraction of a full vendor mark's width (`豆包AI生成` is ~5 CJK widths,
+# `AI生成` ~3), and the prefix length differs per vendor, so the size is genuinely
+# unknown here -- unlike a registered mark, whose fraction is calibrated. Hence a wide,
+# dense ladder: this is the case a dense ladder is actually for.
+TAIL_SCALES = tuple(round(0.30 * (1.08**i), 4) for i in range(18))  # ~0.30 .. ~1.15
+
+
+def build_tail_silhouette() -> np.ndarray:
+    """Font-rendered `AI生成` silhouette, cached next to this script (data-safe)."""
+    if TAIL_PNG.exists():
+        img = cv2.imread(str(TAIL_PNG), cv2.IMREAD_GRAYSCALE)
+        if img is not None:
+            return img
+    from render_vendor_silhouettes import render
+
+    sil = render("AI生成", width=200)
+    cv2.imwrite(str(TAIL_PNG), sil)
+    return sil
+
+
+def tail_score(engine: Any, image: np.ndarray, sil: np.ndarray) -> tuple[float, float]:
+    """Best tail correlation over the size ladder; returns (score, winning_scale)."""
+    loc = engine.locate(image)
+    resp = engine.tophat_response(image, loc)
+    if resp is None:
+        return (0.0, 0.0)
+    c = engine.config
+    base = engine.scale_base(image)
+    ar = sil.shape[0] / max(1, sil.shape[1])
+    best, best_s = 0.0, 0.0
+    for s in TAIL_SCALES:
+        gw = max(12, int(c.alpha_width_frac * base * s))
+        gh = max(6, int(gw * ar))
+        if gw >= resp.shape[1] or gh >= resp.shape[0]:
+            continue
+        t = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA)
+        v = float(cv2.matchTemplate(resp, t, cv2.TM_CCOEFF_NORMED).max())
+        if v > best:
+            best, best_s = v, s
+    return (best, best_s)
+
+
+def _one(path_str: str) -> dict[str, Any] | None:
+    from remove_ai_watermarks.api import visible_provenance
+    from remove_ai_watermarks.doubao_engine import DoubaoEngine
+    from remove_ai_watermarks.identify import identify
+    from remove_ai_watermarks.image_io import imread
+    from remove_ai_watermarks.watermark_registry import detect_marks
+
+    path = Path(path_str)
+    try:
+        prov = visible_provenance(path)
+    except Exception:
+        return None
+    tc260 = bool({"doubao", "jimeng"} & prov)
+
+    if not tc260:
+        try:
+            if identify(path, check_visible=False).signals:
+                return None  # some other provenance: neither target nor clean control
+        except Exception:
+            return None
+
+    img = imread(path_str)
+    if img is None or min(img.shape[:2]) < 200:
+        return None
+    try:
+        fired = [d.key for d in detect_marks(img) if d.detected]
+    except Exception:
+        return None
+
+    if fired:
+        arm = "covered"
+    elif tc260:
+        arm = "uncovered"
+    else:
+        arm = "clean"
+
+    score, scale = tail_score(DoubaoEngine(), img, build_tail_silhouette())
+    return {
+        "src": path.name,
+        "arm": arm,
+        "fired": fired,
+        "tail_score": round(score, 4),
+        "tail_scale": scale,
+    }
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--n", type=int, default=6000)
+    ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2))
+    ap.add_argument("--out", type=Path, default=OUT)
+    ap.add_argument("--report-only", action="store_true")
+    ap.add_argument(
+        "--sheet-at", type=float, default=0.0, help="write a contact sheet of `uncovered` crops scoring >= this"
+    )
+    a = ap.parse_args()
+
+    if a.report_only:
+        rows = [json.loads(x) for x in a.out.read_text(encoding="utf-8").splitlines() if x.strip()]
+        report(rows)
+        if a.sheet_at:
+            contact_sheet(rows, a.sheet_at)
+        return
+
+    build_tail_silhouette()
+    pool = glob.glob(str(CORPUS / "*" / "*"))
+    random.Random(19).shuffle(pool)  # noqa: S311 -- deterministic sampling, not crypto
+    pool = pool[: a.n]
+    print(f"scanning {len(pool)} corpus files  workers={a.workers}")
+    print(f"tail ladder {TAIL_SCALES[0]} .. {TAIL_SCALES[-1]} ({len(TAIL_SCALES)} rungs)\n", flush=True)
+
+    rows: list[dict[str, Any]] = []
+    a.out.parent.mkdir(parents=True, exist_ok=True)
+    with open(a.out, "w", encoding="utf-8") as fh, ProcessPoolExecutor(max_workers=a.workers) as ex:
+        futures = [ex.submit(_one, p) for p in pool]
+        for i, fut in enumerate(as_completed(futures), 1):
+            try:
+                rec = fut.result()
+            except Exception:  # noqa: S112 -- one bad file must not kill the sweep
+                continue
+            if rec is None:
+                continue
+            fh.write(json.dumps(rec) + "\n")
+            rows.append(rec)
+            if i % 500 == 0:
+                fh.flush()
+                print(f"  {i}/{len(pool)}  usable={len(rows)}", flush=True)
+    report(rows)
+    if a.sheet_at:
+        contact_sheet(rows, a.sheet_at)
+
+
+def report(rows: list[dict[str, Any]]) -> None:
+    arms = {k: [r for r in rows if r["arm"] == k] for k in ("covered", "uncovered", "clean")}
+    print(f"\n{'=' * 80}\nGENERIC CJK TAIL PROBE  ({', '.join(f'{k}={len(v)}' for k, v in arms.items())})\n{'=' * 80}")
+    print("\nScore distribution of the shared `AI生成` tail on the tophat response\n")
+    print(f"{'arm':11s} {'n':>6s} {'p50':>7s} {'p75':>7s} {'p90':>7s} {'p95':>7s} {'p99':>7s} {'max':>7s}")
+    for k, v in arms.items():
+        if not v:
+            continue
+        s = np.array([r["tail_score"] for r in v])
+        qs = [np.percentile(s, q) for q in (50, 75, 90, 95, 99)]
+        print(f"{k:11s} {len(v):6d} " + " ".join(f"{q:7.3f}" for q in qs) + f" {s.max():7.3f}")
+
+    if arms["clean"] and arms["uncovered"]:
+        clean = np.array([r["tail_score"] for r in arms["clean"]])
+        unc = np.array([r["tail_score"] for r in arms["uncovered"]])
+        print("\n\nOPERATING POINTS -- threshold set on the CLEAN arm, yield read on `uncovered`")
+        print("`clean` fires are the cost; `uncovered` fires are candidate recoveries that")
+        print("still need eyeballing, since TC260 provenance does not imply a visible mark.\n")
+        print(f"{'threshold':>10s} {'clean fire':>12s} {'uncovered fire':>16s} {'candidates':>12s}")
+        for q in (95, 97.5, 99, 99.5, 99.9):
+            t = float(np.percentile(clean, q))
+            cf = float((clean >= t).mean())
+            uf = float((unc >= t).mean())
+            print(f"{t:10.3f} {100 * cf:11.2f}% {100 * uf:15.1f}% {int((unc >= t).sum()):12d}")
+
+    cov = arms["covered"]
+    if cov:
+        s = np.array([r["tail_score"] for r in cov])
+        print(f"\n\nSANITY: on frames where a registered mark fires, the tail scores median {np.median(s):.3f}.")
+        print("A tail template that cannot score these is broken, whatever it does elsewhere.")
+
+
+def contact_sheet(rows: list[dict[str, Any]], thresh: float, limit: int = 30) -> None:
+    """Crop the corner of the top-scoring `uncovered` frames so the fires can be judged."""
+    from remove_ai_watermarks.doubao_engine import DoubaoEngine
+    from remove_ai_watermarks.image_io import imread
+
+    eng = DoubaoEngine()
+    picks = sorted(
+        (r for r in rows if r["arm"] == "uncovered" and r["tail_score"] >= thresh),
+        key=lambda r: -r["tail_score"],
+    )[:limit]
+    tiles = []
+    for r in picks:
+        hits = glob.glob(str(CORPUS / "*" / r["src"]))
+        if not hits:
+            continue
+        img = imread(hits[0])
+        if img is None:
+            continue
+        loc = eng.locate(img)
+        crop = img[loc.y : loc.y + loc.h, loc.x : loc.x + loc.w]
+        if crop.size:
+            tiles.append(cv2.resize(crop, (320, 96), interpolation=cv2.INTER_AREA))
+    if tiles:
+        dest = REPO / "data" / "spaces" / "_cjk_tail_sheet.png"
+        cv2.imwrite(str(dest), np.vstack(tiles))
+        print(f"\ncontact sheet ({len(tiles)} crops, score >= {thresh:.3f}) -> {dest}")
+        print("scores: " + ", ".join(f"{r['tail_score']:.2f}" for r in picks[: len(tiles)]))
+
+
+if __name__ == "__main__":
+    main()

scripts/controlnet_sweep.py

@@ -0,0 +1,348 @@
+"""ControlNet-as-removal-pipeline prototype sweep (issue #35 / Jacob).
+
+Research prototype, NOT a shipped pipeline. It tests whether a full-image
+SDXL-native ControlNet-conditioned img2img can REPLACE plain SDXL img2img as the
+watermark remover: a single structure-guided regeneration that scrubs an invisible
+robust watermark (SynthID) everywhere while keeping fine detail and small/CJK text
+legible. See docs/controlnet-removal-pipeline-research.md for the full rationale.
+
+The make-or-break tension (from the watermark-removal-attack literature): the
+denoise strength high enough to scrub the watermark deforms text, while the
+conditioning strong enough to keep text may spare the watermark. There is no local
+SynthID detector, so this script CANNOT decide removal on its own -- it produces
+one output per (control, strength, conditioning-scale) cell plus an index, and YOU
+verify each output by hand in the Gemini app ("Verify with SynthID") and judge text
+legibility visually. Fill the verdict columns in the emitted index, then read off
+the Pareto cell (oracle clean AND text legible).
+
+Pipeline: stabilityai/stable-diffusion-xl-base-1.0 +
+  - canny: xinsir/controlnet-canny-sdxl-1.0  (control = cv2.Canny(gray, 100, 200))
+  - tile:  xinsir/controlnet-tile-sdxl-1.0   (control = the resized original, no preproc)
+StableDiffusionXLControlNetImg2ImgPipeline (image=init, control_image=control).
+
+Needs the gpu extra (torch + diffusers) and cv2. Runs locally on 32 GB MPS in
+fp32 (MPS fp16 decodes to all-black NaN -- issue #29 -- so fp32 is the default on
+mps/cpu, fp16 only on cuda/xpu); a dedicated GPU is not required for 1024 px. Run:
+
+    uv run python scripts/controlnet_sweep.py path/to/watermarked.png -o sweep_out
+    uv run python scripts/controlnet_sweep.py img.png --control canny tile \\
+        --strength 0.3 0.5 0.7 1.0 --scale 0.6 1.0 --size 1024
+"""
+
+from __future__ import annotations
+
+# torch/diffusers/cv2 ship no usable types; relax the unknown-type + private-import
+# rules for this boundary script (mirrors scripts/visible_alpha_solve.py and the
+# cv2/torch engine modules). Pure-logic helpers here stay correct regardless.
+# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false, reportPrivateImportUsage=false
+import argparse
+import contextlib
+import csv
+import importlib.util
+import logging
+import sys
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
+
+from _plain_console import Console, Table
+
+if TYPE_CHECKING:
+    from PIL import Image
+
+logging.basicConfig(level=logging.INFO, format="%(message)s")
+log = logging.getLogger(__name__)
+console = Console()
+
+BASE_MODEL = "stabilityai/stable-diffusion-xl-base-1.0"
+FP16_VAE = "madebyollin/sdxl-vae-fp16-fix"
+CONTROLNETS = {
+    "canny": "xinsir/controlnet-canny-sdxl-1.0",
+    "tile": "xinsir/controlnet-tile-sdxl-1.0",
+}
+
+# A neutral quality prompt: the goal is faithful regeneration, not creative edits.
+PROMPT = "best quality, high quality, sharp, detailed, photographic"
+NEGATIVE_PROMPT = "blurry, lowres, deformed, distorted text, garbled text, watermark, jpeg artifacts"
+
+
+def pick_device(requested: str) -> str:
+    """Resolve the inference device without the CUDA-reinstaller side effect.
+
+    Deliberately does NOT call the package ``get_device`` (which can trigger a
+    torch-CUDA reinstall+restart). A research script should never do that.
+    """
+    import torch
+
+    if requested != "auto":
+        return requested
+    if torch.cuda.is_available():
+        return "cuda"
+    if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
+        return "mps"
+    return "cpu"
+
+
+def resolve_dtype(device: str, requested: str) -> Any:
+    """fp16 only on cuda/xpu; fp32 on cpu AND mps, unless overridden.
+
+    MPS fp16 produces all-black NaN output here (the SDXL UNet/VAE overflows on
+    the Metal backend -- issue #29; even the fp16-fix VAE does not save it), so the
+    production pipeline runs fp32 on MPS and so do we. fp32 SDXL + an SDXL ControlNet
+    at 1024 fits 32 GB unified memory with vae-tiling + attention-slicing.
+    """
+    import torch
+
+    if requested == "fp16":
+        return torch.float16
+    if requested == "fp32":
+        return torch.float32
+    return torch.float16 if device in {"cuda", "xpu"} else torch.float32
+
+
+def fit_size(image: Image.Image, long_side: int) -> Image.Image:
+    """Resize so the long side is ``long_side``, each dim a multiple of 8 (SDXL)."""
+    from PIL import Image as PILImage
+
+    w, h = image.size
+    scale = long_side / max(w, h)
+    nw = max(8, round(w * scale) // 8 * 8)
+    nh = max(8, round(h * scale) // 8 * 8)
+    if (nw, nh) == (w, h):
+        return image
+    return image.resize((nw, nh), PILImage.Resampling.LANCZOS)
+
+
+def make_control_image(init: Image.Image, control: str) -> Image.Image:
+    """Build the ControlNet conditioning image for the given control type.
+
+    canny: cv2.Canny(gray, 100, 200) -> 3-channel edge map (xinsir canny recipe).
+    tile:  the init image itself, no preprocessing (xinsir tile recipe).
+    """
+    import cv2
+    import numpy as np
+    from PIL import Image as PILImage
+
+    if control == "tile":
+        return init
+    rgb = np.array(init.convert("RGB"))
+    gray = cv2.cvtColor(rgb, cv2.COLOR_RGB2GRAY)
+    edges = cv2.Canny(gray, 100, 200)
+    edges_rgb = np.stack([edges, edges, edges], axis=-1)
+    return PILImage.fromarray(edges_rgb)
+
+
+def psnr(a: Image.Image, b: Image.Image) -> float:
+    """Coarse global fidelity proxy vs the original (NOT a text or watermark metric)."""
+    import numpy as np
+
+    x = np.asarray(a.convert("RGB"), dtype=np.float64)
+    y = np.asarray(b.convert("RGB").resize(a.size), dtype=np.float64)
+    mse = float(np.mean((x - y) ** 2))
+    if mse == 0.0:
+        return 99.0
+    return float(10.0 * np.log10((255.0**2) / mse))
+
+
+def load_pipeline(control: str, device: str, dtype: Any) -> Any:
+    """Load SDXL base + the chosen SDXL ControlNet as an img2img pipeline."""
+    import torch
+    from diffusers import (
+        AutoencoderKL,
+        ControlNetModel,
+        StableDiffusionXLControlNetImg2ImgPipeline,
+    )
+
+    console.print(f"Loading {CONTROLNETS[control]} ({control}) ...")
+    controlnet = ControlNetModel.from_pretrained(CONTROLNETS[control], torch_dtype=dtype)
+    load_kwargs: dict[str, Any] = {"controlnet": controlnet, "torch_dtype": dtype}
+    if dtype == torch.float16:
+        # The stock SDXL VAE decodes to NaN/black in fp16; the fp16-fix VAE is the
+        # same swap the production pipeline uses (_SDXL_FP16_VAE_ID).
+        load_kwargs["vae"] = AutoencoderKL.from_pretrained(FP16_VAE, torch_dtype=dtype)
+    pipe = StableDiffusionXLControlNetImg2ImgPipeline.from_pretrained(BASE_MODEL, **load_kwargs)
+    pipe = pipe.to(device)
+    pipe.set_progress_bar_config(disable=True)
+    if device != "cpu":
+        # Keep the 1024 px + extra-ControlNet peak inside 32 GB unified memory.
+        with contextlib.suppress(Exception):
+            pipe.enable_vae_tiling()
+        with contextlib.suppress(Exception):
+            pipe.enable_attention_slicing()
+    return pipe
+
+
+def run_cell(
+    pipe: Any,
+    init: Image.Image,
+    control_image: Image.Image,
+    strength: float,
+    scale: float,
+    steps: int,
+    guidance: float,
+    seed: int,
+) -> Image.Image:
+    """Run one ControlNet img2img cell and return the regenerated image.
+
+    The generator is created on CPU intentionally: a CPU generator is portable
+    across the mps/cuda/cpu backends (diffusers rejects a device-mismatched one),
+    matching the production runner's fallback behavior.
+    """
+    import torch
+
+    generator = torch.Generator(device="cpu").manual_seed(seed)
+    result = pipe(
+        prompt=PROMPT,
+        negative_prompt=NEGATIVE_PROMPT,
+        image=init,
+        control_image=control_image,
+        controlnet_conditioning_scale=float(scale),
+        strength=float(strength),
+        num_inference_steps=steps,
+        guidance_scale=guidance,
+        generator=generator,
+    )
+    return result.images[0]
+
+
+def parse_args() -> argparse.Namespace:
+    p = argparse.ArgumentParser(description="ControlNet-as-removal-pipeline prototype sweep.")
+    p.add_argument("image", type=Path, help="Watermarked input image.")
+    p.add_argument("-o", "--out", type=Path, default=Path("controlnet_sweep_out"), help="Output directory.")
+    p.add_argument("--control", nargs="+", choices=list(CONTROLNETS), default=["canny", "tile"])
+    p.add_argument("--strength", nargs="+", type=float, default=[0.3, 0.5, 0.7, 1.0])
+    p.add_argument("--scale", nargs="+", type=float, default=[0.6, 1.0], help="controlnet_conditioning_scale values.")
+    p.add_argument("--size", type=int, default=1024, help="Long-side resolution (multiple of 8).")
+    p.add_argument("--steps", type=int, default=30)
+    p.add_argument("--guidance", type=float, default=7.5)
+    p.add_argument("--seed", type=int, default=0)
+    p.add_argument("--device", default="auto", choices=["auto", "mps", "cuda", "cpu"])
+    p.add_argument("--dtype", default="auto", choices=["auto", "fp16", "fp32"])
+    return p.parse_args()
+
+
+def main() -> int:
+    args = parse_args()
+    if not args.image.exists():
+        log.error("Input image not found: %s", args.image)
+        return 1
+
+    try:
+        from PIL import Image as PILImage
+    except ImportError:
+        log.error("Pillow is required. Install the gpu extra: uv sync --extra gpu --extra dev")
+        return 1
+    if importlib.util.find_spec("diffusers") is None or importlib.util.find_spec("torch") is None:
+        log.error("diffusers/torch are required. Install: uv sync --extra gpu --extra dev")
+        return 1
+
+    device = pick_device(args.device)
+    dtype = resolve_dtype(device, args.dtype)
+    console.print(f"Device: {device} | dtype: {str(dtype).split('.')[-1]}")
+
+    init_full = PILImage.open(args.image).convert("RGB")
+    init = fit_size(init_full, args.size)
+    console.print(f"Input: {args.image.name} {init_full.size[0]}x{init_full.size[1]} -> {init.size[0]}x{init.size[1]}")
+
+    args.out.mkdir(parents=True, exist_ok=True)
+    stem = args.image.stem
+    init_path = args.out / f"{stem}__INPUT.png"
+    init.save(init_path)
+
+    rows: list[dict[str, Any]] = []
+    table = Table(title="ControlNet sweep")
+    for col in ("control", "strength", "scale", "psnr_vs_input", "file"):
+        table.add_column(col)
+
+    # Group by control so SDXL + the ControlNet load once per control type.
+    for control in args.control:
+        pipe = load_pipeline(control, device, dtype)
+        control_image = make_control_image(init, control)
+        if control == "canny":
+            control_image.save(args.out / f"{stem}__canny_edges.png")
+        for strength in args.strength:
+            for scale in args.scale:
+                tag = f"{control}_s{strength:g}_c{scale:g}"
+                console.print(f"Running {tag} ...")
+                try:
+                    out = run_cell(
+                        pipe,
+                        init,
+                        control_image,
+                        strength,
+                        scale,
+                        args.steps,
+                        args.guidance,
+                        args.seed,
+                    )
+                except Exception as exc:
+                    log.warning("Cell %s failed: %s", tag, exc)
+                    continue
+                fname = f"{stem}__{tag}.png"
+                out.save(args.out / fname)
+                quality = psnr(init, out)
+                rows.append(
+                    {
+                        "control": control,
+                        "strength": strength,
+                        "scale": scale,
+                        "psnr_vs_input": round(quality, 2),
+                        "file": fname,
+                        "synthid_oracle": "",  # fill: clean / present
+                        "text_legible": "",  # fill: yes / no / partial
+                    }
+                )
+                table.add_row(control, f"{strength:g}", f"{scale:g}", f"{quality:.2f}", fname)
+        del pipe
+        _free_memory(device)
+
+    _write_index(args.out, stem, rows, init_path.name)
+    console.print(table)
+    console.print(f"\nWrote {len(rows)} cells to {args.out}/")
+    console.print(f"Next: open {args.out}/sweep_index.csv, run each PNG through the Gemini SynthID oracle,")
+    console.print("fill synthid_oracle (clean/present) + text_legible (yes/no/partial), find the Pareto cell.")
+    return 0
+
+
+def _free_memory(device: str) -> None:
+    import gc
+
+    gc.collect()
+    with contextlib.suppress(Exception):
+        import torch
+
+        if device == "cuda":
+            torch.cuda.empty_cache()
+        elif device == "mps" and hasattr(torch, "mps"):
+            torch.mps.empty_cache()
+
+
+def _write_index(out: Path, stem: str, rows: list[dict[str, Any]], input_name: str) -> None:
+    """Write the CSV index (with empty verdict columns) and a README."""
+    fields = ["control", "strength", "scale", "psnr_vs_input", "file", "synthid_oracle", "text_legible"]
+    with (out / "sweep_index.csv").open("w", newline="", encoding="utf-8") as fh:
+        writer = csv.DictWriter(fh, fieldnames=fields)
+        writer.writeheader()
+        writer.writerows(rows)
+    readme = (
+        f"# ControlNet sweep for {stem}\n\n"
+        f"Input (resized): {input_name}\n\n"
+        "Each row in sweep_index.csv is one (control, strength, scale) cell. PSNR vs the\n"
+        "resized input is a COARSE global-fidelity proxy only -- it does NOT measure text\n"
+        "legibility or watermark presence. Decide those two by hand:\n\n"
+        "1. synthid_oracle: open the PNG in the Gemini app, 'Verify with SynthID'. Mark\n"
+        "   'clean' if no SynthID is detected, 'present' if it still is. (No local detector\n"
+        "   exists; this manual check is the only valid SynthID oracle.)\n"
+        "2. text_legible: eyeball the small/CJK text. Mark yes / partial / no.\n\n"
+        "The Pareto cell is the one where synthid_oracle=clean AND text_legible=yes at the\n"
+        "lowest strength. If no cell satisfies both, the canny/tile-ControlNet middle path\n"
+        "is dead for text and a glyph re-render is required (see\n"
+        "docs/text-protection-research.md). Record the outcome in\n"
+        "docs/controlnet-removal-pipeline-research.md.\n"
+    )
+    (out / "sweep_README.md").write_text(readme, encoding="utf-8")
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/corpus_gap_scan.py

@@ -30,8 +30,7 @@ from collections import Counter
 from pathlib import Path
 
 import click
-from rich.console import Console
-from rich.table import Table
+from _plain_console import Console, Table
 
 from remove_ai_watermarks.identify import identify
 from remove_ai_watermarks.metadata import _png_late_metadata

scripts/detector_response.py

@@ -0,0 +1,319 @@
+"""Tier B2: detector response curves -- recall vs mark size, contrast, background and aspect.
+
+WHY THIS EXISTS
+  Every recall number the project quotes rests on found positives: doubao 89% on n=240,
+  jimeng 71% on n=14, the pill 50% on n=6. Those tell you how the detectors do on the
+  marks a corpus happened to contain, and nothing about WHERE they fall over. Two bugs
+  have now shipped in exactly that blind spot:
+
+    * `scale_basis` -- doubao detected 0 of 435 LANDSCAPE marks, a 100% miss, because
+      every fraction was calibrated on portrait captures where width == short side.
+    * the front-end mismatch -- detection moved to the continuous `tophat` response while
+      the removal mask still came from the BINARIZED blob, so ~8% of doubao detections
+      produced an empty mask and removal was a silent no-op.
+
+  Both are geometry/plumbing failures at the edge of the operating range, and both were
+  invisible to a corpus sweep because the corpus is concentrated in the middle of it.
+  This harness constructs the edges instead of waiting for them to turn up.
+
+THE AGGREGATE HERE IS NOT RECALL
+  The grid deliberately visits sizes and opacities the engines were never calibrated for,
+  so a mark counted as missed at size=0.6 is not evidence of a weak detector -- it is the
+  point of the sweep. Quote the NOMINAL cell (size=1.0, alpha=1.0) as the reference, and
+  read the per-axis tables for shape. Production recall comes from an unbiased corpus
+  sample (`scripts/visible_recall_sample.py`), never from here.
+
+WHAT IT MEASURES -- TWO NUMBERS, NOT ONE
+  `detected`  the detector fires.
+  `maskable`  the same call path then yields a NON-EMPTY removal mask.
+
+  The second is the one that has no natural home anywhere else. A mark that is detected
+  but not maskable is reported by `identify` and silently skipped by `visible` -- the user
+  sees a mark the tool says it removed. Anything that measures only detection scores that
+  bug as a success. They are reported side by side and their GAP is the headline.
+
+THE CONSTRUCTION
+  Take a verified-clean corpus image, stamp a real mark onto it with the forward model the
+  reverse-alpha work established (`stamped = (1-a)*bg + a*white`), and sweep two knobs the
+  engine's own geometry normally pins:
+
+    size    multiplies the glyph box the engine would use (a re-rendered or rescaled mark)
+    alpha   multiplies the captured opacity (bold stamp -> faint translucent overlay)
+
+  Background texture and frame aspect are not swept -- they come from the corpus and are
+  recorded, so the grid is crossed with real backgrounds rather than synthetic flats.
+
+WHY EVERY SOURCE ALSO RUNS UNSTAMPED
+  A recall curve with no false-fire baseline is uninterpretable in the same way a fill
+  PSNR with no damage baseline is: "detected on 70% of faint marks" is a triumph or a
+  scandal depending on whether the detector also fires on 70% of the clean frames those
+  marks were stamped onto. The control runs the identical call path on the identical
+  image with no stamp.
+
+READING `mask_hit`
+  Fraction of the stamped glyph box the removal mask covers -- real localization accuracy
+  for the text marks and gemini, because the mask has to be placed from a detection.
+  For `jimeng_pill` it is CONSTRUCTED and means nothing: the pill's footprint is a fixed
+  top-left geometry box, so it covers the stamp by definition. Reported, and flagged.
+
+DATA SAFETY
+  Corpus images are user uploads: read-only, local analysis, gitignored output under
+  data/spaces/. Records source filenames and measurements, never image content.
+
+    uv run python scripts/detector_response.py --n 12          # trial, measures throughput
+    uv run python scripts/detector_response.py --n 150         # the real run, resumable
+"""
+
+from __future__ import annotations
+
+import argparse
+import collections
+import json
+import math
+import os
+import sys
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+sys.path.insert(0, str(Path(__file__).parent))
+
+from fill_quality import SLOT_STAMPABLE, STAMPABLE, clean_sources, stamp_any, texture_of
+
+REPO = Path(__file__).resolve().parents[1]
+OUT = REPO / "data" / "spaces" / "_detector_response.jsonl"
+
+# 1.0 = the geometry/opacity the engine's own constants assume. The sweep reaches below
+# it (a mark rendered smaller, or a faint translucent overlay -- the class the tophat
+# front-end was introduced for) and above it (a larger re-render, or an upscaled upload).
+SIZES = (0.6, 0.8, 1.0, 1.3)
+ALPHAS = (0.3, 0.5, 0.75, 1.0)
+MARKS = (*STAMPABLE, *SLOT_STAMPABLE)
+
+
+def aspect_of(image: np.ndarray) -> str:
+    h, w = image.shape[:2]
+    r = w / max(1, h)
+    return "portrait" if r < 0.9 else ("landscape" if r > 1.1 else "square")
+
+
+def _probe(image: np.ndarray, mark_key: str, box: tuple[int, int, int, int] | None) -> dict[str, Any]:
+    """Run the product detect -> localize path once and score it.
+
+    ``box`` is the stamped glyph box, or None for the unstamped control.
+    """
+    from remove_ai_watermarks.watermark_registry import get_mark
+
+    mark = get_mark(mark_key)
+    loc = mark.localize(image, force=False)
+    mask_px = 0 if loc.mask is None else int(np.count_nonzero(loc.mask))
+    hit = None
+    if box is not None and loc.mask is not None:
+        x, y, w, h = box
+        inside = loc.mask[y : y + h, x : x + w]
+        hit = round(float(np.count_nonzero(inside)) / max(1, w * h), 3)
+    return {
+        "detected": bool(loc.detected),
+        "confidence": round(float(loc.confidence), 4),
+        "maskable": mask_px > 0,
+        "mask_px": mask_px,
+        "mask_hit": hit,
+    }
+
+
+def _one_source(path_str: str) -> list[dict[str, Any]]:
+    """Every (mark, size, alpha) cell for one clean background, plus its controls."""
+    from remove_ai_watermarks.image_io import imread
+
+    base = imread(path_str)
+    if base is None:
+        return []
+    name = Path(path_str).name
+    asp = aspect_of(base)
+    h, w = base.shape[:2]
+    rows: list[dict[str, Any]] = []
+
+    for key in MARKS:
+        common = {"src": name, "mark": key, "aspect": asp, "px": w * h}
+        # Control first: the same call path on the same frame with nothing stamped.
+        rows.append({**common, "cell": "control", "size": None, "alpha": None, **_probe(base, key, None)})
+        for size in SIZES:
+            for alpha in ALPHAS:
+                st = stamp_any(base, key, size_mult=size, alpha_mult=alpha)
+                if st is None:
+                    continue
+                stamped, box = st
+                # Texture of the region the mark sits on, from the CLEAN frame -- the
+                # same median-Sobel proxy the pill's flatness gate uses.
+                x, y, gw, gh = box
+                rows.append(
+                    {
+                        **common,
+                        "cell": "stamped",
+                        "size": size,
+                        "alpha": alpha,
+                        "texture": round(texture_of(base[y : y + gh, x : x + gw]), 2),
+                        **_probe(stamped, key, box),
+                    }
+                )
+    return rows
+
+
+def wilson(k: int, n: int, z: float = 1.96) -> tuple[float, float]:
+    if n == 0:
+        return (0.0, 0.0)
+    p, d = k / n, 1 + z * z / n
+    c = (p + z * z / (2 * n)) / d
+    hw = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
+    return (100 * (c - hw), 100 * (c + hw))
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--n", type=int, default=150, help="verified-clean source images")
+    ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2))
+    ap.add_argument("--out", type=Path, default=OUT)
+    ap.add_argument("--restart", action="store_true")
+    ap.add_argument("--report-only", action="store_true", help="re-read --out and print the report")
+    a = ap.parse_args()
+
+    if a.report_only:
+        rows = [json.loads(line) for line in a.out.read_text(encoding="utf-8").splitlines() if line.strip()]
+        report(rows)
+        return
+
+    if a.restart and a.out.exists():
+        a.out.unlink()
+    done: set[str] = set()
+    if a.out.exists():
+        with open(a.out, encoding="utf-8") as fh:
+            for line in fh:
+                try:
+                    done.add(json.loads(line)["src"])
+                except Exception:  # noqa: S112 -- tolerate a torn last line
+                    continue
+
+    print(f"selecting {a.n} verified-clean sources (already measured: {len(done)})...", flush=True)
+    sources = [p for p in clean_sources(a.n) if p.name not in done]
+    print(f"to do {len(sources)}  workers {a.workers}  grid {len(MARKS)}x{len(SIZES)}x{len(ALPHAS)}\n", flush=True)
+
+    rows: list[dict[str, Any]] = []
+    a.out.parent.mkdir(parents=True, exist_ok=True)
+    with open(a.out, "a", encoding="utf-8") as fh, ProcessPoolExecutor(max_workers=a.workers) as ex:
+        futures = {ex.submit(_one_source, str(p)): p for p in sources}
+        for i, fut in enumerate(as_completed(futures), 1):
+            try:
+                got = fut.result()
+            except Exception as e:  # a crashed source must not kill the sweep
+                print(f"  source failed: {type(e).__name__}: {e}", flush=True)
+                continue
+            for r in got:
+                fh.write(json.dumps(r) + "\n")
+            rows += got
+            if i % 10 == 0:
+                fh.flush()
+                print(f"  {i}/{len(sources)}", flush=True)
+
+    if done:  # merge the resumed rows so the report covers the whole file
+        rows = [json.loads(line) for line in a.out.read_text(encoding="utf-8").splitlines() if line.strip()]
+    report(rows)
+
+
+def _rate(rs: list[dict[str, Any]], field: str) -> tuple[float, float, float, int]:
+    k, n = sum(bool(r[field]) for r in rs), len(rs)
+    lo, hi = wilson(k, n)
+    return (100 * k / n if n else 0.0, lo, hi, n)
+
+
+def report(rows: list[dict[str, Any]]) -> None:
+    stamped = [r for r in rows if r["cell"] == "stamped"]
+    control = [r for r in rows if r["cell"] == "control"]
+    if not stamped:
+        print("no measurements")
+        return
+    srcs = len({r["src"] for r in rows})
+    print(f"\n{'=' * 84}\nDETECTOR RESPONSE  {len(stamped)} stamped cells over {srcs} clean backgrounds\n{'=' * 84}")
+
+    # Lead with the nominal cell. The grid deliberately includes sizes and opacities the
+    # engines were never calibrated for, so an aggregate over the whole grid is an
+    # adversarial score, NOT production recall -- quoting it as recall would understate
+    # the detectors badly. The nominal cell is the honest reference point: the mark at the
+    # exact geometry and opacity the engine assumes, on a real clean background.
+    print("\nNOMINAL CELL (size=1.0, alpha=1.0) -- a mark exactly as the engine expects it.")
+    print("Read every other number against THIS, not as production recall: the grid is")
+    print("adversarial by design and its aggregate is not a recall figure.\n")
+    print(f"{'mark':13s} {'n':>5s} {'detected':>10s} {'maskable':>10s}")
+    for key in MARKS:
+        rs = [r for r in stamped if r["mark"] == key and r["size"] == 1.0 and r["alpha"] == 1.0]
+        if rs:
+            print(f"{key:13s} {len(rs):5d} {_rate(rs, 'detected')[0]:9.1f}% {_rate(rs, 'maskable')[0]:9.1f}%")
+
+    print("\n\nFALSE FIRE on the unstamped controls (the baseline every recall below is read against)\n")
+    print(f"{'mark':13s} {'n':>5s} {'fires':>8s} {'95% CI':>16s}")
+    for key in MARKS:
+        rs = [r for r in control if r["mark"] == key]
+        if rs:
+            pct, lo, hi, n = _rate(rs, "detected")
+            print(f"{key:13s} {n:5d} {pct:7.1f}% {f'{lo:.1f}-{hi:.1f}':>16s}")
+
+    print("\n\nDETECTED vs MASKABLE by mark. The GAP is the silent-no-op rate:")
+    print("a mark counted here as detected-but-not-maskable is reported by `identify` and")
+    print("skipped by `visible`, so the user is told it was removed when it was not.\n")
+    print(f"{'mark':13s} {'n':>6s} {'detected':>10s} {'maskable':>10s} {'gap':>7s} {'mask_hit':>9s}")
+    for key in MARKS:
+        rs = [r for r in stamped if r["mark"] == key]
+        if not rs:
+            continue
+        det = [r for r in rs if r["detected"]]
+        d, m = _rate(rs, "detected")[0], _rate(rs, "maskable")[0]
+        gap = (100 * sum(1 for r in det if not r["maskable"]) / len(det)) if det else 0.0
+        hits = [r["mask_hit"] for r in det if r["mask_hit"] is not None]
+        note = "  (constructed)" if key == "jimeng_pill" else ""
+        hv = f"{float(np.median(hits)):.2f}" if hits else "-"
+        print(f"{key:13s} {len(rs):6d} {d:9.1f}% {m:9.1f}% {gap:6.1f}% {hv:>9s}{note}")
+
+    for axis, label in (("alpha", "CONTRAST (alpha multiplier)"), ("size", "SIZE (glyph box multiplier)")):
+        print(f"\n\nRECALL BY {label}  -- 1.0 is the geometry/opacity the engine assumes\n")
+        vals = sorted({r[axis] for r in stamped})
+        print(f"{'mark':13s}" + "".join(f"{v:>12}" for v in vals))
+        for key in MARKS:
+            cells = []
+            for v in vals:
+                rs = [r for r in stamped if r["mark"] == key and r[axis] == v]
+                cells.append(f"{_rate(rs, 'detected')[0]:10.0f}%" if rs else f"{'-':>11s}")
+            print(f"{key:13s}" + "".join(f"{c:>12}" for c in cells))
+
+    print("\n\nRECALL BY FRAME ASPECT  -- the axis the `scale_basis` bug lived on\n")
+    asps = ["portrait", "square", "landscape"]
+    print(f"{'mark':13s}" + "".join(f"{x:>14s}" for x in asps))
+    for key in MARKS:
+        cells = []
+        for asp in asps:
+            rs = [r for r in stamped if r["mark"] == key and r["aspect"] == asp]
+            cells.append(f"{_rate(rs, 'detected')[0]:.0f}% (n={len(rs)})" if rs else "-")
+        print(f"{key:13s}" + "".join(f"{c:>14s}" for c in cells))
+
+    tex = sorted(r["texture"] for r in stamped if "texture" in r)
+    if tex:
+        t1, t2 = tex[len(tex) // 3], tex[2 * len(tex) // 3]
+        print(f"\n\nRECALL BY BACKGROUND TEXTURE  (terciles at {t1:.1f} / {t2:.1f} median-Sobel)\n")
+        print(f"{'mark':13s}" + "".join(f"{x:>14s}" for x in ("flat", "mid", "textured")))
+        for key in MARKS:
+            cells = []
+            for lo, hi in ((-1, t1), (t1, t2), (t2, 1e9)):
+                rs = [r for r in stamped if r["mark"] == key and lo < r.get("texture", -2) <= hi]
+                cells.append(f"{_rate(rs, 'detected')[0]:.0f}% (n={len(rs)})" if rs else "-")
+            print(f"{key:13s}" + "".join(f"{c:>14s}" for c in cells))
+
+    worst = collections.Counter((r["mark"], r["size"], r["alpha"]) for r in stamped if not r["detected"]).most_common(8)
+    if worst:
+        print("\n\nWORST CELLS (most misses)\n")
+        for (mark, size, alpha), n in worst:
+            print(f"  {mark:13s} size={size}  alpha={alpha}   {n} misses")
+
+
+if __name__ == "__main__":
+    main()

scripts/fidelity_metrics.py

@@ -0,0 +1,418 @@
+# /// script
+# requires-python = ">=3.10"
+# dependencies = [
+#   "click",
+#   "numpy",
+#   "opencv-python-headless",
+#   "pillow",
+#   "scikit-image",
+#   "rapidfuzz",
+#   "torch",
+#   "lpips",
+#   "paddleocr",
+#   "paddlepaddle",
+#   "insightface",
+#   "onnxruntime",
+# ]
+# ///
+"""Objective fidelity metrics for comparing watermark-removal outputs.
+
+Given an ORIGINAL (the reference) and one or more cleaned VARIANTS that have all
+ALREADY passed the scrub oracle, this scores how much real detail each variant
+preserved -- so "closer to the original" is the right axis here (between two
+equally-scrubbed outputs, the one that deviates less from the original wins).
+
+It is a standalone eval tool, NOT part of the package: PEP 723 inline deps let
+``uv run`` build a throwaway env so the heavy models (PaddleOCR, insightface,
+LPIPS) never touch uv.lock or the shipped library. Metrics self-gate: face
+metrics run only where faces are detected, text metrics only where text is.
+
+Two subcommands:
+
+  ocr      -- OCR images (PaddleOCR PP-OCRv6) into a JSON {basename: text} file.
+              Run this on the ORIGINALS, hand-verify/correct the file, and it
+              becomes the ground truth for ``compare --ground-truth`` -- the clean
+              way to score text, since OCR-vs-OCR is doubly noisy (errors on both
+              images + reading-order differences inflate CER even on identical text).
+
+  compare  -- Score each VARIANT against the ORIGINAL across four groups:
+              1. Text  -- character error rate (CER) of the variant's OCR vs the
+                 verified ground truth (or the original's OCR if no --ground-truth).
+              2. Face identity -- insightface (buffalo_l) ArcFace cosine similarity.
+              3. Face texture  -- LPIPS + Laplacian-variance ratio on face crops
+                 (catches "plastication": ratio < 1 = smoother than the original).
+              4. Whole image   -- LPIPS / SSIM / PSNR vs the original.
+
+Usage:
+    uv run scripts/fidelity_metrics.py ocr O1.png O2.png --langs en,ru,ch --out gt.json
+    # (edit gt.json by hand to fix any OCR slips, then:)
+    uv run scripts/fidelity_metrics.py compare --original O1.png \
+        --variant controlnet=C.png --variant qwen=Q.png \
+        --ocr-langs en,ru,ch --ground-truth gt.json
+"""
+
+from __future__ import annotations
+
+import json
+import unicodedata
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any
+
+import click
+import cv2
+import numpy as np
+from _plain_console import Console, Table
+
+console = Console()
+
+
+# ── helpers ──────────────────────────────────────────────────────────
+
+
+def _load_bgr(path: str) -> np.ndarray:
+    img = cv2.imread(path, cv2.IMREAD_COLOR)
+    if img is None:
+        raise click.ClickException(f"cannot read image: {path}")
+    return img
+
+
+def _match_size(variant: np.ndarray, ref: np.ndarray) -> np.ndarray:
+    """Resize a variant to the reference size (outputs differ by a grid-round)."""
+    if variant.shape[:2] != ref.shape[:2]:
+        variant = cv2.resize(variant, (ref.shape[1], ref.shape[0]), interpolation=cv2.INTER_LANCZOS4)
+    return variant
+
+
+def _norm(text: str) -> str:
+    """Normalize for CER: NFC + drop ALL whitespace (segmentation-order agnostic)."""
+    return "".join(unicodedata.normalize("NFC", text).split())
+
+
+# ── text: PaddleOCR (PP-OCRv6) ───────────────────────────────────────
+
+# Our lang codes -> PaddleOCR lang. The 'ch' model also reads Latin; 'ru' reads
+# Cyrillic + Latin. Multiple langs in one image -> run each model, union detections.
+_PADDLE_LANG = {"en": "en", "ru": "ru", "ch": "ch", "ch_sim": "ch", "latin": "latin"}
+_paddle_cache: dict[str, Any] = {}
+
+
+def _paddle(lang: str) -> Any:
+    if lang not in _paddle_cache:
+        from paddleocr import PaddleOCR
+
+        _paddle_cache[lang] = PaddleOCR(
+            lang=lang,
+            use_doc_orientation_classify=False,
+            use_doc_unwarping=False,
+            use_textline_orientation=False,
+        )
+    return _paddle_cache[lang]
+
+
+def _box_xyxy(box: Any) -> tuple[float, float, float, float]:
+    """Axis-aligned (x1, y1, x2, y2) of a PaddleOCR rec box ([x1,y1,x2,y2]) or poly (4x2)."""
+    arr = np.asarray(box, dtype=np.float32).reshape(-1)
+    if arr.size == 4:
+        return float(arr[0]), float(arr[1]), float(arr[2]), float(arr[3])
+    pts = arr.reshape(-1, 2)
+    return float(pts[:, 0].min()), float(pts[:, 1].min()), float(pts[:, 0].max()), float(pts[:, 1].max())
+
+
+def _iou(a: tuple[float, float, float, float], b: tuple[float, float, float, float]) -> float:
+    ix1, iy1 = max(a[0], b[0]), max(a[1], b[1])
+    ix2, iy2 = min(a[2], b[2]), min(a[3], b[3])
+    iw, ih = max(0.0, ix2 - ix1), max(0.0, iy2 - iy1)
+    inter = iw * ih
+    if inter <= 0:
+        return 0.0
+    area_a = (a[2] - a[0]) * (a[3] - a[1])
+    area_b = (b[2] - b[0]) * (b[3] - b[1])
+    return inter / (area_a + area_b - inter + 1e-9)
+
+
+def _ocr_lines(bgr: np.ndarray, langs: list[str], min_score: float = 0.5) -> list[str]:
+    """Detected text lines in reading order, unioned across lang models with spatial NMS.
+
+    Several language models over one image re-detect the same lines -- and crucially the
+    WRONG-script models read e.g. Cyrillic as confident Latin gibberish. So instead of a
+    naive union, keep the HIGHEST-score detection per physical location (greedy IoU NMS):
+    the model that actually fits a line wins it (the 'ru' model takes the Cyrillic, 'ch'
+    the CJK, 'en' the Latin), and the cross-script garbage is dropped.
+    """
+    raw: list[tuple[float, tuple[float, float, float, float], str]] = []
+    for lang in langs:
+        plang = _PADDLE_LANG.get(lang, lang)
+        for page in _paddle(plang).predict(bgr):
+            texts = page.get("rec_texts", [])
+            scores = page.get("rec_scores", [])
+            boxes = page.get("rec_boxes", None)
+            if boxes is None or len(boxes) == 0:
+                boxes = page.get("rec_polys", [])
+            for text, score, box in zip(texts, scores, boxes, strict=False):
+                if score < min_score or not text.strip():
+                    continue
+                raw.append((float(score), _box_xyxy(box), text.strip()))
+
+    raw.sort(key=lambda d: d[0], reverse=True)
+    kept: list[tuple[tuple[float, float, float, float], str]] = []
+    for _score, box, text in raw:
+        if any(_iou(box, kbox) > 0.3 for kbox, _ in kept):
+            continue
+        kept.append((box, text))
+    kept.sort(key=lambda d: (round(d[0][1] / 20.0), d[0][0]))  # reading order: y then x
+    return [t for _, t in kept]
+
+
+def _cer(ref: str, hyp: str) -> float:
+    from rapidfuzz.distance import Levenshtein
+
+    return Levenshtein.normalized_distance(_norm(ref), _norm(hyp))
+
+
+# ── face: detection + ArcFace + texture ──────────────────────────────
+
+
+@dataclass
+class FaceStats:
+    n_faces: int = 0
+    identity: list[float] = field(default_factory=list)
+    lpips: list[float] = field(default_factory=list)
+    lapvar_ratio: list[float] = field(default_factory=list)
+
+
+def _lap_var(bgr: np.ndarray) -> float:
+    gray = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY)
+    return float(cv2.Laplacian(gray, cv2.CV_64F).var())
+
+
+def _bbox_center(bbox: Any) -> tuple[float, float]:
+    return (bbox[0] + bbox[2]) / 2, (bbox[1] + bbox[3]) / 2
+
+
+def _bbox_diag(bbox: Any) -> float:
+    return float(((bbox[2] - bbox[0]) ** 2 + (bbox[3] - bbox[1]) ** 2) ** 0.5)
+
+
+def assign_faces_one_to_one(
+    ref_centers: list[tuple[float, float]],
+    var_centers: list[tuple[float, float]],
+    ref_diags: list[float],
+    max_frac: float = 0.6,
+) -> dict[int, int]:
+    """One-to-one nearest-center face assignment (pure; unit-tested without insightface).
+
+    Per-face nearest matching collides on multi-face images -- two original faces can both
+    pick the SAME variant face (e.g. when regeneration drops a face, so the variant has fewer
+    detections), corrupting the identity metric (the lapvar/LPIPS metrics are immune: they are
+    anchored to the ORIGINAL bbox on both images). This greedy-by-distance assignment is
+    collision-free: it walks candidate pairs nearest-first and never reuses a ref or a variant
+    face. Faces are spatially well-separated, so greedy equals the optimal (Hungarian) result
+    here without the scipy dependency. A pair is dropped when the center distance exceeds
+    ``max_frac`` of the original face diagonal (no plausible match -- the face was lost).
+
+    Returns a dict mapping ref-face index -> variant-face index for matched faces only.
+    """
+    pairs: list[tuple[float, int, int]] = []
+    for i, (rx, ry) in enumerate(ref_centers):
+        for j, (vx, vy) in enumerate(var_centers):
+            pairs.append((((rx - vx) ** 2 + (ry - vy) ** 2) ** 0.5, i, j))
+    pairs.sort()
+    used_ref: set[int] = set()
+    used_var: set[int] = set()
+    matched: dict[int, int] = {}
+    for dist, i, j in pairs:
+        if i in used_ref or j in used_var:
+            continue
+        if dist > max_frac * ref_diags[i]:
+            continue
+        matched[i] = j
+        used_ref.add(i)
+        used_var.add(j)
+    return matched
+
+
+def _cosine(a: np.ndarray, b: np.ndarray) -> float:
+    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-9))
+
+
+def _crop(bgr: np.ndarray, bbox: Any) -> np.ndarray:
+    h, w = bgr.shape[:2]
+    x1, y1, x2, y2 = (int(max(0, bbox[0])), int(max(0, bbox[1])), int(min(w, bbox[2])), int(min(h, bbox[3])))
+    return bgr[y1:y2, x1:x2]
+
+
+# ── whole image: LPIPS / SSIM / PSNR ─────────────────────────────────
+
+
+def _lpips_model() -> tuple[Any, Any]:
+    import lpips
+    import torch
+
+    model = lpips.LPIPS(net="alex", verbose=False)
+    model.eval()
+    return model, torch
+
+
+def _lpips_distance(model_torch: tuple[Any, Any], a_bgr: np.ndarray, b_bgr: np.ndarray) -> float:
+    model, torch = model_torch
+
+    def _t(img: np.ndarray) -> Any:
+        rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB).astype(np.float32) / 127.5 - 1.0
+        return torch.from_numpy(rgb).permute(2, 0, 1).unsqueeze(0)
+
+    with torch.no_grad():
+        return float(model(_t(a_bgr), _t(b_bgr)).item())
+
+
+def _ssim_psnr(a_bgr: np.ndarray, b_bgr: np.ndarray) -> tuple[float, float]:
+    from skimage.metrics import peak_signal_noise_ratio, structural_similarity
+
+    a = cv2.cvtColor(a_bgr, cv2.COLOR_BGR2GRAY)
+    b = cv2.cvtColor(b_bgr, cv2.COLOR_BGR2GRAY)
+    return float(structural_similarity(a, b)), float(peak_signal_noise_ratio(a, b))
+
+
+# ── reporting ────────────────────────────────────────────────────────
+
+
+def _mean(xs: list[float]) -> float | None:
+    return sum(xs) / len(xs) if xs else None
+
+
+def _fmt(v: float | None, nd: int = 3) -> str:
+    return "-" if v is None else f"{v:.{nd}f}"
+
+
+# ── CLI ──────────────────────────────────────────────────────────────
+
+
+@click.group()
+def cli() -> None:
+    """Objective fidelity metrics for watermark-removal outputs."""
+
+
+@cli.command("ocr")
+@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True))
+@click.option("--langs", default="en", help="Comma list of OCR langs (en,ru,ch).")
+@click.option("--out", type=click.Path(), default=None, help="Write {basename: text} JSON here (for ground truth).")
+def ocr_cmd(images: tuple[str, ...], langs: str, out: str | None) -> None:
+    """OCR images into a ground-truth seed -- hand-verify the result before using it."""
+    lang_list = [x.strip() for x in langs.split(",") if x.strip()]
+    result: dict[str, str] = {}
+    for path in images:
+        lines = _ocr_lines(_load_bgr(path), lang_list)
+        text = "\n".join(lines)
+        result[Path(path).name] = text
+        console.print(f"\n=== {Path(path).name} ===")
+        console.print(text or "(no text detected)")
+    if out:
+        Path(out).write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
+        console.print(f"\n  Wrote {out} -- verify/correct it by hand, then pass it to `compare --ground-truth`.")
+
+
+@cli.command("compare")
+@click.option("--original", required=True, type=click.Path(exists=True), help="Reference (unprocessed) image.")
+@click.option(
+    "--variant", "variants", multiple=True, required=True, help="LABEL=PATH of a cleaned output (repeatable)."
+)
+@click.option("--ocr-langs", default="en", help="Comma list of OCR langs (en,ru,ch). Empty = skip text.")
+@click.option("--ground-truth", type=click.Path(exists=True), default=None, help="Verified {basename: text} JSON.")
+@click.option("--no-faces", is_flag=True, help="Skip face metrics.")
+def compare(original: str, variants: tuple[str, ...], ocr_langs: str, ground_truth: str | None, no_faces: bool) -> None:
+    """Score each VARIANT against ORIGINAL across the four fidelity groups."""
+    ref = _load_bgr(original)
+    parsed: list[tuple[str, np.ndarray]] = []
+    for spec in variants:
+        if "=" not in spec:
+            raise click.ClickException(f"--variant must be LABEL=PATH, got {spec!r}")
+        label, path = spec.split("=", 1)
+        parsed.append((label, _match_size(_load_bgr(path), ref)))
+
+    langs = [x.strip() for x in ocr_langs.split(",") if x.strip()]
+    lp = _lpips_model()  # AlexNet LPIPS, loaded once and reused for face crops + whole image
+
+    # ── text ──
+    ocr_cer: dict[str, float | None] = {label: None for label, _ in parsed}
+    if langs:
+        ref_text: str | None = None
+        if ground_truth:
+            gt = json.loads(Path(ground_truth).read_text(encoding="utf-8"))
+            ref_text = gt.get(Path(original).name)
+            if ref_text is None:
+                console.print(f"  (no ground-truth entry for {Path(original).name}; skipping text)")
+        else:
+            console.print(f"  OCR original ({','.join(langs)})...")
+            ref_text = "\n".join(_ocr_lines(ref, langs))
+        if ref_text:
+            console.print(f"  OCR variants ({','.join(langs)})...")
+            for label, img in parsed:
+                ocr_cer[label] = _cer(ref_text, "\n".join(_ocr_lines(img, langs)))
+
+    # ── faces ──
+    face_stats: dict[str, FaceStats] = {label: FaceStats() for label, _ in parsed}
+    if not no_faces:
+        console.print("  Faces (insightface buffalo_l)...")
+        from insightface.app import FaceAnalysis
+
+        app = FaceAnalysis(name="buffalo_l", providers=["CPUExecutionProvider"])
+        app.prepare(ctx_id=-1, det_size=(640, 640))
+        ref_faces = app.get(ref)
+        if ref_faces:
+            ref_centers = [_bbox_center(of.bbox) for of in ref_faces]
+            ref_diags = [_bbox_diag(of.bbox) for of in ref_faces]
+            for label, img in parsed:
+                vfaces = app.get(img)
+                st = face_stats[label]
+                # One-to-one assignment for identity (collision-free); lapvar/LPIPS stay
+                # anchored to the original bbox below, so they need no match.
+                matched = assign_faces_one_to_one(ref_centers, [_bbox_center(vf.bbox) for vf in vfaces], ref_diags)
+                for oi, of in enumerate(ref_faces):
+                    st.n_faces += 1
+                    vf = vfaces[matched[oi]] if oi in matched else None
+                    if vf is not None:
+                        st.identity.append(_cosine(of.normed_embedding, vf.normed_embedding))
+                    oc, vc = _crop(ref, of.bbox), _crop(img, of.bbox)
+                    if oc.size == 0 or vc.size == 0:
+                        continue
+                    vc_r = cv2.resize(vc, (oc.shape[1], oc.shape[0]), interpolation=cv2.INTER_LANCZOS4)
+                    st.lpips.append(_lpips_distance(lp, oc, vc_r))
+                    ov = _lap_var(oc)
+                    st.lapvar_ratio.append(_lap_var(vc_r) / ov if ov > 1e-6 else 0.0)
+        else:
+            console.print("  (no faces detected in the original; skipping face metrics)")
+
+    # ── whole image ──
+    console.print("  Whole-image LPIPS/SSIM/PSNR...")
+    whole: dict[str, tuple[float, float, float]] = {}
+    for label, img in parsed:
+        ssim, psnr = _ssim_psnr(ref, img)
+        whole[label] = (_lpips_distance(lp, ref, img), ssim, psnr)
+
+    # ── report ──
+    table = Table(title=f"Fidelity vs {Path(original).name} (reference)")
+    for col in ("variant", "text CER↓", "faces", "ID cos↑", "face LPIPS↓", "lapvar↑", "img LPIPS↓", "SSIM↑", "PSNR↑"):
+        table.add_column(col)
+    for label, _ in parsed:
+        st = face_stats[label]
+        wl, ws, wp = whole[label]
+        table.add_row(
+            label,
+            _fmt(ocr_cer[label]),
+            str(st.n_faces),
+            _fmt(_mean(st.identity)),
+            _fmt(_mean(st.lpips)),
+            _fmt(_mean(st.lapvar_ratio)),
+            _fmt(wl),
+            _fmt(ws),
+            _fmt(wp, 1),
+        )
+    console.print(table)
+    console.print(
+        "  Legend: CER lower=better; ID cos higher=better; face LPIPS lower=better; "
+        "lapvar ratio ~1=detail kept, <1=smoothed/plastic; img LPIPS lower=better; SSIM/PSNR higher=closer."
+    )
+
+
+if __name__ == "__main__":
+    cli()

scripts/fill_quality.py

@@ -0,0 +1,345 @@
+"""Tier B1: measure visible-mark FILL quality against a constructed ground truth.
+
+THE PROBLEM THIS SOLVES
+  A real watermarked image has no clean counterpart, so fill quality has only ever been
+  eyeballed ("cv2 smears on texture, LaMa is best") and never carried a number. The docs
+  state a preference order for `--backend auto` that rests on no measurement.
+
+THE CONSTRUCTION
+  Take a VERIFIED-CLEAN corpus image, stamp a known mark onto it using the mark's own
+  captured alpha map (the forward model the reverse-alpha work established:
+  `stamped = (1-a)*bg + a*white`), then remove it and compare against the TRUE original.
+  The original is the answer key that reality never provides.
+
+TWO MEASUREMENT DECISIONS THAT MAKE OR BREAK THIS
+  * Score INSIDE the footprint only. The fill touches a tiny corner, so whole-frame PSNR
+    sits near 60 dB whatever the backend does and would rank them all "excellent".
+  * Report the DAMAGE baseline (stamped vs original) next to the recovery (filled vs
+    original). Without it a PSNR is uninterpretable: 30 dB is a triumph if the mark cost
+    12 dB and a failure if it cost 29 dB. The honest metric is how much of the gap the
+    fill closes.
+
+WHAT IT DOES NOT MEASURE
+  Detection. The mark is localized with `force=True`, so a miss cannot contaminate the
+  fill numbers -- this isolates the FILL. Detection accuracy is Tier C's job.
+
+DATA SAFETY
+  Corpus images are user uploads: read-only, local analysis, output under a gitignored
+  data/spaces/ path. No image content is written into the report.
+
+    uv run python scripts/fill_quality.py --n 60
+"""
+
+from __future__ import annotations
+
+import argparse
+import collections
+import glob
+import json
+import random
+import sys
+from pathlib import Path
+from typing import Any
+
+import cv2
+import numpy as np
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+sys.path.insert(0, str(Path(__file__).parent))
+
+from invisible_quality_audit import _ssim  # reuse, do not reimplement a third SSIM
+
+REPO = Path(__file__).resolve().parents[1]
+CORPUS = REPO / "data" / "spaces" / "originals"
+OUT = REPO / "data" / "spaces" / "_fill_quality.jsonl"
+
+# Text marks: a bundled alpha PNG plus the engine's own corner geometry.
+STAMPABLE = ("doubao", "jimeng", "samsung")
+# Marks with no bundled alpha asset. Both expose a default footprint via
+# `footprint_mask(force=True)`, so they are stamped by fitting their own alpha source
+# into that slot: Gemini's alpha is derived from its background captures, the pill's
+# from its synthetic font-rendered silhouette.
+SLOT_STAMPABLE = ("gemini", "jimeng_pill")
+
+
+def psnr(a: np.ndarray, b: np.ndarray) -> float:
+    mse = float(np.mean((a.astype(np.float64) - b.astype(np.float64)) ** 2))
+    return float("inf") if mse == 0 else 10 * float(np.log10(255.0**2 / mse))
+
+
+def texture_of(box: np.ndarray) -> float:
+    """Median Sobel magnitude -- the same texture proxy the pill's flatness gate uses."""
+    g = cv2.cvtColor(box, cv2.COLOR_BGR2GRAY) if box.ndim == 3 else box
+    gx = cv2.Sobel(g, cv2.CV_32F, 1, 0, ksize=3)
+    gy = cv2.Sobel(g, cv2.CV_32F, 0, 1, ksize=3)
+    return float(np.median(cv2.magnitude(gx, gy)))
+
+
+def engine_for(mark_key: str) -> Any:
+    """The TextMarkEngine instance for a mark key (the registry does not expose one)."""
+    import importlib
+
+    mod = importlib.import_module(f"remove_ai_watermarks.{mark_key}_engine")
+    cls = next(
+        obj
+        for name, obj in vars(mod).items()
+        # Exclude the imported base class: it takes a `config` arg, the subclasses do not.
+        if isinstance(obj, type) and name.endswith("Engine") and name != "TextMarkEngine"
+    )
+    return cls()
+
+
+def composite(image: np.ndarray, alpha: np.ndarray, x: int, y: int) -> np.ndarray:
+    """The single forward model every stamp uses: ``stamped = (1-a)*bg + a*white``.
+
+    ``alpha`` is float and clipped to [0,1] here, so a caller may pre-multiply it by an
+    opacity factor without worrying about overflow. Shared by ``stamp``/``stamp_slot``
+    here and by ``scripts/detector_response.py`` (which imports it) so the model is
+    written ONCE.
+    """
+    gh, gw = alpha.shape[:2]
+    out = image.copy()
+    roi = out[y : y + gh, x : x + gw].astype(np.float32)
+    a3 = np.clip(alpha, 0.0, 1.0)[..., None]
+    out[y : y + gh, x : x + gw] = np.clip(roi * (1 - a3) + 255.0 * a3, 0, 255).astype(np.uint8)
+    return out
+
+
+def stamp(
+    image: np.ndarray, mark_key: str, *, size_mult: float = 1.0, alpha_mult: float = 1.0
+) -> tuple[np.ndarray, tuple[int, int, int, int]] | None:
+    """Composite a mark's alpha glyph into its canonical corner. Returns (stamped, bbox).
+
+    ``size_mult`` scales the glyph box off the engine's nominal geometry and ``alpha_mult``
+    its opacity -- both default to 1.0 (the geometry the engine assumes), and
+    `scripts/detector_response.py` sweeps them to build response curves.
+    """
+    from remove_ai_watermarks._text_mark_engine import load_alpha_template
+
+    engine = engine_for(mark_key)
+    cfg = engine.config
+    alpha = load_alpha_template(cfg.asset_name)
+    if alpha is None:
+        return None
+
+    loc = engine.locate(image)
+    base = engine.scale_base(image)
+    gw = max(cfg.min_gw, int(cfg.alpha_width_frac * base * size_mult))
+    gh = max(4, int(cfg.alpha_height_frac * base * size_mult))
+    if gw < 8 or gh < 4 or gw > loc.w or gh > loc.h:
+        return None
+
+    a = cv2.resize(alpha, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32) * alpha_mult
+    x = loc.x + (loc.w - gw) // 2
+    y = loc.y + (loc.h - gh) // 2
+    return composite(image, a, x, y), (x, y, gw, gh)
+
+
+def _slot_alpha(mark_key: str) -> np.ndarray | None:
+    """The alpha source for a mark that ships no bundled alpha PNG."""
+    if mark_key == "gemini":
+        from remove_ai_watermarks.gemini_engine import _shared_engine
+
+        # _shared_engine is the package's own lru_cache singleton: constructing
+        # GeminiEngine() per image reloads its captures, recomputes both alpha maps and
+        # rebuilds the whole template ladder -- the exact cost that singleton exists to
+        # avoid. Private attribute access is deliberate (no accessor exists); a
+        # measurement script may reach in, product code must not.
+        return np.asarray(_shared_engine()._alpha_large, dtype=np.float32)
+    if mark_key == "jimeng_pill":
+        from remove_ai_watermarks._text_mark_engine import load_alpha_template
+
+        return load_alpha_template("jimeng_pill.png")
+    return None
+
+
+def stamp_slot(
+    image: np.ndarray, mark_key: str, *, size_mult: float = 1.0, alpha_mult: float = 1.0
+) -> tuple[np.ndarray, tuple[int, int, int, int]] | None:
+    """Stamp a mark into its OWN default footprint slot (the `--no-detect` geometry).
+
+    Used for gemini and the pill, which have no bundled alpha asset with corner
+    fractions. The mark is fitted into the middle of its slot so the fill has to
+    recover the same kind of region it would in production. ``size_mult``/``alpha_mult``
+    match ``stamp`` (both default to 1.0).
+    """
+    from remove_ai_watermarks.watermark_registry import get_mark
+
+    alpha = _slot_alpha(mark_key)
+    if alpha is None:
+        return None
+    loc = get_mark(mark_key).localize(image, force=True)
+    if loc.mask is None:
+        return None
+    ys, xs = np.nonzero(loc.mask)
+    if ys.size == 0:
+        return None
+    y0, y1, x0, x1 = int(ys.min()), int(ys.max()), int(xs.min()), int(xs.max())
+    bw, bh = x1 - x0 + 1, y1 - y0 + 1
+    gw, gh = max(8, int(bw * 0.7 * size_mult)), max(8, int(bh * 0.7 * size_mult))
+    a = cv2.resize(alpha, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32) * alpha_mult
+    x, y = x0 + (bw - gw) // 2, y0 + (bh - gh) // 2
+    # A large size_mult can grow the glyph past its slot and push x/y negative; guard so
+    # composite never writes out of bounds (numpy would wrap a negative index silently).
+    if x < 0 or y < 0 or x + gw > image.shape[1] or y + gh > image.shape[0]:
+        return None
+    return composite(image, a, x, y), (x, y, gw, gh)
+
+
+def stamp_any(
+    image: np.ndarray, mark_key: str, *, size_mult: float = 1.0, alpha_mult: float = 1.0
+) -> tuple[np.ndarray, tuple[int, int, int, int]] | None:
+    fn = stamp if mark_key in STAMPABLE else stamp_slot
+    return fn(image, mark_key, size_mult=size_mult, alpha_mult=alpha_mult)
+
+
+def clean_sources(n: int, seed: int = 11) -> list[Path]:
+    """Corpus images with NO metadata signal and NO mark detection -- verified clean."""
+    from remove_ai_watermarks.identify import identify
+    from remove_ai_watermarks.image_io import imread
+    from remove_ai_watermarks.watermark_registry import detect_marks
+
+    pool = glob.glob(str(CORPUS / "*" / "*"))
+    random.Random(seed).shuffle(pool)  # noqa: S311 -- deterministic sampling, not crypto
+    out: list[Path] = []
+    for p in pool:
+        if len(out) >= n:
+            break
+        path = Path(p)
+        try:
+            img = imread(str(path))
+            if img is None or min(img.shape[:2]) < 400:
+                continue
+            # check_visible=False: identify would otherwise decode the file a second
+            # time and run the very same visible detectors as detect_marks below.
+            if identify(path, check_visible=False).signals:
+                continue
+            if any(d.detected for d in detect_marks(img)):
+                continue
+        except Exception:  # noqa: S112 -- a bad corpus file just is not a candidate
+            continue
+        out.append(path)
+    return out
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--n", type=int, default=60, help="clean source images")
+    ap.add_argument("--out", type=Path, default=OUT)
+    a = ap.parse_args()
+
+    from remove_ai_watermarks.image_io import imread
+    from remove_ai_watermarks.region_eraser import lama_available, migan_available
+    from remove_ai_watermarks.watermark_registry import fill, get_mark
+
+    backends = ["cv2"] + (["migan"] if migan_available() else []) + (["lama"] if lama_available() else [])
+    print(f"backends under test: {backends}")
+    print(f"selecting {a.n} verified-clean sources...", flush=True)
+    sources = clean_sources(a.n)
+    print(f"got {len(sources)}\n", flush=True)
+
+    rows: list[dict[str, Any]] = []
+    with open(a.out, "w", encoding="utf-8") as fh:
+        for i, src in enumerate(sources, 1):
+            base = imread(str(src))
+            if base is None:
+                continue
+            for key in (*STAMPABLE, *SLOT_STAMPABLE):
+                st = stamp_any(base, key)
+                if st is None:
+                    continue
+                stamped, (x, y, w, h) = st
+                # Score a slightly padded box: the fill dilates, so a tight glyph box
+                # would miss damage the backend does just outside the glyph.
+                pad = 8
+                y0, y1 = max(0, y - pad), min(base.shape[0], y + h + pad)
+                x0, x1 = max(0, x - pad), min(base.shape[1], x + w + pad)
+                truth = base[y0:y1, x0:x1]
+                dmg_psnr = psnr(stamped[y0:y1, x0:x1], truth)
+                dmg_ssim = _ssim(
+                    cv2.cvtColor(stamped[y0:y1, x0:x1], cv2.COLOR_BGR2GRAY), cv2.cvtColor(truth, cv2.COLOR_BGR2GRAY)
+                )
+                tex = texture_of(truth)
+
+                mark = get_mark(key)
+                loc = mark.localize(stamped, force=True)
+                if loc.mask is None:
+                    continue
+                for backend in backends:
+                    try:
+                        filled = fill(stamped, loc.mask, backend=backend)
+                    except Exception as e:
+                        rows.append({"src": src.name, "mark": key, "backend": backend, "error": str(e)[:150]})
+                        continue
+                    # Invariant: the fill must not touch anything outside its mask.
+                    outside = loc.mask == 0
+                    untouched = bool(np.array_equal(filled[outside], stamped[outside]))
+                    rec = {
+                        "src": src.name,
+                        "mark": key,
+                        "backend": backend,
+                        "texture": round(tex, 2),
+                        "damage_psnr": round(dmg_psnr, 2),
+                        "damage_ssim": round(dmg_ssim, 4),
+                        "filled_psnr": round(psnr(filled[y0:y1, x0:x1], truth), 2),
+                        "filled_ssim": round(
+                            _ssim(
+                                cv2.cvtColor(filled[y0:y1, x0:x1], cv2.COLOR_BGR2GRAY),
+                                cv2.cvtColor(truth, cv2.COLOR_BGR2GRAY),
+                            ),
+                            4,
+                        ),
+                        "outside_mask_untouched": untouched,
+                    }
+                    rows.append(rec)
+                    fh.write(json.dumps(rec) + "\n")
+            if i % 10 == 0:
+                fh.flush()
+                print(f"  {i}/{len(sources)}", flush=True)
+
+    report(rows, a.out)
+
+
+def report(rows: list[dict[str, Any]], out: Path) -> None:
+    good = [r for r in rows if "error" not in r]
+    if not good:
+        print("no measurements")
+        return
+    tex = sorted(r["texture"] for r in good)
+    t1, t2 = tex[len(tex) // 3], tex[2 * len(tex) // 3]
+
+    def bucket(t: float) -> str:
+        return "flat" if t <= t1 else ("mid" if t <= t2 else "textured")
+
+    print(f"\n{'=' * 78}\nFILL QUALITY  n={len(good)} measurements  (texture terciles at {t1:.1f} / {t2:.1f})")
+    print(f"{'=' * 78}")
+    print("recovered = filled PSNR minus damaged PSNR; how much of the mark's damage the fill undoes\n")
+    hdr = f"{'mark':9s} {'backend':8s} {'bg':9s} {'n':>4s} {'damaged':>9s} {'filled':>9s}"
+    print(f"{hdr} {'recovered':>10s} {'ssim':>7s}")
+    agg: dict[tuple[str, str, str], list[dict[str, Any]]] = collections.defaultdict(list)
+    for r in good:
+        agg[(r["mark"], r["backend"], bucket(r["texture"]))].append(r)
+    for (mark, backend, bg), rs in sorted(agg.items()):
+        # MEDIAN, not mean: a fill that reproduces a flat background EXACTLY scores
+        # PSNR=inf, and a single inf makes the mean inf -- which reported "+inf" for
+        # every flat bucket on the first run and hid the real numbers.
+        finite = [r for r in rs if np.isfinite(r["filled_psnr"])]
+        perfect = len(rs) - len(finite)
+        d = float(np.median([r["damage_psnr"] for r in rs]))
+        f = float(np.median([r["filled_psnr"] for r in finite])) if finite else float("inf")
+        sv = float(np.median([r["filled_ssim"] for r in rs]))
+        # Recovery is the MEDIAN OF THE PER-IMAGE DIFFERENCES, not the difference of the
+        # two medians. Those are not the same statistic on skewed data and they can even
+        # disagree in SIGN: the first version of this report subtracted medians and
+        # produced a backend ranking that a paired comparison did not support.
+        rec = float(np.median([r["filled_psnr"] - r["damage_psnr"] for r in finite])) if finite else float("inf")
+        tail = f"  ({perfect} exact)" if perfect else ""
+        print(f"{mark:9s} {backend:8s} {bg:9s} {len(rs):4d} {d:9.2f} {f:9.2f} {rec:+10.2f} {sv:7.4f}{tail}")
+
+    bad = [r for r in good if not r["outside_mask_untouched"]]
+    print(f"\noutside-mask invariant violations: {len(bad)}" + (" <-- BUG" if bad else " (none)"))
+    print(f"records: {out}")
+
+
+if __name__ == "__main__":
+    main()

scripts/invisible_quality_audit.py

@@ -0,0 +1,117 @@
+"""Audit invisible-removal output quality by pairing originals with cleaned outputs.
+
+The spaces routine writes ``<hash>_src.<ext>`` originals and ``<hash>_clean.<ext>``
+cleaned outputs. For each pair this computes a structural-similarity score plus
+cheap content proxies (detail via Laplacian variance, resolution, aspect), so the
+WORST-preserved images can be surfaced and then visually classified.
+
+SSIM alone does NOT equal "bad": a high-texture image legitimately changes under
+the SDXL scrub. Use the ranked output to pick candidates, then look at them to
+name the failure classes (garbled text, deformed faces, over-smoothed detail).
+
+Operates on gitignored data only (data/spaces/...); writes nothing tracked.
+
+    uv run python scripts/invisible_quality_audit.py \
+        --originals data/spaces/originals/2026-06-03 \
+        --cleaned   data/spaces/results/2026-06-03 \
+        --out data/spaces/_quality_audit.csv --worst 25
+"""
+
+from __future__ import annotations
+
+import csv
+import logging
+from pathlib import Path
+
+import click
+import cv2
+import numpy as np
+
+from remove_ai_watermarks import image_io
+
+log = logging.getLogger(__name__)
+
+
+def _ssim(a: np.ndarray, b: np.ndarray) -> float:
+    """Grayscale SSIM (single-window Gaussian, the Wang et al. formulation)."""
+    a = a.astype(np.float64)
+    b = b.astype(np.float64)
+    c1, c2 = (0.01 * 255) ** 2, (0.03 * 255) ** 2
+    k = (11, 11)
+    mu_a = cv2.GaussianBlur(a, k, 1.5)
+    mu_b = cv2.GaussianBlur(b, k, 1.5)
+    mu_a2, mu_b2, mu_ab = mu_a * mu_a, mu_b * mu_b, mu_a * mu_b
+    sa = cv2.GaussianBlur(a * a, k, 1.5) - mu_a2
+    sb = cv2.GaussianBlur(b * b, k, 1.5) - mu_b2
+    sab = cv2.GaussianBlur(a * b, k, 1.5) - mu_ab
+    ssim_map = ((2 * mu_ab + c1) * (2 * sab + c2)) / ((mu_a2 + mu_b2 + c1) * (sa + sb + c2))
+    return float(ssim_map.mean())
+
+
+def _stem(name: str) -> str:
+    """Strip the _src/_clean suffix and extension to get the pairing key."""
+    base = name.rsplit(".", 1)[0]
+    for suf in ("_src", "_clean"):
+        if base.endswith(suf):
+            return base[: -len(suf)]
+    return base
+
+
+@click.command()
+@click.option("--originals", type=click.Path(exists=True, file_okay=False, path_type=Path), required=True)
+@click.option("--cleaned", type=click.Path(exists=True, file_okay=False, path_type=Path), required=True)
+@click.option("--out", type=click.Path(path_type=Path), default=Path("data/spaces/_quality_audit.csv"))
+@click.option("--worst", type=int, default=25, help="Print the N lowest-SSIM pairs.")
+def main(originals: Path, cleaned: Path, out: Path, worst: int) -> None:
+    logging.basicConfig(level=logging.WARNING, format="%(message)s")
+    src_by_key = {_stem(p.name): p for p in originals.iterdir() if p.is_file()}
+    clean_by_key = {_stem(p.name): p for p in cleaned.iterdir() if p.is_file()}
+    keys = sorted(src_by_key.keys() & clean_by_key.keys())
+    click.echo(f"Pairs: {len(keys)} ({len(src_by_key)} src, {len(clean_by_key)} clean)")
+
+    rows: list[dict[str, str]] = []
+    with click.progressbar(keys, label="ssim") as bar:
+        for key in bar:
+            a = image_io.imread(src_by_key[key], cv2.IMREAD_COLOR)
+            b = image_io.imread(clean_by_key[key], cv2.IMREAD_COLOR)
+            if a is None or b is None:
+                continue
+            if a.shape[:2] != b.shape[:2]:
+                b = cv2.resize(b, (a.shape[1], a.shape[0]), interpolation=cv2.INTER_LANCZOS4)
+            ga = cv2.cvtColor(a, cv2.COLOR_BGR2GRAY)
+            gb = cv2.cvtColor(b, cv2.COLOR_BGR2GRAY)
+            h, w = ga.shape
+            rows.append(
+                {
+                    "key": key,
+                    "ssim": f"{_ssim(ga, gb):.4f}",
+                    "laplacian_var": f"{cv2.Laplacian(ga, cv2.CV_64F).var():.1f}",
+                    "width": str(w),
+                    "height": str(h),
+                    "megapixels": f"{w * h / 1e6:.2f}",
+                    "aspect": f"{w / h:.2f}",
+                }
+            )
+
+    rows.sort(key=lambda r: float(r["ssim"]))
+    out.parent.mkdir(parents=True, exist_ok=True)
+    with out.open("w", newline="") as f:
+        wtr = csv.DictWriter(f, fieldnames=["key", "ssim", "laplacian_var", "width", "height", "megapixels", "aspect"])
+        wtr.writeheader()
+        wtr.writerows(rows)
+
+    ssims = [float(r["ssim"]) for r in rows]
+    if ssims:
+        arr = np.array(ssims)
+        click.echo(f"\nSSIM mean={arr.mean():.3f} p10={np.percentile(arr, 10):.3f} min={arr.min():.3f}")
+        click.echo(f"Pairs with SSIM < 0.70: {(arr < 0.70).sum()}  | < 0.60: {(arr < 0.60).sum()}")
+        click.echo(f"\nWorst {worst} (lowest SSIM):")
+        for r in rows[:worst]:
+            click.echo(
+                f"  ssim={r['ssim']}  lap={r['laplacian_var']:>8}  {r['megapixels']}MP {r['aspect']}  {r['key']}"
+            )
+    click.echo(f"\nReport: {out}")
+
+
+if __name__ == "__main__":
+    main()

scripts/ladder_headroom.py

@@ -0,0 +1,305 @@
+"""How much recall is the coarse scale ladder costing, and what would a denser one cost?
+
+THE FINDING THIS MEASURES
+  `_tophat_score` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end
+  sweeps none at all. Measured on stamped marks over controlled backgrounds
+  (`scripts/detector_response.py`), the response is a COMB: doubao scores 0.99 exactly at
+  each rung and collapses to 0.37-0.48 between them, against a 0.50 gate. So a mark whose
+  rendered size lands mid-gap is missed at FULL contrast. Jimeng (binary, one nominal
+  size) holds a single 0.90-1.20 lobe; samsung only 0.95-1.05.
+
+WHY A SYNTHETIC SWEEP IS NOT ENOUGH TO JUSTIFY A FIX
+  Dead zones only cost recall if real marks land in them. The fractions were calibrated on
+  real captures, so it is entirely possible that real marks cluster at ratio 1.0 and the
+  gaps are never visited. That question is not answerable by stamping -- it needs the
+  corpus.
+
+THE MEASUREMENT
+  Positives are images carrying INDEPENDENT vendor provenance (TC260 / C2PA metadata
+  naming the vendor), which is evidence that does not come from the pixel detector we are
+  grading -- the same discriminator that settled the pill gate. For every such image the
+  detector currently MISSES, rescore at a dense ladder and ask whether it would now cross
+  the gate, and at which scale.
+
+  Negatives are images with NO metadata signal at all. That is deliberately NOT "and no
+  mark was detected": defining the set by the detector's own verdict would make its
+  false-fire rate 0 by construction and the comparison vacuous. The cost is that a
+  metadata-stripped screenshot of a marked image sits in the negative set and its correct
+  detection is counted as a false fire -- but that impurity is identical under both
+  ladders, so the DELTA between them, which is what the fix is judged on, stays sound.
+
+  Two more impurities to keep in view when reading the positive arm. Provenance says the
+  vendor produced the file, not that a visible mark is on it. And the TC260 label names no
+  specific vendor, so `visible_provenance` maps it to BOTH doubao and jimeng -- a "doubao
+  positive" may carry a jimeng mark or none. Both inflate the miss count, so the recovery
+  percentage is an upper bound on what a denser ladder buys.
+
+DATA SAFETY
+  Corpus images are user uploads: read-only, local analysis, gitignored output.
+
+    uv run python scripts/ladder_headroom.py --mark doubao --n 4000
+"""
+
+from __future__ import annotations
+
+import argparse
+import collections
+import glob
+import json
+import math
+import os
+import random
+import sys
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from pathlib import Path
+from typing import Any
+
+import cv2
+import numpy as np
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+REPO = Path(__file__).resolve().parents[1]
+CORPUS = REPO / "data" / "spaces" / "originals"
+OUT = REPO / "data" / "spaces" / "_ladder_headroom.jsonl"
+
+# The rungs the product ships today, and the dense ladder under evaluation. The dense one
+# is geometric with a ~6% step, chosen from the measured half-width of a rung's lobe
+# (doubao holds >=0.5 for about +/-6% around each peak), so no gap is left uncovered.
+SHIPPED = (0.8, 1.0, 1.25)
+LADDERS: dict[str, tuple[float, ...]] = {
+    # A geometric ~6% step across the whole plausible range. Measures the CEILING of what
+    # any ladder change can buy, and the worst case of what it costs in false fires.
+    "dense": tuple(round(0.70 * (1.06**i), 4) for i in range(13)),  # 0.70 .. ~1.49
+    # The targeted alternative. On the full run, 26 of 28 recoveries came from ONE rung
+    # (~1.116) and 22 of 28 were LANDSCAPE frames -- that is a geometry gap, not a density
+    # gap, so the honest comparison is one extra rung against thirteen.
+    "plus_one": (0.8, 1.0, 1.1157, 1.25),
+    "shipped": SHIPPED,
+}
+DENSE = LADDERS["dense"]
+# Score every rung any candidate ladder might use and store them all, so a new candidate
+# is evaluated by re-reading the JSONL instead of re-running the hour-long sweep.
+PROBE_SCALES: tuple[float, ...] = tuple(sorted({s for rungs in LADDERS.values() for s in rungs}))
+
+
+def wilson(k: int, n: int, z: float = 1.96) -> tuple[float, float]:
+    if n == 0:
+        return (0.0, 0.0)
+    p, d = k / n, 1 + z * z / n
+    c = (p + z * z / (2 * n)) / d
+    hw = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
+    return (100 * (c - hw), 100 * (c + hw))
+
+
+def score_at_scales(engine: Any, image: np.ndarray, scales: tuple[float, ...]) -> dict[float, float]:
+    """`_tophat_score`'s inner loop, opened up so the ladder is a parameter.
+
+    Deliberately reaches into the engine (`tophat_response`, `_glyph_silhouette`): a
+    measurement script may, product code may not. Kept a faithful copy of the shipped
+    scoring -- if it drifts, the numbers below stop describing the product.
+    """
+    c = engine.config
+    loc = engine.locate(image)
+    resp = engine.tophat_response(image, loc)
+    sil = engine._glyph_silhouette()
+    if resp is None or sil is None:
+        return {}
+    base = engine.scale_base(image)
+    out: dict[float, float] = {}
+    for scale in scales:
+        gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
+        gh = max(4, int(c.alpha_height_frac * base * scale))
+        if gw >= resp.shape[1] or gh >= resp.shape[0]:
+            continue
+        tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32)
+        if c.template_blur > 0:
+            tmpl = cv2.GaussianBlur(tmpl, (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur)
+        out[scale] = float(cv2.matchTemplate(resp, tmpl.astype(np.uint8), cv2.TM_CCOEFF_NORMED).max())
+    return out
+
+
+def _one(args: tuple[str, str]) -> dict[str, Any] | None:
+    path_str, mark_key = args
+    from remove_ai_watermarks.api import visible_provenance
+    from remove_ai_watermarks.identify import identify
+    from remove_ai_watermarks.image_io import imread
+    from remove_ai_watermarks.watermark_registry import get_mark
+
+    path = Path(path_str)
+    try:
+        prov = visible_provenance(path)
+    except Exception:
+        return None
+    positive = mark_key in prov
+
+    if not positive:
+        # A negative is a frame with NO metadata signal at all -- evidence independent of
+        # the detector being graded. Do NOT also require that no mark was detected: that
+        # defines the negative set by the very verdict under test, and "false fire today"
+        # would then be 0 by construction on every run.
+        try:
+            if identify(path, check_visible=False).signals:
+                return None
+        except Exception:
+            return None
+
+    img = imread(path_str)
+    if img is None or min(img.shape[:2]) < 200:
+        return None
+
+    mark = get_mark(mark_key)
+    det = mark.detect(img)
+
+    import importlib
+
+    mod = importlib.import_module(f"remove_ai_watermarks.{mark_key}_engine")
+    cls = next(
+        o for n, o in vars(mod).items() if isinstance(o, type) and n.endswith("Engine") and n != "TextMarkEngine"
+    )
+    engine = cls()
+    scores = score_at_scales(engine, img, PROBE_SCALES)
+    if not scores:
+        return None
+    gate = engine.config.detect_ncc_threshold
+    dense = {s: v for s, v in scores.items() if s in LADDERS["dense"]}
+    best_scale = max(dense, key=lambda s: dense[s]) if dense else 0.0
+    h, w = img.shape[:2]
+    return {
+        "src": path.name,
+        "arm": "positive" if positive else "negative",
+        "detected_now": bool(det.detected),
+        "conf_now": round(float(det.confidence), 4),
+        "gate": gate,
+        # Every rung, so a new candidate ladder is evaluated by re-reading this file.
+        "scores": {str(s): round(v, 4) for s, v in scores.items()},
+        "dense_best": round(dense[best_scale], 4) if dense else 0.0,
+        "dense_best_scale": best_scale,
+        "dense_crosses": bool(dense) and dense[best_scale] >= gate,
+        "aspect": "portrait" if w / h < 0.9 else ("landscape" if w / h > 1.1 else "square"),
+    }
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    # doubao only, and that is not laziness. `score_at_scales` reproduces the TOPHAT
+    # front-end, which is the only one with a ladder to widen; scoring it for jimeng or
+    # samsung would measure a code path their detectors never run. Doubao is also the one
+    # mark that declares NO rival, so crossing its NCC gate really is the whole verdict --
+    # for a mark with rivals, `dense_crosses` would ignore the competitive margin and read
+    # optimistically. Widening the binary front-end is a separate experiment.
+    ap.add_argument("--mark", default="doubao", choices=["doubao"])
+    ap.add_argument("--n", type=int, default=4000, help="corpus files to scan")
+    ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2))
+    ap.add_argument("--out", type=Path, default=OUT)
+    ap.add_argument("--report-only", action="store_true")
+    a = ap.parse_args()
+
+    out = a.out.with_name(f"{a.out.stem}_{a.mark}{a.out.suffix}")
+    if a.report_only:
+        report([json.loads(x) for x in out.read_text(encoding="utf-8").splitlines() if x.strip()], a.mark)
+        return
+
+    pool = glob.glob(str(CORPUS / "*" / "*"))
+    random.Random(19).shuffle(pool)  # noqa: S311 -- deterministic sampling, not crypto
+    pool = pool[: a.n]
+    print(f"mark={a.mark}  scanning {len(pool)} corpus files  workers={a.workers}")
+    print(f"shipped ladder {SHIPPED}\ndense ladder   {DENSE}\n", flush=True)
+
+    rows: list[dict[str, Any]] = []
+    out.parent.mkdir(parents=True, exist_ok=True)
+    with open(out, "w", encoding="utf-8") as fh, ProcessPoolExecutor(max_workers=a.workers) as ex:
+        futures = [ex.submit(_one, (p, a.mark)) for p in pool]
+        for i, fut in enumerate(as_completed(futures), 1):
+            try:
+                rec = fut.result()
+            except Exception:  # noqa: S112 -- one bad file must not kill the sweep
+                continue
+            if rec is None:
+                continue
+            fh.write(json.dumps(rec) + "\n")
+            rows.append(rec)
+            if i % 500 == 0:
+                fh.flush()
+                pos = sum(1 for r in rows if r["arm"] == "positive")
+                print(f"  {i}/{len(pool)}  usable={len(rows)} (pos={pos})", flush=True)
+    report(rows, a.mark)
+
+
+def report(rows: list[dict[str, Any]], mark: str) -> None:
+    pos = [r for r in rows if r["arm"] == "positive"]
+    neg = [r for r in rows if r["arm"] == "negative"]
+    print(f"\n{'=' * 82}\nLADDER HEADROOM  mark={mark}   positives={len(pos)}  verified-clean negatives={len(neg)}")
+    print(f"{'=' * 82}")
+    if not pos and not neg:
+        print("no usable rows")
+        return
+
+    if pos:
+        miss = [r for r in pos if not r["detected_now"]]
+        rec = [r for r in miss if r["dense_crosses"]]
+        now = len(pos) - len(miss)
+        lo, hi = wilson(len(rec), len(miss)) if miss else (0.0, 0.0)
+        print("\nPOSITIVE ARM  (images whose METADATA names this vendor -- an upper bound,")
+        print("provenance says the vendor produced the file, not that a mark is visible on it)\n")
+        print(f"  detected today            {now:5d} / {len(pos)}  ({100 * now / len(pos):.1f}%)")
+        print(f"  of the {len(miss)} misses, the dense ladder crosses the gate on {len(rec)}")
+        if miss:
+            print(f"                            ({100 * len(rec) / len(miss):.1f}%, 95% CI {lo:.1f}-{hi:.1f})")
+            after = now + len(rec)
+            print(f"  detection would go        {100 * now / len(pos):.1f}%  ->  {100 * after / len(pos):.1f}%")
+        if rec:
+            by_scale = collections.Counter(r["dense_best_scale"] for r in rec)
+            print("\n  which rung recovers them (a rung near a SHIPPED one recovering many means")
+            print("  the gain is the finer STEP, not the wider RANGE):")
+            for s, n in sorted(by_scale.items()):
+                near = "  <- shipped" if any(abs(s - x) < 0.02 for x in SHIPPED) else ""
+                print(f"    scale {s:<6} {n:4d}{near}")
+            by_asp = collections.Counter(r["aspect"] for r in rec)
+            print(f"\n  by aspect: {dict(by_asp)}")
+
+    if neg:
+        fires_now = sum(1 for r in neg if r["detected_now"])
+        fires_dense = sum(1 for r in neg if r["dense_crosses"])
+        lo0, hi0 = wilson(fires_now, len(neg))
+        lo1, hi1 = wilson(fires_dense, len(neg))
+        print("\n\nNEGATIVE ARM  (no metadata signal; NOT filtered on the detector's own verdict)\n")
+        p0, p1 = 100 * fires_now / len(neg), 100 * fires_dense / len(neg)
+        print(f"  false fire today          {fires_now:4d} / {len(neg)}  {p0:.2f}%  (CI {lo0:.2f}-{hi0:.2f})")
+        print(f"  false fire, dense ladder  {fires_dense:4d} / {len(neg)}  {p1:.2f}%  (CI {lo1:.2f}-{hi1:.2f})")
+        print("\n  This is the price. A denser ladder gives a spurious blob more chances to")
+        print("  match at SOME scale, so the gain above is only real if this line barely moves.")
+
+    ladder_table(pos, neg)
+
+
+def _crosses(row: dict[str, Any], rungs: tuple[float, ...]) -> bool:
+    """Would this frame cross its gate on the given ladder, from the stored per-rung scores."""
+    sc = row.get("scores") or {}
+    gate = row["gate"]
+    return any(v >= gate for k, v in sc.items() if float(k) in rungs)
+
+
+def ladder_table(pos: list[dict[str, Any]], neg: list[dict[str, Any]]) -> None:
+    """Every candidate ladder side by side: what it recovers against what it costs."""
+    if not (pos and neg) or "scores" not in pos[0]:
+        return  # older run without per-rung scores
+    miss = [r for r in pos if not r["detected_now"]]
+    base_fire = sum(1 for r in neg if r["detected_now"])
+    print("\n\nLADDER CANDIDATES -- recovered marks against the false fires they cost\n")
+    print("A candidate is only worth shipping if the recovered column beats the added-fires")
+    print("column by enough to survive the base rates: clean frames vastly outnumber marked")
+    print("ones in real traffic, so a small percentage on the negative arm is a large count.\n")
+    hdr = f"{'ladder':10s} {'rungs':>6s} {'recovered':>10s} {'of misses':>10s}"
+    print(f"{hdr} {'false fire':>11s} {'added':>7s} {'ratio':>7s}")
+    for name, rungs in LADDERS.items():
+        rec = sum(1 for r in miss if _crosses(r, rungs))
+        fire = sum(1 for r in neg if _crosses(r, rungs) or r["detected_now"])
+        added = fire - base_fire
+        ratio = f"{rec / added:.1f}:1" if added > 0 else ("inf" if rec else "-")
+        pct = 100 * rec / len(miss) if miss else 0.0
+        print(f"{name:10s} {len(rungs):6d} {rec:10d} {pct:9.1f}% {100 * fire / len(neg):10.2f}% {added:7d} {ratio:>7s}")
+
+
+if __name__ == "__main__":
+    main()

scripts/metadata_removal_audit.py

@@ -0,0 +1,256 @@
+"""Audit AI-metadata REMOVAL over a local image corpus (detection<->removal parity).
+
+`corpus_gap_scan.py` proves the DETECTOR sees a marker; this proves the STRIPPER
+reaches it. For every file that carries an AI-metadata signal, run
+``remove_ai_metadata`` and re-scan the output with the SAME oracle
+(``get_ai_metadata``). Any signal that survives is a real parity bug: a re-served
+file still reads as AI. Also assert the strip is lossless -- decoded pixels
+(and alpha) bit-identical before/after -- since the removal must only touch
+metadata, never the coded image.
+
+A no-op control set (clean images with no AI metadata) verifies the stripper
+neither ADDS a signal nor corrupts pixels on files it should leave alone.
+
+Operates on gitignored data only (data/spaces/...); writes nothing tracked.
+
+    uv run python scripts/metadata_removal_audit.py \
+        --corpus data/spaces/originals --identify data/spaces/identify \
+        --out data/spaces/_metadata_removal_audit.csv --jobs 8
+"""
+
+from __future__ import annotations
+
+import csv
+import json
+import logging
+import random
+import tempfile
+from collections import Counter
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from pathlib import Path
+
+import click
+import numpy as np
+
+from remove_ai_watermarks.noai.constants import SUPPORTED_FORMATS
+
+log = logging.getLogger(__name__)
+
+# identify-JSON watermark substrings that imply a METADATA-borne signal (as
+# opposed to a purely visual sparkle/text mark). Used only to pick the candidate
+# population fast; get_ai_metadata is the per-file ground truth.
+_META_HINTS = (
+    "C2PA",
+    "Content Credentials",
+    "IPTC",
+    "Made with AI",
+    "AIGC",
+    "TC260",
+    "EXIF",
+    "Signature",
+    "SynthID",
+    "hf-job",
+    "HuggingFace",
+    "Samsung",
+    "soft-binding",
+    "metadata",
+)
+
+
+def _pixels(path: Path) -> tuple[np.ndarray | None, np.ndarray | None]:
+    """Decoded BGR + alpha, for the lossless-strip integrity check."""
+    from remove_ai_watermarks import image_io
+
+    return image_io.read_bgr_and_alpha(path)
+
+
+def _same_pixels(a: Path, b: Path) -> bool | None:
+    """True/False if both decode; None if either is undecodable (can't compare)."""
+    try:
+        bgr_a, al_a = _pixels(a)
+        bgr_b, al_b = _pixels(b)
+    except Exception:
+        return None
+    if bgr_a is None or bgr_b is None:
+        return None
+    if bgr_a.shape != bgr_b.shape or not np.array_equal(bgr_a, bgr_b):
+        return False
+    if (al_a is None) != (al_b is None):
+        return False
+    return al_a is None or al_b is None or np.array_equal(al_a, al_b)
+
+
+def _audit_one(path_str: str) -> dict[str, object]:
+    """Worker: detect -> strip -> re-detect + pixel-integrity for one file."""
+    from remove_ai_watermarks.metadata import get_ai_metadata, remove_ai_metadata
+
+    path = Path(path_str)
+    row: dict[str, object] = {
+        "path": path.name,
+        "ext": path.suffix.lower(),
+        "carrier": False,
+        "before": "",
+        "after": "",
+        "parity_ok": "",
+        "pixels_identical": "",
+        "status": "ok",
+    }
+    try:
+        before = get_ai_metadata(path)
+    except Exception as exc:
+        row["status"] = f"scan_error:{type(exc).__name__}"
+        return row
+    row["before"] = "|".join(sorted(before))
+    row["carrier"] = bool(before)
+
+    with tempfile.TemporaryDirectory() as td:
+        out = Path(td) / f"clean{path.suffix.lower()}"
+        try:
+            remove_ai_metadata(path, out)
+        except Exception as exc:
+            row["status"] = f"strip_error:{type(exc).__name__}"
+            return row
+        if not out.exists():
+            row["status"] = "no_output"
+            return row
+        try:
+            after = get_ai_metadata(out)
+        except Exception as exc:
+            row["status"] = f"rescan_error:{type(exc).__name__}"
+            return row
+        row["after"] = "|".join(sorted(after))
+        row["parity_ok"] = not after  # every AI signal must be gone
+        same = _same_pixels(path, out)
+        row["pixels_identical"] = "" if same is None else same
+    return row
+
+
+def _candidate_paths(corpus: Path, identify: Path | None, clean_sample: int) -> tuple[list[Path], list[Path]]:
+    """Return (carriers, clean_controls). Uses identify JSONs when present to pick
+    metadata carriers fast; falls back to scanning every file."""
+    if identify is None or not identify.exists():
+        files = sorted(p for p in corpus.rglob("*") if p.is_file() and p.suffix.lower() in SUPPORTED_FORMATS)
+        return files, []
+
+    carriers: list[Path] = []
+    clean: list[Path] = []
+    for jf in identify.rglob("*.json"):
+        try:
+            d = json.loads(jf.read_text())
+        except Exception:  # noqa: S112 -- skip an unreadable identify JSON, not security-relevant
+            continue
+        src = d.get("src")
+        if not src:
+            continue
+        img = corpus / jf.parent.name / src
+        if not img.exists() or img.suffix.lower() not in SUPPORTED_FORMATS:
+            continue
+        wm = " | ".join(d.get("watermarks") or [])
+        if any(h in wm for h in _META_HINTS):
+            carriers.append(img)
+        elif not d.get("is_ai_generated"):
+            clean.append(img)
+    rng = random.Random(0)  # noqa: S311 -- deterministic sampling seed, not cryptographic
+    rng.shuffle(clean)
+    return carriers, clean[:clean_sample]
+
+
+@click.command()
+@click.option(
+    "--corpus", type=click.Path(exists=True, file_okay=False, path_type=Path), default=Path("data/spaces/originals")
+)
+@click.option(
+    "--identify",
+    type=click.Path(path_type=Path),
+    default=Path("data/spaces/identify"),
+    help="identify-JSON dir to pick carriers (skip = scan all).",
+)
+@click.option("--out", type=click.Path(path_type=Path), default=Path("data/spaces/_metadata_removal_audit.csv"))
+@click.option(
+    "--clean-sample", type=int, default=1500, help="No-op control: N clean images to prove the strip is a no-op."
+)
+@click.option("--limit", type=int, default=0, help="Cap carriers scanned (0 = all).")
+@click.option("--jobs", type=int, default=8)
+def main(corpus: Path, identify: Path | None, out: Path, clean_sample: int, limit: int, jobs: int) -> None:
+    logging.basicConfig(level=logging.ERROR, format="%(message)s")
+    carriers, clean = _candidate_paths(corpus, identify, clean_sample)
+    if limit:
+        carriers = carriers[:limit]
+    tasks = [(p, "carrier") for p in carriers] + [(p, "clean") for p in clean]
+    click.echo(f"Carriers: {len(carriers)} | clean controls: {len(clean)} | jobs {jobs}")
+
+    rows: list[dict[str, object]] = []
+    with ProcessPoolExecutor(max_workers=jobs) as ex:
+        futs = {ex.submit(_audit_one, str(p)): k for p, k in tasks}
+        for done, fut in enumerate(as_completed(futs), 1):
+            row = fut.result()
+            row["kind"] = futs[fut]
+            rows.append(row)
+            if done % 500 == 0:
+                click.echo(f"  {done}/{len(tasks)}")
+
+    out.parent.mkdir(parents=True, exist_ok=True)
+    fields = ["kind", "path", "ext", "carrier", "before", "after", "parity_ok", "pixels_identical", "status"]
+    with out.open("w", newline="") as f:
+        w = csv.DictWriter(f, fieldnames=fields)
+        w.writeheader()
+        for r in rows:
+            w.writerow({k: r.get(k, "") for k in fields})
+
+    # ---- Summary ----
+    car = [r for r in rows if r["kind"] == "carrier"]
+    ctl = [r for r in rows if r["kind"] == "clean"]
+    real_carriers = [r for r in car if r["carrier"] and r["status"] == "ok"]
+    parity_fail = [r for r in real_carriers if r["parity_ok"] is False]
+    pixel_fail = [r for r in real_carriers if r["pixels_identical"] is False]
+    errors = [r for r in rows if r["status"] != "ok"]
+
+    click.echo("\n===== METADATA REMOVAL PARITY =====")
+    click.echo(f"Carrier candidates:      {len(car)}")
+    click.echo(f"Confirmed carriers:      {len(real_carriers)} (get_ai_metadata non-empty)")
+    click.echo(f"Parity FAILS (signal survives strip): {len(parity_fail)}")
+    click.echo(f"Pixel-integrity FAILS (strip altered pixels): {len(pixel_fail)}")
+    click.echo(f"Errors (scan/strip/decode): {len(errors)}")
+
+    # Per-signal parity breakdown.
+    sig_total: Counter[str] = Counter()
+    sig_fail: Counter[str] = Counter()
+    for r in real_carriers:
+        for s in str(r["before"]).split("|"):
+            if s:
+                sig_total[s] += 1
+        if r["parity_ok"] is False:
+            for s in str(r["after"]).split("|"):
+                if s:
+                    sig_fail[s] += 1
+    click.echo("\nPer-signal (carriers / surviving-after-strip):")
+    for s, n in sig_total.most_common():
+        click.echo(f"  {s:24} {n:6}  survived: {sig_fail.get(s, 0)}")
+
+    click.echo("\n===== NO-OP CONTROL (clean images) =====")
+    ctl_ok = [r for r in ctl if r["status"] == "ok"]
+    added = [r for r in ctl_ok if r["after"]]  # strip must not ADD a signal
+    corrupted = [r for r in ctl_ok if r["pixels_identical"] is False]
+    click.echo(f"Clean controls scanned:  {len(ctl_ok)}")
+    click.echo(f"Strip ADDED a signal:    {len(added)}")
+    click.echo(f"Strip corrupted pixels:  {len(corrupted)}")
+
+    if parity_fail:
+        click.echo("\n--- Parity failures (first 30) ---")
+        for r in parity_fail[:30]:
+            click.echo(f"  {r['ext']:6} survived=[{r['after']}]  {r['path']}")
+    if pixel_fail:
+        click.echo("\n--- Pixel-integrity failures (first 30) ---")
+        for r in pixel_fail[:30]:
+            click.echo(f"  {r['ext']:6} {r['path']}")
+    if errors:
+        ec: Counter[str] = Counter(str(r["status"]) for r in errors)
+        click.echo("\n--- Errors by kind ---")
+        for k, n in ec.most_common():
+            click.echo(f"  {n:5}  {k}")
+
+    click.echo(f"\nReport: {out}")
+
+
+if __name__ == "__main__":
+    main()

scripts/pill_gate_audit.py

@@ -0,0 +1,224 @@
+"""Measure the Jimeng pill on the PRODUCT path, not the raw detector.
+
+WHY THIS EXISTS
+  `visible_removal_audit.py` calls `get_mark(key).remove` directly, which bypasses
+  `_keep_pill`. For the pill that is the wrong path and it reads as a disaster: on a
+  300-image sample the raw path was "detector still fires after removal" 68-75% of the
+  time. The product does not do that -- `--mark auto` gates the pill hard (never on
+  Doubao; unrestricted only when the bottom-right wordmark fired; otherwise only on a
+  flat footprint). Measured through the gate on the same sample, 7 of 80 raw detections
+  survived to removal and 6 of those were corroborated real.
+
+  That sample was too small to state a precision (95% CI 49-97%). This script runs the
+  gated path over EVERY pill positive in the corpus so the interval is usable.
+
+THE CORROBORATION PROXY AND ITS BIAS
+  "Real pill" here means: TC260 metadata names Jimeng, OR the bottom-right "★ 即梦AI"
+  wordmark is detected. Both are independent of the pill detector, which is the point.
+  But the proxy MISSES a real pill on a metadata-stripped screenshot with no visible
+  wordmark -- so measured precision is a LOWER BOUND, not a point estimate. Do not quote
+  it as if it were exact.
+
+Corpus images are user uploads: read-only, local analysis, gitignored output.
+
+    uv run python scripts/pill_gate_audit.py --jobs 7
+"""
+
+from __future__ import annotations
+
+import argparse
+import collections
+import json
+import math
+import os
+import sys
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from concurrent.futures import TimeoutError as FutureTimeout
+from concurrent.futures.process import BrokenProcessPool
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+REPO = Path(__file__).resolve().parents[1]
+POSITIVES = REPO / "data" / "spaces" / "_visible_positives.jsonl"
+OUT = REPO / "data" / "spaces" / "_pill_gate_audit.jsonl"
+
+
+def wilson(k: int, n: int, z: float = 1.96) -> tuple[float, float]:
+    if n == 0:
+        return (0.0, 0.0)
+    p = k / n
+    d = 1 + z * z / n
+    c = (p + z * z / (2 * n)) / d
+    h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
+    return (100 * (c - h), 100 * (c + h))
+
+
+def _one(path_str: str) -> dict[str, object]:
+    from remove_ai_watermarks.api import visible_provenance
+    from remove_ai_watermarks.image_io import imread
+    from remove_ai_watermarks.pill_engine import PillEngine
+    from remove_ai_watermarks.watermark_registry import detect_marks, get_mark, remove_auto_marks
+
+    rec: dict[str, object] = {"path": Path(path_str).name, "status": "ok"}
+    try:
+        img = imread(path_str)
+        if img is None:
+            return {**rec, "status": "unreadable"}
+        prov = visible_provenance(Path(path_str))
+        dets = {d.key: d for d in detect_marks(img) if d.detected}
+        wordmark = "jimeng" in dets
+        tc260 = "jimeng" in prov
+        eng = PillEngine()
+        _, labels = remove_auto_marks(img, sensitivity="auto", provenance=prov, backend="cv2")
+        # Match the EXACT pill label. A substring test on "AI生成" also matches Doubao's
+        # label ("Doubao 豆包AI生成 text"), which counted Doubao removals as pill removals
+        # and inflated the pill's measured precision.
+        pill_label = get_mark("jimeng_pill").label
+        removed = any(str(x) == pill_label for x in labels)
+        # The gate reads the ARBITRATED keys, not strict detection: jimeng can be accepted
+        # on its relaxed (provenance-confirmed) arm, which `detect_marks` never shows. Take
+        # the arm from what was actually removed, or every such case lands in "other".
+        jimeng_decided = any(str(x) == get_mark("jimeng").label for x in labels)
+        return {
+            **rec,
+            "removed_by_product": bool(removed),
+            "corroborated": bool(wordmark or tc260),
+            "wordmark": bool(wordmark),
+            "jimeng_decided": bool(jimeng_decided),
+            "tc260": bool(tc260),
+            "doubao_fired": "doubao" in dets,
+            "footprint_flat": bool(eng.footprint_is_flat(img)),
+        }
+    except Exception as e:
+        return {**rec, "status": f"error:{type(e).__name__}"}
+
+
+def _batch(paths: list[str], jobs: int, timeout: int) -> list[dict[str, object]]:
+    try:
+        with ProcessPoolExecutor(max_workers=jobs) as ex:
+            futs = [ex.submit(_one, p) for p in paths]
+            return [f.result() for f in as_completed(futs, timeout=timeout)]
+    except (FutureTimeout, BrokenProcessPool, OSError, RuntimeError):
+        out = []
+        for p in paths:
+            try:
+                out.append(_one(p))
+            except BaseException:  # a native crash costs one file, not the sweep
+                out.append({"path": Path(p).name, "status": "crashed"})
+        return out
+
+
+def report(rows: list[dict[str, object]]) -> None:
+    ok = [r for r in rows if r.get("status") == "ok"]
+    if not ok:
+        print("no usable rows")
+        return
+    removed = [r for r in ok if r["removed_by_product"]]
+    corro = [r for r in ok if r["corroborated"]]
+    tp = [r for r in removed if r["corroborated"]]
+    missed = [r for r in corro if not r["removed_by_product"]]
+
+    print(f"\n{'=' * 76}\nJIMENG PILL ON THE PRODUCT PATH   n={len(ok)} raw detections\n{'=' * 76}")
+    print(f"raw detections corroborated as real : {len(corro):5d}  ({100 * len(corro) / len(ok):.1f}%)")
+    print(f"the gate lets through to removal    : {len(removed):5d}  ({100 * len(removed) / len(ok):.1f}%)")
+    if removed:
+        lo, hi = wilson(len(tp), len(removed))
+        pct = 100 * len(tp) / len(removed)
+        print(f"  of those, corroborated real       : {len(tp):5d}  -> precision {pct:.1f}% (95% CI {lo:.1f}-{hi:.1f})")
+    if corro:
+        lo, hi = wilson(len(tp), len(corro))
+        pct = 100 * len(tp) / len(corro)
+        head = f"corroborated pills removed          : {len(tp):5d}/{len(corro)}"
+        print(f"{head}  -> recall {pct:.1f}% (95% CI {lo:.1f}-{hi:.1f})")
+    print(f"corroborated pills the gate suppressed: {len(missed):5d}")
+    print("\nprecision is a LOWER BOUND: the corroboration proxy cannot see a real pill on a")
+    print("metadata-stripped image whose wordmark is absent or missed.\n")
+
+    print("which gate arm let it through:")
+    arms: collections.Counter[str] = collections.Counter()
+    for r in removed:
+        if r.get("jimeng_decided"):
+            arms["jimeng wordmark accepted -> unrestricted"] += 1
+        elif r["tc260"]:
+            arms["tc260 metadata + flat footprint"] += 1
+        else:
+            arms["other (unexpected -- the gate has no third arm)"] += 1
+    for k, v in arms.most_common():
+        print(f"   {v:5d}  {k}")
+
+    print("\nwhy the rest were suppressed:")
+    sup: collections.Counter[str] = collections.Counter()
+    for r in ok:
+        if r["removed_by_product"]:
+            continue
+        if r["doubao_fired"]:
+            sup["doubao fired (pill never rides on it)"] += 1
+        elif not r["corroborated"]:
+            sup["no corroboration (no wordmark, no TC260)"] += 1
+        elif not r["footprint_flat"]:
+            sup["corroborated but footprint too textured"] += 1
+        else:
+            sup["other"] += 1
+    for k, v in sup.most_common():
+        print(f"   {v:5d}  {k}")
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--jobs", type=int, default=max(1, (os.cpu_count() or 4) - 2))
+    ap.add_argument("--limit", type=int, default=0)
+    ap.add_argument("--batch", type=int, default=200)
+    ap.add_argument("--timeout", type=int, default=900)
+    ap.add_argument("--out", type=Path, default=OUT)
+    ap.add_argument("--restart", action="store_true")
+    a = ap.parse_args()
+
+    paths = []
+    with open(POSITIVES, encoding="utf-8") as fh:
+        for line in fh:
+            try:
+                r = json.loads(line)
+            except Exception:  # noqa: S112
+                continue
+            if "jimeng_pill" in (r.get("keys") or []):
+                paths.append(r["path"])
+    paths.sort()
+    if a.limit:
+        paths = paths[: a.limit]
+
+    done: set[str] = set()
+    if a.restart and a.out.exists():
+        a.out.unlink()  # --restart must TRUNCATE; appending kept the old rows in the report
+    if a.out.exists() and not a.restart:
+        with open(a.out, encoding="utf-8") as fh:
+            for line in fh:
+                try:
+                    done.add(json.loads(line)["path"])
+                except Exception:  # noqa: S112
+                    continue
+    todo = [p for p in paths if Path(p).name not in done]
+    print(f"pill positives {len(paths)}  done {len(done)}  to do {len(todo)}  jobs {a.jobs}", flush=True)
+
+    rows: list[dict[str, object]] = []
+    with open(a.out, "a", encoding="utf-8") as fh:
+        for i in range(0, len(todo), a.batch):
+            for rec in _batch(todo[i : i + a.batch], a.jobs, a.timeout):
+                fh.write(json.dumps(rec) + "\n")
+                rows.append(rec)
+            fh.flush()
+            print(f"  {len(rows)}/{len(todo)}", flush=True)
+
+    with open(a.out, encoding="utf-8") as fh:
+        allrows = []
+        for line in fh:
+            try:
+                allrows.append(json.loads(line))
+            except Exception:  # noqa: S112
+                continue
+    report(allrows)
+    print(f"records -> {a.out}")
+
+
+if __name__ == "__main__":
+    main()

scripts/qwen_scrub_prototype.py

@@ -0,0 +1,128 @@
+# /// script
+# requires-python = ">=3.10"
+# dependencies = [
+#   "diffusers>=0.35.0",
+#   "transformers>=4.51.0",
+#   "torch",
+#   "accelerate",
+#   "pillow",
+#   "click",
+# ]
+# ///
+"""Isolated GPU prototype: does a low-strength Qwen-Image img2img pass scrub the
+invisible watermark while keeping text/structure legible?
+
+This is the oracle-gated experiment behind Library roadmap P1#5 (migrate the
+invisible pipeline onto Qwen-Image-Edit). It is DELIBERATELY standalone:
+
+  * It is NOT imported by the package and NOT in ``uv.lock``. Qwen-Image needs a
+    newer ``diffusers``/``transformers`` (Qwen2.5-VL text encoder) than the SDXL
+    pipeline is pinned to, so wiring it into the locked env would risk the
+    certified SDXL/ControlNet pipeline (the ``cannot import Qwen3VL...`` trap).
+    PEP 723 inline metadata lets ``uv run`` build a throwaway env for it instead.
+  * Qwen-Image is ~20B, so it needs a real GPU (CUDA) -- it will not fit on MPS.
+
+Run (on a GPU box / Modal), then eyeball the outputs AND submit them to the
+matching oracle (openai.com/verify for OpenAI, the Gemini app for Google):
+
+    uv run scripts/qwen_scrub_prototype.py INPUT.png -o out/ --strengths 0.1,0.2,0.3,0.4
+
+What to look for:
+  * SCRUB: the oracle no longer reports the watermark at some strength.
+  * FIDELITY: text stays legible and faces/structure stay faithful at that same
+    strength -- the whole point of trying Qwen over SDXL (which garbles text).
+The smallest strength that clears the oracle while keeping fidelity is the result
+to compare against the SDXL/ControlNet floors (OpenAI 0.10 / Google 0.15).
+"""
+
+from __future__ import annotations
+
+import logging
+from pathlib import Path
+
+import click
+
+log = logging.getLogger("qwen_proto")
+
+# A neutral, faithful-regeneration prompt (we want to scrub, not restyle); mirrors
+# the intent of the SDXL controlnet prompt. Qwen renders text natively, so a light
+# pass should keep captions legible where SDXL would garble them.
+_PROMPT = "high quality, sharp, detailed, faithful to the original"
+_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
+
+
+def _pick_device(requested: str) -> tuple[str, object]:
+    import torch
+
+    if requested != "auto":
+        device = requested
+    elif torch.cuda.is_available():
+        device = "cuda"
+    elif getattr(torch.backends, "mps", None) is not None and torch.backends.mps.is_available():
+        device = "mps"
+    else:
+        device = "cpu"
+    # bf16 on CUDA (Qwen's reference dtype); fp32 elsewhere for numerical safety.
+    dtype = torch.bfloat16 if device == "cuda" else torch.float32
+    return device, dtype
+
+
+@click.command()
+@click.argument("source", type=click.Path(exists=True, path_type=Path))
+@click.option("-o", "--output-dir", type=click.Path(path_type=Path), default=Path("qwen_out"))
+@click.option("--strengths", default="0.1,0.2,0.3,0.4", help="Comma-separated img2img strengths to sweep.")
+@click.option("--steps", type=int, default=40, help="Inference steps.")
+@click.option("--cfg", type=float, default=4.0, help="true_cfg_scale (Qwen's CFG; reference default 4.0).")
+@click.option("--model", default="Qwen/Qwen-Image", help="HF model id (Qwen-Image img2img base).")
+@click.option("--device", default="auto", type=click.Choice(["auto", "cuda", "mps", "cpu"]))
+@click.option("--seed", type=int, default=0, help="Reproducible seed.")
+def main(
+    source: Path,
+    output_dir: Path,
+    strengths: str,
+    steps: int,
+    cfg: float,
+    model: str,
+    device: str,
+    seed: int,
+) -> None:
+    """Sweep Qwen-Image img2img strength over SOURCE and save one output per strength."""
+    logging.basicConfig(level=logging.INFO, format="%(message)s")
+    import torch
+    from diffusers import QwenImageImg2ImgPipeline
+    from PIL import Image
+
+    dev, dtype = _pick_device(device)
+    log.info("Loading %s on %s (%s)...", model, dev, dtype)
+    pipe = QwenImageImg2ImgPipeline.from_pretrained(model, torch_dtype=dtype)
+    pipe = pipe.to(dev)
+
+    init_image = Image.open(source).convert("RGB")
+    output_dir.mkdir(parents=True, exist_ok=True)
+    values = [float(s) for s in strengths.split(",") if s.strip()]
+
+    for strength in values:
+        generator = torch.Generator(device="cpu").manual_seed(seed)
+        log.info("Generating strength=%.2f ...", strength)
+        result = pipe(
+            prompt=_PROMPT,
+            negative_prompt=_NEGATIVE,
+            image=init_image,
+            strength=strength,
+            num_inference_steps=steps,
+            true_cfg_scale=cfg,
+            generator=generator,
+        )
+        out_path = output_dir / f"{source.stem}_qwen_s{strength:.2f}.png"
+        result.images[0].save(out_path)
+        log.info("  saved %s", out_path)
+
+    log.info(
+        "\nDone. Eyeball text/face fidelity, then submit each output to the matching oracle "
+        "(openai.com/verify / Gemini app). The smallest strength that clears the oracle while "
+        "keeping fidelity is the number to compare against the SDXL floors (OpenAI 0.10 / Google 0.15)."
+    )
+
+
+if __name__ == "__main__":
+    main()

scripts/real_examples_e2e.py

@@ -0,0 +1,328 @@
+"""End-to-end confidence run: drive the ACTUAL CLI over REAL corpus examples.
+
+WHY THIS EXISTS AND WHAT IT IS NOT
+  The 849-test suite and `smoke_matrix.py` prove the code paths behave on fixtures and
+  synthetic inputs. This is the other half: run the real `remove-ai-watermarks` entry point,
+  as a user would, over real corpus images spanning every command and every provenance
+  class, and CHECK THE OUTPUT -- not that it exited 0, but that it did the right thing (the
+  mark is actually gone on re-detect, the metadata actually strips, the diffusion actually
+  writes a changed image). A green exit is not evidence the work happened.
+
+WHAT IT COVERS
+  identify   one real image per provenance class -> the verdict is right
+  metadata   real AI-metadata files -> --check detects, --remove strip-and-verifies clean
+  visible    real marked images per mark -> the mark is gone on re-detect, output written
+  erase      a real image, each fill backend (cv2 / migan / lama) -> output written
+  invisible  a real SynthID image on MPS at reduced resolution -> a CHANGED image is written
+  all        a real marked image through the full pipeline -> output written
+  batch      a real directory -> every input produces an output
+
+  invisible/all run the diffusion model, so they are gated behind --diffusion and run at a
+  small --max-resolution on MPS (the user's "reduced size on MPS" path). Everything else is
+  cv2/numpy and fast.
+
+DATA SAFETY
+  Corpus images are user uploads: read-only, local analysis, outputs to a gitignored temp
+  dir. Records example uids and pass/fail, never image content.
+
+    uv run python scripts/real_examples_e2e.py                 # fast surface (no diffusion)
+    uv run python scripts/real_examples_e2e.py --diffusion     # + invisible/all on MPS
+"""
+
+from __future__ import annotations
+
+import argparse
+import glob
+import json
+import random
+import shutil
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+REPO = Path(__file__).resolve().parents[1]
+CORPUS = REPO / "data" / "spaces" / "originals"
+DATASETS = REPO / "data" / "spaces" / "_visible_datasets"
+SAMPLES = REPO / "data" / "samples"
+_UV = shutil.which("uv") or "uv"  # full path avoids the partial-executable lint
+
+
+def run(args: list[str], timeout: int = 300) -> tuple[int, str]:
+    """Invoke the real installed CLI. Returns (exit_code, combined output)."""
+    proc = subprocess.run(  # noqa: S603 -- fixed argv, the whole point is the real entry point
+        [_UV, "run", "remove-ai-watermarks", *args],
+        cwd=REPO,
+        capture_output=True,
+        text=True,
+        timeout=timeout,
+        check=False,
+    )
+    return proc.returncode, (proc.stdout + proc.stderr)
+
+
+def find_visible_positive(mark: str) -> Path | None:
+    """A real corpus image the parity run bucketed as carrying this mark, that the current
+    detector STILL fires on (the bucket was built by an older run; re-confirm live)."""
+    from remove_ai_watermarks.image_io import imread
+    from remove_ai_watermarks.watermark_registry import detect_marks
+
+    pool = sorted(glob.glob(str(DATASETS / mark / "*")))
+    random.Random(3).shuffle(pool)  # noqa: S311 -- deterministic sampling, not cryptography
+    for p in pool[:60]:
+        img = imread(p)
+        if img is None:
+            continue
+        if any(d.detected and d.key == mark for d in detect_marks(img)):
+            return Path(p)
+    return None
+
+
+class Results:
+    """Collects one row per checked behaviour.
+
+    A FAIL keeps the command's OUTPUT. That is not cosmetic: the first run of this harness
+    discarded it, an `all` invocation failed once with exit 1, and because the output was
+    gone there was no way to tell which of that command's three distinct `SystemExit(1)`
+    paths had fired -- the failure was undiagnosable and did not reproduce. Keep the tail
+    so a transient is at least identifiable after the fact.
+    """
+
+    def __init__(self) -> None:
+        self.rows: list[tuple[str, str, bool, str, str]] = []
+
+    def add(self, cmd: str, example: str, ok: bool, detail: str, output: str = "") -> None:
+        self.rows.append((cmd, example, ok, detail, "" if ok else output[-800:]))
+        mark = "PASS" if ok else "FAIL"
+        print(f"  [{mark}] {cmd:28s} {example:26s} {detail}", flush=True)
+
+    def report(self) -> int:
+        n = len(self.rows)
+        bad = [r for r in self.rows if not r[2]]
+        print(f"\n{'=' * 78}\nREAL-EXAMPLE E2E  {n - len(bad)}/{n} passed")
+        print(f"{'=' * 78}")
+        for cmd, ex, ok, detail, out in self.rows:
+            if not ok:
+                print(f"  FAIL  {cmd}  {ex}  {detail}")
+                if out.strip():
+                    print("        --- command output (tail) ---")
+                    for line in out.strip().splitlines()[-12:]:
+                        print(f"        {line}")
+        if not bad:
+            print("  every command produced the right result on real corpus examples")
+        return 1 if bad else 0
+
+
+def check_identify(res: Results) -> None:
+    """One real image per provenance class -> the verdict is right (--json must parse)."""
+    print("\nidentify -- real image per provenance class")
+    cases = [
+        ("chatgpt-1.png", True),
+        ("firefly-1.png", True),
+        ("mj-1.png", True),
+        ("doubao-1.png", True),
+        ("grok-1.jpg", True),
+        ("flux-1.jpg", True),
+    ]
+    for name, want_ai in cases:
+        p = SAMPLES / name
+        if not p.exists():
+            res.add("identify", name, False, "sample missing")
+            continue
+        code, out = run(["identify", str(p), "--json"])
+        try:
+            data = json.loads(out[out.index("{") : out.rindex("}") + 1])
+        except (ValueError, json.JSONDecodeError):
+            res.add("identify", name, False, f"json did not parse (exit {code})")
+            continue
+        is_ai = data.get("is_ai_generated")
+        plat = (data.get("platform") or "").lower()
+        ok = bool(is_ai) is want_ai
+        res.add("identify", name, ok, f"is_ai={is_ai} platform={plat or '-'}")
+
+
+def check_metadata(res: Results, tmp: Path) -> None:
+    """Real AI-metadata files -> --check detects, --remove strip-and-verifies clean."""
+    print("\nmetadata -- real AI-metadata files, detect then strip-and-verify")
+    for name in ("chatgpt-1.png", "doubao-1.png", "mj-1.png", "grok-1.jpg", "flux-1.jpg"):
+        p = SAMPLES / name
+        if not p.exists():
+            res.add("metadata --check", name, False, "sample missing")
+            continue
+        code, out = run(["metadata", str(p), "--check"])
+        detected = "no ai" not in out.lower() and code == 0
+        res.add("metadata --check", name, detected, "AI metadata detected" if detected else "nothing detected")
+
+        outp = tmp / f"stripped_{name}"
+        code, out = run(["metadata", str(p), "--remove", "-o", str(outp)])
+        # strip-and-verify: the command re-scans the OUTPUT and fails loudly on leftovers.
+        clean = outp.exists() and code == 0 and "still" not in out.lower()
+        res.add("metadata --remove", name, clean, "output re-scanned clean" if clean else f"exit {code}")
+
+
+def check_visible(res: Results, tmp: Path) -> None:
+    """Real marked images -> the PRODUCT'S DECISION is honoured, and a removed mark clears.
+
+    The success criterion is not "the mark is always gone" -- it is "the product did what it
+    decided, and the decision is right". Two designed behaviours make a blind re-detect
+    misleading:
+      * The pill is GATED (`_keep_pill`): a low-confidence pill with no corroboration is
+        deliberately NOT removed, so `visible` correctly writes nothing and exits 2. That is
+        the gate working, not a miss -- checked by asking `remove_auto_marks` whether it
+        chose to act.
+      * Samsung is the faintest mark (peak alpha ~0.38) at a razor-thin 0.40 gate, so a
+        borderline positive can be reduced yet re-detect just above threshold. Reported as
+        the measured before->after confidence, not a bare pass/fail.
+    """
+    from remove_ai_watermarks.image_io import imread
+    from remove_ai_watermarks.watermark_registry import detect_marks, get_mark, remove_auto_marks
+
+    print("\nvisible --mark auto -- real marked image per mark, product decision then re-detect")
+    for mark in ("doubao", "jimeng", "qwen", "gemini", "samsung", "jimeng_pill"):
+        src = find_visible_positive(mark)
+        if src is None:
+            res.add("visible", mark, True, "no live positive in bucket (skipped, not a failure)")
+            continue
+        img = imread(str(src))
+        conf_before = next((d.confidence for d in detect_marks(img) if d.detected and d.key == mark), 0.0)
+        # What does the product DECIDE to do? (labels lists the marks it removed.)
+        _out_img, labels = remove_auto_marks(img, sensitivity="auto", provenance=frozenset(), backend="cv2")
+        acted = get_mark(mark).label in labels
+
+        outp = tmp / f"visible_{mark}{src.suffix}"
+        code, out = run(["visible", str(src), "--mark", "auto", "-o", str(outp)])
+
+        if not acted:
+            # The gate declined this mark. The CLI must then write nothing and exit 2 --
+            # that is the correct outcome, so verify the CLI agrees with the decision.
+            ok = code == 2 and not outp.exists()
+            res.add(
+                "visible", mark, ok, f"gate declined (conf {conf_before:.2f}); CLI exit {code}, no output -- correct"
+            )
+            continue
+
+        if not outp.exists():
+            res.add("visible", mark, False, f"product removed it but CLI wrote no output (exit {code})", out)
+            continue
+        cleaned = imread(str(outp))
+        conf_after = next((d.confidence for d in detect_marks(cleaned) if d.detected and d.key == mark), 0.0)
+        clean = conf_after == 0.0
+        if clean:
+            res.add("visible", mark, True, f"removed, re-detect clean ({conf_before:.2f} -> below gate)")
+        else:
+            # Reduced but still over the gate: a real residual. Honest partial, flagged.
+            res.add(
+                "visible",
+                mark,
+                False,
+                f"reduced {conf_before:.2f} -> {conf_after:.2f} but still over gate (faint-mark residual)",
+            )
+
+
+def check_erase(res: Results, tmp: Path) -> None:
+    """A real image, each fill backend actually runs and writes an output."""
+    from remove_ai_watermarks.region_eraser import lama_available, migan_available
+
+    print("\nerase --region -- each fill backend on a real image")
+    src = next((Path(p) for p in sorted(glob.glob(str(CORPUS / "*" / "*"))) if Path(p).stat().st_size > 50_000), None)
+    if src is None:
+        res.add("erase", "-", False, "no corpus image")
+        return
+    backends = ["cv2"] + (["migan"] if migan_available() else []) + (["lama"] if lama_available() else [])
+    for backend in backends:
+        outp = tmp / f"erase_{backend}{src.suffix}"
+        code, out = run(["erase", str(src), "--region", "10,10,120,60", "--backend", backend, "-o", str(outp)])
+        ok = outp.exists() and code == 0 and outp.stat().st_size > 0
+        res.add(f"erase --backend {backend}", src.name[:12], ok, "output written" if ok else f"exit {code}", out)
+
+
+def check_diffusion(res: Results, tmp: Path, gemini_src: str, openai_src: str) -> None:
+    """The GPU path: invisible + all on MPS at reduced resolution -> a CHANGED image."""
+    import numpy as np
+
+    from remove_ai_watermarks.image_io import imread
+
+    print("\ninvisible / all -- real SynthID image on MPS, reduced resolution")
+    for label, src in (("invisible/gemini", gemini_src), ("invisible/openai", openai_src)):
+        if not src:
+            res.add(label, "-", True, "no real positive found (skipped)")
+            continue
+        sp = Path(src)
+        outp = tmp / f"inv_{sp.stem}.png"
+        code, out = run(
+            ["invisible", src, "-o", str(outp), "--device", "mps", "--max-resolution", "512", "--seed", "0"],
+            timeout=1200,
+        )
+        if not outp.exists():
+            res.add(label, sp.name[:12], False, f"no output (exit {code})", out)
+            continue
+        before, after = imread(src), imread(str(outp))
+        # Diffusion regenerates every pixel; the output must actually differ from the input.
+        # A DIFFERENT SHAPE is itself proof it changed (the pipeline resizes), so treat it
+        # as changed rather than comparing arrays that cannot be compared. The first
+        # version wrote `array_equal(after, before if shapes match else after)`, which
+        # compares `after` WITH ITSELF on the mismatch branch and is therefore always
+        # "unchanged" -- it would have reported a genuinely resized output as a no-op.
+        changed = (
+            after is not None
+            and before is not None
+            and (before.shape != after.shape or not np.array_equal(before, after))
+        )
+        res.add(label, sp.name[:12], changed, "diffusion wrote a changed image" if changed else "output == input", out)
+
+    if gemini_src:
+        sp = Path(gemini_src)
+        outp = tmp / f"all_{sp.stem}.png"
+        code, out = run(
+            ["all", gemini_src, "-o", str(outp), "--device", "mps", "--max-resolution", "512", "--seed", "0"],
+            timeout=1200,
+        )
+        ok = outp.exists() and outp.stat().st_size > 0
+        res.add("all (full pipeline)", sp.name[:12], ok, "output written" if ok else f"exit {code}", out)
+
+
+def check_batch(res: Results, tmp: Path) -> None:
+    """A real directory -> every supported input produces an output."""
+    print("\nbatch -- a real directory")
+    indir = tmp / "batch_in"
+    indir.mkdir(exist_ok=True)
+    picks = sorted(glob.glob(str(DATASETS / "doubao" / "*")))[:5]
+    for p in picks:
+        shutil.copy2(p, indir / Path(p).name)
+    n_in = len(list(indir.glob("*")))
+    if n_in == 0:
+        res.add("batch", "-", False, "no inputs to seed")
+        return
+    outdir = tmp / "batch_out"
+    code, out = run(["batch", str(indir), "-o", str(outdir), "--mode", "visible"], timeout=600)
+    n_out = len(list(outdir.glob("*"))) if outdir.exists() else 0
+    ok = code == 0 and n_out >= n_in
+    res.add("batch --mode visible", f"{n_in} imgs", ok, f"{n_out}/{n_in} outputs (exit {code})", out)
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--diffusion", action="store_true", help="also run invisible/all on MPS (slow)")
+    ap.add_argument("--gemini", default="")
+    ap.add_argument("--openai", default="")
+    a = ap.parse_args()
+
+    res = Results()
+    with tempfile.TemporaryDirectory(prefix="raiw_e2e_") as td:
+        tmp = Path(td)
+        check_identify(res)
+        check_metadata(res, tmp)
+        check_visible(res, tmp)
+        check_erase(res, tmp)
+        check_batch(res, tmp)
+        if a.diffusion:
+            check_diffusion(res, tmp, a.gemini, a.openai)
+        else:
+            print("\n(diffusion skipped -- pass --diffusion to run invisible/all on MPS)")
+    raise SystemExit(res.report())
+
+
+if __name__ == "__main__":
+    main()

scripts/render_pill_silhouette.py

@@ -0,0 +1,60 @@
+"""Render the SYNTHETIC 'AI生成' pill silhouette asset (data-safe, font-rendered).
+
+The Jimeng-basic TC260 visible label is a rounded pill with 'AI生成' in the TOP-LEFT
+corner (issue #54). Unlike the reverse-alpha marks, this is a CAPTURE-LESS mark:
+the committed asset is a font-rendered binary SILHOUETTE (mark shape only, zero photo
+content), used ONLY to (a) detect the pill by edge-NCC in the top-left corner and
+(b) build the inpaint mask. It is NOT an alpha map -- removal quality comes from the
+inpaint backend (MI-GAN/cv2), so the silhouette need not be pixel-accurate, and the
+synthetic render keeps corpus/user content out of the tracked repo (data-safety).
+
+Detection was calibrated on the retained local corpus (61 real positives + jimeng
+negatives): edge-NCC threshold ~0.22 in the top-left ROI. Re-run to regenerate the
+asset:  uv run python scripts/render_pill_silhouette.py
+
+Requires a CJK font (macOS STHeiti by default); the asset itself is committed, so this
+script only runs when regenerating it (never in CI).
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+import numpy as np
+from PIL import Image, ImageDraw, ImageFont
+
+_ASSET = Path(__file__).resolve().parents[1] / "src" / "remove_ai_watermarks" / "assets" / "jimeng_pill.png"
+_FONT = "/System/Library/Fonts/STHeiti Medium.ttc"
+
+
+def render_silhouette(w: int = 320) -> np.ndarray:
+    """Rounded-pill outline + 'AI生成' text as a binary silhouette (255 = mark)."""
+    h = int(w * 0.5)
+    im = Image.new("L", (w, h), 0)
+    d = ImageDraw.Draw(im)
+    pad = int(w * 0.03)
+    r = (h - 2 * pad) // 3
+    d.rounded_rectangle([pad, pad, w - pad, h - pad], radius=r, outline=255, width=max(2, w // 90))
+    fsz = int(h * 0.42)
+    font = ImageFont.truetype(_FONT, fsz)
+    txt = "AI生成"
+    tb = d.textbbox((0, 0), txt, font=font)
+    tw, th = tb[2] - tb[0], tb[3] - tb[1]
+    d.text(((w - tw) // 2 - tb[0], (h - th) // 2 - tb[1]), txt, font=font, fill=255)
+    return np.array(im)
+
+
+def main() -> None:
+    try:
+        sil = render_silhouette()
+    except OSError as e:
+        print(f"Font not found ({e}); install a CJK font or edit _FONT.", file=sys.stderr)
+        raise SystemExit(1) from e
+    _ASSET.parent.mkdir(parents=True, exist_ok=True)
+    Image.fromarray(sil).save(_ASSET)
+    print(f"wrote {_ASSET}  ({sil.shape[1]}x{sil.shape[0]})")
+
+
+if __name__ == "__main__":
+    main()

scripts/render_vendor_silhouettes.py

@@ -0,0 +1,144 @@
+"""Render SYNTHETIC detection silhouettes for the CJK vendor text marks (data-safe).
+
+Adding a mark needs only a DETECTION silhouette, and it must be font-rendered rather
+than derived from user uploads: the corpus is real user content and may never reach a
+tracked asset (see the repo CLAUDE.md data-safety rule). Seeing real samples to learn
+the glyphs, weight and layout is fine; the committed template stays synthetic.
+
+Covered here:
+  qwen    "千问AI生成"  -- Alibaba Tongyi Qianwen, bottom-right, 3-lobed logo + text
+  xinghui "星绘AI生成"  -- ByteDance 星绘, bottom-right, 4-point sparkle + text
+
+The leading LOGO is deliberately NOT rendered. It is the part that varies most between
+releases and is hardest to reproduce synthetically, while the CJK run is stable and is
+what actually discriminates one vendor from another (the shared `AI生成` tail is exactly
+what does NOT discriminate -- see the rival-margin mechanism in _text_mark_engine).
+
+Regenerate with:  uv run python scripts/render_vendor_silhouettes.py
+
+STATUS 2026-07-21: `qwen_alpha.png` IS registered (`qwen_engine.py`) -- the 2026-07-18
+blocker quoted below turned out to be mis-sized GEOMETRY (two size modes + a locate box
+that clipped the first glyph), not segmentation, and was solved by the TC260-producer
+cohort harvest + `vendor_mark_calibrate.py` (117 labelled frames; full record in
+`docs/verification-plan.md`). `xinghui_alpha.png` is still NOT registered: one confirmed
+corpus example is nothing to calibrate a gate against.
+
+--- the 2026-07-18 record, kept as the history of the failed first attempt ---
+Measured on 14 hand-verified 千问 positives from the corpus,
+the then-current detect architecture (top-hat glyph blob -> binary TM_CCOEFF_NORMED)
+could not see this mark AT ALL:
+
+    same pipeline, each mark scored with its OWN template, on real positives
+      doubao   n=40   mean NCC 0.723   median 0.835   >= 0.40 gate: 82%
+      qwen     n=14   mean NCC 0.170   median 0.179   >= 0.40 gate:  0%
+
+Three checks ruled out the obvious explanations, in order:
+  1. NOT the synthetic render. A template cut from an ACTUAL Qwen mark scores the same
+     as the font-rendered one (real-vs-real 0.307 vs synthetic 0.308) -- and real masks
+     do not even match EACH OTHER.
+  2. NOT the morphology kernel size. Scaling MORPH_OPEN/CLOSE with the box height (they
+     are fixed 5px, ~9% of a 57px-tall box) gained only +0.014 mean and moved nothing
+     across the gate.
+  3. NOT the appearance thresholds. Sweeping tophat_delta / logo_min_luma / kernel
+     reached at best mean 0.35 with 4/14 over the gate.
+
+The blocker was named SEGMENTATION on a faint mark: Doubao is stamped bold and opaque,
+so the white top-hat returns a clean glyph blob; the Qwen mark is a thin translucent
+overlay that shatters into specks, and no template can match a blob that is not there.
+The `tophat` front-end (built later, for doubao) removed that blocker -- and 千问 STILL
+did not register, because the real residual was geometry. See the 2026-07-21 status
+above.
+
+星绘 additionally has only ONE confirmed example in the corpus, so even a working
+front-end could not have its threshold calibrated yet.
+
+UPDATE 2026-07-20: the named blocker is GONE, and the retry is still inconclusive.
+`detect_frontend="tophat"` (built later, for doubao) is exactly the "grayscale correlation
+on the raw top-hat" this note asked for, so the 2026-07-18 ruling rests on a premise that
+no longer holds and must not simply be inherited. Two things were measured against it, and
+neither settles the question:
+
+  * A GENERIC template of the shared `AI生成` tail -- attractive because GB 45438-2025
+    guarantees that run across vendors, so one template would cover 千问 / 百度 / 星绘 and
+    anything compliant that ships next. Measured on the tophat front-end at the shipped
+    3-rung ladder: a bold 千问 positive scores 0.407 against clean corners at p99 0.298 /
+    max 0.321. It separates on that one frame, but only by a hair, and a 4-glyph template
+    is inherently less specific than a 6-glyph one -- the shorter the run, the more
+    arbitrary corner structure correlates with it.
+  * The FULL 千问 template on the same front-end scores 0.248 against a clean max of 0.537,
+    i.e. no separation at all -- WORSE than the generic tail, which is the opposite of
+    what the specificity argument predicts and is itself a reason to distrust n=1.
+
+The blocker is now EVIDENCE, not architecture: this session found exactly one 千问 and one
+百度 positive (both by eyeballing doubao-provenance misses), and the 14 positives quoted
+above were not preserved anywhere the current scripts can reach. Nothing should be
+registered off a single frame.
+
+UPDATE 2026-07-21 (the resolution): the evidence arrived via the TC260 producer-USCC
+cohort trick (`scripts/vendor_cohort_harvest.py` -- 117 labelled 千问 frames from metadata
+alone), and the registration shipped the same day (`qwen_engine.py`). The "no separation
+at all" reading above was the MIS-SIZED geometry, not the mark: at the fitted geometry the
+full template separates the cohort from clean corners 0.662 vs 0.134 (p50). The traps
+below still bind any NEXT vendor: score with `alpha_height_frac`, not the silhouette's own
+aspect ratio (the latter inflated the clean p99 from 0.30 to 0.58 and made every
+comparison meaningless); keep the ladder at the shipped rungs for gate-setting, since a
+wide sweep hands clean corners many extra chances to match; and re-filter the clean arm
+per candidate -- the 2026-07-18 `present: []` labels mean "no REGISTERED mark", so qwen
+-cohort frames visibly carrying 千问AI生成 sat in it (see `vendor_mark_calibrate.load_sets`).
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+import numpy as np
+from PIL import Image, ImageDraw, ImageFont
+
+_ASSETS = Path(__file__).resolve().parents[1] / "src" / "remove_ai_watermarks" / "assets"
+# STHeiti Medium approximates the semibold CJK sans these marks are set in; the exact
+# family is unpublished for every vendor (GB 45438-2025 only requires a legible face).
+_FONT = "/System/Library/Fonts/STHeiti Medium.ttc"
+
+MARKS = {
+    "qwen_alpha.png": "千问AI生成",
+    "xinghui_alpha.png": "星绘AI生成",
+}
+
+
+def render(text: str, width: int = 335) -> np.ndarray:
+    """Binary glyph silhouette (255 = glyph), sized to the doubao asset's convention.
+
+    Matching doubao's 335px asset width keeps the `alpha_*_frac` numbers transferable,
+    since these marks are the same house style and scale.
+    """
+    probe = Image.new("L", (10, 10))
+    d0 = ImageDraw.Draw(probe)
+    size = 8
+    while size < 200:  # grow until the run fills the target width
+        f = ImageFont.truetype(_FONT, size)
+        if d0.textbbox((0, 0), text, font=f)[2] >= width * 0.98:
+            break
+        size += 1
+    font = ImageFont.truetype(_FONT, size)
+    bb = d0.textbbox((0, 0), text, font=font)
+    w, h = bb[2] - bb[0], bb[3] - bb[1]
+    pad = max(2, int(h * 0.12))
+    im = Image.new("L", (w + 2 * pad, h + 2 * pad), 0)
+    ImageDraw.Draw(im).text((pad - bb[0], pad - bb[1]), text, font=font, fill=255)
+    return np.array(im)
+
+
+def main() -> None:
+    try:
+        for name, text in MARKS.items():
+            sil = render(text)
+            Image.fromarray(sil).save(_ASSETS / name)
+            print(f"wrote {_ASSETS / name}  ({sil.shape[1]}x{sil.shape[0]})  text={text!r}")
+    except OSError as e:
+        print(f"Font not found ({e}); install a CJK font or edit _FONT.", file=sys.stderr)
+        raise SystemExit(1) from e
+
+
+if __name__ == "__main__":
+    main()

scripts/resource_ceilings.py

@@ -0,0 +1,156 @@
+"""Tier B4: peak RSS and wall time per fill backend across input sizes.
+
+WHY THIS EXISTS
+  The docs describe the backends in CAPABILITY prose -- "MI-GAN is the memory-tight pick",
+  "big-LaMa does not fit a minimal droplet", "~0.6-0.9 GB regardless of upload size". Those
+  numbers were measured once, informally, and are now load-bearing for a real deployment
+  decision (the free tier runs `migan`, the paid tier runs `lama`). This measures them.
+
+WHAT IT MEASURES
+  For each (backend, input size): peak RSS of a FRESH process doing exactly one erase, and
+  the wall time. A fresh subprocess per measurement is the point -- peak RSS inside a
+  long-lived process is contaminated by whatever ran before it, and the question here is
+  what a per-request worker actually needs.
+
+  The mask is a fixed, small corner region at every size, because the claim under test is
+  that the learned backends crop around the mask and so their memory is bounded by the MARK
+  size, not the image size. If that holds, the curve is flat in input size; if it does not,
+  it climbs and the droplet sizing is wrong.
+
+READING IT
+  RSS is the peak resident set of the whole worker, which includes the interpreter, numpy,
+  cv2 and (for the learned backends) onnxruntime plus the model. That is the honest number
+  for sizing a container -- not the model tensor alone.
+
+DATA SAFETY
+  Generates its own synthetic inputs. Reads nothing from the corpus, writes nothing tracked.
+
+    uv run python scripts/resource_ceilings.py                  # cv2 + whatever is installed
+    uv run python scripts/resource_ceilings.py --max-mp 25      # push to 25 MP
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import shutil
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+REPO = Path(__file__).resolve().parents[1]
+_UV = shutil.which("uv") or "uv"
+
+# (label, width, height) -- 1 MP up to 25 MP, the range a phone photo upload spans.
+SIZES = (("1MP", 1000, 1000), ("4MP", 2000, 2000), ("12MP", 4000, 3000), ("25MP", 5000, 5000))
+
+# The child process: build an image, erase one small corner region, report peak RSS.
+# Kept as a string so each measurement is a genuinely fresh interpreter.
+_CHILD = """
+import json, resource, sys, time
+import numpy as np
+from remove_ai_watermarks.region_eraser import erase
+
+w, h, backend = int(sys.argv[1]), int(sys.argv[2]), sys.argv[3]
+rng = np.random.default_rng(0)
+# Textured, not flat: a flat image compresses and can let a backend shortcut work.
+img = rng.integers(0, 255, (h, w, 3), dtype=np.uint8)
+box = (w - 320, h - 90, 300, 70)   # a mark-sized corner region at every input size
+t0 = time.monotonic()
+try:
+    out = erase(img, boxes=[box], backend=backend)
+    # Assert the fill actually RAN. A wrong call signature or a silently-declining
+    # backend would otherwise report the process's numpy footprint as if it were the
+    # backend's cost -- the first version of this script did exactly that on all 12
+    # cells and the numbers looked plausible.
+    # Compare ONLY the mask box. A full-frame `(out != img).any()` allocates a boolean
+    # temp the size of the image BEFORE getrusage is read -- 75 MB at 25 MP, ~17% of the
+    # cv2 figure, and it GROWS with the input, so the harness would partly manufacture
+    # the very "cv2 scales with input size" conclusion it is measuring.
+    bx, by, bw_, bh_ = box
+    changed = int((out[by:by+bh_, bx:bx+bw_] != img[by:by+bh_, bx:bx+bw_]).any())
+    err = "" if changed else "NO-OP: backend did not modify the masked region"
+except Exception as e:
+    err = f"{type(e).__name__}: {e}"[:200]
+elapsed = time.monotonic() - t0
+peak_kb = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
+print("RESULT" + json.dumps({"peak_kb": peak_kb, "sec": round(elapsed, 2), "error": err}))
+"""
+
+
+def peak_rss_mb(peak_kb: int) -> float:
+    """ru_maxrss is BYTES on macOS and KILOBYTES on Linux -- normalize to MB.
+
+    Getting this wrong silently reports 1024x off, which would look like a dramatic
+    finding rather than a unit bug.
+    """
+    return peak_kb / (1024 * 1024) if sys.platform == "darwin" else peak_kb / 1024
+
+
+def measure(backend: str, w: int, h: int, timeout: int = 900) -> dict[str, object] | None:
+    with tempfile.NamedTemporaryFile("w", suffix=".py", delete=False) as fh:
+        fh.write(_CHILD)
+        child = fh.name
+    try:
+        p = subprocess.run(  # noqa: S603 -- fixed argv, our own child script
+            [_UV, "run", "python", child, str(w), str(h), backend],
+            cwd=REPO,
+            capture_output=True,
+            text=True,
+            timeout=timeout,
+            check=False,
+        )
+    except subprocess.TimeoutExpired:
+        return {"error": f"TIMEOUT >{timeout}s", "sec": float(timeout), "peak_mb": float("nan")}
+    finally:
+        Path(child).unlink(missing_ok=True)
+
+    line = next((x for x in p.stdout.splitlines() if x.startswith("RESULT")), None)
+    if line is None:
+        return {"error": (p.stderr or p.stdout)[-200:], "sec": 0.0, "peak_mb": float("nan")}
+    data = json.loads(line[len("RESULT") :])
+    return {"error": data["error"], "sec": data["sec"], "peak_mb": round(peak_rss_mb(data["peak_kb"]), 1)}
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--max-mp", type=int, default=25, help="skip sizes above this megapixel count")
+    a = ap.parse_args()
+
+    from remove_ai_watermarks.region_eraser import lama_available, migan_available
+
+    backends = ["cv2"] + (["migan"] if migan_available() else []) + (["lama"] if lama_available() else [])
+    sizes = [s for s in SIZES if (s[1] * s[2]) / 1e6 <= a.max_mp + 0.5]
+    print(f"backends: {backends}\nsizes: {[s[0] for s in sizes]}")
+    print("\nOne FRESH process per cell; the mask is a fixed small corner at every size.")
+    print("If the learned backends really crop around the mask, RSS stays flat in input size.\n")
+
+    print(f"{'backend':8s} {'size':6s} {'peak RSS':>11s} {'wall':>8s}  note")
+    rows: list[tuple[str, str, dict[str, object]]] = []
+    for backend in backends:
+        for label, w, h in sizes:
+            r = measure(backend, w, h)
+            if r is None:
+                continue
+            rows.append((backend, label, r))
+            note = str(r["error"])[:48] or "ok"
+            print(f"{backend:8s} {label:6s} {r['peak_mb']:8} MB {r['sec']:7}s  {note}", flush=True)
+
+    print(f"\n{'=' * 72}\nRESOURCE CEILINGS\n{'=' * 72}")
+    for backend in backends:
+        cells = [(lbl, r) for b, lbl, r in rows if b == backend and not r["error"]]
+        if not cells:
+            continue
+        peaks = [float(r["peak_mb"]) for _, r in cells]  # type: ignore[arg-type]
+        lo, hi = min(peaks), max(peaks)
+        growth = "flat in input size" if hi <= lo * 1.6 else f"GROWS {hi / max(lo, 0.1):.1f}x with input size"
+        print(f"  {backend:6s} peak {lo:.0f}-{hi:.0f} MB across {cells[0][0]}..{cells[-1][0]}  -- {growth}")
+    print("\nRSS is the whole worker (interpreter + numpy + cv2 + any model), i.e. the")
+    print("number to size a container with, not the model tensor alone.")
+
+
+if __name__ == "__main__":
+    main()

scripts/robustness_suite.py

@@ -0,0 +1,341 @@
+"""Tier E: does the real CLI fail GRACEFULLY on adversarial and degenerate inputs?
+
+WHAT "PASS" MEANS HERE
+  Not "it succeeded" -- most of these inputs SHOULD be rejected. A pass is a graceful
+  outcome: a clear message, a sane exit code, and **no unhandled traceback**, within the
+  timeout. The failure modes this is hunting are the ones a user actually hits and that no
+  unit test covers, because unit tests feed well-formed fixtures:
+
+    * an unhandled traceback  -- the library crashed instead of reporting
+    * a hang                  -- worse than a crash for a batch caller
+    * a silent success on garbage -- it "processed" a corrupt file and wrote something
+
+  A non-zero exit with a readable error is a PASS. A Python traceback is a FAIL even when
+  the exit code looks tidy.
+
+WHY THESE INPUTS
+  Every case is drawn from something real: ~0.2% of corpus uploads are truncated, ~2% carry
+  a mismatched extension, Unicode filenames were issue #17, and a wrapping service will run
+  concurrent jobs against one path. Decompression bombs and absurd geometry are the cheap
+  denial-of-service shapes any tool taking user uploads must survive.
+
+DATA SAFETY
+  Builds its own inputs (synthetic, or truncated copies of committed fixtures) inside a
+  temp dir. Reads corpus images read-only for the one large-input case. Writes nothing
+  tracked.
+
+    uv run python scripts/robustness_suite.py
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+import shutil
+import stat
+import subprocess
+import sys
+import tempfile
+import zlib
+from concurrent.futures import ThreadPoolExecutor
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+REPO = Path(__file__).resolve().parents[1]
+SAMPLES = REPO / "data" / "samples"
+_UV = shutil.which("uv") or "uv"
+
+# A traceback in the output means the failure escaped the error handling, whatever the
+# exit code says.
+_CRASH_MARKERS = ("Traceback (most recent call last)", "Fatal Python error", "Segmentation fault")
+
+
+class Results:
+    def __init__(self) -> None:
+        self.rows: list[tuple[str, str, bool, str, str]] = []
+
+    def add(self, case: str, cmd: str, ok: bool, detail: str, output: str = "") -> None:
+        self.rows.append((case, cmd, ok, detail, "" if ok else output[-600:]))
+        print(f"  [{'PASS' if ok else 'FAIL'}] {case:34s} {cmd:10s} {detail}", flush=True)
+
+    def report(self) -> int:
+        bad = [r for r in self.rows if not r[2]]
+        print(f"\n{'=' * 78}\nROBUSTNESS (Tier E)  {len(self.rows) - len(bad)}/{len(self.rows)} graceful")
+        print(f"{'=' * 78}")
+        for case, cmd, ok, detail, out in self.rows:
+            if not ok:
+                print(f"  FAIL  {case}  ({cmd})  {detail}")
+                for line in out.strip().splitlines()[-10:]:
+                    print(f"        {line}")
+        if not bad:
+            print("  every adversarial input was handled without a crash or a hang")
+        return 1 if bad else 0
+
+
+def run(args: list[str], timeout: int = 120) -> tuple[int, str, bool]:
+    """Returns (exit_code, output, timed_out)."""
+    try:
+        p = subprocess.run(  # noqa: S603 -- fixed argv, driving our own CLI on purpose
+            [_UV, "run", "remove-ai-watermarks", *args],
+            cwd=REPO,
+            capture_output=True,
+            text=True,
+            timeout=timeout,
+            check=False,
+        )
+    except subprocess.TimeoutExpired:
+        return (-1, "", True)
+    return (p.returncode, p.stdout + p.stderr, False)
+
+
+def graceful(res: Results, case: str, cmd: str, args: list[str], timeout: int = 120) -> None:
+    """Run one adversarial case and score it on gracefulness, not on success."""
+    code, out, timed_out = run(args, timeout=timeout)
+    if timed_out:
+        res.add(case, cmd, False, f"HUNG (> {timeout}s)", out)
+        return
+    crashed = next((m for m in _CRASH_MARKERS if m in out), None)
+    if crashed:
+        res.add(case, cmd, False, f"unhandled {crashed} (exit {code})", out)
+        return
+    res.add(case, cmd, True, f"handled cleanly, exit {code}")
+
+
+def make_inputs(tmp: Path) -> dict[str, Path]:
+    """Build the adversarial corpus. Each entry is something a real upload can be."""
+    import numpy as np
+
+    from remove_ai_watermarks.image_io import imwrite
+
+    made: dict[str, Path] = {}
+
+    # A well-formed baseline, so a failure elsewhere is attributable to the input.
+    good = tmp / "good.png"
+    imwrite(good, np.full((600, 800, 3), 128, np.uint8))
+    made["good"] = good
+
+    # Truncated: a real PNG cut mid-stream (~0.2% of real uploads).
+    src = SAMPLES / "chatgpt-1.png"
+    if src.exists():
+        raw = src.read_bytes()
+        trunc = tmp / "truncated.png"
+        trunc.write_bytes(raw[: len(raw) // 3])
+        made["truncated"] = trunc
+
+    # Corrupt: correct magic bytes, garbage body.
+    corrupt = tmp / "corrupt.png"
+    corrupt.write_bytes(b"\x89PNG\r\n\x1a\n" + os.urandom(4096))
+    made["corrupt"] = corrupt
+
+    # Zero-byte file with a valid extension.
+    empty = tmp / "empty.png"
+    empty.write_bytes(b"")
+    made["empty"] = empty
+
+    # Not an image at all, but named like one.
+    text = tmp / "actually_text.jpg"
+    text.write_bytes(b"this is not an image, it is a text file pretending\n" * 50)
+    made["not_an_image"] = text
+
+    # Degenerate geometry: a 1x1, and a 1-pixel-tall sliver (the shape that once faulted
+    # cv2's GaussianBlur natively on Windows).
+    imwrite(tmp / "tiny.png", np.full((1, 1, 3), 200, np.uint8))
+    made["tiny_1x1"] = tmp / "tiny.png"
+    imwrite(tmp / "sliver.png", np.full((1, 4000, 3), 200, np.uint8))
+    made["sliver_1x4000"] = tmp / "sliver.png"
+
+    # Decompression bomb: a tiny file that decodes to a huge canvas. Hand-built so the
+    # on-disk size stays trivial while the declared dimensions are enormous.
+    bomb = tmp / "bomb.png"
+    bomb.write_bytes(_png_bomb(16000, 16000))
+    made["decompression_bomb"] = bomb
+
+    # Unicode + RTL filenames (issue #17 was Unicode-safe IO).
+    uni = tmp / "изображение-测试-🎨.png"
+    shutil.copy2(good, uni)
+    made["unicode_filename"] = uni
+    rtl = tmp / "صورة-اختبار.png"
+    shutil.copy2(good, rtl)
+    made["rtl_filename"] = rtl
+
+    # Mismatched extension: PNG content named .jpg (~2% of real uploads).
+    mismatch = tmp / "png_named_jpg.jpg"
+    shutil.copy2(good, mismatch)
+    made["mismatched_extension"] = mismatch
+
+    return made
+
+
+def _png_bomb(w: int, h: int) -> bytes:
+    """A valid PNG header declaring a huge canvas over highly-compressible data."""
+
+    def chunk(tag: bytes, data: bytes) -> bytes:
+        return len(data).to_bytes(4, "big") + tag + data + zlib.crc32(tag + data).to_bytes(4, "big")
+
+    ihdr = w.to_bytes(4, "big") + h.to_bytes(4, "big") + bytes([8, 2, 0, 0, 0])  # 8-bit RGB
+    raw = b"".join(b"\x00" + b"\x00" * (w * 3) for _ in range(h))
+    return b"\x89PNG\r\n\x1a\n" + chunk(b"IHDR", ihdr) + chunk(b"IDAT", zlib.compress(raw, 9)) + chunk(b"IEND", b"")
+
+
+def check_bad_inputs(res: Results, tmp: Path, inputs: dict[str, Path]) -> None:
+    print("\nmalformed and degenerate inputs -- every command must refuse, not crash")
+    out = tmp / "out.png"
+    for case in ("truncated", "corrupt", "empty", "not_an_image", "tiny_1x1", "sliver_1x4000"):
+        src = inputs.get(case)
+        if src is None:
+            continue
+        graceful(res, case, "identify", ["identify", str(src)])
+        graceful(res, case, "visible", ["visible", str(src), "-o", str(out)])
+        graceful(res, case, "metadata", ["metadata", str(src), "--check"])
+
+
+def check_bomb(res: Results, tmp: Path, inputs: dict[str, Path]) -> None:
+    print("\ndecompression bomb -- must not exhaust memory or hang")
+    bomb = inputs.get("decompression_bomb")
+    if bomb is None:
+        return
+    size_kb = bomb.stat().st_size / 1024
+    print(f"  (bomb is {size_kb:.0f} KB on disk, declares 16000x16000)")
+    graceful(res, "decompression_bomb", "identify", ["identify", str(bomb)], timeout=180)
+    graceful(res, "decompression_bomb", "visible", ["visible", str(bomb), "-o", str(tmp / "b.png")], timeout=180)
+
+
+def check_filenames(res: Results, tmp: Path, inputs: dict[str, Path]) -> None:
+    """Unicode/RTL paths must round-trip on BOTH read and write (issue #17)."""
+    print("\nunicode / RTL / mismatched-extension paths")
+    for case in ("unicode_filename", "rtl_filename", "mismatched_extension"):
+        src = inputs.get(case)
+        if src is None:
+            continue
+        # Write to a Unicode OUTPUT path too -- the read side alone does not prove the IO.
+        outp = tmp / f"вывод-{case}-📤.png"
+        code, out, timed = run(["identify", str(src)])
+        if timed or any(m in out for m in _CRASH_MARKERS):
+            res.add(case, "identify", False, "crashed or hung", out)
+        else:
+            res.add(case, "identify", True, f"read fine, exit {code}")
+        code, out, timed = run(["erase", str(src), "--region", "5,5,50,30", "-o", str(outp)])
+        wrote = outp.exists() and outp.stat().st_size > 0
+        crash = any(m in out for m in _CRASH_MARKERS)
+        res.add(case, "erase", wrote and not crash and not timed, f"unicode output written={wrote} exit={code}", out)
+
+
+def check_output_paths(res: Results, tmp: Path, inputs: dict[str, Path]) -> None:
+    print("\nhostile output paths")
+    good = inputs["good"]
+
+    # A nested output dir that does not exist yet -- the CLI should create it.
+    nested = tmp / "a" / "b" / "c" / "out.png"
+    code, out, timed = run(["erase", str(good), "--region", "5,5,40,20", "-o", str(nested)])
+    crash = any(m in out for m in _CRASH_MARKERS)
+    res.add(
+        "nonexistent_nested_outdir",
+        "erase",
+        nested.exists() and not crash and not timed,
+        f"created={nested.exists()} exit={code}",
+        out,
+    )
+
+    # A READ-ONLY output directory -- must report, not traceback.
+    ro = tmp / "readonly"
+    ro.mkdir(exist_ok=True)
+    ro.chmod(stat.S_IRUSR | stat.S_IXUSR)
+    try:
+        graceful(
+            res, "readonly_output_dir", "erase", ["erase", str(good), "--region", "5,5,40,20", "-o", str(ro / "x.png")]
+        )
+    finally:
+        ro.chmod(stat.S_IRWXU)  # restore so the temp dir can be cleaned up
+
+    # A directory where a file is expected.
+    graceful(res, "directory_as_input", "identify", ["identify", str(tmp)])
+
+    # A path that simply is not there.
+    graceful(res, "missing_input", "identify", ["identify", str(tmp / "nope.png")])
+
+
+def check_concurrency(res: Results, tmp: Path, inputs: dict[str, Path]) -> None:
+    """A wrapping service will run jobs in parallel against one input path."""
+    print("\nconcurrent runs against a single input")
+    good = inputs["good"]
+
+    def one(i: int) -> tuple[int, str, bool]:
+        return run(["erase", str(good), "--region", "5,5,40,20", "-o", str(tmp / f"conc_{i}.png")])
+
+    with ThreadPoolExecutor(max_workers=4) as ex:
+        outs = list(ex.map(one, range(4)))
+    crashed = [o for o in outs if any(m in o[1] for m in _CRASH_MARKERS) or o[2]]
+    all_written = all((tmp / f"conc_{i}.png").exists() for i in range(4))
+    res.add(
+        "4x concurrent on one input",
+        "erase",
+        not crashed and all_written,
+        f"{sum(1 for i in range(4) if (tmp / f'conc_{i}.png').exists())}/4 outputs, {len(crashed)} crashed",
+        "".join(o[1] for o in crashed),
+    )
+
+
+def check_batch_silent_loss(res: Results, tmp: Path, inputs: dict[str, Path]) -> None:
+    """The nastiest shape: NO output files AND a success exit code.
+
+    `graceful()` cannot see this class -- it scores exit code and traceback markers, and a
+    run that writes nothing while exiting 0 has neither. Corpus-reproduced 2026-07-20:
+    `batch --mode visible` into a read-only directory wrote 0 of 2 files and exited 0, so a
+    wrapping service would treat an empty output directory as a completed run. Any check
+    for a silent no-op must assert on the ARTIFACTS, not on the status.
+    """
+    print("\nbatch into a read-only output dir -- must NOT exit 0 with nothing written")
+    indir = tmp / "loss_in"
+    indir.mkdir(exist_ok=True)
+    for i in range(2):
+        shutil.copy2(inputs["good"], indir / f"img{i}.png")
+    ro = tmp / "loss_out"
+    ro.mkdir(exist_ok=True)
+    ro.chmod(stat.S_IRUSR | stat.S_IXUSR)
+    try:
+        code, out, timed = run(["batch", str(indir), "-o", str(ro)], timeout=180)
+        written = len(list(ro.glob("*")))
+    finally:
+        ro.chmod(stat.S_IRWXU)
+    ok = not timed and not (code == 0 and written == 0)
+    res.add(
+        "batch_readonly_outdir", "batch", ok, f"exit {code}, {written}/2 written (exit 0 + 0 files = data loss)", out
+    )
+
+
+def check_batch_edges(res: Results, tmp: Path) -> None:
+    print("\nbatch edge cases")
+    empty_dir = tmp / "empty_dir"
+    empty_dir.mkdir(exist_ok=True)
+    graceful(res, "batch_on_empty_dir", "batch", ["batch", str(empty_dir), "-o", str(tmp / "bo")])
+
+    # A directory of junk: nothing decodable, must not crash the whole run.
+    junk = tmp / "junk_dir"
+    junk.mkdir(exist_ok=True)
+    (junk / "a.png").write_bytes(b"\x89PNG\r\n\x1a\n" + os.urandom(500))
+    (junk / "b.jpg").write_bytes(b"not an image at all")
+    graceful(res, "batch_on_undecodable_dir", "batch", ["batch", str(junk), "-o", str(tmp / "jo")], timeout=180)
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.parse_args()
+    res = Results()
+    with tempfile.TemporaryDirectory(prefix="raiw_robust_") as td:
+        tmp = Path(td)
+        print("building adversarial inputs...")
+        inputs = make_inputs(tmp)
+        print(f"built {len(inputs)} inputs\n")
+        check_bad_inputs(res, tmp, inputs)
+        check_bomb(res, tmp, inputs)
+        check_filenames(res, tmp, inputs)
+        check_output_paths(res, tmp, inputs)
+        check_concurrency(res, tmp, inputs)
+        check_batch_silent_loss(res, tmp, inputs)
+        check_batch_edges(res, tmp)
+    raise SystemExit(res.report())
+
+
+if __name__ == "__main__":
+    main()

scripts/sidecar_regression.py

@@ -0,0 +1,216 @@
+"""Tier A1: diff today's `identify` against the verdicts recorded in the corpus sidecars.
+
+`data/spaces/identify/<day>/<uid>.json` holds the verdict a past run produced for
+`data/spaces/originals/<day>/<uid>_src.<ext>`. Re-running identify and diffing turns the
+corpus into a ~39k-image behavioral regression suite that needs no new labelling.
+
+WHAT A DIFF MEANS -- READ THIS BEFORE PANICKING
+  The sidecars were written by OLDER versions, so an intended improvement shows up as a
+  diff exactly like a regression does. The output is therefore CLASSIFIED, not pass/fail:
+
+    lost_ai        verdict was AI, now is not          <- the alarm; almost always a bug
+    lost_signal    a signal family stopped firing      <- the alarm
+    new_ai         verdict was not AI, now is          <- usually an improvement
+    new_signal     a signal family started firing      <- usually an improvement
+    platform       attribution changed
+    confidence     confidence level changed
+    unchanged      identical on every compared axis
+
+  Only `lost_*` is a regression by default. Everything else needs a look before the
+  baseline is moved.
+
+WHY FAMILIES, NOT RAW STRINGS
+  Watermark descriptions are human-readable prose and their WORDING has changed between
+  versions ("C2PA Content Credentials (OpenAI)" vs "... (OpenAI, Truepic)"). Diffing raw
+  strings would report every rewording as a lost+new signal pair and bury the real
+  regressions. Signals are normalized to families (c2pa, synthid, visible_sparkle, ...)
+  so the comparison tracks BEHAVIOR, not phrasing.
+
+DATA SAFETY
+  Corpus images are user uploads: read-only, local analysis. Output goes to a gitignored
+  path under data/spaces/ and records uids, never image content.
+
+    uv run python scripts/sidecar_regression.py --sample 500     # representative trial
+    uv run python scripts/sidecar_regression.py                  # full corpus, resumable
+
+Use `--sample` for a trial, never `--limit`: the sidecar list is sorted by day, so
+`--limit N` reads only the EARLIEST day and its result does not generalize (the first
+trial run of this script took 200 files that were all 2026-05-29).
+"""
+
+from __future__ import annotations
+
+import argparse
+import collections
+import glob
+import json
+import os
+import random
+import sys
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+REPO = Path(__file__).resolve().parents[1]
+IDENTIFY_DIR = REPO / "data" / "spaces" / "identify"
+ORIGINALS = REPO / "data" / "spaces" / "originals"
+OUT = REPO / "data" / "spaces" / "_sidecar_regression.jsonl"
+
+# Map a watermark description to a stable behavior family. Order matters: the first
+# matching pattern wins, so put the specific tokens above the generic ones.
+_FAMILIES: tuple[tuple[str, tuple[str, ...]], ...] = (
+    ("synthid", ("synthid",)),
+    ("visible_sparkle", ("sparkle",)),
+    ("visible_doubao", ("豆包", "doubao")),
+    ("visible_jimeng", ("即梦", "jimeng")),
+    ("visible_samsung", ("galaxy ai", "generati dall")),
+    ("aigc_tc260", ("aigc", "tc260")),
+    ("iptc", ("iptc", "digitalsourcetype", "made with ai", "made-with-ai")),
+    ("c2pa", ("c2pa", "content credentials")),
+    ("trustmark", ("trustmark",)),
+    ("open_invisible", ("open invisible", "dwt", "stable diffusion xl")),
+    ("xai_signature", ("xai", "grok signature")),
+    ("exif_generator", ("exif", "software tag", "png text")),
+)
+
+
+def family_of(description: str) -> str:
+    low = description.lower()
+    for name, tokens in _FAMILIES:
+        if any(t in low for t in tokens):
+            return name
+    return "other"
+
+
+def families(descriptions: list[str]) -> set[str]:
+    return {family_of(d) for d in descriptions or []}
+
+
+def compare(sidecar: dict, report: object) -> dict:
+    """Classify the difference between a recorded verdict and a fresh one."""
+    old_ai, new_ai = sidecar.get("is_ai_generated"), getattr(report, "is_ai_generated", None)
+    old_fam = families(sidecar.get("watermarks") or [])
+    new_fam = families(list(getattr(report, "watermarks", []) or []))
+    old_plat, new_plat = sidecar.get("platform"), getattr(report, "platform", None)
+    old_conf, new_conf = sidecar.get("confidence"), getattr(report, "confidence", None)
+
+    classes: list[str] = []
+    # Treat only a real True->not-True transition as lost. None means "unknown", and the
+    # library never asserts False, so None->None is not a change.
+    if bool(old_ai) and not bool(new_ai):
+        classes.append("lost_ai")
+    if not bool(old_ai) and bool(new_ai):
+        classes.append("new_ai")
+    if old_fam - new_fam:
+        classes.append("lost_signal")
+    if new_fam - old_fam:
+        classes.append("new_signal")
+    if old_plat != new_plat:
+        classes.append("platform")
+    if old_conf != new_conf:
+        classes.append("confidence")
+
+    return {
+        "classes": classes or ["unchanged"],
+        "old_ai": old_ai,
+        "new_ai": new_ai,
+        "lost_families": sorted(old_fam - new_fam),
+        "new_families": sorted(new_fam - old_fam),
+        "old_platform": old_plat,
+        "new_platform": new_plat,
+        "old_confidence": old_conf,
+        "new_confidence": new_conf,
+    }
+
+
+def _one(sidecar_path: str) -> dict:
+    uid = os.path.basename(sidecar_path)[:-5]
+    day = os.path.basename(os.path.dirname(sidecar_path))
+    try:
+        with open(sidecar_path, encoding="utf-8") as fh:
+            sidecar = json.load(fh)
+    except Exception as e:  # a corrupt sidecar must not kill the sweep
+        return {"uid": uid, "day": day, "status": "sidecar_unreadable", "error": str(e)[:200]}
+
+    src = sidecar.get("src") or ""
+    image = ORIGINALS / day / src
+    if not src or not image.exists():
+        found = glob.glob(str(ORIGINALS / day / f"{uid}*"))
+        if not found:
+            return {"uid": uid, "day": day, "status": "image_missing"}
+        image = Path(found[0])
+
+    try:
+        from remove_ai_watermarks.identify import identify
+
+        report = identify(image)
+    except Exception as e:  # record and continue; a crash IS a finding
+        return {"uid": uid, "day": day, "status": "identify_raised", "error": f"{type(e).__name__}: {e}"[:300]}
+
+    return {"uid": uid, "day": day, "status": "ok", **compare(sidecar, report)}
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--limit", type=int, default=0, help="only N sidecars (trial run)")
+    ap.add_argument(
+        "--sample",
+        type=int,
+        default=0,
+        help="randomly sample N across ALL days -- use this for a trial, not --limit: "
+        "the sidecar list is sorted, so --limit takes one day and is not representative",
+    )
+    ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2))
+    ap.add_argument("--out", type=Path, default=OUT)
+    ap.add_argument("--restart", action="store_true", help="ignore existing output and start over")
+    a = ap.parse_args()
+
+    sidecars = sorted(glob.glob(str(IDENTIFY_DIR / "*" / "*.json")))
+    if a.sample:
+        random.Random(7).shuffle(sidecars)  # noqa: S311 -- deterministic sampling, not crypto
+        sidecars = sorted(sidecars[: a.sample])
+    elif a.limit:
+        sidecars = sidecars[: a.limit]
+    if not sidecars:
+        raise SystemExit(f"no sidecars under {IDENTIFY_DIR}")
+
+    # Resume: a full sweep is ~1 h, so never redo work an interrupted run already did.
+    done: set[str] = set()
+    if a.restart and a.out.exists():
+        a.out.unlink()  # --restart must TRUNCATE; the file is reopened in append mode below
+    if a.out.exists() and not a.restart:
+        with open(a.out, encoding="utf-8") as fh:
+            for line in fh:
+                try:
+                    done.add(json.loads(line)["uid"])
+                except Exception:  # noqa: S112 -- tolerate a torn last line from an interrupted run
+                    continue
+    todo = [s for s in sidecars if os.path.basename(s)[:-5] not in done]
+    print(f"sidecars {len(sidecars)}  already done {len(done)}  to do {len(todo)}  workers {a.workers}")
+
+    counts: collections.Counter[str] = collections.Counter()
+    a.out.parent.mkdir(parents=True, exist_ok=True)
+    with open(a.out, "a", encoding="utf-8") as fh, ProcessPoolExecutor(max_workers=a.workers) as ex:
+        futures = {ex.submit(_one, s): s for s in todo}
+        for i, fut in enumerate(as_completed(futures), 1):
+            rec = fut.result()
+            fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
+            if rec["status"] != "ok":
+                counts[rec["status"]] += 1
+            else:
+                for c in rec["classes"]:
+                    counts[c] += 1
+            if i % 500 == 0:
+                fh.flush()
+                print(f"  {i}/{len(todo)}  " + "  ".join(f"{k}={v}" for k, v in counts.most_common(6)), flush=True)
+
+    print(f"\n{'=' * 70}\nSIDECAR REGRESSION  processed={len(todo)}\n{'=' * 70}")
+    for k, v in counts.most_common():
+        flag = "  <-- REGRESSION" if k.startswith("lost_") or k in ("identify_raised",) else ""
+        print(f"  {k:22s} {v:7d}{flag}")
+    print(f"\nfull records: {a.out}")
+
+
+if __name__ == "__main__":
+    main()

scripts/smoke_matrix.py

@@ -0,0 +1,621 @@
+"""Release smoke matrix: exercise every CLI parameter CHOICE on REAL data.
+
+Run before a release. This is not a unit-test substitute -- it drives the real CLI as a
+subprocess, so it covers argument parsing, exit-code semantics and the file-writing
+contracts that unit tests with fakes cannot.
+
+WHAT IT COVERS AND WHY THAT SHAPE
+  * Every choice-valued flag at least once (`--backend`, `--sensitivity`, `--mark`,
+    `--mode`, `--inpaint-method`, ...). Full permutation is combinatorially large and
+    mostly meaningless; dead branches and typos hide in the CHOICES, not in their
+    products.
+  * Every input FORMAT and shape edge case through the pixel paths -- PNG/JPEG/WebP/
+    HEIC/AVIF, alpha, unicode names, misnamed extensions, truncated files, tiny and
+    landscape frames. Real-world breakage lives here far more than in flag combos.
+  * CONTRACTS, not just exit codes: `visible` must write NO output and exit 2 when no
+    mark is found (re-serving the input reads as success -- the recurring "it didn't
+    work" report); a no-op must be byte-identical; `metadata --remove` must actually
+    strip; a JPEG strip must not touch the pixels. Note the pixel-lossless contract is
+    the DEFAULT path's -- `--remove-all` deliberately re-encodes (see metadata.py).
+  * The diffusion bodies under `--diffusion`, at `--max-resolution 512` so they fit MPS
+    (~1 min/image on 32 GB unified memory). Not just exit codes: `invisible` must
+    restore the input resolution and must NOT re-stamp SDXL's own open watermark.
+
+WHAT IT DOES NOT COVER, DELIBERATELY AND LOUDLY
+  * Without `--diffusion`, the model-running bodies are reported as SKIPPED with a
+    reason, never as passes -- a green run that quietly skipped half the surface is
+    worse than a red one.
+  * Removal STRENGTH is never certified here. Whether a watermark is actually gone
+    needs the per-vendor oracles (docs/known-limitations.md); these rows prove the
+    paths run and keep their contracts, nothing more.
+  * The re-embed row is gated on a POSITIVE CONTROL. imwatermark is positive-only and
+    fails to round-trip on some pristine carriers, so "no watermark found" proves
+    nothing there; the row degrades to a skip rather than a false pass.
+
+    uv run python scripts/smoke_matrix.py                 # corpus + fixtures
+    uv run python scripts/smoke_matrix.py --quick         # fixtures only, no corpus
+    uv run python scripts/smoke_matrix.py --diffusion     # + the SDXL model paths
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import random
+import shutil
+import subprocess
+import tempfile
+from dataclasses import dataclass, field
+from pathlib import Path
+
+REPO = Path(__file__).resolve().parents[1]
+SAMPLES = REPO / "data" / "samples"
+CORPUS = REPO / "data" / "spaces" / "originals"
+EXIT_NO_VISIBLE_MARK = 2
+
+
+def _capture(args: list[str]) -> str:
+    """Run the CLI and return stdout (for the rows that inspect output, not exit code)."""
+    exe = shutil.which("uv") or "uv"
+    p = subprocess.run(  # noqa: S603
+        [exe, "run", "remove-ai-watermarks", *args], capture_output=True, text=True, cwd=REPO, check=False
+    )
+    return p.stdout
+
+
+@dataclass
+class Result:
+    name: str
+    status: str  # pass | FAIL | skip
+    detail: str = ""
+    cmd: str = ""
+
+
+@dataclass
+class Runner:
+    tmp: Path
+    results: list[Result] = field(default_factory=list)
+
+    def run(self, name: str, args: list[str], *, expect_exit: int | None = 0, timeout: int = 180) -> Result:
+        exe = shutil.which("uv") or "uv"
+        cmd = [exe, "run", "remove-ai-watermarks", *args]
+        try:
+            p = subprocess.run(  # noqa: S603
+                cmd, capture_output=True, text=True, timeout=timeout, cwd=REPO, check=False
+            )
+        except subprocess.TimeoutExpired:
+            r = Result(name, "FAIL", f"timeout after {timeout}s", " ".join(args))
+            self.results.append(r)
+            return r
+        ok = expect_exit is None or p.returncode == expect_exit
+        detail = "" if ok else f"exit {p.returncode} (want {expect_exit}): {(p.stderr or p.stdout).strip()[-200:]}"
+        r = Result(name, "pass" if ok else "FAIL", detail, " ".join(args))
+        self.results.append(r)
+        return r
+
+    def check(self, name: str, ok: bool, detail: str = "") -> None:
+        self.results.append(Result(name, "pass" if ok else "FAIL", "" if ok else detail))
+
+    def skip(self, name: str, why: str) -> None:
+        self.results.append(Result(name, "skip", why))
+
+
+def corpus_pick(n: int, suffixes: tuple[str, ...]) -> list[Path]:
+    """Real uploads, chosen deterministically so a failure is reproducible."""
+    if not CORPUS.exists():
+        return []
+    pool = [p for p in CORPUS.glob("*/*") if p.suffix.lower() in suffixes]
+    random.Random(7).shuffle(pool)  # noqa: S311 -- deterministic sampling, not cryptography
+    return pool[:n]
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--quick", action="store_true", help="fixtures only; skip the corpus rows")
+    ap.add_argument(
+        "--diffusion", action="store_true", help="also run the model-running paths (SDXL weights, ~1 min/image)"
+    )
+    a = ap.parse_args()
+
+    tmp = Path(tempfile.mkdtemp(prefix="raiw-smoke-"))
+    r = Runner(tmp)
+    doubao = SAMPLES / "doubao-1.png"
+    chatgpt = SAMPLES / "chatgpt-1.png"
+    grok = SAMPLES / "grok-1.jpg"
+
+    # ---- identify: every flag, and JSON must actually parse -------------------
+    r.run("identify plain", ["identify", str(doubao)])
+    r.run("identify --no-visible", ["identify", str(doubao), "--no-visible"])
+    try:
+        json.loads(_capture(["identify", str(doubao), "--json"]))
+        r.check("identify --json parses", True)
+    except Exception as e:
+        r.check("identify --json parses", False, str(e))
+
+    # ---- metadata: check/remove, and the strip must be REAL ------------------
+    r.run("metadata --check", ["metadata", str(doubao), "--check"])
+    out = tmp / "meta.png"
+    r.run("metadata --remove", ["metadata", str(doubao), "--remove", "-o", str(out)])
+    if out.exists():
+        try:
+            rep = json.loads(_capture(["identify", str(out), "--json"]))
+            # Scope the assert to METADATA. doubao-1 also carries a visible pixel mark,
+            # which `metadata --remove` must NOT touch -- demanding an empty watermark
+            # list would fail on correct behavior (and hide a real metadata leak behind
+            # a permanently-red row).
+            leftover = [s for s in rep.get("signals", []) if not s.get("name", "").startswith("visible_")]
+            r.check(
+                "metadata --remove strips every metadata signal",
+                not leftover and not rep.get("ai_from_metadata"),
+                f"still reports metadata signals {[s.get('name') for s in leftover]}",
+            )
+            r.check(
+                "metadata --remove leaves the visible mark alone",
+                any(s.get("name", "").startswith("visible_") for s in rep.get("signals", [])),
+                "the pixel mark vanished -- a metadata strip must not touch pixels",
+            )
+        except Exception as e:
+            r.check("metadata --remove strips every metadata signal", False, str(e))
+
+    # The pixel-lossless contract belongs to the DEFAULT path. `--remove-all`
+    # (keep_standard=False) deliberately falls through to the full PIL re-encode
+    # (metadata.py: the lossless marker walk preserves standard segments, so it cannot
+    # serve a strip-everything caller), which is lossy for JPEG -- measured ~49 dB.
+    # Asserting losslessness there tested a contract the code never made.
+    import numpy as np
+
+    from remove_ai_watermarks.image_io import imread
+
+    out2 = tmp / "meta2.jpg"
+    r.run("metadata --remove (jpeg, default)", ["metadata", str(grok), "--remove", "-o", str(out2)])
+    if out2.exists():
+        a_, b_ = imread(str(grok)), imread(str(out2))
+        same = a_ is not None and b_ is not None and a_.shape == b_.shape and bool(np.array_equal(a_, b_))
+        r.check("jpeg metadata strip is pixel-lossless", same, "pixels changed -- the strip re-encoded")
+    out3 = tmp / "meta3.jpg"
+    r.run("metadata --remove --remove-all (jpeg)", ["metadata", str(grok), "--remove", "--remove-all", "-o", str(out3)])
+    if out3.exists():
+        rep3 = json.loads(_capture(["identify", str(out3), "--json"]))
+        r.check(
+            "--remove-all strips the AI metadata too",
+            not [s for s in rep3.get("signals", []) if not s.get("name", "").startswith("visible_")],
+            f"still reports {rep3.get('signals')}",
+        )
+
+    # ---- visible: every --mark choice, every --backend, every --sensitivity ---
+    for mark in ("auto", "gemini", "doubao", "jimeng", "samsung", "jimeng_pill"):
+        # doubao-1 carries only the doubao mark, so every other --mark must exit 2
+        want = 0 if mark in ("auto", "doubao") else EXIT_NO_VISIBLE_MARK
+        r.run(
+            f"visible --mark {mark}",
+            ["visible", str(doubao), "--mark", mark, "-o", str(tmp / f"m_{mark}.png")],
+            expect_exit=want,
+        )
+    # Every backend CHOICE, not just the two that need no extra. migan/lama were
+    # measured at library level (scripts/fill_quality.py) but had never been driven
+    # through the CLI, which is a different code path (resolve_backend + the warning).
+    from remove_ai_watermarks.region_eraser import lama_available, migan_available
+
+    for backend in ("auto", "cv2", "migan", "lama"):
+        have = {"migan": migan_available(), "lama": lama_available()}.get(backend, True)
+        if not have:
+            r.skip(f"visible --backend {backend}", f"the `{backend}` extra is not installed")
+            continue
+        r.run(
+            f"visible --backend {backend}",
+            ["visible", str(doubao), "--backend", backend, "-o", str(tmp / f"b_{backend}.png")],
+            timeout=600,
+        )
+    for sens in ("auto", "strict"):
+        r.run(
+            f"visible --sensitivity {sens}",
+            ["visible", str(doubao), "--sensitivity", sens, "-o", str(tmp / f"s_{sens}.png")],
+        )
+    r.run("visible --keep-metadata", ["visible", str(doubao), "--keep-metadata", "-o", str(tmp / "keep.png")])
+    r.run("visible --no-detect (forced)", ["visible", str(doubao), "--no-detect", "-o", str(tmp / "force.png")])
+    for removed in ("assume-ai", "aggressive"):
+        r.run(
+            f"visible rejects --sensitivity {removed}",
+            ["visible", str(doubao), "--sensitivity", removed],
+            expect_exit=2,
+        )
+
+    # CONTRACT: no mark -> no output file, exit 2 (never re-serve the input as success)
+    noout = tmp / "must_not_exist.png"
+    r.run("visible no-mark exits 2", ["visible", str(chatgpt), "-o", str(noout)], expect_exit=EXIT_NO_VISIBLE_MARK)
+    r.check("visible no-mark writes NO output", not noout.exists(), "wrote an output for an undetected mark")
+
+    # ---- erase: backends, methods, repeated regions, dilate ------------------
+    for method in ("telea", "ns"):
+        r.run(
+            f"erase --inpaint-method {method}",
+            [
+                "erase",
+                str(doubao),
+                "--region",
+                "10,10,60,30",
+                "--inpaint-method",
+                method,
+                "-o",
+                str(tmp / f"e_{method}.png"),
+            ],
+        )
+    r.run(
+        "erase repeated --region",
+        ["erase", str(doubao), "--region", "10,10,40,20", "--region", "80,80,40,20", "-o", str(tmp / "e_multi.png")],
+    )
+    r.run(
+        "erase --dilate",
+        ["erase", str(doubao), "--region", "10,10,40,20", "--dilate", "5", "-o", str(tmp / "e_dil.png")],
+    )
+    for backend in ("cv2", "migan", "lama"):
+        have = {"migan": migan_available(), "lama": lama_available()}.get(backend, True)
+        if not have:
+            r.skip(f"erase --backend {backend}", f"the `{backend}` extra is not installed")
+            continue
+        r.run(
+            f"erase --backend {backend}",
+            [
+                "erase",
+                str(doubao),
+                "--region",
+                "10,10,40,20",
+                "--backend",
+                backend,
+                "-o",
+                str(tmp / f"e_{backend}.png"),
+            ],
+            timeout=600,
+        )
+    r.run(
+        "erase --dilate 0 (no dilation)",
+        ["erase", str(doubao), "--region", "10,10,40,20", "--dilate", "0", "-o", str(tmp / "e_d0.png")],
+    )
+    r.run("erase rejects a malformed --region", ["erase", str(doubao), "--region", "not,a,box"], expect_exit=2)
+
+    # ---- global + explicit-default flags that had never been exercised -------
+    r.run("--verbose", ["--verbose", "identify", str(doubao)])
+    r.run("--version", ["--version"])
+    r.run("metadata --keep-standard (explicit)", ["metadata", str(doubao), "--check", "--keep-standard"])
+    r.run(
+        "visible --strip-metadata (explicit)",
+        ["visible", str(doubao), "--strip-metadata", "-o", str(tmp / "sm.png")],
+    )
+    r.run("visible --detect (explicit default)", ["visible", str(doubao), "--detect", "-o", str(tmp / "det.png")])
+
+    # ---- batch: the non-diffusion modes -------------------------------------
+    bd = tmp / "batch_in"
+    bd.mkdir()
+    for f in (doubao, chatgpt):
+        shutil.copy(f, bd / f.name)
+    for mode in ("visible", "metadata"):
+        r.run(
+            f"batch --mode {mode}",
+            ["batch", str(bd), "--mode", mode, "-o", str(tmp / f"batch_{mode}")],
+            expect_exit=None,
+        )
+
+    # ---- diffusion: argument handling always; the model body under --diffusion ----
+    r.run("invisible --help parses full knob set", ["invisible", "--help"])
+    r.run("all --help parses full knob set", ["all", "--help"])
+    _media_rows(r, tmp)
+    if a.diffusion:
+        _diffusion_rows(r, tmp, doubao)
+    else:
+        for name in ("invisible", "all", "batch --mode invisible"):
+            r.skip(f"{name} (model-running body)", "pass --diffusion to exercise it (needs the SDXL weights)")
+
+    # ---- real-data formats and shapes ---------------------------------------
+    if not a.quick:
+        picks: list[tuple[str, Path]] = []
+        for suf, label in ((".heic", "heic"), (".avif", "avif"), (".webp", "webp"), (".jpeg", "jpeg")):
+            picks += [(label, p) for p in corpus_pick(2, (suf,))]
+        picks += [("png", p) for p in corpus_pick(3, (".png",))]
+        if not picks:
+            r.skip("real-format rows", "corpus not present (data/spaces/originals)")
+        for label, p in picks:
+            r.run(f"identify real {label}", ["identify", str(p), "--json"])
+            r.run(
+                f"visible auto real {label}",
+                ["visible", str(p), "-o", str(tmp / f"r_{label}_{p.stem[:8]}.png")],
+                expect_exit=None,
+            )  # 0 or 2 are both correct; a CRASH is not
+
+        # unicode + misnamed extension + truncated: the documented real-world traps
+        if picks:
+            src = picks[0][1]
+            uni = tmp / "тест изображение 测试.png"
+            shutil.copy(src, uni)
+            r.run("unicode filename", ["identify", str(uni), "--json"])
+            mis = tmp / "actually_png.jpg"  # content PNG, extension JPEG
+            shutil.copy(SAMPLES / "chatgpt-1.png", mis)
+            r.run("misnamed extension", ["identify", str(mis), "--json"])
+            trunc = tmp / "truncated.png"
+            trunc.write_bytes((SAMPLES / "chatgpt-1.png").read_bytes()[:4096])
+            r.run("truncated file does not crash", ["identify", str(trunc), "--json"], expect_exit=None)
+
+    # ---- report --------------------------------------------------------------
+    bad = [x for x in r.results if x.status == "FAIL"]
+    skipped = [x for x in r.results if x.status == "skip"]
+    ok = [x for x in r.results if x.status == "pass"]
+    print(f"\n{'=' * 74}\nSMOKE MATRIX  pass={len(ok)}  FAIL={len(bad)}  skipped={len(skipped)}\n{'=' * 74}")
+    for x in skipped:
+        print(f"  SKIP  {x.name:46s} {x.detail}")
+    for x in bad:
+        print(f"  FAIL  {x.name:46s} {x.detail}")
+        if x.cmd:
+            print(f"        cmd: {x.cmd}")
+    if not bad:
+        print("  no failures")
+    print(f"\ntmp artifacts: {tmp}")
+    raise SystemExit(1 if bad else 0)
+
+
+def _knob_rows(r: Runner, tmp: Path, img: Path) -> None:
+    """Every diffusion knob the matrix never touched.
+
+    Deliberately cheap (`--steps 4`, `--max-resolution 384`): these rows answer "is the
+    knob accepted and does the run complete", NOT "is the output good". Quality per knob
+    needs a per-knob oracle and most of them have none (`--humanize` has no oracle at
+    all), so claiming more here would be dishonest.
+    """
+    from remove_ai_watermarks import upscaler
+
+    # --steps 20 is the floor that WORKS, not an arbitrary choice: effective timesteps
+    # are int(steps * strength), so at the default strength 0.15 anything below
+    # --steps 7 rounds to ZERO and the pipeline dies inside torch. The first version of
+    # these rows used --steps 4 and every single one failed with the same reshape error.
+    fast = ["--max-resolution", "384", "--min-resolution", "0", "--steps", "20", "--force", "--seed", "0"]
+
+    def run(name: str, extra: list[str], *, tag: str, expect: int | None = 0) -> None:
+        r.run(
+            name,
+            ["invisible", str(img), "-o", str(tmp / f"k_{tag}.png"), *fast, *extra],
+            expect_exit=expect,
+            timeout=2400,
+        )
+
+    run("--pipeline sdxl", ["--pipeline", "sdxl"], tag="sdxl")
+    run("--pipeline controlnet", ["--pipeline", "controlnet"], tag="cnet")
+    run("--strength", ["--strength", "0.2"], tag="strength")
+    run("--guidance-scale", ["--guidance-scale", "5.0"], tag="gs")
+    run("--controlnet-scale", ["--controlnet-scale", "0.5"], tag="cns")
+    run("--humanize", ["--humanize", "0.3"], tag="hum")
+    run("--unsharp", ["--unsharp", "0.5"], tag="uns")
+    run("--no-adaptive-polish", ["--no-adaptive-polish"], tag="nap")
+    run("--tile", ["--tile", "--tile-size", "256", "--tile-overlap", "64"], tag="tile")
+    run("--device mps", ["--device", "mps"], tag="mps")
+    run("--upscaler lanczos", ["--upscaler", "lanczos"], tag="lanczos")
+    run("--auto (deprecated no-op)", ["--auto"], tag="auto")
+
+    if upscaler.is_available():
+        run("--upscaler esrgan", ["--upscaler", "esrgan"], tag="esrgan")
+    else:
+        r.skip("--upscaler esrgan", "the `esrgan` extra is not installed")
+
+    # CPU is correctness-relevant (it is the documented MPS-OOM fallback) but slow, so
+    # it gets the smallest possible run rather than being skipped.
+    r.run(
+        "--device cpu",
+        [
+            "invisible",
+            str(img),
+            "-o",
+            str(tmp / "k_cpu.png"),
+            "--device",
+            "cpu",
+            "--max-resolution",
+            "256",
+            "--min-resolution",
+            "0",
+            "--steps",
+            "20",
+            "--force",
+            "--seed",
+            "0",
+        ],
+        timeout=3600,
+    )
+
+    # qwen is CUDA-class by design (bf16 MMDiT, no MPS fallback). On this host the
+    # honest outcome is a CLEAN failure, not a crash -- assert it does not hang or
+    # dump a traceback at the user.
+    try:
+        import torch
+
+        cuda = bool(torch.cuda.is_available())
+    except Exception:
+        cuda = False
+    if cuda:
+        run("--pipeline qwen", ["--pipeline", "qwen"], tag="qwen")
+    else:
+        r.skip("--pipeline qwen", "CUDA-class pipeline; no CUDA device on this host")
+
+    # --model and --hf-token are deliberately not exercised: one would download a second
+    # multi-GB checkpoint, the other needs a real credential. Skipped loudly, not passed.
+    r.skip("--model", "would download a second multi-GB checkpoint")
+    r.skip("--hf-token", "needs a real credential; cannot be exercised meaningfully here")
+
+    # CONTRACT, not just execution: the same seed must reproduce the same pixels.
+    a_out, b_out = tmp / "seed_a.png", tmp / "seed_b.png"
+    for out in (a_out, b_out):
+        r.run(
+            f"seed determinism run ({out.name})",
+            ["invisible", str(img), "-o", str(out), *fast],
+            timeout=2400,
+        )
+    if a_out.exists() and b_out.exists():
+        import numpy as np
+
+        from remove_ai_watermarks.image_io import imread
+
+        x, y = imread(str(a_out)), imread(str(b_out))
+        r.check(
+            "same --seed reproduces identical pixels",
+            x is not None and y is not None and x.shape == y.shape and bool(np.array_equal(x, y)),
+            "two runs with the same seed differed",
+        )
+
+
+def _media_rows(r: Runner, tmp: Path) -> None:
+    """Audio/video metadata strip via ffmpeg -- a supported path with no corpus coverage.
+
+    The corpus is images only, so the media is synthesized here with ffmpeg rather than
+    left untested.
+    """
+    if not shutil.which("ffmpeg"):
+        r.skip("audio/video metadata strip", "ffmpeg not on PATH")
+        return
+    for name, gen in (
+        ("mp4", ["-f", "lavfi", "-i", "testsrc=duration=1:size=128x128:rate=8", "-pix_fmt", "yuv420p"]),
+        ("mp3", ["-f", "lavfi", "-i", "sine=frequency=440:duration=1"]),
+    ):
+        src = tmp / f"media.{name}"
+        ff = shutil.which("ffmpeg") or "ffmpeg"
+        made = subprocess.run(  # noqa: S603
+            [ff, "-y", *gen, "-metadata", "comment=Made with AI", str(src)],
+            capture_output=True,
+            check=False,
+        )
+        if made.returncode != 0 or not src.exists():
+            r.skip(f"{name} metadata strip", "ffmpeg could not synthesize the fixture")
+            continue
+        out = tmp / f"media_clean.{name}"
+        r.run(f"{name} metadata strip runs", ["metadata", str(src), "--remove", "-o", str(out)], expect_exit=None)
+        if out.exists():
+            r.check(f"{name} strip produced a non-empty file", out.stat().st_size > 0, "empty output")
+
+
+def _sdxl_watermark_bits(img: object) -> float:
+    """Bits of the open SDXL DWT-DCT watermark recovered from `img` (128 = perfect)."""
+    import numpy as np
+    from imwatermark import WatermarkDecoder
+
+    truth = np.frombuffer(b"StableDiffusionV1"[:16], dtype=np.uint8)
+    rec = WatermarkDecoder("bytes", 128).decode(img, "dwtDct")
+    return float(128 - np.unpackbits(truth ^ np.frombuffer(bytes(rec), dtype=np.uint8)).sum())
+
+
+def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
+    """Exercise the model-running bodies at a reduced resolution (MPS-friendly).
+
+    Bounded with `--max-resolution 512` and a fixed seed: the point is that the paths
+    RUN and keep their contracts, not to certify removal strength (that needs the
+    per-vendor oracles, see docs/known-limitations.md).
+    """
+    import numpy as np
+
+    from remove_ai_watermarks.image_io import imread
+
+    small = ["--max-resolution", "512", "--seed", "0"]
+
+    # `invisible` must restore the original resolution and NOT re-stamp SDXL's own
+    # open watermark (add_watermarker=False; a remover that re-marks its output is the
+    # regression this row exists for).
+    #
+    # The carrier matters: imwatermark is positive-only and fails to round-trip on some
+    # pristine images, so an "absent" verdict on a fragile carrier proves NOTHING. mj-1
+    # is used because it round-trips at 128/128; the control is re-checked on the actual
+    # OUTPUT and the row degrades to a skip rather than a false pass if it goes fragile.
+    mj = SAMPLES / "mj-1.png"
+    inv = tmp / "inv_mj.png"
+    res = r.run("invisible runs (mps, 512px)", ["invisible", str(mj), "-o", str(inv), "--force", *small], timeout=1800)
+    if res.status == "pass" and inv.exists():
+        src_img, out_img = imread(str(mj)), imread(str(inv))
+        r.check(
+            "invisible restores the input resolution",
+            src_img is not None and out_img is not None and src_img.shape == out_img.shape,
+            f"{None if src_img is None else src_img.shape} -> {None if out_img is None else out_img.shape}",
+        )
+        try:
+            from imwatermark import WatermarkEncoder
+
+            enc = WatermarkEncoder()
+            enc.set_watermark("bytes", b"StableDiffusionV1"[:16])
+            control = _sdxl_watermark_bits(enc.encode(np.array(out_img).copy(), "dwtDct"))
+            if control < 118:
+                r.skip("invisible does not re-embed an SDXL watermark", f"carrier fragile (control {control:.0f}/128)")
+            else:
+                bits = _sdxl_watermark_bits(out_img)
+                r.check(
+                    "invisible does not re-embed an SDXL watermark",
+                    bits < 118,
+                    f"re-embedded: {bits:.0f}/128 recovered (control {control:.0f}/128)",
+                )
+        except ImportError:
+            r.skip("invisible does not re-embed an SDXL watermark", "imwatermark absent (extra `detect`)")
+
+    # `all`: every stage must land -- the visible mark AND the metadata both gone.
+    allout = tmp / "all_out.png"
+    res = r.run("all runs (mps, 512px)", ["all", str(doubao), "-o", str(allout), *small], timeout=1800)
+    if res.status == "pass" and allout.exists():
+        rep = json.loads(_capture(["identify", str(allout), "--json"]))
+        r.check(
+            "all clears visible + metadata in one pass",
+            not rep.get("signals"),
+            f"still reports {[s.get('name') for s in rep.get('signals', [])]}",
+        )
+
+    _knob_rows(r, tmp, mj)
+
+    # `batch --mode invisible`: every input must produce an output (a silent short
+    # write is the failure this row guards).
+    bd = tmp / "batch_inv"
+    bd.mkdir(exist_ok=True)
+    for f in (SAMPLES / "chatgpt-2.png", mj):
+        shutil.copy(f, bd / f.name)
+    bout = tmp / "batch_inv_out"
+    res = r.run(
+        "batch --mode invisible runs (mps, 512px)",
+        ["batch", str(bd), "--mode", "invisible", "-o", str(bout), *small],
+        expect_exit=None,
+        timeout=3600,
+    )
+    if res.status == "pass":
+        produced = len(list(bout.glob("*"))) if bout.exists() else 0
+        r.check("batch invisible writes one output per input", produced == 2, f"{produced} outputs for 2 inputs")
+
+    # `batch --mode all` -- the only --mode value the matrix never ran.
+    aout = tmp / "batch_all_out"
+    r.run(
+        "batch --mode all runs (mps, 512px)",
+        ["batch", str(bd), "--mode", "all", "-o", str(aout), *small],
+        expect_exit=None,
+        timeout=3600,
+    )
+
+    # The AI-enhanced composite path: regenerate ONLY a region and feather it back,
+    # leaving everything outside the box pixel-exact. Library-level -- the CLI has no
+    # flag for it, so it would otherwise never be exercised on real data.
+    try:
+        import numpy as np
+
+        from remove_ai_watermarks.image_io import imread
+        from remove_ai_watermarks.noai.watermark_remover import WatermarkRemover
+
+        src = imread(str(mj))
+        h, w = src.shape[:2]
+        box = (w // 4, h // 4, w // 4, h // 4)
+        rem = WatermarkRemover(pipeline="controlnet")
+        rout = tmp / "region_composite.png"
+        rem.remove_watermark(mj, rout, strength=0.15, num_inference_steps=20, seed=0, region=box)
+        got = imread(str(rout))
+        if got is None or got.shape != src.shape:
+            r.check("region composite keeps the frame outside the box", False, "shape changed or unreadable")
+        else:
+            mask = np.ones(src.shape[:2], dtype=bool)
+            x, y, bw, bh = box
+            # Outside the box PLUS the feather margin must be untouched.
+            pad = 96
+            mask[max(0, y - pad) : y + bh + pad, max(0, x - pad) : x + bw + pad] = False
+            r.check(
+                "region composite keeps the frame outside the box",
+                bool(np.array_equal(src[mask], got[mask])),
+                "pixels changed outside the regenerated region",
+            )
+    except Exception as e:
+        r.skip("region composite (remove_watermark(region=...))", f"{type(e).__name__}: {e}"[:120])
+
+
+if __name__ == "__main__":
+    main()

scripts/synthid_corpus.py

@@ -26,9 +26,8 @@ from datetime import datetime, timezone
 from pathlib import Path
 
 import click
+from _plain_console import Console, Table
 from PIL import Image
-from rich.console import Console
-from rich.table import Table
 
 from remove_ai_watermarks.noai.c2pa import extract_c2pa_info
 

scripts/synthid_pixel_probe.py

@@ -30,8 +30,8 @@ from typing import TYPE_CHECKING
 
 import click
 import numpy as np
+from _plain_console import Console
 from PIL import Image
-from rich.console import Console
 
 if TYPE_CHECKING:
     from numpy.typing import NDArray

scripts/text_detection_benchmark.py

@@ -1,150 +0,0 @@
-"""Multilingual recall benchmark for the text-protection detector.
-
-Measures the core lever of text protection (`text_protector.TextProtector`): if
-the PP-OCRv3 DB detector misses a text region, that text is NOT preserved during
-the SDXL watermark-removal pass and gets deformed (issue #14). This renders short
-text in several scripts at several font sizes on two canvas sizes, runs detection,
-and reports the fraction of each known text bbox the detector covers.
-
-Findings (2026-05-29):
-  - Detection is script-agnostic: DB segments text *regions*, not characters, so
-    Latin / Cyrillic / CJK / Hangul / Arabic / digits score identically. Language
-    was never the lever.
-  - The only lever is resolution. A fixed small detector input downscaled large
-    canvases so far that small text was missed. Detecting at the native long side
-    (capped, see ``text_protector._DET_MAX_LONG_SIDE``) lifts overall hit-rate
-    from 0.91 to 1.00 and the worst cell (~16 px text on a 2048 canvas) from
-    0.06 to 1.00.
-
-This needs the detector model (downloaded on first use) and a font that covers
-all the scripts (macOS "Arial Unicode"; on Linux install a Noto super-font).
-No GPU. Run:
-
-    uv run python scripts/text_detection_benchmark.py
-"""
-
-from __future__ import annotations
-
-import logging
-import sys
-from collections import defaultdict
-from pathlib import Path
-from typing import Any
-
-import cv2
-import numpy as np
-from PIL import Image, ImageDraw, ImageFont
-from rich.console import Console
-
-from remove_ai_watermarks import text_protector as tp
-
-log = logging.getLogger(__name__)
-console = Console()
-
-# A single font covering every tested script isolates "language" from "font".
-_FONT_CANDIDATES = [
-    "/System/Library/Fonts/Supplemental/Arial Unicode.ttf",
-    "/Library/Fonts/Arial Unicode.ttf",
-    "/usr/share/fonts/truetype/noto/NotoSansCJK-Regular.ttc",
-    "/usr/share/fonts/truetype/unifont/unifont.ttf",
-]
-SCRIPTS = {
-    "Latin": "Generated by AI",
-    "Cyrillic": "Сгенерировано ИИ",
-    "CJK": "豆包AI生成内容",
-    "Hangul": "AI로 생성됨",
-    "Arabic": "أنشئ بالذكاء",
-    "Digits": "0123456789",
-}
-FONT_SIZES = [16, 24, 32, 48, 64]
-CANVASES = [1024, 2048]
-PLACEMENTS = [(0.08, 0.15), (0.30, 0.55), (0.10, 0.82)]
-BG_COLORS = [(35, 40, 60), (210, 205, 200)]
-
-
-def _find_font() -> str:
-    for path in _FONT_CANDIDATES:
-        if Path(path).exists():
-            return path
-    raise SystemExit(
-        "No multi-script font found. Install one (macOS ships 'Arial Unicode'; "
-        "on Linux: a Noto CJK/super font) and add its path to _FONT_CANDIDATES."
-    )
-
-
-def _render(
-    font_path: str,
-    canvas: int,
-    text: str,
-    font_size: int,
-    place: tuple[float, float],
-    bg: tuple[int, int, int],
-) -> tuple[Image.Image, tuple[int, int, int, int]]:
-    img = Image.new("RGB", (canvas, canvas), bg)
-    draw = ImageDraw.Draw(img)
-    font = ImageFont.truetype(font_path, font_size)
-    x, y = int(place[0] * canvas), int(place[1] * canvas)
-    fg = (245, 245, 245) if sum(bg) < 360 else (20, 20, 20)
-    draw.text((x, y), text, font=font, fill=fg)
-    return img, draw.textbbox((x, y), text, font=font)
-
-
-def _coverage(boxes: list[Any], bbox: tuple[int, int, int, int], h: int, w: int) -> float | None:
-    gt = np.zeros((h, w), np.uint8)
-    cv2.rectangle(gt, (bbox[0], bbox[1]), (bbox[2], bbox[3]), 1, -1)
-    area = int(gt.sum())
-    if area == 0:
-        return None
-    det = np.zeros((h, w), np.uint8)
-    if boxes:
-        cv2.fillPoly(det, [np.asarray(b, np.int32) for b in boxes], 1)
-    return int((gt & det).sum()) / area
-
-
-def _hitrate(values: list[float], thr: float = 0.5) -> float:
-    return sum(c >= thr for c in values) / len(values) if values else float("nan")
-
-
-def main() -> int:
-    logging.basicConfig(level=logging.WARNING)
-    if not tp.is_available():
-        raise SystemExit("text detector unavailable (need opencv with cv2.dnn.TextDetectionModel_DB)")
-    font_path = _find_font()
-    detector = tp.TextProtector()
-
-    by_script_size: dict[tuple[str, int], list[float]] = defaultdict(list)
-    by_size_canvas: dict[tuple[int, int], list[float]] = defaultdict(list)
-    by_script: dict[str, list[float]] = defaultdict(list)
-
-    for canvas in CANVASES:
-        for script, text in SCRIPTS.items():
-            for font_size in FONT_SIZES:
-                for idx, place in enumerate(PLACEMENTS):
-                    img, bbox = _render(font_path, canvas, text, font_size, place, BG_COLORS[idx % len(BG_COLORS)])
-                    bgr = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
-                    cov = _coverage(detector.detect_text_boxes(bgr), bbox, canvas, canvas)
-                    if cov is None:
-                        continue
-                    by_script_size[(script, font_size)].append(cov)
-                    by_size_canvas[(font_size, canvas)].append(cov)
-                    by_script[script].append(cov)
-
-    console.print("=== hit-rate (coverage>=0.5) by script x font-size ===")
-    console.print("script".ljust(10) + "".join(f"{fs:>7}" for fs in FONT_SIZES))
-    for script in SCRIPTS:
-        console.print(
-            script.ljust(10) + "".join(f"{_hitrate(by_script_size[(script, fs)]):>7.2f}" for fs in FONT_SIZES)
-        )
-
-    console.print("\n=== hit-rate by font-size x canvas (the downscale effect) ===")
-    console.print("size".ljust(8) + "".join(f"{c:>8}" for c in CANVASES))
-    for fs in FONT_SIZES:
-        console.print(str(fs).ljust(8) + "".join(f"{_hitrate(by_size_canvas[(fs, c)]):>8.2f}" for c in CANVASES))
-
-    overall = _hitrate([c for vals in by_script.values() for c in vals])
-    console.print(f"\nOVERALL hit-rate: {overall:.2f}  (detector max long side = {tp._DET_MAX_LONG_SIDE})")
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main())

scripts/vendor_cohort_harvest.py

@@ -0,0 +1,241 @@
+"""Partition China-AIGC carriers into VENDOR COHORTS by their TC260 producer code.
+
+THE PROBLEM THIS SOLVES
+  Coverage of uncovered vendors is the largest remaining detection lever
+  (`docs/verification-plan.md`, "Where detection work should go next"), and it is
+  blocked on EVIDENCE: nothing may be registered off a single frame, and the
+  previous session found exactly one confirmed positive each for `千问` and `百度`.
+  Harvesting more by PIXELS is circular -- a detector is what we are trying to
+  build -- and the generic shared-tail probe is too weak to label with (0.407 on a
+  bold positive against a clean p99 of 0.298; see `cjk_tail_probe.py`).
+
+THE KEY
+  The TC260 label is not anonymous. Its `ContentProducer` field carries the
+  producer's Chinese Unified Social Credit Code (USCC), e.g.
+  ``001191110102MACQD9K64010000`` -> USCC ``91110102MACQD9K640``, which names a
+  specific legal entity. So the metadata partitions carriers into per-ENTITY
+  cohorts without looking at a single pixel. A cohort is a LABEL: once one frame
+  in it is eyeballed, every frame in it is a labelled example of that vendor's
+  mark. That is what turns "one confirmed positive" into "30+ per vendor".
+
+  CLAUDE.md's "the generic TC260 label names no specific vendor" is about the
+  label MARKER (the bare presence of `TC260:AIGC`), which indeed names nobody.
+  The producer FIELD inside the block is a different thing and does name one.
+
+  Caveat kept in view: the code names the SIGNING ENTITY, which is not always the
+  consumer brand (an aggregator or a cloud host signs for several apps, and one
+  vendor can hold several codes). So a cohort is a strong grouping key and a
+  hypothesis about the brand -- the brand itself is settled by reading the crop,
+  which is what `--sheets` is for.
+
+WHAT IT COSTS
+  Metadata only. The expensive pixel pass is NOT re-run: which detectors fired is
+  joined from `_visible_positives.jsonl` (the completed full-corpus artifact), per
+  the standing rule against relaunching finished sweeps to re-check them.
+
+DATA SAFETY
+  Corpus images are real user uploads: read-only, local analysis, gitignored
+  output. Contact sheets stay under `data/spaces/`; nothing here is committed.
+
+    uv run python scripts/vendor_cohort_harvest.py
+    uv run python scripts/vendor_cohort_harvest.py --report-only --sheets 12
+"""
+
+from __future__ import annotations
+
+import argparse
+import glob
+import json
+import os
+import sys
+from collections import Counter
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from pathlib import Path
+from typing import Any
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+sys.path.insert(0, str(Path(__file__).parent))
+
+REPO = Path(__file__).resolve().parents[1]
+CORPUS = REPO / "data" / "spaces" / "originals"
+OUT = REPO / "data" / "spaces" / "_vendor_cohorts.jsonl"
+FIRED = REPO / "data" / "spaces" / "_visible_positives.jsonl"
+SHEET_DIR = REPO / "data" / "spaces" / "_vendor_cohort_sheets"
+
+# A producer code is `001` + `1` + USCC(18) + a 5-digit app/product suffix, so two
+# codes sharing the USCC are the same legal entity registering different products.
+# Slicing is defensive: anything not matching the layout is grouped by its raw value.
+_USCC_START, _USCC_END = 4, 22
+
+
+def uscc_of(code: str) -> str:
+    """The 18-char Unified Social Credit Code embedded in a TC260 producer code."""
+    if len(code) >= _USCC_END and code[:3] == "001":
+        return code[_USCC_START:_USCC_END]
+    return code
+
+
+def _one(path_str: str) -> dict[str, Any] | None:
+    from remove_ai_watermarks.metadata import aigc_label
+
+    try:
+        label = aigc_label(Path(path_str))
+    except Exception:
+        return None
+    if not label:
+        return None
+    producer = str(label.get("ContentProducer") or "")
+    return {
+        "path": path_str,
+        "producer": producer,
+        "uscc": uscc_of(producer),
+        "propagator": str(label.get("ContentPropagator") or ""),
+        "service_provider": str(label.get("ServiceProvider") or ""),
+    }
+
+
+def load_fired() -> dict[str, list[str]]:
+    """path -> detector keys that fired, from the completed full-corpus artifact."""
+    if not FIRED.exists():
+        print(f"WARNING: {FIRED.name} missing; cohorts will show no detector state")
+        return {}
+    out: dict[str, list[str]] = {}
+    for line in FIRED.read_text(encoding="utf-8").splitlines():
+        if not line.strip():
+            continue
+        rec = json.loads(line)
+        out[rec["path"]] = rec.get("keys") or []
+    return out
+
+
+def scan(limit: int, workers: int, out_path: Path) -> list[dict[str, Any]]:
+    pool = sorted(glob.glob(str(CORPUS / "*" / "*")))
+    if limit:
+        pool = pool[:limit]
+    print(f"scanning {len(pool)} corpus files for TC260 labels  workers={workers}", flush=True)
+    rows: list[dict[str, Any]] = []
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    with open(out_path, "w", encoding="utf-8") as fh, ProcessPoolExecutor(max_workers=workers) as ex:
+        futures = [ex.submit(_one, p) for p in pool]
+        for i, fut in enumerate(as_completed(futures), 1):
+            try:
+                rec = fut.result()
+            except Exception:  # noqa: S112 -- one bad file must not kill the scan
+                continue
+            if rec is not None:
+                fh.write(json.dumps(rec) + "\n")
+                rows.append(rec)
+            if i % 5000 == 0:
+                fh.flush()
+                print(f"  {i}/{len(pool)}  carriers={len(rows)}", flush=True)
+    return rows
+
+
+def report(rows: list[dict[str, Any]], fired: dict[str, list[str]], min_size: int) -> None:
+    by_uscc: dict[str, list[dict[str, Any]]] = {}
+    for r in rows:
+        by_uscc.setdefault(r["uscc"], []).append(r)
+
+    print(f"\n{'=' * 92}\nVENDOR COHORTS  ({len(rows)} TC260 carriers, {len(by_uscc)} distinct entities)\n{'=' * 92}")
+    print("\n`fires` = share of the cohort where SOME registered detector fires.")
+    print("A large cohort with a low fire rate is an uncovered vendor -- the harvest target.\n")
+    print(f"{'entity (USCC)':22s} {'n':>6s} {'fires':>7s}  {'detectors seen':38s} {'products':>8s}")
+    print("-" * 92)
+
+    cohorts = sorted(by_uscc.items(), key=lambda kv: -len(kv[1]))
+    for uscc, members in cohorts:
+        if len(members) < min_size:
+            continue
+        keys: Counter[str] = Counter()
+        hit = 0
+        for m in members:
+            ks = fired.get(m["path"], [])
+            if ks:
+                hit += 1
+            keys.update(ks)
+        seen = ", ".join(f"{k}:{c}" for k, c in keys.most_common(4)) or "-- none --"
+        products = len({m["producer"] for m in members})
+        print(f"{uscc:22s} {len(members):6d} {100 * hit / len(members):6.1f}%  {seen:38s} {products:8d}")
+
+    small = sum(1 for _, m in cohorts if len(m) < min_size)
+    if small:
+        print(f"\n({small} cohorts below --min-size {min_size} not shown)")
+
+
+def _bands(img: Any, width: int, band: int) -> list[Any]:
+    """Full-width top and bottom bands, scaled to a readable common width.
+
+    An unregistered vendor's placement is unknown, so cropping the bottom-RIGHT
+    corner (where the marks we already cover happen to sit) would beg the
+    question. Full-width bands catch any horizontal position, and the two bands
+    together cover every corner the standard's implementers actually use.
+    """
+    import cv2
+
+    h = img.shape[0]
+    strip = max(24, int(h * band))
+    out = []
+    for piece in (img[:strip], img[h - strip :]):
+        scale = width / max(1, piece.shape[1])
+        out.append(cv2.resize(piece, (width, max(12, int(piece.shape[0] * scale))), interpolation=cv2.INTER_AREA))
+    return out
+
+
+def sheets(rows: list[dict[str, Any]], fired: dict[str, list[str]], per: int, min_size: int) -> None:
+    """Top/bottom bands per uncovered cohort, so the vendor and mark can be read off."""
+    import cv2
+    import numpy as np
+
+    from remove_ai_watermarks.image_io import imread
+
+    by_uscc: dict[str, list[dict[str, Any]]] = {}
+    for r in rows:
+        by_uscc.setdefault(r["uscc"], []).append(r)
+
+    SHEET_DIR.mkdir(parents=True, exist_ok=True)
+    print(f"\nwriting contact sheets -> {SHEET_DIR}")
+
+    for uscc, members in sorted(by_uscc.items(), key=lambda kv: -len(kv[1])):
+        if len(members) < min_size:
+            continue
+        quiet = [m for m in members if not fired.get(m["path"])]
+        if not quiet:
+            continue
+        width = 900
+        tiles: list[Any] = []
+        for m in quiet[:per]:
+            img = imread(m["path"])
+            if img is None:
+                continue
+            for b in _bands(img, width, 0.10):
+                tiles.append(b)
+                tiles.append(np.full((3, width, 3), 60, np.uint8))
+        if tiles:
+            dest = SHEET_DIR / f"{uscc}_n{len(members)}_quiet{len(quiet)}.png"
+            cv2.imwrite(str(dest), np.vstack(tiles))
+            print(f"  {dest.name}  ({len(tiles) // 4} frames)")
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--limit", type=int, default=0, help="cap files scanned (0 = whole corpus)")
+    ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2))
+    ap.add_argument("--out", type=Path, default=OUT)
+    ap.add_argument("--report-only", action="store_true")
+    ap.add_argument("--min-size", type=int, default=5, help="hide cohorts smaller than this")
+    ap.add_argument("--sheets", type=int, default=0, help="crops per cohort contact sheet")
+    a = ap.parse_args()
+
+    if a.report_only:
+        rows = [json.loads(x) for x in a.out.read_text(encoding="utf-8").splitlines() if x.strip()]
+    else:
+        rows = scan(a.limit, a.workers, a.out)
+
+    fired = load_fired()
+    report(rows, fired, a.min_size)
+    if a.sheets:
+        sheets(rows, fired, a.sheets, a.min_size)
+
+
+if __name__ == "__main__":
+    main()

scripts/vendor_mark_calibrate.py

@@ -0,0 +1,598 @@
+"""Calibrate a candidate text-mark detector for an UNCOVERED vendor, on real positives.
+
+WHERE THE POSITIVES COME FROM
+  `vendor_cohort_harvest.py` partitions China-AIGC carriers into per-entity cohorts by
+  their TC260 producer code, so a cohort is a vendor LABEL that owes nothing to any
+  pixel detector. That is what makes this calibration non-circular: the previous attempt
+  (`render_vendor_silhouettes.py`, 2026-07-18) died at n=1 because the only way it knew
+  to find 千问 frames was to eyeball the misses of a detector that cannot see them.
+
+  A cohort is NOT automatically a set of visible-mark positives: TC260 provenance is
+  metadata, and a vendor may label a frame without stamping it. So the cohort is the
+  CANDIDATE pool, and mark presence is settled by eye -- `--sheets` writes the corner
+  crops sorted by score, which makes that pass cheap and makes the separation (or its
+  absence) visible directly.
+
+NEGATIVES
+  The 432 frames hand-labelled `present: []` in the 2026-07-18 round -- already-adjudicated
+  no-visible-mark images, so the false-fire arm rests on human labels rather than on the
+  absence of a detection.
+
+THE TRAPS, INHERITED FROM THE 2026-07-18 MEASUREMENT
+  Both are encoded below rather than left to the caller:
+    * size the template with `alpha_height_frac`, NOT the silhouette's own aspect ratio
+      (the latter inflated the clean p99 from 0.30 to 0.58 and made comparison meaningless)
+    * keep the ladder at the shipped 3 rungs -- a wide sweep hands clean corners extra
+      chances to match, which flatters the positives and the negatives alike
+
+DATA SAFETY
+  Corpus images are real user uploads: read-only, local, gitignored output. The template
+  is font-rendered synthetic (`render_vendor_silhouettes.py`), never cut from an upload.
+
+    uv run python scripts/vendor_mark_calibrate.py --cohort 91440101MA9Y9T4H7A \\
+        --asset qwen_alpha.png --sheets
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from pathlib import Path
+from typing import Any
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+sys.path.insert(0, str(Path(__file__).parent))
+
+REPO = Path(__file__).resolve().parents[1]
+COHORTS = REPO / "data" / "spaces" / "_vendor_cohorts.jsonl"
+SHEET_DIR = REPO / "data" / "spaces" / "_vendor_calib_sheets"
+OUT = REPO / "data" / "spaces" / "_vendor_calibration.jsonl"
+
+
+def build_config(
+    asset: str,
+    name: str,
+    scale_basis: str = "short",
+    overrides: dict[str, Any] | None = None,
+) -> Any:
+    """A candidate config: doubao's tuned geometry with this vendor's silhouette.
+
+    Transferring doubao's numbers is justified by LAYOUT, not by hope: every one of these
+    marks is the same GB 45438-2025 house style -- a 2-glyph vendor prefix, then the
+    mandated `AI生成` tail, set in a semibold CJK sans in the bottom-right corner. So
+    `豆包AI生成` and `千问AI生成` are the same 6 glyph cells at the same scale, and the
+    width/height fractions carry over. The NCC gate does NOT carry over and is what this
+    script exists to measure. Any tuned value can be overridden with what
+    `--fit-geometry` measured -- inheriting the locate box blindly clipped the big-mode
+    qwen mark, which is exactly the trap this tool exists to avoid.
+    """
+    import dataclasses
+
+    from remove_ai_watermarks._text_mark_engine import TextMarkConfig
+    from remove_ai_watermarks.doubao_engine import _CONFIG
+
+    return dataclasses.replace(
+        TextMarkConfig(**dataclasses.asdict(_CONFIG)),
+        name=name,
+        asset_name=asset,
+        scale_basis=scale_basis,
+        **(overrides or {}),
+    )
+
+
+ScoreArgs = tuple[str, str, str, str, "dict[str, Any]"]
+
+
+def _score(args: ScoreArgs) -> dict[str, Any] | None:
+    path_str, asset, name, basis, overrides = args
+    from remove_ai_watermarks._text_mark_engine import TextMarkEngine
+    from remove_ai_watermarks.image_io import imread
+
+    img = imread(path_str)
+    if img is None or min(img.shape[:2]) < 64:
+        return None
+    eng = TextMarkEngine(build_config(asset, name, basis, overrides))
+    loc = eng.locate(img)
+    try:
+        score, box = eng._tophat_best(img, loc)
+    except Exception:
+        return None
+    return {"path": path_str, "score": round(float(score), 4), "box": box}
+
+
+NEGATIVES = REPO / "data" / "spaces" / "_research_20260718_textmark_relaxation" / "groundtruth.jsonl"
+
+
+def load_sets(cohort: str) -> tuple[list[str], list[str]]:
+    pos = [
+        json.loads(x)["path"]
+        for x in COHORTS.read_text(encoding="utf-8").splitlines()
+        if x.strip() and json.loads(x)["uscc"] == cohort
+    ]
+    # The 2026-07-18 labels are in the vocabulary of the REGISTERED marks only
+    # (gemini/doubao/jimeng/jimeng_pill): `present: []` means "no registered mark", NOT
+    # "no mark at all" -- 146 of the 432 sit in a TC260 cohort, and qwen-cohort frames
+    # visibly carrying 千问AI生成 are labelled `present: []` there (measured 2026-07-21:
+    # they made up the clean arm's whole top tail, clean p99 0.37 -> 0.69). A gate read
+    # off that arm is meaningless, so the clean arm excludes every frame in ANY TC260
+    # cohort -- cohort membership is the cheap proxy for "may carry a CJK AI label".
+    in_any_cohort = {
+        str(Path(json.loads(x)["path"]).resolve())
+        for x in COHORTS.read_text(encoding="utf-8").splitlines()
+        if x.strip()
+    }
+    neg: list[str] = []
+    dropped = 0
+    for line in NEGATIVES.read_text(encoding="utf-8").splitlines():
+        if not line.strip():
+            continue
+        rec = json.loads(line)
+        if rec.get("present"):
+            continue
+        p = str((REPO / rec["path"]).resolve())
+        if p in in_any_cohort:
+            dropped += 1
+            continue
+        neg.append(p)
+    if dropped:
+        print(f"clean arm: dropped {dropped} negatives that sit in a TC260 cohort (contamination guard)")
+    return pos, neg
+
+
+def run(
+    paths: list[str],
+    asset: str,
+    name: str,
+    workers: int,
+    basis: str = "short",
+    overrides: dict[str, Any] | None = None,
+) -> list[dict[str, Any]]:
+    out: list[dict[str, Any]] = []
+    with ProcessPoolExecutor(max_workers=workers) as ex:
+        futs = [ex.submit(_score, (p, asset, name, basis, overrides or {})) for p in paths]
+        for f in as_completed(futs):
+            try:
+                r = f.result()
+            except Exception:  # noqa: S112 -- one bad file must not kill the sweep
+                continue
+            if r:
+                out.append(r)
+    return out
+
+
+def report(pos: list[dict[str, Any]], neg: list[dict[str, Any]], name: str) -> None:
+    import numpy as np
+
+    p = np.array([r["score"] for r in pos])
+    n = np.array([r["score"] for r in neg])
+    print(f"\n{'=' * 78}\n{name}: candidate-cohort vs hand-labelled clean\n{'=' * 78}")
+    print(f"\n{'arm':10s} {'n':>5s} {'p10':>7s} {'p50':>7s} {'p90':>7s} {'p95':>7s} {'p99':>7s} {'max':>7s}")
+    for label, arr in (("cohort", p), ("clean", n)):
+        if not len(arr):
+            continue
+        qs = [np.percentile(arr, q) for q in (10, 50, 90, 95, 99)]
+        print(f"{label:10s} {len(arr):5d} " + " ".join(f"{q:7.3f}" for q in qs) + f" {arr.max():7.3f}")
+
+    if len(p) and len(n):
+        print("\n\nOPERATING POINTS -- gate set on the CLEAN arm")
+        print("`cohort fire` is an UPPER BOUND on recall: the cohort also holds")
+        print("metadata-only frames that carry no visible mark to find.\n")
+        print(f"{'gate':>7s} {'clean fire':>12s} {'cohort fire':>13s} {'cohort n':>10s}")
+        for q in (90, 95, 99, 99.5, 100):
+            t = float(np.percentile(n, q))
+            cf, pf = 100 * float((n >= t).mean()), 100 * float((p >= t).mean())
+            print(f"{t:7.3f} {cf:11.2f}% {pf:12.1f}% {int((p >= t).sum()):10d}")
+
+
+def sheets(pos: list[dict[str, Any]], name: str, per_sheet: int = 24) -> None:
+    """Corner crops sorted by score, so mark presence and the gate are read in one pass."""
+    import cv2
+    import numpy as np
+
+    from remove_ai_watermarks._text_mark_engine import TextMarkEngine
+    from remove_ai_watermarks.image_io import imread
+
+    eng = TextMarkEngine(build_config("doubao_alpha.png", "roi"))
+    SHEET_DIR.mkdir(parents=True, exist_ok=True)
+    ranked = sorted(pos, key=lambda r: -r["score"])
+    width = 660
+    for start in range(0, len(ranked), per_sheet):
+        chunk = ranked[start : start + per_sheet]
+        tiles: list[Any] = []
+        for rank, r in enumerate(chunk, start + 1):
+            img = imread(r["path"])
+            if img is None:
+                continue
+            loc = eng.locate(img)
+            crop = img[loc.y : loc.y + loc.h, loc.x : loc.x + loc.w]
+            if not crop.size:
+                continue
+            tile = cv2.resize(crop, (width, 96), interpolation=cv2.INTER_AREA)
+            cv2.rectangle(tile, (0, 0), (118, 22), (0, 0, 0), -1)
+            cv2.putText(tile, f"#{rank} {r['score']:.3f}", (4, 16), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1)
+            tiles.append(tile)
+            tiles.append(np.full((2, width, 3), 70, np.uint8))
+        if tiles:
+            dest = SHEET_DIR / f"{name}_ranked_{start // per_sheet:02d}.png"
+            cv2.imwrite(str(dest), np.vstack(tiles))
+            print(f"  {dest.name}  (#{start + 1}..#{start + len(chunk)})")
+
+
+# Wide and dense, for the geometry FIT only. An unregistered vendor's glyph size is
+# genuinely unknown, which is the one case a dense ladder earns its cost -- but it also
+# hands clean corners extra chances to match, so it must never set a gate.
+_FIT_SCALES = tuple(round(0.4 * (1.03**i), 4) for i in range(80))  # 0.40 .. ~4.1
+# The ladder the product actually ships (`_text_mark_engine._tophat_best`).
+_SHIPPED_LADDER = (0.8, 1.0, 1.25)
+
+
+def _fit_one(args: tuple[str, str]) -> dict[str, Any] | None:
+    """Best match over the WIDE ladder, reported as a mark width in pixels.
+
+    Also measures the template ASPECT at the winning width: the mark's true height is
+    fitted by sweeping gh at the winning gw and reading the argmax, because
+    `alpha_height_frac` must be measured, not taken from the silhouette's own aspect
+    (that inflated the clean p99 from 0.30 to 0.58 on the 2026-07-18 attempt) and not
+    inherited from doubao.
+    """
+    path_str, asset = args
+    import cv2
+    import numpy as np
+
+    from remove_ai_watermarks._text_mark_engine import TextMarkEngine
+    from remove_ai_watermarks.image_io import imread
+
+    cfg = build_config(asset, "fit", "width")
+    eng = TextMarkEngine(cfg)
+    img = imread(path_str)
+    if img is None:
+        return None
+    loc = eng.locate(img)
+    resp = eng.tophat_response(img, loc)
+    sil = eng._glyph_silhouette()
+    if resp is None or sil is None:
+        return None
+    w = img.shape[1]
+    best, best_gw = 0.0, 0
+    best_tl = (0, 0)
+    for s in _FIT_SCALES:
+        gw = max(cfg.min_gw, int(cfg.alpha_width_frac * w * s))
+        gh = max(4, int(cfg.alpha_height_frac * w * s))
+        if gw >= resp.shape[1] or gh >= resp.shape[0]:
+            continue
+        t = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA)
+        res = cv2.matchTemplate(resp, t, cv2.TM_CCOEFF_NORMED)
+        _, v, _, tl = cv2.minMaxLoc(res)
+        if v > best:
+            best, best_gw, best_tl = v, gw, (int(tl[0]), int(tl[1]))
+    # Aspect fit at the winning width: sweep gh/gw and keep the argmax. Range covers
+    # everything between samsung's 0.12 and jimeng's 0.29 house styles, plus slack.
+    best_aspect = 0.0
+    if best_gw > 0:
+        best_gh_score = -1.0
+        for ratio in np.arange(0.12, 0.42, 0.01):
+            gh = max(4, int(best_gw * float(ratio)))
+            if gh >= resp.shape[0]:
+                continue
+            t = cv2.resize(sil, (best_gw, gh), interpolation=cv2.INTER_AREA)
+            v = float(cv2.matchTemplate(resp, t, cv2.TM_CCOEFF_NORMED).max())
+            if v > best_gh_score:
+                best_gh_score, best_aspect = v, float(ratio)
+    # The ABSOLUTE mark rect, so the LOCATE box fractions can be fitted too: inheriting
+    # doubao's corner box clipped the big-mode qwen mark's first glyph (the qwen mark
+    # sits ~0.025 of the short side off the right edge, doubao's box assumes ~0.004),
+    # which collapsed an exact-size template to 0.26.
+    ax = loc.x + best_tl[0]
+    ay = loc.y + best_tl[1]
+    return {
+        "path": path_str,
+        "best": round(best, 4),
+        "mark_w": best_gw,
+        "aspect": round(best_aspect, 3),
+        "x": ax,
+        "y": ay,
+        "w": w,
+        "h": img.shape[0],
+    }
+
+
+def fit_geometry(paths: list[str], asset: str, workers: int, floor: float = 0.50, paths_name: str = "cohort") -> None:
+    """Which basis and fraction does this vendor's mark actually scale with?
+
+    Only frames matching above ``floor`` are used: below it the winning size is the
+    ladder's best fit to background texture, not a measurement of the mark.
+    """
+    import numpy as np
+
+    rows: list[dict[str, Any]] = []
+    with ProcessPoolExecutor(max_workers=workers) as ex:
+        for f in as_completed([ex.submit(_fit_one, (p, asset)) for p in paths]):
+            try:
+                r = f.result()
+            except Exception:  # noqa: S112 -- one bad file must not kill the fit
+                continue
+            if r:
+                rows.append(r)
+
+    strong = [r for r in rows if r["best"] >= floor]
+    fit_out = REPO / "data" / "spaces" / f"_vendor_fit_{paths_name}.jsonl"
+    fit_out.write_text("\n".join(json.dumps(r) for r in rows), encoding="utf-8")
+    print(f"\n{'=' * 78}\nGEOMETRY FIT  (n={len(rows)}, usable best>={floor}: {len(strong)})\n{'=' * 78}")
+    print(f"rows -> {fit_out}")
+    if len(strong) < 20:
+        print("too few strong frames to fit a basis -- do not ship a fraction off this")
+        return
+
+    mw = np.array([r["mark_w"] for r in strong], float)
+    w = np.array([r["w"] for r in strong], float)
+    h = np.array([r["h"] for r in strong], float)
+    bases = {
+        "width": w,
+        "height": h,
+        "short": np.minimum(w, h),
+        "long": np.maximum(w, h),
+        "sqrt(w*h)": np.sqrt(w * h),
+        "diagonal": np.hypot(w, h),
+    }
+    print(f"\n{'basis':12s} {'mean frac':>10s} {'CV':>8s} {'p10':>8s} {'p90':>8s}")
+    print("-" * 52)
+    for nm, b in sorted(bases.items(), key=lambda kv: float(np.std(mw / kv[1]) / np.mean(mw / kv[1]))):
+        r = mw / b
+        print(
+            f"{nm:12s} {np.mean(r):10.4f} {float(np.std(r) / np.mean(r)):8.3f} "
+            f"{np.percentile(r, 10):8.4f} {np.percentile(r, 90):8.4f}"
+        )
+
+    lo_l, hi_l = _SHIPPED_LADDER[0], _SHIPPED_LADDER[-1]
+    print(f"\nCoverage by a single fraction on the SHIPPED ladder ({lo_l} .. {hi_l}, span {hi_l / lo_l:.3f}x):")
+    print(f"{'basis':12s} {'frac':>7s} {'window':>16s} {'covered':>9s}")
+    for nm in ("short", "sqrt(w*h)", "width"):
+        fs = mw / bases[nm]
+        best_f, best_cov = 0.0, -1.0
+        for f in np.arange(float(fs.min()) * 0.9, float(fs.max()) * 1.1, 0.002):
+            cov = float(((fs >= f * lo_l) & (fs <= f * hi_l)).mean())
+            if cov > best_cov:
+                best_f, best_cov = float(f), cov
+        print(f"{nm:12s} {best_f:7.3f} {best_f * lo_l:7.3f}-{best_f * hi_l:.3f} {100 * best_cov:8.1f}%")
+
+    # The raw distribution behind the coverage number: where the mark actually sits,
+    # so the mode structure (and what a 4th rung would recover) is visible directly.
+    fs = mw / bases["short"]
+    qs = [np.percentile(fs, q) for q in (5, 25, 50, 75, 95)]
+    print(f"\nfrac_short distribution: p5 {qs[0]:.3f} p25 {qs[1]:.3f} p50 {qs[2]:.3f} p75 {qs[3]:.3f} p95 {qs[4]:.3f}")
+    hist, edges = np.histogram(fs, bins=16)
+    for c, e0, e1 in zip(hist, edges[:-1], edges[1:], strict=True):
+        print(f"  {e0:.3f}-{e1:.3f}  {'#' * c}")
+
+    # Template aspect at the winning width -> the alpha_height_frac recommendation.
+    # Measured, per the standing rule: not the silhouette's own aspect, not doubao's.
+    aspects = np.array([r["aspect"] for r in strong if r["aspect"] > 0], float)
+    if len(aspects) >= 20:
+        med = float(np.median(aspects))
+        print(
+            f"\nASPECT FIT (n={len(aspects)}): p10 {np.percentile(aspects, 10):.3f} "
+            f"p50 {med:.3f} p90 {np.percentile(aspects, 90):.3f}"
+        )
+        print("alpha_height_frac = alpha_width_frac * p50(aspect), per basis:")
+        for nm in ("short", "sqrt(w*h)", "width"):
+            fxs = mw / bases[nm]
+            best_f = max(
+                np.arange(float(fxs.min()) * 0.9, float(fxs.max()) * 1.1, 0.002),
+                key=lambda f: float(((fxs >= f * lo_l) & (fxs <= f * hi_l)).mean()),
+            )
+            print(f"  {nm:12s} width {best_f:.3f}  ->  height {best_f * med:.4f}")
+
+    # LOCATE-box fit. The box fractions are as mark-specific as the template size:
+    # doubao's box clipped qwen's big-mode mark (see _fit_one). Derive the box from the
+    # measured absolute mark rects: margins must not exceed the mark's own (else the
+    # mark exits the anchored box), and the box must cover the mark plus NCC slack.
+    if len(aspects) >= 20:
+        short = np.minimum(w, h).astype(float)
+        mark_h = np.array([r["mark_w"] * r["aspect"] for r in strong], float)
+        ax = np.array([r["x"] for r in strong], float)
+        ay = np.array([r["y"] for r in strong], float)
+        right = (w - (ax + mw)) / short  # frame right edge to mark right edge
+        bottom = (h - (ay + mark_h)) / short
+        print(f"\nLOCATE FIT (basis=short, n={len(strong)}):")
+        print(f"  right-margin frac  p5 {np.percentile(right, 5):.4f} p50 {np.percentile(right, 50):.4f}")
+        print(f"  bottom-margin frac p5 {np.percentile(bottom, 5):.4f} p50 {np.percentile(bottom, 50):.4f}")
+        print(
+            f"  mark height frac   p50 {np.percentile(mark_h / short, 50):.4f} "
+            f"p95 {np.percentile(mark_h / short, 95):.4f}"
+        )
+        mx = max(0.002, float(np.percentile(right, 5)) - 0.004)
+        mb = max(0.002, float(np.percentile(bottom, 5)) - 0.004)
+        need_w = float(np.percentile(mw / short + right, 95)) - mx + 0.02
+        need_h = float(np.percentile(mark_h / short + bottom, 95)) - mb + 0.015
+        print(f"  recommended: margin_x_frac={mx:.4f} margin_bottom_frac={mb:.4f}")
+        print(f"               width_frac={need_w:.3f} height_frac={need_h:.3f}")
+
+    print("\nThese are DIAGNOSTIC. Re-score both arms on the shipped ladder with the")
+    print("fitted geometry before reading any gate off the clean arm.")
+
+
+FIRED = REPO / "data" / "spaces" / "_visible_positives.jsonl"
+
+
+def _fired_pool(mark: str, limit: int, seed: int = 7) -> list[str]:
+    """Paths where ``mark`` fired, from the COMPLETED full-corpus artifact -- the
+    standing rule: detector firings are joined, never re-run."""
+    import random
+
+    pool = [
+        json.loads(x)["path"]
+        for x in FIRED.read_text(encoding="utf-8").splitlines()
+        if x.strip() and mark in (json.loads(x).get("keys") or [])
+    ]
+    rng = random.Random(seed)  # noqa: S311 -- reproducible sampling, not crypto
+    rng.shuffle(pool)
+    return pool[:limit]
+
+
+def _cross_score(args: tuple[str, Any, Any]) -> dict[str, Any] | None:
+    """One frame scored by BOTH the candidate and the doubao production configs."""
+    path_str, cand_cfg, db_cfg = args
+    from remove_ai_watermarks._text_mark_engine import TextMarkEngine
+    from remove_ai_watermarks.image_io import imread
+
+    img = imread(path_str)
+    if img is None or min(img.shape[:2]) < 200:
+        return None
+    out: dict[str, Any] = {"path": path_str}
+    for key, cfg in (("cand", cand_cfg), ("doubao", db_cfg)):
+        eng = TextMarkEngine(cfg)
+        loc = eng.locate(img)
+        try:
+            score, _ = eng._tophat_best(img, loc)
+        except Exception:
+            return None
+        out[key] = round(float(score), 4)
+    return out
+
+
+def crossfire(
+    pools: dict[str, list[str]],
+    cand_cfg: Any,
+    workers: int,
+    gate: float,
+    margin: float = 0.10,
+) -> None:
+    """Score the candidate AND doubao's production template on the same frames.
+
+    The registration question a cohort-vs-clean run cannot answer: the candidate shares
+    the mandated `AI生成` tail with doubao (4 of 6 glyph cells), so its template will
+    correlate with doubao marks too. If the candidate fires on the doubao pool at the
+    candidate gate, registering it double-fills every doubao frame and mislabels it --
+    unless the rival margin suppresses it, which then has to be shown NOT to kill the
+    candidate on its own marks. Measured here in the tophat domain (the gate's domain);
+    production's `_rival_margin_ok` runs the same comparison on the binary blob.
+    """
+    import numpy as np
+
+    from remove_ai_watermarks.doubao_engine import _CONFIG as db_cfg
+
+    print(f"\n{'=' * 78}\nCROSSFIRE -- candidate vs doubao, same frames, tophat domain\n{'=' * 78}")
+    print(f"candidate gate {gate:.3f} | rival margin {margin:.2f}\n")
+    print(
+        f"{'pool':8s} {'n':>5s} {'cand p50':>9s} {'cand p90':>9s} {'db p50':>7s} {'db p90':>7s} "
+        f"{'m-d p10':>8s} {'m-d p50':>8s} {'fire':>7s} {'fire+m':>7s}"
+    )
+    with ProcessPoolExecutor(max_workers=workers) as ex:
+        for pool_name, paths in pools.items():
+            rows: list[dict[str, Any]] = []
+            futs = [ex.submit(_cross_score, (p, cand_cfg, db_cfg)) for p in paths]
+            for f in as_completed(futs):
+                try:
+                    r = f.result()
+                except Exception:  # noqa: S112 -- one bad file must not kill the pool
+                    continue
+                if r:
+                    rows.append(r)
+            if not rows:
+                continue
+            c = np.array([r["cand"] for r in rows])
+            d = np.array([r["doubao"] for r in rows])
+            m = c - d
+            fire = c >= gate
+            fire_m = fire & (m >= margin)
+            print(
+                f"{pool_name:8s} {len(rows):5d} {np.percentile(c, 50):9.3f} {np.percentile(c, 90):9.3f} "
+                f"{np.percentile(d, 50):7.3f} {np.percentile(d, 90):7.3f} "
+                f"{np.percentile(m, 10):8.3f} {np.percentile(m, 50):8.3f} "
+                f"{100 * float(fire.mean()):6.1f}% {100 * float(fire_m.mean()):6.1f}%"
+            )
+    print("\nReading: on `qwen` the margin column must stay high (the candidate keeps its")
+    print("own marks); on `doubao` fire+m must sit near zero (the candidate stays off")
+    print("doubao marks). If fire is high on `doubao` and fire+m is not, the rival margin")
+    print("is load-bearing for registration; if both are high, the mark cannot be")
+    print("registered on this front-end at all.")
+
+
+def _parse_ladder(raw: str) -> tuple[float, ...] | None:
+    if not raw:
+        return None
+    return tuple(float(x) for x in raw.split(","))
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--cohort", required=True, help="cohort USCC from vendor_cohort_harvest.py")
+    ap.add_argument("--asset", required=True, help="silhouette asset name, e.g. qwen_alpha.png")
+    ap.add_argument("--name", default="", help="label for output files (defaults to the asset stem)")
+    ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2))
+    ap.add_argument("--scale-basis", choices=("short", "width"), default="short")
+    ap.add_argument("--sheets", action="store_true")
+    ap.add_argument(
+        "--fit-geometry",
+        action="store_true",
+        help="fit the basis + fraction + template aspect the mark scales with",
+    )
+    ap.add_argument("--width-frac", type=float, default=None, help="fitted alpha_width_frac (default: inherit doubao)")
+    ap.add_argument(
+        "--height-frac", type=float, default=None, help="fitted alpha_height_frac (default: inherit doubao)"
+    )
+    ap.add_argument("--ladder", default="", help="comma scale rungs, e.g. 0.8,1.0,1.25,1.6 (default: shipped 3)")
+    ap.add_argument("--gate", type=float, default=0.45, help="candidate gate for the crossfire fire rates")
+    ap.add_argument("--box-width-frac", type=float, default=None, help="fitted locate width_frac")
+    ap.add_argument("--box-height-frac", type=float, default=None, help="fitted locate height_frac")
+    ap.add_argument("--margin-x-frac", type=float, default=None, help="fitted locate margin_x_frac")
+    ap.add_argument("--margin-bottom-frac", type=float, default=None, help="fitted locate margin_bottom_frac")
+    ap.add_argument(
+        "--crossfire",
+        action="store_true",
+        help="score the candidate AND doubao on the cohort, the doubao/jimeng pools and the clean arm",
+    )
+    a = ap.parse_args()
+    name = a.name or a.asset.split("_")[0]
+    ladder = _parse_ladder(a.ladder)
+    overrides: dict[str, Any] = {}
+    for arg, field in (
+        (a.width_frac, "alpha_width_frac"),
+        (a.height_frac, "alpha_height_frac"),
+        (a.box_width_frac, "width_frac"),
+        (a.box_height_frac, "height_frac"),
+        (a.margin_x_frac, "margin_x_frac"),
+        (a.margin_bottom_frac, "margin_bottom_frac"),
+    ):
+        if arg is not None:
+            overrides[field] = arg
+    if ladder is not None:
+        overrides["ladder"] = ladder
+
+    pos_paths, neg_paths = load_sets(a.cohort)
+    if a.fit_geometry:
+        print(f"cohort {a.cohort}: {len(pos_paths)} candidates")
+        fit_geometry(pos_paths, a.asset, a.workers, paths_name=name)
+        return
+
+    if a.crossfire:
+        cand = build_config(a.asset, name, a.scale_basis, overrides)
+        pools = {
+            "qwen": pos_paths,
+            "doubao": _fired_pool("doubao", 400),
+            "jimeng": _fired_pool("jimeng", 300),
+            "clean": neg_paths,
+        }
+        print("pools: " + ", ".join(f"{k}={len(v)}" for k, v in pools.items()))
+        crossfire(pools, cand, a.workers, a.gate)
+        return
+
+    print(f"cohort {a.cohort}: {len(pos_paths)} candidates | clean: {len(neg_paths)} hand-labelled")
+    print(f"scale_basis={a.scale_basis} overrides={overrides}")
+    pos = run(pos_paths, a.asset, name, a.workers, a.scale_basis, overrides)
+    neg = run(neg_paths, a.asset, name, a.workers, a.scale_basis, overrides)
+    OUT.write_text(
+        "\n".join(json.dumps({**r, "arm": arm}) for arm, rows in (("cohort", pos), ("clean", neg)) for r in rows),
+        encoding="utf-8",
+    )
+    report(pos, neg, name)
+    if a.sheets:
+        print(f"\nsheets -> {SHEET_DIR}")
+        sheets(pos, name)
+
+
+if __name__ == "__main__":
+    main()

scripts/visible_alpha_solve.py

@@ -0,0 +1,271 @@
+"""Rebuild the visible-watermark alpha assets from controlled captures.
+
+The committed, reproducible build of the bundled visible-mark assets -- the inputs
+live in ``data/<engine>_capture/captures/`` (committed solid-colour captures run
+through the generator). Re-run after re-capturing.
+
+**Doubao "豆包AI生成" strip and Jimeng "★ 即梦AI" wordmark** are fixed
+semi-transparent white overlays; the asset is their recovered per-pixel alpha map
+(``assets/<engine>_alpha.png``). The "careful" solve (issue #13) -- a naive build
+(max-over-channels, coarse background, blur, truncated halo, or a black-dominated
+least-squares fit) leaves a visible outline because the alpha is wrong at the glyph
+edges:
+
+1. Locate the mark on the BLACK capture (bright pixels in the bottom-right).
+2. Fit a smooth CUBIC background per channel over the GRAY capture's non-glyph
+   pixels (a cubic captures the gentle gradient without bleeding glyph values).
+3. Solve ``a = (I - B) / (255 - B)`` on the gray capture, AVERAGED over channels,
+   at FULL halo extent (down to a~0.02) and UNBLURRED. Gray (background ~130-200)
+   is the reference because the mark sits on bright photo content in real use, not
+   on black; the white capture only confirms the logo is white.
+
+**Gemini sparkle** is a different type: a single icon stamped on PURE BLACK, so the
+engine reads ``alpha = max(R,G,B)/255`` directly (no background fit). Its assets are
+the sparkle-on-black capture cropped to two fixed logo sizes (``gemini_bg_{96,48}.png``).
+
+Usage::
+
+    uv run python scripts/visible_alpha_solve.py doubao
+    uv run python scripts/visible_alpha_solve.py jimeng
+    uv run python scripts/visible_alpha_solve.py gemini
+    uv run python scripts/visible_alpha_solve.py all
+"""
+
+# cv2/numpy boundary: third-party libs ship no usable element types; relax the
+# unknown-type rules for this file only (mirrors the engine modules).
+# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false
+from __future__ import annotations
+
+import logging
+import sys
+from dataclasses import dataclass
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+import click
+import cv2
+import numpy as np
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from remove_ai_watermarks import image_io
+
+if TYPE_CHECKING:
+    from numpy.typing import NDArray
+
+log = logging.getLogger(__name__)
+
+_ROOT = Path(__file__).resolve().parents[1]
+
+
+@dataclass(frozen=True)
+class EngineSpec:
+    """Per-engine capture inputs and the alpha asset to rebuild."""
+
+    name: str
+    capture_dir: Path
+    black: str
+    gray: str
+    asset: Path
+    native_width: int = 2048
+    corner: str = "br"  # which corner the mark sits in: "br" (Doubao/Jimeng) or "bl" (Samsung)
+
+
+_SPECS: dict[str, EngineSpec] = {
+    "doubao": EngineSpec(
+        "doubao",
+        _ROOT / "data" / "doubao_capture" / "captures",
+        "doubao_black_1x1_1.png",
+        "doubao_gray_1x1_1.png",
+        _ROOT / "src" / "remove_ai_watermarks" / "assets" / "doubao_alpha.png",
+    ),
+    "jimeng": EngineSpec(
+        "jimeng",
+        _ROOT / "data" / "jimeng_capture" / "captures",
+        "jimeng_cap_A.png",  # black seed
+        "jimeng_cap_C.png",  # gray seed
+        _ROOT / "src" / "remove_ai_watermarks" / "assets" / "jimeng_alpha.png",
+    ),
+    "samsung": EngineSpec(
+        "samsung",
+        _ROOT / "data" / "samsung_capture" / "captures",
+        "samsung_black_1.png",  # black flat edit (mark on true black, bottom-left)
+        "samsung_gray_1.png",  # gray flat edit
+        _ROOT / "src" / "remove_ai_watermarks" / "assets" / "samsung_alpha.png",
+        # The flat captures arrive at the phone's flat-edit size (1086 wide); the
+        # mark is a fixed FRACTION of width (~0.31), consistent with the 2958-wide
+        # real photos, so geometry is emitted relative to the capture width.
+        native_width=1086,
+        corner="bl",
+    ),
+}
+
+_CUBIC_BG_PAD = 30  # px of background margin around the mark for the cubic fit
+_GLYPH_BODY = 0.08  # alpha above this is the solid glyph body (for the bbox)
+_MIN_PART_AREA = 25  # drop connected glyph-mask blobs smaller than this (cubic-fit specks)
+_HALO_PAD = 7  # keep this many px of halo around the glyph body in the saved asset
+
+# Gemini is a different watermark TYPE: a single sparkle icon stamped on a
+# PURE-BLACK background (so the engine reads alpha = max(R,G,B)/255 directly, no
+# background fit). Its assets are the sparkle-on-black CAPTURE at two fixed logo
+# sizes (the engine interpolates between them), not an alpha map.
+_GEMINI_CAPTURE = _ROOT / "data" / "gemini_capture" / "captures" / "gemini_black_2048.png"
+_GEMINI_ASSETS: dict[int, Path] = {
+    96: _ROOT / "src" / "remove_ai_watermarks" / "assets" / "gemini_bg_96.png",
+    48: _ROOT / "src" / "remove_ai_watermarks" / "assets" / "gemini_bg_48.png",
+}
+
+
+def _union_bbox(mask: NDArray[np.uint8], err: str) -> tuple[int, int, int, int]:
+    """Union bbox ``(x0, x1, y0, y1)`` of ``mask``'s connected components with area
+    >= ``_MIN_PART_AREA``. The mark is several separate glyphs, so the union spans
+    the whole word while a stray small speck/blotch is dropped by the area filter.
+    Raises ``ValueError(err)`` if nothing qualifies."""
+    n, _labels, stats, _c = cv2.connectedComponentsWithStats(mask, connectivity=8)
+    parts = [i for i in range(1, n) if stats[i, cv2.CC_STAT_AREA] >= _MIN_PART_AREA]
+    if not parts:
+        raise ValueError(err)
+    x0 = min(int(stats[i, cv2.CC_STAT_LEFT]) for i in parts)
+    y0 = min(int(stats[i, cv2.CC_STAT_TOP]) for i in parts)
+    x1 = max(int(stats[i, cv2.CC_STAT_LEFT] + stats[i, cv2.CC_STAT_WIDTH]) for i in parts)
+    y1 = max(int(stats[i, cv2.CC_STAT_TOP] + stats[i, cv2.CC_STAT_HEIGHT]) for i in parts)
+    return x0, x1, y0, y1
+
+
+def _locate_on_black(black: NDArray[np.float32], corner: str = "br") -> tuple[int, int, int, int]:
+    """Bounding box of the white mark on the black capture, in the given corner.
+
+    Thresholds well above the blotchy near-black background, then unions the
+    sufficiently-large bright components so the box spans the whole word. ``corner``
+    is ``"br"`` (bottom-right, Doubao/Jimeng) or ``"bl"`` (bottom-left, Samsung).
+    The horizontal window is kept generous (the Samsung text strip is ~0.31 of the
+    width, so a corner *quarter* would clip it) while still excluding any centered
+    generated content the flat edit hallucinated.
+    """
+    h, w = black.shape[:2]
+    lum = black.mean(axis=2)
+    br = lum > 40  # comfortably above the ~5-30 background blotches
+    br[: h * 3 // 4, :] = False  # bottom quarter only
+    if corner == "bl":
+        br[:, w // 2 :] = False  # left half only
+    else:
+        br[:, : w * 3 // 4] = False  # right quarter only
+    bright = cv2.morphologyEx(br.astype(np.uint8) * 255, cv2.MORPH_CLOSE, np.ones((9, 9), np.uint8))
+    return _union_bbox(bright, f"no mark found on the black capture ({corner} corner is empty)")
+
+
+def _cubic_background(crop: NDArray[np.float32], glyph: NDArray[np.bool_]) -> NDArray[np.float32]:
+    """Per-channel cubic surface fit over the non-glyph pixels of ``crop``."""
+    h, w = crop.shape[:2]
+    yy, xx = np.mgrid[0:h, 0:w].astype(np.float32)
+    yy /= h
+    xx /= w
+    terms = [np.ones_like(xx), xx, yy, xx * xx, xx * yy, yy * yy, xx**3, xx * xx * yy, xx * yy * yy, yy**3]
+    basis = np.stack(terms, axis=-1).reshape(-1, len(terms))
+    keep = (~glyph).reshape(-1)
+    out = np.zeros_like(crop)
+    for ch in range(3):
+        values = crop[..., ch].reshape(-1)
+        coef, *_ = np.linalg.lstsq(basis[keep], values[keep], rcond=None)
+        out[..., ch] = (basis @ coef).reshape(h, w)
+    return out
+
+
+def solve_alpha(spec: EngineSpec) -> NDArray[np.uint8]:
+    """Solve the careful gray-self alpha map for one engine (uint8, a*255)."""
+    black = image_io.imread(str(spec.capture_dir / spec.black), cv2.IMREAD_COLOR)
+    gray = image_io.imread(str(spec.capture_dir / spec.gray), cv2.IMREAD_COLOR)
+    if black is None or gray is None:
+        raise FileNotFoundError(f"missing captures in {spec.capture_dir} (expected {spec.black}, {spec.gray})")
+    black_f = black.astype(np.float32)
+    gray_f = gray.astype(np.float32)
+
+    img_h, img_w = black_f.shape[:2]
+    mx0, mx1, my0, my1 = _locate_on_black(black_f, spec.corner)
+    pad = _CUBIC_BG_PAD
+    rx0, rx1 = max(0, mx0 - pad), min(img_w, mx1 + pad)
+    ry0, ry1 = max(0, my0 - pad), min(img_h, my1 + pad)
+    cg = gray_f[ry0:ry1, rx0:rx1]
+    cb = black_f[ry0:ry1, rx0:rx1]
+
+    glyph = cv2.dilate((cb.mean(axis=2) > 8).astype(np.uint8), np.ones((9, 9), np.uint8)) > 0
+    bg = _cubic_background(cg, glyph)
+    alpha = np.clip((cg - bg).mean(axis=2) / np.clip(255.0 - bg.mean(axis=2), 1e-3, None), 0.0, 1.0)
+
+    # Crop to the UNION of the glyph parts (the mark is several disconnected
+    # glyphs), padded by _HALO_PAD -- this keeps the real anti-aliased halo while
+    # dropping the small cubic-fit specks at the crop edges (< _MIN_PART_AREA) that
+    # a bare a>floor box would otherwise inflate the asset with.
+    body = (alpha > _GLYPH_BODY).astype(np.uint8)
+    bx, bex, by, bey = _union_bbox(body, "solved alpha has no glyph body -- check the gray capture background")
+    cx0 = max(0, bx - _HALO_PAD)
+    cy0 = max(0, by - _HALO_PAD)
+    cx1 = min(alpha.shape[1], bex + _HALO_PAD)
+    cy1 = min(alpha.shape[0], bey + _HALO_PAD)
+    tight = alpha[cy0:cy1, cx0:cx1]
+    aw, ah = tight.shape[1], tight.shape[0]
+    # Absolute asset position in the capture, for the engine's geometry constants.
+    abs_x0, abs_y0 = rx0 + cx0, ry0 + cy0
+    # Horizontal margin depends on the anchor corner: left margin for "bl", right
+    # margin (distance from the right edge) for "br".
+    h_margin = abs_x0 if spec.corner == "bl" else img_w - (abs_x0 + aw)
+    log.info(
+        "%s: alpha %dx%d max %.3f | WIDTH_FRAC %.4f HEIGHT_FRAC %.4f "
+        "MARGIN_%s_FRAC %.4f MARGIN_BOTTOM_FRAC %.4f (native_width %d)",
+        spec.name,
+        aw,
+        ah,
+        float(tight.max()),
+        aw / spec.native_width,
+        ah / spec.native_width,
+        "LEFT" if spec.corner == "bl" else "RIGHT",
+        h_margin / spec.native_width,
+        (img_h - (abs_y0 + ah)) / spec.native_width,
+        spec.native_width,
+    )
+    return (np.clip(tight, 0.0, 1.0) * 255.0).astype(np.uint8)
+
+
+def solve_gemini() -> dict[int, NDArray[np.uint8]]:
+    """Extract the Gemini sparkle-on-black region from the black capture at each
+    bundled logo size (the bg-capture asset format; the engine derives the alpha).
+    Returns ``{size: bgr_image}``."""
+    black = image_io.imread(str(_GEMINI_CAPTURE), cv2.IMREAD_COLOR)
+    if black is None:
+        raise FileNotFoundError(f"missing Gemini capture {_GEMINI_CAPTURE}")
+    h, w = black.shape[:2]
+    bright = np.zeros((h, w), np.uint8)
+    reg = black.astype(np.float32).mean(axis=2) > 60  # sparkle is ~0.5*255 on black
+    reg[: h * 3 // 4, :] = False
+    reg[:, : w * 3 // 4] = False
+    bright[reg] = 255
+    bright = cv2.morphologyEx(bright, cv2.MORPH_CLOSE, np.ones((7, 7), np.uint8))
+    x0, x1, y0, y1 = _union_bbox(bright, "no sparkle found on the Gemini black capture")
+    crop = black[y0:y1, x0:x1]
+    log.info("gemini: sparkle %dx%d at margin_frac %.4f", x1 - x0, y1 - y0, (w - x1) / w)
+    return {size: cv2.resize(crop, (size, size), interpolation=cv2.INTER_AREA) for size in _GEMINI_ASSETS}
+
+
+@click.command()
+@click.argument("engine", type=click.Choice([*_SPECS, "gemini", "all"]))
+def main(engine: str) -> None:
+    """Rebuild the alpha asset(s) for ENGINE (doubao / jimeng / gemini / all)."""
+    logging.basicConfig(level=logging.INFO, format="%(message)s")
+
+    def _write(path: Path, img: NDArray[np.uint8], label: str) -> None:
+        path.parent.mkdir(parents=True, exist_ok=True)
+        if not image_io.imwrite(str(path), img):
+            raise OSError(f"failed to write {path}")
+        log.info("%s: wrote %s", label, path.relative_to(_ROOT))
+
+    if engine in (*_SPECS, "all"):
+        specs = list(_SPECS.values()) if engine == "all" else [_SPECS[engine]]
+        for spec in specs:
+            _write(spec.asset, solve_alpha(spec), spec.name)
+    if engine in ("gemini", "all"):
+        for size, img in solve_gemini().items():
+            _write(_GEMINI_ASSETS[size], img, f"gemini-{size}")
+
+
+if __name__ == "__main__":
+    main()

scripts/visible_eval.py

@@ -0,0 +1,164 @@
+"""Benchmark harness for the visible-mark detectors.
+
+Run this before AND after any detector change. It re-runs perception over the
+hand-labelled ground truth and reports, per mark, how often a fire is correct --
+with Wilson intervals, so a change inside the noise is visible as such.
+
+    uv run python scripts/visible_eval.py                 # score current code
+    uv run python scripts/visible_eval.py --save baseline # snapshot for comparison
+    uv run python scripts/visible_eval.py --vs baseline   # diff against a snapshot
+
+WHAT THIS SET CAN AND CANNOT MEASURE -- read before quoting a number:
+
+  * PRECISION: sound. Every labelled crop is centred on the region a detector
+    pointed at, so "the detector fired mark K here, was K actually there" is
+    exactly the question the labels answer.
+  * RECALL: NOT measurable here, and the harness refuses to print it. The labelled
+    images were SAMPLED WHERE DETECTORS FIRED (relaxation additions plus controls),
+    so images carrying a mark that every detector missed are absent by construction.
+    Computing recall on this set would divide by a denominator that excludes exactly
+    the failures recall is meant to expose, and would report a flattering number.
+    Recall needs a RANDOM corpus sample laballed exhaustively -- a separate round.
+
+  * `other_ai_label` (千问 / 百度 / 星绘 / 抖音) counts as a FALSE fire for any
+    registered mark, because it is a different vendor's label. It is tracked
+    separately in the confusion output since it is the dominant jimeng failure.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import sys
+from collections import Counter
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
+
+from remove_ai_watermarks import watermark_registry as wr
+from remove_ai_watermarks.image_io import imread
+
+GT = Path("data/spaces/_research_20260718_textmark_relaxation/groundtruth.jsonl")
+SNAP = Path("data/spaces/_research_20260718_textmark_relaxation/snapshots")
+MARKS = ("gemini", "doubao", "jimeng", "samsung", "jimeng_pill")
+
+
+def wilson(k: int, n: int) -> tuple[float, float]:
+    if n == 0:
+        return (0.0, 0.0)
+    z, p = 1.96, k / n
+    d = 1 + z * z / n
+    c = p + z * z / (2 * n)
+    s = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n))
+    return ((c - s) / d, (c + s) / d)
+
+
+def provenance_for(rec: dict) -> frozenset[str]:
+    """The provenance production would have: read from the file's own METADATA.
+
+    Never derive this from the labels. A relaxation arm only fires when provenance
+    names the vendor, so label-derived provenance silently tells the detector the
+    answer and every arm scores near-perfect (observed: gemini 99% instead of 34%).
+    """
+    return frozenset(rec.get("provenance", []))
+
+
+def score(sensitivity: str = "auto") -> dict:
+    recs = [json.loads(line) for line in GT.open()]
+    per: dict[str, Counter] = {m: Counter() for m in MARKS}
+    confusion: dict[str, Counter] = {m: Counter() for m in MARKS}
+    missing = 0
+    for rec in recs:
+        img = imread(rec["path"])
+        if img is None:
+            missing += 1
+            continue
+        cands = wr._build_candidates(img)
+        ctx = wr.Context(sensitivity=sensitivity, provenance=provenance_for(rec))
+        fired = {d.candidate.key for d in wr.decide(cands, ctx)}
+        for m in MARKS:
+            # Only score a mark on images whose shown crop could rule on it. Scoring
+            # outside that scope books real detections as false fires -- see the
+            # adjudication note in visible_groundtruth.py.
+            if m not in rec.get("adjudicated", []):
+                continue
+            if m not in fired:
+                per[m]["fn"] += 1 if m in rec["present"] else 0
+                continue
+            if m in rec["present"]:
+                per[m]["tp"] += 1
+            else:
+                per[m]["fp"] += 1
+                for s in rec["seen"]:
+                    confusion[m][s] += 1
+    scope = {m: sum(1 for r in recs if m in r.get("adjudicated", [])) for m in MARKS}
+    return {
+        "per": {m: dict(c) for m, c in per.items()},
+        "scope": scope,
+        "confusion": {m: dict(c) for m, c in confusion.items()},
+        "missing": missing,
+        "n": len(recs),
+        "sensitivity": sensitivity,
+    }
+
+
+def report(res: dict, prev: dict | None = None) -> None:
+    print(f"\nground truth: {res['n']} images ({res['missing']} unreadable)   sensitivity={res['sensitivity']}")
+    print("=" * 78)
+    print(
+        f"{'mark':12s} {'scope':>6s} {'fires':>6s} {'correct':>8s} "
+        f"{'precision':>11s} {'95% CI':>13s}  {'missed':>7s}  delta"
+    )
+    print("-" * 78)
+    for m in MARKS:
+        c = res["per"][m]
+        tp, fp, fn = c.get("tp", 0), c.get("fp", 0), c.get("fn", 0)
+        n = tp + fp
+        scope = res["scope"].get(m, 0)
+        if n == 0:
+            print(f"{m:12s} {scope:6d} {0:6d} {'-':>8s} {'-':>11s} {'-':>13s} {fn:7d}")
+            continue
+        lo, hi = wilson(tp, n)
+        delta = ""
+        if prev:
+            pc = prev["per"][m]
+            pn = pc.get("tp", 0) + pc.get("fp", 0)
+            if pn:
+                d = tp / n - pc.get("tp", 0) / pn
+                delta = f"{d:+.1%} (fires {pn}->{n})"
+        print(f"{m:12s} {scope:6d} {n:6d} {tp:8d} {tp / n:10.0%} {lo:5.0%}-{hi:<6.0%} {fn:7d}  {delta}")
+    print("\nwhat the FALSE fires actually were:")
+    for m in MARKS:
+        conf = {k: v for k, v in res["confusion"][m].items() if k != m}
+        if conf:
+            print(f"  {m:12s} {dict(sorted(conf.items(), key=lambda kv: -kv[1]))}")
+    print("\n'scope' = images whose crop could rule on that mark; 'missed' = labelled marks it did not fire on.")
+    print("NOTE: 'missed' is NOT recall -- this set was sampled where detectors fired, so images")
+    print("      every detector missed are absent by construction. Use it only to catch a change")
+    print("      LOSING marks it used to find; an unbiased random sample is needed for true recall.")
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--save", metavar="NAME")
+    ap.add_argument("--vs", metavar="NAME")
+    ap.add_argument("--sensitivity", default="auto")
+    a = ap.parse_args()
+    res = score(a.sensitivity)
+    prev = None
+    if a.vs:
+        f = SNAP / f"{a.vs}.json"
+        if f.exists():
+            prev = json.loads(f.read_text())
+        else:
+            print(f"(no snapshot {f}; showing absolute numbers)")
+    report(res, prev)
+    if a.save:
+        SNAP.mkdir(parents=True, exist_ok=True)
+        (SNAP / f"{a.save}.json").write_text(json.dumps(res, indent=1))
+        print(f"\nsaved snapshot -> {SNAP / f'{a.save}.json'}")
+
+
+if __name__ == "__main__":
+    main()

scripts/visible_groundtruth.py

@@ -0,0 +1,134 @@
+"""Consolidate the hand-labelled contact-sheet rounds into ONE ground-truth file.
+
+Ground truth is `uid -> the set of visible marks actually present`, hand-labelled
+blind against contact sheets with a two-sided control in every round. Rounds so far:
+
+  2026-07-18 text-mark/pill round : 423 cells (doubao / jimeng / jimeng_pill arms)
+  2026-07-18 gemini round         : 356 cells (gemini relaxation additions)
+
+DATA SAFETY: the corpus is real user uploads. This script reads the gitignored
+corpus and writes a gitignored ground-truth file. Neither the images nor this
+output may be committed; only the harness is. See the repo CLAUDE.md.
+
+The labels record what the LABELLER SAW in the crop, one of:
+  doubao | jimeng | pill | sparkle | other_ai_label | none | uncertain
+`other_ai_label` is a real visible AI label from a vendor we do NOT have a mark for
+(千问 / 百度 / 星绘 / 抖音); it is NOT a positive for any registered mark, but it is
+also not "clean" -- it is exactly what the relaxed jimeng detector confuses.
+`uncertain` rows are EXCLUDED from scoring rather than coerced, so a labeller's
+honest doubt never becomes a fabricated data point.
+"""
+
+from __future__ import annotations
+
+import csv
+import json
+import sys
+from pathlib import Path
+
+SEEN_TO_MARK = {
+    "doubao": "doubao",
+    "jimeng": "jimeng",
+    "pill": "jimeng_pill",
+    "sparkle": "gemini",
+}
+# Which marks a crop centred on `key` lets the labeller rule on (same corner = visible
+# in the same crop). Doubao and Jimeng share the bottom-right corner.
+_ADJUDICATES = {
+    "doubao": ("doubao", "jimeng"),
+    "jimeng": ("doubao", "jimeng"),
+    "jimeng_pill": ("jimeng_pill",),
+    "gemini": ("gemini",),
+    "samsung": ("samsung",),
+}
+
+ROUNDS = [
+    ("textmark", "labels.csv", "manifest.csv"),
+    ("gemini", "gemini_labels.csv", "gemini_manifest.csv"),
+]
+
+
+def metadata_provenance(path: str) -> list[str]:
+    """The vendor keys LOCAL METADATA confirms -- what cli._visible_provenance reads.
+
+    Must come from the file's own metadata, never from the labels: deriving it from
+    the ground truth would hand the detector the answer it is being scored on (a
+    relaxation only fires when provenance names the vendor, so label-derived
+    provenance makes every arm look near-perfect). Read from the corpus `identify`
+    sidecar, which is the same signal production computes.
+    """
+    p = Path(path)
+    sidecar = Path(str(p.parent).replace("/originals/", "/identify/")) / (p.name.split("_src")[0] + ".json")
+    if not sidecar.exists():
+        return []
+    try:
+        with sidecar.open() as fh:
+            wm = " | ".join(json.load(fh).get("watermarks", []))
+    except Exception:
+        return []
+    keys: list[str] = []
+    if "China AIGC label" in wm:
+        keys += ["doubao", "jimeng"]
+    if "C2PA Content Credentials (Google LLC" in wm:
+        keys.append("gemini")
+    if "Samsung Galaxy AI" in wm:
+        keys.append("samsung")
+    return keys
+
+
+def main() -> None:
+    root = Path(sys.argv[1] if len(sys.argv) > 1 else "data/spaces/_research_20260718_textmark_relaxation")
+    out = root / "groundtruth.jsonl"
+    rows: dict[str, dict] = {}
+    stats: dict[str, int] = {}
+    for round_name, labels_file, manifest_file in ROUNDS:
+        with (root / labels_file).open() as fh:
+            labels = {int(r["idx"]): r["seen"] for r in csv.DictReader(fh)}
+        with (root / manifest_file).open() as fh:
+            manifest_rows = list(csv.DictReader(fh))
+        for m in manifest_rows:
+            seen = labels.get(int(m["idx"]))
+            if seen is None:
+                continue
+            stats[seen] = stats.get(seen, 0) + 1
+            if seen == "uncertain":
+                continue  # excluded by design -- never coerce a doubt into a label
+            rec = rows.setdefault(
+                m["uid"],
+                {"uid": m["uid"], "path": m["path"], "present": [], "seen": [], "rounds": [], "adjudicated": []},
+            )
+            # ADJUDICATION SCOPE -- load-bearing. A crop centred on one mark only lets
+            # the labeller rule on marks visible IN THAT CROP. A pill crop (top-left)
+            # says nothing about a bottom-right wordmark, so scoring jimeng against a
+            # pill-round image would book real detections as false fires (~61% of pills
+            # carry a wordmark). Bottom-right marks co-adjudicate each other: one crop
+            # of that corner shows whichever of Doubao/Jimeng is there.
+            for k in _ADJUDICATES.get(m["key"], (m["key"],)):
+                if k not in rec["adjudicated"]:
+                    rec["adjudicated"].append(k)
+            mark = SEEN_TO_MARK.get(seen)
+            if mark and mark not in rec["present"]:
+                rec["present"].append(mark)
+            if seen not in rec["seen"]:
+                rec["seen"].append(seen)
+            if round_name not in rec["rounds"]:
+                rec["rounds"].append(round_name)
+
+    for r in rows.values():
+        r["provenance"] = metadata_provenance(r["path"])
+    with out.open("w") as fh:
+        for r in rows.values():
+            fh.write(json.dumps(r) + "\n")
+    print(f"wrote {out}  images={len(rows)}")
+    print("label distribution across all rounds:", dict(sorted(stats.items(), key=lambda kv: -kv[1])))
+    n_pos = sum(1 for r in rows.values() if r["present"])
+    print(f"images with at least one registered mark: {n_pos}; clean-of-registered-marks: {len(rows) - n_pos}")
+    adj: dict[str, int] = {}
+    for r in rows.values():
+        for k in r["adjudicated"]:
+            adj[k] = adj.get(k, 0) + 1
+    print("images each mark can be SCORED on (adjudication scope):", dict(sorted(adj.items())))
+
+
+if __name__ == "__main__":
+    main()

scripts/visible_positives.py

@@ -0,0 +1,141 @@
+"""Parallel detection pass: list every corpus image carrying a known visible mark.
+
+Why this exists separately from `visible_removal_audit.py`: that audit is single-process,
+so a full-corpus sweep costs ~10 h and running it once per backend costs ~30 h. But its
+expensive half is DETECTION, and detection does not depend on the fill backend. Splitting
+it out means detecting once in parallel and then feeding the positives to the audit via
+its `--paths-file` seam, over a few thousand images instead of forty thousand.
+
+CRASH TOLERANCE IS NOT OPTIONAL AT THIS SCALE
+  cv2/libpng decode native-crash on some real uploads. A plain `ProcessPoolExecutor.map`
+  over 39k files then DEADLOCKS: the worker dies without a Python traceback and the parent
+  waits forever on a result that never arrives (observed 2026-07-19 -- 26 min of work lost
+  because results were only written at the end). So this script:
+    * writes every result to JSONL as it arrives -- a kill never costs more than a batch;
+    * is resumable -- an interrupted run skips what it already recorded;
+    * runs a FRESH pool per batch with a timeout, so one poisoned file costs one batch,
+      and that batch is retried serially to find and record the offender.
+
+Corpus images are user uploads: read-only, local analysis, gitignored output.
+
+    uv run python scripts/visible_positives.py --jobs 6
+"""
+
+from __future__ import annotations
+
+import argparse
+import collections
+import glob
+import json
+import os
+import sys
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from concurrent.futures import TimeoutError as FutureTimeout
+from concurrent.futures.process import BrokenProcessPool
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+# The package's own format set. An inlined copy here silently skipped .heif, which
+# CLAUDE.md documents as supported.
+from remove_ai_watermarks.noai.constants import SUPPORTED_FORMATS as _EXTS
+
+REPO = Path(__file__).resolve().parents[1]
+CORPUS = REPO / "data" / "spaces" / "originals"
+OUT = REPO / "data" / "spaces" / "_visible_positives.jsonl"
+PATHS = REPO / "data" / "spaces" / "_visible_positives.txt"
+
+
+def _one(path: str) -> dict[str, object]:
+    from remove_ai_watermarks.image_io import imread
+    from remove_ai_watermarks.watermark_registry import detect_marks
+
+    try:
+        img = imread(path)
+        if img is None:
+            return {"path": path, "keys": [], "status": "unreadable"}
+        return {"path": path, "keys": [d.key for d in detect_marks(img) if d.detected], "status": "ok"}
+    except Exception as e:
+        return {"path": path, "keys": [], "status": f"error:{type(e).__name__}"}
+
+
+def _run_batch(batch: list[str], jobs: int, timeout: int) -> list[dict[str, object]]:
+    """One batch in a fresh pool. On a native worker crash or timeout, retry serially so
+    the poisoned file is identified and recorded instead of stalling the whole sweep."""
+    try:
+        with ProcessPoolExecutor(max_workers=jobs) as ex:
+            futs = {ex.submit(_one, p): p for p in batch}
+            return [f.result() for f in as_completed(futs, timeout=timeout)]
+    except (FutureTimeout, BrokenProcessPool, OSError, RuntimeError):
+        out: list[dict[str, object]] = []
+        for p in batch:
+            try:
+                out.append(_one(p))
+            except BaseException:  # a native crash here kills only this file, not the sweep
+                out.append({"path": p, "keys": [], "status": "crashed"})
+        return out
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--jobs", type=int, default=max(1, (os.cpu_count() or 4) - 2))
+    ap.add_argument("--limit", type=int, default=0)
+    ap.add_argument("--batch", type=int, default=200)
+    ap.add_argument("--timeout", type=int, default=600, help="seconds per batch before falling back to serial")
+    ap.add_argument("--out", type=Path, default=OUT)
+    ap.add_argument("--restart", action="store_true")
+    a = ap.parse_args()
+
+    files = sorted(p for p in glob.glob(str(CORPUS / "*" / "*")) if Path(p).suffix.lower() in _EXTS)
+    if a.limit:
+        files = files[: a.limit]
+
+    done: set[str] = set()
+    if a.restart and a.out.exists():
+        a.out.unlink()  # --restart must TRUNCATE; the file is reopened in append mode below
+    if a.out.exists() and not a.restart:
+        with open(a.out, encoding="utf-8") as fh:
+            for line in fh:
+                try:
+                    done.add(json.loads(line)["path"])
+                except Exception:  # noqa: S112 -- tolerate a torn last line
+                    continue
+    todo = [p for p in files if p not in done]
+    print(f"images {len(files)}  already done {len(done)}  to do {len(todo)}  jobs {a.jobs}", flush=True)
+
+    counts: collections.Counter[str] = collections.Counter()
+    status: collections.Counter[str] = collections.Counter()
+    seen = 0
+    with open(a.out, "a", encoding="utf-8") as fh:
+        for start in range(0, len(todo), a.batch):
+            for rec in _run_batch(todo[start : start + a.batch], a.jobs, a.timeout):
+                fh.write(json.dumps(rec) + "\n")
+                status[str(rec["status"])] += 1
+                for k in rec["keys"]:  # type: ignore[union-attr]
+                    counts[str(k)] += 1
+                seen += 1
+            fh.flush()
+            print(f"  {seen}/{len(todo)}  {dict(counts)}  {dict(status)}", flush=True)
+
+    # Rebuild the paths file from the FULL record set, not just this run's slice.
+    hits = []
+    with open(a.out, encoding="utf-8") as fh:
+        for line in fh:
+            try:
+                r = json.loads(line)
+            except Exception:  # noqa: S112
+                continue
+            if r.get("keys"):
+                hits.append(r["path"])
+    # Derive the paths file from --out so a trial run with a scratch --out cannot
+    # overwrite the shared list a full sweep produced.
+    paths_out = a.out.with_suffix(".txt")
+    paths_out.write_text("\n".join(sorted(set(hits))) + "\n", encoding="utf-8")
+    print(f"\npositives: {len(set(hits))} images")
+    for k, v in counts.most_common():
+        print(f"   {v:6d}  {k}")
+    print(f"statuses: {dict(status)}\npaths -> {paths_out}\nrecords -> {a.out}")
+
+
+if __name__ == "__main__":
+    main()

scripts/visible_recall_sample.py

@@ -0,0 +1,126 @@
+"""Build an UNBIASED random sample for measuring visible-mark RECALL.
+
+Every earlier labelling round sampled where detectors FIRED, so images that every
+detector missed were absent by construction and recall was unmeasurable. This round
+samples at random within a provenance class and shows the labeller the corners where
+a mark can physically be, so a MISSED mark is visible as such.
+
+Design decisions that matter:
+
+* SAMPLING FRAME is per provenance class, not the whole corpus. Recall is only
+  meaningful against a denominator where the mark CAN occur: TC260 carriers for the
+  ByteDance marks and the pill, Google-C2PA for the sparkle. Reporting one blended
+  recall over all uploads would mostly measure how often each vendor appears.
+* NATIVE RESOLUTION crops, never a downscaled whole image: a 220px preview destroys a
+  faint mark (measured in an earlier round), which would inflate the miss count with
+  the labeller's own blindness rather than the detector's.
+* BOTH corners per image (top-left pill, bottom-right wordmark/strip/sparkle), so one
+  pass adjudicates every registered mark instead of one mark per crop.
+* The detector's verdict is NOT shown and is not in the sheet order -- the manifest
+  holds it and must not be opened until labelling ends.
+"""
+
+from __future__ import annotations
+
+import csv
+import json
+import random
+import sys
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+import cv2
+import numpy as np
+
+if TYPE_CHECKING:
+    from numpy.typing import NDArray
+
+sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
+from remove_ai_watermarks.image_io import imread
+
+CELL_W = 300
+COLS, ROWS = 4, 3
+PER = COLS * ROWS
+
+
+def corner_strip(img: NDArray[Any]) -> NDArray[Any] | None:
+    """Top-left and bottom-right corners stacked, at (near) native scale.
+
+    Crop size is a fraction of the SHORT side because that is what marks scale with
+    (China's GB 45438-2025 sizes the mandated label off the shortest side).
+    """
+    h, w = img.shape[:2]
+    short = min(h, w)
+    cw, ch = int(short * 0.46), int(short * 0.17)
+    cw, ch = min(cw, w), min(ch, h)
+    tl = img[0:ch, 0:cw]
+    br = img[h - ch : h, w - cw : w]
+    strip = np.vstack([tl, np.full((6, cw, 3), 90, np.uint8), br])
+    s = min(1.0, CELL_W / strip.shape[1])  # never UPSCALE past native
+    if s < 1.0:
+        strip = cv2.resize(strip, (CELL_W, max(1, int(strip.shape[0] * s))), interpolation=cv2.INTER_AREA)
+    return strip
+
+
+def main() -> None:
+    scan = Path(sys.argv[1])
+    out = Path(sys.argv[2])
+    n_tc260 = int(sys.argv[3]) if len(sys.argv) > 3 else 160
+    n_google = int(sys.argv[4]) if len(sys.argv) > 4 else 80
+    out.mkdir(parents=True, exist_ok=True)
+
+    recs = [json.loads(line) for line in scan.open() if '"marks"' in line]
+    seen: dict[tuple, dict] = {}
+    for r in recs:  # exact-duplicate uploads share the whole NCC vector
+        seen.setdefault(tuple(r.get("shape", ())) + tuple(sorted((k, m["conf"]) for k, m in r["marks"].items())), r)
+    uniq = list(seen.values())
+
+    tc = [r for r in uniq if r["cls"] == "tc260"]
+    goog = [r for r in uniq if r["cls"] == "neg" and "Google" in r.get("platform", "")]
+    rng = random.Random(2026)  # noqa: S311 -- sampling, not cryptography
+    rng.shuffle(tc)
+    rng.shuffle(goog)
+    picked = [("tc260", r) for r in tc[:n_tc260]] + [("google", r) for r in goog[:n_google]]
+    rng.shuffle(picked)
+
+    manifest, cells = [], []
+    for cls, r in picked:
+        img = imread(r["path"])
+        if img is None:
+            continue
+        strip = corner_strip(img)
+        if strip is None:
+            continue
+        cells.append(strip)
+        manifest.append(
+            {
+                "idx": len(cells) - 1,
+                "uid": r["uid"],
+                "cls": cls,
+                "path": r["path"],
+                "fired": "|".join(sorted(k for k, m in r["marks"].items() if m["strict"])),
+                **{f"ncc_{k}": m["conf"] for k, m in r["marks"].items()},
+            }
+        )
+
+    cell_h = max(c.shape[0] for c in cells)
+    for si in range(0, len(cells), PER):
+        chunk = cells[si : si + PER]
+        sheet = np.full((ROWS * (cell_h + 26), COLS * (CELL_W + 8), 3), 40, np.uint8)
+        for i, c in enumerate(chunk):
+            rr, cc = divmod(i, COLS)
+            y0, x0 = rr * (cell_h + 26) + 22, cc * (CELL_W + 8) + 4
+            sheet[y0 : y0 + c.shape[0], x0 : x0 + c.shape[1]] = c
+            cv2.putText(sheet, f"{si + i:03d}", (x0, y0 - 6), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1)
+        cv2.imwrite(str(out / f"r{si // PER:02d}.png"), sheet)
+
+    with (out / "MANIFEST_DO_NOT_OPEN.csv").open("w", newline="") as fh:
+        w = csv.DictWriter(fh, fieldnames=list(manifest[0]))
+        w.writeheader()
+        w.writerows(manifest)
+    print(f"cells={len(cells)}  sheets={(len(cells) + PER - 1) // PER} -> {out}")
+    print("each cell = TOP-LEFT corner above, BOTTOM-RIGHT corner below, near native scale")
+
+
+if __name__ == "__main__":
+    main()

scripts/visible_removal_audit.py

@@ -0,0 +1,150 @@
+"""Audit visible-watermark removal over a local image corpus.
+
+For every image the registry detects a known visible mark in, run that mark's
+removal and re-detect on the output, recording before/after confidence and
+whether the detector still fires. Also bucket the detected-positive originals
+into per-mark dataset dirs so the visible-mark corpora are reproducible.
+
+Detector-clean after removal is necessary but, for the Doubao/Jimeng text marks,
+NOT sufficient (their NCC detector is fooled by a thin residual outline -- see
+CLAUDE.md). Treat a detector-clean Doubao/Jimeng as "detector passes"; visual
+residual is a separate check.
+
+Backend: for a REALISTIC quality audit pass ``--backend migan`` (the production
+fill), not ``cv2``. Removal SUCCESS (detector-clean) is backend-independent, so cv2
+is fine for a fast pass/fail sweep, but only migan/lama reflect the recovered-region
+quality a user actually gets. Run migan when validating the visible pipeline for real.
+
+Operates on gitignored data only (data/spaces/...); writes nothing tracked.
+
+    uv run python scripts/visible_removal_audit.py \
+        --corpus data/spaces/originals --out data/spaces/_visible_audit.csv \
+        --dataset-root data/spaces/_visible_datasets
+"""
+
+from __future__ import annotations
+
+import csv
+import logging
+import shutil
+from pathlib import Path
+
+import click
+
+from remove_ai_watermarks import image_io
+from remove_ai_watermarks.watermark_registry import detect_marks, get_mark
+
+log = logging.getLogger(__name__)
+
+_EXTS = {".png", ".jpg", ".jpeg", ".webp", ".avif", ".heic"}
+
+
+def _rel(p: Path, corpus: Path) -> str:
+    try:
+        return str(p.relative_to(corpus))
+    except ValueError:
+        return p.name
+
+
+@click.command()
+@click.option(
+    "--corpus", type=click.Path(exists=True, file_okay=False, path_type=Path), default=Path("data/spaces/originals")
+)
+@click.option("--out", type=click.Path(path_type=Path), default=Path("data/spaces/_visible_audit.csv"))
+@click.option("--dataset-root", type=click.Path(path_type=Path), default=Path("data/spaces/_visible_datasets"))
+@click.option(
+    "--paths-file",
+    type=click.Path(exists=True, path_type=Path),
+    default=None,
+    help="Audit only these paths (one per line), skipping the full rglob.",
+)
+@click.option("--limit", type=int, default=0, help="Scan at most N files (0 = all).")
+@click.option(
+    "--backend",
+    type=click.Choice(["auto", "cv2", "migan", "lama"]),
+    default="auto",
+    help="Fill backend for removal. Removal SUCCESS (detector-clean) is backend-independent; "
+    "cv2 is fastest for a bulk audit, migan/lama only change recovered-region quality.",
+)
+def main(corpus: Path, out: Path, dataset_root: Path, paths_file: Path | None, limit: int, backend: str) -> None:
+    logging.basicConfig(level=logging.WARNING, format="%(message)s")
+    if paths_file is not None:
+        files = [Path(s) for line in paths_file.read_text().splitlines() if (s := line.strip()) and Path(s).is_file()]
+    else:
+        files = sorted(p for p in corpus.rglob("*") if p.is_file() and p.suffix.lower() in _EXTS)
+    if limit:
+        files = files[:limit]
+    click.echo(f"Scanning {len(files)} files under {corpus} ...")
+
+    rows: list[dict[str, str]] = []
+    n_detected = 0
+    n_clean_after = 0
+    fails: list[tuple[str, str, float]] = []
+
+    with click.progressbar(files, label="audit") as bar:
+        for p in bar:
+            img = image_io.imread(p)
+            if img is None:
+                continue
+            for det in detect_marks(img, include_explicit=False):
+                if not det.detected:
+                    continue
+                n_detected += 1
+                mark = get_mark(det.key)
+                # Bucket the positive original into the per-mark dataset.
+                ddir = dataset_root / det.key
+                ddir.mkdir(parents=True, exist_ok=True)
+                if not (ddir / p.name).exists():
+                    shutil.copy2(p, ddir / p.name)
+                # Remove, then re-detect with the SAME mark's detector.
+                try:
+                    cleaned, _ = mark.remove(img, backend=backend)
+                    after = mark.detect(cleaned)
+                except Exception as exc:
+                    log.warning("remove failed on %s (%s): %s", p.name, det.key, exc)
+                    rows.append(
+                        {
+                            "path": _rel(p, corpus),
+                            "mark": det.key,
+                            "conf_before": f"{det.confidence:.3f}",
+                            "conf_after": "",
+                            "removed": "error",
+                        }
+                    )
+                    continue
+                removed = not after.detected
+                n_clean_after += int(removed)
+                if not removed:
+                    fails.append((_rel(p, corpus), det.key, after.confidence))
+                rows.append(
+                    {
+                        "path": _rel(p, corpus),
+                        "mark": det.key,
+                        "conf_before": f"{det.confidence:.3f}",
+                        "conf_after": f"{after.confidence:.3f}",
+                        "removed": str(removed),
+                    }
+                )
+
+    out.parent.mkdir(parents=True, exist_ok=True)
+    with out.open("w", newline="") as f:
+        w = csv.DictWriter(f, fieldnames=["path", "mark", "conf_before", "conf_after", "removed"])
+        w.writeheader()
+        w.writerows(rows)
+
+    by_mark: dict[str, list[bool]] = {}
+    for r in rows:
+        if r["removed"] in ("True", "False"):
+            by_mark.setdefault(r["mark"], []).append(r["removed"] == "True")
+    click.echo(f"\nDetected positives: {n_detected}; detector-clean after removal: {n_clean_after}")
+    for k, v in sorted(by_mark.items()):
+        click.echo(f"  {k:8} removed {sum(v)}/{len(v)} ({100 * sum(v) // max(1, len(v))}%)")
+    if fails:
+        click.echo(f"\nDetector still fires after removal ({len(fails)}):")
+        for path, key, conf in fails[:30]:
+            click.echo(f"  {key:8} {conf:.3f}  {path}")
+    click.echo(f"\nReport: {out}  |  Datasets: {dataset_root}/<mark>/")
+
+
+if __name__ == "__main__":
+    main()

scripts/visible_sheets.py

@@ -0,0 +1,101 @@
+"""Build BLIND contact sheets for hand-labelling relaxation additions.
+
+Crops are centered on the DETECTED REGION (not the corner), padded by ~0.9x the mark
+size, and resized to 240px with INTER_NEAREST -- a downscaled preview destroys a faint
+mark, so nothing here may smooth. The manifest is written to a separate file that must
+NOT be read until labelling is finished.
+
+Each sheet mixes three strata in shuffled order:
+  add    - the relaxation additions whose precision we are measuring
+  pos    - strict-consistent detections (a mark is really there): labeller sensitivity
+  clean  - verified-clean negatives (no mark can be there): labeller specificity
+The two control strata are what make a low measured precision trustworthy.
+"""
+
+import csv
+import json
+import random
+import sys
+from pathlib import Path
+from typing import Any
+
+import cv2
+import numpy as np
+from numpy.typing import NDArray
+
+sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
+from remove_ai_watermarks import watermark_registry as wr
+from remove_ai_watermarks.image_io import imread
+
+CELL = 240
+COLS, ROWS = 6, 3
+PER = COLS * ROWS
+
+
+def region_for(path: str, key: str) -> tuple[int, int, int, int] | None:
+    """Re-detect to recover the mark bbox (Candidate carries no region)."""
+    img = imread(path)
+    if img is None:
+        return None
+    mark = next(m for m in wr._REGISTRY if m.key == key)
+    d = mark.detect(img, provenance=True)
+    return d.region if d.region else None
+
+
+def crop(path: str, region: tuple[int, int, int, int] | None, pad_factor: float = 0.9) -> NDArray[Any] | None:
+    img = imread(path)
+    if img is None:
+        return None
+    h, w = img.shape[:2]
+    if region:
+        x, y, rw, rh = region
+    else:
+        return None
+    px, py = int(rw * pad_factor), int(rh * pad_factor)
+    x0, y0 = max(0, x - px), max(0, y - py)
+    x1, y1 = min(w, x + rw + px), min(h, y + rh + py)
+    c = img[y0:y1, x0:x1]
+    if c.size == 0:
+        return None
+    s = CELL / max(c.shape[0], c.shape[1])
+    return cv2.resize(c, (max(1, int(c.shape[1] * s)), max(1, int(c.shape[0] * s))), interpolation=cv2.INTER_NEAREST)
+
+
+def main() -> None:
+    with open(sys.argv[1]) as fh:
+        items = json.load(fh)  # [{uid,path,key,stratum,conf}]
+    outdir = Path(sys.argv[2])
+    outdir.mkdir(parents=True, exist_ok=True)
+    random.Random(1234).shuffle(items)  # noqa: S311 -- sheet ordering, not cryptography
+
+    manifest = []
+    cells = []
+    for it in items:
+        reg = region_for(it["path"], it["key"])
+        c = crop(it["path"], reg)
+        if c is None:
+            continue
+        cells.append((it, c))
+
+    for si in range(0, len(cells), PER):
+        chunk = cells[si : si + PER]
+        sheet = np.full((ROWS * (CELL + 26), COLS * (CELL + 8), 3), 40, np.uint8)
+        for i, (it, c) in enumerate(chunk):
+            r, col = divmod(i, COLS)
+            y0 = r * (CELL + 26) + 22
+            x0 = col * (CELL + 8) + 4
+            sheet[y0 : y0 + c.shape[0], x0 : x0 + c.shape[1]] = c
+            label = f"{si + i:04d}"  # index ONLY -- no stratum, no confidence
+            cv2.putText(sheet, label, (x0, y0 - 6), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1)
+            manifest.append({"idx": si + i, **it})
+        cv2.imwrite(str(outdir / f"sheet_{si // PER:03d}.png"), sheet)
+
+    with open(outdir / "MANIFEST_DO_NOT_OPEN.csv", "w", newline="") as fh:
+        w = csv.DictWriter(fh, fieldnames=list(manifest[0]))
+        w.writeheader()
+        w.writerows(manifest)
+    print(f"sheets={(len(cells) + PER - 1) // PER} cells={len(cells)} -> {outdir}")
+
+
+if __name__ == "__main__":
+    main()

src/remove_ai_watermarks/__init__.py

@@ -1,3 +1,43 @@
-"""Remove-AI-Watermarks: Unified tool for removing visible and invisible AI watermarks."""
+"""Remove-AI-Watermarks: Unified tool for removing visible and invisible AI watermarks.
 
-__version__ = "0.7.0"
+High-level API (lazy, so ``import remove_ai_watermarks`` stays cheap)::
+
+    import remove_ai_watermarks as raiw
+    raiw.remove_visible("in.png", "out.png")            # clean a file (provenance auto)
+    result, removed = raiw.remove_visible(bgr_array)    # array -> array
+    raiw.visible_provenance("in.png")                   # -> frozenset of confirmed vendors
+
+For a provenance verdict use the ``identify`` submodule::
+
+    from remove_ai_watermarks.identify import identify
+    report = identify("in.png")
+"""
+
+import os as _os
+import warnings as _warnings
+from typing import TYPE_CHECKING
+
+# transformers prints a noisy deprecation for the Siglip2ImageProcessorFast
+# alias when it is imported (by the optional GPU/ML path). Silence it before
+# any submodule pulls transformers in, so the CLI startup stays quiet. Uses
+# setdefault so a user-set TRANSFORMERS_VERBOSITY still wins.
+_os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
+_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
+
+
+__version__ = "0.16.0"
+
+__all__ = ["__version__", "remove_visible", "visible_provenance"]
+
+if TYPE_CHECKING:
+    from remove_ai_watermarks.api import remove_visible, visible_provenance
+
+
+def __getattr__(name: str) -> object:
+    """Lazily resolve the high-level API (PEP 562), so the heavy imports (cv2, the
+    metadata/identify stack) load only when a caller actually reaches for them."""
+    if name in ("remove_visible", "visible_provenance"):
+        from remove_ai_watermarks import api
+
+        return getattr(api, name)
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

src/remove_ai_watermarks/_text_mark_engine.py

@@ -0,0 +1,615 @@
+"""Shared base for the visible text-mark detectors/localizers (localize -> fill).
+
+The Doubao "豆包AI生成", Jimeng "★ 即梦AI", and Samsung "✦ Contenuti generati
+dall'AI" marks are the SAME algorithm: anchor a bottom-corner box by geometry
+relative to the image's SHORT side, extract the light low-saturation glyph candidate (white top-hat), detect
+by matching the bundled alpha-glyph silhouette via ``TM_CCOEFF_NORMED``, and build a
+removal MASK from the glyph blob's bounding box (:meth:`footprint_mask`) for the
+shared fill (region_eraser). The mask is template-FREE -- the top-hat glyph bbox, not
+a fixed alpha-template placement -- so a re-rendered or differently-placed mark (e.g.
+a non-Italian Samsung string) is still masked. The old reverse-alpha pixel recovery
+(``original = (wm - a*logo)/(1-a)``) is gone.
+
+They differ ONLY in a bounded set of tuned values captured by :class:`TextMarkConfig`:
+the constants, the bundled silhouette asset, the corner (Doubao/Jimeng bottom-right,
+Samsung bottom-left), and a few structural knobs. Each engine module is a thin
+:class:`TextMarkEngine` subclass plus the test-facing module constants/helpers.
+
+Gemini stays a SEPARATE engine (``gemini_engine``): its multi-size sparkle model is
+genuinely different, not a tuned variant of this one.
+"""
+
+# cv2/numpy boundary: third-party libs ship no usable element types; relax the
+# unknown-type rules for this file only.
+# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false, reportOptionalMemberAccess=false, reportOptionalCall=false, reportOptionalSubscript=false, reportOptionalOperand=false, reportAttributeAccessIssue=false, reportPrivateImportUsage=false, reportPrivateUsage=false, reportInvalidTypeForm=false, reportConstantRedefinition=false, reportUnnecessaryComparison=false
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+from pathlib import Path
+from typing import TYPE_CHECKING, Any, Literal
+
+import cv2
+import numpy as np
+
+from remove_ai_watermarks import image_io
+
+if TYPE_CHECKING:
+    from numpy.typing import NDArray
+
+logger = logging.getLogger(__name__)
+
+# Minimum image short side (px) for text-mark DETECTION. Below this the glyph
+# template degrades to the ``min_gw`` floor (~8 px) and TM_CCOEFF_NORMED on a few
+# pixels is noise, so an unrelated small geometric shape can spuriously correlate
+# with the CJK silhouette (2026-06-26 FP: a 48x48 app icon -- a blue chevron --
+# scored Doubao 0.41 / Jimeng 0.47, both above their thresholds). The FP is purely
+# a small-size artifact: the same icon upscaled collapses to ~0.06-0.10 NCC at 256
+# px and above. A real AI-generation text label is stamped on a full-resolution
+# render (the captured samples are 1086-2048 px wide), so 200 px sits far below any
+# genuine mark while killing the icon/thumbnail noise band (<=96 px). Detection is
+# skipped (verdict stays "unknown", the safe default) rather than risk a false
+# positive; removal is gated on detection, so it is suppressed too.
+_MIN_DETECT_SHORT_SIDE = 200
+
+# Provenance-confirmed NCC relaxation. When external metadata already confirms the
+# vendor (so the mark is present with high prior), a faint or slightly re-rendered
+# glyph that scores just below the standard NCC gate is still trusted. The relaxed
+# gate is ``detect_ncc_threshold * provenance_ncc_factor``; the coverage gate still
+# applies on top.
+#
+# This used to be ONE shared 0.7 for every text mark. Measured 2026-07-18 on the
+# `auto` path (the default -- no flag, driven by TC260 metadata), it turned out to
+# mean two completely different things per mark. Blind hand-label of the ADDITIONS
+# (accepted with provenance, rejected without) over 4417 unique TC260 carriers,
+# two-sided control (labeller sensitivity 100%/96%, specificity 100%/100%):
+#
+#   mark     band            precision   95% CI     n
+#   doubao   whole arm             76%    61-87%    42
+#            [0.280,0.340)         58%    36-77%    19
+#            [0.340,0.400)         91%    73-98%    23
+#   jimeng   whole arm             17%    10-27%    82
+#            [0.315,0.383)         12%     6-22%    68
+#            [0.383,0.450)         43%    21-67%    14
+#
+# Doubao stays at 0.70: both its bands return more true marks than false fills, so
+# tightening would cost 11 genuine recoveries to prevent 8 false ones.
+#
+# Jimeng moves to 0.85. Its relaxed detector does not key on the "★ 即梦AI" wordmark
+# any more -- it keys on "some text in the bottom-right corner": of 68 false
+# additions, 33 were DOUBAO marks and 17 were other vendors' AI labels (千问, 百度,
+# 星绘, 抖音). 45 of those 68 fill a corner nothing else would touch (the other 23
+# are harmless -- doubao fires strictly there and fills the same box anyway). At
+# 0.85 the [0.315,0.383) band is dropped: 8 genuine recoveries lost, 60 false fills
+# prevented (7.5:1). A false fill is the worse error -- it destroys pixels AND makes
+# the caller report a removal that did not happen, while a miss leaves the image
+# untouched.
+#
+# NOTE: 0.85 is a patch on a detector problem, not a fix. Jimeng's silhouette is not
+# discriminative against Doubao's (same corner, same script, both ByteDance), and no
+# threshold repairs that -- it needs a better detection silhouette.
+_DEFAULT_PROVENANCE_NCC_FACTOR = 0.7
+
+
+@dataclass(frozen=True)
+class TextMarkConfig:
+    """All per-mark tuning for a text-mark detector/localizer."""
+
+    name: str  # short label for log lines (e.g. "Doubao")
+    asset_name: str  # bundled alpha PNG under assets/ (e.g. "doubao_alpha.png")
+    corner: Literal["br", "bl"]  # bottom-right (Doubao/Jimeng) or bottom-left (Samsung)
+    margin_floor: int  # min margin in px for locate (4 for br marks, 2 for Samsung)
+    # locate geometry (fraction of scale_base -- see scale_base())
+    width_frac: float
+    height_frac: float
+    margin_x_frac: float  # right margin (br) or left margin (bl)
+    margin_bottom_frac: float
+    # glyph appearance
+    max_saturation: float
+    logo_min_luma: float
+    tophat_delta: float
+    morph_open_size: int  # MORPH_OPEN kernel side (5 for br marks, 3 for Samsung)
+    # detection
+    detect_min_coverage: float
+    detect_ncc_threshold: float
+    # alpha-map glyph geometry (fraction of scale_base) emitted by
+    # scripts/visible_alpha_solve.py, sizing the detection silhouette for
+    # template_match_score
+    alpha_width_frac: float
+    alpha_height_frac: float
+    min_gw: int  # minimum glyph width for the template match (8 br, 16 Samsung)
+    # Asset names of RIVAL marks that occupy the same corner and can therefore be
+    # scored against the same glyph blob. Detection becomes COMPETITIVE: this mark's
+    # template must beat every rival's by `rival_margin`. See _rival_margin_ok.
+    # Detection front-end. "binary" thresholds the top-hat into a glyph blob and
+    # correlates a binary silhouette against it; "tophat" correlates the CONTINUOUS
+    # top-hat response against a soft template and never binarizes. See
+    # TextMarkEngine.tophat_response for the measurement that motivated the split.
+    detect_frontend: Literal["binary", "tophat"] = "binary"
+    # Gaussian sigma applied to the template in the "tophat" front-end (0 = none).
+    template_blur: float = 0.0
+    # Which image dimension the mark's size and margins scale with. VENDOR-SPECIFIC,
+    # measured, not assumed -- see TextMarkEngine.scale_base. "short" = min(h, w), "width" = w.
+    scale_basis: Literal["short", "width"] = "width"
+    # Scale rungs ``_tophat_best`` sweeps (the detection comb). PER-MARK: a vendor
+    # whose stamp sizes do not land on the shared 3-rung comb carries its own ladder
+    # (measured for 千问, whose marks sit in two size modes ~1.6x apart -- one fraction
+    # on 3 rungs covers only ~75% of them). Densifying the SHARED ladder for everyone
+    # was measured and rejected (false fire 2.52% -> 3.05%; see docs/verification-plan.md
+    # B2), so the default stays the shipped 3 rungs and a deviation must be calibrated
+    # per mark on real positives, never ported.
+    ladder: tuple[float, ...] = (0.8, 1.0, 1.25)
+    rivals: tuple[str, ...] = ()
+    rival_margin: float = 0.10
+    # Multiplier applied to detect_ncc_threshold when provenance confirms the vendor.
+    # Per-mark, NOT shared: see _DEFAULT_PROVENANCE_NCC_FACTOR for the measured
+    # precision that forced the split. Last field so it can carry a default.
+    provenance_ncc_factor: float = _DEFAULT_PROVENANCE_NCC_FACTOR
+
+
+@dataclass
+class TextMarkLocation:
+    """Located watermark box, in absolute pixel coordinates."""
+
+    x: int
+    y: int
+    w: int
+    h: int
+    is_fallback: bool = True  # geometry anchor (no template match) -> always True for now
+
+    @property
+    def bbox(self) -> tuple[int, int, int, int]:
+        return self.x, self.y, self.w, self.h
+
+
+@dataclass
+class TextMarkDetection:
+    """Result of visible text-mark detection."""
+
+    detected: bool = False
+    confidence: float = 0.0
+    region: tuple[int, int, int, int] = (0, 0, 0, 0)
+    coverage: float = 0.0  # fraction of the box occupied by glyph pixels
+
+
+# Alpha / silhouette templates, cached per asset name (the originals cached per
+# module global; this keys by asset so the three engines share the loader without
+# re-reading). Only SUCCESSFUL loads are cached, so a missing asset is retried.
+_alpha_cache: dict[str, NDArray[Any]] = {}
+_silhouette_cache: dict[str, NDArray[Any]] = {}
+
+
+def load_alpha_template(asset_name: str) -> NDArray[Any] | None:
+    """Lazily load the bundled alpha template (float [0,1]) for ``asset_name``, or None."""
+    cached = _alpha_cache.get(asset_name)
+    if cached is not None:
+        return cached
+    path = Path(__file__).parent / "assets" / asset_name
+    img = image_io.imread(str(path), cv2.IMREAD_GRAYSCALE)
+    if img is None:
+        return None
+    _alpha_cache[asset_name] = img.astype(np.float32) / 255.0
+    return _alpha_cache[asset_name]
+
+
+def glyph_silhouette(asset_name: str) -> NDArray[Any] | None:
+    """Binary glyph silhouette (255 = glyph) from the bundled alpha map, or None."""
+    cached = _silhouette_cache.get(asset_name)
+    if cached is not None:
+        return cached
+    at = load_alpha_template(asset_name)
+    if at is None:
+        return None
+    _silhouette_cache[asset_name] = (at > 0.15).astype(np.uint8) * 255
+    return _silhouette_cache[asset_name]
+
+
+_RIVAL_MODULES = {
+    "doubao_alpha.png": "remove_ai_watermarks.doubao_engine",
+    "jimeng_alpha.png": "remove_ai_watermarks.jimeng_engine",
+    "samsung_alpha.png": "remove_ai_watermarks.samsung_engine",
+}
+
+
+def _rival_config(asset_name: str, fallback: TextMarkConfig) -> TextMarkConfig:
+    """The rival mark's own config, for scoring its template on a shared blob.
+
+    Looked up LAZILY by asset name: a rival's template geometry
+    (``alpha_*_frac`` / ``min_gw``) is its own, and scoring it with this mark's
+    geometry would compare a correctly-sized template against a mis-sized one and
+    hand the margin a free win. Lazy because the engine modules import this one.
+    """
+    mod_path = _RIVAL_MODULES.get(asset_name)
+    if mod_path is None:
+        return fallback
+    from importlib import import_module
+
+    try:
+        return import_module(mod_path)._CONFIG
+    except Exception:  # a missing/renamed engine must not break detection
+        logger.debug("rival config %s unavailable; skipping its margin check.", asset_name)
+        return fallback
+
+
+def template_match_score(box_mask: NDArray[Any], scale_base: int, config: TextMarkConfig) -> float:
+    """Zero-mean normalized correlation of the alpha-template glyph silhouette
+    (scaled to the mark's expected size) against the candidate ``box_mask``.
+
+    ``TM_CCOEFF_NORMED`` keys on glyph SHAPE, not coverage, so a dense textured
+    corner does not score highly -- only the actual glyph shape does.
+
+    ``scale_base`` is the mark's own scaling dimension (:meth:`TextMarkEngine.scale_base`),
+    not always the width: sizing the template on the wrong basis stretches it by the
+    aspect ratio on landscape inputs and the correlation collapses.
+    """
+    sil = glyph_silhouette(config.asset_name)
+    if sil is None or box_mask.size == 0:
+        return 0.0
+    gw = min(box_mask.shape[1] - 1, max(config.min_gw, int(config.alpha_width_frac * scale_base)))
+    gh = min(box_mask.shape[0] - 1, max(4, int(config.alpha_height_frac * scale_base)))
+    if gw < config.min_gw or gh < 4:
+        return 0.0
+    template = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_NEAREST)
+    return float(cv2.matchTemplate(box_mask, template, cv2.TM_CCOEFF_NORMED).max())
+
+
+class TextMarkEngine:
+    """Visible text-mark detector/localizer (locate -> mask -> detect; mask feeds the fill)."""
+
+    def __init__(self, config: TextMarkConfig) -> None:
+        self.config = config
+
+    # ── Templates (delegate to the asset-keyed module cache) ────────────
+
+    def _alpha_template(self) -> NDArray[Any] | None:
+        return load_alpha_template(self.config.asset_name)
+
+    def _glyph_silhouette(self) -> NDArray[Any] | None:
+        return glyph_silhouette(self.config.asset_name)
+
+    def _template_match_score(self, box_mask: NDArray[Any], scale_base: int) -> float:
+        return template_match_score(box_mask, scale_base, self.config)
+
+    def _rival_margin_ok(self, score: float, box_mask: NDArray[Any], scale_base: int) -> bool:
+        """Whether this mark's template beats every same-corner RIVAL's on the SAME blob.
+
+        Detection was purely ABSOLUTE -- each engine scored its own template and
+        compared against its own threshold, so nothing ever asked the discriminative
+        question "does this blob look more like the neighbour's mark than like mine?".
+        Two marks sharing a corner and a script (Doubao "豆包AI生成" and Jimeng
+        "★ 即梦AI", both bottom-right, both near-white CJK) survive binarization into
+        very similar blobs, so an absolute gate cannot separate them -- and under the
+        provenance relaxation it stopped trying: 33 of jimeng's 68 false additions
+        were Doubao marks (corpus-measured 2026-07-18).
+
+        Measured separability on hand-labelled examples, scoring BOTH templates
+        against the same glyph blob (n=40 jimeng / 75 doubao / 20 other-vendor labels
+        / 89 clean):
+
+            feature                       separability   (0.5 = useless, 1.0 = perfect)
+            absolute ncc_jimeng                   0.96
+            ncc_jimeng MINUS ncc_doubao           0.99
+
+        At a 0.10 margin: real Jimeng wordmarks pass 100%, Doubao strips 8%, other
+        vendors' AI labels (千问/百度/星绘/抖音) 55%, no-mark corners 12%. Because the
+        real marks pass at 100%, this costs NO recall -- it is a pure precision gain,
+        unlike raising the threshold, which trades recall away.
+
+        Marks with no same-corner rival declare `rivals=()` and are unaffected.
+        """
+        c = self.config
+        if not c.rivals:
+            return True
+        for rival_asset in c.rivals:
+            rival = _rival_config(rival_asset, c)
+            if score - template_match_score(box_mask, scale_base, rival) < c.rival_margin:
+                logger.debug("%s detect: loses the %s rival margin; rejecting.", c.name, rival_asset)
+                return False
+        return True
+
+    # ── Locate ──────────────────────────────────────────────────────────
+
+    def tophat_response(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any] | None:
+        """The CONTINUOUS white top-hat in the located box -- the glyph signal, unbinarized.
+
+        :meth:`extract_mask` thresholds this same response into a 0/255 glyph blob. That
+        is fine for a mark stamped bold and opaque, and destructive for a faint one: a
+        thin translucent overlay shatters into specks under the threshold, and no
+        template can match a blob that is not there.
+
+        Measured 2026-07-18 on hand-verified corpus positives (40 doubao, 14 千问, 60
+        verified-clean), scoring each mark with its own template:
+
+            front-end   doubao   clean neg   AUC doubao/neg
+            binary       0.723       ~0.12             --
+            tophat       0.781        0.122          1.00
+
+        The gates that were hard cuts in the binary path (saturation, absolute luma)
+        become WEIGHTS here, so a faint stroke contributes in proportion to its strength
+        instead of being dropped at a threshold. The response is max-normalized, which
+        makes the score contrast-invariant -- the point of the exercise.
+
+        Kept per-mark (``detect_frontend``) rather than switched globally, because a
+        front-end change must be measured per mark before it ships.
+        """
+        c = self.config
+        x, y, bw, bh = loc.bbox
+        if bh < 16 or bw < 16:
+            return None
+        roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
+        luma = roi.mean(axis=2)
+        sat = roi.max(axis=2) - roi.min(axis=2)
+        sigma = max(4.0, bh * 0.4)
+        tophat = luma - cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma)
+        resp = np.clip(tophat, 0, None) * (sat < c.max_saturation)
+        peak = float(resp.max())
+        if peak <= 1e-6:
+            return None
+        return (resp / peak * 255).astype(np.uint8)
+
+    def _tophat_best(
+        self, image: NDArray[Any], loc: TextMarkLocation
+    ) -> tuple[float, tuple[int, int, int, int] | None]:
+        """Best TM_CCOEFF_NORMED of a soft template against the continuous response, and
+        the ROI-local box (x0, y0, x1, y1) where that best match sits.
+
+        Sweeps the mark's scale ladder: the nominal glyph size is derived from the mark's
+        geometry, but a vendor re-rasterization shifts it by a few percent and the
+        continuous response is sharp enough that an exact-size template would miss. The
+        ladder is per-mark (``TextMarkConfig.ladder``), defaulting to the shipped 3 rungs.
+
+        Detection and the removal mask BOTH read this one method -- the score gates
+        detection, the box bounds the fill. Sharing it is deliberate: the standing rule is
+        that detection and the mask use the same front-end, and the way that rule was last
+        broken was a drift between two separate implementations. One method makes the drift
+        impossible instead of merely discouraged.
+        """
+        c = self.config
+        resp = self.tophat_response(image, loc)
+        sil = self._glyph_silhouette()
+        if resp is None or sil is None:
+            return (0.0, None)
+        base = self.scale_base(image)
+        best_score = 0.0
+        best_box: tuple[int, int, int, int] | None = None
+        for scale in c.ladder:
+            gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
+            gh = max(4, int(c.alpha_height_frac * base * scale))
+            if gw >= resp.shape[1] or gh >= resp.shape[0]:
+                continue
+            tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32)
+            if c.template_blur > 0:
+                tmpl = cv2.GaussianBlur(tmpl, (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur)
+            result = cv2.matchTemplate(resp, tmpl.astype(np.uint8), cv2.TM_CCOEFF_NORMED)
+            _, score, _, top_left = cv2.minMaxLoc(result)
+            if score > best_score:
+                tx, ty = int(top_left[0]), int(top_left[1])
+                best_score, best_box = float(score), (tx, ty, tx + gw - 1, ty + gh - 1)
+        return (best_score, best_box)
+
+    def _tophat_score(self, image: NDArray[Any], loc: TextMarkLocation) -> float:
+        """The detection score alone -- the box the removal mask needs is discarded here."""
+        return self._tophat_best(image, loc)[0]
+
+    def scale_base(self, image: NDArray[Any]) -> int:
+        """The image dimension this mark's geometry scales with.
+
+        Per-mark, and MEASURED -- a single shared basis is wrong. The tuned fractions
+        were all calibrated on PORTRAIT captures, where width and short side coincide,
+        so the basis was never exercised until landscape inputs were measured.
+
+        Corpus-measured 2026-07-18 (2572 unique TC260 carriers; the harness is
+        `scripts/visible_eval.py`). Before any fix, doubao detection by aspect ratio:
+        portrait 60%, square 41%, **landscape 0% (0 of 435)** -- the width-scaled box
+        is inflated by the aspect ratio on a wide image and the glyph never lands in
+        it. Re-running the previously-undetected set with a short-side basis recovered
+        **56% of landscape** images (12% square, 4% portrait).
+
+        But the same switch took JIMENG's labelled landscape positives from 13/13 to
+        0/13: its wordmark tracks the WIDTH. Both marks are ByteDance and share a
+        corner, and they still scale differently -- so this is a per-mark measurement,
+        not a house rule to generalize. Samsung keeps ``width`` because there is no
+        corpus evidence either way (1 addition corpus-wide) and an unmeasured change
+        is not an improvement.
+
+        China's GB 45438-2025 clause 5.2(e) mandates glyph height >= 5% of "the
+        shortest side" for CN marks, which is why a short-side basis is the natural
+        prior -- but Jimeng's measured behaviour overrides the prior, and measurement
+        wins over the standard's wording.
+        """
+        return min(image.shape[:2]) if self.config.scale_basis == "short" else image.shape[1]
+
+    def locate(self, image: NDArray[Any]) -> TextMarkLocation:
+        """Anchor the watermark box in the configured corner, scaled by ``scale_basis``.
+
+        Every fraction is taken against ``scale_base(image)`` -- see
+        :data:`TextMarkConfig.scale_basis`, which is per-mark because the vendors
+        genuinely differ.
+        """
+        c = self.config
+        h, w = image.shape[:2]
+        base = self.scale_base(image)
+        wm_w = max(40, int(base * c.width_frac))
+        wm_h = max(16, int(base * c.height_frac))
+        margin_x = max(c.margin_floor, int(base * c.margin_x_frac))
+        margin_b = max(c.margin_floor, int(base * c.margin_bottom_frac))
+        x = max(0, w - margin_x - wm_w) if c.corner == "br" else min(margin_x, max(0, w - wm_w))
+        y = max(0, h - margin_b - wm_h)
+        wm_w = min(wm_w, w - x)
+        wm_h = min(wm_h, h - y)
+        return TextMarkLocation(x=x, y=y, w=wm_w, h=wm_h, is_fallback=True)
+
+    # ── Mask ────────────────────────────────────────────────────────────
+
+    def extract_mask(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any]:
+        """Build a box-sized uint8 mask (255 = watermark glyph) for ``loc``.
+
+        Returns just the glyph mask of the located box (shape ``(loc.h, loc.w)``),
+        not a full-frame array: every caller immediately crops to ``loc.bbox``, so
+        allocating a full ``(h, w)`` mask and embedding the box was O(image) work
+        and memory for an O(box) result -- a wasted full-frame uint8 allocation on
+        each detect (~12 MB on a 12 MP frame, recomputed per text-mark detector on
+        the memory-tight identify path). The box mask is byte-identical to the old
+        full-frame mask cropped to ``loc.bbox``.
+
+        Polarity-aware: the mark is a light, low-saturation gray rendered brighter
+        than the local background (white top-hat), so a white-paper document is left
+        untouched (nothing brighter than its surroundings is masked there).
+        """
+        c = self.config
+        x, y, bw, bh = loc.bbox
+        # A degenerate ROI (a sliver from an extremely wide/short image) cannot hold
+        # the mark and would feed cv2's GaussianBlur/morphology a ~1-px-tall array,
+        # which can fault native code on some platforms. Skip the cv2 pipeline.
+        if bh < 16 or bw < 16:
+            return np.zeros((bh, bw), np.uint8)
+        # Normalize the ROI to 3-channel BGR (grayscale / BGRA would break axis=2).
+        roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
+
+        luma = roi.mean(axis=2)
+        sat = roi.max(axis=2) - roi.min(axis=2)
+        grayish = sat < c.max_saturation
+
+        # Local background model: a strong Gaussian blur (sigma ~ box height); the
+        # white top-hat (luma - local_bg) lights up bright thin strokes regardless
+        # of the absolute background level.
+        sigma = max(4.0, bh * 0.4)
+        local_bg = cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma)
+        tophat = luma - local_bg
+
+        cand = grayish & (tophat > c.tophat_delta) & (luma > c.logo_min_luma)
+        glyph = cand.astype(np.uint8) * 255
+        glyph = cv2.morphologyEx(glyph, cv2.MORPH_CLOSE, np.ones((5, 5), np.uint8))
+        k = c.morph_open_size
+        return cv2.morphologyEx(glyph, cv2.MORPH_OPEN, np.ones((k, k), np.uint8))
+
+    # ── Detect ──────────────────────────────────────────────────────────
+
+    def detect(self, image: NDArray[Any], *, provenance: bool = False) -> TextMarkDetection:
+        """Detect the mark by matching the alpha-template glyph silhouette against
+        the corner candidate (``TM_CCOEFF_NORMED``); keys on glyph SHAPE, not coverage.
+
+        ``provenance`` signals that external metadata already confirms this vendor
+        (China-AIGC / byteimg for Doubao/Jimeng, ``samsung_genai`` for Samsung); the
+        NCC gate exists to keep a corner texture on an UNRELATED image from matching
+        the glyph silhouette, so when provenance confirms the vendor it is relaxed by
+        the mark's own ``provenance_ncc_factor`` to recover a faint or slightly
+        re-rendered mark (per-mark, not shared -- see _DEFAULT_PROVENANCE_NCC_FACTOR).
+        """
+        c = self.config
+        det = TextMarkDetection()
+        if image is None or image.size == 0:
+            return det
+        # Guard against the small-image NCC-noise false positive (see
+        # _MIN_DETECT_SHORT_SIDE): an icon/thumbnail is too small to carry a real
+        # text label, and the degraded few-pixel template spuriously correlates.
+        if min(image.shape[:2]) < _MIN_DETECT_SHORT_SIDE:
+            logger.debug(
+                "%s detect: image short side %d < %d; too small to carry the mark, skipping.",
+                c.name,
+                min(image.shape[:2]),
+                _MIN_DETECT_SHORT_SIDE,
+            )
+            return det
+        loc = self.locate(image)
+        box = self.extract_mask(image, loc)  # box-sized mask (== old full-frame cropped to bbox)
+        _x, _y, bw, bh = loc.bbox
+        coverage = float((box > 0).sum()) / float(max(1, bw * bh))
+        det.region = loc.bbox
+        det.coverage = coverage
+        if c.detect_frontend == "tophat":
+            # The continuous front-end does not depend on the binarized blob, so the
+            # coverage gate (a blob-area heuristic) does not apply to it.
+            score = self._tophat_score(image, loc)
+            threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
+            det.confidence = score
+            det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
+            logger.debug("%s detect (tophat): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
+            return det
+        if coverage >= c.detect_min_coverage:
+            score = self._template_match_score(box, self.scale_base(image))
+            threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
+            det.confidence = score
+            det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
+            logger.debug(
+                "%s detect: coverage=%.3f ncc=%.2f thr=%.2f detected=%s",
+                c.name,
+                coverage,
+                score,
+                threshold,
+                det.detected,
+            )
+        return det
+
+    # ── Inpaint footprint (for the inpaint-fallback removal path) ────────
+
+    # Minimum glyph pixels for a template-free footprint. Below this the corner has
+    # no real wordmark (a few top-hat specks), so without ``force`` there is nothing
+    # to mask. A real strip covers hundreds of pixels.
+    _MIN_GLYPH_PIXELS = 20
+
+    def footprint_mask(
+        self, image: NDArray[Any], *, force: bool = False, dilate: int | None = None
+    ) -> NDArray[Any] | None:
+        """Full-frame uint8 mask (255 = mark) of the mark footprint, for the shared
+        fill removal path (cv2 / MI-GAN / LaMa), or None if no glyph is found.
+
+        Template-FREE: localize the glyph blob with the top-hat :meth:`extract_mask`,
+        take its bounding box in the corner, and fill that box solid (plus a small
+        margin + dilation). Filling the enclosing rectangle -- not the sparse glyph
+        strokes -- is what makes it robust: the top-hat under-segments individual
+        strokes (which used to leave a "三包"-style residual ghost when the strokes
+        themselves were the mask), but the inpaint reconstructs the whole wordmark
+        rectangle from its surroundings, so a stroke missed by the top-hat is still
+        covered. This drops the fixed alpha-template dependency, so a re-rendered or
+        differently-localized mark (e.g. a non-Italian Samsung string) is still masked.
+
+        With ``force`` and no glyph found, falls back to the whole geometry box (the
+        ``--no-detect`` path). The caller gates on detection.
+        """
+        if image is None or image.size == 0:
+            return None  # guard before to_bgr (cvtColor raises on an empty Mat); mirror detect()
+        image = image_io.to_bgr(image)
+        h, w = image.shape[:2]
+        if h < 32 or w < 64:
+            return None
+        loc = self.locate(image)
+        bx, by, bw, bh = loc.bbox
+        glyph = self.extract_mask(image, loc)  # box-sized, 255 = glyph
+        ys, xs = np.where(glyph > 0)
+        box: tuple[int, int, int, int] | None = None
+        if xs.size >= self._MIN_GLYPH_PIXELS:
+            box = (int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max()))
+        elif self.config.detect_frontend == "tophat" and self.detect(image).detected:
+            # A mark found only by the CONTINUOUS front-end has no binary glyph blob to
+            # bound, so the mask came back empty and removal was a silent no-op while
+            # `identify` still reported the mark (corpus-measured 2026-07-20: 57 of 60
+            # sampled still-detected Doubao marks were untouched, ~8% of its detections).
+            # Use the DETECTOR'S OWN best-match box: the correlation already located the
+            # mark at a position and scale, and thresholding the response was a strictly
+            # worse proxy for that. An earlier fix thresholded the max-normalized uint8
+            # response at 0.5 -- which selects every non-zero pixel, not "half the peak" as
+            # its comment claimed -- and filled ~120% of the corner box on textured frames
+            # (measured: whole corner vs 58.7% for the match box, both detector-clean).
+            # Gated on an actual detection: on a clean corner the box would be spurious.
+            _, box = self._tophat_best(image, loc)
+        if box is not None:
+            gx0, gy0, gx1, gy1 = box
+            pad = max(4, int(0.10 * bh))
+            rx1 = max(0, bx + gx0 - pad)
+            rx2 = min(w, bx + gx1 + 1 + pad)
+            ry1 = max(0, by + gy0 - pad)
+            ry2 = min(h, by + gy1 + 1 + pad)
+        elif force:
+            rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
+        else:
+            return None
+        if rx1 >= rx2 or ry1 >= ry2:
+            return None
+        # Rectangular footprint + dilation is exactly region_eraser.boxes_to_mask (the
+        # same primitive the shared fill uses); reuse it instead of re-inlining the
+        # zeros/fill/MORPH_ELLIPSE-dilate here.
+        from remove_ai_watermarks import region_eraser
+
+        d = dilate if dilate is not None else max(3, int(0.02 * bw))
+        return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)

src/remove_ai_watermarks/api.py

@@ -0,0 +1,178 @@
+"""High-level convenience API: clean an image (or array) in one call.
+
+The low-level building blocks live in ``watermark_registry`` (localize -> fill) and
+``image_io`` (Unicode-safe, alpha-preserving IO). This module ties them into the two
+calls a caller usually wants, so a library user does not have to decode images, wire
+up metadata provenance, or preserve the alpha channel by hand:
+
+    import remove_ai_watermarks as raiw
+    raiw.remove_visible("in.png", "out.png")                 # path -> file, provenance auto
+    result, removed = raiw.remove_visible(bgr_array)         # array -> array
+    raiw.remove_visible("shot.png", "out.png", sensitivity="strict")
+    raiw.visible_provenance("in.png")                        # -> frozenset({"gemini"})
+
+Imports stay lazy (inside the functions), so ``import remove_ai_watermarks`` is cheap.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    from numpy.typing import NDArray
+
+    from remove_ai_watermarks.watermark_registry import Backend, Sensitivity
+
+
+@dataclass(frozen=True)
+class _VisibleInput:
+    """Normalized visible-removal input with its file-only context."""
+
+    bgr: NDArray[Any]
+    alpha: NDArray[Any] | None = None
+    path: Path | None = None
+    provenance: frozenset[str] = frozenset()
+
+
+def visible_provenance(source: str | Path) -> frozenset[str]:
+    """Vendor keys that the file's local metadata confirms, the evidence that drives
+    the ``auto`` sensitivity (relaxing a corroborated mark's detection trust gate).
+
+    Mapping: a Google/Gemini C2PA issuer -> ``"gemini"``; a China-AIGC (TC260) label
+    -> ``"doubao"``/``"jimeng"``; a ``samsung_genai`` marker -> ``"samsung"``.
+    Best-effort: any read error yields an empty set (no relaxation). Metadata-only, so
+    it never loads cv2/torch.
+    """
+    import contextlib
+
+    path = Path(source)
+    with contextlib.suppress(Exception):
+        from remove_ai_watermarks import identify
+
+        rep = identify.identify(path, check_visible=False, check_invisible=False)
+        signal_names = {signal.name for signal in rep.signals}
+        keys: set[str] = set()
+        platform = (rep.platform or "").lower()
+        if "google" in platform or "gemini" in platform:
+            keys.add("gemini")
+        if "aigc" in signal_names:
+            keys |= {"doubao", "jimeng"}
+        if "samsung_genai" in signal_names:
+            keys.add("samsung")
+        return frozenset(keys)
+    return frozenset()
+
+
+def _load_visible_input(source: str | Path | NDArray[Any]) -> _VisibleInput:
+    """Normalize a path/array source without making the public operation stateful."""
+    if not isinstance(source, (str, Path)):
+        return _VisibleInput(source)
+
+    from remove_ai_watermarks import image_io
+
+    path = Path(source)
+    bgr, alpha = image_io.read_bgr_and_alpha(path)
+    if bgr is None:
+        raise ValueError(f"Could not read image: {source}")
+    return _VisibleInput(bgr=bgr, alpha=alpha, path=path, provenance=visible_provenance(path))
+
+
+def _write_visible_result(
+    loaded: _VisibleInput,
+    result: NDArray[Any],
+    removed: list[str],
+    output: str | Path,
+    *,
+    strip_metadata: bool,
+    write_noop: bool,
+) -> None:
+    """Write one visible-removal result while preserving a true no-op losslessly."""
+    if not removed and not write_noop:
+        return
+
+    from remove_ai_watermarks import image_io
+
+    out_path = Path(output)
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    source_path = loaded.path
+    if not removed and source_path is not None and source_path.suffix.lower() == out_path.suffix.lower():
+        # Copy the ORIGINAL bytes instead of lossily re-encoding a no-op. An in-place
+        # call needs no copy and would otherwise raise shutil.SameFileError.
+        if source_path.resolve() != out_path.resolve():
+            import shutil
+
+            shutil.copyfile(source_path, out_path)
+    else:
+        # imwrite is contractually NON-RAISING, so this bool is the only signal the file
+        # was not created. Unchecked, the metadata strip below ran on a nonexistent path
+        # and surfaced as a confusing "cannot read image <INPUT>" naming the OUTPUT path
+        # (Tier E, 2026-07-20). Raise here so a library caller and the CLI both get an
+        # accurate message about the write.
+        if not image_io.write_bgr_with_alpha(out_path, result, loaded.alpha):
+            raise OSError(f"failed to write output (is the destination writable?): {out_path}")
+
+    if strip_metadata:
+        from remove_ai_watermarks import metadata
+
+        metadata.remove_ai_metadata(out_path, out_path)
+
+
+def remove_visible(
+    source: str | Path | NDArray[Any],
+    output: str | Path | None = None,
+    *,
+    sensitivity: Sensitivity = "auto",
+    backend: Backend = "auto",
+    strip_metadata: bool = True,
+    write_noop: bool = True,
+) -> tuple[NDArray[Any], list[str]]:
+    """Remove every detected known visible AI mark (Gemini sparkle, Doubao/Jimeng/
+    Samsung text, the Jimeng pill) via localize -> fill, returning ``(result_bgr,
+    [labels removed])``.
+
+    ``source`` is a file path OR a BGR ndarray. For a PATH, metadata provenance is read
+    automatically (so ``sensitivity="auto"`` recovers a moved/faint mark whenever the
+    file still carries its provenance) and the alpha channel is preserved on write; for
+    an ARRAY there is no metadata to read and no separate alpha plane. When ``output``
+    is given the cleaned image is written there (alpha rejoined for a path source); the
+    array is always returned as well, so an empty ``removed`` list tells a caller nothing
+    known was found (e.g. route to the diffusion ``all`` path or ``erase``).
+
+    ``sensitivity`` (``auto``/``strict``) and ``backend``
+    (``auto``/``cv2``/``migan``/``lama``) are the same knobs as the CLI. Pass
+    corner if the guess is wrong).
+
+    ``strip_metadata`` (default True, matching the CLI ``visible --strip-metadata``)
+    also strips AI provenance metadata (C2PA/EXIF/XMP/IPTC) from the written output via
+    the lossless :func:`metadata.remove_ai_metadata`, so a library call does exactly
+    what the CLI does. Only applies when ``output`` is given.
+
+    ``write_noop`` (default True) controls whether ``output`` is written when NOTHING was
+    removed: True writes a clean passthrough copy (an idempotent clean); False leaves the
+    output path untouched, so a caller that treats "no mark" as "produce nothing" (the CLI
+    ``visible`` no-mark contract) does not clobber a pre-existing file at that path.
+    """
+    from remove_ai_watermarks import watermark_registry
+
+    # Reject a removed sensitivity loudly; `Sensitivity` is a Literal and not enforced
+    # at runtime, so a 0.15 caller would otherwise get `auto` behaviour in silence.
+    watermark_registry.validate_sensitivity(sensitivity)
+    loaded = _load_visible_input(source)
+    result, removed = watermark_registry.remove_auto_marks(
+        loaded.bgr,
+        sensitivity=sensitivity,
+        provenance=loaded.provenance,
+        backend=backend,
+    )
+    if output is not None:
+        _write_visible_result(
+            loaded,
+            result,
+            removed,
+            output,
+            strip_metadata=strip_metadata,
+            write_noop=write_noop,
+        )
+    return result, removed