From d38b9a6122df604d432e0856353ed7f7ec4e8ae8 Mon Sep 17 00:00:00 2001 From: Victor Kuznetsov Date: Mon, 8 Jun 2026 12:22:43 -0700 Subject: [PATCH] docs: correct controlnet/restore SynthID-removal claims from the 2026-06-04 oracle pass Oracle validation (openai.com/verify + the Gemini app) overturned three claims that were on main, and consolidates the controlnet findings into one authoritative place. - controlnet does NOT reliably remove SynthID at the low vendor-adaptive strength: removal is content x pipeline dependent and the survivors FLIP by content type (photoreal survives controlnet / clears default; flat graphic survives default / clears controlnet; flat text clears both). Root cause is insufficient strength, not the pipeline; controlnet needs a higher, per-vendor floor than default. - removal near the threshold is SEED-non-deterministic (same image+pipeline+strength can pass or fail run-to-run); a single clean run does not certify a strength. - `--restore-faces` RE-INTRODUCES SynthID: GFPGAN runs on the ORIGINAL watermarked face at weight 0.5 and composites it back over the cleaned result (clean A/B: a Gemini face stayed detected through controlnet 0.15/0.20/0.25 WITH restore, cleared at 0.20 with --no-restore-faces). The old "GFPGAN scrubs SynthID" claim was wrong. Corrected in CLAUDE.md (watermark_remover controlnet bullet, controlnet Known-limitations bullet, face_restore bullet, vendor-adaptive strength bullet) and docs/synthid.md (5.1 controlnet/face-identity, 5.2 strength floors, new 5.5 oracle validation log). docs/controlnet-removal-pipeline-research.md gains an authoritative "Oracle validation 2026-06-04" section that the others point to as the single source. Docs only; no code change. Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 10 +- docs/controlnet-removal-pipeline-research.md | 73 ++++++++++ docs/synthid.md | 133 +++++++++++++++++-- 3 files changed, 202 insertions(+), 14 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index aa6dc16..9044cff 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -45,9 +45,9 @@ You are a **principal Python engineer** maintaining a CLI tool and library for r - `region_eraser.py` — universal region eraser (`erase` CLI). `erase(image, boxes=|mask=, backend=)` accepts grayscale (2D) and RGBA (4-channel) inputs on **both** backends (`erase_cv2` and `erase_lama` each split off any alpha plane and re-attach it unchanged, and promote grayscale to BGR for processing — LaMa would otherwise crash on grayscale and drop alpha on BGRA): `boxes_to_mask` → `cv2.inpaint` (`cv2` backend, default, no deps) or big-LaMa via onnxruntime (`lama` backend, extra `lama`, `Carve/LaMa-ONNX` Apache-2.0 model downloaded on first use, never bundled). `erase_lama` crops a padded region around the mask, runs LaMa at its fixed 512² input, pastes only masked pixels back (untouched areas stay pixel-exact). Lazy `_get_lama_session` singleton; `lama_available()` guards the optional import. **LaMa-ONNX costs ~3.5-4 GB peak RAM and ~5-6 s/call on CPU** (FFC working set, not arena — `enable_cpu_mem_arena=False` does not help), so it does NOT fit a minimal droplet; the cv2 backend (tens of MB, ~30 ms) does. LaMa quality at low RAM = serverless/GPU, mirroring how raiw.cc offloads SDXL to fal. - `invisible_watermark.py` — `detect_invisible_watermark(path)` decodes the OPEN DWT-DCT watermarks (public decoder, no key) embedded by Stable Diffusion / SDXL / FLUX via the `imwatermark` library. Known fixed patterns (verified against upstream source) live in `_BITS_48` (SDXL 48-bit, FLUX.2 48-bit) and `_SD1_STRING` ("StableDiffusionV1", SD 1.x/2.x). Optional dep (extra `detect`); returns None when absent. The `detect` extra pulls **torch** transitively (invisible-watermark declares torch a hard dep, and `WatermarkDecoder` eagerly imports `rivaGan` -> `torch` at import time), so detection needs torch present even though dwtDct runs CPU-only on cv2/numpy/pywavelets — no GPU and no separate `gpu` extra required. **Unlike SynthID this is locally detectable**, but the watermark is fragile (does not survive JPEG re-encode/resize — verified gone after JPEG q90), so it confirms origin only on pristine files. Add new known patterns here. The file carries a top-of-module pyright pragma because imwatermark/cv2 ship no type stubs. - `trustmark_detector.py` — `detect_trustmark(path)` decodes the OPEN, keyless **Adobe TrustMark** watermark (the soft binding behind Adobe Durable Content Credentials, `alg` `com.adobe.trustmark.P`) via the optional `trustmark` package (extra `trustmark`; pulls torch, downloads model weights on first use). Mirrors `invisible_watermark.py` (lazy singleton guarded by a double-checked `threading.Lock` so concurrent callers do not double-download the weights, top-of-module pyright pragma, returns None when absent). It detects *provenance*, not AI origin as such (TrustMark also marks human-authored content), so `identify` lists it as a watermark without setting `is_ai_generated`. Other soft-binding vendors (Digimarc/Imatag/Steg.AI/...) have no public decoder — they are only *named* via the `C2PA_SOFT_BINDINGS` scan, not decoded. **False-positive gate (added 2026-05-29):** TrustMark's `wm_present` is a BCH error-correction validity flag that spuriously validates on a content-correlated fraction of un-watermarked images — AI-generated textures trip it far more than camera photos (verified 2026-05-29 on real files: it fires on Gemini/OpenAI/Doubao output that *cannot* carry Adobe's watermark, with a random-bytes decoded secret, while signal-free camera photos did not trip it). A genuine TrustMark is a *durable* soft binding engineered to survive re-encoding, so `detect_trustmark` re-decodes after a mild JPEG round-trip (`_survives_reencode`, `_REENCODE_QUALITY` 95) and requires the same schema both times; every observed false positive collapsed (none survived even q95), so the gate is the durability property the watermark guarantees. The second decode runs only on the rare initial hit, so the cost is negligible. Do NOT remove the gate to "catch more" — a lone TrustMark hit without it is almost always content noise. -- `noai/watermark_remover.py` — the `WatermarkRemover` class has two diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id` -- both use the same SDXL base, `DEFAULT_MODEL_ID`). **`default`** runs plain SDXL img2img (`_run_img2img`). **`controlnet`** (**EXPERIMENTAL, opt-in**; `_run_controlnet`, `_load_controlnet_pipeline`) runs `StableDiffusionXLControlNetImg2ImgPipeline` with the SDXL-native canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` (`watermark_profiles.CONTROLNET_CANNY_MODEL`): the control image is `cv2.Canny(gray, 100, 200)` stacked to 3 channels (`_CANNY_LOW`/`_CANNY_HIGH`, prompt `_CONTROLNET_PROMPT` / `_CONTROLNET_NEGATIVE`). **Removal still comes from the img2img regeneration (`strength`); the ControlNet only PRESERVES text and face STRUCTURE via the edge map -- no original pixels are copied or frozen, so SynthID does not survive.** Canny holds face STRUCTURE but NOT identity (the regenerated face drifts in likeness -- canny carries edges, not identity; face identity is preserved by the optional `--restore-faces` GFPGAN post-pass (EXPERIMENTAL, opt-in, OFF by default) -- see `face_restore.py`). `controlnet_conditioning_scale` (ctor arg, default 1.0) is the structure-preservation knob. Same dtype rule as `default` (fp32 on cpu/mps, fp16 only on cuda/xpu; the fp16-fixed SDXL VAE `_SDXL_FP16_VAE_ID` is swapped in on fp16 GPUs -- issue #29) and the same MPS->CPU fallback (reload on cpu/fp32, drop a non-cpu generator, retry once). -- `face_restore.py` — optional GFPGAN face-restoration post-pass (cv2/torch/gfpgan boundary, top-of-file pyright pragma). **EXPERIMENTAL, opt-in, OFF by default.** Runs AFTER the diffusion removal pass (`InvisibleEngine.remove_watermark`, params `restore_faces=False` / `restore_faces_weight=0.5`; CLI `--restore-faces`/`--no-restore-faces` + `--restore-faces-weight` on `invisible`/`all`/`batch`). **Restores face IDENTITY while still scrubbing the pixel watermark:** GFPGAN re-synthesizes each face from a StyleGAN2 prior (codebook/GAN pixels, NOT the original), so the composited face regions carry no watermark and no pixel-copy -- oracle-validated clean at weight 0.5 with identity preserved. Flow: GFPGANer.enhance runs on the ORIGINAL (watermarked) image -> identity faces + RetinaFace boxes (`restorer.face_helper.det_faces`); `_composite_faces` feather-composites those restored face REGIONS into the diffusion-cleaned image. `is_available()` gates on gfpgan + facexlib; lazily-built `GFPGANer` singleton forces CPU unless CUDA (the pip GFPGANer has an MPS device-mismatch bug; it is a cheap post-pass on a few face crops). `_apply_basicsr_shim()` recreates the removed `torchvision.transforms.functional_tensor` module that basicsr imports. The pure `_composite_faces` helper (Gaussian-feathered rectangular alpha per box, `out = restored*a + base*(1-a)`) is unit-tested without the model (`tests/test_face_restore.py`); the model-running path is gated behind `is_available()`. **Commercial-safe** (GFPGAN Apache-2.0 + RetinaFace MIT); the CodeFormer alternative is NON-COMMERCIAL and is NOT shipped. The `restore` extra (gfpgan/facexlib/basicsr) is kept OUT of `all` (heavy + the GFPGANv1.4 + RetinaFace weights download on first use, never bundled). **`restore` pins numpy<2** (same trap class as the removed faceid/insightface extra): basicsr/gfpgan/facexlib are an old ecosystem, so the extra caps `scipy<1.18` (>=1.18 uses `np.long`, gone in numpy 1.24-1.26) and `numba<0.60` to keep the whole env on one numpy 1.26 resolution; verified the `--extra dev --extra gpu` gate env stays numpy 1.26.4 + `diffusers.loaders.peft` importable with `restore` present. **basicsr 1.4.2 builds only on Python <3.13** (its `setup.py get_version()` uses `exec(...)` + `locals()['__version__']`, which the 3.13 fast-locals change broke -> `KeyError: '__version__'`), so the project is pinned to Python 3.12 via `.python-version` and `[tool.uv.extra-build-dependencies] basicsr = ["setuptools<69"]`. basicsr ships sdist-only (no wheel). -- `auto_config.py` — the `--auto` quality-mode planner (EXPERIMENTAL). `plan(image_path) -> AutoConfig | None` inspects the INPUT image (before the diffusion model loads) and picks the pipeline modes, so the run adapts to content. **Designed to run as the FIRST step of the invisible/all pipeline, wherever that runs** — locally or the raiw.cc Modal GPU worker — **never on the 512 MB web host** (image work there OOM-crashes the container; the planner is `_apply_auto` in `cli.py` for the CLI, and raiw-app would call `plan()` inside `RaiwProtect.remove`). **Quality-priority routing:** ControlNet (text/face-structure preservation) is the default; it is skipped for `default` (plain SDXL) only on a clearly structure-less image (`not has_face and not has_text and edge_density < _STRUCTURELESS_EDGE_MAX` 0.008). `restore_faces` is on when a face is present. When a smoothing pass (controlnet/restore) ran, the **adaptive polish** (`humanizer.adaptive_polish`) is applied: it targets the input's Laplacian variance (detail level) with a capped unsharp + edge-masked grain, restoring photo/face texture while **sparing text** (text is already high-frequency, so the deficit is tiny and almost no polish lands -- the old fixed unsharp/grain speckled small text; validated 2026-06-03 on gemini_3 lap-var 84->334 toward the 592 original, openai_1 text near-untouched). **Detection is cv2-only and torch-free** (~100 MB peak RSS, a few ms — measured): OpenCV **YuNet** (`cv2.FaceDetectorYN`, MIT, 232 KB model bundled at `assets/face_detection_yunet_2023mar.onnx`) for faces, **DBNet** (PP-OCRv3 differentiable-binarization via `cv2.dnn.TextDetectionModel_DB`, a 2.4 MB Apache-2.0 model bundled at `assets/text_detection_ppocrv3_2023may.onnx`) for text, with the old Canny+MSER region heuristic kept as a fallback if the DBNet model can't load (`_detect_text_dbnet` returns None → `_detect_text_mser`). The en/cn opencv_zoo PP-OCRv3 detection models are byte-identical, so it is bundled language-neutral. Text only ever ADDS controlnet, so a miss is backstopped by edge-density and a false positive only costs a controlnet run. Plus `edge_density`. `min_resolution` stays 1024. **Every auto decision is independently overridable** (interface principle): `_apply_auto` (cli.py) overrides only the three content-adaptive modes the user left at their click default (`ctx.get_parameter_source(...) == DEFAULT`) — `--pipeline`, `--restore-faces`/`--no-restore-faces`, and **`--adaptive-polish`/`--no-adaptive-polish`** always win; `--min-resolution`/`--strength`/`--unsharp`/`--humanize` are independent knobs. `--adaptive-polish` also works WITHOUT `--auto` (manual detail-targeted polish; the engine's `adaptive_polish` param uses the full-res original as the detail reference). Prints the chosen plan (`AutoConfig.reason`). Wired into `cmd_all`/`cmd_invisible`/`cmd_batch` — in `batch` the plan is recomputed per image and the invisible engine is cached **per resolved pipeline** (`ctx.obj["_inv_engines"]`, keyed `default`/`controlnet`) instead of a single shared instance, so a mixed directory builds at most one engine of each kind. **Adds ZERO new pip deps** (all cv2 core + the bundled MIT YuNet + Apache-2.0 DBNet models + the cv2-only adaptive polish). The auto plan does NOT select the `esrgan` upscaler (that needs the optional extra and would make auto's behavior install-dependent); `--upscaler esrgan` stays a separate manual knob. Unit-tested without a heavy download (`tests/test_auto_config.py`): flat/text synthetic images for routing (the bundled DBNet fires on a real text card), monkeypatched `detect_face`/`_detect_text_dbnet`/`_detect_text_mser` for the face/text/fallback branches (a real detectable-face fixture is private, never committed). Production adoption path for raiw.cc: validate (must keep SynthID removed, not hallucinate micro-text, beat plain SDXL on the real upload distribution), then bump the library SHA in `modal_app.py` and pass `auto=True`. +- `noai/watermark_remover.py` — the `WatermarkRemover` class has two diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id` -- both use the same SDXL base, `DEFAULT_MODEL_ID`). **`default`** runs plain SDXL img2img (`_run_img2img`). **`controlnet`** (**EXPERIMENTAL, opt-in**; `_run_controlnet`, `_load_controlnet_pipeline`) runs `StableDiffusionXLControlNetImg2ImgPipeline` with the SDXL-native canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` (`watermark_profiles.CONTROLNET_CANNY_MODEL`): the control image is `cv2.Canny(gray, 100, 200)` stacked to 3 channels (`_CANNY_LOW`/`_CANNY_HIGH`, prompt `_CONTROLNET_PROMPT` / `_CONTROLNET_NEGATIVE`). **Removal comes from the img2img regeneration (`strength`); the ControlNet only PRESERVES text and face STRUCTURE via the edge map.** No original pixels are copied or frozen, BUT **validation 2026-06-04 disproved the old "so SynthID does not survive" claim: SynthID CAN survive controlnet on photoreal/high-detail content.** At the shared low removal strength the canny edge-conditioning keeps the regeneration so close to the original that the pixel perturbation that destroys SynthID does not happen (oracle-confirmed: an OpenAI bracelet photo + a 9-face grid read **SynthID-detected** after controlnet at strength 0.10/0.15, but **SynthID-not-detected** after the `default` pipeline at the SAME strength + resolution -- only the pipeline differed). **But the reverse also holds: a flat-graphic logo/poster SURVIVED `default` while clearing controlnet** -- removal at the low strength is content×pipeline dependent and neither pipeline is universally safe; the real lever is a higher strength. See the controlnet Known-limitations bullet for the full table + root cause. Canny holds face STRUCTURE but NOT identity (the regenerated face drifts in likeness -- canny carries edges, not identity; face identity is preserved by the optional `--restore-faces` GFPGAN post-pass (EXPERIMENTAL, opt-in, OFF by default) -- see `face_restore.py`). `controlnet_conditioning_scale` (ctor arg, default 1.0) is the structure-preservation knob. Same dtype rule as `default` (fp32 on cpu/mps, fp16 only on cuda/xpu; the fp16-fixed SDXL VAE `_SDXL_FP16_VAE_ID` is swapped in on fp16 GPUs -- issue #29) and the same MPS->CPU fallback (reload on cpu/fp32, drop a non-cpu generator, retry once). +- `face_restore.py` — optional GFPGAN face-restoration post-pass (cv2/torch/gfpgan boundary, top-of-file pyright pragma). **EXPERIMENTAL, opt-in, OFF by default.** Runs AFTER the diffusion removal pass (`InvisibleEngine.remove_watermark`, params `restore_faces=False` / `restore_faces_weight=0.5`; CLI `--restore-faces`/`--no-restore-faces` + `--restore-faces-weight` on `invisible`/`all`/`batch`). **WARNING -- this pass can RE-INTRODUCE SynthID into the face regions (oracle-confirmed 2026-06-04); the old "scrubs the watermark / oracle-validated clean" claim was WRONG.** Flow: GFPGANer.enhance runs on the **ORIGINAL (watermarked)** image -> identity faces + RetinaFace boxes (`restorer.face_helper.det_faces`); `_composite_faces` feather-composites those restored face REGIONS into the diffusion-cleaned image. At the default fidelity weight `0.5` GFPGAN BLENDS ~half the original face pixels with the StyleGAN2 prior (it is not a pure GAN re-synthesis), and **SynthID is robust to that partial blend**, so the composited face carries the watermark back IN -- overwriting what the diffusion pass removed. **Confirmed by a clean A/B:** an OpenAI/Gemini face image (`gemini_3`) read SynthID-DETECTED after controlnet @ strength 0.20/0.25 WITH restore, but SynthID-NOT-detected after the SAME controlnet @ 0.20 with `--no-restore-faces` (only restore differed). It is **content-dependent** (a second face image cleared WITH restore -- smaller faces / different blend), which is why the earlier single-image validation read "clean". **So `--restore-faces` as currently wired is a footgun for removal: it can re-add the watermark it is supposed to be scrubbing. Removal-priority callers (raiw.cc) must NOT use restore-on-original, or must switch to one of the fixes below.** **Fix directions (engineering follow-up, not yet done):** (a) run GFPGAN on the DIFFUSION-CLEANED image instead of the original, so the restored face is derived from already-clean pixels (loses some identity sharpness); (b) drop `--restore-faces-weight` well below 0.5 (more StyleGAN2 synthesis, less original -> less SynthID, but identity drifts); (c) leave restore OFF when removal is the priority. Each needs its own oracle re-validation. `is_available()` gates on gfpgan + facexlib; lazily-built `GFPGANer` singleton forces CPU unless CUDA (the pip GFPGANer has an MPS device-mismatch bug; it is a cheap post-pass on a few face crops). `_apply_basicsr_shim()` recreates the removed `torchvision.transforms.functional_tensor` module that basicsr imports. The pure `_composite_faces` helper (Gaussian-feathered rectangular alpha per box, `out = restored*a + base*(1-a)`) is unit-tested without the model (`tests/test_face_restore.py`); the model-running path is gated behind `is_available()`. **Commercial-safe** (GFPGAN Apache-2.0 + RetinaFace MIT); the CodeFormer alternative is NON-COMMERCIAL and is NOT shipped. The `restore` extra (gfpgan/facexlib/basicsr) is kept OUT of `all` (heavy + the GFPGANv1.4 + RetinaFace weights download on first use, never bundled). **`restore` pins numpy<2** (same trap class as the removed faceid/insightface extra): basicsr/gfpgan/facexlib are an old ecosystem, so the extra caps `scipy<1.18` (>=1.18 uses `np.long`, gone in numpy 1.24-1.26) and `numba<0.60` to keep the whole env on one numpy 1.26 resolution; verified the `--extra dev --extra gpu` gate env stays numpy 1.26.4 + `diffusers.loaders.peft` importable with `restore` present. **basicsr 1.4.2 builds only on Python <3.13** (its `setup.py get_version()` uses `exec(...)` + `locals()['__version__']`, which the 3.13 fast-locals change broke -> `KeyError: '__version__'`), so the project is pinned to Python 3.12 via `.python-version` and `[tool.uv.extra-build-dependencies] basicsr = ["setuptools<69"]`. basicsr ships sdist-only (no wheel). +- `auto_config.py` — the `--auto` quality-mode planner (EXPERIMENTAL). `plan(image_path) -> AutoConfig | None` inspects the INPUT image (before the diffusion model loads) and picks the pipeline modes, so the run adapts to content. **Designed to run as the FIRST step of the invisible/all pipeline, wherever that runs** — locally or the raiw.cc Modal GPU worker — **never on the 512 MB web host** (image work there OOM-crashes the container; the planner is `_apply_auto` in `cli.py` for the CLI, and raiw-app would call `plan()` inside `RaiwProtect.remove`). **Quality-priority routing:** ControlNet (text/face-structure preservation) is the default; it is skipped for `default` (plain SDXL) only on a clearly structure-less image (`not has_face and not has_text and edge_density < _STRUCTURELESS_EDGE_MAX` 0.008). **CAVEAT (oracle-validated 2026-06-04, see the controlnet Known-limitations bullet): at the low vendor-adaptive strength NEITHER pipeline removes SynthID on all content -- it is content×pipeline dependent (photoreal SURVIVES controlnet / clears default; flat graphics SURVIVE default / clear controlnet; flat text clears both). So `--auto` picking controlnet for faces/photos leaves SynthID on exactly those, and plain `default` would leave it on flat graphics -- pipeline choice alone does NOT guarantee removal. The real lever is a HIGHER strength, oracle-validated per content type. Removal-priority callers (raiw.cc) must oracle-validate strength across content types BEFORE adopting auto; the "must keep SynthID removed" gate in the adoption note below is the blocker this caught.** `restore_faces` is on when a face is present. When a smoothing pass (controlnet/restore) ran, the **adaptive polish** (`humanizer.adaptive_polish`) is applied: it targets the input's Laplacian variance (detail level) with a capped unsharp + edge-masked grain, restoring photo/face texture while **sparing text** (text is already high-frequency, so the deficit is tiny and almost no polish lands -- the old fixed unsharp/grain speckled small text; validated 2026-06-03 on gemini_3 lap-var 84->334 toward the 592 original, openai_1 text near-untouched). **Detection is cv2-only and torch-free** (~100 MB peak RSS, a few ms — measured): OpenCV **YuNet** (`cv2.FaceDetectorYN`, MIT, 232 KB model bundled at `assets/face_detection_yunet_2023mar.onnx`) for faces, **DBNet** (PP-OCRv3 differentiable-binarization via `cv2.dnn.TextDetectionModel_DB`, a 2.4 MB Apache-2.0 model bundled at `assets/text_detection_ppocrv3_2023may.onnx`) for text, with the old Canny+MSER region heuristic kept as a fallback if the DBNet model can't load (`_detect_text_dbnet` returns None → `_detect_text_mser`). The en/cn opencv_zoo PP-OCRv3 detection models are byte-identical, so it is bundled language-neutral. Text only ever ADDS controlnet, so a miss is backstopped by edge-density and a false positive only costs a controlnet run. Plus `edge_density`. `min_resolution` stays 1024. **Every auto decision is independently overridable** (interface principle): `_apply_auto` (cli.py) overrides only the three content-adaptive modes the user left at their click default (`ctx.get_parameter_source(...) == DEFAULT`) — `--pipeline`, `--restore-faces`/`--no-restore-faces`, and **`--adaptive-polish`/`--no-adaptive-polish`** always win; `--min-resolution`/`--strength`/`--unsharp`/`--humanize` are independent knobs. `--adaptive-polish` also works WITHOUT `--auto` (manual detail-targeted polish; the engine's `adaptive_polish` param uses the full-res original as the detail reference). Prints the chosen plan (`AutoConfig.reason`). Wired into `cmd_all`/`cmd_invisible`/`cmd_batch` — in `batch` the plan is recomputed per image and the invisible engine is cached **per resolved pipeline** (`ctx.obj["_inv_engines"]`, keyed `default`/`controlnet`) instead of a single shared instance, so a mixed directory builds at most one engine of each kind. **Adds ZERO new pip deps** (all cv2 core + the bundled MIT YuNet + Apache-2.0 DBNet models + the cv2-only adaptive polish). The auto plan does NOT select the `esrgan` upscaler (that needs the optional extra and would make auto's behavior install-dependent); `--upscaler esrgan` stays a separate manual knob. Unit-tested without a heavy download (`tests/test_auto_config.py`): flat/text synthetic images for routing (the bundled DBNet fires on a real text card), monkeypatched `detect_face`/`_detect_text_dbnet`/`_detect_text_mser` for the face/text/fallback branches (a real detectable-face fixture is private, never committed). Production adoption path for raiw.cc: validate (must keep SynthID removed, not hallucinate micro-text, beat plain SDXL on the real upload distribution), then bump the library SHA in `modal_app.py` and pass `auto=True`. - `upscaler.py` — optional Real-ESRGAN pre-diffusion super-resolution for small inputs (spandrel boundary, top-of-file pyright pragma). `is_available()` gates on spandrel+torch (via `importlib.util.find_spec`); `upscale(bgr, device=None)` loads a lazily-built spandrel `ImageModelDescriptor` singleton (double-checked lock) and upscales by the model's native factor (x2), with a non-CPU→CPU device fallback mirroring the diffusion engine's MPS→CPU retry. Weights (`RealESRGAN_x2plus.pth`, BSD-3-Clause) download on first use to the `torch.hub` checkpoints cache; never bundled. Used only when UPscaling to the `min_resolution` floor (a `max_resolution` downscale always uses Lanczos). The wiring is `InvisibleEngine._esrgan_upscale(pil, target)` — Real-ESRGAN at native factor, then a Lanczos resize to the exact target, falling back to a plain Lanczos resize if the extra is absent or the model errors (so an optional upscaler can never break removal). The default `--upscaler` is `lanczos` (cv2, no deps). **ESRGAN is a generic photo/texture GAN with no face/glyph prior**, so it best fits photo/texture content and can degrade faces (glassy/asymmetric eyes -- the diffusion pass regenerates faces so the full-pipeline final recovers; that is what GFPGAN/`--restore-faces` is for) and thin/small text (the GAN invents wrong strokes, and low-strength diffusion will not fix it). Verified 2026-06-04: isolated upscale lap-var ~5x Lanczos on faces+textures but glassy eyes; end-to-end `invisible` final lap-var 1634 vs Lanczos 663 with natural faces (diffusion cleaned the artifact). Kept a **manual opt-in knob** (the auto plan never selects it) with `lanczos` the default; not content-gated by design (use Lanczos for text-heavy inputs). spandrel is MIT and pulls no basicsr, unlike the `restore` extra. Unit-tested without the model: `tests/test_upscaler.py` (availability guard + the not-installed RuntimeError) and `tests/test_invisible_engine.py::TestEsrganUpscale` (the three `_esrgan_upscale` branches via a monkeypatched `upscaler`). - `image_io.py` — Unicode-safe cv2 IO (issue #17). `imread(path, flags=None)` / `imwrite(path, img)` wrap `np.fromfile`+`cv2.imdecode` / `cv2.imencode`+`tofile` so non-ASCII paths work on Windows -- bare `cv2.imread`/`cv2.imwrite` use the platform ANSI code-page API there and fail (empty decode + `can't open/read file`) on Chinese/Cyrillic/accented filenames. `imread` keeps `cv2.imread` semantics (defaults to `IMREAD_COLOR`, returns `None` on missing/empty/undecodable). **Every cv2 file read/write in the package routes through here; do not call `cv2.imread`/`cv2.imwrite` directly.** `imwrite` returns `False` on an unwritable path (`OSError` caught) instead of raising, matching `cv2.imwrite` semantics. macOS/Linux already accept UTF-8 paths, so it is behavior-neutral there (the bug only reproduces on Windows). cv2/numpy are imported lazily inside the functions, so the module is cheap to import in a bare env. @@ -90,6 +90,6 @@ Who embeds what, and whether it is locally detectable (so we know which gaps are - **SynthID detection is metadata-only.** There is no reliable *local* detector of the SynthID *pixel* watermark — Google's decoder is proprietary, no public spec or API (only a waitlisted portal). Authoritative confirmation: Google DeepMind's own paper "SynthID-Image: Image watermarking at internet scale" (Gowal et al., arXiv:2510.09263) states the verification service is restricted to "trusted testers" and does not release detector weights or a reproducible algorithm — so a local pixel detector is infeasible by design, not just unbuilt. https://arxiv.org/abs/2510.09263 We detect SynthID by its C2PA companion (`synthid_source` / `SYNTHID_C2PA_ISSUERS`), which is reliable while the manifest is intact but says nothing once C2PA is stripped. **Surface-dependent blind spot (verified 2026-05-24):** the same Google model emits different metadata per surface -- the Gemini *app* wraps outputs in Google C2PA, but the *API/playground* (AI Studio, Nano Banana / gemini-2.5-flash-image) emits the SynthID *pixel* watermark (confirmed via the Gemini-app oracle) + the visible sparkle but **no C2PA/IPTC at all**, so `synthid_source` returns None despite SynthID being present. Only the pixel oracle or the visible-sparkle detector catches those. (Meta AI is another surface mismatch: it writes the IPTC `digitalSourceType=trainedAlgorithmicMedia` marker, not C2PA and not SynthID.) Google→SynthID is long-standing; OpenAI→SynthID is confirmed by OpenAI's Help Center (ChatGPT/Codex/API "include both C2PA metadata and SynthID watermarks", updated 2026-05-21) but time-gated (pre-rollout OpenAI images carry C2PA without SynthID), so the OpenAI verdict is hedged "likely". Oracles: Gemini app "Verify with SynthID" (Google), openai.com/verify (OpenAI). **Each vendor's oracle detects only its OWN content (verified on the page 2026-05-31):** `openai.com/research/verify` states verbatim "OpenAI generation signals will only be detected if the image was generated with our tools" and "Content could also still be AI-generated by another company's model, which the tool currently does not detect" -- SynthID is shared tech but the verifier is keyed to its own vendor's payload, so a Google-SynthID image reads clean on OpenAI's verifier and vice-versa. **This explains the recurring "oracle says clean but `identify` still flags SynthID" report (#14):** the oracle reads the *pixel* watermark (gone after our SDXL pass), while `identify` reads the *C2PA-metadata proxy* (still present if the manifest survived). Different signals, not a contradiction -- strip the metadata too (`metadata --remove` / `all`) and the proxy goes quiet, but a quiet proxy is not proof the pixel watermark is gone. **SynthID is durable to JPEG re-encode by design, so a GitHub-recompressed issue attachment is still a valid SynthID test subject** (verified 2026-06-01 on issue #14's pic3: the GitHub-served JPEG survived re-encoding and openai.com/verify still detected SynthID). Do NOT dismiss issue-attachment JPEGs as "not faithful originals" when reproducing a SynthID-survival report: the recompression strips the **C2PA metadata** (so `identify` reads Unknown on the attachment) but NOT the **pixel watermark** that openai.com/verify reads. A true byte-original only matters for the metadata/C2PA path, not for the pixel-SynthID-removal test. (Contrast the open imwatermark above, which IS fragile to JPEG.) The spectral phase-coherence approach from `github.com/aloshdenny/reverse-SynthID` was evaluated (May 2026) and **does not work for real-content detection**: on its own shipped codebook + validation set, watermarked and cleaned images were indistinguishable (conf within noise, cleaned often higher); it only fires on pure-black 1024x1024 reference images at exact resolution (the controlled case it was calibrated on). The README's "90% / conf=0.91" reproduces only in that lab condition. Do not build a production detector on it; if revisited, it is experimental/diagnostic only and needs a per-resolution, per-model reference corpus. A from-scratch gpt-image pilot (2026-05-24) confirmed this independently: 5 independent solid-black gpt-image outputs share a near-identical fixed signature (pairwise residual correlation **0.92**, avg-template retains 97% energy), so the watermark/carrier IS strongly present and consistent on flat content — but the carrier frequencies extracted from it do NOT discriminate real content (carrier-to-random ratio: cleaned 1.86 > watermarked 1.53; a non-gpt-image image scored highest at 3.67). The signature drowns in content texture. Net: a perfectly consistent solid-color signature still yields no real-content pixel detector with magnitude/carrier methods. A corpus discrimination test (2026-05-24, `scripts/synthid_pixel_probe.py`, raw zero-mean residual NCC) independently re-confirms this: at matched resolution, SynthID positives do NOT cluster apart from negatives (within-Gemini 0.07; at 1024 px pos-vs-neg >= pos-vs-pos). The only high correlations were near-duplicate *content* (5 ChatGPT renders of one prompt at ~0.92, while a distinct ChatGPT image scored ~0 against them) — content, not a carrier. The probe is solid-fills-only and EXPERIMENTAL/DIAGNOSTIC; do not use it on real content. **Correction (deeper re-examination 2026-05-25):** the carrier IS real on solid fills — the earlier "no carrier" was a *method* artifact of using spatial / FFT-magnitude NCC, which can't see it. The carrier is a fixed *phase* at specific low frequencies, so the right metric is **per-bin phase coherence**. On 8 white `gemini-2.5-flash-image` fills (generated via the reverse-SynthID trick: identity-edit prompt "Recreate this image exactly as it is" on a synthetic pure-white PNG — this bypasses the recitation block that rejects text prompts for pure colors), phase coherence at the white carriers `(0,±7..±12,±20..±23)` = **0.86** vs **0.31** random; single-image leave-one-out phase-match **+0.83** vs real photos **-0.24**. (Black `2.5-flash` fills clip to std≈0 — SynthID can't push values below 0, so no carrier in black; the repo's dark carriers come from nano-banana-pro.) **But it does not generalize:** (a) carriers are model-version + resolution + color specific — the repo's v4 codebook (built for `gemini-3.1-flash-image-preview` + `nano-banana-pro-preview`) scores ~0.527 on my 2.5-flash white fills, indistinguishable from negatives (~0.50), i.e. carriers shift across model versions and need a per-model codebook; (b) on real content (30 `2.5-flash` images) the carrier collapses — set phase coherence at carriers 0.37 ≈ random 0.42, and the repo's v4 detector gives content 0.518 ≈ negatives 0.504 (no separation; a faint +0.24 single-image lean is likely a brightness confound). Net: the spectral/phase approach is a real *controlled-fill* characterizer, NOT an arbitrary-real-content detector, and is brittle to model version. Metadata proxy + visible sparkle + online oracles remain the ceiling for real content. - **External AI-vs-real classifier models are out of scope (decided 2026-05-24).** Generic HuggingFace detectors (`Organika/sdxl-detector` Swin Transformer, `umm-maybe/AI-image-detector`, and fine-tunes) exist and report ~0.98 on their *own* SDXL-vs-real validation sets, but they are per-generator and the model cards themselves note degraded accuracy off-distribution; they are untested on gpt-image / Gemini Nano Banana (the metadata-stripped surfaces we care about), and our own light SDXL pass would likely defeat them the same way it defeats SynthID. Detection here stays local + signal-based (metadata + visible sparkle); do not add a bundled classifier dependency. -- **DEFAULT STRENGTH IS NOW VENDOR-ADAPTIVE (2026-06-01, SUPERSEDES every fixed-default claim in this bullet and the next).** `resolve_strength(strength, profile, vendor)` + `vendor_for_strength(path)` (`watermark_profiles.py`) read the C2PA issuer (`metadata.synthid_source`) on the ORIGINAL input and pick `OPENAI_STRENGTH` **0.10** / `GEMINI_STRENGTH` **0.15** / `UNKNOWN_STRENGTH` **0.15** when `--strength` is unset; explicit `--strength` always wins. The CLI detects the vendor from the pristine source (before the visible pass / metadata-strip removes C2PA from the temp file) and passes it to the engine, so display and execution agree; `cmd_invisible`/`cmd_all`/`batch` + the module-level `remove_watermark` all thread `vendor`. **This replaces the single 0.30 default AND the prior "do NOT build a vendor-adaptive default" policy** -- both came from the now-debunked region-rescrub-contaminated study (the per-region re-scrub that contaminated those numbers was removed in the controlnet refactor). Basis: the oracle-verified June 2026 controlled study (clean v0.8.6, protect OFF): OpenAI clears at 0.05 across 1024-1600 (n=4, resolution-independent); Google needs 0.15 on the capped-1536 path (n=4). `docs/synthid.md` §2.2 (data) + §5.2 (the adaptive default) are authoritative. CAVEAT: Google's 0.15 was validated only on `--max-resolution 1536`; native large Gemini (2816) was not locally measurable (OOM on M-series) and is pending GPU validation on raiw.cc -- if it survives 0.15 native, raise `--strength`. **Everything below in this bullet about a fixed 0.10/0.30 default is HISTORICAL; trust the vendor-adaptive constants + docs/synthid.md.** +- **DEFAULT STRENGTH IS NOW VENDOR-ADAPTIVE (2026-06-01, SUPERSEDES every fixed-default claim in this bullet and the next).** `resolve_strength(strength, profile, vendor)` + `vendor_for_strength(path)` (`watermark_profiles.py`) read the C2PA issuer (`metadata.synthid_source`) on the ORIGINAL input and pick `OPENAI_STRENGTH` **0.10** / `GEMINI_STRENGTH` **0.15** / `UNKNOWN_STRENGTH` **0.15** when `--strength` is unset; explicit `--strength` always wins. The CLI detects the vendor from the pristine source (before the visible pass / metadata-strip removes C2PA from the temp file) and passes it to the engine, so display and execution agree; `cmd_invisible`/`cmd_all`/`batch` + the module-level `remove_watermark` all thread `vendor`. **This replaces the single 0.30 default AND the prior "do NOT build a vendor-adaptive default" policy** -- both came from the now-debunked region-rescrub-contaminated study (the per-region re-scrub that contaminated those numbers was removed in the controlnet refactor). Basis: the oracle-verified June 2026 controlled study (clean v0.8.6, protect OFF): OpenAI clears at 0.05 across 1024-1600 (n=4, resolution-independent); Google needs 0.15 on the capped-1536 path (n=4). `docs/synthid.md` §2.2 (data) + §5.2 (the adaptive default) are authoritative. **CAVEAT (oracle pass 2026-06-04): the OpenAI 0.10 default is content-dependent, NOT universal -- a flat-graphic OpenAI logo/poster still read SynthID-detected after `default` at 0.10, and photoreal images after controlnet at 0.10/0.15 (low-change regions under-perturbed). Removal at 0.10/0.15 is content×pipeline dependent (see the controlnet Known-limitations bullet); the lever is a higher strength, oracle-revalidated per content type. Do NOT assume the vendor-adaptive default clears every image.** CAVEAT: Google's 0.15 was validated only on `--max-resolution 1536`; native large Gemini (2816) was not locally measurable (OOM on M-series) and is pending GPU validation on raiw.cc -- if it survives 0.15 native, raise `--strength`. **Everything below in this bullet about a fixed 0.10/0.30 default is HISTORICAL; trust the vendor-adaptive constants + docs/synthid.md.** - **SynthID removal: strength + oracle scope.** Default strength is vendor-adaptive (see the bullet above); `docs/synthid.md` §2.2 is authoritative for the numbers. **Oracle scope (load-bearing):** the Gemini app "Verify with SynthID" is the ONLY valid SynthID oracle (detects Google's mark on any image); `openai.com/verify` is scoped to OpenAI provenance (its own C2PA), NOT a SynthID oracle -- a negative there is meaningless for SynthID. There is no local SynthID detector, so the tool cannot self-check; if the oracle still reads SynthID, raise `--strength` to the lowest value that verifies clean. Only the `default` (plain SDXL img2img) and `controlnet` (SDXL + canny ControlNet) profiles exist; the local `invisible` default is weight-for-weight identical to raiw.cc prod (`fal-ai/fast-sdxl` = `stabilityai/stable-diffusion-xl-base-1.0`, runtime-downloaded, not bundled). **Forensic-stealth caveat** (arXiv:2605.09203): defeating the SynthID verifier is NOT forensic invisibility -- independent detectors flag *removal-processed* images vs genuinely-clean ones at >98% TPR@1%FPR, so do not over-claim "indistinguishable from a real photo". -- **`controlnet` pipeline (text/face STRUCTURE preservation, EXPERIMENTAL, opt-in `--pipeline controlnet`).** SDXL + the canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` via `StableDiffusionXLControlNetImg2ImgPipeline` (`watermark_remover._run_controlnet` / `_load_controlnet_pipeline`). **Removal still comes from the img2img regeneration (`strength`); the ControlNet only PRESERVES text and face STRUCTURE by conditioning on the canny edge map** (`cv2.Canny(gray, 100, 200)`, 3-channel). Canny preserves edges, NOT face identity (a regenerated face drifts in likeness); face identity is preserved by the optional `--restore-faces` GFPGAN post-pass (EXPERIMENTAL, opt-in, OFF by default -- see `face_restore.py`, the `restore` extra), which re-synthesizes each face from a StyleGAN2 prior so the composited face pixels carry no watermark. The CodeFormer alternative stays NON-COMMERCIAL and is not shipped. The earlier `--face-id` IP-Adapter FaceID layer was REMOVED (footgun: it needs high strength and corrupts faces at the low removal strength). **No original pixels are copied or frozen, so SynthID does not survive** -- unlike the deleted text/face-protection subsystems, which restored or re-scrubbed original pixels and could shield the watermark. `controlnet_conditioning_scale` (CLI `--controlnet-scale`, default 1.0) is the structure-preservation knob (higher = closer to the original structure). It shares the SDXL base, so it uses the SAME vendor-adaptive strength as `default` (`resolve_strength`); fp32 on cpu/mps, fp16-fixed VAE on cuda/xpu. The `controlnet` profile is threaded explicitly (`WatermarkRemover(pipeline=...)` / `InvisibleEngine(pipeline=...)`), NOT inferred from `model_id`. This productionizes the `scripts/controlnet_sweep.py` prototype; see `docs/controlnet-removal-pipeline-research.md`. **Forensic-stealth caveat still applies** (arXiv:2605.09203): defeating the SynthID verifier is not forensic invisibility -- a "this image went through a removal pipeline" classifier can still flag the output. +- **`controlnet` pipeline (text/face STRUCTURE preservation, EXPERIMENTAL, opt-in `--pipeline controlnet`).** SDXL + the canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` via `StableDiffusionXLControlNetImg2ImgPipeline` (`watermark_remover._run_controlnet` / `_load_controlnet_pipeline`). **Removal still comes from the img2img regeneration (`strength`); the ControlNet only PRESERVES text and face STRUCTURE by conditioning on the canny edge map** (`cv2.Canny(gray, 100, 200)`, 3-channel). Canny preserves edges, NOT face identity (a regenerated face drifts in likeness); face identity is preserved by the optional `--restore-faces` GFPGAN post-pass (EXPERIMENTAL, opt-in, OFF by default -- see `face_restore.py`, the `restore` extra) -- **but WARNING: that pass can RE-INTRODUCE SynthID into the face regions (oracle-confirmed 2026-06-04, since GFPGAN runs on the ORIGINAL watermarked face and blends ~half its pixels at weight 0.5), so it is a footgun for removal; see the `face_restore.py` bullet.** The CodeFormer alternative stays NON-COMMERCIAL and is not shipped. The earlier `--face-id` IP-Adapter FaceID layer was REMOVED (footgun: it needs high strength and corrupts faces at the low removal strength). No original pixels are copied or frozen, **BUT removal at the low vendor-adaptive strength is CONTENT × PIPELINE dependent and NEITHER pipeline clears all content -- oracle-validated against the OpenAI verifier 2026-06-04 (8 images, strength 0.10/0.15, `--max-resolution 1536`).** The survivors FLIP by content type: **photoreal** (a 9-face grid, a bracelet product photo) SURVIVES controlnet but CLEARS `default` (controlnet's dense edge map keeps the regen too close to the original, so the SynthID-destroying perturbation never happens; plain img2img perturbs photoreal texture enough); **flat graphic** (a logo/poster with large flat color fills) SURVIVES `default` but CLEARS controlnet (at low strength img2img barely changes flat fills so SynthID persists there, while controlnet repaints them more freely); a flat **text** card cleared under both. **Root cause is insufficient STRENGTH, not the pipeline: at 0.10 the low-change regions -- dense-edge photoreal under controlnet, large flat fills under `default` -- are not perturbed enough to destroy SynthID. The vendor-adaptive 0.10 from the June study is NOT universally sufficient (that study's content happened to clear at 0.10).** The robust fix is a HIGHER strength, oracle-revalidated per content type (controlnet can be cranked harder without losing structure; a lower `controlnet_conditioning_scale` also frees the regen on photoreal). So at today's default strength **both pipelines AND `--auto` can LEAVE SynthID on some content** -- a removal-priority caller (raiw.cc) MUST oracle-validate strength across content types before adopting, not pick a pipeline and assume removal. **Follow-up same day: re-running the two photoreal survivors through controlnet at an explicit `--strength 0.15` cleared BOTH on the oracle -- BUT one of them (the bracelet) had SURVIVED the SAME 0.15 controlnet config in the first pass (only the random, unset seed differed). So removal near the threshold is SEED-NON-DETERMINISTIC: the same image+pipeline+strength+resolution can pass or fail run-to-run (img2img uses `seed=None`/random unless `--seed` is passed, and there is no local SynthID detector to self-verify). 0.15 is the borderline, NOT a robust floor -- pick a strength with MARGIN (controlnet ~>= 0.20) rather than exactly on it; the content×pipeline table's 0.15 data point is near-threshold noise. A confirming run at `--strength 0.20` controlnet cleared BOTH photoreal survivors on the oracle (ladder: 0.10 grid detected → 0.15 borderline/non-deterministic → 0.20 both clean), so **0.20 is the recommended robust controlnet floor for OpenAI photoreal** (one margin run, not an N-run repeatability proof -- a service should add margin or verify repeatability since there is no local SynthID detector to self-check). **Engineering follow-up for raiw.cc: the controlnet pipeline should use a HIGHER vendor strength than `default` -- it currently shares `resolve_strength` (0.10/0.15, tuned for plain img2img), but controlnet's edge map preserves structure so it needs ~0.20+; calibrate per vendor/content on the GPU worker, do NOT just reuse the `default` ladder.** See `docs/synthid.md` §5.5.** **Lesson: visual-quality + face-recovery validation does NOT prove watermark removal -- only the SynthID oracle does, across MULTIPLE content types; never infer removal from sharpness/identity, and never conclude from a partial result (the photoreal-only data first read as "controlnet shields, default removes" -- the flat-graphic result reversed it).** `controlnet_conditioning_scale` (CLI `--controlnet-scale`, default 1.0) is the structure-preservation knob (higher = closer to the original structure); fp32 on cpu/mps, fp16-fixed VAE on cuda/xpu. The `controlnet` profile is threaded explicitly (`WatermarkRemover(pipeline=...)` / `InvisibleEngine(pipeline=...)`), NOT inferred from `model_id`. This productionizes the `scripts/controlnet_sweep.py` prototype; see `docs/controlnet-removal-pipeline-research.md`. **Forensic-stealth caveat still applies** (arXiv:2605.09203): defeating the SynthID verifier is not forensic invisibility -- a "this image went through a removal pipeline" classifier can still flag the output. diff --git a/docs/controlnet-removal-pipeline-research.md b/docs/controlnet-removal-pipeline-research.md index 49391fa..8dd82bc 100644 --- a/docs/controlnet-removal-pipeline-research.md +++ b/docs/controlnet-removal-pipeline-research.md @@ -39,6 +39,79 @@ AND a text-legibility check. The central tension may be fundamental and must be conditioning strong enough to keep text legible may suppress regeneration enough to let SynthID survive; the regeneration strong enough to scrub may deform text regardless of edges. +## Oracle validation 2026-06-04 — measured answer (AUTHORITATIVE; supersedes pre-oracle "scrubs SynthID" claims below) + +The central tension the summary predicted is now MEASURED against the live oracles +(OpenAI `openai.com/verify` for OpenAI content; the Gemini app "Verify with SynthID" +for Google content — each detects only its own vendor's payload). **Verdict: at the +shipped low vendor-adaptive strength, controlnet is NOT a drop-in SynthID remover.** +It preserves structure so well that the watermark survives on exactly the photoreal +content it protects. Controlnet is the text/structure PRESERVATION pipeline; removal +is set by STRENGTH, separately calibrated, not by the pipeline choice. + +This section is the single consolidated reference for the controlnet pipeline's +removal behavior. (Mirrored briefly in `docs/synthid.md` §5.5 and the CLAUDE.md +controlnet / face_restore bullets, which point here.) + +### What we measured (real gpt-image + Gemini originals) + +**1. Content × pipeline — neither pipeline clears all content at low strength.** +OpenAI set, strength = vendor-adaptive (0.10 OpenAI / 0.15 no-C2PA), `--max-resolution 1536`: + +| content | controlnet (`--auto`) | plain `default` | +|---|---|---| +| flat text card | clean | clean | +| flat graphic (logo/poster, large flat fills) | clean | **SynthID detected** | +| photoreal (9-face grid) | **SynthID detected** | clean | +| photoreal (bracelet product photo) | **SynthID detected** | clean | + +Mechanism: controlnet's dense edge map keeps the regeneration very close to the original +on photoreal, so the SynthID-destroying perturbation never happens; but it freely +repaints large flat fills. Plain img2img at low strength perturbs photoreal texture +enough yet barely touches flat fills. So the survivors FLIP by content type — pipeline +choice alone does not guarantee removal. + +**2. Seed non-determinism near threshold.** img2img uses a random seed unless `--seed` +is passed, and there is no local SynthID detector to self-verify. The bracelet survived +controlnet @0.15 in one run and CLEARED @0.15 in another (same pipeline+strength+res). +So a single clean run does NOT establish a strength as safe — characterizing a reliable +floor needs a seed-repeatability sweep (N runs, varied seed), not one pass. + +**3. Per-vendor controlnet strength ladder (photoreal, oracle):** +- **OpenAI:** 0.10 detected → 0.15 borderline/non-deterministic → 0.20 cleared (with margin). +- **Gemini** (harder vendor; default 0.15 vs OpenAI 0.10): most cleared at 0.15–0.25. +- **Resolution is NOT the lever:** SynthID is robust to downscaling, and the study's + trend says LOWER processing res needs LESS strength, so 1024 was never the wall. A + Gemini face that resisted 0.15/0.20/0.25 was blocked by face-restore (#4), not strength. + +**4. `--restore-faces` RE-INTRODUCES SynthID (was the "stubborn face" mystery).** A +Gemini face image stayed SynthID-detected through controlnet 0.15/0.20/0.25 WITH restore, +but CLEARED at 0.20 with `--no-restore-faces` (clean single-variable A/B). GFPGAN runs on +the **ORIGINAL watermarked** face at fidelity weight 0.5, blends ~half its pixels with the +StyleGAN2 prior, and composites that back OVER the diffusion-cleaned face → the watermark +returns in the face region. Content-dependent (smaller faces can clear with restore). So +raising strength cannot fix it — the face is re-pasted from the original after diffusion. +This also corrects the prior "GFPGAN scrubs SynthID / oracle-confirmed clean" claim (it was +checked on one lucky image). + +### Recommendations for a removal pipeline (raiw.cc) + +- **Treat controlnet as PRESERVATION, not removal.** Choose it for text/structure content, + `default` for photoreal; removal efficacy comes from STRENGTH in both. +- **Give controlnet a higher, per-vendor strength than `default`** (today both share + `resolve_strength` 0.10/0.15, tuned for plain img2img). Oracle floors so far: OpenAI + ~0.20 with margin, Gemini ~0.20–0.25. Calibrate on the GPU worker with a seed-repeat + sweep (cheap there), not single local runs. +- **Rework `--restore-faces` before any removal use:** run GFPGAN on the diffusion-CLEANED + image (not the original), or drop the weight well below 0.5, or leave it off — then + re-validate on the oracle. +- **No local SynthID detector exists** → the service can't self-verify; bake in strength + margin and periodic oracle spot-checks. +- **Lesson:** visual-quality / face-identity recovery does NOT prove removal — only the + oracle does, across MULTIPLE content types; never conclude from a partial result (the + photoreal-only data first read as "controlnet shields, default removes"; the flat-graphic + result reversed it; the face mystery was restore, not strength). + ## Findings (with confidence and sources) ### Finding 1 — confidence: high diff --git a/docs/synthid.md b/docs/synthid.md index 07ec056..908cbd0 100644 --- a/docs/synthid.md +++ b/docs/synthid.md @@ -387,17 +387,44 @@ the plain `default` SDXL img2img pass is the shippable path. - **Text + structure:** `--pipeline controlnet` (SDXL img2img + a canny ControlNet, experimental/opt-in) conditions the regeneration on the edge map, so text and - structure stay sharp - while every pixel is still regenerated -- SynthID is removed everywhere. Verified - better than plain img2img at the same strength (text stays legible where plain - garbles it), and the controlnet background scrub reads clean on the oracle. + structure stay sharp while every pixel is still regenerated. Text legibility is + better than plain img2img at the same strength (text stays readable where plain + garbles it). **BUT removal efficacy at the low vendor-adaptive strength is CONTENT × + PIPELINE dependent and NEITHER pipeline clears all content -- oracle-validated + 2026-06-04 (8 OpenAI images, strength 0.10/0.15, max-res 1536).** The survivors FLIP + by content type: **photoreal** (a 9-face grid, a bracelet product photo) SURVIVES + controlnet but CLEARS `default`; **flat graphic** (a logo/poster with large flat + color fills) SURVIVES `default` but CLEARS controlnet; a flat **text** card cleared + under both. Why: controlnet's dense edge map keeps the regen too close to the + original on photoreal (so SynthID survives) but freely repaints flat fills (so it + clears them); plain img2img at low strength perturbs photoreal texture enough but + barely touches flat fills. **Root cause = insufficient STRENGTH, not the pipeline: + the vendor-adaptive 0.10 is NOT universally sufficient (the June numbers below held + for the content they were measured on). The robust fix is a HIGHER strength, + oracle-revalidated per content type (controlnet can be cranked harder without losing + structure; a lower `controlnet_conditioning_scale` also frees the regen on + photoreal).** So neither `--pipeline controlnet` nor plain `default` is a drop-in + removal guarantee at today's strength -- pick by what you must PRESERVE (controlnet + for text/structure), then raise strength until the oracle reads clean. (The earlier + "reads clean on the oracle" claim held only for the one flat/text-background case it + was checked on; it does not generalize.) - **Face identity:** canny holds face *structure* but not *identity*. Shipped as the optional `--restore-faces` GFPGAN post-pass (`face_restore.py`, the `restore` extra, experimental/opt-in, off by default). It runs GFPGAN on the ORIGINAL - faces and feather-composites the restored face REGIONS into the cleaned image: - GFPGAN RE-SYNTHESIZES each face from a StyleGAN2 prior (GAN pixels, not original - -> scrubs SynthID) at a low fidelity weight (`--restore-faces-weight`, default - `0.5`). Oracle-confirmed clean in face regions with identity preserved. Commercial- + faces and feather-composites the restored face REGIONS into the cleaned image. + **WARNING (oracle-confirmed 2026-06-04): this pass can RE-INTRODUCE SynthID into + the face regions -- the earlier "GFPGAN re-synthesizes from a StyleGAN2 prior -> + scrubs SynthID -> oracle-confirmed clean" claim was WRONG.** At the default fidelity + weight `0.5` GFPGAN blends ~half the ORIGINAL (watermarked) face pixels with the + prior, and SynthID is robust to that partial blend, so the composited face carries + the watermark back in -- over the diffusion-cleaned face. Confirmed by a clean A/B: + `gemini_3` read SynthID-detected after controlnet @ 0.20/0.25 WITH restore, but + NOT-detected after the same controlnet @ 0.20 with `--no-restore-faces` (only + restore differed). Content-dependent (a second face image cleared WITH restore), + which is why a single-image check earlier read "clean". **Fix directions (not yet + done): run GFPGAN on the diffusion-CLEANED image not the original; or drop the + weight well below 0.5; or leave restore OFF for removal -- each needs oracle + re-validation.** Commercial- safe (GFPGAN Apache-2.0 + RetinaFace MIT); the CodeFormer alternative is NON-COMMERCIAL and is not shipped. (An IP-Adapter FaceID approach was tried and REMOVED -- it needs high denoise strength and corrupts faces at removal strength; @@ -408,7 +435,14 @@ the plain `default` SDXL img2img pass is the shippable path. There is no single permanent correct strength, but the controlled June 2026 study (section 2.2) gives empirical floors: -- **OpenAI**: 0.05 clears across 1024-1600 (n=4). 0.30 is large overkill here. +- **OpenAI**: 0.05 clears across 1024-1600 (n=4) -- **but content-dependent, NOT + universal.** The follow-up oracle pass (2026-06-04, 8 images) found a flat-graphic + OpenAI logo/poster still SynthID-detected after `default` at 0.10, and photoreal + images still detected after controlnet at 0.10/0.15: at low strength the + low-change regions (large flat fills under `default`, dense edges under controlnet) + are not perturbed enough. So the 0.05 floor held only for the n=4 content it was + measured on; treat it as a lower bound, not a guarantee, and raise + oracle-recheck + per content type (see §5.1 controlnet bullet). - **Google (capped 1536)**: 0.15 (n=4); 0.05 and 0.10 do not clear. - **Google native 2816**: not locally measured; likely needs >= 0.30 (vendor + resolution stack). Use a GPU or `--max-resolution 1536`. @@ -447,6 +481,87 @@ diffusion-based removers are simultaneously the most forensically detectable (AUROC up to 0.9999). The tradeoff is unavoidable with current diffusion-based approaches: defeating the vendor's verifier is not the same as being clean. +### 5.5 Oracle validation log -- 2026-06-04 OpenAI pass + +Eight OpenAI `gpt-image` originals run through both pipelines and checked on +openai.com/verify (the OpenAI SynthID oracle). `--max-resolution 1536`; strength +is the vendor-adaptive default (`vendor_for_strength`): images with an OpenAI C2PA +manifest get `OPENAI_STRENGTH` 0.10, the one without C2PA falls to +`UNKNOWN_STRENGTH` 0.15. "detected" = SynthID still found (removal FAILED); +"clean" = SynthID not detected. + +| image | content type | size | strength | `--auto`/controlnet | `default` | +|---|---|---|---|---|---| +| typography card | flat text | 1122x1402 | 0.10 | clean | clean | +| raiw.cc poster | flat graphic (logo + flat fills) | 1024x1536 | 0.10 | clean | **detected** | +| 9-face grid | photoreal | 1448x1086 | 0.10 | **detected** | clean | +| bracelet product photo | photoreal | 1600x1600 | 0.15 | **detected** | clean | + +(The other four cleared under both and are omitted.) **Reading:** at this strength +NEITHER pipeline removes SynthID on all content -- the survivors flip by content +type. Photoreal survives controlnet / clears `default`; flat graphic survives +`default` / clears controlnet; flat text clears both. + +**Follow-up: removal near the threshold is NON-DETERMINISTIC (seed-dependent).** +Re-running the two photoreal survivors through controlnet at an explicit +`--strength 0.15` (`--auto`, same `--max-resolution 1536`) cleared BOTH on the +oracle (SynthID not detected). But the bracelet had SURVIVED controlnet at the +SAME 0.15 in the first pass (it was the no-C2PA image, so its vendor-adaptive +strength was already 0.15) -- same pipeline + strength + resolution, only the +random (unset) seed differed between runs. So **0.15 is the borderline floor for +controlnet photoreal, not a robust guarantee**: at the threshold the same +image+settings can pass or fail run-to-run. img2img runs with `seed=None` (random) +unless `--seed` is passed, so a removal SERVICE gets a coin-flip near threshold and +has no local SynthID detector to self-verify. + +**Controlnet strength ladder on the two photoreal images (oracle, `--auto`, +`--max-resolution 1536`):** + +| controlnet strength | 9-face grid | bracelet photo | +|---|---|---| +| 0.10 | detected | (was 0.15) | +| 0.15 | clean | **non-deterministic** (survived pass 1, clean pass 2) | +| **0.20** | **clean** | **clean** | + +**Recommended robust controlnet strength = 0.20** (0.05 of margin above the 0.15 +non-deterministic borderline); both photoreal survivors cleared at 0.20. Honest +caveat: 0.20 is one confirming run WITH margin, not an N-run repeatability proof -- +for a removal service, add a little more margin or validate repeatability, since +there is no local SynthID detector to self-check. **Implications:** (1) the +content×pipeline table above conflates a borderline/non-deterministic 0.15 result +with deterministic content behavior -- the photoreal-survives-controlnet effect is +solid at 0.10 but at 0.15 it is near-threshold noise; (2) for reliable removal pick +a strength with MARGIN above the borderline (controlnet >= 0.20), not exactly on +it; (3) **engineering follow-up for raiw.cc: the controlnet pipeline should use a +HIGHER vendor strength than `default` (it currently shares `resolve_strength`) -- +e.g. controlnet floor 0.20 -- calibrated per vendor/content on the GPU worker where +batches are cheap. The shared 0.10/0.15 is tuned for `default`, not controlnet.** +Source images are private (faces / product shots), not committed; reproduce on any +photoreal + flat-graphic gpt-image pair, varying the seed, and re-checking the +oracle. + +**Gemini pass + the face-restore re-introduction (2026-06-04).** Four Gemini +originals via `--auto` (controlnet) at `--max-resolution 1024`, checked on the +Gemini "Verify with SynthID" oracle (Google content needs the Google oracle, not +openai.com/verify): +- Most cleared at controlnet 0.15-0.25; `gemini_3` (a large central FACE, +restore) + stayed **SynthID-detected at controlnet 0.15, 0.20 AND 0.25** -- raising strength + did not crack it. +- **Root cause was the face-restore pass, not strength/resolution.** `gemini_3` at + controlnet 0.20 with `--no-restore-faces` read **SynthID-NOT-detected** (clean + A/B, only restore differed). GFPGAN runs on the ORIGINAL watermarked face and at + weight 0.5 blends ~half its pixels back, re-introducing SynthID into the + composited face over the diffusion-cleaned result (see §5.1 face-identity bullet). +- (Side note: reducing the processing resolution does NOT weaken SynthID -- it is + robust to downscaling by design, and the study's resolution trend says LOWER + processing res needs LESS strength, so 1024 was never the wall.) + +**Net for raiw.cc:** (1) controlnet needs a higher, per-vendor strength than +`default` (OpenAI ~0.20, Gemini >= ~0.20-0.25), oracle-calibrated; (2) the +`--restore-faces` pass can re-add SynthID and must be reworked (restore on the +cleaned image / lower weight / off) before it is safe in a removal pipeline; (3) +removal near threshold is seed-non-deterministic, so use margin. + --- ## References