mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-07 06:28:36 +02:00
Add high-fidelity Qwen Z-Image removal pipeline
This commit is contained in:
@@ -96,7 +96,7 @@ checked on one lucky image).
|
||||
|
||||
### Certified controlnet strength floors (Modal GPU sweep + oracle, 2026-06-04)
|
||||
|
||||
Run via the isolated `raiw-controlnet-cert` Modal app (`raiw-app/modal_cert.py`):
|
||||
Run via an isolated Modal certification harness:
|
||||
controlnet, `restore_faces` OFF (it re-introduces SynthID), `--max-resolution 1536`,
|
||||
each image checked on ITS OWN vendor oracle (OpenAI -> openai.com/verify, Gemini -> the
|
||||
Gemini app; the two payloads are vendor-specific and never cross-checked):
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 2.1 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 2.0 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 8.2 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 5.8 MiB |
@@ -41,12 +41,14 @@ that cannot afford LaMa's ~4.7 GB peak pins `--backend migan` explicitly.
|
||||
|
||||
### Tiled diffusion for large inputs (`--tile`, issue #10)
|
||||
|
||||
`--tile` (OFF by default; `--tile-size` default 1024, `--tile-overlap` default 128) processes the diffusion pass in overlapping sliding-window tiles instead of one forward pass, so a large image is regenerated at **native resolution** without the OOM and without the lossy `--max-resolution` downscale round-trip. It engages only when the long side exceeds `--tile-size`; a sub-tile image runs a single pass unchanged. `WatermarkRemover.remove_watermark` refactors the single-image `_generate` into a per-tile `_generate_one` (the ControlNet canny edge map is rebuilt per tile, so structure preservation works tile-local) and routes it through `noai.tiling.run_tiled` when tiling is active. The geometry and blend math are pure helpers, unit-tested without the model (`tests/test_tiling.py`):
|
||||
`--tile` (OFF by default; `--tile-size` default 1024, `--tile-overlap` default 128) processes the diffusion pass in overlapping sliding-window tiles instead of one forward pass, so a large image is regenerated at **native resolution** without the OOM and without the lossy `--max-resolution` downscale round-trip. It engages only when the long side exceeds `--tile-size`; a sub-tile image runs a single pass unchanged. The SDXL, ControlNet, and base Qwen paths refactor the single-image `_generate` into a per-tile `_generate_one` (the ControlNet canny edge map is rebuilt per tile, so structure preservation works tile-local) and route it through `noai.tiling.run_tiled` when tiling is active. `qwen-zimage` instead tiles only its global Qwen pass, feather-blends that result, and then runs YuNet, SAM, and Z-Image once against the full original/result pair. The geometry and blend math are pure helpers, unit-tested without the model (`tests/test_tiling.py`):
|
||||
|
||||
- `plan_tiles(w, h, tile_size, overlap)` lays out a row-major grid where every tile is exactly `tile_size` (the last tile on each axis is pulled back flush to the far edge, simply overlapping its predecessor more). Uniform tile size keeps each diffusion pass at SDXL's preferred dimension.
|
||||
- `feather_weights(w, h, overlap)` is a separable linear taper, ~1 in the interior and ramping toward each edge, kept **strictly positive** so the normalized accumulate-and-divide blend (`accum / weight_sum`) is a partition of unity: a region covered by one feathered edge (an image corner) still divides cleanly. Identical (unchanged) tiles therefore reconstruct the input exactly -- the seam-free guarantee, asserted in `test_identity_generate_reconstructs_image`.
|
||||
|
||||
CAVEAT: each tile is an **independent** low-strength regeneration. At the certified removal strengths (0.20-0.30) the per-tile drift is small and the feather blend hides the seams, but tiling is a memory workaround, not a quality upgrade over a single native pass -- a 32 GB MPS box that clears the native UNet peak should prefer no tiling. The MPS->CPU fallback still applies per tile; if the first tile falls back to CPU, the device stays CPU for the rest of the image.
|
||||
CAVEAT: each tile is an **independent** low-strength regeneration. At the current SDXL/ControlNet defaults (0.10-0.15) the per-tile drift is small and the feather blend hides the seams, but tiling is a memory workaround, not a quality upgrade over a single native pass -- a 32 GB MPS box that clears the native UNet peak should prefer no tiling. The MPS->CPU fallback still applies per tile; if the first tile falls back to CPU, the device stays CPU for the rest of the image.
|
||||
|
||||
For `qwen-zimage`, the global denoise is still computed from the full-frame megapixel count and the same resolved seed is reused for every tile. The profile defaults to seed 0, matching the release-candidate oracle run; an explicit seed overrides it. Running the face stage only after blending avoids duplicate regeneration and boundary-local face misses. A real H100 smoke on 2026-07-25 exercised the shipped branch on a 4096x3072 input (20 tiles at 1024 with 128 px overlap, seed 0, strength 0.154): it completed in 653.367 seconds after 43.741 seconds of setup, preserved the exact dimensions, and peaked at 22.732 GiB allocated / 23.861 GiB reserved CUDA memory. Visual inspection found no tile seams. The worst tile-boundary gradient-change line was at the 98.563 percentile of all image lines (2.522 standard deviations), below the preselected 99th-percentile outlier threshold; overview fidelity was MAE 3.332%, PSNR 26.564 dB, and global SSIM 0.988627. This no-face input validates the global tiled execution and blend, not the post-blend face path. The July 25 seed-0 oracle result still certifies exact non-tiled candidate bytes only; tiled SynthID efficacy requires a separate provider-oracle check.
|
||||
|
||||
**Concrete MPS data points (the OOM is memory-tier-dependent, NOT a hard MPS limit):** on a ~24 GB unified-memory machine (verified 2026-05-25, 1254x1254 gpt-image SDXL, fp32) native res OOMs at the *UNet* step (peak ~17 GiB), not only the VAE decode, and the auto-fallback in `img2img_runner` reloads on CPU and finishes (slow, ~13 min) -- the output is still weight-identical and defeats SynthID, so "looks hung/crashed" on Mac is usually this CPU fallback, not a pipeline error. On a **32 GB** unified-memory machine the same default SDXL pass runs entirely on MPS with **no CPU fallback** (verified 2026-05-31, 1122x1402 gpt-image, `all`/default, ~155 s end-to-end), so 32 GB clears the native-res UNet peak that 24 GB could not. Adding `enable_vae_tiling()` alone does NOT prevent the 24 GB OOM (the peak is the UNet, not the VAE). The fast Mac workarounds for memory-constrained machines are fp16 on MPS (roughly halves memory) or `--max-resolution` to cap the long side; neither is wired as the default. The `controlnet` pipeline adds the canny ControlNet weights on top of SDXL, so its peak is a bit higher than the plain `default` pass; the same MPS->CPU fallback covers an OOM. The native-vs-cap-vs-floor decision lives in the pure helper `invisible_engine._target_size(w, h, max_resolution, min_resolution)` (returns `None` for native, a target tuple for a downscale cap OR an upscale floor; cap takes precedence, the floor is skipped on a min>max misconfig) so it is unit-tested (`tests/test_invisible_engine.py::TestTargetSize`, the #10/#15/#36 regression guard) without loading the model -- keep that logic in the helper, don't re-inline it.
|
||||
|
||||
@@ -128,7 +130,7 @@ CAVEAT: Google's 0.15 was validated only on `--max-resolution 1536`; native larg
|
||||
|
||||
Default strength is vendor-adaptive (see the bullet above); `docs/synthid.md` §2.2 is authoritative for the numbers.
|
||||
|
||||
**Oracle scope (load-bearing):** the Gemini app "Verify with SynthID" is the ONLY valid SynthID oracle (detects Google's mark on any image); `openai.com/verify` is scoped to OpenAI provenance (its own C2PA), NOT a SynthID oracle -- a negative there is meaningless for SynthID. There is no local SynthID detector, so the tool cannot self-check; if the oracle still reads SynthID, raise `--strength` to the lowest value that verifies clean. Only the `sdxl` (plain SDXL img2img; `default` is a back-compat alias) and `controlnet` (SDXL + canny ControlNet) profiles exist; the local `invisible` default is weight-for-weight identical to raiw.cc prod (`fal-ai/fast-sdxl` = `stabilityai/stable-diffusion-xl-base-1.0`, runtime-downloaded, not bundled).
|
||||
**Oracle scope (load-bearing):** the Gemini app "Verify with SynthID" is the ONLY valid SynthID oracle (detects Google's mark on any image); `openai.com/verify` is scoped to OpenAI provenance (its own C2PA), NOT a SynthID oracle -- a negative there is meaningless for SynthID. There is no local SynthID detector, so the tool cannot self-check; if the oracle still reads SynthID, raise `--strength` to the lowest value that verifies clean. The profiles are `sdxl` (plain SDXL img2img; `default` is a back-compat alias), `controlnet` (SDXL + canny ControlNet), `qwen` (Qwen-Image img2img), and the experimental `qwen-zimage` two-stage stack.
|
||||
|
||||
**Forensic-stealth caveat** (arXiv:2605.09203): defeating the SynthID verifier is NOT forensic invisibility -- independent detectors flag *removal-processed* images vs genuinely-clean ones at >98% TPR@1%FPR, so do not over-claim "indistinguishable from a real photo".
|
||||
|
||||
@@ -148,9 +150,9 @@ The robust fix is a HIGHER strength, oracle-revalidated per content type (contro
|
||||
|
||||
**Follow-up same day: re-running the two photoreal survivors through controlnet at an explicit `--strength 0.15` cleared BOTH on the oracle -- BUT one of them (the bracelet) had SURVIVED the SAME 0.15 controlnet config in the first pass (only the random, unset seed differed). So removal near the threshold is SEED-NON-DETERMINISTIC: the same image+pipeline+strength+resolution can pass or fail run-to-run (img2img uses `seed=None`/random unless `--seed` is passed, and there is no local SynthID detector to self-verify). 0.15 is the borderline, NOT a robust floor -- pick a strength with MARGIN (controlnet ~>= 0.20) rather than exactly on it; the content×pipeline table's 0.15 data point is near-threshold noise. A confirming run at `--strength 0.20` controlnet cleared BOTH photoreal survivors on the oracle (ladder: 0.10 grid detected → 0.15 borderline/non-deterministic → 0.20 both clean), so **0.20 is the recommended robust controlnet floor for OpenAI photoreal** (one margin run, not an N-run repeatability proof -- a service should add margin or verify repeatability since there is no local SynthID detector to self-check).
|
||||
|
||||
**Engineering follow-up DONE 2026-06-09 (three coupled changes):** (1) **strength raised + unified** -- `resolve_strength(strength, vendor)` now applies ONE vendor-adaptive ladder (the certified controlnet floors 0.20/0.30/0.30) to BOTH pipelines; see the DEFAULT STRENGTH bullet above for why one ladder covers `sdxl`. (2) **`controlnet` is now the DEFAULT pipeline** (CLI `--pipeline` default = `controlnet` + both engine ctors). Rationale: with the certified higher ladder it clears BOTH content classes that flipped in the content-x-pipeline table (photoreal AND flat graphic), whereas plain SDXL left SynthID on flat graphics -- so controlnet is the more removal-robust default. Cost: every non-`--auto` run now downloads the canny ControlNet weights + a higher memory peak (MPS->CPU fallback covers OOM). (3) **the plain-SDXL profile was renamed `default` -> `sdxl`** (`watermark_profiles.SDXL_PROFILE`/`normalize_profile`); `default` stays as a back-compat CLI/ctor alias (the `--pipeline` Choice accepts `sdxl`/`controlnet`/`default`, a click callback `_normalize_pipeline` maps `default`->`sdxl` AND warns that `default` is deprecated). (4) **the content-detection layer + `--auto` planner were removed and `--auto` was retired to a deprecated alias for `--adaptive-polish`** -- see the dedicated `auto_config.py`-removal bullet above (controlnet is the default pipeline and the polish self-gates, so detection changed nothing). raiw.cc still needs its own per-vendor/content calibration on the GPU worker for native resolution. The Gemini-native resolution caveat stands: controlnet 0.30 is certified only <=1536.** **CERTIFIED 2026-06-04 via the isolated `raiw-controlnet-cert` Modal app (`raiw-app/modal_cert.py`), restore OFF, ≤1536, each vendor on its own oracle: controlnet floors are OpenAI 0.20 (2 photoreal × 3 seeds = 6/6 clean; the 0.15-flipper is seed-robust at 0.20) and Gemini 0.30 (0.20 detected → 0.30 clean on 2/2 seeds). OpenAI 0.20 transfers to prod (resolution-independent); Gemini 0.30 holds only ≤1536 — Gemini is resolution-sensitive and raiw.cc runs NATIVE (`max_resolution=0`), so cap Gemini ≤1536 + use 0.30, or native-calibrate (~0.35+). Prod recipe: controlnet + per-vendor floor in `resolve_strength` (not the default ladder) + FIXED seed (kills the non-determinism).
|
||||
**Engineering follow-up DONE 2026-06-09 (three coupled changes):** (1) **strength raised + unified** -- `resolve_strength(strength, vendor)` now applies ONE vendor-adaptive ladder (the certified controlnet floors 0.20/0.30/0.30) to BOTH pipelines; see the DEFAULT STRENGTH bullet above for why one ladder covers `sdxl`. (2) **`controlnet` is now the DEFAULT pipeline** (CLI `--pipeline` default = `controlnet` + both engine ctors). Rationale: with the certified higher ladder it clears BOTH content classes that flipped in the content-x-pipeline table (photoreal AND flat graphic), whereas plain SDXL left SynthID on flat graphics -- so controlnet is the more removal-robust default. Cost: every non-`--auto` run now downloads the canny ControlNet weights + a higher memory peak (MPS->CPU fallback covers OOM). (3) **the plain-SDXL profile was renamed `default` -> `sdxl`** (`watermark_profiles.SDXL_PROFILE`/`normalize_profile`); `default` stays as a back-compat CLI/ctor alias (the `--pipeline` Choice accepts `sdxl`/`controlnet`/`default`, a click callback `_normalize_pipeline` maps `default`->`sdxl` AND warns that `default` is deprecated). (4) **the content-detection layer + `--auto` planner were removed and `--auto` was retired to a deprecated alias for `--adaptive-polish`** -- see the dedicated `auto_config.py`-removal bullet above (controlnet is the default pipeline and the polish self-gates, so detection changed nothing). A production caller still needs its own per-vendor/content calibration at its deployed native resolution. The Gemini-native resolution caveat stands: controlnet 0.30 is certified only <=1536.** **CERTIFIED 2026-06-04 via an isolated Modal certification harness, restore OFF, ≤1536, each vendor on its own oracle: controlnet floors are OpenAI 0.20 (2 photoreal × 3 seeds = 6/6 clean; the 0.15-flipper is seed-robust at 0.20) and Gemini 0.30 (0.20 detected → 0.30 clean on 2/2 seeds). OpenAI 0.20 transfers to production (resolution-independent); Gemini 0.30 holds only ≤1536 — Gemini is resolution-sensitive, so a native-resolution caller should cap Gemini to ≤1536 at 0.30 or calibrate its native path (~0.35+). Production recipe: controlnet + per-vendor floor in `resolve_strength` (not the default ladder) + FIXED seed (kills the non-determinism).
|
||||
|
||||
**No face-restore in the library:** every approach evaluated (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned, 2026-06-04 - 2026-06-08 cert sweeps) regenerated the face via SDXL diffusion -- the output face inherited SDXL "clean skin" gloss and lost original identity precision, looking MORE AI-generated than the cleaned image, not less. The drifted face from controlnet 0.20 is the least-AI state we can reach; for a paid service that's the prod output. See `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up".**
|
||||
**No face-restore runs in the default controlnet profile:** every earlier approach evaluated there (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned, 2026-06-04 - 2026-06-08 cert sweeps) regenerated the face via SDXL diffusion -- the output face inherited SDXL "clean skin" gloss and lost original identity precision, looking MORE AI-generated than the cleaned image, not less. The separate experimental `qwen-zimage` profile now tests a different architecture, Z-Image regeneration from the original SAM-masked face crop. Its first ArcFace/LPIPS run is recorded below, but it still needs its own oracle and multi-image face/text matrix.**
|
||||
|
||||
See `docs/synthid.md` §5.5 + `docs/controlnet-removal-pipeline-research.md` (certified floors table).** **Lesson: visual-quality + face-recovery validation does NOT prove watermark removal -- only the SynthID oracle does, across MULTIPLE content types; never infer removal from sharpness/identity, and never conclude from a partial result (the photoreal-only data first read as "controlnet shields, default removes" -- the flat-graphic result reversed it).**
|
||||
|
||||
@@ -172,6 +174,37 @@ The scrub still comes from the img2img `strength` (same lever as SDXL); the call
|
||||
|
||||
**Conclusion: Qwen wins TEXT only for clean body text on a plain background with NO faces; controlnet wins faces AND display/decorative text in a scene. So `qwen` is a MANUAL `--pipeline qwen` opt-in, not a routed lane.** A content `--pipeline auto` router + a faces+text mixed dual-pass were prototyped and DROPPED (2026-06-20): on the canonical faces+text case (the abba poster, faces + display text) controlnet won EVERY metric incl. text (CER 0.114 vs qwen 0.379), so grafting qwen text only hurts; and "text→qwen" is undecidable cheaply (body-vs-display text is what matters). Caveat: `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`, Gemini 0.25), so `--pipeline qwen` gets the 0.25 Gemini floor automatically — the old manual `--strength 0.25` workaround is retired. `_build_qwen_kwargs` now passes an explicit height/width (qwen squished non-square inputs to 1024² without it). Flat-graphic content was not in the sample.
|
||||
|
||||
**Improving Qwen (ship vs improve):** the cited research lives in `docs/qwen-improvement-research.md` -- read it before extending the `qwen` pipeline. Verdict: shippable as an opt-in text lane. **The "add a Qwen-Image ControlNet to fix face smoothing" lead was built, measured, and CLOSED (2026-06-20):** a DiffSynth-Studio Qwen + Apache-2.0 blockwise-canny ControlNet at the Gemini floor 0.25 did NOT restore face skin texture (face Laplacian-variance retention flat 0.40 -> 0.40, 13/16 faces within +-0.02; the SDXL+canny target 0.62 was not approached), because canny carries edges not skin grain and Qwen's higher Gemini floor (0.25 vs SDXL+canny 0.15) forces more smoothing -- and a deep-research sweep confirmed NO permissively-licensed Qwen tile/detail/realism/skin ControlNet exists anywhere (every Qwen conditioning is geometry). So **faces stay on SDXL+controlnet; Qwen is the text lane, not a face fix.** The strongest remaining lead is **Z-Image-Turbo** (6B, Apache-2.0, `ZImageImg2ImgPipeline`, scrub mechanism preserved) -- its own SynthID floor and face/text fidelity are UNMEASURED; that is the next experiment. Non-regenerative high-frequency detail re-injection is NOT safe by assumption (the "clean-output high frequencies do not carry the watermark" claim was refuted) -- it must be oracle-gated. Always validate any improvement at the certified floors with `scripts/fidelity_metrics.py` first.
|
||||
**Improving Qwen (ship vs improve):** the cited research lives in `docs/qwen-improvement-research.md` -- read it before extending the `qwen` pipeline. Verdict: shippable as an opt-in text lane. **The "add a Qwen-Image ControlNet to fix face smoothing" lead was built, measured, and CLOSED (2026-06-20):** a DiffSynth-Studio Qwen + Apache-2.0 blockwise-canny ControlNet at the Gemini floor 0.25 did NOT restore face skin texture (face Laplacian-variance retention flat 0.40 -> 0.40, 13/16 faces within +-0.02; the SDXL+canny target 0.62 was not approached), because canny carries edges not skin grain and Qwen's higher Gemini floor (0.25 vs SDXL+canny 0.15) forces more smoothing -- and a deep-research sweep confirmed NO permissively-licensed Qwen tile/detail/realism/skin ControlNet exists anywhere (every Qwen conditioning is geometry). So **base Qwen stays the text lane, not a face fix.** The distinct Z-Image face-crop lead is now implemented as `qwen-zimage`; direct face comparisons are below, and its exact current six-output candidate is negative in the corresponding provider oracles. Broad seeded removal and text behavior remain unmeasured. Non-regenerative high-frequency detail re-injection is NOT safe by assumption (the "clean-output high frequencies do not carry the watermark" claim was refuted) -- it must be oracle-gated.
|
||||
|
||||
**Seed as a quality lever (measured, openai_1 at 0.10, seeds 0-4):** the seed barely moves whole-image fidelity (img LPIPS 0.062-0.065, SSIM 0.855-0.857, PSNR 28.5-28.7 — flat) but does shift TEXT legibility (OCR CER 0.241-0.290, ~17% spread) -- the seed changes WHICH details get regenerated, not the overall level. So a per-image best-of-N-seed selection is a WEAK, text-only lever (pick the lowest-CER seed that still scrubs; fidelity selection needs no oracle). Not worth the N× cost for general use -- pin one decent seed in prod; reserve best-of-N for text-heavy premium cases.
|
||||
|
||||
## `qwen-zimage` pipeline
|
||||
|
||||
`--pipeline qwen-zimage` is the recommended high-quality SynthID removal mode when CUDA capacity is available and fidelity matters more than latency or cost. It remains a manual opt-in so the broadly compatible, much cheaper ControlNet path can stay the default. The profile ports the upstream two-stage workflow: an input-resolution Qwen-Image-2512 Lightning Canny pass regenerates the frame, then original face crops are segmented and regenerated with Z-Image Turbo before a feathered paste. DiffSynth requires both pixel inputs and the requested dimensions to use the same /16 latent grid, so each stage makes that small alignment resize internally and restores the global result to the original dimensions. The profile defaults to deterministic seed 0 because the release-candidate oracle evidence was produced at that seed; explicit callers can still override it.
|
||||
|
||||
The port is architectural, not bit-identical. The active graph was traced from upstream commit `3007d0351596ae0a78b7074dae7ad179710b1e48`, including its linked Impact Pack implementation. It confirms that the active face path is YOLO + SAM; the MediaPipe node visible on the canvas is unconnected. The port keeps the two adaptive-denoise formulas, four-step Qwen Lightning stage, Canny thresholds and scale, AuraFlow shift 3 equivalent, original-image face source, SAM center + box prompts, IoU-0.93 proposal union with highest-score fallback, detector-box intersection, crop factor 2.5, 768 face guide, 1024 crop cap, eight-step face stage, and paste feather 10.
|
||||
|
||||
Four runtime differences remain. This package uses full safetensors instead of the source graph's quantized GGUF models, YuNet instead of Ultralytics YOLO to avoid an AGPL runtime, DiffSynth FlowMatch samplers instead of ComfyUI's DPM++ 2M / SGM Uniform and `res_2s` / `bong_tangent` pairs, and no latent-space 20 px detailer noise-mask feather. The face crop is regenerated in full, then only the feathered SAM pixels are composited back, so generated pixels outside that mask are discarded. These differences prevent an exact-output claim even though the architecture and active decision path match.
|
||||
|
||||
The default full-frame denoise is resolution-adaptive, not vendor-adaptive. The face denoise is separate and scales from the largest detected face. `--strength` overrides only the global Qwen stage. The profile fixes the global step count at four because its Lightning LoRA is distilled for that schedule; the face stage uses its own eight-step schedule. `--model` is unsupported. `--tile` follows the global-only route described above, with one full-frame face stage after blending.
|
||||
|
||||
Direct comparison now covers two official upstream before/after pairs plus the existing crowded `gemini_3` fixture. The published upstream examples were scored against their own original inputs, with the upstream output resized back only for metric alignment where necessary:
|
||||
|
||||
| Case | Result | ArcFace identity | Face LPIPS | Texture retention | Image LPIPS | SSIM |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| Upstream example 10 | published upstream | 0.976 | 0.172 | 0.166 | 0.259 | 0.627 |
|
||||
| Upstream example 10 | local `qwen-zimage` | 0.950 | 0.045 | 0.570 | 0.167 | 0.765 |
|
||||
| Upstream example 10 | current polished ControlNet | 0.701 | 0.105 | 0.941 | 0.094 | 0.781 |
|
||||
| Upstream example 12, matched size | published upstream | 0.976 | 0.014 | 0.873 | 0.111 | 0.777 |
|
||||
| Upstream example 12, matched size | local `qwen-zimage` | 0.947 | 0.015 | 0.708 | 0.085 | 0.896 |
|
||||
| Upstream example 12, matched size | current polished ControlNet | 0.548 | 0.061 | 0.961 | 0.105 | 0.887 |
|
||||
|
||||
The result reproduces the upstream architecture's main advantage: identity retention is far stronger than the current ControlNet path. On the group example, local face LPIPS nearly matches the published upstream output and whole-image fidelity is better; upstream still leads slightly on ArcFace identity and texture retention. ControlNet preserves more global detail and, on example 10, lower provisional OCR CER, but its faces drift to different identities. The OCR reference for example 10 came from the original image's OCR rather than hand transcription, so it is supporting evidence, not a text certification. The published upstream outputs are also downscaled relative to their originals, which penalizes their detail metrics but is the actual result the repository presents.
|
||||
|
||||
The comparison exposed a real implementation defect on a non-/16 input: the requested DiffSynth dimensions were floored while the PIL image remained at its original size, so the VAE latent and noise grid disagreed. Regression tests were written to fail on that mismatch, then both global and face inputs were changed to use the exact same aligned grid as their `height` and `width`.
|
||||
|
||||
**Final candidate oracle result (2026-07-25):** the user checked every image in the provider-separated `full-clean-final-candidate-2026-07-25-by-oracle` bundle with the corresponding provider oracle and confirmed that none of the six outputs retained SynthID or the provider generation signal. These are the current seed-0 bytes after the complete `visible -> qwen-zimage -> metadata` route, including the calibrated YuNet 0.5 gate and the prompt-cache/model-residency optimizations. This supersedes the earlier first-port batch check as the release-candidate result. It certifies these exact outputs, not every seed, resolution, or content class.
|
||||
|
||||
YuNet's score threshold is 0.5, not the upstream graph's YOLO threshold of 0.2: detector scores are not interchangeable. The copied 0.2 threshold admitted false/duplicate boxes and multiplied serial Z-Image calls. The calibrated gate retained every visible face in the public and upstream fixtures while reducing `gemini_3` from 36 boxes to 18 and the poster from 30 to 10. Serial face regeneration still scales with the retained detector count. Visual QA also found that the smallest multilingual text degraded on the typography sheet even though the larger headings survived. Keep `controlnet` as the compatibility and cost default, but recommend `qwen-zimage` when the user prioritizes output fidelity, especially face identity. The final exact-output oracle check covers the current YuNet threshold and runtime optimizations; do not call the profile broadly certified until a wider seeded face/text matrix is complete.
|
||||
|
||||
**Modal runtime measurement (2026-07-24 through 2026-07-25, seed 0, GPU stage only):** the exact paired A100-40GB run measured ControlNet at 3.342-12.543 seconds per image. `qwen-zimage` took 133.556-188.493 seconds on the three zero-face images and 1212.496 seconds on the 18-face group. The same group initially took 262.072 seconds on an exact H100, including 181.764 seconds in serial face regeneration. On H100 the three zero-face cases took 45.029-65.071 seconds. The shipped fast-load resident placement reduced the group to 133.543 seconds total and 38.272 seconds for face regeneration while producing a pixel-identical output; peak CUDA allocation rose from 24.364 to 43.477 GiB. Setup increased from 32.282 to 43.960 seconds, so even a cold one-request total fell from 294.354 to 177.503 seconds. Reusing the fixed prompt embeddings reduced a warm 18-face request further to 78.474 seconds after an earlier request populated the Qwen embedding; the cached and uncached outputs were pixel-identical, and peak VRAM was unchanged. The Qwen cache helps from the second request in one container, while the Z-Image cache helps after the first face in a multi-face request. Residency is automatic at 64 GiB VRAM or above; smaller cards retain offload. H100 remains both faster and cheaper at the live Modal rates for this workload. Pricing is intentionally not copied here; calculate from the current Modal rate and the recorded GPU seconds. Model setup must be added to an un-warmed single call or amortized over a warm batch.
|
||||
|
||||
@@ -321,11 +321,31 @@ Lazy `_get_{lama,migan}_session` singletons; `{lama,migan}_available()` guard th
|
||||
|
||||
## `noai/watermark_remover.py`
|
||||
|
||||
`noai/watermark_remover.py` — the `WatermarkRemover` class has three diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id`). `sdxl`/`controlnet` share the SDXL base (`DEFAULT_MODEL_ID`); `qwen` is its own base (`QWEN_MODEL_ID`).
|
||||
`noai/watermark_remover.py` — the `WatermarkRemover` class has four diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id`). `sdxl`/`controlnet` share the SDXL base (`DEFAULT_MODEL_ID`); `qwen` is its own base (`QWEN_MODEL_ID`); `qwen-zimage` delegates to the fixed two-stage stack in `noai/qwen_zimage_pipeline.py`.
|
||||
|
||||
**`sdxl`** (renamed from `default` 2026-06-09; `default` kept as a back-compat alias via `normalize_profile`) runs plain SDXL img2img (`_run_img2img`); it is the lighter opt-down alternative (no ControlNet weights).
|
||||
|
||||
**`qwen`** (`_run_qwen`, `_load_qwen_pipeline`) runs `QwenImageImg2ImgPipeline` on `Qwen/Qwen-Image` (20B MMDiT, Apache-2.0 code AND weights). The scrub still comes from the img2img `strength`; Qwen's value is **text preservation** (incl. CJK and small text). **Metric-measured nuance (2026-06-19, `scripts/fidelity_metrics.py`, do NOT trust the eyeball here — it misled). Compare ONLY at each pipeline's oracle-confirmed scrub floor (outputs where SynthID is removed in BOTH — an equal-strength compare is invalid where it leaves one un-scrubbed; Qwen at 0.15 does not clear Gemini): Qwen wins TEXT (lower OCR CER across EN/RU/ZH, perfect Chinese) but controlnet wins FACES (higher Laplacian-variance retention and lower LPIPS — Qwen smooths faces MORE; ArcFace identity favors controlnet 0.546 vs 0.331 at the Gemini floors).** So Qwen is the better text-preserving remover, NOT a universal fidelity win — controlnet's canny edge map holds face skin detail better. Specifics: bf16 on CUDA (fp16 risks overflow on the 20B MMDiT — see the dtype branch in `__init__`); loads `QWEN_MODEL_ID` unless `--model` is overridden; the call shape lives in the pure module helper `_build_qwen_kwargs` (unit-tested without torch in `tests/test_platform.py::TestQwenKwargs`), which uses Qwen's `true_cfg_scale` (NOT SDXL's `guidance_scale` — the CLI `--guidance-scale` maps onto it; ~4.0 is typical, the SDXL default 7.5 is high for Qwen) and an explicit `negative_prompt` (`_QWEN_PROMPT`/`_QWEN_NEGATIVE`). It is CUDA/cloud-class (the 20B does not fit MPS), so `_run_qwen` has NO MPS->CPU fallback — an error propagates. `_load_qwen_pipeline` raises a clear ImportError if the installed diffusers lacks `QwenImageImg2ImgPipeline`. **CERTIFIED oracle floors (Modal A100-80GB, 2026-06-20): OpenAI 0.10 (seed-robust — clean on seeds 0-4), Gemini 0.25 (seed 0 verified on 2 images; the Gemini oracle rate-limits volume seed-repeat, so PIN a seed in prod). The Gemini floor (0.25) is HIGHER than the certified controlnet Gemini floor (0.15); `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`), so `--pipeline qwen` gets the 0.25 Gemini floor automatically -- the old manual `--strength 0.25` workaround is retired. `_build_qwen_kwargs` passes an explicit `height`/`width` from the input (floored to /16 via the pure `_qwen_target_size`); WITHOUT it the img2img pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (the abba 2816x1536 case came back 1024x1024, distorting the scene and garbling text — fixed 2026-06-20, tested in `TestQwenKwargs`).** Fidelity vs controlnet was measured at the certified floors (`scripts/fidelity_metrics.py`), NOT eyeballed. **`qwen` is a MANUAL opt-in only — there is NO auto-router (one was prototyped and DROPPED, see below).** It wins ONE niche: clean body text on a plain background, NO faces (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES and **display/decorative text in a scene** (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes; qwen re-renders and garbles them). **`--pipeline auto` + a faces+text mixed dual-pass were built and DROPPED (2026-06-20):** on the canonical faces+text case (abba) controlnet wins EVERY metric incl. text, so grafting qwen text would only hurt; and "text→qwen" is undecidable cheaply (it is body-vs-display text that matters). The router/detector/mixed modules were removed; the geometry fix + the Qwen strength ladder were kept (they make the manual `--pipeline qwen` correct). **Do NOT retry "add a Qwen ControlNet to close the face gap" — it was built, measured, and CLOSED 2026-06-20:** a DiffSynth blockwise-canny Qwen ControlNet did not restore face skin texture (lapvar flat 0.40, canny carries edges not skin grain) and no permissively-licensed Qwen tile/detail/skin ControlNet exists anywhere (all conditioning is geometry). Faces stay on controlnet; the next improvement lead is Z-Image-Turbo (Apache-2.0, unmeasured floor). Full record + the deep-research sweep in `docs/qwen-improvement-research.md`.
|
||||
**`qwen`** (`_run_qwen`, `_load_qwen_pipeline`) runs `QwenImageImg2ImgPipeline` on `Qwen/Qwen-Image` (20B MMDiT, Apache-2.0 code AND weights). The scrub still comes from the img2img `strength`; Qwen's value is **text preservation** (incl. CJK and small text). **Metric-measured nuance (2026-06-19, `scripts/fidelity_metrics.py`, do NOT trust the eyeball here — it misled). Compare ONLY at each pipeline's oracle-confirmed scrub floor (outputs where SynthID is removed in BOTH — an equal-strength compare is invalid where it leaves one un-scrubbed; Qwen at 0.15 does not clear Gemini): Qwen wins TEXT (lower OCR CER across EN/RU/ZH, perfect Chinese) but controlnet wins FACES (higher Laplacian-variance retention and lower LPIPS — Qwen smooths faces MORE; ArcFace identity favors controlnet 0.546 vs 0.331 at the Gemini floors).** So Qwen is the better text-preserving remover, NOT a universal fidelity win — controlnet's canny edge map holds face skin detail better. Specifics: bf16 on CUDA (fp16 risks overflow on the 20B MMDiT — see the dtype branch in `__init__`); loads `QWEN_MODEL_ID` unless `--model` is overridden; the call shape lives in the pure module helper `_build_qwen_kwargs` (unit-tested without torch in `tests/test_platform.py::TestQwenKwargs`), which uses Qwen's `true_cfg_scale` (NOT SDXL's `guidance_scale` — the CLI `--guidance-scale` maps onto it; ~4.0 is typical, the SDXL default 7.5 is high for Qwen) and an explicit `negative_prompt` (`_QWEN_PROMPT`/`_QWEN_NEGATIVE`). It is CUDA/cloud-class (the 20B does not fit MPS), so `_run_qwen` has NO MPS->CPU fallback — an error propagates. `_load_qwen_pipeline` raises a clear ImportError if the installed diffusers lacks `QwenImageImg2ImgPipeline`. **CERTIFIED oracle floors (Modal A100-80GB, 2026-06-20): OpenAI 0.10 (seed-robust — clean on seeds 0-4), Gemini 0.25 (seed 0 verified on 2 images; the Gemini oracle rate-limits volume seed-repeat, so PIN a seed in prod). The Gemini floor (0.25) is HIGHER than the certified controlnet Gemini floor (0.15); `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`), so `--pipeline qwen` gets the 0.25 Gemini floor automatically -- the old manual `--strength 0.25` workaround is retired. `_build_qwen_kwargs` passes an explicit `height`/`width` from the input (floored to /16 via the pure `_qwen_target_size`); WITHOUT it the img2img pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (the abba 2816x1536 case came back 1024x1024, distorting the scene and garbling text — fixed 2026-06-20, tested in `TestQwenKwargs`).** Fidelity vs controlnet was measured at the certified floors (`scripts/fidelity_metrics.py`), NOT eyeballed. **`qwen` is a MANUAL opt-in only — there is NO auto-router (one was prototyped and DROPPED, see below).** It wins ONE niche: clean body text on a plain background, NO faces (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES and **display/decorative text in a scene** (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes, qwen re-renders and garbles them). **`--pipeline auto` + a faces+text mixed dual-pass were built and DROPPED (2026-06-20):** on the canonical faces+text case controlnet wins EVERY metric incl. text, so grafting qwen text would only hurt; and "text→qwen" is undecidable cheaply (it is body-vs-display text that matters). The router/detector/mixed modules were removed; the geometry fix + the Qwen strength ladder were kept (they make the manual `--pipeline qwen` correct). **Do NOT retry "add a Qwen ControlNet to close the face gap" — it was built, measured, and CLOSED 2026-06-20:** a DiffSynth blockwise-canny Qwen ControlNet did not restore face skin texture (lapvar flat 0.40, canny carries edges not skin grain) and no permissively-licensed Qwen tile/detail/skin ControlNet exists anywhere (all conditioning is geometry). The Z-Image face-crop lead is now implemented as the separate `qwen-zimage` profile and has direct face metrics on two official upstream examples plus one crowded fixture. Its exact current six-output candidate is negative in the corresponding provider oracles, while broad seeded removal and text behavior remain unmeasured. Full record + the deep-research sweep in `docs/qwen-improvement-research.md`.
|
||||
|
||||
## `noai/qwen_zimage_pipeline.py`
|
||||
|
||||
`qwen-zimage` is the recommended high-quality, manual CUDA profile ported from `cebeuq/Synthid-Bypass` v2. `controlnet` remains the default for compatibility and cost; callers that prioritize output fidelity, especially face identity, should select `qwen-zimage`. The full-frame stage uses DiffSynth `QwenImagePipeline` with `Qwen/Qwen-Image-2512`, `lightx2v/Qwen-Image-2512-Lightning` at four steps, and `DiffSynth-Studio/Qwen-Image-Blockwise-ControlNet-Canny`. The Lightning scheduler uses `exponential_shift_mu=log(3)`, the DiffSynth equivalent of the source graph's AuraFlow shift 3. Its default denoise is the source custom node's exact megapixel formula at adaptive level 6; an explicit `--strength` overrides that global value. Its profile seed defaults to `0`, matching the oracle-negative release candidate; an explicit seed still wins. Other profiles keep their existing random default.
|
||||
|
||||
The face stage detects boxes on the original input with OpenCV YuNet and follows the active Impact Pack SAM path from the source graph: each box supplies both the `center-1` positive point and the box prompt; proposals at predicted IoU >= 0.93 are unioned, or the highest-IoU proposal is used when none passes; the result is intersected with the detector box. The inactive MediaPipe node in the workflow has no downstream link. YuNet uses its own calibrated score threshold, 0.5: copying the upstream YOLO threshold of 0.2 admitted background/decorative false positives and duplicate boxes, which multiplies the serial face-stage cost. The 0.5 gate retained all visible faces in the public and upstream comparison fixtures while reducing the crowded group from 36 boxes to 18 and the poster from 30 to 10. Crops expand by the source graph's factor 2.5 and run `Tongyi-MAI/Z-Image-Turbo` for eight steps. Every face uses the denoise derived from the largest face's area ratio, matching the source graph's `largest_face` mode, then pastes through the clipped SAM mask with feather 10. If SAM fails, a box-derived ellipse mask is used rather than aborting the global removal.
|
||||
|
||||
The active graph was traced from upstream commit `3007d0351596ae0a78b7074dae7ad179710b1e48` and its linked Impact Pack implementation, not inferred from the README or node names. YuNet is one intentional substitution: the active reference path uses an Ultralytics YOLO face detector, while this package avoids adding its AGPL runtime. The first-use model download targets GitHub's media endpoint rather than the repository's 131-byte Git LFS pointer and verifies the published 232589-byte model by SHA-256 before caching it. The other runtime differences are full safetensors rather than quantized GGUF models, DiffSynth's first-order Qwen Lightning and Z-Image FlowMatch samplers rather than the graph's DPM++ 2M / SGM Uniform and `res_2s` / `bong_tangent` pairs, and the absence of the detailer's 20 px latent noise-mask feather. This port regenerates the expanded crop and composites only the feathered SAM mask; generated pixels outside the face mask are discarded. The architecture and active decision path match the graph, but the runtime is not bit-identical to ComfyUI.
|
||||
|
||||
The implementation has its own `qwen-zimage` optional dependency group because DiffSynth, torchvision, and the additional model downloads are large. `--model` is rejected for this fixed profile. `--tile` runs only the global Qwen stage through `noai.tiling.run_tiled`; the global denoise is still derived from the full-frame megapixel count and the same seed is reused for each deterministic tile. After feather blending, YuNet, SAM, and Z-Image run once against the full original/global result, so faces are neither duplicated nor dropped at tile boundaries. The SDXL minimum-resolution floor is disabled, and CLI adaptive polish defaults off for this profile, so the two-stage result is not followed by a repository-specific post-process. DiffSynth requires the PIL input, Canny control, and explicit dimensions to agree on the same /16 latent grid: `_resize_to_target` aligns global and face pixels before `build_global_kwargs` / `build_face_kwargs`, and the global output is restored to the exact original size. Passing floored dimensions with unaligned pixels caused a real VAE/noise-grid shape mismatch on the official example 12 input; the call-shape assertions were observed failing before the fix. An explicit `--adaptive-polish` still opts in. Pure helpers cover both adaptive denoise formulas, /16 dimensions, call shapes, Canny generation, masked compositing, and the qwen-zimage tiling seam; an integration test guards dispatch from `WatermarkRemover`. A real 4096x3072, 20-tile H100 smoke completed through this exact branch on 2026-07-25 with dimensions preserved and no visible or 99th-percentile gradient outlier at a tile boundary; the measured runtime, memory, and fidelity figures are in `docs/known-limitations.md`. The exact seed-0 non-tiled release candidate is oracle-verified; tiled outputs still require their own provider-oracle check.
|
||||
|
||||
DiffSynth normally offloads the Z-Image text encoder, DiT, and VAE to CPU after every face call. That placement dominated crowded-scene latency even though the eight diffusion steps themselves were fast. `resolve_face_model_residency` keeps the full face stack on CUDA when total VRAM is at least 64 GiB; smaller cards preserve the original offload path. Callers can explicitly override the decision through `QwenZImagePipeline.keep_face_models_on_device`. The implementation intentionally loads the stack with the normal CPU-managed config, rewrites the managed modules' offload/onload/preparing placement to their CUDA computation device, and moves them once. Loading the same models directly into CUDA cut face inference further but increased setup from 32.282 to 254.632 seconds, making a cold single request more expensive; that variant was rejected. The shipped fast-load residency changes only model placement, not weights, dtypes, prompts, seeds, schedules, masks, or compositing. On the 18-face H100 fixture it produced a pixel-identical output versus offload while reducing face regeneration from 181.764 to 38.272 seconds and total inference from 262.072 to 133.543 seconds. Setup rose only from 32.282 to 43.960 seconds, so cold setup plus inference fell from 294.354 to 177.503 seconds. Peak CUDA allocation rose from 24.364 to 43.477 GiB.
|
||||
|
||||
Both stage prompts are constants, but DiffSynth 2.0.18 exposes their embeddings only through internal PipelineUnits and re-runs the corresponding text encoder on every call. `_cache_static_prompt_embeddings` wraps the exact prompt unit selected by its output signature and memoizes its returned tensors by prompt text. It bypasses the cache whenever `edit_image` participates, so image-conditioned embeddings cannot be reused accidentally. With CFG 1.0, the unit runner already shares the positive result with the negative branch. On a warm H100 sequence, a preceding no-face request populated the Qwen prompt cache; the 18-face case then fell from 133.543 to 78.474 seconds. The global stage fell from 89.720 to 43.334 seconds, and the face stage from 38.272 to 30.592 seconds as the fixed Z-Image prompt was encoded only once. Both the no-face and 18-face cached outputs were pixel-identical to their no-cache references, and peak allocation remained 43.477 GiB. The Qwen saving applies from the second request in a container; the Z-Image saving applies within the first multi-face request after its first face.
|
||||
|
||||
Validation status is deliberately narrower than the existing `qwen` certification. The user reported the upstream workflow as Gemini-oracle negative. The current port has API, unit, dispatch, GPU integration runs, and a direct comparison with two official upstream before/after pairs. On 2026-07-25 the user checked all six current outputs in the provider-separated `full-clean-final-candidate-2026-07-25-by-oracle` bundle with the corresponding provider oracles and confirmed that none retained SynthID or the provider generation signal. The checked bytes used the complete `visible -> qwen-zimage -> metadata` route, the calibrated YuNet 0.5 gate, and the shipped prompt-cache/model-residency optimizations. This supersedes the earlier first-port batch check as the release-candidate result. It certifies the exact seed-0 outputs, not every seed, resolution, or content class. Broad text certification remains open, so the profile stays an experimental manual opt-in even though it is the recommended quality mode.
|
||||
|
||||
**Direct port measurement (2026-07-24, seed 0):** on official upstream examples 10 and 12, local `qwen-zimage` retained ArcFace identity at 0.950 and 0.947 versus 0.701 and 0.548 for the current polished ControlNet output. Face LPIPS was 0.045 and 0.015 versus ControlNet's 0.105 and 0.061. The published upstream outputs retained identity at 0.976 on both and face LPIPS at 0.172 and 0.014; upstream example 10 is strongly penalized by its published downscale, while example 12 was compared at the same published dimensions. Local whole-image LPIPS / SSIM were 0.167 / 0.765 and 0.085 / 0.896, better than the published upstream 0.259 / 0.627 and 0.111 / 0.777. ControlNet still preserved more texture, but the faces drifted. The earlier crowded `gemini_3` run showed the same identity direction, 0.795 versus 0.587/0.588, while smoothing skin and changing the full frame more. The final July 25 oracle check supplies the removal verdict for the exact current candidate bytes only.
|
||||
|
||||
Two integration failures from the first Modal passes are regression-guarded. SAM model pixels must be cast to the model's bfloat16 while geometric prompts remain float32; casting everything either fails or changes prompt semantics. SAM `pred_masks` and `iou_scores` must then be converted through float32 before NumPy because NumPy rejects bfloat16. A third visually severe failure came from accepting an unconstrained SAM mask: it split faces with hard seams. The center point plus box prompt and the final detector-box intersection are both load-bearing.
|
||||
|
||||
**`controlnet`** (**the DEFAULT pipeline since 2026-06-09** for `invisible`/`all`/`batch` and both engine ctors; `_run_controlnet`, `_load_controlnet_pipeline`) runs `StableDiffusionXLControlNetImg2ImgPipeline` with the SDXL-native canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` (`watermark_profiles.CONTROLNET_CANNY_MODEL`): the control image is `cv2.Canny(gray, 100, 200)` stacked to 3 channels (`_CANNY_LOW`/`_CANNY_HIGH`, prompt `_CONTROLNET_PROMPT` / `_CONTROLNET_NEGATIVE`).
|
||||
|
||||
@@ -337,11 +357,11 @@ At the shared low removal strength the canny edge-conditioning keeps the regener
|
||||
|
||||
**But the reverse also holds: a flat-graphic logo/poster SURVIVED `default` while clearing controlnet** -- removal at the low strength is content×pipeline dependent and neither pipeline is universally safe; the real lever is a higher strength. See the controlnet Known-limitations bullet for the full table + root cause. Canny holds face STRUCTURE but NOT identity (the regenerated face drifts in likeness -- canny carries edges, not identity). The drifted cleaned face is the LEAST-AI state we can reach without re-introducing SynthID; the library does NOT ship a face-restore extra. Every restore approach we evaluated (GFPGAN-on-cleaned, PhotoMaker-V2 txt2img, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps, 2026-06-04 - 2026-06-08 Modal cert sweeps) regenerated the face from an ArcFace embedding via SDXL diffusion -- which makes the output face look MORE AI-generated, not less. Empirical conclusion in `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up". For production face preservation, ship the cleaned image as-is. `controlnet_conditioning_scale` (ctor arg, default 1.0) is the structure-preservation knob. Same dtype rule as `default` (fp32 on cpu/mps, fp16 only on cuda/xpu; the fp16-fixed SDXL VAE `_SDXL_FP16_VAE_ID` is swapped in on fp16 GPUs -- issue #29) and the same MPS->CPU fallback (reload on cpu/fp32, drop a non-cpu generator, retry once).
|
||||
|
||||
**Tiled diffusion (`tile`/`tile_size`/`tile_overlap` ctor-path args, CLI `--tile`, issue #10):** for large inputs that OOM at native resolution, `remove_watermark` can process the diffusion pass in overlapping sliding-window tiles instead of one forward pass — the lossless alternative to a `--max-resolution` downscale. The single-image generation closure was refactored into `_generate_one(img)` (dispatches controlnet/img2img, generator shared so the seed advances deterministically across tiles), and `_generate()` routes it through `noai.tiling.run_tiled` when `tile` is set AND `max(init_image.size) > tile_size` (a sub-tile image runs one pass unchanged). The ControlNet canny edge map is rebuilt per tile inside `_generate_one`, so structure preservation is tile-local. See `noai/tiling.py` below and the tiled-diffusion subsection in `docs/known-limitations.md` for the geometry, the partition-of-unity blend, and the quality caveat.
|
||||
**Tiled diffusion (`tile`/`tile_size`/`tile_overlap` ctor-path args, CLI `--tile`, issue #10):** for large inputs that OOM at native resolution, `remove_watermark` can process the diffusion pass in overlapping sliding-window tiles instead of one forward pass — the lossless alternative to a `--max-resolution` downscale. For SDXL, ControlNet, and base Qwen, the single-image generation closure was refactored into `_generate_one(img)` (dispatches controlnet/img2img, generator shared so the seed advances deterministically across tiles), and `_generate()` routes it through `noai.tiling.run_tiled` when `tile` is set AND `max(init_image.size) > tile_size` (a sub-tile image runs one pass unchanged). The ControlNet canny edge map is rebuilt per tile inside `_generate_one`, so structure preservation is tile-local. `qwen-zimage` takes a different route: `_generate()` dispatches once to `QwenZImagePipeline.run`, that runtime tiles only `_run_global`, then performs one full-frame face stage after blending. See `noai/tiling.py` below and the tiled-diffusion subsection in `docs/known-limitations.md` for the geometry, the partition-of-unity blend, and the quality caveat.
|
||||
|
||||
## `noai/tiling.py`
|
||||
|
||||
Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL only). `plan_tiles(w, h, tile_size, overlap)` returns a row-major grid of uniform-size `Tile` boxes — every tile is exactly `tile_size` (the SDXL training size), with the last tile on each axis pulled back flush to the far edge (`_axis_positions` clamps a pathological `overlap >= tile` to `tile - 1` so the step stays >= 1). `feather_weights(w, h, overlap)` is a separable linear taper (1 in the interior, ramping toward each edge) floored at `_WEIGHT_EPS` so it is **strictly positive everywhere** — that makes the normalized `accum / weight_sum` blend a partition of unity, so identical/unchanged tiles reconstruct the input exactly (the seam-free guarantee). `run_tiled(generate_tile, image, tile_size, overlap, set_progress)` is the orchestration loop: crop each planned tile, call `generate_tile` (one diffusion pass on a single PIL tile — injected, so this stays decoupled from the pipeline), resize a latent-grid-rounded result back to the exact tile size, and feather-accumulate. All three are unit-tested without the model (`tests/test_tiling.py`: axis math, grid coverage, taper shape/symmetry/positivity, identity reconstruction, per-tile call count, and the resize-back path). New blend tuning belongs in these pure helpers, not inlined into the runner.
|
||||
Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL only). `plan_tiles(w, h, tile_size, overlap)` returns a row-major grid of uniform-size `Tile` boxes — every tile is exactly `tile_size`, with the last tile on each axis pulled back flush to the far edge (`_axis_positions` clamps a pathological `overlap >= tile` to `tile - 1` so the step stays >= 1). `feather_weights(w, h, overlap)` is a separable linear taper (1 in the interior, ramping toward each edge) floored at `_WEIGHT_EPS` so it is **strictly positive everywhere** — that makes the normalized `accum / weight_sum` blend a partition of unity, so identical/unchanged tiles reconstruct the input exactly (the seam-free guarantee). `run_tiled(generate_tile, image, tile_size, overlap, set_progress)` is the orchestration loop: crop each planned tile, call `generate_tile` (one diffusion pass on a single PIL tile — injected, so this stays decoupled from the pipeline), resize a latent-grid-rounded result back to the exact tile size, and feather-accumulate. All three are unit-tested without the model (`tests/test_tiling.py`: axis math, grid coverage, taper shape/symmetry/positivity, identity reconstruction, per-tile call count, and the resize-back path). New blend tuning belongs in these pure helpers, not inlined into the runner.
|
||||
|
||||
`feather_region_composite(base, regenerated, box, *, feather)` is the pure region-targeted compositor for **AI-enhanced composites** (roadmap P1#8; `identify` `ai_source_kind == "enhanced"`, digitalSourceType `compositeWithTrainedAlgorithmicMedia`). It blends `regenerated` over `base` inside `box = (x, y, w, h)` with a separable linear taper of `feather` px at the box edges (the taper anchors to ~0 at the boundary, so unlike `feather_weights` it is NOT floored — the result equals `base` EXACTLY outside the box), preserving dtype and supporting HxW or HxWxC. It backs `WatermarkRemover.remove_watermark(region=..., region_feather=...)`: the remover regenerates the frame (or tiles), then composites only the AI box back over the original input, so the real photo outside the box stays pixel-exact and only the AI region is scrubbed. The box is caller-supplied (a C2PA composite manifest carries no reliable machine-readable region); the no-model lossless region path remains `region_eraser.erase`. Unit-tested in `tests/test_tiling.py::TestFeatherRegionComposite` (outside-box exactness, interior == regenerated, hard-paste at feather 0, monotonic seam ramp, dtype/grayscale/clamp/empty-box/shape-mismatch).
|
||||
|
||||
@@ -351,7 +371,7 @@ Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL on
|
||||
|
||||
History: `auto_config.plan()` was a content-adaptive planner that detected faces/text/edges (bundled OpenCV YuNet + PP-OCRv3 DBNet models) to route the pipeline and toggle the adaptive polish. Once `controlnet` became the default-and-only auto pipeline (it no longer downgrades a structure-less image to `sdxl`) and the adaptive polish was confirmed to **self-gate by detail level** (`humanizer.adaptive_polish` no-ops when the cleaned image already meets the input's Laplacian variance, so it does real work only on over-smoothed photo/face texture and ~nothing on text/flat), the detection no longer changed any behavior — it only annotated a `reason` string. So the whole layer was deleted: `auto_config.py`, `tests/test_auto_config.py`, and the two detection assets (`assets/face_detection_yunet_2023mar.onnx`, `assets/text_detection_ppocrv3_2023may.onnx`, ~2.6 MB).
|
||||
|
||||
**`--auto` is now a DEPRECATED no-op** (`cli._resolve_auto_polish`): controlnet is already the default pipeline AND the adaptive polish is ON by default, so `--auto` has nothing left to do — it only prints a deprecation warning and passes `adaptive_polish` through unchanged (an explicit `--no-adaptive-polish` still wins). (Originally it re-enabled the polish; once the polish default flipped to ON the same day, the parameter-source branch became dead and was dropped.) The **adaptive polish itself lives on** in `humanizer.adaptive_polish` (CLI `--adaptive-polish/--no-adaptive-polish`, **ON by default since 2026-06-09** — it self-gates to a no-op where there is no detail deficit, so default-on is safe; uses the full-res original as the detail reference) — see the `humanizer` test note. `batch` resolves the polish once before the loop (one warning) and caches the invisible engine per pipeline (`ctx.obj["_inv_engines"]`).
|
||||
**`--auto` is now a DEPRECATED no-op** (`cli._resolve_auto_polish`): controlnet is already the default pipeline AND the adaptive polish is ON by default, so `--auto` has nothing left to do — it only prints a deprecation warning and passes `adaptive_polish` through unchanged (an explicit `--no-adaptive-polish` still wins). (Originally it re-enabled the polish; once the polish default flipped to ON the same day, the parameter-source branch became dead and was dropped.) The **adaptive polish itself lives on** in `humanizer.adaptive_polish` (CLI `--adaptive-polish/--no-adaptive-polish`, **ON by default since 2026-06-09 for the original profiles** — it self-gates to a no-op where there is no detail deficit; `qwen-zimage` defaults it off to preserve its upstream-matching output, and an explicit flag overrides either default) — see the `humanizer` test note. `batch` resolves the polish once before the loop (one warning) and caches the invisible engine per pipeline (`ctx.obj["_inv_engines"]`).
|
||||
|
||||
## Content `--pipeline auto` router + faces+text mixed dual-pass — PROTOTYPED and DROPPED (2026-06-20)
|
||||
|
||||
@@ -390,7 +410,7 @@ Full pipeline (visible + invisible + metadata). Same diffusion knobs as `invisib
|
||||
|
||||
### `invisible`
|
||||
|
||||
Diffusion SynthID removal. The `--tile/--no-tile` knob is the *lossless* alternative to a `--max-resolution` downscale for large inputs that OOM on MPS/GPU: it engages only when the long side exceeds `--tile-size` (default 1024); tiles are feather-blended over `--tile-overlap` px (default 128); pair with `--max-resolution 0`. `--adaptive-polish` is a detail-targeted polish that self-gates to a no-op where there is no deficit. `--auto` is deprecated and now a no-op that only warns (the polish it used to enable is ON by default). **No-signal skip (P0#5, roadmap):** before the diffusion runs, the command checks `identify.has_invisible_target(source)` (the `ProvenanceReport.ai_from_metadata` union: C2PA AI-issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, open DWT-DCT / TrustMark — visible marks do NOT count, they are a separate pass). When nothing is locally detectable it does NOT regenerate (that would only degrade a clean image — the dominant paid score-0 cause on no-watermark uploads): it writes NO output, prints guidance that does NOT claim the image is clean (a pixel SynthID is undetectable once its metadata proxy is gone), and exits **`EXIT_NO_INVISIBLE_SIGNAL` (2)** — same value/role as the visible `EXIT_NO_VISIBLE_MARK`. `--force/--no-force` (**default skip = ON**) runs the scrub regardless. The check fails SAFE (a detector exception → run, since leaving a watermark on a paid removal is worse than over-regenerating). Helpers `cli._no_invisible_signal_exit` + `identify.has_invisible_target`; regression-guarded by `tests/test_cli.py::TestInvisibleCommand::{test_invisible_no_signal_skips_and_exits_two,test_invisible_force_runs_scrub_on_no_signal,test_invisible_runs_without_force_when_signal_present}` and `tests/test_identify.py::TestHasInvisibleTargetFailSafe`. **Test trap:** any `invisible`/`all`/`batch` test that exercises the diffusion path on a signal-LESS fixture (e.g. the synthetic `sample_png`) MUST pass `--force`, or the new gate skips step 2 (so `mock_engine.remove_watermark` is never called / `invisible` exits 2).
|
||||
Diffusion SynthID removal. The `--tile/--no-tile` knob is the *lossless* alternative to a `--max-resolution` downscale for large inputs that OOM on MPS/GPU: it engages only when the long side exceeds `--tile-size` (default 1024); tiles are feather-blended over `--tile-overlap` px (default 128); pair with `--max-resolution 0`. `--adaptive-polish` is a detail-targeted polish that self-gates to a no-op where there is no deficit; it defaults off only on `qwen-zimage`. `--auto` is deprecated and now a no-op that only warns. **No-signal skip (P0#5, roadmap):** before the diffusion runs, the command checks `identify.has_invisible_target(source)` (the `ProvenanceReport.ai_from_metadata` union: C2PA AI-issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, open DWT-DCT / TrustMark — visible marks do NOT count, they are a separate pass). When nothing is locally detectable it does NOT regenerate (that would only degrade a clean image — the dominant paid score-0 cause on no-watermark uploads): it writes NO output, prints guidance that does NOT claim the image is clean (a pixel SynthID is undetectable once its metadata proxy is gone), and exits **`EXIT_NO_INVISIBLE_SIGNAL` (2)** — same value/role as the visible `EXIT_NO_VISIBLE_MARK`. `--force/--no-force` (**default skip = ON**) runs the scrub regardless. The check fails SAFE (a detector exception → run, since leaving a watermark on a paid removal is worse than over-regenerating). Helpers `cli._no_invisible_signal_exit` + `identify.has_invisible_target`; regression-guarded by `tests/test_cli.py::TestInvisibleCommand::{test_invisible_no_signal_skips_and_exits_two,test_invisible_force_runs_scrub_on_no_signal,test_invisible_runs_without_force_when_signal_present}` and `tests/test_identify.py::TestHasInvisibleTargetFailSafe`. **Test trap:** any `invisible`/`all`/`batch` test that exercises the diffusion path on a signal-LESS fixture (e.g. the synthetic `sample_png`) MUST pass `--force`, or the new gate skips step 2 (so `mock_engine.remove_watermark` is never called / `invisible` exits 2).
|
||||
|
||||
### `visible`
|
||||
|
||||
@@ -398,4 +418,4 @@ Known-visible-mark removal by **localize -> fill**: each detected mark is locali
|
||||
|
||||
### `batch`
|
||||
|
||||
Process every supported image in a directory (output defaults to `<directory>_clean/`, set with `-o`). `--mode visible|invisible|metadata|all` (default `visible`); the invisible/all path reuses the **full `invisible` knob set** (`--strength`/`--steps`/`--guidance-scale`/`--pipeline`/`--controlnet-scale`/`--model`/`--device`/`--max-resolution`/`--min-resolution`/`--upscaler`/`--seed`/`--hf-token`/`--humanize`/`--unsharp`/`--adaptive-polish`/`--tile`/`--tile-size`/`--tile-overlap`/`--force`), plus `--backend` for the visible localize -> fill pass. `--adaptive-polish` is ON by default; `--auto` is deprecated and a no-op that only warns. **No-signal skip (P0#5):** in invisible/all mode each image runs the same `has_invisible_target` gate — a signal-less image is skipped (no diffusion); in `invisible` mode the input is copied through to the output dir so it stays complete, in `all` mode the visible-removed result is kept and metadata is still stripped. `--force` scrubs every image regardless. One engine cached per pipeline; the polish is resolved once before the loop. **Exit code (`batch` used to always exit 0, hiding failures):** `cmd_batch` raises `SystemExit(1)` when any image errored, OR when a `--mode invisible`/`all` image carried an invisible signal but the GPU extra was absent so its SynthID scrub was skipped — mirroring single `all`, it emits a loud "the invisible watermark was NOT removed on N image(s)" warning and (invisible mode) copies the input through so the output dir stays complete, rather than silently dropping the signal-bearing files that most needed processing. `_process_batch_image` returns that skipped-scrub flag; the loop tallies it. Regression-guarded by `tests/test_cli.py::TestBatchCommand::{test_batch_errors_exit_nonzero, test_batch_invisible_gpu_missing_writes_output_and_exits_nonzero}`.
|
||||
Process every supported image in a directory (output defaults to `<directory>_clean/`, set with `-o`). `--mode visible|invisible|metadata|all` (default `visible`); the invisible/all path reuses the **full `invisible` knob set** (`--strength`/`--steps`/`--guidance-scale`/`--pipeline`/`--controlnet-scale`/`--model`/`--device`/`--max-resolution`/`--min-resolution`/`--upscaler`/`--seed`/`--hf-token`/`--humanize`/`--unsharp`/`--adaptive-polish`/`--tile`/`--tile-size`/`--tile-overlap`/`--force`), plus `--backend` for the visible localize -> fill pass. `--adaptive-polish` is ON by default except on `qwen-zimage`; `--auto` is deprecated and a no-op that only warns. **No-signal skip (P0#5):** in invisible/all mode each image runs the same `has_invisible_target` gate — a signal-less image is skipped (no diffusion); in `invisible` mode the input is copied through to the output dir so it stays complete, in `all` mode the visible-removed result is kept and metadata is still stripped. `--force` scrubs every image regardless. One engine cached per pipeline; the polish is resolved once before the loop. **Exit code (`batch` used to always exit 0, hiding failures):** `cmd_batch` raises `SystemExit(1)` when any image errored, OR when a `--mode invisible`/`all` image carried an invisible signal but the GPU extra was absent so its SynthID scrub was skipped — mirroring single `all`, it emits a loud "the invisible watermark was NOT removed on N image(s)" warning and (invisible mode) copies the input through so the output dir stays complete, rather than silently dropping the signal-bearing files that most needed processing. `_process_batch_image` returns that skipped-scrub flag; the loop tallies it. Regression-guarded by `tests/test_cli.py::TestBatchCommand::{test_batch_errors_exit_nonzero, test_batch_invisible_gpu_missing_writes_output_and_exits_nonzero}`.
|
||||
|
||||
@@ -25,9 +25,10 @@ faces). The strongest verified improvement path is to **add structure conditioni
|
||||
(a Qwen-Image ControlNet) to the existing base pass, the direct analog of the SDXL +
|
||||
canny conditioning that wins on faces. Separately, **Z-Image / Z-Image-Turbo** (6B,
|
||||
Apache-2.0) is the best-verified lighter alternative to evaluate before committing to
|
||||
the 20B cost. None of the improvements has measured face-fidelity numbers at our
|
||||
scrub floors yet, so each must be validated with `scripts/fidelity_metrics.py` plus
|
||||
the oracle before shipping.
|
||||
the 20B cost. At research time none of the improvements had measured face-fidelity
|
||||
numbers at our scrub floors. The later `qwen-zimage` follow-up below adds a crowded
|
||||
fixture, two direct upstream comparisons, and a provider-oracle-negative final candidate.
|
||||
A broader seeded text/face matrix is still needed for general certification.
|
||||
|
||||
## Follow-up: ControlNet experiment + deeper research (2026-06-20)
|
||||
|
||||
@@ -60,8 +61,8 @@ Measured on `gemini_3` (18 faces) at the Gemini scrub floor 0.25 vs base-Qwen 0.
|
||||
fix faces" lead is closed for good.**
|
||||
- **[high, unanimous] Z-Image / Z-Image-Turbo (6B, Apache-2.0 on code AND weights, ~1/3 of
|
||||
Qwen 20B)** ships a documented `ZImageImg2ImgPipeline` with standard strength denoising, so
|
||||
it preserves the scrub mechanism. Its own SynthID scrub floor and face/text fidelity are
|
||||
UNMEASURED -- this is the strongest concrete NEXT experiment.
|
||||
it preserves the scrub mechanism. Its own SynthID scrub floor and broad text fidelity
|
||||
remain unmeasured. The later `qwen-zimage` follow-up provides direct face metrics.
|
||||
- **[medium] Lowering Qwen's scrub floor has no off-the-shelf SynthID answer:** the "partial
|
||||
img2img ~0.3 breaks robust watermarks" literature tests open schemes
|
||||
(StegaStamp/TrustMark/VINE), NEVER SynthID (proprietary decoder) -- analogy, not proof. No
|
||||
@@ -70,9 +71,58 @@ Measured on `gemini_3` (18 faces) at the Gemini scrub floor 0.25 vs base-Qwen 0.
|
||||
not carry the watermark back." So non-regenerative detail transfer is NOT safe by
|
||||
assumption -- the transferred high-frequency band must be gated against the SynthID oracle.
|
||||
|
||||
**Net for the pipeline:** **faces stay on SDXL+controlnet**; there is no Qwen face-fix.
|
||||
The live frontier is Z-Image-Turbo (next experiment) and oracle-gated non-regenerative detail
|
||||
re-injection.
|
||||
**Net for the single-pass `qwen` pipeline:** faces stay on SDXL+controlnet; Canny alone is not
|
||||
a Qwen face fix. The next distinct architecture was Z-Image-Turbo on original masked face
|
||||
crops, not another Qwen geometry conditioner.
|
||||
|
||||
**Implementation follow-up (2026-07-24):** that distinct architecture now exists as the
|
||||
manual `qwen-zimage` profile. It ports the upstream Synthid-Bypass v2 graph: Qwen-Image-2512
|
||||
Lightning + DiffSynth Canny for the full frame, then SAM-masked Z-Image Turbo regeneration
|
||||
from original face crops. The upstream result supplied by the user was Gemini-oracle negative.
|
||||
The active upstream face path is YOLO + SAM, not the unconnected MediaPipe node. The port
|
||||
matches its center-point + box prompts, IoU-0.93 proposal selection, detector-box intersection,
|
||||
crop factor, and paste feather; YuNet is the intentional detector substitution.
|
||||
|
||||
The first exact-path Modal run completed without SAM fallback or face seams. On one crowded
|
||||
18-face `gemini_3` fixture, ArcFace identity improved materially over controlnet
|
||||
(0.795 vs 0.587/0.588 raw/polished), while face LPIPS was 0.082 vs 0.087/0.078.
|
||||
Two official upstream before/after pairs then reproduced the same identity advantage:
|
||||
local `qwen-zimage` scored 0.950/0.947 ArcFace identity versus polished ControlNet's
|
||||
0.701/0.548. The published upstream outputs remained slightly higher at 0.976/0.976.
|
||||
On the matched-size group example, local face LPIPS essentially matched upstream
|
||||
(0.015 vs 0.014) while whole-image LPIPS and SSIM were better (0.085/0.896 vs
|
||||
0.111/0.777). ControlNet retained more texture but changed the identities.
|
||||
|
||||
That comparison also caught a real /16 alignment defect: DiffSynth dimensions were floored
|
||||
without resizing the corresponding PIL input. The official non-grid input failed with a
|
||||
VAE/noise-grid shape mismatch. Regression assertions were observed failing before global
|
||||
and face pixels were aligned to the same grid as their explicit dimensions.
|
||||
|
||||
The final verifier follow-up is positive but exact-output scoped. On 2026-07-25 the user
|
||||
checked all six current outputs in the provider-separated
|
||||
`full-clean-final-candidate-2026-07-25-by-oracle` bundle with the corresponding provider
|
||||
oracles and confirmed that none retained SynthID or the provider generation signal. The
|
||||
checked bytes used the complete `visible -> qwen-zimage -> metadata` route, the calibrated
|
||||
YuNet 0.5 gate, and the shipped prompt-cache/model-residency optimizations. This supersedes
|
||||
the earlier first-port batch check as the release-candidate result, but it is not a
|
||||
certification across seeds, resolutions, and content classes. YuNet's threshold was
|
||||
calibrated independently from upstream YOLO: 0.5 retained the visible faces in the
|
||||
comparison fixtures while removing the false and duplicate boxes admitted by the copied
|
||||
0.2 gate. The remaining questions are now narrower: how well does text hold across a
|
||||
broader set and how stable removal is across seeds and Google content types. See
|
||||
`docs/known-limitations.md`.
|
||||
|
||||
The shipped profile therefore resolves an omitted seed to `0`; explicit seeds still
|
||||
override it, and every other pipeline keeps its existing random default. This makes the
|
||||
documented `--pipeline qwen-zimage` command reproduce the seed condition used for the
|
||||
release-candidate oracle evidence without claiming cross-seed certification.
|
||||
|
||||
**Product recommendation:** keep ControlNet as the default because it is much cheaper and
|
||||
supports CUDA, XPU, MPS, and CPU. Treat it as the compatibility baseline, not the
|
||||
highest-fidelity result. When CUDA is available and visual quality matters more than
|
||||
latency or cost, recommend `qwen-zimage`, especially for face-heavy content. This
|
||||
recommendation is based on the measured identity advantage and the oracle-negative exact
|
||||
candidate; it does not upgrade that exact-output result into broad certification.
|
||||
|
||||
**Follow-up (2026-06-20) — the content-routed lane / mixed dual-pass was tested and DROPPED.**
|
||||
A `--pipeline auto` router (Haar+MSER → text→qwen / faces→controlnet / both→mixed) and a
|
||||
@@ -148,11 +198,10 @@ original bbox and stayed collision-immune. Regression-guarded by
|
||||
**non-regenerative detail-restoration** technique (high-frequency residual transfer,
|
||||
guided filtering) that recovers smoothed faces without re-introducing the watermark.
|
||||
Research angle 4 produced zero surviving claims, so it is unanswered.
|
||||
2. No claim provides measured face-fidelity numbers (ArcFace/LPIPS/Laplacian) for ANY
|
||||
recommended intervention at the project's scrub floors. All fidelity evidence is the
|
||||
project's own internal measurement. The improvements are mechanistically sound but
|
||||
unproven for this exact metric, so validate with `scripts/fidelity_metrics.py`
|
||||
before shipping.
|
||||
2. No external claim provides measured face-fidelity numbers (ArcFace/LPIPS/Laplacian) for
|
||||
any recommended intervention at the project's scrub floors. The later direct
|
||||
`qwen-zimage` comparisons are the project's own measurements, not external evidence or
|
||||
a certification.
|
||||
3. Several vendor model cards are marketing-register primary sources (Qwen blog,
|
||||
Z-Image card). Load-bearing facts (license, params, API levers) are independently
|
||||
corroborated, but comparative quality framings are author glosses.
|
||||
@@ -172,14 +221,11 @@ original bbox and stayed collision-immune. Regression-guarded by
|
||||
- What **non-regenerative detail-restoration** method recovers smoothed faces WITHOUT
|
||||
re-introducing SynthID? Note: residual transfer from the ORIGINAL risks copying back
|
||||
watermark-carrying high frequencies, so it must be verified against the SynthID oracle.
|
||||
- Does adding Qwen-Image-ControlNet (canny/depth) at the certified floors (OpenAI 0.10,
|
||||
Gemini 0.25) actually raise face Laplacian/LPIPS toward the SDXL+ControlNet numbers
|
||||
(0.62 / 0.09) WITHOUT re-introducing SynthID, or does the structure constraint
|
||||
preserve the watermark the way ControlNet can on photoreal content (the existing
|
||||
"SynthID CAN survive controlnet at low strength" caveat)?
|
||||
- Head-to-head: does Z-Image-Turbo at its scrub floor match Qwen's text advantage
|
||||
(CJK+Cyrillic CER) while not worsening faces, and what are Z-Image's own SynthID
|
||||
scrub floors and seed-robustness (none exist yet)?
|
||||
- Head-to-head: does `qwen-zimage` retain its measured ArcFace gain across more portraits
|
||||
and mixed scenes, match Qwen's text advantage (CJK+Cyrillic CER), and clear SynthID
|
||||
robustly across content types and seeds?
|
||||
- Can YuNet's serial face workload be bounded to foreground/relevant faces without losing the
|
||||
small faces the upstream YOLO path would process?
|
||||
|
||||
## Refuted claims (do NOT rely on these)
|
||||
|
||||
|
||||
@@ -249,7 +249,7 @@ from the test set + this doc).
|
||||
img2img at the certified strength (0.20 OpenAI, 0.30 Gemini-capped-1536)
|
||||
with the canny edge map.
|
||||
3. **Oracle validation** on the cert sweep: run the new PhotoMaker variant
|
||||
through `raiw-app/modal_cert.py` over the same 6 image set, certify on the
|
||||
through the isolated Modal certification harness over the same 6 image set, certify on the
|
||||
per-vendor oracles. Expected: SynthID cleared (the regeneration is the same)
|
||||
AND identity recovered (the embedding adds it back).
|
||||
4. **Honest exit criteria.** Ship only if BOTH oracle reads clean AND a small
|
||||
|
||||
+11
-1
@@ -421,6 +421,16 @@ conditioning, never by copying original pixels.**
|
||||
The plain-SDXL profile was also renamed `default` -> `sdxl` (`default` stays as an
|
||||
alias). The 0.10/0.15 numbers in this analysis are the PRE-raise values it was
|
||||
measured at. See §5.2.**
|
||||
- **Highest-fidelity CUDA option:** `--pipeline qwen-zimage` is the recommended
|
||||
quality mode when preserving face identity matters more than latency, model size,
|
||||
and GPU cost. ControlNet remains the default because it is much cheaper and supports
|
||||
CUDA, XPU, MPS, and CPU, but canny conditioning preserves edges rather than identity.
|
||||
On two direct upstream comparisons, `qwen-zimage` retained substantially more
|
||||
ArcFace identity than polished ControlNet. On 2026-07-25 the exact six-output
|
||||
`visible -> qwen-zimage -> metadata` candidate was negative in the corresponding
|
||||
OpenAI and Gemini oracles. This is a quality recommendation for the measured content,
|
||||
not broad removal certification; very small text can still degrade.
|
||||
See `docs/known-limitations.md` for the metrics, runtime, and validation scope.
|
||||
- **Face identity:** canny holds face *structure* but not *identity*. Shipped as the
|
||||
optional `--restore-faces` GFPGAN post-pass (`face_restore.py`, the `restore`
|
||||
extra, experimental/opt-in, off by default). It runs GFPGAN on the ORIGINAL
|
||||
@@ -589,7 +599,7 @@ openai.com/verify):
|
||||
robust to downscaling by design, and the study's resolution trend says LOWER
|
||||
processing res needs LESS strength, so 1024 was never the wall.)
|
||||
|
||||
**Certified controlnet floors (Modal GPU sweep `raiw-app/modal_cert.py` + oracle,
|
||||
**Certified controlnet floors (isolated Modal GPU sweep + oracle,
|
||||
restore OFF, <= 1536, each vendor on its own oracle):** OpenAI **0.20** (2 photoreal x
|
||||
seed {1,2,3} = 6/6 clean; the 0.15-flipper is seed-robust at 0.20) and Gemini **0.30**
|
||||
(0.20 detected -> 0.30 clean on 2/2 seeds). OpenAI 0.20 transfers to prod
|
||||
|
||||
Reference in New Issue
Block a user