mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-06 22:18:36 +02:00
Finish CPU offload support on current main
This commit is contained in:
@@ -321,11 +321,31 @@ Lazy `_get_{lama,migan}_session` singletons; `{lama,migan}_available()` guard th
|
||||
|
||||
## `noai/watermark_remover.py`
|
||||
|
||||
`noai/watermark_remover.py` — the `WatermarkRemover` class has three diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id`). `sdxl`/`controlnet` share the SDXL base (`DEFAULT_MODEL_ID`); `qwen` is its own base (`QWEN_MODEL_ID`).
|
||||
`noai/watermark_remover.py` — the `WatermarkRemover` class has four diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id`). `sdxl`/`controlnet` share the SDXL base (`DEFAULT_MODEL_ID`); `qwen` is its own base (`QWEN_MODEL_ID`); `qwen-zimage` delegates to the fixed two-stage stack in `noai/qwen_zimage_pipeline.py`.
|
||||
|
||||
**`sdxl`** (renamed from `default` 2026-06-09; `default` kept as a back-compat alias via `normalize_profile`) runs plain SDXL img2img (`_run_img2img`); it is the lighter opt-down alternative (no ControlNet weights).
|
||||
|
||||
**`qwen`** (`_run_qwen`, `_load_qwen_pipeline`) runs `QwenImageImg2ImgPipeline` on `Qwen/Qwen-Image` (20B MMDiT, Apache-2.0 code AND weights). The scrub still comes from the img2img `strength`; Qwen's value is **text preservation** (incl. CJK and small text). **Metric-measured nuance (2026-06-19, `scripts/fidelity_metrics.py`, do NOT trust the eyeball here — it misled). Compare ONLY at each pipeline's oracle-confirmed scrub floor (outputs where SynthID is removed in BOTH — an equal-strength compare is invalid where it leaves one un-scrubbed; Qwen at 0.15 does not clear Gemini): Qwen wins TEXT (lower OCR CER across EN/RU/ZH, perfect Chinese) but controlnet wins FACES (higher Laplacian-variance retention and lower LPIPS — Qwen smooths faces MORE; ArcFace identity favors controlnet 0.546 vs 0.331 at the Gemini floors).** So Qwen is the better text-preserving remover, NOT a universal fidelity win — controlnet's canny edge map holds face skin detail better. Specifics: bf16 on CUDA (fp16 risks overflow on the 20B MMDiT — see the dtype branch in `__init__`); loads `QWEN_MODEL_ID` unless `--model` is overridden; the call shape lives in the pure module helper `_build_qwen_kwargs` (unit-tested without torch in `tests/test_platform.py::TestQwenKwargs`), which uses Qwen's `true_cfg_scale` (NOT SDXL's `guidance_scale` — the CLI `--guidance-scale` maps onto it; ~4.0 is typical, the SDXL default 7.5 is high for Qwen) and an explicit `negative_prompt` (`_QWEN_PROMPT`/`_QWEN_NEGATIVE`). It is CUDA/cloud-class (the 20B does not fit MPS), so `_run_qwen` has NO MPS->CPU fallback — an error propagates. `_load_qwen_pipeline` raises a clear ImportError if the installed diffusers lacks `QwenImageImg2ImgPipeline`. **CERTIFIED oracle floors (Modal A100-80GB, 2026-06-20): OpenAI 0.10 (seed-robust — clean on seeds 0-4), Gemini 0.25 (seed 0 verified on 2 images; the Gemini oracle rate-limits volume seed-repeat, so PIN a seed in prod). The Gemini floor (0.25) is HIGHER than the certified controlnet Gemini floor (0.15); `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`), so `--pipeline qwen` gets the 0.25 Gemini floor automatically -- the old manual `--strength 0.25` workaround is retired. `_build_qwen_kwargs` passes an explicit `height`/`width` from the input (floored to /16 via the pure `_qwen_target_size`); WITHOUT it the img2img pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (the abba 2816x1536 case came back 1024x1024, distorting the scene and garbling text — fixed 2026-06-20, tested in `TestQwenKwargs`).** Fidelity vs controlnet was measured at the certified floors (`scripts/fidelity_metrics.py`), NOT eyeballed. **`qwen` is a MANUAL opt-in only — there is NO auto-router (one was prototyped and DROPPED, see below).** It wins ONE niche: clean body text on a plain background, NO faces (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES and **display/decorative text in a scene** (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes; qwen re-renders and garbles them). **`--pipeline auto` + a faces+text mixed dual-pass were built and DROPPED (2026-06-20):** on the canonical faces+text case (abba) controlnet wins EVERY metric incl. text, so grafting qwen text would only hurt; and "text→qwen" is undecidable cheaply (it is body-vs-display text that matters). The router/detector/mixed modules were removed; the geometry fix + the Qwen strength ladder were kept (they make the manual `--pipeline qwen` correct). **Do NOT retry "add a Qwen ControlNet to close the face gap" — it was built, measured, and CLOSED 2026-06-20:** a DiffSynth blockwise-canny Qwen ControlNet did not restore face skin texture (lapvar flat 0.40, canny carries edges not skin grain) and no permissively-licensed Qwen tile/detail/skin ControlNet exists anywhere (all conditioning is geometry). Faces stay on controlnet; the next improvement lead is Z-Image-Turbo (Apache-2.0, unmeasured floor). Full record + the deep-research sweep in `docs/qwen-improvement-research.md`.
|
||||
**`qwen`** (`_run_qwen`, `_load_qwen_pipeline`) runs `QwenImageImg2ImgPipeline` on `Qwen/Qwen-Image` (20B MMDiT, Apache-2.0 code AND weights). The scrub still comes from the img2img `strength`; Qwen's value is **text preservation** (incl. CJK and small text). **Metric-measured nuance (2026-06-19, `scripts/fidelity_metrics.py`, do NOT trust the eyeball here — it misled). Compare ONLY at each pipeline's oracle-confirmed scrub floor (outputs where SynthID is removed in BOTH — an equal-strength compare is invalid where it leaves one un-scrubbed; Qwen at 0.15 does not clear Gemini): Qwen wins TEXT (lower OCR CER across EN/RU/ZH, perfect Chinese) but controlnet wins FACES (higher Laplacian-variance retention and lower LPIPS — Qwen smooths faces MORE; ArcFace identity favors controlnet 0.546 vs 0.331 at the Gemini floors).** So Qwen is the better text-preserving remover, NOT a universal fidelity win — controlnet's canny edge map holds face skin detail better. Specifics: bf16 on CUDA (fp16 risks overflow on the 20B MMDiT — see the dtype branch in `__init__`); loads `QWEN_MODEL_ID` unless `--model` is overridden; the call shape lives in the pure module helper `_build_qwen_kwargs` (unit-tested without torch in `tests/test_platform.py::TestQwenKwargs`), which uses Qwen's `true_cfg_scale` (NOT SDXL's `guidance_scale` — the CLI `--guidance-scale` maps onto it; ~4.0 is typical, the SDXL default 7.5 is high for Qwen) and an explicit `negative_prompt` (`_QWEN_PROMPT`/`_QWEN_NEGATIVE`). It is CUDA/cloud-class (the 20B does not fit MPS), so `_run_qwen` has NO MPS->CPU fallback — an error propagates. `_load_qwen_pipeline` raises a clear ImportError if the installed diffusers lacks `QwenImageImg2ImgPipeline`. **CERTIFIED oracle floors (Modal A100-80GB, 2026-06-20): OpenAI 0.10 (seed-robust — clean on seeds 0-4), Gemini 0.25 (seed 0 verified on 2 images; the Gemini oracle rate-limits volume seed-repeat, so PIN a seed in prod). The Gemini floor (0.25) is HIGHER than the certified controlnet Gemini floor (0.15); `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`), so `--pipeline qwen` gets the 0.25 Gemini floor automatically -- the old manual `--strength 0.25` workaround is retired. `_build_qwen_kwargs` passes an explicit `height`/`width` from the input (floored to /16 via the pure `_qwen_target_size`); WITHOUT it the img2img pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (the abba 2816x1536 case came back 1024x1024, distorting the scene and garbling text — fixed 2026-06-20, tested in `TestQwenKwargs`).** Fidelity vs controlnet was measured at the certified floors (`scripts/fidelity_metrics.py`), NOT eyeballed. **`qwen` is a MANUAL opt-in only — there is NO auto-router (one was prototyped and DROPPED, see below).** It wins ONE niche: clean body text on a plain background, NO faces (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES and **display/decorative text in a scene** (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes, qwen re-renders and garbles them). **`--pipeline auto` + a faces+text mixed dual-pass were built and DROPPED (2026-06-20):** on the canonical faces+text case controlnet wins EVERY metric incl. text, so grafting qwen text would only hurt; and "text→qwen" is undecidable cheaply (it is body-vs-display text that matters). The router/detector/mixed modules were removed; the geometry fix + the Qwen strength ladder were kept (they make the manual `--pipeline qwen` correct). **Do NOT retry "add a Qwen ControlNet to close the face gap" — it was built, measured, and CLOSED 2026-06-20:** a DiffSynth blockwise-canny Qwen ControlNet did not restore face skin texture (lapvar flat 0.40, canny carries edges not skin grain) and no permissively-licensed Qwen tile/detail/skin ControlNet exists anywhere (all conditioning is geometry). The Z-Image face-crop lead is now implemented as the separate `qwen-zimage` profile and has direct face metrics on two official upstream examples plus one crowded fixture. Its exact current six-output candidate is negative in the corresponding provider oracles, while broad seeded removal and text behavior remain unmeasured. Full record + the deep-research sweep in `docs/qwen-improvement-research.md`.
|
||||
|
||||
## `noai/qwen_zimage_pipeline.py`
|
||||
|
||||
`qwen-zimage` is the recommended high-quality, manual CUDA profile ported from `cebeuq/Synthid-Bypass` v2. `controlnet` remains the default for compatibility and cost; callers that prioritize output fidelity, especially face identity, should select `qwen-zimage`. The full-frame stage uses DiffSynth `QwenImagePipeline` with `Qwen/Qwen-Image-2512`, `lightx2v/Qwen-Image-2512-Lightning` at four steps, and `DiffSynth-Studio/Qwen-Image-Blockwise-ControlNet-Canny`. The Lightning scheduler uses `exponential_shift_mu=log(3)`, the DiffSynth equivalent of the source graph's AuraFlow shift 3. Its default denoise is the source custom node's exact megapixel formula at adaptive level 6; an explicit `--strength` overrides that global value. Its profile seed defaults to `0`, matching the oracle-negative release candidate; an explicit seed still wins. Other profiles keep their existing random default.
|
||||
|
||||
The face stage detects boxes on the original input with OpenCV YuNet and follows the active Impact Pack SAM path from the source graph: each box supplies both the `center-1` positive point and the box prompt; proposals at predicted IoU >= 0.93 are unioned, or the highest-IoU proposal is used when none passes; the result is intersected with the detector box. The inactive MediaPipe node in the workflow has no downstream link. YuNet uses its own calibrated score threshold, 0.5: copying the upstream YOLO threshold of 0.2 admitted background/decorative false positives and duplicate boxes, which multiplies the serial face-stage cost. The 0.5 gate retained all visible faces in the public and upstream comparison fixtures while reducing the crowded group from 36 boxes to 18 and the poster from 30 to 10. Crops expand by the source graph's factor 2.5 and run `Tongyi-MAI/Z-Image-Turbo` for eight steps. Every face uses the denoise derived from the largest face's area ratio, matching the source graph's `largest_face` mode, then pastes through the clipped SAM mask with feather 10. If SAM fails, a box-derived ellipse mask is used rather than aborting the global removal.
|
||||
|
||||
The active graph was traced from upstream commit `3007d0351596ae0a78b7074dae7ad179710b1e48` and its linked Impact Pack implementation, not inferred from the README or node names. YuNet is one intentional substitution: the active reference path uses an Ultralytics YOLO face detector, while this package avoids adding its AGPL runtime. The first-use model download targets GitHub's media endpoint rather than the repository's 131-byte Git LFS pointer and verifies the published 232589-byte model by SHA-256 before caching it. The other runtime differences are full safetensors rather than quantized GGUF models, DiffSynth's first-order Qwen Lightning and Z-Image FlowMatch samplers rather than the graph's DPM++ 2M / SGM Uniform and `res_2s` / `bong_tangent` pairs, and the absence of the detailer's 20 px latent noise-mask feather. This port regenerates the expanded crop and composites only the feathered SAM mask; generated pixels outside the face mask are discarded. The architecture and active decision path match the graph, but the runtime is not bit-identical to ComfyUI.
|
||||
|
||||
The implementation has its own `qwen-zimage` optional dependency group because DiffSynth, torchvision, and the additional model downloads are large. `--model` is rejected for this fixed profile. `--tile` runs only the global Qwen stage through `noai.tiling.run_tiled`; the global denoise is still derived from the full-frame megapixel count and the same seed is reused for each deterministic tile. After feather blending, YuNet, SAM, and Z-Image run once against the full original/global result, so faces are neither duplicated nor dropped at tile boundaries. The SDXL minimum-resolution floor is disabled, and CLI adaptive polish defaults off for this profile, so the two-stage result is not followed by a repository-specific post-process. DiffSynth requires the PIL input, Canny control, and explicit dimensions to agree on the same /16 latent grid: `_resize_to_target` aligns global and face pixels before `build_global_kwargs` / `build_face_kwargs`, and the global output is restored to the exact original size. Passing floored dimensions with unaligned pixels caused a real VAE/noise-grid shape mismatch on the official example 12 input; the call-shape assertions were observed failing before the fix. An explicit `--adaptive-polish` still opts in. Pure helpers cover both adaptive denoise formulas, /16 dimensions, call shapes, Canny generation, masked compositing, and the qwen-zimage tiling seam; an integration test guards dispatch from `WatermarkRemover`. A real 4096x3072, 20-tile H100 smoke completed through this exact branch on 2026-07-25 with dimensions preserved and no visible or 99th-percentile gradient outlier at a tile boundary; the measured runtime, memory, and fidelity figures are in `docs/known-limitations.md`. The exact seed-0 non-tiled release candidate is oracle-verified; tiled outputs still require their own provider-oracle check.
|
||||
|
||||
DiffSynth normally offloads the Z-Image text encoder, DiT, and VAE to CPU after every face call. That placement dominated crowded-scene latency even though the eight diffusion steps themselves were fast. `resolve_face_model_residency` keeps the full face stack on CUDA when total VRAM is at least 64 GiB; smaller cards preserve the original offload path. Callers can explicitly override the decision through `QwenZImagePipeline.keep_face_models_on_device`. The implementation intentionally loads the stack with the normal CPU-managed config, rewrites the managed modules' offload/onload/preparing placement to their CUDA computation device, and moves them once. Loading the same models directly into CUDA cut face inference further but increased setup from 32.282 to 254.632 seconds, making a cold single request more expensive; that variant was rejected. The shipped fast-load residency changes only model placement, not weights, dtypes, prompts, seeds, schedules, masks, or compositing. On the 18-face H100 fixture it produced a pixel-identical output versus offload while reducing face regeneration from 181.764 to 38.272 seconds and total inference from 262.072 to 133.543 seconds. Setup rose only from 32.282 to 43.960 seconds, so cold setup plus inference fell from 294.354 to 177.503 seconds. Peak CUDA allocation rose from 24.364 to 43.477 GiB.
|
||||
|
||||
Both stage prompts are constants, but DiffSynth 2.0.18 exposes their embeddings only through internal PipelineUnits and re-runs the corresponding text encoder on every call. `_cache_static_prompt_embeddings` wraps the exact prompt unit selected by its output signature and memoizes its returned tensors by prompt text. It bypasses the cache whenever `edit_image` participates, so image-conditioned embeddings cannot be reused accidentally. With CFG 1.0, the unit runner already shares the positive result with the negative branch. On a warm H100 sequence, a preceding no-face request populated the Qwen prompt cache; the 18-face case then fell from 133.543 to 78.474 seconds. The global stage fell from 89.720 to 43.334 seconds, and the face stage from 38.272 to 30.592 seconds as the fixed Z-Image prompt was encoded only once. Both the no-face and 18-face cached outputs were pixel-identical to their no-cache references, and peak allocation remained 43.477 GiB. The Qwen saving applies from the second request in a container; the Z-Image saving applies within the first multi-face request after its first face.
|
||||
|
||||
Validation status is deliberately narrower than the existing `qwen` certification. The user reported the upstream workflow as Gemini-oracle negative. The current port has API, unit, dispatch, GPU integration runs, and a direct comparison with two official upstream before/after pairs. On 2026-07-25 the user checked all six current outputs in the provider-separated `full-clean-final-candidate-2026-07-25-by-oracle` bundle with the corresponding provider oracles and confirmed that none retained SynthID or the provider generation signal. The checked bytes used the complete `visible -> qwen-zimage -> metadata` route, the calibrated YuNet 0.5 gate, and the shipped prompt-cache/model-residency optimizations. This supersedes the earlier first-port batch check as the release-candidate result. It certifies the exact seed-0 outputs, not every seed, resolution, or content class. Broad text certification remains open, so the profile stays an experimental manual opt-in even though it is the recommended quality mode.
|
||||
|
||||
**Direct port measurement (2026-07-24, seed 0):** on official upstream examples 10 and 12, local `qwen-zimage` retained ArcFace identity at 0.950 and 0.947 versus 0.701 and 0.548 for the current polished ControlNet output. Face LPIPS was 0.045 and 0.015 versus ControlNet's 0.105 and 0.061. The published upstream outputs retained identity at 0.976 on both and face LPIPS at 0.172 and 0.014; upstream example 10 is strongly penalized by its published downscale, while example 12 was compared at the same published dimensions. Local whole-image LPIPS / SSIM were 0.167 / 0.765 and 0.085 / 0.896, better than the published upstream 0.259 / 0.627 and 0.111 / 0.777. ControlNet still preserved more texture, but the faces drifted. The earlier crowded `gemini_3` run showed the same identity direction, 0.795 versus 0.587/0.588, while smoothing skin and changing the full frame more. The final July 25 oracle check supplies the removal verdict for the exact current candidate bytes only.
|
||||
|
||||
Two integration failures from the first Modal passes are regression-guarded. SAM model pixels must be cast to the model's bfloat16 while geometric prompts remain float32; casting everything either fails or changes prompt semantics. SAM `pred_masks` and `iou_scores` must then be converted through float32 before NumPy because NumPy rejects bfloat16. A third visually severe failure came from accepting an unconstrained SAM mask: it split faces with hard seams. The center point plus box prompt and the final detector-box intersection are both load-bearing.
|
||||
|
||||
**`controlnet`** (**the DEFAULT pipeline since 2026-06-09** for `invisible`/`all`/`batch` and both engine ctors; `_run_controlnet`, `_load_controlnet_pipeline`) runs `StableDiffusionXLControlNetImg2ImgPipeline` with the SDXL-native canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` (`watermark_profiles.CONTROLNET_CANNY_MODEL`): the control image is `cv2.Canny(gray, 100, 200)` stacked to 3 channels (`_CANNY_LOW`/`_CANNY_HIGH`, prompt `_CONTROLNET_PROMPT` / `_CONTROLNET_NEGATIVE`).
|
||||
|
||||
@@ -337,11 +357,11 @@ At the shared low removal strength the canny edge-conditioning keeps the regener
|
||||
|
||||
**But the reverse also holds: a flat-graphic logo/poster SURVIVED `default` while clearing controlnet** -- removal at the low strength is content×pipeline dependent and neither pipeline is universally safe; the real lever is a higher strength. See the controlnet Known-limitations bullet for the full table + root cause. Canny holds face STRUCTURE but NOT identity (the regenerated face drifts in likeness -- canny carries edges, not identity). The drifted cleaned face is the LEAST-AI state we can reach without re-introducing SynthID; the library does NOT ship a face-restore extra. Every restore approach we evaluated (GFPGAN-on-cleaned, PhotoMaker-V2 txt2img, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps, 2026-06-04 - 2026-06-08 Modal cert sweeps) regenerated the face from an ArcFace embedding via SDXL diffusion -- which makes the output face look MORE AI-generated, not less. Empirical conclusion in `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up". For production face preservation, ship the cleaned image as-is. `controlnet_conditioning_scale` (ctor arg, default 1.0) is the structure-preservation knob. Same dtype rule as `default` (fp32 on cpu/mps, fp16 only on cuda/xpu; the fp16-fixed SDXL VAE `_SDXL_FP16_VAE_ID` is swapped in on fp16 GPUs -- issue #29) and the same MPS->CPU fallback (reload on cpu/fp32, drop a non-cpu generator, retry once).
|
||||
|
||||
**Tiled diffusion (`tile`/`tile_size`/`tile_overlap` ctor-path args, CLI `--tile`, issue #10):** for large inputs that OOM at native resolution, `remove_watermark` can process the diffusion pass in overlapping sliding-window tiles instead of one forward pass — the lossless alternative to a `--max-resolution` downscale. The single-image generation closure was refactored into `_generate_one(img)` (dispatches controlnet/img2img, generator shared so the seed advances deterministically across tiles), and `_generate()` routes it through `noai.tiling.run_tiled` when `tile` is set AND `max(init_image.size) > tile_size` (a sub-tile image runs one pass unchanged). The ControlNet canny edge map is rebuilt per tile inside `_generate_one`, so structure preservation is tile-local. See `noai/tiling.py` below and the tiled-diffusion subsection in `docs/known-limitations.md` for the geometry, the partition-of-unity blend, and the quality caveat.
|
||||
**Tiled diffusion (`tile`/`tile_size`/`tile_overlap` ctor-path args, CLI `--tile`, issue #10):** for large inputs that OOM at native resolution, `remove_watermark` can process the diffusion pass in overlapping sliding-window tiles instead of one forward pass — the lossless alternative to a `--max-resolution` downscale. For SDXL, ControlNet, and base Qwen, the single-image generation closure was refactored into `_generate_one(img)` (dispatches controlnet/img2img, generator shared so the seed advances deterministically across tiles), and `_generate()` routes it through `noai.tiling.run_tiled` when `tile` is set AND `max(init_image.size) > tile_size` (a sub-tile image runs one pass unchanged). The ControlNet canny edge map is rebuilt per tile inside `_generate_one`, so structure preservation is tile-local. `qwen-zimage` takes a different route: `_generate()` dispatches once to `QwenZImagePipeline.run`, that runtime tiles only `_run_global`, then performs one full-frame face stage after blending. See `noai/tiling.py` below and the tiled-diffusion subsection in `docs/known-limitations.md` for the geometry, the partition-of-unity blend, and the quality caveat.
|
||||
|
||||
## `noai/tiling.py`
|
||||
|
||||
Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL only). `plan_tiles(w, h, tile_size, overlap)` returns a row-major grid of uniform-size `Tile` boxes — every tile is exactly `tile_size` (the SDXL training size), with the last tile on each axis pulled back flush to the far edge (`_axis_positions` clamps a pathological `overlap >= tile` to `tile - 1` so the step stays >= 1). `feather_weights(w, h, overlap)` is a separable linear taper (1 in the interior, ramping toward each edge) floored at `_WEIGHT_EPS` so it is **strictly positive everywhere** — that makes the normalized `accum / weight_sum` blend a partition of unity, so identical/unchanged tiles reconstruct the input exactly (the seam-free guarantee). `run_tiled(generate_tile, image, tile_size, overlap, set_progress)` is the orchestration loop: crop each planned tile, call `generate_tile` (one diffusion pass on a single PIL tile — injected, so this stays decoupled from the pipeline), resize a latent-grid-rounded result back to the exact tile size, and feather-accumulate. All three are unit-tested without the model (`tests/test_tiling.py`: axis math, grid coverage, taper shape/symmetry/positivity, identity reconstruction, per-tile call count, and the resize-back path). New blend tuning belongs in these pure helpers, not inlined into the runner.
|
||||
Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL only). `plan_tiles(w, h, tile_size, overlap)` returns a row-major grid of uniform-size `Tile` boxes — every tile is exactly `tile_size`, with the last tile on each axis pulled back flush to the far edge (`_axis_positions` clamps a pathological `overlap >= tile` to `tile - 1` so the step stays >= 1). `feather_weights(w, h, overlap)` is a separable linear taper (1 in the interior, ramping toward each edge) floored at `_WEIGHT_EPS` so it is **strictly positive everywhere** — that makes the normalized `accum / weight_sum` blend a partition of unity, so identical/unchanged tiles reconstruct the input exactly (the seam-free guarantee). `run_tiled(generate_tile, image, tile_size, overlap, set_progress)` is the orchestration loop: crop each planned tile, call `generate_tile` (one diffusion pass on a single PIL tile — injected, so this stays decoupled from the pipeline), resize a latent-grid-rounded result back to the exact tile size, and feather-accumulate. All three are unit-tested without the model (`tests/test_tiling.py`: axis math, grid coverage, taper shape/symmetry/positivity, identity reconstruction, per-tile call count, and the resize-back path). New blend tuning belongs in these pure helpers, not inlined into the runner.
|
||||
|
||||
`feather_region_composite(base, regenerated, box, *, feather)` is the pure region-targeted compositor for **AI-enhanced composites** (roadmap P1#8; `identify` `ai_source_kind == "enhanced"`, digitalSourceType `compositeWithTrainedAlgorithmicMedia`). It blends `regenerated` over `base` inside `box = (x, y, w, h)` with a separable linear taper of `feather` px at the box edges (the taper anchors to ~0 at the boundary, so unlike `feather_weights` it is NOT floored — the result equals `base` EXACTLY outside the box), preserving dtype and supporting HxW or HxWxC. It backs `WatermarkRemover.remove_watermark(region=..., region_feather=...)`: the remover regenerates the frame (or tiles), then composites only the AI box back over the original input, so the real photo outside the box stays pixel-exact and only the AI region is scrubbed. The box is caller-supplied (a C2PA composite manifest carries no reliable machine-readable region); the no-model lossless region path remains `region_eraser.erase`. Unit-tested in `tests/test_tiling.py::TestFeatherRegionComposite` (outside-box exactness, interior == regenerated, hard-paste at feather 0, monotonic seam ramp, dtype/grayscale/clamp/empty-box/shape-mismatch).
|
||||
|
||||
@@ -351,7 +371,7 @@ Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL on
|
||||
|
||||
History: `auto_config.plan()` was a content-adaptive planner that detected faces/text/edges (bundled OpenCV YuNet + PP-OCRv3 DBNet models) to route the pipeline and toggle the adaptive polish. Once `controlnet` became the default-and-only auto pipeline (it no longer downgrades a structure-less image to `sdxl`) and the adaptive polish was confirmed to **self-gate by detail level** (`humanizer.adaptive_polish` no-ops when the cleaned image already meets the input's Laplacian variance, so it does real work only on over-smoothed photo/face texture and ~nothing on text/flat), the detection no longer changed any behavior — it only annotated a `reason` string. So the whole layer was deleted: `auto_config.py`, `tests/test_auto_config.py`, and the two detection assets (`assets/face_detection_yunet_2023mar.onnx`, `assets/text_detection_ppocrv3_2023may.onnx`, ~2.6 MB).
|
||||
|
||||
**`--auto` is now a DEPRECATED no-op** (`cli._resolve_auto_polish`): controlnet is already the default pipeline AND the adaptive polish is ON by default, so `--auto` has nothing left to do — it only prints a deprecation warning and passes `adaptive_polish` through unchanged (an explicit `--no-adaptive-polish` still wins). (Originally it re-enabled the polish; once the polish default flipped to ON the same day, the parameter-source branch became dead and was dropped.) The **adaptive polish itself lives on** in `humanizer.adaptive_polish` (CLI `--adaptive-polish/--no-adaptive-polish`, **ON by default since 2026-06-09** — it self-gates to a no-op where there is no detail deficit, so default-on is safe; uses the full-res original as the detail reference) — see the `humanizer` test note. `batch` resolves the polish once before the loop (one warning) and caches the invisible engine per pipeline (`ctx.obj["_inv_engines"]`).
|
||||
**`--auto` is now a DEPRECATED no-op** (`cli._resolve_auto_polish`): controlnet is already the default pipeline AND the adaptive polish is ON by default, so `--auto` has nothing left to do — it only prints a deprecation warning and passes `adaptive_polish` through unchanged (an explicit `--no-adaptive-polish` still wins). (Originally it re-enabled the polish; once the polish default flipped to ON the same day, the parameter-source branch became dead and was dropped.) The **adaptive polish itself lives on** in `humanizer.adaptive_polish` (CLI `--adaptive-polish/--no-adaptive-polish`, **ON by default since 2026-06-09 for the original profiles** — it self-gates to a no-op where there is no detail deficit; `qwen-zimage` defaults it off to preserve its upstream-matching output, and an explicit flag overrides either default) — see the `humanizer` test note. `batch` resolves the polish once before the loop (one warning) and caches the invisible engine per pipeline (`ctx.obj["_inv_engines"]`).
|
||||
|
||||
## Content `--pipeline auto` router + faces+text mixed dual-pass — PROTOTYPED and DROPPED (2026-06-20)
|
||||
|
||||
@@ -390,7 +410,7 @@ Full pipeline (visible + invisible + metadata). Same diffusion knobs as `invisib
|
||||
|
||||
### `invisible`
|
||||
|
||||
Diffusion SynthID removal. The `--tile/--no-tile` knob is the *lossless* alternative to a `--max-resolution` downscale for large inputs that OOM on MPS/GPU: it engages only when the long side exceeds `--tile-size` (default 1024); tiles are feather-blended over `--tile-overlap` px (default 128); pair with `--max-resolution 0`. `--adaptive-polish` is a detail-targeted polish that self-gates to a no-op where there is no deficit. `--auto` is deprecated and now a no-op that only warns (the polish it used to enable is ON by default). **No-signal skip (P0#5, roadmap):** before the diffusion runs, the command checks `identify.has_invisible_target(source)` (the `ProvenanceReport.ai_from_metadata` union: C2PA AI-issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, open DWT-DCT / TrustMark — visible marks do NOT count, they are a separate pass). When nothing is locally detectable it does NOT regenerate (that would only degrade a clean image — the dominant paid score-0 cause on no-watermark uploads): it writes NO output, prints guidance that does NOT claim the image is clean (a pixel SynthID is undetectable once its metadata proxy is gone), and exits **`EXIT_NO_INVISIBLE_SIGNAL` (2)** — same value/role as the visible `EXIT_NO_VISIBLE_MARK`. `--force/--no-force` (**default skip = ON**) runs the scrub regardless. The check fails SAFE (a detector exception → run, since leaving a watermark on a paid removal is worse than over-regenerating). Helpers `cli._no_invisible_signal_exit` + `identify.has_invisible_target`; regression-guarded by `tests/test_cli.py::TestInvisibleCommand::{test_invisible_no_signal_skips_and_exits_two,test_invisible_force_runs_scrub_on_no_signal,test_invisible_runs_without_force_when_signal_present}` and `tests/test_identify.py::TestHasInvisibleTargetFailSafe`. **Test trap:** any `invisible`/`all`/`batch` test that exercises the diffusion path on a signal-LESS fixture (e.g. the synthetic `sample_png`) MUST pass `--force`, or the new gate skips step 2 (so `mock_engine.remove_watermark` is never called / `invisible` exits 2).
|
||||
Diffusion SynthID removal. The `--tile/--no-tile` knob is the *lossless* alternative to a `--max-resolution` downscale for large inputs that OOM on MPS/GPU: it engages only when the long side exceeds `--tile-size` (default 1024); tiles are feather-blended over `--tile-overlap` px (default 128); pair with `--max-resolution 0`. `--cpu-offload/--no-cpu-offload` trades speed for lower CUDA VRAM use: SDXL, ControlNet, and base Qwen call Diffusers `enable_model_cpu_offload(device="cuda")`, which moves whole model components between CPU and GPU; `qwen-zimage` instead forces its face stack to use the existing offload path rather than automatic high-VRAM residency. The flag has no effect on CPU/MPS and fails loudly if a CUDA Diffusers pipeline lacks the offload method. `--adaptive-polish` is a detail-targeted polish that self-gates to a no-op where there is no deficit; it defaults off only on `qwen-zimage`. `--auto` is deprecated and now a no-op that only warns. **No-signal skip (P0#5, roadmap):** before the diffusion runs, the command checks `identify.has_invisible_target(source)` (the `ProvenanceReport.ai_from_metadata` union: C2PA AI-issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, open DWT-DCT / TrustMark — visible marks do NOT count, they are a separate pass). When nothing is locally detectable it does NOT regenerate (that would only degrade a clean image — the dominant paid score-0 cause on no-watermark uploads): it writes NO output, prints guidance that does NOT claim the image is clean (a pixel SynthID is undetectable once its metadata proxy is gone), and exits **`EXIT_NO_INVISIBLE_SIGNAL` (2)** — same value/role as the visible `EXIT_NO_VISIBLE_MARK`. `--force/--no-force` (**default skip = ON**) runs the scrub regardless. The check fails SAFE (a detector exception → run, since leaving a watermark on a paid removal is worse than over-regenerating). Helpers `cli._no_invisible_signal_exit` + `identify.has_invisible_target`; regression-guarded by `tests/test_cli.py::TestInvisibleCommand::{test_invisible_no_signal_skips_and_exits_two,test_invisible_force_runs_scrub_on_no_signal,test_invisible_runs_without_force_when_signal_present,test_invisible_cpu_offload_flows_to_engine}`, `tests/test_cli.py::TestAllCommand::test_all_cpu_offload_flows_to_engine`, `tests/test_cli.py::TestBatchCommand::test_batch_cpu_offload_flows_to_cached_engine`, and `tests/test_identify.py::TestHasInvisibleTargetFailSafe`. **Test trap:** any `invisible`/`all`/`batch` test that exercises the diffusion path on a signal-LESS fixture (e.g. the synthetic `sample_png`) MUST pass `--force`, or the new gate skips step 2 (so `mock_engine.remove_watermark` is never called / `invisible` exits 2).
|
||||
|
||||
### `visible`
|
||||
|
||||
@@ -398,4 +418,4 @@ Known-visible-mark removal by **localize -> fill**: each detected mark is locali
|
||||
|
||||
### `batch`
|
||||
|
||||
Process every supported image in a directory (output defaults to `<directory>_clean/`, set with `-o`). `--mode visible|invisible|metadata|all` (default `visible`); the invisible/all path reuses the **full `invisible` knob set** (`--strength`/`--steps`/`--guidance-scale`/`--pipeline`/`--controlnet-scale`/`--model`/`--device`/`--max-resolution`/`--min-resolution`/`--upscaler`/`--seed`/`--hf-token`/`--humanize`/`--unsharp`/`--adaptive-polish`/`--tile`/`--tile-size`/`--tile-overlap`/`--force`), plus `--backend` for the visible localize -> fill pass. `--adaptive-polish` is ON by default; `--auto` is deprecated and a no-op that only warns. **No-signal skip (P0#5):** in invisible/all mode each image runs the same `has_invisible_target` gate — a signal-less image is skipped (no diffusion); in `invisible` mode the input is copied through to the output dir so it stays complete, in `all` mode the visible-removed result is kept and metadata is still stripped. `--force` scrubs every image regardless. One engine cached per pipeline; the polish is resolved once before the loop. **Exit code (`batch` used to always exit 0, hiding failures):** `cmd_batch` raises `SystemExit(1)` when any image errored, OR when a `--mode invisible`/`all` image carried an invisible signal but the GPU extra was absent so its SynthID scrub was skipped — mirroring single `all`, it emits a loud "the invisible watermark was NOT removed on N image(s)" warning and (invisible mode) copies the input through so the output dir stays complete, rather than silently dropping the signal-bearing files that most needed processing. `_process_batch_image` returns that skipped-scrub flag; the loop tallies it. Regression-guarded by `tests/test_cli.py::TestBatchCommand::{test_batch_errors_exit_nonzero, test_batch_invisible_gpu_missing_writes_output_and_exits_nonzero}`.
|
||||
Process every supported image in a directory (output defaults to `<directory>_clean/`, set with `-o`). `--mode visible|invisible|metadata|all` (default `visible`); the invisible/all path reuses the **full `invisible` knob set** (`--strength`/`--steps`/`--guidance-scale`/`--pipeline`/`--controlnet-scale`/`--model`/`--device`/`--max-resolution`/`--min-resolution`/`--upscaler`/`--seed`/`--hf-token`/`--humanize`/`--unsharp`/`--adaptive-polish`/`--tile`/`--tile-size`/`--tile-overlap`/`--cpu-offload`/`--force`), plus `--backend` for the visible localize -> fill pass. `--adaptive-polish` is ON by default except on `qwen-zimage`; `--auto` is deprecated and a no-op that only warns. **No-signal skip (P0#5):** in invisible/all mode each image runs the same `has_invisible_target` gate — a signal-less image is skipped (no diffusion); in `invisible` mode the input is copied through to the output dir so it stays complete, in `all` mode the visible-removed result is kept and metadata is still stripped. `--force` scrubs every image regardless. One engine cached per pipeline; the polish is resolved once before the loop. **Exit code (`batch` used to always exit 0, hiding failures):** `cmd_batch` raises `SystemExit(1)` when any image errored, OR when a `--mode invisible`/`all` image carried an invisible signal but the GPU extra was absent so its SynthID scrub was skipped — mirroring single `all`, it emits a loud "the invisible watermark was NOT removed on N image(s)" warning and (invisible mode) copies the input through so the output dir stays complete, rather than silently dropping the signal-bearing files that most needed processing. `_process_batch_image` returns that skipped-scrub flag; the loop tallies it. Regression-guarded by `tests/test_cli.py::TestBatchCommand::{test_batch_errors_exit_nonzero, test_batch_invisible_gpu_missing_writes_output_and_exits_nonzero}`.
|
||||
|
||||
Reference in New Issue
Block a user