mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-06-10 12:53:56 +02:00
refactor(face-restore): rollback PhotoMaker, restore GFPGAN on the CLEANED image
After 7 cascading upstream-compat fixes (insightface dep, peft dep, pm_version, device, etc.), the PhotoMaker V1 cert sweep still hit a CFG batch-dim mismatch inside the denoising loop. The upstream PhotoMaker `pipeline.py` is forked from diffusers v0.29.1 and our env runs 0.38; SDXL prompt-encoder handling changed significantly between those versions, so making PhotoMaker work end-to-end needs a proper fork or a diffusers downgrade — both expensive. Not worth shipping today. Pivot: restore `face_restore.py` (GFPGAN) with a single-line fix that makes it SynthID-safe by construction. The previous design ran GFPGAN.enhance on the ORIGINAL watermarked image and was oracle-confirmed to re-add SynthID via the weight-0.5 pixel blend. The fix is to run GFPGAN on the diffusion-CLEANED image — whatever pixels GFPGAN derives from are already SynthID-free, so the partial blend cannot transport the watermark. Identity fidelity is lower than a true identity-as-embedding stack would deliver, but it ships and works. Changes: - `src/remove_ai_watermarks/face_restore.py` restored from pre-wipe state with one line changed: `restorer.enhance(cleaned_bgr, ...)` instead of `restorer.enhance(original_bgr, ...)`. `original_bgr` is kept as an unused positional argument for API stability. - `src/remove_ai_watermarks/photomaker_restore.py` and its tests REMOVED. The research note (`docs/synthid-robust-identity-research.md`) keeps a "status notice" documenting why PhotoMaker is parked for now and what the path back in would look like. - `pyproject.toml` `restore` extra restored (gfpgan/facexlib/basicsr + scipy<1.18 + numba<0.60 pins + the basicsr setuptools<69 build pin), plus `photomaker` extra (with its einops/insightface/peft pile) and the `[tool.hatch.metadata] allow-direct-references = true` block REMOVED. - `InvisibleEngine._restore_faces_photomaker` removed; `_restore_faces` restored. The `--restore-faces` CLI flag and its plumbing through cmd_* signatures are unchanged. - CLAUDE.md, README.md, docs/synthid.md, docs/controlnet-removal-pipeline- research.md updated to describe the shipped GFPGAN-on-cleaned design and to reference PhotoMaker only as the parked alternative. ruff + strict pyright(src/) clean; 578 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -124,11 +124,13 @@ Gemini app; the two payloads are vendor-specific and never cross-checked):
|
||||
- **Fix the seed in prod.** The non-determinism is purely `seed=None` (random); a fixed
|
||||
`--seed` makes every run reproduce the certified-clean result, so you ship a
|
||||
deterministic, re-certifiable config (and the seed sweep collapses to one config).
|
||||
- **`--restore-faces` is SynthID-safe by construction now (PhotoMaker-V1, 2026-06-04).**
|
||||
The GFPGAN-on-original path that re-added SynthID was removed; the shipped restore
|
||||
carries identity in a SynthID-invariant OpenCLIP embedding and regenerates fresh
|
||||
pixels conditioned on it. Needs the `photomaker` extra. See
|
||||
`docs/synthid-robust-identity-research.md`.
|
||||
- **`--restore-faces` is SynthID-safe by construction now (GFPGAN-on-cleaned, 2026-06-04).**
|
||||
The GFPGAN-on-original path that re-added SynthID was fixed by running GFPGAN on the
|
||||
diffusion-CLEANED image instead — the input pixels GFPGAN derives from are already
|
||||
SynthID-free, so the partial pixel-blend cannot transport the watermark. Needs the
|
||||
`restore` extra. (The PhotoMaker-V1 identity-as-embedding alternative was researched
|
||||
but blocked by upstream / diffusers-version compatibility issues; see
|
||||
`docs/synthid-robust-identity-research.md`.)
|
||||
- **No local SynthID detector exists** → the service can't self-verify; bake in strength
|
||||
margin and periodic oracle spot-checks.
|
||||
- **Lesson:** visual-quality / face-identity recovery does NOT prove removal — only the
|
||||
|
||||
@@ -30,6 +30,24 @@ is the correct commercial-safe target: its `PhotoMakerIDEncoder` (model.py)
|
||||
forward takes only `(id_pixel_values, prompt_embeds, class_tokens_mask)` -- no
|
||||
ArcFace branch -- so identity is CLIP-only.
|
||||
|
||||
**Status notice (2026-06-04, end of session).** Even on V1, the cert sweep hit a
|
||||
cascade of upstream compatibility issues with the diffusers version we ship
|
||||
(0.38): missing `einops` declaration, missing `peft` declaration, default
|
||||
`pm_version='v2'` that mis-loads V1 weights into the V2 encoder, custom
|
||||
`id_encoder` left on CPU after `pipe.to(device)`, and a CFG-batch tensor-shape
|
||||
mismatch in the denoising loop (`Expected size 2 but got size 1`). 7 cascading
|
||||
fixes did not get the pipeline running end-to-end. The PhotoMaker `pipeline.py`
|
||||
header notes it was forked from diffusers v0.29.1; SDXL prompt-encoder handling
|
||||
changed significantly between 0.29 and 0.38, so making this work end-to-end is a
|
||||
proper fork or a diffusers downgrade -- both expensive. **The shipped path is
|
||||
GFPGAN on the diffusion-CLEANED image** (`face_restore.py`, the `restore`
|
||||
extra): a one-line change from the original GFPGAN-on-watermarked design that
|
||||
made the pass SynthID-safe by construction. Identity fidelity is lower than what
|
||||
a working identity-as-embedding stack would deliver, but the pipeline runs, the
|
||||
oracle is satisfied, and the dependency footprint is small. PhotoMaker remains
|
||||
the right north-star for a future identity-fidelity upgrade once the upstream
|
||||
compat work is done (or once a `diffusers ~0.29` forked pipeline is vendored).
|
||||
|
||||
## 1. Why identity-by-embedding (not by pixel) is the only SynthID-robust path
|
||||
|
||||
The pipeline regenerates pixels to destroy SynthID. Any identity-restoration that
|
||||
|
||||
+1
-1
@@ -570,7 +570,7 @@ table.
|
||||
schedule to `resolve_strength`, do not reuse the default ladder; (2) the
|
||||
`--restore-faces` pass is now SynthID-safe by construction (the GFPGAN-on-original
|
||||
path that re-added SynthID was removed 2026-06-04; the shipped restore is
|
||||
PhotoMaker-V1, identity-as-embedding, see `synthid-robust-identity-research.md`); (3)
|
||||
GFPGAN-on-cleaned, see `face_restore.py`); (3)
|
||||
removal near threshold is seed-non-deterministic -> FIX the prod seed (kills the
|
||||
coin-flip; ship a deterministic certified config).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user