mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-06-10 12:53:56 +02:00
refactor(face-restore): drop GFPGAN, ship PhotoMaker-V2 as the sole restore (non-commercial)
Visual review of the GFPGAN-on-cleaned output (9-face grid, 1448x1086) showed it only polished the already-drifted face without restoring identity — useless for the "restore who is in the photo" intent. Dropping it. The shipped restore path is now PhotoMaker-V2, which delivers true identity-from- embedding face regeneration via a CLIP+ArcFace dual encoder. The ArcFace branch pulls InsightFace antelopev2/buffalo_l model packs at runtime, which InsightFace releases under a research-only license, so the whole extra is **NON-COMMERCIAL**. raiw.cc and any monetized deployment must NOT install the `photomaker` extra. This is called out at every entry point: CLI flag help, module docstring, pyproject extra block, CLAUDE.md extras bullet, README install snippet. Changes: - Deleted `src/remove_ai_watermarks/face_restore.py` and its tests. - Deleted the `restore` extra (gfpgan/facexlib/basicsr + scipy<1.18 / numba<0.60 pins) and the basicsr setuptools<69 build pin from pyproject.toml. - Restored `src/remove_ai_watermarks/photomaker_restore.py` (V2 this time: `TencentARC/PhotoMaker-V2`, `photomaker-v2.bin`, no `pm_version='v1'` override). - Restored the `photomaker` extra in pyproject with all the upstream-compat pins (einops, peft, onnxruntime, insightface) and the `allow-direct-references` hatch metadata block. - `InvisibleEngine` swapped `_restore_faces` -> `_restore_faces_photomaker`; `--restore-faces-method` removed (only one method, no choice). - CLI flag help, CLAUDE.md, README, docs/synthid.md, and docs/controlnet-removal-pipeline-research.md all updated. - docs/synthid-robust-identity-research.md status notice rewritten to list both abandoned commercial-safe attempts (V1 + GFPGAN-on-cleaned) and the non-commercial trade-off we accepted. ruff + strict pyright(src/) clean; 578 tests pass (the 9 GFPGAN tests are gone, the 11 PhotoMaker tests stay green). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -124,13 +124,14 @@ Gemini app; the two payloads are vendor-specific and never cross-checked):
|
||||
- **Fix the seed in prod.** The non-determinism is purely `seed=None` (random); a fixed
|
||||
`--seed` makes every run reproduce the certified-clean result, so you ship a
|
||||
deterministic, re-certifiable config (and the seed sweep collapses to one config).
|
||||
- **`--restore-faces` is SynthID-safe by construction now (GFPGAN-on-cleaned, 2026-06-04).**
|
||||
The GFPGAN-on-original path that re-added SynthID was fixed by running GFPGAN on the
|
||||
diffusion-CLEANED image instead — the input pixels GFPGAN derives from are already
|
||||
SynthID-free, so the partial pixel-blend cannot transport the watermark. Needs the
|
||||
`restore` extra. (The PhotoMaker-V1 identity-as-embedding alternative was researched
|
||||
but blocked by upstream / diffusers-version compatibility issues; see
|
||||
`docs/synthid-robust-identity-research.md`.)
|
||||
- **`--restore-faces` is PhotoMaker-V2 (NON-COMMERCIAL).** The GFPGAN-on-cleaned path
|
||||
was tried and rejected: it polished but did not restore identity. PhotoMaker-V2
|
||||
regenerates faces from a CLIP+ArcFace embedding (so pixels are fresh, SynthID is not
|
||||
re-introduced) but pulls InsightFace antelopev2/buffalo_l model packs at runtime,
|
||||
which are research-only. Needs the `photomaker` extra; **a paid service MUST NOT
|
||||
use this flag.** PhotoMaker-V1 was attempted as a commercial-safe alternative but
|
||||
blocked by a CFG batch-dim mismatch in the upstream pipeline (forked from diffusers
|
||||
0.29; we ship 0.38) — see `docs/synthid-robust-identity-research.md`.
|
||||
- **No local SynthID detector exists** → the service can't self-verify; bake in strength
|
||||
margin and periodic oracle spot-checks.
|
||||
- **Lesson:** visual-quality / face-identity recovery does NOT prove removal — only the
|
||||
|
||||
@@ -30,23 +30,39 @@ is the correct commercial-safe target: its `PhotoMakerIDEncoder` (model.py)
|
||||
forward takes only `(id_pixel_values, prompt_embeds, class_tokens_mask)` -- no
|
||||
ArcFace branch -- so identity is CLIP-only.
|
||||
|
||||
**Status notice (2026-06-04, end of session).** Even on V1, the cert sweep hit a
|
||||
cascade of upstream compatibility issues with the diffusers version we ship
|
||||
(0.38): missing `einops` declaration, missing `peft` declaration, default
|
||||
`pm_version='v2'` that mis-loads V1 weights into the V2 encoder, custom
|
||||
`id_encoder` left on CPU after `pipe.to(device)`, and a CFG-batch tensor-shape
|
||||
mismatch in the denoising loop (`Expected size 2 but got size 1`). 7 cascading
|
||||
fixes did not get the pipeline running end-to-end. The PhotoMaker `pipeline.py`
|
||||
header notes it was forked from diffusers v0.29.1; SDXL prompt-encoder handling
|
||||
changed significantly between 0.29 and 0.38, so making this work end-to-end is a
|
||||
proper fork or a diffusers downgrade -- both expensive. **The shipped path is
|
||||
GFPGAN on the diffusion-CLEANED image** (`face_restore.py`, the `restore`
|
||||
extra): a one-line change from the original GFPGAN-on-watermarked design that
|
||||
made the pass SynthID-safe by construction. Identity fidelity is lower than what
|
||||
a working identity-as-embedding stack would deliver, but the pipeline runs, the
|
||||
oracle is satisfied, and the dependency footprint is small. PhotoMaker remains
|
||||
the right north-star for a future identity-fidelity upgrade once the upstream
|
||||
compat work is done (or once a `diffusers ~0.29` forked pipeline is vendored).
|
||||
**Status notice (2026-06-04, end of session).** Two commercial-safe paths were
|
||||
tried and abandoned:
|
||||
|
||||
1. **PhotoMaker-V1** (commercial-safe by license but blocked by upstream compat).
|
||||
The cert sweep hit a cascade of upstream compatibility issues with the diffusers
|
||||
version we ship (0.38): missing `einops` declaration, missing `peft` declaration,
|
||||
default `pm_version='v2'` that mis-loads V1 weights into the V2 encoder, custom
|
||||
`id_encoder` left on CPU after `pipe.to(device)`, and a CFG-batch tensor-shape
|
||||
mismatch in the denoising loop (`Expected size 2 but got size 1`). 7 cascading
|
||||
fixes did not get the pipeline running end-to-end. The PhotoMaker `pipeline.py`
|
||||
header notes it was forked from diffusers v0.29.1; SDXL prompt-encoder handling
|
||||
changed significantly between 0.29 and 0.38.
|
||||
2. **GFPGAN on the diffusion-CLEANED image** (commercial-safe, but no identity
|
||||
recovery). A one-line change made it SynthID-safe (input pixels are already
|
||||
clean, so the partial blend cannot transport the watermark), but visual
|
||||
inspection of the cert output showed it only polished the already-drifted face
|
||||
without actually restoring identity. Trade-off was real and the value too low
|
||||
to keep.
|
||||
|
||||
**The shipped path is PhotoMaker-V2** (`photomaker_restore.py`, the `photomaker`
|
||||
extra). V2 uses a DUAL ID encoder (CLIP image features + ArcFace embedding),
|
||||
which delivers true identity-from-embedding face regeneration. The cost is that
|
||||
the ArcFace embedding comes from InsightFace's `antelopev2`/`buffalo_l` model
|
||||
packs, which are released under a non-commercial / research-only license. **So
|
||||
the shipped restore path is NON-COMMERCIAL.** raiw.cc and any other monetized
|
||||
deployment must NOT install the `photomaker` extra. The CLI flag and module
|
||||
docstring both call this out at every entry point.
|
||||
|
||||
A future commercial-safe path would need either (a) the PhotoMaker upstream to
|
||||
land its diffusers 0.38 compat fix so V1 can run, or (b) an equally good
|
||||
ArcFace-class face-recognition model released under a permissive license that
|
||||
PhotoMaker-V2 can be retargeted to. Neither is on a near-term horizon as of
|
||||
this writing.
|
||||
|
||||
## 1. Why identity-by-embedding (not by pixel) is the only SynthID-robust path
|
||||
|
||||
|
||||
+1
-1
@@ -570,7 +570,7 @@ table.
|
||||
schedule to `resolve_strength`, do not reuse the default ladder; (2) the
|
||||
`--restore-faces` pass is now SynthID-safe by construction (the GFPGAN-on-original
|
||||
path that re-added SynthID was removed 2026-06-04; the shipped restore is
|
||||
GFPGAN-on-cleaned, see `face_restore.py`); (3)
|
||||
PhotoMaker-V2, NON-COMMERCIAL, see `photomaker_restore.py`); (3)
|
||||
removal near threshold is seed-non-deterministic -> FIX the prod seed (kills the
|
||||
coin-flip; ship a deterministic certified config).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user