refactor(face-restore): drop GFPGAN, ship PhotoMaker-V2 as the sole restore (non-commercial)

Visual review of the GFPGAN-on-cleaned output (9-face grid, 1448x1086) showed it
only polished the already-drifted face without restoring identity — useless for the
"restore who is in the photo" intent. Dropping it.

The shipped restore path is now PhotoMaker-V2, which delivers true identity-from-
embedding face regeneration via a CLIP+ArcFace dual encoder. The ArcFace branch
pulls InsightFace antelopev2/buffalo_l model packs at runtime, which InsightFace
releases under a research-only license, so the whole extra is **NON-COMMERCIAL**.
raiw.cc and any monetized deployment must NOT install the `photomaker` extra.
This is called out at every entry point: CLI flag help, module docstring,
pyproject extra block, CLAUDE.md extras bullet, README install snippet.

Changes:
- Deleted `src/remove_ai_watermarks/face_restore.py` and its tests.
- Deleted the `restore` extra (gfpgan/facexlib/basicsr + scipy<1.18 / numba<0.60
  pins) and the basicsr setuptools<69 build pin from pyproject.toml.
- Restored `src/remove_ai_watermarks/photomaker_restore.py` (V2 this time:
  `TencentARC/PhotoMaker-V2`, `photomaker-v2.bin`, no `pm_version='v1'` override).
- Restored the `photomaker` extra in pyproject with all the upstream-compat
  pins (einops, peft, onnxruntime, insightface) and the `allow-direct-references`
  hatch metadata block.
- `InvisibleEngine` swapped `_restore_faces` -> `_restore_faces_photomaker`;
  `--restore-faces-method` removed (only one method, no choice).
- CLI flag help, CLAUDE.md, README, docs/synthid.md, and
  docs/controlnet-removal-pipeline-research.md all updated.
- docs/synthid-robust-identity-research.md status notice rewritten to list both
  abandoned commercial-safe attempts (V1 + GFPGAN-on-cleaned) and the
  non-commercial trade-off we accepted.

ruff + strict pyright(src/) clean; 578 tests pass (the 9 GFPGAN tests are gone,
the 11 PhotoMaker tests stay green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Victor Kuznetsov
2026-06-08 18:41:01 -07:00
parent 01fe98bf54
commit 65de8df5c5
13 changed files with 704 additions and 1263 deletions
+8 -7
View File
@@ -124,13 +124,14 @@ Gemini app; the two payloads are vendor-specific and never cross-checked):
- **Fix the seed in prod.** The non-determinism is purely `seed=None` (random); a fixed
`--seed` makes every run reproduce the certified-clean result, so you ship a
deterministic, re-certifiable config (and the seed sweep collapses to one config).
- **`--restore-faces` is SynthID-safe by construction now (GFPGAN-on-cleaned, 2026-06-04).**
The GFPGAN-on-original path that re-added SynthID was fixed by running GFPGAN on the
diffusion-CLEANED image instead — the input pixels GFPGAN derives from are already
SynthID-free, so the partial pixel-blend cannot transport the watermark. Needs the
`restore` extra. (The PhotoMaker-V1 identity-as-embedding alternative was researched
but blocked by upstream / diffusers-version compatibility issues; see
`docs/synthid-robust-identity-research.md`.)
- **`--restore-faces` is PhotoMaker-V2 (NON-COMMERCIAL).** The GFPGAN-on-cleaned path
was tried and rejected: it polished but did not restore identity. PhotoMaker-V2
regenerates faces from a CLIP+ArcFace embedding (so pixels are fresh, SynthID is not
re-introduced) but pulls InsightFace antelopev2/buffalo_l model packs at runtime,
which are research-only. Needs the `photomaker` extra; **a paid service MUST NOT
use this flag.** PhotoMaker-V1 was attempted as a commercial-safe alternative but
blocked by a CFG batch-dim mismatch in the upstream pipeline (forked from diffusers
0.29; we ship 0.38) — see `docs/synthid-robust-identity-research.md`.
- **No local SynthID detector exists** → the service can't self-verify; bake in strength
margin and periodic oracle spot-checks.
- **Lesson:** visual-quality / face-identity recovery does NOT prove removal — only the
+33 -17
View File
@@ -30,23 +30,39 @@ is the correct commercial-safe target: its `PhotoMakerIDEncoder` (model.py)
forward takes only `(id_pixel_values, prompt_embeds, class_tokens_mask)` -- no
ArcFace branch -- so identity is CLIP-only.
**Status notice (2026-06-04, end of session).** Even on V1, the cert sweep hit a
cascade of upstream compatibility issues with the diffusers version we ship
(0.38): missing `einops` declaration, missing `peft` declaration, default
`pm_version='v2'` that mis-loads V1 weights into the V2 encoder, custom
`id_encoder` left on CPU after `pipe.to(device)`, and a CFG-batch tensor-shape
mismatch in the denoising loop (`Expected size 2 but got size 1`). 7 cascading
fixes did not get the pipeline running end-to-end. The PhotoMaker `pipeline.py`
header notes it was forked from diffusers v0.29.1; SDXL prompt-encoder handling
changed significantly between 0.29 and 0.38, so making this work end-to-end is a
proper fork or a diffusers downgrade -- both expensive. **The shipped path is
GFPGAN on the diffusion-CLEANED image** (`face_restore.py`, the `restore`
extra): a one-line change from the original GFPGAN-on-watermarked design that
made the pass SynthID-safe by construction. Identity fidelity is lower than what
a working identity-as-embedding stack would deliver, but the pipeline runs, the
oracle is satisfied, and the dependency footprint is small. PhotoMaker remains
the right north-star for a future identity-fidelity upgrade once the upstream
compat work is done (or once a `diffusers ~0.29` forked pipeline is vendored).
**Status notice (2026-06-04, end of session).** Two commercial-safe paths were
tried and abandoned:
1. **PhotoMaker-V1** (commercial-safe by license but blocked by upstream compat).
The cert sweep hit a cascade of upstream compatibility issues with the diffusers
version we ship (0.38): missing `einops` declaration, missing `peft` declaration,
default `pm_version='v2'` that mis-loads V1 weights into the V2 encoder, custom
`id_encoder` left on CPU after `pipe.to(device)`, and a CFG-batch tensor-shape
mismatch in the denoising loop (`Expected size 2 but got size 1`). 7 cascading
fixes did not get the pipeline running end-to-end. The PhotoMaker `pipeline.py`
header notes it was forked from diffusers v0.29.1; SDXL prompt-encoder handling
changed significantly between 0.29 and 0.38.
2. **GFPGAN on the diffusion-CLEANED image** (commercial-safe, but no identity
recovery). A one-line change made it SynthID-safe (input pixels are already
clean, so the partial blend cannot transport the watermark), but visual
inspection of the cert output showed it only polished the already-drifted face
without actually restoring identity. Trade-off was real and the value too low
to keep.
**The shipped path is PhotoMaker-V2** (`photomaker_restore.py`, the `photomaker`
extra). V2 uses a DUAL ID encoder (CLIP image features + ArcFace embedding),
which delivers true identity-from-embedding face regeneration. The cost is that
the ArcFace embedding comes from InsightFace's `antelopev2`/`buffalo_l` model
packs, which are released under a non-commercial / research-only license. **So
the shipped restore path is NON-COMMERCIAL.** raiw.cc and any other monetized
deployment must NOT install the `photomaker` extra. The CLI flag and module
docstring both call this out at every entry point.
A future commercial-safe path would need either (a) the PhotoMaker upstream to
land its diffusers 0.38 compat fix so V1 can run, or (b) an equally good
ArcFace-class face-recognition model released under a permissive license that
PhotoMaker-V2 can be retargeted to. Neither is on a near-term horizon as of
this writing.
## 1. Why identity-by-embedding (not by pixel) is the only SynthID-robust path
+1 -1
View File
@@ -570,7 +570,7 @@ table.
schedule to `resolve_strength`, do not reuse the default ladder; (2) the
`--restore-faces` pass is now SynthID-safe by construction (the GFPGAN-on-original
path that re-added SynthID was removed 2026-06-04; the shipped restore is
GFPGAN-on-cleaned, see `face_restore.py`); (3)
PhotoMaker-V2, NON-COMMERCIAL, see `photomaker_restore.py`); (3)
removal near threshold is seed-non-deterministic -> FIX the prod seed (kills the
coin-flip; ship a deterministic certified config).