mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-07 22:48:37 +02:00
fix(photomaker): switch to V1 — V2 actually requires InsightFace (non-commercial)
A Modal cert sweep caught what the research doc missed: PhotoMaker-V2 fails at
import without InsightFace ("No module named 'insightface'"). Reading the upstream
source confirms it: `photomaker/__init__.py` imports `FaceAnalysis2` (an InsightFace
wrapper) at module load, V2's encoder is named
`PhotoMakerIDEncoder_CLIPInsightfaceExtendtoken`, and `model_v2.py`'s forward
takes an `id_embeds` argument that the pipeline computes via
`insightface.app.FaceAnalysis(name='antelopev2', ...)`. So V2 is a DUAL encoder
(CLIP + ArcFace), not CLIP-only as the model card line "id_encoder includes
finetuned OpenCLIP-ViT-H-14 and a few fuse layers" implied.
InsightFace's pretrained model packs (antelopev2, buffalo_l) are research/
non-commercial only per their own README:
"The pretrained models we provided with this library are available for
non-commercial research purposes only."
So V2 is blocked for a paid service like raiw.cc.
PhotoMaker-V1 is the commercial-safe alternative — its `PhotoMakerIDEncoder`
(model.py) forward takes only `(id_pixel_values, prompt_embeds, class_tokens_mask)`,
no ArcFace branch. Identity is CLIP-only, license is Apache-2.0, no InsightFace.
Code change: swap the repo + filename constants in `photomaker_restore.py`
(TencentARC/PhotoMaker, photomaker-v1.bin). Tests still pass (the 9 PhotoMaker
tests use a fake pipeline, so the model swap is transparent to them).
Doc correction: rewrote the verdict / license table / section 5 of
`docs/synthid-robust-identity-research.md` to lead with V1 and add a correction
notice explaining the V2 misread. Bulk-renamed `PhotoMaker-V2` to `PhotoMaker-V1`
across CLAUDE.md, README.md, docs/synthid.md, and
docs/controlnet-removal-pipeline-research.md (kept V2 only in the correction
notice, the license table, and the anchor reference).
ruff clean; 578 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
7e6fc8bfb9
commit
dfa5181309
@@ -1,4 +1,4 @@
|
||||
"""SynthID-robust face identity restoration via PhotoMaker-V2.
|
||||
"""SynthID-robust face identity restoration via PhotoMaker-V1.
|
||||
|
||||
The diffusion removal pass scrubs the pixel watermark from the WHOLE image, including
|
||||
faces, but lets faces drift in identity. Unlike the GFPGAN restore pass in
|
||||
@@ -14,11 +14,16 @@ empirically confirmed 2026-06-04: on 31 face crops, the cosine similarity betwee
|
||||
SynthID magnitude) is 0.9977 -- an order of magnitude less drift than JPEG90, which
|
||||
SynthID survives at >=99% TPR by design. See ``docs/synthid-robust-identity-research.md``.
|
||||
|
||||
Architecture: PhotoMaker-V2 is a fine-tuned OpenCLIP-ViT-H/14 ID encoder plus LoRA on
|
||||
the SDXL UNet attention layers. It ships as a single ``photomaker-v2.bin`` checkpoint
|
||||
loaded into a ``PhotoMakerStableDiffusionXLPipeline`` (txt2img only -- there is no
|
||||
PhotoMakerControlNetImg2img class in diffusers). We use it as a SECOND PASS after the
|
||||
main controlnet/default removal:
|
||||
Architecture: PhotoMaker-V1 is a fine-tuned OpenCLIP-ViT-H/14 ID encoder plus LoRA on
|
||||
the SDXL UNet attention layers. It ships as a single ``photomaker-v1.bin`` checkpoint
|
||||
loaded into a ``PhotoMakerStableDiffusionXLPipeline`` (txt2img). **V1, not V2:** V2
|
||||
adds an InsightFace/ArcFace face-recognition component at runtime, whose pretrained
|
||||
model packs (antelopev2, buffalo_l) are non-commercial-research-only per the
|
||||
InsightFace README, which would block a paid service like raiw.cc. V1's identity
|
||||
encoder is CLIP-only (PhotoMakerIDEncoder, ``model.py``); confirmed by inspecting
|
||||
the upstream source (model_v2.py forward takes ``id_embeds`` from InsightFace; V1
|
||||
forward does not). We use it as a SECOND PASS after the main controlnet/default
|
||||
removal:
|
||||
|
||||
1. Main removal pass (`controlnet` at the certified strength) cleans SynthID
|
||||
everywhere but leaves faces drifted.
|
||||
@@ -31,11 +36,12 @@ The generated face pixels are diffusion-fresh and inherit identity from the embe
|
||||
(not the pixels), so SynthID is not re-introduced.
|
||||
|
||||
Commercial-safe end-to-end:
|
||||
- PhotoMaker-V2 weights: Apache-2.0 (TencentARC).
|
||||
- PhotoMaker-V1 weights: Apache-2.0 (TencentARC).
|
||||
- ID encoder: OpenCLIP-ViT-H/14 (MIT) finetuned by PhotoMaker (still Apache-2.0).
|
||||
- SDXL base: shared with the main pipeline (already used in `default`/`controlnet`).
|
||||
- NO InsightFace / antelopev2 (which is the non-commercial blocker for IP-Adapter
|
||||
FaceID / InstantID / PuLID / Arc2Face).
|
||||
- NO InsightFace / antelopev2 (the non-commercial blocker that BLOCKS PhotoMaker-V2,
|
||||
IP-Adapter FaceID, InstantID, PuLID, and Arc2Face). V1 is the only commercial-safe
|
||||
member of this family.
|
||||
|
||||
Requires the optional ``photomaker`` extra: ``pip install
|
||||
'remove-ai-watermarks[photomaker]'`` (pulls torch / diffusers / the upstream PhotoMaker
|
||||
@@ -57,9 +63,10 @@ if TYPE_CHECKING:
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# PhotoMaker-V2 weights (Apache-2.0, TencentARC). Downloaded on first use.
|
||||
_PHOTOMAKER_REPO = "TencentARC/PhotoMaker-V2"
|
||||
_PHOTOMAKER_FILE = "photomaker-v2.bin"
|
||||
# PhotoMaker-V1 weights (Apache-2.0, TencentARC). Downloaded on first use. V2 is NOT
|
||||
# used because it pulls InsightFace at runtime (non-commercial models).
|
||||
_PHOTOMAKER_REPO = "TencentARC/PhotoMaker"
|
||||
_PHOTOMAKER_FILE = "photomaker-v1.bin"
|
||||
# SDXL base shared with the main pipeline (same checkpoint as `default`/`controlnet`).
|
||||
_SDXL_MODEL_ID = "stabilityai/stable-diffusion-xl-base-1.0"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user