mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-07-12 11:06:33 +02:00
docs(restore): document that restore methods REGENERATE, not preserve
Empirical conclusion from the 2026-06-04 - 2026-06-08 cert sweeps: every shipped face-restore method (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned at three parameter settings) regenerates the face from an ArcFace embedding via SDXL diffusion. Output face pixels are diffusion-fresh, which makes the regenerated face look MORE AI-generated than the cleaned image (gloss, symmetric pores, SDXL "clean skin" aesthetic) regardless of license. The cleaned image from the main controlnet 0.20 removal pass is the LEAST-AI state we can reach without re-introducing SynthID; any restore on top trades original-look for embedding-driven regeneration. The fundamental issue is structural: ArcFace encodes "general look" at 512 dimensions, SDXL decodes that into pixels with the inherent SDXL aesthetic. Stronger identity push (higher strength + IP-Adapter scale) makes the face closer to the embedding but more AI-looking; weaker push leaves identity to drift further. No parameter setting recovers original identity AND looks less AI than cleaned. Production conclusion: do not ship `--restore-faces` in any monetized deployment. The extras (`instantid`, `photomaker`) stay in the library for research / personal use where users explicitly want regeneration. Documented at every entry point: - CLAUDE.md: new "Face restore trade-off" bullet + every restore mention rewritten to "REGENERATES, does NOT recover"; controlnet bullet updated - README.md: feature bullet + callout + secondary mention all updated - docs/synthid-robust-identity-research-2026-06-08.md: appended "Empirical follow-up" section documenting the InstantID sweep phases (Phase 1 txt2img v1/v2/v3, Phase 2 img2img defaults + stronger params) - docs/controlnet-removal-pipeline-research.md: updated restore-faces bullet to reflect the empirical conclusion - CLI help: `_restore_faces_options` docstring + `--restore-faces` / `--restore-faces-method` help text all updated Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -124,14 +124,19 @@ Gemini app; the two payloads are vendor-specific and never cross-checked):
|
||||
- **Fix the seed in prod.** The non-determinism is purely `seed=None` (random); a fixed
|
||||
`--seed` makes every run reproduce the certified-clean result, so you ship a
|
||||
deterministic, re-certifiable config (and the seed sweep collapses to one config).
|
||||
- **`--restore-faces` is PhotoMaker-V2 (NON-COMMERCIAL).** The GFPGAN-on-cleaned path
|
||||
was tried and rejected: it polished but did not restore identity. PhotoMaker-V2
|
||||
regenerates faces from a CLIP+ArcFace embedding (so pixels are fresh, SynthID is not
|
||||
re-introduced) but pulls InsightFace antelopev2/buffalo_l model packs at runtime,
|
||||
which are research-only. Needs the `photomaker` extra; **a paid service MUST NOT
|
||||
use this flag.** PhotoMaker-V1 was attempted as a commercial-safe alternative but
|
||||
blocked by a CFG batch-dim mismatch in the upstream pipeline (forked from diffusers
|
||||
0.29; we ship 0.38) — see `docs/synthid-robust-identity-research.md`.
|
||||
- **`--restore-faces` is OFF in prod and stays opt-in.** Two methods ship
|
||||
(`instantid` default, `photomaker`), both NON-COMMERCIAL. They REGENERATE the face
|
||||
from an ArcFace embedding via SDXL diffusion, making the output face look more
|
||||
AI-generated than the cleaned image (gloss, symmetric pores, SDXL "clean skin"
|
||||
aesthetic). For production face preservation the cleaned image from controlnet 0.20
|
||||
is the LEAST-AI state we can reach — any restore on top trades original-look for
|
||||
embedding-driven regeneration. Empirical sweep summary: GFPGAN-on-cleaned polished
|
||||
without identity recovery; PhotoMaker-V2 produced a different person; InstantID
|
||||
txt2img produced studio-portrait patchwork on group photos; InstantID
|
||||
img2img-on-cleaned with three parameter settings integrated scene context cleanly
|
||||
but never recovered original identity precisely — every setting traded one problem
|
||||
for another. See `docs/synthid-robust-identity-research-2026-06-08.md`
|
||||
"Empirical follow-up" for the full sweep.
|
||||
- **No local SynthID detector exists** → the service can't self-verify; bake in strength
|
||||
margin and periodic oracle spot-checks.
|
||||
- **Lesson:** visual-quality / face-identity recovery does NOT prove removal — only the
|
||||
|
||||
@@ -126,4 +126,60 @@ Six claims were refuted in adversarial verification, two of them load-bearing: A
|
||||
- [source](https://github.com/IrvingMeng/MagFace/blob/main/LICENSE)
|
||||
- [source](https://github.com/askerlee/AdaFace-dev)
|
||||
- [source](https://openreview.net/forum?id=Hc2ZwCYgmB)
|
||||
- [source](https://github.com/tencent-ailab/IP-Adapter/wiki/IP%E2%80%90Adapter%E2%80%90Face)
|
||||
- [source](https://github.com/tencent-ailab/IP-Adapter/wiki/IP%E2%80%90Adapter%E2%80%90Face)
|
||||
|
||||
## Empirical follow-up (2026-06-08, end of session)
|
||||
|
||||
After the research synthesis above, InstantID was integrated end-to-end and cert-swept
|
||||
on Modal A100 in two phases:
|
||||
|
||||
1. **Phase 1: InstantID txt2img per-face crop + composite.** Per-face InstantID
|
||||
txt2img with the upstream `pipeline_stable_diffusion_xl_instantid`, ArcFace
|
||||
embedding from the original face, landmark stick figure. Three composite
|
||||
iterations:
|
||||
- v1 (rectangular Gaussian alpha on the 2x square_box around each face):
|
||||
visible patchwork on group photos, generated 1024 backgrounds clashing.
|
||||
- v2 (tight crop on YuNet-detected face in the generated 1024 + elliptical
|
||||
alpha 0.45*bw x 0.55*bh + soft feather): ellipse axis exceeded bbox
|
||||
vertically, clipped forehead/chin on single portrait, group still had
|
||||
visible elliptical seams + cool-vs-warm tone clash with scene.
|
||||
- v3 (tighter ellipse 0.32*bw x 0.42*bh + per-channel mean color match to
|
||||
local cleaned canvas + softer feather): patchwork visually softened; faces
|
||||
still read as studio portraits inserted into the scene, not as people
|
||||
shot in the scene. Single portrait identity drifted (tatsunari -> "round
|
||||
Asian male" vs original's thin face).
|
||||
2. **Phase 2: InstantID img2img on cleaned crop.** Switched to the upstream
|
||||
`pipeline_stable_diffusion_xl_instantid_img2img` (downloaded at first use
|
||||
from raw.githubusercontent.com; requires `trust_remote_code=True`). Same
|
||||
ArcFace + landmark conditioning but the SDXL diffusion source is the
|
||||
CLEANED face crop, so the diffusion sees scene lighting / shoulders /
|
||||
shadow direction directly. Multi-face composition jumped substantially:
|
||||
faces sit in the bar scene with matching warm tone, no more elliptical
|
||||
seams. Single-portrait identity at the default (`strength=0.55`,
|
||||
`ip_adapter_scale=0.8`, `controlnet_conditioning_scale=0.8`) was "similar
|
||||
person, not exactly the original"; raising to `strength=0.7`,
|
||||
`ip_adapter_scale=1.0`, `controlnet_scale=1.0` brought identity closer to
|
||||
original but introduced more "SDXL gloss / clean skin" aesthetic.
|
||||
|
||||
**Net finding for raiw.cc (load-bearing).** The fundamental issue is structural:
|
||||
ArcFace encodes "this person's general look" (ethnicity, gender, basic facial
|
||||
geometry) at 512 dimensions; SDXL decodes that embedding into pixels with the
|
||||
inherent SDXL aesthetic (smooth skin, symmetric pores, AI-photoreal look).
|
||||
Stronger identity push (higher strength / IP-Adapter scale) makes the face
|
||||
CLOSER to the embedded identity but MORE AI-looking; weaker push leaves
|
||||
identity to drift but face looks less AI-generated. There is no parameter
|
||||
setting that simultaneously recovers original identity AND looks less AI than
|
||||
the cleaned image, because the cleaned image is itself a controlnet-light
|
||||
denoise of the original (closer to original pixels) while a restore pass is a
|
||||
full SDXL regeneration (further from original pixels).
|
||||
|
||||
**Operational conclusion.** Do not ship `--restore-faces` in any monetized
|
||||
deployment. The cleaned image from the main controlnet 0.20 pass is the
|
||||
LEAST-AI state we can reach without re-introducing SynthID; every restore
|
||||
method tested (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img,
|
||||
InstantID img2img-on-cleaned at three parameter sweeps) trades original-look
|
||||
for embedding-driven regeneration and makes the face read as "AI-generated"
|
||||
rather than "the original person". The `instantid` and `photomaker` extras
|
||||
stay in the library as opt-in for research / personal use where users
|
||||
explicitly want identity regeneration; the CLI flag and module docstrings
|
||||
state the trade-off at every entry point.
|
||||
Reference in New Issue
Block a user