Commit Graph
11 Commits
Author SHA1 Message Date
Victor KuznetsovandClaude Opus 5 13095fb45c Verify every doc claim against the source and fix what drifted
Every code-referencing claim in the docs, the README and the rules files was
checked against src/, and each finding was re-derived independently before it
was applied. 35 held, 5 were false positives.

Two of them were code, not text. `InvisibleOptions` promises in its docstring to
mirror `InvisibleEngine`, and two defaults had silently stopped:
`max_resolution=None` reached `_target_size`'s `max_resolution > 0` and raised
`TypeError` on every library call that left the options alone, and
`cpu_offload=True` made a library run slower than the identical CLI run. Both are
fixed, and `TestInvisibleOptionsMirrorTheEngine` compares the two signatures
field by field rather than pinning the two values that happen to be known. A
companion assertion in `TestTargetSize` reads the engine's own declared default,
so a drift on the engine side -- which the mirror check alone would accept,
because both sides would still agree -- fails too.

The user-facing docs: README called `invisible` GPU-optional where it raises
without CUDA, and gave the image `metadata` command `video metadata`'s output
rule, promising the source survives a command that overwrites it. Yuanbao was
missing from the supported-mark list. `veo` was listed among the video policies
that require a run anchor, though its row sets no `anchor_iou`.
`known-limitations` called ControlNet the default profile and contradicted
itself ninety lines below. An unescaped pipe truncated the `hailuo` table row.
The `dev` extra, the CI shape, ffmpeg's role, the sdist boundary and the
strength-curve range were corrected, and `remove_all`/`remove_batch`, the pill
gate, `erase --keep-metadata` and `all`'s CUDA failure mode were documented.

Research notes that described removed modules, extras and flags in the present
tense now say so once in the page banner instead of sentence by sentence, which
covers the whole page rather than the lines that happened to be noticed, and one
fixture is referred to by role rather than by name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 11:38:04 -07:00
Victor KuznetsovandClaude Opus 5 b0ca2054f6 Keep only the two-stage profiles and make CUDA a precondition
qwen-zimage becomes the default and sdxl-zimage the only alternative. The
controlnet, sdxl, qwen and default profiles are gone, and with them the CPU and
MPS paths for invisible-watermark removal: neither matched the two-stage
recipe's face preservation, so keeping them advertised a quality this library no
longer delivers. Visible-mark removal and every identify command still run
anywhere.

Retired names are rejected rather than remapped. Silently routing --pipeline
sdxl onward would run an old script at a different strength, on a different
model, at a different quality, and report success.

CUDA is now checked when the remover is constructed instead of when the model
loads. Auto-detection cheerfully returned mps on a Mac, so the failure arrived
several layers down, after the dependency check and the pipeline import, in a
message naming whichever internal pipeline happened to raise. _DEVICES collapses
to {"cuda"} and the cpu/mps float32 branch goes with it.

resolve_strength stays total. It briefly returned None for qwen-zimage, meaning
"ask the resolution curve", which pushed a branch onto both callers and left one
of the two strength policies outside the strength module; the CLI copy had
already grown an `or 0.0` guarding a path its own comment called unreachable. It
now takes the image size and answers for both profiles, so the displayed value
cannot drift from the executed one.

Deletion fallout removed with it: img2img_runner and progress.py (the MPS
recovery path and its progress monitor had no callers left), viable_steps, the
fp16 degenerate-output retry, the fp16 VAE fix, and the Qwen img2img call
builders. try_empty_device_cache moved into watermark_remover rather than
leaving a module whose docstring outlived its code. _HAS_DIFFUSERS routes
through optional_deps.module_available, which is what the rest of the library
uses and what correctly rejects a pruned namespace remnant.

--steps, --guidance-scale and --model now have exactly one legal value each and
are still accepted at parse time, then rejected in remove(). Their help text
says so, but validating them beside the option would be better.

Not addressed, and worth its own decision: invisible_engine forces
min_resolution to 0 for both profiles, so the --min-resolution floor, --upscaler,
_esrgan_upscale, upscaler.py and the esrgan extra are all unreachable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 12:16:14 -07:00
Victor Kuznetsov 08dc078d91 Release 0.22.0 with composable feature extras 2026-07-31 10:39:13 -07:00
Victor Kuznetsov 03cd00f132 Restructure documentation, validate metadata removal, consolidate assets 2026-07-25 21:08:04 -07:00
Victor Kuznetsov b6204a24ce Add high-fidelity Qwen Z-Image removal pipeline 2026-07-25 16:53:22 -07:00
Victor KuznetsovandClaude Opus 4.8 20d7eda96a remove: drop all face-restore code (regeneration, not preservation)
Empirical conclusion from the 2026-06-04 - 2026-06-08 Modal cert sweeps:
every face-restore approach we built (GFPGAN-on-cleaned, PhotoMaker-V2,
InstantID txt2img, InstantID img2img-on-cleaned at three parameter
settings) regenerates the face via SDXL diffusion rather than preserves
it. Output face pixels are diffusion-fresh, so the regenerated face
inherits SDXL "clean skin" aesthetic and loses original identity
precision -- it looks MORE AI-generated than the cleaned image, not
less. The cleaned image from the main controlnet 0.20 removal pass is
the least-AI face state we can reach without re-introducing SynthID.

Nothing in the restore family achieves the actual goal (preserve the
original person's face). Keeping them around as opt-in invites users to
ship something that defeats the point. Removing entirely.

Library changes:
- Deleted src/remove_ai_watermarks/instantid_restore.py
- Deleted src/remove_ai_watermarks/photomaker_restore.py
- Deleted tests/test_instantid_restore.py
- Deleted tests/test_photomaker_restore.py
- Removed `instantid` and `photomaker` extras from pyproject.toml
- Removed `[tool.hatch.metadata] allow-direct-references = true` (was
  only needed for the photomaker git+ URL)
- InvisibleEngine.remove_watermark: dropped `restore_faces` +
  `restore_faces_method` params, removed both `_restore_faces_instantid`
  and `_restore_faces_photomaker` private methods, removed dispatch
- CLI: dropped `_restore_faces_options` decorator, all four cmd_*
  signatures lose `restore_faces` + `restore_faces_method`, kwarg passes
  to remove_watermark dropped
- _apply_auto: dropped `restore_faces` from tuple shape (was unused after
  the engine no longer takes it)
- auto_config.AutoConfig: dropped `restore_faces` field; `plan()` no
  longer sets it; `reason` no longer mentions it
- Tests updated accordingly (test_auto_config.TestReason no longer asserts
  "face-restore on" in the reason string)

Docs updated:
- CLAUDE.md: removed the photomaker extras bullet, the Face restore
  trade-off bullet, the instantid_restore.py + photomaker_restore.py
  module bullets; replaced restore mentions in watermark_remover and
  controlnet bullets and prod recipe with the empirical conclusion
- README.md: removed both `--restore-faces` callouts and the install
  snippet; the feature bullet and auto-mode comment updated
- docs/synthid-robust-identity-research.md: added Status-retired notice
  at the top pointing at the 2026-06-08 followup

raiw-app:
- modal_cert.py: dropped `--restore-faces` flag entirely; sweep() no
  longer takes restore_faces; pinned _LIB_SPEC to `[gpu]` extras (no
  `photomaker` / `instantid` extras), points at main

ruff + strict pyright clean; 569 tests pass; 18 restore-specific tests
gone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 21:21:58 -07:00
Victor KuznetsovandClaude Opus 4.8 65de8df5c5 refactor(face-restore): drop GFPGAN, ship PhotoMaker-V2 as the sole restore (non-commercial)
Visual review of the GFPGAN-on-cleaned output (9-face grid, 1448x1086) showed it
only polished the already-drifted face without restoring identity — useless for the
"restore who is in the photo" intent. Dropping it.

The shipped restore path is now PhotoMaker-V2, which delivers true identity-from-
embedding face regeneration via a CLIP+ArcFace dual encoder. The ArcFace branch
pulls InsightFace antelopev2/buffalo_l model packs at runtime, which InsightFace
releases under a research-only license, so the whole extra is **NON-COMMERCIAL**.
raiw.cc and any monetized deployment must NOT install the `photomaker` extra.
This is called out at every entry point: CLI flag help, module docstring,
pyproject extra block, CLAUDE.md extras bullet, README install snippet.

Changes:
- Deleted `src/remove_ai_watermarks/face_restore.py` and its tests.
- Deleted the `restore` extra (gfpgan/facexlib/basicsr + scipy<1.18 / numba<0.60
  pins) and the basicsr setuptools<69 build pin from pyproject.toml.
- Restored `src/remove_ai_watermarks/photomaker_restore.py` (V2 this time:
  `TencentARC/PhotoMaker-V2`, `photomaker-v2.bin`, no `pm_version='v1'` override).
- Restored the `photomaker` extra in pyproject with all the upstream-compat
  pins (einops, peft, onnxruntime, insightface) and the `allow-direct-references`
  hatch metadata block.
- `InvisibleEngine` swapped `_restore_faces` -> `_restore_faces_photomaker`;
  `--restore-faces-method` removed (only one method, no choice).
- CLI flag help, CLAUDE.md, README, docs/synthid.md, and
  docs/controlnet-removal-pipeline-research.md all updated.
- docs/synthid-robust-identity-research.md status notice rewritten to list both
  abandoned commercial-safe attempts (V1 + GFPGAN-on-cleaned) and the
  non-commercial trade-off we accepted.

ruff + strict pyright(src/) clean; 578 tests pass (the 9 GFPGAN tests are gone,
the 11 PhotoMaker tests stay green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 18:41:01 -07:00
Victor KuznetsovandClaude Opus 4.8 01fe98bf54 refactor(face-restore): rollback PhotoMaker, restore GFPGAN on the CLEANED image
After 7 cascading upstream-compat fixes (insightface dep, peft dep, pm_version,
device, etc.), the PhotoMaker V1 cert sweep still hit a CFG batch-dim mismatch
inside the denoising loop. The upstream PhotoMaker `pipeline.py` is forked from
diffusers v0.29.1 and our env runs 0.38; SDXL prompt-encoder handling changed
significantly between those versions, so making PhotoMaker work end-to-end
needs a proper fork or a diffusers downgrade — both expensive. Not worth
shipping today.

Pivot: restore `face_restore.py` (GFPGAN) with a single-line fix that makes it
SynthID-safe by construction. The previous design ran GFPGAN.enhance on the
ORIGINAL watermarked image and was oracle-confirmed to re-add SynthID via the
weight-0.5 pixel blend. The fix is to run GFPGAN on the diffusion-CLEANED
image — whatever pixels GFPGAN derives from are already SynthID-free, so the
partial blend cannot transport the watermark. Identity fidelity is lower than
a true identity-as-embedding stack would deliver, but it ships and works.

Changes:
- `src/remove_ai_watermarks/face_restore.py` restored from pre-wipe state with
  one line changed: `restorer.enhance(cleaned_bgr, ...)` instead of
  `restorer.enhance(original_bgr, ...)`. `original_bgr` is kept as an unused
  positional argument for API stability.
- `src/remove_ai_watermarks/photomaker_restore.py` and its tests REMOVED. The
  research note (`docs/synthid-robust-identity-research.md`) keeps a "status
  notice" documenting why PhotoMaker is parked for now and what the path back
  in would look like.
- `pyproject.toml` `restore` extra restored (gfpgan/facexlib/basicsr +
  scipy<1.18 + numba<0.60 pins + the basicsr setuptools<69 build pin), plus
  `photomaker` extra (with its einops/insightface/peft pile) and the
  `[tool.hatch.metadata] allow-direct-references = true` block REMOVED.
- `InvisibleEngine._restore_faces_photomaker` removed; `_restore_faces`
  restored. The `--restore-faces` CLI flag and its plumbing through cmd_*
  signatures are unchanged.
- CLAUDE.md, README.md, docs/synthid.md, docs/controlnet-removal-pipeline-
  research.md updated to describe the shipped GFPGAN-on-cleaned design and to
  reference PhotoMaker only as the parked alternative.

ruff + strict pyright(src/) clean; 578 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 16:55:45 -07:00
Victor KuznetsovandClaude Opus 4.8 dfa5181309 fix(photomaker): switch to V1 — V2 actually requires InsightFace (non-commercial)
A Modal cert sweep caught what the research doc missed: PhotoMaker-V2 fails at
import without InsightFace ("No module named 'insightface'"). Reading the upstream
source confirms it: `photomaker/__init__.py` imports `FaceAnalysis2` (an InsightFace
wrapper) at module load, V2's encoder is named
`PhotoMakerIDEncoder_CLIPInsightfaceExtendtoken`, and `model_v2.py`'s forward
takes an `id_embeds` argument that the pipeline computes via
`insightface.app.FaceAnalysis(name='antelopev2', ...)`. So V2 is a DUAL encoder
(CLIP + ArcFace), not CLIP-only as the model card line "id_encoder includes
finetuned OpenCLIP-ViT-H-14 and a few fuse layers" implied.

InsightFace's pretrained model packs (antelopev2, buffalo_l) are research/
non-commercial only per their own README:
  "The pretrained models we provided with this library are available for
   non-commercial research purposes only."
So V2 is blocked for a paid service like raiw.cc.

PhotoMaker-V1 is the commercial-safe alternative — its `PhotoMakerIDEncoder`
(model.py) forward takes only `(id_pixel_values, prompt_embeds, class_tokens_mask)`,
no ArcFace branch. Identity is CLIP-only, license is Apache-2.0, no InsightFace.

Code change: swap the repo + filename constants in `photomaker_restore.py`
(TencentARC/PhotoMaker, photomaker-v1.bin). Tests still pass (the 9 PhotoMaker
tests use a fake pipeline, so the model swap is transparent to them).

Doc correction: rewrote the verdict / license table / section 5 of
`docs/synthid-robust-identity-research.md` to lead with V1 and add a correction
notice explaining the V2 misread. Bulk-renamed `PhotoMaker-V2` to `PhotoMaker-V1`
across CLAUDE.md, README.md, docs/synthid.md, and
docs/controlnet-removal-pipeline-research.md (kept V2 only in the correction
notice, the license table, and the anchor reference).

ruff clean; 578 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 16:05:58 -07:00
Victor KuznetsovandClaude Opus 4.8 f8f247308b docs(identity): smoke test confirms OpenCLIP embedding is invariant to SynthID-magnitude noise
Empirical confirmation of the load-bearing assumption in the PhotoMaker-V2 path: the
identity embedding cannot transport an invisible pixel watermark.

Tested OpenCLIP-ViT-H/14 (laion2B-s32B-b79K — the same encoder PhotoMaker-V2
fine-tunes) on 31 face crops from gemini_3/gemini_4/openai_3 grid. cosine
similarity between embed(orig) and embed(perturbed):

- synthid_proxy (±2 LSB low-frequency noise, the regime SynthID actually lives in):
  mean 0.9977, min 0.9937. Embedding moves by 0.002 — an order of magnitude less
  than JPEG90 (mean 0.928), which SynthID survives at >=99% TPR by design.
- noise3 / jpeg70 / blur1: 0.89-0.95, all clearly above the SynthID floor.
- self check: 1.0000 (pipeline sane).

So the embedder discards exactly the dimensions SynthID hides in. PhotoMaker-V2
conditioned on a watermarked face will see the same identity vector as a clean
face of that person, so the generated face inherits identity, not the watermark.

This unblocks step 2 of the research plan: prototype PhotoMaker-V2 in the
controlnet pipeline. The previously logged ad-hoc "cos(orig, SDXL-cleaned)"
numbers (0.56-0.93) measured diffusion drift, not watermark invariance, and are
not relevant to the hypothesis.

Docs only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:05:15 -07:00
Victor KuznetsovandClaude Opus 4.8 310ce912ba docs: SynthID-robust identity research — PhotoMaker-V2 is the only commercial-safe SDXL stack
After GFPGAN restore was oracle-confirmed to RE-INTRODUCE SynthID (it is a fidelity-
restoration net conditioned on the watermarked input), the only identity path that
will not transport the watermark is identity-by-EMBEDDING: a semantic vector that
conditions a fresh generation. That requires a face-recognition / ArcFace-class or
CLIP-image embedder.

Verified the license stack of every credible 2025-2026 SDXL identity adapter by
fetching primary sources directly (HuggingFace model cards, insightface.ai):

- IP-Adapter FaceID family, InstantID, PuLID, Arc2Face -> all blocked. Each
  depends at runtime on InsightFace's antelopev2/buffalo_l ArcFace packs, and
  insightface.ai explicitly states "Code is MIT licensed; models require separate
  commercial licensing." IP-Adapter FaceID's own model card flags itself non-
  commercial for the same reason.
- PhotoMaker-V2 is the single commercial-safe end-to-end stack today: Apache-2.0
  adapter weights with identity encoded as a fine-tuned OpenCLIP-ViT-H/14 (the
  model card's exact phrase: "id_encoder includes finetuned OpenCLIP-ViT-H-14
  and a few fuse layers"). No InsightFace.

Mechanistic argument that an identity embedding cannot transport SynthID: the
embedder is trained to be invariant to low-amplitude pixel changes (JPEG, resize,
brightness, noise), which is exactly the regime SynthID hides in by design. So
the embedding extracted from a watermarked face should be ~identical to the
embedding from the cleaned face, and the embedding cannot carry the watermark
into a freshly generated face. Flagged explicitly as not-yet-measured -- the
first integration step is a cosine-similarity smoke test (no codegen) before
investing in a PhotoMaker prototype.

Process note: the deep-research harness was run but its verifier subagents failed
to call StructuredOutput (same harness bug as a prior session), so its synthesis
was unusable; the license claims here are direct quotes from the primary
sources, fetched and verified, not from the workflow synthesis.

Docs only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 14:58:11 -07:00