mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-06 22:18:36 +02:00
fix(invisible): disable diffusers SDXL watermarker on removal output
The SDXL removal pipelines (sdxl + controlnet) were built without add_watermarker=False, so diffusers embedded its default open "Stable Diffusion XL" DWT-DCT invisible watermark on every output whenever invisible-watermark is installed (the detect extra). A watermark REMOVER was therefore replacing one detectable AI watermark (SynthID) with another: the cleaned output re-read as AI (identify -> "Open invisible watermark: Stable Diffusion XL"), observed on the SynthID validation sample. Both SDXL loaders now call a shared _disable_sdxl_watermarker helper (mirrors _maybe_add_fp16_vae; the ControlNetModel sub-model and the Qwen loader never call it, since only the pipeline accepts the kwarg). Verified end to end: the affected outputs re-run clean (is_ai=None, no open watermark). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
7f22852593
commit
183793fbf2
@@ -53,7 +53,7 @@ module.
|
||||
|
||||
**Independence is source-grouped (`_CLASH_SOURCE`, added 2026-06-02):** the C2PA issuer attribution (`c2pa`) and the SynthID proxy (`synthid`) are NOT independent — the proxy is inferred from the *same* manifest — so they share one source and two vendors named within a single manifest do not clash. This killed a false-positive class found on the spaces corpus: legitimate multi-actor manifests where a product wraps another vendor's engine (Microsoft Designer on OpenAI → `OpenAI, Microsoft`; Microsoft on Google → `Microsoft, Google LLC, Google C2PA Core Generator Library`) or an edit chain re-signs (Adobe over a Gemini original → Adobe c2pa + Google synthid) — 19 such files across the 2026-06-01/02 batches read as clashes before the fix. Rule 1 still fires when a manifest vendor disagrees with a genuinely independent stamp (EXIF/XMP generator, IPTC `AISystemUsed`, AIGC, xAI); each non-`c2pa`/`synthid` family is its own source (`test_identify.py::TestIntegrityClashes::{test_multi_actor_manifest_no_clash,test_manifest_vendor_vs_independent_signal_clashes}`). Vendor normalization is `_vendor_of` over `_AI_VENDOR_TOKENS` (so a C2PA "Google (Gemini)" issuer and a SynthID-Google proxy agree, while different vendors clash). `_AI_VENDOR_TOKENS` covers ByteDance (all brands: bytedance/doubao/jimeng/dreamina/volcengine), Canva, ElevenLabs, and Black Forest Labs in addition to the OpenAI/Google/Adobe/... set — without them a transplanted ByteDance/Canva/BFL C2PA manifest next to an independent conflicting stamp was silently missed. **The generic `China AIGC (TC260)` label names no SPECIFIC vendor** (any Chinese generator applies it), so it cannot vendor-conflict in the spoofing sense: when a Chinese TC260-applying vendor (`_TC260_VENDORS`, today `{ByteDance}`) is co-attributed, Rule 1 attributes the label to that vendor (a legit Doubao image carries BOTH a ByteDance C2PA manifest and its own TC260 label and must not clash); against a NON-TC260 vendor (OpenAI etc.) the label stays generic and still clashes as a laundering tell (`test_bytedance_c2pa_plus_own_aigc_no_clash`, `test_foreign_vendor_plus_aigc_still_clashes`, `test_bytedance_c2pa_plus_foreign_generator_clashes`). Corpus-validated: 0 new clashes on 5000 ByteDance/AIGC/Canva/FLUX carriers.
|
||||
|
||||
**High-precision by design:** only hard generator stamps feed it (C2PA-issuer when source is AI, SynthID, EXIF/XMP generator, IPTC `AISystemUsed`, xAI, AIGC); the fuzzy visible sparkle and the open invisible watermark are **excluded** (the latter can be a by-product of our own SDXL removal pass). The c2pa vendor is classified from the issuer attribution / generator, NOT the resolved `platform` (a camera label like "Google Pixel" would mis-normalize to "Google"). All real single-origin fixtures (chatgpt/firefly/doubao/grok/mj) verified to produce **zero** clashes (false-positive guard in `test_identify.py::TestRealSamplesHaveNoClash`).
|
||||
**High-precision by design:** only hard generator stamps feed it (C2PA-issuer when source is AI, SynthID, EXIF/XMP generator, IPTC `AISystemUsed`, xAI, AIGC); the fuzzy visible sparkle and the open invisible watermark are **excluded** (both are low-precision/positive-only signals; the open watermark was also historically a by-product of our own SDXL removal pass, until `watermark_remover` was fixed to load the SDXL pipelines with `add_watermarker=False` — it stays excluded as a fuzzy signal regardless). The c2pa vendor is classified from the issuer attribution / generator, NOT the resolved `platform` (a camera label like "Google Pixel" would mis-normalize to "Google"). All real single-origin fixtures (chatgpt/firefly/doubao/grok/mj) verified to produce **zero** clashes (false-positive guard in `test_identify.py::TestRealSamplesHaveNoClash`).
|
||||
|
||||
**`ai_from_metadata` field + `has_invisible_target` helper (P0#5, 2026-06-22):** the high-confidence union (everything that sets `confidence == "high"`: C2PA AI-issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, open DWT-DCT / TrustMark — the medium-confidence `hf_only`/`visible_only`/`samsung_only` are excluded) is now surfaced as the public `ProvenanceReport.ai_from_metadata` boolean, so callers gate on intent rather than on the `confidence` string. `has_invisible_target(path)` wraps `identify(path, check_visible=False, check_invisible=True)` and returns that field — it is the decision gate for the diffusion scrub (the CLI `invisible`/`all`/`batch` no-signal skip, `cli._no_invisible_signal_exit`): a visible-only or no-signal image has it False, so regeneration (which would only degrade a clean image) does not run. It fails SAFE — any detector exception returns True so the removal still runs (leaving a watermark on a paid removal is worse than over-regenerating). It does NOT prove a pixel SynthID is absent (SynthID is detectable only via its metadata proxy, gone once stripped), so a False means "no locally-detectable target", never "clean". Guarded by `test_identify.py::{TestIdentifyRealSamples::test_has_invisible_target_*,TestHasInvisibleTargetFailSafe}`.
|
||||
|
||||
|
||||
@@ -447,6 +447,19 @@ class WatermarkRemover:
|
||||
self._set_progress("Loading fp16-fixed SDXL VAE (avoids black output)...")
|
||||
load_kwargs["vae"] = AutoencoderKL.from_pretrained(_SDXL_FP16_VAE_ID, torch_dtype=torch.float16)
|
||||
|
||||
@staticmethod
|
||||
def _disable_sdxl_watermarker(load_kwargs: dict[str, Any]) -> None:
|
||||
"""Turn off the diffusers default invisible watermarker on an SDXL pipeline.
|
||||
|
||||
diffusers embeds an open "Stable Diffusion XL" DWT-DCT invisible watermark on
|
||||
EVERY SDXL output whenever ``invisible-watermark`` is installed (the ``detect``
|
||||
extra). A watermark REMOVER must not re-stamp a detectable AI watermark, or the
|
||||
cleaned output re-reads as AI (``identify`` -> "Open invisible watermark: Stable
|
||||
Diffusion XL"). Shared by both SDXL loaders; the ``ControlNetModel`` sub-model
|
||||
and the Qwen loader never call it (only the pipeline accepts the kwarg).
|
||||
"""
|
||||
load_kwargs["add_watermarker"] = False
|
||||
|
||||
def _move_to_device_and_optimize(self, pipeline: Any) -> Any:
|
||||
"""Move a freshly-loaded pipeline to ``self.device`` + enable memory opts.
|
||||
|
||||
@@ -520,6 +533,7 @@ class WatermarkRemover:
|
||||
load_kwargs = self._base_load_kwargs()
|
||||
load_kwargs["safety_checker"] = None
|
||||
load_kwargs["requires_safety_checker"] = False
|
||||
self._disable_sdxl_watermarker(load_kwargs)
|
||||
self._maybe_add_fp16_vae(load_kwargs)
|
||||
|
||||
pipeline = self._load_from_pretrained(AutoImg2ImgPipeline, self.model_id, **load_kwargs) # type: ignore
|
||||
@@ -549,6 +563,7 @@ class WatermarkRemover:
|
||||
|
||||
load_kwargs = self._base_load_kwargs()
|
||||
load_kwargs["controlnet"] = controlnet
|
||||
self._disable_sdxl_watermarker(load_kwargs)
|
||||
self._maybe_add_fp16_vae(load_kwargs)
|
||||
|
||||
self._set_progress(f"Loading model weights: {self.model_id}")
|
||||
|
||||
@@ -178,6 +178,54 @@ class TestFp16WeightVariant:
|
||||
assert "variant" not in cls.from_pretrained.call_args.kwargs
|
||||
|
||||
|
||||
class TestNoReembeddedWatermark:
|
||||
"""F2 regression: the SDXL removal pipelines must disable the diffusers default
|
||||
invisible watermarker (``add_watermarker=False``).
|
||||
|
||||
diffusers stamps an open "Stable Diffusion XL" DWT-DCT watermark onto every SDXL
|
||||
output whenever ``invisible-watermark`` is installed. A watermark REMOVER that left
|
||||
it on would replace one detectable AI watermark (SynthID) with another -- the cleaned
|
||||
output re-reads as AI. The ControlNet sub-model load must NOT receive the kwarg (it
|
||||
is not a pipeline and does not accept it).
|
||||
"""
|
||||
|
||||
def _remover(self, profile: str):
|
||||
if not is_watermark_removal_available():
|
||||
pytest.skip("torch/diffusers not installed")
|
||||
from remove_ai_watermarks.noai.watermark_remover import WatermarkRemover
|
||||
|
||||
return WatermarkRemover(device="cpu", pipeline=profile)
|
||||
|
||||
def _capture(self, monkeypatch, remover):
|
||||
from remove_ai_watermarks.noai.watermark_remover import WatermarkRemover
|
||||
|
||||
calls: list[tuple[str, dict]] = []
|
||||
|
||||
def fake_load(self, cls, model_id, **kwargs):
|
||||
calls.append((getattr(cls, "__name__", str(cls)), kwargs))
|
||||
return MagicMock()
|
||||
|
||||
monkeypatch.setattr(WatermarkRemover, "_load_from_pretrained", fake_load)
|
||||
monkeypatch.setattr(WatermarkRemover, "_move_to_device_and_optimize", lambda self, p: p)
|
||||
return calls
|
||||
|
||||
def test_sdxl_pipeline_disables_watermarker(self, monkeypatch: pytest.MonkeyPatch):
|
||||
remover = self._remover("sdxl")
|
||||
calls = self._capture(monkeypatch, remover)
|
||||
remover._load_pipeline()
|
||||
assert any(kw.get("add_watermarker") is False for _, kw in calls), calls
|
||||
|
||||
def test_controlnet_pipeline_disables_watermarker(self, monkeypatch: pytest.MonkeyPatch):
|
||||
remover = self._remover("controlnet")
|
||||
calls = self._capture(monkeypatch, remover)
|
||||
remover._load_controlnet_pipeline()
|
||||
by_cls = dict(calls)
|
||||
# the SDXL pipeline load disables the watermarker...
|
||||
assert by_cls["StableDiffusionXLControlNetImg2ImgPipeline"].get("add_watermarker") is False
|
||||
# ...but the ControlNet sub-model load must not carry the kwarg (it would error).
|
||||
assert "add_watermarker" not in by_cls["ControlNetModel"]
|
||||
|
||||
|
||||
class _StubImage:
|
||||
"""Minimal PIL.Image stand-in: just the ``width``/``height`` the pure helper reads."""
|
||||
|
||||
|
||||
Reference in New Issue
Block a user