diffusers 0.38's auto-pipeline registry imports a transformers 5.x-only
symbol, so the gpu extra needs transformers>=5. Cap tokenizers to the
stable 0.22 line so the global prerelease="allow" no longer drags in the
0.23.0rc0 whose CLIP tokenizer breaks SDXL loading.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
HEIF/AVIF store XMP as a meta-box `mime` item whose bytes live in mdat/idat, out
of reach of the top-level uuid/jumb box stripper. An AI-label XMP packet there
(TC260 AIGC, IPTC "Made with AI", IPTC 2025.1) was therefore left in place.
isobmff.blank_ai_xmp_packets locates each XMP packet by its <?xpacket begin ...
end?> delimiters and, if it carries an AI marker (_AI_LABEL_MARKERS), overwrites
it with spaces of the SAME length. Equal length means no box size or iloc offset
shifts -- the coded image stays bit-for-bit intact, the item stays structurally
valid, only the AI label content is destroyed. Plain (non-AI) XMP is left alone,
mirroring the top-level XMP-uuid content match. Wired into remove_ai_metadata's
ISOBMFF branch after strip_c2pa_boxes.
Chosen over exiftool (a non-bundled binary dep) to stay pure-Python and
droplet-compatible; over full iinf/iloc surgery to avoid offset-rewrite
corruption risk. The AI labels we target are all XMP, so this closes the
practical gap. An Exif *item* inside the meta box (rare) still needs iinf/iloc
surgery or exiftool -- documented.
4 new tests (TestMetaBoxXmpBlanking): AI packet blanked (same length, marker
gone, surrounding image bytes intact), plain XMP preserved, no-packet no-op, and
end-to-end remove_ai_metadata on a .heic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Provenance detection no longer relies on a fixed first-MB read. In a streaming /
non-faststart MP4 the C2PA manifest sits AFTER a multi-megabyte mdat, beyond the
1 MB scan window, so it was missed.
- isobmff.scan_c2pa_region(path): a file-seeking top-level box walker that
returns the payloads of uuid/jumb (provenance) boxes, seeking past mdat by
size without reading it -- works on multi-GB files. Returns b"" for
non-ISOBMFF or on read error. Mirrors the box-size encoding of the existing
in-memory _iter_top_level_boxes (largesize / size==0).
- metadata.scan_head(path, size): the shared input for every C2PA/AIGC/IPTC
byte scan -- first __TEXT __DATA __OBJC others dec hex bytes plus, for ISOBMFF, the late provenance-box
payloads. Behavior-neutral (f.read(size)) for non-ISOBMFF inputs.
- Routed all six metadata scan sites (has_ai_metadata, aigc_label,
iptc_ai_system, synthid_source, exif_generator XMP, get_ai_metadata
soft-binding) and identify's head read through scan_head.
6 new tests: late box found by scan_c2pa_region / scan_head, the fixed window
provably misses it, non-ISOBMFF -> b"", front-placed (faststart) regression.
The remaining gap stays documented: EXIF/XMP stored as items inside the meta
box (AVIF/HEIF stills) still needs meta-box surgery or exiftool.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Surface contradictions between independent provenance signals instead of
collapsing to a single verdict -- a strong tell of spoofed, transplanted, or
laundered metadata. Inspired by arXiv:2603.02378.
Two rules in the new _integrity_clashes helper:
- Conflicting AI-origin attributions: two or more distinct AI vendors named by
independent generator stamps (e.g. a C2PA OpenAI manifest on an image whose
EXIF says Make="Ideogram AI").
- Camera + AI: a camera-capture C2PA device (Pixel/Leica/Sony/Nikon/Truepic)
coexisting with an AI-generation marker -- a genuine capture is not AI.
High-precision by design: only hard generator stamps feed it (C2PA issuer when
the source is AI, SynthID proxy, EXIF/XMP generator, IPTC AISystemUsed, xAI,
AIGC). The fuzzy visible sparkle and the open invisible watermark are excluded
-- the latter can be a by-product of our own SDXL removal pass. Vendor
normalization (_vendor_of over _AI_VENDOR_TOKENS) keeps consistent signals from
clashing (C2PA "Google (Gemini)" + SynthID-Google agree); the C2PA vendor is
read from the issuer attribution, not the resolved platform, so a camera label
like "Google Pixel" cannot mis-normalize to an AI vendor.
Surfaced as ProvenanceReport.integrity_clashes (red in the table view, included
in --json). 19 new tests; all real single-origin fixtures (chatgpt/firefly/
doubao/grok/mj) verified to produce zero clashes (false-positive guard).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two CLI/IO robustness bugs surfaced by issues #17 and #19.
#17 -- non-ASCII image paths (Chinese/Cyrillic/accented) failed on Windows:
cv2.imread/imwrite use the platform ANSI code-page API, so the decode came back
empty with a "can't open/read file" warning. New image_io.imread/imwrite route
through np.fromfile+cv2.imdecode / cv2.imencode+tofile (Unicode-safe, byte-
identical output, cv2.imread None-semantics preserved); all 8 cv2 read/write
call sites now go through it. Behavior-neutral on macOS/Linux (already accept
UTF-8 paths), so the fix is correct-by-construction for the Windows-only bug.
#19 (incidental) -- rich parsed the "[gpu]" in the GPU-extra install hint as a
style tag and dropped it, so the printed command was the un-installable
"pip install 'remove-ai-watermarks'". Escaped as \[gpu] at both call sites.
Tests: test_image_io.py (non-ASCII round-trip, alpha, missing/empty/garbage
semantics); test_cli.py::TestGpuHintMarkup (install hint keeps the extra).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prevents an unmapped C2PA device whose manifest incidentally contains a mapped
issuer substring (e.g. the "Adobe XMP" toolkit string in a Canon/Sony camera
capture) from being mislabeled as that AI generator ("Adobe Firefly").
_attribute_platform now names a specific AI-generator platform only when the
digital-source-type is trainedAlgorithmicMedia; otherwise it degrades to the
neutral "C2PA signer: X" label. Real Firefly/OpenAI/Google output carries the
AI source-type and is unaffected (verified: chatgpt-1.png->OpenAI,
firefly-1.png->Adobe Firefly still attribute). Closes the only real downside of
leaving Canon/Samsung/Bria device signers unmapped: detection and removal were
already unaffected; now the platform label degrades gracefully too.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
remove_ai_metadata now handles non-ISOBMFF audio/video (which the box walker
can't reach) by shelling out to ffmpeg with a lossless stream copy
(`-map_metadata -1 -map_chapters -1 -c copy`): codec data is untouched, only
container tags/chapters (ID3 / RIFF / Vorbis comments / EBML tags) are dropped.
Requires ffmpeg on PATH; raises a clear RuntimeError if absent or if ffmpeg
can't parse the input (instead of crashing in the image path).
Verified end-to-end: a real ffmpeg-made WAV/MP3 with a "Suno AI" title tag ->
tag gone, audio bytes preserved.
NOT built (evaluated, deliberate): Resemble PerTh audio *detection* --
`get_watermark()` returns a raw bit array with no presence/confidence flag, so
reliably telling watermarked from clean needs Resemble's fixed payload or a
confidence API (neither public; no real sample to calibrate). Same wall as the
SynthID pixel detector. AVIF/HEIF meta-box EXIF/XMP stripping also stays a gap
(needs exiftool, a non-installed binary). Both documented in CLAUDE.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds Sony to _DEVICE_C2PA_PLATFORM, matching Sony's own `sony.sig` / `sony.cert`
C2PA assertion namespace (NOT bare "Sony", which is a common EXIF Make). Verified
against a real Sony-signed file (Sony PXW-Z300, signer "Sony Corporation") found
in the Security4Media/c2pa-video-player repo. The sample is video (MP4) -- our
ISOBMFF C2PA path detects it; Sony Alpha stills likely share the namespace.
Verified device set is now Leica, Nikon, Google Pixel, Sony, Truepic. Canon /
Samsung / Bria still have no public direct-download C2PA sample to verify.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the claim-generator-string match with a distinctive device-token scan
of the manifest bytes (_device_platform / _DEVICE_C2PA_PLATFORM), which is more
robust: it catches devices where the generator name lives under a non-standard
CBOR key (Pixel uses `claim_generator_info`, so it has no `claim_generator`).
- Adds Google Pixel, verified against a real Pixel 10 Pro C2PA file (attached to
c2pa-rs issue #1609/#1554): cert CN "Pixel Camera", digitalSourceType
`computationalCapture` -> capture authenticity, not AI (is_ai stays None).
- Token distinctiveness is load-bearing: bare "Truepic" matched the OpenAI
chatgpt-1.png fixture (Truepic is a trust-chain signing authority), so the
token is the specific "Truepic_Lens"; "Pixel Camera" (cert CN) not "Pixel".
- Verified Leica/Nikon/Truepic/Pixel attribute correctly and OpenAI/Adobe/MJ
do not regress. Sony/Canon/Samsung/Bria stay unmapped: no public direct-
download C2PA sample exists to verify their in-manifest string.
- Regression tests: device token beats incidental issuer mentions (Leica,
Pixel-vs-Google).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verified on real signed files that the issuer byte-scan mis-attributes
multi-entity manifests: Leica read as "Truepic" (timestamp authority in the
chain), Nikon as "Adobe Firefly" (XMP-toolkit "Adobe" + the sample's
"Adobe_MAX" name), Truepic as "Google". Platform attribution now prefers the
claim generator (what produced the asset) and falls back to the issuer scan.
- New _CLAIM_GENERATOR_PLATFORM map + _platform_from_generator; claim generator
read for non-PNG via the now-public c2pa.cbor_text_after.
- Device tokens listed only where verified against a real C2PA file (Leica
lc_c2pa, Nikon, Truepic Lens); Pixel/Samsung/Sony/Canon/Bria deferred until a
real sample confirms the in-manifest string. Camera C2PA marks capture
authenticity, so these never set is_ai.
- cbor_text_after made public (was _cbor_text_after); call sites + tests updated.
- Regression test: claim_generator beats incidental Adobe/Google/Truepic tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Broadens metadata provenance coverage at the detection and container-strip level.
Detection:
- C2PA soft-binding `alg` -> forensic-watermark vendor (Adobe TrustMark,
Digimarc, Imatag, Steg.AI, Microsoft, ...) via C2PA_SOFT_BINDINGS +
soft_binding_vendors_in(); names the watermark vendor even when the watermark
itself can't be decoded.
- IPTC Photo Metadata 2025.1 AI-disclosure XMP fields (AISystemUsed etc.) via
iptc_ai_system() + IPTC_AI_FIELD_MARKERS.
- Adobe TrustMark open keyless decoder (trustmark_detector.py, optional extra
`trustmark`) -- the watermark behind Adobe Durable Content Credentials.
Detects provenance, not AI origin, so it does not assert is_ai.
Removal / containers:
- isobmff.strip_c2pa_boxes now also drops a top-level XMP uuid box that carries
an AI label (matched by AI-marker content, byte-order-robust; plain XMP kept).
- remove_ai_metadata routes MP4/MOV/M4V/M4A (and any ftyp-sniffed ISOBMFF)
through the box stripper; raises a clear error for non-ISOBMFF audio/video
(WebM/MP3/WAV) instead of crashing in the image path.
Tests: soft-binding scan, IPTC element/attribute/presence, MP4 + M4A detect/
strip, ISOBMFF XMP surgical strip, content-sniff, unsupported-container guard,
TrustMark absent-safety + identify integration. ruff clean; pyright clean on
all new modules.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
remove_ai_metadata now scrubs AI tags from the JPEG EXIF instead of passing
the block through wholesale. Closes the v0.5.5 follow-up: the xAI/Grok
Signature + UUID-Artist pair was detected but not removed.
- metadata._scrub_ai_exif(): deletes the xAI signature pair and any
Software/Make/Artist/ImageDescription tag carrying an AI_GENERATOR_TOKENS
token (so Ideogram's Make="Ideogram AI" is scrubbed too), keeping genuine
camera/editor EXIF intact.
- Shared _is_xai_signature_pair / _exif_text helpers (module-level compiled
regexes) are now the single source of truth, used by both xai_signature
and _scrub_ai_exif.
- Tests: Grok signature stripped on JPEG output, Ideogram Make stripped,
real-camera Make ("Apple") preserved. 325 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
xAI Grok (Aurora) images carry no C2PA/SynthID/IPTC -- their only provenance
signal is an EXIF pair: ImageDescription "Signature: <base64>" + a UUID Artist.
Verified stable across 3 genuine generations (a real download previously read
as unknown / "no AI metadata").
- metadata.xai_signature(): matches the Signature blob + UUID Artist pair;
wired into has_ai_metadata, get_ai_metadata, and identify (platform
"xAI (Grok / Aurora)").
- data/samples/grok-1.jpg: real Grok fixture (neutral content; the Artist UUID
is the public image id, not PII).
- Tests: synthetic-fixture unit tests, real-sample assertion, identify
integration (322 passing).
Docs (research refresh, May 2026):
- C2PA 2.4 Durable Content Credentials (soft-binding re-discovery after the
embedded manifest is stripped).
- New AI-labeling laws, primary-source verified: EU AI Act Art 50 (2026-08-02),
South Korea AI Framework Act Art 31(3), California AB 853.
- Hedge removal claims: defeating the SynthID verifier is not forensic
invisibility (arXiv:2605.09203); cite SynthID-Image (arXiv:2510.09263).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The native-vs-downscale decision in InvisibleEngine.remove_watermark (the
issue #10/#15 fix: max_resolution=0 must not pre-downscale, since any
downscale both loses quality and lets SynthID survive) had no test. Extract
it into a pure helper invisible_engine._target_size(w, h, max_resolution)
and cover it with tests/test_invisible_engine.py::TestTargetSize so a
re-introduced forced downscale fails CI instead of silently regressing #15.
Also:
- Clamp the short side to >=1 in _target_size: extreme aspect ratios (e.g.
5000x3 with --max-resolution 1024) truncated it to 0 and crashed
image.resize(). Pre-existing in the inline math; fixed now that it is a
named, tested function.
- Consolidate the two duplicated temp-file save blocks into one
unconditional save (behavior unchanged: the EXIF-transposed image is
still always persisted before WatermarkRemover reloads it by path), and
drop the now-redundant `_tmp_path is not None` guard in finally.
- Bump version 0.5.3 -> 0.5.4 (pyproject, __init__, uv.lock); document the
helper as the regression guard in CLAUDE.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- feat(identify): detect the China TC260 <TC260:AIGC> XMP label (Doubao
and other China-served generators); reports platform + ContentProducer.
Removal already strips it via the existing metadata cleaner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- fix(invisible): process at native resolution by default; the forced
downscale-to-1024 -> upscale-back round-trip was the main quality loss
(#10). Matches the raiw.cc backend (fal fast-sdxl = sdxl-base-1.0).
New --max-resolution opt-in cap for GPU/MPS memory.
- docs: verified fal checkpoint, native-res, gpt-image-2 SynthID.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Research found one locally-fillable detection gap: Stable Diffusion, SDXL,
and FLUX all embed an open DWT-DCT watermark via the invisible-watermark
(imwatermark) library -- a PUBLIC decoder, no secret key, unlike SynthID.
New invisible_watermark.py decodes the known fixed patterns (verified
against upstream source: diffusers SDXL WATERMARK_MESSAGE, FLUX.2
src/flux2/watermark.py, and the 'StableDiffusionV1' default string) and
identify() reports the scheme as a high-confidence signal.
Verified locally end-to-end: embedding SDXL's exact 48-bit message and
decoding it back recovers 48/48 bits; a clean image and our own fal-SDXL
outputs decode to ~21/48 (no match). Caveat baked into the report: the
watermark is fragile -- gone after JPEG q90 -- so it confirms origin only
on pristine files; absence is never proof.
imwatermark is an optional dep (extra 'detect'; pulls non-headless opencv),
so the import is guarded and the signal is skipped when absent. CLI
--no-visible now means metadata-only (skips both pixel-domain detectors).
Also records the broader watermarking landscape in CLAUDE.md: which
services are locally detectable (SD/SDXL/FLUX), C2PA-covered (Bing/Canva/
Getty/Shutterstock unsampled), or proprietary-only like SynthID (Amazon
Titan/Nova, Kakao). Midjourney embeds neither C2PA nor an invisible mark.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Bump diffusers minimum to 0.38.0 (closes GHSA-98h9-4798-4q5v).
- Refresh uv.lock to pull urllib3 2.7.0 (closes GHSA-qccp-gfcp-xxvc and
GHSA-mf9v-mfxr-j63j via transitive update from requests / huggingface-hub).
- Allow pre-releases globally (`[tool.uv] prerelease = "allow"`) because
diffusers 0.38.0 declares a dependency on safetensors>=0.8.0rc0. Drop
once safetensors 0.8.0 stable is published or diffusers re-pins.
uv-secure --ignore-unfixed now reports zero vulnerabilities.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reverse alpha blending applied at the assumed default position painted
a visible inverse-sparkle artifact onto clean or edited images. The
function now returns an unmodified copy when detection fails, instead
of falling back to the hardcoded Gemini corner. Bump to 0.3.5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- metadata --check now shows claim_generator, c2pa_spec, digital_source_type,
c2pa_actions, signer instead of empty table for C2PA-only files
- reuses existing extract_c2pa_chunk() from noai/c2pa.py — no more duplicate
PNG chunk parsing or full-file reads
- adds data/samples/openai-images-2/amur-leopard.png: real gpt-image-2 output
with C2PA manifest signed by OpenAI OpCo LLC / Trufo CA (spec 2.2.0)
- removes stale data/samples/nano-banana-1/2.png (no longer referenced)
- updates README: new Images 2.0 row in supported models table
- documents known text-degradation limitation in CLAUDE.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove opencv-python from [gpu] extra (conflicts with headless in base deps)
- Add graceful fallback in 'invisible' and 'all' commands when GPU deps missing
- Cache InvisibleEngine in batch mode (avoid reloading model per image)
- Fix --humanize help text (was '0.0-1.0', actual range is 0-6.0+)
- Fix stale docstring referencing non-existent [invisible] extra
- Add [gpu] extra install instructions to README
- Fix broken NeuralBleach placeholder URL in Credits
Move torch, diffusers, transformers, accelerate, controlnet-aux,
ultralytics, and safetensors into [project.optional-dependencies.gpu].
Core install now only includes lightweight deps (~20 MB vs ~1 GB):
pillow, piexif, numpy, opencv-python-headless, click, rich.
This allows web apps using fal.ai cloud GPU to skip installing
1+ GB of ML packages, reducing Docker images from 3 GB to ~300 MB
and deploy times from 14 minutes to ~3-4 minutes.
Usage:
pip install remove-ai-watermarks # core only (visible + metadata)
pip install remove-ai-watermarks[gpu] # full local GPU support
pip install remove-ai-watermarks[all] # gpu + dev tools
- Rewrite README for SEO: Nano Banana, SynthID, Made with AI, C2PA keywords
- Add Supported Models table with 7 AI services
- Add 'Made with AI' label removal to features
- Rename sections for search discoverability
- Add samples: ChatGPT/DALL-E, Midjourney, Adobe Firefly
- Reorganize data/samples with flat structure and clear naming
- Unify 'all' defaults to match 'invisible' (strength=0.02, steps=100)
- Reorder CLI docs: 'all' command first, individual commands second
- HuggingFace token is now documented as optional
- Remove 'additional setup' label from invisible section
Changes since 0.1.0:
- Fix phantom model param bug in invisible/all commands
- Fix macOS SSL certificate issue for YOLO downloads
- Use temp file in 'all' pipeline to hide intermediate output
- Add legal disclaimer and fix license attribution
- Add troubleshooting and upgrade docs to README
- Expand test suite to 137 tests covering all CLI modes
- Clean up dependencies and pyright config
- CLI with visible, invisible, all, metadata, and batch commands
- Gemini watermark removal via reverse alpha blending
- Invisible watermark removal via diffusion regeneration (SynthID, TreeRing)
- AI metadata stripping (EXIF, PNG text, C2PA)
- Face protection (YOLO/Haar) and analog humanizer
- 137 tests covering all CLI modes and core engines
- Ruff and Pyright clean
Victor Kuznetsov