mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-14 21:40:47 +02:00
Victor KuznetsovandClaude Opus 5.5
7ba07869d2
Collect provider samples from Higgsfield, Runway and Dreamina plus Google Photos, Gemini, Grok and YouTube re-encodes into data/captures/2026-09 with a hash-checked manifest, and use them to fix attribution and false positives: - Attribute aggregator outputs: Higgsfield job metadata, Dreamina's re-signed manifests, Runway's own C2PA signer, TC260 video producers. - Veto any visible video mark when a valid C2PA claim names another vendor (a Gemini Omni clip read as Sora on steam), and gate Kling and Hailuo matches on edge, anchor and white-label evidence. - Veto the Gemini sparkle when trusted provenance names another vendor. - Report SynthID as present for Google Photos AI edits: Google's checker found it on all four measured edits, though manifests record no action. - Add Vidu video and Wan image marks, TC260 audio placements, and record the AIGC key in metadata records. - Document the research sweep, the Dreamina badge decision, and verified Google statements on SynthID in Photos. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
74 lines
4.0 KiB
Markdown
74 lines
4.0 KiB
Markdown
# External watermark benchmarks: ETI and W-Bench licensing verdict
|
|
|
|
Investigation completed 2026-09-07 against the primary sources, closing the
|
|
last open item of the original research map: whether an honest external
|
|
benchmark profile can be built on ETI or W-Bench. The verdict is split:
|
|
W-Bench is legally usable, ETI is license-blocked.
|
|
|
|
## ETI (NeurIPS 2024 "Erasing the Invisible")
|
|
|
|
Sources verified: the Hugging Face dataset
|
|
`furonghuang-lab/ETI_Competition_Data` at revision `d80024580b5a` and the
|
|
organizers' evaluation repository `erasinginvisible/eval-program`.
|
|
|
|
- **License: absent everywhere.** The dataset carries no license tag and no
|
|
license section in its card; the evaluation repository has no LICENSE file.
|
|
With no grant of rights, the corpus is unusable in this public repository,
|
|
and downloading it for local evaluation builds results that cannot be
|
|
published. The only unblock is written permission from the organizers -
|
|
an external action that needs an explicit decision, not a default.
|
|
- **Composition (verified from the card):** 300 Beige-track images
|
|
(Gaussian Shading on Stable Diffusion 2.1, StegaStamp on Flux.1) and 300
|
|
Black-track images (JigMark, PRC, StableSignature, Trufo, plus
|
|
Gaussian-Shading+JigMark and StableSignature+StegaStamp doubles), plus
|
|
~1,072 Beige and ~1,650 Black valid submissions with per-watermark
|
|
detection results, image-quality metrics, and final scores.
|
|
- One earlier blocker is now stale: the attacked submission images ARE part
|
|
of the release per the card note, not missing. The license blocker alone
|
|
remains decisive.
|
|
|
|
## W-Bench / VINE (ICLR 2025)
|
|
|
|
Sources verified: the Hugging Face dataset `Shilin-LU/W-Bench` at revision
|
|
`c81f3924f0fb` and the VINE repository `Shilin-LU/VINE`.
|
|
|
|
- **Dataset license: MIT**, declared on the dataset card. The benchmark is
|
|
legally usable as a benchmark, but the MIT tag does not relicense the
|
|
underlying COCO, Flickr and ShareGPT4V images, whose own terms still apply
|
|
to any redistribution.
|
|
- **Composition (verified from the card):** 10,000 instances sourced from
|
|
COCO, Flickr, ShareGPT4V and similar datasets, eleven watermarking
|
|
methods, evaluated across four editing profiles - regeneration, global
|
|
editing, local editing, image-to-video - plus image distortions. An
|
|
earlier internal note said "15k images"; the card says 10,000, and the
|
|
card wins.
|
|
- **Code license: NTUITIVE Non-Commercial** (NTU technology transfer;
|
|
commercial use requires contacting the authors). The VINE code cannot be
|
|
vendored into this Apache-2.0-licensed project, and pipeline outputs derived
|
|
from running it would inherit that restriction. What remains lawful and
|
|
useful: consuming the MIT dataset directly with our own pinned oracles
|
|
and benchmark kernel, adopting the attack taxonomy as protocol
|
|
definitions reimplemented in our stack.
|
|
|
|
## Verdict for the map item
|
|
|
|
An external W-Bench profile is buildable: MIT dataset, our kernel, our
|
|
oracles, no VINE code. Whether to build it is a scoping decision, not a
|
|
legal one - the image cohorts already cover detection/removal/fidelity
|
|
questions with pinned local evidence, and W-Bench would add its
|
|
editing-profile attack axis (regeneration, image-to-video). The ETI profile
|
|
stays closed until the organizers grant a license; nothing in this
|
|
repository should download or derive from it meanwhile.
|
|
|
|
One protocol detail is worth keeping as a candidate, not a commitment: the
|
|
ETI scoring program runs submissions through median blur and JPEG
|
|
compression before decoding, so an attack only wins if the watermark stays
|
|
dead under ordinary handling - that filter separates fragile
|
|
adversarial-perturbation removals from structural ones. Our removal paths
|
|
are structural (fill, regeneration), so no driving question exists today;
|
|
if removal durability under platform re-encoding ever becomes one, the
|
|
follow-up is an hour-sized handling arm over removed-state artifacts in the
|
|
existing cohorts. Participant attack code was never published (the release
|
|
carries attacked images and metrics only), so there is no implementation to
|
|
learn from even where reading would be lawful.
|