mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-19 07:37:14 +02:00
Victor KuznetsovandClaude Opus 5.5
7ba07869d2
Collect provider samples from Higgsfield, Runway and Dreamina plus Google Photos, Gemini, Grok and YouTube re-encodes into data/captures/2026-09 with a hash-checked manifest, and use them to fix attribution and false positives: - Attribute aggregator outputs: Higgsfield job metadata, Dreamina's re-signed manifests, Runway's own C2PA signer, TC260 video producers. - Veto any visible video mark when a valid C2PA claim names another vendor (a Gemini Omni clip read as Sora on steam), and gate Kling and Hailuo matches on edge, anchor and white-label evidence. - Veto the Gemini sparkle when trusted provenance names another vendor. - Report SynthID as present for Google Photos AI edits: Google's checker found it on all four measured edits, though manifests record no action. - Add Vidu video and Wan image marks, TC260 audio placements, and record the AIGC key in metadata records. - Document the research sweep, the Dreamina badge decision, and verified Google statements on SynthID in Photos. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
121 lines
6.2 KiB
Markdown
121 lines
6.2 KiB
Markdown
# Watermark forgery study
|
|
|
|
> Audit correction, 2026-09-08: The historical VideoSeal results and
|
|
> cross-scheme conclusions are superseded by the corrected video run below.
|
|
> Historical video observations were taken before encoding
|
|
> rather than from the hashed output artifact (R12); rectangular video
|
|
> decoding also required correction (R01). AudioSeal observations remain
|
|
> separate from those invalidated video comparisons.
|
|
|
|
## Corrected VideoSeal measurements (2026-09-08)
|
|
|
|
Twelve observations now measure decoded saved artifacts, including controls
|
|
and double embeds. All 12 recorded SHA-256 digests match the retained MP4s;
|
|
the decoder also checks that each file remains unchanged during measurement.
|
|
|
|
| Cell | Gradient vs A | Gradient vs B | Real Sora vs A | Real Sora vs B |
|
|
| --- | ---: | ---: | ---: | ---: |
|
|
| clean | 0.516 | 0.555 | 0.559 | 0.535 |
|
|
| marked A | 1.000 | 0.469 | 1.000 | 0.469 |
|
|
| removed CRF 23 | 0.559 | 0.496 | 1.000 | 0.469 |
|
|
| forged B on clean | 0.469 | 1.000 | 0.469 | 1.000 |
|
|
| forged B on marked A | 0.469 | 1.000 | 0.473 | 0.996 |
|
|
| forged B on removed A | 0.469 | 1.000 | 0.469 | 1.000 |
|
|
|
|
Both marked controls meet the fixed-message A rule. All six B-embedding
|
|
outputs fall below its 0.9 threshold, while matching B at 0.996 or above.
|
|
CRF 23 removes the matched A verdict on the gradient but preserves it on
|
|
this Sora carrier. These two fixed carriers support this scoped observation;
|
|
they do not establish complete erasure of every trace of A or a universal
|
|
double-embedding rule. AudioSeal was not rerun in this correction.
|
|
|
|
The run used the same pinned checkpoint, Torch 2.14.0, ffmpeg 9.0.1, and
|
|
two CPU threads as the corrected [temporal study](videoseal-temporal-evaluation.md#corrected-artifact-measurements-2026-09-08).
|
|
No model downloads or provider calls ran. Study source SHA-256:
|
|
`031d9e0c58af58ab5f9227ea52a500eb86644fc464f9c37be1e894ce0b6b4935`.
|
|
Corrected case rows SHA-256:
|
|
`f8c5a65e5853de0f1ff67c2ef0549fbd81df971e13c79539c85c1b0255eb60a8`.
|
|
Artifacts remain outside the repository.
|
|
|
|
## Historical study (2026-09-07, superseded video results)
|
|
|
|
Development-only study, run on 2026-09-07 from
|
|
`scripts/watermark_forgery_study.py`, opening the forgery/remover-forensics
|
|
direction from the original research map. It measures the four-state model
|
|
(clean / marked / removed / forged) and the double-embedding question for both
|
|
pinned oracles: does embedding message B over an A-marked artifact leave A,
|
|
B, or mutual interference?
|
|
|
|
Carriers: the tone-stack audio carrier, the synthetic moving-gradient video
|
|
clip, and the real committed Sora clip. Message B is the oracle's message
|
|
generator at seed 8. Everything local; artifacts under `.local-eval/`.
|
|
|
|
## The double-embedding answer: the last writer wins, completely
|
|
|
|
| Cell | AudioSeal acc vs A | vs B | VideoSeal (gradient) vs A | vs B | VideoSeal (real Sora) vs A | vs B |
|
|
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
| marked A | 1.000 | 0.44 | 1.000 | 0.47 | 1.000 | 0.47 |
|
|
| removed | 0.50 (presence 0.28) | 0.44 | 0.57 | 0.48 | 1.000 | 0.47 |
|
|
| forged B on clean | 0.44 | 1.000 | 0.47 | 1.000 | 0.47 | 1.000 |
|
|
| **forged B on marked A** | **0.44** | **1.000** | **0.47** | **1.000** | **0.48** | **0.99** |
|
|
| forged B on removed | 0.56 | 0.875 | 0.47 | 1.000 | 0.47 | 1.00 |
|
|
|
|
Embedding B over marked A reads as pure B: the original message decays to
|
|
coin-flip accuracy under both schemes, on synthetic and real content alike.
|
|
There is no mutual interference and no residue of A. With an unkeyed, openly
|
|
published embedder this is the expected outcome - and now it is a measurement,
|
|
not an assumption. The forensic consequence: payload attribution without key
|
|
control is meaningless against an attacker who can run the same open embedder;
|
|
only presence detection survives, and even that carries the attacker's
|
|
payload. Provider schemes that rely on secret keys are a different question
|
|
this study cannot answer.
|
|
|
|
Two boundary observations travel with the table:
|
|
|
|
- The removal transform is carrier-dependent, as the cohorts already showed:
|
|
10 dB noise removes the tone stack (presence 0.28) and crf 23 removes the
|
|
gradient clip (0.57), but the real Sora clip keeps a perfect 1.000 decode
|
|
through the same crf 23 - the same number the temporal study recorded.
|
|
- A forged row reads `detected` under the AudioSeal presence rule (the label
|
|
carries the foreign payload) and `not_detected` under the VideoSeal
|
|
message-accuracy rule. Both cohort fixtures encode their scheme's correct
|
|
expected value, and the benchmark docs state the two rules side by side.
|
|
|
|
## Remover-trace probe (scoped first evidence)
|
|
|
|
High-to-low frequency band energy ratio on clean versus noise-removed audio:
|
|
the tone stack jumps from 8.6e-05 to 2.3e-02 (a 269x trace), the pinkish
|
|
carrier 0.016 to 0.039, and the white-noise carrier does not separate at all
|
|
(0.330 to 0.332) - a noise carrier absorbs a noise remover. This detects the
|
|
trace of THIS study's gaussian-noise removal only; it is not a general
|
|
remover detector, and the probe JSON says so in its scope field.
|
|
|
|
## Cohort arms
|
|
|
|
The audio cohort (v2, 20 cases) and video cohort (v2, 15 cases) now carry the
|
|
four states as strict manifest rows: `removed` rows where the transform
|
|
measurably destroys the mark (audio: noise on tone stack; video: crf 23 on
|
|
both synthetic carriers), and `forged` rows embedding message B with the
|
|
scheme-correct expected verdict. Zero expected-result mismatches in both
|
|
recorded runs.
|
|
|
|
## Pins
|
|
|
|
Study script
|
|
`sha256:3985522c862ac44a0c7ef8aa02946a11701439a94d67f5867f00819f638c032b`,
|
|
case rows
|
|
`sha256:03d446c08a5daed623a4be2980e097bb8bbc0cd7a6d3ab3f931d25aa58570a16`.
|
|
Oracles and checkpoints are pinned as in the kernel document.
|
|
|
|
## Follow-ups
|
|
|
|
- A keyed-scheme perspective needs a scheme with actual key control. WmForger
|
|
([arXiv:2510.20468](https://arxiv.org/abs/2510.20468), NeurIPS 2025) is a
|
|
gradient attack on an image preference model, not a diffusion attack, and its
|
|
code ships MIT-licensed in `facebookresearch/videoseal`, so it can run against
|
|
the pinned VideoSeal oracle and the local TrustMark decoder. WMCopier
|
|
([arXiv:2503.22330](https://arxiv.org/abs/2503.22330)) has public code but no
|
|
license, so its outputs cannot be published here.
|
|
- A general remover-trace detector would need features beyond one band ratio
|
|
and validation against removals this project did not produce.
|