mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-22 08:38:22 +02:00
Victor Kuznetsov
464fd142ea
Extend the development-only benchmark family from images to audio and video, and close four research directions from the watermark-landscape map with local, revision-pinned evidence. Benchmark schema: media_type audio and video rows, audioseal and videoseal adapters with media-checked adapter binding, sample-domain audio fidelity (SNR with unbounded_identical), frame-domain video fidelity (mean PSNR, changed-frame fraction), and per-frame temporal evidence in videoseal detection records. The report accepts all three media types and carries SNR columns. Audio: shared pinned AudioSeal oracle, synthetic and system-voice speech cohorts with matched negatives, codec, noise, removed, and forged arms, and the provenance experiment proving visual-only video cleaning leaves the audio bitstream and watermark verdict invariant. Video: TorchScript-only VideoSeal oracle (the pip package is unusable on macOS arm64 and its loader resolves paths relative to cwd), video cohort with crf/scale/fps attacks and four-state arms, temporal evaluation measuring the crf decay curve, the frame-aggregation matrix (computed locally after finding the TorchScript build ignores the aggregation argument), and real provider clips as carriers. Forgery: double-embedding study showing the last writer wins completely under both oracles, scoped remover-trace probe, and the forged/ removed cohort arms encoding each scheme's correct expected verdict (presence rule for audioseal, message rule for videoseal). C2PA resolution: measured the five vendors' resolver contracts and the C2PA 2.2 18.10.5 privacy gradient; product decision recorded that this tool never resolves over the network. A registry guard pins the published resolution-API set. Gate: ruff and pyright src/ clean, 2114 passed / 3 skipped; maintain.sh stops only on the known unfixed lightning 2.6.5 CVEs, pre-existing.
77 lines
3.9 KiB
Markdown
77 lines
3.9 KiB
Markdown
# Watermark forgery study
|
|
|
|
Development-only study, run on 2026-09-07 from
|
|
`scripts/watermark_forgery_study.py`, opening the forgery/remover-forensics
|
|
direction from the original research map. It measures the four-state model
|
|
(clean / marked / removed / forged) and the double-embedding question for both
|
|
pinned oracles: does embedding message B over an A-marked artifact leave A,
|
|
B, or mutual interference?
|
|
|
|
Carriers: the tone-stack audio carrier, the synthetic moving-gradient video
|
|
clip, and the real committed Sora clip. Message B is the oracle's message
|
|
generator at seed 8. Everything local; artifacts under `.local-eval/`.
|
|
|
|
## The double-embedding answer: the last writer wins, completely
|
|
|
|
| Cell | AudioSeal acc vs A | vs B | VideoSeal (gradient) vs A | vs B | VideoSeal (real Sora) vs A | vs B |
|
|
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
| marked A | 1.000 | 0.44 | 1.000 | 0.47 | 1.000 | 0.47 |
|
|
| removed | 0.50 (presence 0.28) | 0.44 | 0.57 | 0.48 | 1.000 | 0.47 |
|
|
| forged B on clean | 0.44 | 1.000 | 0.47 | 1.000 | 0.47 | 1.000 |
|
|
| **forged B on marked A** | **0.44** | **1.000** | **0.47** | **1.000** | **0.48** | **0.99** |
|
|
| forged B on removed | 0.56 | 0.875 | 0.47 | 1.000 | 0.47 | 1.00 |
|
|
|
|
Embedding B over marked A reads as pure B: the original message decays to
|
|
coin-flip accuracy under both schemes, on synthetic and real content alike.
|
|
There is no mutual interference and no residue of A. With an unkeyed, openly
|
|
published embedder this is the expected outcome - and now it is a measurement,
|
|
not an assumption. The forensic consequence: payload attribution without key
|
|
control is meaningless against an attacker who can run the same open embedder;
|
|
only presence detection survives, and even that carries the attacker's
|
|
payload. Provider schemes that rely on secret keys are a different question
|
|
this study cannot answer.
|
|
|
|
Two boundary observations travel with the table:
|
|
|
|
- The removal transform is carrier-dependent, as the cohorts already showed:
|
|
10 dB noise removes the tone stack (presence 0.28) and crf 23 removes the
|
|
gradient clip (0.57), but the real Sora clip keeps a perfect 1.000 decode
|
|
through the same crf 23 - the same number the temporal study recorded.
|
|
- A forged row reads `detected` under the AudioSeal presence rule (the label
|
|
carries the foreign payload) and `not_detected` under the VideoSeal
|
|
message-accuracy rule. Both cohort fixtures encode their scheme's correct
|
|
expected value, and the benchmark docs state the two rules side by side.
|
|
|
|
## Remover-trace probe (scoped first evidence)
|
|
|
|
High-to-low frequency band energy ratio on clean versus noise-removed audio:
|
|
the tone stack jumps from 8.6e-05 to 2.3e-02 (a 269x trace), the pinkish
|
|
carrier 0.016 to 0.039, and the white-noise carrier does not separate at all
|
|
(0.330 to 0.332) - a noise carrier absorbs a noise remover. This detects the
|
|
trace of THIS study's gaussian-noise removal only; it is not a general
|
|
remover detector, and the probe JSON says so in its scope field.
|
|
|
|
## Cohort arms
|
|
|
|
The audio cohort (v2, 20 cases) and video cohort (v2, 15 cases) now carry the
|
|
four states as strict manifest rows: `removed` rows where the transform
|
|
measurably destroys the mark (audio: noise on tone stack; video: crf 23 on
|
|
both synthetic carriers), and `forged` rows embedding message B with the
|
|
scheme-correct expected verdict. Zero expected-result mismatches in both
|
|
recorded runs.
|
|
|
|
## Pins
|
|
|
|
Study script
|
|
`sha256:3985522c862ac44a0c7ef8aa02946a11701439a94d67f5867f00819f638c032b`,
|
|
case rows
|
|
`sha256:03d446c08a5daed623a4be2980e097bb8bbc0cd7a6d3ab3f931d25aa58570a16`.
|
|
Oracles and checkpoints are pinned as in the kernel document.
|
|
|
|
## Follow-ups
|
|
|
|
- A keyed-scheme perspective needs a scheme with actual key control; WmForger
|
|
and WMCopier diffusion-attack baselines remain the map's heavyweight item.
|
|
- A general remover-trace detector would need features beyond one band ratio
|
|
and validation against removals this project did not produce.
|