mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-09 23:50:40 +02:00
Make the video SynthID operating point measurable and hard to move silently
The shipped profile was certified by one oracle row, but only noise_std was pinned: long_side and fps -- two thirds of what the verifier was actually shown -- could move with a green suite. The test now derives the pin from data/evaluations/video-synthid-oracle.csv, so a default without a certifying row fails. The certified profile is a perturbation-to-signal ratio, not a bare noise_std. sd-vae-ft-mse publishes no scaling_factor key, so 0.18215 comes from the AutoencoderKL class default under an upper-unbounded diffusers pin. The loader now gates that value, carries it on VideoVaeRuntime, and passes it into encode and decode so the validated value is the applied value. video_synthid_sweep.py loads through the same function: the harness producing the certified rows was the one path exempt from the gate it exists to feed. psnr_db is measured against the already-resized frame and before the encoder, so it cannot see the downscale, the decimation, or the codec, and no in-loop metric can. scripts/video_fidelity_probe.py scores the delivered file end to end, streaming the way the engine does and sharing its frame-selection rule rather than copying it -- a frame-count check cannot catch a rule that reorders frames without changing how many. The manifest gains source geometry, vae, track, verbatim verdict and session fields. The two 2026-07-31 rows keep them empty: they were never recorded and are not recoverable. Verdicts now have four states, because the verifier's unclear reading logged as not_detected is the silent regression the manifest exists to prevent. docs/video-synthid-quality-research.md records the research behind this: the noise axis is worth about 2 dB and is nearly exhausted, resolution is the real prize but is an uncertified destruction axis rather than a free win, and every proposed autoencoder swap was refuted. First local measurements included. Verified: engine output is byte-identical before and after the refactor on a locally built clip, at noise_std 0.00 and 0.15. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
f481e6f944
commit
8fe0b0110f
@@ -398,6 +398,14 @@ verifier verdict blank:
|
||||
uv run --extra video --extra diffusion python scripts/video_synthid_sweep.py input.mp4 -o sweep/
|
||||
```
|
||||
|
||||
To score what an output actually cost, use `scripts/video_fidelity_probe.py`:
|
||||
the engine's own PSNR is measured before the resize and the encode, so only the
|
||||
probe sees the delivered picture.
|
||||
|
||||
```bash
|
||||
uv run --extra video python scripts/video_fidelity_probe.py input.mp4 input_clean.mp4
|
||||
```
|
||||
|
||||
The control must still be SynthID-positive before a negative candidate can
|
||||
count as removal evidence. In the 2026-07-29 two-clip calibration, both matched
|
||||
controls were positive in Gemini's built-in SynthID verifier; the stronger
|
||||
|
||||
Reference in New Issue
Block a user