mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-09 23:50:40 +02:00
Victor KuznetsovandClaude Opus 5
8fe0b0110f
The shipped profile was certified by one oracle row, but only noise_std was pinned: long_side and fps -- two thirds of what the verifier was actually shown -- could move with a green suite. The test now derives the pin from data/evaluations/video-synthid-oracle.csv, so a default without a certifying row fails. The certified profile is a perturbation-to-signal ratio, not a bare noise_std. sd-vae-ft-mse publishes no scaling_factor key, so 0.18215 comes from the AutoencoderKL class default under an upper-unbounded diffusers pin. The loader now gates that value, carries it on VideoVaeRuntime, and passes it into encode and decode so the validated value is the applied value. video_synthid_sweep.py loads through the same function: the harness producing the certified rows was the one path exempt from the gate it exists to feed. psnr_db is measured against the already-resized frame and before the encoder, so it cannot see the downscale, the decimation, or the codec, and no in-loop metric can. scripts/video_fidelity_probe.py scores the delivered file end to end, streaming the way the engine does and sharing its frame-selection rule rather than copying it -- a frame-count check cannot catch a rule that reorders frames without changing how many. The manifest gains source geometry, vae, track, verbatim verdict and session fields. The two 2026-07-31 rows keep them empty: they were never recorded and are not recoverable. Verdicts now have four states, because the verifier's unclear reading logged as not_detected is the silent regression the manifest exists to prevent. docs/video-synthid-quality-research.md records the research behind this: the noise axis is worth about 2 dB and is nearly exhausted, resolution is the real prize but is an uncertified destruction axis rather than a free win, and every proposed autoencoder swap was refuted. First local measurements included. Verified: engine output is byte-identical before and after the refactor on a locally built clip, at noise_std 0.00 and 0.15. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
107 lines
7.0 KiB
Markdown
107 lines
7.0 KiB
Markdown
---
|
|
globs: ["src/**/*.py", "tests/**/*.py", "scripts/**/*.py", "pyproject.toml", "uv.lock", "maintain.sh", ".github/workflows/*.yml"]
|
|
description: Command contracts, project gate, typing boundaries, model-adjacent test invariants, and the detection-path measurement rule.
|
|
---
|
|
|
|
# Development invariants
|
|
|
|
## Command contracts
|
|
|
|
Every single-image command declares `source` with `dir_okay=False`; `batch` declares its directory with `file_okay=False`. Keep `tests/test_cli_robustness.py::TestDirectoryInputIsRejected` as the regression guard.
|
|
|
|
Exit-code and no-signal behavior is a public contract. Read the command-line section of [`../../docs/module-internals.md`](../../docs/module-internals.md) before changing it.
|
|
|
|
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
|
|
|
|
`device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. It is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds.
|
|
|
|
The same rule applies to install hints: name the extra that actually makes the command work (`qwen-zimage`, not `diffusion`), and keep the printed command shell-quoted -- bare `pkg[extra]` is a glob in zsh.
|
|
|
|
## Local gate
|
|
|
|
Run `bash maintain.sh` from the repository root. The authoritative type gate is scoped to `src/`; full-project Pyright can exhaust Node memory on the ML dependency graph.
|
|
|
|
Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.
|
|
|
|
## Model-adjacent tests
|
|
|
|
Do not classify an entire module as untestable because its main path downloads a model. Keep pure behavior covered without downloads, including:
|
|
|
|
- target-size selection in `test_invisible_engine.py`;
|
|
- unsharp and adaptive-polish helpers in `test_humanizer.py`;
|
|
- tiling geometry and blending in `test_tiling.py`;
|
|
- prompt-embedding cache keying, storage round-trip, and the cross-pipeline reuse
|
|
that lets a stack load without its text encoder, in `test_qwen_zimage_pipeline.py`;
|
|
- the face stack's dtype, in `test_qwen_zimage_pipeline.py`. A subclass that changes
|
|
the pipeline dtype for its own global model must not change the inherited face
|
|
stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a
|
|
face. When one profile inherits another's stage, guard the invariants that stage
|
|
relies on, not just the code path.
|
|
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
|
|
mirror another, compare them field by field rather than pinning the values you happen
|
|
to know about, so the next field added on one side and not the other fails at the
|
|
seam. Two of these defaults drifted in practice and neither needed a GPU to catch;
|
|
the incident is recorded in `docs/module-internals.md`. Keep the comparison free of
|
|
an exception table: a field that needs one is a field that belongs elsewhere, which
|
|
is what `force` turned out to be.
|
|
|
|
A defaults comparison is not a forwarding test, and the two fail differently. Pin the
|
|
VALUE at the seam, not just the name -- `_run_invisible` passed the whole suite with
|
|
`controlnet_conditioning_scale` hardcoded, because nothing asserted the caller's value
|
|
arrived. `test_every_field_arrives_at_the_engine_with_the_caller_s_value` drives the
|
|
real seam with every field set off its default, so one test covers the whole bag
|
|
instead of one assertion per knob.
|
|
|
|
Count the seams before believing a knob is covered. Each of `force` and
|
|
`controlnet_conditioning_scale` reaches the engine through TWO paths -- `remove_all`
|
|
versus `remove_batch(mode="all")` for the first, `_run_invisible` versus `_batch_engine`
|
|
for the second -- and in both cases guarding one path left the other free to hardcode a
|
|
constant with a green suite. The mode-parametrized guards in
|
|
`TestRemoveBatchLibrary::test_force_reaches_the_scrub_gate_in_every_scrubbing_mode` and
|
|
`TestBatchCommand::test_batch_controlnet_scale_flows_to_the_cached_engine` exist because
|
|
that is what actually happened.
|
|
|
|
Use availability checks only for paths that actually load large models.
|
|
|
|
## One measurement, one gate seam
|
|
|
|
A detector is split into a trust-level-blind scan and a verdict that applies the
|
|
threshold, so `detect` and `detect_both` reach the same numbers by construction. Two
|
|
rules follow, and both were broken in practice before they were written down:
|
|
|
|
- A per-mark demotion goes in the `_post_gate` hook (or, for a whole-scan precondition
|
|
like LibLibAI's size floor, in `_scan`) -- never in a `detect` override. An override
|
|
is invisible to `detect_both`, so the RunningHub and Yuanbao anchor gates silently
|
|
stopped applying on the arbiter's perception path. `TestSinglePassPerception` is the
|
|
guard: it asserts `detect_both` equals two `detect` calls field for field.
|
|
- Detection and the removal mask must read ONE sweep. The winning box travels on
|
|
`TextMarkDetection.match_box` and the registry threads the detection into the mask
|
|
builder; a mask path that re-runs its own sweep is how the two drift apart.
|
|
|
|
Before changing anything in the detection path, record the detectors' exact verdicts
|
|
over a local sample first and diff them after. A refactor here is only correct if that
|
|
record is byte-identical, and a green test suite does not establish that on its own.
|
|
|
|
## A certified operating point is data, not a constant
|
|
|
|
The video SynthID default is only meaningful as a row in
|
|
`data/evaluations/video-synthid-oracle.csv`, so
|
|
`test_shipped_defaults_match_a_certified_manifest_row` derives the pin from that
|
|
manifest instead of restating literals. Pinning `noise_std` alone had let `long_side`
|
|
and `fps` -- two thirds of what the oracle was actually shown -- move with a green
|
|
suite.
|
|
|
|
The certified profile is a perturbation-to-signal ratio, not a bare `noise_std`, so
|
|
the latent scaling factor is gated in `load_video_vae_runtime`, carried on
|
|
`VideoVaeRuntime`, and passed into encode and decode: the validated value and the
|
|
applied value are one measurement. Anything that produces oracle evidence loads
|
|
through that function. `video_synthid_sweep.py` hand-rolled the load and was the one
|
|
path exempt from the gate it exists to feed, which is exactly backwards.
|
|
|
|
Prove a video-path refactor the same way the detection path is proven, and without
|
|
needing an oracle carrier: build a clip from a tracked fixture with ffmpeg, run the
|
|
engine before and after, and require an identical output sha256. Keep the generated
|
|
media outside the repository.
|
|
|
|
Environment setup, dependency recovery, CI behavior, and fixture policy: [`../../docs/development.md`](../../docs/development.md).
|