Files
remove-ai-watermarks/.claude/rules/development.md
T
Victor KuznetsovandClaude Opus 5 8fe0b0110f Make the video SynthID operating point measurable and hard to move silently
The shipped profile was certified by one oracle row, but only noise_std was
pinned: long_side and fps -- two thirds of what the verifier was actually shown
-- could move with a green suite. The test now derives the pin from
data/evaluations/video-synthid-oracle.csv, so a default without a certifying row
fails.

The certified profile is a perturbation-to-signal ratio, not a bare noise_std.
sd-vae-ft-mse publishes no scaling_factor key, so 0.18215 comes from the
AutoencoderKL class default under an upper-unbounded diffusers pin. The loader
now gates that value, carries it on VideoVaeRuntime, and passes it into encode
and decode so the validated value is the applied value. video_synthid_sweep.py
loads through the same function: the harness producing the certified rows was
the one path exempt from the gate it exists to feed.

psnr_db is measured against the already-resized frame and before the encoder, so
it cannot see the downscale, the decimation, or the codec, and no in-loop metric
can. scripts/video_fidelity_probe.py scores the delivered file end to end,
streaming the way the engine does and sharing its frame-selection rule rather
than copying it -- a frame-count check cannot catch a rule that reorders frames
without changing how many.

The manifest gains source geometry, vae, track, verbatim verdict and session
fields. The two 2026-07-31 rows keep them empty: they were never recorded and
are not recoverable. Verdicts now have four states, because the verifier's
unclear reading logged as not_detected is the silent regression the manifest
exists to prevent.

docs/video-synthid-quality-research.md records the research behind this: the
noise axis is worth about 2 dB and is nearly exhausted, resolution is the real
prize but is an uncertified destruction axis rather than a free win, and every
proposed autoencoder swap was refuted. First local measurements included.

Verified: engine output is byte-identical before and after the refactor on a
locally built clip, at noise_std 0.00 and 0.15.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 11:17:38 -07:00

7.0 KiB

globs, description
globs description
src/**/*.py
tests/**/*.py
scripts/**/*.py
pyproject.toml
uv.lock
maintain.sh
.github/workflows/*.yml
Command contracts, project gate, typing boundaries, model-adjacent test invariants, and the detection-path measurement rule.

Development invariants

Command contracts

Every single-image command declares source with dir_okay=False; batch declares its directory with file_okay=False. Keep tests/test_cli_robustness.py::TestDirectoryInputIsRejected as the regression guard.

Exit-code and no-signal behavior is a public contract. Read the command-line section of ../../docs/module-internals.md before changing it.

Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, InvisibleEngine, or WatermarkRemover -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.

device is the deliberate exception and stays a library parameter: None/"auto" detect, "cuda" pins without detecting, and everything else raises at construction. It is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds.

The same rule applies to install hints: name the extra that actually makes the command work (qwen-zimage, not diffusion), and keep the printed command shell-quoted -- bare pkg[extra] is a glob in zsh.

Local gate

Run bash maintain.sh from the repository root. The authoritative type gate is scoped to src/; full-project Pyright can exhaust Node memory on the ML dependency graph.

Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.

Model-adjacent tests

Do not classify an entire module as untestable because its main path downloads a model. Keep pure behavior covered without downloads, including:

  • target-size selection in test_invisible_engine.py;
  • unsharp and adaptive-polish helpers in test_humanizer.py;
  • tiling geometry and blending in test_tiling.py;
  • prompt-embedding cache keying, storage round-trip, and the cross-pipeline reuse that lets a stack load without its text encoder, in test_qwen_zimage_pipeline.py;
  • the face stack's dtype, in test_qwen_zimage_pipeline.py. A subclass that changes the pipeline dtype for its own global model must not change the inherited face stage's; sdxl-zimage shipped doing exactly that and crashed on every image with a face. When one profile inherits another's stage, guard the invariants that stage relies on, not just the code path.
  • the InvisibleOptions defaults, in test_api.py. When one signature promises to mirror another, compare them field by field rather than pinning the values you happen to know about, so the next field added on one side and not the other fails at the seam. Two of these defaults drifted in practice and neither needed a GPU to catch; the incident is recorded in docs/module-internals.md. Keep the comparison free of an exception table: a field that needs one is a field that belongs elsewhere, which is what force turned out to be.

A defaults comparison is not a forwarding test, and the two fail differently. Pin the VALUE at the seam, not just the name -- _run_invisible passed the whole suite with controlnet_conditioning_scale hardcoded, because nothing asserted the caller's value arrived. test_every_field_arrives_at_the_engine_with_the_caller_s_value drives the real seam with every field set off its default, so one test covers the whole bag instead of one assertion per knob.

Count the seams before believing a knob is covered. Each of force and controlnet_conditioning_scale reaches the engine through TWO paths -- remove_all versus remove_batch(mode="all") for the first, _run_invisible versus _batch_engine for the second -- and in both cases guarding one path left the other free to hardcode a constant with a green suite. The mode-parametrized guards in TestRemoveBatchLibrary::test_force_reaches_the_scrub_gate_in_every_scrubbing_mode and TestBatchCommand::test_batch_controlnet_scale_flows_to_the_cached_engine exist because that is what actually happened.

Use availability checks only for paths that actually load large models.

One measurement, one gate seam

A detector is split into a trust-level-blind scan and a verdict that applies the threshold, so detect and detect_both reach the same numbers by construction. Two rules follow, and both were broken in practice before they were written down:

  • A per-mark demotion goes in the _post_gate hook (or, for a whole-scan precondition like LibLibAI's size floor, in _scan) -- never in a detect override. An override is invisible to detect_both, so the RunningHub and Yuanbao anchor gates silently stopped applying on the arbiter's perception path. TestSinglePassPerception is the guard: it asserts detect_both equals two detect calls field for field.
  • Detection and the removal mask must read ONE sweep. The winning box travels on TextMarkDetection.match_box and the registry threads the detection into the mask builder; a mask path that re-runs its own sweep is how the two drift apart.

Before changing anything in the detection path, record the detectors' exact verdicts over a local sample first and diff them after. A refactor here is only correct if that record is byte-identical, and a green test suite does not establish that on its own.

A certified operating point is data, not a constant

The video SynthID default is only meaningful as a row in data/evaluations/video-synthid-oracle.csv, so test_shipped_defaults_match_a_certified_manifest_row derives the pin from that manifest instead of restating literals. Pinning noise_std alone had let long_side and fps -- two thirds of what the oracle was actually shown -- move with a green suite.

The certified profile is a perturbation-to-signal ratio, not a bare noise_std, so the latent scaling factor is gated in load_video_vae_runtime, carried on VideoVaeRuntime, and passed into encode and decode: the validated value and the applied value are one measurement. Anything that produces oracle evidence loads through that function. video_synthid_sweep.py hand-rolled the load and was the one path exempt from the gate it exists to feed, which is exactly backwards.

Prove a video-path refactor the same way the detection path is proven, and without needing an oracle carrier: build a clip from a tracked fixture with ffmpeg, run the engine before and after, and require an identical output sha256. Keep the generated media outside the repository.

Environment setup, dependency recovery, CI behavior, and fixture policy: ../../docs/development.md.