The shipped profile was certified by one oracle row, but only noise_std was pinned: long_side and fps -- two thirds of what the verifier was actually shown -- could move with a green suite. The test now derives the pin from data/evaluations/video-synthid-oracle.csv, so a default without a certifying row fails. The certified profile is a perturbation-to-signal ratio, not a bare noise_std. sd-vae-ft-mse publishes no scaling_factor key, so 0.18215 comes from the AutoencoderKL class default under an upper-unbounded diffusers pin. The loader now gates that value, carries it on VideoVaeRuntime, and passes it into encode and decode so the validated value is the applied value. video_synthid_sweep.py loads through the same function: the harness producing the certified rows was the one path exempt from the gate it exists to feed. psnr_db is measured against the already-resized frame and before the encoder, so it cannot see the downscale, the decimation, or the codec, and no in-loop metric can. scripts/video_fidelity_probe.py scores the delivered file end to end, streaming the way the engine does and sharing its frame-selection rule rather than copying it -- a frame-count check cannot catch a rule that reorders frames without changing how many. The manifest gains source geometry, vae, track, verbatim verdict and session fields. The two 2026-07-31 rows keep them empty: they were never recorded and are not recoverable. Verdicts now have four states, because the verifier's unclear reading logged as not_detected is the silent regression the manifest exists to prevent. docs/video-synthid-quality-research.md records the research behind this: the noise axis is worth about 2 dB and is nearly exhausted, resolution is the real prize but is an uncertified destruction axis rather than a free win, and every proposed autoencoder swap was refuted. First local measurements included. Verified: engine output is byte-identical before and after the refactor on a locally built clip, at noise_std 0.00 and 0.15. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
7.0 KiB
globs, description
| globs | description | |||||||
|---|---|---|---|---|---|---|---|---|
|
Command contracts, project gate, typing boundaries, model-adjacent test invariants, and the detection-path measurement rule. |
Development invariants
Command contracts
Every single-image command declares source with dir_okay=False; batch declares its directory with file_okay=False. Keep tests/test_cli_robustness.py::TestDirectoryInputIsRejected as the regression guard.
Exit-code and no-signal behavior is a public contract. Read the command-line section of ../../docs/module-internals.md before changing it.
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, InvisibleEngine, or WatermarkRemover -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
device is the deliberate exception and stays a library parameter: None/"auto" detect, "cuda" pins without detecting, and everything else raises at construction. It is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds.
The same rule applies to install hints: name the extra that actually makes the command work (qwen-zimage, not diffusion), and keep the printed command shell-quoted -- bare pkg[extra] is a glob in zsh.
Local gate
Run bash maintain.sh from the repository root. The authoritative type gate is scoped to src/; full-project Pyright can exhaust Node memory on the ML dependency graph.
Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.
Model-adjacent tests
Do not classify an entire module as untestable because its main path downloads a model. Keep pure behavior covered without downloads, including:
- target-size selection in
test_invisible_engine.py; - unsharp and adaptive-polish helpers in
test_humanizer.py; - tiling geometry and blending in
test_tiling.py; - prompt-embedding cache keying, storage round-trip, and the cross-pipeline reuse
that lets a stack load without its text encoder, in
test_qwen_zimage_pipeline.py; - the face stack's dtype, in
test_qwen_zimage_pipeline.py. A subclass that changes the pipeline dtype for its own global model must not change the inherited face stage's;sdxl-zimageshipped doing exactly that and crashed on every image with a face. When one profile inherits another's stage, guard the invariants that stage relies on, not just the code path. - the
InvisibleOptionsdefaults, intest_api.py. When one signature promises to mirror another, compare them field by field rather than pinning the values you happen to know about, so the next field added on one side and not the other fails at the seam. Two of these defaults drifted in practice and neither needed a GPU to catch; the incident is recorded indocs/module-internals.md. Keep the comparison free of an exception table: a field that needs one is a field that belongs elsewhere, which is whatforceturned out to be.
A defaults comparison is not a forwarding test, and the two fail differently. Pin the
VALUE at the seam, not just the name -- _run_invisible passed the whole suite with
controlnet_conditioning_scale hardcoded, because nothing asserted the caller's value
arrived. test_every_field_arrives_at_the_engine_with_the_caller_s_value drives the
real seam with every field set off its default, so one test covers the whole bag
instead of one assertion per knob.
Count the seams before believing a knob is covered. Each of force and
controlnet_conditioning_scale reaches the engine through TWO paths -- remove_all
versus remove_batch(mode="all") for the first, _run_invisible versus _batch_engine
for the second -- and in both cases guarding one path left the other free to hardcode a
constant with a green suite. The mode-parametrized guards in
TestRemoveBatchLibrary::test_force_reaches_the_scrub_gate_in_every_scrubbing_mode and
TestBatchCommand::test_batch_controlnet_scale_flows_to_the_cached_engine exist because
that is what actually happened.
Use availability checks only for paths that actually load large models.
One measurement, one gate seam
A detector is split into a trust-level-blind scan and a verdict that applies the
threshold, so detect and detect_both reach the same numbers by construction. Two
rules follow, and both were broken in practice before they were written down:
- A per-mark demotion goes in the
_post_gatehook (or, for a whole-scan precondition like LibLibAI's size floor, in_scan) -- never in adetectoverride. An override is invisible todetect_both, so the RunningHub and Yuanbao anchor gates silently stopped applying on the arbiter's perception path.TestSinglePassPerceptionis the guard: it assertsdetect_bothequals twodetectcalls field for field. - Detection and the removal mask must read ONE sweep. The winning box travels on
TextMarkDetection.match_boxand the registry threads the detection into the mask builder; a mask path that re-runs its own sweep is how the two drift apart.
Before changing anything in the detection path, record the detectors' exact verdicts over a local sample first and diff them after. A refactor here is only correct if that record is byte-identical, and a green test suite does not establish that on its own.
A certified operating point is data, not a constant
The video SynthID default is only meaningful as a row in
data/evaluations/video-synthid-oracle.csv, so
test_shipped_defaults_match_a_certified_manifest_row derives the pin from that
manifest instead of restating literals. Pinning noise_std alone had let long_side
and fps -- two thirds of what the oracle was actually shown -- move with a green
suite.
The certified profile is a perturbation-to-signal ratio, not a bare noise_std, so
the latent scaling factor is gated in load_video_vae_runtime, carried on
VideoVaeRuntime, and passed into encode and decode: the validated value and the
applied value are one measurement. Anything that produces oracle evidence loads
through that function. video_synthid_sweep.py hand-rolled the load and was the one
path exempt from the gate it exists to feed, which is exactly backwards.
Prove a video-path refactor the same way the detection path is proven, and without needing an oracle carrier: build a clip from a tracked fixture with ffmpeg, run the engine before and after, and require an identical output sha256. Keep the generated media outside the repository.
Environment setup, dependency recovery, CI behavior, and fixture policy: ../../docs/development.md.