mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-18 15:31:06 +02:00
MiniMax signs its TC260 label with SM3withSM2 per TC260-PG-202511A. A verification-only SM3/SM2 (no dependency provides the SM2 curve) checks the appendix A.1 label message; a signature under the pinned MiniMax key is a high-confidence aigc_signature signal, a failing one is a caveat, and every result is the aigc_signature metadata marker. Pinned by the GB/T 32905 SM3 vectors, OpenSSL's SM2 verify vectors and real MiniMax files. Over 411 local files the only verdict-record change is the marker on nine MiniMax videos. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2893 lines
176 KiB
Markdown
2893 lines
176 KiB
Markdown
# Module internals
|
|
|
|
This page documents the current implementation contract. It intentionally
|
|
avoids experiment logs, corpus counts, and calibration history. Those records
|
|
live in [the verification plan](verification-plan.md) and the research archive
|
|
listed in [the documentation index](index.md).
|
|
|
|
Read the relevant section before changing a subsystem. When this page and the
|
|
code disagree, the code and its tests are authoritative and this page must be
|
|
updated in the same change.
|
|
|
|
## Architecture
|
|
|
|
The package has four main paths, plus an opt-in pixel classifier that is not
|
|
on the provenance or removal graph:
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
Input[Input file] --> Identify[Identify provenance]
|
|
Input --> Visible[Visible mark removal]
|
|
Input --> Invisible[Diffusion regeneration]
|
|
Input --> Metadata[Metadata stripping]
|
|
Input --> Classify[classify_pixels]
|
|
|
|
Identify --> Report[ProvenanceReport]
|
|
Visible --> VisibleOutput[Localized and filled image]
|
|
Invisible --> InvisibleOutput[Regenerated image]
|
|
Metadata --> MetadataOutput[Container with AI metadata removed]
|
|
Classify --> PixelClassification
|
|
```
|
|
|
|
`identify` does not call `classify_pixels`. `has_invisible_target` and `all`
|
|
do not read it.
|
|
|
|
The `all` command runs visible removal, optional invisible regeneration, and
|
|
metadata stripping in that order.
|
|
|
|
## Command line interface
|
|
|
|
[`cli.py`](../src/remove_ai_watermarks/cli.py) owns command parsing and
|
|
user-facing exit behavior.
|
|
|
|
Important contracts:
|
|
|
|
- Single-image arguments reject directories.
|
|
- `visible` writes no output when no registered mark is selected and exits with
|
|
`EXIT_NO_VISIBLE_MARK`.
|
|
- `invisible` writes no output when no supported local signal is found, unless
|
|
`--force` is supplied.
|
|
- The two no-signal conditions currently share exit code `2`.
|
|
- A quiet `metadata --check` and a successful `metadata --remove` (same for
|
|
`video metadata`) end by repeating the `identify` limit: the pixel channel is
|
|
untouched and a watermark such as SynthID has no local decoder once its
|
|
metadata proxy is gone, so neither outcome is a clean verdict.
|
|
- `visible`, `erase`, `all` and `batch` report a missing pixel stack as an
|
|
install hint naming `remove-ai-watermarks[visible]`, not as a traceback. The
|
|
default package ships without those dependencies and the Homebrew formula
|
|
installs exactly that build, so a first run following the project's own
|
|
instructions used to die on `ModuleNotFoundError: No module named 'cv2'`.
|
|
`_pixels_required` wraps the four image commands. Video pixel-processing paths
|
|
reach the same build through `video._require_video_runtime`, which names the
|
|
`video` extra, while `video metadata` and `video identify --no-visible` bypass
|
|
the pixel runtime. `remove_batch` re-raises the same ImportError instead of
|
|
counting it once per file, because one absent package is not N broken images.
|
|
All three image sites ask `optional_deps.pixels_available()`
|
|
rather than matching the exception's module name: an ImportError raised inside
|
|
`cv2/__init__.py` carries `cv2.cv2` and a numpy ABI mismatch carries `None`, so
|
|
a name match let exactly the tracebacks this guard exists for through.
|
|
- Hard processing and write failures exit with code `1`.
|
|
- `all` can still write the completed visible and metadata stages when the
|
|
diffusion dependencies are unavailable, but exits with code `1` so the
|
|
partial result is not reported as complete.
|
|
- `batch` counts per-file failures and exits nonzero if any file failed or an
|
|
applicable invisible stage was skipped because its dependencies were absent.
|
|
|
|
The batch CLI prints plain text and forwards verbose progress events directly.
|
|
Its library summary derives counts from the recorded per-file results and errors.
|
|
|
|
The decorators for diffusion options are shared by `invisible`, `all`, and
|
|
`batch`. The runtime help generated by Click is the source of truth for option
|
|
names and defaults.
|
|
|
|
`--adaptive-polish` is tri-state: it declares `default=None`, so "the user did not
|
|
choose" is a value the CLI passes through rather than a default it has to invent.
|
|
`resolve_adaptive_polish` in `watermark_profiles.py` turns that `None` into the
|
|
profile's answer (off for `qwen-zimage`, whose output already matches the input's
|
|
detail level; on for `sdxl-zimage`). The same call runs inside
|
|
`InvisibleEngine.remove_watermark`, so a library caller and a CLI caller on one
|
|
profile get the same output.
|
|
|
|
It used to read Click's parameter source in the CLI instead. That put per-profile
|
|
data in the argument-parsing layer, left the engine declaring the opposite default,
|
|
and silently lost the polish for anything supplying the flag non-interactively (an
|
|
envvar default or a wrapper calling `main()` with a defaulted list is classified
|
|
`DEFAULT`). The seed follows the same rule: the CLI does not pre-resolve it either.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_cli.py`](../tests/test_cli.py)
|
|
- [`test_cli_robustness.py`](../tests/test_cli_robustness.py)
|
|
- [`test_optional_deps.py`](../tests/test_optional_deps.py)
|
|
|
|
## High-level Python API
|
|
|
|
[`api.py`](../src/remove_ai_watermarks/api.py) provides the visible-mark entry
|
|
points:
|
|
|
|
- `remove_visible`
|
|
- `remove_visible_detailed`, returning a `VisibleRemovalResult` with per-mark
|
|
post-fill validation
|
|
- `visible_provenance`
|
|
|
|
and the image pipeline that the `all` and `batch` commands are thin wrappers
|
|
over:
|
|
|
|
- `remove_all`, returning a `RemoveAllResult` after the visible, invisible, and
|
|
metadata stages. Its legacy `visible_label` remains, while `visible_status`
|
|
and `visible_marks` retain the aggregate and per-mark post-fill result
|
|
- `remove_batch`, returning a `BatchSummary` for one directory and one mode.
|
|
`items` contains one lightweight `BatchItemResult` per source, including the
|
|
same visible status and per-mark records for visible/all modes without keeping
|
|
image arrays alive across the directory
|
|
- `InvisibleOptions`, the invisible stage's knobs as one immutable value. Engine
|
|
knobs only, under the engine's own names and defaults, so a bare
|
|
`InvisibleOptions()` behaves exactly like calling the engine with no arguments.
|
|
The engine takes them across two callables, `__init__` for what shapes the
|
|
loaded stack and `remove_watermark` for the per-image ones, so `_run_invisible`
|
|
forwards each field to the right one rather than splatting the whole bag. Two
|
|
defaults silently stopped
|
|
mirroring: `max_resolution=None` reached `_target_size`'s `max_resolution > 0`
|
|
and raised `TypeError` on every library call, and `cpu_offload=True` made a
|
|
library run slower than the identical CLI run.
|
|
`TestInvisibleOptionsMirrorTheEngine` compares the two signatures field by
|
|
field, and deliberately keeps no exception table: a field needing one is a
|
|
field that belongs elsewhere. `force` was such a field, and it decides whether
|
|
the engine runs rather than how, so it is a parameter of `remove_all` and
|
|
`remove_batch` next to `backend` and `sensitivity`
|
|
- `MetadataStripIncomplete`, raised before any write when AI metadata survives
|
|
|
|
`remove_all` reports progress as `(stage, detail)` pairs of stable tokens, not
|
|
prose the caller has to parse back. Both pipeline entry points call
|
|
`remove_auto_marks_detailed` directly; carrying validation into their result
|
|
objects adds no detector or fill pass. The original positional constructors for
|
|
`RemoveAllResult` and `BatchSummary` remain valid through defaulted appended
|
|
fields.
|
|
|
|
An invisible batch no-op replaces any existing output with the current input;
|
|
it cannot count a stale file as this run's result. A successful invisible
|
|
result names the published output even when the engine used a temporary
|
|
working file. `remove_all` carries display tags through its intermediate,
|
|
and keeps the original alpha aligned if diffusion bakes EXIF orientation
|
|
into the RGB raster.
|
|
|
|
The package root exposes all of them lazily through
|
|
[`__getattr__`](../src/remove_ai_watermarks/__init__.py), keeping a plain package
|
|
import free of the heavier image and model imports.
|
|
|
|
For path inputs, both visible-removal entry points read provenance metadata,
|
|
preserve alpha, and optionally write and strip metadata. Array inputs are
|
|
treated as BGR arrays and have no file provenance or separate alpha plane.
|
|
|
|
When no visible mark is removed, a same-format path copy preserves the original
|
|
bytes. `write_noop=False` leaves the requested output path untouched instead.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_api.py`](../tests/test_api.py)
|
|
- [`test_image_io.py`](../tests/test_image_io.py)
|
|
|
|
[`video.py`](../src/remove_ai_watermarks/video.py) provides the high-level
|
|
video entry point:
|
|
|
|
- `identify_video`
|
|
- `inspect_video_metadata`
|
|
- `remove_video_all`
|
|
- `remove_video_batch`
|
|
- `remove_video_invisible`
|
|
- `remove_video_metadata`
|
|
- `remove_video_visible`
|
|
|
|
The video API validates both the supported extension and container signature,
|
|
then delegates all metadata detection and stripping to `metadata.py`. It
|
|
requires a separate same-container output, defaulting to `<source>_clean`, so
|
|
the product path does not overwrite an original. The package root exposes all
|
|
functions lazily.
|
|
|
|
`identify_video` runs the same stable-mark selection helper as
|
|
`remove_video_visible`, so a provenance report cannot authorize a mark that the
|
|
removal path would reject. It reports an empty local result as unknown rather
|
|
than clean. A C2PA manifest alone is not an AI signal: `_video_markers_claim_ai`
|
|
requires what the image path requires, an AI digital source type or disclosure,
|
|
another AI marker (SynthID, TC260, xAI signature), or an AI-generator identity
|
|
(a registered AI signer or an AI product as claim generator). Since YouTube
|
|
re-signs every upload as Google LLC with only `opened` and `transcoded`, the
|
|
earlier any-manifest rule read a re-encoded xAI Grok video as Google AI with
|
|
SynthID (measured 2026-09-24, fixture `youtube-reencode-grok.mp4`); on the
|
|
local video corpus the change also moved an Adobe Premiere Pro export and a
|
|
Claude-provided file from AI to unknown and nothing else. Identification skips the separate per-frame timestamp probe because
|
|
it never encodes frames. `remove_video_all` is the predictable-output
|
|
composition: visible removal plus verified metadata stripping by default, with
|
|
a same-container passthrough when neither signal exists. The lossy invisible
|
|
removal stage is an explicit opt-in through the oracle-certified profile.
|
|
`remove_video_batch` applies those contracts sequentially across a top-level
|
|
directory, returns every per-file failure, and byte-copies visible no-ops so a
|
|
successful output set has no silent holes. An invisible batch loads one VAE
|
|
runtime and reuses it across every compatible file; a failed model load is
|
|
reported per file without retrying the same multi-GB initialization.
|
|
|
|
Cleanup results separate an action from a watermark verdict. The retained
|
|
`invisible_removed` boolean means only that the visual regeneration stage ran;
|
|
`visual_invisible_action` exposes `not_run` or `regenerated` without implying a
|
|
fresh oracle result. `VideoAudioStatus` records `copied_if_present` and
|
|
`unverified` for metadata, visible, invisible, all, and batch outputs. It is
|
|
static result metadata: no extra probe, decode, model, or network call runs. A
|
|
visible no-op or failed batch item with no published output reports
|
|
`stream_action=not_written` instead.
|
|
|
|
Native MP4/MOV TC260 labels follow TC260-PG-20257A:
|
|
`moov.udta.meta.keys` maps an `AIGC` key to a raw JSON value in `ilst`.
|
|
[`_internal/isobmff.py`](../src/remove_ai_watermarks/_internal/isobmff.py) walks those
|
|
nested boxes by seeking, so detection reaches a tail `moov` without reading the
|
|
preceding `mdat`. The metadata record stores each such value behind its `AIGC`
|
|
key: the bare JSON gave the record's scan parser no anchor, so three Higgsfield
|
|
MP4s with a tail `moov` read as labeled from the file and unknown from their
|
|
record until the key was kept (2026-09-24). Two Doubao iOS variants sit outside that normative placement
|
|
and are covered by the same walker (2026-08-17 corpus findings, both previously
|
|
undetected): a QuickTime-form `meta` box as a *direct* `moov` child (no FullBox
|
|
header, disambiguated by probing the child-box offset), and a QuickTime
|
|
`hdlr=mdir` metadata list under `udta.meta` whose `ilst` data items carry the
|
|
validated JSON with no `keys` box at all (content-validated, so only genuine
|
|
TC260 JSON matches; the keyless entry has no key name to blank, so its value
|
|
alone is spaced out). The MP4/MOV/M4V/M4A removal path first validates the top-level
|
|
box walk, then copies the source to a sibling temporary file in bounded chunks.
|
|
Supported C2PA/JUMBF/AI-label boxes become same-size `free` boxes with blank
|
|
payloads; TC260 removal changes the four-byte key to `free` and blanks only the
|
|
validated JSON value with same-length spaces. Keyed generation metadata such
|
|
as `workflow` and `prompt` uses the same structural walk; removal blanks the
|
|
complete key and value in place. This preserves every box size,
|
|
`stco`/`co64` offset, encoded stream byte, and source-sized memory bound.
|
|
Publication is atomic, and a malformed top-level walk is copied unchanged. A
|
|
generic `AIGC` key whose value has no TC260 field is ignored.
|
|
|
|
[`_internal/ebml.py`](../src/remove_ai_watermarks/_internal/ebml.py) provides the
|
|
corresponding bounded Matroska/WebM reader. It seeks over clusters and accepts
|
|
only a `Segment.Tags.Tag.SimpleTag` pairing `TagName=AIGC` with a JSON
|
|
`TagString` carrying a TC260 field. The existing ffmpeg stream-copy path removes
|
|
those container tags without transcoding the encoded streams.
|
|
|
|
[`_internal/riff.py`](../src/remove_ai_watermarks/_internal/riff.py) and
|
|
[`_internal/flv.py`](../src/remove_ai_watermarks/_internal/flv.py) implement the remaining
|
|
normative TC260 video placements. The RIFF walker reads only AVI
|
|
`LIST/INFO/AIGC` children. The FLV walker skips media tags and parses the AMF0
|
|
`script.onMetaData.AIGC` string. Both require a recognized TC260 JSON field and
|
|
use the verified ffmpeg stream-copy path for removal.
|
|
|
|
[`video_encoding.py`](../src/remove_ai_watermarks/video_encoding.py) owns the
|
|
ffmpeg command and pipe lifecycle shared by visible removal and invisible
|
|
regeneration. It centralizes container codecs, optional audio stream copying,
|
|
metadata/chapter policy, encode-failure reporting, and atomic same-directory
|
|
publication. Each mapped stream is allowed to reach its own end, so a copied
|
|
audio tail is not shortened to the frame-input duration.
|
|
Both the raw-BGR and timestamped-NUT stdin modes redirect ffmpeg stderr to a
|
|
temporary file while frames are written. Waiting to read diagnostics until
|
|
after stdin closed allowed stderr backpressure to stop ffmpeg's frame reads,
|
|
which in turn blocked the producer before it could close stdin. The file consumes
|
|
no pipe capacity or RAM while ffmpeg runs; completion reports a bounded head and
|
|
tail when diagnostics are unusually large. Aborts release it even when ffmpeg
|
|
has already exited. A real subprocess regression writes diagnostics beyond pipe
|
|
capacity while streaming frames, checks bounded failure reporting, and the Linux
|
|
full-clip CI job guards the complete path.
|
|
Frame encoding and source-audio copying run as two ffmpeg processes in sequence.
|
|
The streaming encoder has only the frame pipe as input, so input probing or
|
|
demux queues cannot deadlock the producer against a second input. After that
|
|
pipe reaches EOF, a finite stream-copy mux combines the encoded video with the
|
|
source audio and applies the requested metadata/chapter policy. Both stages use
|
|
sibling temporary files, and only the completed mux is published atomically.
|
|
The mux also redirects diagnostics to disk and reports only a bounded head and
|
|
tail. Command regressions assert the single-input encoder and final map targets;
|
|
failure regressions cover bounded mux diagnostics and atomic cleanup.
|
|
`probe_video_encode_profile` reads the first source video stream with ffprobe
|
|
and preserves the supported properties that survive the 8-bit BGR boundary:
|
|
`yuv420p`/`yuv422p`/`yuv444p` chroma sampling, recognized color tags, encoder
|
|
time base, MP4/MOV track timescale, source pixel format, and component depth.
|
|
Both raw-CFR and timestamped-NUT inputs use ffmpeg's passthrough FPS mode. This
|
|
keeps one encoded frame per supplied frame when an older ffmpeg receives a
|
|
fine-grained source encoder time base such as `1/90000`; implicit synchronization
|
|
can otherwise synthesize thousands of duplicate frames between CFR timestamps.
|
|
HDR transfer functions and component depths above 8 bits are rejected before
|
|
encoding so the OpenCV boundary cannot silently reduce them to SDR 8-bit.
|
|
`probe_video_timestamps` reads authoritative per-frame display PTS through
|
|
ffprobe. OpenCV timestamps are only a count-matched fallback when ffprobe is
|
|
unavailable or fails; this avoids decoder anomalies such as one spurious
|
|
negative first-frame timestamp turning a CFR clip into false VFR. A uniform sequence keeps the
|
|
cheap raw-BGR pipe unless the source starts at a non-zero PTS. A variable or
|
|
offset sequence is packetized by the lazy PyAV bridge as rawvideo in an
|
|
in-memory NUT stream with explicit PTS. System ffmpeg reads that stream with
|
|
`-fps_mode passthrough`; `-copyts` additionally retains a non-zero video start
|
|
and the corresponding copied-audio offset. No temporary frame sequence or
|
|
second video encoder is introduced.
|
|
|
|
[`video_temporal.py`](../src/remove_ai_watermarks/video_temporal.py) owns the
|
|
shared optical-flow maps and temporal residual metric. Visible removal uses
|
|
`stabilize_filled_frame` after the selected image backend: it works on a
|
|
bounded crop around adjacent masks, backward-warps the prior cleaned frame,
|
|
requires high warped-mask coverage, and gates blending on an unmasked
|
|
source-context ring. Only covered current-mask pixels change. Scene cuts,
|
|
disjoint marks, and poor motion matches therefore retain the independent
|
|
current-frame fill. The same module supplies the motion-compensated metric used
|
|
by the invisible-video sweep.
|
|
|
|
[`video_invisible.py`](../src/remove_ai_watermarks/video_invisible.py)
|
|
implements the calibrated video-pixel SynthID-removal engine. It samples frames
|
|
uniformly, resizes to a VAE-aligned geometry, encodes each frame to latent
|
|
space, applies one seeded spatial-noise field across the entire sequence, and
|
|
decodes fresh pixels. Reusing a single noise field avoids independent
|
|
frame-to-frame noise. The shipped path retains only one configured frame batch,
|
|
updates PSNR and temporal residuals incrementally, and streams BGR frames
|
|
directly to the video-only ffmpeg encoder. A separate stream-copy mux then adds
|
|
optional source audio and drops all source metadata. The result is written
|
|
through same-directory temporary files and atomically replaced only after both
|
|
stages succeed.
|
|
|
|
The engine returns PSNR and a motion-compensated temporal-residual ratio as
|
|
quality measurements. Neither is a watermark detector. The high-level result
|
|
reports completed removal without a separate verification-status flag. The companion
|
|
`scripts/video_synthid_sweep.py` imports the same engine helpers to build a
|
|
matched control and candidate grid, preventing research and shipped
|
|
regeneration paths from drifting.
|
|
|
|
The engine's `psnr_db` is measured against the already-resized frame and before
|
|
the encoder, so it scores the VAE round trip plus latent noise and cannot see
|
|
the downscale, the decimation, or the codec. No in-loop metric can: the
|
|
candidate frame is captured before it reaches the encoder pipe.
|
|
`scripts/video_fidelity_probe.py` covers the rest by decoding the delivered file
|
|
after muxing, upscaling it back to the source geometry, and scoring it against
|
|
the untouched source frames. It also reports the delivered file's bitrate, so a
|
|
fixed-crf bitrate rise cannot read as unchanged quality; because the mux copies
|
|
source audio verbatim, that figure is a container bitrate, not a video one. The
|
|
probe streams and accumulates the same way the engine does, so its peak memory
|
|
does not grow with clip length. It drives the source through the engine's own
|
|
`_iter_sampled_frames` at the source geometry rather than repeating the
|
|
selection rule: a frame-count check cannot catch a rule that reorders frames
|
|
without changing how many, so the rule itself has to be shared.
|
|
|
|
`load_video_vae_runtime` asserts the default model's latent scaling factor
|
|
against `VIDEO_SYNTHID_VAE_SCALING_FACTOR` and warns that no certified profile
|
|
exists for any other model. The published `sd-vae-ft-mse` config carries no
|
|
`scaling_factor` key, so the value is a `diffusers` class default under an
|
|
upper-unbounded pin, and the certified profile is a perturbation-to-signal ratio
|
|
rather than a bare `noise_std`. A library bump that moved that default would
|
|
otherwise rescale every perturbation with a green suite. The validated factor is
|
|
carried on `VideoVaeRuntime` and passed into encode and decode, so the gated
|
|
value and the applied value are one measurement rather than three independent
|
|
reads. `scripts/video_synthid_sweep.py` loads through the same function: the
|
|
harness that produces the certified rows is the last place that should be exempt
|
|
from the gate. The full-clip oracle floor is
|
|
`noise_std=0.15`: on the public eight-second Veo carrier, `0.10` remained
|
|
detected while `0.15` did not.
|
|
|
|
[`video_visible.py`](../src/remove_ai_watermarks/video_visible.py) implements
|
|
the first pixel stages for Sora, Veo, Seedance, Doubao, Dola, Hailuo AI, Vidu, and Kling AI. The
|
|
Sora detector searches a normalized frame with fully synthetic
|
|
mascot-and-text and mascot-only silhouettes at several scales. The mascot-only
|
|
candidate must also correlate with the complete grayscale shape, including the
|
|
paired eyes; top-hat edge correlation alone can score a Veo diamond or scene
|
|
texture above the Sora confidence floor. The shape floor is calibrated below
|
|
the independently rendered mascot and above the measured false matches.
|
|
The Veo detector uses separate
|
|
synthetic silhouettes for the current four-point diamond and legacy `Veo`
|
|
text. Seedance uses a synthetic rounded boxed-`AI` silhouette, while Dola uses
|
|
an OpenCV Hershey-font `Dola AI` silhouette. Hailuo AI uses a synthetic waveform,
|
|
MINIMAX/Hailuo AI text, separator, and ring. Vidu uses two outlined capsules
|
|
leaning into a V plus a font-rendered `Vidu AI`, sized from the measured 1080p
|
|
mark (232 x 50 px); its floors (0.52 weak, 0.58 strong) sit between the one real
|
|
clip (0.72 at 480-1080p, 0.60 at 360p) and the maximum over the 956 other local
|
|
videos (0.41), measured 2026-09-24. Kling AI combines synthetic font
|
|
and capitalization variants with a ring approximation of its swirl. The Dola and
|
|
Kling Hershey renders are frozen PNG assets drawn by OpenCV 4
|
|
(`scripts/render_video_hershey_templates.py`): OpenCV 5 draws the same `putText`
|
|
call smaller and heavier, and with runtime rendering the Dola gallery clip was
|
|
selected as Kling; the logo path rescues
|
|
wordmarks whose version or font differs, while the edge and white-label gates
|
|
reject recurring scene texture. All fixed-mark searches are bounded to the
|
|
expected lower-frame area and calibrated independently. A strong relocated Veo
|
|
diamond may bypass the known layout anchors, but weak free-corner matches never
|
|
enter the temporal arbiter.
|
|
|
|
The default `auto` route decodes each frame once, shares its grayscale and
|
|
normalized representations across all detectors, and caches resized synthetic
|
|
template features for the fixed stream geometry. Provider confidence scales
|
|
are not comparable: selection applies each provider's temporal arbiter and
|
|
takes the first stable result in specificity order (`sora`, `veo`, `seedance`,
|
|
`doubao`, `dola`, `hailuo`, `vidu`, `kling`). `vidu` precedes `kling` because the
|
|
Kling detector also scores the Vidu wordmark. An explicit mark uses the same scan path with one
|
|
candidate. Removal also collects authoritative per-frame timestamps for the
|
|
encoder, while identification omits that unused ffprobe pass.
|
|
Cross-matches between providers are settled by the provenance veto below: a
|
|
Sora run on a clip with valid Google AI-video provenance and no Sora provenance
|
|
is vetoed, so a stable Veo run selects Veo. Metadata alone never selects a
|
|
visible mark. A narrower rule that switched to Veo only when the Veo diamond sat
|
|
inside the Sora box, or Veo covered every frame and Sora did not, became
|
|
unreachable once the veto covered every mark and was removed at the 2026-09-26
|
|
merge; its credential rule survives in the veto.
|
|
For metadata-only video identification, a recognized `claim_generator` takes
|
|
precedence over certificate issuer names. Searching all marker values at once
|
|
previously labeled a Google-generated video as OpenAI when both organizations
|
|
appeared in its issuer field. Unknown generators still fall back to the
|
|
existing issuer/marker registry.
|
|
|
|
Every per-frame result is untrusted. Each provider's floors, minimum-run policy,
|
|
fill padding and mask style are one row in `VISIBLE_MARK_POLICIES`, and every mark
|
|
enters the same `stabilize_localizations` entry point; the recurrence
|
|
implementation underneath knows nothing about providers. That policy row also
|
|
carries `accepts_provenance`, which forces `provenance=False` for Kling AI.
|
|
Hailuo AI accepts an explicit MiniMax TC260 producer, and Vidu the ShengShu
|
|
USCC `91110108MACC4D63XF`. Doubao accepts its
|
|
registry-listed TC260 producer codes, including their structured USCC form;
|
|
an unrelated or absent producer does not confirm Doubao. Provenance can relax a low-contrast run only
|
|
after recurring visual evidence exists. Sora transition frames follow the
|
|
nearest confirmed moving position only with Sora provenance. Seedance, Doubao,
|
|
Dola, Hailuo AI, and Kling AI additionally require candidates to remain anchored to the
|
|
start of a run. This rejects slowly drifting scene details that still have
|
|
high frame-to-frame overlap. Technical encoder tags do not establish provider
|
|
provenance. Hailuo AI confirmation requires its explicit producer marker,
|
|
while Kling AI remains visual-only. Provenance can still veto any video mark
|
|
(`contradicts_video_provenance`): a C2PA manifest that records
|
|
`trainedAlgorithmicMedia` under an issuer outside the mark's own vendor
|
|
(`_VIDEO_MARK_ISSUER_TOKENS`), and for Kling also a TC260 producer outside the
|
|
Kling row's codes (which include the bare `kling` Higgsfield writes). The C2PA
|
|
rule came from a Veo 3.1 Lite clip whose wood texture matched Kling on every
|
|
frame at 0.61, and was widened to every mark after a Gemini Omni 1.1 Flash clip
|
|
from Runway (2026-09-25) matched Sora on a steam wisp at 0.61-0.67 for five
|
|
frames, a stable run above the strict floor. A re-signing platform such as
|
|
YouTube records no source type and vetoes nothing. An invalid credential
|
|
(`c2pa_credential_level`) vetoes nothing, and provenance that also confirms the
|
|
mark's own vendor (`confirms_video_provenance`: a `Sora` claim generator under
|
|
a Google issuer) confirms instead of vetoing. Provenance cannot veto a clip that really is Kling, so the swirl arm of
|
|
`detect_kling_frame` now passes the same `_reaches_kling_edge` test as the
|
|
wordmark arm: on Higgsfield's Cinema Studio v2 (a Kling model, TC260 producer
|
|
`kling`) the swirl matched wood grain at 0.52-0.56 with its box 11% short of the
|
|
right edge, and the run covered every frame. `detect_hailuo_frame` likewise
|
|
rejects a box with under `_HAILUO_MIN_WHITE_FRACTION` bright low-saturation
|
|
pixels: Hailuo 2.3 scored 0.33-0.35 on wood grain, above the real label's 0.31,
|
|
with no white pixel in the box (real label 1.00, synthetic example 0.15).
|
|
Neither gate stopped a Higgsfield Genjutsu clip whose wordmark arm matched wood
|
|
grain at the frame edge (0.22-0.32, white highlights included). Across 1008
|
|
local videos the per-frame Kling score was no discriminator at all: the top
|
|
clip, an OpenArt export, scored 0.81 against the real overlay's 0.63, and 68
|
|
inspected clips between 0.24 and 0.81 showed no Kling mark. What does separate
|
|
them is geometry over time: the real overlay's box is identical on every frame,
|
|
texture matches drift, so the Kling policy tightens `anchor_iou` from 0.80 to
|
|
0.95.
|
|
|
|
A video TC260 label names its platform through `_tc260_video_platform`: the
|
|
Kling, Doubao, Qwen and Wan registry codes, the Vidu constant, and the bare
|
|
`MiniMax`. The labels name the producer organization, because the Tongyi Yunqi
|
|
code signs both Wan and HappyHorse. An unregistered producer keeps the generic
|
|
"China AIGC-labeled content (TC260 standard)". The image path keeps that generic
|
|
platform for every producer and records the manufacturer only for clash checks.
|
|
|
|
Removal runs in a second decode pass. Sora, legacy Veo text, Dola text,
|
|
Seedance, Hailuo AI, and Kling AI use box masks. Seedance deliberately fills the
|
|
complete localized box: a synthetic outline mask passed repeat detection but
|
|
left part of the real translucent border visible during visual end-to-end
|
|
review. Hailuo AI expands beyond the matched core to cover both provider icons.
|
|
Kling AI expands around the wordmark or swirl to include the version and optional
|
|
`PRO` suffix. The square Veo diamond uses a synthetic shape mask so transparent
|
|
corners do not erase unrelated pixels. Every mask goes through the shared
|
|
`watermark_registry.fill` backends. ffmpeg encodes the changed video stream and
|
|
copies optional audio. The default OpenCV fill is the speed floor; structured
|
|
backgrounds need MI-GAN or LaMa for better reconstruction. Invisible video
|
|
stages must continue to reuse the image and metadata implementations rather
|
|
than copying their logic.
|
|
|
|
Apache-2.0 temporal video inpainting was measured on 2026-09-27 and is not a
|
|
backend: VACE and ROSE had higher masked frame-to-frame error than MI-GAN on
|
|
all three publication-cleared Sora, Veo, and Kling clips. VACE left a gray
|
|
block on Veo, while ROSE visibly reproduced the wordmark and made the Veo
|
|
detector accept 17/17 output frames. The model pins, mask-conditioning audit,
|
|
decoded metrics, H100 cost, and frame review are in the
|
|
[temporal video inpainting evaluation](video-inpainting-research.md).
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_video.py`](../tests/test_video.py), including a real ffmpeg full-clip
|
|
Sora/OpenCV path that generates a synthetic marked MP4 with AAC audio and
|
|
C2PA provenance, runs both `remove_video_visible` and the composed
|
|
`remove_video_all` API without mocks, and verifies complete removal, frame
|
|
count, frame rate, duration, untouched-region PSNR, paired temporal deltas
|
|
inside the filled region, byte-identical copied audio packets, source stream
|
|
properties, metadata stripping, and a large-`mdat` metadata case that rejects
|
|
any full-source `read_bytes()` call. CI installs ffmpeg explicitly for this
|
|
test so the integration gate cannot silently skip.
|
|
- [`test_video_fidelity_probe.py`](../tests/test_video_fidelity_probe.py), which
|
|
builds its clips from solid-color frames whose index is recoverable from the
|
|
pixels, so a decimation that keeps the frame count but shifts the phase scores
|
|
far worse instead of passing unnoticed. It also asserts that the probe binds the
|
|
engine's sampler rather than a copy of it, and it holds the suite's only
|
|
constraint on that sampler's phase. Nothing here needs a model, but it does need
|
|
ffmpeg, so CI runs this file in the same job that installs it.
|
|
|
|
## Metadata and provenance
|
|
|
|
### C2PA
|
|
|
|
[`_internal/c2pa.py`](../src/remove_ai_watermarks/_internal/c2pa.py) reads C2PA with the
|
|
official `c2pa-python` reader first. Its byte-level PNG parser remains a fallback
|
|
for partial and synthetic fixtures that the official reader rejects.
|
|
|
|
Structured extraction is limited to the active manifest and the ingredient
|
|
manifests reachable from it. Validation is preserved as separate dimensions:
|
|
asset binding integrity, claim signature, signer trust, and signer certificate
|
|
validity. A matching asset hash and valid claim signature do not make an
|
|
untrusted or expired signer trusted, so those stay separate fields and separate
|
|
caveats -- but they do not lower confidence. `identify` assigns high confidence
|
|
when the asset binding and the claim signature both validate, medium confidence
|
|
to a claim that validated nothing (fallback parsing, unknown dimensions), and no
|
|
origin verdict to a binding failure, a signature failure, or a revoked signing
|
|
credential. The failed claim remains in the marker inventory and keeps the removal
|
|
gate fail-safe because a post-signing container edit can invalidate C2PA without
|
|
removing a declared pixel watermark.
|
|
|
|
Revocation reaches the disqualifying branch through `signer_validity`, not through
|
|
binding or signature. A check that read only the latter two returned a confident AI
|
|
verdict off a credential the issuer had disowned, with an empty `integrity_clashes`
|
|
-- quieter than a hash mismatch on the same file. Certificate expiry is deliberately
|
|
not disqualifying: an expired certificate does not imply the signed bytes changed,
|
|
and a signature actually made outside validity already arrives as
|
|
`claimSignature.outsideValidity`.
|
|
|
|
One rule, one place. `_validation_fields` maps status codes to the four dimensions
|
|
and also emits `c2pa_failed_codes`, the subset of failures that actually drove a
|
|
dimension to invalid; `c2pa_info_has_invalid_credential` maps those dimensions to
|
|
disqualified, and both the ingredient-reachability walk and the report consume that
|
|
single path. The displayed reason comes from `c2pa_failed_codes` rather than a
|
|
substring rescan of the full code list, so what a caller is shown as the cause
|
|
cannot become a looser rule than the verdict it explains.
|
|
|
|
The structured walk also treats an exact known AI product in a reachable
|
|
`claim_generator` as an AI assertion. This covers update chains where the active
|
|
manifest names only `c2pa-tool` while a validated ingredient names Dreamina, and
|
|
Firefly chains that identify `Adobe_Firefly` without repeating a digital source
|
|
type. Unreachable manifests remain excluded.
|
|
|
|
Reachable `c2pa.soft-binding*` assertions retain their exact `alg` and bounded,
|
|
printable block `value` in addition to the normalized vendor label. A block value
|
|
without its algorithm is not surfaced because it cannot be attributed to a
|
|
decoder or registry entry. The labels and watermark/fingerprint classification
|
|
come from the generated `_internal/_generated_c2pa_soft_bindings.py` snapshot of
|
|
the official C2PA registry. Refresh it with
|
|
`uv run python scripts/sync_c2pa_soft_bindings.py`; the command validates the
|
|
upstream schema invariants, records the exact source revision and writes a
|
|
deterministic module. Runtime inspection remains offline, and registry membership
|
|
is name-only evidence rather than proof that a compatible decoder ships.
|
|
The GitHub API revision lookup sends `GITHUB_TOKEN` (or `GH_TOKEN`) when set,
|
|
because the unauthenticated 60 requests/hour per IP limit is exhausted on
|
|
shared CI runners; `.env.example` documents the variable.
|
|
|
|
`com.microsoft.invismark.1` uses its block value as the
|
|
pixel-watermark identifier in Microsoft Paint output. An InvisMark soft binding
|
|
keeps the invisible-removal gate fail-safe even when the C2PA asset binding has
|
|
since become invalid, because metadata damage does not prove the pixel carrier
|
|
disappeared. `identify` retains the generic `soft_binding` signal for schema-1
|
|
compatibility and adds `invismark` as the stable pixel-removal signal.
|
|
Registry entries of type `fingerprint` remain durable-provenance signals but are
|
|
not added to the watermark inventory, do not trigger pixel regeneration, and do
|
|
not suppress independently established SynthID watermark evidence. A registered
|
|
watermark or an unknown soft-binding algorithm keeps that inference fail-safe.
|
|
|
|
The local Paint reader treats C2PA as corroboration, not as the pixel detector.
|
|
It extracts 4-by-4 blocks from the luminance Haar LL plane, reads the largest
|
|
singular-value residue of the 15 DCT AC coefficients, and votes over the repeating
|
|
12-by-12 map. A payload is actionable only when every bit has at least three
|
|
supporting carriers and the `0x4c` prefix and GUID checksum validate. A caller can
|
|
also require the decoded Windows-order GUID to equal the one in pristine C2PA.
|
|
Direct disruption moves each confirmed carrier to the opposite residue center,
|
|
encodes to a temporary file, and re-runs the reader before atomically publishing
|
|
it. Any missing or ambiguous
|
|
condition returns no detection so a caller can retain its existing fallback. The
|
|
implementation contains no Microsoft binary, PDB data, or extracted writer-quality
|
|
forest.
|
|
|
|
The SDK default enables trust verification but supplies no production trust
|
|
anchors. Consequently, an installation without an explicitly maintained C2PA
|
|
trust bundle reports every otherwise valid signer chain as untrusted --
|
|
`signingCredential.trusted` appears in no default installation, from any vendor.
|
|
Confidence therefore must not depend on it. It did from 0.27.0 through 0.30.0, which made
|
|
the high-confidence branch unreachable in production while a hand-built fixture
|
|
stamping `signingCredential.trusted` kept it green in the suite; measured on the
|
|
committed provenance fixtures, every C2PA file from OpenAI, Adobe and Black Forest
|
|
Labs came back untrusted, and an intact manifest scored the same medium as a
|
|
fallback parse that validated nothing. That collapsed the one distinction the
|
|
official reader exists to draw.
|
|
`tests/test_identify.py::TestIdentifyRealSamples::test_no_committed_fixture_reports_a_trusted_signer`
|
|
is the reachability guard: it asserts the fixtures really are untrusted and still
|
|
reach high confidence.
|
|
|
|
Read `untrusted` here as a missing input, not a finding: nothing was checked,
|
|
because there was nothing to check against. If a maintained trust bundle is ever
|
|
configured, that stops being true and the confidence mapping in
|
|
`c2pa_credential_level` (in `_internal/c2pa.py`) must be re-read, because only then does a failed trust
|
|
check mean the signer was rejected. Shipping or fetching an official trust bundle
|
|
requires a separate update, provenance, and availability policy; do not silently
|
|
convert `signingCredential.untrusted` into trusted based on a vendor-name match.
|
|
|
|
Vendor attribution comes from the registry in
|
|
[`_internal/constants.py`](../src/remove_ai_watermarks/_internal/constants.py). Derived
|
|
issuer and platform maps should not be maintained separately.
|
|
A recognized product in `claim_generator` takes precedence over the certificate
|
|
issuer: an application can sign through an upstream model provider without
|
|
becoming that provider's product. Exact products are also useful provenance on a
|
|
non-AI edit, such as CapCut; recognizing the product does not itself change the AI
|
|
verdict. An unknown claim generator falls back to issuer attribution. An unmapped
|
|
issuer org reads as unknown-signer C2PA with no platform; that is how Ideogram was
|
|
surfaced (4 private-corpus files signed "Ideogram, Inc", 2026-08-08) before its
|
|
vendor row was added on 2026-08-27.
|
|
|
|
### Metadata scanning and stripping
|
|
|
|
[`metadata.py`](../src/remove_ai_watermarks/metadata.py) contains the shared
|
|
metadata scanners and `remove_ai_metadata`.
|
|
|
|
Key contracts:
|
|
|
|
- `scan_head` is the shared cached input for bounded byte scans. It fills the buffer
|
|
in two layers. Structural readers first, one per container, each seeking past the
|
|
pixel payload to reach metadata placed beyond the window: `isobmff.scan_c2pa_region`,
|
|
`_png_late_metadata`, `_riff_late_metadata`. A decoder-backed fallback last,
|
|
`_decoder_visible_text`, for metadata the raw bytes do not spell at all — a
|
|
zlib-compressed PNG `zTXt` packet is readable only after inflation. Compressed
|
|
text is inspected even when the whole file fits within the head window. The layers are
|
|
ordered that way because the structural readers work on files no decoder can open.
|
|
- A C2PA reader failure is logged at warning, not debug. It returns the same `None` as
|
|
a file with no manifest, so nothing downstream can distinguish "no credentials" from
|
|
"the credentials could not be read", and the second silently downgrades a verdict.
|
|
- JPEG stripping walks metadata segments and preserves the entropy-coded image
|
|
scan.
|
|
- Exact app-export JSON disclosures in EXIF `ImageDescription` or `UserComment`
|
|
share one parser. Known AI-product provenance is removable without by itself
|
|
asserting that the pixels were generated; explicit `aigc_info` discriminator
|
|
values and Dreamina `exportType=generation` do assert AI origin. Ordinary
|
|
Aweme, retouch, and `lv` editor exports are preserved.
|
|
- EXIF reads fall back to Pillow's decoded `getexif()` mapping when a container
|
|
exposes no ordinary `info["exif"]` blob. This covers ImageMagick PNG raw EXIF
|
|
profiles and keeps portable-record identification aligned with the file path.
|
|
- Structured `generationParams.modelName` JSON is reduced to its recognized
|
|
model name before attribution, so prompts and the surrounding JSON never
|
|
become a platform label.
|
|
- ISOBMFF containers use
|
|
[`_internal/isobmff.py`](../src/remove_ai_watermarks/_internal/isobmff.py).
|
|
- Native MP4/MOV TC260 `AIGC` entries are read from
|
|
`moov.udta.meta.keys/ilst` and blanked without changing box sizes.
|
|
- TC260 `SecurityData` signatures are checked by
|
|
[`_internal/tc260_signature.py`](../src/remove_ai_watermarks/_internal/tc260_signature.py)
|
|
on top of a verification-only SM3/SM2 in
|
|
[`_internal/sm2.py`](../src/remove_ai_watermarks/_internal/sm2.py), because no
|
|
dependency provides the SM2 curve. The label message is the guide's appendix
|
|
A.1 string (`"Label":"1","ContentProducer":"...","ProduceID":"..."`) under the
|
|
default SM2 user ID, and MiniMax's `LabelMataData` spelling is accepted beside
|
|
the guide's `Md`. Only the label signature is checked: MiniMax's binding
|
|
signature also verifies, but no verdict reads it, and its content hash is not
|
|
recomputed because MiniMax names no `CntSel`. The key travels in the file, so
|
|
only a key pinned in `TC260_SIGNING_KEYS` names a signer, and a pinned signer
|
|
is checked with the pinned point whatever parity the file claims; an unpinned
|
|
key that verifies proves integrity only and adds no signal. `ReservedCode2`,
|
|
the propagator's slot, is not read because no sample fills it. Verifications
|
|
are memoized, since `get_ai_metadata` and the verdict check the same label.
|
|
`tests/test_sm2.py` pins the GB/T 32905 SM3 vectors, OpenSSL's SM2 verify
|
|
vectors, and the real MiniMax signatures. Over 411 local images and videos the
|
|
only verdict-record change was the added marker on the nine MiniMax videos.
|
|
- Native MP4/MOV `workflow` and `prompt` entries use that keyed metadata-list
|
|
reader and the same offset-preserving removal path.
|
|
- Native MKV/WebM TC260 `AIGC` entries are read from
|
|
`Segment.Tags.Tag.SimpleTag` and removed through the ffmpeg stream-copy path.
|
|
- Native AVI and FLV TC260 entries are read from `LIST/INFO/AIGC` and
|
|
`script.onMetaData.AIGC`, respectively, then removed through ffmpeg stream
|
|
copying.
|
|
- Supported non-ISOBMFF audio and video containers use ffmpeg stream copying
|
|
with explicit `-map 0` to preserve every input stream. A same-directory
|
|
temporary output is published atomically, including for in-place removal.
|
|
- `keep_standard=False` removes EXIF (including GPS) and ICC profiles rather
|
|
than restoring them through the Pillow save arguments.
|
|
- The low-level remover is fail-safe and can copy an undecodable file through
|
|
unchanged.
|
|
- A caller that reports success must use `strip_and_verify`, which scans the
|
|
written output for surviving markers. If the metadata-preserving decoder
|
|
rejected the container but `image_io` can still decode its raster,
|
|
`strip_and_verify` normalizes that raster and scans again. A truly undecodable
|
|
file keeps the surviving-marker result.
|
|
|
|
Detection and removal must stay in parity. A new marker is incomplete until the
|
|
scanner can find it, the remover can reach every supported placement, and a
|
|
test proves that it no longer appears in the output.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_metadata.py`](../tests/test_metadata.py)
|
|
- [`test_metadata_internals.py`](../tests/test_metadata_internals.py)
|
|
- [`test_security_clamp.py`](../tests/test_security_clamp.py)
|
|
|
|
### Provenance report
|
|
|
|
[`identify.py`](../src/remove_ai_watermarks/identify.py) separates file-backed
|
|
metadata extraction from verdict logic:
|
|
|
|
- `extract_provenance_evidence` reads the supported metadata signals into
|
|
`ProvenanceEvidence`.
|
|
- `evidence_from_metadata_record` normalizes an externally collected nested
|
|
metadata record into the same evidence type without file access. Versioned native
|
|
records accept only source-derived fields; filenames, hashes, timings, errors,
|
|
prior verdicts, and pixel results cannot become evidence. Unknown native schema
|
|
versions and other record types are rejected.
|
|
- The vendor registries are matched over `_metadata_region(head)`, not the whole scan
|
|
buffer: they see the container's metadata and not its coded pixels. The tokens are
|
|
raw substrings and the shortest are four and five bytes, so over a megabyte of
|
|
compressed data one turns up by chance, and the entry it hits may assert AI. The
|
|
trim happens only when the container parses -- a malformed or unknown one is left
|
|
whole, because dropping real evidence to avoid a chance match is the wrong trade.
|
|
When the reader could not open the manifest, the issuer and signer scan narrows
|
|
further to the manifest store the container locates (`_c2pa_store_bytes`: JPEG
|
|
APP11, PNG `caBX`): an XMP packet names Adobe in its toolkit string, and a vivo
|
|
X300 capture was labeled "C2PA signer: Adobe" from it (2026-09-26).
|
|
- Registry tokens are matched two ways, by what the text is. Decoded manifest
|
|
strings (signer, claim generator, software agent, and the video metadata
|
|
markers) use `registry_word_matches`: the token must stand as a whole word, so
|
|
`Canva` no longer matches `Nova Canvas`. Raw bytes cannot use that rule. Over
|
|
every C2PA file in the local corpus (1,379 files, 2026-09-26), 374 of 37,325
|
|
token matches in the manifest store had a letter or digit beside them, nearly all
|
|
of it framing: a CBOR text header before the token (`iMicrosoft`, 0x69 = length
|
|
9) or a DER tag after it (`OpenAI1`, `Sora0`). The only continuation that named
|
|
the wrong vendor was `Canva|s`; `Adobe|Product` sat in an Adobe manifest. So raw
|
|
bytes decide nothing once the reader has decoded the manifest: not the signer
|
|
(an unregistered one stays unknown, where the byte scan had read the Amazon
|
|
Bedrock image as Canva), not a signer-platform token, not the AI-tool fallback
|
|
for the generator, not a raw `claim_generator` CBOR value, not a soft-binding
|
|
identifier, and not the SynthID vendor inference, in `identify` and in
|
|
`metadata.synthid_source` alike; the verdict reads SynthID from the decoded
|
|
C2PA info when the metadata dict lacks it. Claim-generator tokens also match as whole
|
|
words (`claim_generator_platform`: `suno` no longer matches `Sunoco`), and so do
|
|
the C2PA vendor needles the video path reads (`c2pa_vendor_platform`). The byte scans remain only for
|
|
manifests the reader cannot open (1 of the 1,379), and verdicts over all 1,379
|
|
were unchanged by the rule. Device tokens (`NIKON`, `Pixel Camera`, `sony.sig`)
|
|
exist only in raw bytes, so they stay a byte scan, but of the manifest store:
|
|
the same word as an EXIF Make beside an unrelated AI manifest is not a verified
|
|
capture. The source-kind scan is left on the whole buffer on purpose, because it
|
|
also reads IPTC `digitalSourceType` in XMP, a signal outside C2PA. The byte
|
|
fallback's `c2pa.ai-disclosure` kind now uses the structured path's rule
|
|
(`_disclosure_kind`): `human_validated` beside `prompt_guided` is a generation.
|
|
- `identify_from_evidence` evaluates that evidence without reopening the source. Rules
|
|
that decide a verdict live here, not in extraction: extraction has two
|
|
implementations, and a rule in only one of them is a rule the other lacks. The
|
|
SynthID provenance evidence is the worked example — its structured form comes from the manifest,
|
|
and the byte-scan fallback for containers no parser reaches runs in the verdict, so
|
|
both extractors reach the same answer. It did not, and the record path silently
|
|
reported no SynthID for images the file path flagged.
|
|
- `identify` preserves the path-based API and adds the optional registered
|
|
visible-mark and open invisible-watermark detectors after extraction.
|
|
- When the pixel stack is absent, the visible arm still no-ops (that is the
|
|
historical `get_or_none` contract), but the report now carries a caveat saying
|
|
the detectors did NOT run. Without it, an install lacking the `visible` extra
|
|
dropped `Visible Gemini sparkle` from a marked image and read exactly like a
|
|
clean scan -- a silent false negative on the one install path Homebrew
|
|
produces. The caveat names WHICH silence happened: an ImportError on a build
|
|
`pixels_available()` calls incomplete is the absent extra, and anything else is
|
|
a file that could not be decoded. The two need different fixes, so naming the
|
|
wrong one sends the user to reinstall a working install.
|
|
|
|
### SynthID periodic carrier detector (research only)
|
|
|
|
The local lattice expert is not part of the public package. Runtime code lives
|
|
in [`scripts/synthid_runtime/`](../scripts/synthid_runtime/) and the campaign
|
|
log is [`synthid-detector-research.md`](synthid-detector-research.md). The
|
|
notes below are the calibration history of that research expert.
|
|
|
|
[`synthid_detector.py`](../scripts/synthid_runtime/synthid_detector.py) is the
|
|
runtime form of the frozen 2048x2048 periodic-tile experiment. It folds a
|
|
Gaussian high-pass residual modulo 16x16 within a calibrated pixel-count range
|
|
and compares the normalized RGB tile with the bundled float64 template
|
|
`scripts/synthid_runtime/synthid_periodic_tile_2048_v1.npz`. Exact multiples use the original
|
|
reshape-and-mean path; other sizes use count-correct modulo folding,
|
|
without resize. Channels are filtered and folded sequentially, and partial edge
|
|
blocks are accumulated without a full-frame padding buffer so the 18-megapixel
|
|
ceiling does not require multiple three-channel float workspaces. The model hash
|
|
is pinned by a test, and the unchanged operating threshold is
|
|
`0.17357069773071196` through 10 megapixels.
|
|
|
|
That fixed threshold is not production-qualified. A later source-fresh
|
|
Open Images test-split challenge produced 5 crossings among the 211 images in
|
|
its supported geometry. A precision-first `0.28` replacement is frozen as a
|
|
research candidate after retaining all 12 available source-diverse native
|
|
positives and rejecting those five crossings. It then failed the second
|
|
untouched holdout at 1/213, with score `0.322542963`. The replacement is
|
|
rejected and fixed-v2 remains an explicit diagnostic only.
|
|
|
|
The direct API returns `detected`, `indeterminate`, or `unsupported`. Passing
|
|
`register_scale=False` selects fixed-v2 from 1,000,000 through 10,000,000
|
|
decoded pixels as an explicit diagnostic. Its frozen threshold accepted
|
|
none of 5,000 public COCO views balanced across every observed target geometry,
|
|
and none of a separate 5,000-view challenge over 256 generated geometries
|
|
covering every pair of modulo-16 edge remainders. The original 2048x2048
|
|
verdicts and exact scores remain unchanged. Runtime matches do not attribute a
|
|
provider. `identify` adds only positive matches as high-confidence
|
|
evidence and never turns a local negative into a clean verdict.
|
|
The result envelope also names the signal family, provider scope, backend,
|
|
whether metadata contributed to the verdict, whether pixels were preserved,
|
|
and an explicit reason for unsupported or indeterminate results. These fields
|
|
are shared with the development OpenAI oracle's JSON boundary.
|
|
|
|
The production router selects `synthid-periodic-tile-large-v1` above 10 through 18
|
|
megapixels when both dimensions are at least 2,048 pixels. It evaluates all
|
|
phase-aligned 2,048-square windows and combines the minimum fixed-template,
|
|
Red-minus-Green, and Blue-minus-Yellow spatial correlations with the most
|
|
negative Blue-minus-Yellow mid-band correlation. The 3072x5504 portrait
|
|
geometry also applies a Green mid-band alias veto. Each component is normalized
|
|
to its frozen gate and the public threshold is `1.0`.
|
|
|
|
All 37 inferred large candidates cross the rule, and all seven metadata-free,
|
|
pixel-identical candidates checked by the official Gemini verifier were
|
|
detected. The constants rejected all 17,417 exposed external controls. A
|
|
post-freeze production-path challenge then rejected all 2,637 decoded-pixel-
|
|
unique controls drawn from 2,000 COCO images excluded from the earlier large
|
|
color-phase challenge and 637 deduplicated Picsum controls. Four large
|
|
geometries and four resampling kernels were balanced; the maximum score was
|
|
`0.0592777965`. The source collections were not freshly acquired, so this is a
|
|
feature-unseen holdout rather than a fresh-source estimate.
|
|
|
|
A separate post-freeze Open Images download yielded 41 completed,
|
|
decoded-pixel-unique controls after excluding incomplete `.aria2` files and all
|
|
prior Open Images hashes. The frozen production path accepted 0/41 and reached
|
|
a maximum score of `0.4083013324`. This source-fresh audit is too small to
|
|
replace the main holdout interval but checks the acquisition boundary.
|
|
|
|
The same seven official positives were then re-encoded at unchanged dimensions.
|
|
JPEG-95 and JPEG-90 each reduced detection from 7/7 native files to 0/7. The
|
|
large operating point is therefore native-pixel and lossless-copy support, not
|
|
a codec-robust claim.
|
|
|
|
Arbitrary geometry is not the same as arbitrary spatial resampling. On a
|
|
stratified 80-image fixed-positive sample, one-step resizes at seven nonidentity
|
|
scales from 0.5 through 1.5 reduced the unchanged 16x16 detector from 80 accepted
|
|
sources to zero at every scale. Restoring the original geometry recovered 58-80
|
|
sources, showing that the carrier period scaled with the pixels. A discovery
|
|
bank that scaled the template to integer periods 8, 10, 12, 14, 18, 20, and 24
|
|
was promising: a threshold frozen above 3,000 resized COCO controls accepted no
|
|
view in a 2,000-control final partition and accepted 672 of 800 source-disjoint
|
|
provider positives. It also stayed below threshold on the tracked OpenAI and
|
|
Adobe controls. The branch remains research-only because noninteger periods at
|
|
scales 0.8, 0.9, 1.1, 1.2, and 1.333 collapsed, while separate per-period
|
|
thresholds accepted five final controls. The runtime therefore keeps only the
|
|
fixed 16-pixel lattice.
|
|
|
|
A follow-up fractional-period probe sampled the 30 strongest template harmonics
|
|
over a continuous 7.5-24.5 period range. The correct period appeared within
|
|
0.05 pixels among the top three candidates for 58 of 60 transformed positives.
|
|
Testing nine neighboring reconstructed geometries then recovered 44 of 60 at
|
|
the native threshold, compared with an upper bound of 48 when the true source
|
|
geometry was supplied. The complete search still failed its small frozen
|
|
control split: a threshold above 250 development controls accepted two of 150
|
|
final controls. Multiplying the canonical score by spectral-period confidence
|
|
also accepted two. That baseline was rejected rather than shipped at its
|
|
discovery threshold.
|
|
|
|
The research-only
|
|
[`synthid_affine_lattice_probe.py`](../scripts/synthid_affine_lattice_probe.py)
|
|
adds split-confirm synchronization. It estimates complex harmonic coherence on
|
|
one checkerboard of patches, confirms the selected period on the other, and
|
|
reports amplitude-aware confirmation, a locally content-whitened multichannel
|
|
code match, and phase-preserving and cyclically registered template scores. A
|
|
0.1-pixel grid recovered the expected
|
|
12.8, 14.4, 17.6, and 19.2 periods in 20/20 transformed views from five official
|
|
positive parents. A provisional conjunction accepted none of 800 corresponding
|
|
views from 200 oracle-negative parents, none of 469 OpenAI-labeled rows, and
|
|
three of 276 broad non-Google rows whose TC260 or Samsung provenance prevents
|
|
treating them as clean oracle negatives. This probe is not runtime routing:
|
|
positive diversity is still inadequate, period 8 remains rejected, and the
|
|
amplitude stage currently handles isotropic scale at zero rotation only. Full
|
|
protocol and caveats are in the detector research plan.
|
|
|
|
The whitened match uses neighboring noncarrier bins to estimate complex
|
|
Green/opponent-color covariance around every selected harmonic. It corrected a
|
|
period-24 alias on one native official positive, giving the broad native search
|
|
the correct period 16 on all five ordinary-size positives. It was retained only
|
|
as a candidate reranker: its positive-to-negative margin was smaller than the
|
|
existing spatial-template margin in both the native pilot and a locked 0.8
|
|
resize challenge. Two of the three TC260/Samsung-provenance challenge crossings
|
|
also matched the full whitened code more strongly than the weakest official
|
|
large positive, so the pixel result identifies a compatible signal family, not
|
|
a provider.
|
|
|
|
The same probe also exposes a payload-agnostic H5 confirmation. It estimates a
|
|
complex harmonic vector independently on the two checkerboards and measures
|
|
their fixed relative-phase inner product; the codeword need not match the known
|
|
template. Free cyclic-shift selection was rejected because natural phase aliases
|
|
overlapped the controls. The fixed all-harmonic statistic separated five native
|
|
official positives from 130 controls with margin `0.1619`, and five 0.8-resized
|
|
views from 200 fresh-parent controls with margin `0.1541`; all seven large
|
|
official positives also passed the observed gap. It remains research-only
|
|
because these tests supplied the period, reuse one negative source family, and
|
|
contain only 12 independent positive parents.
|
|
|
|
The separate research-only
|
|
[`synthid_cyclostationary_probe.py`](../scripts/synthid_cyclostationary_probe.py)
|
|
measures full complex cross-channel spectral correlation at carrier shifts
|
|
against neighboring-shift same-image nulls. It recognizes synthetic amplitude
|
|
modulation and rejects both independent noise and a purely additive carrier.
|
|
The real native pilot overlapped completely: five official-positive joint
|
|
contrasts ranged from `-0.0027` to `0.0337`, while 130 controls reached `0.0593`.
|
|
The cyclostationary branch is rejected rather than tuned on new frequency bands.
|
|
|
|
The affine-lattice research CLI also supports symmetric in-memory JPEG and
|
|
top-left crop challenges. At fixed native period 16, codec-specific
|
|
phase-preserving template thresholds retained all five repeated positive views
|
|
and accepted none of 80 locked controls at JPEG qualities 95, 90, and 80. The
|
|
locked margins declined from `0.1054` to `0.0257`; whitened and unknown-codeword
|
|
scores were rejected for lossy JPEG. A separate registered crop conjunction
|
|
retained all five views and accepted none of 80 controls after both 5% and 10%
|
|
top-left crops, with registered-template margins above `0.22`. These are
|
|
research pilots, not runtime experts: the period was supplied, positive parents
|
|
were reused, and only one codec implementation and one negative family were
|
|
tested.
|
|
|
|
The `register_scale=True` path is the calibrated ordinary-size production
|
|
expert. It
|
|
keeps the harmonic search, adds three-level scale-space consistency and quadrant
|
|
agreement. Its calibrated decision requires the canonical winner to be the
|
|
strongest spectral-period candidate, its three-way amplitude to cross the
|
|
threshold for that period bin, and both independent high-frequency template
|
|
bands to correlate by at least `0.075`. The public registered score is the
|
|
minimum normalized margin across those gates, so its decision threshold is
|
|
`1.0`.
|
|
|
|
The earlier single-threshold version produced 68 false positives among 11,273
|
|
symmetrically downscaled private-corpus controls. A top-candidate plus period-bin version
|
|
removed those but still produced 6 false positives among 1,000 source-independent
|
|
Open Images controls. The high-band gate removed them and then accepted none of
|
|
499 previously untouched Open Images reserve controls. The resulting rule
|
|
retained 229 of 355 source-disjoint transformed positives: 0/65 at scale 0.5 and
|
|
229/290 from scale 0.65 through 1.5. The explicit period-8 rescue is rejected
|
|
because resize lattices fully overlap its positive distribution.
|
|
|
|
A subsequently frozen 1,000-image Open Images reserve accepted zero in
|
|
registered mode. The fixed expert supported only 81 of those geometries and
|
|
accepted seven, so fixed and registered results cannot safely be unioned. In
|
|
overlapping geometry the registered decision remains the validated path;
|
|
fixed-only evidence is a diagnostic rather than a universal-cascade positive.
|
|
|
|
A later source-fresh 3,000-image Open Images test-split challenge superseded the
|
|
registered-v2 precision claim: it accepted 4 of 2,997 supported controls. The
|
|
registered-v3 confirmation uses disjoint patch groups for phase, cyclic
|
|
amplitude, and held-out-codeword evidence. It rejected all four crossings,
|
|
retained 147 of 148 base-accepted source-diverse positives, and retained all 359
|
|
base-accepted views in a dense 0.65-1.50 transform matrix over 12 independent
|
|
parents. Frozen unchanged, it then accepted 0/2,996 controls from a second
|
|
nonoverlapping Open Images cohort and 0/2,366 supported controls from a
|
|
3,000-image COCO second-family challenge. Registered-v3 is now the default
|
|
ordinary-size positive route. The exact gates, acquisition hashes, and rejected
|
|
weak-signal rescue are recorded in the detector research plan.
|
|
|
|
Every one of those control rates is photographic. Against 223 corpus images
|
|
whose C2PA names a non-Google generator, the unchanged entry point accepted 29
|
|
(`0.130`, Adobe Firefly `0.241`, highest foreign score `3.01`), all from
|
|
registered-v3. The branch reads a lattice shared across generation pipelines,
|
|
which is why it must not be reported as a watermark. The public `identify`
|
|
path no longer calls this expert.
|
|
|
|
The branch is also phase-locked to the image origin, exactly like the large
|
|
expert. A two-pixel diagonal crop killed all 28 in-geometry foreign detections
|
|
and all 8 detected Google provenance positives (maximum remaining scores
|
|
`0.779` and `0.311` against the `1.0` threshold); the signal recovers only at
|
|
offsets that are multiples of four. Registered-v3 therefore detects the same
|
|
crop-destroyed generation-pipeline lattice as `large-v1`, on ordinary sizes.
|
|
|
|
When registered-v3 abstains, opponent-registered-v1 searches the same frozen
|
|
template in Red-minus-Green and Blue-minus-Yellow space. It reranks three
|
|
separated scale candidates with fixed RGB and two spatial opponent-color gates,
|
|
then accepts only periods 7.9-12.0 on 1-10 megapixel rasters whose sides are at
|
|
least 768 pixels. Period-8 candidates additionally require Red-Green and
|
|
Blue-Yellow 8-pixel edge ratios no greater than `1.05`; this vetoes the
|
|
deterministic JPEG block lattice without using container metadata. The final
|
|
rule recovered 49/49 lossless 0.5x-0.75x views from seven official-positive
|
|
parents. It rejected all 1,790 measured period-8 codec crossings, while 350
|
|
identically resized controls had no base crossing and the earlier period-band
|
|
rule accepted 0/1,000 post-freeze Picsum controls. Period 12.8 remains excluded,
|
|
and lossy JPEG/WebP views remain inconclusive.
|
|
|
|
The runtime precedence is registered-v3, the bounded opponent fallback, then
|
|
large-v1 above 10 megapixels. Passing `register_scale=False` selects the legacy
|
|
fixed diagnostic explicitly. The research bank in
|
|
`scripts/synthid_routed_expert_bank.py` preserves five explicit expert
|
|
identities for audits: fixed, registered-v3, large, opponent-registered-v1,
|
|
and opponent-fine-registered-v1. An inactive fallback is unsupported, not a
|
|
new independent score: two runtime calls populate the five slots. Schema-2
|
|
exports retain the actual routed identity; legacy schema-1 registered
|
|
observations must be rescored, and conformal calibration must match the new
|
|
expert set. Each expert uses
|
|
its own threshold, including 1.05 for the fine fallback.
|
|
Neither runtime nor research routing returns a clean-image verdict.
|
|
|
|
An unchanged registered challenge from 10 to 18 megapixels retained only 1 of
|
|
37 Google candidates and zero of 89 non-Google controls. Twenty-eight positives
|
|
cleared amplitude, 21 had matching spatial and spectral periods, but only three
|
|
cleared high-band agreement. The 10-megapixel ceiling therefore remains.
|
|
Phase-aligned 2,048-square fixed windows did not provide a fallback: median
|
|
consensus retained 36 positives and accepted 10 controls, while all-window
|
|
consensus retained 36 and accepted eight.
|
|
One frozen full-frame pre-resize to eight megapixels also retained only the same
|
|
1 of 37 positives and zero controls; just three positives cleared high-band
|
|
agreement. Large images therefore cannot be routed through a canonical-size
|
|
registered fallback; the later native opponent-color branch is separate.
|
|
The remaining phase-aligned window variants closed this branch: a single center
|
|
2,048-square registered window retained 1 of 37 positives and zero of 89
|
|
controls, while accepting any phase-aligned 2,048-square window retained 2 of
|
|
37 and zero controls. The latter control maximum was already 0.968 against the
|
|
1.0 decision threshold. Neither the coverage nor the exposed specificity
|
|
margin supports a registered-window expert; these results do not apply to the
|
|
later native opponent-color branch.
|
|
|
|
A separate half-scale patch-consensus branch initially looked promising. Its
|
|
64-pixel, 90th-percentile patch statistic retained 33 of 49 validation positives
|
|
and zero of 166 controls, then 27 of 52 locked-test positives and zero of 140
|
|
controls. The frozen broad challenge rejected it: 61 of 10,906 symmetrically
|
|
half-scaled private-corpus controls and 22 of 1,000 independent Open Images controls
|
|
crossed the unchanged threshold. A post-challenge Green negative-phase spread
|
|
gate removed those errors but retained only 2 of 234 later Google rows and zero
|
|
of 689 later controls. Half-scale support therefore remains explicitly absent.
|
|
|
|
An independent reproduction of the public `aloshdenny/reverse-SynthID` V4
|
|
cross-color codebook did not provide another expert. Its best-of-two-model phase
|
|
rule retained 141 of 355 positives but accepted 191 of 499 controls and 386 of
|
|
a fresh 1,000-control reserve; AUC was 0.517. The repository's stricter
|
|
aspect-ratio routing reduced coverage but preserved the same approximately 38%
|
|
positive and control acceptance within supported rows. Its older V3 phase score
|
|
had AUC 0.473, and two prespecified amplitude-aware V4 rerankings reached only
|
|
0.521 and 0.524 AUC. These external rules are research baselines only. Their useful
|
|
contribution is the solid-reference phase-consensus construction, not either
|
|
published single-image threshold. Full methodology and the DALL-E reference-set
|
|
confound are recorded in the detector research plan.
|
|
A later exact repeat on the hash-disjoint v7 challenge confirmed the same
|
|
failure: the published cut accepted 77/200 Google and 76/200 OpenAI, so it did
|
|
not separate the two target providers.
|
|
A direct exact-1024 reproduction of the same V4 artifact confirmed rather than
|
|
rescued that verdict: the union of its two published profiles accepted 177 of
|
|
443 Google rows and 60 of 162 controls. The artifact's floor left only about two
|
|
active bins in the inspected profile/channel, explaining why its score acts
|
|
as a weak generic phase-coherence test rather than a specific codebook match.
|
|
|
|
An exact-1086x1448 OpenAI follow-up also rejected the solid-reference transfer.
|
|
A low-texture 256-bin phase codebook achieved native and JPEG-95 test AUCs of
|
|
0.512, and independent train halves shared only two exact bins. The paper's
|
|
content-dependent alternative was tested with 60 OKLab masking-alignment
|
|
features. Its native/JPEG minimum reached test AUC 0.593 and 15/562 positives
|
|
versus 1/228 controls at the frozen boundary, but deterministic half-fits shared
|
|
no positive decision and had coefficient cosine 0.148. Neither branch is a
|
|
runtime OpenAI expert.
|
|
|
|
An exact-1254 assertion-contrast search selected an sRGB period-8 Blue-channel
|
|
direction at 45 of 259 validation asserted rows and zero of 16 same- or
|
|
other-provider validation rows. It failed its locked specificity check at 39 of
|
|
233 asserted rows and one of 18 controls. Across the complete exact-geometry
|
|
audit it accepted 185 of 992 asserted rows, one of 55 same-provider rows without
|
|
an assertion, and one of 13 other-provider rows. Raising the threshold above
|
|
the wider controls reduced recall to a small minority, so this branch remains
|
|
research-only.
|
|
|
|
Three attempted veto or missing-codeword rescues also failed. The official
|
|
InvisMark decoder checkpoint passed its own embedded-watermark self-test at
|
|
0.997 confidence and 0.97 bit accuracy, but all OpenAI and provider-control
|
|
cohorts clustered around 0.18 confidence with no useful separation. DTCWT
|
|
modulus before periodic folding retained 5 of 158 locked-test OpenAI rows and
|
|
zero of 135 controls, then zero of 40 fresh OpenAI rows. A 16-codeword whitened
|
|
period-8 model retained 5 of 158 and zero controls, then 1 of 40 fresh OpenAI
|
|
rows and 1 of 50 fresh controls. They neither explain the known oracle-positive
|
|
miss nor justify runtime routing.
|
|
|
|
An image-level patch-bag model over nine normalized high-pass patches was the
|
|
strongest later local OpenAI classifier. It accepted 324/1,364 development-test
|
|
OpenAI rows with 4/3,373 heterogeneous controls, then transferred to 120/486
|
|
OpenAI rows, 8/504 general controls, and 5/33 same-provider non-assertion rows
|
|
in the later temporal challenge. Assertion enrichment over the same-provider
|
|
stratum was not independently significant (`p = 0.151`, one-sided exact test).
|
|
JPEG-95 reduced the unchanged conjunction to
|
|
1/486 positives and zero controls. The native temporal hits were complementary
|
|
to the signed period-8 hits, but the same-provider rate and codec collapse
|
|
identify another export noiseprint rather than a runtime SynthID expert.
|
|
|
|
The separate OpenAI period-8 DTCWT component is persistent rather than tied to
|
|
one short rollout: exact-generator asserted hits were 3/16, 46/365, and 26/200
|
|
from May through July, with 75/581 overall versus 1/52 same-generator rows
|
|
without an assertion. Its native/JPEG minimum score reached 0.721 AUC between
|
|
those indeterminate strata. Sorting all 64 cyclic correlation scores removed
|
|
absolute phase but also removed locked-test discrimination at 0/158 positives
|
|
and 0/135 controls. This remains research evidence for a weak signed carrier,
|
|
not a runtime OpenAI detector.
|
|
|
|
A four-family open-proxy challenge also failed to justify a generic neural
|
|
watermark expert. A fixed residual frontend and cross-family residual mixing
|
|
were trained on three of TrustMark P, VideoSeal, DWT-DCT, and WAM while the
|
|
fourth encoder and its test sources remained unseen. Held-out AUCs ranged from
|
|
0.437 to 0.562. Equal-power phase-scrambled hard negatives prevented simple
|
|
spectral-energy shortcuts, but did not produce architecture transfer. A
|
|
separate translation-invariant Gemini bicoherence search selected none of 20
|
|
development positives and finished at 0/50 positives, 0/199 controls, and AUC
|
|
0.374. Neither branch is part of runtime routing; full split and oracle details
|
|
are in the detector research plan.
|
|
|
|
The separately measured registered geometry range remains 250,000 through
|
|
10,000,000 decoded pixels with both sides at least 256 pixels. The default path
|
|
in the development-only research runtime uses registered-v3, then the narrower opponent-registered-v1
|
|
fallback in its 1-10 megapixel domain, and large-v1 above 10 megapixels. A
|
|
20-image real-corpus drift check was byte-identical after the earlier v2 integration.
|
|
The calibration history and caveats are in the linked detector research plan.
|
|
The separate `scripts/synthid_affine_lattice_probe.py` now emits schema 14
|
|
with `period_selection="selection-only-whitened-amplitude-v1"`. Period
|
|
selection cannot read confirmation patches; schema 13 and earlier multi-period
|
|
probe results need rerunning and recalibration. This does not change the
|
|
frozen runtime detector's independent implementation or thresholds. Learned
|
|
folds may include partial edge tiles when both raster dimensions support at
|
|
least one full tile. A residual-correlation comparison requires compatible
|
|
geometry; a mismatch is an error rather than measured attenuation.
|
|
External `not_detected` controls require either the matching target verifier
|
|
or an explicit independent `evidence_reference`.
|
|
|
|
### Development OpenAI SynthID oracle
|
|
|
|
[`openai_provenance.py`](../src/remove_ai_watermarks/openai_provenance.py)
|
|
provides a development-only remote oracle. It is absent from the installed CLI
|
|
and top-level Python API, and `identify` never imports it. Development
|
|
dependencies include the optional OpenAI SDK.
|
|
|
|
The backend accepts only PNG, JPEG, and WebP. It computes a decoded RGBA pixel
|
|
fingerprint, removes AI provenance metadata into a temporary file through
|
|
`metadata.strip_and_verify`, recomputes the fingerprint, and aborts before any
|
|
request if metadata survived, the format changed, the pixels changed, or the
|
|
sanitized file exceeds the endpoint's 50 MiB limit. It then sends exactly one
|
|
multipart file to `content_provenance_checks.create` and parses exactly one
|
|
`type == "synthid"` result. The independent C2PA entry is never returned or
|
|
used as fallback evidence. Missing, duplicate, or unknown SynthID outcomes are
|
|
errors rather than negative detections.
|
|
|
|
The default SDK client has a 120-second request timeout, zero automatic retries,
|
|
and an HTTP transport with `trust_env=False`, so process proxy variables cannot
|
|
reroute it. A caller may provide one explicit API key without mutating the process
|
|
environment. One upload acknowledgement therefore authorizes at most one media
|
|
transmission rather than inheriting the SDK's retry default. Request logs keep the
|
|
endpoint, temporary basename, media type, byte count, timeout, retry policy,
|
|
duration, HTTP status, error code, and request id when available, but omit the
|
|
source path, image bytes, credentials, and decoded-pixel fingerprint.
|
|
|
|
The internal `OpenAIProvenanceError` preserves the status, API error code, request id,
|
|
`Retry-After` value, and a transient-only `retryable` flag. The library does not
|
|
automatically act on that flag: an explicit caller invocation is required for
|
|
every additional upload. Transport and schema failures remain errors rather
|
|
than becoming `not_detected` or a local detector result.
|
|
|
|
The development result remains provider-scoped and positive-evidence-only. `not_detected`
|
|
does not mean human-created, and the official endpoint's published prohibition
|
|
on repeated reverse-engineering or evasion queries prevents using this backend
|
|
as an adaptive training or removal oracle.
|
|
|
|
### Portable metadata record
|
|
|
|
[`metadata_record.py`](../src/remove_ai_watermarks/metadata_record.py) produces the
|
|
record `evidence_from_metadata_record` consumes, so collection and verdict can run
|
|
on different machines. Its contract is equality with the file path, and the three
|
|
defects found while establishing that equality are the reason each rule exists:
|
|
|
|
- It walks the file's RAW head, never the `scan_head` buffer. That buffer is the
|
|
head concatenated with late metadata payloads, so a structural walk runs off the
|
|
end of the real head and parses appended bytes as chunks, inflating the record and
|
|
creating false signals.
|
|
- Samsung Galaxy AI splits its evidence: the `PhotoEditor_Re_Edit_Data` marker sits
|
|
in the post-EOI trailer while the `genAIType` value it is gated on can sit inside
|
|
the entropy-coded scan. A marked file therefore keeps the whole tail window, not
|
|
just the trailer.
|
|
- PIL's info keys are emitted in the file path's own candidate order
|
|
(`Software`, `Source`, `Title`, `Description`, then EXIF). `generator_from_metadata`
|
|
returns the FIRST candidate carrying a known token, so preserving candidate order
|
|
is part of verdict equivalence.
|
|
|
|
The transport is independently versioned as `provenance_metadata` schema 1. Native
|
|
records require the exact integer schema version and a `complete` status. Source
|
|
or head-read failures produce `error`; later region, trailer, decoder, or C2PA
|
|
read failures retain collected evidence with `partial` and stage-specific
|
|
`issues`. Neither incomplete status can be judged as a complete scan. WebP walks the full
|
|
declared RIFF container by seeking over `VP8`, `VP8L`, `ALPH`, and `ANMF`, so late
|
|
XMP/C2PA remains visible without shipping coded frames or parsing appended trailer
|
|
bytes as chunks.
|
|
|
|
Pixel forensics are deliberately absent: the provenance path does not read them.
|
|
Verdict equivalence is checked over tracked fixtures and a separate local evaluation
|
|
corpus.
|
|
|
|
### Broad forensic metadata
|
|
|
|
[`forensic_metadata.py`](../src/remove_ai_watermarks/forensic_metadata.py) owns the
|
|
wide metadata-only inspection record: hashes and timestamps, full EXIF/IPTC, C2PA,
|
|
container inventories, bounded binary metadata, and embedded-thumbnail forensics.
|
|
It is a separate `forensic_metadata` record type and is deliberately rejected by the
|
|
provenance normalizer. Integration code publishes the strict
|
|
`ProvenanceReport.to_dict()` alongside it rather than letting operational fields or
|
|
derived results influence detection.
|
|
|
|
### Photo pixel classifier
|
|
|
|
[`classify.py`](../src/remove_ai_watermarks/classify.py) is the 2026-08-31
|
|
photo freeze: CLIP-L-ft ridge AND freeze MLP, then 124-d focal heads only on
|
|
DEFINITELY. Since 2026-09-02 a linear receipt-document gate runs first on
|
|
that DEFINITELY path, on the same CLIP vector: a hit publishes `unknown`
|
|
(detector stays `definitely`, provider is not read, 124-d extraction is
|
|
skipped). The head ships with the model: since 2026-09-07 under the
|
|
STABLE name `receipt-gate.npz` in the Hub snapshot or
|
|
`RAIW_CLASSIFY_WEIGHTS` directory, with the legacy dated spelling
|
|
`receipt-gate-2026-09-02.npz` still readable and the package asset
|
|
`src/remove_ai_watermarks/assets/receipt-gate-2026-09-02.npz` as
|
|
the fallback for weights directories frozen before the gate existed (the
|
|
head is fitted on the freeze CLIP-L-ft embedding space, so it versions
|
|
with the model, not with the code). Head and threshold are read from ONE
|
|
artifact: the threshold travels inside the npz and
|
|
`classify_from_scores` resolves it from the loaded gate, so a model-side
|
|
gate update needs no lib release. `RECEIPT_GATE_THRESHOLD` in
|
|
`classify.py` is only the legacy pinned value for the package fallback
|
|
and its pinned tests.
|
|
Training data and certification live in
|
|
`receipt-gate-shipped-2026-09-02/report.json` in the research tree; positives
|
|
are CORD-v2 train (800, CC BY 4.0, disjoint from the CORD test split the
|
|
eval corpus uses) plus regenerable synthetic receipts, negatives are ai_train
|
|
rows. `tests/test_classify.py` pins the asset threshold, the downgrade, and
|
|
that the gate skips forensics. The public label is `ai` / `human` /
|
|
`unknown`. POSSIBLY is `unknown`. The named class is `openai` / `google` /
|
|
`muse-image` / `bytedance` / `None`. `bytedance` is the shared ByteDance
|
|
generator lineage (Doubao and Jimeng render with one model family;
|
|
measured 83.9% on the frozen test cell). `tc260` covers the REST of
|
|
China's generator ecosystem, providers that are peers of
|
|
openai/google/meta; no mixed head can honestly name the group, so its
|
|
argmax win abstains to `None` (measured 2026-09-07: without the veto
|
|
207/379 China test rows would be falsely named openai/google/muse-image). The freeze file keys the Muse
|
|
Image head `meta_muse_image`. Tests in `tests/test_classify.py` pin the
|
|
gate without downloads and pin that `identify` does not import this
|
|
module.
|
|
|
|
Weights stay out of git. The Hub snapshot is `wiltodelta/raiw-photo-classify`.
|
|
`RAIW_CLASSIFY_WEIGHTS` overrides it. The extra is `classify`. The optional
|
|
CPU ONNX vision runtime is the `classify-onnx` extra. The runtime
|
|
narrows its snapshot download to the selected backend's model and shared heads.
|
|
`WEIGHTS_ALLOW_PATTERNS` exports the union so an offline deploy can pre-cache
|
|
both backends explicitly (`tests/test_classify.py` pins the seam). User guide:
|
|
[photo-classify.md](photo-classify.md). Hub card:
|
|
[photo-classify-hf/README.md](photo-classify-hf/README.md).
|
|
|
|
### Source-pipeline classifier
|
|
|
|
[`source_classify.py`](../src/remove_ai_watermarks/source_classify.py) is a
|
|
separate, lightweight OpenAI/Google/unknown source-export classifier. It fuses
|
|
the existing 124-dimensional patch descriptor with the 768-dimensional
|
|
phase-free spectrum in
|
|
[`_internal/source_spectral.py`](../src/remove_ai_watermarks/_internal/source_spectral.py),
|
|
then evaluates the frozen 892 -> 128 -> 64 -> 3 GELU MLP in NumPy. Provider
|
|
labels clear independent calibrated margins; every other row is `unknown`.
|
|
The NumPy decision treats values within `1e-5` of the stored margin as the
|
|
inclusive training threshold, absorbing float32 BLAS differences; this matches
|
|
the frozen training-runtime decisions on all 27,811 cached internal and
|
|
external feature rows.
|
|
|
|
The runtime pins both the immutable Hub revision and the NPZ SHA-256, loads
|
|
with `allow_pickle=False`, and rejects any field, shape, class, activation, or
|
|
non-finite-value drift. `RAIW_SOURCE_CLASSIFY_WEIGHTS` is the offline override.
|
|
The `source-classify` extra adds `pixels` and Hugging Face Hub but no Torch or
|
|
Transformers. `identify` never imports or runs this path. Its output names a
|
|
complete source/export pipeline and is not SynthID detection. Unit contracts
|
|
and the threshold mutation guard live in `tests/test_source_classify.py`;
|
|
evaluation and attack limits live in
|
|
[`synthid-classifiers.md`](synthid-classifiers.md).
|
|
|
|
`source_classify_v2.py` provides a separate, opt-in `classify_image_source`
|
|
entry point. Its NPZ schema contains only complex spectral filters, spatial
|
|
templates, and thresholds; the runtime pins the Hub revision and SHA-256,
|
|
rejects unknown fields and invalid shapes, and never loads pickles. The result
|
|
has `watermark_truth="unknown"` even when a provider pattern matches. A
|
|
simultaneous provider match abstains. `PROVIDERS` exports only the calibrated
|
|
provider labels for callers that route processing from a source hint. Tests in
|
|
`tests/test_source_classify_v2.py` cover the public seam and changed-threshold
|
|
behavior; the [model card](image-source-hf/README.md) records the frozen
|
|
aggregate evidence and limits. This does not change the earlier
|
|
`classify_source` API.
|
|
|
|
The CLIP loader suppresses discarded random parameter initialization with
|
|
Transformers' `no_init_weights` context before applying the complete frozen
|
|
state dict. Normal `CLIPModel(config)` construction initialized 427 million
|
|
parameters that the checkpoint immediately replaced. On 2026-09-10, three
|
|
fresh local CPU processes over the same synthetic 512x384 PNG fell from
|
|
9.024/7.558/7.783 seconds to 2.124/2.059/2.065 seconds. This is a process-cold,
|
|
OS-cache-warm comparison, not a hosted-container startup estimate. All 42
|
|
tracked image fixtures kept byte-identical classification records before and
|
|
after. `tests/test_clip_l_ft.py` rejects a loader that invokes random Linear
|
|
initialization and verifies that checkpoint parameters and constructor-created
|
|
buffers are fully materialized.
|
|
|
|
The optional CPU-only `backend="onnx"` path reads a static batch-one,
|
|
vision-only FP32 graph produced by `scripts/export_photo_classify_onnx.py`.
|
|
The graph is published in the pinned Hub snapshot; PyTorch remains the default.
|
|
It is 1,216,102,506 bytes versus the 1,710,697,163-byte full
|
|
PyTorch checkpoint; the removed text tower alone is 494,601,216 bytes. A
|
|
2026-09-10 stress comparison covered 500 locally available catalog photographs
|
|
closest to either Model 1 threshold plus 500 deterministic family-stratified
|
|
photographs. The minimum embedding cosine was 0.999999979, mean absolute delta
|
|
was 1.54e-7, and no detector or receipt-gate decision changed. All 42 tracked
|
|
fixtures also kept byte-identical public records.
|
|
|
|
Three alternating fresh-process CPU pairs on the same synthetic 512x384 PNG
|
|
gave PyTorch 7.037/5.235/5.003 seconds and ONNX 6.058/4.956/4.528 seconds under
|
|
the then-current machine load: ONNX won all three pairs, but its 4.956-second
|
|
median was only 5% below PyTorch's 5.235 seconds. Median peak RSS fell from
|
|
3.78 GB to 1.92 GB. These are process-cold, OS-cache-warm local results, not a
|
|
hosted-container estimate; the memory reduction is the stronger result. ONNX
|
|
Runtime graph optimization is disabled because rebuilding the already
|
|
constant-folded 1.2 GB graph increased session startup. CoreML is not selected:
|
|
its static session took 18.8 seconds to construct and its dynamic-batch probe
|
|
failed. Dynamic INT8 reduced the graph to 305,462,280 bytes but changed 3 of the
|
|
42 fixture verdicts in both directions across the public decision boundary, so
|
|
it remains rejected without retraining and a new operating point. Splitting
|
|
Model 2 does not address cold start: its detector and provider files total about
|
|
2 MB and loaded in roughly 3 ms in the same stage profile; CLIP-L Model 1
|
|
dominates.
|
|
|
|
### Pixel forensics
|
|
|
|
[`pixel_evidence.py`](../src/remove_ai_watermarks/pixel_evidence.py) measures six
|
|
families of scale-robust pixel statistics (block-DCT histograms and Benford
|
|
deviation, FFT band energies and CFA peaks, high-pass residual, error level,
|
|
gradient, color) in a single decode, sharing the intermediate maps between them.
|
|
|
|
It remains independent of verdict and removal. `PixelEvidence.to_dict()` is the
|
|
versioned service boundary: it omits the local path, exposes complete/partial/error
|
|
status, keeps exception details in logs, and can include opt-in per-stage timings.
|
|
|
|
The provenance metadata collector, broad forensic collector, provenance report,
|
|
and pixel report all accept an explicit output `schema_version`. Package releases
|
|
may add an output schema while retaining older serializers, so a rolling consumer
|
|
can keep requesting the version it already understands. Within one schema, changes
|
|
are additive; existing fields, types, meanings, signal names, and watermark labels
|
|
remain stable. Unsupported selections raise before a different shape is returned.
|
|
|
|
`artifacts=True` additionally returns the spatial layer: a perceptual hash, a 128px
|
|
JPEG thumbnail, and coarse ELA, residual and phase maps. Those identify the source
|
|
image rather than describe it, which is why they are opt-in and a separate field: a
|
|
caller storing them is handling image content, not statistics about it.
|
|
|
|
The DWT-DCT detector and the visible-mark stage share a single decode of the
|
|
source, held by
|
|
a per-call `_SharedDecode`. It exposes two accessors because the two arms need
|
|
opposite failure handling: the visible arm swallows a decode failure (no cv2, no
|
|
visible marks, metadata verdict untouched), while the invisible arm re-raises it
|
|
so `has_invisible_target` reaches its documented fail-safe `True`. Swallowing it
|
|
there would skip a diffusion scrub on a file that used to get one. TrustMark
|
|
deliberately keeps its own Pillow decode: cv2 and Pillow disagree on EXIF
|
|
orientation and on 16-bit PNG, so substituting one for the other is not
|
|
behavior-preserving.
|
|
|
|
The metadata probes (`aigc_label`, `xai_signature`, `iptc_ai_system`,
|
|
`huggingface_job`, `samsung_genai`) and `extract_c2pa_info` are memoized on
|
|
`(path, mtime_ns, size)`. One `identify` reaches each of them twice, and each
|
|
re-walks the container or re-runs the manifest reader. Size is in the key as well
|
|
as mtime because this package rewrites files in place, and an in-place rewrite
|
|
can land inside one mtime tick. The C2PA key additionally carries the
|
|
reader-availability flag: with the official reader the manifest comes back as a
|
|
store and without it from the PNG chunk parser, so the answer depends on process
|
|
state and not on the file alone.
|
|
|
|
Native-container TC260 readers (`isobmff`, `ebml`, `riff`, `flv`) all run, in that
|
|
order, on every file. Each self-gates on its own magic bytes after a 4-12 byte
|
|
read, so gating the AVI and FLV ones on the file extension as well was redundant
|
|
and made a correctly formatted container served under the wrong name invisible.
|
|
WebP is the one input class the now-unconditional RIFF reader newly touches; its
|
|
`AVI ` form check is what rejects it.
|
|
|
|
`api._SourceEvidence` extracts that metadata once per `remove_all` call and serves
|
|
both the visible pass (which vendor is confirmed) and the scrub gate (is there an
|
|
invisible target). It is per-call, never module-level: `batch` may write its
|
|
output over its input, and a holder that outlived one call would answer the scrub
|
|
gate from pre-write evidence. In `batch` each stage builds its own holder after
|
|
any write that precedes it, for the same reason. Every accessor fails safe the way
|
|
the function it replaces does — no provenance means no relaxation, and an unknown
|
|
invisible target means scrub rather than skip.
|
|
|
|
The `detect` extra composes the shared `pixels` runtime with PyWavelets. Its
|
|
in-tree [`dwt_dct.py`](../src/remove_ai_watermarks/dwt_dct.py) decoder reproduces
|
|
the upstream algorithm's output bit for bit without installing Torch or
|
|
non-headless OpenCV; the block scan is vectorized rather than transcribed, so
|
|
the file no longer reads line by line against `maxDct.py`. The upstream MIT
|
|
notice ships inside the wheel under `licenses/`.
|
|
|
|
`is_ai_generated` is `True` or `None`; absence of evidence is not reported as a
|
|
human-made verdict. `ai_source_kind` distinguishes fully generated content from
|
|
AI-enhanced composites when the source metadata provides that distinction.
|
|
|
|
TrustMark is reported as a watermark signal but does not by itself assert AI
|
|
origin because it can also protect human-authored content. The decoder requests
|
|
binary mode, matching Adobe's Durable Content Credentials example, then requires
|
|
the same payload and schema after a quality-95 JPEG round-trip. Only Variant P
|
|
schemas 0-2 count as positives. Schema 3 is below the precision threshold: all
|
|
38 measured historical false-positive candidates used it, and six retained the
|
|
same false payload after re-encoding. The official Adobe Variant P schema-1
|
|
fixture in `data/fixtures/provenance/` is the positive regression control.
|
|
|
|
#### Why the DWT-DCT decoder looks the way it does
|
|
|
|
Two things in `dwt_dct.py` are load-bearing and neither is obvious from the code.
|
|
|
|
`_approximation` calls `pywt.dwt` twice instead of `pywt.dwt2`, and transposes
|
|
before each pass. A factorial ablation over both axes separates the two:
|
|
skipping the three detail bands `dwt2` computes and this code discards is worth
|
|
**4%**, while the transposes are worth **2.8x**, because pywt walks the axis it
|
|
transforms and on axis 0 of a C-contiguous plane that is a column walk. The
|
|
arithmetic saving is the intuitive explanation and it is the small term; four
|
|
independent profiles named it as the mechanism before the ablation contradicted
|
|
them.
|
|
|
|
Bit-identity is a hard requirement, not a preference. For uint8 input the exact
|
|
Haar LL value is a multiple of 0.5 and the bit test is `peak % 36 > 18.0`, a
|
|
threshold sitting exactly on a representable value that ~1 block in 72 lands on,
|
|
so a 1-ulp difference deterministically flips real bits. That is why the ~16x
|
|
available from a hand-rolled numpy Haar is unreachable rather than merely
|
|
untaken: pywt's C convolution contracts into an FMA that numpy has no ufunc for,
|
|
and `np.longdouble` is 64-bit on arm64 macOS.
|
|
|
|
Each pass is one flat `pywt.downcoef` call over a raveled strip rather than
|
|
`pywt.dwt(..., axis=1)[0]`, and the plane is processed in strips of `_STRIP`
|
|
block-rows so no full-plane float64 intermediate is ever materialized. The strip
|
|
height is not a tuned value: 8 through 64 all scored inside each other's noise
|
|
with unstable ordering, and only "strips at all" versus whole-plane matters.
|
|
|
|
`_approximation`'s even-last-axis check is load-bearing, not defensive. Haar's
|
|
filter is length 2, so an even row length keeps every pair inside its own row;
|
|
on an odd width the pairs walk across row boundaries and the reshape still
|
|
succeeds whenever the total is even, which would be wrong bits with no
|
|
exception. Both call sites are even by construction today, and
|
|
`TestRaveledHaarPass` pins both halves -- the `downcoef`/`dwt` equivalence,
|
|
which a pywt upgrade could take away, and the raise on an odd width.
|
|
|
|
Measured on a 1536x2816 image, all arms timed in one process: the decoder went
|
|
0.112 s (the original per-block Python loop) to 0.011 s vectorized to 0.007 s
|
|
with strips, and a warm `identify()` 1.757 s to 1.365 s on the first step and a
|
|
further 0.4% on the second. That last figure is the point at which this target
|
|
is finished: the decoder is now under 2% of `identify()`, so speed here has
|
|
stopped buying anything. What the strips buy is peak RSS in the stage, 111 MB to
|
|
21 MB on a 4.3 MP image, which is what matters on the memory-limited Space.
|
|
|
|
Both steps were verified by recording decoder output and detector verdict over
|
|
200 sampled `data/` images plus two synthesized carriers before and after: the
|
|
record is byte-identical, as are seven degenerate shapes (`1x65536` through
|
|
`8x8192`, plus an odd width) that clear the caller's area check.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_identify.py`](../tests/test_identify.py)
|
|
- [`test_trustmark_detector.py`](../tests/test_trustmark_detector.py)
|
|
- [`test_invisible_watermark.py`](../tests/test_invisible_watermark.py) --
|
|
note `test_in_tree_decoder_matches_upstream` is the parity guard against
|
|
upstream's own decoder, and the whole module is `skipif(not is_available())`.
|
|
A green run without the `detect` extra installed has not checked parity at
|
|
all, so a decoder change still owes the before/after verdict record.
|
|
|
|
## Visible mark removal
|
|
|
|
### Registry and decision flow
|
|
|
|
[`watermark_registry.py`](../src/remove_ai_watermarks/watermark_registry.py) is
|
|
the only visible-mark registry. `mark_keys()` supplies the CLI choices, so the
|
|
CLI must not maintain a separate mark list.
|
|
|
|
Automatic removal has three distinct stages:
|
|
|
|
1. Perception: each registered detector produces strict and relaxed candidates.
|
|
2. Decision: the pure `decide` arbiter applies sensitivity and corroborating
|
|
provenance.
|
|
3. Action: each selected mark is localized to a mask and passed to the shared
|
|
fill function.
|
|
|
|
`sensitivity="strict"` never relaxes a detector. `sensitivity="auto"` can relax
|
|
one only when metadata or a sufficiently strong same-product sibling confirms
|
|
that product. The removed blanket `assume_ai` mode is rejected explicitly.
|
|
|
|
The Jimeng pill has an additional decision gate because its visual detector is
|
|
weaker than the other registered marks. Keep that policy in the registry, not
|
|
inside unrelated detector engines.
|
|
|
|
Everything about a mark is one registry row: its product family, manufacturer,
|
|
label regime, the platform sentence `identify` reports for it, and the metadata
|
|
signals that confirm its vendor. The manufacturer is distinct from the label
|
|
regime: Alibaba, ByteDance, Kuaishou, Tencent, and other companies all use TC260,
|
|
but their marks do not belong to one manufacturer family.
|
|
`identify._VISIBLE_MARK_PLATFORM` and the signal mapping in
|
|
`api.visible_provenance` are derived from those rows rather than hand-maintained
|
|
beside them, so registering a mark is one edit. Two marks carry no platform of
|
|
their own: the Gemini sparkle has its own higher-confidence path, and the pill
|
|
alone is too weak to attribute.
|
|
|
|
The set of marks that veto the Jimeng pill is derived from the registry rows:
|
|
every other product from the same manufacturer. Doubao therefore vetoes the pill
|
|
because both products are ByteDance, while Qwen, Kling, Yuanbao, RunningHub,
|
|
Baidu, and LiblibAI do not become siblings merely because they use the same TC260
|
|
standard. Marks from other manufacturers cannot enable or veto the
|
|
ByteDance-specific arm.
|
|
|
|
A TC260 label relaxes the vendor its `ContentProducer` names, resolved through
|
|
`KnownMark.tc260_producer_codes`. The label itself is vendor-agnostic, so this used to
|
|
relax ByteDance's two products on every China-AIGC image -- which both risked a
|
|
false fill on an image carrying some other vendor's mark and denied that vendor's
|
|
own mark the relaxed gate its `provenance_ncc_factor` was calibrated for. An
|
|
absent or unmapped producer confirms no particular product and leaves every detector
|
|
strict.
|
|
|
|
`remove_auto_marks` removes every selected mark, not only the strongest one.
|
|
This matters for images that carry marks in more than one corner.
|
|
|
|
The automatic path has four stages: perception, decision, action, then read-only
|
|
validation. The first action reuses the exact detection produced by perception,
|
|
so the common one-mark path pays for one pre-fill and one post-fill detector pass
|
|
rather than two pre-fill passes plus validation. Later co-firing marks retain the
|
|
historical re-detection on the progressively edited image instead of acting on
|
|
stale geometry.
|
|
|
|
After one fill, the same detector runs at the same resolved trust level. A
|
|
post-fill detection counts as a residual only if its region overlaps the actual
|
|
mask bounding box. The per-mark status is `cleaned`, `partial`, or `unvalidated`;
|
|
the aggregate also has `no_watermark`. Validation never expands the mask or
|
|
retries the fill. Because it uses the same detector, this is a consistency check,
|
|
not an independent oracle of visual quality.
|
|
|
|
Runtime was measured on 2026-09-03 with 12 tracked public visible fixtures, five
|
|
warmed repetitions each, explicit `cv2`, alternating old/new order, and no
|
|
concurrent benchmark load. Across 60 paired runs, the old path's median was
|
|
373.9 ms and the validated path's median was 379.2 ms; the median paired delta
|
|
was -0.9% (interquartile range -3.6% to +1.5%) and the aggregate delta was
|
|
-1.5%. All output pixels and removed-label lists matched the old path. These
|
|
figures show no measurable regression on that corpus, not a cross-machine
|
|
latency guarantee. A separate exact snapshot check kept every strict/relaxed
|
|
verdict, confidence, and region unchanged for all 12 detectors on all 12
|
|
fixtures.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_watermark_registry.py`](../tests/test_watermark_registry.py)
|
|
- [`test_api.py`](../tests/test_api.py)
|
|
|
|
### Gemini sparkle
|
|
|
|
[`gemini_engine.py`](../src/remove_ai_watermarks/gemini_engine.py) uses a
|
|
multi-scale shape search and a false-positive gate. Its captured sparkle assets
|
|
serve detection and mask geometry only. Pixel recovery is performed by the
|
|
shared fill backend.
|
|
|
|
`detect_sparkle_confidence` uses a process-wide shared engine because its loaded
|
|
assets and template ladder are immutable.
|
|
|
|
A decorative sparkle/glint/star glyph in stylized typography -- a bright,
|
|
near-white four-point accent, the same silhouette family the engine's own
|
|
template captures -- is a known, unresolved false-positive class. Incident
|
|
2026-08-27 (raiw-app, Spaces uid 7e01ef8cb99949bba14ab4607a66ae06): a book-cover
|
|
title's decorative diamond accent inside a neon-glow letterform scored 0.64, just
|
|
under `_SPARKLE_FP_CONF`, and the removal step painted a permanent smudge over
|
|
the artwork. Investigation (margin, gradient, saturation, and a prototyped local
|
|
surrounding-edge-density check) found no signal that separates this class from a
|
|
genuine weak Gemini sparkle over a noisy/textured photo background -- a
|
|
synthetic counter-example lands in the same confidence band with the same
|
|
clean-margin, clean-gradient, near-white-core signature that the
|
|
`_SPARKLE_KEEP_CONF` rescue exists to protect (see
|
|
`test_bright_white_blurred_sparkle_rescued`). Tightening any of those signals to
|
|
exclude the glyph excludes the legitimate case too. Closing this gap needs a
|
|
real labeled corpus of decorative-glyph occurrences, the way
|
|
`watermark_registry.GEMINI_SPARKLE_TRUST_CONF` was calibrated against 954 Google-
|
|
metadata uploads, not another synthetic guess. `_apply_false_positive_gate` logs
|
|
`DEBUG` on the exact bypass path this incident hit, to build that corpus from
|
|
production occurrences.
|
|
|
|
Texture reaches the trust gate too. A local sweep on 2026-09-24 found non-Gemini
|
|
images scoring up to 0.67 (whiteboard photos, UI screenshots, digital art) while
|
|
real Google-signed sparkles went as low as 0.55, and two Higgsfield downloads
|
|
scored 0.53 on wood grain. No threshold separates those, so `identify` passes
|
|
`_collect_visible_signals` the first non-Google AI vendor from the trusted
|
|
vendor claims (C2PA only when verified) and drops a sparkle on such an image
|
|
with `_SPARKLE_VETOED_CAVEAT`. Google provenance never vetoes. The removal
|
|
arbitration does not see these claims.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_gemini_engine.py`](../tests/test_gemini_engine.py), including
|
|
`TestDecorativeGlyphFalsePositive` pinning the confirmed false positive and its
|
|
blocking counter-example.
|
|
- [`test_identify.py`](../tests/test_identify.py)
|
|
`test_non_google_provenance_vetoes_a_sparkle_on_texture` and
|
|
`test_google_provenance_keeps_the_sparkle`.
|
|
|
|
### Text mark engines
|
|
|
|
[`_text_mark_engine.py`](../src/remove_ai_watermarks/_text_mark_engine.py)
|
|
provides common localization, detection front ends, template caching, rival
|
|
comparison, and footprint construction.
|
|
|
|
Each vendor module supplies a `TextMarkConfig` and only the behavior that cannot
|
|
be represented by the shared base:
|
|
|
|
- [`doubao_engine.py`](../src/remove_ai_watermarks/doubao_engine.py)
|
|
- [`jimeng_engine.py`](../src/remove_ai_watermarks/jimeng_engine.py)
|
|
- [`qwen_engine.py`](../src/remove_ai_watermarks/qwen_engine.py)
|
|
- [`kling_engine.py`](../src/remove_ai_watermarks/kling_engine.py)
|
|
- [`yuanbao_engine.py`](../src/remove_ai_watermarks/yuanbao_engine.py)
|
|
- [`samsung_engine.py`](../src/remove_ai_watermarks/samsung_engine.py)
|
|
- [`runninghub_engine.py`](../src/remove_ai_watermarks/runninghub_engine.py)
|
|
- [`baidu_engine.py`](../src/remove_ai_watermarks/baidu_engine.py)
|
|
- [`liblib_engine.py`](../src/remove_ai_watermarks/liblib_engine.py)
|
|
- [`microsoft_engine.py`](../src/remove_ai_watermarks/microsoft_engine.py)
|
|
- [`generic_ai_label_engine.py`](../src/remove_ai_watermarks/generic_ai_label_engine.py)
|
|
|
|
Qwen is the one image registry row that currently covers two layout families.
|
|
The shared text engine handles `千问AI生成`; `QwenEngine` also scans a square
|
|
bottom-right region for Qwen Create's three-lobe symbol. Its 0.65 NCC gate was
|
|
measured against the cleared provider fixture (0.848) and 132 decoded tracked
|
|
controls other than that positive (maximum 0.516 on 2026-09-10, zero fires).
|
|
NCC is not portable across OpenCV builds: issue 108 reported 0.81 on
|
|
`data/fixtures/provenance/chatgpt-1.png` (sand and denim, no glyph) while this
|
|
tree's OpenCV scores 0.437 on the same file (4.11, and again 5.0 on 2026-09-23). A fire also requires
|
|
binarized IoU of the bright residual with the three-lobe silhouette at or
|
|
above 0.70; the true fixture stays at or above 0.75 down to 512 px, and
|
|
chatgpt-1 sits at 0.55. `provenance_ncc_factor` remains 1.0, so OpenAI C2PA
|
|
cannot relax this mark.
|
|
The removal mask reuses the aligned three-lobe alpha and expands it by 6% of
|
|
the detected side; a solid square was rejected because classical inpainting
|
|
left a visible rectangular blur.
|
|
`scripts/visible_alpha_solve.py qwen_symbol` rebuilds the template from the
|
|
cleared fixture with a cubic local-background fit and retains only the three
|
|
interior lobes.
|
|
|
|
LiblibAI has two registry entries under one product. `liblib` retains the
|
|
historical bottom-center wordmark detector. `liblib_pill` covers the compact
|
|
top-left `AI生成` pill and never fires at strict trust: the shape is generic, so
|
|
automatic removal requires either LiblibAI TC260 metadata or a confident
|
|
bottom-center sibling detection. The cleared current provider fixture scores
|
|
`0.216367` against the reused synthetic pill silhouette, above the corroborated
|
|
`0.20` gate but below the standalone Jimeng pill's `0.22` gate. A flat-footprint
|
|
check remains mandatory. The pill is a separate action rather than a distant
|
|
component unioned into the bottom mask, keeping LaMa and MI-GAN crops local to
|
|
each mark.
|
|
|
|
Kling keeps two independently gated silhouettes under one registry key. The
|
|
older measured cohort uses `可灵AI 3.0`; a direct authenticated IMAGE 3.0 export
|
|
on 2026-09-04 uses `KlingAI 3.0`. The Latin template scored 0.429 on that 1024 x
|
|
1024 provider original. Its narrow 0.9/1.0/1.1 scale ladder scored at most 0.349
|
|
across the 94 available neighboring real and synthetic image controls, while an
|
|
adversarial solid corner blob reached 0.379, so its strict gate is 0.40. This is
|
|
one-positive provisional calibration, intentionally separate from the older
|
|
CJK detector rather than presented as broad recall evidence.
|
|
|
|
The measured Microsoft badge variant (2026-08-27 registration) is the first
|
|
`tr`-corner mark and the first `long`-side scale basis: the pill tracks the
|
|
render dimension, so a
|
|
1024x1536 portrait carries the same pill as 1536x1024, and a width basis
|
|
undersized the template by the aspect ratio (portrait carriers fell to
|
|
0.15-0.32 NCC until the basis was measured). The silhouette is a white pill with
|
|
its synthetic internal shapes knocked out - the holes are what separate it from
|
|
any other bright rounded corner element (a plain white pill scores below the gate
|
|
in the tests). It does not claim coverage of Microsoft's other documented icon,
|
|
wording, or position variants.
|
|
|
|
The 2026-08-27 rerun used the registered engine through
|
|
`scripts/registered_mark_calibrate.py`, rather than a copied detector
|
|
configuration. The manifest kept three evidence classes separate: 17 visually
|
|
confirmed carriers, 343 Microsoft-provenance files without a visual adjudication,
|
|
and 1200 non-overlapping no-signal controls. At the strict 0.38 gate, 15/17
|
|
confirmed carriers fired (min 0.249, p50 0.519, p90 0.578, max 0.579), while
|
|
0/1200 controls fired (p99 0.200, max 0.293). The provenance cohort produced
|
|
78/343 fires, but that is not a recall measurement because provenance identifies
|
|
the provider, not the presence of this visible layout. That cohort was later
|
|
labeled by an OCR badge census (2026-08-28: 86 badge carriers, 257 badge-less,
|
|
badge-less max 0.251 / p99 0.213), which is what enabled the shipped provenance
|
|
relaxation 0.7: the relaxed band [0.251, 0.38) holds three faint badges and
|
|
zero false fills, re-verified as 3 band detections with 0/257 badge-less fires
|
|
on the provenance path.
|
|
|
|
The detector and removal mask must use compatible geometry. A detector that
|
|
fires while producing an empty or misplaced mask is a removal failure even if
|
|
the detection test passes. That parity is now structural rather than a
|
|
convention: the three continuous front ends share one ladder sweep
|
|
(`_ladder_best`), and the winning box travels to the mask on
|
|
`TextMarkDetection.match_box` instead of being swept a second time.
|
|
|
|
Detection is split into a trust-level-blind `_scan` and a `_verdict` that applies
|
|
the threshold. `detect_both` returns the strict and relaxed verdicts from one
|
|
scan, which is what the arbiter's perception stage calls. A per-mark demotion
|
|
belongs in the `_post_gate` hook, never in a `detect` override: an override is
|
|
invisible to the single-pass path, and the RunningHub and Yuanbao anchor gates
|
|
were briefly skipped there for exactly that reason.
|
|
|
|
A mark whose removable rectangular footprint differs from what the detector localizes
|
|
overrides `_footprint_rect` (which policy) and `_extend_match_box` (how far the
|
|
box grows), not the whole `footprint_mask`. Baidu extends right to the corner tag
|
|
and LiblibAI extends left to the triangle logo; both inherit every guard around
|
|
that arithmetic.
|
|
|
|
Doubao, Kling, and Samsung are deliberate sparse-mask exceptions because their alpha assets
|
|
supply more information than a rectangle. Doubao's continuous top-hat response
|
|
locates the mark, then `DoubaoEngine.footprint_mask` resizes the alpha to that same winning
|
|
`match_box` and masks only the glyphs. The canonical 2048-pixel fixture has bright
|
|
branch texture behind the bottom-edge wordmark; bounding the thresholded response,
|
|
padding it, and dilating it produced a solid 404-by-134 mask clamped to the bottom
|
|
and right edges. OpenCV then had no context beyond either edge and filled the hole
|
|
with large triangular wedges. The aligned sparse mask keeps both frame edges
|
|
untouched and still clears the detector. `force` cannot align a missing detection,
|
|
so it retains the shared geometry-box fallback.
|
|
|
|
Doubao keeps the original low-saturation scan for pale corners. When enough of
|
|
its corner is strongly colored and that scan falls below the strict gate, a
|
|
wider color filter searches for white glyphs tinted by the background. The
|
|
stronger scan wins, preserving previously accepted pale-background marks.
|
|
|
|
Kling keeps the pixel-derived footprint for release-specific strokes, then unions
|
|
it with the synthetic alpha aligned to the detector's winning match box. This is
|
|
necessary because its faint canonical overlay produced a blob covering only the
|
|
left half of `可灵AI 3.0`, leaving `3.0` after fill. On 60 constructed pairs from 20
|
|
tracked source images crossed with the three detector scales, the hybrid covered
|
|
100% of the known stamped footprint and cleared 60/60, versus 29/60 for the blob
|
|
alone. In the same paired region the hybrid improved median PSNR by 6.59 dB and
|
|
median SSIM by 0.062; both moved in the favorable direction on 52 pairs, tied on
|
|
seven, and regressed on one. The union is deliberate: replacing the old footprint
|
|
with the synthetic core alone would overfit the font render and could discard a
|
|
real variant stroke the pixel mask already found.
|
|
|
|
Samsung uses the continuous top-hat front end with an exact `(1.0,)` scale ladder.
|
|
The three retained real positives (two controlled captures and one real-content
|
|
photo) scored 0.82-0.90, while 421 public clean controls reached at most 0.38, so
|
|
the existing 0.40 threshold did not need to move. The exact rung matches the measured
|
|
width at both 1086 and 2958 pixels and avoids giving clean corners extra scale trials.
|
|
Samsung is strict-only under provenance: the inherited 0.70 factor would lower the
|
|
gate to 0.28 and admitted 14/421 clean controls without recovering any retained real
|
|
positive.
|
|
On the same 421 backgrounds, nominal constructed detection rose from 81/421 through
|
|
the binary front end to 321/421 through continuous top-hat, with no control fires in
|
|
either arm. This constructed arm measures the known failure mode on textured content;
|
|
it is not a production-recall estimate.
|
|
|
|
The winning template location travels with the score. `SamsungEngine.footprint_mask`
|
|
aligns the captured alpha to that same box and masks only pixels above the measured
|
|
0.05 alpha floor, followed by one pixel of dilation. The asset already contains the
|
|
solved opacity, whose peak is about 0.38; research renderers must composite it toward
|
|
pure white exactly once. Multiplying it by 0.38 again produced an unrealistically
|
|
faint synthetic mark and invalidated the first Samsung response curve.
|
|
|
|
The mask geometry was checked on both retained flat provider captures and by a
|
|
paired 60-image constructed-reference ablation on 2026-09-04. At the native
|
|
1086-pixel capture width, the mask fell from about 19.7k pixels to 4.9k while one
|
|
fill still cleared the detector. Against the old rectangle, regional PSNR improved
|
|
on 55/60 images for OpenCV, 55/60 for MI-GAN, and 57/60 for LaMa; median paired
|
|
gains were 14.98, 10.38, and 14.87 dB respectively (two-sided sign-test
|
|
`p=1.04e-11`, `1.04e-11`, and `6.25e-14`). Explicit `force` has no detected box to
|
|
align and deliberately retains the conservative geometry fallback.
|
|
|
|
For the 321 newly detected constructed marks, the winning box matched the stamped
|
|
box exactly and every detection produced a non-empty mask. One OpenCV fill remained
|
|
detectable and is surfaced by post-removal validation; the detector does not trigger
|
|
an automatic retry.
|
|
|
|
Yuanbao uses the polarity-independent `contrast` front end because its standard
|
|
two-line mark can be light on dark scenes or dark on light scenes. Its detector
|
|
and footprint both use the same best-match box. The separate one-line overlay
|
|
variant is not covered.
|
|
|
|
#### Generic bare "AI生成" label
|
|
|
|
Registered 2026-09-21 for a confirmed production miss: a bottom-right "AI生成"
|
|
text watermark with no brand wordmark in front of it was left completely
|
|
unremoved because none of the brand-tuned detectors above match a bare,
|
|
brand-less corner stamp -- `identify` found only the TC260 metadata label, with
|
|
nothing for `suggested_mode=visible` to localize. No customer image bytes exist
|
|
for that case in any repository; this engine is built from research plus
|
|
synthetic fixtures.
|
|
|
|
Research found the bare wording is not tied to one dominant generator. vivo's
|
|
own support documentation confirms its Gallery app stamps a bare "AI生成"
|
|
watermark after AI消除/AI扩图/大片模式 editing (togglable under Album Settings),
|
|
and independent reporting documents the same bare watermark on Xiaomi Gallery's
|
|
AI-edit output. Community reporting suggests Samsung's China-locale Galaxy AI
|
|
tools render "AI生成" too, distinct from the Italian string the `samsung` engine
|
|
targets. No independent confirmation of OPPO/Huawei's exact wording was found.
|
|
Because the label is a compliance stamp mandated by GB 45438-2025 rather than a
|
|
brand mark, several OEM gallery apps plausibly share the same wording with only
|
|
styling differences -- which is why this is one generic fallback rather than
|
|
another per-OEM tuned config.
|
|
|
|
Unlike a captured brand mark, there is no vendor screenshot to solve an alpha
|
|
map from: the mark has no logo element, only literal text, so it shares
|
|
`scripts/render_vendor_silhouettes.py` -- the same capture-less font-rendering
|
|
pipeline that already produces Qwen/Baidu/Kling/Yuanbao's own text-only
|
|
silhouettes -- which renders "AI生成" directly (white on black, already the
|
|
alpha map) and crops to the glyph body, following `visible_alpha_solve.py`'s
|
|
asset contract (a grayscale PNG under `assets/`) so it plugs into the same
|
|
`TextMarkConfig`/`TextMarkEngine` machinery as every captured mark.
|
|
|
|
Every brand-specific rival mark in this family literally contains the
|
|
substring "AI生成" in its own glyphs (Doubao, Qwen, Baidu, Kling, Yuanbao --
|
|
Jimeng's "★ 即梦AI" does not and is excluded), so this generic template is at
|
|
real risk of firing a second time on an already-attributed mark. At this
|
|
asset's glyph shape the usual 0.10 `rival_margin` already separates it from
|
|
Qwen's own binarized glyph blob once the absolute NCC gate clears 0.45, so
|
|
`generic_ai_label_engine.py` keeps the shared default rather than widening it
|
|
per mark.
|
|
|
|
CALIBRATION IS SYNTHETIC ONLY. `scripts/calibrate_generic_ai_label.py` sweeps
|
|
the gate and rival margin over 36 positives (the rendered glyph composited onto
|
|
generated backgrounds at 4 sizes, 3 scale rungs, 3 opacities), 40 generated
|
|
clean/textured negatives, and 8 rival marks' own committed gallery examples. At
|
|
gate 0.45 or 0.50 / the default 0.10 margin the sweep reads the same 67% recall,
|
|
100% precision, zero rival cross-fires; a lower 0.35-0.40 gate reaches the same
|
|
67% recall but admits one Qwen cross-fire. 0.50 is the shipped choice: it costs
|
|
nothing on this corpus and clears a faint inpainting residual the library's own
|
|
removal leaves behind, which otherwise re-fires detection at 0.45. No real
|
|
"AI生成"-generator screenshot was available to validate either number against
|
|
actual vendor output -- treat this
|
|
detector as weaker evidence than the brand-tuned engines above until it is
|
|
recalibrated on real captures.
|
|
|
|
#### Wan logo and wordmark
|
|
|
|
Registered 2026-09-24 from one real Wan 2.7 Pro export (2048 px square, neutral
|
|
scene, `data/fixtures/visible/wan/provider-original.png`). The mark measures
|
|
0.131 x 0.050 of the short side with ~0.005 margins; the synthetic silhouette is
|
|
`draw_wan` in `scripts/render_vendor_silhouettes.py` (a cut hexagon plus Avenir
|
|
Next Demi Bold `Wan`, parameters picked by best match on the real mark, 0.66).
|
|
NCC alone separates poorly: 3092 local images (every tracked fixture plus 3000
|
|
sampled) reached 0.519. The mark hugs the corner, so `WanEngine._post_gate`
|
|
also requires the match to end within 0.015 of the short side from both edges;
|
|
with that anchor no negative scored 0.45 or above. The gate is NCC >= 0.55 plus
|
|
the anchor, strict only. The TC260 producer is Tongyi Yunqi (Hangzhou), USCC
|
|
`91330106MA2CFLDG4R`. Calibration stays provisional until a wider Wan cohort
|
|
exists.
|
|
|
|
#### OpenArt wordmark
|
|
|
|
Registered 2026-09-21 for a confirmed production miss reported through raiw-app:
|
|
a free `visible`-mode run (2026-09-11, 1528x2712 portrait, Spaces uid
|
|
`aefb1e07306f4a7280615a8daea01a39`) left a large, prominent OpenArt wordmark
|
|
(a bowtie/infinity icon followed by "OpenArt" in bold sans-serif, semi-
|
|
transparent white) completely unremoved -- pixel-identical before and after.
|
|
`identify` found nothing at all for that case: no C2PA, EXIF AI tag, or China
|
|
AIGC/TC260 label, so the visible mark is the only known attribution route for
|
|
a bare OpenArt export. The user rated the result 0 ("Didn't work").
|
|
|
|
**This is the only registered mark placed at the FRAME CENTER rather than a
|
|
corner.** Every other engine in this family anchors to one of the four corners
|
|
or a bottom/top edge midpoint; `_text_mark_engine.py` gained a new `"cc"`
|
|
corner literal (both axes centered) specifically for this row, following the
|
|
same per-vendor-measured-geometry precedent as the existing `"bc"` (LiblibAI)
|
|
and `"tr"`/`"long"`-basis (Microsoft) additions. A frame-center locate box sits
|
|
over far more varied image content than a corner box, which is why its NCC
|
|
gate (0.62) is set well above Doubao/Jimeng's measured 0.45-0.50 rather than
|
|
matching them.
|
|
|
|
OpenArt (openart.ai) requires an OAuth-authenticated account for image
|
|
generation -- there is no anonymous or API-key access (confirmed via its own
|
|
help center and MCP/CLI docs, 2026-09-21) -- so this repository's tooling
|
|
cannot generate a second real capture to survey placement variance by aspect
|
|
ratio, watermark configurability, or whether a paid tier still embeds C2PA;
|
|
only the one customer-reported case above is confirmed. OpenArt's own pricing
|
|
page lists "Watermark-free" as a paid-plan-only feature, consistent with every
|
|
free-plan output carrying this mark, and its help center separately confirms
|
|
free-plan VIDEO exports carry a removable watermark toggle at download --
|
|
independent evidence that a visible free-tier watermark is real product
|
|
behavior, not one-off.
|
|
|
|
Unlike every captured brand mark in this file, there is no committed real
|
|
capture to solve an alpha map from (see above -- no account access) and no
|
|
public example image was reconstructed by hand. `scripts/build_openart_alpha.py`
|
|
instead PROCEDURALLY DRAWS a reconstruction of OpenArt's own public brand mark
|
|
(the bowtie icon + wordmark shown on openart.ai's own marketing pages) as a
|
|
grayscale alpha PNG, the same bundled-asset contract every other engine uses.
|
|
Its wordmark is a Hershey `putText` render, so like the video Hershey templates
|
|
the recipe refuses anything but OpenCV 4.x, whose output the committed asset is.
|
|
|
|
**CALIBRATION IS PROVISIONAL, NOT MEASURED.** There is no recall/precision
|
|
sweep like Doubao's or the generic-label engine's: no positive corpus exists
|
|
at all beyond the one production case, real or synthetic-composited. The 0.62
|
|
NCC threshold is a conservative guess, not a measured operating point; a quick
|
|
false-fire check against 15 random Gaussian-blurred noise images at plausible
|
|
aspect ratios and sizes found zero false fires, which is reassurance, not
|
|
calibration. Do not cite this mark in any auto-scan precision claim, and do
|
|
not lower its threshold, without a real captured corpus under
|
|
`data/calibration/openart/`.
|
|
|
|
`openart_engine.py`'s module docstring carries the same caveats in code. Its
|
|
tests (`tests/test_openart_engine.py`) mirror `test_doubao_engine.py`'s
|
|
structure entirely on synthetic composites of the bundled asset -- there is no
|
|
`TestRealSample` analogue.
|
|
|
|
#### Bare-upload pill arm: measured impossible with the current detector (2026-08-28)
|
|
|
|
`_keep_pill` never removes a metadata-bare pill, and the question "can a bare arm
|
|
(score + footprint-flatness) be opened" was measured to a closed NO over the local
|
|
private corpus: 68 OCR-confirmed bare pills (an independent pixel-level label: the
|
|
band sweep read a flush-left `AI生成`) score p50 0.181 / max 0.317, while 799 clean
|
|
no-signal negatives reach 0.353 and 18 of them already pass the shipped 0.22 raw
|
|
gate. The true-pill and clean distributions overlap completely -- there is no
|
|
threshold -- and the engine's own high scores (0.34-0.48 on the flagged cohort)
|
|
correlate with textured corners, not with pills: the edge-NCC keys on "text-like
|
|
structure top-left". As with the Jimeng wordmark's silhouette, no threshold repairs
|
|
this; a bare arm needs a detector that keys on the pill's rounded-rectangle
|
|
geometry, and until one exists the bare-arm question is settled by this
|
|
measurement, not by gate tuning. The local calibration harness is not tracked.
|
|
(cohorts A/B/C over the no-signal pool).
|
|
|
|
The weak Jimeng pill detector lives in
|
|
[`pill_engine.py`](../src/remove_ai_watermarks/pill_engine.py). It uses a
|
|
synthetic silhouette for detection and a fixed top-left footprint. Its public,
|
|
published real-world regression carrier is
|
|
`data/fixtures/visible/jimeng_pill/provider-published-example.jpg`; the direct
|
|
detector scores the pill at 0.28 and the same image's Jimeng wordmark at 0.61.
|
|
|
|
Each engine has a corresponding test module under [`tests/`](../tests/).
|
|
Shared behavior is covered by:
|
|
|
|
- [`test_text_mark_engine.py`](../tests/test_text_mark_engine.py)
|
|
- [`test_text_mark_faint_mask.py`](../tests/test_text_mark_faint_mask.py)
|
|
- [`test_text_mark_memory.py`](../tests/test_text_mark_memory.py)
|
|
|
|
#### Doubao video mark (2026-08-28 registration)
|
|
|
|
`detect_doubao_frame` reuses the image engine's synthetic `doubao_alpha.png` as
|
|
its video template (same `豆包AI生成` run, bottom-right) through
|
|
`_asset_template`, so image and video cannot drift apart on the glyph. Search
|
|
profile: heights 3.2-6.0 percent of the short side from origin (0.60, 0.76),
|
|
measured on the corpus. Policy is seedance-style single-floor: weak 0.35 /
|
|
strong 0.55 / anchor IoU 0.80, calibrated by sequential decode over 39
|
|
TC260-confirmed Doubao videos and 25 negatives -- a 12-frame stable run at 0.35
|
|
accepts 33/39 positives and 0/25 negatives, while 0.30 already accepted one
|
|
negative, so the bar stays at 0.35. Through the shipped arbiter
|
|
(`identify_video`): 32/39 selected as doubao, 0/25 false; the remainder split
|
|
between no stable run (5) and cross-template ties handed to earlier table
|
|
entries (sora 1, kling 1) -- the registry's known order-decides behavior, not a
|
|
doubao-specific defect. The local sequence-calibration harness is not tracked.
|
|
|
|
Two further video candidates were measured and PARKED the same day (2026-08-28,
|
|
local RunningHub/Jimeng video calibration): the Jimeng CN wordmark does not
|
|
separate on video (1/5 positives vs 2/25 negatives hold a 12-frame 0.35 run --
|
|
the same silhouette-generality the image jimeng engine documents), and
|
|
RunningHub video separates (1/2 positives, 0/25 negatives) but a two-positive
|
|
cohort is below the registration bar; both await corpus growth. The top-left
|
|
"AI生成" pill on video is parked by analysis, not measurement: its image-engine
|
|
false-fire mode is a STATIC textured corner, which is temporally stable, so the
|
|
recurrence arbiter cannot screen it.
|
|
|
|
### Fill backends and region erasing
|
|
|
|
[`region_eraser.py`](../src/remove_ai_watermarks/region_eraser.py) implements the
|
|
same backends used by visible removal and the user-directed `erase` command:
|
|
|
|
- `cv2`
|
|
- `migan`
|
|
- `lama`
|
|
|
|
`watermark_registry.resolve_backend` selects LaMa first, then MI-GAN, then
|
|
OpenCV for `auto`. A memory-constrained caller should explicitly select MI-GAN
|
|
or OpenCV instead of relying on `auto`.
|
|
|
|
MI-GAN and LaMa crop around the mask before model inference and paste back only
|
|
masked pixels. Their model sessions are loaded lazily. MI-GAN uses the inverse
|
|
mask polarity expected by its ONNX model.
|
|
|
|
Depth is the source's, and only the filled pixels can lose it. `cv2.inpaint`
|
|
accepts 16-bit as a single channel only (its colour path is 8-bit) and the shipped
|
|
MI-GAN ONNX declares a uint8 input tensor, so both narrow the frame through
|
|
`_erase_via_uint8`, fill, and write the result back into the original array --
|
|
everything outside the mask stays bit-exact at 16 bits, and an alpha plane remains
|
|
bit-exact everywhere. LaMa is float32 and needs no narrowing: it normalises by the
|
|
source's full scale and returns the source dtype.
|
|
Before that, a 16-bit colour image raised a bare `icvInpaint` "Unsupported format"
|
|
error on the cv2 backend, and LaMa divided the crop by a hardcoded 255 -- feeding the
|
|
model ~235 where it expected ~0.92 and then wrapping the uint8 cast into near-black
|
|
pixels, which is the worse failure of the two because it is silent. A float image is
|
|
now refused by name rather than reaching a backend that cannot take it.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_region_eraser.py`](../tests/test_region_eraser.py)
|
|
- [`test_inpaint_fallback.py`](../tests/test_inpaint_fallback.py)
|
|
|
|
## Invisible watermark regeneration
|
|
|
|
### Profiles and strength
|
|
|
|
[`_internal/watermark_profiles.py`](../src/remove_ai_watermarks/_internal/watermark_profiles.py)
|
|
is the source of truth for:
|
|
|
|
- profile names and their underscore spellings;
|
|
- the fixed seed;
|
|
- the SDXL global-stage checkpoint id (`SDXL_MODEL_ID`) and the Canny ControlNet id;
|
|
- strength resolution for every profile.
|
|
|
|
The current profiles are `qwen-zimage` (the default), `sdxl-zimage`,
|
|
`chroma-zimage`, and `auto`, and all four are CUDA-only. `controlnet`, `sdxl`,
|
|
`qwen` and `default` were removed rather than kept as a CPU path, and are
|
|
rejected rather than aliased onward. `auto` is a deterministic per-cohort
|
|
selection policy:
|
|
sdxl-zimage for Google, chroma-zimage for Microsoft, and qwen-zimage for
|
|
OpenAI, Meta, and unknown.
|
|
It does not run a learned router or classify the image's genre.
|
|
|
|
`qwen-zimage` normally resolves global denoise from image area for unknown content.
|
|
Measured provider cohorts bypass that curve with flat operating points. The values,
|
|
measurement derivations, and corpus limits are canonical in
|
|
[`known-limitations.md`](known-limitations.md#strength-is-content-and-seed-dependent).
|
|
|
|
For serverless cold starts, `InvisibleEngine.preload(global_only=True)` loads the
|
|
mandatory global stage and YuNet while leaving the optional Z-Image and SAM face
|
|
stack lazy until a face is detected. The default `preload()` still loads every
|
|
stage.
|
|
|
|
**What is deliberately not a parameter.** Model id, step count and CFG are fixed
|
|
by the profile, so none of them appears in `WatermarkRemover.__init__`,
|
|
`remove_watermark`, `InvisibleEngine`, or the CLI. They used to be accepted and
|
|
then rejected several frames down; a signature that refuses the argument outright
|
|
fails where the caller can act on it, and stops a wrapper from threading a value
|
|
that would silently do nothing. The step count and CFG live with the stage that
|
|
runs them: `GLOBAL_STEPS`/`GLOBAL_CFG` with the Qwen global stage
|
|
(`qwen_zimage_pipeline.py`), `SDXL_STEPS` with the SDXL one
|
|
(`sdxl_zimage_pipeline.py`), and `FACE_STEPS`/`FACE_CFG` with the shared face
|
|
stage (`two_stage_pipeline.py`). The dtype is likewise profile-owned: see "Face-stage
|
|
dtype" for what an override cost the last time one existed.
|
|
|
|
`device` is the exception and remains a library parameter: `None` or `"auto"`
|
|
detect, `"cuda"` pins without detecting (which is what a container that knows its
|
|
hardware wants), and any other value raises at construction. It is not a CLI
|
|
option, because the only useful value a user could type is the one detection
|
|
already returns.
|
|
|
|
[`invisible_engine.py`](../src/remove_ai_watermarks/invisible_engine.py) handles
|
|
image sizing, postprocessing, and the public engine
|
|
interface. It delegates model execution to
|
|
[`_internal/watermark_remover.py`](../src/remove_ai_watermarks/_internal/watermark_remover.py).
|
|
|
|
`get_device` in that module answers only `cuda` or `cpu`. An `mps` or `xpu` answer
|
|
would travel one frame to the same CUDA-only refusal while costing a device probe,
|
|
and reporting it implied an Apple-silicon or Intel-GPU path that does not exist.
|
|
The refusal names the *resolved* device, so `device=None` on a CUDA-less host says
|
|
`'cpu'` rather than `'None'`.
|
|
|
|
The Python engine and the CLI now resolve the same defaults: the CLI forwards an
|
|
unset `--adaptive-polish` and `--seed` as `None` and the engine applies the
|
|
profile's answer, so a library caller and a CLI caller on one profile produce the
|
|
same pixels. They diverged before, in opposite directions, for exactly this knob.
|
|
|
|
The global and face prompts are calibrated model inputs, and the Canny edge map
|
|
uses fixed thresholds of `_CANNY_LOW = 13` / `_CANNY_HIGH = 64`
|
|
(`two_stage_pipeline.py`). Treat those values as behavioral compatibility
|
|
contracts: a refactor must preserve them, and any deliberate change requires
|
|
image-quality evaluation rather than only a unit-test pass. The prompt and
|
|
edge-map regression guards are
|
|
`test_qwen_zimage_pipeline.py::test_global_kwargs_use_lightning_and_diffsynth_controlnet_shape`,
|
|
`::test_face_kwargs_use_project_zimage_settings` and
|
|
`::test_canny_control_image_is_three_channel_and_detects_an_edge`.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_invisible_engine.py`](../tests/test_invisible_engine.py)
|
|
- [`test_qwen_zimage_pipeline.py`](../tests/test_qwen_zimage_pipeline.py)
|
|
- [`test_platform.py`](../tests/test_platform.py)
|
|
|
|
### CPU offload
|
|
|
|
CPU offload is enabled only when requested. Nothing calls Diffusers'
|
|
`enable_model_cpu_offload` any more -- that belonged to the deleted single-stage
|
|
profiles. `--cpu-offload` sets both residency fields, and they do not travel the
|
|
same distance: the face field is read by the shared base and so applies to every
|
|
profile, while the global field is read by `QwenZImagePipeline` alone.
|
|
`ChromaZImagePipeline` and `SdxlZImagePipeline` load their global stack with a
|
|
plain `.to(device)` and never consult it.
|
|
|
|
`GLOBAL_OFFLOAD_PROFILES` in `watermark_profiles.py` is the list, and
|
|
`global_offload_supported` the question; `WatermarkRemover._warn_if_global_offload_unsupported`
|
|
warns ahead of the model load rather than at construction, because `auto` picks
|
|
its engine per-image. An explicit `auto` preload uses the qwen-zimage fallback and
|
|
therefore keeps global offload active before that resolution. Below
|
|
`RESIDENT_FACE_MODEL_MIN_VRAM_GIB` the flag is a no-op for the other two profiles --
|
|
the face stack was offloading anyway and the global one does not ask -- which is
|
|
what the warning exists to say before a download starts.
|
|
|
|
Residency is otherwise chosen from the card's total VRAM, once per stack:
|
|
`resolve_global_model_residency` gates the mandatory Qwen stack at
|
|
`RESIDENT_GLOBAL_MODEL_MIN_VRAM_GIB` and `resolve_face_model_residency` gates the
|
|
optional Z-Image stack at `RESIDENT_FACE_MODEL_MIN_VRAM_GIB`.
|
|
|
|
Below the global floor, `_qwen_vram_config` streams the stack from disk, which is
|
|
what makes a 20B model runnable on a consumer card. At or above it, streaming is
|
|
pure waste and the weights stay on the GPU. The difference is not marginal:
|
|
DiffSynth offloads by dropping the weights to the meta device and re-reading every
|
|
parameter through its `DiskMap` on the next onload, and the pipeline moves between
|
|
text encoder, transformer and VAE on each pass. Measured on an H100 (80 GiB) in
|
|
August 2026, a warm global pass took 37.3 s at 0.8 GiB resident with the streaming
|
|
config, against 2.2 s at 28.7 GiB with the stack resident; both stacks resident
|
|
peaked at 48.0 GiB. Faster storage cannot close that gap, because the cost is the
|
|
reload itself rather than the read.
|
|
|
|
The resident config deliberately passes no `"disk"` value anywhere. DiffSynth latches
|
|
`disk_offload` once, from `offload_dtype`, so leaving the sentinel in place while
|
|
pointing every device at CUDA would keep the meta-drop and re-read.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_cpu_offload.py`](../tests/test_cpu_offload.py)
|
|
- [`test_qwen_zimage_pipeline.py`](../tests/test_qwen_zimage_pipeline.py)
|
|
|
|
### Qwen plus Z-Image
|
|
|
|
The profiles are siblings of one shared recipe rather than a chain:
|
|
[`_internal/two_stage_pipeline.py`](../src/remove_ai_watermarks/_internal/two_stage_pipeline.py)
|
|
holds `TwoStageZImagePipeline`, the base that owns everything profile-independent
|
|
(the Z-Image face stage with YuNet detection and SAM masks, sizing, compositing,
|
|
the prompt cache, and the `run`/`preload` orchestration). A profile subclass
|
|
implements `_load_global` and `_run_global` for its regeneration model and may
|
|
implement `_vae_roundtrip` to expose that stack's VAE as a verified-text donor;
|
|
nothing else differs, and a test asserts the shared methods are the same objects
|
|
on every subclass. Adding a third global stage means implementing those hooks,
|
|
not inheriting from another profile's pipeline.
|
|
|
|
[`_internal/qwen_zimage_pipeline.py`](../src/remove_ai_watermarks/_internal/qwen_zimage_pipeline.py)
|
|
implements the fixed CUDA-only two-stage profile on that base:
|
|
|
|
1. Qwen Image with Canny conditioning regenerates the frame.
|
|
2. YuNet locates faces, SAM builds masks, and Z-Image regenerates the selected
|
|
face regions.
|
|
|
|
The profile rejects a custom model identifier. Its global and face model stack
|
|
is fixed by the implementation. When tiling is enabled, only the global stage
|
|
is tiled; the face stage runs once after the tiles are blended.
|
|
|
|
#### How the face stage actually composites
|
|
|
|
The mechanism is easy to misread from the parameter names, so state it plainly.
|
|
`_run_faces` crops the expanded box from the **original** image, resizes it toward the
|
|
768 px face guide, runs Z-Image over **the entire crop**, resizes back, and only then
|
|
merges through `composite_face`, which cross-fades on a Gaussian-blurred SAM mask with
|
|
`feather=10`. The base it merges into is the global Qwen result.
|
|
|
|
Two consequences follow, and both matter when tuning:
|
|
|
|
- **Everything inside the crop is regenerated, including the pixels the mask later
|
|
discards.** The generation is therefore conditioned on a fully noised neighbourhood,
|
|
not on an intact one. An alternative design passes the mask into the sampler as a
|
|
latent noise mask, so only masked pixels are ever denoised and the edge transition
|
|
happens inside the generation rather than as a post-hoc blend. That approach has
|
|
never been tried in this runtime and is an open lever, particularly since the face
|
|
stage is the largest measured quality contributor: removing it costs 3.5 dB inside
|
|
the face boxes on one fixture and 6.1 dB on another.
|
|
- **`FACE_DENOISE_SCALE = 0.5` is best understood as compensation for the above.**
|
|
Regenerating a whole crop and blending is a stronger operation than denoising only
|
|
inside a mask, so the halved strength brings the visible result back into range. Read
|
|
it as coupled to the compositing design rather than as an independently calibrated
|
|
constant: changing the compositing without revisiting the scale would change output
|
|
strength by roughly a factor of two.
|
|
|
|
The maintained implementation preserves the previously oracle-tested strength,
|
|
conditioning, crop, and sampler parameters as compatibility contracts. Its Python
|
|
orchestration, YuNet integration, SAM selection, masks, sizing helpers, and pixel
|
|
compositing are implemented for this runtime. Changing a calibrated model input
|
|
requires the same provider-oracle and identity evaluation as a model change.
|
|
|
|
#### Verified text restoration
|
|
|
|
[`_internal/text_restoration.py`](../src/remove_ai_watermarks/_internal/text_restoration.py)
|
|
implements the opt-in `vae-glyphs` stage. A versioned manifest carries manually
|
|
reviewed strings and source-space line boxes in schema 1, or verified source-space
|
|
geometry alone in schema 2, plus a SHA-256 over decoded RGB width, height, and pixels.
|
|
Validation happens before model loading. The library never treats OCR confidence as
|
|
verification, and geometry-only operators do not need to invent text or script fields.
|
|
|
|
When enabled, the one profile selected before model loading reconstructs the source
|
|
once through its already loaded VAE, runs the ordinary global and face stages, and
|
|
calls the shared restoration compositor. Qwen, SDXL, and Chroma implement this donor
|
|
hook. The optional Qwen-only fidelity anchor first blends 15% of the VAE
|
|
reconstruction into the clean result; it is off by default because the blend returned
|
|
detector-visible OpenAI SynthID in the measured poster fixtures. The compositor derives
|
|
binary source and candidate silhouettes, groups nearby lines, uses LaMa for the initial
|
|
and residual-glyph erase passes, paints fresh silhouette edges, then copies the profile-VAE
|
|
core with a 0.5-pixel feather. The evaluation script
|
|
imports these same mask and compositing helpers so the two implementations cannot
|
|
silently drift. Silhouette crops start 12% of line height beyond each horizontal
|
|
side, then expand each side independently while a foreground component anchored
|
|
inside the detector box still reaches that boundary, up to one line height. They
|
|
extend 8% above and 25% below. The anchored-component gate covers clipped leading
|
|
flourishes, trailing punctuation, icons, and descenders without walking into
|
|
disconnected decoration or background texture.
|
|
|
|
The stage is deliberately narrower than the engine: it rejects tiles, resolution
|
|
caps, humanize, unsharp, and explicitly enabled adaptive polish. An unset polish
|
|
value resolves to off in this mode, including for SDXL, because those combinations change
|
|
geometry or final pixels after the verified layer and have no measured oracle result.
|
|
It remains opt-in because annotations are manual and provider verdicts apply only to
|
|
the exact tested output hashes, not to the mechanism in general.
|
|
|
|
The first full-path H100 check on 2026-08-31 ran each generative engine exactly
|
|
once per source and used that SAME engine's VAE donor; it did not compose Qwen and
|
|
Chroma in one production result. On the two verified multilingual OpenAI posters,
|
|
restoration reduced PaddleOCR CER from 0.303 to 0.160 and 0.279 to 0.136 for Qwen,
|
|
and from 0.213 to 0.130 and 0.204 to 0.108 for Chroma. PSNR improved on all four
|
|
arms (Qwen 26.1 to 29.9 dB and 27.7 to 31.6 dB; Chroma 29.6 to 33.1 dB and 29.7
|
|
to 32.9 dB). The verified CJK Google fixture is the counterexample that prevents
|
|
an unconditional quality claim: Qwen CER stayed 0.074, while Chroma moved from
|
|
0.037 to 0.074 despite better LPIPS, SSIM, PSNR, and text-box pixel error. The
|
|
stage is therefore implemented and fidelity-positive on both OpenAI sources, but
|
|
not assumed to improve every OCR reading. This is a fidelity check, not a provider
|
|
oracle certification. These restoration-study CER values are distinct from
|
|
the longer-string-normalized NED emitted by `scripts/fidelity_metrics.py`;
|
|
the latter's former CER label was corrected without changing its values.
|
|
Exact output hashes and per-box metrics are produced by
|
|
the `chroma_text_restoration_study.py` harness (kept outside this repository)
|
|
under the gitignored
|
|
`out/text-restoration-engine-study/` directory.
|
|
|
|
The 2026-09-15 rerun added the SDXL donor implementation and exercised all three
|
|
profiles on the same three manifests. SDXL improved text-box MAE from 11.319 to
|
|
7.970 and from 10.704 to 7.657 on the two OpenAI posters, while PSNR rose from
|
|
25.83 to 29.24 dB and from 26.17 to 29.04 dB. On the tracked Google CJK carrier
|
|
(`4affd7f27767a445db6abf741355743ba8d95108ad922c9fff045feed8492236`, already
|
|
recorded as SynthID-positive), SDXL at the calibrated 0.50 strength improved
|
|
text-box MAE from 10.738 to 8.564 and PSNR from 23.93 to 24.00 dB. The exact
|
|
restored output
|
|
`035e4919e1d84129a347006c98f19eb4431eb5772788f6d2097b94a2c6baec84`
|
|
was submitted to the Google Gemini oracle after provenance stripping preserved
|
|
its decoded pixels; the sanitized upload hash was
|
|
`3e4f06274a7b735f7d06863fd54dc8146f60aeb9cc8315f86294180deb660b82`.
|
|
The oracle returned `not_detected` on 2026-09-16. This
|
|
certifies that one carrier, manifest, seed, runtime, and output hash; it does not
|
|
turn the compositor into a provider-wide guarantee.
|
|
|
|
A matched stage-isolation check on the 18-face Gemini portrait grid confirms the
|
|
division of responsibility. The visible-cleaned, metadata-stripped control and the
|
|
Z-Image face-only output were both SynthID-positive; Qwen global-only and the full
|
|
Qwen-then-Z-Image output were both clean. The face stage raised identity cosine from
|
|
0.589 to 0.852 and reduced face LPIPS from 0.217 to 0.050 without reintroducing a
|
|
detectable whole-image signal. Thus Z-Image is a masked fidelity repair stage here,
|
|
not the watermark-removal stage. Exact hashes, metrics, strengths, and the one-fixture,
|
|
one-seed caveat are recorded in
|
|
[`data/evaluations/fidelity/face-stage-isolation-2026-08-13.csv`](../data/evaluations/fidelity/face-stage-isolation-2026-08-13.csv).
|
|
|
|
The public Synthid-Bypass v2 graph was subsequently audited at upstream commit
|
|
`3007d035`. Its saved-output path confirms the same division: Qwen-Image-2512
|
|
Lightning plus Canny is global, and Z-Image Turbo exists only inside the masked
|
|
face detailer. The connected face path is YOLOv8-face plus SAM; the MediaPipe nodes
|
|
described by the upstream README and the 1.2-megapixel normalization node do not
|
|
reach `SaveImage`. Upstream also applies its adaptive face strength directly,
|
|
whereas this implementation multiplies it by `FACE_DENOISE_SCALE = 0.5`.
|
|
|
|
A close reproduction on the same portrait fixture kept the control positive and
|
|
made both Qwen global-only and full Qwen-then-Z-Image outputs clean. Applying the
|
|
upstream-strength face pass raised identity from 0.589 to 0.783 and reduced face
|
|
LPIPS from 0.217 to 0.083, but remained worse than this profile's 0.852 and 0.050.
|
|
The published upstream pair 12 was also independently checked positive before and
|
|
clean after, with 0.975 identity. Exact workflow provenance, hashes, metrics,
|
|
oracle outcomes, and the DiffSynth/GGUF, scheduler, detector, and seed caveats are
|
|
recorded in
|
|
[`data/evaluations/fidelity/upstream-v2-reproduction-2026-08-13.csv`](../data/evaluations/fidelity/upstream-v2-reproduction-2026-08-13.csv).
|
|
|
|
### Chroma1 plus Z-Image
|
|
|
|
[`_internal/chroma_zimage_pipeline.py`](../src/remove_ai_watermarks/_internal/chroma_zimage_pipeline.py)
|
|
runs the same two-stage recipe on a Chroma1-HD (`lodestones/Chroma1-HD`,
|
|
Apache-2.0) global pass through diffusers' `ChromaImg2ImgPipeline`. It is the
|
|
answer to issue #88's FLUX.2 request through the model that actually exposes
|
|
strength-controlled img2img in that family. diffusers has since added
|
|
`Flux2KleinInpaintPipeline` with a real strength path. A FLUX.2 klein 4B global
|
|
stage was measured on 2026-09-27 and not adopted: the stock pipeline feeds the
|
|
clean source back as reference tokens and keeps SynthID, and without them klein
|
|
clears both vendors only at lower face and CJK-text fidelity than `qwen-zimage`.
|
|
The full research record is
|
|
[`chroma1-engine-research.md`](chroma1-engine-research.md).
|
|
|
|
The profile also implements the shared verified-text donor hook through its already
|
|
loaded Chroma VAE. It edge-pads native geometry to the /16 latent grid, takes the
|
|
deterministic mode of the encoder distribution rather than sampling it, decodes with
|
|
Diffusers' own image processor, and crops back to the exact source dimensions. Thus
|
|
`auto` still loads and runs only the profile selected by the measured vendor table;
|
|
the presence of a manifest does not introduce a second generative model or override
|
|
that decision. A CPU smoke run against the official Chroma1-HD `AutoencoderKL`
|
|
weights on 2026-08-31 verified the real encode/decode contract and exact restoration
|
|
of a non-grid-aligned source size; model-free tests pin mode-not-sample and padding.
|
|
Chroma-specific full-pipeline provider-oracle certification remains an output-hash
|
|
measurement, not something either check can establish.
|
|
|
|
The profile swaps only the global stage (same inheritance invariant as
|
|
sdxl-zimage, asserted by the same test shape). Three things are bound to the
|
|
calibration and must not drift: the NEUTRAL prompt (`high quality, sharp,
|
|
detailed, faithful to the original` -- deliberately NOT the shared canny-stage
|
|
prompt, which was never calibrated against Chroma1), guidance 5.0, and the
|
|
calibrated `ceil(4 / strength)` requested-step schedule.
|
|
`CHROMA_NOMINAL_STEPS = 4` is a nominal count: Chroma rounds the start
|
|
index, so strengths 0.09 and 0.17 execute five steps, while 0.20, 0.125,
|
|
and 0.50 execute four. Preserve those requests rather than changing the
|
|
calibration to force four effective steps. There is no Canny
|
|
conditioning: the floors were measured on a plain strength pass.
|
|
|
|
The flat vendor floors (`CHROMA_ZIMAGE_*_STRENGTH` in watermark_profiles):
|
|
OpenAI 0.20, Microsoft 0.125, Google 0.50, and Meta 0.17. The source-fresh
|
|
Google sweep supersedes the older 0.40 zero-face / 0.125 face split: fresh CJK
|
|
remained detected at 0.40, and a face-plus-text carrier did not support treating
|
|
face presence as evidence for the easy arm. The runtime has no independently
|
|
measured text-content gate, so every Google carrier now uses the replicated global
|
|
floor. The matched-strength addendum in the research doc remains useful history:
|
|
at the strength each image actually needs, Chroma1 regenerates better almost
|
|
everywhere except face identity (which the inherited Z-Image face stage supplies),
|
|
but high flat floors can destroy dense text and face identity. A Meta arm was measured
|
|
on 2026-08-30/31 (docs/chroma1-engine-research.md, expansion section) and
|
|
does not ship: first-cleans from 0.03 to 0.10 form a continuum,
|
|
`flat_ratio` overlaps the hard and easy clusters, and a high-flat
|
|
easy-arm would misroute the harvest-1 portraits. The worst seed-0
|
|
first-clean is still 0.10; botanical seeds 1 and 2 stay DETECTED at
|
|
that rung, which is the margin the 0.17 floor already holds. A later OpenAI
|
|
holdout invalidated the flat 0.09 conclusion: two carriers first cleared under
|
|
Chroma at 0.10625 and 0.1375, while qwen-zimage cleared both at its existing
|
|
0.07675 operating point. A later source-fresh n=8 Qwen calibration raised its
|
|
OpenAI floor to 0.15625; that does not change the engine decision. Explicit
|
|
Chroma uses the spread-derived 0.20, verified clean three times on both holdout
|
|
carriers, and `auto` routes OpenAI to qwen-zimage.
|
|
|
|
A separate content-balanced check on 2026-08-31 tested the exact production
|
|
OpenAI and Meta floors over 19 prompt-matched image strata per provider. Each
|
|
engine ran in its own sequential H100 invocation, and the comparison isolated
|
|
the global stage because the shared face repair does not change with this choice.
|
|
On OpenAI content, Chroma won SSIM and PSNR on 19/19 pairs, MAE on 18/19, edge
|
|
F1 on 15/19, and LPIPS on 12/19. On Meta content, Qwen won LPIPS and edge F1 on
|
|
18/19 pairs; Chroma's better PSNR on 16/19 did not overcome its systematically
|
|
worse perceptual and edge preservation at the higher required floor. Exact
|
|
two-sided sign-test p-values were 3.8e-6 for the OpenAI SSIM and PSNR directions
|
|
and 7.6e-5 for the Meta LPIPS and edge-F1 directions. The tracked discovery
|
|
inputs and manifests are in
|
|
[`data/evaluations/engine-selection/`](../data/evaluations/engine-selection/),
|
|
the run is reproduced by the `engine_selection_study.py` harness (kept outside
|
|
this repository), and paired analysis
|
|
by `scripts/analyze_engine_selection_study.py`. These content files were locally
|
|
re-encoded and are not watermark oracles; they validate fidelity at already
|
|
oracle-calibrated floors, not removal by themselves. No content-stratum override
|
|
is supported by this pass, so `auto` keeps the measured provenance-cohort table.
|
|
|
|
### SDXL plus Z-Image
|
|
|
|
[`_internal/sdxl_zimage_pipeline.py`](../src/remove_ai_watermarks/_internal/sdxl_zimage_pipeline.py)
|
|
runs the same two-stage recipe on an SDXL global pass. `SdxlZImagePipeline`
|
|
subclasses `TwoStageZImagePipeline` and implements the global stage
|
|
(`_load_global`, `_run_global`) plus the deterministic verified-text VAE donor, so
|
|
the face stage is inherited rather than copied
|
|
and cannot drift between the profiles; a test asserts the shared methods are the
|
|
same objects.
|
|
|
|
Four things are architecture-bound and swap with the model: the ControlNet
|
|
(`xinsir/controlnet-canny-sdxl-1.0`), the four-step distillation LoRA
|
|
(`ByteDance/SDXL-Lightning` at its documented strength 1.0, not the reference graph's
|
|
0.8, which belongs to a different LoRA), the sampler (Euler with trailing spacing, no
|
|
AuraFlow shift), and the latent grid (8 px against Qwen's 16).
|
|
|
|
**Strength is architecture-bound too, and that is the easy mistake.** An SDXL global
|
|
pass can leave SynthID at the strength Qwen needs. The source-fresh 2026-09-15
|
|
matrix found its limiting Google CJK carrier detected through 0.45 and clean at
|
|
0.50, with independent flat-text and face-plus-text replication at 0.50. So this
|
|
profile takes a vendor policy (`SDXL_ZIMAGE_OPENAI_STRENGTH` 0.15,
|
|
`SDXL_ZIMAGE_GEMINI_STRENGTH` 0.50, unknown following Google) rather than
|
|
`resolution_adaptive_denoise`. Flat values are what was measured; no size dependence
|
|
has been established for this stage, so none is asserted.
|
|
|
|
#### Cross-profile strength sweep, 2026-09-14
|
|
|
|
A fresh single-source, seed-0 sweep compared qwen-zimage, chroma-zimage, and
|
|
sdxl-zimage on the same provider-specific carriers. The original content matrix covered
|
|
photorealistic scenes, flat low-detail art, single and multiple faces, dense Latin,
|
|
CJK and Cyrillic text, multilingual posters, and faces plus text. Meta completed
|
|
48/48 real variants, Microsoft 12/12, the original OpenAI sweep 36/36, and Google
|
|
44/48. Thirteen unique adaptive OpenAI extension candidates later closed the two
|
|
missing content quadrants. The minimum
|
|
tested global strengths that cleared every completed stratum were:
|
|
|
|
| Provider | qwen-zimage | chroma-zimage | sdxl-zimage |
|
|
| --- | ---: | ---: | ---: |
|
|
| OpenAI | 0.06 | 0.12 | 0.06 |
|
|
| Microsoft | 0.10 | <= 0.04 | <= 0.08 |
|
|
| Meta | 0.06 | 0.10 | <= 0.08 |
|
|
| Google | 0.35 | 0.50 | 0.50 |
|
|
|
|
`<=` means the first tested rung already cleared, not that its lower boundary is
|
|
known. Google qwen cleared both CJK and Cyrillic only at 0.35; its face series was
|
|
non-monotonic. Google SDXL cleared Cyrillic at 0.35 after detections through 0.28,
|
|
but CJK remained detected at 0.35, 0.40, and 0.45 before clearing at 0.50. Google
|
|
Chroma CJK remained detected through 0.40 and cleared at 0.50; Cyrillic cleared at
|
|
0.40. Chroma face+text first cleared at
|
|
0.22 after detections at 0.08 and 0.16; the face carrier cleared at 0.22 after a
|
|
detection at 0.08. A recovered-quota recheck on 2026-09-15 also detected the face
|
|
carrier at 0.16, closing that tested bracket at `0.16 detected -> 0.22 clean` and
|
|
showing that the shipped 0.125 face arm is too low for this carrier. These are
|
|
candidate rungs at this stage of the study, not replacement defaults: there is one
|
|
source and one seed per case. The independent three-carrier replication below later
|
|
closed the production seed-0 decision; size, crop/resize, JPEG, local-edit, and broader
|
|
multi-seed robustness remain separate holdouts.
|
|
|
|
The original OpenAI set had faces-only and multilingual text-only carriers but no
|
|
explicit no-face/no-text carrier. A fresh 1536x1024 photorealistic rocky-coast scene
|
|
closed that stratum on 2026-09-15. The official OpenAI verifier found Qwen 0.04 and
|
|
Chroma 0.10, then cleared Qwen 0.06 and Chroma 0.12. It found SDXL 0.04 and cleared
|
|
0.06. These are 0.02-grid brackets, not mathematical minima. The public Web verifier
|
|
settled the earlier points and exposed SynthID separately from C2PA. A later Chroma
|
|
0.12 Web request remained in `Verifying` for more than 90 seconds, so it was recorded
|
|
as indeterminate; the preferred official Content Provenance API returned
|
|
`not_detected` for that same processed source hash. Three independent official API
|
|
requests returned HTTP 200 for Chroma 0.12, SDXL 0.04, and SDXL 0.06 respectively. No source control was
|
|
submitted.
|
|
|
|
The other missing OpenAI stratum, one face plus multilingual text, was added with a
|
|
second 1536x1024 carrier. The official API found Qwen 0.04 and cleared 0.06, while
|
|
Chroma 0.10 and SDXL 0.04 already cleared. All responses were HTTP 200 with C2PA
|
|
`not_present`, and original-size review found the face identity, Latin, CJK, Cyrillic,
|
|
and date text preserved at every generated rung. This stratum does not raise any
|
|
global OpenAI candidate; the no-face/no-text scene remains the limiting case for
|
|
Chroma and SDXL.
|
|
|
|
At the original non-Google candidate rungs, Chroma had the best mean quality in all
|
|
three provider cohorts. For Meta, mean LPIPS/SSIM was 0.0320/0.8882 for Chroma 0.10,
|
|
0.0357/0.8595 for Qwen 0.06, and 0.0531/0.8503 for SDXL 0.08. For OpenAI it was
|
|
0.0183/0.8763, 0.0192/0.8595, and 0.0304/0.8459 respectively over the original three
|
|
carriers. Recomputed over all five OpenAI carriers at the freshly verified candidates,
|
|
Qwen 0.06 has the best LPIPS, SSIM, and edge F1 at 0.03073, 0.82461, and 0.89141.
|
|
Chroma 0.12 has the best MAE by a narrow margin at 6.42, but its LPIPS/SSIM is worse
|
|
at 0.05057/0.80940 and Qwen narrowly leads PSNR 27.99 to 27.97 dB; Chroma's
|
|
no-face/no-text scene is the main loss.
|
|
SDXL 0.06 is third on the aggregate perceptual and structural metrics at
|
|
0.03766/0.77202, while keeping the mean Laplacian ratio closest to one. Original-size
|
|
review agrees that there is no single winner on every criterion, but Qwen is the best
|
|
overall OpenAI candidate when perceptual similarity, structure, edges, faces, and text
|
|
are weighted together. Microsoft's
|
|
single flat carrier likewise favored Chroma 0.04 at 0.00265/0.99621 over Qwen
|
|
0.10 at 0.00347/0.99426 and SDXL 0.08 at 0.00324/0.99566; this is a carrier
|
|
comparison, not a multi-source mean. At Google 0.35, SDXL preserved substantially
|
|
more quality than Qwen (mean LPIPS 0.0655 versus 0.1576, SSIM 0.7285 versus
|
|
0.5457), although SDXL needed 0.50 rather than 0.35 on the limiting CJK carrier.
|
|
On that carrier, SDXL 0.50 measured LPIPS/SSIM/PSNR 0.1041/0.7437/23.93 dB,
|
|
substantially better than Chroma 0.50 at 0.2129/0.5853/20.74 dB and Qwen 0.35 at
|
|
0.1834/0.5550/18.93 dB. The newly cleared Chroma 0.22 face and face+text points also lost to
|
|
SDXL at its corresponding tested rungs on every reported metric: LPIPS/SSIM was
|
|
0.0476/0.6657 versus 0.0318/0.7036 for faces, and 0.0506/0.6359 versus
|
|
0.0426/0.7016 for face+text. Original-resolution visual review agreed with that
|
|
direction: Qwen
|
|
0.35 changed the last glyph of the Cyrillic `УЛЫБКОЙ` into a Latin-looking `N` and
|
|
altered substantially more wood grain and background structure; Chroma 0.40 and
|
|
SDXL 0.35 retained the word, with SDXL visually closest to the source. Chroma 0.22
|
|
preserved foreground face identity and text legibility but visibly changed
|
|
background people, room structure, lighting, and confetti. The
|
|
official-oracle access and rolling-quota caveats are recorded in
|
|
[`provider-oracles.md`](provider-oracles.md).
|
|
|
|
The original Google carriers omitted the explicit no-face/no-text stratum. A
|
|
2026-09-14 extension added two real Gemini-generated 2752x1536 carriers: a
|
|
photorealistic alpine lake and a low-detail geometric desert. At the current
|
|
candidate rungs, the flat carrier remained close under all three profiles;
|
|
LPIPS/SSIM was 0.0237/0.9654 for Qwen 0.35, 0.0120/0.9706 for Chroma 0.40,
|
|
and 0.0134/0.9655 for SDXL 0.35. On the photographic carrier, every profile
|
|
visibly regenerated scene detail. SDXL 0.35 was least destructive at
|
|
0.1174/0.4933, ahead of Qwen 0.35 at 0.2070/0.4120 and Chroma 0.40 at
|
|
0.2216/0.4229. Original-resolution review found major mountain, shoreline,
|
|
vegetation, rock, and reflection changes in Qwen and Chroma; SDXL retained the
|
|
main composition and geometry but still changed fine detail. Full four-rung
|
|
ladders produced 24 variants across the two carriers and three profiles. After the
|
|
independent session allowances recovered on 2026-09-15, the official verifier cleared
|
|
both Qwen carriers at the first 0.15 rung. Chroma's flat carrier cleared at 0.12,
|
|
while its photographic carrier closed at `0.20 detected -> 0.30 clean`. SDXL's
|
|
photographic carrier cleared at 0.15 and its quality-qualified flat 0.35 carrier
|
|
cleared directly. These scene cases therefore do not raise the global Google
|
|
candidate above the text-bound values in the table.
|
|
The SDXL flat ladder was quality-non-monotonic: 0.15 introduced severe texture
|
|
and scored LPIPS 0.1283, while 0.35 looked cleaner and scored 0.0134. Repeating
|
|
0.15 and 0.22 with adaptive polish disabled changed the polished outputs by only
|
|
0.51 and 0.57 mean byte levels per channel and left LPIPS effectively unchanged.
|
|
The artifact therefore comes from the low-step SDXL reconstruction, not polish.
|
|
The 0.15 flat variant is rejected before oracle use; 0.35 is the
|
|
quality-qualified SynthID candidate for that one path.
|
|
|
|
A Chroma extension was run on an isolated CUDA research worker from the same
|
|
`c4edc282` checkout. Strength 0.50 retained readable
|
|
CJK and Cyrillic but was already borderline: LPIPS/SSIM/PSNR was
|
|
0.2129/0.5853/20.74 dB and 0.2268/0.5490/20.83 dB respectively. At 0.60 both
|
|
scripts were visibly destroyed and LPIPS exceeded 0.42. The official verifier
|
|
subsequently detected CJK 0.40 and cleared 0.50, making the borderline 0.50 rung the
|
|
fresh global Chroma candidate. Strength 0.60 remains rejected on fidelity.
|
|
|
|
The corresponding SDXL CJK extension used the same checkout and an isolated CUDA
|
|
research worker. The official verifier detected 0.40 and 0.45 and cleared 0.50.
|
|
At 0.50, LPIPS/SSIM/PSNR was 0.1041/0.7437/23.93 dB; original-resolution review
|
|
found the CJK text fully legible and the scene recognizably preserved. SDXL is the
|
|
best-quality Google candidate on the limiting carrier.
|
|
|
|
#### Google boundary replication, 2026-09-15
|
|
|
|
Three new Gemini-generated controls replicated the hard boundary on independent
|
|
content: a busy night market with dense CJK and many faces, a flat CJK poster without
|
|
faces, and a portrait reading a dense CJK newspaper. An isolated CUDA run generated
|
|
both seed 0 and seed 1 at the candidate and neighboring lower rungs, 36 variants total. Production
|
|
uses the fixed seed 0, and only processed seed-0 variants were sent to the official
|
|
verifier; source controls were not uploaded.
|
|
|
|
At the candidate rungs, the official verifier returned `not_detected` for all nine
|
|
profile-by-carrier cells: Qwen 0.35, Chroma 0.50, and SDXL 0.50. A separate lower-rung
|
|
night-market check also cleared Chroma 0.40 and SDXL 0.45, but those two results do
|
|
not lower the policy because the original fresh CJK carrier remained detected there.
|
|
Three quota failures and one duplicate post-success tool-call failure were recorded
|
|
as `unreachable` in separate immutable batches and excluded from the watermark
|
|
count. Every settled batch passed source, upload, manifest, and result hash
|
|
verification.
|
|
|
|
Quality over all three new carriers and both seeds selected SDXL for Google auto
|
|
routing. Mean LPIPS/SSIM/PSNR was 0.0609/0.8178/25.15 dB for SDXL 0.50,
|
|
0.0980/0.7055/21.85 dB for Qwen 0.35, and 0.1269/0.6909/22.53 dB for Chroma 0.50.
|
|
Original-resolution review agreed: SDXL best preserved the night-market headline,
|
|
newspaper layout, faces, and the flat poster. Therefore the production Google floors
|
|
are Qwen 0.35, Chroma 0.50, and SDXL 0.50, and `auto` routes Google to SDXL.
|
|
|
|
A localized follow-up checked that aggregate image metrics were not hiding face or
|
|
text failures. InsightFace `buffalo_l` matched three faces in the newspaper scene and
|
|
15 in the night market one-to-one. Across the four scene-seed pairs, SDXL beat Qwen
|
|
on ArcFace cosine in three and lost one, and beat Chroma in all four. That identity
|
|
result is directional but small: the two-sided sign-test p-values are 0.625 and
|
|
0.125, respectively. Face-crop LPIPS favored SDXL in all four pairs against both
|
|
other profiles, also p=0.125 with only four non-tied pairs. This supports, but does
|
|
not by itself prove, an identity advantage over Qwen.
|
|
|
|
Text was scored separately in seven manually verified fixed regions whose originals
|
|
all produced an exact OCR line: the newspaper masthead, headline, two halves of its
|
|
subhead, the night-market headline, and two poster lines. The metric was the closest
|
|
OCR-line normalized edit distance to that fixed target, across both seeds. SDXL's
|
|
mean was 0.0119 with 13/14 exact region-seed results, versus 0.3770 and 7/14 for Qwen
|
|
and 0.5374 and 4/14 for Chroma. In paired sign tests SDXL was better than Qwen in
|
|
seven non-tied cells and worse in none (two-sided p=0.015625), and better than Chroma
|
|
in ten and worse in none (p=0.001953125). These results cover the selected prominent
|
|
lines, not arbitrary text preservation: fine newspaper body copy was regenerated as
|
|
pseudotext by every profile. The safe claim is therefore that SDXL was the relative
|
|
quality winner on these replicated Google carriers, not that it preserves faces or
|
|
text losslessly.
|
|
|
|
Two current-floor extensions closed the remaining content quadrants. On the
|
|
face-only carrier, rerun at Qwen 0.35, Chroma 0.50, and SDXL 0.50 for seeds 0 and
|
|
1, InsightFace matched all 18 faces. SDXL had the highest mean ArcFace cosine in
|
|
both seeds, 0.828 and 0.842, versus 0.780/0.794 for Chroma and 0.744/0.773 for
|
|
Qwen. Chroma narrowly won face-crop LPIPS at 0.059/0.058, versus 0.060/0.063 for
|
|
SDXL, while SDXL was much closer over the whole frame. Its face Laplacian ratios
|
|
of 1.152/1.176 also show slight texture amplification; Qwen and Chroma instead
|
|
smoothed face detail to roughly half the source level. Original-resolution review
|
|
therefore favors SDXL for overall composition and identity, but not every local
|
|
face metric.
|
|
|
|
The no-face/no-text photographic and flat-graphic carriers were also rerun at the
|
|
same current floors and two seeds. On the photographic scene, SDXL had the best
|
|
LPIPS/SSIM in both seeds: 0.345/0.457 and 0.339/0.461, ahead of Qwen at
|
|
0.413/0.412 and 0.404/0.414 and Chroma at 0.568/0.395 and 0.579/0.401.
|
|
Original-resolution review likewise found SDXL closest, although every profile
|
|
regenerated major landscape detail. The flat graphic reversed the LPIPS result:
|
|
Qwen scored 0.045/0.042, Chroma 0.068/0.065, and SDXL 0.099/0.088; all SSIM values
|
|
were 0.962-0.965. Visual review found model-specific gradient, object, or sun-halo
|
|
artifacts in every output. One carrier and two seeds do not justify a new
|
|
content-adaptive route. Together with the text-only and face-plus-text replication,
|
|
these extensions cover all four primary face/text quadrants at the shipped Google
|
|
floors, while also showing why `auto` is a relative default rather than a universal
|
|
per-image optimum.
|
|
|
|
`requested_steps` is shared by SDXL and Chroma in `two_stage_pipeline.py`,
|
|
which also owns `_target_size` with each profile's grid passed explicitly.
|
|
The step helper exists because the two runtimes truncate differently. DiffSynth sets
|
|
`sigma_start = denoising_strength` and runs every requested step across the shortened
|
|
sigma range; Diffusers img2img truncates the step *count*
|
|
(`init_timestep = int(steps * strength)`), so asking it for four steps at 0.15 executes
|
|
**zero** and returns a bare VAE round-trip.
|
|
|
|
**The face stage keeps its own dtype, and 0.23.0 shipped without that.** The remover
|
|
gives this profile `torch.float16`, because SDXL ships fp16 weights and an fp16-safe
|
|
VAE. That dtype reached the inherited `_load_zimage`, while `_zimage_vram_config()`
|
|
hardcodes bfloat16 for its offload, onload and computation dtypes -- so the Z-Image
|
|
modules were built bf16 and handed fp16 latents, and every image containing a face
|
|
died in the VAE with `Input type (c10::Half) and bias type (c10::BFloat16) should be
|
|
the same`. Every face-stage loader now reads `_face_stage_dtype()`, which returns the
|
|
computation dtype of the VRAM config it is paired with, so the two cannot drift again.
|
|
|
|
Two things hid this. Zero-face inputs never enter `_run_faces`, so the profile looked
|
|
healthy on exactly the images used to time it; and the profile's tests deliberately
|
|
avoid model downloads, so nothing exercised the loader. The lesson is narrower than
|
|
"add a GPU test": inheriting a stage means inheriting its *invariants*, and this one
|
|
was a dtype the subclass silently changed out from under it.
|
|
|
|
Note what the seam is, because it decides where the fix belongs.
|
|
`SdxlZImagePipeline._load_global` hardcodes fp16 for its own ControlNet, VAE and
|
|
pipeline, so `self.torch_dtype` was never actually the global stage's dtype on this
|
|
profile -- its only remaining readers were face-stage code. SAM was the second one:
|
|
it never crashed, because it casts its own inputs and leaves through `.float()`, but it
|
|
was reading the same wrong field and would have re-landed the bug for the next profile
|
|
with a different global dtype. It is routed through the same accessor, which for
|
|
`qwen-zimage` is the bfloat16 it already used.
|
|
|
|
The guard is `test_face_stage_loads_in_its_own_dtype_when_the_global_stage_differs`.
|
|
It asserts the dtype the Z-Image and SAM loaders actually receive, not the accessor
|
|
against the config it is derived from -- that comparison would restate the
|
|
implementation and pass for any consistently wrong value. Both assertions were
|
|
mutation-tested against the pre-fix line. For `qwen-zimage` the whole change is a
|
|
strict no-op: the remover already handed it bfloat16, the same value
|
|
`_face_stage_dtype()` returns.
|
|
|
|
This profile is not deployed. Before it could be, it needs the other three Gemini
|
|
originals, OpenAI re-verified at 0.15, a flat-graphic content class, and a low
|
|
resolution case -- every verdict so far comes from one fixture and one seed.
|
|
|
|
### Measured provider boundaries for qwen-zimage
|
|
|
|
Both ends of the shipped curve now have oracle verdicts, and the shipped curve clears
|
|
everything it has been tested at:
|
|
|
|
A source-fresh GPT Image 2.5 check on 2026-09-15 exercised the complete shipped
|
|
profile rather than only a strength sweep. One 1024 x 1024 API generation from
|
|
Flare and one from Sunburst both validated as current OpenAI C2PA, routed to the
|
|
flat OpenAI strength, and remained OpenAI SynthID-positive after pixel-identical
|
|
metadata removal. The complete `qwen-zimage` profile at the then-current
|
|
0.07675, seed 0, cleared
|
|
both in the official OpenAI Content Provenance API. The Flare output measured
|
|
37.48 dB PSNR / 0.9329 SSIM and the Sunburst output 37.01 dB / 0.9179 against
|
|
their sources. Those fidelity numbers describe two neutral studio-photo
|
|
carriers, not a content-wide quality estimate, and the source-positive/output-negative
|
|
oracle pairs do not replace the wider OpenAI calibration that set the floor.
|
|
|
|
The source-fresh 2026-09-21 expansion supplied that wider calibration. Six
|
|
independent `gpt-image-2` API generations - three YuNet-positive and three
|
|
YuNet-zero-face - each retained SynthID after metadata stripping with decoded RGB
|
|
unchanged. Per-file Qwen ladders, seed 0, were bracketed to 0.00625 against the
|
|
official API:
|
|
|
|
| content | detected through | first clean |
|
|
|---|---:|---:|
|
|
| 1 face, studio portrait | 0.0375 | 0.04375 |
|
|
| 7 faces, group portrait | 0.0875 | 0.09375 |
|
|
| 3 faces plus text | 0.05625 | 0.0625 |
|
|
| zero-face architecture | 0.0375 | 0.04375 |
|
|
| zero-face typography | 0.025 | 0.03125 |
|
|
| zero-face watercolor | 0.075 | 0.08125 |
|
|
|
|
Together with the earlier first-cleans at 0.06225 and 0.0695, the cohort is n=8.
|
|
The observed extrema are 0.03125 and 0.09375, so the established rule gives
|
|
`0.09375 + (0.09375 - 0.03125) = 0.15625`. Both face strata span 0.05 internally,
|
|
while their worst boundaries differ by 0.0125; face presence therefore does not
|
|
support separate floors.
|
|
|
|
| oracle | fixture size | detected at | clean from |
|
|
|---|---|---|---|
|
|
| openai.com/verify | 1.57 MP | 0.06 | 0.08 |
|
|
| Gemini app | 4.33 MP | 0.08 | 0.10 |
|
|
| Gemini app | 0.57 MP | -- | 0.0896 (the curve's own value) |
|
|
| Gemini app | 1.40 MP | -- | 0.1066 (the curve's own value) |
|
|
|
|
Read the last two rows before concluding the curve's low end is under-driven. Against
|
|
the 4.33 MP Gemini boundary the sub-1 MP rungs of 0.084-0.098 look short, but at those
|
|
sizes the curve's own values verify clean, which is what a resolution-scaled
|
|
requirement would predict. There is no measured size at which the shipped curve fails,
|
|
so it is left alone.
|
|
|
|
### Meta Content Seal boundaries for qwen-zimage
|
|
|
|
Measured 2026-08-26 against the anonymous oracle `meta.ai/identification`
|
|
(verdicts read from the settled page text; corpus and full row-level provenance in
|
|
`data/contentseal/manifest.csv`). The original measurement used the
|
|
default resolution-adaptive curve. The retained standalone AI IPTC tag is a
|
|
shared standard and no longer infers Meta. Files need explicit `--vendor meta`
|
|
to select the measured Meta floor from independently known provenance.
|
|
|
|
| source (2.56 MP generation) | detected at | clean from |
|
|
|---|---|---|
|
|
| gen_lighthouse_watercolor | 0.0525 | 0.06 |
|
|
| gen_fox_forest | 0.03 | 0.0375 |
|
|
| gen_night_city | 0.03 | 0.0375 |
|
|
| gen_studio_mug | -- | 0.03 |
|
|
| gen_text_poster | -- | 0.015 |
|
|
|
|
Full spread: worst first-clean boundary (0.0525, 0.06] on lighthouse, easiest
|
|
source already clean at 0.015. Following the same derivation as the OpenAI and
|
|
Microsoft floors (worst clean boundary plus one full observed cross-source
|
|
spread): 0.06 + (0.0525 - 0.015) = 0.0975, rounded up to **0.1**. Shipped as
|
|
`QWEN_ZIMAGE_META_STRENGTH`; `--vendor meta` / `InvisibleOptions.vendor` names
|
|
the cohort explicitly, implying the scrub runs (naming the cohort asserts the
|
|
watermark is present). sdxl-zimage has no measured Meta rung and an
|
|
explicit meta vendor there falls to the conservative unknown 0.50. The default
|
|
resolution-adaptive curve (~0.1305 at 2.56 MP) also clears every measured
|
|
source, so default behavior needed no change. Oracle verdicts carry a
|
|
generation ID and creation timestamp embedded in the watermark payload; both
|
|
survived the 512 px resize and JPEG q85 rows, so payload recovery outlives the
|
|
detection threshold. Oracle session limits are per-IP, server-side, and
|
|
sliding-window: clearing cookies and storage does not reset them, and a burst
|
|
exhausts the window minutes after it reopens.
|
|
|
|
### Static prompt embeddings
|
|
|
|
Both stages prompt with module constants, and at CFG 1.0 DiffSynth's
|
|
`PipelineUnitRunner` reuses the positive embedding for the negative side instead of
|
|
encoding it. So exactly one embedding per stage is ever computed, from text that
|
|
cannot vary at runtime, which makes it cacheable across containers rather than only
|
|
within one pipeline.
|
|
|
|
`_cache_static_prompt_embeddings` therefore persists what the text encoder produced
|
|
under `_model_cache_dir()/prompt-embeddings`, keyed by cache version, model id,
|
|
pipeline output params, and the exact prompt string. Once that file exists,
|
|
`_load_global` and `_load_zimage` drop the text-encoder `ModelConfig` from the model
|
|
stack entirely and serve the stored tensors instead. Measured on an H100 volume in
|
|
August 2026, that removes **15.45 GiB** (Qwen2.5-VL) and **7.49 GiB** (Z-Image) of a
|
|
**87.6 GiB** per-request read, worth a median **11.76 s** and **4.10 s** of load time
|
|
(paired within five containers). The output is **byte-identical** -- the stored
|
|
tensors are the encoder's own -- so this needs no provider-oracle re-verification.
|
|
|
|
Three properties are load-bearing:
|
|
|
|
- **The key self-heals.** A model bump or a prompt edit changes the key, so the next
|
|
container recomputes rather than reading a stale embedding. `_PROMPT_CACHE_VERSION`
|
|
covers a change to the stored shape itself.
|
|
- **The write is atomic.** A torn write must never be readable as a cache hit, so the
|
|
payload lands in a temp file and is renamed into place.
|
|
- **A miss after the encoder was dropped raises.** `require_cache` records that the
|
|
stack was built without a text encoder on the strength of the file; falling back
|
|
would call a model that is not loaded, which surfaces as an opaque crash.
|
|
|
|
`_model_cache_dir()` prefers `HF_HOME` for the same reason: on a scale-to-zero runner
|
|
that is the only persistently mounted path, and anything below it is re-derived per
|
|
request. The YuNet download follows the same root.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_qwen_zimage_pipeline.py`](../tests/test_qwen_zimage_pipeline.py)
|
|
- [`test_cpu_offload.py`](../tests/test_cpu_offload.py)
|
|
|
|
### Tiling
|
|
|
|
[`_internal/tiling.py`](../src/remove_ai_watermarks/_internal/tiling.py) contains pure
|
|
tile planning, feather weights, and tile orchestration.
|
|
|
|
Tiling engages only when requested and the long side exceeds the tile size.
|
|
It avoids an explicit full-image downscale but does not make diffusion
|
|
pixel-preserving. Each tile is still regenerated.
|
|
|
|
It also held a `feather_region_composite` for AI-*enhanced* composites, where only
|
|
the edited region should change. Nothing ever reached it: the `erase` command
|
|
inpaints through `region_eraser`, and the remover's `region` argument was only
|
|
reachable from a module-level convenience wrapper with no callers. Both went.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_tiling.py`](../tests/test_tiling.py)
|
|
|
|
### Postprocessing
|
|
|
|
[`humanizer.py`](../src/remove_ai_watermarks/humanizer.py) contains explicit
|
|
grain, unsharp masking, and adaptive polish helpers.
|
|
|
|
Unsharp masking rounds its floating-point output before clipping to uint8.
|
|
Truncation can turn a flat 128-valued image into 127 on OpenCV builds with
|
|
slightly different floating-point kernels. The explicit rounding regression
|
|
also covers fractional values and saturation at both byte limits.
|
|
|
|
`_apply_postprocessing` in `invisible_engine.py` owns the stage ORDER, and the
|
|
order is a contract: restore the original resolution, unsharp, adaptive polish,
|
|
humanize. Grain runs LAST because `adaptive_polish` measures the image it is given
|
|
against the reference's Laplacian variance, so grain applied first is read as detail
|
|
the image already has. Grain ran first until 0.36.0, and above about `--humanize` 6
|
|
that made the polish a bit-for-bit no-op: measured on a 1092x1440 photo (source
|
|
variance 354.4, blurred stand-in 9.6), `--humanize` 9 and 12 left the polish
|
|
contributing nothing while 0 and 3 let it work. The flag was accepted, the progress
|
|
line still printed, and the run exited 0, so the composition failure was invisible
|
|
from outside. Running the polish first also puts the grain at output resolution
|
|
instead of letting the Lanczos restore smear it.
|
|
|
|
`adaptive_polish` still self-limits when there is no deficit -- a caller driving the
|
|
helpers directly can reach it, and so can an output already sharper than its source
|
|
-- but it now reports that through its `on_skip` callback instead of returning
|
|
silently.
|
|
|
|
`upscaler.py` held an optional Real-ESRGAN path, reachable only when enlarging a
|
|
small image to the minimum-resolution floor. That floor existed to lift small
|
|
inputs toward SDXL's ~1024 training size; when the standalone SDXL and ControlNet
|
|
profiles were removed (`sdxl-zimage` is a later, surviving profile) it
|
|
was forced to 0 on every path, so the module, the `--min-resolution` and
|
|
`--upscaler` options and the `esrgan` extra were all unreachable and went with
|
|
it. Only the `max_resolution` cap can move geometry now, and it only scales down.
|
|
Tiled removal bypasses that cap and processes the native image geometry;
|
|
`tile_size` controls the per-tile workload.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_humanizer.py`](../tests/test_humanizer.py)
|
|
|
|
## Image input and output
|
|
|
|
[`image_io.py`](../src/remove_ai_watermarks/image_io.py) is the shared image
|
|
codec boundary.
|
|
|
|
Contracts:
|
|
|
|
- All package OpenCV file reads and writes use `image_io.imread` and
|
|
`image_io.imwrite`.
|
|
- `to_bgr` normalizes grayscale and alpha-bearing arrays.
|
|
- `read_bgr_and_alpha` and `write_bgr_with_alpha` preserve the alpha plane.
|
|
- `imwrite` returns a success flag; every caller must check it.
|
|
- HEIC, HEIF, and AVIF pixel reads fall back to Pillow plus `pillow-heif` from
|
|
the independent `heif` extra. Metadata scanning does not require that plugin.
|
|
- A visible no-op can preserve the original file bytes.
|
|
- `read_bgr_and_alpha` reads with `IMREAD_UNCHANGED`, so a 16-bit source stays
|
|
16-bit through the pixel path and out through `imwrite`. Anything downstream that
|
|
needs 8 bits narrows a copy for itself.
|
|
Lossless-write tests likewise compare explicit unchanged decodes: default
|
|
color reads may apply EXIF orientation, including for WebP on newer OpenCV builds.
|
|
- `imwrite` accepts `display_tags_from` naming the file whose decode produced the
|
|
pixels, and carries that file's ICC profile and EXIF orientation into the
|
|
re-encoded output (issue #98: cv2's encoders write no container metadata, so a
|
|
Display P3 portrait used to come back desaturated and sideways). PNG and JPEG
|
|
get the tags spliced into the encoded bytes without a second encode; WebP is
|
|
re-saved losslessly through Pillow (libwebp hides spliced chunks from readers
|
|
unless a hand-built VP8X announces them); HEIF writers bake orientation into
|
|
pixels on save. Callers that know the decode state pass
|
|
`orientation_applied=False` for raw pixels and `True` for a transposed
|
|
raster. Dimensions alone cannot distinguish mirrors or 180-degree rotation.
|
|
TIFF and HEIF are the container exceptions on the explicit claim. Every
|
|
reader in the stack (cv2's libtiff path under `IMREAD_UNCHANGED` too, and
|
|
Pillow's TIFF plugin, identically) applies the TIFF orientation tag on
|
|
decode, so `_read_display_tags` ignores the claim for TIFF and decides on
|
|
the IFD's stored `ImageWidth`/`ImageLength` against the raster, via
|
|
`_stored_size` -- `Image.size` already reports the upright geometry there,
|
|
which is what let a turned raster be tagged into a second rotation
|
|
(issue #106). HEIF blanks the tag instead: pillow-heif's opener rewrites
|
|
the EXIF orientation to 1 (the real value waits in
|
|
`info["original_orientation"]`), and libheif turns the raster exactly
|
|
when the container declares an effective `irot`/`imir` transform
|
|
(`isobmff.heif_transform_applied` scans the `meta`/`iprp`/`ipco`
|
|
property store), so `Image.size` matches the raster in both cases and the
|
|
shape heuristic has nothing to compare. A camera HEIC (stored raster,
|
|
EXIF-only rotation, issue #105) re-declares its turn on the output; a
|
|
re-encoded one whose writer baked `irot` decodes upright and is not
|
|
tagged again. AVIF keeps the generic path: this plugin build neither
|
|
blanks its tag nor applies the transform there. The omitted value retains
|
|
the compatibility shape heuristic. Display tags
|
|
are captured before encoding, including when source and destination match.
|
|
|
|
The metadata strip is the other half of that contract: Pillow cannot hold 16-bit
|
|
colour, so `remove_ai_metadata`'s open+save path silently returned a 16-bit PNG at
|
|
8 bits. A PNG whose IHDR declares more than 8 bits now goes through
|
|
`_strip_png_metadata_lossless`, which walks the chunk list, drops the AI-bearing
|
|
text chunks plus `caBX`, re-emits `eXIf` scrubbed of AI tags, and copies IDAT
|
|
verbatim -- the PNG analogue of `_strip_jpeg_metadata_lossless`. A standard
|
|
`iCCP` colour profile survives the default strip and is removed by `--remove-all`;
|
|
the PIL path forwards the profile and the scrubbed EXIF (orientation included) for
|
|
8-bit PNG and WebP outputs too, where they used to be dropped with the rest of
|
|
EXIF. The gate is the depth, not the format: at 8 bits PIL is not lossy, so
|
|
ordinary PNGs keep the shipped path.
|
|
|
|
Regression coverage:
|
|
|
|
- [`test_image_io.py`](../tests/test_image_io.py)
|
|
- [`test_cli_robustness.py`](../tests/test_cli_robustness.py)
|
|
|
|
## Bounded image audit workers
|
|
|
|
`scripts/_isolated_image_workers.py` supervises a fresh Python subprocess for
|
|
each image in `visible_positives.py` and `pill_gate_audit.py`. Each child has
|
|
its own timeout and is killed and reaped when that bound expires. A native
|
|
crash, timeout, or malformed worker reply remains an explicit failed row;
|
|
the parent never retries unsafe decoding in its own process. Threads only
|
|
supervise children. Recall sampling deduplicates by source content SHA-256,
|
|
not by equal detector-confidence vectors.
|
|
|
|
## Adding or changing behavior
|
|
|
|
For a new visible mark:
|
|
|
|
1. create a synthetic detection silhouette;
|
|
2. add or extend a vendor engine;
|
|
3. add one registry entry;
|
|
4. test detection, false positives, localization, and actual pixel change;
|
|
5. update [supported signals](supported-signals.md).
|
|
|
|
For a new metadata signal:
|
|
|
|
1. add the scanner;
|
|
2. add every supported removal placement;
|
|
3. verify the output through `strip_and_verify`;
|
|
4. add identification and removal tests;
|
|
5. update [supported signals](supported-signals.md) and, when relevant,
|
|
[the watermarking landscape](watermarking-landscape.md).
|
|
|
|
For a diffusion change:
|
|
|
|
1. keep model-free logic in pure helpers where possible;
|
|
2. test option propagation and dispatch without downloading models;
|
|
3. run a real model smoke for the changed model path;
|
|
4. treat provider-verifier results as specific to the exact checked output;
|
|
5. update [known limitations](known-limitations.md).
|