Compare commits
No files matched your search
@@ -1,6 +1,6 @@
|
||||
---
|
||||
globs: ["src/**/*.py", "tests/**/*.py", "scripts/**/*.py", "pyproject.toml", "uv.lock", "maintain.sh", ".github/workflows/*.yml"]
|
||||
description: Command contracts, project gate, typing boundaries, and model-adjacent test invariants.
|
||||
description: Command contracts, project gate, typing boundaries, model-adjacent test invariants, and the detection-path measurement rule.
|
||||
---
|
||||
|
||||
# Development invariants
|
||||
@@ -11,23 +11,158 @@ Every single-image command declares `source` with `dir_okay=False`; `batch` decl
|
||||
|
||||
Exit-code and no-signal behavior is a public contract. Read the command-line section of [`../../docs/module-internals.md`](../../docs/module-internals.md) before changing it.
|
||||
|
||||
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
|
||||
|
||||
`device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. It is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds.
|
||||
|
||||
The same rule applies to install hints: name the extra that actually makes the command work (`qwen-zimage`, not `diffusion`), and keep the printed command shell-quoted -- bare `pkg[extra]` is a glob in zsh.
|
||||
|
||||
## Local gate
|
||||
|
||||
Run `bash maintain.sh` from the repository root. The authoritative type gate is scoped to `src/`; full-project Pyright can exhaust Node memory on the ML dependency graph.
|
||||
|
||||
Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.
|
||||
|
||||
From a worktree, `uv run` imports the package from the MAIN checkout -- that is where the editable install points. A script measuring a worktree's edit must insert that worktree's `src` at `sys.path[0]` and assert `module.__file__` resolves inside it, or it silently compares unmodified code against itself.
|
||||
|
||||
## Visible-mark example gallery
|
||||
|
||||
Every registered mark carries a committed example: `data/fixtures/visible/<key>/example.png`
|
||||
(image marks) and `example.mp4` (video marks), regenerated by
|
||||
`scripts/render_visible_examples.py`. `tests/test_visible_examples.py` is the canary: it
|
||||
fails when a mark is registered without an example and when an engine stops detecting its
|
||||
own example. The examples are deterministic synthetic composites of the committed
|
||||
silhouettes. When the generator fails after a geometry or gate change, fix the generator
|
||||
(and the engine) together; do not hand-edit the binaries.
|
||||
|
||||
Visible-mark content must distinguish the registry key, the provider's canonical
|
||||
product name, and the literal text rendered in the mark whenever those differ. Verify
|
||||
external provider names against the provider's official source rather than inferring
|
||||
them from an asset filename or OCR text.
|
||||
|
||||
## Model-adjacent tests
|
||||
|
||||
Do not classify an entire module as untestable because its main path downloads a model. Keep pure behavior covered without downloads, including:
|
||||
|
||||
- target-size selection in `test_invisible_engine.py`;
|
||||
- unsharp and adaptive-polish helpers in `test_humanizer.py`;
|
||||
- mocked device fallback in `test_img2img_runner.py`;
|
||||
- tiling geometry and blending in `test_tiling.py`;
|
||||
- prompt-embedding cache keying, storage round-trip, and the cross-pipeline reuse
|
||||
that lets a stack load without its text encoder, in `test_qwen_zimage_pipeline.py`.
|
||||
that lets a stack load without its text encoder, in `test_qwen_zimage_pipeline.py`;
|
||||
- the face stack's dtype, in `test_qwen_zimage_pipeline.py`. A subclass that changes
|
||||
the pipeline dtype for its own global model must not change the inherited face
|
||||
stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a
|
||||
face. When one profile inherits another's stage, guard the invariants that stage
|
||||
relies on, not just the code path.
|
||||
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
|
||||
mirror another, compare them field by field rather than pinning the values you happen
|
||||
to know about, so the next field added on one side and not the other fails at the
|
||||
seam. Two of these defaults drifted in practice and neither needed a GPU to catch;
|
||||
the incident is recorded in `docs/module-internals.md`. Keep the comparison free of
|
||||
an exception table: a field that needs one is a field that belongs elsewhere, which
|
||||
is what `force` turned out to be.
|
||||
|
||||
A defaults comparison is not a forwarding test, and the two fail differently. Pin the
|
||||
VALUE at the seam, not just the name -- `_run_invisible` passed the whole suite with
|
||||
`controlnet_conditioning_scale` hardcoded, because nothing asserted the caller's value
|
||||
arrived. `test_every_field_arrives_at_the_engine_with_the_caller_s_value` drives the
|
||||
real seam with every field set off its default, so one test covers the whole bag
|
||||
instead of one assertion per knob.
|
||||
|
||||
Count the seams before believing a knob is covered. Each of `force` and
|
||||
`controlnet_conditioning_scale` reaches the engine through TWO paths -- `remove_all`
|
||||
versus `remove_batch(mode="all")` for the first, `_run_invisible` versus `_batch_engine`
|
||||
for the second -- and in both cases guarding one path left the other free to hardcode a
|
||||
constant with a green suite. The mode-parametrized guards in
|
||||
`TestRemoveBatchLibrary::test_force_reaches_the_scrub_gate_in_every_scrubbing_mode` and
|
||||
`TestBatchCommand::test_batch_controlnet_scale_flows_to_the_cached_engine` exist because
|
||||
that is what actually happened.
|
||||
|
||||
Use availability checks only for paths that actually load large models.
|
||||
|
||||
A verdict branch must be reachable with the dependency stack as shipped. High-confidence
|
||||
C2PA attribution required `signingCredential.trusted`, a code the reader emits only when a
|
||||
trust anchor list is loaded, and none ships -- so the branch was dead in production for
|
||||
every vendor while a hand-built info dict stamping that code kept it green from 0.27.0
|
||||
through 0.30.0. Assert reachability against the committed fixtures rather than a synthesized
|
||||
status set; `TestIdentifyRealSamples::test_no_committed_fixture_reports_a_trusted_signer`
|
||||
is that guard. Read a capability the stack does not have as a missing input, never as a
|
||||
negative finding.
|
||||
|
||||
## One measurement, one gate seam
|
||||
|
||||
A detector is split into a trust-level-blind scan and a verdict that applies the
|
||||
threshold, so `detect` and `detect_both` reach the same numbers by construction. Two
|
||||
rules follow, and both were broken in practice before they were written down:
|
||||
|
||||
- A per-mark demotion goes in the `_post_gate` hook (or, for a whole-scan precondition
|
||||
like LibLibAI's size floor, in `_scan`) -- never in a `detect` override. An override
|
||||
is invisible to `detect_both`, so the RunningHub and Yuanbao anchor gates silently
|
||||
stopped applying on the arbiter's perception path. `TestSinglePassPerception` is the
|
||||
guard: it asserts `detect_both` equals two `detect` calls field for field.
|
||||
- Detection and the removal mask must read ONE sweep. The winning box travels on
|
||||
`TextMarkDetection.match_box` and the registry threads the detection into the mask
|
||||
builder; a mask path that re-runs its own sweep is how the two drift apart.
|
||||
|
||||
The C2PA manifest-store JSON is NOT stable across reads: the reader regenerates manifest
|
||||
URNs and instance ids. Compare the derived `c2pa_info`, never the raw store.
|
||||
|
||||
A third seam reaches the same verdict: `collect_metadata_record` ->
|
||||
`evidence_from_metadata_record` -> `identify_from_evidence`, the path a caller uses when
|
||||
collection and verdict run on different machines. Its contract is equality with
|
||||
`identify(path, check_visible=False, check_invisible=False)` on the same image, and it
|
||||
can break from EITHER side -- a region the collector stops walking, or a placement the
|
||||
file path learns to read and the record does not. `tests/test_metadata_record.py` pins
|
||||
it over the tracked fixtures; a separate local evaluation corpus catches placements the
|
||||
fixtures do not cover. Change either side and re-run both.
|
||||
|
||||
Before changing anything in the detection path, record the detectors' exact verdicts
|
||||
over a local sample first and diff them after. A refactor here is only correct if that
|
||||
record is byte-identical, and a green test suite does not establish that on its own. A
|
||||
change that is meant to FIX detection is the exception that proves the rule: the diff
|
||||
must then be exactly the files you intended to change, named in advance.
|
||||
|
||||
When the baseline is a published release rather than the previous commit, get it with
|
||||
`uv run --isolated --no-project --with 'remove-ai-watermarks[heif]==<version>' python <script>`:
|
||||
it resolves that release from PyPI without touching the working tree or the editable
|
||||
install. Quote the extra -- bare `pkg[extra]` is a glob in zsh.
|
||||
|
||||
Two corollaries in `dwt_dct.py`, where "close enough" has an exact meaning. The bit test
|
||||
is `peak % 36 > 18.0` and for uint8 input the exact Haar value is a multiple of 0.5, so it
|
||||
lands ON the threshold once per 72 blocks and a 1-ulp difference flips real bits: leave the
|
||||
transform to `pywt` however slow it looks, because its C convolution contracts into an FMA
|
||||
that numpy has no ufunc for (`docs/module-internals.md` carries what that costs). And
|
||||
`tests/test_invisible_watermark.py` is `skipif(not is_available())` -- without the `detect`
|
||||
extra the upstream-parity test never runs, so a green suite is not evidence here and the
|
||||
verdict record is not optional.
|
||||
|
||||
## A certified operating point is data, not a constant
|
||||
|
||||
The video SynthID default is only meaningful as a row in
|
||||
`data/evaluations/video-synthid-oracle.csv`, so
|
||||
`test_shipped_defaults_match_a_certified_manifest_row` derives the pin from that
|
||||
manifest instead of restating literals. Pinning `noise_std` alone had let `long_side`
|
||||
and `fps` -- two thirds of what the oracle was actually shown -- move with a green
|
||||
suite.
|
||||
|
||||
The certified profile is a perturbation-to-signal ratio, not a bare `noise_std`, so
|
||||
the latent scaling factor is gated in `load_video_vae_runtime`, carried on
|
||||
`VideoVaeRuntime`, and passed into encode and decode: the validated value and the
|
||||
applied value are one measurement. Anything that produces oracle evidence loads
|
||||
through that function. `video_synthid_sweep.py` hand-rolled the load and was the one
|
||||
path exempt from the gate it exists to feed, which is exactly backwards.
|
||||
|
||||
Prove a video-path refactor the same way the detection path is proven, and without
|
||||
needing an oracle carrier: build a clip from a tracked fixture with ffmpeg, run the
|
||||
engine before and after, and require an identical output sha256. Keep the generated
|
||||
media outside the repository.
|
||||
|
||||
Frame sampling is compared as `timestamp + 1e-9 >= next_sample_time`, so mutating
|
||||
that `>=` to `>` changes nothing and a green suite proves nothing. Mutate the phase
|
||||
or the period instead -- starting the accumulator half a period late shifts the
|
||||
selection by one frame while keeping the count identical, which is the drift a
|
||||
frame-count check cannot see and what
|
||||
`test_pairing_follows_the_engine_sampling_rule_not_just_the_frame_count` exists to
|
||||
catch.
|
||||
|
||||
Environment setup, dependency recovery, CI behavior, and fixture policy: [`../../docs/development.md`](../../docs/development.md).
|
||||
@@ -1,14 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"WebSearch",
|
||||
"WebFetch"
|
||||
]
|
||||
},
|
||||
"enabledPlugins": {
|
||||
"pyright-lsp@claude-plugins-official": true,
|
||||
"context7@claude-plugins-official": true,
|
||||
"code-simplifier@claude-plugins-official": true,
|
||||
"claude-md-management@claude-plugins-official": true
|
||||
}
|
||||
}
|
||||
@@ -1,11 +1,10 @@
|
||||
name: Distribute on release
|
||||
|
||||
# Fans a published GitHub Release out to the channels that need a nudge.
|
||||
# PyPI is handled by publish.yml; the conda-forge channel is handled by its
|
||||
# autotick bot. This workflow updates the repository recipe and event-drives
|
||||
# the three channels that would otherwise be manual:
|
||||
# PyPI is handled by publish.yml. This workflow event-drives the three channels
|
||||
# that would otherwise be manual:
|
||||
# - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
|
||||
# - HF Space: factory-rebuild so it reinstalls the latest sdist from PyPI.
|
||||
# - HF Space: factory-rebuild the version already pinned in the Space repo.
|
||||
# - ComfyUI: sync, test, and publish the node against the exact library release.
|
||||
# All wait for the freshly published sdist to appear on PyPI first, since the
|
||||
# Release event fires in parallel with publish.yml's upload.
|
||||
@@ -89,34 +88,6 @@ jobs:
|
||||
git commit -m "remove-ai-watermarks $VERSION" || { echo "Formula already current"; exit 0; }
|
||||
git push
|
||||
|
||||
conda-recipe:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- name: Checkout main
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
ref: main
|
||||
- name: Bump the repository conda recipe
|
||||
env:
|
||||
SHA: ${{ needs.resolve.outputs.sha }}
|
||||
VERSION: ${{ needs.resolve.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
python scripts/sync_conda_recipe.py --version "$VERSION" --sha256 "$SHA"
|
||||
git diff --check
|
||||
if git diff --quiet -- packaging/conda/recipe.yaml; then
|
||||
echo "Conda recipe already current"
|
||||
exit 0
|
||||
fi
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git add packaging/conda/recipe.yaml
|
||||
git commit -m "Sync conda recipe with v$VERSION"
|
||||
git push
|
||||
|
||||
hf-space:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -28,13 +28,26 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Cross-OS x Python-floor/recent. The default-install surface only
|
||||
# needs the core deps plus dev tooling (no GPU extra): the model-running
|
||||
# paths are availability-gated and skip when torch/diffusers are absent,
|
||||
# so this matrix exercises metadata, identify, visible, and the cv2
|
||||
# eraser on every OS without pulling the heavy ML stack.
|
||||
os: [ubuntu-latest, macos-latest, windows-latest]
|
||||
python-version: ["3.10", "3.12"]
|
||||
# Exercise every supported minor. The floor and latest minor run on all
|
||||
# operating systems; intermediate minors run on Linux to bound the
|
||||
# matrix without leaving a declared Python version untested.
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.11"
|
||||
- os: macos-latest
|
||||
python-version: "3.11"
|
||||
- os: windows-latest
|
||||
python-version: "3.11"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.12"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.13"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.14"
|
||||
- os: macos-latest
|
||||
python-version: "3.14"
|
||||
- os: windows-latest
|
||||
python-version: "3.14"
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
@@ -45,6 +58,27 @@ jobs:
|
||||
- name: Run tests
|
||||
run: uv run pytest -q
|
||||
|
||||
transport-contract-python-314:
|
||||
name: versioned transport contracts py3.14
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
python-version: "3.14"
|
||||
- name: Create clean runtime contract environment
|
||||
run: uv venv --python 3.14 .venv-contract
|
||||
- name: Install runtime contract dependencies
|
||||
run: >-
|
||||
uv pip install --python .venv-contract/bin/python
|
||||
".[pixels,heif]" "numpy>=2,<3" "pytest>=8"
|
||||
- name: Run versioned transport contract tests
|
||||
run: >-
|
||||
.venv-contract/bin/python -m pytest -q
|
||||
tests/test_forensic_metadata.py
|
||||
tests/test_metadata_record.py
|
||||
tests/test_pixel_evidence.py
|
||||
|
||||
video-e2e:
|
||||
name: video full-clip end-to-end
|
||||
runs-on: ubuntu-latest
|
||||
@@ -62,3 +96,4 @@ jobs:
|
||||
run: >-
|
||||
uv run pytest -vv -o faulthandler_timeout=60
|
||||
tests/test_video.py::TestVideoVisibleFullClip
|
||||
tests/test_video_fidelity_probe.py
|
||||
@@ -20,8 +20,7 @@ Thumbs.db
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# SynthID corpus reference fills (synthetic black/white calibration tiles,
|
||||
# regenerable; the labeled pos/neg/cleaned images ARE tracked, see README)
|
||||
# Generated calibration inputs
|
||||
data/synthid_corpus/refs/
|
||||
|
||||
# Reference materials
|
||||
@@ -31,15 +30,15 @@ _refs/
|
||||
yolov8n.pt
|
||||
.coverage
|
||||
|
||||
# Local prototype and evaluation outputs (scripts/chroma_scrub_prototype.py and
|
||||
# friends); the durable record of an oracle verdict lives under data/evaluations/
|
||||
out/
|
||||
|
||||
# Claude Code local settings
|
||||
.claude/settings.local.json
|
||||
.claude/scheduled_tasks.lock
|
||||
|
||||
# Visible-watermark alpha calibration. The solid black/gray/white CAPTURES are
|
||||
# committed (content-free: a solid colour + the watermark; the source for
|
||||
# scripts/visible_alpha_solve.py so the alpha assets are reproducible). The
|
||||
# synthetic seeds (regenerable) and any real-content validation download (a real
|
||||
# generated scene, kept local for privacy) are NOT committed.
|
||||
# Generated calibration inputs and local evaluation artifacts
|
||||
data/doubao_capture/seeds/
|
||||
data/jimeng_capture/seeds/
|
||||
data/jimeng_capture/captures/jimeng_content_*.png
|
||||
@@ -48,13 +47,18 @@ data/gemini_capture/captures/gemini_content_*.png
|
||||
data/samsung_capture/seeds/
|
||||
data/samsung_capture/captures/samsung_content_*
|
||||
|
||||
# Leftover GFPGAN weights dir from the retired face-restore experiments
|
||||
# (GFPGAN wrote RetinaFace/parsing weights to a CWD ./gfpgan/weights/ working
|
||||
# dir on first use). Runtime artifact, never committed.
|
||||
# Runtime model artifacts
|
||||
gfpgan/
|
||||
|
||||
# Qwen ControlNet experiment outputs (throwaway eval; never the committed corpus)
|
||||
scripts/_qwen_exp_out/
|
||||
|
||||
# Local-only working data for analysis (not a committed corpus; never tracked)
|
||||
# Local evaluation data
|
||||
.local-eval/
|
||||
|
||||
# Other harnesses' generated config. Claude's sync owns these; a committed copy
|
||||
# is a second source nobody reconciles. Anchored so a same-named source file
|
||||
# deeper in the tree is not caught.
|
||||
/.agents/
|
||||
/.codex/
|
||||
/sch_*.yaml
|
||||
/.opencode/
|
||||
/opencode.json
|
||||
/opencode.jsonc
|
||||
@@ -33,7 +33,7 @@ Optional features and installation groups are documented in [`docs/installation.
|
||||
|
||||
Command, gate, typing, and model-test invariants auto-load from [`.claude/rules/development.md`](.claude/rules/development.md). Environment recovery, CI behavior, and fixture policy live in [`docs/development.md`](docs/development.md).
|
||||
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Keep the source-distribution public allowlist.
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Treat the release as complete only after PyPI, Homebrew, the Hugging Face Space, and the ComfyUI Registry are verified; conda is not published. Keep the source-distribution public allowlist.
|
||||
|
||||
## Module architecture
|
||||
|
||||
@@ -51,4 +51,4 @@ Topic-specific rules live in `.claude/rules/*.md` and are auto-loaded when match
|
||||
|
||||
| File | Covers |
|
||||
|---|---|
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, and model-adjacent tests |
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, and the detection-path measurement rule |
|
||||
@@ -2,17 +2,19 @@
|
||||
|
||||
Remove AI provenance marks from images and video you generated yourself:
|
||||
|
||||
- known visible labels such as the Gemini sparkle and vendor text marks;
|
||||
- known visible labels such as the Google Gemini sparkle watermark and vendor text marks;
|
||||
- invisible pixel watermarks through diffusion regeneration;
|
||||
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
|
||||
|
||||
Video support covers provenance identification, complete visible-plus-metadata
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo, and
|
||||
Kling mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo AI, and
|
||||
Kling AI mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
removal.
|
||||
|
||||
> Try it online at [raiw.cc](https://raiw.cc) if you do not want to install Python
|
||||
> or run diffusion models locally.
|
||||
> [raiw.cc](https://raiw.cc) runs this library as a hosted service, with the GPU
|
||||
> included and nothing to install. Visible mark and metadata removal at Standard
|
||||
> output up to 12 MP are free there; original resolution above 12 MP and invisible
|
||||
> watermark removal are paid.
|
||||
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
@@ -39,10 +41,14 @@ removal.
|
||||
| Remove a registered visible AI mark from video | `video visible` | No |
|
||||
| Process a directory of videos | `video batch` | Depends on mode |
|
||||
| Remove video SynthID with the certified VAE profile | `video invisible` | Recommended |
|
||||
| Regenerate an image to disrupt invisible watermarks | `invisible` | Recommended |
|
||||
| Regenerate an image to disrupt invisible watermarks | `invisible` | Required (CUDA) |
|
||||
| Run visible, invisible, and metadata removal | `all` | Recommended |
|
||||
| Process a directory | `batch` | Depends on mode |
|
||||
|
||||
Microsoft Paint and Photos InvisMark declarations are routed automatically to
|
||||
pixel regeneration. The `all` command removes both the hidden pixel watermark and
|
||||
its linked C2PA manifest; metadata stripping alone removes only the manifest.
|
||||
|
||||
## Installation modes
|
||||
|
||||
| Need | Install |
|
||||
@@ -52,13 +58,13 @@ removal.
|
||||
| Visible video processing | `remove-ai-watermarks[video]` |
|
||||
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
|
||||
| Torch-free DWT-DCT detection | `remove-ai-watermarks[detect]` |
|
||||
| Diffusion removal | `remove-ai-watermarks[diffusion]` |
|
||||
| Every production feature | `remove-ai-watermarks[all]` |
|
||||
| Invisible image removal (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| Every production feature available on the active Python | `remove-ai-watermarks[all]` |
|
||||
|
||||
Lower-level and specialized extras include `pixels`, `heif`, `trustmark`,
|
||||
`migan`, `lama`, `esrgan`, and `qwen-zimage`. The
|
||||
`migan`, `lama`, and `diffusion`. The
|
||||
[installation guide](docs/installation.md#feature-extras) documents their exact
|
||||
dependency composition and model requirements.
|
||||
dependency composition, Python compatibility, and model requirements.
|
||||
|
||||
## Quick start
|
||||
|
||||
@@ -92,6 +98,8 @@ Strip metadata without running visible inpainting or diffusion:
|
||||
remove-ai-watermarks metadata image.png --remove -o clean.png
|
||||
```
|
||||
|
||||
Without `-o` this command overwrites the source in place.
|
||||
|
||||
Inspect or remove AI metadata from an MP4, MOV, M4V, WebM, MKV, AVI, or FLV
|
||||
file:
|
||||
|
||||
@@ -100,10 +108,13 @@ remove-ai-watermarks video metadata input.mp4 --check
|
||||
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
|
||||
```
|
||||
|
||||
The metadata command does not transcode video or audio streams. When `-o` is
|
||||
omitted it writes `<source>_clean` and preserves the original. MP4 and MOV
|
||||
The `video metadata` command does not transcode video or audio streams. Unlike
|
||||
the image command above, when `-o` is omitted it writes `<source>_clean` and
|
||||
preserves the original. MP4 and MOV
|
||||
inspection includes the native TC260 `AIGC` tag in
|
||||
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload.
|
||||
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload,
|
||||
plus the QuickTime-form `meta` variants Doubao's iOS export writes (a bare
|
||||
`meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list).
|
||||
MKV and WebM inspection reads the normative
|
||||
`Segment.Tags.Tag.SimpleTag` placement. AVI uses `LIST/INFO/AIGC`, while FLV
|
||||
uses `script.onMetaData.AIGC`. The non-ISOBMFF formats are remuxed with stream
|
||||
@@ -158,8 +169,8 @@ the specificity order shown below. Pass an explicit mark to restrict detection
|
||||
to one provider.
|
||||
Sora covers the moving Sora 2 mascot and wordmark. Veo covers both the current
|
||||
four-point diamond and the legacy `Veo` text. Seedance covers the fixed boxed
|
||||
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo covers the composite
|
||||
`MINIMAX | hailuo AI` label, and Kling covers the bottom-right `KLING AI`
|
||||
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo AI covers the composite
|
||||
`MINIMAX | hailuo AI` label, and Kling AI covers the bottom-right `KLING AI`
|
||||
label with its version suffix. A completed encode is published atomically. No
|
||||
output is written when no stable mark is found.
|
||||
HDR, PQ/HLG, and greater-than-8-bit inputs are rejected before encoding rather
|
||||
@@ -180,10 +191,11 @@ check. Google does not publish a local decoder, so a fresh provider check
|
||||
remains useful for unusually important files or after provider changes, but it
|
||||
is not a product result state.
|
||||
|
||||
For invisible watermark removal, install the diffusion dependencies:
|
||||
For invisible watermark removal, install the `qwen-zimage` extra. **An NVIDIA GPU
|
||||
is required**: all profiles are CUDA-only, and there is no CPU or MPS fallback.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[diffusion]"
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png
|
||||
```
|
||||
|
||||
@@ -194,6 +206,20 @@ image came from an AI generator, add `--force`:
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
Typography-heavy images can opt into the experimental verified-text post-pass.
|
||||
It accepts manually reviewed strings and line boxes or an operator-verified
|
||||
geometry-only manifest; it never treats raw OCR output as ground truth or runs
|
||||
automatically:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
See the [CLI guide](docs/cli.md#restore-operator-verified-text) for the manifest
|
||||
schema, compatibility restrictions, and oracle caveats.
|
||||
|
||||
See the [installation guide](docs/installation.md) for Homebrew, uv, optional
|
||||
features, and development setup.
|
||||
|
||||
@@ -205,16 +231,25 @@ features, and development setup.
|
||||
| --- | --- |
|
||||
|  |  |
|
||||
|
||||
The `after` raster is generated from the tracked `before` raster by the public path:
|
||||
|
||||
```bash
|
||||
uv run remove-ai-watermarks visible demo_banana_before.png \
|
||||
--backend cv2 -o demo_banana_after.png
|
||||
```
|
||||
|
||||
### High quality invisible removal
|
||||
|
||||
The `qwen-zimage` profile is the highest fidelity option for face heavy images.
|
||||
It is CUDA only and uses a much larger model stack than the default ControlNet
|
||||
profile.
|
||||
`qwen-zimage` is the default profile: a Qwen-Image-2512 Lightning pass under Canny
|
||||
ControlNet, followed by SAM-masked Z-Image repair of any detected face. The
|
||||
alternative, `sdxl-zimage`, swaps the global stage for SDXL and keeps the same face
|
||||
stage. A third profile, `chroma-zimage`, uses the Apache-2.0 Chroma1 global pass
|
||||
with its own flat vendor floors; see `docs/chroma1-engine-research.md` for the
|
||||
calibration. All are CUDA only.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --force
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
| OpenAI example before | OpenAI example after |
|
||||
@@ -273,7 +308,7 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
```
|
||||
|
||||
CPU offload lowers CUDA memory pressure by moving model components between CPU
|
||||
and GPU. It is slower and has no effect on CPU or MPS.
|
||||
and GPU, at the cost of speed.
|
||||
|
||||
### Process a directory
|
||||
|
||||
@@ -286,8 +321,10 @@ remove-ai-watermarks batch ./images --mode all
|
||||
|
||||
Visible mark support includes:
|
||||
|
||||
- Google Gemini and Nano Banana sparkle;
|
||||
- Doubao, Jimeng, Qwen, Kling, Baidu, LibLibAI, and RunningHub labels;
|
||||
- Google Gemini and Nano Banana visible sparkle watermark;
|
||||
- Doubao, the Jimeng wordmark and top-left `AI生成` pill, Qwen, Kling AI,
|
||||
Yuanbao, Baidu, LiblibAI, and RunningHub labels;
|
||||
- one calibrated Microsoft top-right white AI-badge variant;
|
||||
- one calibrated Samsung Galaxy AI label variant.
|
||||
|
||||
Metadata and provenance inspection covers C2PA, EXIF, XMP, IPTC, common
|
||||
@@ -296,7 +333,9 @@ signals. Optional decoders add support for open DWT-DCT watermarks and Adobe
|
||||
TrustMark.
|
||||
|
||||
The exact support matrix, including important locale and detector limits, lives
|
||||
in [supported signals](docs/supported-signals.md).
|
||||
in [supported signals](docs/supported-signals.md). The
|
||||
[synthetic example gallery](data/fixtures/visible/README.md) carries one canonical
|
||||
sample for every registered image and video mark.
|
||||
|
||||
## How it works
|
||||
|
||||
@@ -350,8 +389,9 @@ The high level API accepts a file path or a BGR NumPy array. For path inputs it
|
||||
also reads provenance metadata, preserves alpha, and can strip AI metadata from
|
||||
the written result.
|
||||
|
||||
See the [Python API guide](docs/python-api.md) for visible removal, provenance
|
||||
inspection, metadata stripping, and diffusion usage.
|
||||
See the [Python API guide](docs/python-api.md) for visible removal, the full
|
||||
`remove_all` and `remove_batch` pipeline, provenance inspection, metadata
|
||||
stripping, and diffusion usage.
|
||||
|
||||
## ComfyUI
|
||||
|
||||
@@ -370,7 +410,7 @@ invisible removal.
|
||||
detail.
|
||||
- Visible video removal recognizes the moving Sora 2 wordmark, the current Veo
|
||||
diamond plus legacy `Veo` text, the Seedance boxed `AI` label, and the fixed
|
||||
Dola, Hailuo, and Kling labels. It does not recognize the older Sora Turbo
|
||||
Dola, Hailuo AI, and Kling AI labels. It does not recognize the older Sora Turbo
|
||||
corner swirl or unregistered layouts from those providers.
|
||||
The classical OpenCV backend can smear structured backgrounds; use MI-GAN or
|
||||
LaMa when recovery quality matters.
|
||||
@@ -378,8 +418,9 @@ invisible removal.
|
||||
The shipped profile is oracle-certified, but no public local decoder can
|
||||
certify an arbitrary output at runtime. Recheck unusually important outputs
|
||||
after provider changes.
|
||||
- `qwen-zimage` requires CUDA. The other diffusion profiles also support the
|
||||
devices listed by `remove-ai-watermarks invisible --help`.
|
||||
- Invisible-watermark removal requires CUDA. All profiles refuse any other
|
||||
device at construction rather than falling back to one that cannot run them.
|
||||
Visible removal, metadata stripping and `identify` still run anywhere.
|
||||
- Provider watermark systems can change. Validate important outputs with the
|
||||
provider's own verifier when one is available.
|
||||
|
||||
@@ -392,6 +433,14 @@ verifier verdict blank:
|
||||
uv run --extra video --extra diffusion python scripts/video_synthid_sweep.py input.mp4 -o sweep/
|
||||
```
|
||||
|
||||
To score what an output actually cost, use `scripts/video_fidelity_probe.py`:
|
||||
the engine's own PSNR is measured before the resize and the encode, so only the
|
||||
probe sees the delivered picture.
|
||||
|
||||
```bash
|
||||
uv run --extra video python scripts/video_fidelity_probe.py input.mp4 input_clean.mp4
|
||||
```
|
||||
|
||||
The control must still be SynthID-positive before a negative candidate can
|
||||
count as removal evidence. In the 2026-07-29 two-clip calibration, both matched
|
||||
controls were positive in Gemini's built-in SynthID verifier; the stronger
|
||||
|
||||
@@ -6,6 +6,9 @@ Tracked data is organized by purpose:
|
||||
data/
|
||||
fixtures/
|
||||
provenance/ Real format and provenance fixtures used by tests
|
||||
(source records live in fixtures/README.md)
|
||||
visible/ Synthetic per-mark example gallery (one committed example per
|
||||
registered visible mark; see fixtures/visible/README.md)
|
||||
calibration/
|
||||
<vendor>/ Minimal controlled inputs needed to rebuild detector assets
|
||||
synthid/
|
||||
@@ -31,10 +34,50 @@ data/
|
||||
commands, hashes, and oracle verdicts instead of committing another corpus
|
||||
copy. A small curated before-and-after example may live in `docs/images/`
|
||||
when it is part of the public documentation.
|
||||
6. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
6. Keep third-party fixture license notices in `licenses/`, outside directories
|
||||
that tests enumerate as media inputs.
|
||||
7. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
Unregistered research candidates belong in
|
||||
`scripts/assets/visible-mark-candidates/` so they are not shipped in the
|
||||
wheel.
|
||||
|
||||
The source distribution excludes `data/`; the wheel contains only package
|
||||
runtime assets.
|
||||
|
||||
## Video SynthID oracle manifest
|
||||
|
||||
`evaluations/video-synthid-oracle.csv` is the only evidence that the shipped
|
||||
video removal profile works, so it is also the source of truth for three shipped
|
||||
defaults: `tests/test_video_invisible.py` asserts that `noise_std`, `long_side`,
|
||||
and `fps` together match a row this manifest records as certified. Changing one
|
||||
of those three without adding the row that certifies it fails the suite. Both
|
||||
historical rows mark `vae` as `unrecorded`: the repository must not infer a model
|
||||
identity from the current default. Add it to the assertion in the same commit as
|
||||
the first oracle row that records and verifies one.
|
||||
|
||||
| Column | Meaning |
|
||||
| --- | --- |
|
||||
| `date`, `source_url`, `source_sha256` | Identify the carrier. |
|
||||
| `source_width`, `source_height`, `source_fps` | Carrier geometry. Without it the actual downscale factor of a row cannot be recovered later. |
|
||||
| `duration_seconds`, `source_verdict` | Clip length submitted and the verifier's reading of the untouched carrier. |
|
||||
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The run configuration. `unrecorded` means the historical run did not preserve the VAE identity. |
|
||||
| `output_sha256` | Identifies the exact submitted file. |
|
||||
| `output_verdict` | One of `detected`, `not_detected`, `indeterminate`, `refused`. |
|
||||
| `output_verdict_text` | The verifier's wording, verbatim. |
|
||||
| `output_detected_range` | Time range the verifier reported for the output. |
|
||||
| `track` | `visual`, `audio`, `both`, or empty. The verifier scores tracks separately and this path copies source audio unchanged. |
|
||||
| `session_id` | Groups rows submitted in one oracle session, so per-session drift stays visible. |
|
||||
| `stratum` | Content class of the carrier, for stratified certification. |
|
||||
| `psnr_db`, `temporal_residual_ratio` | Fidelity measurements. Neither is a watermark verdict. |
|
||||
|
||||
Record `indeterminate` when the verifier answers with its unclear state rather
|
||||
than a negative: an unclear reading logged as `not_detected` is exactly the
|
||||
silent regression this manifest exists to prevent. Leave a field empty when it
|
||||
was not recorded, and never backfill it with a plausible value.
|
||||
|
||||
`psnr_db` is measured against the already-resized frame and before the encode,
|
||||
so it excludes the downscale, the decimation, and the codec. Rows stay
|
||||
comparable to each other only while that definition holds.
|
||||
|
||||
The two 2026-07-31 rows predate this schema; their empty fields were never
|
||||
recorded and are not recoverable from the row.
|
||||
@@ -0,0 +1,101 @@
|
||||
# Content Seal oracle corpus
|
||||
|
||||
Muse Image (`muse-image-1.0`) generations with externally verified Content Seal
|
||||
verdicts, produced through the Meta Model API on 2026-08-26 and checked against
|
||||
the public detector at `https://meta.ai/identification` (anonymous session, no
|
||||
login). Follows the `data/synthid/originals` pattern: binaries live in
|
||||
`originals/`, every derived variant is recorded in `manifest.csv` as a recipe
|
||||
plus hash and is not stored.
|
||||
|
||||
## What this corpus establishes
|
||||
|
||||
- The Meta Model API image endpoint (`POST /v1/images/generations`) stamps the
|
||||
same Content Seal pixel watermark as the consumer Meta AI app: all five
|
||||
generations verified positive with attribution "Muse Image 1 - Meta".
|
||||
- The detector response carries a per-generation ID and creation timestamp
|
||||
embedded in the watermark payload. Both survived a 512 px LANCZOS resize and
|
||||
a full-size JPEG q85 re-encode (same ID returned), so the payload is more
|
||||
robust than the detection threshold.
|
||||
- Three checked center crops lost the seal: 50% and 33% linear crops of the fox
|
||||
and the 50% crop of the text poster returned "No AI signatures from Meta were
|
||||
found". The text poster's 33% crop was not checked because the daily oracle
|
||||
limit was reached, so its empty verdict is not evidence either way. The checked
|
||||
results are consistent with the Reuters 2026-07-11 analysis (55% missed after
|
||||
cropping).
|
||||
- API outputs carry XMP `iptcExt:DigitalSourceType =
|
||||
trainedAlgorithmicMedia`, so local `identify` flags them via the existing
|
||||
Made-with-AI path. Metadata-stripping transforms fall back to unknown, and
|
||||
Content Seal has no local decoder in this project: the oracle is the only
|
||||
reader.
|
||||
- Drift finding: the 512 px resize of `gen_fox_forest` triggers a
|
||||
medium-confidence false positive "Tencent Yuanbao (visible 元宝 / AI生成
|
||||
mark)" in this project's `identify`. Recorded here as a reproducible case.
|
||||
|
||||
## Oracle limits and wire format
|
||||
|
||||
There is no public or documented checking API. Verified against the official
|
||||
developer documentation on 2026-08-26 (`https://dev.meta.ai/docs/`): the full
|
||||
Meta Model API reference lists only Responses, Chat Completions, Messages,
|
||||
Files, Images (`/v1/images/generations`, `/v1/images/edits`), and Models, with
|
||||
no identification, detection, or watermark endpoint, and the image-generation,
|
||||
Muse Image cookbook, and pricing pages never mention watermark, Content Seal,
|
||||
or provenance at all. The API applies the seal (every generation in this corpus
|
||||
carries it) while documenting nothing about it. The web tool drives an internal
|
||||
REST pair, captured from the browser network log on 2026-08-26:
|
||||
|
||||
1. `POST https://rupload.meta.ai/gen_ai_document_gen_ai_tenant/<uuid>` with the
|
||||
raw file bytes, `x-entity-type`, `x-entity-length`, `ai_detector_upload: true`,
|
||||
and an anonymous `authorization: OAuth ecto1:<token>` session token minted by
|
||||
the page.
|
||||
2. `POST https://meta.ai/api/ai-detector` with `Bearer ecto1:<token>` and body
|
||||
`{"media_id": "...", "fileName": "...", "mimeType": "..."}`.
|
||||
|
||||
The rate limit is enforced at that endpoint, server-side, and keyed beyond the
|
||||
browser session: the API itself returns `429 {"errorType": "rate_limited"}`,
|
||||
and clearing cookies and storage changed nothing, so driving the internal pair
|
||||
directly does not bypass it. The Meta Model API (`api.meta.ai/v1`, where the
|
||||
generation key works) has no identification endpoint; plausible paths all
|
||||
return 404. Rows with an empty `oracle_verdict` were transformed but not yet
|
||||
checkable. Read a verdict only from the settled page text after the
|
||||
result-complete state ("Upload another file"): a wait for a verdict string can
|
||||
match the previous upload's text, and the fresh-navigation protocol used for
|
||||
the calibration rows below is the race-free variant.
|
||||
|
||||
## Strength floor calibration (qwen-zimage, seed 0)
|
||||
|
||||
The library resolves strength per vendor with measured floors (OpenAI
|
||||
0.07675 / Google 0.27 / Microsoft InvisMark 0.15 in
|
||||
`_internal/watermark_profiles.py`). Meta Content Seal had no floor before this
|
||||
calibration; these rows measure one by the same methodology: independent
|
||||
generations, each one's first-clean boundary, floor = worst boundary plus the
|
||||
observed cross-source spread.
|
||||
|
||||
Measured (2026-08-26/27, oracle `meta.ai/identification`):
|
||||
|
||||
- Default pipeline clears Content Seal: tested samples came back clean at the
|
||||
default resolution-adaptive strength (~0.1305 at 2.56 MP), including the
|
||||
worst source.
|
||||
- Five independent generations bracketed. First-clean boundaries:
|
||||
lighthouse (0.0525, 0.06], fox (0.03, 0.0375], night_city (0.03, 0.0375],
|
||||
mug <= 0.03, text <= 0.015. Cross-source spread is wide (a factor of four
|
||||
between easiest and hardest).
|
||||
- Derived Meta floor by the existing worst-boundary-plus-cross-source-spread
|
||||
method: 0.06 + (0.0525 - 0.015) = 0.0975, rounded to **0.1**.
|
||||
- Shipped as `QWEN_ZIMAGE_META_STRENGTH`: auto mode routes standalone-AI-IPTC
|
||||
files onto the cohort, and `--vendor meta` / `InvisibleOptions.vendor`
|
||||
names it explicitly on stripped files (implying the scrub runs).
|
||||
## Regeneration
|
||||
|
||||
The eight deterministic crop, resize, and JPEG variants can be reproduced and
|
||||
hash-checked from the tracked originals:
|
||||
|
||||
```bash
|
||||
uv run python scripts/contentseal_transforms.py /tmp/contentseal-derived
|
||||
```
|
||||
|
||||
The Meta API generations and remote GPU outputs are not reproducible from this
|
||||
repository alone. Their prompts, exact output hashes, model/profile settings,
|
||||
and oracle results are recorded in `manifest.csv`, but the generation API is
|
||||
stochastic and the private worker environment is not tracked. `MUSE_API_KEY` and
|
||||
the anonymous detector session are deliberately absent. A new calibration must
|
||||
therefore create new manifest rows rather than claiming to recreate these bytes.
|
||||
@@ -0,0 +1,44 @@
|
||||
name,file,origin,source_url,prompt,requested_size,delivered_size,sha256,xmp_iptc_trainedAlgorithmicMedia,oracle_verdict,oracle_attribution,oracle_generation_id,oracle_created_utc,checked_at_utc,notes
|
||||
gen_fox_forest,originals/gen_fox_forest.webp,meta-model-api,,A red fox resting on a mossy log in a misty forest at dawn photorealistic shallow depth of field,1024x1024,1600x1600,6d7cecf322a5ac4b2547e2f32c767b3cb3e8a0e49f274cfd00aa31be6697eed4,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:11Z,checked twice; ID stable across uploads
|
||||
gen_text_poster,originals/gen_text_poster.webp,meta-model-api,,A cheerful flat-design poster that says 'WATERMARK FIELD DAY 2026' in bold rounded letters over a coral and cream striped background,1024x1024,1600x1600,873927662d8198e9c38b4ae71a2919ba5d417dfcd714649db1b0ae231e8cbf9b,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbJuIKXPAYxsd,2026-08-26T19:08Z,2026-08-26T19:08Z,
|
||||
gen_lighthouse_watercolor,originals/gen_lighthouse_watercolor.webp,meta-model-api,,A watercolor illustration of a lighthouse on a cliff at golden hour seagulls in the distance soft loose washes white paper texture,1024x1024,1600x1600,69bee7c376608568361085602c5c7406900885849bd94c5279c3924cf793b27c,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbB5uWbAmWmwp,2026-08-26T19:08Z,2026-08-26T19:08Z,
|
||||
gen_night_city,originals/gen_night_city.webp,meta-model-api,,A night city skyline in the rain neon reflections on wet asphalt cinematic wide shot moody,1536x1024,1920x1280,345adf6b9f777b15ef7ce7bbdfb3b355a1d9e882740809e966e9f6caafd94d48,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbH5uWbAmWmwp,2026-08-26T19:08Z,2026-08-26T19:08Z,
|
||||
gen_studio_mug,originals/gen_studio_mug.webp,meta-model-api,,A minimalist studio photo of a single matte ceramic mug on a pastel blue background soft shadow clean commercial look,1024x1024,1600x1600,2b7bc7fd56fa2250c66ef74f8b3e49d9666fe147d894d5baf9461b5e36077a07,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbAsSozV2XCzw,2026-08-26T19:08Z,2026-08-26T19:08Z,
|
||||
fox_crop50,,derived:gen_fox_forest,,center crop 0.5 linear each side then WEBP q95,,800x800,80b91995560a7c4b08596a26b3401a29afb82691fad193502697a49785eab7be,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost; PIL re-encode strips XMP
|
||||
fox_crop33,,derived:gen_fox_forest,,center crop 0.33 linear each side then WEBP q95,,528x528,3b3aded8f4fcb98d0599431abccb73b9581854ae34c6bc869724cd5c86a1e35e,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost
|
||||
fox_res512,,derived:gen_fox_forest,,resize long side 512 LANCZOS then WEBP q95,,512x512,19b85f615d56f5e9f55cd97bae0ee343f4f432e6bfa0a6fc7acb07dec2cc1d81,no_stripped_by_reencode,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:13Z,generation ID and creation time identical to the original generation: watermark payload survives the resize
|
||||
fox_jpeg85,,derived:gen_fox_forest,,full-size JPEG q85 re-encode,,1600x1600,3bc7dc11bf3c16c83814df1e6485cdb6e49e21141511dda56d210cc920a8c5e3,no_stripped_by_reencode,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:13Z,payload survives JPEG recompression; local identify returns unknown (metadata stripped)
|
||||
text_crop50,,derived:gen_text_poster,,center crop 0.5 linear each side then WEBP q95,,800x800,3f0d4075012ee4fec820209c5c1c32bb739998813325f49b1124f379162f9837,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost
|
||||
text_crop33,,derived:gen_text_poster,,center crop 0.33 linear each side then WEBP q95,,528x528,95cef87002961831b21f6f47cd9a0824a67e2ee6f8503356fd5db6a3fcfcc998,no_stripped_by_reencode,,,,,,daily oracle limit reached before this row could be checked
|
||||
text_res512,,derived:gen_text_poster,,resize long side 512 LANCZOS then WEBP q95,,512x512,b319727d02e908f22e59f148050e6b4686cbd1a622bd69db51fe4072a6f66b34,no_stripped_by_reencode,,,,,,daily oracle limit reached; also the fox twin of this transform triggered the Yuanbao false positive below
|
||||
text_jpeg85,,derived:gen_text_poster,,full-size JPEG q85 re-encode,,1600x1600,76f00fcde1e974549afa5d90e1879c705e3c3631ef1f35cab901fc912cd15fa4,no_stripped_by_reencode,,,,,,daily oracle limit reached before this row could be checked
|
||||
blog_qr_manhwa,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1807972953501959&version=1783318590&transcode_extension=webp,,,1920x1280,8ee7700dff59f21d7de393a619664a0b9360f148d18df289364ecea939b3dea9,yes,detected,Muse Image 1 - Meta,,,2026-08-26T18:59Z,not stored; re-downloadable from source_url
|
||||
blog_fractal_poster,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1499727434782014&version=1783727678&transcode_extension=webp,,,2096x992,2a96168db026444b47ac37807fd99a88c8b7fa6866e1c93a54ab427fbe581f76,no,detected,Muse Image 1 - Meta,,,2026-08-26T18:59Z,seal survived CDN transcode without any metadata
|
||||
blog_flipbook_punch,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=2521328538303639&version=1783396590&transcode_extension=webp,,,1920x1280,35f28dc2cbdf44aed190bc8efaa0fe7bfa288a2196638712dd83bc941cc294d5,yes,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,
|
||||
blog_teen_cat,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1325391919706734&version=1783319609&transcode_extension=webp,,,1600x1600,c2e7df000ffa6520589d8efc5f0bcbeb759175cdaa0c827189c28a64f75d3dd7,yes,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,
|
||||
blog_hero_1,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=2140990336800598&version=1783727678,,,1344x1792,7655639acb9f826cdb864d869530153efc1e1467ff414b5a63e80aa7b39b2fb7,no,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,seal survived CDN transcode without any metadata
|
||||
control_synth_gradient,,synthetic-local,,PIL gradient with rectangle JPEG q90; never touched any AI pipeline,,1024x768,25f29c0b993ed1a8e41060ed50de778d875a6d94847ced180aafc5a6ad429f82,no,not_detected,,,,2026-08-26T19:01Z,negative control: No AI signatures from Meta were found
|
||||
fox_nometa,,derived:gen_fox_forest,,metadata --remove only (no pixel change),,1600x1600,685d3a5c5d74e653ec7615fc7a347a0584c277fc7fb59594d388ac619ffe3985,no_stripped,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T22:19Z,metadata stripping does NOT remove Content Seal; seal lives in pixels; same generation ID as original
|
||||
fox_modal_invisible,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,6daa3e9b5dde5cb443d8071c5294262742f001c02b694243764a50345769d8b8,no_stripped,not_detected,,,,2026-08-26T22:21Z,current default pipeline clears Content Seal
|
||||
text_modal_invisible,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,8530a82fb856b0a00651076ac1ad8a7544b5d8873cac9bd5780704fbe10f53f8,no_stripped,not_detected,,,,2026-08-26T22:22Z,current default pipeline clears Content Seal
|
||||
fox_str0.06000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,7b5e5ae75ca20f2a3cdfcfa7666104752d9557b08ac236db6f1548434940a1f3,no_stripped,not_detected,,,,2026-08-26T22:28Z,strength calibration ladder
|
||||
fox_str0.04500,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,26da15694b3096e2f1717154db213e814cdd27f2fc40e5f924db2324213cf960,no_stripped,not_detected,,,,2026-08-27T02:28Z,strength calibration ladder; re-confirmed clean at 0.045
|
||||
fox_str0.03750,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,b7eb0616d10042933edf22cfc30aff4df915384828289b409f6cfa4cdf2807d4,no_stripped,not_detected,,,,2026-08-26T22:30Z,fresh-page protocol; fox first-clean boundary is in (0.03 0.0375]
|
||||
fox_str0.03000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,cff80d18d9d00000f47e0f97c419a37c13a13831795768aee8727aa9516d1401,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:31Z,fresh-page protocol; seal survives 0.03
|
||||
fox_str0.02000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,b19183ec60b63a5b73d3e1c723e7943584912857d51a4b5f85259c5c77da0e53,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:28Z,settled-read protocol; monotonic below the 0.03 boundary
|
||||
text_str0.03750,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,bacfd4eca7b13616502f74497cbe4f02ef6893918870f13f9e76fdb93b43107f,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol
|
||||
text_str0.03000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,d5c6cffbf471ef8bb74ee8c0d22cc17d9da2c446b83e46c4be67e6ea4cdd09de,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol; text first-clean boundary is at or below 0.03 - cross-source spread vs fox confirmed
|
||||
lh_str0.03750,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,f5169debc5fd2792399c0ea3fa88cb1d05f523d95004bec240c73db0e9aaee51,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.0375 where fox is already clean
|
||||
lh_str0.03000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,110d2112d91dd53df5ccd1839e0e2b41c44aa1832adb0090032660e071fb1fd4,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,fresh-page protocol; lighthouse holds its seal below the fox boundary: cross-source spread is wide
|
||||
text_str0.02500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,f46a6c53b4f3d39fccb1d62cca82ced2e163e4424d13c98e6468a8e083c0fa54,no_stripped,not_detected,,,,2026-08-27T02:28Z,mid rung inside the clean zone
|
||||
text_str0.02000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,a19cbe22313e0f19768c98567ea89f2d9dc30827691fc7ec2971c52f3e7c6cc6,no_stripped,not_detected,,,,2026-08-27T02:28Z,text clean at 0.02: boundary <= 0.02 - the easiest source measured
|
||||
text_str0.01500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.015,,1600x1600,aa3816c75e38f3ab5afc378d349caafad56f2bb6da859a5ee68fdea57ad00d73,no_stripped,not_detected,,,,2026-08-27T02:41Z,text clean at 0.015: boundary <= 0.015 - the easiest source measured
|
||||
lh_str0.02500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,989c4a7a804bc8420ec4d86b9445bc3a37382c1834782a7f8842f781d69b05e2,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
|
||||
lh_str0.02000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,6a4a4ec94292a145e7d73e6900b8ec6d0eede17137f2b91fba7757ec0bdba5ec,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
|
||||
nc_str0.03750,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1920x1280,46f322335f801ee54427eb39275216465f9a7cfa3d918c698e4d90a2439a0815,no_stripped,not_detected,,,,2026-08-27T02:28Z,boundary (0.03; 0.0375]
|
||||
nc_str0.03000,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1920x1280,3f42632319248ad1a91ef2457c7e0b13ed17e1d2569decbbaab67c2072e9a305,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,night_city holds seal at 0.03
|
||||
mug_str0.03750,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,3b6109ae7e86b8ea74ce33315e3f2a7eda1cadb724d429f0951f13c3ead34b0f,no_stripped,not_detected,,,,2026-08-27T02:28Z,upper rung for the mug bracket
|
||||
mug_str0.03000,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,1c24c984677e1528da83003ac40594f6e5ef1fa9351f1368b6d31a0101308583,no_stripped,not_detected,,,,2026-08-27T02:28Z,mug clean at 0.03: boundary <= 0.03
|
||||
lh_str0.05250,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0525,,1600x1600,8912ba1ff4a26790f69224c49dd649e46835e277c375276ac9604f5339775b2a,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:41Z,bisection: lighthouse first-clean boundary bracketed to (0.0525; 0.06]
|
||||
lh_str0.04500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,cfdcf7bbb27c740846a1de4edffee3b25a624c2a8e1aba2362feaa25b682e3ae,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.045: lighthouse is the worst source measured
|
||||
lh_str0.06000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,4e214be00a89dc9f1f32ec39483c5e9668a1ec7f38b15e5794942479c839fcca,no_stripped,not_detected,,,,2026-08-27T02:28Z,lighthouse first-clean boundary bracketed to (0.045 0.06]
|
||||
|
|
After Width: | Height: | Size: 585 KiB |
|
After Width: | Height: | Size: 510 KiB |
|
After Width: | Height: | Size: 652 KiB |
|
After Width: | Height: | Size: 83 KiB |
|
After Width: | Height: | Size: 192 KiB |
@@ -7,31 +7,394 @@ ground truth and instructions.
|
||||
|
||||
| Original | Provider | Content | Exercises |
|
||||
| --- | --- | --- | --- |
|
||||
| `ChatGPT Image May 31, 2026, 02_02_23 PM.png` | OpenAI | Light multilingual typography | Text preservation |
|
||||
| `ChatGPT Image May 31, 2026, 02_03_55 PM.png` | OpenAI | Multilingual typography | Text preservation |
|
||||
| `Gemini_Generated_Image_633uuy633uuy633u.png` | Google | Landscape with a Chinese sign | CJK text preservation |
|
||||
| `Gemini_Generated_Image_y48j3cy48j3cy48j.png` | Google | Portrait grid | Face identity and skin texture |
|
||||
|
||||
## Text ground truth
|
||||
|
||||
`ground-truth.json` contains hand-verified OCR for the two text-bearing
|
||||
`ground-truth.json` contains hand-verified OCR for the three text-bearing
|
||||
originals. To regenerate an OCR seed:
|
||||
|
||||
```bash
|
||||
uv run scripts/fidelity_metrics.py ocr \
|
||||
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png" \
|
||||
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png" \
|
||||
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
|
||||
--langs en,ru,ch \
|
||||
--out data/evaluations/fidelity/ground-truth.json
|
||||
```
|
||||
|
||||
Verify and correct the generated text by hand before using it as ground truth.
|
||||
`text-lines.json` contains the verified per-line strings and source-space boxes
|
||||
used by the evaluation-only selective-restoration experiment. It is not an
|
||||
automatic scene-text annotation set.
|
||||
|
||||
`scripts/infer_text_lines.py` can generate a draft from source pixels without
|
||||
existing annotations. On the two posters it proposed 20 and 18 lines at the
|
||||
default threshold, but exact-text precision was only 90.0% and 94.4% because
|
||||
high-confidence OCR still dropped an English comma and replaced a Chinese comma
|
||||
with ASCII. Its output therefore requires manual verification of every line;
|
||||
`accepted` means crop-stable, not ground-truth-correct.
|
||||
|
||||
## Text-preservation benchmark
|
||||
|
||||
`text-preservation-2026-08-13.csv` records a fixed-seed comparison of the two
|
||||
current profiles and a global Z-Image Turbo prototype on all three text
|
||||
fixtures. Each candidate ran through the
|
||||
complete `visible -> invisible -> metadata` route with its profile default
|
||||
strength and adaptive-polish setting. The Z-Image prototype instead sweeps
|
||||
0.08, 0.10, 0.15, 0.20, and 0.25 without polish because its provider-specific
|
||||
removal floor is not known. The output hash identifies the exact bytes
|
||||
measured; generated outputs remain outside the repository.
|
||||
|
||||
The character-weighted added CER is 0.262 for `qwen-zimage` and 0.256 for
|
||||
`sdxl-zimage`. That 0.006 absolute difference is not a
|
||||
stable ordering: SDXL wins the light poster, Qwen wins the dark poster, and the
|
||||
Chinese sign is tied. A paired sign test on the two non-ties is 1-1 (`p=1.0`).
|
||||
The measured sample therefore does not support a general text-preservation
|
||||
winner. Both profiles substantially degrade the smallest multilingual poster
|
||||
text and preserve the larger Chinese sign.
|
||||
|
||||
Z-Image Turbo is the clear fidelity lead. At strength 0.10 its
|
||||
character-weighted CER is 0.093, against the unchanged sources' 0.124 OCR
|
||||
floor, while whole-image LPIPS stays between 0.047 and 0.086. Visual inspection
|
||||
still finds substitutions in the smallest Cyrillic and Chinese poster lines,
|
||||
so a favorable OCR score does not mean pixel-exact text. Quality drops quickly
|
||||
above 0.15; at 0.25 the weighted CER is 0.428.
|
||||
|
||||
Provider-oracle checks on 2026-08-13 bracket the OpenAI requirement at no more
|
||||
than 0.10 for this sample. Both original OpenAI controls were detected by
|
||||
`openai.com/verify`; the light poster was clean at 0.08, while the dark poster
|
||||
was detected at 0.08 and clean at 0.10. The light 0.10 output was not separately
|
||||
checked. Google fails the entire tested sweep. The original Gemini image was
|
||||
detected through C2PA, and a pixel-identical copy with AI metadata stripped was
|
||||
separately detected by Gemini's built-in SynthID verifier. Z-Image outputs at
|
||||
0.08, 0.10, 0.15, 0.20, and 0.25 were all still detected. At 0.25 the weighted
|
||||
CER has already risen to 0.428, worse than the raw weighted CER of both current
|
||||
profiles (Qwen 0.387, SDXL 0.381). Increasing strength beyond the measured grid
|
||||
would therefore no longer serve the text-preservation objective without a new
|
||||
mechanism or hypothesis. Z-Image is not a viable global replacement on this
|
||||
evidence: no clean Google operating point was found before it lost its fidelity
|
||||
advantage.
|
||||
|
||||
Qwen-Image-2.0 was not added to the numeric comparison. Its weights are not
|
||||
published, and its hosted edit API exposes an editing instruction and seed but
|
||||
no low-strength denoise control. It can be evaluated as a separate hosted edit
|
||||
strategy when credentials are available, but it is not a drop-in replacement
|
||||
for the partial-regeneration mechanism measured here.
|
||||
|
||||
The OCR floor is the source image scored against the hand-verified text. Use
|
||||
`added_cer = text_cer - ocr_floor` when interpreting pipeline damage, because the
|
||||
unchanged poster sources already score 0.127 CER. `oracle_rechecked=true` marks
|
||||
the exact Z-Image bytes checked above; the remaining rows were not rechecked.
|
||||
The table does not certify other seeds, content classes, or strengths beyond
|
||||
the recorded provider verdicts.
|
||||
|
||||
## Text-restoration prototype
|
||||
|
||||
`text-restoration-2026-08-13.csv` evaluates an OCR-driven post-pass on the exact
|
||||
Qwen outputs above. The prototype recognizes English and Russian with macOS
|
||||
Vision and CJK with PaddleOCR, derives glyph masks independently from the
|
||||
source and Qwen output, removes both sets of glyphs with block-wise LaMa, and
|
||||
draws the recognized strings with new system-font pixels. It never composites
|
||||
source pixels back into the result.
|
||||
|
||||
On the two multilingual posters, character-weighted CER fell from 0.338 and
|
||||
0.305 to 0.007 on both. OpenAI Verify reported no OpenAI signals for both Qwen
|
||||
controls and both restored outputs in the same run. The improvement comes with
|
||||
a substantial whole-image fidelity cost: LPIPS rose from 0.107 to 0.174 and
|
||||
from 0.095 to 0.162, while PSNR fell by about 10.5-11.4 dB. Visual inspection
|
||||
found one residual shadow in the smallest English line of the light poster;
|
||||
the dark poster was clean but the substitute fonts visibly changed typography.
|
||||
|
||||
The Chinese sign did not improve: CER rose from 0.074 to 0.111 because the OCR
|
||||
and renderer changed punctuation. The Gemini verifier returned detected for
|
||||
both the restored output and its byte-identical Qwen control on a third work
|
||||
account, although that Qwen hash had previously returned clean on another work
|
||||
account. This run therefore does not isolate a restoration-stage regression;
|
||||
its Google verdict is inconclusive until a source-positive, Qwen-negative
|
||||
control can be reproduced in the same available account.
|
||||
|
||||
This prototype is not ready to ship. Its strongest result establishes that
|
||||
fresh-glyph reconstruction can recover literal text without reintroducing an
|
||||
OpenAI signal, but portable OCR, font/style reconstruction, a tighter mask, and
|
||||
a reproducible Google oracle control remain prerequisites.
|
||||
|
||||
### Selective restoration follow-up
|
||||
|
||||
`selective-text-restoration-2026-08-13.csv` compares that full compositor with
|
||||
a selective prototype on the same two posters. The prototype leaves a Qwen line
|
||||
unchanged when padded source and output recognition agree, and applies the same
|
||||
LaMa plus fresh-system-font reconstruction only to lines whose recognized text
|
||||
changed. This reduced the edited area from 15.4% and 17.0% to 5.5% on both
|
||||
posters.
|
||||
|
||||
Under one consistent Paddle `en+ru+ch` measurement route, selective restoration
|
||||
reduced Qwen CER from 0.378 to 0.101 on the light poster and from 0.413 to 0.112
|
||||
on the dark poster. Its image LPIPS was 0.120 and 0.103, substantially closer to
|
||||
Qwen than the full compositor's 0.174 and 0.162. These CER values must not be
|
||||
mixed with the preceding table's Vision/Paddle hybrid values: the comparison
|
||||
file remeasures all three variants through Paddle so their relative result is
|
||||
valid on one OCR route.
|
||||
|
||||
OpenAI Verify returned `No OpenAI signals detected` for both selective outputs,
|
||||
then detected the original light poster as `Generated with OpenAI tools` in the
|
||||
same Chrome sequence. Visual inspection found the dark output clean, but the
|
||||
light output still retained a local shadow around one replaced fine-text line.
|
||||
Selective restoration is therefore the strongest current direction, not a
|
||||
production-ready default. The next implementation needs automatic line matching
|
||||
and a tighter source-plus-candidate glyph mask before it can be proposed for the
|
||||
pipeline.
|
||||
|
||||
A mask-only follow-up added two pixels of dilation around every selected glyph.
|
||||
It visually removed the light poster's shadow and improved its CER from 0.101 to
|
||||
0.081 with effectively unchanged LPIPS, but increased the edited area from 5.5%
|
||||
to 7.0%. Applying the same rule to the dark poster worsened CER from 0.112 to
|
||||
0.147. These expanded-mask rows were not resubmitted to the provider oracle.
|
||||
The opposing result rules out one global dilation constant and makes per-line
|
||||
residual detection the next fidelity gate.
|
||||
|
||||
An automatic follow-up removed the hand-selected line list. For each known
|
||||
source line, it runs a recognition-only model on padded, upscaled source and
|
||||
Qwen crops. It preserves a line only when both confidence scores are at least
|
||||
0.75, the source recognition is within 0.25 normalized edit distance of the
|
||||
verified line, and normalized source and candidate recognition match exactly.
|
||||
This reproduced the manual dark-poster line selection and additionally kept one
|
||||
correct large Russian line on the light poster.
|
||||
|
||||
The automatic prototype also replaced uniform dilation with a second measured
|
||||
LaMa pass. After the first erase, it extracts contrast remaining at the original
|
||||
glyph positions, dilates only that residual, and inpaints it before drawing fresh
|
||||
text. Visual inspection found no remaining double glyphs on either output. The
|
||||
effective edited fractions were 4.1% and 5.7%. Image LPIPS was 0.113 and 0.104,
|
||||
against 0.120 and 0.103 for the manual selective version. Detector-based page CER
|
||||
was 0.123 and 0.119; the light value is inflated by the detector splitting the
|
||||
preserved Cyrillic line into Latin-like fragments even though the padded Russian
|
||||
recognizer reads it exactly. This is another reason that line decisions use
|
||||
recognition-only crops rather than the page detector's aggregate CER.
|
||||
|
||||
Both exact automatic output hashes returned `No OpenAI signals detected`, and
|
||||
the original light poster returned `Generated with OpenAI tools` immediately
|
||||
afterward in the same Chrome sequence. The tracked script reproduced the dark
|
||||
hash exactly. Its light output differed from the initial prototype in only 386
|
||||
pixels by at most one channel value because separate ONNX environments rounded
|
||||
differently; the script's exact `900def5a...` hash was therefore submitted and
|
||||
also returned clean before the positive source control.
|
||||
|
||||
The script and its required verified line annotations are tracked as
|
||||
`scripts/selective_text_restoration.py` and `text-lines.json`. For example:
|
||||
|
||||
```bash
|
||||
PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK=True \
|
||||
uv run scripts/selective_text_restoration.py \
|
||||
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
|
||||
path/to/qwen-candidate.png \
|
||||
--restoration vae-glyphs \
|
||||
--glyph-donor path/to/qwen-vae-reconstruction.png \
|
||||
--glyph-feather 0.5 \
|
||||
--output path/to/selective-output.png \
|
||||
--manifest path/to/manifest.json
|
||||
```
|
||||
|
||||
The restoration mode is deliberately explicit. `vae-glyphs` requires a
|
||||
separately generated VAE reconstruction in `--glyph-donor`; it takes exact
|
||||
thresholded glyph cores from that donor, uses a fresh silhouette beneath them,
|
||||
and defaults to a narrow 0.5-pixel donor edge. The donor and the resulting exact bytes still require
|
||||
full-pipeline oracle evaluation. `rerender` reproduces the tracked
|
||||
fresh-system-font experiment. `source-glyphs` is only for compositing a
|
||||
separately regenerated, oracle-evaluated layer; feeding it the watermarked
|
||||
original would paste provenance-bearing pixels back into the result. The
|
||||
`--keep-background --composite-mask boxes` combination is an aligned-layer
|
||||
experiment and is not a production text restorer.
|
||||
|
||||
The `source-silhouette` follow-up keeps the source glyph geometry but discards
|
||||
its pixel amplitudes: it thresholds each source line to a binary shape, samples
|
||||
one foreground color, and synthesizes new antialiasing over the scrubbed image.
|
||||
On a 24-image matrix spanning serif, Latin-diacritic, Cyrillic, CJK, tiny-UI,
|
||||
and rotated text, the original outer-feather compositor preserved all 34
|
||||
source-readable lines exactly under crop OCR, with median text-box SSIM 0.859.
|
||||
An inner-antialias variant raised median text-box SSIM to 0.902 and left
|
||||
whole-image SSIM unchanged at 0.672, but preserved 33/34 lines: OCR read one
|
||||
middle dot as a colon. The higher-fidelity antialiasing is retained in the
|
||||
evaluation script, with that punctuation miss recorded as a caveat. Its exact
|
||||
outputs were not submitted to the provider oracle because the
|
||||
public verifier reached its request limit; a clean verdict from the earlier
|
||||
outer-feather bytes does not transfer to the new hashes.
|
||||
|
||||
Follow-up visual review on a typography-rich control rejected both silhouette
|
||||
compositors despite their OCR and SSIM scores. They preserved literal content
|
||||
and approximate glyph geometry, but changed stroke weight, color variation,
|
||||
edge antialiasing, and small decorative details enough to be plainly different
|
||||
from the source. OCR exactness and text-box SSIM are therefore screening metrics,
|
||||
not acceptance gates for source-typography preservation. A candidate must also
|
||||
retain source-like edge pixels and pass direct visual comparison at native size.
|
||||
|
||||
A narrower Qwen-VAE donor follow-up keeps the scrubbed background, then copies
|
||||
only VAE-reconstructed pixels through the source-silhouette mask with a
|
||||
0.5-pixel feather. Across the 48-case typography matrix (548 annotated lines),
|
||||
text-box SSIM improved in 47/48 cases and glyph-edge error improved in 48/48.
|
||||
Median text-box SSIM rose from 0.854 to 0.914, while median glyph-edge MAE fell
|
||||
from 37.59 to 32.42. The median nonzero alpha area was 3.62%; on dense or large
|
||||
text this is still substantially wider than the target control. These are
|
||||
fidelity results only; the 48 matrix outputs were not submitted to the provider
|
||||
oracle. On a separate untracked dense-typography control, the exact core-only
|
||||
Qwen-VAE donor with a 1.43% source-silhouette area returned `No OpenAI signals
|
||||
detected`. Its 0.5-pixel feathered sibling, which raises the nonzero donor area
|
||||
to 2.75%, returned the same verdict and improved mean text-box SSIM from 0.957
|
||||
to 0.960. Crop OCR recovered from 7/15 exact lines on the raw pass to 14/15,
|
||||
matching the source OCR floor. Whole-image LPIPS was 0.082, but only 0.108% of
|
||||
pixels were exactly unchanged and the detected face retained 0.670 of source
|
||||
Laplacian variance. The tracked script reproduced the feathered file byte for
|
||||
byte. These two exact-byte verdicts do not certify other images or the larger
|
||||
matrix masks, and the global smoothing fails a strict unchanged-image criterion.
|
||||
|
||||
The opt-in production port was rechecked separately on 2026-08-15. Its current
|
||||
LaMa runtime did not reproduce the earlier evaluation PNG byte for byte, but all
|
||||
changed pixels were confined to the erased background outside the donor glyph
|
||||
core. The exact production artifact returned `No OpenAI signals detected` in
|
||||
3/3 OpenAI Verify runs, while the matched source control returned `Generated
|
||||
with OpenAI tools` in 2/2 runs in the same Chrome session; expanded details
|
||||
identified SynthID and no C2PA manifest on the control. The private control and
|
||||
artifact hashes remain outside the public repository. This certifies only that
|
||||
runtime, verified manifest, and output, not arbitrary text masks or images.
|
||||
|
||||
The Google result is negative. On the synthetic CJK sign case, two separate
|
||||
work-account runs both detected SynthID in the resaved source control and in the
|
||||
exact Qwen-VAE donor output. The candidate improved mean text-box SSIM from
|
||||
0.791 to 0.835 and glyph-edge MAE from 35.48 to 22.49 with a 3.67% donor area,
|
||||
but Gemini still detected it. The intermediate Qwen silhouette base and the
|
||||
earlier original-sign Qwen baseline were also detected in the same account.
|
||||
This isolates the current blocker upstream of text restoration: the global
|
||||
Google pass did not reach the SynthID removal floor, so the OpenAI-clean
|
||||
`vae-glyphs` configuration cannot be used as a Google operating point.
|
||||
|
||||
On the large Chinese-sign control, source and candidate recognition agreed on
|
||||
all three lines. The script selected no changed lines, emitted a zero mask, and
|
||||
copied the candidate byte for byte instead of re-encoding it. This corrects the
|
||||
earlier unnecessary CJK rerender and proves the no-edit branch. It does not add
|
||||
a Google-negative oracle result: the available Google account still cannot
|
||||
reproduce a source-positive, Qwen-negative control sequence.
|
||||
|
||||
The earlier automatic rerender was the first variant in the experiment to pass
|
||||
the recorded visual, fidelity, and OpenAI-oracle gates on its two poster
|
||||
fixtures. The later typography-rich control shows that result does not
|
||||
generalize to source-typography preservation. It remains evaluation-only: it
|
||||
depends on verified source text and source line boxes, uses macOS system fonts,
|
||||
and has not been validated on natural scene text, rotated text, or automatic
|
||||
line-box discovery.
|
||||
|
||||
An opt-in `--detect-boxes` follow-up tested automatic geometry. Grouping Paddle
|
||||
word detections by vertical overlap found exactly 20/20, 20/20, and 3/3 lines;
|
||||
mean IoU with verified boxes was 0.857, 0.847, and 1.000. Reusing the annotation
|
||||
crop padding was unstable and preserved only 4/20 dark-poster lines. Reducing
|
||||
vertical recognition padding to 10% restored the exact 8/20 and 7/20 selection
|
||||
decisions, but detector CER was 0.127 and 0.154 rather than 0.123 and 0.119. The
|
||||
dark regression failed the fidelity gate, so these hashes were not submitted to
|
||||
the provider oracle. The flag is retained only to reproduce that negative
|
||||
evaluation and still requires verified strings and an exact line-count match.
|
||||
|
||||
### AnyText2 glyph-conditioned follow-up
|
||||
|
||||
`anytext2-restoration-2026-08-13.csv` tests the official Apache-2.0
|
||||
AnyText2 checkpoint as a local text-editing pass over the exact Qwen Chinese-sign
|
||||
output. The checkpoint hash was verified against ModelScope. Its own edit example
|
||||
successfully replaced a masked blackboard line with the requested `DADDY`, so the
|
||||
runtime and checkpoint were functional before the tracked fixture was measured.
|
||||
|
||||
AnyText2 failed the fidelity gate. The standard full-image detector scored the
|
||||
default edit at CER 0.185 and the source-font mimic at 0.222, against 0.074 for
|
||||
Qwen; image LPIPS moved from 0.289 to 0.338 and 0.345. A padded crop-recognition
|
||||
cross-check removed the detector's punctuation misses: Qwen scored 0.000, the
|
||||
default edit still scored 0.074 with two substituted characters, and the mimic
|
||||
edit stayed at 0.222. Visual inspection agreed, with additional line-shape and
|
||||
punctuation errors in the mimic output. No provider-oracle submission was made
|
||||
because neither candidate preserved the requested content better than its
|
||||
already-clean Qwen input.
|
||||
|
||||
This result covers the published SD1.5 checkpoint, three Chinese lines of no more
|
||||
than 20 characters, one seed, and one fixture. The repository still describes
|
||||
AnyText2XL as unreleased, and the inference wrapper truncates each requested line
|
||||
after 20 characters, so the published model cannot directly cover the longer
|
||||
English and Russian poster lines.
|
||||
|
||||
The padded cross-check also changes the interpretation of this fixture's earlier
|
||||
restoration result. Its Qwen image already contains all three correct Chinese
|
||||
lines; the reported 0.074 came from the detector cropping the final punctuation
|
||||
from two lines. A deterministic rerender with horizontally padded recognition
|
||||
and a mask-matched Hiragino Sans GB W6 font reached detector CER 0.000, but image
|
||||
LPIPS worsened from 0.289 to 0.327 and the font was visibly heavier. The correct
|
||||
policy on this large-text fixture is therefore to leave a Qwen line unchanged
|
||||
when padded recognition confirms the source and candidate strings agree.
|
||||
|
||||
## Face-stage isolation
|
||||
|
||||
`face-stage-isolation-2026-08-13.csv` separates the two stages on the 18-face
|
||||
Gemini portrait-grid fixture. All variants use the same visible-cleaned,
|
||||
metadata-stripped input and seed 0. The prepared input remained positive in the
|
||||
Gemini SynthID verifier, so the surrounding visible and metadata stages did not
|
||||
create the clean verdicts.
|
||||
|
||||
| Route | Gemini oracle | ID cosine | Face LPIPS | Laplacian ratio | Image LPIPS |
|
||||
| --- | --- | ---: | ---: | ---: | ---: |
|
||||
| Prepared control | detected | - | - | - | - |
|
||||
| Z-Image face stage only | detected | 0.927 | 0.028 | 0.612 | 0.013 |
|
||||
| Qwen global stage only | clean | 0.589 | 0.217 | 0.331 | 0.288 |
|
||||
| Full Qwen then Z-Image | clean | 0.852 | 0.050 | 0.492 | 0.252 |
|
||||
|
||||
This isolates the roles: the Qwen global pass removes the watermark, while the
|
||||
masked Z-Image pass restores face identity and detail without reintroducing a
|
||||
detectable whole-image signal. Z-Image face-only does not remove SynthID. The
|
||||
result explains why the face stage succeeded while global Z-Image failed: the
|
||||
earlier clean verdict belonged to the two-stage pipeline, not to Z-Image as a
|
||||
standalone remover. This is one fixture, one seed, and one oracle pass per
|
||||
variant; it does not establish a general reintroduction threshold for mask size.
|
||||
|
||||
## Upstream Synthid-Bypass v2 reproduction
|
||||
|
||||
`upstream-v2-reproduction-2026-08-13.csv` records a source-level audit and a
|
||||
close reproduction of the public
|
||||
[`Synthid-Bypass-v2.0.json`](https://github.com/cebeuq/Synthid-Bypass/blob/3007d0351596ae0a78b7074dae7ad179710b1e48/Synthid-Bypass-v2.0.json).
|
||||
The audited workflow is SHA-256
|
||||
`41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6` at
|
||||
upstream commit `3007d0351596ae0a78b7074dae7ad179710b1e48`.
|
||||
|
||||
Tracing links backward from `SaveImage` confirms that the active global stage is
|
||||
Qwen-Image-2512 Q4 with the Lightning four-step LoRA at 0.8, DiffSynth Canny at
|
||||
1.0, AuraFlow shift 3, `dpmpp_2m` plus `sgm_uniform`, CFG 1, and four steps. Its
|
||||
resolution helper uses adaptive level 6 over 0.08..0.15, which resolves to the
|
||||
0.154 ceiling for the 2816x1536 portrait fixture. Z-Image Turbo is used only by
|
||||
the face detailer: eight steps, CFG 1, `res_2s` plus `bong_tangent`, a 768 px
|
||||
guide, 1024 px cap, and direct adaptive denoise
|
||||
`clamp(0.10 * largest_face_ratio / 0.03, 0.05, 0.28)`. The active face regions
|
||||
come from YOLOv8-face plus SAM. Although the README describes MediaPipe as part
|
||||
of the strict path, the MediaPipe nodes do not reach the saved output. The
|
||||
1.2-megapixel scaler is also bypassed (`mode=4`).
|
||||
|
||||
The published upstream pair 12 changed from SynthID-positive to clean in the
|
||||
Gemini verifier and scored 0.975 face identity. On the project portrait fixture,
|
||||
the prepared control was positive, while both the close global reproduction and
|
||||
the close full reproduction were clean. The global stage scored 0.589 identity
|
||||
and 0.217 face LPIPS; the upstream-strength face stage improved those to 0.783
|
||||
and 0.083. The maintained profile's weaker face pass scored 0.852 and 0.050 on
|
||||
the same control, so copying upstream's roughly doubled face strength would be a
|
||||
quality regression on this fixture.
|
||||
|
||||
The close reproduction is intentionally labeled rather than presented as an
|
||||
exact ComfyUI run. It uses full-precision DiffSynth bf16 weights instead of the
|
||||
Q4 GGUF files, the maintained DiffSynth Lightning scheduler approximation
|
||||
instead of the ComfyUI sampler pair, YuNet plus SAM instead of YOLOv8-face plus
|
||||
SAM, and fixed seed 0 instead of randomized seeds. The two independent oracle
|
||||
controls and the published upstream pair establish the direction of the result;
|
||||
they do not make the local output byte-equivalent to upstream.
|
||||
|
||||
## Compare
|
||||
|
||||
```bash
|
||||
uv run scripts/fidelity_metrics.py compare \
|
||||
--original data/synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png \
|
||||
--variant controlnet=<out>.png \
|
||||
--variant qwen=<out>.png \
|
||||
--variant qwen-zimage=<qwen-out>.png \
|
||||
--variant sdxl-zimage=<sdxl-out>.png \
|
||||
--ocr-langs ""
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
date,source_commit,fixture,variant,model,model_revision,model_sha256,ocr_route,text_cer,img_lpips,ssim,psnr_db,output_sha256,provider_oracle,notes
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,,,,"Paddle ch",0.074,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_run,Exact Qwen baseline reused from text-restoration benchmark
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-default,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.185,0.338,0.642,18.2,87729fe7e0482d2cead6d3b7c7dd386ecc75ab3d8658131c54e67528b8f9feeb,not_run,Fidelity gate failed; padded crop recognition CER 0.074 still shows two substituted characters; locally composited three-line edit; 20 DDIM steps; seed 0
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-mimic,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.222,0.345,0.644,17.9,f5198d8f2365b79880aba8ea6b0c3885142b9de21c9f7f4f6712ac4a0db8dd85,not_run,Fidelity gate failed; source glyph masks supplied as font hints; 20 DDIM steps; seed 0
|
||||
|
@@ -0,0 +1,5 @@
|
||||
date,source_commit,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,zimage-face-only,0,,0.06604512532552083,18,0.927,0.028,0.612,0.013,0.959,34.7,c388e9bbf737bb9b22a7cb2216ffb8026321ca632f9db29e1e456b7c14a8553f,synthid_detected
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-global-only,0,0.153999996,,18,0.589,0.217,0.331,0.288,0.646,24.2,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-zimage-full,0,0.153999996,0.06604512532552083,18,0.852,0.050,0.492,0.252,0.671,24.7,69de191bddbfbf53348c7638bdba661b6d45d257c00a8a2646b2558670c5a4c2,clean
|
||||
|
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
|
||||
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png": "每天都是一个新的机会。\n用微笑开始它。\n世界也会向你微笑。"
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Paddle en+ru+ch",0.378,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.048,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.101,0.120,0.796,19.5,0.054799,15b40b8cdf9e97558c66d4acdcb97d46769e64f04f143f8061746bc731fbbc21,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.081,0.119,0.796,19.5,0.070267,5a8a0bcbe5763d1b1af0d64149cec7e35b16c5751e995d3bacf17581d64177ec,not_checked,fidelity-only follow-up
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.123,0.113,0.808,21.2,0.041279,900def5a433c039aa474789eddc4dc4c63b8126ba1aff6829d2e8d600b9f9c71,no_openai_signals_detected,source detected immediately afterward in the same Chrome sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Paddle en+ru+ch",0.413,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.086,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.112,0.103,0.780,20.8,0.054912,846f39db0ad4a48a734a5019f0b0ff5d110eecb3210223407e72529dfa4fcf54,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.147,0.104,0.779,20.8,0.072556,8ae76e1dedeec26bebb8d35e71fd92d77f142dbb0f3440b330cdf33b79558afe,not_checked,fidelity-only follow-up
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.119,0.104,0.779,20.8,0.057110,ac4201bd2e342b1a37737319438cedb2203f6bb58cd9113f825cf9b80d913755,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-selective-automatic-passthrough,"Paddle detector ch; horizontal-only padded recognizer selection",0.074,0.289,0.696,23.3,0.000000,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_checked,all three source and candidate recognitions agreed exactly; output is byte-identical to candidate
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.127,0.113,0.805,20.9,0.041280,70c52fed8b8a1d98f4cda8558ae08825313e510dfb372e448370d8dcf1da73b0,not_checked,fidelity-only automatic-box follow-up; same 8/20 preserve decisions
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.154,0.104,0.778,20.8,0.057146,1fa01ce706042c003a61429d5d97fc6c928ec32278071a91926d588f1e760b00,not_checked,fidelity-only automatic-box follow-up; same 7/20 preserve decisions but worse CER
|
||||
|
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": [
|
||||
{"box": [156, 206, 709, 296], "text": "This is a longer", "script": "alphabetic"},
|
||||
{"box": [157, 293, 939, 379], "text": "sample text in English.", "script": "alphabetic"},
|
||||
{"box": [165, 393, 774, 434], "text": "Typography can flow smoothly from", "script": "alphabetic"},
|
||||
{"box": [161, 434, 521, 477], "text": "large to medium size.", "script": "alphabetic"},
|
||||
{"box": [165, 495, 566, 522], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
|
||||
{"box": [165, 524, 354, 550], "text": "rhythm, and clarity.", "script": "alphabetic"},
|
||||
{"box": [165, 564, 448, 585], "text": "Even the finest text should remain", "script": "alphabetic"},
|
||||
{"box": [165, 593, 330, 613], "text": "clean and readable.", "script": "alphabetic"},
|
||||
{"box": [158, 673, 903, 740], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
|
||||
{"box": [162, 753, 697, 793], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
|
||||
{"box": [164, 807, 524, 829], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
|
||||
{"box": [163, 845, 483, 868], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
|
||||
{"box": [162, 951, 780, 1009], "text": "Это более длинный пример", "script": "alphabetic"},
|
||||
{"box": [161, 1019, 723, 1062], "text": "текста на русском языке.", "script": "alphabetic"},
|
||||
{"box": [165, 1076, 639, 1104], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
|
||||
{"box": [165, 1111, 505, 1133], "text": "крупного размера к среднему.", "script": "alphabetic"},
|
||||
{"box": [165, 1151, 513, 1172], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
|
||||
{"box": [167, 1178, 287, 1192], "text": "ритм и ясность.", "script": "alphabetic"},
|
||||
{"box": [165, 1210, 417, 1227], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
|
||||
{"box": [165, 1235, 381, 1247], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
|
||||
],
|
||||
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": [
|
||||
{"box": [165, 204, 754, 299], "text": "This is a longer", "script": "alphabetic"},
|
||||
{"box": [167, 295, 1006, 385], "text": "sample text in English.", "script": "alphabetic"},
|
||||
{"box": [171, 401, 831, 446], "text": "Typography can flow smoothly from", "script": "alphabetic"},
|
||||
{"box": [169, 446, 560, 491], "text": "large to medium size.", "script": "alphabetic"},
|
||||
{"box": [173, 507, 605, 536], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
|
||||
{"box": [173, 538, 379, 567], "text": "rhythm, and clarity.", "script": "alphabetic"},
|
||||
{"box": [175, 583, 481, 603], "text": "Even the finest text should remain", "script": "alphabetic"},
|
||||
{"box": [173, 611, 352, 632], "text": "clean and readable.", "script": "alphabetic"},
|
||||
{"box": [170, 693, 956, 757], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
|
||||
{"box": [174, 775, 740, 812], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
|
||||
{"box": [175, 829, 561, 852], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
|
||||
{"box": [174, 869, 534, 892], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
|
||||
{"box": [169, 958, 824, 1022], "text": "Это более длинный пример", "script": "alphabetic"},
|
||||
{"box": [171, 1029, 764, 1074], "text": "текста на русском языке.", "script": "alphabetic"},
|
||||
{"box": [175, 1090, 678, 1119], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
|
||||
{"box": [175, 1125, 546, 1151], "text": "крупного размера к среднему.", "script": "alphabetic"},
|
||||
{"box": [175, 1169, 552, 1192], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
|
||||
{"box": [175, 1196, 312, 1217], "text": "ритм и ясность.", "script": "alphabetic"},
|
||||
{"box": [175, 1233, 479, 1253], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
|
||||
{"box": [177, 1259, 440, 1272], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
|
||||
],
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png": [
|
||||
{"box": [1281, 650, 2357, 818], "text": "每天都是一个新的机会。", "script": "cjk"},
|
||||
{"box": [1466, 788, 2167, 957], "text": "用微笑开始它。", "script": "cjk"},
|
||||
{"box": [1367, 910, 2264, 1112], "text": "世界也会向你微笑。", "script": "cjk"}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
date,source_commit,fixture,profile,route,seed,strength,adaptive_polish,text_cer,ocr_floor,added_cer,img_lpips,ssim,psnr_db,output_sha256,oracle_rechecked
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.378,0.127,0.251,0.107,0.832,24.4,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.356,0.127,0.229,0.125,0.707,25.3,9bb932d6e66b52f34ff07c3fcdbdb9c33eb601ae2cd84bec6b8c7b5e30df9926,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.413,0.127,0.286,0.095,0.818,26.0,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.422,0.127,0.295,0.099,0.754,25.9,7605b1175b0c99f481ae6929acc488f34d98bc0f2571d42a1d2972b671d9b8cb,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,original,reference,,,,0.037,0.037,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,visible-invisible-metadata,0,0.153999996,false,0.074,0.037,0.037,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,sdxl-zimage,visible-invisible-metadata,0,0.25,true,0.074,0.037,0.037,0.176,0.769,26.1,d9f89465c31bd2d38ffb111775b9bbea4577f40713fcbebf421e05d344ab27ad,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.068,0.127,-0.059,0.042,0.874,31.4,534de6422a3a03cfde44377aa9bf824a8267db3b73511240f8f07fb8dcb95eff,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.086,0.127,-0.041,0.047,0.862,30.8,4dba8211e8d84ebbfff72de00f54d4365d0d202394acd412d1419ae93d1e9f71,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.177,0.127,0.050,0.065,0.838,29.1,fc45c5fff6d33f15d30e05dd02580fe0addd97098b735b2894015f9abdaa43a7,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.248,0.127,0.121,0.093,0.813,27.3,1e55e11f0e289a752098370e9f1d53fed6cd4fab7ceff46fe86dd20c927f9ffb,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.343,0.127,0.216,0.120,0.787,25.7,f5a2d2829e88a790cff5bedc50fca1f878a92cdb392d530afb49d0d95c23b884,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.143,0.127,0.016,0.042,0.877,31.5,5cfa466e276ab37cf532f7458d6dd86ea4f0807e2cbdb6788b7c01ca6ee35a1a,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.101,0.127,-0.026,0.047,0.864,30.9,ade869ec7c5b08098495d9b9d67ee900773f189f94bbb411b043f242f5c2fa37,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.208,0.127,0.081,0.060,0.839,29.3,a11976963e3f2ca856ce66278e22064d51345d449fa6d672029039a79517d60a,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.341,0.127,0.214,0.070,0.819,27.6,1ab1ae075565e9d39960a80cf5d61e5b0c50d7e23283493ea0277056b3e964ab,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.538,0.127,0.411,0.084,0.799,25.6,5ff5a364d84f738de4c56ead7d4950a2f9a1aaf4bec16e97bda6a65915744a57,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.08,false,0.074,0.037,0.037,0.068,0.865,30.8,81419eddc3e5150956859dd10c26dd841bb93d9c8b34c01f344e6d69bca6edbf,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.10,false,0.074,0.037,0.037,0.086,0.844,29.8,4d1f32bd0dac0e05885ff943e7c10230b06de7d8547e6215f8e2941f732a1df5,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.15,false,0.000,0.037,-0.037,0.135,0.793,27.6,526c7987b34896f9513f7439b63b8a87573b59d15930fa307f3bb924435f3cec,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.20,false,0.000,0.037,-0.037,0.179,0.746,25.9,325364cf7a3b5d39ac0c44af82181753ca4d6fde0f1b522c896414191725e8fc,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.25,false,0.000,0.037,-0.037,0.218,0.703,24.5,f01d64dc0cabd45d66a93c1be39b819b988f29cda1f0a7bc0d5a1954571d807e,true
|
||||
|
@@ -0,0 +1,7 @@
|
||||
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.338,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.305,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,"Paddle ch",0.074,0.289,0.696,23.3,,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,synthid_detected,inconclusive account: clean baseline not reproduced
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-restored,"Paddle ch",0.111,0.328,0.653,18.1,0.051,2ab7666a4161ff0e12c53d9412b23d013970c7613a8a55a1b1ae8a2744e3290d,synthid_detected,inconclusive account: qwen-zimage control also detected
|
||||
|
@@ -0,0 +1,6 @@
|
||||
date,upstream_commit,workflow_sha256,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle,implementation
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/before/12.png,official-before,,,,10,,,,,,,056276464387c883f5fd89bd2f1415faeb425abb3153cd10efb25d1b9053bdc2,synthid_detected,upstream-published-bytes
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/after/12.png,official-after,,,,10,0.975,0.093,0.354,0.172,0.820,23.8,a822d805666224adb3e360b3ee89e14f20a341f1e7c4d3a5d3d5251e9286aefe,clean,upstream-published-bytes
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected,project-runtime-control
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-global-close-reproduction,0,0.153999996,,18,0.589,0.217,0.331,0.286,0.647,24.3,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean,diffsynth-bf16-yunet-sam
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-full-close-reproduction,0,0.153999996,0.13209025065104166,18,0.783,0.083,0.470,0.260,0.659,24.7,2d7a8f76f98b905f14d69e5f9e11979610c72c6be7b6ac298dec4659f17bd819,clean,diffsynth-bf16-yunet-sam
|
||||
|
@@ -1,3 +1,3 @@
|
||||
date,source_url,source_sha256,duration_seconds,source_verdict,noise_std,long_side,fps,seed,output_sha256,output_verdict,psnr_db,temporal_residual_ratio
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,8,detected_all_frames,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,26.2932,1.0072
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,8,detected_all_frames,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,25.3911,1.0578
|
||||
date,source_url,source_sha256,source_width,source_height,source_fps,duration_seconds,source_verdict,vae,noise_std,long_side,fps,seed,output_sha256,output_verdict,output_verdict_text,output_detected_range,track,session_id,stratum,psnr_db,temporal_residual_ratio
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,unrecorded,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,unrecorded,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
|
||||
|
@@ -0,0 +1,35 @@
|
||||
# Provenance fixtures
|
||||
|
||||
These files exercise real container, metadata, and pixel-provenance formats.
|
||||
They are test inputs, not evaluation corpora.
|
||||
|
||||
## Adobe TrustMark
|
||||
|
||||
`adobe-trustmark-p.png` is Adobe's official `images/ghost_P.png` Variant P
|
||||
example from [`adobe/trustmark`](https://github.com/adobe/trustmark) at commit
|
||||
`0ed40cbe8188f664fd9cbbeacd969807de27440a`.
|
||||
|
||||
- Source: `https://raw.githubusercontent.com/adobe/trustmark/0ed40cbe8188f664fd9cbbeacd969807de27440a/images/ghost_P.png`
|
||||
- SHA-256: `e58c5825ed7e5d9fb04710ea541b61bd55879cad65554c0d46260aa24b3d0755`
|
||||
- Expected signal: TrustMark Variant P, schema 1
|
||||
- License: MIT, reproduced in `../licenses/adobe-trustmark-MIT.txt`
|
||||
|
||||
|
||||
## Ideogram C2PA attribution
|
||||
|
||||
`ideogram-c2pa.png` is a SYNTHETIC signed manifest, not a vendor download: a
|
||||
64x64 flat PNG carrying a C2PA claim (`trainedAlgorithmicMedia`,
|
||||
`claim_generator "Ideogram/3.0"`) signed by a throwaway test CA whose leaf
|
||||
certificate subject is `O=Ideogram, Inc, CN=Ideogram, Inc`. It exercises the
|
||||
issuer-to-platform ATTRIBUTION path end to end through the real reader; the
|
||||
signer is deliberately untrusted (no trust anchors ship), which is also the
|
||||
fixture's second assertion: attribution and signer trust are separate
|
||||
dimensions.
|
||||
|
||||
- Expected signal: C2PA, platform `Ideogram`, `ai_from_metadata` True
|
||||
- Expected signer: `untrusted` (test CA; attribution must not imply trust)
|
||||
- Recipe (no keys are committed): create a CA and an ES256 leaf with
|
||||
`digitalSignature` key usage and subject `CN=Ideogram, Inc`, then
|
||||
`c2pa.Builder.from_json(...).sign_file(png, out, Signer.from_info(
|
||||
C2paSignerInfo(b"es256", leaf_pem + ca_pem, pkcs8_key, None)))`.
|
||||
- No user upload and no Ideogram artifact contributes pixels or bytes.
|
||||
|
After Width: | Height: | Size: 360 KiB |
|
After Width: | Height: | Size: 43 KiB |
@@ -0,0 +1,41 @@
|
||||
# Visible-mark example gallery
|
||||
|
||||
One committed example per registered visible mark, so the repository carries a
|
||||
working sample of everything it supports. `tests/test_visible_examples.py` holds
|
||||
both sides to it: a mark registered without an example fails the suite, and so
|
||||
does an engine that stops detecting its own example.
|
||||
|
||||
## What these files are
|
||||
|
||||
Every example is SYNTHETIC: `scripts/render_visible_examples.py` composites the
|
||||
mark's committed silhouette (the same font-rendered asset the detector matches)
|
||||
onto a deterministic generated base photo at the engine's measured geometry.
|
||||
The silhouettes are repository-owned font renders generated by
|
||||
`scripts/render_vendor_silhouettes.py`; the gallery does not copy vendor raster
|
||||
assets.
|
||||
|
||||
The examples demonstrate DETECTION geometry and house style, not vendor raster
|
||||
fidelity; real-world variants (fonts, opacities, sizes) are covered by the
|
||||
engines' recorded calibration evidence.
|
||||
|
||||
## Regeneration
|
||||
|
||||
uv run python scripts/render_visible_examples.py
|
||||
|
||||
The generator self-verifies: it fails (exit 1) if any registered mark does not
|
||||
detect on its own example, so regeneration is the fix point for drift.
|
||||
|
||||
## Layout
|
||||
|
||||
<mark-key>/example.png 1536x2048..2048x2048 PNG, one per image mark
|
||||
<mark-key>/example.mp4 960x540 90-frame clip, one per video mark
|
||||
(kling carries both: it is registered in both registries)
|
||||
|
||||
Special cases: `gemini` composites the sparkle alpha map at the provider's
|
||||
configured position; `jimeng_pill` is the capture-less pill at the measured
|
||||
3:4 portrait geometry; `microsoft` is the opaque white pill with dark text
|
||||
holes (the discriminator its detector keys on). Video examples composite the
|
||||
detector's own synthetic template on every frame; where two marks share a
|
||||
shape family the example carries the discriminative variant (`veo` the legacy
|
||||
text form, `kling` the logo-plus-wordmark pair flush to the edge), because the
|
||||
temporal selection resolves cross-template ties by table order.
|
||||
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 2.6 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
@@ -0,0 +1,25 @@
|
||||
Copyright 2023 Adobe
|
||||
All Rights Reserved.
|
||||
|
||||
NOTICE: Adobe permits you to use, modify, and distribute this file in
|
||||
accordance with the terms of the license agreement accompanying it.
|
||||
|
||||
MIT License
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -56,7 +56,9 @@ Ground-truth quality, strongest first:
|
||||
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
|
||||
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
|
||||
- `synthid-portal` — checked via Google's SynthID Detector portal.
|
||||
- `c2pa-metadata` — issuer-only proxy (Google/OpenAI C2PA manifest present). Weaker: the C2PA can be stripped while the pixel watermark remains.
|
||||
- `c2pa-metadata` — supported provenance evidence (Google AI C2PA, or OpenAI
|
||||
C2PA with an explicit `c2pa.watermarked.*` action). Weaker than a provider
|
||||
oracle: C2PA can be stripped while the pixel watermark remains.
|
||||
- `third-party` — label asserted by an external dataset, not independently verified.
|
||||
- `none` — unverified.
|
||||
|
||||
@@ -158,7 +160,7 @@ sparkle; the SynthID *pixel* itself has no local detector (oracle only).
|
||||
|
||||
Key takeaways:
|
||||
- The same model differs by *surface*: Gemini app wraps C2PA, AI Studio (API/playground) emits none -- only the pixel + sparkle survive.
|
||||
- Microsoft Designer's DALL-E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL-E).
|
||||
- Microsoft Designer's DALL·E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL·E).
|
||||
- Meta uses the IPTC `digitalSourceType` marker, not C2PA or SynthID.
|
||||
- The open imwatermark fires only on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL) -- not from re-hosts (Krea, Stability hosted SDXL) or re-encoded design exports (Recraft, Canva). Ideogram's only signal is the EXIF `Make` tag.
|
||||
- Bing and Grok web UIs are uncooperative for autonomous capture (no document_idle for screenshots; blob downloads intermittently no-op; low-res in-chat previews). Use their native download button manually if a full-res sample is needed.
|
||||
|
Before Width: | Height: | Size: 3.6 MiB After Width: | Height: | Size: 5.8 MiB |
@@ -0,0 +1,306 @@
|
||||
# Chroma1 engine research (2026-08-29/30)
|
||||
|
||||
> Research archive. This page records experiments and decisions from the dates
|
||||
> above. Current package behavior is defined by the
|
||||
> [supported signals](supported-signals.md), [known limitations](known-limitations.md),
|
||||
> and [module internals](module-internals.md). Nothing here is shipped.
|
||||
|
||||
Cited experiment behind issue #88 ("Please integrate Flux 2"): FLUX.2 has no
|
||||
native strength img2img anywhere (BFL reference code or diffusers; its image
|
||||
inputs are reference conditioning and outputs start from pure noise), so the
|
||||
requestable scrub lane in the FLUX family is **Chroma1** -- the FLUX.1
|
||||
architecture re-trained on open data, Apache-2.0 on weights, with a native
|
||||
`ChromaImg2ImgPipeline` strength parameter. This page records what was measured
|
||||
before any integration decision.
|
||||
|
||||
## Setup
|
||||
|
||||
All Chroma1 runs: `lodestones/Chroma1-HD` bf16 on Modal H100, seed 0, the
|
||||
neutral scrub prompt (`high quality, sharp, detailed, faithful to the original`
|
||||
/ `blurry, lowres, distorted text, garbled text, artifacts`), guidance 5.0, and
|
||||
**step compensation**: Diffusers truncates the step COUNT (`int(steps *
|
||||
strength)`), so the requested count is scaled to always spend 4 effective
|
||||
denoising steps, the same semantics as `sdxl_zimage_pipeline.requested_steps`.
|
||||
Without it, strength 0.05 runs a single step and the floor reads artificially
|
||||
high (the first, uncompensated sweep put the OpenAI boundary at (0.05, 0.10];
|
||||
compensated it is (0.05, 0.06]).
|
||||
|
||||
Two Flux-family traps found while wiring the prototype:
|
||||
|
||||
- without explicit `width`/`height`, the img2img pipeline silently generates
|
||||
1024x1024 regardless of input aspect;
|
||||
- dimensions floor to the /16 latent-patch grid, so the result must be resized
|
||||
back to the input size for pixel-exact comparison.
|
||||
|
||||
Scripts: `scripts/chroma_scrub_prototype.py` (first sweep),
|
||||
`scripts/engine_quality_price_probe.py` (head-to-head with the production
|
||||
qwen-zimage path), `scripts/engine_cohort_calibration.py` (the cohort ladders
|
||||
below), `scripts/chroma_meta_high_rungs.py` (the Meta rungs above 0.08).
|
||||
Generated outputs stay outside the repository (`out/`, gitignored); the durable
|
||||
verdicts are the tables on this page.
|
||||
|
||||
## Measured floors
|
||||
|
||||
Oracle per cohort: OpenAI via openai.com/verify (SynthID read separately from
|
||||
C2PA); Meta via the anonymous meta.ai/identification oracle. Each fixture's
|
||||
rungs were checked ascending and stopped at the first clean verdict.
|
||||
|
||||
| cohort | fixture | detected at | clean from |
|
||||
|---|---|---|---|
|
||||
| OpenAI | typography card 1122x1402 | 0.05 | 0.06 |
|
||||
| OpenAI | 9-face grid 1448x1086 | 0.06 | 0.075 |
|
||||
| OpenAI | full-pipeline quality fixture 1122x1402 | 0.05 | 0.06 |
|
||||
| Meta | gen_lighthouse_watercolor 1600x1600 | 0.08 | 0.10 |
|
||||
| Meta | gen_fox_forest 1600x1600 | 0.06 | 0.08 |
|
||||
| Meta | gen_night_city 1920x1280 | 0.045 | 0.06 |
|
||||
| Meta | gen_studio_mug 1600x1600 | 0.03 | 0.045 |
|
||||
| Meta | gen_text_poster 1600x1600 | -- | <=0.03 |
|
||||
|
||||
Derived operating points by the shipped rule (worst first-clean boundary plus
|
||||
one observed cross-source spread):
|
||||
|
||||
- **OpenAI: 0.09** (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and
|
||||
sdxl-zimage 0.15. The typography boundary (0.05, 0.06] is seed-stable across
|
||||
seeds 0, 1, 2.
|
||||
- **Meta: 0.17** (0.10 + (0.10 - 0.03)), oracle-verified clean on the worst
|
||||
fixture at 0.17. Against qwen-zimage 0.1. Chroma needs ~1.7x qwen's strength
|
||||
on Content Seal.
|
||||
- **Google: 0.40, measured 2026-08-30 across all four fixtures.** A
|
||||
metadata-stripped original verified as pixel-DETECTED through the @synthid
|
||||
Verify AI agent (the C2PA-carrying original answers about C2PA instead,
|
||||
which is not a pixel verdict).
|
||||
|
||||
| fixture | detected at | clean from |
|
||||
|---|---|---|
|
||||
| Gemini_Generated_Image_633uuy | 0.20 | 0.25 |
|
||||
| Gemini_Generated_Image_akdbei | 0.20 | 0.25 |
|
||||
| Gemini_Generated_Image_y48j3c | 0.08 | 0.12 |
|
||||
| Gemini_Generated_Image_3mc4t9 | 0.08 | 0.12 |
|
||||
|
||||
Derived by the shipped rule: worst first-clean 0.25 + spread (0.25 - 0.12
|
||||
= 0.13) = 0.38, rounded up to the measured rung **0.40**, then
|
||||
oracle-verified clean on the worst fixture. Against qwen-zimage 0.27:
|
||||
Chroma needs ~1.5x the strength on Google SynthID. At 0.15-0.20 the tool
|
||||
still reads positive while its wording degrades from "most or all" to
|
||||
"part of", so the boundary band is real signal decay, not noise.
|
||||
|
||||
Oracle mechanics learned while bracketing: the quota is per ACCOUNT and
|
||||
additionally dedupes SIMILAR images ("quota for checking images similar to
|
||||
this one"), roughly three checks per account window; switching accounts
|
||||
inside one browser works. A fresh Playwright context with the same Google
|
||||
session cookies (exported browser-wide via CDP `Storage.getCookies`, never
|
||||
through the transcript) carries working sign-in and its own check budget.
|
||||
The agent's own instructions leak into the answer on some renders and
|
||||
include an abstention rule for text-dominated images - always read the TOOL
|
||||
outcome, never the prose.
|
||||
- **Microsoft: 0.125**, measured 2026-08-30 on the SAME three valid Paint
|
||||
carriers behind the qwen floor (staged under
|
||||
`out/cohort-calibration/microsoft/` with pixel-identical metadata-stripped
|
||||
controls; they are real raiw-corpus user uploads and are never committed).
|
||||
Controls re-verified `Microsoft AI detected` on the day of the run. Against
|
||||
qwen-zimage 0.15: Chroma needs LESS strength on InvisMark.
|
||||
|
||||
| source (Paint) | detected at | clean from | qwen first-clean |
|
||||
|---|---|---|---|
|
||||
| paint-1 (47d87e99, 1536x1024) | 0.06 | 0.08 | 0.055 |
|
||||
| paint-2 (a579fb70, 1536x1024) | -- | <=0.04 | 0.04125 |
|
||||
| paint-3 (883191b5, worst qwen source) | 0.06 | 0.08 | 0.095 |
|
||||
|
||||
Derived by the shipped rule: 0.08 + (0.08 - 0.04) = 0.12, rounded to the
|
||||
measured rung **0.125**, which was then oracle-verified clean on both worst
|
||||
sources. Oracle: the public `ai.azure.com/nextgen/validate` page; its
|
||||
`Inconclusive` verdict is the site's watermark-negative, weaker than an
|
||||
API-level `Watermark: false`, the same caveat the qwen cohort carries.
|
||||
|
||||
## Fidelity at the floors (scripts/fidelity_metrics.py)
|
||||
|
||||
qwen-zimage rows ran through the package's own `QwenZImagePipeline` on the same
|
||||
H100 (production path, both stacks resident, prompt cache warm).
|
||||
|
||||
OpenAI typography (chroma 0.06 vs qwen 0.07675):
|
||||
|
||||
| | CER | img LPIPS | SSIM | PSNR |
|
||||
|---|---|---|---|---|
|
||||
| qwen-zimage | 0.241 | 0.083 | 0.834 | 27.7 |
|
||||
| chroma1 | **0.138** | **0.049** | **0.881** | **31.2** |
|
||||
|
||||
OpenAI full-pipeline fixture (chroma 0.09 vs qwen 0.07675): chroma wins every
|
||||
metric (CER 0.155 vs 0.259, LPIPS 0.056 vs 0.094, PSNR 29.6 vs 26.1).
|
||||
|
||||
OpenAI 9-face grid (chroma 0.075 global-only vs qwen 0.07675 with its Z-Image
|
||||
face stage):
|
||||
|
||||
| | ID cos | face LPIPS | img LPIPS | SSIM | PSNR |
|
||||
|---|---|---|---|---|---|
|
||||
| qwen-zimage | **0.857** | **0.051** | 0.065 | 0.865 | 31.1 |
|
||||
| chroma1 (no face stage) | 0.788 | 0.067 | **0.060** | **0.890** | **32.4** |
|
||||
|
||||
The face stage regenerates from the ORIGINAL crops and is engine-independent,
|
||||
so an integrated `chroma-zimage` would composite the same faces over the
|
||||
chroma global; whether that closes the identity gap is an integration-time
|
||||
measurement, not a given.
|
||||
|
||||
Meta at the respective floors (chroma 0.17 vs qwen 0.1): qwen wins perceptual
|
||||
fidelity on textured content (LPIPS: fox 0.163 vs 0.211, lighthouse 0.173 vs
|
||||
0.248, night city 0.187 vs 0.277), chroma wins the text poster decisively
|
||||
(LPIPS 0.008 vs 0.012, CER 0.000 vs 0.021) and the studio mug is a split. The
|
||||
extra 0.07 of strength costs real fidelity on photoreal content.
|
||||
|
||||
Google at the respective floors (chroma 0.40 vs qwen 0.27, qwen row includes
|
||||
its Z-Image face stage):
|
||||
|
||||
| fixture | qwen LPIPS/SSIM/ID cos | chroma LPIPS/SSIM/ID cos |
|
||||
|---|---|---|
|
||||
| 633uuy | **0.405 / 0.577** / - | 0.499 / 0.576 / - (CER 1.000 vs 0.333) |
|
||||
| akdbei | **0.440 / 0.552** / - | 0.540 / 0.557 / - (CER 0.796 vs 0.367) |
|
||||
| y48j3c (18 faces) | **0.436 / 0.608 / 0.801** | 0.560 / 0.548 / 0.279 |
|
||||
| 3mc4t9 (5 faces) | **0.422 / 0.529 / 0.921** | 0.562 / 0.461 / 0.143 |
|
||||
|
||||
qwen wins every axis that matters on every fixture; at 0.40 the Chroma global
|
||||
regeneration itself collapses face identity (0.14-0.28 cosine) and destroys
|
||||
dense text (CER up to 1.0). The extra 0.13 of strength is simply too much
|
||||
regeneration. This is the Meta pattern amplified.
|
||||
|
||||
Microsoft at the respective floors (chroma 0.125 vs qwen 0.15): chroma wins
|
||||
EVERY metric on EVERY source while scrubbing at lower strength --
|
||||
|
||||
| source | CER q/ch | img LPIPS q/ch | SSIM q/ch | PSNR q/ch |
|
||||
|---|---|---|---|---|
|
||||
| paint-1 | 0.741 / 0.241 | 0.164 / 0.106 | 0.742 / 0.847 | 24.8 / 29.3 |
|
||||
| paint-2 | 0.826 / 0.391 | 0.253 / 0.108 | 0.536 / 0.633 | 22.5 / 25.5 |
|
||||
| paint-3 | 0.000 / 0.000 | 0.038 / 0.020 | 0.762 / 0.953 | 36.3 / 42.2 |
|
||||
|
||||
The Microsoft cohort therefore behaves like the OpenAI cohort, not like Meta:
|
||||
for InvisMark carriers Chroma1 is the better engine at a lower operating point.
|
||||
|
||||
## Price (warm H100 container)
|
||||
|
||||
| | qwen-zimage | chroma1 |
|
||||
|---|---|---|
|
||||
| seconds per image | 2.44 | 3.62 |
|
||||
| VRAM peak | 31.9 GiB | 29.2 GiB |
|
||||
| USD per 1000 images ($3.95/h) | $2.68 | $3.97 |
|
||||
| weights in cache | ~70 GB | ~28 GB |
|
||||
|
||||
The chroma number includes a T5 encode per call; the shipped qwen path caches
|
||||
prompt embeddings, and the same treatment would put chroma near 1.5-2 s/image.
|
||||
Both stacks fit an 80 GB card; chroma's smaller footprint likely fits smaller
|
||||
cards untested.
|
||||
|
||||
## Matched-strength comparison: the floors hide the engine (2026-08-30 addendum)
|
||||
|
||||
The floor-based tables above compare chroma at its DERIVED floor (worst case
|
||||
+ spread) against qwen at its shipped floor - different perturbation levels.
|
||||
At the strength where each fixture ACTUALLY turns clean (chroma first-clean
|
||||
rungs, all oracle-verified above), the picture reverses:
|
||||
|
||||
Google (chroma@first-clean vs qwen@0.27):
|
||||
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | CER q/ch | ID q/ch |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 633uuy | 0.25 | 0.405/**0.375** | 0.577/**0.673** | 18.8/**22.3** | **0.333**/0.667 | - |
|
||||
| akdbei | 0.25 | 0.440/**0.409** | 0.552/**0.650** | 18.6/**22.5** | 0.367/**0.163** | - |
|
||||
| y48j3c | 0.12 | 0.436/**0.329** | 0.608/**0.700** | 21.0/**25.8** | 1.0/1.0 | **0.801**/0.611 |
|
||||
| 3mc4t9 | 0.12 | 0.422/**0.318** | 0.529/**0.644** | 17.7/**21.3** | 0.443/**0.229** | **0.921**/0.410 |
|
||||
|
||||
Meta (chroma@first-clean vs qwen@0.1):
|
||||
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | CER q/ch |
|
||||
|---|---|---|---|---|---|
|
||||
| lighthouse | 0.10 (equal) | 0.173/**0.097** | 0.783/**0.886** | 25.6/**30.8** | - |
|
||||
| fox | 0.08 | 0.163/**0.083** | 0.778/**0.907** | 25.6/**31.4** | - |
|
||||
| night_city | 0.06 | 0.187/**0.092** | 0.748/**0.893** | 23.9/**29.6** | 0.850/**0.250** |
|
||||
| studio_mug | 0.045 | 0.024/**0.009** | 0.984/**0.992** | **39.0**/37.8 | - |
|
||||
| text_poster | 0.03 | 0.012/**0.002** | 0.984/**0.994** | 34.0/**41.2** | 0.021/**0.007** |
|
||||
|
||||
At effective-clean strength Chroma1 wins whole-image fidelity on essentially
|
||||
every fixture (often by 4-6 dB) and text on five of six; qwen's remaining
|
||||
edge is face identity, which its Z-Image face stage buys (0.80-0.92 vs
|
||||
0.41-0.61 cosine) and a chroma-zimage profile would inherit on top of its own
|
||||
global pass.
|
||||
|
||||
**Why the floors still matter:** a shipped strength must clear the worst
|
||||
unseen content, and Chroma1's per-fixture boundaries scatter much wider
|
||||
(Google 0.12-0.25, Meta 0.03-0.10) than the spread the flat floors were built
|
||||
to absorb - so its derived floors (0.40 / 0.17) pay a large margin penalty
|
||||
and at those floors the extra strength genuinely destroys quality. The
|
||||
engines differ less in "quality per strength" than in how predictable their
|
||||
boundary is. The integration lever this exposes: a content-adaptive strength
|
||||
policy for chroma-zimage (the project already ships resolution-adaptive
|
||||
denoise for qwen) could capture chroma's low per-content boundaries; the two
|
||||
hard Google fixtures are both dense-text cards and the easy ones are
|
||||
photoreal, so a content-class signal may predict the rung.
|
||||
|
||||
## Pre-ship validation (2026-08-30)
|
||||
|
||||
With the `chroma-zimage` profile implemented
|
||||
(`_internal/chroma_zimage_pipeline.py` on `TwoStageZImagePipeline`, floors in
|
||||
`watermark_profiles`), two ship-gate checks were run:
|
||||
|
||||
1. **Seed stability at the worst Meta boundary.** lighthouse @ 0.10 with
|
||||
seeds 1 and 2: both CLEAN on the meta.ai oracle. The Meta boundary is not a
|
||||
seed-0 fluke. The Google seed probes (633uuy @ 0.25, seeds 1-2) were
|
||||
generated but the oracle's per-image-class quota blocked them on every
|
||||
available account; they remain queued, and the shipped 0.40 Google floor
|
||||
carries 0.15 of margin over that boundary regardless.
|
||||
2. **Full-path face-stage interaction.** The complete profile run (Chroma1
|
||||
global at the shipped Google floor 0.40, then the inherited
|
||||
YuNet/SAM/Z-Image face stage) on the 5-face fixture 3mc4t9: CLEAN on the
|
||||
@synthid oracle. The composited face regions do not re-introduce a
|
||||
detectable signal on top of the floor.
|
||||
|
||||
Generation script: `scripts/chroma_preship_validation.py`; outputs under
|
||||
`out/preship-chroma/` (gitignored).
|
||||
|
||||
## Verdict
|
||||
|
||||
Engine quality is cohort-dependent, and after measuring all four cohorts the
|
||||
split is even and sharp:
|
||||
|
||||
- On OpenAI-provenance content Chroma1 is better at every measured fidelity
|
||||
axis at its floor (0.09 vs 0.07675), for comparable money.
|
||||
- On Microsoft InvisMark content Chroma1 is better at every measured fidelity
|
||||
axis at a LOWER floor than qwen (0.125 vs 0.15).
|
||||
- On Meta Content Seal Chroma1 needs 1.7x the strength (0.17 vs 0.1) and loses
|
||||
LPIPS on textured content at its floor.
|
||||
- On Google SynthID Chroma1 needs ~1.5x the strength (0.40 vs 0.27) and loses
|
||||
every fidelity axis on every fixture, with face identity collapsing at 0.40
|
||||
(0.14-0.28 cosine vs qwen's 0.80-0.92).
|
||||
|
||||
A single-engine `chroma-zimage` default at a flat per-vendor floor is NOT
|
||||
supported: Google and Meta floors land too high and the quality at those
|
||||
floors is worse than qwen's. But the matched-strength addendum changes what
|
||||
IS supported: at the strength each image actually needs, Chroma1 regenerates
|
||||
better than qwen almost everywhere except face identity (which the shared
|
||||
face stage would supply). The two honest integration options:
|
||||
|
||||
1. Per-cohort engine router at flat floors: chroma for OpenAI/Microsoft,
|
||||
qwen for Google/Meta - conservative, ships on the measurements as they
|
||||
stand.
|
||||
2. `chroma-zimage` with a content-adaptive strength policy that predicts the
|
||||
per-image rung - potentially better everywhere, but the predictor is new
|
||||
research (the measured signal so far: dense-text content needs the high
|
||||
rungs, photoreal the low ones) and needs its own oracle validation before
|
||||
any ship.
|
||||
|
||||
Both need the seed-sensitivity and face-stage composition checks called out
|
||||
above before shipping. Both checks have now been run (Pre-ship validation
|
||||
above): the Meta boundary is seed-stable across seeds 0-2 and the full
|
||||
profile path stays clean through the face stage on the shipped Google floor.
|
||||
The profile ships with option 1's floors; option 2's adaptive policy remains
|
||||
the documented follow-up.
|
||||
|
||||
## Oracle session notes
|
||||
|
||||
- openai.com/verify throttles anonymous sessions through Cloudflare Turnstile
|
||||
after ~15 rapid checks; a fresh browser context clears it. The programmatic
|
||||
`POST /v1/content_provenance_checks` API is the better oracle (separate
|
||||
SynthID/C2PA outcomes) but the checked `OPENAI_API_KEY` returned 401
|
||||
org-wide, including `/v1/models`.
|
||||
- meta.ai/identification enforces a sliding per-IP window: a burst of ~15
|
||||
checks exhausted it twice; it reopens minutes later, and the page sometimes
|
||||
overstates this as a daily limit. Pacing of roughly 2-3 checks per window
|
||||
worked.
|
||||
- The Gemini app oracle requires the user's logged-in session; driving the
|
||||
user's live Chrome while it is in active use was refused (wrong-tab risk),
|
||||
so those 20 checks remain queued.
|
||||
@@ -16,17 +16,17 @@ defaults. This page focuses on choosing the right command.
|
||||
| `metadata` and metadata-only `identify` | Default package |
|
||||
| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` |
|
||||
| Adobe TrustMark signals in `identify` | `remove-ai-watermarks[trustmark]` |
|
||||
| Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` |
|
||||
| `visible` and `erase` with OpenCV | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| `visible` or `erase` with MI-GAN | `remove-ai-watermarks[migan]` |
|
||||
| `visible` or `erase` with big-LaMa | `remove-ai-watermarks[lama]` |
|
||||
| `invisible` | `remove-ai-watermarks[diffusion]` |
|
||||
| `invisible --pipeline qwen-zimage` | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| `invisible` and `all` (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| `video metadata` and `video identify --no-visible` | Default package |
|
||||
| `video identify`, `video visible`, and visible/all batch modes | `remove-ai-watermarks[video]` |
|
||||
| `video invisible` and `video all --invisible` | `remove-ai-watermarks[video,diffusion]` |
|
||||
| `video identify` | `remove-ai-watermarks[video]` |
|
||||
| `video visible`, `video all`, and visible/all batch modes | `remove-ai-watermarks[video]` plus ffmpeg on PATH |
|
||||
| `video invisible` and `video all --invisible` | `remove-ai-watermarks[video,diffusion]` plus ffmpeg on PATH |
|
||||
| HEIC/HEIF/AVIF pixel input | Add `remove-ai-watermarks[heif]` |
|
||||
| Every production command and backend | `remove-ai-watermarks[all]` |
|
||||
| Every production command and Python-compatible backend | `remove-ai-watermarks[all]` |
|
||||
|
||||
`batch` requires the same extra as its selected mode. Extras can be combined in
|
||||
one installation, for example `remove-ai-watermarks[visible,detect,heif]`.
|
||||
@@ -40,7 +40,15 @@ remove-ai-watermarks identify image.png
|
||||
`identify` always inspects supported metadata. When pixel extras are installed,
|
||||
it also evaluates supported visible and invisible pixel signals. When no signal
|
||||
is found, it reports the origin as unknown. It does not claim the image is
|
||||
clean.
|
||||
clean. For C2PA files, the text report shows asset integrity, claim-signature,
|
||||
signer-trust, and signer-validity results separately. Confidence follows the
|
||||
binding: an intact asset binding and claim signature is high confidence, and an
|
||||
unanchored or expired signer is reported as a caveat rather than a lower score,
|
||||
because no trust anchor list ships with the reader. A failed asset binding or
|
||||
signature, or a revoked signing credential, does not confirm the claimed origin. When a structured
|
||||
C2PA soft binding is present, the report also names its exact algorithm and
|
||||
signed value; removing the manifest does not remove the referenced pixel
|
||||
watermark or content fingerprint.
|
||||
|
||||
Machine readable output:
|
||||
|
||||
@@ -68,7 +76,8 @@ remove-ai-watermarks visible image.png -o clean.png
|
||||
The default behavior:
|
||||
|
||||
- checks every registered visible mark;
|
||||
- removes every detected match;
|
||||
- removes every detected match, except the weakly detected Jimeng label pill,
|
||||
which needs corroboration (see [supported signals](supported-signals.md));
|
||||
- selects the best installed fill backend;
|
||||
- strips AI metadata from the output.
|
||||
|
||||
@@ -129,6 +138,13 @@ remove-ai-watermarks erase image.png \
|
||||
`erase` accepts `cv2`, `migan`, and `lama`. The corresponding optional extra
|
||||
must be installed for a learned backend.
|
||||
|
||||
Two more knobs tune the fill. `--dilate N` (default 3) grows every box by `N`
|
||||
pixels before inpainting, which helps when a mark has a soft edge or a drop
|
||||
shadow just outside the box you measured; it applies to every backend because it
|
||||
shapes the mask. `--inpaint-method telea|ns` selects the classical algorithm and
|
||||
only affects the `cv2` backend. Like `visible`, `erase` strips AI metadata from
|
||||
the output by default; pass `--keep-metadata` to retain it.
|
||||
|
||||
## Strip AI metadata
|
||||
|
||||
Inspect metadata:
|
||||
@@ -146,6 +162,11 @@ remove-ai-watermarks metadata image.png --remove -o clean.png
|
||||
When `-o` is omitted, removal overwrites the source. Standard metadata is kept
|
||||
unless you pass `--remove-all`.
|
||||
|
||||
A quiet `--check` or a successful `--remove` is not a clean verdict. The command
|
||||
only inspects and strips embedded AI metadata; a pixel watermark such as SynthID
|
||||
has no local decoder once that metadata proxy is gone. `identify` reports the
|
||||
same limit.
|
||||
|
||||
The command also supports the audio and video containers listed in
|
||||
[supported signals](supported-signals.md). ffmpeg must be available for the
|
||||
non-ISOBMFF audio and video path.
|
||||
@@ -214,11 +235,17 @@ remove-ai-watermarks video metadata input.mp4 --check
|
||||
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
|
||||
```
|
||||
|
||||
As with the generic `metadata` command, a quiet check or a successful strip is
|
||||
not a clean verdict: video SynthID is not decoded locally after the metadata
|
||||
proxy is gone.
|
||||
|
||||
Supported containers are MP4, MOV, M4V, WebM, MKV, AVI, and FLV. The operation
|
||||
delegates to the same verified metadata scanner and stripper as the generic
|
||||
`metadata` command, so detection and removal stay in parity. Video and audio
|
||||
streams are not transcoded. For MP4 and MOV, this includes the native TC260
|
||||
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`. The inspector
|
||||
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`, plus the
|
||||
QuickTime-form `meta` variants Doubao's iOS export writes (a bare `meta` box
|
||||
as a direct `moov` child, and a keyless `hdlr=mdir` metadata list). The inspector
|
||||
seeks past a large `mdat` to find a tail `moov`. Removal stream-copies the
|
||||
container in bounded chunks, converts supported top-level provenance boxes to
|
||||
same-size `free` boxes, and blanks the TC260 key/value in place. Box sizes,
|
||||
@@ -290,7 +317,7 @@ remove-ai-watermarks video visible kling.mp4 --mark kling -o kling_clean.mp4
|
||||
|
||||
The command supports the moving Sora mascot and wordmark, two Veo
|
||||
corner variants, the Seedance boxed `AI` label, the `Dola AI` text label, the
|
||||
composite `MINIMAX | hailuo AI` label, and the bottom-right Kling label. Sora
|
||||
composite `MINIMAX | hailuo AI` label, and the bottom-right Kling AI label. Sora
|
||||
searches the whole frame at multiple scales. The other detectors search bounded
|
||||
lower-frame regions with separate synthetic silhouettes. Kling additionally
|
||||
requires its bright low-saturation label near the frame edge. Every mark
|
||||
@@ -300,8 +327,8 @@ provider provenance may relax the visual score only for registered
|
||||
provenance-aware marks; metadata alone never creates a detection.
|
||||
|
||||
`--mark auto` is the default. It evaluates all providers in one decode pass and
|
||||
selects the first stable match in specificity order: Sora, Veo, Seedance, Dola,
|
||||
Hailuo, then Kling. Their confidence scores are independently calibrated and
|
||||
selects the first stable match in specificity order: Sora, Veo, Seedance,
|
||||
Doubao, Dola, Hailuo AI, then Kling AI. Their confidence scores are independently calibrated and
|
||||
are not compared across providers. Pass an explicit `--mark` to scan only that
|
||||
provider.
|
||||
|
||||
@@ -340,10 +367,11 @@ failed encode does not overwrite an existing result.
|
||||
|
||||
## Remove invisible watermarks
|
||||
|
||||
Install the diffusion dependencies first:
|
||||
Install the removal dependencies first. All profiles are CUDA-only and all
|
||||
run the DiffSynth Z-Image face stage, so this is the extra either one needs:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[diffusion]"
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
```
|
||||
|
||||
Then run:
|
||||
@@ -359,14 +387,36 @@ detected. Use `--force` when you know the image should be processed:
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
### Choose a strength cohort
|
||||
|
||||
`--vendor` selects the cohort the default strength resolves from. `auto`
|
||||
(the default) derives it from provenance: the C2PA issuer for
|
||||
OpenAI/Google/Microsoft, and the standalone AI IPTC tag for Meta Content Seal
|
||||
(Muse output carries no C2PA; the tag is a standard code, so C2PA evidence
|
||||
always wins first). An explicit value both names the cohort for stripped files
|
||||
and implies the scrub runs -- naming the cohort asserts the pixel watermark is
|
||||
present -- exactly like `--force` plus a measured floor:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks invisible muse_output.webp -o clean.png --vendor meta
|
||||
```
|
||||
|
||||
The same option exists on `all` and `batch`, and as
|
||||
`InvisibleOptions(vendor="meta")` in the Python API.
|
||||
|
||||
### Choose a pipeline
|
||||
|
||||
| Pipeline | When to use it |
|
||||
| --- | --- |
|
||||
| `controlnet` | Default compatibility profile with structural conditioning |
|
||||
| `sdxl` | Lighter plain SDXL regeneration |
|
||||
| `qwen` | Large CUDA oriented Qwen Image profile |
|
||||
| `qwen-zimage` | CUDA only high fidelity profile with a separate face stage |
|
||||
| `qwen-zimage` | Default. Qwen-Image-2512 global pass plus a SAM-masked Z-Image face stage |
|
||||
| `sdxl-zimage` | The same recipe and face stage on an SDXL global pass, at a higher denoise |
|
||||
|
||||
**Both are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
removal. The former `controlnet`, `sdxl`, `qwen` and `default` profiles were removed
|
||||
rather than kept as a CPU path: none of them matched this recipe's face preservation,
|
||||
so offering them implied a quality the library no longer delivers. Passing a retired
|
||||
name is rejected at parse time rather than remapped. Visible-mark removal and every
|
||||
identify path still run anywhere.
|
||||
|
||||
Example:
|
||||
|
||||
@@ -375,8 +425,55 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --force
|
||||
```
|
||||
|
||||
The legacy `default` value is an alias for `sdxl`. The `--auto` option is
|
||||
deprecated, emits a warning, and changes nothing.
|
||||
There is no `--model`, `--steps`, `--guidance-scale` or `--device` option, and the
|
||||
deprecated `--auto` is gone. Each profile pins its model stack, its per-stage
|
||||
schedule, CFG 1.0 and CUDA, so every one of those flags existed only to be refused
|
||||
several layers down. They are not parsed at all now, which fails at the point the
|
||||
user can act on rather than after a model load.
|
||||
|
||||
### Restore operator-verified text
|
||||
|
||||
`--text-manifest` enables the experimental `vae-glyphs` post-pass. It reconstructs
|
||||
the source with the Qwen VAE, erases the annotated candidate glyphs with LaMa, and
|
||||
composites only the reconstructed glyph cores through source-derived silhouettes. It
|
||||
does not run OCR or choose which strings are correct. The optional `--fidelity-anchor`
|
||||
described below additionally blends 15% of the reconstruction across the full frame.
|
||||
|
||||
Install the combined extra and run only with an operator-verified manifest:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
``verified: true`` may also be set by an automated operator that attests
|
||||
machine-verified geometry: stability-gated detector boxes inside sane caps. Such
|
||||
operators should use the geometry-only schema 2, which carries no transcription
|
||||
or script metadata.
|
||||
|
||||
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default: it
|
||||
was measured to return detector-visible OpenAI SynthID on poster-scale manifests
|
||||
(official Content Provenance API, 2026-08-19). `--fidelity-anchor` restores the
|
||||
0.27.0 research behavior; text-box fidelity lost by the default is well under one
|
||||
MAE point on the measured fixtures.
|
||||
|
||||
The manifest is a JSON object with `verified: true`, decoded RGB dimensions,
|
||||
`source_pixel_sha256`, and a non-empty `lines` array. Schema 1 is retained for
|
||||
manually reviewed annotations: each line has an integer `[x1, y1, x2, y2]` box,
|
||||
exact `text`, a non-empty `script`, and an optional angle from -30 to 30 degrees.
|
||||
Schema 2 is geometry-only: each line has the box and optional angle, with no
|
||||
required `text` or `script`. Lines must be in top-to-bottom, left-to-right order.
|
||||
The hash binds the annotations to decoded RGB geometry and pixels, so metadata-only
|
||||
container changes remain valid while a resized or edited source fails closed. The
|
||||
experimental helper
|
||||
`remove_ai_watermarks._internal.text_restoration.source_pixel_sha256` computes it.
|
||||
|
||||
This mode is supported only by `qwen-zimage` at native geometry with
|
||||
`humanize=0`, `unsharp=0`, and adaptive polish disabled. `all` also accepts the flag,
|
||||
but its manifest must match the pixels entering the invisible stage; if visible-mark
|
||||
removal changes those pixels, the hash check rejects the run. One oracle verdict does
|
||||
not certify another manifest, seed, model/runtime version, or output hash.
|
||||
|
||||
### Work with limited memory
|
||||
|
||||
@@ -409,8 +506,8 @@ It is a memory strategy, not a guarantee of better quality.
|
||||
|
||||
The `all` command and the `all` installation extra are separate concepts. The
|
||||
command runs every applicable stage. Installing `remove-ai-watermarks[all]`
|
||||
makes every production backend available; a smaller installation such as
|
||||
`remove-ai-watermarks[visible,diffusion]` can also run the command with fewer
|
||||
makes every backend compatible with the active Python available; a smaller installation such as
|
||||
`remove-ai-watermarks[visible,qwen-zimage]` can also run the command with fewer
|
||||
optional backends.
|
||||
|
||||
```bash
|
||||
@@ -425,10 +522,16 @@ The command runs:
|
||||
|
||||
The visible options and diffusion options are also available on `all`.
|
||||
|
||||
If diffusion is required but the `diffusion` extra is unavailable, `all` still
|
||||
writes the result of the visible and metadata stages, prints a prominent
|
||||
warning, and exits with code 1. This prevents a partial result from being
|
||||
reported as complete.
|
||||
When the `qwen-zimage` extra is unavailable, `all` still writes the result of
|
||||
the visible and metadata stages, prints a prominent warning, and exits with code
|
||||
1. That happens on every run without the extra: the skipped stage is what would
|
||||
have decided whether a signal was there. This prevents a partial result from
|
||||
being reported as complete.
|
||||
|
||||
If the extra is installed but the machine has no CUDA, which is the usual macOS
|
||||
case, the run fails at engine construction instead: `all` prints
|
||||
`Error: Invisible-watermark removal is CUDA-only ...`, writes no output at all,
|
||||
and exits with code 1.
|
||||
|
||||
## Process a directory
|
||||
|
||||
|
||||
@@ -7,3 +7,15 @@ This page records notices required by source dependencies and licensed derivativ
|
||||
- The DWT-DCT implementation derives from ShieldMnt's
|
||||
[`invisible-watermark`](https://github.com/ShieldMnt/invisible-watermark), licensed
|
||||
under MIT. Its notice ships in `src/remove_ai_watermarks/licenses/invisible-watermark-MIT.txt`.
|
||||
The decode path is a vectorized reformulation, not a transcription: it produces
|
||||
the same bits and is checked against upstream's own decoder, but it no longer
|
||||
corresponds line by line to `imwatermark/maxDct.py`. Diffing the two files will
|
||||
show structurally different code, which is expected and does not mean the
|
||||
derivation notice is stale.
|
||||
|
||||
## Licensed test fixtures
|
||||
|
||||
- `data/fixtures/provenance/adobe-trustmark-p.png` is Adobe's official
|
||||
TrustMark Variant P example, licensed under MIT. Its source commit, digest,
|
||||
and reproduced license are recorded beside the fixture in
|
||||
`data/fixtures/README.md` and `data/licenses/adobe-trustmark-MIT.txt`.
|
||||
@@ -3,7 +3,10 @@
|
||||
> Research archive. This document records experiments, superseded defaults, and
|
||||
> deployment considerations from the time of the study. It does not define the
|
||||
> current CLI or Python API. See `README.md`, `docs/cli.md`, and
|
||||
> `docs/known-limitations.md` for current behavior.
|
||||
> `docs/known-limitations.md` for current behavior. Everything it names on the
|
||||
> face-restoration path was removed after this study: the `--restore-faces` and
|
||||
> `--face-id` flags, the `instantid`, `photomaker` and `faceid` extras, and their
|
||||
> modules. So were the `controlnet`, `sdxl`, `qwen` and `default` profiles.
|
||||
|
||||
Date: 2026-06-02. Source: a manual primary-source pass (WebSearch + WebFetch over the
|
||||
watermark-removal-attack and SDXL-ControlNet literature). Prompted by issue #35
|
||||
@@ -55,8 +58,7 @@ content it protects. Controlnet is the text/structure PRESERVATION pipeline; rem
|
||||
is set by STRENGTH, separately calibrated, not by the pipeline choice.
|
||||
|
||||
This section is the single consolidated reference for the controlnet pipeline's
|
||||
removal behavior. (Mirrored briefly in `docs/synthid.md` §5.5 and the CLAUDE.md
|
||||
controlnet / face_restore bullets, which point here.)
|
||||
removal behavior. (Mirrored briefly in `docs/synthid.md` §5.5, which points here.)
|
||||
|
||||
### What we measured (real gpt-image + Gemini originals)
|
||||
|
||||
@@ -288,8 +290,8 @@ text-vs-scrub tension as an empirical question to measure, not assume.
|
||||
**Prototype (runs locally on 32 GB MPS — no dedicated GPU required):**
|
||||
|
||||
Compute is NOT the bottleneck. On a 32 GB Apple-silicon machine (M5 here) native SDXL already
|
||||
runs entirely on MPS with no CPU fallback (~155 s at 1122x1402, verified — see `synthid.md` /
|
||||
CLAUDE.md). The prototype runs at **1024** (fewer pixels than that) with SDXL base + an SDXL
|
||||
runs entirely on MPS with no CPU fallback (~155 s at 1122x1402, verified 2026-05-31).
|
||||
The prototype runs at **1024** (fewer pixels than that) with SDXL base + an SDXL
|
||||
ControlNet + activations in **fp32** (MPS fp16 decodes to all-black NaN — issue #29 — confirmed
|
||||
on run 1 below; fp32 is the required default on mps/cpu) — fits the 32 GB budget with vae-tiling +
|
||||
attention-slicing; ~1-2 min/image, so a coarse sweep is a sub-hour background run. A dedicated GPU
|
||||
@@ -519,7 +521,7 @@ held by its masked ArcFace vector -- no original pixel copied.
|
||||
drift, NOT a perfect face swap. Set expectations; PuLID/InstantID are the higher-fidelity (heavier)
|
||||
paths if needed.
|
||||
- **Value scales with strength:** at low strength (OpenAI 0.10) faces barely drift, so FaceID is
|
||||
marginal; at the higher strength a hard vendor (Google 0.30) needs, FaceID earns its keep.
|
||||
marginal; at the higher strength then used for a hard vendor (Google 0.30), FaceID earned its keep.
|
||||
|
||||
### Build plan (staged)
|
||||
|
||||
@@ -534,7 +536,8 @@ https://instantid.github.io/ · https://arxiv.org/abs/2308.06721 (IP-Adapter)
|
||||
|
||||
### FaceID prototype run 1 -- 2026-06-03 (NEGATIVE on dense small-face groups)
|
||||
|
||||
Built and shipped the masked multi-face FaceID layer (`--face-id`, `face_id.py`, `faceid` extra).
|
||||
Built and shipped the masked multi-face FaceID layer (`--face-id`, `face_id.py`, `faceid` extra;
|
||||
all three were removed after this run and never reached the tracked tree).
|
||||
First real run on the gemini_3 group photo (Google, s015, scale 0.6, native 2816 via cap 1536):
|
||||
insightface detected **17 faces**, the masked multi-face pass composed and ran end-to-end (non-black
|
||||
output), so the API is correct. **At s015 the result is a clear FAILURE: every face corrupted --
|
||||
@@ -638,7 +641,7 @@ photo only).
|
||||
|
||||
**Sources.** https://arxiv.org/abs/2206.11253 (CodeFormer) · https://github.com/sczhou/CodeFormer ·
|
||||
https://arxiv.org/pdf/2401.07519 (InstantID) ·
|
||||
https://openaccess.thecvf.com/content/WACV2024/papers/Suin_Diffuse_and_Restore... (region-adaptive) ·
|
||||
https://openaccess.thecvf.com/content/WACV2024/html/Suin_Diffuse_and_Restore_A_Region-Adaptive_Diffusion_Model_for_Identity-Preserving_Blind_WACV_2024_paper.html (region-adaptive) ·
|
||||
https://arxiv.org/pdf/2504.12809 (saliency-aware watermark removal)
|
||||
|
||||
## Provenance
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# Development
|
||||
|
||||
Standalone evaluation, calibration, and release utilities are catalogued in
|
||||
[`scripts/README.md`](../scripts/README.md). They are maintainer tools rather than
|
||||
installed commands, and their local data inputs remain outside the repository.
|
||||
|
||||
Read this reference for environment setup, dependency recovery, CI behavior, and fixture policy. The always-loaded invariants remain in [`.claude/rules/development.md`](../.claude/rules/development.md).
|
||||
|
||||
## Local environment
|
||||
@@ -12,9 +16,18 @@ Read this reference for environment setup, dependency recovery, CI behavior, and
|
||||
|
||||
The optional TrustMark decoder downloads weights into its installed package directory. After pruning that extra, a leftover weights directory can make availability checks see an empty namespace package. If Pyright reports an unknown `TrustMark` import and `find_spec("trustmark")` returns a loader-less spec, remove that regenerable remnant from the active virtual environment and resync.
|
||||
|
||||
### Known security-gate blocks
|
||||
|
||||
`maintain.sh` fails while PyPI ships no fixed release for a transitive CVE. Current case (triaged 2026-08-17): `lightning` 2.6.5, pulled only by the optional `trustmark` extra, carries PYSEC-2026-3624 (RCE via `load_from_checkpoint` on an attacker-crafted checkpoint). The vulnerable API is unreachable here, because the TrustMark decoder loads only its own pinned weights downloaded from the TrustMark release, never a user-supplied checkpoint. The upstream fix is merged but unreleased, and ignores are never added, so run and report the remaining core checks (Ruff, Pyright, tests) separately until a fixed `lightning` release lands; bump it with `uv lock --upgrade-package lightning` as soon as one does.
|
||||
|
||||
## CI
|
||||
|
||||
`.github/workflows/test.yml` runs Ruff and a cross-platform supported-Python test matrix with default plus development dependencies. Diffusion and model-running tests skip in that matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the OpenCV eraser remain covered across operating systems.
|
||||
`.github/workflows/test.yml` runs Ruff, a test matrix over every supported Python
|
||||
minor with default plus development dependencies, cross-platform coverage at the
|
||||
oldest and newest minors, and a separate job that installs ffmpeg on Ubuntu to
|
||||
run the full-clip video test. Diffusion and model-running tests skip in that
|
||||
matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the
|
||||
OpenCV eraser remain covered across operating systems.
|
||||
|
||||
Keep `uv.lock` compatible with `uv sync --frozen`. Dependency pull-request checks use GitHub's merge result against current `main`; if `main` moves, merge it locally and rerun the full gate because a newer linter can expose stale directives in later code.
|
||||
|
||||
|
||||
@@ -19,9 +19,10 @@ to run the tool. Use the maintainer references only when changing the code.
|
||||
| Page | Purpose |
|
||||
| --- | --- |
|
||||
| [Module internals](module-internals.md) | Current architecture, invariants, and regression guards by module. |
|
||||
| [Development](development.md) | Environment setup, dependency recovery, CI behavior, and fixture policy. |
|
||||
| [Code provenance](code-provenance.md) | Required notices for licensed derivative work. |
|
||||
| [Verification plan](verification-plan.md) | Verification methods, completed measurements, and remaining validation gaps. |
|
||||
| [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, and release workflow. |
|
||||
| [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, ComfyUI Registry, and the release workflow. |
|
||||
| [Watermarking landscape](watermarking-landscape.md) | Vendor signals and detection approaches. |
|
||||
| [SynthID technical reference](synthid.md) | Mechanism, detector access, robustness, and implications for this project. |
|
||||
|
||||
@@ -36,4 +37,6 @@ The current behavior is defined by the code, tests, README, and user guides.
|
||||
- [Doubao reverse-alpha research](research-doubao-distillation.md)
|
||||
- [SynthID identity research](synthid-robust-identity-research.md)
|
||||
- [SynthID identity follow-up](synthid-robust-identity-research-2026-06-08.md)
|
||||
- [Video SynthID quality research](video-synthid-quality-research.md)
|
||||
- [Text protection research](text-protection-research.md)
|
||||
- [Chroma1 engine research](chroma1-engine-research.md)
|
||||
@@ -1,6 +1,6 @@
|
||||
# Installation
|
||||
|
||||
Python 3.10.1 or newer is required.
|
||||
Python 3.11 through 3.14 are supported.
|
||||
|
||||
## Default metadata mode
|
||||
|
||||
@@ -48,7 +48,10 @@ uv tool install --force "remove-ai-watermarks[visible,heif]"
|
||||
|
||||
## Video processing
|
||||
|
||||
Video metadata inspection and stripping work with the default package. Stable
|
||||
Video metadata inspection works with the default package, and MP4 and MOV
|
||||
stripping uses the in-tree ISOBMFF box walker. Stripping the non-ISOBMFF
|
||||
containers (MKV, WebM, AVI, FLV, and the audio formats) and writing any cleaned
|
||||
video need ffmpeg on PATH, for example `brew install ffmpeg`. Stable
|
||||
visible-mark identification and removal, full video cleaning, and visible/all
|
||||
batch modes need the `video` extra:
|
||||
|
||||
@@ -65,22 +68,36 @@ uv tool install --force "remove-ai-watermarks[video,diffusion]"
|
||||
|
||||
## Invisible watermark removal
|
||||
|
||||
Diffusion based removal needs the `diffusion` extra:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[diffusion]"
|
||||
```
|
||||
|
||||
The code supports CUDA, XPU, MPS, and CPU devices. A GPU is recommended because
|
||||
CPU inference is slow.
|
||||
|
||||
For the CUDA only Qwen Image plus Z-Image profile:
|
||||
Install the `qwen-zimage` extra:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
```
|
||||
|
||||
The `qwen-zimage` extra includes the normal `diffusion` dependencies.
|
||||
Both remaining profiles run a Z-Image face stage on the DiffSynth runtime, so
|
||||
both need this extra. The third profile, `chroma-zimage`, uses diffusers'
|
||||
ChromaImg2ImgPipeline instead of DiffSynth for its global stage but inherits
|
||||
the same face stage, so it needs this extra too. It includes the `diffusion`
|
||||
dependencies; `diffusion` on its own covers the torch and diffusers imports
|
||||
but not the face stage, so it is not enough to run a removal.
|
||||
|
||||
**An NVIDIA GPU is required.** `qwen-zimage`, `sdxl-zimage` and `chroma-zimage`
|
||||
are CUDA-only, and
|
||||
construction refuses any other device rather than falling back to a slow or broken
|
||||
one. There is no CPU, MPS or XPU path for invisible-watermark removal. Visible-mark
|
||||
removal, metadata stripping and every `identify` command still run anywhere.
|
||||
|
||||
Video SynthID regeneration is a separate VAE path and does still run on CPU or MPS;
|
||||
it needs the `diffusion` extra, not this one.
|
||||
|
||||
The experimental verified-text post-pass additionally needs LaMa:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
```
|
||||
|
||||
That extra includes `qwen-zimage` and `lama`; it does not add OCR. Text strings and
|
||||
line boxes must be reviewed before the run.
|
||||
|
||||
## Feature extras
|
||||
|
||||
@@ -94,14 +111,15 @@ application actually uses:
|
||||
| `visible` | Visible mark detection, OpenCV inpainting, and manual erasing | `pixels` | No |
|
||||
| `video` | Visible video identification/removal and timestamp preservation | `visible`, PyAV | No |
|
||||
| `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No |
|
||||
| `trustmark` | Adobe TrustMark detection | trustmark | Yes |
|
||||
| `diffusion` | Diffusion-based invisible watermark removal | `pixels`, Torch, Diffusers | Yes |
|
||||
| `trustmark` | Adobe TrustMark detection on Python 3.11-3.12 | trustmark | Yes |
|
||||
| `diffusion` | Torch and Diffusers runtime; video SynthID regeneration | `pixels`, Torch, Diffusers | Yes |
|
||||
| `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `esrgan` | Real-ESRGAN upscaling before diffusion | `pixels`, spandrel | Yes |
|
||||
| `qwen-zimage` | CUDA-only Qwen Image plus Z-Image pipeline | `diffusion`, DiffSynth | Yes |
|
||||
| `all` | Every production feature | All rows above | Yes |
|
||||
| `dev` | Tests, linting, typing, and upstream parity checks | `visible`, `detect`, upstream invisible-watermark | Yes, for parity tests |
|
||||
| `qwen-zimage` | Invisible image-watermark removal, both CUDA-only profiles | `diffusion`, DiffSynth | Yes |
|
||||
| `text-restoration` | Opt-in verified Qwen-VAE glyph restoration | `qwen-zimage`, `lama` | Yes |
|
||||
| `text-draft` | Draft OCR proposals for operator verification | PaddleOCR, PaddlePaddle | Model download, no Torch |
|
||||
| `all` | Every production feature available on the active Python | All compatible rows above | Yes |
|
||||
| `dev` | Tests, linting, typing, and upstream parity checks | `video`, `detect`, upstream invisible-watermark | Yes, for parity tests |
|
||||
|
||||
Dependency composition:
|
||||
|
||||
@@ -113,15 +131,21 @@ flowchart LR
|
||||
diffusion --> pixels
|
||||
migan --> visible
|
||||
lama --> visible
|
||||
esrgan --> pixels
|
||||
qwen["qwen-zimage"] --> diffusion
|
||||
text["text-restoration"] --> qwen
|
||||
text --> lama
|
||||
draft["text-draft"]
|
||||
heif
|
||||
trustmark
|
||||
```
|
||||
|
||||
`heif` and `trustmark` are independent branches. Combine them explicitly with
|
||||
another feature when required. The `all` bundle contains every production
|
||||
branch but never includes `dev`.
|
||||
`heif`, `trustmark`, and `text-draft` are independent branches. Combine them
|
||||
explicitly with another feature when required. `text-draft` is excluded from
|
||||
`all` because it proposes unverified OCR annotations and is not a production
|
||||
removal path. TrustMark requires NumPy 1.x, which has no
|
||||
CPython 3.13 or 3.14 wheels, so that branch is available only on Python
|
||||
3.11-3.12. The `all` bundle contains every production branch compatible with
|
||||
the active Python and never includes `dev`.
|
||||
|
||||
Examples:
|
||||
|
||||
@@ -138,7 +162,7 @@ uv tool install --force "remove-ai-watermarks[video]"
|
||||
# DWT-DCT and TrustMark detection without diffusion removal
|
||||
uv tool install --force "remove-ai-watermarks[detect,trustmark]"
|
||||
|
||||
# Every production capability
|
||||
# Every production capability compatible with this Python
|
||||
uv tool install --force "remove-ai-watermarks[all]"
|
||||
|
||||
# An arbitrary minimal combination
|
||||
@@ -188,8 +212,10 @@ Run the complete project gate:
|
||||
bash maintain.sh
|
||||
```
|
||||
|
||||
The script runs dependency checks, linting, formatting checks, type checking,
|
||||
and the test suite.
|
||||
The script syncs every optional backend on top of the `dev` environment above,
|
||||
then runs dependency checks, linting, formatting, type checking, and the test
|
||||
suite. It applies Ruff fixes and formatting in place rather than only reporting
|
||||
them.
|
||||
|
||||
## Hugging Face authentication
|
||||
|
||||
@@ -216,5 +242,7 @@ The normal behavior is to skip diffusion when no supported local signal is
|
||||
found. A missing signal does not prove that the image is clean. If you know the
|
||||
image came from a relevant generator, use `--force`.
|
||||
|
||||
If the CLI reports that diffusion dependencies are unavailable, install the
|
||||
`diffusion` extra. Video SynthID removal needs both `video` and `diffusion`.
|
||||
If the CLI reports that the removal dependencies are unavailable, install the
|
||||
`qwen-zimage` extra. `diffusion` alone covers Torch and Diffusers but not the
|
||||
DiffSynth face stage that all profiles run. Video SynthID removal is a separate
|
||||
path and needs `video` and `diffusion`.
|
||||
@@ -31,11 +31,14 @@ an unsupported locale, a different position, or a crop may be missed.
|
||||
|
||||
Known examples:
|
||||
|
||||
- The Microsoft detector covers one calibrated top-right white pill. Microsoft's
|
||||
[documented feature](https://support.microsoft.com/en-us/topic/include-a-watermark-when-content-from-microsoft-365-is-ai-generated-b00a656e-ae61-4692-8086-67d004421030)
|
||||
can instead use a Copilot icon, `AI-Generated` text, or another position.
|
||||
- Samsung detection is calibrated for the Italian
|
||||
`Contenuti generati dall'AI` text variant.
|
||||
- The Jimeng top-left pill has a weak visual detector and is intentionally
|
||||
subject to additional product and background checks.
|
||||
- Kling support covers the calibrated variants rather than every Kling label.
|
||||
- Kling AI support covers the calibrated variants rather than every Kling AI label.
|
||||
|
||||
Use `erase --region` when you can see and select an unsupported or missed mark.
|
||||
|
||||
@@ -57,18 +60,49 @@ Invisible removal does not decode and delete a payload. It regenerates the
|
||||
image through a diffusion pipeline. Faces, text, colors, and fine detail can
|
||||
change even when the watermark is successfully disrupted.
|
||||
|
||||
ControlNet is the default compatibility profile. It conditions on edges to
|
||||
preserve structure, but edges do not preserve identity or exact texture.
|
||||
Text, tables, and UI screenshots are the worst case for a forced scrub: the
|
||||
model redraws glyphs as plausible-but-wrong shapes. Do not run the invisible
|
||||
stage on such content unless a locally-detectable invisible signal exists.
|
||||
The default `all`, `invisible`, and `batch` commands already skip it then, and
|
||||
`visible` (which strips AI metadata by default) or `metadata` alone never
|
||||
redraws a glyph outside the filled watermark box. `--force` is for content you
|
||||
know carries a pixel watermark despite no local signal; on a clean screenshot
|
||||
it only buys distortion. Preserving or pasting back the original text pixels
|
||||
during a real scrub is deliberately not offered: an invisible watermark such
|
||||
as SynthID lives in the pixels everywhere, including inside the glyphs, so
|
||||
frozen text regions would keep the watermark
|
||||
([text protection research](text-protection-research.md)).
|
||||
|
||||
The CUDA only `qwen-zimage` profile adds a separate face stage and is the
|
||||
highest fidelity option in the current implementation. It is larger, slower,
|
||||
and still may alter small text or difficult faces.
|
||||
`qwen-zimage` is the default profile and `sdxl-zimage` the only alternative.
|
||||
Both are CUDA only and differ only in the global regeneration model: each
|
||||
conditions that stage on a canny edge map, which preserves structure but not
|
||||
identity or exact texture, and each then runs the same face stage.
|
||||
Existing face evaluations favor `qwen-zimage`, but there is no blanket fidelity
|
||||
ordering across content types. A fixed-seed, three-scene text comparison at the
|
||||
profile defaults found no stable winner: SDXL won one poster, Qwen won one, and
|
||||
the Chinese sign tied. Both are large, slow, and may still alter small text or
|
||||
difficult faces. The measurements and their OCR and oracle caveats are tracked
|
||||
in [`data/evaluations/fidelity/`](../data/evaluations/fidelity/README.md).
|
||||
A global Z-Image Turbo prototype preserved text substantially better at low
|
||||
strength, but it has no useful cross-provider operating point and is not a
|
||||
supported profile. Automatic text restorers also remain research-only:
|
||||
fresh-font and silhouette variants visibly changed typography. The higher-fidelity
|
||||
`vae-glyphs` route is available only as an experimental opt-in with verified strings
|
||||
and line geometry. It builds its donor internally but still requires an independently
|
||||
clean global anchor. Automatic OCR and line-box proposals are not reliable enough to
|
||||
remove those requirements, and exact oracle results do not establish a general mask,
|
||||
seed, runtime, or provider operating range. Qwen-Image-2.0 is hosted-only and exposes
|
||||
no equivalent low-strength denoise control. Exact experiments, controls, and pass
|
||||
rates are kept in
|
||||
[`text-protection-research.md`](text-protection-research.md) and the
|
||||
[`fidelity` evaluation record](../data/evaluations/fidelity/README.md).
|
||||
|
||||
### Removal cannot be verified locally for proprietary SynthID
|
||||
|
||||
The project has no public local SynthID pixel decoder. It can infer likely
|
||||
presence from supported provenance metadata, but a missing metadata proxy is
|
||||
not a negative pixel verdict.
|
||||
The project has no public local SynthID pixel decoder. It recognizes presence
|
||||
from supported provenance metadata: Google AI C2PA under Google's all-media
|
||||
policy, or current OpenAI C2PA with an explicit watermark action. A missing
|
||||
provenance signal is not a negative pixel verdict.
|
||||
|
||||
For important outputs:
|
||||
|
||||
@@ -80,6 +114,14 @@ For important outputs:
|
||||
Provider systems can change, so a result verified on one file, seed, or version
|
||||
is not a permanent certification.
|
||||
|
||||
Meta Content Seal (Muse Image) is in the same family: no local decoder, presence
|
||||
recognizable only through the XMP `trainedAlgorithmicMedia` companion tag while
|
||||
that metadata survives, and removal verifiable only through the anonymous
|
||||
`meta.ai/identification` oracle, which rate-limits by IP per day. The oracle's
|
||||
negative is weaker than its positive: Reuters measured it missing 55% of cropped
|
||||
Muse images, so treat a negative on a cropped or heavily edited file as
|
||||
inconclusive rather than clean.
|
||||
|
||||
### Video SynthID removal is lossy and content-dependent
|
||||
|
||||
The `video invisible` command and `remove_video_invisible` API regenerate video
|
||||
@@ -111,6 +153,21 @@ measured 25.39 dB paired PSNR and a 1.058 motion-compensated temporal-residual
|
||||
ratio. This is one carrier, not a universal guarantee; hashes and exact verdicts
|
||||
are tracked in `data/evaluations/video-synthid-oracle.csv`.
|
||||
|
||||
### The invisible video path does not touch the audio track
|
||||
|
||||
`remove_video_invisible` regenerates the video stream and copies the source audio
|
||||
verbatim: the extracted audio bitstream of input and output has an identical
|
||||
sha256, measured on two carriers. It also strips every metadata marker, so the
|
||||
output can report clean from `get_ai_metadata` and `identify_video` while carrying
|
||||
an untouched generated audio track. Google's verifier scores audio and visual
|
||||
tracks separately, and a track this path never modifies is a track it cannot have
|
||||
cleaned.
|
||||
|
||||
Whether a given carrier's audio actually holds a mark the verifier reads has not
|
||||
been established -- that needs a provider verdict, which has not been obtained.
|
||||
Treat a clean local report on a clip with generated audio as unproven, not as a
|
||||
guarantee, and check the audio separately when it matters.
|
||||
|
||||
The shipped engine streams sampled frames in bounded batches, computes its
|
||||
fidelity metrics incrementally, and pipes regenerated pixels directly to
|
||||
ffmpeg. Its frame and latent memory is therefore bounded by `--batch-size`
|
||||
@@ -118,45 +175,90 @@ rather than clip duration. Runtime still grows linearly with duration, and the
|
||||
separate multi-candidate research sweep deliberately retains its short sampled
|
||||
prefix so it can reuse identical latents across candidate strengths.
|
||||
|
||||
The reported `psnr_db` is measured against the already-resized frame and before
|
||||
the H.264 encode, so it excludes the downscale, the frame decimation, and the
|
||||
encoder. It measures the VAE round trip plus latent noise at the working geometry.
|
||||
[`video-synthid-quality-research.md`](video-synthid-quality-research.md) records
|
||||
what the manifest rows constrain, what a higher-resolution or higher-frame-rate
|
||||
profile would need in order to be certified, and the audio-track question this
|
||||
path has not yet answered.
|
||||
|
||||
### Strength is content and seed dependent
|
||||
|
||||
For SDXL and ControlNet, the CLI resolves an unset strength from the detected
|
||||
vendor:
|
||||
The profiles resolve an unset strength differently, because different things
|
||||
were measured for each.
|
||||
|
||||
- OpenAI: `0.10`;
|
||||
- Google: `0.15`;
|
||||
- unknown: `0.15`.
|
||||
`qwen-zimage` reads unknown content from the resolution-adaptive denoise curve.
|
||||
Measured provider cohorts instead take flat operating points: OpenAI `0.07675`,
|
||||
Google `0.27`, and Microsoft InvisMark `0.15`. OpenAI and Microsoft add one full
|
||||
observed cross-source boundary spread to the worst clean source; Microsoft's three
|
||||
first-clean boundaries were `0.04125`, `0.055`, and `0.095`, giving `0.14875` before
|
||||
rounding up. Google's candidate was separately repeated across three valid sources
|
||||
and three accounts. The small corpora make these operating points, not universal
|
||||
thresholds.
|
||||
|
||||
An explicit `--strength` overrides these defaults. The defaults are operating
|
||||
points, not universal guarantees. Near a removal threshold, different content
|
||||
or a different random seed may change the verifier result.
|
||||
`sdxl-zimage` reads it from the C2PA issuer, on a flat ladder:
|
||||
|
||||
The base Qwen and `qwen-zimage` profiles have profile specific strength
|
||||
behavior. Consult `remove-ai-watermarks invisible --help` and the source of
|
||||
[`watermark_profiles.py`](../src/remove_ai_watermarks/_internal/watermark_profiles.py)
|
||||
for the current resolver.
|
||||
- OpenAI: `0.15`;
|
||||
- Google: `0.25`;
|
||||
- unknown: `0.25`, following the stricter of the two.
|
||||
|
||||
An SDXL global pass needs more denoise than Qwen at the same fidelity, and the
|
||||
values are flat rather than a curve because flat values are what was measured: each
|
||||
verdict came from a fixed strength at one size, and no size dependence has been
|
||||
established for that stage.
|
||||
|
||||
`chroma-zimage` also reads a flat ladder from the provenance cohort, measured
|
||||
2026-08-29/30 across the four vendor oracles (full record:
|
||||
[`chroma1-engine-research.md`](chroma1-engine-research.md)):
|
||||
|
||||
- OpenAI: `0.09` (below qwen's `0.07675` is false economy neither way -- the
|
||||
matched-strength fidelity favors Chroma1 on this cohort);
|
||||
- Microsoft InvisMark: `0.125` (below qwen's `0.15`);
|
||||
- Google: `0.40` (above qwen's `0.27`; at this floor the regeneration destroys
|
||||
dense text and collapses face identity -- the tradeoff that motivates the
|
||||
documented content-adaptive follow-up);
|
||||
- Meta Content Seal: `0.17` (above qwen's `0.1`);
|
||||
- unknown: `0.40`, following the strictest measured cohort.
|
||||
|
||||
An explicit `--strength` overrides all three. The defaults are operating points,
|
||||
not universal guarantees. Near a removal threshold, different content or a
|
||||
different random seed may change the verifier result, which is why the profiles
|
||||
are certified at a fixed seed. The live resolver is
|
||||
[`watermark_profiles.py`](../src/remove_ai_watermarks/_internal/watermark_profiles.py).
|
||||
|
||||
### Pipelines have different quality tradeoffs
|
||||
|
||||
| Pipeline | Main limit |
|
||||
| --- | --- |
|
||||
| `controlnet` | Edge conditioning can preserve a watermark carrying region too closely, and faces may drift. |
|
||||
| `sdxl` | Flat graphics and precise structure may receive too little or unhelpful change. |
|
||||
| `qwen` | Large CUDA oriented model; face smoothing can still be significant. |
|
||||
| `qwen-zimage` | CUDA only, large model stack, and limited broad certification across seeds and content. |
|
||||
| `sdxl-zimage` | CUDA only. Its strength ladder is flat per vendor, not a resolution curve, because flat values are what was measured. |
|
||||
|
||||
The legacy `default` profile name maps to `sdxl`. The `--auto` flag is
|
||||
deprecated, emits a warning, and changes nothing.
|
||||
Only manually verified `vae-glyphs` is an optional production stage, and it is
|
||||
experimental rather than a default.
|
||||
OCR plus LaMa recovered literal poster text but changed fonts and worsened whole-image
|
||||
fidelity. Restricting it to OCR-mismatched lines improved the tradeoff but still
|
||||
left a local shadow on one poster. The published AnyText2 SD1.5 checkpoint
|
||||
substituted Chinese characters and increased CER on the sign fixture. AnyText2XL
|
||||
is not published, and the released wrapper truncates individual text lines after
|
||||
20 characters.
|
||||
|
||||
The `controlnet`, `sdxl`, `qwen` and `default` profiles were removed, not aliased
|
||||
onward: a retired name is rejected at parse time rather than routed into a profile
|
||||
the caller never chose. There is no `--model`, `--steps`, `--guidance-scale`,
|
||||
`--device` or `--auto` option either; each profile pins its model stack, its
|
||||
per-stage schedule, CFG 1.0 and CUDA.
|
||||
|
||||
## Resolution and memory
|
||||
|
||||
### Small images are enlarged before SDXL based diffusion
|
||||
### Small images are processed at their native size
|
||||
|
||||
The SDXL, ControlNet, and base Qwen paths use a default minimum long side of
|
||||
`1024`. Smaller inputs are enlarged before diffusion and restored to their
|
||||
original dimensions afterward. Set `--min-resolution 0` to disable the floor.
|
||||
There is no minimum-resolution floor. It existed to enlarge small inputs toward
|
||||
SDXL's ~1024 training resolution and was removed with the SDXL profiles, which
|
||||
never applied it anyway. Both surviving profiles run at native geometry, so a
|
||||
small input is neither enlarged before diffusion nor restored afterward.
|
||||
|
||||
`qwen-zimage` does not apply this SDXL minimum resolution floor.
|
||||
`--max-resolution` still caps very large inputs, and only ever scales down.
|
||||
|
||||
### Large images stay at native resolution unless capped
|
||||
|
||||
@@ -169,23 +271,25 @@ overlapping tiles. It avoids the explicit downscale, but it is not pixel
|
||||
lossless: each tile is independently regenerated. With `qwen-zimage`, only the
|
||||
global Qwen stage is tiled; the face stage runs after tile blending.
|
||||
|
||||
### CPU offload is CUDA only
|
||||
### CPU offload trades speed for VRAM
|
||||
|
||||
`--cpu-offload` moves Diffusers model components between CPU and CUDA instead of
|
||||
keeping the complete standard pipeline in GPU memory. For `qwen-zimage`, it
|
||||
forces the face stack to use its offload path.
|
||||
`--cpu-offload` forces both stacks of the two-stage profile out of automatic
|
||||
device residency, streaming weights instead of pinning them. It reduces CUDA
|
||||
memory pressure at the cost of speed.
|
||||
|
||||
The option reduces CUDA memory pressure at the cost of speed. It has no effect
|
||||
on CPU or MPS and fails loudly when a CUDA Diffusers pipeline does not expose
|
||||
the required offload method.
|
||||
Residency is otherwise chosen from the card's total VRAM. On a card large enough
|
||||
to hold a stack, offloading is pure waste: DiffSynth drops weights to the meta
|
||||
device and re-reads every parameter from disk on each stage transition.
|
||||
|
||||
### MPS may fall back to CPU
|
||||
### There is no CPU or MPS fallback
|
||||
|
||||
The SDXL paths include an MPS out-of-memory fallback that reloads on CPU. A run
|
||||
that appears much slower after an MPS failure may be continuing on CPU.
|
||||
Invisible-watermark removal refuses any device but CUDA at construction. There is
|
||||
no MPS out-of-memory fallback that continues on CPU, and no lighter profile to
|
||||
drop to: all three profiles need an NVIDIA GPU.
|
||||
|
||||
Memory needs depend on the pipeline, input size, dtype, and machine. Use tiling,
|
||||
a resolution cap, or a lighter pipeline when necessary.
|
||||
When a run does not fit, the levers are `--tile` (native geometry, tiled
|
||||
diffusion), `--max-resolution` (an explicit downscale), and `--cpu-offload`.
|
||||
Memory needs depend on the profile, input size, dtype, and card.
|
||||
|
||||
## Metadata and formats
|
||||
|
||||
@@ -222,11 +326,11 @@ supported AI provenance metadata without transcoding streams.
|
||||
|
||||
`video visible` and `remove_video_visible` additionally support the moving
|
||||
Sora 2 mascot and wordmark, the current Veo four-point diamond, the legacy
|
||||
`Veo` text, the Seedance boxed `AI` label, the fixed `Dola AI` text, the Hailuo
|
||||
MINIMAX/Hailuo composite label, and the bottom-right Kling label with its
|
||||
`Veo` text, the Seedance boxed `AI` label, the fixed `Dola AI` text, the Hailuo AI
|
||||
MINIMAX/Hailuo AI composite label, and the bottom-right Kling AI label with its
|
||||
version suffix. Detection requires a recurring visual candidate across
|
||||
adjacent frames. Fixed-mark candidates must remain anchored rather than
|
||||
drifting with a scene object. Kling also requires a bright low-saturation
|
||||
drifting with a scene object. Kling AI also requires a bright low-saturation
|
||||
candidate near the expected frame edge. Provider provenance can recover
|
||||
low-contrast runs only after visual evidence exists for the marks that define a
|
||||
provenance prior, so metadata alone does not erase a clean API export.
|
||||
@@ -237,7 +341,7 @@ policies and selects the first stable result in specificity order. Use an
|
||||
explicit mark when the provider is already known.
|
||||
Historical Sora Turbo exports use a small OpenAI swirl in the corner rather
|
||||
than the moving mascot-and-wordmark design; that earlier variant is not
|
||||
detected by the `sora` video mark. Hailuo and Kling coverage is specific to the
|
||||
detected by the `sora` video mark. Hailuo AI and Kling AI coverage is specific to the
|
||||
verified lower-edge layouts; a new provider layout needs a separate calibrated
|
||||
silhouette. Other provider video labels are not supported yet. Google video
|
||||
SynthID has an oracle-certified VAE removal path, while other proprietary
|
||||
@@ -323,8 +427,17 @@ Diffusion, SDXL, and FLUX workflows. That decoder is sensitive to the carrier
|
||||
and transformations. A negative result is not a universal negative.
|
||||
|
||||
The `trustmark` extra adds Adobe TrustMark decoding. The implementation retains
|
||||
an additional JPEG re-encode gate because isolated decoder hits can otherwise
|
||||
be content noise.
|
||||
an additional JPEG re-encode gate and requires the binary payload and schema to
|
||||
remain identical because isolated decoder hits can otherwise be content noise.
|
||||
It accepts Variant P schemas 0-2. Variant Q requires a different model, and
|
||||
schema 3 is rejected at the measured precision threshold. Its NumPy 1.x runtime
|
||||
limits the extra to Python 3.11-3.12; the rest of the package remains supported
|
||||
through Python 3.14. The current TrustMark dependency line also resolves lightning 2.6.5
|
||||
(PYSEC-2026-3624, no fixed release yet); the vulnerable `load_from_checkpoint`
|
||||
path is unreachable here because TrustMark loads its checksummed checkpoints
|
||||
with plain `torch.load`. Bump lightning and cut a patch release when a fix
|
||||
ships. The calibration history is in
|
||||
[module internals](module-internals.md#metadata-and-provenance).
|
||||
|
||||
External AI versus real image classifiers are out of scope. The project
|
||||
identifies concrete local provenance signals instead of shipping a generic
|
||||
|
||||
@@ -6,8 +6,9 @@ and pipeline modules are intended for maintainers and specialized workflows.
|
||||
Dependency groups are identical for the CLI and Python API. The default install
|
||||
covers metadata extraction, normalization, verdict logic, and stripping.
|
||||
Array/pixel APIs use `pixels`; visible removal uses `visible`; DWT-DCT detection
|
||||
uses `detect`; diffusion removal uses `diffusion`; and visible video processing
|
||||
uses `video`. Video SynthID removal combines `video` and `diffusion`. Add `heif`
|
||||
uses `detect`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and
|
||||
visible video processing uses `video`. Video SynthID removal is a separate VAE
|
||||
path that still runs on CPU and combines `video` and `diffusion`. Add `heif`
|
||||
independently when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See
|
||||
the complete [feature-extra matrix](installation.md#feature-extras).
|
||||
|
||||
@@ -76,6 +77,73 @@ image = cv2.imread("input.png")
|
||||
result, removed = raiw.remove_visible(image, backend="cv2")
|
||||
```
|
||||
|
||||
## Run the full pipeline
|
||||
|
||||
`remove_all` is the library form of the `all` command: visible marks, then the
|
||||
invisible watermark, then AI metadata. Stages are chained through a file in the
|
||||
system temp directory, so a partial result never appears at the output path.
|
||||
|
||||
```python
|
||||
import remove_ai_watermarks as raiw
|
||||
|
||||
result = raiw.remove_all("input.png", "clean.png") # -> RemoveAllResult
|
||||
print(result.output) # the path written
|
||||
print(result.visible_label) # the marks removed, or None
|
||||
print(result.invisible) # "removed" | "no-signal" | "unavailable"
|
||||
```
|
||||
|
||||
`invisible` is the field to check. `"unavailable"` means the GPU extra is not
|
||||
installed, so the output *looks* processed but still carries the watermark;
|
||||
`"no-signal"` means the scrub was deliberately skipped because nothing was
|
||||
locally detectable, which is a successful run.
|
||||
|
||||
Pass `InvisibleOptions` to tune the diffusion stage, and `engine` to reuse one
|
||||
loaded model across many calls:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks import InvisibleOptions
|
||||
|
||||
raiw.remove_all(
|
||||
"input.png",
|
||||
"clean.png",
|
||||
invisible=InvisibleOptions(strength=0.35),
|
||||
force=True,
|
||||
progress=print,
|
||||
)
|
||||
```
|
||||
|
||||
`vendor="meta"` names a strength cohort explicitly (the measured Content Seal
|
||||
floor) for a stripped file whose provenance no longer carries the AI IPTC tag,
|
||||
and implies the scrub runs:
|
||||
|
||||
```python
|
||||
raiw.remove_all(
|
||||
"muse_output.webp",
|
||||
"clean.png",
|
||||
invisible=InvisibleOptions(vendor="meta"),
|
||||
)
|
||||
```
|
||||
|
||||
`InvisibleOptions` carries only what `InvisibleEngine` itself takes, and uses the
|
||||
engine's own parameter names and defaults. `force`, which decides whether the
|
||||
engine runs at all, is a parameter of `remove_all` and `remove_batch` alongside
|
||||
`backend` and `sensitivity`.
|
||||
|
||||
If AI metadata survives the strip, `remove_all` raises `MetadataStripIncomplete`
|
||||
**before** writing anything: an AI-readable output is worse than no output.
|
||||
|
||||
`remove_batch` runs one mode over a directory and never lets a single bad file
|
||||
end the run:
|
||||
|
||||
```python
|
||||
summary = raiw.remove_batch("in_dir", "out_dir", mode="visible") # -> BatchSummary
|
||||
print(summary.processed, summary.failed, summary.errors)
|
||||
print(summary.invisible_unavailable) # outputs that still carry the watermark
|
||||
```
|
||||
|
||||
`mode` is `all`, `visible`, `invisible`, or `metadata`. Pass a constructed
|
||||
`InvisibleEngine` as `engine` to load the model once for the whole directory.
|
||||
|
||||
## Inspect provenance
|
||||
|
||||
The default installation evaluates file metadata. Add `visible`, `detect`, or
|
||||
@@ -99,8 +167,34 @@ from remove_ai_watermarks.identify import identify
|
||||
report = identify(Path("input.png"))
|
||||
print(report.platform)
|
||||
print(report.signals)
|
||||
print(report.c2pa_validation)
|
||||
```
|
||||
|
||||
`c2pa_validation`, when present, reports `integrity`, `signature`,
|
||||
`signer_trust`, and `signer_validity` independently, plus the reader status
|
||||
codes. A valid hash and signature is a high-confidence signed claim; an
|
||||
unanchored or expired signer appears in `caveats` and in these fields, not as a
|
||||
lower confidence, because the reader ships no trust anchors to check against and
|
||||
`signer_trust` is therefore a missing input rather than a finding. A hash or
|
||||
signature failure, or a revoked signing credential, does not confirm the claimed
|
||||
platform or AI origin.
|
||||
|
||||
A consumer must read `integrity_clashes`. When a credential fails validation,
|
||||
`is_ai_generated` becomes `None`, because a claim that cannot be tied to these
|
||||
bytes cannot establish origin -- the manifest may have been transplanted from a
|
||||
real AI image onto anything -- and the failure is reported in
|
||||
`integrity_clashes` instead. That is a different question from whether an AI
|
||||
watermark is physically present in the pixels, which is what
|
||||
`has_invisible_target` answers, and it stays fail-safe `True` on the same file.
|
||||
Reading only `is_ai_generated` turns a broken vendor manifest into silence.
|
||||
|
||||
`c2pa_validation["state"]` is the reader's own aggregate and is carried for
|
||||
diagnostics only; no verdict is derived from it, because it collapses a
|
||||
transplanted manifest and a merely expired certificate into one `Invalid`, and
|
||||
its `Trusted` level depends on anchors no default installation has. Fallback parsing reports unknown validation
|
||||
dimensions, while a raw marker in an unsupported or malformed container can
|
||||
leave `c2pa_validation` as `None`.
|
||||
|
||||
Use `check_visible=False` and `check_invisible=False` for metadata-only
|
||||
inspection through the compatible path-based API:
|
||||
|
||||
@@ -126,8 +220,66 @@ evidence = extract_provenance_evidence(Path("input.png"))
|
||||
report = identify_from_evidence(evidence)
|
||||
```
|
||||
|
||||
If metadata was collected by another component, normalize its nested record
|
||||
without reopening the original file:
|
||||
### Collect once, judge elsewhere
|
||||
|
||||
`collect_metadata_record` splits the two halves apart: it is the only step that
|
||||
touches the file, and it returns a JSON-serializable record the verdict can be
|
||||
built from on another machine, in another process, or later.
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
from remove_ai_watermarks.identify import identify_metadata_record
|
||||
from remove_ai_watermarks.metadata_record import collect_metadata_record
|
||||
|
||||
record = collect_metadata_record(Path("input.png"), schema_version=1) # reads the file
|
||||
blob = json.dumps(record) # ship it anywhere
|
||||
|
||||
report = identify_metadata_record(json.loads(blob), path=Path("input.png")) # reads nothing
|
||||
payload = report.to_dict(schema_version=1) # versioned JSON contract
|
||||
```
|
||||
|
||||
The collection record has `record_type="provenance_metadata"`,
|
||||
`schema_version=1`, and a `status`. A vanished or unreadable source produces an
|
||||
`error` record with structured `issues`; `identify_metadata_record` rejects that
|
||||
record instead of turning a collection failure into an unknown-image verdict.
|
||||
Unknown schema versions, non-integer aliases, and native records without a
|
||||
`complete` collection status are rejected explicitly.
|
||||
|
||||
The verdict is the same one `identify(path, check_visible=False,
|
||||
check_invisible=False)` returns for that file. That equality is the record's whole
|
||||
contract and is verified over the tracked provenance fixtures and a separate local
|
||||
evaluation corpus. `ProvenanceReport.to_dict()` is the stable service boundary: it
|
||||
adds a `schema_version`, contains only JSON-safe values, and deliberately omits the
|
||||
local source path.
|
||||
|
||||
Package and transport versions evolve independently. Long-lived consumers should
|
||||
request the schema they implement, as above, instead of assuming the installed
|
||||
package's latest schema. Within schema 1, existing fields, types, meanings,
|
||||
`signals[].name` values, and `watermarks[]` labels remain compatible; releases may
|
||||
add fields that consumers must ignore. A breaking change requires a new schema while
|
||||
the schema 1 serializer remains available for rolling upgrades. Asking a release for
|
||||
an unsupported schema raises `ValueError` rather than silently returning another
|
||||
shape.
|
||||
|
||||
Microsoft InvisMark declarations emit both the compatible generic `soft_binding`
|
||||
signal and the additive `invismark` signal. Consumers should use `invismark` to route
|
||||
the image through pixel removal; the generic signal also covers content fingerprints
|
||||
that must not trigger regeneration.
|
||||
|
||||
A record carries metadata regions, not the primary coded-pixel stream: marker
|
||||
segments before the JPEG scan, every PNG chunk except `IDAT`, RIFF chunks except the
|
||||
coded image, the ISOBMFF provenance boxes, the container's trailer, the parsed EXIF tags the
|
||||
verdict reads by name, PIL's info mapping, and the C2PA manifest store. Record size
|
||||
is bounded by those metadata regions and trailers; images with large embedded
|
||||
manifests naturally produce larger records.
|
||||
|
||||
The `path` argument is metadata: it labels the report and is never opened by
|
||||
either function, so a record collected elsewhere can be judged against a path that
|
||||
does not exist locally.
|
||||
|
||||
If metadata was collected by another component instead, normalize its nested
|
||||
record the same way:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.identify import (
|
||||
@@ -143,16 +295,70 @@ evidence = evidence_from_metadata_record(record, path=Path("input.png"))
|
||||
report = identify_from_evidence(evidence)
|
||||
```
|
||||
|
||||
The normalizer recursively preserves text and byte values. It also decodes
|
||||
Unversioned third-party records are normalized recursively for compatibility. The
|
||||
normalizer preserves text and byte values. It also decodes
|
||||
strings prefixed with `hex:` and fields named `base64` or ending in
|
||||
`_base64`. Diagnostic values under `error` and `kind` are ignored because they
|
||||
describe the collector rather than the source file. Pass a C2PA manifest-store
|
||||
dictionary in `record["c2pa_store"]`, or through the explicit
|
||||
`_base64`. Diagnostic, transport, timing, hash, provenance-result, and pixel-result
|
||||
subtrees are ignored because they describe the collector or a derived result rather
|
||||
than the source file. A versioned portable record is stricter still: only
|
||||
`metadata_base64`, `tail_base64`, `pil`, `exif`, and `c2pa_store` are accepted as
|
||||
source evidence. Other `record_type` values are rejected, so do not pass a broad
|
||||
forensic inspection record to this API. Pass a C2PA manifest-store dictionary in
|
||||
`record["c2pa_store"]`, or through the explicit
|
||||
`c2pa_manifest_store` argument.
|
||||
|
||||
`identify_from_evidence` does not reopen the source file. It evaluates metadata
|
||||
only; registered visible marks and pixel-backed invisible watermarks remain in
|
||||
the path-based `identify` call.
|
||||
### Broad metadata inspection
|
||||
|
||||
`collect_forensic_metadata` provides the wide metadata-only record used by forensic
|
||||
inspection and migration adapters. It preserves hashes and timestamps, full EXIF and
|
||||
IPTC, C2PA, container inventories, bounded raw metadata payloads, and embedded
|
||||
thumbnail forensics. It does not calculate a provenance verdict or pixel statistics.
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.forensic_metadata import collect_forensic_metadata
|
||||
|
||||
record = collect_forensic_metadata(Path("input.png"), schema_version=1)
|
||||
assert record["record_type"] == "forensic_metadata"
|
||||
```
|
||||
|
||||
This record is intentionally not accepted by `identify_metadata_record`. Collect the
|
||||
small strict provenance record separately and publish the resulting
|
||||
`ProvenanceReport.to_dict()` as the detector contract.
|
||||
|
||||
### Pixel evidence
|
||||
|
||||
`extract_pixel_evidence` decodes once and calculates the DCT, FFT, residual, ELA,
|
||||
gradient, and color families. Its versioned `to_dict()` result has a semantic
|
||||
`status`: `complete`, `partial` when an individual family failed, or `error` when the
|
||||
source could not be decoded. Transported errors contain only the exception class, so
|
||||
local paths stay in the caller's logs rather than crossing the service boundary.
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.pixel_evidence import extract_pixel_evidence
|
||||
|
||||
pixels = extract_pixel_evidence(Path("input.png"), artifacts=False, timings=True)
|
||||
payload = pixels.to_dict(schema_version=1)
|
||||
```
|
||||
|
||||
Timings and spatial artifacts are opt-in. Artifacts include image-identifying data
|
||||
such as a thumbnail and perceptual hash; aggregate feature families do not.
|
||||
|
||||
`identify_from_evidence` does not reopen the source file by default: it evaluates
|
||||
metadata only, and registered visible marks and pixel-backed invisible watermarks
|
||||
remain in the path-based `identify` call.
|
||||
|
||||
Pass `image_path` together with `check_visible` or `check_invisible` to add those
|
||||
pixel detectors on top of the SAME evidence. That is how a caller asking one file
|
||||
two provenance questions — which vendor is confirmed, and is there an invisible
|
||||
target — pays for the metadata extraction once:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.identify import extract_provenance_evidence, identify_from_evidence
|
||||
|
||||
evidence = extract_provenance_evidence(source)
|
||||
metadata_only = identify_from_evidence(evidence)
|
||||
with_pixels = identify_from_evidence(evidence, image_path=source, check_invisible=True)
|
||||
```
|
||||
|
||||
## Strip metadata
|
||||
|
||||
@@ -259,7 +465,8 @@ must use the same container extension as the source.
|
||||
The returned `VideoMetadataResult` records the source, output, metadata detected
|
||||
before removal, and any markers remaining after the verified strip. MP4/MOV
|
||||
inspection recognizes the native TC260 `AIGC` entry in
|
||||
`moov.udta.meta.keys/ilst`; its removal preserves container size and encoded
|
||||
`moov.udta.meta.keys/ilst` and the QuickTime-form `meta` variants Doubao's iOS
|
||||
export writes; its removal preserves container size and encoded
|
||||
stream bytes. MP4/MOV/M4V are copied in bounded chunks, so a large `mdat` is not
|
||||
loaded into memory; publication is atomic. MKV/WebM inspection recognizes the corresponding
|
||||
`Segment.Tags.Tag.SimpleTag` representation; its removal requires ffmpeg for a
|
||||
@@ -356,8 +563,8 @@ independently and are not compared across detectors. Pass one of those explicit
|
||||
values to restrict the scan to a single provider. The Veo detector recognizes
|
||||
the current four-point diamond and the
|
||||
legacy `Veo` text. Seedance recognizes the boxed `AI` label, Dola recognizes
|
||||
its compact text label, Hailuo recognizes the composite MINIMAX/Hailuo label,
|
||||
and Kling recognizes its bottom-right logo, wordmark, and version suffix. Each
|
||||
its compact text label, Hailuo AI recognizes the composite MINIMAX/Hailuo AI label,
|
||||
and Kling AI recognizes its bottom-right logo, wordmark, and version suffix. Each
|
||||
variant has an independent synthetic silhouette and calibrated temporal policy.
|
||||
After each accepted frame is filled, `temporal_consistency=True` motion-aligns
|
||||
the preceding accepted fill and blends it only when the warped prior mask
|
||||
@@ -380,8 +587,8 @@ to 8-bit SDR.
|
||||
|
||||
## Remove invisible watermarks
|
||||
|
||||
Install `remove-ai-watermarks[diffusion]` for the standard pipelines or
|
||||
`remove-ai-watermarks[qwen-zimage]` for the CUDA-only high-fidelity profile.
|
||||
Install `remove-ai-watermarks[qwen-zimage]`. All three profiles need it, and all
|
||||
need an NVIDIA GPU.
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
@@ -389,7 +596,7 @@ from pathlib import Path
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine
|
||||
|
||||
engine = InvisibleEngine(
|
||||
pipeline="controlnet",
|
||||
pipeline="qwen-zimage", # the default; also "sdxl-zimage" or "chroma-zimage"
|
||||
device=None,
|
||||
cpu_offload=False,
|
||||
)
|
||||
@@ -400,29 +607,87 @@ engine.remove_watermark(
|
||||
)
|
||||
```
|
||||
|
||||
`device=None` selects the device automatically. Supported explicit values are
|
||||
defined by the CLI and runtime device resolver.
|
||||
`device=None` and `device="auto"` both run detection. `"cuda"` pins it without
|
||||
detecting. Every other value raises at construction rather than deferring a
|
||||
guaranteed failure to model-load time.
|
||||
|
||||
For limited CUDA memory:
|
||||
|
||||
```python
|
||||
engine = InvisibleEngine(
|
||||
pipeline="controlnet",
|
||||
pipeline="qwen-zimage",
|
||||
cpu_offload=True,
|
||||
)
|
||||
```
|
||||
|
||||
For the CUDA only high fidelity profile:
|
||||
All profiles are CUDA-only, so on a machine without an NVIDIA GPU `device=None`
|
||||
resolves to `cpu` and construction raises. For the SDXL global stage instead of
|
||||
Qwen:
|
||||
|
||||
```python
|
||||
engine = InvisibleEngine(pipeline="qwen-zimage")
|
||||
engine = InvisibleEngine(pipeline="sdxl-zimage")
|
||||
```
|
||||
|
||||
The `qwen-zimage` extra must be installed for that profile.
|
||||
The `qwen-zimage` extra is required for all three profiles: each runs the same
|
||||
DiffSynth Z-Image face stage.
|
||||
|
||||
The full `remove_watermark` signature includes strength, steps, guidance,
|
||||
seeding, tiling, resolution, upscaling, and postprocessing controls. Read the
|
||||
method signature in
|
||||
The opt-in verified-text stage uses the same `text_manifest` argument as the CLI:
|
||||
|
||||
```python
|
||||
engine.remove_watermark(
|
||||
Path("watermarked.png"),
|
||||
Path("clean.png"),
|
||||
text_manifest=Path("verified-lines.json"),
|
||||
)
|
||||
```
|
||||
|
||||
Install `remove-ai-watermarks[text-restoration]`. The manifest schema and safety
|
||||
constraints are documented in the CLI guide. The engine verifies its decoded RGB
|
||||
hash before loading the diffusion models and rejects SDXL, downscaling, and
|
||||
postprocessing combinations that were not evaluated. Tiling is also rejected because
|
||||
the combined tiled-restoration path has no provider-oracle calibration. `InvisibleOptions`
|
||||
exposes the same field for `remove_all`; after a visible-stage edit, the manifest must
|
||||
be built against the staged pixels rather than the pristine source.
|
||||
|
||||
Use manifest schema 1 for manually reviewed text plus script metadata. Automated
|
||||
operators that verify only text-region geometry should emit schema 2 lines with a
|
||||
`box` and optional `angle`; no placeholder transcription or script is required.
|
||||
|
||||
Since 0.27.1 the mode's global 15% Qwen-VAE fidelity-anchor blend is **off by
|
||||
default** (`fidelity_anchor=False`): that whole-frame blend was measured to
|
||||
return detector-visible OpenAI SynthID on poster-scale manifests (official
|
||||
Content Provenance API, 2026-08-19 - detected x6 with the anchor, clean x6
|
||||
without it, controls and base outputs validated in the same sessions). Pass
|
||||
`fidelity_anchor=True` to reproduce the 0.27.0 research behavior.
|
||||
|
||||
### Drafting manifest lines
|
||||
|
||||
`remove_ai_watermarks.text_draft` proposes lines for a manifest; it never
|
||||
produces verified ones:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.text_draft import draft_text_lines
|
||||
|
||||
draft = draft_text_lines(Path("watermarked.png"))
|
||||
for line in draft.accepted:
|
||||
print(line.box, line.script, line.min_score, line.text)
|
||||
```
|
||||
|
||||
Install `remove-ai-watermarks[text-draft]` (CPU, no torch: PaddleOCR detection
|
||||
plus three script-chosen recognition engines). A line lands in `accepted` only
|
||||
when three crop paddings normalize identically and every confidence clears
|
||||
`min_score` (default 0.85); `accepted` means crop-stable, NOT ground-truth-
|
||||
correct - on the reference posters the draft's exact-text precision was 90.0%
|
||||
and 94.4% because high-confidence OCR still lost punctuation. Every accepted
|
||||
line needs a human yes/no before a manifest may claim `verified: true`.
|
||||
`source_pixel_sha256` is re-exported here for building the manifest's
|
||||
pixel-binding hash against the exact source the engine will decode.
|
||||
|
||||
`remove_watermark` takes strength, seed, tiling, resolution, and postprocessing
|
||||
controls. It takes no model id, step count or guidance scale, and neither does the
|
||||
constructor: each profile pins its model stack, its per-stage schedule and CFG
|
||||
1.0, so passing one raises `TypeError` at the call rather than being accepted and
|
||||
refused several layers down. Read the method signature in
|
||||
[`invisible_engine.py`](../src/remove_ai_watermarks/invisible_engine.py) or use
|
||||
the CLI guide for the concepts.
|
||||
Defaults can differ between the Python method and CLI profile resolution, so
|
||||
|
||||
@@ -3,6 +3,21 @@
|
||||
This page describes the release behavior defined in this repository. External
|
||||
registry state can change independently, so verify it during a release.
|
||||
|
||||
## Published surfaces
|
||||
|
||||
A release is complete only after all four published surfaces are verified:
|
||||
|
||||
| Surface | Published result | Automation |
|
||||
|---|---|---|
|
||||
| PyPI | The `remove-ai-watermarks` wheel and source distribution | `publish.yml` |
|
||||
| Homebrew | The `remove-ai-watermarks` formula in `wiltodelta/homebrew-tap` | `distribute.yml` |
|
||||
| Hugging Face Space | The Space demo deployed on the new release through a requirements-pin bump in `wiltodelta/raiw-hf-space` | `raiw-hf-space` sync workflow; the pin bump is manual, `distribute.yml` only re-installs the pinned version |
|
||||
| ComfyUI Registry | A compatible release of `wiltodelta/ComfyUI-remove-ai-watermarks` with its own node version | `distribute.yml` and the node repository's workflows |
|
||||
|
||||
The GitHub Release is the trigger and release record for this flow. Conda is
|
||||
not a supported publishing surface: this repository has no conda recipe or
|
||||
conda publication job.
|
||||
|
||||
## Release sources of truth
|
||||
|
||||
The package version appears in:
|
||||
@@ -14,6 +29,10 @@ The package version appears in:
|
||||
Update the first two, then refresh the lock file with uv. Do not edit a
|
||||
line-number-specific location in `uv.lock`; its package order changes.
|
||||
|
||||
Before choosing a version, verify it is absent from git tags, GitHub Releases,
|
||||
and PyPI. An unpublished remote tag still reserves that version; never move it,
|
||||
publish the next version instead.
|
||||
|
||||
## Publish flow
|
||||
|
||||
PyPI publishing is triggered by a published GitHub Release, not by a tag push
|
||||
@@ -43,25 +62,23 @@ PyPI API token from the repository.
|
||||
waits for the matching source distribution to appear on PyPI, then:
|
||||
|
||||
- updates the Homebrew tap formula URL and SHA-256;
|
||||
- updates the repository conda recipe version and source-distribution SHA-256;
|
||||
- triggers a factory rebuild of the Hugging Face Space.
|
||||
- triggers a factory rebuild of the Hugging Face Space: this re-installs the
|
||||
version pinned in the Space repo, it does NOT upgrade the demo;
|
||||
- synchronizes, tests, versions, and publishes the ComfyUI nodes.
|
||||
|
||||
Upgrading the Hugging Face Space is a separate, manual step: bump
|
||||
`remove-ai-watermarks[visible,heif]` in `wiltodelta/raiw-hf-space` (`pyproject.toml`,
|
||||
`uv lock`, and the re-exported `requirements.txt`), then push. That repository's
|
||||
`sync-to-hf.yml` mirrors the files onto the Space, which rebuilds on the new
|
||||
pin. Its callback smoke tests run in that repository's CI on every push; they
|
||||
are the scenario check for the demo surface.
|
||||
|
||||
The workflow can also be started manually with an optional version input.
|
||||
Conda-forge updates are outside this workflow.
|
||||
|
||||
If a distribution job fails because a repository or Hugging Face credential is
|
||||
invalid, rotate the corresponding GitHub secret and rerun the failed job. A
|
||||
manual Homebrew formula update is the fallback when its automation is blocked.
|
||||
|
||||
The conda job uses the published artifact rather than a locally built archive
|
||||
as the hash source and commits the resulting recipe change to `main`. Runtime
|
||||
dependency mapping remains review-controlled: keep it aligned with the default
|
||||
metadata dependencies in `pyproject.toml`, do not copy optional pixel extras
|
||||
into the default recipe, and document any conda-forge package that is
|
||||
unavailable and must be omitted.
|
||||
The optional `video` extra carries PyAV with Python-version-specific bounds; it
|
||||
does not belong in the default metadata-focused conda recipe.
|
||||
|
||||
## Source distribution boundary
|
||||
|
||||
The wheel includes the package under `src/`.
|
||||
@@ -71,9 +88,10 @@ The source distribution uses an explicit allowlist for `/src`, `/LICENSE`,
|
||||
`[tool.hatch.build.targets.sdist]` in `pyproject.toml`. It also defensively
|
||||
excludes `/data`, `/tmp`, and `/.sc`. Keep both controls: calibration captures,
|
||||
test corpora, generated research outputs, and local session state do not belong
|
||||
in the published package archive. The matching local-root entries in
|
||||
`.gitignore` prevent accidental commits, but are not a substitute for the build
|
||||
boundary because hatchling may include untracked files.
|
||||
in the published package archive. Hatchling always adds the root `.gitignore` to
|
||||
the sdist, so keep its comments generic and free of local operational context.
|
||||
Ignore rules are not the build boundary: `data/` is deliberately tracked, while
|
||||
the sdist configuration keeps it and the other excluded paths out of the archive.
|
||||
|
||||
## Build backend
|
||||
|
||||
@@ -82,8 +100,6 @@ The package uses hatchling through the unpinned `hatchling` build requirement in
|
||||
|
||||
## Other channels
|
||||
|
||||
The repository includes a conda recipe under `packaging/conda/recipe.yaml`.
|
||||
|
||||
The ComfyUI nodes are maintained and versioned in their own repository. After
|
||||
the matching source distribution appears on PyPI, `distribute.yml` dispatches
|
||||
that repository's sync workflow with the exact library version and waits for it
|
||||
@@ -96,13 +112,36 @@ write access.
|
||||
|
||||
## Release verification
|
||||
|
||||
Forensic transports are versioned independently from the package. Before publishing
|
||||
a change to provenance metadata, provenance reports, broad forensic metadata, or
|
||||
pixel evidence, run their schema 1 contract tests. Additive fields are compatible;
|
||||
renaming a field, changing its type or meaning, changing a signal name or watermark
|
||||
label, or removing a field requires a new output schema. Add the new serializer
|
||||
without removing schema 1 so long-lived consumers can update separately. A package
|
||||
release must never silently substitute its latest schema when a caller explicitly
|
||||
requests an older supported one.
|
||||
|
||||
After publication, verify:
|
||||
|
||||
- both wheel and source distribution exist on PyPI;
|
||||
- the package version matches the tag;
|
||||
- the Homebrew formula points to the new source distribution;
|
||||
- the distribution workflow completed successfully;
|
||||
- the repository's conda recipe matches the published version and source
|
||||
distribution;
|
||||
- the ComfyUI Registry node requires the new library version;
|
||||
- the Hugging Face Space serves the new release: its `requirements.txt` pin
|
||||
matches (raw file via the HF API), the runtime stage is `RUNNING` after the
|
||||
rebuild, and one live `identify` and one live `visible` API call succeed
|
||||
against the running Space;
|
||||
- a clean install can run `remove-ai-watermarks --version`.
|
||||
|
||||
A clean-install check run immediately after publication can fail with "no
|
||||
version of remove-ai-watermarks==X.Y.Z" even though the simple index already
|
||||
lists it: uv serves its cached pre-release view of the index. Re-run with a
|
||||
fresh cache (`UV_CACHE_DIR=$(mktemp -d)`) before treating it as a release
|
||||
failure.
|
||||
|
||||
The ComfyUI sync run can fail the same way on its own: it resolves the version
|
||||
from the PyPI JSON API (which updates first) but installs the test dependency
|
||||
with pip against the simple index, whose CDN edges lag by minutes. Rerun the
|
||||
failed `distribute.yml` comfyui job once the simple index lists the release;
|
||||
nothing about the release itself is wrong.
|
||||
@@ -10,7 +10,7 @@ The `visible` command registers these mark keys:
|
||||
|
||||
| Key | Mark | Expected area | Important limit |
|
||||
| --- | --- | --- | --- |
|
||||
| `gemini` | Google Gemini sparkle | Usually bottom right | Detection includes a false positive gate. |
|
||||
| `gemini` | Google Gemini visible watermark (sparkle) | Usually bottom right | Detection includes a false positive gate. |
|
||||
| `doubao` | `豆包AI生成` | Bottom right | Vendor specific text detector. |
|
||||
| `jimeng` | `★ 即梦AI` | Bottom right | Vendor specific text detector. |
|
||||
| `qwen` | `千问AI生成` | Bottom right | Strict visual gate. |
|
||||
@@ -19,7 +19,8 @@ The `visible` command registers these mark keys:
|
||||
| `samsung` | `✦ Contenuti generati dall'AI` | Bottom left | Calibrated for the Italian text variant. |
|
||||
| `runninghub` | `RunningHub AI生成` | Top left | Strict visual and position gates. |
|
||||
| `baidu` | `百度 AI生成` | Bottom right | Detector and extended removal footprint. |
|
||||
| `liblib` | `LibLibAI` | Bottom center | Includes a minimum image size gate. |
|
||||
| `liblib` | `LiblibAI` | Bottom center | Includes a minimum image size gate. |
|
||||
| `microsoft` | One Microsoft white AI-badge variant | Top right | Strict uses the visual gate; auto can use Microsoft provenance for the measured [relaxed gate](module-internals.md#visible-mark-removal). Other documented icon, text, and position variants are not covered. |
|
||||
| `jimeng_pill` | `AI生成` pill | Top left | Weak detector with additional product and background gates. |
|
||||
|
||||
`--mark auto` evaluates all registered marks and removes every selected match.
|
||||
@@ -29,16 +30,21 @@ the masked area.
|
||||
Marks from other vendors are not detected automatically. Use `erase --region`
|
||||
when you can select the affected area yourself.
|
||||
|
||||
Synthetic examples for every registered image and video mark live in the
|
||||
[visible-mark gallery](../data/fixtures/visible/README.md). They demonstrate the
|
||||
detectors' canonical geometry and house style, not vendor raster fidelity.
|
||||
|
||||
### Visible video marks
|
||||
|
||||
| Key | Mark | Motion | Important limit |
|
||||
| --- | --- | --- | --- |
|
||||
| `sora` | Sora 2 mascot and wordmark | Moves among frame positions | Requires a temporally recurring visual match; the older Sora Turbo corner swirl is a different unsupported mark. |
|
||||
| `veo` | Current four-point diamond and legacy `Veo` text | Fixed bottom-right corner | Uses separate silhouettes and requires a recurring match; learned fill is preferable on structured backgrounds. |
|
||||
| `doubao` | `豆包AI生成` text run | Fixed bottom-right corner | Reuses the image engine's synthetic alpha as the template; a stable 12-frame run above 0.35 is required. |
|
||||
| `seedance` | Boxed `AI` label | Fixed bottom-right corner | Requires an anchored recurring match; the full localized box is filled because a thinner synthetic shape mask leaves the real translucent rim behind. |
|
||||
| `dola` | `Dola AI` text | Fixed bottom-right corner | Requires an anchored recurring match; ByteDance or BytePlus provenance can relax only an existing visual run. |
|
||||
| `hailuo` | `MINIMAX | hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. |
|
||||
| `kling` | Kling swirl, `KLING AI`, version, and optional `PRO` suffix | Fixed bottom-right edge | Combines a synthetic logo rescue with font variants, an edge gate, a white-label gate, and anchored temporal recurrence. |
|
||||
| `hailuo` | `MINIMAX \| hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. A TC260 label naming MiniMax as producer can relax only an existing stable run. |
|
||||
| `kling` | Kling AI swirl, `KLING AI`, version, and optional `PRO` suffix | Fixed bottom-right edge | Combines a synthetic logo rescue with font variants, an edge gate, a white-label gate, and anchored temporal recurrence. |
|
||||
|
||||
`video identify`, `video visible`, and `video all` share this registry and the
|
||||
same temporal arbiter. It is separate from the image registry because selection
|
||||
@@ -67,6 +73,9 @@ The inspection and stripping code handles signals in these groups:
|
||||
|
||||
- C2PA Content Credentials and supported cloud manifest references;
|
||||
- EXIF and XMP generator fields;
|
||||
- exact app-export provenance and AIGC disclosures from supported
|
||||
ByteDance-family products, with product-only provenance excluded from the
|
||||
generated-image verdict;
|
||||
- IPTC AI disclosure fields;
|
||||
- PNG text chunks and embedded generation parameters;
|
||||
- China TC260 AIGC labels in supported image placements and the normative
|
||||
@@ -77,11 +86,19 @@ The inspection and stripping code handles signals in these groups:
|
||||
- Samsung AI editing markers;
|
||||
- Hugging Face job metadata;
|
||||
- open Stable Diffusion style DWT-DCT watermarks with the `detect` extra;
|
||||
- Adobe TrustMark with the `trustmark` extra.
|
||||
- Adobe TrustMark Variant P schemas 0-2 with the `trustmark` extra. Variant Q
|
||||
needs a different model, while schema 3 is deliberately rejected because it
|
||||
produced persistent false positives on unrelated generators.
|
||||
|
||||
`identify` combines detected signals into a `ProvenanceReport`. It reports
|
||||
unknown when evidence is absent. It never treats missing metadata as proof that
|
||||
an image is human made.
|
||||
an image is human made. C2PA presence alone is not a verified identity: the
|
||||
report distinguishes asset binding, claim signature, signer trust, and signer
|
||||
validity. High-confidence C2PA attribution requires an intact asset binding and
|
||||
claim signature; signer trust and certificate expiry are reported as their own
|
||||
dimensions and as caveats, since no trust anchor list ships to evaluate them.
|
||||
Fallback claims, which validate nothing, remain medium-confidence, while a failed
|
||||
binding or signature or a revoked credential contributes no origin verdict.
|
||||
|
||||
## File and container formats
|
||||
|
||||
@@ -114,17 +131,74 @@ The `invisible` command uses diffusion regeneration. It targets watermark
|
||||
patterns by changing the image rather than decoding and deleting a known
|
||||
payload.
|
||||
|
||||
Current pipeline values:
|
||||
Current pipeline values, both CUDA-only:
|
||||
|
||||
- `controlnet`;
|
||||
- `sdxl`;
|
||||
- `qwen`;
|
||||
- `qwen-zimage`;
|
||||
- legacy alias `default`, which resolves to `sdxl`.
|
||||
- `qwen-zimage`, the default;
|
||||
- `sdxl-zimage`, the same recipe and the same face stage on an SDXL global pass.
|
||||
|
||||
SynthID does not have a public local pixel decoder in this project. The tool can
|
||||
infer likely presence from supported provenance metadata, but after that
|
||||
metadata is removed a local negative result is inconclusive.
|
||||
The `controlnet`, `sdxl`, `qwen` and `default` values were removed. A retired name
|
||||
is rejected at parse time rather than remapped onto a surviving profile.
|
||||
|
||||
SynthID does not have a public local pixel decoder in this project. The tool
|
||||
recognizes presence from supported provenance: Google AI C2PA under Google's
|
||||
all-media watermark policy, and current OpenAI C2PA carrying an explicit
|
||||
`c2pa.watermarked.*` action. Legacy OpenAI C2PA without that action does not
|
||||
assert SynthID. After provenance metadata is removed, a local negative result
|
||||
is still inconclusive.
|
||||
|
||||
Microsoft Paint can name `com.microsoft.invismark.1` in a C2PA soft-binding
|
||||
assertion. Inspection reports both that exact algorithm and its signed `value`,
|
||||
which Paint uses as the identifier carried by the pixel watermark, and emits an
|
||||
additive `invismark` signal so callers can select pixel removal without parsing
|
||||
the generic `soft_binding` detail. Photos uses a parallel local writer path.
|
||||
Metadata stripping removes only the embedded manifest; `invisible` and `all`
|
||||
guarantee the supported InvisMark removal contract by regenerating the pixel
|
||||
layer as well. The project has no validated local InvisMark decoder, so local
|
||||
inspection cannot independently verify the output. Microsoft's official
|
||||
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
|
||||
is the external oracle: it reports pixel `Watermark` and embedded `C2PA` results
|
||||
separately; a control-positive, output-negative pair is the available per-file
|
||||
verification path. The API needs Azure credentials; the page a human can check
|
||||
without an account is <https://ai.azure.com/nextgen/validate>. Its verdict is
|
||||
weaker than the API's: it collapses watermark and C2PA into one rendered result
|
||||
and tops out at `Inconclusive` rather than a watermark-negative, so treat
|
||||
`Inconclusive` on a processed file as "not confirmed", not as "detected still".
|
||||
|
||||
Meta Muse Image stamps every output with Content Seal, a proprietary invisible
|
||||
pixel watermark, and ships no visible mark (the legacy `Imagined with AI`
|
||||
corner mark belongs to the pre-2026 Imagine pipeline and is not registered).
|
||||
This project has no local Content Seal decoder. Meta Model API outputs and
|
||||
Meta CDN copies carry an XMP `iptcExt:DigitalSourceType =
|
||||
trainedAlgorithmicMedia` companion tag, which `identify` reports through the
|
||||
existing Made-with-AI path; that IPTC code is a standard, not a Meta-exclusive
|
||||
signal, so it cannot key a strength cohort the way the C2PA issuer does. Since
|
||||
0.33.0 a standalone AI digital-source tag (no C2PA manifest) additionally emits
|
||||
the additive `content_seal` signal - the strength router's Meta bet as evidence,
|
||||
medium confidence, with the same caveat - so clients select pixel removal from
|
||||
the signal list exactly the way InvisMark is additive over `soft_binding`. It
|
||||
is an attribution, never a decode. The
|
||||
external oracle is `https://meta.ai/identification`: anonymous, no login,
|
||||
accepts image, video, and audio, enforces an unspecified daily identification
|
||||
limit, and answers with model attribution (`Muse Image 1 - Meta`) plus a
|
||||
per-generation ID and creation timestamp read from the watermark payload. No
|
||||
identification endpoint exists in the Meta Model API itself (the reference at
|
||||
dev.meta.ai documents only generation and edits for images), and the official
|
||||
documentation never mentions the seal. The
|
||||
default `qwen-zimage` profile clears Content Seal at the default
|
||||
resolution-adaptive strength (oracle-verified on 2.56 MP generations); measured
|
||||
strength boundaries are recorded in `data/contentseal/manifest.csv` and
|
||||
[module internals](module-internals.md#meta-content-seal-boundaries-for-qwen-zimage).
|
||||
The derived Meta floor (0.1 by the standard spread method) ships as a measured
|
||||
cohort: auto mode routes a file whose only provenance is the standalone AI IPTC
|
||||
tag to it (the tag is not Meta-exclusive; other tag users ship no invisible
|
||||
watermark this profile targets, and Google/OpenAI/Microsoft C2PA evidence
|
||||
always wins first), and `invisible --vendor meta` (also `all` and `batch`, and
|
||||
`InvisibleOptions.vendor` in the API) names the cohort explicitly on stripped
|
||||
files. An explicit vendor implies the scrub runs: naming the cohort asserts the
|
||||
pixel watermark is present.
|
||||
The seal survives resizing, JPEG recompression, and metadata stripping; it dies
|
||||
to center crops of a third to a half, matching the Reuters 2026-07-11 finding
|
||||
that Meta's detector missed 55% of cropped Muse images.
|
||||
|
||||
For MP4, MOV, and M4V, `video invisible` or the explicit
|
||||
`video all --invisible` option can regenerate the video through a VAE and strip
|
||||
@@ -146,16 +220,20 @@ not a universal clean verdict.
|
||||
| Google Gemini | Sparkle | Diffusion regeneration for SynthID | C2PA and related source signals |
|
||||
| Google Veo video | Veo diamond and legacy text | Oracle-certified VAE removal for SynthID | C2PA and related source signals |
|
||||
| OpenAI image generators | None registered | Diffusion regeneration for supported invisible signals | C2PA and generator provenance |
|
||||
| Meta Muse Image | None on Muse output (legacy `Imagined with AI` unregistered) | Diffusion regeneration for Content Seal, oracle-verified on the default profile | XMP IPTC `trainedAlgorithmicMedia` companion tag; no local Content Seal decoder |
|
||||
| Microsoft Paint and Photos | None registered | External Microsoft oracle for InvisMark; no validated local decoder | Paint C2PA soft-binding algorithm and identifier |
|
||||
| Microsoft image outputs (measured variant) | One top-right white AI-badge variant | No registered pixel decoder | C2PA attribution |
|
||||
| Stable Diffusion and SDXL | None registered | Diffusion regeneration; optional open decoder | Embedded parameters and text metadata |
|
||||
| FLUX | None registered | Diffusion regeneration; optional open decoder | C2PA for supported sources |
|
||||
| Adobe Firefly | None registered | No proprietary local decoder | C2PA; optional TrustMark decoder |
|
||||
| Adobe Firefly | None registered | Optional TrustMark Variant P decoder | C2PA |
|
||||
| Midjourney | None registered | No registered pixel decoder | EXIF, XMP, and IPTC signals |
|
||||
| ByteDance generators | Doubao and Jimeng marks | No registered pixel decoder | TC260 AIGC and supported C2PA signals |
|
||||
| Luma AI | None registered | No registered pixel decoder | PNG text generator tags (Uni-1) |
|
||||
| ByteDance generators | Doubao and Jimeng marks | No registered pixel decoder | TC260 AIGC, supported C2PA, and exact app-export AIGC disclosures |
|
||||
| Qwen | Qwen mark | No registered pixel decoder | TC260 AIGC |
|
||||
| Kling | Kling image and video marks | No registered pixel decoder | TC260 AIGC |
|
||||
| Hailuo / MiniMax video | Hailuo composite video label | No registered pixel decoder | TC260 AIGC where present |
|
||||
| Kling AI | Kling AI image and video marks | No registered pixel decoder | TC260 AIGC |
|
||||
| Hailuo AI / MiniMax video | Hailuo AI composite video label | No registered pixel decoder | TC260 AIGC where present |
|
||||
| Baidu | Baidu mark | No registered pixel decoder | TC260 AIGC |
|
||||
| LibLibAI | LibLibAI mark | No registered pixel decoder | TC260 AIGC |
|
||||
| LiblibAI | LiblibAI mark | No registered pixel decoder | TC260 AIGC |
|
||||
| RunningHub | RunningHub mark | No registered pixel decoder | TC260 AIGC |
|
||||
| Samsung Galaxy AI | One locale specific mark | No registered pixel decoder | C2PA and Samsung markers |
|
||||
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# Deep research: SynthID-safe face-identity recovery for SDXL (2026-06-08)
|
||||
|
||||
> Research archive. This dated follow-up records evidence available during the
|
||||
> study. It is not the current command reference.
|
||||
> study. It is not the current command reference. The `instantid` and `photomaker`
|
||||
> extras it discusses, the `--restore-faces` flag, and their modules were all removed
|
||||
> from the library afterwards.
|
||||
|
||||
**Stats:** {"angles": 6, "sourcesFetched": 28, "claimsExtracted": 104, "claimsVerified": 25, "confirmed": 19, "killed": 6, "afterSynthesis": 6, "urlDupes": 1, "budgetDropped": 7, "agentCalls": 111}
|
||||
|
||||
@@ -149,8 +151,9 @@ on Modal A100 in two phases:
|
||||
- v3 (tighter ellipse 0.32*bw x 0.42*bh + per-channel mean color match to
|
||||
local cleaned canvas + softer feather): patchwork visually softened; faces
|
||||
still read as studio portraits inserted into the scene, not as people
|
||||
shot in the scene. Single portrait identity drifted (tatsunari -> "round
|
||||
Asian male" vs original's thin face).
|
||||
shot in the scene. Single portrait identity drifted (the single-portrait
|
||||
fixture regenerated as a generic "round Asian male" vs the original's thin
|
||||
face).
|
||||
2. **Phase 2: InstantID img2img on cleaned crop.** Switched to the upstream
|
||||
`pipeline_stable_diffusion_xl_instantid_img2img` (downloaded at first use
|
||||
from raw.githubusercontent.com; requires `trust_remote_code=True`). Same
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
> Research archive. This document evaluates approaches considered during the
|
||||
> study. It does not describe the current public interface. The current
|
||||
> pipelines and limits are documented in `docs/cli.md` and
|
||||
> `docs/known-limitations.md`.
|
||||
> `docs/known-limitations.md`. The path it calls shipped -- `photomaker_restore.py`
|
||||
> and the `photomaker` extra -- was removed from the library afterwards.
|
||||
|
||||
> **Status (2026-06-08): retired.** Every approach described below was empirically
|
||||
> tested and rejected -- see `docs/synthid-robust-identity-research-2026-06-08.md`
|
||||
@@ -246,7 +247,9 @@ from the test set + this doc).
|
||||
- If no -> the assumption is wrong; PhotoMaker would re-introduce the
|
||||
watermark. Stop and reconsider.
|
||||
2. **PhotoMaker-V1 prototype** in the existing `controlnet` pipeline:
|
||||
- Mirror the `_load_controlnet_pipeline` path: add a PhotoMaker variant that
|
||||
- Mirror the `_load_controlnet_pipeline` path (removed in 0.24.0 with the
|
||||
controlnet profile; the equivalent seam is now `SdxlZImagePipeline._load_global`):
|
||||
add a PhotoMaker variant that
|
||||
loads SDXL + canny ControlNet + PhotoMaker adapter on the same engine.
|
||||
- Extract the OpenCLIP face embedding from the watermarked face crops (use
|
||||
OpenCV YuNet, already bundled for `auto`, to find the face boxes).
|
||||
|
||||
@@ -289,14 +289,19 @@ The external variant SynthID-O is available "through partnerships" only. Our
|
||||
tool cannot locally detect SynthID presence or absence -- this is by design,
|
||||
not a gap we can fill.
|
||||
|
||||
### 3.2 How our tool detects SynthID (metadata proxy)
|
||||
### 3.2 How our tool recognizes SynthID from provenance
|
||||
|
||||
We detect SynthID indirectly: if the image's C2PA manifest is signed by a
|
||||
known SynthID-using issuer (Google, OpenAI), we infer SynthID is present. This
|
||||
is a **metadata proxy**, not a pixel watermark decode. It works while the C2PA
|
||||
manifest is intact, and is silent once the manifest is stripped or the image
|
||||
is re-encoded without C2PA (e.g., a screenshot, a social-media re-upload, or
|
||||
after `metadata --remove`).
|
||||
We recognize SynthID indirectly from supported C2PA evidence; this is not a
|
||||
pixel watermark decode. Google states that all media generated by its tools is
|
||||
watermarked, so Google AI C2PA establishes SynthID. OpenAI C2PA predates its
|
||||
SynthID rollout, but current manifests add the signed
|
||||
`c2pa.watermarked.unbound` action; OpenAI provenance establishes SynthID only
|
||||
when that action is present. Legacy OpenAI C2PA without it remains valid origin
|
||||
evidence but does not assert a pixel watermark.
|
||||
|
||||
This works while the C2PA manifest is intact and is silent once the manifest is
|
||||
stripped or the image is re-encoded without C2PA (e.g., a screenshot, a
|
||||
social-media re-upload, or after `metadata --remove`).
|
||||
|
||||
This is why:
|
||||
- `identify` on a GitHub-recompressed issue attachment returns Unknown (C2PA is
|
||||
@@ -372,6 +377,11 @@ itself. `0.15` is therefore the shipped default. The tracked manifest
|
||||
`data/evaluations/video-synthid-oracle.csv` records the public source URL,
|
||||
hashes, fidelity metrics, and verdicts without committing generated videos.
|
||||
|
||||
What that calibration does and does not constrain, the ranked experiment program
|
||||
for trading less quality for the same removal, and the change candidates that were
|
||||
refuted along the way are recorded in
|
||||
[`video-synthid-quality-research.md`](video-synthid-quality-research.md).
|
||||
|
||||
The VAE perturbation follows the general regeneration-attack construction from
|
||||
Zhao et al. The video-specific control and temporal metric are local additions.
|
||||
VideoMarkBench motivates testing frame aggregation and matched perturbations,
|
||||
@@ -404,8 +414,9 @@ OpenAI confirmed SynthID adoption (Help Center, updated 2026-05-21):
|
||||
> "ChatGPT images include both C2PA metadata and SynthID watermarks."
|
||||
|
||||
This is time-gated: pre-rollout ChatGPT/gpt-image images carry C2PA without
|
||||
SynthID. Our C2PA proxy therefore over-reports SynthID presence on old images
|
||||
(hence the `_OPENAI_CAVEAT` hedging flag in the codebase).
|
||||
SynthID. Current OpenAI manifests distinguish the watermarked output with the
|
||||
signed `c2pa.watermarked.unbound` action. The detector requires that action, so
|
||||
old OpenAI C2PA remains an origin signal without becoming a SynthID claim.
|
||||
|
||||
### 4.3 Other vendors
|
||||
|
||||
@@ -448,6 +459,14 @@ the payload, reconstituting SynthID in text. The lesson held and shaped the
|
||||
current design: **content is preserved by REGENERATING it under structural
|
||||
conditioning, never by copying original pixels.**
|
||||
|
||||
> **Superseded in 0.24.0.** The `controlnet`, `sdxl`, `qwen` and `default` profiles
|
||||
> were removed, and a retired name is now rejected at parse time rather than routed
|
||||
> onward. `--pipeline` accepts only `qwen-zimage` (the default) and `sdxl-zimage`;
|
||||
> both are CUDA-only, both condition their global stage on a canny edge map, and both
|
||||
> run the same Z-Image face stage. The bullets below are a record of what was measured
|
||||
> on the removed profiles, kept because the oracle verdicts are still the evidence the
|
||||
> current design rests on.
|
||||
|
||||
- **Text + structure:** `--pipeline controlnet` (SDXL img2img + a canny ControlNet) is
|
||||
**THE DEFAULT pipeline since 2026-06-09** (`--pipeline default` opts down to plain
|
||||
SDXL img2img for inputs without text/faces). It conditions the regeneration on the
|
||||
@@ -480,21 +499,21 @@ conditioning, never by copying original pixels.**
|
||||
measured at. See §5.2.**
|
||||
- **Highest-fidelity CUDA option:** `--pipeline qwen-zimage` is the recommended
|
||||
quality mode when preserving face identity matters more than latency, model size,
|
||||
and GPU cost. ControlNet remains the default because it is much cheaper and supports
|
||||
and GPU cost. ControlNet was then the default, because it was much cheaper and ran on
|
||||
CUDA, XPU, MPS, and CPU, but canny conditioning preserves edges rather than identity.
|
||||
On two direct upstream comparisons, `qwen-zimage` retained substantially more
|
||||
ArcFace identity than polished ControlNet. On 2026-07-25 the exact six-output
|
||||
`visible -> qwen-zimage -> metadata` candidate was negative in the corresponding
|
||||
OpenAI and Gemini oracles. This is a quality recommendation for the measured content,
|
||||
not broad removal certification; very small text can still degrade.
|
||||
See `docs/known-limitations.md` for the metrics, runtime, and validation scope.
|
||||
- **Face identity:** canny holds face *structure* but not *identity*. The standard
|
||||
SDXL and ControlNet profiles do not run a separate face-restoration option.
|
||||
Earlier GFPGAN, PhotoMaker, and FaceID experiments were removed after they
|
||||
degraded identity or risked reintroducing source pixels. The separate
|
||||
`qwen-zimage` profile now provides the only shipped face-specific stage:
|
||||
YuNet and SAM locate faces, then Z-Image regenerates the selected original
|
||||
face crops before a feathered composite. See
|
||||
See `docs/qwen-improvement-research.md` for the identity and text metrics and the
|
||||
validation scope of those comparisons.
|
||||
- **Face identity:** canny holds face *structure* but not *identity*. The removed
|
||||
SDXL and ControlNet profiles did not run a separate face-restoration stage, and
|
||||
earlier GFPGAN, PhotoMaker, and FaceID experiments were dropped after they
|
||||
degraded identity or risked reintroducing source pixels. Both shipped profiles
|
||||
now run the same face-specific stage: YuNet and SAM locate faces, then Z-Image
|
||||
regenerates the selected original face crops before a feathered composite. See
|
||||
`docs/controlnet-removal-pipeline-research.md` for the historical experiments.
|
||||
|
||||
### 5.2 Strength setting
|
||||
@@ -514,10 +533,17 @@ study (section 2.2) gives empirical floors:
|
||||
- **Google native 2816**: 0.15 clears (n=2, deployed controlnet worker, 2026-06-14) --
|
||||
the same rung as capped 1536, so no resolution penalty was observed.
|
||||
|
||||
The default is **vendor-adaptive** (`watermark_profiles.resolve_strength` +
|
||||
`vendor_for_strength`): the tool reads the C2PA issuer on the original input and picks
|
||||
> **Superseded in 0.24.0.** The `sdxl`, `controlnet`, `qwen` and `default` profiles
|
||||
> were removed, and `OPENAI_STRENGTH` / `GEMINI_STRENGTH` / `UNKNOWN_STRENGTH` went
|
||||
> with them. Everything from here to the end of this section is a record of what was
|
||||
> measured on those profiles, kept because the oracle verdicts are still the evidence
|
||||
> base. For the strength policy in force now see
|
||||
> [`known-limitations.md`](known-limitations.md#strength-is-content-and-seed-dependent).
|
||||
|
||||
The default was **vendor-adaptive** (`watermark_profiles.resolve_strength` +
|
||||
`vendor_for_strength`): the tool read the C2PA issuer on the original input and picked
|
||||
`OPENAI_STRENGTH` 0.10 / `GEMINI_STRENGTH` 0.15 / `UNKNOWN_STRENGTH` 0.15 **(LOWERED
|
||||
2026-06-14 from the 2026-06-04 cert floors 0.20/0.30/0.30)**. **The SAME ladder applies
|
||||
2026-06-14 from the 2026-06-04 cert floors 0.20/0.30/0.30)**. **The SAME ladder applied
|
||||
to both pipelines** (`sdxl` and `controlnet`). The 2026-06-14 re-test on the deployed
|
||||
Modal controlnet worker (v0.10.0) cleared SynthID on the oracle at OpenAI 0.10 (2
|
||||
photoreal) and Google 0.15 (2 NATIVE 2816x1536, contradicting the "native >= 0.30" guess
|
||||
@@ -538,7 +564,7 @@ for `sdxl`, not a separate certification (no local SynthID detector to self-veri
|
||||
The higher strength costs little quality where it matters, because `controlnet` is now
|
||||
the default pipeline, so `sdxl` is reached only via an explicit `--pipeline sdxl` (a
|
||||
deliberate opt-down), where over-regeneration has no faces/text to damage.
|
||||
This uses the vendor signal we DO have locally (the C2PA SynthID proxy) to avoid the
|
||||
This uses the vendor signal we DO have locally (the C2PA SynthID provenance) to avoid the
|
||||
overkill of a single high default on OpenAI images, without needing a local pixel
|
||||
detector. An explicit `--strength` always wins. If the watermark still survives (e.g. a
|
||||
large native Gemini beyond the capped-1536 validation), raise toward 0.35-0.40 (0.40
|
||||
@@ -597,7 +623,7 @@ Two constraints on reading this:
|
||||
- **The correct oracle for OpenAI images is openai.com/verify**, not the Gemini
|
||||
app. The two oracles detect different payloads.
|
||||
- **A quiet `identify` output after processing is not proof of removal.** It
|
||||
means the metadata proxy is gone. The pixel watermark state is unknown without
|
||||
means the provenance evidence is gone. The pixel watermark state is unknown without
|
||||
an oracle check.
|
||||
- **After removal, the output may carry forensic artifacts** detectable by an
|
||||
independent classifier even if the vendor oracle reads negative. Defeating the
|
||||
@@ -665,9 +691,9 @@ solid at 0.10 but at 0.15 it is near-threshold noise; (2) for reliable removal p
|
||||
a strength with MARGIN above the borderline (controlnet >= 0.20), not exactly on
|
||||
it; (3) **historical engineering conclusion:** this dated run argued for a
|
||||
higher ControlNet strength than the then-current default. That proposal was
|
||||
later superseded. The current resolver intentionally shares the 0.10/0.15
|
||||
ladder between SDXL and ControlNet and uses a separate Qwen ladder; see
|
||||
`_internal/watermark_profiles.py`.
|
||||
later superseded. The resolver of that period shared the 0.10/0.15 ladder between
|
||||
SDXL and ControlNet; the current policies are recorded in
|
||||
[`known-limitations.md`](known-limitations.md#strength-is-content-and-seed-dependent).
|
||||
Source images are private (faces / product shots), not committed; reproduce on any
|
||||
photoreal + flat-graphic gpt-image pair, varying the seed, and re-checking the
|
||||
oracle.
|
||||
@@ -707,10 +733,11 @@ end has simply never been through the Gemini oracle on any pipeline. Do not reas
|
||||
a resolution trend here; measure it.
|
||||
|
||||
**Current implication:** the old floor table remains evidence about the dated
|
||||
test set, not the current resolver. The shipped SDXL and ControlNet defaults are
|
||||
defined in `watermark_profiles.py`, and face restoration is available only
|
||||
through the separate `qwen-zimage` profile. Removal near a threshold remains
|
||||
seed dependent, so reproducible verification requires a fixed seed.
|
||||
test set, not the current resolver. The SDXL and ControlNet profiles it measured
|
||||
no longer exist; the shipped defaults are defined in `watermark_profiles.py`, and
|
||||
both surviving profiles run face repair as a built-in second stage rather than as
|
||||
an optional restore. Removal near a threshold remains seed dependent, so
|
||||
reproducible verification requires a fixed seed.
|
||||
|
||||
---
|
||||
|
||||
@@ -742,3 +769,15 @@ seed dependent, so reproducible verification requires a fixed seed.
|
||||
8. Jiang et al. (2025). **VideoMarkBench: Benchmarking Robustness of Video
|
||||
Watermarking.** arXiv:2505.21620.
|
||||
https://arxiv.org/abs/2505.21620
|
||||
|
||||
**Google floor on qwen-zimage (0.27.2, remeasured 2026-08-25).** The bracket above did not
|
||||
survive the full production path: on the 4.33 MP CJK-sign fixture (visible-stage
|
||||
sparkle removal -> qwen-zimage seed 0 at the curve's 0.154 top -> resize-back ->
|
||||
metadata strip), the Gemini verifier detected SynthID x3 with a valid
|
||||
pixel-identical stripped control in the same session. Google-provenance content
|
||||
initially took a flat `0.30` floor instead of the area curve. The later provider
|
||||
remeasurement established a common pass at `0.25` across three valid sources and
|
||||
separately repeated `QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.27` across those sources and
|
||||
three accounts. OpenAI now has its own measured flat operating point; unknown
|
||||
content keeps the resolution curve. An explicit strength still wins. Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-18/` (the failing re-baseline).
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
> Research archive. This page records evaluated ideas, including rejected
|
||||
> prototypes. Current behavior is documented in the user guides and source.
|
||||
> The `text_protector.py` module it describes was removed from the library
|
||||
> afterwards.
|
||||
|
||||
Date: 2026-05-29. Source: a deep-research run (104 agents, 5 search angles, sources
|
||||
fetched and 3-vote adversarially verified). Not committed automatically — saved as a
|
||||
@@ -18,8 +20,8 @@ filter applied to every candidate below.
|
||||
## Problem recap
|
||||
|
||||
The `invisible` pipeline is SDXL base 1.0 img2img to defeat SynthID. The default
|
||||
strength has risen over time as Google hardens SynthID (0.05 -> 0.10 -> **~0.30**, the
|
||||
current threshold for fresh Gemini output); higher strength deforms text more, which is
|
||||
strength rose over time as Google hardened SynthID (0.05 -> 0.10 -> **~0.30** in
|
||||
this experiment; the later measured operating point is 0.27); higher strength deforms text more, which is
|
||||
exactly why text protection matters. Text is protected via Differential Diffusion with a
|
||||
per-pixel change map (`preserve` ~0.9) driven by the PP-OCRv3 DB detector
|
||||
(`text_protector.py`). Large text survives; **small text (sub ~8 px strokes) softens or
|
||||
@@ -141,3 +143,229 @@ Deep-research workflow run `wf_118b9a03-3eb` (2026-05-29). Findings adversariall
|
||||
(2/3 refutes required to kill a claim). This note records research only; no code change is
|
||||
implied until a prototype validates fidelity and the SynthID-scrub guarantee on the
|
||||
restored output.
|
||||
|
||||
## 2026-08-13 OCR plus LaMa prototype
|
||||
|
||||
A lightweight version of the recommended two-stage architecture was tested on
|
||||
the three tracked text fixtures. It used the already-clean Qwen result as the
|
||||
base, recognized the source text, removed source and Qwen glyph contours with
|
||||
LaMa, and rendered the recognized strings as fresh pixels. No source pixels
|
||||
were pasted back.
|
||||
|
||||
The result validates only part of the hypothesis. Character-weighted CER on the
|
||||
two multilingual posters fell from 0.338/0.305 to 0.007/0.007, and OpenAI Verify
|
||||
remained clean for both matched Qwen and restored pairs. However, replacement
|
||||
fonts changed the design, whole-image LPIPS worsened by 0.067 on each poster,
|
||||
and the light poster retained a shadow in one small English line. On the Chinese
|
||||
sign, punctuation errors moved CER from 0.074 to 0.111. The Google verifier run
|
||||
was inconclusive because the available account detected both the restored image
|
||||
and the byte-identical Qwen control that a previous account had called clean.
|
||||
|
||||
The experiment supports a proper glyph-conditioned restorer, not shipping the
|
||||
plain OCR/system-font compositor. Exact hashes, metrics, masks, and provider
|
||||
verdicts are tracked in
|
||||
[`data/evaluations/fidelity/text-restoration-2026-08-13.csv`](../data/evaluations/fidelity/text-restoration-2026-08-13.csv).
|
||||
|
||||
## 2026-08-13 AnyText2 follow-up
|
||||
|
||||
The official AnyText2 SD1.5 checkpoint was then tested as the glyph-conditioned
|
||||
restorer. Its ModelScope entry and code are Apache 2.0, and the downloaded
|
||||
checkpoint matched the published SHA-256. An official blackboard edit completed
|
||||
successfully before the fixture run, establishing that the runtime reproduced
|
||||
the model rather than silently exercising a fallback.
|
||||
|
||||
On the Chinese sign, a three-line local edit over the exact Qwen output scored
|
||||
CER 0.185 under the standard detector. Font mimic from the source glyph masks
|
||||
scored 0.222. Whole-image LPIPS changed from 0.289 to 0.338 and 0.345 respectively.
|
||||
A padded crop-recognition check showed why the comparison matters: Qwen contained
|
||||
the full correct text and scored 0.000, while default AnyText2 still scored 0.074
|
||||
with two substituted characters and mimic remained at 0.222. The provider oracle
|
||||
was deliberately not called because these variants had already failed the
|
||||
content-fidelity gate.
|
||||
|
||||
AnyText2 is therefore not the missing production restorer in its published form.
|
||||
The negative result is narrower than the model family: only the SD1.5 checkpoint
|
||||
is public, while AnyText2XL remains listed as future work. The current wrapper
|
||||
also truncates each quoted text line to 20 characters, which prevents a direct
|
||||
test of several tracked English and Russian poster lines. Exact model provenance,
|
||||
output hashes, and metrics are recorded in
|
||||
[`data/evaluations/fidelity/anytext2-restoration-2026-08-13.csv`](../data/evaluations/fidelity/anytext2-restoration-2026-08-13.csv).
|
||||
|
||||
The same cross-check exposes a measurement bug in the earlier lightweight
|
||||
restoration result. Paddle detection made tight boxes that omitted the final
|
||||
Chinese full stop from two lines, producing Qwen CER 0.074 although the pixels
|
||||
and padded recognition were correct. Adding 16-32 pixels of horizontal context
|
||||
restored every punctuation mark. A deterministic rerender using the full OCR
|
||||
strings and the closest of nine mask-scored CJK fonts also reached detector CER
|
||||
0.000, but its heavier Hiragino Sans GB W6 glyphs raised LPIPS from 0.289 to 0.327.
|
||||
The actionable design is selective restoration: compare padded source and output
|
||||
recognition first, and preserve the Qwen output when they already match.
|
||||
|
||||
That policy was tested on the two multilingual posters. A manual prototype kept
|
||||
the lines whose padded source and Qwen recognition agreed and rerendered only the
|
||||
changed lines. The edited mask fell from 15.4%/17.0% in the full compositor to
|
||||
5.5%/5.5%. On a single Paddle `en+ru+ch` route, CER changed from 0.378/0.413 for
|
||||
Qwen to 0.101/0.112, while LPIPS was only 0.120/0.103 rather than the full
|
||||
compositor's 0.174/0.162. The different OCR route is intentional and all three
|
||||
variants were remeasured through it; these CER values are not directly
|
||||
comparable with the earlier Vision/Paddle hybrid results.
|
||||
|
||||
Both selective outputs returned `No OpenAI signals detected` in OpenAI Verify,
|
||||
and the original light poster returned `Generated with OpenAI tools` immediately
|
||||
afterward as the positive control. The dark poster was visually clean. The light
|
||||
poster still showed one local old-text shadow, so mask construction remains the
|
||||
blocking defect. Exact hashes, metrics, mask fractions, and oracle controls are
|
||||
in
|
||||
[`data/evaluations/fidelity/selective-text-restoration-2026-08-13.csv`](../data/evaluations/fidelity/selective-text-restoration-2026-08-13.csv).
|
||||
|
||||
Uniformly expanding every selected glyph mask by two pixels removed that light
|
||||
poster shadow and moved CER from 0.101 to 0.081 without a meaningful LPIPS cost.
|
||||
It also expanded the edited area from 5.5% to 7.0%, and the identical rule made
|
||||
the dark poster worse at CER 0.147 rather than 0.112. Those follow-up bytes have
|
||||
not been oracle-checked. The next prototype should detect residual source glyphs
|
||||
per line and expand only the failing component, rather than introducing another
|
||||
poster-wide tuning constant.
|
||||
|
||||
The manual selection was then replaced with per-line padded recognition. A line
|
||||
is left untouched only when the source recognizer is sufficiently consistent
|
||||
with the verified line and normalized source/candidate recognition agree
|
||||
exactly. The automatic rule reproduced the manual selection on the dark poster
|
||||
and correctly kept one additional large Russian line on the light poster. It
|
||||
reduced that poster's effective edited area from 5.5% to 4.1%.
|
||||
|
||||
Mask cleanup became a measured second pass rather than a global dilation. After
|
||||
the first LaMa erase, the prototype finds contrast that remains specifically at
|
||||
the original glyph positions, dilates only that residual, and erases it before
|
||||
fresh text is drawn. This removed the visible double glyphs from both posters.
|
||||
The automatic outputs scored LPIPS 0.113/0.104 and detector CER 0.123/0.119, with
|
||||
effective edited fractions of 4.1%/5.7%. The light aggregate CER is pessimistic:
|
||||
the page detector fragments its untouched Cyrillic line into Latin-like pieces,
|
||||
while the padded Russian recognizer returns the exact expected text.
|
||||
|
||||
OpenAI Verify returned `No OpenAI signals detected` for both new hashes and then
|
||||
`Generated with OpenAI tools` for the original light poster in the same browser
|
||||
sequence. Reproducing the prototype from the tracked script exposed harmless
|
||||
ONNX rounding on the light image: 386 pixels differed by at most one channel
|
||||
value from the initially checked hash. The exact tracked output hash
|
||||
`900def5a...` was therefore submitted separately, returned clean, and was
|
||||
followed immediately by another positive source result. The dark tracked output
|
||||
was byte-identical to the initially checked hash.
|
||||
|
||||
The Chinese sign provides a no-edit control for the selector. Detector boxes for
|
||||
its three large lines overlap vertically, so Latin-style vertical padding
|
||||
incorrectly mixed neighboring lines. Horizontal-only CJK padding captured the
|
||||
terminal punctuation without mixing neighbors and made source and candidate
|
||||
recognition agree exactly on all three lines.
|
||||
The script then produced a zero mask and copied the Qwen candidate byte for byte.
|
||||
This prevents the heavier-font regression seen in the earlier CJK compositor,
|
||||
but does not resolve the unavailable Google-negative oracle control.
|
||||
|
||||
The result clears the measured OpenAI gate without a manual line selection, but
|
||||
is not a production design yet. It still assumes verified source strings and
|
||||
line boxes, relies on macOS system fonts, and needs evaluation on uncurated
|
||||
layouts, rotated text, false OCR matches, automatic line-box discovery, and a
|
||||
reproducible Google-negative control.
|
||||
|
||||
The later source-silhouette experiment removed the font lookup but did not meet
|
||||
the actual visual requirement. Native-size review found changed stroke weight,
|
||||
color variation, edge antialiasing, and decorative details even where crop OCR
|
||||
was exact. A narrower `vae-glyphs` experiment uses the silhouette only as a
|
||||
support mask: exact pixels come from a Qwen-VAE reconstruction, while a fresh
|
||||
flat-color silhouette supplies only the outer edge beneath that core. Across 48
|
||||
typography cases (548 annotated lines), this improved glyph-edge MAE in 48/48
|
||||
and text-box SSIM in 47/48. The median values moved from 37.59 to 32.42 and from
|
||||
0.854 to 0.914 respectively. One separate untracked core-only output with a
|
||||
1.43% donor area returned `No OpenAI signals detected`; the tracked script then
|
||||
reproduced those exact bytes. A 0.5-pixel feathered sibling with 2.75% nonzero
|
||||
donor area improved mean text-box SSIM from 0.957 to 0.960, with a 0.918 minimum
|
||||
across 15 verified lines, while full-image RGB SSIM reached 0.864. Crop OCR
|
||||
recovered from 7/15 exact lines on the raw pass to 14/15, equal to the source's
|
||||
own recognizer floor. Whole-image LPIPS was 0.082, but only 0.108% of pixels
|
||||
remained exactly equal to the source and the detected face's Laplacian-variance
|
||||
ratio was 0.670, confirming residual global smoothing. A same-session
|
||||
OpenAI Verify sequence detected the exact source control in 1/1 check, returned
|
||||
clean for the raw global pass and clean-fidelity anchor in 1/1 each, and returned
|
||||
clean for the feathered output in 3/3 repeats. The tracked script reproduced
|
||||
that feathered file byte for byte. This proves two materially better exact-output
|
||||
Pareto points, not a general operating range. The 48 matrix outputs and other
|
||||
mask sizes remain oracle-unverified.
|
||||
|
||||
The first Google oracle sample did not transfer. For the synthetic CJK sign,
|
||||
two separate work accounts detected SynthID in both the resaved positive source
|
||||
control and the exact Qwen-VAE donor candidate. The candidate's text-box SSIM
|
||||
improved from 0.791 to 0.835 and its glyph-edge MAE from 35.48 to 22.49, but its
|
||||
3.67% donor layer did not create a clean operating point. The Qwen silhouette
|
||||
base was already detected, as was the earlier original-sign Qwen baseline in the
|
||||
same account. The measured cause is therefore insufficient global Google
|
||||
regeneration, not a demonstrated watermark regression from the text donor.
|
||||
Google needs a stronger oracle-clean global anchor before the narrow donor can
|
||||
be evaluated there.
|
||||
|
||||
A targeted follow-up supplied that missing anchor for one CJK case. An explicit
|
||||
Qwen global pass at strength 0.30 returned no reliable SynthID signal in Gemini,
|
||||
while the matched source control was detected. Applying the 0.5-pixel
|
||||
`vae-glyphs` layer to that clean anchor changed 1.17% of the image and raised
|
||||
mean text-box SSIM from 0.765 to 0.964 (minimum 0.963), with mean text-box MAE
|
||||
falling from 20.74 to 5.38. The post-donor output then returned clean in 3/3
|
||||
Gemini checks across two work accounts. A separate 18-face fixture with four
|
||||
tiny UI-text lines also had a clean strength-0.30 anchor; its donor layer changed
|
||||
0.70% of the image, raised mean text-box SSIM from 0.617 to 0.976 (minimum
|
||||
0.972), and returned clean in 3/3 checks across the same two accounts. One
|
||||
quota-exceeded response is excluded from both pass rates. These are two exact
|
||||
oracle-certified outputs, not a general operating range: other layouts, masks,
|
||||
strengths, seeds, and donor areas remain unverified.
|
||||
|
||||
An automatic-box follow-up merged Paddle word detections by vertical overlap.
|
||||
It found exactly 20/20 poster lines and 3/3 sign lines, with mean IoU 0.857,
|
||||
0.847, and 1.000 against the verified boxes. That structural match was not
|
||||
sufficient: reusing the annotation crop padding changed recognition decisions,
|
||||
reduced preserved dark-poster lines from 7 to 4, and expanded its edited
|
||||
fraction from 5.7% to 11.2%.
|
||||
|
||||
A recognition-only sweep isolated the crop sensitivity. Limiting vertical
|
||||
padding around detector boxes to 8-12% of line height reproduced the verified-
|
||||
box decision vectors on both posters; 10% was used for a full follow-up. It kept
|
||||
8/20 and 7/20 lines with edited fractions of 4.1% and 5.7%, and visual inspection
|
||||
found no double glyphs. Whole-image LPIPS remained 0.113/0.104, but detector CER
|
||||
was 0.127/0.154 instead of 0.123/0.119. The dark regression failed the fidelity
|
||||
gate, so these hashes were not submitted to the provider oracle. The opt-in flag
|
||||
remains only to reproduce the negative result. Count, IoU, and matching decision
|
||||
vectors are therefore insufficient gates for automatic boxes; the next design
|
||||
needs box rectification or recognition stability under crop jitter before it
|
||||
can replace verified geometry.
|
||||
|
||||
Removing verified strings was tested separately with an annotation-seed dry
|
||||
run. It detects boxes, chooses `en`, `ru`, or `ch` from Unicode script, and
|
||||
accepts a draft only when three crop paddings normalize identically and every
|
||||
confidence is at least 0.85. One execution proposed 20 and 18 poster lines, but
|
||||
exact-text precision against the held-out annotations was only 90.0% and 94.4%.
|
||||
The stable errors were punctuation: one lost English comma and an ideographic
|
||||
comma consistently replaced by ASCII. Thus crop agreement and confidence do
|
||||
not verify literal text. The helper is retained only to seed annotations; every
|
||||
proposal needs manual review before it can enter the restoration compositor.
|
||||
|
||||
## Production-path anchor leak and fix (2026-08-19)
|
||||
|
||||
The engine `text_manifest=` path (unlike the research compositor above) blends a
|
||||
15% Qwen-VAE fidelity anchor across the WHOLE frame before glyph restoration.
|
||||
Certified through the official OpenAI Content Provenance API on the exact
|
||||
production path (qwen-zimage, seed 0, no strength override, untiled, LANCZOS
|
||||
resize-back, metadata strip; library 0.27.0 @ 50d9f448):
|
||||
|
||||
- controls (pixel-identical metadata-stripped sources): `synthid: detected` -
|
||||
the pixel channel was proven live before every session;
|
||||
- base outputs: `clean` x6 (two 20-line posters, 3 attempts each);
|
||||
- text-restored WITH the anchor: `synthid: detected` x6, deterministic;
|
||||
- text-restored with `FIDELITY_BLEND_ALPHA = 0` (runtime ablation, base bytes
|
||||
identical between runs): `clean` x6.
|
||||
|
||||
The anchor was therefore the leak channel: a 15% global donor admixture over a
|
||||
scrubbed frame returns enough OpenAI SynthID for the provider detector. Fidelity
|
||||
cost of dropping it (source-referenced text-box MAE): 11.60->11.72, 7.79->7.86,
|
||||
and 7.57->8.13 across the two posters and the CJK sign - well under one point
|
||||
against base MAEs of 24.7/16.8/10.2. Since 0.27.1 the anchor is off by default
|
||||
(`fidelity_anchor` parameter on `remove_watermark` / `InvisibleOptions` /
|
||||
`--fidelity-anchor`); Google rows were not re-queried because the default
|
||||
strength leaves that fixture's base detected (the vendor-strength lane tracks
|
||||
that separately). Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-19-A1/` (sha256-stamped rows).
|
||||
@@ -24,7 +24,7 @@ Performance depends on format, enabled extras, and hardware. Measure it locally
|
||||
|
||||
## Tier A -- self-evident oracles (full corpus, unattended)
|
||||
|
||||
Properties that are true or false without anyone labelling anything. These are the
|
||||
Properties that are true or false without anyone labeling anything. These are the
|
||||
backbone: they scale to large local datasets and catch regressions with zero human cost.
|
||||
|
||||
### A1. Recorded-verdict regression
|
||||
@@ -39,6 +39,15 @@ confidence), reviewed once, then re-baselined. Lost detections are the alarm.
|
||||
|
||||
Implemented as `scripts/sidecar_regression.py` (resumable, ~1.5 h at 8 workers).
|
||||
|
||||
A 2026-08-15 metadata-only C2PA regression audit over the local historical
|
||||
corpus caught two structured-parser gaps before release: an exact Firefly claim
|
||||
generator without a repeated source type, and a Dreamina generator carried by a
|
||||
reachable ingredient under a generic update manifest. Both now have focused
|
||||
tests. The same audit confirmed that invalid hash/signature claims lose origin
|
||||
attribution without losing the C2PA inventory, and exposed the absence of
|
||||
production trust anchors in the SDK defaults. Dataset-derived counts and
|
||||
identifiers remain in the gitignored audit output.
|
||||
|
||||
#### Local run protocol
|
||||
|
||||
Re-run `identify` against locally recorded sidecars, classify losses separately from intended new detections, and keep generated reports under `.local-eval/`. Do not commit dataset-derived counts or identifiers.
|
||||
@@ -72,8 +81,10 @@ Run detection, removal, and re-detection over a representative local set. Confir
|
||||
|
||||
### A5. Contract sweep across every parameter choice
|
||||
|
||||
`scripts/smoke_matrix.py` (exists, 68 rows, 0 skipped with `--diffusion`) covers every
|
||||
choice-valued flag on fixtures. Extend from fixtures to a stratified corpus slice
|
||||
`scripts/smoke_matrix.py` covers every choice-valued flag on fixtures. Its knob rows
|
||||
were rewritten when the CPU/MPS profiles and the ESRGAN chain were removed: most now
|
||||
assert a knob is REJECTED, and the accepted-knob rows skip without a CUDA device, so
|
||||
the row count is host-dependent rather than the fixed 68 recorded here before. Extend from fixtures to a stratified corpus slice
|
||||
(~500 images spanning format x provenance x aspect ratio), asserting exit-code semantics
|
||||
rather than just absence of crash.
|
||||
|
||||
@@ -84,32 +95,40 @@ sweep asserts on stderr, or the codes get split -- the latter is the better fix.
|
||||
#### Coverage method
|
||||
|
||||
Compare the flags and values exercised by the matrix against the options declared by
|
||||
the CLI. Include optional backends, batch modes, tiling, region-targeted composition,
|
||||
the ESRGAN upscaler, and the ffmpeg audio/video strip. The gap to find is not only
|
||||
the CLI. Include optional backends, batch modes, tiling, and the ffmpeg audio/video
|
||||
strip. (Region-targeted diffusion composition was on this list until the parameter
|
||||
was deleted: nothing but a caller-less convenience wrapper ever reached it, and the
|
||||
user-facing region path is `erase`, which inpaints instead.) The gap to find is not only
|
||||
"logic untested" but
|
||||
"never executed on real data", which is precisely what this campaign is for.
|
||||
|
||||
#### Bug found by the extension: `--steps` below ~7 crashes inside torch
|
||||
#### Bug found by the extension: `--steps` below ~7 crashed inside torch
|
||||
|
||||
Effective timesteps are `int(steps * strength)`. At the vendor-adaptive default strength
|
||||
(0.15, or 0.10 for OpenAI) any `--steps` under 7 rounds to **zero**, and the pipeline dies
|
||||
with a raw traceback:
|
||||
**Fixed by deletion.** `--steps` no longer exists, on the CLI or in the Python API,
|
||||
so this class of failure is unreachable. Kept as a record of why.
|
||||
|
||||
Effective timesteps were `int(steps * strength)`. At the vendor-adaptive default
|
||||
strength (0.15, or 0.10 for OpenAI) any `--steps` under 7 rounded to **zero**, and the
|
||||
pipeline died with a raw traceback:
|
||||
|
||||
```
|
||||
$ remove-ai-watermarks invisible img.png --steps 5
|
||||
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1, 1, 512]
|
||||
```
|
||||
|
||||
Fully valid CLI arguments, no special flags, no `--force`. The value is accepted, the
|
||||
crash is a torch internal, and nothing tells the user that steps and strength interact.
|
||||
Fix is either a clamp to >=1 effective step or an up-front validation naming both values.
|
||||
Fully valid CLI arguments, no special flags, no `--force`. The value was accepted, the
|
||||
crash was a torch internal, and nothing told the user that steps and strength interact.
|
||||
The considered fixes were a clamp to >=1 effective step or an up-front validation
|
||||
naming both values; what shipped instead is that each stage owns its own distilled
|
||||
schedule and no caller can set it. The general lesson stands: a knob whose valid range
|
||||
depends on another knob's value needs the interaction validated where both are known,
|
||||
or it needs to not be a knob.
|
||||
|
||||
Method note: the first run of the knob rows failed 12 times with this identical error,
|
||||
which read like twelve broken features. It was one bad harness parameter (`--steps 4`)
|
||||
sitting on top of one real bug. An error that is IDENTICAL across unrelated rows is
|
||||
evidence of a common cause, not of many faults -- check the shared input first.
|
||||
|
||||
## Tier B -- constructed ground truth (automatable, no labelling)
|
||||
## Tier B -- constructed ground truth (automatable, no labeling)
|
||||
|
||||
Where reality gives no answer key, build one. This is the tier that closes the two biggest
|
||||
holes: fill quality and detector response at the edge of the operating range.
|
||||
@@ -150,7 +169,7 @@ Peak RSS and wall time per backend x input size, up to 25 MP. The memory-constra
|
||||
tier is a real constraint (MI-GAN must stay ~0.6-0.9 GB by cropping around the mask); a
|
||||
regression here is invisible today and would only surface under load.
|
||||
|
||||
## Tier C -- human-labelled accuracy (bounded by labelling effort)
|
||||
## Tier C -- human-labeled accuracy (bounded by labeling effort)
|
||||
|
||||
The machinery exists: `visible_recall_sample.py` -> `visible_sheets.py` ->
|
||||
`visible_groundtruth.py` -> `visible_eval.py`.
|
||||
@@ -167,10 +186,14 @@ report recall from the detector-sampled set.
|
||||
|
||||
## Tier D -- external oracles (manual, not automatable here)
|
||||
|
||||
SynthID removal cannot be verified locally by design -- no public decoder exists. Each
|
||||
vendor has its own oracle and it covers only that vendor's content: `openai.com/verify` for
|
||||
OpenAI (more accessible, the automation candidate), the Gemini app for Google (manual,
|
||||
rate-limited). A quiet metadata proxy is **not** proof the pixel watermark is gone.
|
||||
Proprietary watermark removal cannot be verified locally by design -- no public decoder
|
||||
exists. Each vendor has its own oracle and it covers only that vendor's content:
|
||||
`openai.com/verify` for OpenAI, the Gemini app for Google, and Microsoft's
|
||||
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
|
||||
for InvisMark. The Microsoft API reports `Watermark` and `C2PA` separately. It therefore
|
||||
needs a pixel-identical metadata-stripped control: the control must lose `C2PA` while
|
||||
remaining `Watermark`-positive before a candidate's negative result can be attributed to
|
||||
pixel regeneration. A quiet metadata proxy is **not** proof the pixel watermark is gone.
|
||||
|
||||
Scope honestly: this tier certifies strength floors on a handful of images per vendor, and
|
||||
that is all it can do. See `docs/synthid.md`.
|
||||
@@ -268,7 +291,7 @@ on one file. The bar is never "handles it" but **never raises and never silently
|
||||
4. **B2 detector curves** -- cheap, and directly guards the geometry class of bug.
|
||||
5. **A5 contract sweep over a representative local set**.
|
||||
6. **B4 resource ceilings**, **E robustness**.
|
||||
7. **C recall expansion** -- gated by labelling appetite.
|
||||
7. **C recall expansion** -- gated by labeling appetite.
|
||||
8. **D oracles** -- manual, per release.
|
||||
|
||||
Every tier writes a versioned snapshot so runs are comparable over time; a run that cannot
|
||||
@@ -303,7 +326,7 @@ Per mark, what actually goes away when metadata is stripped:
|
||||
|
||||
### Measure before improving
|
||||
|
||||
Use Tier B2 detector-response curves before tuning a detector with sparse labelled
|
||||
Use Tier B2 detector-response curves before tuning a detector with sparse labeled
|
||||
examples. Sweep size, contrast, aspect, and background texture so geometry regressions
|
||||
are visible without exposing private evaluation statistics.
|
||||
|
||||
|
||||
@@ -0,0 +1,607 @@
|
||||
# Video SynthID quality research (2026-08-05)
|
||||
|
||||
> Research archive. This page records experiments and decisions from the date
|
||||
> above. It may mention prototypes or defaults that were later changed. Use the
|
||||
> user guides and current source code for the supported interface.
|
||||
|
||||
Cited research behind the question **"can the video SynthID path keep removing the
|
||||
mark while giving up far less quality than the shipped 512 px / 12 fps profile?"**
|
||||
Produced by a 13-agent workflow: 6 parallel scouts (pipeline ablation, SynthID
|
||||
internals, attack literature, autoencoder landscape, perceptual masking, experiment
|
||||
design), 11 change proposals from 3 independent design angles, and 3 adversarial
|
||||
critics (signal theory, repository engineering, verifiability). No experiment was
|
||||
run against the provider oracle for this page; every claim is labeled MEASURED,
|
||||
REPORTED, or INFERRED.
|
||||
|
||||
Repository claims below were re-verified against source after the workflow
|
||||
returned: the PSNR reference frame, the audio stream copy, the missing `_fit_size`
|
||||
clamp, the absent `enable_tiling` call, the crf asymmetry, the test pins, and the
|
||||
absent `scaling_factor` key in the cached checkpoint config.
|
||||
|
||||
## Context
|
||||
|
||||
`remove_video_invisible` regenerates video pixels through `sd-vae-ft-mse`: frames
|
||||
are decimated to 12 fps, resized to a 512 px long side, encoded to latents with
|
||||
`latent_dist.mode()`, perturbed by one seeded spatial noise field shared across
|
||||
every frame, decoded, and streamed to an H.264 encoder at crf 18. A separate
|
||||
stream-copy mux re-adds source audio and strips metadata. The shipped
|
||||
`noise_std=0.15` is certified by one oracle row on one carrier.
|
||||
|
||||
The quality complaint is real: a 1080p source is delivered at roughly a quarter of
|
||||
its linear resolution and half its frame rate. This page asks what part of that
|
||||
cost is buying removal and what part is buying nothing.
|
||||
|
||||
## What the two oracle rows do and do not prove
|
||||
|
||||
`data/evaluations/video-synthid-oracle.csv`, one carrier (Veo 3 off-road,
|
||||
sha256 `79a552b9...`), one seed, one geometry:
|
||||
|
||||
| noise_std | long_side | fps | verdict | psnr_db | temporal_residual_ratio |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 0.10 | 512 | 12 | detected | 26.2932 | 1.0072 |
|
||||
| 0.15 | 512 | 12 | not_detected | 25.3911 | 1.0578 |
|
||||
|
||||
They prove exactly one thing: the envelope of 512 px, 12 fps, crf 18, and a full
|
||||
VAE round trip does **not** silence the oracle on its own. The verdict flip is
|
||||
bought by the last 0.05 of latent noise.
|
||||
|
||||
They cannot decompose that conjunction. No row varies `long_side` or `fps`, so the
|
||||
contribution of the downscale is unmeasured. The frequent reading "the 512 px
|
||||
downscale was probably doing the removal work" is not supported, and neither is its
|
||||
opposite.
|
||||
|
||||
## Measured facts from the repository
|
||||
|
||||
- **The fidelity metric is blind to the expensive steps.** `_iter_sampled_frames`
|
||||
applies `cv2.resize` at [`video_invisible.py:236`](../src/remove_ai_watermarks/video_invisible.py);
|
||||
the same generator feeds the loop at `:401-407`; the accumulator at `:429-435`
|
||||
zips those already-resized frames against `regenerated` captured at `:423`, which
|
||||
is before `frame_pipe.write` at `:430`. `psnr_db` therefore excludes the
|
||||
downscale, the decimation, and the encoder. It measures the VAE round trip plus
|
||||
latent noise at the working geometry, nothing else.
|
||||
- **Frame decimation cannot destroy the carrier.** `_iter_sampled_frames`
|
||||
(`:224-238`) is pure subset selection: the only statement inside the threshold
|
||||
test is `yield cv2.resize(...)`, with no arithmetic across frames. The
|
||||
perturbation is equally time-blind: `shared_noise.expand(latents.shape[0], -1, -1, -1)`
|
||||
at `:295` broadcasts one CHW field across the batch and never mixes across time.
|
||||
Any fps effect on the verdict is a detection-probability effect, not carrier
|
||||
destruction.
|
||||
- **`_fit_size` has no clamp at 1.0.** `scale = long_side / max(width, height)` at
|
||||
`:88` upscales any source whose long side is below 512, and both axes are floored
|
||||
independently to a multiple of 8 at `:89-96`, which introduces an anamorphic
|
||||
shift at scale 1.0 (1366x768 becomes 1360x768).
|
||||
- **`enable_tiling` is never called** anywhere in `src/`, `scripts/`, or `tests/`.
|
||||
`enable_slicing()` at `:135` already splits both encode and decode to single
|
||||
frames, so activation VRAM is set by one frame and does not scale with
|
||||
`--batch-size`.
|
||||
- **crf asymmetry:** 18 on the invisible path (`:321`, `:386`) against 14 on the
|
||||
visible path ([`video_visible.py:1343`](../src/remove_ai_watermarks/video_visible.py))
|
||||
through the same encoder.
|
||||
- **No HDR guard on the invisible path.** `_HDR_TRANSFERS` (`video_visible.py:97`)
|
||||
and the `component_depth > 8` rejection (`:1322`) exist only for visible removal.
|
||||
- **Audio is byte-copied.** `-map 0:v:0`, `-map 1:a?`, `-c copy` at
|
||||
[`video_encoding.py:413`](../src/remove_ai_watermarks/video_encoding.py).
|
||||
- **Only `noise_std` is pinned.** `tests/test_video_invisible.py:285` asserts
|
||||
`DEFAULT_VIDEO_SYNTHID_NOISE_STD == 0.15`. Neither `long_side` nor `fps` is
|
||||
pinned anywhere, so changing either breaks no test.
|
||||
- **`scaling_factor` 0.18215 is a class default, not a checkpoint fact.** The
|
||||
cached `config.json` for `stabilityai/sd-vae-ft-mse` has no `scaling_factor` key
|
||||
at all (verified locally: `_class_name`, `latent_channels`, `block_out_channels`,
|
||||
`sample_size`, and block types only). The value comes from the `AutoencoderKL`
|
||||
class default under `diffusers>=0.38.0` with no upper bound (`pyproject.toml:94`,
|
||||
`uv.lock` resolves 0.39.0), while `maintain.sh` runs `uv-outdated`. A dependency
|
||||
bump can move the certified operating point with a fully green test suite.
|
||||
- **The manifest schema cannot record what a real program needs:** no source
|
||||
geometry, no control row, no track column, no verbatim verdict, no indeterminate
|
||||
state.
|
||||
|
||||
## Measured facts from external sources
|
||||
|
||||
- Gemini video verification quota: 10 checks per rolling 24 hours, up to 5 minutes
|
||||
of video total, under 90 seconds and 100 MB per file. The verifier reports which
|
||||
parts of the video carry the mark, and it has a **third** state beyond detected
|
||||
and not detected: unclear, with documented causes including "not enough details
|
||||
to watermark".
|
||||
- The verifier scores **audio and visual tracks separately**. Google's published
|
||||
example verdict reads as SynthID detected in the audio over a time range with no
|
||||
SynthID detected in the visuals.
|
||||
- SynthID-Image (arXiv:2510.09263) is a post-hoc, model-independent pixel-space
|
||||
watermark: a separate encoder network stamps an already-decoded image rather than
|
||||
being injected into the generator's latents. Its published payload figure is 136
|
||||
bits within a 512x512 image, and its product setup runs at 1536x1536.
|
||||
|
||||
## Inferred, with the reasoning that makes them weak
|
||||
|
||||
- **The noise axis is nearly exhausted.** Fitting `MSE = A + B * noise_std^2` to the
|
||||
two measured rows gives A = 124.5 and B = 2815, so a pure round trip
|
||||
(`noise_std = 0`) lands near 27.2 dB. The entire noise budget is worth at most
|
||||
**+1.79 dB**; everything else is the autoencoder. This is a two-parameter fit to
|
||||
two points with zero degrees of freedom, and the assumptions that MSE is
|
||||
quadratic in `noise_std` and that reconstruction and noise errors are orthogonal
|
||||
are untested. One local run at `noise_std=0` replaces it with a measurement.
|
||||
- **Statistical power of the current certification.** With zero failures at n = 1,
|
||||
the exact one-sided Clopper-Pearson bound `1 - 0.05^(1/n)` is 95%: the data are
|
||||
compatible with removal failing almost always. n = 15 gives 18.1%, n = 30 gives
|
||||
9.5%.
|
||||
- **Size of the geometry prize.** At a 1280x720 source, `_fit_size(1280, 720, 512)`
|
||||
is `(512, 288)` (pinned in `tests/test_video_synthid_sweep.py:30`), so 6.25x of
|
||||
the pixels are discarded by the downscale and another 2x by decimation from a
|
||||
24 fps source. Ladder rungs: 768 gives 2.25x the current area, 1024 gives 4.0x.
|
||||
The carrier's own geometry is recorded nowhere, so this is conditional.
|
||||
- **The direction of the resolution effect is disputed, and both sides are
|
||||
inference.** Against raising it: the absolute frequency ceiling below which an f8
|
||||
VAE reconstructs faithfully is tied to the latent pitch, so 512x288 (a 64x36
|
||||
latent) preserves roughly up to 32 cycles per frame width while 1920 (a 240x135
|
||||
latent) preserves up to about 120. Raising resolution moves the carrier band out
|
||||
of the regime the decoder synthesizes and into the regime it reproduces
|
||||
faithfully, handing the detector more evidence. For raising it: Google's product
|
||||
operating point is 1536x1536, so native is closer to the distribution the
|
||||
watermark encoder targets. Note that the second argument cuts against the
|
||||
proposal rather than for it.
|
||||
|
||||
## First local measurements (2026-08-05)
|
||||
|
||||
Run without the oracle on a locally built carrier: a 6-second 1280x720 24 fps
|
||||
clip panning across `data/fixtures/provenance/doubao-1.png`, processed by the
|
||||
shipped path at 512 px / 12 fps on MPS. Generated media stayed outside the
|
||||
repository.
|
||||
|
||||
| noise_std | engine `psnr_db` | end-to-end PSNR | end-to-end SSIM | bitrate |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 0.00 | 27.8049 | 27.6935 | 0.6955 | 2564 kbps |
|
||||
| 0.15 | 25.8853 | 25.8541 | 0.6703 | 2716 kbps |
|
||||
|
||||
Three readings, all MEASURED, none of them about SynthID:
|
||||
|
||||
1. **The two-point fit's shape survives contact with independent content.** The
|
||||
pure round trip lands at 27.80 dB against the 27.2 dB the carrier fit
|
||||
predicted, and the whole noise budget costs 1.92 dB against the predicted
|
||||
1.79 dB. These are not the carrier's numbers, but the decomposition holds:
|
||||
the autoencoder is the floor and the entire `noise_std` axis is worth about
|
||||
2 dB.
|
||||
2. **On this content the downscale is nearly free in PSNR terms.** End-to-end
|
||||
PSNR sits within 0.11 dB of the in-loop number, meaning the 512 px geometry
|
||||
cost almost nothing next to the VAE damage. This clip is a pan over a smooth
|
||||
generated image with little high-frequency detail, so it is the friendly
|
||||
case: real camera texture should widen that gap. Run the probe across content
|
||||
types before trusting any estimate of the geometry prize.
|
||||
3. **`temporal_residual_ratio` is not meaningful on a near-static shot.** It read
|
||||
1.82 at `noise_std=0` and 2.47 at 0.15, far outside the [1.0072, 1.0578] band
|
||||
ever observed before. A slow pan gives the source almost no motion residual,
|
||||
so the `max(temporal_baseline, 1e-6)` denominator collapses and the ratio
|
||||
inflates. This is the predicted defect, now observed rather than argued.
|
||||
|
||||
## Measurements on the real carrier (2026-08-06)
|
||||
|
||||
The public Veo 3 off-road sample was downloaded and its sha256 matched the manifest
|
||||
exactly, so everything below is measured on the same file the 2026-07-31
|
||||
calibration used. Its properties, none of which the manifest recorded at the time:
|
||||
H.264 1920x1080 at 24 fps, 8.000 s, 5,155,063 bps video, plus an **AAC 48 kHz
|
||||
stereo 192,241 bps audio track of 7.914 s**.
|
||||
|
||||
The shipped profile therefore delivers 512x288 at 12 fps: **7.1% of the frame area
|
||||
and 3.6% of the pixels per second.**
|
||||
|
||||
### The geometry prize, measured end to end
|
||||
|
||||
Scored with `scripts/video_fidelity_probe.py` against the untouched 1920x1080
|
||||
source, all at `noise_std=0.15`:
|
||||
|
||||
| Geometry | Frame area | PSNR | SSIM | Temporal | Bitrate |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 512x288 @12 (shipped) | 7.1% | 25.3971 | 0.8215 | 1.0407 | 2187 kbps |
|
||||
| 768x432 @12 | 16.0% | 27.3796 | 0.8556 | 1.0323 | 3782 kbps |
|
||||
| 1024x576 @12 | 28.4% | 28.8542 | 0.8819 | 1.0262 | 5604 kbps |
|
||||
| 512x288 @24 | 7.1% | 25.6142 | 0.8270 | 1.0995 | 3181 kbps |
|
||||
|
||||
**512 to 1024 is worth +3.46 dB, against 1.92 dB for the entire `noise_std` axis.**
|
||||
The geometry lever is roughly twice the noise lever, which is the first hard number
|
||||
this investigation has for the size of the prize. What it still costs in removal is
|
||||
unknown: none of these was ever submitted.
|
||||
|
||||
Two readings that are not obvious from the table:
|
||||
|
||||
- **Temporal residual improves as resolution rises** (1.0407 to 1.0262). The shared
|
||||
noise field lives on the latent grid, so at 1920 it is four times finer relative
|
||||
to the frame than at 512, where it stretches into low-frequency blotches. Higher
|
||||
resolution plausibly makes the perturbation *less* visible, not more.
|
||||
- **The frame-rate arm looks bad here and the metric is the wrong judge.** 24 fps
|
||||
buys +0.22 dB and worsens the temporal ratio to 1.0995, but what 24 fps actually
|
||||
buys is smoothness, which neither PSNR nor SSIM measures at all. The metric sees
|
||||
only the cost. Do not decide the fps question from this table.
|
||||
|
||||
Encoder cost on an M-series MPS device, 8 seconds of source: 51 s at 512/12, 180 s
|
||||
at 768/12, 380 s at 1024/12, 190 s at 512/24. Native 1920x1080 at 24 fps
|
||||
extrapolates to roughly 46 minutes for the same 8 seconds, around 350x real time. A
|
||||
native run was started and killed before producing anything, so that figure remains
|
||||
an extrapolation.
|
||||
|
||||
### The certified row does not reproduce
|
||||
|
||||
Re-running the certified configuration (512/12/0.15, seed 0) on this machine gave
|
||||
25.5002 dB and 1.0675 against the manifest's 25.3911 and 1.0578, and a different
|
||||
output sha256. The manifest records neither device nor dtype, and the original was
|
||||
almost certainly CUDA fp16 while this was MPS fp32. **The oracle certified a
|
||||
specific file that cannot be rebuilt**, so any re-run produces an artifact the
|
||||
verifier never saw. Device and dtype belong in the schema.
|
||||
|
||||
### Audio passes through untouched, and the output still reads clean
|
||||
|
||||
Established on two carriers, by comparing the extracted audio bitstream sha256 of
|
||||
input and output:
|
||||
|
||||
- Veo carrier: `674063df...` in, `674063df...` out.
|
||||
- A Lyria 3 track generated for this purpose: `6ee4c041...` in, `6ee4c041...` out.
|
||||
|
||||
The Lyria case is the sharper one because its provenance is known. Google's own
|
||||
generator stamps it: `get_ai_metadata` reads a C2PA manifest naming
|
||||
`Google C2PA Core Generator Library`, issuer `Google LLC`, `trainedAlgorithmicMedia`.
|
||||
After the shipped path runs, that output reports `{}` -- every metadata marker
|
||||
stripped, `remaining_metadata` empty, visuals regenerated -- while the audio the
|
||||
generator produced is bit-for-bit intact.
|
||||
|
||||
So the pipeline can produce a file that **our own detector calls clean while an
|
||||
untouched Google-generated audio track sits inside it**. The mechanism is proven;
|
||||
whether that audio carries an audible-to-the-verifier SynthID mark is not, and only
|
||||
the oracle can say. If it does, this is a shipped defect rather than a quality
|
||||
question, and it outranks everything else in this document.
|
||||
|
||||
Worth recording alongside it: the Veo carrier itself returns `{}` from
|
||||
`get_ai_metadata`. The public sample carries no C2PA at all, which is exactly why it
|
||||
needed an oracle verdict in the first place.
|
||||
|
||||
## State when this work was parked (2026-08-06)
|
||||
|
||||
Landed in `main` through PR #78: the manifest-backed default pin, the
|
||||
scaling-factor gate and its single seam, the sweep loading through the engine's
|
||||
loader, `scripts/video_fidelity_probe.py` with its tests, the extended manifest
|
||||
schema, and this document. **The shipped operating point did not move**: it is
|
||||
still 512 px / 12 fps / `noise_std=0.15`.
|
||||
|
||||
A product requirement arrived late and reframes the target: output resolution,
|
||||
frame rate and bitrate must not be reduced. Resolution, frame rate and crf are
|
||||
parameters and can be preserved, but zero quality loss is not achievable by this
|
||||
mechanism at all -- removal works by rebuilding pixels, so a regeneration can never
|
||||
return its input. The honest goal is to preserve container properties and make the
|
||||
perceptual loss imperceptible, judged by DISTS and VMAF rather than PSNR. Two
|
||||
consequences follow: crf 18 on this path against crf 14 on the visible path is an
|
||||
unexplained asymmetry and a free quality win, and the oracle's 100 MB limit means a
|
||||
higher-quality profile is *harder* to certify than the one it replaces.
|
||||
|
||||
Nothing is blocked on analysis. Six oracle submissions were prepared and never run,
|
||||
because file upload to the verifier failed at the tool level:
|
||||
|
||||
1. A Lyria carrier and the pipeline's output from it, answering the audio question.
|
||||
2. A multiplexed control-plus-candidate file at the shipped geometry, validating
|
||||
whether the verifier reads a concatenated pair by time range. Verified locally
|
||||
frame by frame: half one is bit-identical to the control, half two to the
|
||||
candidate.
|
||||
3. Three more multiplexed pairs for 768, 1024 and 24 fps, worth three queries
|
||||
instead of six if the instrument validates.
|
||||
|
||||
Every generated artifact for those submissions has since been lost with its
|
||||
scratch directory, which is the intended lifecycle for generated media. Rebuilding
|
||||
costs one download and roughly 25 minutes of encoding; the Lyria track is the
|
||||
exception, since regenerating it would produce a different track, and the copies
|
||||
that survive are the only ones.
|
||||
|
||||
## The single most important unknown
|
||||
|
||||
**How much the 512 px downscale contributes to removal.** Every quality gain routes
|
||||
through this question, and the critics moved it from "probably free" to "direction
|
||||
unknown", which is exactly what makes it the highest-information experiment
|
||||
available.
|
||||
|
||||
## Ranked experiment program
|
||||
|
||||
Ranked by information per oracle query. The budget is roughly 10 checks per 24
|
||||
hours (MEASURED), so the program is paced by calendar, not by money.
|
||||
|
||||
### E0. The zero-oracle tier (0 submissions) - do this first
|
||||
|
||||
Not an experiment on the oracle; the precondition that makes everything else
|
||||
interpretable.
|
||||
|
||||
- `ffprobe` the carrier and record `source_width`, `source_height`, `source_fps`.
|
||||
The actual downscale factor of the existing rows is currently unrecoverable.
|
||||
- Extend the manifest schema: variant/control, source geometry, `vae`, `track`
|
||||
(audio|visual), verdict state in {DETECTED, NOT_DETECTED, INDETERMINATE,
|
||||
REFUSED}, verbatim verdict text, detected time range for the output,
|
||||
`session_id`, content stratum.
|
||||
- **Measure the `noise_std = 0.0` round trip locally** on the same carrier. This
|
||||
converts the inferred 27.2 dB ceiling into a measurement and bounds the whole
|
||||
noise axis for one GPU pass and zero oracle cost.
|
||||
- Freeze the current defaults' metrics on a fixed clip set, matching the
|
||||
record-then-diff rule in `.claude/rules/development.md`.
|
||||
- Determine whether the carrier has an audio track and whether it carries SynthID.
|
||||
|
||||
### E1. Instrument validation: multiplexing and a session anchor (1-2 submissions)
|
||||
|
||||
The verifier reports time ranges, so one file can carry several doses. A 24-second
|
||||
file of `[control 8 s | certified 0.15 8 s | control 8 s]` should read detected on
|
||||
the outer segments and not detected on the middle one, both halves already known
|
||||
from the manifest.
|
||||
|
||||
`encode_video_frames` requires matching dimensions and one frame rate, so
|
||||
multiplexing works **only along the `noise_std` axis**, not across geometries: one
|
||||
multiplexed file per geometric envelope.
|
||||
|
||||
Separately, submitting the existing detected file first in each session costs 10%
|
||||
of the quota, turns "the oracle may have changed" into a per-session gate, and
|
||||
measures the flip rate on a byte-identical file, a quantity every plan silently
|
||||
assumes is zero.
|
||||
|
||||
### E2. Resolution ladder at fixed dose - the decisive axis (2 submissions per rung)
|
||||
|
||||
A fixed value, not a ceiling. Rung 1 is `long_side = 768` with fps held at 12 so
|
||||
exactly one destruction axis moves. Jumping straight to 1920 is a bad first rung:
|
||||
a 3.75x jump makes a detected verdict uninformative about where the boundary lies.
|
||||
|
||||
- **Matched control:** 768 / 12 / `noise_std = 0.10`, expected DETECTED. It is
|
||||
strictly stronger than a plain re-encode control because it discharges the
|
||||
envelope, the VAE round trip, and a nonzero dose at once, and it diffs directly
|
||||
against the existing 512 row.
|
||||
- **Candidate:** 768 / 12 / 0.15.
|
||||
- **Positive:** the resolution axis is open; next rungs 1024 and then native with a
|
||||
clamp, 2 submissions each.
|
||||
- **Negative:** the downscale contributes to removal. This is the most valuable
|
||||
available negative: it kills the native-resolution proposals outright and turns
|
||||
the question into how much dose must be spent to buy resolution back, at the
|
||||
known exchange rate B = 2815.
|
||||
- Watch for a monotonicity violation: 768 / 0.10 reading NOT_DETECTED would
|
||||
overturn the critics' spectral argument.
|
||||
|
||||
### E3. Frame rate at fixed geometry and dose (2 submissions)
|
||||
|
||||
Control 512 / 24 / 0.10 expected DETECTED; candidate 512 / 24 / 0.15 expected
|
||||
not detected. The mechanism is provable from source, so the prior is high, but the
|
||||
only counter-mechanism - per-frame count aggregation - is probabilistic and n = 1
|
||||
on an 8-second clip does not measure it. Certify on the longest clip that fits the
|
||||
90 s / 100 MB limits, because the risk compounds with frame count.
|
||||
|
||||
Ranked below E2 because the outcome is nearly predetermined and the prize (judder)
|
||||
is smaller than the geometry prize. It is the safest bet if a guaranteed win is
|
||||
wanted.
|
||||
|
||||
### E4. `noise_std = 0.0` as an oracle probe (1 submission) - deliberately low rank
|
||||
|
||||
The local half of this probe is the cheapest high-information measurement in the
|
||||
program and already sits in E0 at zero oracle cost. The **oracle** half ranks low:
|
||||
it buys no quality by construction, and its likely DETECTED outcome is nearly
|
||||
deducible from the 0.10 row under monotonicity. Its one real value is that
|
||||
NOT_DETECTED here would be non-monotone against 0.10 and would refute the
|
||||
assumption underneath every ladder and bisection in this program. Worth one query,
|
||||
after E2 and E3.
|
||||
|
||||
### E5. Stratified certification of the surviving operating point (15-30 submissions)
|
||||
|
||||
Only after E2 and E3 produce a winner. Zero detections across at least 15
|
||||
stratified carriers (at least 3 per stratum: face closeup, fast motion, flat sky,
|
||||
dark night, text overlay) bounds the failure rate at 18.1%; 30 carriers bound it at
|
||||
9.5%. Add the project's 1.5x margin convention, at least 2 seeds, at least 2
|
||||
sessions, and per-track verdict recording.
|
||||
|
||||
For the flat-sky stratum, check the source reads detected first: Google documents a
|
||||
"not enough details to watermark" state, so a clean reading there may say nothing
|
||||
about removal.
|
||||
|
||||
## Surviving change candidates
|
||||
|
||||
**S1. Fix the metric's reference frame (0 oracle submissions).** Add
|
||||
`source_psnr_db` (candidate upscaled back to native against the untouched source
|
||||
frame) **and** a separate post-mux pass that decodes the delivered file. The
|
||||
critics killed the original claim that this would expose crf 18: the candidate is
|
||||
captured at `:423`, before `frame_pipe.write` at `:430`, so no in-loop metric can
|
||||
see the codec. Only the post-mux pass covers resize, decimation, crf, and mux
|
||||
together. Use `INTER_AREA` or a fixed analytic size for the upscale so the CPU
|
||||
resize does not dominate the loop, and update both other consumers of the
|
||||
generator in the same change (`read_sampled_frames` at `:188-209`, and
|
||||
`scripts/video_synthid_sweep.py:147` where `np.stack(frames)` must keep receiving
|
||||
resized frames). Do not redefine the existing `psnr_db`: the two manifest rows stay
|
||||
comparable to each other only while that field keeps its exact current meaning.
|
||||
|
||||
**S2. Frame rate as a fixed certified value (after E3).**
|
||||
`DEFAULT_VIDEO_SYNTHID_FPS` 12.0 to 24.0, keeping `min(fps, source_fps)`. A
|
||||
ceiling of 60.0 was **rejected**: it makes the delivered operating point a function
|
||||
of the user's source, so one sample from a family gets certified while the rest
|
||||
ship uncertified, and the risk direction is unfavorable. Port the VFR/PTS bridge
|
||||
(`probe_video_timestamps`, `timestamped_input`, currently only in
|
||||
`video_visible.py`): at native frame rate the output stops reading as a proxy, and
|
||||
silent CFR-ification becomes a master-quality defect. Add `fps` and `long_side`
|
||||
pins next to `tests/test_video_invisible.py:285`, tied to the certifying manifest
|
||||
row.
|
||||
|
||||
**S3. Clamp and align `_fit_size` (0 oracle submissions, with a caveat).**
|
||||
`scale = min(1.0, long_side / max(width, height))` at `:88`; align the long side
|
||||
and derive the short side from the true aspect, rounding to the nearest multiple of
|
||||
8. The pinned `_fit_size(1280, 720, 512) == (512, 288)` survives. The caveat: the
|
||||
clamp is an **uncertified operator change for a whole class of users**, since a
|
||||
320x240 source is upscaled to 512x384 today and would run at 320x240 afterwards,
|
||||
and neither has been tested. Ship it as its own documented change, not as a
|
||||
drive-by fix.
|
||||
|
||||
**S4. Extend the manifest schema and oracle protocol (0 submissions).** A
|
||||
precondition for the whole program. Record the verbatim verdict text, because an
|
||||
unclear state logged as not detected is exactly the silent regression the protocol
|
||||
exists to prevent.
|
||||
|
||||
**S5. A sigma normalization contract (0 submissions, applies to the current path).**
|
||||
No VAE swap survived, but two elements are real risks today: `vae.config.scaling_factor`
|
||||
cannot be trusted, so log `latents.std()` at `:276` and assert the effective
|
||||
scaling factor at load; and any cross-model sigma transfer must match on the **RMS
|
||||
of the decoded pixel perturbation**, not on latent units.
|
||||
|
||||
**S6. Additive texture masking - contingency on an E2 negative only.** Not energy
|
||||
preserving: the certified sigma stays as a floor everywhere and the mask only adds
|
||||
on top in textured regions. The original energy-preserving form was refuted - the
|
||||
mechanism normalizes and then clips, which breaks the claimed invariant by a
|
||||
content-dependent amount, and the proposed unit test asserted the invariant before
|
||||
the clip and would have stayed green. Energy preservation is also unsafe in
|
||||
principle: the detector does not need a full frame, and the verifier reports
|
||||
per-segment, so a flat region perturbed only at the floor is a crop carrying nearly
|
||||
the full carrier. The salvaged version has **no standalone PSNR win**; its only
|
||||
value is freeing distortion budget to spend on resolution if E2 shows the dose must
|
||||
rise. It additionally needs motion compensation for the mask (flow is computed at
|
||||
`:442`, after the decode at `:423`, so the loop must be restructured) and one
|
||||
deliberately bad candidate so `temporal_residual_ratio` acquires a known failing
|
||||
value: it has only ever been observed in [1.0072, 1.0578], so "the ratio looks
|
||||
fine" is currently an unfalsified claim.
|
||||
|
||||
## Refuted proposals - do not resurrect
|
||||
|
||||
- **Carrier-scale, resolution-invariant noise field.** "Carrier scale" is set by
|
||||
our VAE's latent pitch, not the carrier's. The motivating arithmetic treats a
|
||||
field on the latent grid as white at pixel resolution and is wrong by exactly the
|
||||
square of the scale factor: white noise on a 240-wide latent already spans
|
||||
0-120 cycles per frame, entirely inside the band the decoder reproduces. The
|
||||
construction actually band-limits the perturbation from 0-120 to 0-32 cycles,
|
||||
making it smoother - the first thing any spread-spectrum extractor's content
|
||||
suppression removes - while moving its energy toward the peak of the contrast
|
||||
sensitivity function, making it more visible. Bilinear upsampling is also
|
||||
heteroscedastic, and global renormalization by `noise.std()` stamps a visible
|
||||
amplitude lattice rather than fixing it.
|
||||
- **Native resolution plus tiling plus a carrier-scale field.** Inherits the
|
||||
arithmetic above, concedes that native processing destroys less carrier without
|
||||
proposing a replacement mechanism, and bundles four destruction axes into one
|
||||
candidate on a 4-query budget so a detected verdict is undecomposable. Its cost
|
||||
model was also wrong by roughly 10x. The salvageable parts moved into S1 and S3.
|
||||
- **`AutoencoderKLTemporalDecoder`.** Refuted independently by three critics.
|
||||
Removal is a property of the encode-decode composition, so a strictly more
|
||||
faithful decoder preserves more carrier at the same sigma. The class implements
|
||||
neither slicing nor tiling, so `enable_slicing()` at `:135` raises
|
||||
`NotImplementedError` and the documented bounded-memory property dies. Its
|
||||
temporal receptive field needs windows of tens of frames, its license carries a
|
||||
revenue restriction incompatible with an Apache-2.0 default, and its own
|
||||
published table shows a **worse** FID (9.17 against 7.61).
|
||||
- **Wan or another temporal video VAE.** Hard refusal in code: `AutoencoderKLWan._encode`
|
||||
uses `iter_ = 1 + (num_frame - 1) // 4`, so at the shipped `batch_size = 4` only
|
||||
frame 0 is encoded and a 4-frame batch decodes back to 1, which makes
|
||||
`zip(..., strict=True)` at `:429` raise. The proposed remedy of carrying the
|
||||
causal feature cache across chunks is impossible through the public API, since
|
||||
`clear_cache()` runs on both entry and exit of `_encode` and `_decode`.
|
||||
`float(vae.config.scaling_factor)` at `:270` and `:292` also raises, because Wan
|
||||
exposes `latents_mean`/`latents_std` instead. Logically, feeding 4:1 temporal
|
||||
compression frames 83 ms apart either reconstructs well (carrier preserved, no
|
||||
removal gain) or hallucinates (no quality prize); the two claims are mutually
|
||||
exclusive.
|
||||
- **A 16-channel f8 VAE.** Self-refuted and confirmed by the critics. Quadrupling
|
||||
latent channels roughly doubles L in Zhao's theorem, so sigma must roughly double
|
||||
to hold removal, and the VAE frontier in the UnMarker results is monotone with no
|
||||
published point where a more faithful autoencoder removes **more**. Predicted net
|
||||
loss of 2-7.5 dB by its own arithmetic. Its config also carries
|
||||
`scaling_factor = 0.2614`, so an unchanged `noise_std = 0.15` is about 30% weaker
|
||||
in raw latent units. Only the normalization contract survived, as S5.
|
||||
- **A per-channel luma/chroma probe.** The arms match in absolute latent units but
|
||||
not in relative dose per channel, since `scaling_factor` normalizes the aggregate
|
||||
latent, so the very reading the probe exists for is confounded with dose. The
|
||||
chroma arm is additionally subsampled by the yuv420p encode, which no current
|
||||
metric sees. Maximum prize under 1 dB by its own fit.
|
||||
- **Post-processing with unsharp plus grain, as a shipped stage.** The
|
||||
carrier-reimport argument is sound and was verified line by line: `unsharp_mask`
|
||||
reads only its own argument ([`humanizer.py:79`](../src/remove_ai_watermarks/humanizer.py)),
|
||||
and `adaptive_polish` touches the source only through one float. But reimport was
|
||||
never the binding risk:
|
||||
`cv2.addWeighted(img_f, 1.0 + amount, blurred, -amount, 0.0)` at
|
||||
`_ADAPTIVE_MAX_UNSHARP = 1.0` multiplies the **surviving carrier residue** by up
|
||||
to 2x as the terminal operation before encoding, and whether that re-arms a given
|
||||
file depends on that file's unobservable margin. With no local decoder and only a
|
||||
sampled oracle the stage is structurally uncertifiable, not merely expensive to
|
||||
certify. The only rescue is moving sharpening **above** the VAE stage so the
|
||||
removal operator stays terminal. Grain, which monotonically lowers detector SNR,
|
||||
is the safe half and can be separated.
|
||||
|
||||
## Measuring quality properly
|
||||
|
||||
The current `psnr_db` cannot show the improvement this work exists to produce.
|
||||
Minimum upgrade:
|
||||
|
||||
1. **`source_psnr_db`** - candidate upscaled back to native against the untouched
|
||||
source frame. Good for ranking configurations against a common reference,
|
||||
dominated by unrecoverable high-frequency content, so not a measure of what the
|
||||
VAE costs.
|
||||
2. **A post-mux end-to-end pass** - decode the delivered file after
|
||||
`mux_encoded_video` (`:456`) and compare against the source. The only measure
|
||||
covering resize, decimation, crf, and mux together. Keep it out of the streaming
|
||||
loop so `test_stream_batches_consumes_only_one_batch_ahead` stays valid.
|
||||
Implemented as `scripts/video_fidelity_probe.py`, which streams, reports the
|
||||
delivered file's bitrate, and shares the engine's frame-selection rule rather
|
||||
than copying it.
|
||||
3. **DISTS** (arXiv:2004.07728), built to tolerate texture resampling - exactly
|
||||
what the VAE does to foliage, skin, and grass, and exactly what PSNR and LPIPS
|
||||
punish even when the result is perceptually equivalent. It separates "the VAE
|
||||
resampled the grass" from "the VAE destroyed an edge".
|
||||
4. **VMAF** through `ffmpeg` `libvmaf`, cheapest to add, whose ADM/DLM feature
|
||||
names this exact complaint. Its temporal term is only a mean absolute luma
|
||||
difference between neighboring frames, so it is not a flicker detector: keep the
|
||||
motion-compensated ratio.
|
||||
5. **Encoded file size as a third axis.** Every current metric is taken before the
|
||||
pipe, so a bitrate explosion currently reads as "quality did not suffer". This
|
||||
matters most for any perturbation that varies frame to frame: at fixed crf it
|
||||
raises bitrate rather than lowering quality.
|
||||
6. **`temporal_residual_ratio` repairs**: add a p95 across frames next to the mean
|
||||
so localized flicker stops being averaged away, add a long-horizon term against
|
||||
the first frame, and replace `max(temporal_baseline, 1e-6)` with an explicit
|
||||
undefined result on static shots.
|
||||
7. **A no-reference metric** (DOVER or FAST-VQA) for the case where no
|
||||
full-reference measure can compare a 512/12 output against a 1024/24 output on
|
||||
one scale. Relative ranking within a sweep only; absolute values are
|
||||
uncalibrated for this artifact class.
|
||||
|
||||
Nothing local measures removal. `remove_video_invisible` checks `get_ai_metadata`
|
||||
on the output, which is metadata, not pixels. Only a manifest row counts.
|
||||
|
||||
## Risks
|
||||
|
||||
**Error asymmetry is the governing constraint.** Shipping a leak means a user
|
||||
receives a watermarked file believing it is clean, with no local pixel decoder to
|
||||
catch it and no feedback path that would surface it. Staying conservative means a
|
||||
user receives 512 px / 12 fps, a cost that is visible, bounded, and reversible with
|
||||
an explicit flag. A symmetric test is therefore inadmissible, and the burden of
|
||||
proof sits entirely on the new default.
|
||||
|
||||
**n = 1.** The current default is certified by one success on one carrier with one
|
||||
seed. Any quality change inherits that weakness and must not deepen it.
|
||||
|
||||
**Content and seed dependence** is MEASURED on this project's image branch:
|
||||
survivors switch by content type, and near the threshold the same input flipped
|
||||
between runs on seed alone.
|
||||
|
||||
**Oracle instability.** Three verdict states, not two. Per-segment reporting.
|
||||
Separate audio and visual tracks - and the pipeline byte-copies audio, so a Veo 3
|
||||
clip with generated sound leaves with its audio SynthID intact, and the manifest
|
||||
cannot even record which track the 2026-07-31 negative referred to. That is
|
||||
potentially a shipped product hole, not only an experimental confound. Session
|
||||
drift is currently unfalsifiable.
|
||||
|
||||
**Observability narrows exactly where quality rises.** At fixed crf the bitrate
|
||||
grows with pixels per second, so the maximum duration fitting under 100 MB falls as
|
||||
fps and resolution rise. The configuration users actually receive becomes **less**
|
||||
observable than the one it replaces. That is a permanent property, not an
|
||||
inconvenience.
|
||||
|
||||
**Ceilings instead of fixed values.** Any ceiling makes the delivered operating
|
||||
point a function of the user's source, and with it the perturbation's cycles per
|
||||
frame - the quantity carrying the certified margin. Ship fixed values.
|
||||
|
||||
**Documentation and test surface.** Only `DEFAULT_VIDEO_SYNTHID_NOISE_STD` is
|
||||
pinned. Moving `long_side` or `fps` touches hardcoded numbers in `README.md`,
|
||||
`docs/known-limitations.md`, `docs/cli.md`, `docs/python-api.md`,
|
||||
`docs/module-internals.md`, `docs/synthid.md`, and `docs/verification-plan.md`.
|
||||
Drift-prone operational numbers should not live in seven places of prose; their
|
||||
source of truth is the manifest.
|
||||
|
||||
**Silent configuration drift.** `sd-vae-ft-mse/config.json` carries no
|
||||
`scaling_factor`, so 0.18215 is a class default under an upper-unbounded
|
||||
`diffusers>=0.38.0` while `maintain.sh` runs `uv-outdated`. A library bump can move
|
||||
the certified operating point with a green suite.
|
||||
|
||||
**Missing guards that get worse at native resolution.** There is no HDR or >8-bit
|
||||
rejection and no VFR/PTS bridge on the invisible path. A 512 px output plainly
|
||||
reads as a proxy; a native-resolution output reads as a master, and the cost of
|
||||
silently flattening a 10-bit PQ source to 8-bit SDR rises accordingly.
|
||||
|
||||
**Do not bundle axes.** Raising `long_side`, raising `fps`, and lowering crf all
|
||||
reduce total destruction and all require recertification. The metric reference-frame
|
||||
fix is the only exception, because it changes the measurement rather than the
|
||||
pixels. Do not put a crf 18 to 14 change in the same candidate as a resolution
|
||||
change.
|
||||
@@ -10,7 +10,7 @@ Who embeds what, and whether it is locally detectable (so we know which gaps are
|
||||
**Root cause and external confirmation (deep-research 2026-06-19, adversarially verified).** This is the SCHEME's ceiling, not our usage — there is no better decoder to adopt. The imwatermark maintainers state verbatim (both the ShieldMnt and Stability-AI READMEs) that the algorithm "cannot guarantee to decode the original watermarks 100% accurately even though we don't apply any attack." Independent measurement (WMAdapter, arXiv:2406.08337 Table 2) puts dwtDct at only **~0.79 bit accuracy on CLEAN images (~38/48 bits — already below our 44 gate)**, collapsing to ~0.50 (chance) under crop/JPEG. Two code-verified + locally-reproduced mechanisms drive the content-dependent failures: (1) the decoder reads each bit as the **highest-magnitude DCT coefficient per block**, so any content coefficient exceeding the encoded target flips the bit; (2) the default embed is in the **YUV chroma channel, which 8-bit-clamps on white/bright pixels** (a +36 chroma delta survives a white-fill round-trip as only +4, ~89% loss) — this is the mechanism behind the bright-flat / minimalist failures and the all-ones degenerate decode. No maintained fork or detector decodes this scheme reliably: the WAVES benchmark (arXiv:2401.08573) relegates DWT-DCT to supplementary appendix G.5 and targets Stable Signature / Tree-Ring / StegaStamp instead; learned encoder/decoder schemes reach ~0.98-0.99 clean but are a DIFFERENT watermark class (not what SDXL/FLUX stamp). `dwtDctSvd` does not help (SDXL embeds `dwtDct`; dwtDctSvd cannot decode it, and its clean accuracy ~0.72 is lower). **Authoritative conclusion: the open DWT-DCT mark cannot be turned from positive-only into a reliable real-world detector; keep it positive-only and rely on C2PA.** (Refuted along the way: that the library is unmaintained, and that it is robust to JPEG but only fails on geometric attacks — both did not survive verification.)
|
||||
|
||||
Consequence for the FLUX hosted-output question (BFL Playground, FLUX.2 [pro] + FLUX.1 [dev], 2026-06-19): all samples carry the signed C2PA manifest (issuer "Black Forest Labs"); the open DWT-DCT decode returned `None`, but every available FLUX carrier (textured fox AND a minimalist-flat generation) failed the positive control (28/48), so the detector is blind on them and **whether BFL hosted output embeds the open pixel watermark is UNRESOLVED** (an earlier note here wrongly asserted it absent — overstated; a later note blamed "high texture" — also wrong, flat carriers fail too). What IS established: C2PA is the reliable FLUX identifier; the `_BITS_48` pattern is correct (round-trips on chatgpt/firefly/random). Resolving the hosted question needs a hosted FLUX carrier that first passes a >=44/48 positive control, which neither a textured nor a flat prompt produced — low priority (the open mark is only a stripped-metadata fallback).
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Copilot + Designer; Bing Image Creator collected 2026-05-24 still signs as "Microsoft" and now runs **MAI-Image**, NOT OpenAI/DALL·E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
|
||||
**Samsung Galaxy AI** signs supported edits with C2PA and may carry the
|
||||
proprietary `genAIType` marker. The registered visible detector covers the
|
||||
@@ -22,8 +22,15 @@ icon-only variants need separate calibrated silhouettes.
|
||||
|
||||
**Black Forest Labs (FLUX)** API output signs C2PA: `claim_generator_info "Black Forest Labs API"` + a `c2pa.ai_generated_content` assertion + `trainedAlgorithmicMedia` (issuer `b"Black Forest Labs"` added to `C2PA_ISSUERS`, platform "Black Forest Labs (FLUX)").
|
||||
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
|
||||
Some applications sign C2PA through an upstream model or infrastructure
|
||||
provider. For AI claims, exact product mappings in `claim_generator` therefore
|
||||
take precedence over issuer attribution. Supported mappings include Higgsfield
|
||||
AI, Topaz Labs Image API, and TikTok Ad Creative Toolbox; an unknown claim
|
||||
generator still falls back to the certificate issuer.
|
||||
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance Volcano Engine"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance Dreamina" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
|
||||
- **EXIF/XMP/PNG-text generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal). Additional verified generator stamps include **NovelAI** (`Software`, `Source`, and `Title` PNG text chunks), **Reve** (`Software` or XMP `CreatorTool` = `reve.com`), and **Aphrodite AI** (`Make` or `Software` = `Aphrodite AI`).
|
||||
- **App-export provenance and AIGC JSON:** supported ByteDance-family exports can place a JSON object in EXIF `ImageDescription` or `UserComment`, independently of C2PA or TC260. Exact `product` values for Doubao, Xinghui, and Dreamina are removable product provenance, but do not alone prove that the pixels were generated. Dreamina additionally requires `exportType=generation` for that verdict. A nested Aweme `aigc_type=1` or private ByteDance `aigc_label_type=1` / `2` is an AIGC disclosure; `0` is inconclusive and can occur on a Dreamina generation export. Plain Aweme, retouch, and `lv` exports are preserved. The lower-case private field is deliberately not interpreted as the normative TC260 `Label`, whose values `1` / `2` / `3` mean generated / possibly generated / suspected generated under [GB 45438-2025](https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf).
|
||||
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
|
||||
|
||||
Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC** — only EXIF `Artist` = a UUID and EXIF `ImageDescription` = `Signature: <base64>` (a crypto signature, unverifiable locally without xAI's public key). This empirically kills the earlier unverified "xAI signs C2PA as xAI" lead — xAI is not even a C2PA member. `exif_generator` misses it (neither field holds an `AI_GENERATOR_TOKENS` token), so a dedicated detector `xai_signature(path)` matches the pair (`ImageDescription ~ ^Signature: [A-Za-z0-9+/=]{64,}` AND UUID `Artist`); wired into `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify` (signal `xai_signature`, platform "xAI (Grok / Aurora)").
|
||||
@@ -36,9 +43,9 @@ AI generator values while retaining unrelated camera and editor EXIF. The
|
||||
shared `xai_signature_pair` helper is the single source of truth for the
|
||||
pair. On the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
|
||||
in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`).
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). The label says only "this is AI", but its `ContentProducer` names the signing entity — `001` + `1` + an 18-char Unified Social Credit Code + a 5-digit product suffix, normalized by `metadata.uscc_of`, or for a few generators a bare product name. `KnownMark.tc260_producer_codes` maps the codes settled per vendor by `scripts/vendor_cohort_harvest.py` to registry mark keys, so an AIGC image relaxes the detector of the vendor it actually carries rather than a guessed pair; an unmapped or absent producer falls back to ByteDance's two products. A code identifies a legal entity, not necessarily one brand, so a hosting or aggregating platform that signs for several apps is a recall bet rather than a proof.
|
||||
|
||||
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item. The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
|
||||
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item; Doubao's iOS export additionally writes two QuickTime-form variants (a bare `meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list whose `ilst` data items carry the JSON with no key name) — both verified on retained carriers (2026-08-17). The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
|
||||
|
||||
Native MKV/WebM is a sixth serialization. TC260-PG-20257A stores
|
||||
`TagName=AIGC` and the raw JSON `TagString` in
|
||||
@@ -51,24 +58,25 @@ defines two more native serializations. AVI stores an `AIGC` child in
|
||||
bounded RIFF and FLV readers validate the JSON field set and skip media
|
||||
payloads. Removal remuxes either container through ffmpeg with stream copy.
|
||||
|
||||
- **HuggingFace-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** HuggingFace Jobs / Spaces can stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID. It marks the *hosting job*, not a model, so it lifts an Unknown verdict to a tentative AI via `hf_only` but never overrides a hard metadata signal. `_HF_JOB_CAVEAT` states the limit. Removal drops the chunk through the PNG metadata whitelist.
|
||||
- **Hugging Face-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** Hugging Face Jobs / Spaces can stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID. It marks the *hosting job*, not a model, so it lifts an Unknown verdict to a tentative AI via `hf_only` but never overrides a hard metadata signal. `_HF_JOB_CAVEAT` states the limit. Removal drops the chunk through the PNG metadata whitelist.
|
||||
- **No detectable signal on some downloads:** Recraft exports and some hosted
|
||||
FLUX surfaces can arrive without a supported local signal. Midjourney samples
|
||||
may carry IPTC metadata but no registered C2PA or pixel watermark. The open
|
||||
DWT-DCT decoder only applies when the producing pipeline actually ran its
|
||||
encoder and the carrier remains decodable.
|
||||
- **Invisible but NOT locally detectable (proprietary, API/oracle only — same wall as SynthID):** Amazon Titan Image Generator + Nova Canvas (Bedrock `DetectGeneratedContent` API), Kakao (new SynthID image adopter, May 2026), NVIDIA Cosmos (SynthID video). No local detector possible; treat like SynthID.
|
||||
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest (Digimarc/Imatag/Steg.AI/...) stay name-only (proprietary decoders).
|
||||
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, preserve the structured assertion's signed `value`, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest stay name-only unless a compatible decoder is independently verified.
|
||||
- **Microsoft Paint and Photos InvisMark (reverse-engineered 2026-08-20):** Paint receives a per-generation GUID from remote prompt moderation, embeds it into locally generated pixels, and records the same value in `c2pa.soft-binding` under `com.microsoft.invismark.1`. The C2PA soft-binding registry independently identifies that algorithm as Microsoft Responsible AI InvisMark for image and video. Paint's 144-bit writer framing does not match the public repository's 100-bit pretrained checkpoint interface, so compatibility is not assumed. The parser reports the signed identifier; there is no validated local pixel decoder. Microsoft's external Content Provenance Detection API is the removal oracle because it reports `Watermark` separately from `C2PA`; a metadata-stripped, pixel-identical control must remain watermark-positive before an output-negative result is attributed to pixel removal. The no-account web page for the same check is <https://ai.azure.com/nextgen/validate> (its `Inconclusive` verdict is weaker than the API's separate watermark result). Sources: https://xusheng.dev/posts/reversing/mspaint_invisible_watermark/main/, https://github.com/c2pa-org/softbinding-algorithm-list/blob/main/softbinding-algorithm-list.json, and https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection
|
||||
- **Built in the dated batch:** soft-binding vendor detection, IPTC Photo
|
||||
Metadata AI-disclosure fields, C2PA detection and stripping for supported
|
||||
ISOBMFF video, the optional Adobe TrustMark decoder, and temporally stabilized
|
||||
visible Sora, Veo, Seedance, Dola, Hailuo, and Kling removal. Other visible
|
||||
visible Sora, Veo, Seedance, Dola, Hailuo AI, and Kling AI removal. Other visible
|
||||
video logos and proprietary audio-watermark detection remain outside the
|
||||
package.
|
||||
Metadata stripping for supported audio containers is a separate implemented
|
||||
path.
|
||||
|
||||
**Box detection window — now handled (v0.6.8):** detection no longer relies on a fixed first-MB read. `metadata.scan_head(path, size)` reads the first `size` bytes and, for ISOBMFF, appends the payloads of late provenance boxes found by `isobmff.scan_c2pa_region` (a file-seeking top-level box walker that skips past `mdat` by size without reading it), so a C2PA/AIGC/IPTC manifest placed AFTER a large `mdat` in a streaming/non-faststart MP4 is now caught. Every C2PA/marker byte scan (`has_ai_metadata`, `aigc_label`, `iptc_ai_system`, `synthid_source`, `exif_generator` XMP, `get_ai_metadata` soft-binding, and `identify`) goes through `scan_head`; it is behavior-neutral for non-ISOBMFF inputs (exactly `f.read(size)`).
|
||||
**Box detection window — now handled (v0.6.8):** detection no longer relies on a fixed first-MB read. `metadata.scan_head(path, size)` reads the first `size` bytes and, for ISOBMFF, appends the payloads of late provenance boxes found by `isobmff.scan_c2pa_region` (a file-seeking top-level box walker that skips past `mdat` by size without reading it), so a C2PA/AIGC/IPTC manifest placed AFTER a large `mdat` in a streaming/non-faststart MP4 is now caught. Every C2PA/marker byte scan (`has_ai_metadata`, `aigc_label`, `iptc_ai_system`, `synthid_source`, `exif_generator` XMP, `get_ai_metadata` soft-binding, and `identify`) goes through `scan_head`; for PNG it likewise appends the payloads of `tEXt` / `iTXt` / `zTXt` / `eXIf` / `iCCP` chunks that start beyond the window (`_png_late_metadata`, seeking past `IDAT`), which is how a TC260 AIGC label appended after the pixel stream is caught; for WebP it appends the `EXIF` / `XMP ` / `ICCP` / `C2PA` chunks past the window (`_riff_late_metadata`, stepping over the coded image), which is how an IPTC "Made with AI" tag stored after the pixels is caught; and for a file at least `size` bytes long it finally appends the metadata text the decoder reaches but a raw read cannot (`_decoder_visible_text`), which covers a compressed PNG `zTXt` packet no byte scan can spell; for any file that fits inside `size`, it is exactly `f.read(size)`.
|
||||
|
||||
Native TC260 MP4/MOV tags do not live in those top-level provenance boxes.
|
||||
`tc260_aigc_payloads` separately seeks through `moov.udta.meta.keys/ilst`, so
|
||||
@@ -108,12 +116,12 @@ every frame, then requires an anchored temporal run. This extra anchor check
|
||||
was necessary because a moving clean scene detail could retain enough adjacent
|
||||
overlap to pass a recurrence-only gate.
|
||||
|
||||
**Hailuo and Kling use larger fixed composite labels.** Verified Hailuo exports
|
||||
carry a lower-edge waveform, `MINIMAX`, separator, Hailuo ring, and
|
||||
`hailuo AI` text. Verified Kling exports carry a bottom-right swirl,
|
||||
**Hailuo AI and Kling AI use larger fixed composite labels.** Verified Hailuo AI exports
|
||||
carry a lower-edge waveform, `MINIMAX`, separator, Hailuo AI ring, and
|
||||
`hailuo AI` text. Verified Kling AI exports carry a bottom-right swirl,
|
||||
`KLING AI`, a changing version suffix, and sometimes `PRO`. The detectors use
|
||||
only synthetic primitives and fonts. Hailuo expands the matched core to cover
|
||||
the complete composite. Kling combines a version-independent text core with a
|
||||
only synthetic primitives and fonts. Hailuo AI expands the matched core to cover
|
||||
the complete composite. Kling AI combines a version-independent text core with a
|
||||
synthetic ring rescue, then requires the recurring candidate to reach the
|
||||
expected frame edge and contain enough bright low-saturation pixels. Those
|
||||
extra gates were added after clean Luma and PixVerse scene details passed shape
|
||||
@@ -123,17 +131,17 @@ instead uses Florence-2 to identify arbitrary watermarks before LaMa
|
||||
inpainting. That is broader, but it carries a much heavier model and a less
|
||||
auditable detection boundary than the provider-specific synthetic path here.
|
||||
|
||||
**The faint-visible-mark precision/recall wall is fundamental, not a heuristic artifact.** The visible-watermark-detection literature has moved to LEARNED segmentation / object-detection (WDNet WACV'21 arXiv:2012.07616; SLBR ACM MM'21, open code+weights; the PRCV'18 large-scale detector; Su et al. survey 2025), but three verified findings bound what a learned detector actually buys: (1) a claim that a confidence threshold "cleanly separates" true from false matches even with a learned CNN front-end was **REFUTED** in verification (arXiv:1705.08593) -- the precision/recall wall persists even with learned features. (2) Learned detectors need a LARGE, pattern-diverse labeled dataset trained on synthetic composites (PRCV'18: 60k images / 80 watermark classes; CLWD: 60k / 160 marks), and off-distribution degradation is a documented real axis (models trained on limited-pattern LVW transfer worse; diversity of training patterns drives generalization). (3) Inference is cheap (WDNet ~8 ms at 256x256) -- the cost is the data pipeline, not runtime. Net: a learned detector shifts the frontier but does NOT remove the wall; for a SINGLE mark the cheapest next step is a small patch classifier (real-sparkle vs false-positive) on top of the existing NCC localizer, not a full segmentation model. SLBR is a ready baseline. The current NCC + false-positive gate (core-ring brightness margin + gradient-NCC crispness + white-core saturation) is a sound operating point, and the residual miss is the information-theoretic wall the literature confirms.
|
||||
**Learned detectors change the visible-mark precision/recall tradeoff; they do not establish a universal separator.** The visible-watermark-detection literature has moved to learned segmentation and object detection (WDNet WACV'21 arXiv:2012.07616; SLBR ACM MM'21, open code and weights; the PRCV'18 large-scale detector; Su et al. survey 2025). The cited arXiv:1705.08593 paper reports a method that significantly reduces false matches and eliminates them after rejecting a small fraction of matches on its electron-microscopy task; it does not prove an information-theoretic limit for watermark detection. Learned watermark detectors also rely on large, pattern-diverse labeled datasets built with synthetic composites (PRCV'18: 60k images / 80 watermark classes; CLWD: 60k / 160 marks), and transfer depends on the diversity represented in training. Inference can be cheap (WDNet reports roughly 8 ms at 256x256), while the labeled-data and evaluation pipeline remain the larger project cost. A learned detector is therefore a possible future operating point, not evidence that the current NCC gate is theoretically optimal.
|
||||
|
||||
**Visible-mark landscape beyond the registry.** Meta stamps a visible "Imagined with AI" mark (bottom-LEFT, a small symbol) on its OWN Meta AI / "Imagine" output; for third-party images it relies on C2PA / IPTC, not a visible mark. Samsung Galaxy AI additionally uses a **four-star icon** variant in a corner alongside the localized text wordmark `samsung_engine` calibrates (only the Italian text variant is covered) -- the icon is a distinct, uncovered variant. Every source agrees visible + metadata marks are trivially removable (crop / screenshot, ~2 s), which is the tool's premise.
|
||||
**Visible-mark landscape beyond the registry.** Since Muse Image (2026-07) Meta's own generation output carries no visible mark, only the invisible proprietary Content Seal; the legacy visible "Imagined with AI" mark belongs to the pre-2026 Meta AI / "Imagine" pipeline and remains unregistered (string verified, position not). For third-party images Meta relies on C2PA / IPTC, not a visible mark. The Samsung detector covers only the measured Italian text wordmark; no separate icon-only variant has been verified. Visible overlays and embedded metadata can usually be removed without changing the underlying generation model, which is the tool's premise.
|
||||
|
||||
**Regulatory driver -- China GB 45438-2025 is the strongest VISIBLE-mark mandate.** The CAC / TC260 "Measures for Labeling AI-Generated Synthesized Content" (issued March 2025, **effective 2025-09-01**, technical standard **GB 45438-2025**, building on the TC260 Aug-2023 practice guide) MANDATE a **visible** label for AI images -- a visible textual mark whose height must be **>= 5% of the image's shortest side** -- plus the metadata (implicit) label. Several such CJK text marks are now registered; see [supported signals](supported-signals.md) for the current list. By contrast EU AI Act Article 50 mandates only the MACHINE-READABLE mark (enforceable 2026-08-02, grace to 2026-12-02); a visible label is proposed and modality-specific (visible for images) but is NOT a hard "fixed icon" mandate -- a claim that Art 50 requires a clearly-visible fixed icon for images was refuted in verification. Primary-source dates verified against the article/standard text, not search summaries.
|
||||
|
||||
## Uncovered visible marks: implementation specs (deep-research 2026-07-18)
|
||||
|
||||
Compatibility testing showed that TC260-labelled images can still produce no visible-mark
|
||||
Compatibility testing showed that TC260-labeled images can still produce no visible-mark
|
||||
detection. The main causes were a fixed Doubao localization defect and genuinely
|
||||
uncovered vendors. Verification status is labelled per claim; treat (b)/(c) as leads,
|
||||
uncovered vendors. Verification status is labeled per claim; treat (b)/(c) as leads,
|
||||
not ground truth.
|
||||
|
||||
**GB 45438-2025 clause 5.2, the binding constraint for every Chinese mark (VERIFIED (a) -- full standard text extracted from the TC260-hosted PDF).** Verbatim requirements for an image's explicit label:
|
||||
@@ -145,7 +153,7 @@ not ground truth.
|
||||
|
||||
Two consequences we can exploit: (1) the 5% floor is a **scale prior** -- a compliant CN mark's glyphs are large (>= 51 px on a 1024² image), so a CN silhouette ladder can be anchored at ~5-10% of the short side instead of swept broadly, which should cut false fires; (2) every compliant string shares the tail `AI生成` / `AI合成`, so a shared suffix silhouette plus a per-vendor prefix may beat five independent templates. NOT established: whether 文字高度 means cap height, em box, or rendered bounding box (a ~1.3x spread in template scale). Sources: `https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf`, parent CAC measure `https://www.cac.gov.cn/2025-03/14/c_1743654684782215.htm` (the CAC text itself specifies no size or corner).
|
||||
|
||||
**Alibaba Qwen -- two surfaces that differ (API tier VERIFIED (a)).** Model Studio docs state verbatim that the API adds a `Qwen-Image` watermark 在图像右下角 and 默认值为 false -- so API output is **unwatermarked by default**, and when enabled the mark is a LATIN wordmark, not CJK, and not GB-compliant in wording. The consumer app's `千问AI生成` (bottom-right) is (b) secondary only -- no Alibaba primary page states it. So Qwen needs TWO templates, and its absence is never evidence of a clean image. Source: `https://help.aliyun.com/zh/model-studio/qwen-image-api`.
|
||||
**Alibaba Cloud Qwen -- two surfaces that differ (API tier VERIFIED (a)).** Model Studio docs state verbatim that the API adds a `Qwen-Image` watermark 在图像右下角 and 默认值为 false -- so API output is **unwatermarked by default**, and when enabled the mark is a LATIN wordmark, not CJK, and not GB-compliant in wording. The consumer app's `千问AI生成` (bottom-right) is (b) secondary only -- no Alibaba primary page states it. So Qwen needs TWO templates, and its absence is never evidence of a clean image. Source: `https://help.aliyun.com/zh/model-studio/qwen-image-api`.
|
||||
|
||||
**星绘 is ByteDance (VERIFIED (a): Baidu Baike + App Store listing, now branded 豆包旗下, team folded into Doubao April 2025).** So `星绘AI生成` is very likely the Doubao house style -- same typeface, same corner, possibly the same top-left `AI生成` pill. Starting from the Doubao `TextMarkConfig` and swapping the two lead glyphs is the cheap path. String/position themselves are (c) inferred.
|
||||
|
||||
@@ -153,9 +161,11 @@ Two consequences we can exploit: (1) the 5% floor is a **scale prior** -- a comp
|
||||
|
||||
**Tencent Yuanbao: RESOLVED 2026-07-25, registered (`yuanbao_engine.py`).** The standard mark is a compact two-line italic `元宝` over `AI生成` block at bottom-right. It switches between light and dark strokes with the scene, so detection uses polarity-independent local contrast rather than a white top-hat. The separate one-line overlay variant remains evidence-limited to one example.
|
||||
|
||||
**Meta `Imagined with AI` (string VERIFIED (a) from Meta's own newsroom; POSITION NOT VERIFIED).** Sources conflict on placement. Do not encode a corner without a verified sample. `identify` reads the supported IPTC disclosure; it does not decode Meta's proprietary invisible watermark. Source: `https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/`.
|
||||
**Meta Muse Image / Content Seal: current state VERIFIED 2026-08-26 against live artifacts.** Muse Image (launched 2026-07-07, first image model from Meta Superintelligence Labs) puts no visible mark on output. Every output carries Content Seal, a proprietary invisible pixel watermark; research lineage is open (`github.com/facebookresearch/content-seal`, Pixel Seal / VideoSeal / Watermark Anything), but the deployed implementation is proprietary and unpublished. API outputs and Meta CDN copies also carry XMP `iptcExt:DigitalSourceType = trainedAlgorithmicMedia` (a standard IPTC code, not Meta-exclusive). The only reader is the anonymous web oracle `meta.ai/identification`, which returns model attribution plus a per-generation ID and timestamp embedded in the payload. Verified robustness (our corpus, `data/contentseal/`): seal survives CDN WebP transcode, 512 px resize, JPEG q85, and full metadata stripping; it is lost to center crops of 33-50% linear size (Reuters measured 55% detector misses on cropped images, 2026-07-11). Removal is fully supported and oracle-verified on the default profile, with a measured Meta strength floor (0.1) shipped behind auto IPTC routing and an explicit `--vendor meta` override; see [supported signals](supported-signals.md) and [module internals](module-internals.md). Legacy paragraph retained below.
|
||||
|
||||
**Samsung English/other locales: still not established.** Samsung's own support page says only that "A Galaxy AI watermark will appear on AI-generated images" -- no string, no corner. Every community thread carrying the exact English string returned HTTP 403 to WebFetch, so the search paraphrase (bottom-left) is deliberately NOT recorded as fact. Feature-tier detail (b): the mark is applied by Generative Edit / sketch-to-image but reportedly NOT by Object Eraser, so Samsung absence is feature-dependent. The four-star icon variant: nothing found.
|
||||
**Meta `Imagined with AI` (string VERIFIED (a) from Meta's own newsroom; POSITION NOT VERIFIED).** Sources conflict on placement. Do not encode a corner without a verified sample. A dedicated 2026-08-27 sample hunt failed to obtain one: the Feb-2024 newsroom images are UI mockups whose photos carry no in-pixel mark; community posts (Threads `C8_rS_MuEId` titled "lower left corner", a Facebook share) corroborate bottom-LEFT verbally but their files defeat pixel verification; press screenshots predate the mark (Dec 2023); `imagine.meta.com` is dead (redirects to meta.ai) and its Wayback captures are broken SPA error pages. The mark's generator no longer exists (Muse output has no visible mark), so no fresh sample can be made. Removal stays on the generic `erase --region` path until a legacy capture surfaces. Source: `https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/`.
|
||||
|
||||
**Samsung English/other locales: still not established.** Samsung's own support page says only that "A Galaxy AI watermark will appear on AI-generated images" -- no string, no corner. Every community thread carrying the exact English string returned HTTP 403 to WebFetch, so the search paraphrase (bottom-left) is deliberately NOT recorded as fact. Feature-tier detail (b): the mark is applied by Generative Edit / sketch-to-image but reportedly NOT by Object Eraser, so Samsung absence is feature-dependent. No separate icon-only variant was established.
|
||||
|
||||
**The one document that would settle ByteDance placement is BLOCKED.** Douyin's 《抖音关于人工智能生成内容标识的水印与元数据规范》 aims to give AI tools a unified watermark style and position, which would cover Doubao / Jimeng / 星绘 at once. Both mirrors return HTTP 403 to WebFetch; a secondary report (b, unconfirmed) says the watermark is `AI生成` + tool name + company name placed **top-left** -- which would explain the Jimeng pill's top-left position but contradicts the GB annex's bottom-right example. Worth one retry through Chrome MCP with a real browser session.
|
||||
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
schema_version: 1
|
||||
|
||||
context:
|
||||
version: "0.22.0"
|
||||
python_min: "3.10"
|
||||
|
||||
package:
|
||||
name: remove-ai-watermarks
|
||||
version: ${{ version }}
|
||||
|
||||
source:
|
||||
url: https://pypi.org/packages/source/r/remove-ai-watermarks/remove_ai_watermarks-${{ version }}.tar.gz
|
||||
sha256: 00549927083c2b60444dd8404830c7333899a9345aacc41d43b21a77c63b8dd0
|
||||
|
||||
build:
|
||||
noarch: python
|
||||
number: 0
|
||||
script: python -m pip install . -vv --no-deps --no-build-isolation
|
||||
|
||||
requirements:
|
||||
host:
|
||||
- python ${{ python_min }}.*
|
||||
- pip
|
||||
- hatchling
|
||||
run:
|
||||
- python >=${{ python_min }}
|
||||
- pillow >=10.0.0
|
||||
- piexif >=1.1.3
|
||||
- click >=8.0.0
|
||||
- python-dotenv >=1.0.0
|
||||
# c2pa-python is a default PyPI dependency but is not packaged on conda-forge.
|
||||
# The guarded import falls back to the built-in C2PA byte scanner when it is
|
||||
# absent. Add it here once a c2pa-python feedstock exists.
|
||||
|
||||
tests:
|
||||
- python:
|
||||
imports:
|
||||
- remove_ai_watermarks
|
||||
- remove_ai_watermarks.identify
|
||||
- remove_ai_watermarks.metadata
|
||||
python_version: ${{ python_min }}.*
|
||||
# pip_check defaults to true; disable it explicitly. `pip check` fails on
|
||||
# the conda-forge piexif `py_2` build, whose stale 2019 metadata pip reads
|
||||
# as "piexif 1.1.3 is not supported on this platform" -- even though the
|
||||
# package installs, imports, and works. The imports test verifies loading.
|
||||
pip_check: false
|
||||
|
||||
about:
|
||||
homepage: https://github.com/wiltodelta/remove-ai-watermarks
|
||||
summary: Inspect and strip AI provenance metadata from media
|
||||
description: |
|
||||
Inspect and strip AI-provenance metadata (C2PA, EXIF, IPTC, and PNG text
|
||||
chunks) from images and supported video containers. Optional pip extras add
|
||||
visible watermark removal, video processing, SynthID diffusion removal, and
|
||||
additional invisible-watermark detectors.
|
||||
license: Apache-2.0
|
||||
license_file: LICENSE
|
||||
repository: https://github.com/wiltodelta/remove-ai-watermarks
|
||||
|
||||
extra:
|
||||
recipe-maintainers:
|
||||
- wiltodelta
|
||||
@@ -1,9 +1,13 @@
|
||||
[project]
|
||||
# This name is the published PyPI distribution, not a local label: changing it
|
||||
# strands every `pip install remove-ai-watermarks` and every pinned dependency
|
||||
# on it, including the ComfyUI node package. The console script below carries
|
||||
# the same weight, since users have it on PATH.
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.23.0"
|
||||
version = "0.35.0"
|
||||
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10.1"
|
||||
requires-python = ">=3.11,<3.15"
|
||||
license = {text = "Apache-2.0"}
|
||||
keywords = [
|
||||
"ai-watermark",
|
||||
@@ -33,10 +37,10 @@ classifiers = [
|
||||
"License :: OSI Approved :: Apache Software License",
|
||||
"Operating System :: OS Independent",
|
||||
"Programming Language :: Python :: 3 :: Only",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Programming Language :: Python :: 3.14",
|
||||
"Topic :: Multimedia :: Graphics",
|
||||
"Topic :: Multimedia :: Graphics :: Graphics Conversion",
|
||||
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
||||
@@ -60,7 +64,10 @@ dependencies = [
|
||||
|
||||
[project.optional-dependencies]
|
||||
pixels = [
|
||||
"numpy>=1.24.0",
|
||||
"numpy>=1.24.0; python_version < '3.13'",
|
||||
# NumPy 1.x has no CPython 3.13+ wheels. Requiring 2.x prevents installers
|
||||
# from attempting an unsupported source build on the newer Python lines.
|
||||
"numpy>=2; python_version >= '3.13'",
|
||||
"opencv-python-headless>=4.8.0",
|
||||
]
|
||||
# Optional HEIC/AVIF pixel decode. Metadata scanning handles these containers
|
||||
@@ -70,12 +77,10 @@ heif = [
|
||||
]
|
||||
visible = ["remove-ai-watermarks[pixels]"]
|
||||
# Video visible removal uses the shared pixel runtime. PyAV packetizes processed
|
||||
# VFR frames with explicit PTS before system ffmpeg encodes them. PyAV 18 requires
|
||||
# Python 3.11; the 16.x wheel line still covers Python 3.10.
|
||||
# VFR frames with explicit PTS before system ffmpeg encodes them.
|
||||
video = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
"av>=16,<17; python_version < '3.11'",
|
||||
"av>=18,<19; python_version >= '3.11'",
|
||||
"av>=18,<19",
|
||||
]
|
||||
# Open DWT-DCT watermarks used by Stable Diffusion / SDXL / FLUX. The in-tree
|
||||
# decoder avoids the upstream invisible-watermark package's mandatory torch and
|
||||
@@ -86,16 +91,11 @@ detect = [
|
||||
]
|
||||
diffusion = [
|
||||
"remove-ai-watermarks[pixels]",
|
||||
# A CUDA-enabled torch build is required: invisible-watermark removal has no
|
||||
# CPU, MPS or XPU path. The default PyPI wheel carries CUDA on Linux/Windows;
|
||||
# on macOS there is no CUDA build and this extra installs only for the
|
||||
# non-diffusion imports it shares.
|
||||
"torch>=2.0.0",
|
||||
# The default PyPI torch wheel is a CPU/CUDA build. To drive an Intel GPU
|
||||
# (Arc / Data Center) via ``--device xpu`` you need an XPU-enabled torch
|
||||
# from PyTorch's XPU wheel index (Linux/Windows only -- there is no macOS
|
||||
# XPU build). Install that build first, then this extra (torch is then
|
||||
# already satisfied and won't be re-pulled):
|
||||
# pip install torch --index-url https://download.pytorch.org/whl/xpu
|
||||
# pip install 'remove-ai-watermarks[diffusion]'
|
||||
# uv users can target the ``pytorch-xpu`` index declared under [tool.uv]:
|
||||
# uv pip install torch --index-url https://download.pytorch.org/whl/xpu
|
||||
"diffusers>=0.38.0",
|
||||
# diffusers 0.38's auto-pipeline registry imports ``Qwen3VLForConditional
|
||||
# Generation`` (its ``nucleusmoe_image`` pipeline), which only exists in
|
||||
@@ -120,12 +120,27 @@ qwen-zimage = [
|
||||
"diffsynth>=2.0.17,<3",
|
||||
"torchvision>=0.20.0",
|
||||
]
|
||||
# Opt-in verified-text reconstruction over qwen-zimage. LaMa removes the changed
|
||||
# candidate glyphs before exact Qwen-VAE cores are composited back.
|
||||
text-restoration = [
|
||||
"remove-ai-watermarks[qwen-zimage,lama]",
|
||||
]
|
||||
# Draft-only OCR proposals for verified-text manifests. CPU, no torch:
|
||||
# PaddleOCR detection plus three script-chosen recognition engines. The models
|
||||
# download on first use and are cached by huggingface_hub; never bundled.
|
||||
text-draft = [
|
||||
"paddleocr>=3.3.3",
|
||||
"paddlepaddle>=3.0.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Adobe TrustMark decoder -- the open, keyless watermark behind Adobe Durable
|
||||
# Content Credentials (soft-binding alg ``com.adobe.trustmark.P``). Optional
|
||||
# because it pulls torch and downloads model weights on first use. identify()
|
||||
# guards the import and skips the TrustMark signal when absent.
|
||||
trustmark = [
|
||||
"trustmark>=0.8.0",
|
||||
# TrustMark requires NumPy <2, whose last release has no CPython 3.13+
|
||||
# wheels. Keep this feature installable only on its real runtime range.
|
||||
"trustmark>=0.8.0; python_version < '3.13'",
|
||||
]
|
||||
# Universal region eraser backend -- big-LaMa via onnxruntime (Carve/LaMa-ONNX,
|
||||
# Apache-2.0). CPU, no torch. Model (~200 MB) is downloaded on first use and
|
||||
@@ -133,10 +148,7 @@ trustmark = [
|
||||
# eraser backend needs none of this.
|
||||
lama = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
# ONNX Runtime 1.24 dropped CPython 3.10 wheels; keep the project's
|
||||
# supported 3.10 line on the last compatible release series.
|
||||
"onnxruntime>=1.16.0,<1.24; python_version < '3.11'",
|
||||
"onnxruntime>=1.16.0; python_version >= '3.11'",
|
||||
"onnxruntime>=1.24.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Lightweight inpaint backend -- MI-GAN via onnxruntime (andraniksargsyan/migan,
|
||||
@@ -146,23 +158,9 @@ lama = [
|
||||
# LaMa, the quality-first `auto` choice, is too large. Same runtime as `lama`.
|
||||
migan = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
"onnxruntime>=1.16.0,<1.24; python_version < '3.11'",
|
||||
"onnxruntime>=1.16.0; python_version >= '3.11'",
|
||||
"onnxruntime>=1.24.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Optional pre-diffusion super-resolution for small inputs (Real-ESRGAN). Loaded via
|
||||
# spandrel (MIT) -- a pure model-loader with NO basicsr dependency (it pulls only
|
||||
# torch / torchvision / safetensors / numpy / einops).
|
||||
# The Real-ESRGAN weights (BSD-3-Clause) download on first use and are cached; they
|
||||
# are never bundled. CPU works but is slow on large inputs -- it is meant for the
|
||||
# pre-diffusion upscale of SMALL inputs (and the GPU worker). Guarded by
|
||||
# upscaler.is_available(); the default upscaler stays Lanczos (cv2, no deps). The
|
||||
# weights are fetched with torch.hub (bundled with spandrel's torch), so no extra
|
||||
# download dependency is needed.
|
||||
esrgan = [
|
||||
"remove-ai-watermarks[pixels]",
|
||||
"spandrel>=0.3.0",
|
||||
]
|
||||
dev = [
|
||||
"remove-ai-watermarks[video]",
|
||||
"remove-ai-watermarks[detect]",
|
||||
@@ -173,21 +171,10 @@ dev = [
|
||||
"ruff>=0.4.0",
|
||||
"pyright>=1.1.0",
|
||||
"invisible-watermark>=0.2.0",
|
||||
# maintain.sh helpers; they only support newer Pythons, so gate them by
|
||||
# marker to keep the py3.10 resolution (and CI matrix) solvable.
|
||||
"uv-outdated>=0.1.0; python_version >= '3.12'",
|
||||
"uv-secure>=0.12.0; python_version >= '3.12'",
|
||||
]
|
||||
all = ["remove-ai-watermarks[video,heif,detect,trustmark,diffusion,qwen-zimage,lama,migan,esrgan]"]
|
||||
|
||||
# PyTorch Intel-GPU (XPU) wheel index. ``explicit = true`` keeps it inert for
|
||||
# the default CPU/CUDA install: uv consults it only when a torch install
|
||||
# explicitly targets it (see the ``diffusion`` extra comment), so it does not alter
|
||||
# the locked CPU/CUDA resolution. Linux/Windows only -- no macOS XPU build.
|
||||
[[tool.uv.index]]
|
||||
name = "pytorch-xpu"
|
||||
url = "https://download.pytorch.org/whl/xpu"
|
||||
explicit = true
|
||||
# ``qwen-zimage`` already pulls ``diffusion``; naming both would suggest diffusion is
|
||||
# independently sufficient for a removal, which it is not.
|
||||
all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan]"]
|
||||
|
||||
[project.scripts]
|
||||
remove-ai-watermarks = "remove_ai_watermarks.cli:main"
|
||||
@@ -224,7 +211,7 @@ pythonpath = ["src"]
|
||||
addopts = "-v --tb=short"
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py310"
|
||||
target-version = "py311"
|
||||
line-length = 120
|
||||
exclude = ["_refs"]
|
||||
extend-exclude = ["*.md"]
|
||||
@@ -247,7 +234,7 @@ quote-style = "double"
|
||||
indent-style = "space"
|
||||
|
||||
[tool.pyright]
|
||||
pythonVersion = "3.10"
|
||||
pythonVersion = "3.11"
|
||||
typeCheckingMode = "strict"
|
||||
exclude = ["_refs"]
|
||||
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
# Maintainer scripts
|
||||
|
||||
These scripts are development and evaluation tools, not installed CLI commands. Run
|
||||
them from the repository root with `uv run python scripts/<name>.py --help`. Inputs
|
||||
under `.local-eval/` and generated reports remain untracked unless a data README
|
||||
explicitly names a tracked canonical result.
|
||||
|
||||
## Audits and release checks
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `corpus_gap_scan.py` | Compare a local image corpus with the library's `identify` results. |
|
||||
| `detection_timing.py` | Record per-method metadata and verdict timings. |
|
||||
| `detection_timing_report.py` | Aggregate timing records by method and segment. |
|
||||
| `fidelity_metrics.py` | Compute objective image-fidelity metrics for paired outputs. |
|
||||
| `invisible_quality_audit.py` | Pair originals and invisible-removal outputs for quality review. |
|
||||
| `metadata_removal_audit.py` | Check metadata detection/removal parity over a corpus. |
|
||||
| `pill_gate_audit.py` | Measure the Jimeng pill detector on the product path. |
|
||||
| `real_examples_e2e.py` | Run end-to-end confidence checks over local examples. |
|
||||
| `record_parity_audit.py` | Compare record-based and file-based identification. |
|
||||
| `resource_ceilings.py` | Measure peak RSS and runtime for fill backends. |
|
||||
| `robustness_suite.py` | Exercise CLI failures on adversarial and degenerate inputs. |
|
||||
| `sidecar_regression.py` | Compare current identification with recorded sidecars. |
|
||||
| `smoke_matrix.py` | Exercise CLI parameter choices on real local data. |
|
||||
| `video_fidelity_probe.py` | Compare delivered video fidelity with its source. |
|
||||
| `visible_eval.py` | Benchmark registered visible-mark detectors. |
|
||||
| `visible_removal_audit.py` | Audit visible-removal results over a local corpus. |
|
||||
|
||||
## Calibration and corpus preparation
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `contentseal_transforms.py` | Reproduce and hash-check deterministic Content Seal variants. |
|
||||
| `detector_response.py` | Measure detector response over mark size, contrast, background, and aspect. |
|
||||
| `fill_quality.py` | Measure visible-fill quality against constructed ground truth. |
|
||||
| `ladder_headroom.py` | Measure recall cost from the coarse scale ladder. |
|
||||
| `registered_mark_calibrate.py` | Measure a registered detector without conflating visual positives, metadata cohorts, adjudicated negatives, and unlabeled controls. |
|
||||
| `synthid_corpus.py` | Ingest and inspect the local SynthID reference corpus. |
|
||||
| `vendor_cohort_harvest.py` | Partition TC260 carriers by producer code. |
|
||||
| `vendor_mark_calibrate.py` | Calibrate a candidate vendor text detector. |
|
||||
| `visible_alpha_solve.py` | Rebuild visible-watermark alpha assets from controlled captures. |
|
||||
| `visible_groundtruth.py` | Consolidate blinded contact-sheet labels into ground truth. |
|
||||
| `visible_positives.py` | List corpus images carrying a registered visible mark. |
|
||||
| `visible_recall_sample.py` | Build an unbiased blinded sample for recall measurement. |
|
||||
| `visible_sheets.py` | Build blinded contact sheets for relaxation candidates. |
|
||||
|
||||
## Research and diagnostic prototypes
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `cjk_tail_probe.py` | Test a generic template for otherwise uncovered CJK labels. |
|
||||
| `controlnet_sweep.py` | Sweep the historical ControlNet removal prototype. |
|
||||
| `infer_text_lines.py` | Draft stable source-text lines without modifying pixels. |
|
||||
| `qwen_scrub_prototype.py` | Probe low-strength Qwen regeneration on a GPU. |
|
||||
| `selective_text_restoration.py` | Evaluate text restoration over a scrubbed image. |
|
||||
| `synthid_pixel_probe.py` | Run the experimental local SynthID carrier probe. |
|
||||
| `video_synthid_sweep.py` | Build oracle-gated video regeneration candidates. |
|
||||
|
||||
## Generated assets
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `render_pill_silhouette.py` | Render the synthetic Jimeng pill silhouette. |
|
||||
| `render_vendor_silhouettes.py` | Render synthetic vendor text-mark silhouettes. |
|
||||
|
||||
## Shared helpers
|
||||
|
||||
`_plain_console.py` provides plain-text fallbacks for Rich output, and
|
||||
`_text_eval.py` contains normalization helpers shared by text-evaluation scripts.
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Pure text-normalization helpers shared by evaluation scripts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unicodedata
|
||||
|
||||
|
||||
def normalize_text(text: str) -> str:
|
||||
"""Normalize text for layout-independent evaluation comparisons."""
|
||||
return "".join(unicodedata.normalize("NFC", text).casefold().split())
|
||||
|
||||
|
||||
def levenshtein_normalized(left: str, right: str) -> float:
|
||||
"""Return normalized Levenshtein distance without changing either input."""
|
||||
if not left and not right:
|
||||
return 0.0
|
||||
previous = list(range(len(right) + 1))
|
||||
for left_index, left_character in enumerate(left, start=1):
|
||||
current = [left_index]
|
||||
for right_index, right_character in enumerate(right, start=1):
|
||||
current.append(
|
||||
min(
|
||||
current[-1] + 1,
|
||||
previous[right_index] + 1,
|
||||
previous[right_index - 1] + (left_character != right_character),
|
||||
)
|
||||
)
|
||||
previous = current
|
||||
return previous[-1] / max(len(left), len(right))
|
||||
|
||||
|
||||
def normalized_edit_distance(left: str, right: str) -> float:
|
||||
"""Normalize text, then return Levenshtein distance over the result."""
|
||||
return levenshtein_normalized(normalize_text(left), normalize_text(right))
|
||||
@@ -1,436 +0,0 @@
|
||||
"""AI-generation scorer for scanned datasets.
|
||||
|
||||
Trains a gradient-boosted classifier on the metadata-derived labels of a
|
||||
scan_dataset.py output and scores every file, WITHOUT needing the
|
||||
metadata to be present at scoring time: the features are pixel and
|
||||
container statistics, so a metadata-stripped file still gets a score.
|
||||
The model is distribution-specific by design; evaluate it again before
|
||||
using it on an unrelated target distribution.
|
||||
|
||||
Modes:
|
||||
uv run --with scikit-learn python scripts/ai_score.py train <scan_glob> <model.pkl> [v1|v2]
|
||||
uv run --with scikit-learn python scripts/ai_score.py score <scan_glob> <model.pkl> <out.jsonl>
|
||||
|
||||
<scan_glob> is a glob of scan_dataset shards, e.g. 'data/scan/part_*.jsonl'.
|
||||
score output: one JSON line per file with file, sha256, ai_score, and the
|
||||
label evidence when a metadata label existed (for monitoring drift).
|
||||
"""
|
||||
|
||||
import glob
|
||||
import json
|
||||
import math
|
||||
import pickle
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
import numpy as np
|
||||
|
||||
# AI-generator names in C2PA claim_generator_info (metadata-level truth).
|
||||
AI_GENERATORS = (
|
||||
"openai",
|
||||
"adobe firefly",
|
||||
"adobe_firefly",
|
||||
"microsoft responsible ai",
|
||||
"microsoft_designer",
|
||||
"black forest labs",
|
||||
"fal-ai",
|
||||
"bria",
|
||||
"chatgpt",
|
||||
"stability",
|
||||
"dreamina",
|
||||
"canva",
|
||||
)
|
||||
# Human-origin software (negative evidence, NOT proof by itself).
|
||||
HUMAN_SOFTWARE = (
|
||||
"photoshop",
|
||||
"lightroom",
|
||||
"picsart",
|
||||
"snapseed",
|
||||
"paint.net",
|
||||
"gimp",
|
||||
"capture one",
|
||||
"meitu",
|
||||
"xingtu",
|
||||
"snow",
|
||||
)
|
||||
|
||||
_FEAT_SCALAR = [
|
||||
("noise", "noise_std"),
|
||||
("noise", "noise_kurtosis"),
|
||||
("fft", "cfa_peak"),
|
||||
("ela", "ela_mean"),
|
||||
("ela", "ela_p95"),
|
||||
("gradient", "laplacian_var"),
|
||||
("color", "saturation_mean"),
|
||||
("color", "value_mean"),
|
||||
("dct", "benford_mad"),
|
||||
]
|
||||
FeatureSchema = Literal["v1", "v2"]
|
||||
_FFT_BANDS = 8
|
||||
_GRADIENT_BINS = 10
|
||||
_COLOR_BINS = 64
|
||||
_DCT_AC_POSITIONS = 8
|
||||
_DCT_AC_BINS = 21
|
||||
_JPEG_QUANT_TABLES = 2
|
||||
_JPEG_QUANT_VALUES = 64
|
||||
_SCORE_BATCH = 4096
|
||||
|
||||
_V1_FEATURE_NAMES = (
|
||||
*(key for _, key in _FEAT_SCALAR),
|
||||
*(f"fft_band_energy_{index}" for index in range(_FFT_BANDS)),
|
||||
*(f"gradient_hist_{index}" for index in range(_GRADIENT_BINS)),
|
||||
*(f"color_hist_4x4x4_{index}" for index in range(_COLOR_BINS)),
|
||||
"jpeg_444",
|
||||
"jpeg_progressive",
|
||||
"megapixels",
|
||||
"aspect_ratio",
|
||||
"format_jpeg",
|
||||
"format_png",
|
||||
)
|
||||
_V2_EXTRA_FEATURE_NAMES = (
|
||||
"cfa_peak_0",
|
||||
"cfa_peak_1",
|
||||
*(f"dct_ac_{position}_{bin_index}" for position in range(_DCT_AC_POSITIONS) for bin_index in range(_DCT_AC_BINS)),
|
||||
*(f"jpeg_quant_{table}_{index}" for table in range(_JPEG_QUANT_TABLES) for index in range(_JPEG_QUANT_VALUES)),
|
||||
"jpeg_quant_table_count",
|
||||
"jpeg_huffman_table_count",
|
||||
"jpeg_huffman_total_bytes",
|
||||
"jpeg_scan_count",
|
||||
"jpeg_restart_interval",
|
||||
"jpeg_precision_bits",
|
||||
"jpeg_adobe_transform",
|
||||
"jpeg_jfif_present",
|
||||
"format_webp",
|
||||
"format_isobmff",
|
||||
"format_other",
|
||||
)
|
||||
_SCHEMA_FEATURE_NAMES: dict[str, tuple[str, ...]] = {
|
||||
"v1": _V1_FEATURE_NAMES,
|
||||
"v2": _V1_FEATURE_NAMES + _V2_EXTRA_FEATURE_NAMES,
|
||||
}
|
||||
|
||||
|
||||
def parse_schema(value: Any) -> FeatureSchema:
|
||||
"""Validate a feature-schema token (CLI argument or model-bundle field)."""
|
||||
if value == "v1" or value == "v2":
|
||||
return value
|
||||
raise ValueError(f"Unknown feature schema: {value!r}")
|
||||
|
||||
|
||||
def feature_names(schema: FeatureSchema = "v1") -> tuple[str, ...]:
|
||||
"""Return the stable ordered feature names for a model schema."""
|
||||
return _SCHEMA_FEATURE_NAMES[parse_schema(schema)]
|
||||
|
||||
|
||||
def _fixed_values(values: Any, count: int, *, normalize: bool = False) -> list[float]:
|
||||
"""Return a fixed-width float vector, padding absent values with NaN."""
|
||||
if not isinstance(values, (list, tuple)):
|
||||
return [float("nan")] * count
|
||||
result = [float(value) for value in values[:count]]
|
||||
if normalize and result:
|
||||
total = sum(value for value in result if math.isfinite(value))
|
||||
if total > 0:
|
||||
result = [value / total for value in result]
|
||||
return result + [float("nan")] * (count - len(result))
|
||||
|
||||
|
||||
def _optional_float(mapping: dict[str, Any], key: str) -> float:
|
||||
value = mapping.get(key)
|
||||
return float(value) if isinstance(value, (int, float)) else float("nan")
|
||||
|
||||
|
||||
def label_of(record: dict[str, Any]) -> int | None:
|
||||
"""1 = strong metadata AI, 0 = human-origin, None = unlabeled."""
|
||||
store = record.get("c2pa_store") or {}
|
||||
for m in (store.get("manifests") or {}).values():
|
||||
for cgi in m.get("claim_generator_info") or []:
|
||||
if any(g in str(cgi.get("name", "")).lower() for g in AI_GENERATORS):
|
||||
return 1
|
||||
for a in m.get("assertions") or []:
|
||||
d = a.get("data") if isinstance(a.get("data"), dict) else {}
|
||||
if "trainedAlgorithmicMedia" in str(d.get("digitalSourceType", "")):
|
||||
return 1
|
||||
for c in record.get("png_chunks", []):
|
||||
t = c.get("text", "")
|
||||
kw = t.split("\x00")[0]
|
||||
if kw in ("prompt", "workflow", "parameters") or "AIGC" in t[:80]:
|
||||
return 1
|
||||
exif = record.get("exif", {})
|
||||
z = exif.get("0th") or {}
|
||||
e = exif.get("Exif") or {}
|
||||
if z.get("Make") and z.get("Model") and (e.get("MakerNote") or e.get("LensModel") or e.get("LensMake")):
|
||||
return 0
|
||||
for c in record.get("png_chunks", []):
|
||||
if c.get("apple_screenshot_marker"):
|
||||
return 0
|
||||
for v in (record.get("iptc") or {}).values():
|
||||
if str(v).strip() == "Screenshot":
|
||||
return 0
|
||||
if any(t in str(z.get("Software", "")).lower() for t in HUMAN_SOFTWARE):
|
||||
return 0
|
||||
return None
|
||||
|
||||
|
||||
def features_of(record: dict[str, Any], *, schema: FeatureSchema = "v1") -> list[float]:
|
||||
"""The structural feature vector (pixel + container stats, no metadata)."""
|
||||
v = [float((record.get(s) or {}).get(k, np.nan)) for s, k in _FEAT_SCALAR]
|
||||
fft = record.get("fft") or {}
|
||||
gradient = record.get("gradient") or {}
|
||||
color = record.get("color") or {}
|
||||
v.extend(_fixed_values(fft.get("fft_band_energy"), _FFT_BANDS))
|
||||
v.extend(_fixed_values(gradient.get("gradient_hist"), _GRADIENT_BINS))
|
||||
v.extend(_fixed_values(color.get("color_hist_4x4x4"), _COLOR_BINS))
|
||||
jf = record.get("jpeg_forensics") or {}
|
||||
v.append(1.0 if jf.get("subsampling") == "4:4:4" else 0.0)
|
||||
v.append(1.0 if jf.get("progressive") else 0.0)
|
||||
pil = record.get("pil") or {}
|
||||
w, h = pil.get("width") or 0, pil.get("height") or 0
|
||||
v += [float(w * h) / 1e6, float(w) / max(h, 1)]
|
||||
fmt = record.get("content_format")
|
||||
v += [1.0 if fmt == "jpeg" else 0.0, 1.0 if fmt == "png" else 0.0]
|
||||
if schema == "v2":
|
||||
v.extend(_fixed_values(fft.get("cfa_peaks"), 2))
|
||||
dct_hist = (record.get("dct") or {}).get("dct_ac_hist")
|
||||
for position in range(_DCT_AC_POSITIONS):
|
||||
values = dct_hist[position] if isinstance(dct_hist, list) and position < len(dct_hist) else None
|
||||
v.extend(_fixed_values(values, _DCT_AC_BINS, normalize=True))
|
||||
quant_tables = jf.get("quant_tables")
|
||||
for table in range(_JPEG_QUANT_TABLES):
|
||||
values = quant_tables.get(str(table)) if isinstance(quant_tables, dict) else None
|
||||
v.extend(_fixed_values(values, _JPEG_QUANT_VALUES))
|
||||
is_jpeg = fmt == "jpeg"
|
||||
huffman_tables = jf.get("huffman_tables_hex")
|
||||
huffman_values = huffman_tables if isinstance(huffman_tables, list) else []
|
||||
v.extend(
|
||||
[
|
||||
float(len(quant_tables)) if isinstance(quant_tables, dict) else (0.0 if is_jpeg else float("nan")),
|
||||
float(len(huffman_values)) if is_jpeg else float("nan"),
|
||||
(
|
||||
float(sum(len(value) // 2 for value in huffman_values if isinstance(value, str)))
|
||||
if is_jpeg
|
||||
else float("nan")
|
||||
),
|
||||
_optional_float(jf, "scan_count"),
|
||||
_optional_float(jf, "restart_interval"),
|
||||
_optional_float(jf, "precision_bits"),
|
||||
_optional_float(jf, "adobe_transform"),
|
||||
1.0 if jf.get("jfif") else (0.0 if is_jpeg else float("nan")),
|
||||
1.0 if fmt == "webp" else 0.0,
|
||||
1.0 if isinstance(fmt, str) and fmt.startswith("isobmff:") else 0.0,
|
||||
(
|
||||
1.0
|
||||
if fmt is not None and fmt not in {"jpeg", "png", "webp"} and not str(fmt).startswith("isobmff:")
|
||||
else 0.0
|
||||
),
|
||||
]
|
||||
)
|
||||
elif schema != "v1":
|
||||
raise ValueError(f"Unknown feature schema: {schema}")
|
||||
expected = len(feature_names(schema))
|
||||
if len(v) != expected:
|
||||
raise ValueError(f"Feature schema {schema} produced {len(v)} values; expected {expected}")
|
||||
return v
|
||||
|
||||
|
||||
def grouped_stratified_split(
|
||||
labels: np.ndarray,
|
||||
hashes: np.ndarray,
|
||||
*,
|
||||
test_size: float = 0.3,
|
||||
random_state: int = 0,
|
||||
) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""Split records by hash while preserving label balance across hash groups."""
|
||||
if len(labels) != len(hashes):
|
||||
raise ValueError("labels and hashes must have equal length")
|
||||
if not 0 < test_size < 1:
|
||||
raise ValueError("test_size must be between 0 and 1")
|
||||
groups: dict[str, list[int]] = defaultdict(list)
|
||||
group_labels: dict[str, int] = {}
|
||||
for index, (label, digest) in enumerate(zip(labels, hashes, strict=True)):
|
||||
key = str(digest) if digest else f"__row_{index}"
|
||||
int_label = int(label)
|
||||
if key in group_labels and group_labels[key] != int_label:
|
||||
raise ValueError(f"Hash {key!r} has conflicting labels")
|
||||
group_labels[key] = int_label
|
||||
groups[key].append(index)
|
||||
keys_by_label: dict[int, list[str]] = defaultdict(list)
|
||||
for key, label in group_labels.items():
|
||||
keys_by_label[label].append(key)
|
||||
rng = np.random.default_rng(random_state)
|
||||
test_set: set[str] = set()
|
||||
for class_keys in keys_by_label.values():
|
||||
if len(class_keys) < 2:
|
||||
raise ValueError("Each label needs at least two distinct hash groups")
|
||||
shuffled = np.asarray(class_keys)
|
||||
rng.shuffle(shuffled)
|
||||
test_count = min(max(round(len(shuffled) * test_size), 1), len(shuffled) - 1)
|
||||
test_set.update(shuffled[:test_count].tolist())
|
||||
train_set = set(groups) - test_set
|
||||
train = np.asarray([index for key, indices in groups.items() if key in train_set for index in indices])
|
||||
test = np.asarray([index for key, indices in groups.items() if key in test_set for index in indices])
|
||||
return np.sort(train), np.sort(test)
|
||||
|
||||
|
||||
def temporal_holdout_split(
|
||||
dates: np.ndarray,
|
||||
hashes: np.ndarray,
|
||||
*,
|
||||
train_fraction: float = 0.7,
|
||||
) -> tuple[np.ndarray, np.ndarray, str]:
|
||||
"""Split chronologically and remove later copies of training hashes."""
|
||||
if len(dates) != len(hashes):
|
||||
raise ValueError("dates and hashes must have equal length")
|
||||
if not 0 < train_fraction < 1:
|
||||
raise ValueError("train_fraction must be between 0 and 1")
|
||||
unique_dates = sorted({str(date) for date in dates})
|
||||
if len(unique_dates) < 2:
|
||||
raise ValueError("Temporal holdout needs at least two distinct dates")
|
||||
cutoff_index = min(int(len(unique_dates) * train_fraction), len(unique_dates) - 1)
|
||||
cutoff = unique_dates[cutoff_index]
|
||||
train = np.flatnonzero(dates < cutoff)
|
||||
test = np.flatnonzero((dates >= cutoff) & ~np.isin(hashes, hashes[train]))
|
||||
return train, test, cutoff
|
||||
|
||||
|
||||
def model_schema(bundle: dict[str, Any]) -> FeatureSchema:
|
||||
"""Read a model's feature schema, defaulting legacy bundles to v1."""
|
||||
return parse_schema(bundle.get("feature_schema", "v1"))
|
||||
|
||||
|
||||
def iter_records(pattern: str) -> Any:
|
||||
for path in sorted(glob.glob(pattern)):
|
||||
with open(path) as fh:
|
||||
for line in fh:
|
||||
yield json.loads(line)
|
||||
|
||||
|
||||
def cmd_train(pattern: str, model_path: str, schema: FeatureSchema = "v2") -> None:
|
||||
from sklearn.ensemble import HistGradientBoostingClassifier
|
||||
from sklearn.impute import SimpleImputer
|
||||
from sklearn.metrics import average_precision_score, roc_auc_score
|
||||
|
||||
# keep the feature vector, not the record: a parsed scan record is an order of
|
||||
# magnitude larger than the row it collapses to.
|
||||
labeled: dict[str, tuple[list[float], int, str]] = {}
|
||||
missing_hash_index = 0
|
||||
for r in iter_records(pattern):
|
||||
if "noise" not in r:
|
||||
continue
|
||||
lab = label_of(r)
|
||||
if lab is None:
|
||||
continue
|
||||
digest = r.get("sha256")
|
||||
if not digest:
|
||||
digest = f"__missing_{missing_hash_index}"
|
||||
missing_hash_index += 1
|
||||
date = Path(r["file"]).parent.name
|
||||
previous = labeled.get(str(digest))
|
||||
if previous is not None and previous[1] != lab:
|
||||
raise ValueError(f"Hash {digest!r} has conflicting labels")
|
||||
if previous is None or date < previous[2]:
|
||||
labeled[str(digest)] = (features_of(r, schema=schema), lab, date)
|
||||
|
||||
rows, labels, row_dates = zip(*labeled.values(), strict=True)
|
||||
X = np.asarray(rows)
|
||||
y = np.asarray(labels)
|
||||
dates = np.asarray(row_dates)
|
||||
hashes = np.asarray(list(labeled))
|
||||
print(
|
||||
f"unique labeled: {len(y)} "
|
||||
f"(pos {int(y.sum())}, neg {int((1 - y).sum())}, features {X.shape[1]}, schema {schema})"
|
||||
)
|
||||
|
||||
def fit(train: np.ndarray) -> tuple[Any, Any]:
|
||||
rows = X[train]
|
||||
imputer = SimpleImputer(strategy="median")
|
||||
classifier = HistGradientBoostingClassifier(
|
||||
random_state=0,
|
||||
max_iter=300,
|
||||
l2_regularization=5.0,
|
||||
early_stopping=False,
|
||||
)
|
||||
classifier.fit(imputer.fit_transform(rows), y[train])
|
||||
return imputer, classifier
|
||||
|
||||
train, test = grouped_stratified_split(y, hashes, test_size=0.3, random_state=0)
|
||||
imp, clf = fit(train)
|
||||
p = clf.predict_proba(imp.transform(X[test]))[:, 1]
|
||||
print(f"SHA-grouped holdout: AUC {roc_auc_score(y[test], p):.4f} | AP {average_precision_score(y[test], p):.4f}")
|
||||
|
||||
temporal_train, temporal_test, cutoff = temporal_holdout_split(dates, hashes)
|
||||
if len(temporal_train) > 100 and len(temporal_test) > 100:
|
||||
imp_t, clf_t = fit(temporal_train)
|
||||
pt = clf_t.predict_proba(imp_t.transform(X[temporal_test]))[:, 1]
|
||||
auc_t = roc_auc_score(y[temporal_test], pt)
|
||||
ap_t = average_precision_score(y[temporal_test], pt)
|
||||
print(f"temporal (<{cutoff}): AUC {auc_t:.4f} | AP {ap_t:.4f}")
|
||||
|
||||
all_indices = np.arange(len(y))
|
||||
imp, clf = fit(all_indices)
|
||||
bundle = {
|
||||
"imputer": imp,
|
||||
"clf": clf,
|
||||
"feature_schema": schema,
|
||||
"feature_names": feature_names(schema),
|
||||
}
|
||||
with open(model_path, "wb") as f:
|
||||
pickle.dump(bundle, f)
|
||||
print(f"model written: {model_path}")
|
||||
|
||||
|
||||
def cmd_score(pattern: str, model_path: str, out_path: str) -> None:
|
||||
# the model file is produced locally by `train`; do not load third-party pickles
|
||||
with open(model_path, "rb") as f:
|
||||
bundle = pickle.load(f) # noqa: S301
|
||||
imp, clf = bundle["imputer"], bundle["clf"]
|
||||
schema = model_schema(bundle)
|
||||
stored_names = bundle.get("feature_names")
|
||||
if stored_names is not None and tuple(stored_names) != feature_names(schema):
|
||||
raise ValueError(f"Model feature names do not match schema {schema}")
|
||||
n = 0
|
||||
batch_rows: list[list[float]] = []
|
||||
batch_meta: list[dict[str, Any]] = []
|
||||
|
||||
def flush(out: Any) -> None:
|
||||
"""Score one batch: per-record predict_proba is dominated by call overhead."""
|
||||
nonlocal n
|
||||
if not batch_rows:
|
||||
return
|
||||
scores = clf.predict_proba(imp.transform(np.asarray(batch_rows)))[:, 1]
|
||||
for meta, score in zip(batch_meta, scores, strict=True):
|
||||
out.write(json.dumps({**meta, "ai_score": round(float(score), 4)}) + "\n")
|
||||
n += len(batch_rows)
|
||||
batch_rows.clear()
|
||||
batch_meta.clear()
|
||||
print(f" {n}", flush=True)
|
||||
|
||||
with open(out_path, "w") as out:
|
||||
for r in iter_records(pattern):
|
||||
if "noise" not in r:
|
||||
continue
|
||||
batch_rows.append(features_of(r, schema=schema))
|
||||
batch_meta.append({"file": r["file"], "sha256": r.get("sha256"), "metadata_label": label_of(r)})
|
||||
if len(batch_rows) >= _SCORE_BATCH:
|
||||
flush(out)
|
||||
flush(out)
|
||||
print(f"scored {n} -> {out_path}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
if len(sys.argv) < 2 or sys.argv[1] not in ("train", "score"):
|
||||
print(__doc__)
|
||||
sys.exit(2)
|
||||
if sys.argv[1] == "train" and len(sys.argv) in (4, 5):
|
||||
schema = parse_schema(sys.argv[4]) if len(sys.argv) == 5 else "v2"
|
||||
cmd_train(sys.argv[2], sys.argv[3], schema)
|
||||
elif sys.argv[1] == "score" and len(sys.argv) == 5:
|
||||
cmd_score(sys.argv[2], sys.argv[3], sys.argv[4])
|
||||
else:
|
||||
print(__doc__)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,38 @@
|
||||
# Unregistered visible-mark candidates
|
||||
|
||||
Synthetic detection silhouettes for marks that are measured but NOT registered:
|
||||
none of these separates its positives from clean negatives yet, so shipping them
|
||||
would attribute and fill corners on content that carries no mark (a false fill
|
||||
is the worse error). Assets are regenerated by
|
||||
`scripts/render_vendor_silhouettes.py`; a candidate is deleted from here on the
|
||||
day its mark registers (the asset moves to the package `assets/`).
|
||||
|
||||
Measurements below are local calibration snapshots. Candidate pools are
|
||||
provenance cohorts, not automatically visible-mark positives; comparison controls
|
||||
are independently selected no-signal images and are not adjudicated negatives.
|
||||
Each needs capture-solved alphas or vendor-accurate font work before it can
|
||||
ship; `scripts/vendor_mark_calibrate.py` is the candidate-detector harness.
|
||||
Rows through `mindvideo_alpha.png` and the parked candidates were recorded on
|
||||
2026-08-27; rows from `higgsfield_alpha.png` through `gendo_alpha.png` were
|
||||
recorded on 2026-08-28. These are repository record dates, not inferred capture
|
||||
dates. The Mark column quotes the candidate raster text; where that text differs
|
||||
from the provider's canonical name, the provider is named separately.
|
||||
|
||||
| Asset | Mark | Evidence | Result |
|
||||
| --- | --- | --- | --- |
|
||||
| `samsung_en_alpha.png` | Samsung Galaxy AI label, English locale ("AI-generated content", bottom-left) | 5 corpus files | POS 0.11-0.30 vs NEG max 0.40 (binary); tophat/gray tried, no separation. The registered Italian engine scores 0.18-0.31 on the same files -- same layout class, wrong glyph template. |
|
||||
| `gemini_text_alpha.png` | "Generated with Gemini" text label (bottom-right; the registered gemini mark is the sparkle icon) | 3 corpus files | POS 0.07-0.22 vs NEG max 0.32; coverage gate finds the blob (0.29-0.35) but the Arial silhouette misses Google's letterforms. |
|
||||
| `notebooklm_alpha.png` | NotebookLM wordmark (bottom-right) | 12 corpus files | locate geometry not yet fitted; POS max 0.12. |
|
||||
| `dola_alpha.png` | DolaAI on images (the video mark is registered) | 12 corpus files | POS 0.11-0.21 vs NEG max 0.30. |
|
||||
| `mindvideo_alpha.png` | MindVideo.AI (top-right) | 11 corpus files | POS 0.29-0.32 vs NEG max 0.30 -- borderline overlap, not shippable. |
|
||||
| `higgsfield_alpha.png` | `HIGGSFIELD AI` wordmark (Higgsfield AI, bottom-right; the boxed `AI` variant shares the cohort) | 5 wordmark files (16 in the boxed-AI OCR cluster) | POS max 0.26 vs NEG max 0.22 -- no separation; the mark may be two-part (wordmark + boxed AI) and needs a composed template. |
|
||||
| `jianying_alpha.png` | `剪映AI` wordmark (Jianying / 剪映, the China product in the CapCut family; bottom-right) | 2 corpus files | POS 0.29 vs NEG max 0.35. |
|
||||
| `capcut_alpha.png` | CapCut AI pill (top-left; likely pill class, not plain text) | 3 corpus files | POS max 0.15 vs NEG max 0.33 -- locate geometry not yet fitted for the pill form. |
|
||||
| `zsky_alpha.png` | `MADE WITH zsky.ai` wordmark (ZSky AI, bottom-right) | 2 corpus files | POS 0.10 vs NEG max 0.27. |
|
||||
| `chromastudio_alpha.png` | `ChromaStudio.ai` wordmark (ChromaStudio, bottom-right) | 2 corpus files | POS 0.11 vs NEG max 0.29. |
|
||||
| `digenai_alpha.png` | `DIGENAI` wordmark (Digen AI, bottom-right) | 3 corpus files (one 2026-07-24 batch) | POS 0.16 vs NEG max 0.31. |
|
||||
| `gendo_alpha.png` | `GendoAI` wordmark (Gendo, bottom-left) | 3 corpus files | POS 0.08 vs NEG max 0.32. |
|
||||
| `xinghui_alpha.png` | 星绘AI生成 (parked before this set) | -- | prior parking, unchanged. |
|
||||
| `qingyan_alpha.png` | 清言·AI生成 (parked before this set) | -- | prior parking, unchanged. |
|
||||
| `hailuo_alpha.png` | Hailuo AI image wordmark (parked before this set; the VIDEO label is registered) | -- | prior parking, unchanged. |
|
||||
| `catlogo_alpha.png` | outline cat-head + AI生成 (parked before this set) | -- | prior parking, unchanged. |
|
||||
|
After Width: | Height: | Size: 2.7 KiB |
|
After Width: | Height: | Size: 2.6 KiB |
|
After Width: | Height: | Size: 2.9 KiB |
|
After Width: | Height: | Size: 3.4 KiB |
|
After Width: | Height: | Size: 1.9 KiB |
|
After Width: | Height: | Size: 3.4 KiB |
|
After Width: | Height: | Size: 1.9 KiB |
|
After Width: | Height: | Size: 3.9 KiB |
|
After Width: | Height: | Size: 3.0 KiB |
|
After Width: | Height: | Size: 2.7 KiB |
|
After Width: | Height: | Size: 1.7 KiB |
|
After Width: | Height: | Size: 2.7 KiB |
@@ -0,0 +1,114 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["modal>=1.5"]
|
||||
# ///
|
||||
"""Extra high Chroma1 rungs (0.25-0.40) for the Google cohort fixtures.
|
||||
|
||||
The 0.08-0.20 ladder came back detected end-to-end on the first fixture, so
|
||||
this appends the high rungs for every Google fixture without touching the
|
||||
already-oracled lower outputs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import modal
|
||||
|
||||
_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
app = modal.App("chroma-google-high-rungs")
|
||||
|
||||
image = (
|
||||
modal.Image.debian_slim(python_version="3.12")
|
||||
.pip_install(
|
||||
"torch",
|
||||
"diffusers>=0.38,<1",
|
||||
"transformers>=4.53",
|
||||
"accelerate",
|
||||
"sentencepiece",
|
||||
"protobuf",
|
||||
"pillow",
|
||||
"numpy",
|
||||
)
|
||||
.env({"HF_HOME": "/cache"})
|
||||
)
|
||||
|
||||
cache = modal.Volume.from_name("chroma-hf-cache", create_if_missing=True)
|
||||
|
||||
CHROMA_MODEL_ID = "lodestones/Chroma1-HD"
|
||||
RUNGS = (0.25, 0.30, 0.35, 0.40)
|
||||
EFFECTIVE_STEPS = 4
|
||||
_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
|
||||
FIXTURES = (
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png",
|
||||
"Gemini_Generated_Image_akdbeiakdbeiakdb.png",
|
||||
"Gemini_Generated_Image_y48j3cy48j3cy48j.png",
|
||||
"Gemini_Generated_Image_3mc4t93mc4t93mc4.png",
|
||||
)
|
||||
MAX_SIDE = 1536
|
||||
|
||||
|
||||
@app.function(image=image, gpu="H100", volumes={"/cache": cache}, timeout=2400)
|
||||
def run(payload: dict[str, bytes]) -> dict:
|
||||
import io
|
||||
import time
|
||||
|
||||
import torch
|
||||
from diffusers import ChromaImg2ImgPipeline
|
||||
from PIL import Image
|
||||
|
||||
chroma = ChromaImg2ImgPipeline.from_pretrained(CHROMA_MODEL_ID, torch_dtype=torch.bfloat16).to("cuda")
|
||||
out: dict = {}
|
||||
for name, data in payload.items():
|
||||
source = Image.open(io.BytesIO(data)).convert("RGB")
|
||||
if MAX_SIDE and max(source.size) > MAX_SIDE:
|
||||
scale = MAX_SIDE / max(source.size)
|
||||
source = source.resize(
|
||||
(round(source.width * scale), round(source.height * scale)),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
width = max(16, (source.width // 16) * 16)
|
||||
height = max(16, (source.height // 16) * 16)
|
||||
for strength in RUNGS:
|
||||
started = time.perf_counter()
|
||||
generator = torch.Generator(device="cpu").manual_seed(0)
|
||||
image = chroma(
|
||||
prompt=_PROMPT,
|
||||
negative_prompt=_NEGATIVE,
|
||||
image=source,
|
||||
width=width,
|
||||
height=height,
|
||||
strength=strength,
|
||||
num_inference_steps=max(1, math.ceil(EFFECTIVE_STEPS / strength)),
|
||||
guidance_scale=5.0,
|
||||
generator=generator,
|
||||
).images[0]
|
||||
if image.size != source.size:
|
||||
image = image.resize(source.size, Image.Resampling.LANCZOS)
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="PNG")
|
||||
out[f"{Path(name).stem}/chroma_s{strength:.2f}"] = {
|
||||
"png": buffer.getvalue(),
|
||||
"seconds": round(time.perf_counter() - started, 2),
|
||||
}
|
||||
print(f"{name} s={strength} done", flush=True)
|
||||
cache.commit()
|
||||
return out
|
||||
|
||||
|
||||
@app.local_entrypoint()
|
||||
def main(out: str = "out/cohort-calibration/google") -> None:
|
||||
payload = {name: (_ROOT / "data" / "synthid" / "originals" / name).read_bytes() for name in FIXTURES}
|
||||
results = run.remote(payload)
|
||||
timing = {}
|
||||
for key, item in results.items():
|
||||
destination = Path(out) / f"{key}.png"
|
||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||
destination.write_bytes(item["png"])
|
||||
timing[key] = item["seconds"]
|
||||
print(json.dumps(timing, indent=2))
|
||||
@@ -0,0 +1,109 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["modal>=1.5"]
|
||||
# ///
|
||||
"""Extra high Chroma1 rungs for the Meta Content Seal cohort.
|
||||
|
||||
The first calibration ladder topped out at 0.08 and the worst fixture
|
||||
(gen_lighthouse_watercolor) was still detected there, so this appends 0.10,
|
||||
0.12 and 0.15 for every Meta fixture without touching the already-oracled
|
||||
lower rungs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import modal
|
||||
|
||||
app = modal.App("chroma-meta-high-rungs")
|
||||
|
||||
_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
image = (
|
||||
modal.Image.debian_slim(python_version="3.12")
|
||||
.pip_install(
|
||||
"torch",
|
||||
"diffusers>=0.38,<1",
|
||||
"transformers>=4.53",
|
||||
"accelerate",
|
||||
"sentencepiece",
|
||||
"protobuf",
|
||||
"pillow",
|
||||
"numpy",
|
||||
)
|
||||
.env({"HF_HOME": "/cache"})
|
||||
)
|
||||
|
||||
cache = modal.Volume.from_name("chroma-hf-cache", create_if_missing=True)
|
||||
|
||||
CHROMA_MODEL_ID = "lodestones/Chroma1-HD"
|
||||
RUNGS = (0.17,)
|
||||
EFFECTIVE_STEPS = 4
|
||||
_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
|
||||
META_FIXTURES = (
|
||||
"gen_fox_forest.webp",
|
||||
"gen_text_poster.webp",
|
||||
"gen_lighthouse_watercolor.webp",
|
||||
"gen_night_city.webp",
|
||||
"gen_studio_mug.webp",
|
||||
)
|
||||
|
||||
|
||||
@app.function(image=image, gpu="H100", volumes={"/cache": cache}, timeout=1800)
|
||||
def run(payload: dict[str, bytes]) -> dict:
|
||||
import io
|
||||
import time
|
||||
|
||||
import torch
|
||||
from diffusers import ChromaImg2ImgPipeline
|
||||
from PIL import Image
|
||||
|
||||
chroma = ChromaImg2ImgPipeline.from_pretrained(CHROMA_MODEL_ID, torch_dtype=torch.bfloat16).to("cuda")
|
||||
out: dict = {}
|
||||
for name, data in payload.items():
|
||||
source = Image.open(io.BytesIO(data)).convert("RGB")
|
||||
width = max(16, (source.width // 16) * 16)
|
||||
height = max(16, (source.height // 16) * 16)
|
||||
for strength in RUNGS:
|
||||
started = time.perf_counter()
|
||||
generator = torch.Generator(device="cpu").manual_seed(0)
|
||||
image = chroma(
|
||||
prompt=_PROMPT,
|
||||
negative_prompt=_NEGATIVE,
|
||||
image=source,
|
||||
width=width,
|
||||
height=height,
|
||||
strength=strength,
|
||||
num_inference_steps=max(1, math.ceil(EFFECTIVE_STEPS / strength)),
|
||||
guidance_scale=5.0,
|
||||
generator=generator,
|
||||
).images[0]
|
||||
if image.size != source.size:
|
||||
image = image.resize(source.size, Image.Resampling.LANCZOS)
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="PNG")
|
||||
out[f"{Path(name).stem}/chroma_s{strength:.3f}"] = {
|
||||
"png": buffer.getvalue(),
|
||||
"seconds": round(time.perf_counter() - started, 2),
|
||||
}
|
||||
print(f"{name} s={strength} done", flush=True)
|
||||
cache.commit()
|
||||
return out
|
||||
|
||||
|
||||
@app.local_entrypoint()
|
||||
def main(out: str = "out/cohort-calibration/meta") -> None:
|
||||
payload = {name: (_ROOT / "data" / "contentseal" / "originals" / name).read_bytes() for name in META_FIXTURES}
|
||||
results = run.remote(payload)
|
||||
timing = {}
|
||||
for key, item in results.items():
|
||||
destination = Path(out) / f"{key}.png"
|
||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||
destination.write_bytes(item["png"])
|
||||
timing[key] = item["seconds"]
|
||||
print(json.dumps(timing, indent=2))
|
||||
@@ -0,0 +1,142 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["modal>=1.5"]
|
||||
# ///
|
||||
"""Chroma1 ladder for the Microsoft InvisMark cohort (three valid Paint sources)
|
||||
plus the matching qwen-zimage production runs at the shipped 0.15 floor.
|
||||
|
||||
The sources are the same three Paint carriers behind the qwen cohort floor
|
||||
(staged under out/cohort-calibration/microsoft/<label>/original.png with
|
||||
pixel-identical metadata-stripped controls). They are real user uploads from
|
||||
the raiw corpus and must never be committed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import modal
|
||||
|
||||
_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
app = modal.App("chroma-microsoft-ladder")
|
||||
|
||||
image = (
|
||||
modal.Image.debian_slim(python_version="3.12")
|
||||
.pip_install(
|
||||
"torch",
|
||||
"torchvision>=0.20.0",
|
||||
"diffusers>=0.38,<1",
|
||||
"diffsynth>=2.0.17,<3",
|
||||
"transformers>=4.53",
|
||||
"accelerate",
|
||||
"sentencepiece",
|
||||
"protobuf",
|
||||
"pillow",
|
||||
"numpy",
|
||||
"opencv-python-headless",
|
||||
)
|
||||
.env({"HF_HOME": "/cache"})
|
||||
.add_local_dir(_ROOT / "src" / "remove_ai_watermarks", remote_path="/pkg/remove_ai_watermarks")
|
||||
)
|
||||
|
||||
cache = modal.Volume.from_name("chroma-hf-cache", create_if_missing=True)
|
||||
|
||||
CHROMA_MODEL_ID = "lodestones/Chroma1-HD"
|
||||
CHROMA_RUNGS = (0.04, 0.06, 0.08, 0.10, 0.125, 0.15, 0.20)
|
||||
QWEN_FLOOR = 0.15
|
||||
EFFECTIVE_STEPS = 4
|
||||
_SOURCES = ("paint-1", "paint-2", "paint-3")
|
||||
_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
|
||||
|
||||
@app.function(image=image, gpu="H100", volumes={"/cache": cache}, timeout=2400)
|
||||
def run(payload: dict[str, bytes]) -> dict:
|
||||
import io
|
||||
import sys
|
||||
import time
|
||||
|
||||
import torch
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, "/pkg")
|
||||
|
||||
def chroma_call(chroma: object, source: Image.Image, strength: float, seed: int) -> Image.Image:
|
||||
width = max(16, (source.width // 16) * 16)
|
||||
height = max(16, (source.height // 16) * 16)
|
||||
generator = torch.Generator(device="cpu").manual_seed(seed)
|
||||
out = chroma(
|
||||
prompt=_PROMPT,
|
||||
negative_prompt=_NEGATIVE,
|
||||
image=source,
|
||||
width=width,
|
||||
height=height,
|
||||
strength=strength,
|
||||
num_inference_steps=max(1, math.ceil(EFFECTIVE_STEPS / strength)),
|
||||
guidance_scale=5.0,
|
||||
generator=generator,
|
||||
).images[0]
|
||||
return out if out.size == source.size else out.resize(source.size, Image.Resampling.LANCZOS)
|
||||
|
||||
def png_bytes(image: Image.Image) -> bytes:
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="PNG")
|
||||
return buffer.getvalue()
|
||||
|
||||
from diffusers import ChromaImg2ImgPipeline
|
||||
|
||||
chroma = ChromaImg2ImgPipeline.from_pretrained(CHROMA_MODEL_ID, torch_dtype=torch.bfloat16).to("cuda")
|
||||
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import QwenZImagePipeline
|
||||
|
||||
qwen = QwenZImagePipeline(
|
||||
device="cuda",
|
||||
torch_dtype=torch.bfloat16,
|
||||
keep_global_models_on_device=True,
|
||||
keep_face_models_on_device=True,
|
||||
)
|
||||
|
||||
out: dict = {}
|
||||
for label in _SOURCES:
|
||||
source = Image.open(io.BytesIO(payload[label])).convert("RGB")
|
||||
rungs: dict = {}
|
||||
for strength in CHROMA_RUNGS:
|
||||
started = time.perf_counter()
|
||||
image = chroma_call(chroma, source, strength, 0)
|
||||
rungs[f"{strength:.4f}"] = {
|
||||
"png": png_bytes(image),
|
||||
"seconds": round(time.perf_counter() - started, 2),
|
||||
}
|
||||
print(f"{label} chroma s={strength} done", flush=True)
|
||||
started = time.perf_counter()
|
||||
qwen_out = qwen.run(source, strength=QWEN_FLOOR, seed=0)
|
||||
out[label] = {
|
||||
"chroma": rungs,
|
||||
"qwen_seconds": round(time.perf_counter() - started, 2),
|
||||
"qwen_png": png_bytes(qwen_out),
|
||||
}
|
||||
print(f"{label} qwen s={QWEN_FLOOR} done", flush=True)
|
||||
|
||||
cache.commit()
|
||||
return out
|
||||
|
||||
|
||||
@app.local_entrypoint()
|
||||
def main(staged: str = "out/cohort-calibration/microsoft") -> None:
|
||||
staged_root = Path(staged)
|
||||
payload = {label: (staged_root / label / "original.png").read_bytes() for label in _SOURCES}
|
||||
results = run.remote(payload)
|
||||
summary: dict = {}
|
||||
for label, entry in results.items():
|
||||
target = staged_root / label
|
||||
for strength, item in entry["chroma"].items():
|
||||
(target / f"chroma_s{float(strength):.4f}.png").write_bytes(item["png"])
|
||||
(target / "qwen_floor.png").write_bytes(entry["qwen_png"])
|
||||
summary[label] = {
|
||||
"chroma_seconds": {k: v["seconds"] for k, v in entry["chroma"].items()},
|
||||
"qwen_seconds": entry["qwen_seconds"],
|
||||
}
|
||||
print(json.dumps(summary, indent=2))
|
||||
@@ -0,0 +1,152 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["modal>=1.5"]
|
||||
# ///
|
||||
"""Pre-ship validation outputs for the chroma-zimage profile.
|
||||
|
||||
1. Seed sensitivity of the measured boundaries: chroma at the worst-fixture
|
||||
first-clean rungs (google 633uuy @ 0.25, meta lighthouse @ 0.10) with
|
||||
seeds 1 and 2, to check the boundary is not a seed-0 fluke.
|
||||
2. Face-stage interaction: the FULL chroma-zimage pipeline (Chroma1 global at
|
||||
the shipped Google floor 0.40, then the inherited YuNet/SAM/Z-Image face
|
||||
stage) on the 5-face fixture 3mc4t9, to verify the composited face regions
|
||||
do not re-introduce a detectable signal on top of the floor.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import modal
|
||||
|
||||
_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
app = modal.App("chroma-preship-validation")
|
||||
|
||||
image = (
|
||||
modal.Image.debian_slim(python_version="3.12")
|
||||
.pip_install(
|
||||
"torch",
|
||||
"torchvision>=0.20.0",
|
||||
"diffusers>=0.38,<1",
|
||||
"diffsynth>=2.0.17,<3",
|
||||
"transformers>=4.53",
|
||||
"accelerate",
|
||||
"sentencepiece",
|
||||
"protobuf",
|
||||
"pillow",
|
||||
"numpy",
|
||||
"opencv-python-headless",
|
||||
)
|
||||
.env({"HF_HOME": "/cache"})
|
||||
.add_local_dir(_ROOT / "src" / "remove_ai_watermarks", remote_path="/pkg/remove_ai_watermarks")
|
||||
)
|
||||
|
||||
cache = modal.Volume.from_name("chroma-hf-cache", create_if_missing=True)
|
||||
|
||||
CHROMA_MODEL_ID = "lodestones/Chroma1-HD"
|
||||
_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
GOOGLE_FLOOR = 0.40
|
||||
MAX_SIDE = 1536
|
||||
|
||||
|
||||
@app.function(image=image, gpu="H100", volumes={"/cache": cache}, timeout=2400)
|
||||
def run(payload: dict[str, bytes]) -> dict:
|
||||
import io
|
||||
import time
|
||||
|
||||
import torch
|
||||
from diffusers import ChromaImg2ImgPipeline
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, "/pkg")
|
||||
|
||||
def chroma_call(pipe: object, source: Image.Image, strength: float, seed: int) -> Image.Image:
|
||||
width = max(16, (source.width // 16) * 16)
|
||||
height = max(16, (source.height // 16) * 16)
|
||||
generator = torch.Generator(device="cpu").manual_seed(seed)
|
||||
out = pipe(
|
||||
prompt=_PROMPT,
|
||||
negative_prompt=_NEGATIVE,
|
||||
image=source,
|
||||
width=width,
|
||||
height=height,
|
||||
strength=strength,
|
||||
num_inference_steps=max(1, math.ceil(4 / strength)),
|
||||
guidance_scale=5.0,
|
||||
generator=generator,
|
||||
).images[0]
|
||||
return out if out.size == source.size else out.resize(source.size, Image.Resampling.LANCZOS)
|
||||
|
||||
def png_bytes(image: Image.Image) -> bytes:
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="PNG")
|
||||
return buffer.getvalue()
|
||||
|
||||
out: dict = {}
|
||||
|
||||
chroma = ChromaImg2ImgPipeline.from_pretrained(CHROMA_MODEL_ID, torch_dtype=torch.bfloat16).to("cuda")
|
||||
|
||||
# Seed sensitivity at the worst-fixture boundaries.
|
||||
for key, strength in (("google_633uuy", 0.25), ("meta_lighthouse", 0.10)):
|
||||
source = Image.open(io.BytesIO(payload[key])).convert("RGB")
|
||||
if MAX_SIDE and max(source.size) > MAX_SIDE:
|
||||
scale = MAX_SIDE / max(source.size)
|
||||
source = source.resize(
|
||||
(round(source.width * scale), round(source.height * scale)),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
for seed in (1, 2):
|
||||
started = time.perf_counter()
|
||||
image = chroma_call(chroma, source, strength, seed)
|
||||
out[f"seedprobe_{key}_s{strength}_seed{seed}"] = png_bytes(image)
|
||||
print(f"{key} s={strength} seed={seed} ({time.perf_counter() - started:.1f}s)", flush=True)
|
||||
|
||||
del chroma
|
||||
import gc
|
||||
|
||||
gc.collect()
|
||||
torch.cuda.empty_cache()
|
||||
|
||||
# Full chroma-zimage pipeline on the 5-face fixture at the shipped floor.
|
||||
from remove_ai_watermarks._internal.chroma_zimage_pipeline import ChromaZImagePipeline
|
||||
|
||||
full = ChromaZImagePipeline(
|
||||
device="cuda",
|
||||
torch_dtype=torch.bfloat16,
|
||||
keep_global_models_on_device=True,
|
||||
keep_face_models_on_device=True,
|
||||
)
|
||||
source = Image.open(io.BytesIO(payload["google_3mc4t9"])).convert("RGB")
|
||||
if MAX_SIDE and max(source.size) > MAX_SIDE:
|
||||
scale = MAX_SIDE / max(source.size)
|
||||
source = source.resize(
|
||||
(round(source.width * scale), round(source.height * scale)),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
started = time.perf_counter()
|
||||
result = full.run(source, strength=GOOGLE_FLOOR, seed=0)
|
||||
out["fullpath_3mc4t9_s0.40"] = png_bytes(result)
|
||||
print(f"full pipeline 3mc4t9 @ {GOOGLE_FLOOR} ({time.perf_counter() - started:.1f}s)", flush=True)
|
||||
|
||||
cache.commit()
|
||||
return out
|
||||
|
||||
|
||||
@app.local_entrypoint()
|
||||
def main(out: str = "out/preship-chroma") -> None:
|
||||
sources = {
|
||||
"google_633uuy": _ROOT / "data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png",
|
||||
"meta_lighthouse": _ROOT / "data/contentseal/originals/gen_lighthouse_watercolor.webp",
|
||||
"google_3mc4t9": _ROOT / "data/synthid/originals/Gemini_Generated_Image_3mc4t93mc4t93mc4.png",
|
||||
}
|
||||
results = run.remote({k: p.read_bytes() for k, p in sources.items()})
|
||||
root = Path(out)
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
for key, png in results.items():
|
||||
(root / f"{key}.png").write_bytes(png)
|
||||
print(json.dumps(sorted(results), indent=2))
|
||||
@@ -0,0 +1,188 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["modal>=1.5"]
|
||||
# ///
|
||||
"""Isolated Modal GPU prototype: does a low-strength Chroma1 img2img pass scrub
|
||||
the invisible watermark while keeping text/faces faithful?
|
||||
|
||||
Chroma1 (lodestones, Apache-2.0) is the FLUX.1 architecture re-trained on open
|
||||
data, and diffusers ships it a native strength img2img pipeline, which is exactly
|
||||
what FLUX.2 lacks (see docs/synthid.md and issue #88). This script is the same
|
||||
shape as qwen_scrub_prototype.py: standalone, not imported by the package and
|
||||
not in uv.lock, so the locked environment never moves for an experiment.
|
||||
|
||||
Run from the repository root (weights ~34 GB download once into a Modal volume,
|
||||
then every later run starts warm):
|
||||
|
||||
uvx modal run scripts/chroma_scrub_prototype.py \
|
||||
--source data/synthid/originals/'ChatGPT Image May 31, 2026, 02_03_55 PM.png' \
|
||||
--out out/chroma --strengths 0.05,0.1,0.15,0.2
|
||||
|
||||
A different GPU (default H100, ~$3.95/h; the 34 GB bf16 stack also fits
|
||||
A100-80GB at ~$2.50/h):
|
||||
|
||||
CHROMA_GPU=A100-80GB uvx modal run scripts/chroma_scrub_prototype.py ...
|
||||
|
||||
What to look for (same protocol as the Qwen study, docs/synthid.md 2.2):
|
||||
|
||||
* SCRUB: the oracle no longer reports the watermark at some strength
|
||||
(openai.com/verify for OpenAI carriers, the Gemini app for Google ones).
|
||||
* FIDELITY: text and faces stay faithful at that same strength; score with
|
||||
scripts/fidelity_metrics.py after the oracle pass.
|
||||
The smallest strength that clears the oracle while keeping fidelity is the
|
||||
number to compare against the qwen-zimage floors (OpenAI 0.07675 / Google 0.25).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import modal
|
||||
|
||||
_GPU = os.environ.get("CHROMA_GPU", "H100")
|
||||
|
||||
app = modal.App("chroma-scrub-proto")
|
||||
|
||||
image = (
|
||||
modal.Image.debian_slim(python_version="3.12")
|
||||
.pip_install(
|
||||
"torch",
|
||||
"diffusers>=0.38,<1",
|
||||
"transformers>=4.53",
|
||||
"accelerate",
|
||||
"sentencepiece",
|
||||
"protobuf",
|
||||
"pillow",
|
||||
"numpy",
|
||||
)
|
||||
.env({"HF_HOME": "/cache"})
|
||||
)
|
||||
|
||||
# Model weights persist here across runs, so only the first container downloads.
|
||||
cache = modal.Volume.from_name("chroma-hf-cache", create_if_missing=True)
|
||||
|
||||
CHROMA_MODEL_ID = "lodestones/Chroma1-HD"
|
||||
|
||||
# A neutral, faithful-regeneration prompt (scrub, not restyle); the same intent
|
||||
# as the Qwen prototype and the shipped global prompts.
|
||||
_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
|
||||
_pipe = None
|
||||
|
||||
|
||||
def _load_pipe() -> Any:
|
||||
global _pipe
|
||||
if _pipe is not None:
|
||||
return _pipe
|
||||
import torch
|
||||
from diffusers import ChromaImg2ImgPipeline
|
||||
|
||||
print(f"loading {CHROMA_MODEL_ID} (bf16)...", flush=True)
|
||||
_pipe = ChromaImg2ImgPipeline.from_pretrained(CHROMA_MODEL_ID, torch_dtype=torch.bfloat16)
|
||||
_pipe = _pipe.to("cuda")
|
||||
return _pipe
|
||||
|
||||
|
||||
def _psnr(a: Any, b: Any) -> float:
|
||||
import numpy as np
|
||||
|
||||
x = np.asarray(a.convert("RGB"), dtype=np.float64)
|
||||
y = np.asarray(b.convert("RGB"), dtype=np.float64)
|
||||
mse = float(((x - y) ** 2).mean())
|
||||
return 99.0 if mse <= 1e-12 else 10.0 * float(np.log10(255.0**2 / mse))
|
||||
|
||||
|
||||
@app.function(
|
||||
image=image,
|
||||
gpu=_GPU,
|
||||
volumes={"/cache": cache},
|
||||
timeout=2400,
|
||||
)
|
||||
def scrub(
|
||||
source_bytes: bytes,
|
||||
strengths: list[float],
|
||||
steps: int,
|
||||
guidance: float,
|
||||
seed: int,
|
||||
max_side: int,
|
||||
) -> list[tuple[str, bytes, float]]:
|
||||
"""Sweep one fixture over the strengths; return name, PNG bytes, and PSNR."""
|
||||
import io
|
||||
|
||||
import torch
|
||||
from PIL import Image
|
||||
|
||||
pipe = _load_pipe()
|
||||
source = Image.open(io.BytesIO(source_bytes)).convert("RGB")
|
||||
if max_side and max(source.size) > max_side:
|
||||
scale = max_side / max(source.size)
|
||||
source = source.resize(
|
||||
(round(source.width * scale), round(source.height * scale)),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
# The Flux-family img2img pipelines default to 1024x1024 when height/width
|
||||
# are not passed, silently discarding the input aspect. Floor to the /16
|
||||
# latent-patch grid, the same rule the shipped profiles use.
|
||||
width = max(16, (source.width // 16) * 16)
|
||||
height = max(16, (source.height // 16) * 16)
|
||||
|
||||
results: list[tuple[str, bytes, float]] = []
|
||||
for strength in strengths:
|
||||
generator = torch.Generator(device="cpu").manual_seed(seed)
|
||||
print(f"strength={strength:.2f} on {width}x{height}...", flush=True)
|
||||
output = pipe(
|
||||
prompt=_PROMPT,
|
||||
negative_prompt=_NEGATIVE,
|
||||
image=source,
|
||||
width=width,
|
||||
height=height,
|
||||
strength=strength,
|
||||
num_inference_steps=steps,
|
||||
guidance_scale=guidance,
|
||||
generator=generator,
|
||||
).images[0]
|
||||
if output.size != source.size:
|
||||
output = output.resize(source.size, Image.Resampling.LANCZOS)
|
||||
buffer = io.BytesIO()
|
||||
output.save(buffer, format="PNG")
|
||||
results.append((f"s{strength:.2f}", buffer.getvalue(), _psnr(source, output)))
|
||||
print(f" psnr vs source: {results[-1][2]:.2f} dB", flush=True)
|
||||
cache.commit()
|
||||
return results
|
||||
|
||||
|
||||
@app.local_entrypoint()
|
||||
def main(
|
||||
source: str,
|
||||
strengths: str = "0.05,0.1,0.15,0.2",
|
||||
out: str = "out/chroma",
|
||||
steps: int = 35,
|
||||
guidance: float = 5.0,
|
||||
seed: int = 0,
|
||||
max_side: int = 1536,
|
||||
) -> None:
|
||||
"""Sweep Chroma1 img2img strength over SOURCE and save one output per strength."""
|
||||
source_path = Path(source)
|
||||
if not source_path.exists():
|
||||
raise SystemExit(f"source not found: {source_path}")
|
||||
values = [float(s) for s in strengths.split(",") if s.strip()]
|
||||
stem = source_path.stem
|
||||
|
||||
print(f"gpu={_GPU} model={CHROMA_MODEL_ID} steps={steps} guidance={guidance} seed={seed}")
|
||||
outputs = scrub.remote(source_path.read_bytes(), values, steps, guidance, seed, max_side)
|
||||
|
||||
out_dir = Path(out)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
for name, payload, psnr in outputs:
|
||||
destination = out_dir / f"{stem}_chroma_{name}.png"
|
||||
destination.write_bytes(payload)
|
||||
print(f"saved {destination} (psnr {psnr:.2f} dB vs source)")
|
||||
|
||||
print(
|
||||
"\nDone. Submit each output to the matching oracle (openai.com/verify for "
|
||||
"OpenAI carriers, the Gemini app for Google ones). The smallest strength "
|
||||
"that clears the oracle while keeping fidelity is the Chroma1 floor."
|
||||
)
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Can one GENERIC template cover the CJK AI labels no per-vendor detector fires on?
|
||||
|
||||
THE OPPORTUNITY
|
||||
Corpus inspection of doubao-provenance misses turned up `千问AI生成` (Alibaba Qwen) and
|
||||
Corpus inspection of doubao-provenance misses turned up `千问AI生成` (Alibaba Cloud Qwen) and
|
||||
`百度 AI生成` (Baidu) sitting in the same bottom-right corner as the marks we do cover,
|
||||
bold and plainly legible, with no detector able to fire on either. `docs/...landscape`
|
||||
puts uncovered vendors at ~6% of sampled images -- larger than any tuning gain left in
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Reproduce the deterministic Content Seal crop, resize, and JPEG variants."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CORPUS = ROOT / "data" / "contentseal"
|
||||
MANIFEST = CORPUS / "manifest.csv"
|
||||
|
||||
|
||||
def _rows() -> dict[str, dict[str, str]]:
|
||||
with MANIFEST.open(newline="", encoding="utf-8") as stream:
|
||||
return {row["name"]: row for row in csv.DictReader(stream)}
|
||||
|
||||
|
||||
def _write_and_verify(image: Image.Image, path: Path, row: dict[str, str], *, format: str, quality: int) -> None:
|
||||
image.save(path, format=format, quality=quality)
|
||||
digest = hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
if digest != row["sha256"]:
|
||||
raise RuntimeError(f"{row['name']} hash mismatch: expected {row['sha256']}, got {digest}")
|
||||
log.info("Verified %s", path)
|
||||
|
||||
|
||||
def reproduce_transforms(output_dir: Path) -> list[Path]:
|
||||
"""Write and hash-check the eight deterministic manifest variants."""
|
||||
rows = _rows()
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
outputs: list[Path] = []
|
||||
|
||||
for prefix, source_name in (("fox", "gen_fox_forest"), ("text", "gen_text_poster")):
|
||||
with Image.open(CORPUS / rows[source_name]["file"]) as opened:
|
||||
source = opened.convert("RGB")
|
||||
|
||||
for fraction in (0.5, 0.33):
|
||||
width = int(source.width * fraction)
|
||||
height = int(source.height * fraction)
|
||||
left = (source.width - width) // 2
|
||||
top = (source.height - height) // 2
|
||||
name = f"{prefix}_crop{int(fraction * 100)}"
|
||||
path = output_dir / f"{name}.webp"
|
||||
crop = source.crop((left, top, left + width, top + height))
|
||||
_write_and_verify(crop, path, rows[name], format="WEBP", quality=95)
|
||||
outputs.append(path)
|
||||
|
||||
scale = 512 / max(source.size)
|
||||
resized = source.resize(
|
||||
(round(source.width * scale), round(source.height * scale)),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
name = f"{prefix}_res512"
|
||||
path = output_dir / f"{name}.webp"
|
||||
_write_and_verify(resized, path, rows[name], format="WEBP", quality=95)
|
||||
outputs.append(path)
|
||||
|
||||
name = f"{prefix}_jpeg85"
|
||||
path = output_dir / f"{name}.jpg"
|
||||
_write_and_verify(source, path, rows[name], format="JPEG", quality=85)
|
||||
outputs.append(path)
|
||||
|
||||
return outputs
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("output_dir", type=Path, help="Directory for regenerated variants")
|
||||
args = parser.parse_args()
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
|
||||
reproduce_transforms(args.output_dir)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,241 @@
|
||||
"""Per-method wall time for metadata extraction and the verdict built on it.
|
||||
|
||||
WHY THIS EXISTS
|
||||
The detection path is documented by CAPABILITY -- which signals it reads, in what
|
||||
order, with what confidence -- but not by COST. Batch throughput and the question
|
||||
"which probe dominates, and does it depend on container or on file size" have never
|
||||
been measured.
|
||||
|
||||
WHAT IT MEASURES
|
||||
Only the file-backed half: metadata extraction, then the verdict evaluated on the
|
||||
extracted evidence. The pixel-backed detectors are deliberately out of scope.
|
||||
|
||||
COLD pass: one measurement on a file the process has never touched --
|
||||
``extract_provenance_evidence``. Only the first read of a file is genuinely cold,
|
||||
so it buys exactly one number, and that is the one a single-shot run pays.
|
||||
|
||||
WARM pass, with the filesystem cache now hot:
|
||||
1. ``extract_provenance_evidence`` as a whole.
|
||||
2. Its nine components, timed IN THE ORDER the dataclass constructs them and with
|
||||
the per-file caches cleared once beforehand -- so ``extract_c2pa_info`` carries
|
||||
the Rust manifest reader and the later ``get_ai_metadata`` sees the same warm
|
||||
cache it sees in production. Timing them in any other order moves that cost to
|
||||
a different row and flatters whichever ran second.
|
||||
3. ``identify_from_evidence``: pure verdict logic, the source is never reopened.
|
||||
4. ``identify(check_visible=False, check_invisible=False)`` -- extraction plus
|
||||
verdict as one call, the cross-check that 1 + 3 is the whole metadata path.
|
||||
|
||||
Every ``@lru_cache`` in the metadata and C2PA modules is cleared before each timed
|
||||
unit. Without that the second measurement of a file answers from the memo and
|
||||
reports a cost of zero -- the caches are keyed on (path, mtime, size) and this
|
||||
script reads each file several times.
|
||||
|
||||
READING IT
|
||||
Component times do NOT sum to the ``extract_provenance_evidence`` total for free:
|
||||
they come from a separate cache-cleared run, so the sum is a cross-check. A gap
|
||||
means a component is missing from the list. Both numbers are written.
|
||||
|
||||
DATA SAFETY
|
||||
Read-only over a local dataset. Writes only to the given output prefix, which
|
||||
belongs outside the repository.
|
||||
|
||||
uv run python scripts/detection_timing.py <dataset> <prefix> --limit 200
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable, Iterator
|
||||
|
||||
# The package's OWN tree, not the repository root: from a worktree, an editable
|
||||
# install resolves `remove_ai_watermarks` to the MAIN checkout, so a script measuring
|
||||
# this tree would silently import a different one.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
||||
|
||||
from remove_ai_watermarks import identify as identify_mod
|
||||
from remove_ai_watermarks import metadata as metadata_mod
|
||||
from remove_ai_watermarks._internal import c2pa as c2pa_mod
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SUPPORTED = frozenset({".png", ".jpg", ".jpeg", ".webp", ".heic", ".heif", ".avif"})
|
||||
|
||||
# Timed in the order ``ProvenanceEvidence`` is constructed in ``identify.py``. The
|
||||
# report script imports this to label its columns, so the order is the contract.
|
||||
COMPONENTS: tuple[tuple[str, Callable[[Path], Any]], ...] = (
|
||||
("c2pa_info", c2pa_mod.extract_c2pa_info),
|
||||
("ai_metadata", metadata_mod.get_ai_metadata),
|
||||
("scan_head", lambda p: metadata_mod.scan_head(p, identify_mod._SCAN_BYTES)),
|
||||
("iptc_ai_system", metadata_mod.iptc_ai_system),
|
||||
("aigc_label", metadata_mod.aigc_label),
|
||||
("exif_generator", metadata_mod.exif_generator),
|
||||
("xai_signature", metadata_mod.xai_signature),
|
||||
("huggingface_job", metadata_mod.huggingface_job),
|
||||
("samsung_genai", metadata_mod.samsung_genai),
|
||||
)
|
||||
|
||||
|
||||
def _cache_clearers() -> tuple[Callable[[], None], ...]:
|
||||
"""Every per-file memo in the metadata path, found by attribute, not by name list.
|
||||
|
||||
A hand-written list silently goes stale the next time a probe gains a cache, and a
|
||||
stale entry shows up as a suspiciously fast row rather than as an error.
|
||||
"""
|
||||
found: list[Callable[[], None]] = []
|
||||
for module in (metadata_mod, c2pa_mod):
|
||||
for name in dir(module):
|
||||
clear = getattr(getattr(module, name, None), "cache_clear", None)
|
||||
if callable(clear):
|
||||
found.append(clear)
|
||||
return tuple(found)
|
||||
|
||||
|
||||
_CLEARERS = _cache_clearers()
|
||||
|
||||
|
||||
def _clear() -> None:
|
||||
for clear in _CLEARERS:
|
||||
clear()
|
||||
|
||||
|
||||
def _ms(fn: Callable[[], Any]) -> tuple[float, Any]:
|
||||
"""Wall time in milliseconds plus the call's result."""
|
||||
start = time.perf_counter_ns()
|
||||
value = fn()
|
||||
return (time.perf_counter_ns() - start) / 1e6, value
|
||||
|
||||
|
||||
def _pixel_geometry(path: Path) -> tuple[str | None, int | None, int | None]:
|
||||
"""Container format and pixel dimensions from the header alone."""
|
||||
try:
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(path) as img:
|
||||
return img.format, img.width, img.height
|
||||
except Exception: # unreadable or an unsupported container
|
||||
return None, None, None
|
||||
|
||||
|
||||
def _warm_breakdown(path: Path, row: dict[str, Any]) -> None:
|
||||
"""Fill ``row`` with the warm-cache per-method breakdown."""
|
||||
_clear()
|
||||
row["extract_evidence_ms"], evidence = _ms(lambda: identify_mod.extract_provenance_evidence(path))
|
||||
|
||||
_clear()
|
||||
component_sum = 0.0
|
||||
for name, fn in COMPONENTS:
|
||||
elapsed, _ = _ms(lambda fn=fn: fn(path))
|
||||
row[f"meta_{name}_ms"] = elapsed
|
||||
component_sum += elapsed
|
||||
row["meta_components_sum_ms"] = component_sum
|
||||
|
||||
row["verdict_from_evidence_ms"], report = _ms(lambda: identify_mod.identify_from_evidence(evidence))
|
||||
|
||||
_clear()
|
||||
row["identify_metadata_only_ms"], full = _ms(
|
||||
lambda: identify_mod.identify(path, check_visible=False, check_invisible=False)
|
||||
)
|
||||
|
||||
row["scan_bytes"] = len(evidence.scan)
|
||||
row["has_c2pa"] = bool(evidence.c2pa_info)
|
||||
row["has_ai_metadata"] = bool(evidence.ai_metadata)
|
||||
row["is_ai_generated"] = full.is_ai_generated
|
||||
row["confidence"] = full.confidence
|
||||
row["platform"] = full.platform
|
||||
row["signals"] = [signal.name for signal in full.signals]
|
||||
# The two verdict paths must agree; a mismatch means the breakdown timed a
|
||||
# different code path than the end-to-end call and the rows are not comparable.
|
||||
row["verdict_agrees"] = (report.confidence, report.platform) == (full.confidence, full.platform)
|
||||
|
||||
|
||||
def _measure(path: Path) -> dict[str, Any]:
|
||||
row: dict[str, Any] = {"path": str(path), "ext": path.suffix.lower()}
|
||||
try:
|
||||
row["bytes"] = path.stat().st_size
|
||||
except OSError as exc:
|
||||
return {**row, "error": f"stat: {exc}"}
|
||||
|
||||
# COLD first: this is the only moment the file is untouched by this process.
|
||||
_clear()
|
||||
try:
|
||||
row["cold_extract_evidence_ms"], _ = _ms(lambda: identify_mod.extract_provenance_evidence(path))
|
||||
except Exception as exc:
|
||||
return {**row, "error": f"cold extract: {type(exc).__name__}: {exc}"}
|
||||
|
||||
row["format"], row["width"], row["height"] = _pixel_geometry(path)
|
||||
width, height = row["width"], row["height"]
|
||||
row["megapixels"] = round(width * height / 1e6, 3) if width and height else None
|
||||
|
||||
try:
|
||||
_warm_breakdown(path, row)
|
||||
except Exception as exc:
|
||||
row["error"] = f"warm pass: {type(exc).__name__}: {exc}"
|
||||
return row
|
||||
|
||||
|
||||
def _iter_images(root: Path) -> Iterator[Path]:
|
||||
for path in sorted(root.rglob("*")):
|
||||
if path.is_file() and path.suffix.lower() in SUPPORTED:
|
||||
yield path
|
||||
|
||||
|
||||
def _done_paths(out_path: Path) -> set[str]:
|
||||
"""Paths already recorded, so a long run resumes instead of restarting."""
|
||||
if not out_path.exists():
|
||||
return set()
|
||||
done: set[str] = set()
|
||||
with out_path.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
done.add(json.loads(line)["path"])
|
||||
except (ValueError, KeyError):
|
||||
continue
|
||||
return done
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("dataset", type=Path, help="directory of images, scanned recursively")
|
||||
parser.add_argument("out_prefix", type=Path, help="output prefix; writes <prefix>.jsonl")
|
||||
parser.add_argument("--limit", type=int, default=0, help="stop after N files (0 = all)")
|
||||
parser.add_argument("--progress-every", type=int, default=200)
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
|
||||
out_path = args.out_prefix.with_suffix(".jsonl")
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
done = _done_paths(out_path)
|
||||
if done:
|
||||
log.info("resuming: %d files already recorded", len(done))
|
||||
|
||||
processed = 0
|
||||
started = time.monotonic()
|
||||
with out_path.open("a", encoding="utf-8") as handle:
|
||||
for path in _iter_images(args.dataset):
|
||||
if str(path) in done:
|
||||
continue
|
||||
row = _measure(path)
|
||||
handle.write(json.dumps(row, ensure_ascii=False, default=str) + "\n")
|
||||
handle.flush()
|
||||
processed += 1
|
||||
if processed % args.progress_every == 0:
|
||||
log.info("%d files, %.2f files/s", processed, processed / (time.monotonic() - started))
|
||||
if args.limit and processed >= args.limit:
|
||||
break
|
||||
|
||||
elapsed = time.monotonic() - started
|
||||
rate = processed / max(elapsed, 1e-9)
|
||||
log.info("done: %d files in %.1f s (%.2f files/s) -> %s", processed, elapsed, rate, out_path)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,232 @@
|
||||
"""Aggregate ``detection_timing.py`` records into per-method, per-segment tables.
|
||||
|
||||
WHAT IT PRODUCES
|
||||
Reading it back: the per-file JSONL is one row per image with a millisecond field
|
||||
per method. This collapses it into percentiles per method, then repeats that per
|
||||
segment -- container, megapixels, file size, C2PA presence, verdict confidence --
|
||||
because a single median hides a path whose cost is carried entirely by one
|
||||
container or by the files that actually have a manifest.
|
||||
|
||||
Writes ``<prefix>_summary.csv`` (long form: segment_kind, segment, method, n, p50,
|
||||
p90, p99, mean) and prints a markdown report.
|
||||
|
||||
Percentiles are computed by nearest-rank on the sorted sample, not interpolated:
|
||||
every reported number is a time some real file actually took.
|
||||
|
||||
DATA SAFETY
|
||||
Reads and writes only the given prefix, which belongs outside the repository.
|
||||
|
||||
uv run python scripts/detection_timing_report.py .local-eval/timing/full
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import math
|
||||
import statistics
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable, Iterator
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from detection_timing import COMPONENTS as _TIMED
|
||||
|
||||
# Taken from the script that WROTE the rows, in its order, so a probe added, removed
|
||||
# or reordered there cannot silently leave a column missing or mislabeled here.
|
||||
COMPONENTS = tuple(name for name, _ in _TIMED)
|
||||
METHODS = (
|
||||
("cold_extract_evidence_ms", "extract_provenance_evidence (cold)"),
|
||||
("extract_evidence_ms", "extract_provenance_evidence (warm)"),
|
||||
*((f"meta_{name}_ms", f" {name}") for name in COMPONENTS),
|
||||
("meta_components_sum_ms", " (sum of components)"),
|
||||
("verdict_from_evidence_ms", "identify_from_evidence (verdict)"),
|
||||
("identify_metadata_only_ms", "identify(metadata only), end to end"),
|
||||
)
|
||||
# Median-only columns in the per-segment tables; the full percentile set goes to the CSV.
|
||||
HEADLINE_LABELS = (
|
||||
("extract_evidence_ms", "extract p50"),
|
||||
("verdict_from_evidence_ms", "verdict p50"),
|
||||
("identify_metadata_only_ms", "end-to-end p50"),
|
||||
)
|
||||
HEADLINE = tuple(key for key, _ in HEADLINE_LABELS)
|
||||
|
||||
|
||||
def _bucket(value: float | None, edges: tuple[float, ...], labels: tuple[str, ...]) -> str:
|
||||
if value is None:
|
||||
return "unknown"
|
||||
for edge, label in zip(edges, labels, strict=False):
|
||||
if value < edge:
|
||||
return label
|
||||
return labels[-1]
|
||||
|
||||
|
||||
SEGMENTS: tuple[tuple[str, Callable[[dict[str, Any]], str]], ...] = (
|
||||
("container", lambda r: str(r.get("format") or "unknown")),
|
||||
(
|
||||
"megapixels",
|
||||
lambda r: _bucket(r.get("megapixels"), (1, 4, 12), ("<1 MP", "1-4 MP", "4-12 MP", ">12 MP")),
|
||||
),
|
||||
(
|
||||
"file size",
|
||||
lambda r: _bucket(
|
||||
(r["bytes"] / 1e6) if r.get("bytes") is not None else None,
|
||||
(1, 5, 20),
|
||||
("<1 MB", "1-5 MB", "5-20 MB", ">20 MB"),
|
||||
),
|
||||
),
|
||||
("c2pa", lambda r: "with C2PA" if r.get("has_c2pa") else "no C2PA"),
|
||||
("verdict", lambda r: f"confidence={r.get('confidence') or 'n/a'}"),
|
||||
)
|
||||
|
||||
|
||||
def _numeric(records: list[dict[str, Any]], key: str) -> list[float]:
|
||||
"""The numeric values of ``key``, skipping rows where it is absent or non-numeric.
|
||||
|
||||
One definition of "counts as a measurement", so the printed table and the CSV can
|
||||
never disagree about which rows a method was measured on.
|
||||
"""
|
||||
return [float(r[key]) for r in records if isinstance(r.get(key), (int, float))]
|
||||
|
||||
|
||||
def _percentile(sorted_values: list[float], q: float) -> float:
|
||||
"""Nearest-rank percentile: the reported value is one a real file produced."""
|
||||
index = max(0, math.ceil(q * len(sorted_values)) - 1)
|
||||
return sorted_values[index]
|
||||
|
||||
|
||||
def _stats(values: list[float]) -> dict[str, float | int]:
|
||||
ordered = sorted(values)
|
||||
return {
|
||||
"n": len(ordered),
|
||||
"p50": _percentile(ordered, 0.50),
|
||||
"p90": _percentile(ordered, 0.90),
|
||||
"p99": _percentile(ordered, 0.99),
|
||||
"mean": statistics.fmean(ordered),
|
||||
}
|
||||
|
||||
|
||||
def _rows(path: Path) -> Iterator[dict[str, Any]]:
|
||||
with path.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
yield json.loads(line)
|
||||
except ValueError: # a truncated tail while the run is still writing
|
||||
continue
|
||||
|
||||
|
||||
def _table(rows: list[dict[str, Any]], methods: tuple[tuple[str, str], ...]) -> list[str]:
|
||||
out = ["| method | n | p50 | p90 | p99 | mean |", "|---|---:|---:|---:|---:|---:|"]
|
||||
for key, label in methods:
|
||||
values = _numeric(rows, key)
|
||||
if not values:
|
||||
continue
|
||||
s = _stats(values)
|
||||
out.append(f"| {label} | {s['n']} | {s['p50']:.3f} | {s['p90']:.3f} | {s['p99']:.3f} | {s['mean']:.3f} |")
|
||||
return out
|
||||
|
||||
|
||||
def _correlation(rows: list[dict[str, Any]], x_key: str, y_key: str) -> float | None:
|
||||
both = [r for r in rows if isinstance(r.get(x_key), (int, float)) and isinstance(r.get(y_key), (int, float))]
|
||||
if len(both) < 3:
|
||||
return None
|
||||
xs = _numeric(both, x_key)
|
||||
ys = _numeric(both, y_key)
|
||||
try:
|
||||
return statistics.correlation(xs, ys)
|
||||
except statistics.StatisticsError: # a constant column has no correlation
|
||||
return None
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("prefix", type=Path, help="prefix given to detection_timing.py")
|
||||
args = parser.parse_args()
|
||||
|
||||
jsonl = args.prefix.with_suffix(".jsonl")
|
||||
all_rows = list(_rows(jsonl))
|
||||
failed = [r for r in all_rows if "error" in r]
|
||||
rows = [r for r in all_rows if "error" not in r]
|
||||
if not rows:
|
||||
print(f"no usable records in {jsonl}")
|
||||
return 1
|
||||
|
||||
disagreed = [r for r in rows if r.get("verdict_agrees") is False]
|
||||
|
||||
lines = [
|
||||
f"# Detection timing over {len(rows)} images",
|
||||
"",
|
||||
f"Source: `{jsonl}`. Failed rows: {len(failed)}. Verdict-path mismatches: {len(disagreed)}.",
|
||||
"",
|
||||
"All times in milliseconds. Percentiles are nearest-rank.",
|
||||
"",
|
||||
"## Whole corpus",
|
||||
"",
|
||||
*_table(rows, METHODS),
|
||||
"",
|
||||
]
|
||||
|
||||
csv_rows: list[dict[str, Any]] = []
|
||||
for key, label in METHODS:
|
||||
values = _numeric(rows, key)
|
||||
if values:
|
||||
csv_rows.append({"segment_kind": "all", "segment": "all", "method": label.strip(), **_stats(values)})
|
||||
|
||||
for kind, classify in SEGMENTS:
|
||||
groups: dict[str, list[dict[str, Any]]] = defaultdict(list)
|
||||
for row in rows:
|
||||
groups[classify(row)].append(row)
|
||||
lines += [f"## By {kind}", ""]
|
||||
header = ["| segment | n | " + " | ".join(label for _, label in HEADLINE_LABELS) + " |"]
|
||||
header.append("|---|---:|" + "---:|" * len(HEADLINE))
|
||||
lines += header
|
||||
for segment, group in sorted(groups.items(), key=lambda item: -len(item[1])):
|
||||
cells = []
|
||||
for key in HEADLINE:
|
||||
values = _numeric(group, key)
|
||||
cells.append(f"{_percentile(sorted(values), 0.5):.2f}" if values else "-")
|
||||
lines.append(f"| {segment} | {len(group)} | " + " | ".join(cells) + " |")
|
||||
for key, label in METHODS:
|
||||
values = _numeric(group, key)
|
||||
if values:
|
||||
csv_rows.append(
|
||||
{"segment_kind": kind, "segment": segment, "method": label.strip(), **_stats(values)}
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
corr = _correlation(rows, "scan_bytes", "verdict_from_evidence_ms")
|
||||
corr_size = _correlation(rows, "bytes", "extract_evidence_ms")
|
||||
lines += [
|
||||
"## Scaling",
|
||||
"",
|
||||
f"- verdict time vs scan-buffer size: r = {corr:.3f}" if corr is not None else "- verdict correlation: n/a",
|
||||
(
|
||||
f"- extraction time vs file size: r = {corr_size:.3f}"
|
||||
if corr_size is not None
|
||||
else "- extraction correlation: n/a"
|
||||
),
|
||||
"",
|
||||
]
|
||||
|
||||
summary_csv = args.prefix.with_name(args.prefix.name + "_summary.csv")
|
||||
columns = ["segment_kind", "segment", "method", "n", "p50", "p90", "p99", "mean"]
|
||||
with summary_csv.open("w", encoding="utf-8", newline="") as handle:
|
||||
writer = csv.DictWriter(handle, fieldnames=columns)
|
||||
writer.writeheader()
|
||||
writer.writerows(csv_rows)
|
||||
|
||||
report = "\n".join(lines)
|
||||
args.prefix.with_name(args.prefix.name + "_report.md").write_text(report + "\n", encoding="utf-8")
|
||||
print(report)
|
||||
print(f"\nwrote {summary_csv}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,272 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["modal>=1.5"]
|
||||
# ///
|
||||
"""Chroma1 floor calibration across the committed vendor cohorts, plus the
|
||||
matching qwen-zimage production runs at each cohort's shipped floor, for the
|
||||
engine quality/price comparison.
|
||||
|
||||
Every Chroma1 run spends a fixed 4 effective denoising steps (the requested
|
||||
count is scaled up, because Diffusers truncates the step COUNT), matching the
|
||||
semantics the shipped profiles use. Google fixtures are capped to a 1536 px
|
||||
long side, the same practical path the qwen floors were certified on; Meta
|
||||
sources run at native size like their cohort measurement.
|
||||
|
||||
The Microsoft InvisMark cohort is NOT here: its carrier fixtures are not
|
||||
committed to this repository (they were private sources), so no Chroma1 floor
|
||||
can be measured for it until carriers exist. Until then a chroma profile would
|
||||
treat Microsoft like sdxl-zimage treats Meta today: fall back to the
|
||||
conservative unknown floor.
|
||||
|
||||
Run from the repository root (both stacks are warm in the shared volume):
|
||||
|
||||
uvx modal run scripts/engine_cohort_calibration.py
|
||||
|
||||
Outputs land in out/cohort-calibration/<vendor>/<fixture>/ . Oracle each
|
||||
fixture's rungs in ASCENDING strength order and stop at the first clean one:
|
||||
that pair brackets the floor with the fewest checks.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import modal
|
||||
|
||||
_GPU = "H100"
|
||||
|
||||
app = modal.App("engine-cohort-calibration")
|
||||
|
||||
_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
image = (
|
||||
modal.Image.debian_slim(python_version="3.12")
|
||||
.pip_install(
|
||||
"torch",
|
||||
"torchvision>=0.20.0",
|
||||
"diffusers>=0.38,<1",
|
||||
"diffsynth>=2.0.17,<3",
|
||||
"transformers>=4.53",
|
||||
"accelerate",
|
||||
"sentencepiece",
|
||||
"protobuf",
|
||||
"pillow",
|
||||
"numpy",
|
||||
"opencv-python-headless",
|
||||
)
|
||||
.env({"HF_HOME": "/cache"})
|
||||
.add_local_dir(_ROOT / "src" / "remove_ai_watermarks", remote_path="/pkg/remove_ai_watermarks")
|
||||
)
|
||||
|
||||
cache = modal.Volume.from_name("chroma-hf-cache", create_if_missing=True)
|
||||
|
||||
CHROMA_MODEL_ID = "lodestones/Chroma1-HD"
|
||||
QWEN_OPENAI_FLOOR = 0.07675
|
||||
QWEN_GOOGLE_FLOOR = 0.27
|
||||
QWEN_META_FLOOR = 0.1
|
||||
CHROMA_EFFECTIVE_STEPS = 4
|
||||
|
||||
_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
|
||||
# vendor -> fixture -> {chroma rungs, qwen floor strength, max side}
|
||||
FIXTURES: dict[str, dict[str, dict]] = {
|
||||
"openai": {
|
||||
"ChatGPT Image May 30, 2026, 10_31_08 AM.png": { # June study 9-face grid, photoreal
|
||||
"chroma": (0.04, 0.05, 0.06, 0.075, 0.09, 0.12),
|
||||
"qwen": QWEN_OPENAI_FLOOR,
|
||||
"max_side": 1536,
|
||||
},
|
||||
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": { # full-pipeline quality fixture
|
||||
"chroma": (0.04, 0.05, 0.06, 0.075, 0.09),
|
||||
"qwen": QWEN_OPENAI_FLOOR,
|
||||
"max_side": 1536,
|
||||
},
|
||||
},
|
||||
"google": {
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png": {
|
||||
"chroma": (0.08, 0.10, 0.12, 0.15, 0.20),
|
||||
"qwen": QWEN_GOOGLE_FLOOR,
|
||||
"max_side": 1536,
|
||||
},
|
||||
"Gemini_Generated_Image_akdbeiakdbeiakdb.png": {
|
||||
"chroma": (0.08, 0.10, 0.12, 0.15, 0.20),
|
||||
"qwen": QWEN_GOOGLE_FLOOR,
|
||||
"max_side": 1536,
|
||||
},
|
||||
"Gemini_Generated_Image_y48j3cy48j3cy48j.png": {
|
||||
"chroma": (0.08, 0.10, 0.12, 0.15, 0.20),
|
||||
"qwen": QWEN_GOOGLE_FLOOR,
|
||||
"max_side": 1536,
|
||||
},
|
||||
"Gemini_Generated_Image_3mc4t93mc4t93mc4.png": { # the 18-face portrait grid
|
||||
"chroma": (0.08, 0.10, 0.12, 0.15, 0.20),
|
||||
"qwen": QWEN_GOOGLE_FLOOR,
|
||||
"max_side": 1536,
|
||||
},
|
||||
},
|
||||
"meta": {
|
||||
name: {"chroma": (0.03, 0.045, 0.06, 0.08), "qwen": QWEN_META_FLOOR, "max_side": 0}
|
||||
for name in (
|
||||
"gen_fox_forest.webp",
|
||||
"gen_text_poster.webp",
|
||||
"gen_lighthouse_watercolor.webp",
|
||||
"gen_night_city.webp",
|
||||
"gen_studio_mug.webp",
|
||||
)
|
||||
},
|
||||
}
|
||||
|
||||
# Seed sensitivity of the already-bracketed OpenAI typography fixture: the
|
||||
# boundary was (0.05, 0.06] at seed 0; probe the same rungs at two more seeds.
|
||||
SEED_PROBE = {
|
||||
"source": "data/synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png",
|
||||
"rungs": (0.05, 0.06),
|
||||
"seeds": (1, 2),
|
||||
"max_side": 1536,
|
||||
}
|
||||
|
||||
|
||||
def requested_steps(effective_steps: int, strength: float) -> int:
|
||||
return max(1, math.ceil(effective_steps / max(float(strength), 1e-6)))
|
||||
|
||||
|
||||
@app.function(image=image, gpu=_GPU, volumes={"/cache": cache}, timeout=3000)
|
||||
def calibrate(payload: dict[str, bytes]) -> dict:
|
||||
"""Run the chroma ladders, the qwen floors, and the seed probe."""
|
||||
import io
|
||||
import time
|
||||
|
||||
import torch
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, "/pkg")
|
||||
report: dict = {}
|
||||
|
||||
def load(path_key: str, max_side: int) -> Image.Image:
|
||||
source = Image.open(io.BytesIO(payload[path_key])).convert("RGB")
|
||||
if max_side and max(source.size) > max_side:
|
||||
scale = max_side / max(source.size)
|
||||
source = source.resize(
|
||||
(round(source.width * scale), round(source.height * scale)),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
return source
|
||||
|
||||
def chroma_call(chroma: Any, source: Image.Image, strength: float, seed: int) -> Image.Image:
|
||||
width = max(16, (source.width // 16) * 16)
|
||||
height = max(16, (source.height // 16) * 16)
|
||||
generator = torch.Generator(device="cpu").manual_seed(seed)
|
||||
out = chroma(
|
||||
prompt=_PROMPT,
|
||||
negative_prompt=_NEGATIVE,
|
||||
image=source,
|
||||
width=width,
|
||||
height=height,
|
||||
strength=strength,
|
||||
num_inference_steps=requested_steps(CHROMA_EFFECTIVE_STEPS, strength),
|
||||
guidance_scale=5.0,
|
||||
generator=generator,
|
||||
).images[0]
|
||||
return out if out.size == source.size else out.resize(source.size, Image.Resampling.LANCZOS)
|
||||
|
||||
def png_bytes(image: Image.Image) -> bytes:
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="PNG")
|
||||
return buffer.getvalue()
|
||||
|
||||
from diffusers import ChromaImg2ImgPipeline
|
||||
|
||||
print(f"[chroma] loading {CHROMA_MODEL_ID}...", flush=True)
|
||||
chroma = ChromaImg2ImgPipeline.from_pretrained(CHROMA_MODEL_ID, torch_dtype=torch.bfloat16).to("cuda")
|
||||
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import QwenZImagePipeline
|
||||
|
||||
print("[qwen] loading the production two-stage pipeline...", flush=True)
|
||||
qwen = QwenZImagePipeline(
|
||||
device="cuda",
|
||||
torch_dtype=torch.bfloat16,
|
||||
keep_global_models_on_device=True,
|
||||
keep_face_models_on_device=True,
|
||||
)
|
||||
|
||||
for vendor, fixtures in FIXTURES.items():
|
||||
report[vendor] = {}
|
||||
for name, spec in fixtures.items():
|
||||
print(f"[{vendor}] {name}", flush=True)
|
||||
source = load(name, spec["max_side"])
|
||||
entry: dict = {"chroma": {}, "qwen_seconds": None}
|
||||
for strength in spec["chroma"]:
|
||||
started = time.perf_counter()
|
||||
out = chroma_call(chroma, source, strength, 0)
|
||||
seconds = time.perf_counter() - started
|
||||
entry["chroma"][f"{strength:.4f}"] = {
|
||||
"png": png_bytes(out),
|
||||
"seconds": round(seconds, 2),
|
||||
}
|
||||
print(f" chroma s={strength:.3f} {seconds:.1f}s", flush=True)
|
||||
started = time.perf_counter()
|
||||
qwen_out = qwen.run(source, strength=spec["qwen"], seed=0)
|
||||
entry["qwen_seconds"] = round(time.perf_counter() - started, 2)
|
||||
entry["qwen_png"] = png_bytes(qwen_out)
|
||||
print(f" qwen s={spec['qwen']} {entry['qwen_seconds']}s", flush=True)
|
||||
report[vendor][name] = entry
|
||||
|
||||
print("[seeds] openai typography boundary probe...", flush=True)
|
||||
seed_source = load(SEED_PROBE["source"], SEED_PROBE["max_side"])
|
||||
report["seed_probe"] = {}
|
||||
for seed in SEED_PROBE["seeds"]:
|
||||
for strength in SEED_PROBE["rungs"]:
|
||||
out = chroma_call(chroma, seed_source, strength, seed)
|
||||
report["seed_probe"][f"s{strength:.2f}_seed{seed}"] = png_bytes(out)
|
||||
print(f" seed={seed} s={strength:.2f}", flush=True)
|
||||
|
||||
cache.commit()
|
||||
return report
|
||||
|
||||
|
||||
@app.local_entrypoint()
|
||||
def main(out: str = "out/cohort-calibration") -> None:
|
||||
payload: dict[str, bytes] = {}
|
||||
for vendor, fixtures in FIXTURES.items():
|
||||
for name in fixtures:
|
||||
if vendor == "meta":
|
||||
path = _ROOT / "data" / "contentseal" / "originals" / name
|
||||
else:
|
||||
path = _ROOT / "data" / "synthid" / "originals" / name
|
||||
if not path.exists():
|
||||
raise SystemExit(f"fixture not found: {path}")
|
||||
payload[name] = path.read_bytes()
|
||||
payload[SEED_PROBE["source"]] = (_ROOT / SEED_PROBE["source"]).read_bytes()
|
||||
|
||||
report = calibrate.remote(payload)
|
||||
|
||||
out_root = Path(out)
|
||||
summary: dict = {}
|
||||
for vendor, fixtures in report.items():
|
||||
if vendor == "seed_probe":
|
||||
for key, png in fixtures.items():
|
||||
destination = out_root / "seed-probe" / f"chroma_{key}.png"
|
||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||
destination.write_bytes(png)
|
||||
continue
|
||||
summary[vendor] = {}
|
||||
for name, entry in fixtures.items():
|
||||
fixture_dir = out_root / vendor / Path(name).stem
|
||||
fixture_dir.mkdir(parents=True, exist_ok=True)
|
||||
(fixture_dir / "qwen_floor.png").write_bytes(entry["qwen_png"])
|
||||
rungs = {}
|
||||
for strength, run in entry["chroma"].items():
|
||||
(fixture_dir / f"chroma_s{float(strength):.3f}.png").write_bytes(run["png"])
|
||||
rungs[strength] = run["seconds"]
|
||||
summary[vendor][name] = {
|
||||
"chroma_rungs_seconds": rungs,
|
||||
"qwen_seconds": entry["qwen_seconds"],
|
||||
}
|
||||
(out_root / "timing.json").write_text(json.dumps(summary, indent=2))
|
||||
print(json.dumps(summary, indent=2))
|
||||
print(f"\noutputs in {out_root}. Oracle each fixture ascending, stop at first clean.")
|
||||
@@ -0,0 +1,249 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["modal>=1.5"]
|
||||
# ///
|
||||
"""Head-to-head probe: qwen-zimage (the shipped production path) vs Chroma1
|
||||
img2img on one fixture -- warm per-image wall time, VRAM peak, and outputs for
|
||||
the local fidelity metrics, so quality and price can be compared on the same
|
||||
pixels.
|
||||
|
||||
qwen-zimage runs through the package's own ``QwenZImagePipeline`` (DiffSynth,
|
||||
Lightning LoRA, Canny ControlNet, face stage) at its certified OpenAI operating
|
||||
point (0.07675) with both stacks resident, exactly as the deployed worker does.
|
||||
Chroma1 runs the same neutral prompt with the SDXL-style step compensation
|
||||
(``requested_steps`` semantics: always spend 4 effective denoising steps), so a
|
||||
low strength is not silently 1 step of 35.
|
||||
|
||||
Standalone like the other prototypes: not imported by the package, not in
|
||||
uv.lock. The package source is mounted at runtime, never installed.
|
||||
|
||||
Run from the repository root (first run downloads both stacks, ~100 GB total,
|
||||
into the shared volume; later runs start warm):
|
||||
|
||||
uvx modal run scripts/engine_quality_price_probe.py \
|
||||
--source data/synthid/originals/'ChatGPT Image May 31, 2026, 02_03_55 PM.png'
|
||||
|
||||
Oracle the Chroma ladder outputs afterwards (openai.com/verify) and score both
|
||||
engines' floor outputs with scripts/fidelity_metrics.py locally.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import modal
|
||||
|
||||
_GPU = "H100"
|
||||
# Verified 2026-08-29 from modal.com/pricing.
|
||||
_H100_PER_HOUR = 3.95
|
||||
_A100_80_PER_HOUR = 2.50
|
||||
|
||||
app = modal.App("engine-quality-price-probe")
|
||||
|
||||
_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
image = (
|
||||
modal.Image.debian_slim(python_version="3.12")
|
||||
.pip_install(
|
||||
"torch",
|
||||
"torchvision>=0.20.0",
|
||||
"diffusers>=0.38,<1",
|
||||
"diffsynth>=2.0.17,<3",
|
||||
"transformers>=4.53",
|
||||
"accelerate",
|
||||
"sentencepiece",
|
||||
"protobuf",
|
||||
"pillow",
|
||||
"numpy",
|
||||
"opencv-python-headless",
|
||||
)
|
||||
.env({"HF_HOME": "/cache"})
|
||||
.add_local_dir(_ROOT / "src" / "remove_ai_watermarks", remote_path="/pkg/remove_ai_watermarks")
|
||||
)
|
||||
|
||||
# Carries both engines' weights; named for the first prototype that filled it.
|
||||
cache = modal.Volume.from_name("chroma-hf-cache", create_if_missing=True)
|
||||
|
||||
QWEN_OPENAI_FLOOR = 0.07675 # watermark_profiles.QWEN_ZIMAGE_OPENAI_STRENGTH
|
||||
CHROMA_MODEL_ID = "lodestones/Chroma1-HD"
|
||||
CHROMA_STRENGTHS = (0.05, 0.06, 0.075, 0.09, 0.10)
|
||||
CHROMA_EFFECTIVE_STEPS = 4
|
||||
|
||||
_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
|
||||
|
||||
def requested_steps(effective_steps: int, strength: float) -> int:
|
||||
"""Diffusers truncates the step COUNT; spend effective_steps regardless."""
|
||||
import math
|
||||
|
||||
return max(1, math.ceil(effective_steps / max(float(strength), 1e-6)))
|
||||
|
||||
|
||||
def _psnr(a: Any, b: Any) -> float:
|
||||
import numpy as np
|
||||
|
||||
x = np.asarray(a.convert("RGB"), dtype=np.float64)
|
||||
y = np.asarray(b.convert("RGB"), dtype=np.float64)
|
||||
mse = float(((x - y) ** 2).mean())
|
||||
return 99.0 if mse <= 1e-12 else 10.0 * float(np.log10(255.0**2 / mse))
|
||||
|
||||
|
||||
def _png(image: Any) -> bytes:
|
||||
import io
|
||||
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="PNG")
|
||||
return buffer.getvalue()
|
||||
|
||||
|
||||
@app.function(image=image, gpu=_GPU, volumes={"/cache": cache}, timeout=3000)
|
||||
def probe(source_bytes: bytes, max_side: int) -> dict:
|
||||
"""Run both engines; return outputs, timings, and VRAM peaks."""
|
||||
import gc
|
||||
import io
|
||||
import time
|
||||
|
||||
import torch
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, "/pkg")
|
||||
|
||||
source = Image.open(io.BytesIO(source_bytes)).convert("RGB")
|
||||
if max_side and max(source.size) > max_side:
|
||||
scale = max_side / max(source.size)
|
||||
source = source.resize(
|
||||
(round(source.width * scale), round(source.height * scale)),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
|
||||
report: dict = {"source_size": source.size, "engines": {}}
|
||||
|
||||
# --- qwen-zimage, the exact production path ---------------------------------
|
||||
print("[qwen] loading the production two-stage pipeline (resident stacks)...", flush=True)
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import QwenZImagePipeline
|
||||
|
||||
qwen = QwenZImagePipeline(
|
||||
device="cuda",
|
||||
torch_dtype=torch.bfloat16,
|
||||
keep_global_models_on_device=True,
|
||||
keep_face_models_on_device=True,
|
||||
)
|
||||
print(f"[qwen] warmup pass at strength={QWEN_OPENAI_FLOOR}...", flush=True)
|
||||
qwen.run(source, strength=QWEN_OPENAI_FLOOR, seed=0)
|
||||
|
||||
torch.cuda.reset_peak_memory_stats()
|
||||
started = time.perf_counter()
|
||||
qwen_out = qwen.run(source, strength=QWEN_OPENAI_FLOOR, seed=0)
|
||||
qwen_seconds = time.perf_counter() - started
|
||||
qwen_vram_gib = torch.cuda.max_memory_allocated() / (1024**3)
|
||||
report["engines"]["qwen-zimage"] = {
|
||||
"strength": QWEN_OPENAI_FLOOR,
|
||||
"warm_seconds": round(qwen_seconds, 2),
|
||||
"vram_peak_gib": round(qwen_vram_gib, 1),
|
||||
"psnr": round(_psnr(source, qwen_out), 2),
|
||||
"png": _png(qwen_out),
|
||||
}
|
||||
qwen_report = report["engines"]["qwen-zimage"]
|
||||
print(
|
||||
f"[qwen] warm {qwen_seconds:.2f}s, vram peak {qwen_vram_gib:.1f} GiB, psnr {qwen_report['psnr']:.2f} dB",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
del qwen
|
||||
gc.collect()
|
||||
torch.cuda.empty_cache()
|
||||
|
||||
# --- Chroma1, compensated-step ladder ---------------------------------------
|
||||
from diffusers import ChromaImg2ImgPipeline
|
||||
|
||||
print(f"[chroma] loading {CHROMA_MODEL_ID} (bf16)...", flush=True)
|
||||
chroma = ChromaImg2ImgPipeline.from_pretrained(CHROMA_MODEL_ID, torch_dtype=torch.bfloat16).to("cuda")
|
||||
|
||||
width = max(16, (source.width // 16) * 16)
|
||||
height = max(16, (source.height // 16) * 16)
|
||||
ladder: list[dict] = []
|
||||
for strength in CHROMA_STRENGTHS:
|
||||
steps = requested_steps(CHROMA_EFFECTIVE_STEPS, strength)
|
||||
torch.cuda.reset_peak_memory_stats()
|
||||
started = time.perf_counter()
|
||||
generator = torch.Generator(device="cpu").manual_seed(0)
|
||||
out = chroma(
|
||||
prompt=_PROMPT,
|
||||
negative_prompt=_NEGATIVE,
|
||||
image=source,
|
||||
width=width,
|
||||
height=height,
|
||||
strength=strength,
|
||||
num_inference_steps=steps,
|
||||
guidance_scale=5.0,
|
||||
generator=generator,
|
||||
).images[0]
|
||||
seconds = time.perf_counter() - started
|
||||
vram_gib = torch.cuda.max_memory_allocated() / (1024**3)
|
||||
if out.size != source.size:
|
||||
out = out.resize(source.size, Image.Resampling.LANCZOS)
|
||||
ladder.append(
|
||||
{
|
||||
"strength": strength,
|
||||
"requested_steps": steps,
|
||||
"warm_seconds": round(seconds, 2),
|
||||
"vram_peak_gib": round(vram_gib, 1),
|
||||
"psnr": round(_psnr(source, out), 2),
|
||||
"png": _png(out),
|
||||
}
|
||||
)
|
||||
print(
|
||||
f"[chroma] s={strength:.3f} steps={steps}/{steps} warm {seconds:.2f}s "
|
||||
f"vram {vram_gib:.1f} GiB psnr {ladder[-1]['psnr']:.2f} dB",
|
||||
flush=True,
|
||||
)
|
||||
report["engines"]["chroma1"] = {"ladder": ladder}
|
||||
|
||||
cache.commit()
|
||||
return report
|
||||
|
||||
|
||||
@app.local_entrypoint()
|
||||
def main(
|
||||
source: str,
|
||||
out: str = "out/engine-comparison",
|
||||
max_side: int = 1536,
|
||||
) -> None:
|
||||
source_path = Path(source)
|
||||
if not source_path.exists():
|
||||
raise SystemExit(f"source not found: {source_path}")
|
||||
|
||||
report = probe.remote(source_path.read_bytes(), max_side)
|
||||
|
||||
out_dir = Path(out) / source_path.stem
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
for engine, payload in report["engines"].items():
|
||||
if engine == "chroma1":
|
||||
for run in payload["ladder"]:
|
||||
(out_dir / f"chroma_s{run['strength']:.3f}.png").write_bytes(run["png"])
|
||||
else:
|
||||
(out_dir / f"{engine}_s{payload['strength']}.png").write_bytes(payload["png"])
|
||||
|
||||
summary = {
|
||||
"source": str(source_path),
|
||||
"source_size": report["source_size"],
|
||||
"gpu": _GPU,
|
||||
"usd_per_hour": {"H100": _H100_PER_HOUR, "A100-80GB": _A100_80_PER_HOUR},
|
||||
"engines": {
|
||||
engine: (
|
||||
{k: v for k, v in payload.items() if k != "png"}
|
||||
if engine != "chroma1"
|
||||
else {"ladder": [{k: v for k, v in run.items() if k != "png"} for run in payload["ladder"]]}
|
||||
)
|
||||
for engine, payload in report["engines"].items()
|
||||
},
|
||||
}
|
||||
(out_dir / "timing.json").write_text(json.dumps(summary, indent=2))
|
||||
print(json.dumps(summary, indent=2))
|
||||
print(f"\noutputs in {out_dir}; oracle the chroma ladder, then score locally with:")
|
||||
print(f" uv run scripts/fidelity_metrics.py compare --original '{source}' \\")
|
||||
print(f" --variant qwen={out_dir / 'qwen-zimage_s0.0768.png'} --variant chroma={out_dir}/chroma_sXXX.png")
|
||||
@@ -1,12 +1,11 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "numpy",
|
||||
# "opencv-python-headless",
|
||||
# "opencv-python-headless<5",
|
||||
# "pillow",
|
||||
# "scikit-image",
|
||||
# "rapidfuzz",
|
||||
# "torch",
|
||||
# "lpips",
|
||||
# "paddleocr",
|
||||
@@ -29,7 +28,7 @@ metrics run only where faces are detected, text metrics only where text is.
|
||||
|
||||
Two subcommands:
|
||||
|
||||
ocr -- OCR images (PaddleOCR PP-OCRv6) into a JSON {basename: text} file.
|
||||
ocr -- OCR images (PaddleOCR defaults) into a JSON {basename: text} file.
|
||||
Run this on the ORIGINALS, hand-verify/correct the file, and it
|
||||
becomes the ground truth for ``compare --ground-truth`` -- the clean
|
||||
way to score text, since OCR-vs-OCR is doubly noisy (errors on both
|
||||
@@ -54,6 +53,7 @@ Usage:
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import unicodedata
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
@@ -65,7 +65,10 @@ import numpy as np
|
||||
from _plain_console import Console, Table
|
||||
|
||||
console = Console()
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from scripts._text_eval import levenshtein_normalized # noqa: E402
|
||||
|
||||
# ── helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
@@ -89,7 +92,7 @@ def _norm(text: str) -> str:
|
||||
return "".join(unicodedata.normalize("NFC", text).split())
|
||||
|
||||
|
||||
# ── text: PaddleOCR (PP-OCRv6) ───────────────────────────────────────
|
||||
# ── text: PaddleOCR defaults ─────────────────────────────────────────
|
||||
|
||||
# Our lang codes -> PaddleOCR lang. The 'ch' model also reads Latin; 'ru' reads
|
||||
# Cyrillic + Latin. Multiple langs in one image -> run each model, union detections.
|
||||
@@ -165,9 +168,7 @@ def _ocr_lines(bgr: np.ndarray, langs: list[str], min_score: float = 0.5) -> lis
|
||||
|
||||
|
||||
def _cer(ref: str, hyp: str) -> float:
|
||||
from rapidfuzz.distance import Levenshtein
|
||||
|
||||
return Levenshtein.normalized_distance(_norm(ref), _norm(hyp))
|
||||
return levenshtein_normalized(_norm(ref), _norm(hyp))
|
||||
|
||||
|
||||
# ── face: detection + ArcFace + texture ──────────────────────────────
|
||||
|
||||