mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-18 23:37:14 +02:00
Compare commits
No files matched your search
@@ -1,6 +1,6 @@
|
||||
---
|
||||
globs: ["src/**/*.py", "tests/**/*.py", "scripts/**/*.py", "pyproject.toml", "uv.lock", "maintain.sh", ".github/workflows/*.yml"]
|
||||
description: Command contracts, project gate, typing boundaries, and model-adjacent test invariants.
|
||||
description: Command contracts, project gate, typing boundaries, model-adjacent test invariants, and the detection-path measurement rule.
|
||||
---
|
||||
|
||||
# Development invariants
|
||||
@@ -11,21 +11,129 @@ Every single-image command declares `source` with `dir_okay=False`; `batch` decl
|
||||
|
||||
Exit-code and no-signal behavior is a public contract. Read the command-line section of [`../../docs/module-internals.md`](../../docs/module-internals.md) before changing it.
|
||||
|
||||
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
|
||||
|
||||
`device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. It is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds.
|
||||
|
||||
The same rule applies to install hints: name the extra that actually makes the command work (`qwen-zimage`, not `diffusion`), and keep the printed command shell-quoted -- bare `pkg[extra]` is a glob in zsh.
|
||||
|
||||
## Local gate
|
||||
|
||||
Run `bash maintain.sh` from the repository root. The authoritative type gate is scoped to `src/`; full-project Pyright can exhaust Node memory on the ML dependency graph.
|
||||
|
||||
Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.
|
||||
|
||||
From a worktree, `uv run` imports the package from the MAIN checkout -- that is where the editable install points. A script measuring a worktree's edit must insert that worktree's `src` at `sys.path[0]` and assert `module.__file__` resolves inside it, or it silently compares unmodified code against itself.
|
||||
|
||||
## Model-adjacent tests
|
||||
|
||||
Do not classify an entire module as untestable because its main path downloads a model. Keep pure behavior covered without downloads, including:
|
||||
|
||||
- target-size selection in `test_invisible_engine.py`;
|
||||
- unsharp and adaptive-polish helpers in `test_humanizer.py`;
|
||||
- mocked device fallback in `test_img2img_runner.py`;
|
||||
- tiling geometry and blending in `test_tiling.py`.
|
||||
- tiling geometry and blending in `test_tiling.py`;
|
||||
- prompt-embedding cache keying, storage round-trip, and the cross-pipeline reuse
|
||||
that lets a stack load without its text encoder, in `test_qwen_zimage_pipeline.py`;
|
||||
- the face stack's dtype, in `test_qwen_zimage_pipeline.py`. A subclass that changes
|
||||
the pipeline dtype for its own global model must not change the inherited face
|
||||
stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a
|
||||
face. When one profile inherits another's stage, guard the invariants that stage
|
||||
relies on, not just the code path.
|
||||
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
|
||||
mirror another, compare them field by field rather than pinning the values you happen
|
||||
to know about, so the next field added on one side and not the other fails at the
|
||||
seam. Two of these defaults drifted in practice and neither needed a GPU to catch;
|
||||
the incident is recorded in `docs/module-internals.md`. Keep the comparison free of
|
||||
an exception table: a field that needs one is a field that belongs elsewhere, which
|
||||
is what `force` turned out to be.
|
||||
|
||||
A defaults comparison is not a forwarding test, and the two fail differently. Pin the
|
||||
VALUE at the seam, not just the name -- `_run_invisible` passed the whole suite with
|
||||
`controlnet_conditioning_scale` hardcoded, because nothing asserted the caller's value
|
||||
arrived. `test_every_field_arrives_at_the_engine_with_the_caller_s_value` drives the
|
||||
real seam with every field set off its default, so one test covers the whole bag
|
||||
instead of one assertion per knob.
|
||||
|
||||
Count the seams before believing a knob is covered. Each of `force` and
|
||||
`controlnet_conditioning_scale` reaches the engine through TWO paths -- `remove_all`
|
||||
versus `remove_batch(mode="all")` for the first, `_run_invisible` versus `_batch_engine`
|
||||
for the second -- and in both cases guarding one path left the other free to hardcode a
|
||||
constant with a green suite. The mode-parametrized guards in
|
||||
`TestRemoveBatchLibrary::test_force_reaches_the_scrub_gate_in_every_scrubbing_mode` and
|
||||
`TestBatchCommand::test_batch_controlnet_scale_flows_to_the_cached_engine` exist because
|
||||
that is what actually happened.
|
||||
|
||||
Use availability checks only for paths that actually load large models.
|
||||
|
||||
## One measurement, one gate seam
|
||||
|
||||
A detector is split into a trust-level-blind scan and a verdict that applies the
|
||||
threshold, so `detect` and `detect_both` reach the same numbers by construction. Two
|
||||
rules follow, and both were broken in practice before they were written down:
|
||||
|
||||
- A per-mark demotion goes in the `_post_gate` hook (or, for a whole-scan precondition
|
||||
like LibLibAI's size floor, in `_scan`) -- never in a `detect` override. An override
|
||||
is invisible to `detect_both`, so the RunningHub and Yuanbao anchor gates silently
|
||||
stopped applying on the arbiter's perception path. `TestSinglePassPerception` is the
|
||||
guard: it asserts `detect_both` equals two `detect` calls field for field.
|
||||
- Detection and the removal mask must read ONE sweep. The winning box travels on
|
||||
`TextMarkDetection.match_box` and the registry threads the detection into the mask
|
||||
builder; a mask path that re-runs its own sweep is how the two drift apart.
|
||||
|
||||
The C2PA manifest-store JSON is NOT stable across reads: the reader regenerates manifest
|
||||
URNs and instance ids. Compare the derived `c2pa_info`, never the raw store.
|
||||
|
||||
A third seam reaches the same verdict: `collect_metadata_record` ->
|
||||
`evidence_from_metadata_record` -> `identify_from_evidence`, the path a caller uses when
|
||||
collection and verdict run on different machines. Its contract is equality with
|
||||
`identify(path, check_visible=False, check_invisible=False)` on the same image, and it
|
||||
can break from EITHER side -- a region the collector stops walking, or a placement the
|
||||
file path learns to read and the record does not. `tests/test_metadata_record.py` pins
|
||||
it over the tracked fixtures; a separate local evaluation corpus catches placements the
|
||||
fixtures do not cover. Change either side and re-run both.
|
||||
|
||||
Before changing anything in the detection path, record the detectors' exact verdicts
|
||||
over a local sample first and diff them after. A refactor here is only correct if that
|
||||
record is byte-identical, and a green test suite does not establish that on its own. A
|
||||
change that is meant to FIX detection is the exception that proves the rule: the diff
|
||||
must then be exactly the files you intended to change, named in advance.
|
||||
|
||||
Two corollaries in `dwt_dct.py`, where "close enough" has an exact meaning. The bit test
|
||||
is `peak % 36 > 18.0` and for uint8 input the exact Haar value is a multiple of 0.5, so it
|
||||
lands ON the threshold once per 72 blocks and a 1-ulp difference flips real bits: leave the
|
||||
transform to `pywt` however slow it looks, because its C convolution contracts into an FMA
|
||||
that numpy has no ufunc for (`docs/module-internals.md` carries what that costs). And
|
||||
`tests/test_invisible_watermark.py` is `skipif(not is_available())` -- without the `detect`
|
||||
extra the upstream-parity test never runs, so a green suite is not evidence here and the
|
||||
verdict record is not optional.
|
||||
|
||||
## A certified operating point is data, not a constant
|
||||
|
||||
The video SynthID default is only meaningful as a row in
|
||||
`data/evaluations/video-synthid-oracle.csv`, so
|
||||
`test_shipped_defaults_match_a_certified_manifest_row` derives the pin from that
|
||||
manifest instead of restating literals. Pinning `noise_std` alone had let `long_side`
|
||||
and `fps` -- two thirds of what the oracle was actually shown -- move with a green
|
||||
suite.
|
||||
|
||||
The certified profile is a perturbation-to-signal ratio, not a bare `noise_std`, so
|
||||
the latent scaling factor is gated in `load_video_vae_runtime`, carried on
|
||||
`VideoVaeRuntime`, and passed into encode and decode: the validated value and the
|
||||
applied value are one measurement. Anything that produces oracle evidence loads
|
||||
through that function. `video_synthid_sweep.py` hand-rolled the load and was the one
|
||||
path exempt from the gate it exists to feed, which is exactly backwards.
|
||||
|
||||
Prove a video-path refactor the same way the detection path is proven, and without
|
||||
needing an oracle carrier: build a clip from a tracked fixture with ffmpeg, run the
|
||||
engine before and after, and require an identical output sha256. Keep the generated
|
||||
media outside the repository.
|
||||
|
||||
Frame sampling is compared as `timestamp + 1e-9 >= next_sample_time`, so mutating
|
||||
that `>=` to `>` changes nothing and a green suite proves nothing. Mutate the phase
|
||||
or the period instead -- starting the accumulator half a period late shifts the
|
||||
selection by one frame while keeping the count identical, which is the drift a
|
||||
frame-count check cannot see and what
|
||||
`test_pairing_follows_the_engine_sampling_rule_not_just_the_frame_count` exists to
|
||||
catch.
|
||||
|
||||
Environment setup, dependency recovery, CI behavior, and fixture policy: [`../../docs/development.md`](../../docs/development.md).
|
||||
@@ -1,14 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"WebSearch",
|
||||
"WebFetch"
|
||||
]
|
||||
},
|
||||
"enabledPlugins": {
|
||||
"pyright-lsp@claude-plugins-official": true,
|
||||
"context7@claude-plugins-official": true,
|
||||
"code-simplifier@claude-plugins-official": true,
|
||||
"claude-md-management@claude-plugins-official": true
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -1,3 +1,3 @@
|
||||
# HuggingFace token (optional; only needed for gated/private models)
|
||||
# Get yours at: https://huggingface.co/settings/tokens
|
||||
HF_TOKEN=
|
||||
# HF_TOKEN=
|
||||
@@ -1,9 +1,8 @@
|
||||
name: Distribute on release
|
||||
|
||||
# Fans a published GitHub Release out to the channels that need a nudge.
|
||||
# PyPI is handled by publish.yml; the conda-forge channel is handled by its
|
||||
# autotick bot. This workflow updates the repository recipe and event-drives
|
||||
# the three channels that would otherwise be manual:
|
||||
# PyPI is handled by publish.yml. This workflow event-drives the three channels
|
||||
# that would otherwise be manual:
|
||||
# - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
|
||||
# - HF Space: factory-rebuild so it reinstalls the latest sdist from PyPI.
|
||||
# - ComfyUI: sync, test, and publish the node against the exact library release.
|
||||
@@ -89,34 +88,6 @@ jobs:
|
||||
git commit -m "remove-ai-watermarks $VERSION" || { echo "Formula already current"; exit 0; }
|
||||
git push
|
||||
|
||||
conda-recipe:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- name: Checkout main
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
ref: main
|
||||
- name: Bump the repository conda recipe
|
||||
env:
|
||||
SHA: ${{ needs.resolve.outputs.sha }}
|
||||
VERSION: ${{ needs.resolve.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
python scripts/sync_conda_recipe.py --version "$VERSION" --sha256 "$SHA"
|
||||
git diff --check
|
||||
if git diff --quiet -- packaging/conda/recipe.yaml; then
|
||||
echo "Conda recipe already current"
|
||||
exit 0
|
||||
fi
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git add packaging/conda/recipe.yaml
|
||||
git commit -m "Sync conda recipe with v$VERSION"
|
||||
git push
|
||||
|
||||
hf-space:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -28,13 +28,26 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Cross-OS x Python-floor/recent. The default-install surface only
|
||||
# needs the core deps plus dev tooling (no GPU extra): the model-running
|
||||
# paths are availability-gated and skip when torch/diffusers are absent,
|
||||
# so this matrix exercises metadata, identify, visible, and the cv2
|
||||
# eraser on every OS without pulling the heavy ML stack.
|
||||
os: [ubuntu-latest, macos-latest, windows-latest]
|
||||
python-version: ["3.10", "3.12"]
|
||||
# Exercise every supported minor. The floor and latest minor run on all
|
||||
# operating systems; intermediate minors run on Linux to bound the
|
||||
# matrix without leaving a declared Python version untested.
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.11"
|
||||
- os: macos-latest
|
||||
python-version: "3.11"
|
||||
- os: windows-latest
|
||||
python-version: "3.11"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.12"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.13"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.14"
|
||||
- os: macos-latest
|
||||
python-version: "3.14"
|
||||
- os: windows-latest
|
||||
python-version: "3.14"
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
@@ -44,3 +57,43 @@ jobs:
|
||||
run: uv sync --frozen --extra dev
|
||||
- name: Run tests
|
||||
run: uv run pytest -q
|
||||
|
||||
transport-contract-python-314:
|
||||
name: versioned transport contracts py3.14
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
python-version: "3.14"
|
||||
- name: Create clean runtime contract environment
|
||||
run: uv venv --python 3.14 .venv-contract
|
||||
- name: Install runtime contract dependencies
|
||||
run: >-
|
||||
uv pip install --python .venv-contract/bin/python
|
||||
".[pixels,heif]" "numpy>=2,<3" "pytest>=8"
|
||||
- name: Run versioned transport contract tests
|
||||
run: >-
|
||||
.venv-contract/bin/python -m pytest -q
|
||||
tests/test_forensic_metadata.py
|
||||
tests/test_metadata_record.py
|
||||
tests/test_pixel_evidence.py
|
||||
|
||||
video-e2e:
|
||||
name: video full-clip end-to-end
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Install ffmpeg
|
||||
run: sudo apt-get update && sudo apt-get install --yes ffmpeg
|
||||
- name: Sync dev environment
|
||||
run: uv sync --frozen --extra dev
|
||||
- name: Run full-clip video test
|
||||
run: >-
|
||||
uv run pytest -vv -o faulthandler_timeout=60
|
||||
tests/test_video.py::TestVideoVisibleFullClip
|
||||
tests/test_video_fidelity_probe.py
|
||||
+16
-14
@@ -4,6 +4,8 @@ __pycache__/
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
tmp/
|
||||
.sc/
|
||||
|
||||
# Environment secrets
|
||||
.env
|
||||
@@ -18,8 +20,7 @@ Thumbs.db
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# SynthID corpus reference fills (synthetic black/white calibration tiles,
|
||||
# regenerable; the labeled pos/neg/cleaned images ARE tracked, see README)
|
||||
# Generated calibration inputs
|
||||
data/synthid_corpus/refs/
|
||||
|
||||
# Reference materials
|
||||
@@ -33,11 +34,7 @@ yolov8n.pt
|
||||
.claude/settings.local.json
|
||||
.claude/scheduled_tasks.lock
|
||||
|
||||
# Visible-watermark alpha calibration. The solid black/gray/white CAPTURES are
|
||||
# committed (content-free: a solid colour + the watermark; the source for
|
||||
# scripts/visible_alpha_solve.py so the alpha assets are reproducible). The
|
||||
# synthetic seeds (regenerable) and any real-content validation download (a real
|
||||
# generated scene, kept local for privacy) are NOT committed.
|
||||
# Generated calibration inputs and local evaluation artifacts
|
||||
data/doubao_capture/seeds/
|
||||
data/jimeng_capture/seeds/
|
||||
data/jimeng_capture/captures/jimeng_content_*.png
|
||||
@@ -46,13 +43,18 @@ data/gemini_capture/captures/gemini_content_*.png
|
||||
data/samsung_capture/seeds/
|
||||
data/samsung_capture/captures/samsung_content_*
|
||||
|
||||
# Leftover GFPGAN weights dir from the retired face-restore experiments
|
||||
# (GFPGAN wrote RetinaFace/parsing weights to a CWD ./gfpgan/weights/ working
|
||||
# dir on first use). Runtime artifact, never committed.
|
||||
# Runtime model artifacts
|
||||
gfpgan/
|
||||
|
||||
# Qwen ControlNet experiment outputs (throwaway eval; never the committed corpus)
|
||||
scripts/_qwen_exp_out/
|
||||
|
||||
# Local-only working data for analysis (not a committed corpus; never tracked)
|
||||
# Local evaluation data
|
||||
.local-eval/
|
||||
|
||||
# Other harnesses' generated config. Claude's sync owns these; a committed copy
|
||||
# is a second source nobody reconciles. Anchored so a same-named source file
|
||||
# deeper in the tree is not caught.
|
||||
/.agents/
|
||||
/.codex/
|
||||
/sch_*.yaml
|
||||
/.opencode/
|
||||
/opencode.json
|
||||
/opencode.jsonc
|
||||
@@ -33,7 +33,7 @@ Optional features and installation groups are documented in [`docs/installation.
|
||||
|
||||
Command, gate, typing, and model-test invariants auto-load from [`.claude/rules/development.md`](.claude/rules/development.md). Environment recovery, CI behavior, and fixture policy live in [`docs/development.md`](docs/development.md).
|
||||
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Keep the source-distribution exclusion for `data/`.
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Treat the release as complete only after PyPI, Homebrew, the Hugging Face Space, and the ComfyUI Registry are verified; conda is not published. Keep the source-distribution public allowlist.
|
||||
|
||||
## Module architecture
|
||||
|
||||
@@ -51,4 +51,4 @@ Topic-specific rules live in `.claude/rules/*.md` and are auto-loaded when match
|
||||
|
||||
| File | Covers |
|
||||
|---|---|
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, and model-adjacent tests |
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, and the detection-path measurement rule |
|
||||
@@ -1,13 +1,19 @@
|
||||
# Remove AI Watermarks
|
||||
|
||||
Remove AI provenance marks from images you generated yourself:
|
||||
Remove AI provenance marks from images and video you generated yourself:
|
||||
|
||||
- known visible labels such as the Gemini sparkle and vendor text marks;
|
||||
- invisible pixel watermarks through diffusion regeneration;
|
||||
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
|
||||
|
||||
> Try it online at [raiw.cc](https://raiw.cc) if you do not want to install Python
|
||||
> or run diffusion models locally.
|
||||
Video support covers provenance identification, complete visible-plus-metadata
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo, and
|
||||
Kling mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
removal.
|
||||
|
||||
> [raiw.cc](https://raiw.cc) runs this library as a hosted service, with the GPU
|
||||
> included and nothing to install. Visible mark and metadata removal are free
|
||||
> there; invisible watermark removal is paid.
|
||||
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
@@ -28,7 +34,13 @@ Remove AI provenance marks from images you generated yourself:
|
||||
| Remove known visible AI marks | `visible` | No |
|
||||
| Erase a region you select | `erase` | No |
|
||||
| Strip AI metadata | `metadata` | No |
|
||||
| Regenerate an image to disrupt invisible watermarks | `invisible` | Recommended |
|
||||
| Identify supported video provenance | `video identify` | No |
|
||||
| Remove visible marks and AI metadata from video | `video all` | No |
|
||||
| Strip AI metadata from video | `video metadata` | No |
|
||||
| Remove a registered visible AI mark from video | `video visible` | No |
|
||||
| Process a directory of videos | `video batch` | Depends on mode |
|
||||
| Remove video SynthID with the certified VAE profile | `video invisible` | Recommended |
|
||||
| Regenerate an image to disrupt invisible watermarks | `invisible` | Required (CUDA) |
|
||||
| Run visible, invisible, and metadata removal | `all` | Recommended |
|
||||
| Process a directory | `batch` | Depends on mode |
|
||||
|
||||
@@ -38,14 +50,16 @@ Remove AI provenance marks from images you generated yourself:
|
||||
| --- | --- |
|
||||
| Metadata inspection and stripping | `remove-ai-watermarks` |
|
||||
| Visible detection and removal | `remove-ai-watermarks[visible]` |
|
||||
| Visible video processing | `remove-ai-watermarks[video]` |
|
||||
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
|
||||
| Torch-free DWT-DCT detection | `remove-ai-watermarks[detect]` |
|
||||
| Diffusion removal | `remove-ai-watermarks[diffusion]` |
|
||||
| Every production feature | `remove-ai-watermarks[all]` |
|
||||
| Invisible image removal (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| Every production feature available on the active Python | `remove-ai-watermarks[all]` |
|
||||
|
||||
Lower-level and specialized extras include `pixels`, `heif`, `trustmark`,
|
||||
`migan`, `lama`, `esrgan`, and `qwen-zimage`. The
|
||||
`migan`, `lama`, and `diffusion`. The
|
||||
[installation guide](docs/installation.md#feature-extras) documents their exact
|
||||
dependency composition and model requirements.
|
||||
dependency composition, Python compatibility, and model requirements.
|
||||
|
||||
## Quick start
|
||||
|
||||
@@ -79,10 +93,104 @@ Strip metadata without running visible inpainting or diffusion:
|
||||
remove-ai-watermarks metadata image.png --remove -o clean.png
|
||||
```
|
||||
|
||||
For invisible watermark removal, install the diffusion dependencies:
|
||||
Without `-o` this command overwrites the source in place.
|
||||
|
||||
Inspect or remove AI metadata from an MP4, MOV, M4V, WebM, MKV, AVI, or FLV
|
||||
file:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[diffusion]"
|
||||
remove-ai-watermarks video metadata input.mp4 --check
|
||||
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
|
||||
```
|
||||
|
||||
The `video metadata` command does not transcode video or audio streams. Unlike
|
||||
the image command above, when `-o` is omitted it writes `<source>_clean` and
|
||||
preserves the original. MP4 and MOV
|
||||
inspection includes the native TC260 `AIGC` tag in
|
||||
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload,
|
||||
plus the QuickTime-form `meta` variants Doubao's iOS export writes (a bare
|
||||
`meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list).
|
||||
MKV and WebM inspection reads the normative
|
||||
`Segment.Tags.Tag.SimpleTag` placement. AVI uses `LIST/INFO/AIGC`, while FLV
|
||||
uses `script.onMetaData.AIGC`. The non-ISOBMFF formats are remuxed with stream
|
||||
copy for removal.
|
||||
|
||||
Use the product-oriented video path to identify or clean a file:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[video]"
|
||||
remove-ai-watermarks video identify input.mp4
|
||||
remove-ai-watermarks video all input.mp4 -o clean.mp4
|
||||
```
|
||||
|
||||
`video all` removes a stable registered visible mark when present and always
|
||||
strips verified AI metadata. If neither signal is found, it still writes a
|
||||
same-container passthrough, so application callers get one predictable output
|
||||
contract. Proprietary invisible-video removal is excluded by default.
|
||||
`--invisible` opts into the lossy, oracle-certified video SynthID profile.
|
||||
|
||||
Process a directory with the same contract:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks video batch ./videos --mode all
|
||||
```
|
||||
|
||||
Remove a supported visible video mark:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks video visible input.mp4 -o clean.mp4
|
||||
remove-ai-watermarks video visible veo.mp4 --mark veo -o veo_clean.mp4
|
||||
remove-ai-watermarks video visible seedance.mp4 --mark seedance -o seedance_clean.mp4
|
||||
remove-ai-watermarks video visible dola.mp4 --mark dola -o dola_clean.mp4
|
||||
remove-ai-watermarks video visible hailuo.mp4 --mark hailuo -o hailuo_clean.mp4
|
||||
remove-ai-watermarks video visible kling.mp4 --mark kling -o kling_clean.mp4
|
||||
```
|
||||
|
||||
This path scans the complete sequence before changing pixels. It accepts only a
|
||||
mark that repeats at a stable position across adjacent frames, then reuses the
|
||||
same OpenCV, MI-GAN, or LaMa fill backends as image removal. Audio is copied
|
||||
without re-encoding and is allowed to reach its natural end; the video stream
|
||||
is transcoded because its pixels change. By default, a guarded optical-flow
|
||||
pass motion-aligns the preceding accepted fill and blends it only when the
|
||||
nearby source context agrees; use `--no-temporal-consistency` to disable it.
|
||||
The encoder preserves supported
|
||||
8-bit source chroma sampling, color tags, and MP4/MOV track timescale instead
|
||||
of relying on ffmpeg's implicit raw-BGR defaults. Variable frame intervals are
|
||||
preserved through a timestamped in-memory NUT bridge instead of being flattened
|
||||
to the average frame rate. Non-zero source start timestamps are retained
|
||||
together with the copied audio offset. The default `--mark auto`
|
||||
scans all providers in one decode pass and selects the first stable match in
|
||||
the specificity order shown below. Pass an explicit mark to restrict detection
|
||||
to one provider.
|
||||
Sora covers the moving Sora 2 mascot and wordmark. Veo covers both the current
|
||||
four-point diamond and the legacy `Veo` text. Seedance covers the fixed boxed
|
||||
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo covers the composite
|
||||
`MINIMAX | hailuo AI` label, and Kling covers the bottom-right `KLING AI`
|
||||
label with its version suffix. A completed encode is published atomically. No
|
||||
output is written when no stable mark is found.
|
||||
HDR, PQ/HLG, and greater-than-8-bit inputs are rejected before encoding rather
|
||||
than silently reduced through OpenCV's 8-bit BGR boundary.
|
||||
|
||||
Remove video SynthID:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[video,diffusion]"
|
||||
remove-ai-watermarks video invisible input.mp4 -o clean.mp4
|
||||
```
|
||||
|
||||
This path regenerates the complete sequence with one latent-noise field shared
|
||||
across time, copies complete audio, strips source metadata, and publishes the
|
||||
completed encode atomically. The default `noise_std=0.15` profile passed both
|
||||
the two-carrier calibration and a complete public eight-second Veo oracle
|
||||
check. Google does not publish a local decoder, so a fresh provider check
|
||||
remains useful for unusually important files or after provider changes, but it
|
||||
is not a product result state.
|
||||
|
||||
For invisible watermark removal, install the `qwen-zimage` extra. **An NVIDIA GPU
|
||||
is required**: both profiles are CUDA-only, and there is no CPU or MPS fallback.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png
|
||||
```
|
||||
|
||||
@@ -93,6 +201,20 @@ image came from an AI generator, add `--force`:
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
Typography-heavy images can opt into the experimental verified-text post-pass.
|
||||
It accepts manually reviewed strings and line boxes or an operator-verified
|
||||
geometry-only manifest; it never treats raw OCR output as ground truth or runs
|
||||
automatically:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
See the [CLI guide](docs/cli.md#restore-operator-verified-text) for the manifest
|
||||
schema, compatibility restrictions, and oracle caveats.
|
||||
|
||||
See the [installation guide](docs/installation.md) for Homebrew, uv, optional
|
||||
features, and development setup.
|
||||
|
||||
@@ -106,14 +228,14 @@ features, and development setup.
|
||||
|
||||
### High quality invisible removal
|
||||
|
||||
The `qwen-zimage` profile is the highest fidelity option for face heavy images.
|
||||
It is CUDA only and uses a much larger model stack than the default ControlNet
|
||||
profile.
|
||||
`qwen-zimage` is the default profile: a Qwen-Image-2512 Lightning pass under Canny
|
||||
ControlNet, followed by SAM-masked Z-Image repair of any detected face. The
|
||||
alternative, `sdxl-zimage`, swaps the global stage for SDXL and keeps the same face
|
||||
stage. Both are CUDA only.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --force
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
| OpenAI example before | OpenAI example after |
|
||||
@@ -172,7 +294,7 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
```
|
||||
|
||||
CPU offload lowers CUDA memory pressure by moving model components between CPU
|
||||
and GPU. It is slower and has no effect on CPU or MPS.
|
||||
and GPU, at the cost of speed.
|
||||
|
||||
### Process a directory
|
||||
|
||||
@@ -186,7 +308,7 @@ remove-ai-watermarks batch ./images --mode all
|
||||
Visible mark support includes:
|
||||
|
||||
- Google Gemini and Nano Banana sparkle;
|
||||
- Doubao, Jimeng, Qwen, Kling, Baidu, LibLibAI, and RunningHub labels;
|
||||
- Doubao, Jimeng, Qwen, Kling, Yuanbao, Baidu, LibLibAI, and RunningHub labels;
|
||||
- one calibrated Samsung Galaxy AI label variant.
|
||||
|
||||
Metadata and provenance inspection covers C2PA, EXIF, XMP, IPTC, common
|
||||
@@ -206,8 +328,9 @@ Visible removal follows three steps:
|
||||
3. Fill only the masked region with OpenCV, MI-GAN, or LaMa.
|
||||
|
||||
Metadata removal uses format aware stripping. JPEG metadata removal preserves
|
||||
the encoded image scan instead of recompressing it. Other supported containers
|
||||
use their corresponding metadata path.
|
||||
the encoded image scan instead of recompressing it. Native MP4/MOV TC260 values
|
||||
are blanked without changing box sizes or media offsets. Other supported
|
||||
containers use their corresponding metadata path.
|
||||
|
||||
Invisible removal is different. It regenerates the image through a diffusion
|
||||
pipeline to disrupt pixel and frequency domain watermarks. This changes the
|
||||
@@ -226,14 +349,31 @@ import remove_ai_watermarks as raiw
|
||||
|
||||
result, removed = raiw.remove_visible("watermarked.png", "clean.png")
|
||||
print(removed)
|
||||
|
||||
provenance = raiw.identify_video("input.mp4")
|
||||
report = raiw.inspect_video_metadata("input.mp4")
|
||||
complete = raiw.remove_video_all("input.mp4", "clean.mp4")
|
||||
batch = raiw.remove_video_batch("videos", "videos_clean")
|
||||
cleaned = raiw.remove_video_metadata("input.mp4")
|
||||
synthid_cleaned = raiw.remove_video_invisible("input.mp4", "synthid_clean.mp4")
|
||||
visible = raiw.remove_video_visible("input.mp4", "clean.mp4")
|
||||
print(visible.mark)
|
||||
veo = raiw.remove_video_visible("veo.mp4", "veo_clean.mp4", mark="veo")
|
||||
seedance = raiw.remove_video_visible(
|
||||
"seedance.mp4",
|
||||
"seedance_clean.mp4",
|
||||
mark="seedance",
|
||||
)
|
||||
dola = raiw.remove_video_visible("dola.mp4", "dola_clean.mp4", mark="dola")
|
||||
```
|
||||
|
||||
The high level API accepts a file path or a BGR NumPy array. For path inputs it
|
||||
also reads provenance metadata, preserves alpha, and can strip AI metadata from
|
||||
the written result.
|
||||
|
||||
See the [Python API guide](docs/python-api.md) for visible removal, provenance
|
||||
inspection, metadata stripping, and diffusion usage.
|
||||
See the [Python API guide](docs/python-api.md) for visible removal, the full
|
||||
`remove_all` and `remove_batch` pipeline, provenance inspection, metadata
|
||||
stripping, and diffusion usage.
|
||||
|
||||
## ComfyUI
|
||||
|
||||
@@ -250,11 +390,51 @@ invisible removal.
|
||||
and selected fill backend.
|
||||
- Invisible removal changes the whole image and may alter faces, text, or fine
|
||||
detail.
|
||||
- `qwen-zimage` requires CUDA. The other diffusion profiles also support the
|
||||
devices listed by `remove-ai-watermarks invisible --help`.
|
||||
- Visible video removal recognizes the moving Sora 2 wordmark, the current Veo
|
||||
diamond plus legacy `Veo` text, the Seedance boxed `AI` label, and the fixed
|
||||
Dola, Hailuo, and Kling labels. It does not recognize the older Sora Turbo
|
||||
corner swirl or unregistered layouts from those providers.
|
||||
The classical OpenCV backend can smear structured backgrounds; use MI-GAN or
|
||||
LaMa when recovery quality matters.
|
||||
- Video SynthID regeneration changes resolution, frame rate, and image detail.
|
||||
The shipped profile is oracle-certified, but no public local decoder can
|
||||
certify an arbitrary output at runtime. Recheck unusually important outputs
|
||||
after provider changes.
|
||||
- Invisible-watermark removal requires CUDA. Both profiles refuse any other
|
||||
device at construction rather than falling back to one that cannot run them.
|
||||
Visible removal, metadata stripping and `identify` still run anywhere.
|
||||
- Provider watermark systems can change. Validate important outputs with the
|
||||
provider's own verifier when one is available.
|
||||
|
||||
The shipped `video invisible` command uses the certified `noise_std=0.15`
|
||||
profile. The companion `scripts/video_synthid_sweep.py` research harness builds
|
||||
a matched re-encode control plus VAE-regenerated candidates and leaves the
|
||||
verifier verdict blank:
|
||||
|
||||
```bash
|
||||
uv run --extra video --extra diffusion python scripts/video_synthid_sweep.py input.mp4 -o sweep/
|
||||
```
|
||||
|
||||
To score what an output actually cost, use `scripts/video_fidelity_probe.py`:
|
||||
the engine's own PSNR is measured before the resize and the encode, so only the
|
||||
probe sees the delivered picture.
|
||||
|
||||
```bash
|
||||
uv run --extra video python scripts/video_fidelity_probe.py input.mp4 input_clean.mp4
|
||||
```
|
||||
|
||||
The control must still be SynthID-positive before a negative candidate can
|
||||
count as removal evidence. In the 2026-07-29 two-clip calibration, both matched
|
||||
controls were positive in Gemini's built-in SynthID verifier; the stronger
|
||||
candidate was negative on both carriers, while a weaker candidate was
|
||||
negative on one. A later adversarial follow-up that asked ordinary Gemini to
|
||||
reinterpret the pixel result returned `UNAVAILABLE`; that follow-up was not a
|
||||
verifier rerun and does not invalidate the built-in verdicts. A 2026-07-31
|
||||
full-clip check on a public eight-second Veo sample found `0.10` still detected
|
||||
and `0.15` not detected, so `0.15` is now the certified default. The
|
||||
reproducible hashes and verdicts live in
|
||||
`data/evaluations/video-synthid-oracle.csv`.
|
||||
|
||||
## Documentation
|
||||
|
||||
Start with the [documentation index](docs/index.md).
|
||||
|
||||
+43
-1
@@ -6,6 +6,7 @@ Tracked data is organized by purpose:
|
||||
data/
|
||||
fixtures/
|
||||
provenance/ Real format and provenance fixtures used by tests
|
||||
(source records live in fixtures/README.md)
|
||||
calibration/
|
||||
<vendor>/ Minimal controlled inputs needed to rebuild detector assets
|
||||
synthid/
|
||||
@@ -15,6 +16,8 @@ data/
|
||||
Reusable full-pipeline evaluation selection
|
||||
evaluations/
|
||||
fidelity/ Evaluation instructions and hand-verified ground truth
|
||||
video-synthid-oracle.csv
|
||||
Reproducible full-clip Gemini SynthID verdicts
|
||||
```
|
||||
|
||||
## Storage rules
|
||||
@@ -29,10 +32,49 @@ data/
|
||||
commands, hashes, and oracle verdicts instead of committing another corpus
|
||||
copy. A small curated before-and-after example may live in `docs/images/`
|
||||
when it is part of the public documentation.
|
||||
6. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
6. Keep third-party fixture license notices in `licenses/`, outside directories
|
||||
that tests enumerate as media inputs.
|
||||
7. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
Unregistered research candidates belong in
|
||||
`scripts/assets/visible-mark-candidates/` so they are not shipped in the
|
||||
wheel.
|
||||
|
||||
The source distribution excludes `data/`; the wheel contains only package
|
||||
runtime assets.
|
||||
|
||||
## Video SynthID oracle manifest
|
||||
|
||||
`evaluations/video-synthid-oracle.csv` is the only evidence that the shipped
|
||||
video removal profile works, so it is also the source of truth for three shipped
|
||||
defaults: `tests/test_video_invisible.py` asserts that `noise_std`, `long_side`,
|
||||
and `fps` together match a row this manifest records as certified. Changing one
|
||||
of those three without adding the row that certifies it fails the suite. `vae` is
|
||||
deliberately outside that check because neither tracked row records one; add it
|
||||
to the assertion in the same commit as the first row that does.
|
||||
|
||||
| Column | Meaning |
|
||||
| --- | --- |
|
||||
| `date`, `source_url`, `source_sha256` | Identify the carrier. |
|
||||
| `source_width`, `source_height`, `source_fps` | Carrier geometry. Without it the actual downscale factor of a row cannot be recovered later. |
|
||||
| `duration_seconds`, `source_verdict` | Clip length submitted and the verifier's reading of the untouched carrier. |
|
||||
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The full run configuration. |
|
||||
| `output_sha256` | Identifies the exact submitted file. |
|
||||
| `output_verdict` | One of `detected`, `not_detected`, `indeterminate`, `refused`. |
|
||||
| `output_verdict_text` | The verifier's wording, verbatim. |
|
||||
| `output_detected_range` | Time range the verifier reported for the output. |
|
||||
| `track` | `visual`, `audio`, `both`, or empty. The verifier scores tracks separately and this path copies source audio unchanged. |
|
||||
| `session_id` | Groups rows submitted in one oracle session, so per-session drift stays visible. |
|
||||
| `stratum` | Content class of the carrier, for stratified certification. |
|
||||
| `psnr_db`, `temporal_residual_ratio` | Fidelity measurements. Neither is a watermark verdict. |
|
||||
|
||||
Record `indeterminate` when the verifier answers with its unclear state rather
|
||||
than a negative: an unclear reading logged as `not_detected` is exactly the
|
||||
silent regression this manifest exists to prevent. Leave a field empty when it
|
||||
was not recorded, and never backfill it with a plausible value.
|
||||
|
||||
`psnr_db` is measured against the already-resized frame and before the encode,
|
||||
so it excludes the downscale, the decimation, and the codec. Rows stay
|
||||
comparable to each other only while that definition holds.
|
||||
|
||||
The two 2026-07-31 rows predate this schema; their empty fields were never
|
||||
recorded and are not recoverable from the row.
|
||||
@@ -7,31 +7,394 @@ ground truth and instructions.
|
||||
|
||||
| Original | Provider | Content | Exercises |
|
||||
| --- | --- | --- | --- |
|
||||
| `ChatGPT Image May 31, 2026, 02_02_23 PM.png` | OpenAI | Light multilingual typography | Text preservation |
|
||||
| `ChatGPT Image May 31, 2026, 02_03_55 PM.png` | OpenAI | Multilingual typography | Text preservation |
|
||||
| `Gemini_Generated_Image_633uuy633uuy633u.png` | Google | Landscape with a Chinese sign | CJK text preservation |
|
||||
| `Gemini_Generated_Image_y48j3cy48j3cy48j.png` | Google | Portrait grid | Face identity and skin texture |
|
||||
|
||||
## Text ground truth
|
||||
|
||||
`ground-truth.json` contains hand-verified OCR for the two text-bearing
|
||||
`ground-truth.json` contains hand-verified OCR for the three text-bearing
|
||||
originals. To regenerate an OCR seed:
|
||||
|
||||
```bash
|
||||
uv run scripts/fidelity_metrics.py ocr \
|
||||
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png" \
|
||||
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png" \
|
||||
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
|
||||
--langs en,ru,ch \
|
||||
--out data/evaluations/fidelity/ground-truth.json
|
||||
```
|
||||
|
||||
Verify and correct the generated text by hand before using it as ground truth.
|
||||
`text-lines.json` contains the verified per-line strings and source-space boxes
|
||||
used by the evaluation-only selective-restoration experiment. It is not an
|
||||
automatic scene-text annotation set.
|
||||
|
||||
`scripts/infer_text_lines.py` can generate a draft from source pixels without
|
||||
existing annotations. On the two posters it proposed 20 and 18 lines at the
|
||||
default threshold, but exact-text precision was only 90.0% and 94.4% because
|
||||
high-confidence OCR still dropped an English comma and replaced a Chinese comma
|
||||
with ASCII. Its output therefore requires manual verification of every line;
|
||||
`accepted` means crop-stable, not ground-truth-correct.
|
||||
|
||||
## Text-preservation benchmark
|
||||
|
||||
`text-preservation-2026-08-13.csv` records a fixed-seed comparison of the two
|
||||
current profiles and a global Z-Image Turbo prototype on all three text
|
||||
fixtures. Each candidate ran through the
|
||||
complete `visible -> invisible -> metadata` route with its profile default
|
||||
strength and adaptive-polish setting. The Z-Image prototype instead sweeps
|
||||
0.08, 0.10, 0.15, 0.20, and 0.25 without polish because its provider-specific
|
||||
removal floor is not known. The output hash identifies the exact bytes
|
||||
measured; generated outputs remain outside the repository.
|
||||
|
||||
The character-weighted added CER is 0.262 for `qwen-zimage` and 0.256 for
|
||||
`sdxl-zimage`. That 0.006 absolute difference is not a
|
||||
stable ordering: SDXL wins the light poster, Qwen wins the dark poster, and the
|
||||
Chinese sign is tied. A paired sign test on the two non-ties is 1-1 (`p=1.0`).
|
||||
The measured sample therefore does not support a general text-preservation
|
||||
winner. Both profiles substantially degrade the smallest multilingual poster
|
||||
text and preserve the larger Chinese sign.
|
||||
|
||||
Z-Image Turbo is the clear fidelity lead. At strength 0.10 its
|
||||
character-weighted CER is 0.093, against the unchanged sources' 0.124 OCR
|
||||
floor, while whole-image LPIPS stays between 0.047 and 0.086. Visual inspection
|
||||
still finds substitutions in the smallest Cyrillic and Chinese poster lines,
|
||||
so a favorable OCR score does not mean pixel-exact text. Quality drops quickly
|
||||
above 0.15; at 0.25 the weighted CER is 0.428.
|
||||
|
||||
Provider-oracle checks on 2026-08-13 bracket the OpenAI requirement at no more
|
||||
than 0.10 for this sample. Both original OpenAI controls were detected by
|
||||
`openai.com/verify`; the light poster was clean at 0.08, while the dark poster
|
||||
was detected at 0.08 and clean at 0.10. The light 0.10 output was not separately
|
||||
checked. Google fails the entire tested sweep. The original Gemini image was
|
||||
detected through C2PA, and a pixel-identical copy with AI metadata stripped was
|
||||
separately detected by Gemini's built-in SynthID verifier. Z-Image outputs at
|
||||
0.08, 0.10, 0.15, 0.20, and 0.25 were all still detected. At 0.25 the weighted
|
||||
CER has already risen to 0.428, worse than the raw weighted CER of both current
|
||||
profiles (Qwen 0.387, SDXL 0.381). Increasing strength beyond the measured grid
|
||||
would therefore no longer serve the text-preservation objective without a new
|
||||
mechanism or hypothesis. Z-Image is not a viable global replacement on this
|
||||
evidence: no clean Google operating point was found before it lost its fidelity
|
||||
advantage.
|
||||
|
||||
Qwen-Image-2.0 was not added to the numeric comparison. Its weights are not
|
||||
published, and its hosted edit API exposes an editing instruction and seed but
|
||||
no low-strength denoise control. It can be evaluated as a separate hosted edit
|
||||
strategy when credentials are available, but it is not a drop-in replacement
|
||||
for the partial-regeneration mechanism measured here.
|
||||
|
||||
The OCR floor is the source image scored against the hand-verified text. Use
|
||||
`added_cer = text_cer - ocr_floor` when interpreting pipeline damage, because the
|
||||
unchanged poster sources already score 0.127 CER. `oracle_rechecked=true` marks
|
||||
the exact Z-Image bytes checked above; the remaining rows were not rechecked.
|
||||
The table does not certify other seeds, content classes, or strengths beyond
|
||||
the recorded provider verdicts.
|
||||
|
||||
## Text-restoration prototype
|
||||
|
||||
`text-restoration-2026-08-13.csv` evaluates an OCR-driven post-pass on the exact
|
||||
Qwen outputs above. The prototype recognizes English and Russian with macOS
|
||||
Vision and CJK with PaddleOCR, derives glyph masks independently from the
|
||||
source and Qwen output, removes both sets of glyphs with block-wise LaMa, and
|
||||
draws the recognized strings with new system-font pixels. It never composites
|
||||
source pixels back into the result.
|
||||
|
||||
On the two multilingual posters, character-weighted CER fell from 0.338 and
|
||||
0.305 to 0.007 on both. OpenAI Verify reported no OpenAI signals for both Qwen
|
||||
controls and both restored outputs in the same run. The improvement comes with
|
||||
a substantial whole-image fidelity cost: LPIPS rose from 0.107 to 0.174 and
|
||||
from 0.095 to 0.162, while PSNR fell by about 10.5-11.4 dB. Visual inspection
|
||||
found one residual shadow in the smallest English line of the light poster;
|
||||
the dark poster was clean but the substitute fonts visibly changed typography.
|
||||
|
||||
The Chinese sign did not improve: CER rose from 0.074 to 0.111 because the OCR
|
||||
and renderer changed punctuation. The Gemini verifier returned detected for
|
||||
both the restored output and its byte-identical Qwen control on a third work
|
||||
account, although that Qwen hash had previously returned clean on another work
|
||||
account. This run therefore does not isolate a restoration-stage regression;
|
||||
its Google verdict is inconclusive until a source-positive, Qwen-negative
|
||||
control can be reproduced in the same available account.
|
||||
|
||||
This prototype is not ready to ship. Its strongest result establishes that
|
||||
fresh-glyph reconstruction can recover literal text without reintroducing an
|
||||
OpenAI signal, but portable OCR, font/style reconstruction, a tighter mask, and
|
||||
a reproducible Google oracle control remain prerequisites.
|
||||
|
||||
### Selective restoration follow-up
|
||||
|
||||
`selective-text-restoration-2026-08-13.csv` compares that full compositor with
|
||||
a selective prototype on the same two posters. The prototype leaves a Qwen line
|
||||
unchanged when padded source and output recognition agree, and applies the same
|
||||
LaMa plus fresh-system-font reconstruction only to lines whose recognized text
|
||||
changed. This reduced the edited area from 15.4% and 17.0% to 5.5% on both
|
||||
posters.
|
||||
|
||||
Under one consistent Paddle `en+ru+ch` measurement route, selective restoration
|
||||
reduced Qwen CER from 0.378 to 0.101 on the light poster and from 0.413 to 0.112
|
||||
on the dark poster. Its image LPIPS was 0.120 and 0.103, substantially closer to
|
||||
Qwen than the full compositor's 0.174 and 0.162. These CER values must not be
|
||||
mixed with the preceding table's Vision/Paddle hybrid values: the comparison
|
||||
file remeasures all three variants through Paddle so their relative result is
|
||||
valid on one OCR route.
|
||||
|
||||
OpenAI Verify returned `No OpenAI signals detected` for both selective outputs,
|
||||
then detected the original light poster as `Generated with OpenAI tools` in the
|
||||
same Chrome sequence. Visual inspection found the dark output clean, but the
|
||||
light output still retained a local shadow around one replaced fine-text line.
|
||||
Selective restoration is therefore the strongest current direction, not a
|
||||
production-ready default. The next implementation needs automatic line matching
|
||||
and a tighter source-plus-candidate glyph mask before it can be proposed for the
|
||||
pipeline.
|
||||
|
||||
A mask-only follow-up added two pixels of dilation around every selected glyph.
|
||||
It visually removed the light poster's shadow and improved its CER from 0.101 to
|
||||
0.081 with effectively unchanged LPIPS, but increased the edited area from 5.5%
|
||||
to 7.0%. Applying the same rule to the dark poster worsened CER from 0.112 to
|
||||
0.147. These expanded-mask rows were not resubmitted to the provider oracle.
|
||||
The opposing result rules out one global dilation constant and makes per-line
|
||||
residual detection the next fidelity gate.
|
||||
|
||||
An automatic follow-up removed the hand-selected line list. For each known
|
||||
source line, it runs a recognition-only model on padded, upscaled source and
|
||||
Qwen crops. It preserves a line only when both confidence scores are at least
|
||||
0.75, the source recognition is within 0.25 normalized edit distance of the
|
||||
verified line, and normalized source and candidate recognition match exactly.
|
||||
This reproduced the manual dark-poster line selection and additionally kept one
|
||||
correct large Russian line on the light poster.
|
||||
|
||||
The automatic prototype also replaced uniform dilation with a second measured
|
||||
LaMa pass. After the first erase, it extracts contrast remaining at the original
|
||||
glyph positions, dilates only that residual, and inpaints it before drawing fresh
|
||||
text. Visual inspection found no remaining double glyphs on either output. The
|
||||
effective edited fractions were 4.1% and 5.7%. Image LPIPS was 0.113 and 0.104,
|
||||
against 0.120 and 0.103 for the manual selective version. Detector-based page CER
|
||||
was 0.123 and 0.119; the light value is inflated by the detector splitting the
|
||||
preserved Cyrillic line into Latin-like fragments even though the padded Russian
|
||||
recognizer reads it exactly. This is another reason that line decisions use
|
||||
recognition-only crops rather than the page detector's aggregate CER.
|
||||
|
||||
Both exact automatic output hashes returned `No OpenAI signals detected`, and
|
||||
the original light poster returned `Generated with OpenAI tools` immediately
|
||||
afterward in the same Chrome sequence. The tracked script reproduced the dark
|
||||
hash exactly. Its light output differed from the initial prototype in only 386
|
||||
pixels by at most one channel value because separate ONNX environments rounded
|
||||
differently; the script's exact `900def5a...` hash was therefore submitted and
|
||||
also returned clean before the positive source control.
|
||||
|
||||
The script and its required verified line annotations are tracked as
|
||||
`scripts/selective_text_restoration.py` and `text-lines.json`. For example:
|
||||
|
||||
```bash
|
||||
PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK=True \
|
||||
uv run scripts/selective_text_restoration.py \
|
||||
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
|
||||
path/to/qwen-candidate.png \
|
||||
--restoration vae-glyphs \
|
||||
--glyph-donor path/to/qwen-vae-reconstruction.png \
|
||||
--glyph-feather 0.5 \
|
||||
--output path/to/selective-output.png \
|
||||
--manifest path/to/manifest.json
|
||||
```
|
||||
|
||||
The restoration mode is deliberately explicit. `vae-glyphs` requires a
|
||||
separately generated VAE reconstruction in `--glyph-donor`; it takes exact
|
||||
thresholded glyph cores from that donor, uses a fresh silhouette beneath them,
|
||||
and defaults to a narrow 0.5-pixel donor edge. The donor and the resulting exact bytes still require
|
||||
full-pipeline oracle evaluation. `rerender` reproduces the tracked
|
||||
fresh-system-font experiment. `source-glyphs` is only for compositing a
|
||||
separately regenerated, oracle-evaluated layer; feeding it the watermarked
|
||||
original would paste provenance-bearing pixels back into the result. The
|
||||
`--keep-background --composite-mask boxes` combination is an aligned-layer
|
||||
experiment and is not a production text restorer.
|
||||
|
||||
The `source-silhouette` follow-up keeps the source glyph geometry but discards
|
||||
its pixel amplitudes: it thresholds each source line to a binary shape, samples
|
||||
one foreground color, and synthesizes new antialiasing over the scrubbed image.
|
||||
On a 24-image matrix spanning serif, Latin-diacritic, Cyrillic, CJK, tiny-UI,
|
||||
and rotated text, the original outer-feather compositor preserved all 34
|
||||
source-readable lines exactly under crop OCR, with median text-box SSIM 0.859.
|
||||
An inner-antialias variant raised median text-box SSIM to 0.902 and left
|
||||
whole-image SSIM unchanged at 0.672, but preserved 33/34 lines: OCR read one
|
||||
middle dot as a colon. The higher-fidelity antialiasing is retained in the
|
||||
evaluation script, with that punctuation miss recorded as a caveat. Its exact
|
||||
outputs were not submitted to the provider oracle because the
|
||||
public verifier reached its request limit; a clean verdict from the earlier
|
||||
outer-feather bytes does not transfer to the new hashes.
|
||||
|
||||
Follow-up visual review on a typography-rich control rejected both silhouette
|
||||
compositors despite their OCR and SSIM scores. They preserved literal content
|
||||
and approximate glyph geometry, but changed stroke weight, color variation,
|
||||
edge antialiasing, and small decorative details enough to be plainly different
|
||||
from the source. OCR exactness and text-box SSIM are therefore screening metrics,
|
||||
not acceptance gates for source-typography preservation. A candidate must also
|
||||
retain source-like edge pixels and pass direct visual comparison at native size.
|
||||
|
||||
A narrower Qwen-VAE donor follow-up keeps the scrubbed background, then copies
|
||||
only VAE-reconstructed pixels through the source-silhouette mask with a
|
||||
0.5-pixel feather. Across the 48-case typography matrix (548 annotated lines),
|
||||
text-box SSIM improved in 47/48 cases and glyph-edge error improved in 48/48.
|
||||
Median text-box SSIM rose from 0.854 to 0.914, while median glyph-edge MAE fell
|
||||
from 37.59 to 32.42. The median nonzero alpha area was 3.62%; on dense or large
|
||||
text this is still substantially wider than the target control. These are
|
||||
fidelity results only; the 48 matrix outputs were not submitted to the provider
|
||||
oracle. On a separate untracked dense-typography control, the exact core-only
|
||||
Qwen-VAE donor with a 1.43% source-silhouette area returned `No OpenAI signals
|
||||
detected`. Its 0.5-pixel feathered sibling, which raises the nonzero donor area
|
||||
to 2.75%, returned the same verdict and improved mean text-box SSIM from 0.957
|
||||
to 0.960. Crop OCR recovered from 7/15 exact lines on the raw pass to 14/15,
|
||||
matching the source OCR floor. Whole-image LPIPS was 0.082, but only 0.108% of
|
||||
pixels were exactly unchanged and the detected face retained 0.670 of source
|
||||
Laplacian variance. The tracked script reproduced the feathered file byte for
|
||||
byte. These two exact-byte verdicts do not certify other images or the larger
|
||||
matrix masks, and the global smoothing fails a strict unchanged-image criterion.
|
||||
|
||||
The opt-in production port was rechecked separately on 2026-08-15. Its current
|
||||
LaMa runtime did not reproduce the earlier evaluation PNG byte for byte, but all
|
||||
changed pixels were confined to the erased background outside the donor glyph
|
||||
core. The exact production artifact returned `No OpenAI signals detected` in
|
||||
3/3 OpenAI Verify runs, while the matched source control returned `Generated
|
||||
with OpenAI tools` in 2/2 runs in the same Chrome session; expanded details
|
||||
identified SynthID and no C2PA manifest on the control. The private control and
|
||||
artifact hashes remain outside the public repository. This certifies only that
|
||||
runtime, verified manifest, and output, not arbitrary text masks or images.
|
||||
|
||||
The Google result is negative. On the synthetic CJK sign case, two separate
|
||||
work-account runs both detected SynthID in the resaved source control and in the
|
||||
exact Qwen-VAE donor output. The candidate improved mean text-box SSIM from
|
||||
0.791 to 0.835 and glyph-edge MAE from 35.48 to 22.49 with a 3.67% donor area,
|
||||
but Gemini still detected it. The intermediate Qwen silhouette base and the
|
||||
earlier original-sign Qwen baseline were also detected in the same account.
|
||||
This isolates the current blocker upstream of text restoration: the global
|
||||
Google pass did not reach the SynthID removal floor, so the OpenAI-clean
|
||||
`vae-glyphs` configuration cannot be used as a Google operating point.
|
||||
|
||||
On the large Chinese-sign control, source and candidate recognition agreed on
|
||||
all three lines. The script selected no changed lines, emitted a zero mask, and
|
||||
copied the candidate byte for byte instead of re-encoding it. This corrects the
|
||||
earlier unnecessary CJK rerender and proves the no-edit branch. It does not add
|
||||
a Google-negative oracle result: the available Google account still cannot
|
||||
reproduce a source-positive, Qwen-negative control sequence.
|
||||
|
||||
The earlier automatic rerender was the first variant in the experiment to pass
|
||||
the recorded visual, fidelity, and OpenAI-oracle gates on its two poster
|
||||
fixtures. The later typography-rich control shows that result does not
|
||||
generalize to source-typography preservation. It remains evaluation-only: it
|
||||
depends on verified source text and source line boxes, uses macOS system fonts,
|
||||
and has not been validated on natural scene text, rotated text, or automatic
|
||||
line-box discovery.
|
||||
|
||||
An opt-in `--detect-boxes` follow-up tested automatic geometry. Grouping Paddle
|
||||
word detections by vertical overlap found exactly 20/20, 20/20, and 3/3 lines;
|
||||
mean IoU with verified boxes was 0.857, 0.847, and 1.000. Reusing the annotation
|
||||
crop padding was unstable and preserved only 4/20 dark-poster lines. Reducing
|
||||
vertical recognition padding to 10% restored the exact 8/20 and 7/20 selection
|
||||
decisions, but detector CER was 0.127 and 0.154 rather than 0.123 and 0.119. The
|
||||
dark regression failed the fidelity gate, so these hashes were not submitted to
|
||||
the provider oracle. The flag is retained only to reproduce that negative
|
||||
evaluation and still requires verified strings and an exact line-count match.
|
||||
|
||||
### AnyText2 glyph-conditioned follow-up
|
||||
|
||||
`anytext2-restoration-2026-08-13.csv` tests the official Apache-2.0
|
||||
AnyText2 checkpoint as a local text-editing pass over the exact Qwen Chinese-sign
|
||||
output. The checkpoint hash was verified against ModelScope. Its own edit example
|
||||
successfully replaced a masked blackboard line with the requested `DADDY`, so the
|
||||
runtime and checkpoint were functional before the tracked fixture was measured.
|
||||
|
||||
AnyText2 failed the fidelity gate. The standard full-image detector scored the
|
||||
default edit at CER 0.185 and the source-font mimic at 0.222, against 0.074 for
|
||||
Qwen; image LPIPS moved from 0.289 to 0.338 and 0.345. A padded crop-recognition
|
||||
cross-check removed the detector's punctuation misses: Qwen scored 0.000, the
|
||||
default edit still scored 0.074 with two substituted characters, and the mimic
|
||||
edit stayed at 0.222. Visual inspection agreed, with additional line-shape and
|
||||
punctuation errors in the mimic output. No provider-oracle submission was made
|
||||
because neither candidate preserved the requested content better than its
|
||||
already-clean Qwen input.
|
||||
|
||||
This result covers the published SD1.5 checkpoint, three Chinese lines of no more
|
||||
than 20 characters, one seed, and one fixture. The repository still describes
|
||||
AnyText2XL as unreleased, and the inference wrapper truncates each requested line
|
||||
after 20 characters, so the published model cannot directly cover the longer
|
||||
English and Russian poster lines.
|
||||
|
||||
The padded cross-check also changes the interpretation of this fixture's earlier
|
||||
restoration result. Its Qwen image already contains all three correct Chinese
|
||||
lines; the reported 0.074 came from the detector cropping the final punctuation
|
||||
from two lines. A deterministic rerender with horizontally padded recognition
|
||||
and a mask-matched Hiragino Sans GB W6 font reached detector CER 0.000, but image
|
||||
LPIPS worsened from 0.289 to 0.327 and the font was visibly heavier. The correct
|
||||
policy on this large-text fixture is therefore to leave a Qwen line unchanged
|
||||
when padded recognition confirms the source and candidate strings agree.
|
||||
|
||||
## Face-stage isolation
|
||||
|
||||
`face-stage-isolation-2026-08-13.csv` separates the two stages on the 18-face
|
||||
Gemini portrait-grid fixture. All variants use the same visible-cleaned,
|
||||
metadata-stripped input and seed 0. The prepared input remained positive in the
|
||||
Gemini SynthID verifier, so the surrounding visible and metadata stages did not
|
||||
create the clean verdicts.
|
||||
|
||||
| Route | Gemini oracle | ID cosine | Face LPIPS | Laplacian ratio | Image LPIPS |
|
||||
| --- | --- | ---: | ---: | ---: | ---: |
|
||||
| Prepared control | detected | - | - | - | - |
|
||||
| Z-Image face stage only | detected | 0.927 | 0.028 | 0.612 | 0.013 |
|
||||
| Qwen global stage only | clean | 0.589 | 0.217 | 0.331 | 0.288 |
|
||||
| Full Qwen then Z-Image | clean | 0.852 | 0.050 | 0.492 | 0.252 |
|
||||
|
||||
This isolates the roles: the Qwen global pass removes the watermark, while the
|
||||
masked Z-Image pass restores face identity and detail without reintroducing a
|
||||
detectable whole-image signal. Z-Image face-only does not remove SynthID. The
|
||||
result explains why the face stage succeeded while global Z-Image failed: the
|
||||
earlier clean verdict belonged to the two-stage pipeline, not to Z-Image as a
|
||||
standalone remover. This is one fixture, one seed, and one oracle pass per
|
||||
variant; it does not establish a general reintroduction threshold for mask size.
|
||||
|
||||
## Upstream Synthid-Bypass v2 reproduction
|
||||
|
||||
`upstream-v2-reproduction-2026-08-13.csv` records a source-level audit and a
|
||||
close reproduction of the public
|
||||
[`Synthid-Bypass-v2.0.json`](https://github.com/cebeuq/Synthid-Bypass/blob/3007d0351596ae0a78b7074dae7ad179710b1e48/Synthid-Bypass-v2.0.json).
|
||||
The audited workflow is SHA-256
|
||||
`41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6` at
|
||||
upstream commit `3007d0351596ae0a78b7074dae7ad179710b1e48`.
|
||||
|
||||
Tracing links backward from `SaveImage` confirms that the active global stage is
|
||||
Qwen-Image-2512 Q4 with the Lightning four-step LoRA at 0.8, DiffSynth Canny at
|
||||
1.0, AuraFlow shift 3, `dpmpp_2m` plus `sgm_uniform`, CFG 1, and four steps. Its
|
||||
resolution helper uses adaptive level 6 over 0.08..0.15, which resolves to the
|
||||
0.154 ceiling for the 2816x1536 portrait fixture. Z-Image Turbo is used only by
|
||||
the face detailer: eight steps, CFG 1, `res_2s` plus `bong_tangent`, a 768 px
|
||||
guide, 1024 px cap, and direct adaptive denoise
|
||||
`clamp(0.10 * largest_face_ratio / 0.03, 0.05, 0.28)`. The active face regions
|
||||
come from YOLOv8-face plus SAM. Although the README describes MediaPipe as part
|
||||
of the strict path, the MediaPipe nodes do not reach the saved output. The
|
||||
1.2-megapixel scaler is also bypassed (`mode=4`).
|
||||
|
||||
The published upstream pair 12 changed from SynthID-positive to clean in the
|
||||
Gemini verifier and scored 0.975 face identity. On the project portrait fixture,
|
||||
the prepared control was positive, while both the close global reproduction and
|
||||
the close full reproduction were clean. The global stage scored 0.589 identity
|
||||
and 0.217 face LPIPS; the upstream-strength face stage improved those to 0.783
|
||||
and 0.083. The maintained profile's weaker face pass scored 0.852 and 0.050 on
|
||||
the same control, so copying upstream's roughly doubled face strength would be a
|
||||
quality regression on this fixture.
|
||||
|
||||
The close reproduction is intentionally labeled rather than presented as an
|
||||
exact ComfyUI run. It uses full-precision DiffSynth bf16 weights instead of the
|
||||
Q4 GGUF files, the maintained DiffSynth Lightning scheduler approximation
|
||||
instead of the ComfyUI sampler pair, YuNet plus SAM instead of YOLOv8-face plus
|
||||
SAM, and fixed seed 0 instead of randomized seeds. The two independent oracle
|
||||
controls and the published upstream pair establish the direction of the result;
|
||||
they do not make the local output byte-equivalent to upstream.
|
||||
|
||||
## Compare
|
||||
|
||||
```bash
|
||||
uv run scripts/fidelity_metrics.py compare \
|
||||
--original data/synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png \
|
||||
--variant controlnet=<out>.png \
|
||||
--variant qwen=<out>.png \
|
||||
--variant qwen-zimage=<qwen-out>.png \
|
||||
--variant sdxl-zimage=<sdxl-out>.png \
|
||||
--ocr-langs ""
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
date,source_commit,fixture,variant,model,model_revision,model_sha256,ocr_route,text_cer,img_lpips,ssim,psnr_db,output_sha256,provider_oracle,notes
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,,,,"Paddle ch",0.074,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621,not_run,Exact Qwen baseline reused from text-restoration benchmark
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-default,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.185,0.338,0.642,18.2,87729fe7e0482d2cead6d3b7c7dd386ecc75ab3d8658131c54e67528b8f9feeb,not_run,Fidelity gate failed; padded crop recognition CER 0.074 still shows two substituted characters; locally composited three-line edit; 20 DDIM steps; seed 0
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-mimic,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.222,0.345,0.644,17.9,f5198d8f2365b79880aba8ea6b0c3885142b9de21c9f7f4f6712ac4a0db8dd85,not_run,Fidelity gate failed; source glyph masks supplied as font hints; 20 DDIM steps; seed 0
|
||||
|
@@ -0,0 +1,5 @@
|
||||
date,source_commit,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,zimage-face-only,0,,0.06604512532552083,18,0.927,0.028,0.612,0.013,0.959,34.7,c388e9bbf737bb9b22a7cb2216ffb8026321ca632f9db29e1e456b7c14a8553f,synthid_detected
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-global-only,0,0.153999996,,18,0.589,0.217,0.331,0.288,0.646,24.2,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-zimage-full,0,0.153999996,0.06604512532552083,18,0.852,0.050,0.492,0.252,0.671,24.7,69de191bddbfbf53348c7638bdba661b6d45d257c00a8a2646b2558670c5a4c2,clean
|
||||
|
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行，也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
|
||||
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行，也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png": "每天都是一个新的机会。\n用微笑开始它。\n世界也会向你微笑。"
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Paddle en+ru+ch",0.378,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.048,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.101,0.120,0.796,19.5,0.054799,15b40b8cdf9e97558c66d4acdcb97d46769e64f04f143f8061746bc731fbbc21,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.081,0.119,0.796,19.5,0.070267,5a8a0bcbe5763d1b1af0d64149cec7e35b16c5751e995d3bacf17581d64177ec,not_checked,fidelity-only follow-up
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.123,0.113,0.808,21.2,0.041279,900def5a433c039aa474789eddc4dc4c63b8126ba1aff6829d2e8d600b9f9c71,no_openai_signals_detected,source detected immediately afterward in the same Chrome sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Paddle en+ru+ch",0.413,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.086,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.112,0.103,0.780,20.8,0.054912,846f39db0ad4a48a734a5019f0b0ff5d110eecb3210223407e72529dfa4fcf54,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.147,0.104,0.779,20.8,0.072556,8ae76e1dedeec26bebb8d35e71fd92d77f142dbb0f3440b330cdf33b79558afe,not_checked,fidelity-only follow-up
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.119,0.104,0.779,20.8,0.057110,ac4201bd2e342b1a37737319438cedb2203f6bb58cd9113f825cf9b80d913755,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-selective-automatic-passthrough,"Paddle detector ch; horizontal-only padded recognizer selection",0.074,0.289,0.696,23.3,0.000000,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_checked,all three source and candidate recognitions agreed exactly; output is byte-identical to candidate
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.127,0.113,0.805,20.9,0.041280,70c52fed8b8a1d98f4cda8558ae08825313e510dfb372e448370d8dcf1da73b0,not_checked,fidelity-only automatic-box follow-up; same 8/20 preserve decisions
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.154,0.104,0.778,20.8,0.057146,1fa01ce706042c003a61429d5d97fc6c928ec32278071a91926d588f1e760b00,not_checked,fidelity-only automatic-box follow-up; same 7/20 preserve decisions but worse CER
|
||||
|
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": [
|
||||
{"box": [156, 206, 709, 296], "text": "This is a longer", "script": "alphabetic"},
|
||||
{"box": [157, 293, 939, 379], "text": "sample text in English.", "script": "alphabetic"},
|
||||
{"box": [165, 393, 774, 434], "text": "Typography can flow smoothly from", "script": "alphabetic"},
|
||||
{"box": [161, 434, 521, 477], "text": "large to medium size.", "script": "alphabetic"},
|
||||
{"box": [165, 495, 566, 522], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
|
||||
{"box": [165, 524, 354, 550], "text": "rhythm, and clarity.", "script": "alphabetic"},
|
||||
{"box": [165, 564, 448, 585], "text": "Even the finest text should remain", "script": "alphabetic"},
|
||||
{"box": [165, 593, 330, 613], "text": "clean and readable.", "script": "alphabetic"},
|
||||
{"box": [158, 673, 903, 740], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
|
||||
{"box": [162, 753, 697, 793], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
|
||||
{"box": [164, 807, 524, 829], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
|
||||
{"box": [163, 845, 483, 868], "text": "即使是最小的一行，也应该保持清楚易读。", "script": "cjk"},
|
||||
{"box": [162, 951, 780, 1009], "text": "Это более длинный пример", "script": "alphabetic"},
|
||||
{"box": [161, 1019, 723, 1062], "text": "текста на русском языке.", "script": "alphabetic"},
|
||||
{"box": [165, 1076, 639, 1104], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
|
||||
{"box": [165, 1111, 505, 1133], "text": "крупного размера к среднему.", "script": "alphabetic"},
|
||||
{"box": [165, 1151, 513, 1172], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
|
||||
{"box": [167, 1178, 287, 1192], "text": "ритм и ясность.", "script": "alphabetic"},
|
||||
{"box": [165, 1210, 417, 1227], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
|
||||
{"box": [165, 1235, 381, 1247], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
|
||||
],
|
||||
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": [
|
||||
{"box": [165, 204, 754, 299], "text": "This is a longer", "script": "alphabetic"},
|
||||
{"box": [167, 295, 1006, 385], "text": "sample text in English.", "script": "alphabetic"},
|
||||
{"box": [171, 401, 831, 446], "text": "Typography can flow smoothly from", "script": "alphabetic"},
|
||||
{"box": [169, 446, 560, 491], "text": "large to medium size.", "script": "alphabetic"},
|
||||
{"box": [173, 507, 605, 536], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
|
||||
{"box": [173, 538, 379, 567], "text": "rhythm, and clarity.", "script": "alphabetic"},
|
||||
{"box": [175, 583, 481, 603], "text": "Even the finest text should remain", "script": "alphabetic"},
|
||||
{"box": [173, 611, 352, 632], "text": "clean and readable.", "script": "alphabetic"},
|
||||
{"box": [170, 693, 956, 757], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
|
||||
{"box": [174, 775, 740, 812], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
|
||||
{"box": [175, 829, 561, 852], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
|
||||
{"box": [174, 869, 534, 892], "text": "即使是最小的一行，也应该保持清楚易读。", "script": "cjk"},
|
||||
{"box": [169, 958, 824, 1022], "text": "Это более длинный пример", "script": "alphabetic"},
|
||||
{"box": [171, 1029, 764, 1074], "text": "текста на русском языке.", "script": "alphabetic"},
|
||||
{"box": [175, 1090, 678, 1119], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
|
||||
{"box": [175, 1125, 546, 1151], "text": "крупного размера к среднему.", "script": "alphabetic"},
|
||||
{"box": [175, 1169, 552, 1192], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
|
||||
{"box": [175, 1196, 312, 1217], "text": "ритм и ясность.", "script": "alphabetic"},
|
||||
{"box": [175, 1233, 479, 1253], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
|
||||
{"box": [177, 1259, 440, 1272], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
|
||||
],
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png": [
|
||||
{"box": [1281, 650, 2357, 818], "text": "每天都是一个新的机会。", "script": "cjk"},
|
||||
{"box": [1466, 788, 2167, 957], "text": "用微笑开始它。", "script": "cjk"},
|
||||
{"box": [1367, 910, 2264, 1112], "text": "世界也会向你微笑。", "script": "cjk"}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
date,source_commit,fixture,profile,route,seed,strength,adaptive_polish,text_cer,ocr_floor,added_cer,img_lpips,ssim,psnr_db,output_sha256,oracle_rechecked
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.378,0.127,0.251,0.107,0.832,24.4,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.356,0.127,0.229,0.125,0.707,25.3,9bb932d6e66b52f34ff07c3fcdbdb9c33eb601ae2cd84bec6b8c7b5e30df9926,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.413,0.127,0.286,0.095,0.818,26.0,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.422,0.127,0.295,0.099,0.754,25.9,7605b1175b0c99f481ae6929acc488f34d98bc0f2571d42a1d2972b671d9b8cb,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,original,reference,,,,0.037,0.037,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,visible-invisible-metadata,0,0.153999996,false,0.074,0.037,0.037,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,sdxl-zimage,visible-invisible-metadata,0,0.25,true,0.074,0.037,0.037,0.176,0.769,26.1,d9f89465c31bd2d38ffb111775b9bbea4577f40713fcbebf421e05d344ab27ad,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.068,0.127,-0.059,0.042,0.874,31.4,534de6422a3a03cfde44377aa9bf824a8267db3b73511240f8f07fb8dcb95eff,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.086,0.127,-0.041,0.047,0.862,30.8,4dba8211e8d84ebbfff72de00f54d4365d0d202394acd412d1419ae93d1e9f71,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.177,0.127,0.050,0.065,0.838,29.1,fc45c5fff6d33f15d30e05dd02580fe0addd97098b735b2894015f9abdaa43a7,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.248,0.127,0.121,0.093,0.813,27.3,1e55e11f0e289a752098370e9f1d53fed6cd4fab7ceff46fe86dd20c927f9ffb,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.343,0.127,0.216,0.120,0.787,25.7,f5a2d2829e88a790cff5bedc50fca1f878a92cdb392d530afb49d0d95c23b884,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.143,0.127,0.016,0.042,0.877,31.5,5cfa466e276ab37cf532f7458d6dd86ea4f0807e2cbdb6788b7c01ca6ee35a1a,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.101,0.127,-0.026,0.047,0.864,30.9,ade869ec7c5b08098495d9b9d67ee900773f189f94bbb411b043f242f5c2fa37,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.208,0.127,0.081,0.060,0.839,29.3,a11976963e3f2ca856ce66278e22064d51345d449fa6d672029039a79517d60a,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.341,0.127,0.214,0.070,0.819,27.6,1ab1ae075565e9d39960a80cf5d61e5b0c50d7e23283493ea0277056b3e964ab,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.538,0.127,0.411,0.084,0.799,25.6,5ff5a364d84f738de4c56ead7d4950a2f9a1aaf4bec16e97bda6a65915744a57,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.08,false,0.074,0.037,0.037,0.068,0.865,30.8,81419eddc3e5150956859dd10c26dd841bb93d9c8b34c01f344e6d69bca6edbf,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.10,false,0.074,0.037,0.037,0.086,0.844,29.8,4d1f32bd0dac0e05885ff943e7c10230b06de7d8547e6215f8e2941f732a1df5,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.15,false,0.000,0.037,-0.037,0.135,0.793,27.6,526c7987b34896f9513f7439b63b8a87573b59d15930fa307f3bb924435f3cec,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.20,false,0.000,0.037,-0.037,0.179,0.746,25.9,325364cf7a3b5d39ac0c44af82181753ca4d6fde0f1b522c896414191725e8fc,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.25,false,0.000,0.037,-0.037,0.218,0.703,24.5,f01d64dc0cabd45d66a93c1be39b819b988f29cda1f0a7bc0d5a1954571d807e,true
|
||||
|
@@ -0,0 +1,7 @@
|
||||
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.338,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.305,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,"Paddle ch",0.074,0.289,0.696,23.3,,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,synthid_detected,inconclusive account: clean baseline not reproduced
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-restored,"Paddle ch",0.111,0.328,0.653,18.1,0.051,2ab7666a4161ff0e12c53d9412b23d013970c7613a8a55a1b1ae8a2744e3290d,synthid_detected,inconclusive account: qwen-zimage control also detected
|
||||
|
@@ -0,0 +1,6 @@
|
||||
date,upstream_commit,workflow_sha256,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle,implementation
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/before/12.png,official-before,,,,10,,,,,,,056276464387c883f5fd89bd2f1415faeb425abb3153cd10efb25d1b9053bdc2,synthid_detected,upstream-published-bytes
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/after/12.png,official-after,,,,10,0.975,0.093,0.354,0.172,0.820,23.8,a822d805666224adb3e360b3ee89e14f20a341f1e7c4d3a5d3d5251e9286aefe,clean,upstream-published-bytes
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected,project-runtime-control
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-global-close-reproduction,0,0.153999996,,18,0.589,0.217,0.331,0.286,0.647,24.3,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean,diffsynth-bf16-yunet-sam
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-full-close-reproduction,0,0.153999996,0.13209025065104166,18,0.783,0.083,0.470,0.260,0.659,24.7,2d7a8f76f98b905f14d69e5f9e11979610c72c6be7b6ac298dec4659f17bd819,clean,diffsynth-bf16-yunet-sam
|
||||
|
@@ -0,0 +1,3 @@
|
||||
date,source_url,source_sha256,source_width,source_height,source_fps,duration_seconds,source_verdict,vae,noise_std,long_side,fps,seed,output_sha256,output_verdict,output_verdict_text,output_detected_range,track,session_id,stratum,psnr_db,temporal_residual_ratio
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
|
||||
|
@@ -0,0 +1,15 @@
|
||||
# Provenance fixtures
|
||||
|
||||
These files exercise real container, metadata, and pixel-provenance formats.
|
||||
They are test inputs, not evaluation corpora.
|
||||
|
||||
## Adobe TrustMark
|
||||
|
||||
`adobe-trustmark-p.png` is Adobe's official `images/ghost_P.png` Variant P
|
||||
example from [`adobe/trustmark`](https://github.com/adobe/trustmark) at commit
|
||||
`0ed40cbe8188f664fd9cbbeacd969807de27440a`.
|
||||
|
||||
- Source: `https://raw.githubusercontent.com/adobe/trustmark/0ed40cbe8188f664fd9cbbeacd969807de27440a/images/ghost_P.png`
|
||||
- SHA-256: `e58c5825ed7e5d9fb04710ea541b61bd55879cad65554c0d46260aa24b3d0755`
|
||||
- Expected signal: TrustMark Variant P, schema 1
|
||||
- License: MIT, reproduced in `../licenses/adobe-trustmark-MIT.txt`
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 360 KiB |
@@ -0,0 +1,25 @@
|
||||
Copyright 2023 Adobe
|
||||
All Rights Reserved.
|
||||
|
||||
NOTICE: Adobe permits you to use, modify, and distribute this file in
|
||||
accordance with the terms of the license agreement accompanying it.
|
||||
|
||||
MIT License
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -56,7 +56,9 @@ Ground-truth quality, strongest first:
|
||||
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
|
||||
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
|
||||
- `synthid-portal` — checked via Google's SynthID Detector portal.
|
||||
- `c2pa-metadata` — issuer-only proxy (Google/OpenAI C2PA manifest present). Weaker: the C2PA can be stripped while the pixel watermark remains.
|
||||
- `c2pa-metadata` — supported provenance evidence (Google AI C2PA, or OpenAI
|
||||
C2PA with an explicit `c2pa.watermarked.*` action). Weaker than a provider
|
||||
oracle: C2PA can be stripped while the pixel watermark remains.
|
||||
- `third-party` — label asserted by an external dataset, not independently verified.
|
||||
- `none` — unverified.
|
||||
|
||||
|
||||
+296
-21
@@ -16,14 +16,17 @@ defaults. This page focuses on choosing the right command.
|
||||
| `metadata` and metadata-only `identify` | Default package |
|
||||
| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` |
|
||||
| Adobe TrustMark signals in `identify` | `remove-ai-watermarks[trustmark]` |
|
||||
| Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` |
|
||||
| `visible` and `erase` with OpenCV | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| `visible` or `erase` with MI-GAN | `remove-ai-watermarks[migan]` |
|
||||
| `visible` or `erase` with big-LaMa | `remove-ai-watermarks[lama]` |
|
||||
| `invisible` | `remove-ai-watermarks[diffusion]` |
|
||||
| `invisible --pipeline qwen-zimage` | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| `invisible` and `all` (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| `video metadata` and `video identify --no-visible` | Default package |
|
||||
| `video identify` | `remove-ai-watermarks[video]` |
|
||||
| `video visible`, `video all`, and visible/all batch modes | `remove-ai-watermarks[video]` plus ffmpeg on PATH |
|
||||
| `video invisible` and `video all --invisible` | `remove-ai-watermarks[video,diffusion]` plus ffmpeg on PATH |
|
||||
| HEIC/HEIF/AVIF pixel input | Add `remove-ai-watermarks[heif]` |
|
||||
| Every production command and backend | `remove-ai-watermarks[all]` |
|
||||
| Every production command and Python-compatible backend | `remove-ai-watermarks[all]` |
|
||||
|
||||
`batch` requires the same extra as its selected mode. Extras can be combined in
|
||||
one installation, for example `remove-ai-watermarks[visible,detect,heif]`.
|
||||
@@ -37,7 +40,13 @@ remove-ai-watermarks identify image.png
|
||||
`identify` always inspects supported metadata. When pixel extras are installed,
|
||||
it also evaluates supported visible and invisible pixel signals. When no signal
|
||||
is found, it reports the origin as unknown. It does not claim the image is
|
||||
clean.
|
||||
clean. For C2PA files, the text report shows asset integrity, claim-signature,
|
||||
signer-trust, and signer-validity results separately. An intact claim from an
|
||||
untrusted or expired signer is reported at medium confidence; a failed asset
|
||||
binding or signature does not confirm the claimed origin. When a structured
|
||||
C2PA soft binding is present, the report also names its exact algorithm and
|
||||
signed value; removing the manifest does not remove the referenced pixel
|
||||
watermark or content fingerprint.
|
||||
|
||||
Machine readable output:
|
||||
|
||||
@@ -65,7 +74,8 @@ remove-ai-watermarks visible image.png -o clean.png
|
||||
The default behavior:
|
||||
|
||||
- checks every registered visible mark;
|
||||
- removes every detected match;
|
||||
- removes every detected match, except the weakly detected Jimeng label pill,
|
||||
which needs corroboration (see [supported signals](supported-signals.md));
|
||||
- selects the best installed fill backend;
|
||||
- strips AI metadata from the output.
|
||||
|
||||
@@ -126,6 +136,13 @@ remove-ai-watermarks erase image.png \
|
||||
`erase` accepts `cv2`, `migan`, and `lama`. The corresponding optional extra
|
||||
must be installed for a learned backend.
|
||||
|
||||
Two more knobs tune the fill. `--dilate N` (default 3) grows every box by `N`
|
||||
pixels before inpainting, which helps when a mark has a soft edge or a drop
|
||||
shadow just outside the box you measured; it applies to every backend because it
|
||||
shapes the mask. `--inpaint-method telea|ns` selects the classical algorithm and
|
||||
only affects the `cv2` backend. Like `visible`, `erase` strips AI metadata from
|
||||
the output by default; pass `--keep-metadata` to retain it.
|
||||
|
||||
## Strip AI metadata
|
||||
|
||||
Inspect metadata:
|
||||
@@ -143,16 +160,216 @@ remove-ai-watermarks metadata image.png --remove -o clean.png
|
||||
When `-o` is omitted, removal overwrites the source. Standard metadata is kept
|
||||
unless you pass `--remove-all`.
|
||||
|
||||
A quiet `--check` or a successful `--remove` is not a clean verdict. The command
|
||||
only inspects and strips embedded AI metadata; a pixel watermark such as SynthID
|
||||
has no local decoder once that metadata proxy is gone. `identify` reports the
|
||||
same limit.
|
||||
|
||||
The command also supports the audio and video containers listed in
|
||||
[supported signals](supported-signals.md). ffmpeg must be available for the
|
||||
non-ISOBMFF audio and video path.
|
||||
|
||||
## Remove invisible watermarks
|
||||
## Identify and clean video
|
||||
|
||||
Install the diffusion dependencies first:
|
||||
Install the video pixel and timestamp runtime for visible identification,
|
||||
removal, and the complete pipeline:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[diffusion]"
|
||||
uv tool install --force "remove-ai-watermarks[video]"
|
||||
```
|
||||
|
||||
Inspect every locally supported video signal:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks video identify input.mp4
|
||||
remove-ai-watermarks video identify input.mp4 --json
|
||||
remove-ai-watermarks video identify input.mp4 --no-visible
|
||||
```
|
||||
|
||||
The default scans the complete clip for stable registered visible marks and
|
||||
inspects supported metadata. A result with no signals is reported as unknown,
|
||||
not clean, because proprietary pixel watermarks have no public local decoder.
|
||||
`--no-visible` performs metadata-only inspection.
|
||||
|
||||
Use the complete locally verifiable cleaning path:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks video all input.mp4 -o clean.mp4
|
||||
```
|
||||
|
||||
It removes a stable supported visible mark when found and always strips
|
||||
verified AI metadata. When neither signal is found, it writes a same-container
|
||||
passthrough instead of returning a missing output. The source is never
|
||||
overwritten.
|
||||
|
||||
Invisible regeneration is deliberately opt-in:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks video all input.mp4 -o clean.mp4 --invisible
|
||||
```
|
||||
|
||||
That option is supported only for MP4, MOV, and M4V. It is lossy and uses the
|
||||
same oracle-certified profile as `video invisible`.
|
||||
|
||||
Process all supported files in a top-level directory:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks video batch ./videos --mode all
|
||||
remove-ai-watermarks video batch ./videos --mode visible
|
||||
remove-ai-watermarks video batch ./videos --mode metadata
|
||||
```
|
||||
|
||||
The batch runs sequentially, preserves successful outputs when another file
|
||||
fails, and exits nonzero if any item failed. Visible no-op files are copied
|
||||
byte-for-byte so the output directory remains complete. `--invisible` is
|
||||
available only with `--mode all`.
|
||||
|
||||
## Strip AI metadata from video
|
||||
|
||||
Metadata inspection and removal are also available as an isolated operation:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks video metadata input.mp4 --check
|
||||
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
|
||||
```
|
||||
|
||||
As with the generic `metadata` command, a quiet check or a successful strip is
|
||||
not a clean verdict: video SynthID is not decoded locally after the metadata
|
||||
proxy is gone.
|
||||
|
||||
Supported containers are MP4, MOV, M4V, WebM, MKV, AVI, and FLV. The operation
|
||||
delegates to the same verified metadata scanner and stripper as the generic
|
||||
`metadata` command, so detection and removal stay in parity. Video and audio
|
||||
streams are not transcoded. For MP4 and MOV, this includes the native TC260
|
||||
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`, plus the
|
||||
QuickTime-form `meta` variants Doubao's iOS export writes (a bare `meta` box
|
||||
as a direct `moov` child, and a keyless `hdlr=mdir` metadata list). The inspector
|
||||
seeks past a large `mdat` to find a tail `moov`. Removal stream-copies the
|
||||
container in bounded chunks, converts supported top-level provenance boxes to
|
||||
same-size `free` boxes, and blanks the TC260 key/value in place. Box sizes,
|
||||
media offsets, and encoded stream bytes do not move; the result is atomically
|
||||
published only after the complete copy succeeds.
|
||||
|
||||
For MKV and WebM, the inspector reads the native TC260
|
||||
`Segment.Tags.Tag.SimpleTag` entry. Removal uses ffmpeg stream copying to
|
||||
discard container tags and chapters without transcoding the streams.
|
||||
AVI uses the normative `LIST/INFO/AIGC` chunk, while FLV uses the
|
||||
`script.onMetaData.AIGC` AMF0 string. Their bounded readers skip media payloads,
|
||||
and removal also uses ffmpeg stream copying.
|
||||
|
||||
When `-o` is omitted, the command writes `<source>_clean` with the same
|
||||
extension. It never overwrites the source, and it rejects an output with a
|
||||
different container extension.
|
||||
|
||||
Visible video labels and invisible video watermarks are not handled by this
|
||||
command.
|
||||
|
||||
## Remove video SynthID
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[video,diffusion]"
|
||||
remove-ai-watermarks video invisible input.mp4 -o clean.mp4
|
||||
```
|
||||
|
||||
The command supports MP4, MOV, and M4V. It samples the complete
|
||||
sequence at the configured frame rate, resizes frames to the configured long
|
||||
side, regenerates them through a VAE, and applies one deterministic latent-noise
|
||||
field to every frame. Reusing one spatial field avoids the unnecessary flicker
|
||||
caused by independent per-frame noise. Frames are regenerated in bounded
|
||||
batches and streamed directly to ffmpeg, which encodes the result, copies
|
||||
audio, and drops source metadata.
|
||||
|
||||
The default `noise_std=0.15` profile is oracle-certified. The project has no
|
||||
local video SynthID decoder, so an optional per-file recheck is still useful
|
||||
for unusually important files or after provider changes. In a new Gemini chat,
|
||||
upload the original first, invoke the built-in verifier with `@synthid`, and ask:
|
||||
|
||||
> For the video attached to this message, was it created or edited by Google
|
||||
> AI? Use the built-in SynthID content verification result.
|
||||
|
||||
The source must be positive. Then upload the processed result in a separate new
|
||||
chat and repeat the same built-in check. Only a source-positive, output-negative
|
||||
pair is a fresh per-file verification. Do not ask an adversarial follow-up that tells the
|
||||
chat model to ignore the verifier and reason about raw pixels: that is ordinary
|
||||
Gemini reasoning, not a second oracle check.
|
||||
|
||||
The default output is `<source>_clean` in the same container. The
|
||||
source is never overwritten. Use `--noise-std`, `--long-side`, `--fps`,
|
||||
`--batch-size`, `--seed`, and `--device` to control the regeneration. The
|
||||
default noise level is `0.15`. It cleared both carriers in the 2026-07-29
|
||||
short-clip calibration and the complete public eight-second Veo carrier in the
|
||||
2026-07-31 full-clip check; `0.10` remained detected on that complete clip. This
|
||||
calibration certifies the shipped operating point; the paired check above is an
|
||||
optional runtime audit, not a separate result state.
|
||||
|
||||
## Remove a supported visible video mark
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks video visible input.mp4 -o clean.mp4
|
||||
remove-ai-watermarks video visible veo.mp4 --mark veo -o veo_clean.mp4
|
||||
remove-ai-watermarks video visible seedance.mp4 --mark seedance -o seedance_clean.mp4
|
||||
remove-ai-watermarks video visible dola.mp4 --mark dola -o dola_clean.mp4
|
||||
remove-ai-watermarks video visible hailuo.mp4 --mark hailuo -o hailuo_clean.mp4
|
||||
remove-ai-watermarks video visible kling.mp4 --mark kling -o kling_clean.mp4
|
||||
```
|
||||
|
||||
The command supports the moving Sora mascot and wordmark, two Veo
|
||||
corner variants, the Seedance boxed `AI` label, the `Dola AI` text label, the
|
||||
composite `MINIMAX | hailuo AI` label, and the bottom-right Kling label. Sora
|
||||
searches the whole frame at multiple scales. The other detectors search bounded
|
||||
lower-frame regions with separate synthetic silhouettes. Kling additionally
|
||||
requires its bright low-saturation label near the frame edge. Every mark
|
||||
requires a spatially recurring candidate across adjacent frames. Fixed marks
|
||||
must also remain anchored instead of drifting with a scene object. Matching
|
||||
provider provenance may relax the visual score only for registered
|
||||
provenance-aware marks; metadata alone never creates a detection.
|
||||
|
||||
`--mark auto` is the default. It evaluates all providers in one decode pass and
|
||||
selects the first stable match in specificity order: Sora, Veo, Seedance, Dola,
|
||||
Hailuo, then Kling. Their confidence scores are independently calibrated and
|
||||
are not compared across providers. Pass an explicit `--mark` to scan only that
|
||||
provider.
|
||||
|
||||
The video stream is transcoded and the complete original audio stream is
|
||||
copied without truncating an audio tail that extends beyond the final video
|
||||
frame. The encoder probes the source stream and preserves supported 8-bit
|
||||
chroma sampling, color range/matrix/transfer/primaries tags, and MP4/MOV track
|
||||
timescale. For a variable-frame-rate source, decoded PTS are carried through a
|
||||
timestamped in-memory NUT bridge so the output retains the source frame
|
||||
intervals instead of flattening them to a constant rate. A non-zero source
|
||||
start PTS and the copied audio start offset are preserved as well.
|
||||
Supported input and output containers are MP4, MOV, M4V, WebM, MKV, AVI, and
|
||||
FLV; the output extension must match the input. The default `cv2` backend is
|
||||
fast but can smear structured backgrounds. Select `--backend migan` or
|
||||
`--backend lama` for a learned fill, or `--backend auto` to choose the best
|
||||
installed backend.
|
||||
|
||||
`--temporal-consistency` is enabled by default. It motion-aligns the preceding
|
||||
accepted fill, requires overlapping removal masks and matching source context,
|
||||
and blends only the safely covered pixels. Scene cuts, disjoint moving marks,
|
||||
or a poor motion match keep the independent current-frame fill. Use
|
||||
`--no-temporal-consistency` for an exact frame-local baseline.
|
||||
|
||||
The pixel path is intentionally limited to SDR 8-bit video. A high-bit-depth,
|
||||
PQ, or HLG source is rejected before ffmpeg starts, preserving any existing
|
||||
output instead of silently downconverting it through OpenCV's 8-bit boundary.
|
||||
On CPU, MI-GAN is the practical learned tier. LaMa remains an explicit offline
|
||||
quality option because full-sequence inference is too slow and memory-heavy
|
||||
for an online worker.
|
||||
|
||||
AI metadata is stripped from the encoded output by default. Use
|
||||
`--keep-metadata` to retain mapped container metadata. When no temporally stable
|
||||
mark is found, the command writes no output and exits with the no-visible-mark
|
||||
status. The final path is replaced atomically only after ffmpeg completes, so a
|
||||
failed encode does not overwrite an existing result.
|
||||
|
||||
## Remove invisible watermarks
|
||||
|
||||
Install the removal dependencies first. Both profiles are CUDA-only and both
|
||||
run the DiffSynth Z-Image face stage, so this is the extra either one needs:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
```
|
||||
|
||||
Then run:
|
||||
@@ -172,10 +389,15 @@ remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
|
||||
| Pipeline | When to use it |
|
||||
| --- | --- |
|
||||
| `controlnet` | Default compatibility profile with structural conditioning |
|
||||
| `sdxl` | Lighter plain SDXL regeneration |
|
||||
| `qwen` | Large CUDA oriented Qwen Image profile |
|
||||
| `qwen-zimage` | CUDA only high fidelity profile with a separate face stage |
|
||||
| `qwen-zimage` | Default. Qwen-Image-2512 global pass plus a SAM-masked Z-Image face stage |
|
||||
| `sdxl-zimage` | The same recipe and face stage on an SDXL global pass, at a higher denoise |
|
||||
|
||||
**Both are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
removal. The former `controlnet`, `sdxl`, `qwen` and `default` profiles were removed
|
||||
rather than kept as a CPU path: none of them matched this recipe's face preservation,
|
||||
so offering them implied a quality the library no longer delivers. Passing a retired
|
||||
name is rejected at parse time rather than remapped. Visible-mark removal and every
|
||||
identify path still run anywhere.
|
||||
|
||||
Example:
|
||||
|
||||
@@ -184,8 +406,55 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --force
|
||||
```
|
||||
|
||||
The legacy `default` value is an alias for `sdxl`. The `--auto` option is
|
||||
deprecated, emits a warning, and changes nothing.
|
||||
There is no `--model`, `--steps`, `--guidance-scale` or `--device` option, and the
|
||||
deprecated `--auto` is gone. Each profile pins its model stack, its per-stage
|
||||
schedule, CFG 1.0 and CUDA, so every one of those flags existed only to be refused
|
||||
several layers down. They are not parsed at all now, which fails at the point the
|
||||
user can act on rather than after a model load.
|
||||
|
||||
### Restore operator-verified text
|
||||
|
||||
`--text-manifest` enables the experimental `vae-glyphs` post-pass. It reconstructs
|
||||
the source with the Qwen VAE, blends 15% of that reconstruction into the normal
|
||||
`qwen-zimage` result, erases the annotated candidate glyphs with LaMa, and composites
|
||||
only the reconstructed glyph cores through source-derived silhouettes. It does not
|
||||
run OCR or choose which strings are correct.
|
||||
|
||||
Install the combined extra and run only with an operator-verified manifest:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
``verified: true`` may also be set by an automated operator that attests
|
||||
machine-verified geometry: stability-gated detector boxes inside sane caps. Such
|
||||
operators should use the geometry-only schema 2, which carries no transcription
|
||||
or script metadata.
|
||||
|
||||
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default: it
|
||||
was measured to return detector-visible OpenAI SynthID on poster-scale manifests
|
||||
(official Content Provenance API, 2026-08-19). `--fidelity-anchor` restores the
|
||||
0.27.0 research behavior; text-box fidelity lost by the default is well under one
|
||||
MAE point on the measured fixtures.
|
||||
|
||||
The manifest is a JSON object with `verified: true`, decoded RGB dimensions,
|
||||
`source_pixel_sha256`, and a non-empty `lines` array. Schema 1 is retained for
|
||||
manually reviewed annotations: each line has an integer `[x1, y1, x2, y2]` box,
|
||||
exact `text`, a non-empty `script`, and an optional angle from -30 to 30 degrees.
|
||||
Schema 2 is geometry-only: each line has the box and optional angle, with no
|
||||
required `text` or `script`. Lines must be in top-to-bottom, left-to-right order.
|
||||
The hash binds the annotations to decoded RGB geometry and pixels, so metadata-only
|
||||
container changes remain valid while a resized or edited source fails closed. The
|
||||
experimental helper
|
||||
`remove_ai_watermarks._internal.text_restoration.source_pixel_sha256` computes it.
|
||||
|
||||
This mode is supported only by `qwen-zimage` at native geometry with
|
||||
`humanize=0`, `unsharp=0`, and adaptive polish disabled. `all` also accepts the flag,
|
||||
but its manifest must match the pixels entering the invisible stage; if visible-mark
|
||||
removal changes those pixels, the hash check rejects the run. One oracle verdict does
|
||||
not certify another manifest, seed, model/runtime version, or output hash.
|
||||
|
||||
### Work with limited memory
|
||||
|
||||
@@ -218,8 +487,8 @@ It is a memory strategy, not a guarantee of better quality.
|
||||
|
||||
The `all` command and the `all` installation extra are separate concepts. The
|
||||
command runs every applicable stage. Installing `remove-ai-watermarks[all]`
|
||||
makes every production backend available; a smaller installation such as
|
||||
`remove-ai-watermarks[visible,diffusion]` can also run the command with fewer
|
||||
makes every backend compatible with the active Python available; a smaller installation such as
|
||||
`remove-ai-watermarks[visible,qwen-zimage]` can also run the command with fewer
|
||||
optional backends.
|
||||
|
||||
```bash
|
||||
@@ -234,10 +503,16 @@ The command runs:
|
||||
|
||||
The visible options and diffusion options are also available on `all`.
|
||||
|
||||
If diffusion is required but the `diffusion` extra is unavailable, `all` still
|
||||
writes the result of the visible and metadata stages, prints a prominent
|
||||
warning, and exits with code 1. This prevents a partial result from being
|
||||
reported as complete.
|
||||
When the `qwen-zimage` extra is unavailable, `all` still writes the result of
|
||||
the visible and metadata stages, prints a prominent warning, and exits with code
|
||||
1. That happens on every run without the extra: the skipped stage is what would
|
||||
have decided whether a signal was there. This prevents a partial result from
|
||||
being reported as complete.
|
||||
|
||||
If the extra is installed but the machine has no CUDA, which is the usual macOS
|
||||
case, the run fails at engine construction instead: `all` prints
|
||||
`Error: Invisible-watermark removal is CUDA-only ...`, writes no output at all,
|
||||
and exits with code 1.
|
||||
|
||||
## Process a directory
|
||||
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# Code provenance
|
||||
|
||||
This page records notices required by source dependencies and licensed derivative work.
|
||||
|
||||
## Licensed derivative work
|
||||
|
||||
- The DWT-DCT implementation derives from ShieldMnt's
|
||||
[`invisible-watermark`](https://github.com/ShieldMnt/invisible-watermark), licensed
|
||||
under MIT. Its notice ships in `src/remove_ai_watermarks/licenses/invisible-watermark-MIT.txt`.
|
||||
The decode path is a vectorized reformulation, not a transcription: it produces
|
||||
the same bits and is checked against upstream's own decoder, but it no longer
|
||||
corresponds line by line to `imwatermark/maxDct.py`. Diffing the two files will
|
||||
show structurally different code, which is expected and does not mean the
|
||||
derivation notice is stale.
|
||||
|
||||
## Licensed test fixtures
|
||||
|
||||
- `data/fixtures/provenance/adobe-trustmark-p.png` is Adobe's official
|
||||
TrustMark Variant P example, licensed under MIT. Its source commit, digest,
|
||||
and reproduced license are recorded beside the fixture in
|
||||
`data/fixtures/README.md` and `data/licenses/adobe-trustmark-MIT.txt`.
|
||||
@@ -3,7 +3,10 @@
|
||||
> Research archive. This document records experiments, superseded defaults, and
|
||||
> deployment considerations from the time of the study. It does not define the
|
||||
> current CLI or Python API. See `README.md`, `docs/cli.md`, and
|
||||
> `docs/known-limitations.md` for current behavior.
|
||||
> `docs/known-limitations.md` for current behavior. Everything it names on the
|
||||
> face-restoration path was removed after this study: the `--restore-faces` and
|
||||
> `--face-id` flags, the `instantid`, `photomaker` and `faceid` extras, and their
|
||||
> modules. So were the `controlnet`, `sdxl`, `qwen` and `default` profiles.
|
||||
|
||||
Date: 2026-06-02. Source: a manual primary-source pass (WebSearch + WebFetch over the
|
||||
watermark-removal-attack and SDXL-ControlNet literature). Prompted by issue #35
|
||||
@@ -55,8 +58,7 @@ content it protects. Controlnet is the text/structure PRESERVATION pipeline; rem
|
||||
is set by STRENGTH, separately calibrated, not by the pipeline choice.
|
||||
|
||||
This section is the single consolidated reference for the controlnet pipeline's
|
||||
removal behavior. (Mirrored briefly in `docs/synthid.md` §5.5 and the CLAUDE.md
|
||||
controlnet / face_restore bullets, which point here.)
|
||||
removal behavior. (Mirrored briefly in `docs/synthid.md` §5.5, which points here.)
|
||||
|
||||
### What we measured (real gpt-image + Gemini originals)
|
||||
|
||||
@@ -211,7 +213,7 @@ regeneration so strokes exceed the VAE's ~8 px latent floor), but ctrlregen runs
|
||||
at LOW res, the opposite. CtrlRegen's paper gives no resolution/tiling spec to contradict this.
|
||||
|
||||
**Sources.** the former internal
|
||||
`src/remove_ai_watermarks/noai/ctrlregen/engine.py` (removed after this study);
|
||||
`src/remove_ai_watermarks/_internal/ctrlregen/engine.py` (removed after this study);
|
||||
resolution-omission
|
||||
confirmed against https://arxiv.org/html/2410.05470v1
|
||||
|
||||
@@ -288,8 +290,8 @@ text-vs-scrub tension as an empirical question to measure, not assume.
|
||||
**Prototype (runs locally on 32 GB MPS — no dedicated GPU required):**
|
||||
|
||||
Compute is NOT the bottleneck. On a 32 GB Apple-silicon machine (M5 here) native SDXL already
|
||||
runs entirely on MPS with no CPU fallback (~155 s at 1122x1402, verified — see `synthid.md` /
|
||||
CLAUDE.md). The prototype runs at **1024** (fewer pixels than that) with SDXL base + an SDXL
|
||||
runs entirely on MPS with no CPU fallback (~155 s at 1122x1402, verified 2026-05-31).
|
||||
The prototype runs at **1024** (fewer pixels than that) with SDXL base + an SDXL
|
||||
ControlNet + activations in **fp32** (MPS fp16 decodes to all-black NaN — issue #29 — confirmed
|
||||
on run 1 below; fp32 is the required default on mps/cpu) — fits the 32 GB budget with vae-tiling +
|
||||
attention-slicing; ~1-2 min/image, so a coarse sweep is a sub-hour background run. A dedicated GPU
|
||||
@@ -534,7 +536,8 @@ https://instantid.github.io/ · https://arxiv.org/abs/2308.06721 (IP-Adapter)
|
||||
|
||||
### FaceID prototype run 1 -- 2026-06-03 (NEGATIVE on dense small-face groups)
|
||||
|
||||
Built and shipped the masked multi-face FaceID layer (`--face-id`, `face_id.py`, `faceid` extra).
|
||||
Built and shipped the masked multi-face FaceID layer (`--face-id`, `face_id.py`, `faceid` extra;
|
||||
all three were removed after this run and never reached the tracked tree).
|
||||
First real run on the gemini_3 group photo (Google, s015, scale 0.6, native 2816 via cap 1536):
|
||||
insightface detected **17 faces**, the masked multi-face pass composed and ran end-to-end (non-black
|
||||
output), so the API is correct. **At s015 the result is a clear FAILURE: every face corrupted --
|
||||
|
||||
+10
-1
@@ -12,9 +12,18 @@ Read this reference for environment setup, dependency recovery, CI behavior, and
|
||||
|
||||
The optional TrustMark decoder downloads weights into its installed package directory. After pruning that extra, a leftover weights directory can make availability checks see an empty namespace package. If Pyright reports an unknown `TrustMark` import and `find_spec("trustmark")` returns a loader-less spec, remove that regenerable remnant from the active virtual environment and resync.
|
||||
|
||||
### Known security-gate blocks
|
||||
|
||||
`maintain.sh` fails while PyPI ships no fixed release for a transitive CVE. Current case (triaged 2026-08-17): `lightning` 2.6.5, pulled only by the optional `trustmark` extra, carries PYSEC-2026-3624 (RCE via `load_from_checkpoint` on an attacker-crafted checkpoint). The vulnerable API is unreachable here, because the TrustMark decoder loads only its own pinned weights downloaded from the TrustMark release, never a user-supplied checkpoint. The upstream fix is merged but unreleased, and ignores are never added, so run and report the remaining core checks (Ruff, Pyright, tests) separately until a fixed `lightning` release lands; bump it with `uv lock --upgrade-package lightning` as soon as one does.
|
||||
|
||||
## CI
|
||||
|
||||
`.github/workflows/test.yml` runs Ruff and a cross-platform supported-Python test matrix with default plus development dependencies. Diffusion and model-running tests skip in that matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the OpenCV eraser remain covered across operating systems.
|
||||
`.github/workflows/test.yml` runs Ruff, a test matrix over every supported Python
|
||||
minor with default plus development dependencies, cross-platform coverage at the
|
||||
oldest and newest minors, and a separate job that installs ffmpeg on Ubuntu to
|
||||
run the full-clip video test. Diffusion and model-running tests skip in that
|
||||
matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the
|
||||
OpenCV eraser remain covered across operating systems.
|
||||
|
||||
Keep `uv.lock` compatible with `uv sync --frozen`. Dependency pull-request checks use GitHub's merge result against current `main`; if `main` moves, merge it locally and rerun the full gate because a newer linter can expose stale directives in later code.
|
||||
|
||||
|
||||
+5
-2
@@ -8,7 +8,7 @@ to run the tool. Use the maintainer references only when changing the code.
|
||||
| Page | Use it when |
|
||||
| --- | --- |
|
||||
| [Installation](installation.md) | You need the CLI, an optional model backend, or a development environment. |
|
||||
| [CLI guide](cli.md) | You want a command for one image, a directory, or a specific watermark type. |
|
||||
| [CLI guide](cli.md) | You want a command for an image, video metadata or visible marks, a directory, or a specific watermark type. |
|
||||
| [Python API](python-api.md) | You want to call the package from Python. |
|
||||
| [Supported signals](supported-signals.md) | You need to know which visible marks, metadata formats, and invisible signals are covered. |
|
||||
| [Known limitations](known-limitations.md) | You need the quality, device, format, or verification boundaries. |
|
||||
@@ -19,8 +19,10 @@ to run the tool. Use the maintainer references only when changing the code.
|
||||
| Page | Purpose |
|
||||
| --- | --- |
|
||||
| [Module internals](module-internals.md) | Current architecture, invariants, and regression guards by module. |
|
||||
| [Development](development.md) | Environment setup, dependency recovery, CI behavior, and fixture policy. |
|
||||
| [Code provenance](code-provenance.md) | Required notices for licensed derivative work. |
|
||||
| [Verification plan](verification-plan.md) | Verification methods, completed measurements, and remaining validation gaps. |
|
||||
| [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, and release workflow. |
|
||||
| [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, ComfyUI Registry, and the release workflow. |
|
||||
| [Watermarking landscape](watermarking-landscape.md) | Vendor signals and detection approaches. |
|
||||
| [SynthID technical reference](synthid.md) | Mechanism, detector access, robustness, and implications for this project. |
|
||||
|
||||
@@ -35,4 +37,5 @@ The current behavior is defined by the code, tests, README, and user guides.
|
||||
- [Doubao reverse-alpha research](research-doubao-distillation.md)
|
||||
- [SynthID identity research](synthid-robust-identity-research.md)
|
||||
- [SynthID identity follow-up](synthid-robust-identity-research-2026-06-08.md)
|
||||
- [Video SynthID quality research](video-synthid-quality-research.md)
|
||||
- [Text protection research](text-protection-research.md)
|
||||
+65
-22
@@ -1,6 +1,6 @@
|
||||
# Installation
|
||||
|
||||
Python 3.10.1 or newer is required.
|
||||
Python 3.11 through 3.14 are supported.
|
||||
|
||||
## Default metadata mode
|
||||
|
||||
@@ -46,24 +46,55 @@ Add `heif` only when the pixel path must decode HEIC, HEIF, or AVIF:
|
||||
uv tool install --force "remove-ai-watermarks[visible,heif]"
|
||||
```
|
||||
|
||||
## Invisible watermark removal
|
||||
## Video processing
|
||||
|
||||
Diffusion based removal needs the `diffusion` extra:
|
||||
Video metadata inspection works with the default package, and MP4 and MOV
|
||||
stripping uses the in-tree ISOBMFF box walker. Stripping the non-ISOBMFF
|
||||
containers (MKV, WebM, AVI, FLV, and the audio formats) and writing any cleaned
|
||||
video need ffmpeg on PATH, for example `brew install ffmpeg`. Stable
|
||||
visible-mark identification and removal, full video cleaning, and visible/all
|
||||
batch modes need the `video` extra:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[diffusion]"
|
||||
uv tool install --force "remove-ai-watermarks[video]"
|
||||
```
|
||||
|
||||
The code supports CUDA, XPU, MPS, and CPU devices. A GPU is recommended because
|
||||
CPU inference is slow.
|
||||
The extra includes the visible pixel runtime and PyAV for preserving variable
|
||||
frame timestamps. Video SynthID regeneration also needs the diffusion stack:
|
||||
|
||||
For the CUDA only Qwen Image plus Z-Image profile:
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[video,diffusion]"
|
||||
```
|
||||
|
||||
## Invisible watermark removal
|
||||
|
||||
Install the `qwen-zimage` extra:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
```
|
||||
|
||||
The `qwen-zimage` extra includes the normal `diffusion` dependencies.
|
||||
Both remaining profiles run a Z-Image face stage on the DiffSynth runtime, so
|
||||
both need this extra. It includes the `diffusion` dependencies; `diffusion` on its
|
||||
own covers the torch and diffusers imports but not the face stage, so it is not
|
||||
enough to run a removal.
|
||||
|
||||
**An NVIDIA GPU is required.** `qwen-zimage` and `sdxl-zimage` are CUDA-only, and
|
||||
construction refuses any other device rather than falling back to a slow or broken
|
||||
one. There is no CPU, MPS or XPU path for invisible-watermark removal. Visible-mark
|
||||
removal, metadata stripping and every `identify` command still run anywhere.
|
||||
|
||||
Video SynthID regeneration is a separate VAE path and does still run on CPU or MPS;
|
||||
it needs the `diffusion` extra, not this one.
|
||||
|
||||
The experimental verified-text post-pass additionally needs LaMa:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
```
|
||||
|
||||
That extra includes `qwen-zimage` and `lama`; it does not add OCR. Text strings and
|
||||
line boxes must be reviewed before the run.
|
||||
|
||||
## Feature extras
|
||||
|
||||
@@ -75,34 +106,39 @@ application actually uses:
|
||||
| `pixels` | Shared BGR array and image-processing runtime | NumPy, headless OpenCV | No |
|
||||
| `heif` | HEIC, HEIF, and AVIF pixel decoding | pillow-heif | No |
|
||||
| `visible` | Visible mark detection, OpenCV inpainting, and manual erasing | `pixels` | No |
|
||||
| `video` | Visible video identification/removal and timestamp preservation | `visible`, PyAV | No |
|
||||
| `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No |
|
||||
| `trustmark` | Adobe TrustMark detection | trustmark | Yes |
|
||||
| `diffusion` | Diffusion-based invisible watermark removal | `pixels`, Torch, Diffusers | Yes |
|
||||
| `trustmark` | Adobe TrustMark detection on Python 3.11-3.12 | trustmark | Yes |
|
||||
| `diffusion` | Torch and Diffusers runtime; video SynthID regeneration | `pixels`, Torch, Diffusers | Yes |
|
||||
| `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `esrgan` | Real-ESRGAN upscaling before diffusion | `pixels`, spandrel | Yes |
|
||||
| `qwen-zimage` | CUDA-only Qwen Image plus Z-Image pipeline | `diffusion`, DiffSynth | Yes |
|
||||
| `all` | Every production feature | All rows above | Yes |
|
||||
| `dev` | Tests, linting, typing, and upstream parity checks | `visible`, `detect`, upstream invisible-watermark | Yes, for parity tests |
|
||||
| `qwen-zimage` | Invisible image-watermark removal, both CUDA-only profiles | `diffusion`, DiffSynth | Yes |
|
||||
| `text-restoration` | Opt-in verified Qwen-VAE glyph restoration | `qwen-zimage`, `lama` | Yes |
|
||||
| `all` | Every production feature available on the active Python | All compatible rows above | Yes |
|
||||
| `dev` | Tests, linting, typing, and upstream parity checks | `video`, `detect`, upstream invisible-watermark | Yes, for parity tests |
|
||||
|
||||
Dependency composition:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
visible --> pixels
|
||||
video --> visible
|
||||
detect --> pixels
|
||||
diffusion --> pixels
|
||||
migan --> visible
|
||||
lama --> visible
|
||||
esrgan --> pixels
|
||||
qwen["qwen-zimage"] --> diffusion
|
||||
text["text-restoration"] --> qwen
|
||||
text --> lama
|
||||
heif
|
||||
trustmark
|
||||
```
|
||||
|
||||
`heif` and `trustmark` are independent branches. Combine them explicitly with
|
||||
another feature when required. The `all` bundle contains every production
|
||||
branch but never includes `dev`.
|
||||
another feature when required. TrustMark requires NumPy 1.x, which has no
|
||||
CPython 3.13 or 3.14 wheels, so that branch is available only on Python
|
||||
3.11-3.12. The `all` bundle contains every production branch compatible with
|
||||
the active Python and never includes `dev`.
|
||||
|
||||
Examples:
|
||||
|
||||
@@ -113,10 +149,13 @@ uv tool install --force "remove-ai-watermarks[detect]"
|
||||
# Visible removal with HEIC/AVIF support and MI-GAN
|
||||
uv tool install --force "remove-ai-watermarks[migan,heif]"
|
||||
|
||||
# Visible video removal with preserved timestamps
|
||||
uv tool install --force "remove-ai-watermarks[video]"
|
||||
|
||||
# DWT-DCT and TrustMark detection without diffusion removal
|
||||
uv tool install --force "remove-ai-watermarks[detect,trustmark]"
|
||||
|
||||
# Every production capability
|
||||
# Every production capability compatible with this Python
|
||||
uv tool install --force "remove-ai-watermarks[all]"
|
||||
|
||||
# An arbitrary minimal combination
|
||||
@@ -166,8 +205,10 @@ Run the complete project gate:
|
||||
bash maintain.sh
|
||||
```
|
||||
|
||||
The script runs dependency checks, linting, formatting checks, type checking,
|
||||
and the test suite.
|
||||
The script syncs every optional backend on top of the `dev` environment above,
|
||||
then runs dependency checks, linting, formatting, type checking, and the test
|
||||
suite. It applies Ruff fixes and formatting in place rather than only reporting
|
||||
them.
|
||||
|
||||
## Hugging Face authentication
|
||||
|
||||
@@ -194,5 +235,7 @@ The normal behavior is to skip diffusion when no supported local signal is
|
||||
found. A missing signal does not prove that the image is clean. If you know the
|
||||
image came from a relevant generator, use `--force`.
|
||||
|
||||
If the CLI reports that diffusion dependencies are unavailable, install the
|
||||
`gpu` extra.
|
||||
If the CLI reports that the removal dependencies are unavailable, install the
|
||||
`qwen-zimage` extra. `diffusion` alone covers Torch and Diffusers but not the
|
||||
DiffSynth face stage that both profiles run. Video SynthID removal is a separate
|
||||
path and needs `video` and `diffusion`.
|
||||
+256
-44
@@ -57,18 +57,49 @@ Invisible removal does not decode and delete a payload. It regenerates the
|
||||
image through a diffusion pipeline. Faces, text, colors, and fine detail can
|
||||
change even when the watermark is successfully disrupted.
|
||||
|
||||
ControlNet is the default compatibility profile. It conditions on edges to
|
||||
preserve structure, but edges do not preserve identity or exact texture.
|
||||
Text, tables, and UI screenshots are the worst case for a forced scrub: the
|
||||
model redraws glyphs as plausible-but-wrong shapes. Do not run the invisible
|
||||
stage on such content unless a locally-detectable invisible signal exists.
|
||||
The default `all`, `invisible`, and `batch` commands already skip it then, and
|
||||
`visible` (which strips AI metadata by default) or `metadata` alone never
|
||||
redraws a glyph outside the filled watermark box. `--force` is for content you
|
||||
know carries a pixel watermark despite no local signal; on a clean screenshot
|
||||
it only buys distortion. Preserving or pasting back the original text pixels
|
||||
during a real scrub is deliberately not offered: an invisible watermark such
|
||||
as SynthID lives in the pixels everywhere, including inside the glyphs, so
|
||||
frozen text regions would keep the watermark
|
||||
([text protection research](text-protection-research.md)).
|
||||
|
||||
The CUDA only `qwen-zimage` profile adds a separate face stage and is the
|
||||
highest fidelity option in the current implementation. It is larger, slower,
|
||||
and still may alter small text or difficult faces.
|
||||
`qwen-zimage` is the default profile and `sdxl-zimage` the only alternative.
|
||||
Both are CUDA only and differ only in the global regeneration model: each
|
||||
conditions that stage on a canny edge map, which preserves structure but not
|
||||
identity or exact texture, and each then runs the same face stage.
|
||||
Existing face evaluations favor `qwen-zimage`, but there is no blanket fidelity
|
||||
ordering across content types. A fixed-seed, three-scene text comparison at the
|
||||
profile defaults found no stable winner: SDXL won one poster, Qwen won one, and
|
||||
the Chinese sign tied. Both are large, slow, and may still alter small text or
|
||||
difficult faces. The measurements and their OCR and oracle caveats are tracked
|
||||
in [`data/evaluations/fidelity/`](../data/evaluations/fidelity/README.md).
|
||||
A global Z-Image Turbo prototype preserved text substantially better at low
|
||||
strength, but it has no useful cross-provider operating point and is not a
|
||||
supported profile. Automatic text restorers also remain research-only:
|
||||
fresh-font and silhouette variants visibly changed typography. The higher-fidelity
|
||||
`vae-glyphs` route is available only as an experimental opt-in with verified strings
|
||||
and line geometry. It builds its donor internally but still requires an independently
|
||||
clean global anchor. Automatic OCR and line-box proposals are not reliable enough to
|
||||
remove those requirements, and exact oracle results do not establish a general mask,
|
||||
seed, runtime, or provider operating range. Qwen-Image-2.0 is hosted-only and exposes
|
||||
no equivalent low-strength denoise control. Exact experiments, controls, and pass
|
||||
rates are kept in
|
||||
[`text-protection-research.md`](text-protection-research.md) and the
|
||||
[`fidelity` evaluation record](../data/evaluations/fidelity/README.md).
|
||||
|
||||
### Removal cannot be verified locally for proprietary SynthID
|
||||
|
||||
The project has no public local SynthID pixel decoder. It can infer likely
|
||||
presence from supported provenance metadata, but a missing metadata proxy is
|
||||
not a negative pixel verdict.
|
||||
The project has no public local SynthID pixel decoder. It recognizes presence
|
||||
from supported provenance metadata: Google AI C2PA under Google's all-media
|
||||
policy, or current OpenAI C2PA with an explicit watermark action. A missing
|
||||
provenance signal is not a negative pixel verdict.
|
||||
|
||||
For important outputs:
|
||||
|
||||
@@ -80,45 +111,129 @@ For important outputs:
|
||||
Provider systems can change, so a result verified on one file, seed, or version
|
||||
is not a permanent certification.
|
||||
|
||||
### Video SynthID removal is lossy and content-dependent
|
||||
|
||||
The `video invisible` command and `remove_video_invisible` API regenerate video
|
||||
pixels through a VAE. The shipped `noise_std=0.15` profile is oracle-certified,
|
||||
but Google does not publish a local decoder for arbitrary runtime outputs. A
|
||||
quiet metadata scan, paired PSNR, and the temporal-residual metric are fidelity
|
||||
measurements, not independent SynthID verdicts.
|
||||
|
||||
The control must use the same clip, frame rate, dimensions, and final codec as
|
||||
the candidates. The separate `scripts/video_synthid_sweep.py` harness produces
|
||||
that matched control. If the control is not detected by the matching provider
|
||||
oracle, the experiment cannot attribute a quiet candidate to regeneration.
|
||||
|
||||
The 2026-07-29 two-clip calibration used Gemini's built-in content verifier:
|
||||
both matched controls were SynthID-positive, the stronger candidate was
|
||||
negative on both carriers, and a weaker candidate was negative on one. A
|
||||
2026-07-30 adversarial follow-up incorrectly asked the ordinary chat model to
|
||||
reinterpret the verifier while excluding every other input; its `UNAVAILABLE`
|
||||
answer was not another detector run and does not invalidate the original
|
||||
built-in results. The calibrated default remains content-dependent; a fresh
|
||||
source-positive, output-negative pair is an optional audit for unusually
|
||||
important files or after provider changes.
|
||||
|
||||
The 2026-07-31 full-clip check used the public eight-second Veo off-road sample.
|
||||
The source was detected across the full clip, the complete product path at
|
||||
`noise_std=0.10` remained detected, and `0.15` returned no SynthID detection.
|
||||
The default was raised to `0.15`. At 512 px / 12 fps, the accepted candidate
|
||||
measured 25.39 dB paired PSNR and a 1.058 motion-compensated temporal-residual
|
||||
ratio. This is one carrier, not a universal guarantee; hashes and exact verdicts
|
||||
are tracked in `data/evaluations/video-synthid-oracle.csv`.
|
||||
|
||||
### The invisible video path does not touch the audio track
|
||||
|
||||
`remove_video_invisible` regenerates the video stream and copies the source audio
|
||||
verbatim: the extracted audio bitstream of input and output has an identical
|
||||
sha256, measured on two carriers. It also strips every metadata marker, so the
|
||||
output can report clean from `get_ai_metadata` and `identify_video` while carrying
|
||||
an untouched generated audio track. Google's verifier scores audio and visual
|
||||
tracks separately, and a track this path never modifies is a track it cannot have
|
||||
cleaned.
|
||||
|
||||
Whether a given carrier's audio actually holds a mark the verifier reads has not
|
||||
been established -- that needs a provider verdict, which has not been obtained.
|
||||
Treat a clean local report on a clip with generated audio as unproven, not as a
|
||||
guarantee, and check the audio separately when it matters.
|
||||
|
||||
The shipped engine streams sampled frames in bounded batches, computes its
|
||||
fidelity metrics incrementally, and pipes regenerated pixels directly to
|
||||
ffmpeg. Its frame and latent memory is therefore bounded by `--batch-size`
|
||||
rather than clip duration. Runtime still grows linearly with duration, and the
|
||||
separate multi-candidate research sweep deliberately retains its short sampled
|
||||
prefix so it can reuse identical latents across candidate strengths.
|
||||
|
||||
The reported `psnr_db` is measured against the already-resized frame and before
|
||||
the H.264 encode, so it excludes the downscale, the frame decimation, and the
|
||||
encoder. It measures the VAE round trip plus latent noise at the working geometry.
|
||||
[`video-synthid-quality-research.md`](video-synthid-quality-research.md) records
|
||||
what the manifest rows constrain, what a higher-resolution or higher-frame-rate
|
||||
profile would need in order to be certified, and the audio-track question this
|
||||
path has not yet answered.
|
||||
|
||||
### Strength is content and seed dependent
|
||||
|
||||
For SDXL and ControlNet, the CLI resolves an unset strength from the detected
|
||||
vendor:
|
||||
The two profiles resolve an unset strength differently, because different things
|
||||
were measured for each.
|
||||
|
||||
- OpenAI: `0.10`;
|
||||
- Google: `0.15`;
|
||||
- unknown: `0.15`.
|
||||
`qwen-zimage` reads it from image area, through the resolution-adaptive denoise
|
||||
curve. The vendor is deliberately ignored for openai/unknown content: the curve,
|
||||
not the issuer, is what was calibrated. One measured exception (0.27.2):
|
||||
Google-provenance content takes the flat `QWEN_ZIMAGE_GOOGLE_STRENGTH` 0.30 floor
|
||||
- the curve's 0.154 top left a 4.33 MP fixture oracle-detected on the full
|
||||
production path (2026-08-18), while 0.30 anchors measured clean on two fixtures
|
||||
in two accounts.
|
||||
|
||||
An explicit `--strength` overrides these defaults. The defaults are operating
|
||||
points, not universal guarantees. Near a removal threshold, different content
|
||||
or a different random seed may change the verifier result.
|
||||
`sdxl-zimage` reads it from the C2PA issuer, on a flat ladder:
|
||||
|
||||
The base Qwen and `qwen-zimage` profiles have profile specific strength
|
||||
behavior. Consult `remove-ai-watermarks invisible --help` and the source of
|
||||
[`watermark_profiles.py`](../src/remove_ai_watermarks/noai/watermark_profiles.py)
|
||||
for the current resolver.
|
||||
- OpenAI: `0.15`;
|
||||
- Google: `0.25`;
|
||||
- unknown: `0.25`, following the stricter of the two.
|
||||
|
||||
An SDXL global pass needs more denoise than Qwen at the same fidelity, and the
|
||||
values are flat rather than a curve because flat values are what was measured: each
|
||||
verdict came from a fixed strength at one size, and no size dependence has been
|
||||
established for that stage.
|
||||
|
||||
An explicit `--strength` overrides both. The defaults are operating points, not
|
||||
universal guarantees. Near a removal threshold, different content or a different
|
||||
random seed may change the verifier result, which is why both profiles are
|
||||
certified at a fixed seed. The live resolver is
|
||||
[`watermark_profiles.py`](../src/remove_ai_watermarks/_internal/watermark_profiles.py).
|
||||
|
||||
### Pipelines have different quality tradeoffs
|
||||
|
||||
| Pipeline | Main limit |
|
||||
| --- | --- |
|
||||
| `controlnet` | Edge conditioning can preserve a watermark carrying region too closely, and faces may drift. |
|
||||
| `sdxl` | Flat graphics and precise structure may receive too little or unhelpful change. |
|
||||
| `qwen` | Large CUDA oriented model; face smoothing can still be significant. |
|
||||
| `qwen-zimage` | CUDA only, large model stack, and limited broad certification across seeds and content. |
|
||||
| `sdxl-zimage` | CUDA only. Its strength ladder is flat per vendor, not a resolution curve, because flat values are what was measured. |
|
||||
|
||||
The legacy `default` profile name maps to `sdxl`. The `--auto` flag is
|
||||
deprecated, emits a warning, and changes nothing.
|
||||
Only manually verified `vae-glyphs` is an optional production stage, and it is
|
||||
experimental rather than a default.
|
||||
OCR plus LaMa recovered literal poster text but changed fonts and worsened whole-image
|
||||
fidelity. Restricting it to OCR-mismatched lines improved the tradeoff but still
|
||||
left a local shadow on one poster. The published AnyText2 SD1.5 checkpoint
|
||||
substituted Chinese characters and increased CER on the sign fixture. AnyText2XL
|
||||
is not published, and the released wrapper truncates individual text lines after
|
||||
20 characters.
|
||||
|
||||
The `controlnet`, `sdxl`, `qwen` and `default` profiles were removed, not aliased
|
||||
onward: a retired name is rejected at parse time rather than routed into a profile
|
||||
the caller never chose. There is no `--model`, `--steps`, `--guidance-scale`,
|
||||
`--device` or `--auto` option either; each profile pins its model stack, its
|
||||
per-stage schedule, CFG 1.0 and CUDA.
|
||||
|
||||
## Resolution and memory
|
||||
|
||||
### Small images are enlarged before SDXL based diffusion
|
||||
### Small images are processed at their native size
|
||||
|
||||
The SDXL, ControlNet, and base Qwen paths use a default minimum long side of
|
||||
`1024`. Smaller inputs are enlarged before diffusion and restored to their
|
||||
original dimensions afterward. Set `--min-resolution 0` to disable the floor.
|
||||
There is no minimum-resolution floor. It existed to enlarge small inputs toward
|
||||
SDXL's ~1024 training resolution and was removed with the SDXL profiles, which
|
||||
never applied it anyway. Both surviving profiles run at native geometry, so a
|
||||
small input is neither enlarged before diffusion nor restored afterward.
|
||||
|
||||
`qwen-zimage` does not apply this SDXL minimum resolution floor.
|
||||
`--max-resolution` still caps very large inputs, and only ever scales down.
|
||||
|
||||
### Large images stay at native resolution unless capped
|
||||
|
||||
@@ -131,23 +246,25 @@ overlapping tiles. It avoids the explicit downscale, but it is not pixel
|
||||
lossless: each tile is independently regenerated. With `qwen-zimage`, only the
|
||||
global Qwen stage is tiled; the face stage runs after tile blending.
|
||||
|
||||
### CPU offload is CUDA only
|
||||
### CPU offload trades speed for VRAM
|
||||
|
||||
`--cpu-offload` moves Diffusers model components between CPU and CUDA instead of
|
||||
keeping the complete standard pipeline in GPU memory. For `qwen-zimage`, it
|
||||
forces the face stack to use its offload path.
|
||||
`--cpu-offload` forces both stacks of the two-stage profile out of automatic
|
||||
device residency, streaming weights instead of pinning them. It reduces CUDA
|
||||
memory pressure at the cost of speed.
|
||||
|
||||
The option reduces CUDA memory pressure at the cost of speed. It has no effect
|
||||
on CPU or MPS and fails loudly when a CUDA Diffusers pipeline does not expose
|
||||
the required offload method.
|
||||
Residency is otherwise chosen from the card's total VRAM. On a card large enough
|
||||
to hold a stack, offloading is pure waste: DiffSynth drops weights to the meta
|
||||
device and re-reads every parameter from disk on each stage transition.
|
||||
|
||||
### MPS may fall back to CPU
|
||||
### There is no CPU or MPS fallback
|
||||
|
||||
The SDXL paths include an MPS out-of-memory fallback that reloads on CPU. A run
|
||||
that appears much slower after an MPS failure may be continuing on CPU.
|
||||
Invisible-watermark removal refuses any device but CUDA at construction. There is
|
||||
no MPS out-of-memory fallback that continues on CPU, and no lighter profile to
|
||||
drop to: both remaining profiles need an NVIDIA GPU.
|
||||
|
||||
Memory needs depend on the pipeline, input size, dtype, and machine. Use tiling,
|
||||
a resolution cap, or a lighter pipeline when necessary.
|
||||
When a run does not fit, the levers are `--tile` (native geometry, tiled
|
||||
diffusion), `--max-resolution` (an explicit downscale), and `--cpu-offload`.
|
||||
Memory needs depend on the profile, input size, dtype, and card.
|
||||
|
||||
## Metadata and formats
|
||||
|
||||
@@ -177,6 +294,92 @@ WebM, Matroska, MP3, WAV, FLAC, OGG, Opus, and AAC container metadata is strippe
|
||||
through ffmpeg with stream copying. The operation fails if ffmpeg is absent or
|
||||
cannot parse the input.
|
||||
|
||||
### Video pixel removal is provider-specific
|
||||
|
||||
The `video metadata` command and high level video API inspect and strip
|
||||
supported AI provenance metadata without transcoding streams.
|
||||
|
||||
`video visible` and `remove_video_visible` additionally support the moving
|
||||
Sora 2 mascot and wordmark, the current Veo four-point diamond, the legacy
|
||||
`Veo` text, the Seedance boxed `AI` label, the fixed `Dola AI` text, the Hailuo
|
||||
MINIMAX/Hailuo composite label, and the bottom-right Kling label with its
|
||||
version suffix. Detection requires a recurring visual candidate across
|
||||
adjacent frames. Fixed-mark candidates must remain anchored rather than
|
||||
drifting with a scene object. Kling also requires a bright low-saturation
|
||||
candidate near the expected frame edge. Provider provenance can recover
|
||||
low-contrast runs only after visual evidence exists for the marks that define a
|
||||
provenance prior, so metadata alone does not erase a clean API export.
|
||||
The default auto-router evaluates all detectors in one decode pass but does not
|
||||
rank their raw confidence values. Those scores are provider-specific and known
|
||||
to cross-match in some layouts, so the router applies the independent temporal
|
||||
policies and selects the first stable result in specificity order. Use an
|
||||
explicit mark when the provider is already known.
|
||||
Historical Sora Turbo exports use a small OpenAI swirl in the corner rather
|
||||
than the moving mascot-and-wordmark design; that earlier variant is not
|
||||
detected by the `sora` video mark. Hailuo and Kling coverage is specific to the
|
||||
verified lower-edge layouts; a new provider layout needs a separate calibrated
|
||||
silhouette. Other provider video labels are not supported yet. Google video
|
||||
SynthID has an oracle-certified VAE removal path, while other proprietary
|
||||
invisible video watermarks have no registered attack.
|
||||
|
||||
Visible removal transcodes the video stream and copies the complete audio
|
||||
stream without shortening an audio tail. Completed visible and invisible
|
||||
encodes are published atomically, so an encode failure preserves an existing
|
||||
output. Visible removal now applies a guarded motion-compensated blend after
|
||||
the per-frame fill. It uses adjacent optical flow only when the warped prior
|
||||
mask covers the current mask and a source-context ring agrees; scene cuts and
|
||||
disjoint marks keep the independent fill. This reduces measured paired
|
||||
temporal error, but it is not a generative video-inpainting model and cannot
|
||||
recover structure that no frame exposes. OpenCV can still leave a visible
|
||||
smear where the mark overlaps a hard edge or structured texture. MI-GAN
|
||||
improves difficult individual frames and is the practical learned CPU tier.
|
||||
LaMa remains an offline quality option: a full real sequence confirmed its
|
||||
multi-GB memory use and CPU throughput unsuitable for an online worker. The Veo diamond
|
||||
uses a shape mask to limit damage outside the symbol. Seedance fills the full
|
||||
localized box because a synthetic outline mask left part of the real
|
||||
translucent border visible in an end-to-end check. OpenCV may therefore soften
|
||||
texture inside that small box; use MI-GAN or LaMa when reconstruction quality
|
||||
matters. Relative variable-frame timestamps are preserved through a timestamped
|
||||
NUT bridge. A non-zero absolute video start PTS and the corresponding copied
|
||||
audio offset are preserved through ffmpeg timestamp passthrough.
|
||||
The encoder is source-aware for common 8-bit inputs: it probes and preserves
|
||||
supported chroma sampling, recognized color metadata, encoder time base, and
|
||||
MP4/MOV track timescale instead of accepting ffmpeg's implicit `yuv444p`
|
||||
raw-BGR output. OpenCV still decodes through 8-bit BGR, so HDR/high-bit-depth
|
||||
inputs are rejected before encoding rather than silently falling back to
|
||||
`yuv420p`. The synthetic
|
||||
Sora/OpenCV full-clip CI gate covers complete removal, untouched-region PSNR,
|
||||
frame count, frame rate, duration, copied-audio identity, source stream
|
||||
properties, paired temporal deltas against an independently encoded
|
||||
frame-local baseline inside the filled region, and metadata
|
||||
stripping through real ffmpeg. A second synthetic VFR clip verifies all display
|
||||
timestamps to the source time-base tick, including a non-zero source start,
|
||||
and checks both video and audio stream offsets. A separate constant-rate case
|
||||
guards the non-zero-start routing independently of VFR detection. A local
|
||||
full-sequence audit over all six providers passed both OpenCV and MI-GAN for
|
||||
complete-frame removal, quiet second detection, stream starts, duration, and
|
||||
copied audio. A full LaMa sequence passed the same checks but established that
|
||||
the backend belongs in the offline tier on CPU. These bounded local checks are
|
||||
still not universal evidence for every provider layout or source.
|
||||
|
||||
Native TC260 metadata in MP4/MOV is supported at its normative
|
||||
`moov.udta.meta.keys/ilst` placement, including non-faststart files whose
|
||||
`moov` follows a large media payload. MKV/WebM is supported at the normative
|
||||
`Segment.Tags.Tag.SimpleTag` placement and uses ffmpeg for stream-copy removal.
|
||||
AVI is supported at `LIST/INFO/AIGC`, and FLV at
|
||||
`script.onMetaData.AIGC`; both use ffmpeg stream-copy removal. Stock ffmpeg can
|
||||
write and strip the FLV form, but writing the nonstandard AVI child for fixture
|
||||
generation requires dedicated muxer support, so the AVI reader is verified
|
||||
against an exact synthetic RIFF structure.
|
||||
|
||||
MP4/MOV/M4V metadata removal now stream-copies the container in bounded chunks,
|
||||
keeps every box size and media offset fixed, and publishes atomically. The
|
||||
large-`mdat` regression rejects a full-source `read_bytes()` call and verifies
|
||||
that the encoded payload is byte-identical. HEIF/AVIF/JPEG-XL image metadata
|
||||
still uses the in-memory path because their XMP/EXIF items may live inside
|
||||
`mdat`/`idat` and require bounded format-item parsing before that path can
|
||||
stream safely.
|
||||
|
||||
### Metadata transformation is fail safe
|
||||
|
||||
`remove_ai_metadata` may copy an undecodable file through unchanged instead of
|
||||
@@ -199,8 +402,17 @@ Diffusion, SDXL, and FLUX workflows. That decoder is sensitive to the carrier
|
||||
and transformations. A negative result is not a universal negative.
|
||||
|
||||
The `trustmark` extra adds Adobe TrustMark decoding. The implementation retains
|
||||
an additional JPEG re-encode gate because isolated decoder hits can otherwise
|
||||
be content noise.
|
||||
an additional JPEG re-encode gate and requires the binary payload and schema to
|
||||
remain identical because isolated decoder hits can otherwise be content noise.
|
||||
It accepts Variant P schemas 0-2. Variant Q requires a different model, and
|
||||
schema 3 is rejected at the measured precision threshold. Its NumPy 1.x runtime
|
||||
limits the extra to Python 3.11-3.12; the rest of the package remains supported
|
||||
through Python 3.14. Through 0.29.0 the extra also resolves lightning 2.6.5
|
||||
(PYSEC-2026-3624, no fixed release yet); the vulnerable `load_from_checkpoint`
|
||||
path is unreachable here because TrustMark loads its checksummed checkpoints
|
||||
with plain `torch.load`. Bump lightning and cut a patch release when a fix
|
||||
ships. The calibration history is in
|
||||
[module internals](module-internals.md#metadata-and-provenance).
|
||||
|
||||
External AI versus real image classifiers are out of scope. The project
|
||||
identifies concrete local provenance signals instead of shipping a generic
|
||||
|
||||
+866
-58
File diff suppressed because it is too large.
Load diff
+451
-24
@@ -6,9 +6,11 @@ and pipeline modules are intended for maintainers and specialized workflows.
|
||||
Dependency groups are identical for the CLI and Python API. The default install
|
||||
covers metadata extraction, normalization, verdict logic, and stripping.
|
||||
Array/pixel APIs use `pixels`; visible removal uses `visible`; DWT-DCT detection
|
||||
uses `detect`; and diffusion removal uses `diffusion`. Add `heif` independently
|
||||
when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See the complete
|
||||
[feature-extra matrix](installation.md#feature-extras).
|
||||
uses `detect`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and
|
||||
visible video processing uses `video`. Video SynthID removal is a separate VAE
|
||||
path that still runs on CPU and combines `video` and `diffusion`. Add `heif`
|
||||
independently when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See
|
||||
the complete [feature-extra matrix](installation.md#feature-extras).
|
||||
|
||||
## Remove visible marks
|
||||
|
||||
@@ -75,6 +77,61 @@ image = cv2.imread("input.png")
|
||||
result, removed = raiw.remove_visible(image, backend="cv2")
|
||||
```
|
||||
|
||||
## Run the full pipeline
|
||||
|
||||
`remove_all` is the library form of the `all` command: visible marks, then the
|
||||
invisible watermark, then AI metadata. Stages are chained through a file in the
|
||||
system temp directory, so a partial result never appears at the output path.
|
||||
|
||||
```python
|
||||
import remove_ai_watermarks as raiw
|
||||
|
||||
result = raiw.remove_all("input.png", "clean.png") # -> RemoveAllResult
|
||||
print(result.output) # the path written
|
||||
print(result.visible_label) # the marks removed, or None
|
||||
print(result.invisible) # "removed" | "no-signal" | "unavailable"
|
||||
```
|
||||
|
||||
`invisible` is the field to check. `"unavailable"` means the GPU extra is not
|
||||
installed, so the output *looks* processed but still carries the watermark;
|
||||
`"no-signal"` means the scrub was deliberately skipped because nothing was
|
||||
locally detectable, which is a successful run.
|
||||
|
||||
Pass `InvisibleOptions` to tune the diffusion stage, and `engine` to reuse one
|
||||
loaded model across many calls:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks import InvisibleOptions
|
||||
|
||||
raiw.remove_all(
|
||||
"input.png",
|
||||
"clean.png",
|
||||
invisible=InvisibleOptions(strength=0.35),
|
||||
force=True,
|
||||
progress=print,
|
||||
)
|
||||
```
|
||||
|
||||
`InvisibleOptions` carries only what `InvisibleEngine` itself takes, and uses the
|
||||
engine's own parameter names and defaults. `force`, which decides whether the
|
||||
engine runs at all, is a parameter of `remove_all` and `remove_batch` alongside
|
||||
`backend` and `sensitivity`.
|
||||
|
||||
If AI metadata survives the strip, `remove_all` raises `MetadataStripIncomplete`
|
||||
**before** writing anything: an AI-readable output is worse than no output.
|
||||
|
||||
`remove_batch` runs one mode over a directory and never lets a single bad file
|
||||
end the run:
|
||||
|
||||
```python
|
||||
summary = raiw.remove_batch("in_dir", "out_dir", mode="visible") # -> BatchSummary
|
||||
print(summary.processed, summary.failed, summary.errors)
|
||||
print(summary.invisible_unavailable) # outputs that still carry the watermark
|
||||
```
|
||||
|
||||
`mode` is `all`, `visible`, `invisible`, or `metadata`. Pass a constructed
|
||||
`InvisibleEngine` as `engine` to load the model once for the whole directory.
|
||||
|
||||
## Inspect provenance
|
||||
|
||||
The default installation evaluates file metadata. Add `visible`, `detect`, or
|
||||
@@ -98,8 +155,17 @@ from remove_ai_watermarks.identify import identify
|
||||
report = identify(Path("input.png"))
|
||||
print(report.platform)
|
||||
print(report.signals)
|
||||
print(report.c2pa_validation)
|
||||
```
|
||||
|
||||
`c2pa_validation`, when present, reports `integrity`, `signature`,
|
||||
`signer_trust`, and `signer_validity` independently, plus the reader status
|
||||
codes. A valid hash and signature with an untrusted or expired signer is a
|
||||
medium-confidence signed claim. A hash or signature failure does not confirm the
|
||||
claimed platform or AI origin. Fallback parsing reports unknown validation
|
||||
dimensions, while a raw marker in an unsupported or malformed container can
|
||||
leave `c2pa_validation` as `None`.
|
||||
|
||||
Use `check_visible=False` and `check_invisible=False` for metadata-only
|
||||
inspection through the compatible path-based API:
|
||||
|
||||
@@ -125,8 +191,61 @@ evidence = extract_provenance_evidence(Path("input.png"))
|
||||
report = identify_from_evidence(evidence)
|
||||
```
|
||||
|
||||
If metadata was collected by another component, normalize its nested record
|
||||
without reopening the original file:
|
||||
### Collect once, judge elsewhere
|
||||
|
||||
`collect_metadata_record` splits the two halves apart: it is the only step that
|
||||
touches the file, and it returns a JSON-serializable record the verdict can be
|
||||
built from on another machine, in another process, or later.
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
from remove_ai_watermarks.identify import identify_metadata_record
|
||||
from remove_ai_watermarks.metadata_record import collect_metadata_record
|
||||
|
||||
record = collect_metadata_record(Path("input.png"), schema_version=1) # reads the file
|
||||
blob = json.dumps(record) # ship it anywhere
|
||||
|
||||
report = identify_metadata_record(json.loads(blob), path=Path("input.png")) # reads nothing
|
||||
payload = report.to_dict(schema_version=1) # versioned JSON contract
|
||||
```
|
||||
|
||||
The collection record has `record_type="provenance_metadata"`,
|
||||
`schema_version=1`, and a `status`. A vanished or unreadable source produces an
|
||||
`error` record with structured `issues`; `identify_metadata_record` rejects that
|
||||
record instead of turning a collection failure into an unknown-image verdict.
|
||||
Unknown schema versions, non-integer aliases, and native records without a
|
||||
`complete` collection status are rejected explicitly.
|
||||
|
||||
The verdict is the same one `identify(path, check_visible=False,
|
||||
check_invisible=False)` returns for that file. That equality is the record's whole
|
||||
contract and is verified over the tracked provenance fixtures and a separate local
|
||||
evaluation corpus. `ProvenanceReport.to_dict()` is the stable service boundary: it
|
||||
adds a `schema_version`, contains only JSON-safe values, and deliberately omits the
|
||||
local source path.
|
||||
|
||||
Package and transport versions evolve independently. Long-lived consumers should
|
||||
request the schema they implement, as above, instead of assuming the installed
|
||||
package's latest schema. Within schema 1, existing fields, types, meanings,
|
||||
`signals[].name` values, and `watermarks[]` labels remain compatible; releases may
|
||||
add fields that consumers must ignore. A breaking change requires a new schema while
|
||||
the schema 1 serializer remains available for rolling upgrades. Asking a release for
|
||||
an unsupported schema raises `ValueError` rather than silently returning another
|
||||
shape.
|
||||
|
||||
A record carries metadata regions, not the primary coded-pixel stream: marker
|
||||
segments before the JPEG scan, every PNG chunk except `IDAT`, RIFF chunks except the
|
||||
coded image, the ISOBMFF provenance boxes, the container's trailer, the parsed EXIF tags the
|
||||
verdict reads by name, PIL's info mapping, and the C2PA manifest store. Record size
|
||||
is bounded by those metadata regions and trailers; images with large embedded
|
||||
manifests naturally produce larger records.
|
||||
|
||||
The `path` argument is metadata: it labels the report and is never opened by
|
||||
either function, so a record collected elsewhere can be judged against a path that
|
||||
does not exist locally.
|
||||
|
||||
If metadata was collected by another component instead, normalize its nested
|
||||
record the same way:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.identify import (
|
||||
@@ -142,16 +261,70 @@ evidence = evidence_from_metadata_record(record, path=Path("input.png"))
|
||||
report = identify_from_evidence(evidence)
|
||||
```
|
||||
|
||||
The normalizer recursively preserves text and byte values. It also decodes
|
||||
Unversioned third-party records are normalized recursively for compatibility. The
|
||||
normalizer preserves text and byte values. It also decodes
|
||||
strings prefixed with `hex:` and fields named `base64` or ending in
|
||||
`_base64`. Diagnostic values under `error` and `kind` are ignored because they
|
||||
describe the collector rather than the source file. Pass a C2PA manifest-store
|
||||
dictionary in `record["c2pa_store"]`, or through the explicit
|
||||
`_base64`. Diagnostic, transport, timing, hash, provenance-result, and pixel-result
|
||||
subtrees are ignored because they describe the collector or a derived result rather
|
||||
than the source file. A versioned portable record is stricter still: only
|
||||
`metadata_base64`, `tail_base64`, `pil`, `exif`, and `c2pa_store` are accepted as
|
||||
source evidence. Other `record_type` values are rejected, so do not pass a broad
|
||||
forensic inspection record to this API. Pass a C2PA manifest-store dictionary in
|
||||
`record["c2pa_store"]`, or through the explicit
|
||||
`c2pa_manifest_store` argument.
|
||||
|
||||
`identify_from_evidence` does not reopen the source file. It evaluates metadata
|
||||
only; registered visible marks and pixel-backed invisible watermarks remain in
|
||||
the path-based `identify` call.
|
||||
### Broad metadata inspection
|
||||
|
||||
`collect_forensic_metadata` provides the wide metadata-only record used by forensic
|
||||
inspection and migration adapters. It preserves hashes and timestamps, full EXIF and
|
||||
IPTC, C2PA, container inventories, bounded raw metadata payloads, and embedded
|
||||
thumbnail forensics. It does not calculate a provenance verdict or pixel statistics.
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.forensic_metadata import collect_forensic_metadata
|
||||
|
||||
record = collect_forensic_metadata(Path("input.png"), schema_version=1)
|
||||
assert record["record_type"] == "forensic_metadata"
|
||||
```
|
||||
|
||||
This record is intentionally not accepted by `identify_metadata_record`. Collect the
|
||||
small strict provenance record separately and publish the resulting
|
||||
`ProvenanceReport.to_dict()` as the detector contract.
|
||||
|
||||
### Pixel evidence
|
||||
|
||||
`extract_pixel_evidence` decodes once and calculates the DCT, FFT, residual, ELA,
|
||||
gradient, and color families. Its versioned `to_dict()` result has a semantic
|
||||
`status`: `complete`, `partial` when an individual family failed, or `error` when the
|
||||
source could not be decoded. Transported errors contain only the exception class, so
|
||||
local paths stay in the caller's logs rather than crossing the service boundary.
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.pixel_evidence import extract_pixel_evidence
|
||||
|
||||
pixels = extract_pixel_evidence(Path("input.png"), artifacts=False, timings=True)
|
||||
payload = pixels.to_dict(schema_version=1)
|
||||
```
|
||||
|
||||
Timings and spatial artifacts are opt-in. Artifacts include image-identifying data
|
||||
such as a thumbnail and perceptual hash; aggregate feature families do not.
|
||||
|
||||
`identify_from_evidence` does not reopen the source file by default: it evaluates
|
||||
metadata only, and registered visible marks and pixel-backed invisible watermarks
|
||||
remain in the path-based `identify` call.
|
||||
|
||||
Pass `image_path` together with `check_visible` or `check_invisible` to add those
|
||||
pixel detectors on top of the SAME evidence. That is how a caller asking one file
|
||||
two provenance questions — which vendor is confirmed, and is there an invisible
|
||||
target — pays for the metadata extraction once:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.identify import extract_provenance_evidence, identify_from_evidence
|
||||
|
||||
evidence = extract_provenance_evidence(source)
|
||||
metadata_only = identify_from_evidence(evidence)
|
||||
with_pixels = identify_from_evidence(evidence, image_path=source, check_invisible=True)
|
||||
```
|
||||
|
||||
## Strip metadata
|
||||
|
||||
@@ -182,10 +355,206 @@ path preserves the pixels but drops standard metadata. Treat a nonempty
|
||||
undecodable input through unchanged, so its return alone must not be presented
|
||||
as proof that metadata was removed.
|
||||
|
||||
## Identify and clean video
|
||||
|
||||
The high level video API supports MP4, MOV, M4V, WebM, MKV, AVI, and FLV:
|
||||
metadata-only calls work with the default install, while visible identification,
|
||||
removal, and the complete pipeline require `remove-ai-watermarks[video]`.
|
||||
|
||||
```python
|
||||
import remove_ai_watermarks as raiw
|
||||
|
||||
report = raiw.identify_video("input.mp4")
|
||||
print(report.is_ai_generated)
|
||||
print(report.platform)
|
||||
print(report.visible_mark)
|
||||
print(report.metadata_markers)
|
||||
```
|
||||
|
||||
`identify_video` uses the same full-clip temporal arbiter as visible removal.
|
||||
It reports a recurring registered mark and supported AI metadata as positive
|
||||
signals. When neither is present, `is_ai_generated` is `None`, never `False`.
|
||||
The absence of a public local video SynthID decoder is included in `caveats`.
|
||||
Pass `check_visible=False` for a bounded metadata-only inspection.
|
||||
|
||||
For normal product integration, use the complete locally verifiable pipeline:
|
||||
|
||||
```python
|
||||
result = raiw.remove_video_all("input.mp4", "clean.mp4")
|
||||
if result.remaining_metadata:
|
||||
raise RuntimeError(f"AI metadata remains: {result.remaining_metadata}")
|
||||
```
|
||||
|
||||
The default removes one stable supported visible provider mark when present,
|
||||
always strips verified AI metadata, and writes a same-container output even
|
||||
when neither signal is found. This gives callers one predictable output path.
|
||||
It does not run lossy invisible regeneration by default.
|
||||
|
||||
`include_invisible=True` explicitly adds VAE regeneration for MP4, MOV, or M4V.
|
||||
`VideoAllResult.invisible_removed` reports whether the oracle-certified SynthID
|
||||
stage ran.
|
||||
|
||||
Process a top-level directory sequentially:
|
||||
|
||||
```python
|
||||
batch = raiw.remove_video_batch("videos", "videos_clean", mode="all")
|
||||
if batch.failed:
|
||||
for item in batch.items:
|
||||
if item.error:
|
||||
print(item.source, item.error)
|
||||
```
|
||||
|
||||
Batch modes are `all`, `visible`, and `metadata`. Successful visible no-ops are
|
||||
copied byte-for-byte, keeping the output directory complete. Per-file failures
|
||||
are returned in `VideoBatchItem.error`; they do not discard successful outputs.
|
||||
The invisible stage is available only as an explicit opt-in in `all` mode and
|
||||
reuses one loaded VAE runtime across the batch.
|
||||
|
||||
## Inspect and strip video metadata
|
||||
|
||||
Metadata inspection and removal use the same supported video containers:
|
||||
|
||||
```python
|
||||
import remove_ai_watermarks as raiw
|
||||
|
||||
report = raiw.inspect_video_metadata("input.mp4")
|
||||
if report.has_ai_metadata:
|
||||
result = raiw.remove_video_metadata("input.mp4")
|
||||
if result.remaining:
|
||||
raise RuntimeError(f"AI metadata remains: {result.remaining}")
|
||||
```
|
||||
|
||||
`remove_video_metadata` does not transcode video or audio streams. Its default
|
||||
output is `input_clean.mp4`, leaving the source untouched. An explicit output
|
||||
must use the same container extension as the source.
|
||||
|
||||
The returned `VideoMetadataResult` records the source, output, metadata detected
|
||||
before removal, and any markers remaining after the verified strip. MP4/MOV
|
||||
inspection recognizes the native TC260 `AIGC` entry in
|
||||
`moov.udta.meta.keys/ilst` and the QuickTime-form `meta` variants Doubao's iOS
|
||||
export writes; its removal preserves container size and encoded
|
||||
stream bytes. MP4/MOV/M4V are copied in bounded chunks, so a large `mdat` is not
|
||||
loaded into memory; publication is atomic. MKV/WebM inspection recognizes the corresponding
|
||||
`Segment.Tags.Tag.SimpleTag` representation; its removal requires ffmpeg for a
|
||||
stream-copy remux. AVI inspection reads `LIST/INFO/AIGC`, and FLV inspection
|
||||
reads `script.onMetaData.AIGC`; both use the same verified ffmpeg stream-copy
|
||||
removal path.
|
||||
|
||||
## Remove video SynthID
|
||||
|
||||
Install `remove-ai-watermarks[video,diffusion]` before using the video SynthID
|
||||
API.
|
||||
|
||||
```python
|
||||
import remove_ai_watermarks as raiw
|
||||
|
||||
result = raiw.remove_video_invisible(
|
||||
"input.mp4",
|
||||
"clean.mp4",
|
||||
device="auto",
|
||||
)
|
||||
if result.remaining_metadata:
|
||||
raise RuntimeError(f"AI metadata remains: {result.remaining_metadata}")
|
||||
```
|
||||
|
||||
`remove_video_invisible` supports MP4, MOV, and M4V. It regenerates the complete
|
||||
video through a VAE in bounded batches, shares one seeded latent-noise field
|
||||
across all frames, streams pixels to ffmpeg, copies complete audio, strips
|
||||
source metadata, and publishes atomically. The default output is
|
||||
`input_clean.mp4`; a distinct same-container output is required.
|
||||
|
||||
The returned `VideoInvisibleResult` includes output geometry, frame rate, frame
|
||||
count, paired PSNR, and the motion-compensated temporal-residual ratio. Those
|
||||
fields measure fidelity and flicker only. They are not a SynthID detector.
|
||||
The default `noise_std=0.15` is the current full-clip oracle floor; `0.10`
|
||||
remained detected on the public eight-second Veo calibration carrier.
|
||||
The default profile is oracle-certified. Google does not publish a local
|
||||
decoder for this video payload, so a fresh source-positive, output-negative
|
||||
pair from Gemini's built-in SynthID verifier remains an optional per-file audit.
|
||||
A response inferred from a visible logo or metadata is not such a verdict, and
|
||||
an adversarial follow-up asking ordinary Gemini to reinterpret the verifier is
|
||||
not a second oracle run.
|
||||
|
||||
## Remove a supported visible video mark
|
||||
|
||||
```python
|
||||
import remove_ai_watermarks as raiw
|
||||
|
||||
result = raiw.remove_video_visible(
|
||||
"input.mp4",
|
||||
"clean.mp4",
|
||||
backend="cv2",
|
||||
strip_metadata=True,
|
||||
temporal_consistency=True,
|
||||
)
|
||||
if result.output is None:
|
||||
print("No temporally stable supported mark was found")
|
||||
else:
|
||||
print(result.mark)
|
||||
|
||||
veo_result = raiw.remove_video_visible(
|
||||
"veo.mp4",
|
||||
"veo_clean.mp4",
|
||||
mark="veo",
|
||||
)
|
||||
seedance_result = raiw.remove_video_visible(
|
||||
"seedance.mp4",
|
||||
"seedance_clean.mp4",
|
||||
mark="seedance",
|
||||
)
|
||||
dola_result = raiw.remove_video_visible(
|
||||
"dola.mp4",
|
||||
"dola_clean.mp4",
|
||||
mark="dola",
|
||||
)
|
||||
hailuo_result = raiw.remove_video_visible(
|
||||
"hailuo.mp4",
|
||||
"hailuo_clean.mp4",
|
||||
mark="hailuo",
|
||||
)
|
||||
kling_result = raiw.remove_video_visible(
|
||||
"kling.mp4",
|
||||
"kling_clean.mp4",
|
||||
mark="kling",
|
||||
)
|
||||
```
|
||||
|
||||
`remove_video_visible` scans the complete video before writing output. It
|
||||
combines synthetic multi-scale visual matching with temporal consistency, so an
|
||||
isolated lookalike in one frame is not enough to authorize inpainting.
|
||||
`mark="auto"` is the default: it evaluates all providers in one decode pass and
|
||||
selects the first stable match in specificity order (`sora`, `veo`, `seedance`,
|
||||
`dola`, `hailuo`, `kling`). Provider confidence values are calibrated
|
||||
independently and are not compared across detectors. Pass one of those explicit
|
||||
values to restrict the scan to a single provider. The Veo detector recognizes
|
||||
the current four-point diamond and the
|
||||
legacy `Veo` text. Seedance recognizes the boxed `AI` label, Dola recognizes
|
||||
its compact text label, Hailuo recognizes the composite MINIMAX/Hailuo label,
|
||||
and Kling recognizes its bottom-right logo, wordmark, and version suffix. Each
|
||||
variant has an independent synthetic silhouette and calibrated temporal policy.
|
||||
After each accepted frame is filled, `temporal_consistency=True` motion-aligns
|
||||
the preceding accepted fill and blends it only when the warped prior mask
|
||||
covers the current mask and a surrounding source-context ring agrees. Scene
|
||||
cuts and disjoint masks keep the independent current fill. Pass
|
||||
`temporal_consistency=False` for the frame-local baseline.
|
||||
|
||||
The returned `VideoVisibleResult` records the selected `mark`, the total,
|
||||
detected, and removed frame counts, plus any AI metadata that survived the
|
||||
output encode. The function returns `output=None` and writes no file when no
|
||||
stable mark is selected. Video pixels are transcoded through ffmpeg while the
|
||||
complete source audio stream is copied. The encoder preserves supported 8-bit
|
||||
source chroma sampling, color tags, MP4/MOV track timescale, and relative
|
||||
variable-frame timestamps. It also retains a non-zero source start PTS and the
|
||||
copied audio offset. A failed encode preserves any existing output; only a
|
||||
completed result is published atomically.
|
||||
SDR 8-bit video is the supported pixel contract. High-bit-depth, PQ, and HLG
|
||||
sources raise `RuntimeError` before encoding instead of being silently reduced
|
||||
to 8-bit SDR.
|
||||
|
||||
## Remove invisible watermarks
|
||||
|
||||
Install `remove-ai-watermarks[diffusion]` for the standard pipelines or
|
||||
`remove-ai-watermarks[qwen-zimage]` for the CUDA-only high-fidelity profile.
|
||||
Install `remove-ai-watermarks[qwen-zimage]`. Both profiles need it, and both
|
||||
need an NVIDIA GPU.
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
@@ -193,7 +562,7 @@ from pathlib import Path
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine
|
||||
|
||||
engine = InvisibleEngine(
|
||||
pipeline="controlnet",
|
||||
pipeline="qwen-zimage", # the default; the only other value is "sdxl-zimage"
|
||||
device=None,
|
||||
cpu_offload=False,
|
||||
)
|
||||
@@ -204,29 +573,87 @@ engine.remove_watermark(
|
||||
)
|
||||
```
|
||||
|
||||
`device=None` selects the device automatically. Supported explicit values are
|
||||
defined by the CLI and runtime device resolver.
|
||||
`device=None` and `device="auto"` both run detection. `"cuda"` pins it without
|
||||
detecting. Every other value raises at construction rather than deferring a
|
||||
guaranteed failure to model-load time.
|
||||
|
||||
For limited CUDA memory:
|
||||
|
||||
```python
|
||||
engine = InvisibleEngine(
|
||||
pipeline="controlnet",
|
||||
pipeline="qwen-zimage",
|
||||
cpu_offload=True,
|
||||
)
|
||||
```
|
||||
|
||||
For the CUDA only high fidelity profile:
|
||||
Both profiles are CUDA-only, so on a machine without an NVIDIA GPU `device=None`
|
||||
resolves to `cpu` and construction raises. For the SDXL global stage instead of
|
||||
Qwen:
|
||||
|
||||
```python
|
||||
engine = InvisibleEngine(pipeline="qwen-zimage")
|
||||
engine = InvisibleEngine(pipeline="sdxl-zimage")
|
||||
```
|
||||
|
||||
The `qwen-zimage` extra must be installed for that profile.
|
||||
The `qwen-zimage` extra is required for both profiles: each runs the same
|
||||
DiffSynth Z-Image face stage.
|
||||
|
||||
The full `remove_watermark` signature includes strength, steps, guidance,
|
||||
seeding, tiling, resolution, upscaling, and postprocessing controls. Read the
|
||||
method signature in
|
||||
The opt-in verified-text stage uses the same `text_manifest` argument as the CLI:
|
||||
|
||||
```python
|
||||
engine.remove_watermark(
|
||||
Path("watermarked.png"),
|
||||
Path("clean.png"),
|
||||
text_manifest=Path("verified-lines.json"),
|
||||
)
|
||||
```
|
||||
|
||||
Install `remove-ai-watermarks[text-restoration]`. The manifest schema and safety
|
||||
constraints are documented in the CLI guide. The engine verifies its decoded RGB
|
||||
hash before loading the diffusion models and rejects SDXL, downscaling, and
|
||||
postprocessing combinations that were not evaluated. Tiling is allowed: the VAE
|
||||
donor uses the same overlapping tiles as the global pass. `InvisibleOptions` exposes the
|
||||
same field for `remove_all`; after a visible-stage edit, the manifest must be built
|
||||
against the staged pixels rather than the pristine source.
|
||||
|
||||
Use manifest schema 1 for manually reviewed text plus script metadata. Automated
|
||||
operators that verify only text-region geometry should emit schema 2 lines with a
|
||||
`box` and optional `angle`; no placeholder transcription or script is required.
|
||||
|
||||
Since 0.27.1 the mode's global 15% Qwen-VAE fidelity-anchor blend is **off by
|
||||
default** (`fidelity_anchor=False`): that whole-frame blend was measured to
|
||||
return detector-visible OpenAI SynthID on poster-scale manifests (official
|
||||
Content Provenance API, 2026-08-19 - detected x6 with the anchor, clean x6
|
||||
without it, controls and base outputs validated in the same sessions). Pass
|
||||
`fidelity_anchor=True` to reproduce the 0.27.0 research behavior.
|
||||
|
||||
### Drafting manifest lines
|
||||
|
||||
`remove_ai_watermarks.text_draft` proposes lines for a manifest; it never
|
||||
produces verified ones:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.text_draft import draft_text_lines
|
||||
|
||||
draft = draft_text_lines(Path("watermarked.png"))
|
||||
for line in draft.accepted:
|
||||
print(line.box, line.script, line.min_score, line.text)
|
||||
```
|
||||
|
||||
Install `remove-ai-watermarks[text-draft]` (CPU, no torch: PaddleOCR detection
|
||||
plus three script-chosen recognition engines). A line lands in `accepted` only
|
||||
when three crop paddings normalize identically and every confidence clears
|
||||
`min_score` (default 0.85); `accepted` means crop-stable, NOT ground-truth-
|
||||
correct - on the reference posters the draft's exact-text precision was 90.0%
|
||||
and 94.4% because high-confidence OCR still lost punctuation. Every accepted
|
||||
line needs a human yes/no before a manifest may claim `verified: true`.
|
||||
`source_pixel_sha256` is re-exported here for building the manifest's
|
||||
pixel-binding hash against the exact source the engine will decode.
|
||||
|
||||
`remove_watermark` takes strength, seed, tiling, resolution, and postprocessing
|
||||
controls. It takes no model id, step count or guidance scale, and neither does the
|
||||
constructor: each profile pins its model stack, its per-stage schedule and CFG
|
||||
1.0, so passing one raises `TypeError` at the call rather than being accepted and
|
||||
refused several layers down. Read the method signature in
|
||||
[`invisible_engine.py`](../src/remove_ai_watermarks/invisible_engine.py) or use
|
||||
the CLI guide for the concepts.
|
||||
Defaults can differ between the Python method and CLI profile resolution, so
|
||||
|
||||
@@ -79,13 +79,13 @@ Measured on `gemini_3` (18 faces) at the Gemini scrub floor 0.25 vs base-Qwen 0.
|
||||
a Qwen face fix. The next distinct architecture was Z-Image-Turbo on original masked face
|
||||
crops, not another Qwen geometry conditioner.
|
||||
|
||||
**Implementation follow-up (2026-07-24):** that distinct architecture now exists as the
|
||||
manual `qwen-zimage` profile. It ports the upstream Synthid-Bypass v2 graph: Qwen-Image-2512
|
||||
Lightning + DiffSynth Canny for the full frame, then SAM-masked Z-Image Turbo regeneration
|
||||
from original face crops. The upstream result supplied by the user was Gemini-oracle negative.
|
||||
The active upstream face path is YOLO + SAM, not the unconnected MediaPipe node. The port
|
||||
matches its center-point + box prompts, IoU-0.93 proposal selection, detector-box intersection,
|
||||
crop factor, and paste feather; YuNet is the intentional detector substitution.
|
||||
**Implementation follow-up (2026-07-24, revised 2026-07-31):** an early experimental
|
||||
`qwen-zimage` prototype reproduced a broad two-stage shape demonstrated by a public
|
||||
experiment: structure-guided Qwen regeneration followed by masked Z-Image face
|
||||
refinement. The maintained profile was subsequently
|
||||
rewritten with project-owned prompts, adaptive strength and sizing policies, YuNet face
|
||||
detection, SAM selection, masks, and compositing. It does not include the upstream workflow
|
||||
JSON or source code. The upstream result supplied by the user was Gemini-oracle negative.
|
||||
|
||||
The first exact-path Modal run completed without SAM fallback or face seams. On one crowded
|
||||
18-face `gemini_3` fixture, ArcFace identity improved materially over controlnet
|
||||
@@ -108,7 +108,7 @@ checked all six current outputs in the provider-separated
|
||||
oracles and confirmed that none retained SynthID or the provider generation signal. The
|
||||
checked bytes used the complete `visible -> qwen-zimage -> metadata` route, the calibrated
|
||||
YuNet 0.5 gate, and the shipped prompt-cache/model-residency optimizations. This supersedes
|
||||
the earlier first-port batch check as the release-candidate result, but it is not a
|
||||
the earlier prototype batch check as the release-candidate result, but it is not a
|
||||
certification across seeds, resolutions, and content classes. YuNet's threshold was
|
||||
calibrated independently from upstream YOLO: 0.5 retained the visible faces in the
|
||||
comparison fixtures while removing the false and duplicate boxes admitted by the copied
|
||||
|
||||
@@ -3,6 +3,21 @@
|
||||
This page describes the release behavior defined in this repository. External
|
||||
registry state can change independently, so verify it during a release.
|
||||
|
||||
## Published surfaces
|
||||
|
||||
A release is complete only after all four published surfaces are verified:
|
||||
|
||||
| Surface | Published result | Automation |
|
||||
|---|---|---|
|
||||
| PyPI | The `remove-ai-watermarks` wheel and source distribution | `publish.yml` |
|
||||
| Homebrew | The `remove-ai-watermarks` formula in `wiltodelta/homebrew-tap` | `distribute.yml` |
|
||||
| Hugging Face Space | The Space demo deployed on the new release through a requirements-pin bump in `wiltodelta/raiw-hf-space` | `raiw-hf-space` sync workflow; the pin bump is manual, `distribute.yml` only re-installs the pinned version |
|
||||
| ComfyUI Registry | A compatible release of `wiltodelta/ComfyUI-remove-ai-watermarks` with its own node version | `distribute.yml` and the node repository's workflows |
|
||||
|
||||
The GitHub Release is the trigger and release record for this flow. Conda is
|
||||
not a supported publishing surface: this repository has no conda recipe or
|
||||
conda publication job.
|
||||
|
||||
## Release sources of truth
|
||||
|
||||
The package version appears in:
|
||||
@@ -43,31 +58,36 @@ PyPI API token from the repository.
|
||||
waits for the matching source distribution to appear on PyPI, then:
|
||||
|
||||
- updates the Homebrew tap formula URL and SHA-256;
|
||||
- updates the repository conda recipe version and source-distribution SHA-256;
|
||||
- triggers a factory rebuild of the Hugging Face Space.
|
||||
- triggers a factory rebuild of the Hugging Face Space: this re-installs the
|
||||
version pinned in the Space repo, it does NOT upgrade the demo;
|
||||
- synchronizes, tests, versions, and publishes the ComfyUI nodes.
|
||||
|
||||
Upgrading the Hugging Face Space is a separate, manual step: bump
|
||||
`remove-ai-watermarks[visible,heif]` in `wiltodelta/raiw-hf-space` (`pyproject.toml`,
|
||||
`uv lock`, and the re-exported `requirements.txt`), then push. That repository's
|
||||
`sync-to-hf.yml` mirrors the files onto the Space, which rebuilds on the new
|
||||
pin. Its callback smoke tests run in that repository's CI on every push; they
|
||||
are the scenario check for the demo surface.
|
||||
|
||||
The workflow can also be started manually with an optional version input.
|
||||
Conda-forge updates are outside this workflow.
|
||||
|
||||
If a distribution job fails because a repository or Hugging Face credential is
|
||||
invalid, rotate the corresponding GitHub secret and rerun the failed job. A
|
||||
manual Homebrew formula update is the fallback when its automation is blocked.
|
||||
|
||||
The conda job uses the published artifact rather than a locally built archive
|
||||
as the hash source and commits the resulting recipe change to `main`. Runtime
|
||||
dependency mapping remains review-controlled: keep it aligned with the default
|
||||
metadata dependencies in `pyproject.toml`, do not copy optional pixel extras
|
||||
into the default recipe, and document any conda-forge package that is
|
||||
unavailable and must be omitted.
|
||||
|
||||
## Source distribution boundary
|
||||
|
||||
The wheel includes the package under `src/`.
|
||||
|
||||
The source distribution explicitly excludes `/data` through
|
||||
`[tool.hatch.build.targets.sdist]` in `pyproject.toml`. Keep that exclusion:
|
||||
calibration captures and test corpora do not belong in the published package
|
||||
archive.
|
||||
The source distribution uses an explicit allowlist for `/src`, `/LICENSE`,
|
||||
`/README.md`, and `/pyproject.toml` through
|
||||
`[tool.hatch.build.targets.sdist]` in `pyproject.toml`. It also defensively
|
||||
excludes `/data`, `/tmp`, and `/.sc`. Keep both controls: calibration captures,
|
||||
test corpora, generated research outputs, and local session state do not belong
|
||||
in the published package archive. Hatchling always adds the root `.gitignore` to
|
||||
the sdist, so keep its comments generic and free of local operational context.
|
||||
Ignore rules are not the build boundary: `data/` is deliberately tracked, while
|
||||
the sdist configuration keeps it and the other excluded paths out of the archive.
|
||||
|
||||
## Build backend
|
||||
|
||||
@@ -76,8 +96,6 @@ The package uses hatchling through the unpinned `hatchling` build requirement in
|
||||
|
||||
## Other channels
|
||||
|
||||
The repository includes a conda recipe under `packaging/conda/recipe.yaml`.
|
||||
|
||||
The ComfyUI nodes are maintained and versioned in their own repository. After
|
||||
the matching source distribution appears on PyPI, `distribute.yml` dispatches
|
||||
that repository's sync workflow with the exact library version and waits for it
|
||||
@@ -90,13 +108,30 @@ write access.
|
||||
|
||||
## Release verification
|
||||
|
||||
Forensic transports are versioned independently from the package. Before publishing
|
||||
a change to provenance metadata, provenance reports, broad forensic metadata, or
|
||||
pixel evidence, run their schema 1 contract tests. Additive fields are compatible;
|
||||
renaming a field, changing its type or meaning, changing a signal name or watermark
|
||||
label, or removing a field requires a new output schema. Add the new serializer
|
||||
without removing schema 1 so long-lived consumers can update separately. A package
|
||||
release must never silently substitute its latest schema when a caller explicitly
|
||||
requests an older supported one.
|
||||
|
||||
After publication, verify:
|
||||
|
||||
- both wheel and source distribution exist on PyPI;
|
||||
- the package version matches the tag;
|
||||
- the Homebrew formula points to the new source distribution;
|
||||
- the distribution workflow completed successfully;
|
||||
- the repository's conda recipe matches the published version and source
|
||||
distribution;
|
||||
- the ComfyUI Registry node requires the new library version;
|
||||
- the Hugging Face Space serves the new release: its `requirements.txt` pin
|
||||
matches (raw file via the HF API), the runtime stage is `RUNNING` after the
|
||||
rebuild, and one live `identify` and one live `visible` API call succeed
|
||||
against the running Space;
|
||||
- a clean install can run `remove-ai-watermarks --version`.
|
||||
|
||||
A clean-install check run immediately after publication can fail with "no
|
||||
version of remove-ai-watermarks==X.Y.Z" even though the simple index already
|
||||
lists it: uv serves its cached pre-release view of the index. Re-run with a
|
||||
fresh cache (`UV_CACHE_DIR=$(mktemp -d)`) before treating it as a release
|
||||
failure.
|
||||
@@ -26,7 +26,7 @@ color still leaves a persistent ghost outline.
|
||||
|
||||
Diagnosed why, empirically (cached stacks, `/tmp/doubao_distill`): (1) the mark is a clean white overlay with **no dark halo** -- over glyph pixels ~54% are brighter than the clean bg, only ~4% darker -- so the white-logo model `I=(1-α)O+α·255` is correct; (2) but content backgrounds are almost never dark *under* the mark (median darkest available bg over glyph pixels = **58/255**; only ~13% of mark pixels are ever observed on a bg < 40), so on bright backgrounds the equation is ill-conditioned and `α` is unidentifiable; (3) LaMa's `O` is a plausible **hallucination**, not the true pre-mark background, which compounds the error, and per-pixel regression on ~15 obs overfits into color noise.
|
||||
|
||||
**Why Gemini's engine is clean (verified in GeminiWatermarkTool `src/core/watermark_engine.cpp`): its alpha map is the watermark stamped on a PURE-BLACK background**, where `watermarked = α·255 + (1-α)·0 = α·255`, so `alpha = capture/255` exactly -- no estimation. (`gemini_bg_*.png` is literally the sparkle in gray on black.) So the real Doubao unlock is the same controlled capture, **not more content images**. The retained black and gray outputs live in `data/calibration/doubao/`; local solid-color seeds are regenerable and are not committed.
|
||||
**Why Gemini's engine is clean: its alpha map is the watermark stamped on a PURE-BLACK background**, where `watermarked = α·255 + (1-α)·0 = α·255`, so `alpha = capture/255` exactly -- no estimation. (`gemini_bg_*.png` is literally the sparkle in gray on black.) So the real Doubao unlock is the same controlled capture, **not more content images**. The retained black and gray outputs live in `data/calibration/doubao/`; local solid-color seeds are regenerable and are not committed.
|
||||
|
||||
**Until black captures arrive, the shipped direction is precise canonical glyph mask + inpaint (cv2 default, lama optional), NOT reverse-alpha.**
|
||||
|
||||
|
||||
+79
-19
@@ -29,13 +29,34 @@ the masked area.
|
||||
Marks from other vendors are not detected automatically. Use `erase --region`
|
||||
when you can select the affected area yourself.
|
||||
|
||||
### Visible video marks
|
||||
|
||||
| Key | Mark | Motion | Important limit |
|
||||
| --- | --- | --- | --- |
|
||||
| `sora` | Sora 2 mascot and wordmark | Moves among frame positions | Requires a temporally recurring visual match; the older Sora Turbo corner swirl is a different unsupported mark. |
|
||||
| `veo` | Current four-point diamond and legacy `Veo` text | Fixed bottom-right corner | Uses separate silhouettes and requires a recurring match; learned fill is preferable on structured backgrounds. |
|
||||
| `seedance` | Boxed `AI` label | Fixed bottom-right corner | Requires an anchored recurring match; the full localized box is filled because a thinner synthetic shape mask leaves the real translucent rim behind. |
|
||||
| `dola` | `Dola AI` text | Fixed bottom-right corner | Requires an anchored recurring match; ByteDance or BytePlus provenance can relax only an existing visual run. |
|
||||
| `hailuo` | `MINIMAX \| hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. A TC260 label naming MiniMax as producer can relax only an existing stable run. |
|
||||
| `kling` | Kling swirl, `KLING AI`, version, and optional `PRO` suffix | Fixed bottom-right edge | Combines a synthetic logo rescue with font variants, an edge gate, a white-label gate, and anchored temporal recurrence. |
|
||||
|
||||
`video identify`, `video visible`, and `video all` share this registry and the
|
||||
same temporal arbiter. It is separate from the image registry because selection
|
||||
is made over a sequence rather than one raster. The default `auto` mode scans
|
||||
all six entries in one decode pass and selects the first temporally stable
|
||||
match in table order; an explicit mark restricts the scan to that row.
|
||||
Accepted fills are motion-aligned across adjacent frames by default. The prior
|
||||
fill contributes only where its warped mask covers the current removal mask and
|
||||
nearby source context agrees. Scene cuts or disjoint marks retain the
|
||||
independent frame fill.
|
||||
|
||||
## Fill backends
|
||||
|
||||
| Backend | Install | Behavior |
|
||||
| --- | --- | --- |
|
||||
| `cv2` | `remove-ai-watermarks[visible]` | Classical OpenCV inpainting |
|
||||
| `migan` | `remove-ai-watermarks[migan]` | MI-GAN through ONNX Runtime |
|
||||
| `lama` | `remove-ai-watermarks[lama]` | big-LaMa through ONNX Runtime |
|
||||
| `migan` | `remove-ai-watermarks[migan]` | MI-GAN through ONNX Runtime; practical learned CPU video tier |
|
||||
| `lama` | `remove-ai-watermarks[lama]` | big-LaMa through ONNX Runtime; offline video quality tier |
|
||||
| `auto` | Depends on installed extras | Selects LaMa, then MI-GAN, then OpenCV |
|
||||
|
||||
The learned backends download model files on first use.
|
||||
@@ -46,18 +67,30 @@ The inspection and stripping code handles signals in these groups:
|
||||
|
||||
- C2PA Content Credentials and supported cloud manifest references;
|
||||
- EXIF and XMP generator fields;
|
||||
- exact app-export provenance and AIGC disclosures from supported
|
||||
ByteDance-family products, with product-only provenance excluded from the
|
||||
generated-image verdict;
|
||||
- IPTC AI disclosure fields;
|
||||
- PNG text chunks and embedded generation parameters;
|
||||
- China TC260 AIGC labels in supported metadata placements;
|
||||
- China TC260 AIGC labels in supported image placements and the normative
|
||||
MP4/MOV `moov.udta.meta.keys/ilst`, MKV/WebM
|
||||
`Segment.Tags.Tag.SimpleTag`, AVI `LIST/INFO/AIGC`, and FLV
|
||||
`script.onMetaData.AIGC` placements;
|
||||
- xAI and Grok EXIF signature fields;
|
||||
- Samsung AI editing markers;
|
||||
- Hugging Face job metadata;
|
||||
- open Stable Diffusion style DWT-DCT watermarks with the `detect` extra;
|
||||
- Adobe TrustMark with the `trustmark` extra.
|
||||
- Adobe TrustMark Variant P schemas 0-2 with the `trustmark` extra. Variant Q
|
||||
needs a different model, while schema 3 is deliberately rejected because it
|
||||
produced persistent false positives on unrelated generators.
|
||||
|
||||
`identify` combines detected signals into a `ProvenanceReport`. It reports
|
||||
unknown when evidence is absent. It never treats missing metadata as proof that
|
||||
an image is human made.
|
||||
an image is human made. C2PA presence alone is not a verified identity: the
|
||||
report distinguishes asset binding, claim signature, signer trust, and signer
|
||||
validity. High-confidence C2PA attribution requires all four; intact but
|
||||
untrusted or fallback claims are medium-confidence, while a failed binding or
|
||||
signature contributes no origin verdict.
|
||||
|
||||
## File and container formats
|
||||
|
||||
@@ -76,8 +109,8 @@ Metadata inspection and removal additionally have container paths for:
|
||||
|
||||
- JPEG XL metadata;
|
||||
- MP4, MOV, M4V, and M4A;
|
||||
- WebM, MKV, MKA, MP3, WAV, FLAC, OGG, OGA, Opus, and AAC when ffmpeg is
|
||||
available.
|
||||
- WebM, MKV, MKA, AVI, FLV, MP3, WAV, FLAC, OGG, OGA, Opus, and AAC when
|
||||
ffmpeg is available.
|
||||
|
||||
JPEG image metadata stripping removes targeted metadata segments without
|
||||
re-encoding the entropy coded image scan. PNG and WebP removal preserves pixel
|
||||
@@ -90,17 +123,40 @@ The `invisible` command uses diffusion regeneration. It targets watermark
|
||||
patterns by changing the image rather than decoding and deleting a known
|
||||
payload.
|
||||
|
||||
Current pipeline values:
|
||||
Current pipeline values, both CUDA-only:
|
||||
|
||||
- `controlnet`;
|
||||
- `sdxl`;
|
||||
- `qwen`;
|
||||
- `qwen-zimage`;
|
||||
- legacy alias `default`, which resolves to `sdxl`.
|
||||
- `qwen-zimage`, the default;
|
||||
- `sdxl-zimage`, the same recipe and the same face stage on an SDXL global pass.
|
||||
|
||||
SynthID does not have a public local pixel decoder in this project. The tool can
|
||||
infer likely presence from supported provenance metadata, but after that
|
||||
metadata is removed a local negative result is inconclusive.
|
||||
The `controlnet`, `sdxl`, `qwen` and `default` values were removed. A retired name
|
||||
is rejected at parse time rather than remapped onto a surviving profile.
|
||||
|
||||
SynthID does not have a public local pixel decoder in this project. The tool
|
||||
recognizes presence from supported provenance: Google AI C2PA under Google's
|
||||
all-media watermark policy, and current OpenAI C2PA carrying an explicit
|
||||
`c2pa.watermarked.*` action. Legacy OpenAI C2PA without that action does not
|
||||
assert SynthID. After provenance metadata is removed, a local negative result
|
||||
is still inconclusive.
|
||||
|
||||
Microsoft Paint can name `com.microsoft.invismark.1` in a C2PA soft-binding
|
||||
assertion. Inspection reports both that exact algorithm and its signed `value`,
|
||||
which Paint uses as the identifier carried by the pixel watermark. Photos has a
|
||||
parallel local writer path, but no public output sample was available for this
|
||||
work. Metadata stripping removes the embedded manifest, not the pixel carrier.
|
||||
The project has no validated local InvisMark decoder. Microsoft's official
|
||||
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
|
||||
is the external oracle: it reports pixel `Watermark` and embedded `C2PA` results
|
||||
separately. The diffusion profiles have not been broadly certified against that
|
||||
oracle, so a control-positive, output-negative pair is still a per-file result,
|
||||
not a universal guarantee.
|
||||
|
||||
For MP4, MOV, and M4V, `video invisible` or the explicit
|
||||
`video all --invisible` option can regenerate the video through a VAE and strip
|
||||
source metadata. The shipped profile is oracle-certified, but it is not a local
|
||||
decoder. A fresh source-positive, output-negative pair from Gemini's built-in
|
||||
SynthID verifier is an optional per-file audit. A normal Gemini answer may instead
|
||||
infer from a visible logo or metadata; asking it to reinterpret a completed
|
||||
verifier result is not a second oracle run.
|
||||
|
||||
The optional `detect` extra is different: it provides a local decoder for the
|
||||
open DWT-DCT watermark used by some Stable Diffusion, SDXL, and FLUX workflows.
|
||||
@@ -112,14 +168,18 @@ not a universal clean verdict.
|
||||
| Provider or family | Visible | Invisible path | Metadata or provenance |
|
||||
| --- | --- | --- | --- |
|
||||
| Google Gemini | Sparkle | Diffusion regeneration for SynthID | C2PA and related source signals |
|
||||
| Google Veo video | Veo diamond and legacy text | Oracle-certified VAE removal for SynthID | C2PA and related source signals |
|
||||
| OpenAI image generators | None registered | Diffusion regeneration for supported invisible signals | C2PA and generator provenance |
|
||||
| Microsoft Paint and Photos | None registered | External Microsoft oracle for InvisMark; no validated local decoder | Paint C2PA soft-binding algorithm and identifier |
|
||||
| Stable Diffusion and SDXL | None registered | Diffusion regeneration; optional open decoder | Embedded parameters and text metadata |
|
||||
| FLUX | None registered | Diffusion regeneration; optional open decoder | C2PA for supported sources |
|
||||
| Adobe Firefly | None registered | No proprietary local decoder | C2PA; optional TrustMark decoder |
|
||||
| Adobe Firefly | None registered | Optional TrustMark Variant P decoder | C2PA |
|
||||
| Midjourney | None registered | No registered pixel decoder | EXIF, XMP, and IPTC signals |
|
||||
| ByteDance generators | Doubao and Jimeng marks | No registered pixel decoder | TC260 AIGC and supported C2PA signals |
|
||||
| Luma AI | None registered | No registered pixel decoder | PNG text generator tags (Uni-1) |
|
||||
| ByteDance generators | Doubao and Jimeng marks | No registered pixel decoder | TC260 AIGC, supported C2PA, and exact app-export AIGC disclosures |
|
||||
| Qwen | Qwen mark | No registered pixel decoder | TC260 AIGC |
|
||||
| Kling | Kling mark | No registered pixel decoder | TC260 AIGC |
|
||||
| Kling | Kling image and video marks | No registered pixel decoder | TC260 AIGC |
|
||||
| Hailuo / MiniMax video | Hailuo composite video label | No registered pixel decoder | TC260 AIGC where present |
|
||||
| Baidu | Baidu mark | No registered pixel decoder | TC260 AIGC |
|
||||
| LibLibAI | LibLibAI mark | No registered pixel decoder | TC260 AIGC |
|
||||
| RunningHub | RunningHub mark | No registered pixel decoder | TC260 AIGC |
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# Deep research: SynthID-safe face-identity recovery for SDXL (2026-06-08)
|
||||
|
||||
> Research archive. This dated follow-up records evidence available during the
|
||||
> study. It is not the current command reference.
|
||||
> study. It is not the current command reference. The `instantid` and `photomaker`
|
||||
> extras it discusses, the `--restore-faces` flag, and their modules were all removed
|
||||
> from the library afterwards.
|
||||
|
||||
**Stats:** {"angles": 6, "sourcesFetched": 28, "claimsExtracted": 104, "claimsVerified": 25, "confirmed": 19, "killed": 6, "afterSynthesis": 6, "urlDupes": 1, "budgetDropped": 7, "agentCalls": 111}
|
||||
|
||||
@@ -149,8 +151,9 @@ on Modal A100 in two phases:
|
||||
- v3 (tighter ellipse 0.32*bw x 0.42*bh + per-channel mean color match to
|
||||
local cleaned canvas + softer feather): patchwork visually softened; faces
|
||||
still read as studio portraits inserted into the scene, not as people
|
||||
shot in the scene. Single portrait identity drifted (tatsunari -> "round
|
||||
Asian male" vs original's thin face).
|
||||
shot in the scene. Single portrait identity drifted (the single-portrait
|
||||
fixture regenerated as a generic "round Asian male" vs the original's thin
|
||||
face).
|
||||
2. **Phase 2: InstantID img2img on cleaned crop.** Switched to the upstream
|
||||
`pipeline_stable_diffusion_xl_instantid_img2img` (downloaded at first use
|
||||
from raw.githubusercontent.com; requires `trust_remote_code=True`). Same
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
> Research archive. This document evaluates approaches considered during the
|
||||
> study. It does not describe the current public interface. The current
|
||||
> pipelines and limits are documented in `docs/cli.md` and
|
||||
> `docs/known-limitations.md`.
|
||||
> `docs/known-limitations.md`. The path it calls shipped -- `photomaker_restore.py`
|
||||
> and the `photomaker` extra -- was removed from the library afterwards.
|
||||
|
||||
> **Status (2026-06-08): retired.** Every approach described below was empirically
|
||||
> tested and rejected -- see `docs/synthid-robust-identity-research-2026-06-08.md`
|
||||
@@ -246,7 +247,9 @@ from the test set + this doc).
|
||||
- If no -> the assumption is wrong; PhotoMaker would re-introduce the
|
||||
watermark. Stop and reconsider.
|
||||
2. **PhotoMaker-V1 prototype** in the existing `controlnet` pipeline:
|
||||
- Mirror the `_load_controlnet_pipeline` path: add a PhotoMaker variant that
|
||||
- Mirror the `_load_controlnet_pipeline` path (removed in 0.24.0 with the
|
||||
controlnet profile; the equivalent seam is now `SdxlZImagePipeline._load_sdxl`):
|
||||
add a PhotoMaker variant that
|
||||
loads SDXL + canny ControlNet + PhotoMaker adapter on the same engine.
|
||||
- Extract the OpenCLIP face embedding from the watermarked face crops (use
|
||||
OpenCV YuNet, already bundled for `auto`, to find the face boxes).
|
||||
|
||||
+195
-42
@@ -1,4 +1,4 @@
|
||||
# SynthID-Image: technical reference
|
||||
# SynthID: technical reference
|
||||
|
||||
> Technical research reference. Current package behavior is defined by the
|
||||
> [supported signals](supported-signals.md), [known limitations](known-limitations.md),
|
||||
@@ -6,10 +6,10 @@
|
||||
> historical evidence and should not be read as current CLI defaults.
|
||||
|
||||
This document covers how Google SynthID for images works mechanically, what it
|
||||
survives, what removes it, and the current deployment landscape. It is written
|
||||
for engineers working on watermark detection and removal -- specifically to
|
||||
inform decisions about strength settings, test methodology, and what oracle
|
||||
results mean.
|
||||
survives, what removes it, the external video-verification workflow, and the
|
||||
current deployment landscape. It is written for engineers working on watermark
|
||||
detection and removal -- specifically to inform decisions about strength
|
||||
settings, test methodology, and what oracle results mean.
|
||||
|
||||
Primary sources are cited inline. Marketing-only claims are flagged separately
|
||||
from independently-verified results.
|
||||
@@ -289,14 +289,19 @@ The external variant SynthID-O is available "through partnerships" only. Our
|
||||
tool cannot locally detect SynthID presence or absence -- this is by design,
|
||||
not a gap we can fill.
|
||||
|
||||
### 3.2 How our tool detects SynthID (metadata proxy)
|
||||
### 3.2 How our tool recognizes SynthID from provenance
|
||||
|
||||
We detect SynthID indirectly: if the image's C2PA manifest is signed by a
|
||||
known SynthID-using issuer (Google, OpenAI), we infer SynthID is present. This
|
||||
is a **metadata proxy**, not a pixel watermark decode. It works while the C2PA
|
||||
manifest is intact, and is silent once the manifest is stripped or the image
|
||||
is re-encoded without C2PA (e.g., a screenshot, a social-media re-upload, or
|
||||
after `metadata --remove`).
|
||||
We recognize SynthID indirectly from supported C2PA evidence; this is not a
|
||||
pixel watermark decode. Google states that all media generated by its tools is
|
||||
watermarked, so Google AI C2PA establishes SynthID. OpenAI C2PA predates its
|
||||
SynthID rollout, but current manifests add the signed
|
||||
`c2pa.watermarked.unbound` action; OpenAI provenance establishes SynthID only
|
||||
when that action is present. Legacy OpenAI C2PA without it remains valid origin
|
||||
evidence but does not assert a pixel watermark.
|
||||
|
||||
This works while the C2PA manifest is intact and is silent once the manifest is
|
||||
stripped or the image is re-encoded without C2PA (e.g., a screenshot, a
|
||||
social-media re-upload, or after `metadata --remove`).
|
||||
|
||||
This is why:
|
||||
- `identify` on a GitHub-recompressed issue attachment returns Unknown (C2PA is
|
||||
@@ -326,6 +331,63 @@ A Google-SynthID image reads clean on openai.com/verify. An OpenAI image reads
|
||||
clean in the Gemini oracle. They are different payloads within the same
|
||||
framework.
|
||||
|
||||
### 3.4 Video verification and attack harness
|
||||
|
||||
Gemini's built-in verification flow reports whether and where it detects Google
|
||||
SynthID in a video. This remains a proprietary oracle: invoke `@synthid`, use
|
||||
the supported content-verification question, and keep every file in a separate
|
||||
new chat. A normal Gemini answer that discusses visual clues or metadata is not
|
||||
an oracle verdict. Nor is an adversarial follow-up that asks the chat model to
|
||||
ignore and reinterpret a completed verifier result.
|
||||
|
||||
The research harness `scripts/video_synthid_sweep.py` tests a VAE regeneration
|
||||
attack without pretending to detect success locally. It emits:
|
||||
|
||||
1. a re-encode control using the same sampled frames, dimensions, frame rate,
|
||||
and codec as the candidates;
|
||||
2. VAE round-trip candidates with one spatial latent-noise field shared across
|
||||
time;
|
||||
3. paired PSNR and motion-compensated temporal-residual metrics;
|
||||
4. an empty oracle column for the external verdict.
|
||||
|
||||
The control is the first oracle submission. If it is not SynthID-positive, stop:
|
||||
the surrounding transcode already changed the verifier result. Only a
|
||||
control-positive, candidate-negative pair is evidence about the regeneration
|
||||
attack. PSNR and temporal residual measure fidelity and flicker, never watermark
|
||||
presence.
|
||||
|
||||
The shipped `video invisible` command and `remove_video_invisible` API reuse the
|
||||
same VAE regeneration mechanism for a complete input sequence. The shipped
|
||||
default is oracle-certified and does not expose a separate verification-status
|
||||
flag. In the 2026-07-29
|
||||
two-carrier calibration, both matched controls were positive in the built-in
|
||||
verifier; the stronger candidate was negative on both, while a weaker
|
||||
candidate was negative on one. A 2026-07-30 `UNAVAILABLE` response came from an
|
||||
ordinary-model follow-up that asked Gemini to reinterpret the already returned
|
||||
verdict and therefore did not invalidate it. The default is a calibrated,
|
||||
content-dependent operating point. A per-file provider check remains an
|
||||
optional audit after provider changes or for unusually important files.
|
||||
|
||||
The 2026-07-31 full-clip calibration used Google's public eight-second Veo
|
||||
off-road sample through the complete product command. The original was detected
|
||||
across 00:00-00:07, the `noise_std=0.10` output remained detected, and the
|
||||
`0.15` output was not detected. The positive `0.10` result proves that the
|
||||
surrounding 512 px / 12 fps / H.264 path did not create the negative result by
|
||||
itself. `0.15` is therefore the shipped default. The tracked manifest
|
||||
`data/evaluations/video-synthid-oracle.csv` records the public source URL,
|
||||
hashes, fidelity metrics, and verdicts without committing generated videos.
|
||||
|
||||
What that calibration does and does not constrain, the ranked experiment program
|
||||
for trading less quality for the same removal, and the change candidates that were
|
||||
refuted along the way are recorded in
|
||||
[`video-synthid-quality-research.md`](video-synthid-quality-research.md).
|
||||
|
||||
The VAE perturbation follows the general regeneration-attack construction from
|
||||
Zhao et al. The video-specific control and temporal metric are local additions.
|
||||
VideoMarkBench motivates testing frame aggregation and matched perturbations,
|
||||
but it does not evaluate Google's proprietary SynthID, so its findings cannot
|
||||
stand in for the Gemini oracle.
|
||||
|
||||
---
|
||||
|
||||
## 4. Adoption and current state (as of June 2026)
|
||||
@@ -352,8 +414,9 @@ OpenAI confirmed SynthID adoption (Help Center, updated 2026-05-21):
|
||||
> "ChatGPT images include both C2PA metadata and SynthID watermarks."
|
||||
|
||||
This is time-gated: pre-rollout ChatGPT/gpt-image images carry C2PA without
|
||||
SynthID. Our C2PA proxy therefore over-reports SynthID presence on old images
|
||||
(hence the `_OPENAI_CAVEAT` hedging flag in the codebase).
|
||||
SynthID. Current OpenAI manifests distinguish the watermarked output with the
|
||||
signed `c2pa.watermarked.unbound` action. The detector requires that action, so
|
||||
old OpenAI C2PA remains an origin signal without becoming a SynthID claim.
|
||||
|
||||
### 4.3 Other vendors
|
||||
|
||||
@@ -396,6 +459,14 @@ the payload, reconstituting SynthID in text. The lesson held and shaped the
|
||||
current design: **content is preserved by REGENERATING it under structural
|
||||
conditioning, never by copying original pixels.**
|
||||
|
||||
> **Superseded in 0.24.0.** The `controlnet`, `sdxl`, `qwen` and `default` profiles
|
||||
> were removed, and a retired name is now rejected at parse time rather than routed
|
||||
> onward. `--pipeline` accepts only `qwen-zimage` (the default) and `sdxl-zimage`;
|
||||
> both are CUDA-only, both condition their global stage on a canny edge map, and both
|
||||
> run the same Z-Image face stage. The bullets below are a record of what was measured
|
||||
> on the removed profiles, kept because the oracle verdicts are still the evidence the
|
||||
> current design rests on.
|
||||
|
||||
- **Text + structure:** `--pipeline controlnet` (SDXL img2img + a canny ControlNet) is
|
||||
**THE DEFAULT pipeline since 2026-06-09** (`--pipeline default` opts down to plain
|
||||
SDXL img2img for inputs without text/faces). It conditions the regeneration on the
|
||||
@@ -428,21 +499,21 @@ conditioning, never by copying original pixels.**
|
||||
measured at. See §5.2.**
|
||||
- **Highest-fidelity CUDA option:** `--pipeline qwen-zimage` is the recommended
|
||||
quality mode when preserving face identity matters more than latency, model size,
|
||||
and GPU cost. ControlNet remains the default because it is much cheaper and supports
|
||||
and GPU cost. ControlNet was then the default, because it was much cheaper and ran on
|
||||
CUDA, XPU, MPS, and CPU, but canny conditioning preserves edges rather than identity.
|
||||
On two direct upstream comparisons, `qwen-zimage` retained substantially more
|
||||
ArcFace identity than polished ControlNet. On 2026-07-25 the exact six-output
|
||||
`visible -> qwen-zimage -> metadata` candidate was negative in the corresponding
|
||||
OpenAI and Gemini oracles. This is a quality recommendation for the measured content,
|
||||
not broad removal certification; very small text can still degrade.
|
||||
See `docs/known-limitations.md` for the metrics, runtime, and validation scope.
|
||||
- **Face identity:** canny holds face *structure* but not *identity*. The standard
|
||||
SDXL and ControlNet profiles do not run a separate face-restoration option.
|
||||
Earlier GFPGAN, PhotoMaker, and FaceID experiments were removed after they
|
||||
degraded identity or risked reintroducing source pixels. The separate
|
||||
`qwen-zimage` profile now provides the only shipped face-specific stage:
|
||||
YuNet and SAM locate faces, then Z-Image regenerates the selected original
|
||||
face crops before a feathered composite. See
|
||||
See `docs/qwen-improvement-research.md` for the identity and text metrics and the
|
||||
validation scope of those comparisons.
|
||||
- **Face identity:** canny holds face *structure* but not *identity*. The removed
|
||||
SDXL and ControlNet profiles did not run a separate face-restoration stage, and
|
||||
earlier GFPGAN, PhotoMaker, and FaceID experiments were dropped after they
|
||||
degraded identity or risked reintroducing source pixels. Both shipped profiles
|
||||
now run the same face-specific stage: YuNet and SAM locate faces, then Z-Image
|
||||
regenerates the selected original face crops before a feathered composite. See
|
||||
`docs/controlnet-removal-pipeline-research.md` for the historical experiments.
|
||||
|
||||
### 5.2 Strength setting
|
||||
@@ -459,13 +530,20 @@ study (section 2.2) gives empirical floors:
|
||||
measured on; treat it as a lower bound, not a guarantee, and raise + oracle-recheck
|
||||
per content type (see §5.1 controlnet bullet).
|
||||
- **Google (capped 1536)**: 0.15 (n=4); 0.05 and 0.10 do not clear.
|
||||
- **Google native 2816**: not locally measured; likely needs >= 0.30 (vendor +
|
||||
resolution stack). Use a GPU or `--max-resolution 1536`.
|
||||
- **Google native 2816**: 0.15 clears (n=2, deployed controlnet worker, 2026-06-14) --
|
||||
the same rung as capped 1536, so no resolution penalty was observed.
|
||||
|
||||
The default is **vendor-adaptive** (`watermark_profiles.resolve_strength` +
|
||||
`vendor_for_strength`): the tool reads the C2PA issuer on the original input and picks
|
||||
> **Superseded in 0.24.0.** The `sdxl`, `controlnet`, `qwen` and `default` profiles
|
||||
> were removed, and `OPENAI_STRENGTH` / `GEMINI_STRENGTH` / `UNKNOWN_STRENGTH` went
|
||||
> with them. Everything from here to the end of this section is a record of what was
|
||||
> measured on those profiles, kept because the oracle verdicts are still the evidence
|
||||
> base. For the strength policy in force now see `module-internals.md`: `qwen-zimage`
|
||||
> uses `resolution_adaptive_denoise`, `sdxl-zimage` a flat vendor ladder.
|
||||
|
||||
The default was **vendor-adaptive** (`watermark_profiles.resolve_strength` +
|
||||
`vendor_for_strength`): the tool read the C2PA issuer on the original input and picked
|
||||
`OPENAI_STRENGTH` 0.10 / `GEMINI_STRENGTH` 0.15 / `UNKNOWN_STRENGTH` 0.15 **(LOWERED
|
||||
2026-06-14 from the 2026-06-04 cert floors 0.20/0.30/0.30)**. **The SAME ladder applies
|
||||
2026-06-14 from the 2026-06-04 cert floors 0.20/0.30/0.30)**. **The SAME ladder applied
|
||||
to both pipelines** (`sdxl` and `controlnet`). The 2026-06-14 re-test on the deployed
|
||||
Modal controlnet worker (v0.10.0) cleared SynthID on the oracle at OpenAI 0.10 (2
|
||||
photoreal) and Google 0.15 (2 NATIVE 2816x1536, contradicting the "native >= 0.30" guess
|
||||
@@ -486,12 +564,56 @@ for `sdxl`, not a separate certification (no local SynthID detector to self-veri
|
||||
The higher strength costs little quality where it matters, because `controlnet` is now
|
||||
the default pipeline, so `sdxl` is reached only via an explicit `--pipeline sdxl` (a
|
||||
deliberate opt-down), where over-regeneration has no faces/text to damage.
|
||||
This uses the vendor signal we DO have locally (the C2PA SynthID proxy) to avoid the
|
||||
This uses the vendor signal we DO have locally (the C2PA SynthID provenance) to avoid the
|
||||
overkill of a single high default on OpenAI images, without needing a local pixel
|
||||
detector. An explicit `--strength` always wins. If the watermark still survives (e.g. a
|
||||
large native Gemini beyond the capped-1536 validation), raise toward 0.35-0.40 (0.40
|
||||
visibly corrupts dense text), using the lowest value that reads clean on the oracle.
|
||||
|
||||
**qwen-zimage global denoise, Gemini boundary bracketed (2026-08-02).** The profile does
|
||||
not use the vendor ladder above; `resolution_adaptive_denoise` maps megapixels onto
|
||||
roughly 0.084 (sub-0.3 MP) to 0.154 (>= 3.7 MP). A ladder on one native 2816x1536 Gemini
|
||||
original, seed 0, everything else at profile defaults, verified through the Gemini app:
|
||||
|
||||
| global denoise | Gemini app | whole-image PSNR | face-box PSNR | edge IoU |
|
||||
|---|---|---|---|---|
|
||||
| 0.154 (profile top) | clean | 24.72 | 31.19 | 0.188 |
|
||||
| 0.12 | clean | 25.65 | 31.82 | 0.202 |
|
||||
| 0.10 | **clean** | 26.26 | 32.17 | 0.212 |
|
||||
| 0.08 | **SynthID FOUND** | 26.95 | 32.51 | 0.227 |
|
||||
|
||||
So the boundary sits between 0.08 and 0.10 for this image, and the profile's shipped
|
||||
0.154 carries roughly half a rung more strength than that content needed. Fidelity rises
|
||||
monotonically all the way down - dropping to 0.10 buys **+1.54 dB whole-image and
|
||||
+0.98 dB inside the face boxes** - which is exactly why the temptation is to move the
|
||||
ceiling, and exactly why one fixture is not enough to do it.
|
||||
|
||||
Two constraints on reading this:
|
||||
|
||||
- **It brackets, it does not calibrate.** One image, one seed. Shipping the lowest clean
|
||||
rung means shipping at the measured cliff edge; another sample, seed, or content class
|
||||
can sit on the other side of it. Note §5.2's flat-graphic hard cases were not in this
|
||||
set at all.
|
||||
- **The bottom of the curve was the untested end. It has now been measured, and it
|
||||
holds.** Every Gemini oracle fixture is 2816x1536, so the Google-side certification
|
||||
only ever covered 0.154, while `resolution_adaptive_denoise` sends sub-1 MP images to
|
||||
0.084-0.094 - at or below the rung that failed at 4.33 MP. Downscaling a Gemini
|
||||
original (valid test material: SynthID survives it by design) and running the deployed
|
||||
worker on it gives, through the Gemini app:
|
||||
|
||||
| processing size | profile denoise | Gemini app |
|
||||
|---|---|---|
|
||||
| 1024x559 (0.57 MP) | 0.0896 | clean |
|
||||
| 1600x873 (1.40 MP) | 0.1066 | clean |
|
||||
|
||||
So production's own low end clears Google, and the "small images are under-processed"
|
||||
failure mode is ruled out at these two sizes. **Read this as validation of the shipped
|
||||
curve, not as proof that the boundary moves with resolution.** 0.0896 sits inside the
|
||||
untested gap at 4.33 MP, where only 0.08 (found) and 0.10 (clean) were probed, so it
|
||||
may well clear at both sizes. The direction of any resolution dependence remains
|
||||
unproven (§5.5). Also note these are downscales of a 2816x1536 original rather than
|
||||
natively small Gemini outputs, which have still never been tested.
|
||||
|
||||
### 5.3 Test methodology
|
||||
|
||||
- **GitHub-recompressed JPEGs from issue attachments are valid SynthID test
|
||||
@@ -501,7 +623,7 @@ visibly corrupts dense text), using the lowest value that reads clean on the ora
|
||||
- **The correct oracle for OpenAI images is openai.com/verify**, not the Gemini
|
||||
app. The two oracles detect different payloads.
|
||||
- **A quiet `identify` output after processing is not proof of removal.** It
|
||||
means the metadata proxy is gone. The pixel watermark state is unknown without
|
||||
means the provenance evidence is gone. The pixel watermark state is unknown without
|
||||
an oracle check.
|
||||
- **After removal, the output may carry forensic artifacts** detectable by an
|
||||
independent classifier even if the vendor oracle reads negative. Defeating the
|
||||
@@ -571,7 +693,7 @@ it; (3) **historical engineering conclusion:** this dated run argued for a
|
||||
higher ControlNet strength than the then-current default. That proposal was
|
||||
later superseded. The current resolver intentionally shares the 0.10/0.15
|
||||
ladder between SDXL and ControlNet and uses a separate Qwen ladder; see
|
||||
`noai/watermark_profiles.py`.
|
||||
`_internal/watermark_profiles.py`.
|
||||
Source images are private (faces / product shots), not committed; reproduce on any
|
||||
photoreal + flat-graphic gpt-image pair, varying the seed, and re-checking the
|
||||
oracle.
|
||||
@@ -590,24 +712,32 @@ openai.com/verify):
|
||||
weight 0.5 blends ~half its pixels back, re-introducing SynthID into the
|
||||
composited face over the diffusion-cleaned result (see §5.1 face-identity bullet).
|
||||
- (Side note: reducing the processing resolution does NOT weaken SynthID -- it is
|
||||
robust to downscaling by design, and the study's resolution trend says LOWER
|
||||
processing res needs LESS strength, so 1024 was never the wall.)
|
||||
robust to downscaling by design, so 1024 was never the wall. Whether a lower
|
||||
processing resolution then needs more or less removal strength is NOT established;
|
||||
see the note below.)
|
||||
|
||||
**Historical controlnet certification, superseded by the current vendor-adaptive
|
||||
defaults (isolated GPU sweep + oracle,
|
||||
restore OFF, <= 1536, each vendor on its own oracle):** OpenAI **0.20** (2 photoreal x
|
||||
seed {1,2,3} = 6/6 clean; the 0.15-flipper is seed-robust at 0.20) and Gemini **0.30**
|
||||
(0.20 detected -> 0.30 clean on 2/2 seeds). OpenAI 0.20 transfers to prod
|
||||
(resolution-independent); Gemini 0.30 holds only <= 1536 -- Gemini is
|
||||
resolution-sensitive, so a native-resolution deployment should cap Gemini <= 1536 + use 0.30 or
|
||||
native-calibrate (~0.35+). See `docs/controlnet-removal-pipeline-research.md` for the
|
||||
table.
|
||||
(0.20 detected -> 0.30 clean on 2/2 seeds). Both were measured at <= 1536 only. See
|
||||
`docs/controlnet-removal-pipeline-research.md` for the table.
|
||||
|
||||
**Whether Gemini removal is resolution-sensitive is UNPROVEN, in either direction.**
|
||||
This document previously asserted it was, and recommended capping Gemini at 1536 with
|
||||
0.30 or "native-calibrating" to ~0.35+. Nothing measured that. The one relevant
|
||||
measurement points the other way: the 2026-06-14 deployed-worker re-test cleared Gemini
|
||||
at **0.15 on two NATIVE 2816x1536 images**, the same rung as capped 1536. So there is no
|
||||
observed native-resolution penalty, and no observed benefit either -- the low-resolution
|
||||
end has simply never been through the Gemini oracle on any pipeline. Do not reason from
|
||||
a resolution trend here; measure it.
|
||||
|
||||
**Current implication:** the old floor table remains evidence about the dated
|
||||
test set, not the current resolver. The shipped SDXL and ControlNet defaults are
|
||||
defined in `watermark_profiles.py`, and face restoration is available only
|
||||
through the separate `qwen-zimage` profile. Removal near a threshold remains
|
||||
seed dependent, so reproducible verification requires a fixed seed.
|
||||
test set, not the current resolver. The SDXL and ControlNet profiles it measured
|
||||
no longer exist; the shipped defaults are defined in `watermark_profiles.py`, and
|
||||
both surviving profiles run face repair as a built-in second stage rather than as
|
||||
an optional restore. Removal near a threshold remains seed dependent, so
|
||||
reproducible verification requires a fixed seed.
|
||||
|
||||
---
|
||||
|
||||
@@ -628,3 +758,26 @@ seed dependent, so reproducible verification requires a fixed seed.
|
||||
|
||||
5. OpenAI. **Verify tool for AI-generated images.** openai.com/research/verify.
|
||||
Accessed 2026-05-31.
|
||||
|
||||
6. Google. **Verify AI-generated images, videos, and audio.**
|
||||
https://support.google.com/gemini/answer/16722517
|
||||
|
||||
7. Zhao et al. (2024). **Invisible Image Watermarks Are Provably Removable
|
||||
Using Generative AI.** NeurIPS 2024, arXiv:2306.01953.
|
||||
https://arxiv.org/abs/2306.01953
|
||||
|
||||
8. Jiang et al. (2025). **VideoMarkBench: Benchmarking Robustness of Video
|
||||
Watermarking.** arXiv:2505.21620.
|
||||
https://arxiv.org/abs/2505.21620
|
||||
|
||||
**Google floor on qwen-zimage (0.27.2, 2026-08-19).** The bracket above did not
|
||||
survive the full production path: on the 4.33 MP CJK-sign fixture (visible-stage
|
||||
sparkle removal -> qwen-zimage seed 0 at the curve's 0.154 top -> resize-back ->
|
||||
metadata strip), the Gemini verifier detected SynthID x3 with a valid
|
||||
pixel-identical stripped control in the same session. Google-provenance content
|
||||
now takes a flat `QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.30` floor instead of the area
|
||||
curve - 0.30 anchors measured clean in Gemini on two fixtures (CJK sign +
|
||||
18-face) at 3/3 checks across two work accounts, and stayed clean with the
|
||||
vae-glyphs donor layer on top; openai/unknown content keeps the resolution
|
||||
curve. An explicit strength still wins. Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-18/` (the failing re-baseline).
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
> Research archive. This page records evaluated ideas, including rejected
|
||||
> prototypes. Current behavior is documented in the user guides and source.
|
||||
> The `text_protector.py` module it describes was removed from the library
|
||||
> afterwards.
|
||||
|
||||
Date: 2026-05-29. Source: a deep-research run (104 agents, 5 search angles, sources
|
||||
fetched and 3-vote adversarially verified). Not committed automatically — saved as a
|
||||
@@ -141,3 +143,229 @@ Deep-research workflow run `wf_118b9a03-3eb` (2026-05-29). Findings adversariall
|
||||
(2/3 refutes required to kill a claim). This note records research only; no code change is
|
||||
implied until a prototype validates fidelity and the SynthID-scrub guarantee on the
|
||||
restored output.
|
||||
|
||||
## 2026-08-13 OCR plus LaMa prototype
|
||||
|
||||
A lightweight version of the recommended two-stage architecture was tested on
|
||||
the three tracked text fixtures. It used the already-clean Qwen result as the
|
||||
base, recognized the source text, removed source and Qwen glyph contours with
|
||||
LaMa, and rendered the recognized strings as fresh pixels. No source pixels
|
||||
were pasted back.
|
||||
|
||||
The result validates only part of the hypothesis. Character-weighted CER on the
|
||||
two multilingual posters fell from 0.338/0.305 to 0.007/0.007, and OpenAI Verify
|
||||
remained clean for both matched Qwen and restored pairs. However, replacement
|
||||
fonts changed the design, whole-image LPIPS worsened by 0.067 on each poster,
|
||||
and the light poster retained a shadow in one small English line. On the Chinese
|
||||
sign, punctuation errors moved CER from 0.074 to 0.111. The Google verifier run
|
||||
was inconclusive because the available account detected both the restored image
|
||||
and the byte-identical Qwen control that a previous account had called clean.
|
||||
|
||||
The experiment supports a proper glyph-conditioned restorer, not shipping the
|
||||
plain OCR/system-font compositor. Exact hashes, metrics, masks, and provider
|
||||
verdicts are tracked in
|
||||
[`data/evaluations/fidelity/text-restoration-2026-08-13.csv`](../data/evaluations/fidelity/text-restoration-2026-08-13.csv).
|
||||
|
||||
## 2026-08-13 AnyText2 follow-up
|
||||
|
||||
The official AnyText2 SD1.5 checkpoint was then tested as the glyph-conditioned
|
||||
restorer. Its ModelScope entry and code are Apache 2.0, and the downloaded
|
||||
checkpoint matched the published SHA-256. An official blackboard edit completed
|
||||
successfully before the fixture run, establishing that the runtime reproduced
|
||||
the model rather than silently exercising a fallback.
|
||||
|
||||
On the Chinese sign, a three-line local edit over the exact Qwen output scored
|
||||
CER 0.185 under the standard detector. Font mimic from the source glyph masks
|
||||
scored 0.222. Whole-image LPIPS changed from 0.289 to 0.338 and 0.345 respectively.
|
||||
A padded crop-recognition check showed why the comparison matters: Qwen contained
|
||||
the full correct text and scored 0.000, while default AnyText2 still scored 0.074
|
||||
with two substituted characters and mimic remained at 0.222. The provider oracle
|
||||
was deliberately not called because these variants had already failed the
|
||||
content-fidelity gate.
|
||||
|
||||
AnyText2 is therefore not the missing production restorer in its published form.
|
||||
The negative result is narrower than the model family: only the SD1.5 checkpoint
|
||||
is public, while AnyText2XL remains listed as future work. The current wrapper
|
||||
also truncates each quoted text line to 20 characters, which prevents a direct
|
||||
test of several tracked English and Russian poster lines. Exact model provenance,
|
||||
output hashes, and metrics are recorded in
|
||||
[`data/evaluations/fidelity/anytext2-restoration-2026-08-13.csv`](../data/evaluations/fidelity/anytext2-restoration-2026-08-13.csv).
|
||||
|
||||
The same cross-check exposes a measurement bug in the earlier lightweight
|
||||
restoration result. Paddle detection made tight boxes that omitted the final
|
||||
Chinese full stop from two lines, producing Qwen CER 0.074 although the pixels
|
||||
and padded recognition were correct. Adding 16-32 pixels of horizontal context
|
||||
restored every punctuation mark. A deterministic rerender using the full OCR
|
||||
strings and the closest of nine mask-scored CJK fonts also reached detector CER
|
||||
0.000, but its heavier Hiragino Sans GB W6 glyphs raised LPIPS from 0.289 to 0.327.
|
||||
The actionable design is selective restoration: compare padded source and output
|
||||
recognition first, and preserve the Qwen output when they already match.
|
||||
|
||||
That policy was tested on the two multilingual posters. A manual prototype kept
|
||||
the lines whose padded source and Qwen recognition agreed and rerendered only the
|
||||
changed lines. The edited mask fell from 15.4%/17.0% in the full compositor to
|
||||
5.5%/5.5%. On a single Paddle `en+ru+ch` route, CER changed from 0.378/0.413 for
|
||||
Qwen to 0.101/0.112, while LPIPS was only 0.120/0.103 rather than the full
|
||||
compositor's 0.174/0.162. The different OCR route is intentional and all three
|
||||
variants were remeasured through it; these CER values are not directly
|
||||
comparable with the earlier Vision/Paddle hybrid results.
|
||||
|
||||
Both selective outputs returned `No OpenAI signals detected` in OpenAI Verify,
|
||||
and the original light poster returned `Generated with OpenAI tools` immediately
|
||||
afterward as the positive control. The dark poster was visually clean. The light
|
||||
poster still showed one local old-text shadow, so mask construction remains the
|
||||
blocking defect. Exact hashes, metrics, mask fractions, and oracle controls are
|
||||
in
|
||||
[`data/evaluations/fidelity/selective-text-restoration-2026-08-13.csv`](../data/evaluations/fidelity/selective-text-restoration-2026-08-13.csv).
|
||||
|
||||
Uniformly expanding every selected glyph mask by two pixels removed that light
|
||||
poster shadow and moved CER from 0.101 to 0.081 without a meaningful LPIPS cost.
|
||||
It also expanded the edited area from 5.5% to 7.0%, and the identical rule made
|
||||
the dark poster worse at CER 0.147 rather than 0.112. Those follow-up bytes have
|
||||
not been oracle-checked. The next prototype should detect residual source glyphs
|
||||
per line and expand only the failing component, rather than introducing another
|
||||
poster-wide tuning constant.
|
||||
|
||||
The manual selection was then replaced with per-line padded recognition. A line
|
||||
is left untouched only when the source recognizer is sufficiently consistent
|
||||
with the verified line and normalized source/candidate recognition agree
|
||||
exactly. The automatic rule reproduced the manual selection on the dark poster
|
||||
and correctly kept one additional large Russian line on the light poster. It
|
||||
reduced that poster's effective edited area from 5.5% to 4.1%.
|
||||
|
||||
Mask cleanup became a measured second pass rather than a global dilation. After
|
||||
the first LaMa erase, the prototype finds contrast that remains specifically at
|
||||
the original glyph positions, dilates only that residual, and erases it before
|
||||
fresh text is drawn. This removed the visible double glyphs from both posters.
|
||||
The automatic outputs scored LPIPS 0.113/0.104 and detector CER 0.123/0.119, with
|
||||
effective edited fractions of 4.1%/5.7%. The light aggregate CER is pessimistic:
|
||||
the page detector fragments its untouched Cyrillic line into Latin-like pieces,
|
||||
while the padded Russian recognizer returns the exact expected text.
|
||||
|
||||
OpenAI Verify returned `No OpenAI signals detected` for both new hashes and then
|
||||
`Generated with OpenAI tools` for the original light poster in the same browser
|
||||
sequence. Reproducing the prototype from the tracked script exposed harmless
|
||||
ONNX rounding on the light image: 386 pixels differed by at most one channel
|
||||
value from the initially checked hash. The exact tracked output hash
|
||||
`900def5a...` was therefore submitted separately, returned clean, and was
|
||||
followed immediately by another positive source result. The dark tracked output
|
||||
was byte-identical to the initially checked hash.
|
||||
|
||||
The Chinese sign provides a no-edit control for the selector. Detector boxes for
|
||||
its three large lines overlap vertically, so Latin-style vertical padding
|
||||
incorrectly mixed neighboring lines. Horizontal-only CJK padding captured the
|
||||
terminal punctuation without mixing neighbors and made source and candidate
|
||||
recognition agree exactly on all three lines.
|
||||
The script then produced a zero mask and copied the Qwen candidate byte for byte.
|
||||
This prevents the heavier-font regression seen in the earlier CJK compositor,
|
||||
but does not resolve the unavailable Google-negative oracle control.
|
||||
|
||||
The result clears the measured OpenAI gate without a manual line selection, but
|
||||
is not a production design yet. It still assumes verified source strings and
|
||||
line boxes, relies on macOS system fonts, and needs evaluation on uncurated
|
||||
layouts, rotated text, false OCR matches, automatic line-box discovery, and a
|
||||
reproducible Google-negative control.
|
||||
|
||||
The later source-silhouette experiment removed the font lookup but did not meet
|
||||
the actual visual requirement. Native-size review found changed stroke weight,
|
||||
color variation, edge antialiasing, and decorative details even where crop OCR
|
||||
was exact. A narrower `vae-glyphs` experiment uses the silhouette only as a
|
||||
support mask: exact pixels come from a Qwen-VAE reconstruction, while a fresh
|
||||
flat-color silhouette supplies only the outer edge beneath that core. Across 48
|
||||
typography cases (548 annotated lines), this improved glyph-edge MAE in 48/48
|
||||
and text-box SSIM in 47/48. The median values moved from 37.59 to 32.42 and from
|
||||
0.854 to 0.914 respectively. One separate untracked core-only output with a
|
||||
1.43% donor area returned `No OpenAI signals detected`; the tracked script then
|
||||
reproduced those exact bytes. A 0.5-pixel feathered sibling with 2.75% nonzero
|
||||
donor area improved mean text-box SSIM from 0.957 to 0.960, with a 0.918 minimum
|
||||
across 15 verified lines, while full-image RGB SSIM reached 0.864. Crop OCR
|
||||
recovered from 7/15 exact lines on the raw pass to 14/15, equal to the source's
|
||||
own recognizer floor. Whole-image LPIPS was 0.082, but only 0.108% of pixels
|
||||
remained exactly equal to the source and the detected face's Laplacian-variance
|
||||
ratio was 0.670, confirming residual global smoothing. A same-session
|
||||
OpenAI Verify sequence detected the exact source control in 1/1 check, returned
|
||||
clean for the raw global pass and clean-fidelity anchor in 1/1 each, and returned
|
||||
clean for the feathered output in 3/3 repeats. The tracked script reproduced
|
||||
that feathered file byte for byte. This proves two materially better exact-output
|
||||
Pareto points, not a general operating range. The 48 matrix outputs and other
|
||||
mask sizes remain oracle-unverified.
|
||||
|
||||
The first Google oracle sample did not transfer. For the synthetic CJK sign,
|
||||
two separate work accounts detected SynthID in both the resaved positive source
|
||||
control and the exact Qwen-VAE donor candidate. The candidate's text-box SSIM
|
||||
improved from 0.791 to 0.835 and its glyph-edge MAE from 35.48 to 22.49, but its
|
||||
3.67% donor layer did not create a clean operating point. The Qwen silhouette
|
||||
base was already detected, as was the earlier original-sign Qwen baseline in the
|
||||
same account. The measured cause is therefore insufficient global Google
|
||||
regeneration, not a demonstrated watermark regression from the text donor.
|
||||
Google needs a stronger oracle-clean global anchor before the narrow donor can
|
||||
be evaluated there.
|
||||
|
||||
A targeted follow-up supplied that missing anchor for one CJK case. An explicit
|
||||
Qwen global pass at strength 0.30 returned no reliable SynthID signal in Gemini,
|
||||
while the matched source control was detected. Applying the 0.5-pixel
|
||||
`vae-glyphs` layer to that clean anchor changed 1.17% of the image and raised
|
||||
mean text-box SSIM from 0.765 to 0.964 (minimum 0.963), with mean text-box MAE
|
||||
falling from 20.74 to 5.38. The post-donor output then returned clean in 3/3
|
||||
Gemini checks across two work accounts. A separate 18-face fixture with four
|
||||
tiny UI-text lines also had a clean strength-0.30 anchor; its donor layer changed
|
||||
0.70% of the image, raised mean text-box SSIM from 0.617 to 0.976 (minimum
|
||||
0.972), and returned clean in 3/3 checks across the same two accounts. One
|
||||
quota-exceeded response is excluded from both pass rates. These are two exact
|
||||
oracle-certified outputs, not a general operating range: other layouts, masks,
|
||||
strengths, seeds, and donor areas remain unverified.
|
||||
|
||||
An automatic-box follow-up merged Paddle word detections by vertical overlap.
|
||||
It found exactly 20/20 poster lines and 3/3 sign lines, with mean IoU 0.857,
|
||||
0.847, and 1.000 against the verified boxes. That structural match was not
|
||||
sufficient: reusing the annotation crop padding changed recognition decisions,
|
||||
reduced preserved dark-poster lines from 7 to 4, and expanded its edited
|
||||
fraction from 5.7% to 11.2%.
|
||||
|
||||
A recognition-only sweep isolated the crop sensitivity. Limiting vertical
|
||||
padding around detector boxes to 8-12% of line height reproduced the verified-
|
||||
box decision vectors on both posters; 10% was used for a full follow-up. It kept
|
||||
8/20 and 7/20 lines with edited fractions of 4.1% and 5.7%, and visual inspection
|
||||
found no double glyphs. Whole-image LPIPS remained 0.113/0.104, but detector CER
|
||||
was 0.127/0.154 instead of 0.123/0.119. The dark regression failed the fidelity
|
||||
gate, so these hashes were not submitted to the provider oracle. The opt-in flag
|
||||
remains only to reproduce the negative result. Count, IoU, and matching decision
|
||||
vectors are therefore insufficient gates for automatic boxes; the next design
|
||||
needs box rectification or recognition stability under crop jitter before it
|
||||
can replace verified geometry.
|
||||
|
||||
Removing verified strings was tested separately with an annotation-seed dry
|
||||
run. It detects boxes, chooses `en`, `ru`, or `ch` from Unicode script, and
|
||||
accepts a draft only when three crop paddings normalize identically and every
|
||||
confidence is at least 0.85. One execution proposed 20 and 18 poster lines, but
|
||||
exact-text precision against the held-out annotations was only 90.0% and 94.4%.
|
||||
The stable errors were punctuation: one lost English comma and an ideographic
|
||||
comma consistently replaced by ASCII. Thus crop agreement and confidence do
|
||||
not verify literal text. The helper is retained only to seed annotations; every
|
||||
proposal needs manual review before it can enter the restoration compositor.
|
||||
|
||||
## Production-path anchor leak and fix (2026-08-19)
|
||||
|
||||
The engine `text_manifest=` path (unlike the research compositor above) blends a
|
||||
15% Qwen-VAE fidelity anchor across the WHOLE frame before glyph restoration.
|
||||
Certified through the official OpenAI Content Provenance API on the exact
|
||||
production path (qwen-zimage, seed 0, no strength override, untiled, LANCZOS
|
||||
resize-back, metadata strip; library 0.27.0 @ 50d9f448):
|
||||
|
||||
- controls (pixel-identical metadata-stripped sources): `synthid: detected` -
|
||||
the pixel channel was proven live before every session;
|
||||
- base outputs: `clean` x6 (two 20-line posters, 3 attempts each);
|
||||
- text-restored WITH the anchor: `synthid: detected` x6, deterministic;
|
||||
- text-restored with `FIDELITY_BLEND_ALPHA = 0` (runtime ablation, base bytes
|
||||
identical between runs): `clean` x6.
|
||||
|
||||
The anchor was therefore the leak channel: a 15% global donor admixture over a
|
||||
scrubbed frame returns enough OpenAI SynthID for the provider detector. Fidelity
|
||||
cost of dropping it (source-referenced text-box MAE): 11.60->11.72, 7.79->7.86,
|
||||
and 7.57->8.13 across the two posters and the CJK sign - well under one point
|
||||
against base MAEs of 24.7/16.8/10.2. Since 0.27.1 the anchor is off by default
|
||||
(`fidelity_anchor` parameter on `remove_watermark` / `InvisibleOptions` /
|
||||
`--fidelity-anchor`); Google rows were not re-queried because the default
|
||||
strength leaves that fixture's base detected (the vendor-strength lane tracks
|
||||
that separately). Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-19-A1/` (sha256-stamped rows).
|
||||
+113
-16
@@ -39,6 +39,15 @@ confidence), reviewed once, then re-baselined. Lost detections are the alarm.
|
||||
|
||||
Implemented as `scripts/sidecar_regression.py` (resumable, ~1.5 h at 8 workers).
|
||||
|
||||
A 2026-08-15 metadata-only C2PA regression audit over the local historical
|
||||
corpus caught two structured-parser gaps before release: an exact Firefly claim
|
||||
generator without a repeated source type, and a Dreamina generator carried by a
|
||||
reachable ingredient under a generic update manifest. Both now have focused
|
||||
tests. The same audit confirmed that invalid hash/signature claims lose origin
|
||||
attribution without losing the C2PA inventory, and exposed the absence of
|
||||
production trust anchors in the SDK defaults. Dataset-derived counts and
|
||||
identifiers remain in the gitignored audit output.
|
||||
|
||||
#### Local run protocol
|
||||
|
||||
Re-run `identify` against locally recorded sidecars, classify losses separately from intended new detections, and keep generated reports under `.local-eval/`. Do not commit dataset-derived counts or identifiers.
|
||||
@@ -72,8 +81,10 @@ Run detection, removal, and re-detection over a representative local set. Confir
|
||||
|
||||
### A5. Contract sweep across every parameter choice
|
||||
|
||||
`scripts/smoke_matrix.py` (exists, 68 rows, 0 skipped with `--diffusion`) covers every
|
||||
choice-valued flag on fixtures. Extend from fixtures to a stratified corpus slice
|
||||
`scripts/smoke_matrix.py` covers every choice-valued flag on fixtures. Its knob rows
|
||||
were rewritten when the CPU/MPS profiles and the ESRGAN chain were removed: most now
|
||||
assert a knob is REJECTED, and the accepted-knob rows skip without a CUDA device, so
|
||||
the row count is host-dependent rather than the fixed 68 recorded here before. Extend from fixtures to a stratified corpus slice
|
||||
(~500 images spanning format x provenance x aspect ratio), asserting exit-code semantics
|
||||
rather than just absence of crash.
|
||||
|
||||
@@ -84,25 +95,33 @@ sweep asserts on stderr, or the codes get split -- the latter is the better fix.
|
||||
#### Coverage method
|
||||
|
||||
Compare the flags and values exercised by the matrix against the options declared by
|
||||
the CLI. Include optional backends, batch modes, tiling, region-targeted composition,
|
||||
the ESRGAN upscaler, and the ffmpeg audio/video strip. The gap to find is not only
|
||||
the CLI. Include optional backends, batch modes, tiling, and the ffmpeg audio/video
|
||||
strip. (Region-targeted diffusion composition was on this list until the parameter
|
||||
was deleted: nothing but a caller-less convenience wrapper ever reached it, and the
|
||||
user-facing region path is `erase`, which inpaints instead.) The gap to find is not only
|
||||
"logic untested" but
|
||||
"never executed on real data", which is precisely what this campaign is for.
|
||||
|
||||
#### Bug found by the extension: `--steps` below ~7 crashes inside torch
|
||||
#### Bug found by the extension: `--steps` below ~7 crashed inside torch
|
||||
|
||||
Effective timesteps are `int(steps * strength)`. At the vendor-adaptive default strength
|
||||
(0.15, or 0.10 for OpenAI) any `--steps` under 7 rounds to **zero**, and the pipeline dies
|
||||
with a raw traceback:
|
||||
**Fixed by deletion.** `--steps` no longer exists, on the CLI or in the Python API,
|
||||
so this class of failure is unreachable. Kept as a record of why.
|
||||
|
||||
Effective timesteps were `int(steps * strength)`. At the vendor-adaptive default
|
||||
strength (0.15, or 0.10 for OpenAI) any `--steps` under 7 rounded to **zero**, and the
|
||||
pipeline died with a raw traceback:
|
||||
|
||||
```
|
||||
$ remove-ai-watermarks invisible img.png --steps 5
|
||||
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1, 1, 512]
|
||||
```
|
||||
|
||||
Fully valid CLI arguments, no special flags, no `--force`. The value is accepted, the
|
||||
crash is a torch internal, and nothing tells the user that steps and strength interact.
|
||||
Fix is either a clamp to >=1 effective step or an up-front validation naming both values.
|
||||
Fully valid CLI arguments, no special flags, no `--force`. The value was accepted, the
|
||||
crash was a torch internal, and nothing told the user that steps and strength interact.
|
||||
The considered fixes were a clamp to >=1 effective step or an up-front validation
|
||||
naming both values; what shipped instead is that each stage owns its own distilled
|
||||
schedule and no caller can set it. The general lesson stands: a knob whose valid range
|
||||
depends on another knob's value needs the interaction validated where both are known,
|
||||
or it needs to not be a knob.
|
||||
|
||||
Method note: the first run of the knob rows failed 12 times with this identical error,
|
||||
which read like twelve broken features. It was one bad harness parameter (`--steps 4`)
|
||||
@@ -167,10 +186,14 @@ report recall from the detector-sampled set.
|
||||
|
||||
## Tier D -- external oracles (manual, not automatable here)
|
||||
|
||||
SynthID removal cannot be verified locally by design -- no public decoder exists. Each
|
||||
vendor has its own oracle and it covers only that vendor's content: `openai.com/verify` for
|
||||
OpenAI (more accessible, the automation candidate), the Gemini app for Google (manual,
|
||||
rate-limited). A quiet metadata proxy is **not** proof the pixel watermark is gone.
|
||||
Proprietary watermark removal cannot be verified locally by design -- no public decoder
|
||||
exists. Each vendor has its own oracle and it covers only that vendor's content:
|
||||
`openai.com/verify` for OpenAI, the Gemini app for Google, and Microsoft's
|
||||
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
|
||||
for InvisMark. The Microsoft API reports `Watermark` and `C2PA` separately. It therefore
|
||||
needs a pixel-identical metadata-stripped control: the control must lose `C2PA` while
|
||||
remaining `Watermark`-positive before a candidate's negative result can be attributed to
|
||||
pixel regeneration. A quiet metadata proxy is **not** proof the pixel watermark is gone.
|
||||
|
||||
Scope honestly: this tier certifies strength floors on a handful of images per vendor, and
|
||||
that is all it can do. See `docs/synthid.md`.
|
||||
@@ -216,6 +239,43 @@ for the wrong reason reads exactly like success.
|
||||
the control passes -- but that is Google's claim about their own decoder, not our
|
||||
measurement, so it is a hypothesis to test, not a reason to skip the control.
|
||||
|
||||
### D4. Video candidates require a matched transcode control
|
||||
|
||||
Video experiments add frame sampling, resizing, frame-rate conversion, and a
|
||||
final video codec around the actual attack. `scripts/video_synthid_sweep.py`
|
||||
therefore emits `control.mp4` from the same selected frames and encoder settings
|
||||
as every VAE candidate.
|
||||
|
||||
Verify the control first in a new Gemini chat by invoking `@synthid` and using
|
||||
the supported built-in content-verification question. Continue only when the
|
||||
provider oracle still detects SynthID in it. Verify each candidate in its own
|
||||
new chat. A generic response that discusses visual clues or metadata is not an
|
||||
oracle result. Do not ask the chat model to reinterpret or second-guess the
|
||||
built-in verdict; that follow-up is ordinary model reasoning. Record only the
|
||||
explicit built-in SynthID verification verdict in the generated CSV.
|
||||
|
||||
The harness shares one latent-noise field across the sequence to avoid adding
|
||||
independent frame noise. Its temporal-residual metric is a fidelity check, not a
|
||||
watermark detector.
|
||||
|
||||
The 2026-07-29 two-carrier calibration produced genuine built-in verifier
|
||||
results: both matched controls were positive, the stronger candidate
|
||||
was negative on both carriers, and a weaker candidate was negative on one. On
|
||||
2026-07-30, adversarial follow-up prompts returned `UNAVAILABLE` after asking
|
||||
ordinary Gemini to ignore and reinterpret the detector result. Those follow-ups
|
||||
were mistakenly treated as a stricter oracle; they were not detector reruns.
|
||||
The implementation remains exposed as `video invisible` and
|
||||
`remove_video_invisible`. Its oracle-certified default is the product operating
|
||||
point. A fresh control-positive, candidate-negative pair remains an optional
|
||||
per-file audit after provider changes or for unusually important files.
|
||||
|
||||
The 2026-07-31 full-clip check added a public eight-second Veo carrier. The
|
||||
source and the complete `0.10` product output were both detected, proving the
|
||||
surrounding resize / frame-rate / codec path had not silenced the oracle. The
|
||||
complete `0.15` product output was not detected, so `0.15` became the default.
|
||||
The tracked source and output hashes, fidelity metrics, and verdicts are in
|
||||
`data/evaluations/video-synthid-oracle.csv`; generated media remains untracked.
|
||||
|
||||
## Tier E -- robustness and adversarial inputs
|
||||
|
||||
Malformed and hostile inputs, including truncated files:
|
||||
@@ -277,6 +337,43 @@ separately, outside this repo.
|
||||
|
||||
Run `scripts/real_examples_e2e.py` against representative local inputs before releases that affect image handling. The script must read from `.local-eval/`, write only untracked temporary output, and report behavior without exposing dataset provenance or aggregate private measurements.
|
||||
|
||||
The cheap video seam is covered automatically by
|
||||
`TestVideoVisibleFullClip::test_removes_complete_clip_and_preserves_sequence_and_audio`.
|
||||
It constructs a full synthetic Sora-like MP4 with AAC audio and C2PA
|
||||
provenance, then exercises detection, temporal arbitration, OpenCV fill, real
|
||||
ffmpeg encoding, metadata stripping, audio stream copy, and atomic publication
|
||||
through the public API. A separately encoded clean control supplies the paired
|
||||
frame-to-frame deltas inside the filled region, so the gate detects temporal
|
||||
flicker rather than treating all output motion as an error. The default
|
||||
motion-compensated fill must score strictly below a separately encoded
|
||||
frame-local opt-out on the median paired error, while its high-percentile error
|
||||
cannot regress. The dedicated Linux CI job installs ffmpeg explicitly. Keep real-provider and learned-backend
|
||||
sequence evaluation local because those inputs or model downloads do not
|
||||
belong in the core matrix.
|
||||
|
||||
The local real-provider audit runs one complete clip for every registered video
|
||||
mark. OpenCV and MI-GAN must remove every accepted frame, leave the second-pass
|
||||
detector quiet, preserve source stream starts and duration, and copy AAC
|
||||
packets exactly when present. Run one complete LaMa clip to verify wiring and
|
||||
resource tier; its CPU throughput makes a six-provider online matrix
|
||||
counterproductive. Store generated outputs and the detailed CSV only under
|
||||
`.local-eval/`.
|
||||
|
||||
The same full-clip gate runs the public metadata-only path against its real MP4
|
||||
and verifies unchanged file size, decoded frames, stream properties, and AAC
|
||||
packets. A separate synthetic large-`mdat` test rejects full-source
|
||||
`read_bytes()`, hashes the copied media payload, and mutation-checks both C2PA
|
||||
and TC260 survival.
|
||||
|
||||
A companion full-clip VFR case alternates three frame durations, runs the
|
||||
public visible-removal API, and compares every output display timestamp to the
|
||||
source within one source time-base tick. Its source starts at a non-zero PTS,
|
||||
so the test also verifies retained video/audio stream offsets, container
|
||||
duration, and copied AAC identity. Mutations that disable the timestamped NUT
|
||||
bridge or reset its start PTS must fail this gate.
|
||||
A separate constant-rate clip with the same non-zero start guards the
|
||||
start-offset routing without relying on the VFR branch.
|
||||
|
||||
## Standing gap
|
||||
|
||||
None of this is in `maintain.sh`, and it should not all be -- the sweeps take hours. But
|
||||
|
||||
@@ -0,0 +1,607 @@
|
||||
# Video SynthID quality research (2026-08-05)
|
||||
|
||||
> Research archive. This page records experiments and decisions from the date
|
||||
> above. It may mention prototypes or defaults that were later changed. Use the
|
||||
> user guides and current source code for the supported interface.
|
||||
|
||||
Cited research behind the question **"can the video SynthID path keep removing the
|
||||
mark while giving up far less quality than the shipped 512 px / 12 fps profile?"**
|
||||
Produced by a 13-agent workflow: 6 parallel scouts (pipeline ablation, SynthID
|
||||
internals, attack literature, autoencoder landscape, perceptual masking, experiment
|
||||
design), 11 change proposals from 3 independent design angles, and 3 adversarial
|
||||
critics (signal theory, repository engineering, verifiability). No experiment was
|
||||
run against the provider oracle for this page; every claim is labeled MEASURED,
|
||||
REPORTED, or INFERRED.
|
||||
|
||||
Repository claims below were re-verified against source after the workflow
|
||||
returned: the PSNR reference frame, the audio stream copy, the missing `_fit_size`
|
||||
clamp, the absent `enable_tiling` call, the crf asymmetry, the test pins, and the
|
||||
absent `scaling_factor` key in the cached checkpoint config.
|
||||
|
||||
## Context
|
||||
|
||||
`remove_video_invisible` regenerates video pixels through `sd-vae-ft-mse`: frames
|
||||
are decimated to 12 fps, resized to a 512 px long side, encoded to latents with
|
||||
`latent_dist.mode()`, perturbed by one seeded spatial noise field shared across
|
||||
every frame, decoded, and streamed to an H.264 encoder at crf 18. A separate
|
||||
stream-copy mux re-adds source audio and strips metadata. The shipped
|
||||
`noise_std=0.15` is certified by one oracle row on one carrier.
|
||||
|
||||
The quality complaint is real: a 1080p source is delivered at roughly a quarter of
|
||||
its linear resolution and half its frame rate. This page asks what part of that
|
||||
cost is buying removal and what part is buying nothing.
|
||||
|
||||
## What the two oracle rows do and do not prove
|
||||
|
||||
`data/evaluations/video-synthid-oracle.csv`, one carrier (Veo 3 off-road,
|
||||
sha256 `79a552b9...`), one seed, one geometry:
|
||||
|
||||
| noise_std | long_side | fps | verdict | psnr_db | temporal_residual_ratio |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 0.10 | 512 | 12 | detected | 26.2932 | 1.0072 |
|
||||
| 0.15 | 512 | 12 | not_detected | 25.3911 | 1.0578 |
|
||||
|
||||
They prove exactly one thing: the envelope of 512 px, 12 fps, crf 18, and a full
|
||||
VAE round trip does **not** silence the oracle on its own. The verdict flip is
|
||||
bought by the last 0.05 of latent noise.
|
||||
|
||||
They cannot decompose that conjunction. No row varies `long_side` or `fps`, so the
|
||||
contribution of the downscale is unmeasured. The frequent reading "the 512 px
|
||||
downscale was probably doing the removal work" is not supported, and neither is its
|
||||
opposite.
|
||||
|
||||
## Measured facts from the repository
|
||||
|
||||
- **The fidelity metric is blind to the expensive steps.** `_iter_sampled_frames`
|
||||
applies `cv2.resize` at [`video_invisible.py:236`](../src/remove_ai_watermarks/video_invisible.py);
|
||||
the same generator feeds the loop at `:401-407`; the accumulator at `:429-435`
|
||||
zips those already-resized frames against `regenerated` captured at `:423`, which
|
||||
is before `frame_pipe.write` at `:430`. `psnr_db` therefore excludes the
|
||||
downscale, the decimation, and the encoder. It measures the VAE round trip plus
|
||||
latent noise at the working geometry, nothing else.
|
||||
- **Frame decimation cannot destroy the carrier.** `_iter_sampled_frames`
|
||||
(`:224-238`) is pure subset selection: the only statement inside the threshold
|
||||
test is `yield cv2.resize(...)`, with no arithmetic across frames. The
|
||||
perturbation is equally time-blind: `shared_noise.expand(latents.shape[0], -1, -1, -1)`
|
||||
at `:295` broadcasts one CHW field across the batch and never mixes across time.
|
||||
Any fps effect on the verdict is a detection-probability effect, not carrier
|
||||
destruction.
|
||||
- **`_fit_size` has no clamp at 1.0.** `scale = long_side / max(width, height)` at
|
||||
`:88` upscales any source whose long side is below 512, and both axes are floored
|
||||
independently to a multiple of 8 at `:89-96`, which introduces an anamorphic
|
||||
shift at scale 1.0 (1366x768 becomes 1360x768).
|
||||
- **`enable_tiling` is never called** anywhere in `src/`, `scripts/`, or `tests/`.
|
||||
`enable_slicing()` at `:135` already splits both encode and decode to single
|
||||
frames, so activation VRAM is set by one frame and does not scale with
|
||||
`--batch-size`.
|
||||
- **crf asymmetry:** 18 on the invisible path (`:321`, `:386`) against 14 on the
|
||||
visible path ([`video_visible.py:1343`](../src/remove_ai_watermarks/video_visible.py))
|
||||
through the same encoder.
|
||||
- **No HDR guard on the invisible path.** `_HDR_TRANSFERS` (`video_visible.py:97`)
|
||||
and the `component_depth > 8` rejection (`:1322`) exist only for visible removal.
|
||||
- **Audio is byte-copied.** `-map 0:v:0`, `-map 1:a?`, `-c copy` at
|
||||
[`video_encoding.py:413`](../src/remove_ai_watermarks/video_encoding.py).
|
||||
- **Only `noise_std` is pinned.** `tests/test_video_invisible.py:285` asserts
|
||||
`DEFAULT_VIDEO_SYNTHID_NOISE_STD == 0.15`. Neither `long_side` nor `fps` is
|
||||
pinned anywhere, so changing either breaks no test.
|
||||
- **`scaling_factor` 0.18215 is a class default, not a checkpoint fact.** The
|
||||
cached `config.json` for `stabilityai/sd-vae-ft-mse` has no `scaling_factor` key
|
||||
at all (verified locally: `_class_name`, `latent_channels`, `block_out_channels`,
|
||||
`sample_size`, and block types only). The value comes from the `AutoencoderKL`
|
||||
class default under `diffusers>=0.38.0` with no upper bound (`pyproject.toml:94`,
|
||||
`uv.lock` resolves 0.39.0), while `maintain.sh` runs `uv-outdated`. A dependency
|
||||
bump can move the certified operating point with a fully green test suite.
|
||||
- **The manifest schema cannot record what a real program needs:** no source
|
||||
geometry, no control row, no track column, no verbatim verdict, no indeterminate
|
||||
state.
|
||||
|
||||
## Measured facts from external sources
|
||||
|
||||
- Gemini video verification quota: 10 checks per rolling 24 hours, up to 5 minutes
|
||||
of video total, under 90 seconds and 100 MB per file. The verifier reports which
|
||||
parts of the video carry the mark, and it has a **third** state beyond detected
|
||||
and not detected: unclear, with documented causes including "not enough details
|
||||
to watermark".
|
||||
- The verifier scores **audio and visual tracks separately**. Google's published
|
||||
example verdict reads as SynthID detected in the audio over a time range with no
|
||||
SynthID detected in the visuals.
|
||||
- SynthID-Image (arXiv:2510.09263) is a post-hoc, model-independent pixel-space
|
||||
watermark: a separate encoder network stamps an already-decoded image rather than
|
||||
being injected into the generator's latents. Its published payload figure is 136
|
||||
bits within a 512x512 image, and its product setup runs at 1536x1536.
|
||||
|
||||
## Inferred, with the reasoning that makes them weak
|
||||
|
||||
- **The noise axis is nearly exhausted.** Fitting `MSE = A + B * noise_std^2` to the
|
||||
two measured rows gives A = 124.5 and B = 2815, so a pure round trip
|
||||
(`noise_std = 0`) lands near 27.2 dB. The entire noise budget is worth at most
|
||||
**+1.79 dB**; everything else is the autoencoder. This is a two-parameter fit to
|
||||
two points with zero degrees of freedom, and the assumptions that MSE is
|
||||
quadratic in `noise_std` and that reconstruction and noise errors are orthogonal
|
||||
are untested. One local run at `noise_std=0` replaces it with a measurement.
|
||||
- **Statistical power of the current certification.** With zero failures at n = 1,
|
||||
the exact one-sided Clopper-Pearson bound `1 - 0.05^(1/n)` is 95%: the data are
|
||||
compatible with removal failing almost always. n = 15 gives 18.1%, n = 30 gives
|
||||
9.5%.
|
||||
- **Size of the geometry prize.** At a 1280x720 source, `_fit_size(1280, 720, 512)`
|
||||
is `(512, 288)` (pinned in `tests/test_video_synthid_sweep.py:30`), so 6.25x of
|
||||
the pixels are discarded by the downscale and another 2x by decimation from a
|
||||
24 fps source. Ladder rungs: 768 gives 2.25x the current area, 1024 gives 4.0x.
|
||||
The carrier's own geometry is recorded nowhere, so this is conditional.
|
||||
- **The direction of the resolution effect is disputed, and both sides are
|
||||
inference.** Against raising it: the absolute frequency ceiling below which an f8
|
||||
VAE reconstructs faithfully is tied to the latent pitch, so 512x288 (a 64x36
|
||||
latent) preserves roughly up to 32 cycles per frame width while 1920 (a 240x135
|
||||
latent) preserves up to about 120. Raising resolution moves the carrier band out
|
||||
of the regime the decoder synthesizes and into the regime it reproduces
|
||||
faithfully, handing the detector more evidence. For raising it: Google's product
|
||||
operating point is 1536x1536, so native is closer to the distribution the
|
||||
watermark encoder targets. Note that the second argument cuts against the
|
||||
proposal rather than for it.
|
||||
|
||||
## First local measurements (2026-08-05)
|
||||
|
||||
Run without the oracle on a locally built carrier: a 6-second 1280x720 24 fps
|
||||
clip panning across `data/fixtures/provenance/doubao-1.png`, processed by the
|
||||
shipped path at 512 px / 12 fps on MPS. Generated media stayed outside the
|
||||
repository.
|
||||
|
||||
| noise_std | engine `psnr_db` | end-to-end PSNR | end-to-end SSIM | bitrate |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 0.00 | 27.8049 | 27.6935 | 0.6955 | 2564 kbps |
|
||||
| 0.15 | 25.8853 | 25.8541 | 0.6703 | 2716 kbps |
|
||||
|
||||
Three readings, all MEASURED, none of them about SynthID:
|
||||
|
||||
1. **The two-point fit's shape survives contact with independent content.** The
|
||||
pure round trip lands at 27.80 dB against the 27.2 dB the carrier fit
|
||||
predicted, and the whole noise budget costs 1.92 dB against the predicted
|
||||
1.79 dB. These are not the carrier's numbers, but the decomposition holds:
|
||||
the autoencoder is the floor and the entire `noise_std` axis is worth about
|
||||
2 dB.
|
||||
2. **On this content the downscale is nearly free in PSNR terms.** End-to-end
|
||||
PSNR sits within 0.11 dB of the in-loop number, meaning the 512 px geometry
|
||||
cost almost nothing next to the VAE damage. This clip is a pan over a smooth
|
||||
generated image with little high-frequency detail, so it is the friendly
|
||||
case: real camera texture should widen that gap. Run the probe across content
|
||||
types before trusting any estimate of the geometry prize.
|
||||
3. **`temporal_residual_ratio` is not meaningful on a near-static shot.** It read
|
||||
1.82 at `noise_std=0` and 2.47 at 0.15, far outside the [1.0072, 1.0578] band
|
||||
ever observed before. A slow pan gives the source almost no motion residual,
|
||||
so the `max(temporal_baseline, 1e-6)` denominator collapses and the ratio
|
||||
inflates. This is the predicted defect, now observed rather than argued.
|
||||
|
||||
## Measurements on the real carrier (2026-08-06)
|
||||
|
||||
The public Veo 3 off-road sample was downloaded and its sha256 matched the manifest
|
||||
exactly, so everything below is measured on the same file the 2026-07-31
|
||||
calibration used. Its properties, none of which the manifest recorded at the time:
|
||||
H.264 1920x1080 at 24 fps, 8.000 s, 5,155,063 bps video, plus an **AAC 48 kHz
|
||||
stereo 192,241 bps audio track of 7.914 s**.
|
||||
|
||||
The shipped profile therefore delivers 512x288 at 12 fps: **7.1% of the frame area
|
||||
and 3.6% of the pixels per second.**
|
||||
|
||||
### The geometry prize, measured end to end
|
||||
|
||||
Scored with `scripts/video_fidelity_probe.py` against the untouched 1920x1080
|
||||
source, all at `noise_std=0.15`:
|
||||
|
||||
| Geometry | Frame area | PSNR | SSIM | Temporal | Bitrate |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 512x288 @12 (shipped) | 7.1% | 25.3971 | 0.8215 | 1.0407 | 2187 kbps |
|
||||
| 768x432 @12 | 16.0% | 27.3796 | 0.8556 | 1.0323 | 3782 kbps |
|
||||
| 1024x576 @12 | 28.4% | 28.8542 | 0.8819 | 1.0262 | 5604 kbps |
|
||||
| 512x288 @24 | 7.1% | 25.6142 | 0.8270 | 1.0995 | 3181 kbps |
|
||||
|
||||
**512 to 1024 is worth +3.46 dB, against 1.92 dB for the entire `noise_std` axis.**
|
||||
The geometry lever is roughly twice the noise lever, which is the first hard number
|
||||
this investigation has for the size of the prize. What it still costs in removal is
|
||||
unknown: none of these was ever submitted.
|
||||
|
||||
Two readings that are not obvious from the table:
|
||||
|
||||
- **Temporal residual improves as resolution rises** (1.0407 to 1.0262). The shared
|
||||
noise field lives on the latent grid, so at 1920 it is four times finer relative
|
||||
to the frame than at 512, where it stretches into low-frequency blotches. Higher
|
||||
resolution plausibly makes the perturbation *less* visible, not more.
|
||||
- **The frame-rate arm looks bad here and the metric is the wrong judge.** 24 fps
|
||||
buys +0.22 dB and worsens the temporal ratio to 1.0995, but what 24 fps actually
|
||||
buys is smoothness, which neither PSNR nor SSIM measures at all. The metric sees
|
||||
only the cost. Do not decide the fps question from this table.
|
||||
|
||||
Encoder cost on an M-series MPS device, 8 seconds of source: 51 s at 512/12, 180 s
|
||||
at 768/12, 380 s at 1024/12, 190 s at 512/24. Native 1920x1080 at 24 fps
|
||||
extrapolates to roughly 46 minutes for the same 8 seconds, around 350x real time. A
|
||||
native run was started and killed before producing anything, so that figure remains
|
||||
an extrapolation.
|
||||
|
||||
### The certified row does not reproduce
|
||||
|
||||
Re-running the certified configuration (512/12/0.15, seed 0) on this machine gave
|
||||
25.5002 dB and 1.0675 against the manifest's 25.3911 and 1.0578, and a different
|
||||
output sha256. The manifest records neither device nor dtype, and the original was
|
||||
almost certainly CUDA fp16 while this was MPS fp32. **The oracle certified a
|
||||
specific file that cannot be rebuilt**, so any re-run produces an artifact the
|
||||
verifier never saw. Device and dtype belong in the schema.
|
||||
|
||||
### Audio passes through untouched, and the output still reads clean
|
||||
|
||||
Established on two carriers, by comparing the extracted audio bitstream sha256 of
|
||||
input and output:
|
||||
|
||||
- Veo carrier: `674063df...` in, `674063df...` out.
|
||||
- A Lyria 3 track generated for this purpose: `6ee4c041...` in, `6ee4c041...` out.
|
||||
|
||||
The Lyria case is the sharper one because its provenance is known. Google's own
|
||||
generator stamps it: `get_ai_metadata` reads a C2PA manifest naming
|
||||
`Google C2PA Core Generator Library`, issuer `Google LLC`, `trainedAlgorithmicMedia`.
|
||||
After the shipped path runs, that output reports `{}` -- every metadata marker
|
||||
stripped, `remaining_metadata` empty, visuals regenerated -- while the audio the
|
||||
generator produced is bit-for-bit intact.
|
||||
|
||||
So the pipeline can produce a file that **our own detector calls clean while an
|
||||
untouched Google-generated audio track sits inside it**. The mechanism is proven;
|
||||
whether that audio carries an audible-to-the-verifier SynthID mark is not, and only
|
||||
the oracle can say. If it does, this is a shipped defect rather than a quality
|
||||
question, and it outranks everything else in this document.
|
||||
|
||||
Worth recording alongside it: the Veo carrier itself returns `{}` from
|
||||
`get_ai_metadata`. The public sample carries no C2PA at all, which is exactly why it
|
||||
needed an oracle verdict in the first place.
|
||||
|
||||
## State when this work was parked (2026-08-06)
|
||||
|
||||
Landed in `main` through PR #78: the manifest-backed default pin, the
|
||||
scaling-factor gate and its single seam, the sweep loading through the engine's
|
||||
loader, `scripts/video_fidelity_probe.py` with its tests, the extended manifest
|
||||
schema, and this document. **The shipped operating point did not move**: it is
|
||||
still 512 px / 12 fps / `noise_std=0.15`.
|
||||
|
||||
A product requirement arrived late and reframes the target: output resolution,
|
||||
frame rate and bitrate must not be reduced. Resolution, frame rate and crf are
|
||||
parameters and can be preserved, but zero quality loss is not achievable by this
|
||||
mechanism at all -- removal works by rebuilding pixels, so a regeneration can never
|
||||
return its input. The honest goal is to preserve container properties and make the
|
||||
perceptual loss imperceptible, judged by DISTS and VMAF rather than PSNR. Two
|
||||
consequences follow: crf 18 on this path against crf 14 on the visible path is an
|
||||
unexplained asymmetry and a free quality win, and the oracle's 100 MB limit means a
|
||||
higher-quality profile is *harder* to certify than the one it replaces.
|
||||
|
||||
Nothing is blocked on analysis. Six oracle submissions were prepared and never run,
|
||||
because file upload to the verifier failed at the tool level:
|
||||
|
||||
1. A Lyria carrier and the pipeline's output from it, answering the audio question.
|
||||
2. A multiplexed control-plus-candidate file at the shipped geometry, validating
|
||||
whether the verifier reads a concatenated pair by time range. Verified locally
|
||||
frame by frame: half one is bit-identical to the control, half two to the
|
||||
candidate.
|
||||
3. Three more multiplexed pairs for 768, 1024 and 24 fps, worth three queries
|
||||
instead of six if the instrument validates.
|
||||
|
||||
Every generated artifact for those submissions has since been lost with its
|
||||
scratch directory, which is the intended lifecycle for generated media. Rebuilding
|
||||
costs one download and roughly 25 minutes of encoding; the Lyria track is the
|
||||
exception, since regenerating it would produce a different track, and the copies
|
||||
that survive are the only ones.
|
||||
|
||||
## The single most important unknown
|
||||
|
||||
**How much the 512 px downscale contributes to removal.** Every quality gain routes
|
||||
through this question, and the critics moved it from "probably free" to "direction
|
||||
unknown", which is exactly what makes it the highest-information experiment
|
||||
available.
|
||||
|
||||
## Ranked experiment program
|
||||
|
||||
Ranked by information per oracle query. The budget is roughly 10 checks per 24
|
||||
hours (MEASURED), so the program is paced by calendar, not by money.
|
||||
|
||||
### E0. The zero-oracle tier (0 submissions) - do this first
|
||||
|
||||
Not an experiment on the oracle; the precondition that makes everything else
|
||||
interpretable.
|
||||
|
||||
- `ffprobe` the carrier and record `source_width`, `source_height`, `source_fps`.
|
||||
The actual downscale factor of the existing rows is currently unrecoverable.
|
||||
- Extend the manifest schema: variant/control, source geometry, `vae`, `track`
|
||||
(audio|visual), verdict state in {DETECTED, NOT_DETECTED, INDETERMINATE,
|
||||
REFUSED}, verbatim verdict text, detected time range for the output,
|
||||
`session_id`, content stratum.
|
||||
- **Measure the `noise_std = 0.0` round trip locally** on the same carrier. This
|
||||
converts the inferred 27.2 dB ceiling into a measurement and bounds the whole
|
||||
noise axis for one GPU pass and zero oracle cost.
|
||||
- Freeze the current defaults' metrics on a fixed clip set, matching the
|
||||
record-then-diff rule in `.claude/rules/development.md`.
|
||||
- Determine whether the carrier has an audio track and whether it carries SynthID.
|
||||
|
||||
### E1. Instrument validation: multiplexing and a session anchor (1-2 submissions)
|
||||
|
||||
The verifier reports time ranges, so one file can carry several doses. A 24-second
|
||||
file of `[control 8 s | certified 0.15 8 s | control 8 s]` should read detected on
|
||||
the outer segments and not detected on the middle one, both halves already known
|
||||
from the manifest.
|
||||
|
||||
`encode_video_frames` requires matching dimensions and one frame rate, so
|
||||
multiplexing works **only along the `noise_std` axis**, not across geometries: one
|
||||
multiplexed file per geometric envelope.
|
||||
|
||||
Separately, submitting the existing detected file first in each session costs 10%
|
||||
of the quota, turns "the oracle may have changed" into a per-session gate, and
|
||||
measures the flip rate on a byte-identical file, a quantity every plan silently
|
||||
assumes is zero.
|
||||
|
||||
### E2. Resolution ladder at fixed dose - the decisive axis (2 submissions per rung)
|
||||
|
||||
A fixed value, not a ceiling. Rung 1 is `long_side = 768` with fps held at 12 so
|
||||
exactly one destruction axis moves. Jumping straight to 1920 is a bad first rung:
|
||||
a 3.75x jump makes a detected verdict uninformative about where the boundary lies.
|
||||
|
||||
- **Matched control:** 768 / 12 / `noise_std = 0.10`, expected DETECTED. It is
|
||||
strictly stronger than a plain re-encode control because it discharges the
|
||||
envelope, the VAE round trip, and a nonzero dose at once, and it diffs directly
|
||||
against the existing 512 row.
|
||||
- **Candidate:** 768 / 12 / 0.15.
|
||||
- **Positive:** the resolution axis is open; next rungs 1024 and then native with a
|
||||
clamp, 2 submissions each.
|
||||
- **Negative:** the downscale contributes to removal. This is the most valuable
|
||||
available negative: it kills the native-resolution proposals outright and turns
|
||||
the question into how much dose must be spent to buy resolution back, at the
|
||||
known exchange rate B = 2815.
|
||||
- Watch for a monotonicity violation: 768 / 0.10 reading NOT_DETECTED would
|
||||
overturn the critics' spectral argument.
|
||||
|
||||
### E3. Frame rate at fixed geometry and dose (2 submissions)
|
||||
|
||||
Control 512 / 24 / 0.10 expected DETECTED; candidate 512 / 24 / 0.15 expected
|
||||
not detected. The mechanism is provable from source, so the prior is high, but the
|
||||
only counter-mechanism - per-frame count aggregation - is probabilistic and n = 1
|
||||
on an 8-second clip does not measure it. Certify on the longest clip that fits the
|
||||
90 s / 100 MB limits, because the risk compounds with frame count.
|
||||
|
||||
Ranked below E2 because the outcome is nearly predetermined and the prize (judder)
|
||||
is smaller than the geometry prize. It is the safest bet if a guaranteed win is
|
||||
wanted.
|
||||
|
||||
### E4. `noise_std = 0.0` as an oracle probe (1 submission) - deliberately low rank
|
||||
|
||||
The local half of this probe is the cheapest high-information measurement in the
|
||||
program and already sits in E0 at zero oracle cost. The **oracle** half ranks low:
|
||||
it buys no quality by construction, and its likely DETECTED outcome is nearly
|
||||
deducible from the 0.10 row under monotonicity. Its one real value is that
|
||||
NOT_DETECTED here would be non-monotone against 0.10 and would refute the
|
||||
assumption underneath every ladder and bisection in this program. Worth one query,
|
||||
after E2 and E3.
|
||||
|
||||
### E5. Stratified certification of the surviving operating point (15-30 submissions)
|
||||
|
||||
Only after E2 and E3 produce a winner. Zero detections across at least 15
|
||||
stratified carriers (at least 3 per stratum: face closeup, fast motion, flat sky,
|
||||
dark night, text overlay) bounds the failure rate at 18.1%; 30 carriers bound it at
|
||||
9.5%. Add the project's 1.5x margin convention, at least 2 seeds, at least 2
|
||||
sessions, and per-track verdict recording.
|
||||
|
||||
For the flat-sky stratum, check the source reads detected first: Google documents a
|
||||
"not enough details to watermark" state, so a clean reading there may say nothing
|
||||
about removal.
|
||||
|
||||
## Surviving change candidates
|
||||
|
||||
**S1. Fix the metric's reference frame (0 oracle submissions).** Add
|
||||
`source_psnr_db` (candidate upscaled back to native against the untouched source
|
||||
frame) **and** a separate post-mux pass that decodes the delivered file. The
|
||||
critics killed the original claim that this would expose crf 18: the candidate is
|
||||
captured at `:423`, before `frame_pipe.write` at `:430`, so no in-loop metric can
|
||||
see the codec. Only the post-mux pass covers resize, decimation, crf, and mux
|
||||
together. Use `INTER_AREA` or a fixed analytic size for the upscale so the CPU
|
||||
resize does not dominate the loop, and update both other consumers of the
|
||||
generator in the same change (`read_sampled_frames` at `:188-209`, and
|
||||
`scripts/video_synthid_sweep.py:147` where `np.stack(frames)` must keep receiving
|
||||
resized frames). Do not redefine the existing `psnr_db`: the two manifest rows stay
|
||||
comparable to each other only while that field keeps its exact current meaning.
|
||||
|
||||
**S2. Frame rate as a fixed certified value (after E3).**
|
||||
`DEFAULT_VIDEO_SYNTHID_FPS` 12.0 to 24.0, keeping `min(fps, source_fps)`. A
|
||||
ceiling of 60.0 was **rejected**: it makes the delivered operating point a function
|
||||
of the user's source, so one sample from a family gets certified while the rest
|
||||
ship uncertified, and the risk direction is unfavorable. Port the VFR/PTS bridge
|
||||
(`probe_video_timestamps`, `timestamped_input`, currently only in
|
||||
`video_visible.py`): at native frame rate the output stops reading as a proxy, and
|
||||
silent CFR-ification becomes a master-quality defect. Add `fps` and `long_side`
|
||||
pins next to `tests/test_video_invisible.py:285`, tied to the certifying manifest
|
||||
row.
|
||||
|
||||
**S3. Clamp and align `_fit_size` (0 oracle submissions, with a caveat).**
|
||||
`scale = min(1.0, long_side / max(width, height))` at `:88`; align the long side
|
||||
and derive the short side from the true aspect, rounding to the nearest multiple of
|
||||
8. The pinned `_fit_size(1280, 720, 512) == (512, 288)` survives. The caveat: the
|
||||
clamp is an **uncertified operator change for a whole class of users**, since a
|
||||
320x240 source is upscaled to 512x384 today and would run at 320x240 afterwards,
|
||||
and neither has been tested. Ship it as its own documented change, not as a
|
||||
drive-by fix.
|
||||
|
||||
**S4. Extend the manifest schema and oracle protocol (0 submissions).** A
|
||||
precondition for the whole program. Record the verbatim verdict text, because an
|
||||
unclear state logged as not detected is exactly the silent regression the protocol
|
||||
exists to prevent.
|
||||
|
||||
**S5. A sigma normalization contract (0 submissions, applies to the current path).**
|
||||
No VAE swap survived, but two elements are real risks today: `vae.config.scaling_factor`
|
||||
cannot be trusted, so log `latents.std()` at `:276` and assert the effective
|
||||
scaling factor at load; and any cross-model sigma transfer must match on the **RMS
|
||||
of the decoded pixel perturbation**, not on latent units.
|
||||
|
||||
**S6. Additive texture masking - contingency on an E2 negative only.** Not energy
|
||||
preserving: the certified sigma stays as a floor everywhere and the mask only adds
|
||||
on top in textured regions. The original energy-preserving form was refuted - the
|
||||
mechanism normalizes and then clips, which breaks the claimed invariant by a
|
||||
content-dependent amount, and the proposed unit test asserted the invariant before
|
||||
the clip and would have stayed green. Energy preservation is also unsafe in
|
||||
principle: the detector does not need a full frame, and the verifier reports
|
||||
per-segment, so a flat region perturbed only at the floor is a crop carrying nearly
|
||||
the full carrier. The salvaged version has **no standalone PSNR win**; its only
|
||||
value is freeing distortion budget to spend on resolution if E2 shows the dose must
|
||||
rise. It additionally needs motion compensation for the mask (flow is computed at
|
||||
`:442`, after the decode at `:423`, so the loop must be restructured) and one
|
||||
deliberately bad candidate so `temporal_residual_ratio` acquires a known failing
|
||||
value: it has only ever been observed in [1.0072, 1.0578], so "the ratio looks
|
||||
fine" is currently an unfalsified claim.
|
||||
|
||||
## Refuted proposals - do not resurrect
|
||||
|
||||
- **Carrier-scale, resolution-invariant noise field.** "Carrier scale" is set by
|
||||
our VAE's latent pitch, not the carrier's. The motivating arithmetic treats a
|
||||
field on the latent grid as white at pixel resolution and is wrong by exactly the
|
||||
square of the scale factor: white noise on a 240-wide latent already spans
|
||||
0-120 cycles per frame, entirely inside the band the decoder reproduces. The
|
||||
construction actually band-limits the perturbation from 0-120 to 0-32 cycles,
|
||||
making it smoother - the first thing any spread-spectrum extractor's content
|
||||
suppression removes - while moving its energy toward the peak of the contrast
|
||||
sensitivity function, making it more visible. Bilinear upsampling is also
|
||||
heteroscedastic, and global renormalization by `noise.std()` stamps a visible
|
||||
amplitude lattice rather than fixing it.
|
||||
- **Native resolution plus tiling plus a carrier-scale field.** Inherits the
|
||||
arithmetic above, concedes that native processing destroys less carrier without
|
||||
proposing a replacement mechanism, and bundles four destruction axes into one
|
||||
candidate on a 4-query budget so a detected verdict is undecomposable. Its cost
|
||||
model was also wrong by roughly 10x. The salvageable parts moved into S1 and S3.
|
||||
- **`AutoencoderKLTemporalDecoder`.** Refuted independently by three critics.
|
||||
Removal is a property of the encode-decode composition, so a strictly more
|
||||
faithful decoder preserves more carrier at the same sigma. The class implements
|
||||
neither slicing nor tiling, so `enable_slicing()` at `:135` raises
|
||||
`NotImplementedError` and the documented bounded-memory property dies. Its
|
||||
temporal receptive field needs windows of tens of frames, its license carries a
|
||||
revenue restriction incompatible with an Apache-2.0 default, and its own
|
||||
published table shows a **worse** FID (9.17 against 7.61).
|
||||
- **Wan or another temporal video VAE.** Hard refusal in code: `AutoencoderKLWan._encode`
|
||||
uses `iter_ = 1 + (num_frame - 1) // 4`, so at the shipped `batch_size = 4` only
|
||||
frame 0 is encoded and a 4-frame batch decodes back to 1, which makes
|
||||
`zip(..., strict=True)` at `:429` raise. The proposed remedy of carrying the
|
||||
causal feature cache across chunks is impossible through the public API, since
|
||||
`clear_cache()` runs on both entry and exit of `_encode` and `_decode`.
|
||||
`float(vae.config.scaling_factor)` at `:270` and `:292` also raises, because Wan
|
||||
exposes `latents_mean`/`latents_std` instead. Logically, feeding 4:1 temporal
|
||||
compression frames 83 ms apart either reconstructs well (carrier preserved, no
|
||||
removal gain) or hallucinates (no quality prize); the two claims are mutually
|
||||
exclusive.
|
||||
- **A 16-channel f8 VAE.** Self-refuted and confirmed by the critics. Quadrupling
|
||||
latent channels roughly doubles L in Zhao's theorem, so sigma must roughly double
|
||||
to hold removal, and the VAE frontier in the UnMarker results is monotone with no
|
||||
published point where a more faithful autoencoder removes **more**. Predicted net
|
||||
loss of 2-7.5 dB by its own arithmetic. Its config also carries
|
||||
`scaling_factor = 0.2614`, so an unchanged `noise_std = 0.15` is about 30% weaker
|
||||
in raw latent units. Only the normalization contract survived, as S5.
|
||||
- **A per-channel luma/chroma probe.** The arms match in absolute latent units but
|
||||
not in relative dose per channel, since `scaling_factor` normalizes the aggregate
|
||||
latent, so the very reading the probe exists for is confounded with dose. The
|
||||
chroma arm is additionally subsampled by the yuv420p encode, which no current
|
||||
metric sees. Maximum prize under 1 dB by its own fit.
|
||||
- **Post-processing with unsharp plus grain, as a shipped stage.** The
|
||||
carrier-reimport argument is sound and was verified line by line: `unsharp_mask`
|
||||
reads only its own argument ([`humanizer.py:79`](../src/remove_ai_watermarks/humanizer.py)),
|
||||
and `adaptive_polish` touches the source only through one float. But reimport was
|
||||
never the binding risk:
|
||||
`cv2.addWeighted(img_f, 1.0 + amount, blurred, -amount, 0.0)` at
|
||||
`_ADAPTIVE_MAX_UNSHARP = 1.0` multiplies the **surviving carrier residue** by up
|
||||
to 2x as the terminal operation before encoding, and whether that re-arms a given
|
||||
file depends on that file's unobservable margin. With no local decoder and only a
|
||||
sampled oracle the stage is structurally uncertifiable, not merely expensive to
|
||||
certify. The only rescue is moving sharpening **above** the VAE stage so the
|
||||
removal operator stays terminal. Grain, which monotonically lowers detector SNR,
|
||||
is the safe half and can be separated.
|
||||
|
||||
## Measuring quality properly
|
||||
|
||||
The current `psnr_db` cannot show the improvement this work exists to produce.
|
||||
Minimum upgrade:
|
||||
|
||||
1. **`source_psnr_db`** - candidate upscaled back to native against the untouched
|
||||
source frame. Good for ranking configurations against a common reference,
|
||||
dominated by unrecoverable high-frequency content, so not a measure of what the
|
||||
VAE costs.
|
||||
2. **A post-mux end-to-end pass** - decode the delivered file after
|
||||
`mux_encoded_video` (`:456`) and compare against the source. The only measure
|
||||
covering resize, decimation, crf, and mux together. Keep it out of the streaming
|
||||
loop so `test_stream_batches_consumes_only_one_batch_ahead` stays valid.
|
||||
Implemented as `scripts/video_fidelity_probe.py`, which streams, reports the
|
||||
delivered file's bitrate, and shares the engine's frame-selection rule rather
|
||||
than copying it.
|
||||
3. **DISTS** (arXiv:2004.07728), built to tolerate texture resampling - exactly
|
||||
what the VAE does to foliage, skin, and grass, and exactly what PSNR and LPIPS
|
||||
punish even when the result is perceptually equivalent. It separates "the VAE
|
||||
resampled the grass" from "the VAE destroyed an edge".
|
||||
4. **VMAF** through `ffmpeg` `libvmaf`, cheapest to add, whose ADM/DLM feature
|
||||
names this exact complaint. Its temporal term is only a mean absolute luma
|
||||
difference between neighboring frames, so it is not a flicker detector: keep the
|
||||
motion-compensated ratio.
|
||||
5. **Encoded file size as a third axis.** Every current metric is taken before the
|
||||
pipe, so a bitrate explosion currently reads as "quality did not suffer". This
|
||||
matters most for any perturbation that varies frame to frame: at fixed crf it
|
||||
raises bitrate rather than lowering quality.
|
||||
6. **`temporal_residual_ratio` repairs**: add a p95 across frames next to the mean
|
||||
so localized flicker stops being averaged away, add a long-horizon term against
|
||||
the first frame, and replace `max(temporal_baseline, 1e-6)` with an explicit
|
||||
undefined result on static shots.
|
||||
7. **A no-reference metric** (DOVER or FAST-VQA) for the case where no
|
||||
full-reference measure can compare a 512/12 output against a 1024/24 output on
|
||||
one scale. Relative ranking within a sweep only; absolute values are
|
||||
uncalibrated for this artifact class.
|
||||
|
||||
Nothing local measures removal. `remove_video_invisible` checks `get_ai_metadata`
|
||||
on the output, which is metadata, not pixels. Only a manifest row counts.
|
||||
|
||||
## Risks
|
||||
|
||||
**Error asymmetry is the governing constraint.** Shipping a leak means a user
|
||||
receives a watermarked file believing it is clean, with no local pixel decoder to
|
||||
catch it and no feedback path that would surface it. Staying conservative means a
|
||||
user receives 512 px / 12 fps, a cost that is visible, bounded, and reversible with
|
||||
an explicit flag. A symmetric test is therefore inadmissible, and the burden of
|
||||
proof sits entirely on the new default.
|
||||
|
||||
**n = 1.** The current default is certified by one success on one carrier with one
|
||||
seed. Any quality change inherits that weakness and must not deepen it.
|
||||
|
||||
**Content and seed dependence** is MEASURED on this project's image branch:
|
||||
survivors switch by content type, and near the threshold the same input flipped
|
||||
between runs on seed alone.
|
||||
|
||||
**Oracle instability.** Three verdict states, not two. Per-segment reporting.
|
||||
Separate audio and visual tracks - and the pipeline byte-copies audio, so a Veo 3
|
||||
clip with generated sound leaves with its audio SynthID intact, and the manifest
|
||||
cannot even record which track the 2026-07-31 negative referred to. That is
|
||||
potentially a shipped product hole, not only an experimental confound. Session
|
||||
drift is currently unfalsifiable.
|
||||
|
||||
**Observability narrows exactly where quality rises.** At fixed crf the bitrate
|
||||
grows with pixels per second, so the maximum duration fitting under 100 MB falls as
|
||||
fps and resolution rise. The configuration users actually receive becomes **less**
|
||||
observable than the one it replaces. That is a permanent property, not an
|
||||
inconvenience.
|
||||
|
||||
**Ceilings instead of fixed values.** Any ceiling makes the delivered operating
|
||||
point a function of the user's source, and with it the perturbation's cycles per
|
||||
frame - the quantity carrying the certified margin. Ship fixed values.
|
||||
|
||||
**Documentation and test surface.** Only `DEFAULT_VIDEO_SYNTHID_NOISE_STD` is
|
||||
pinned. Moving `long_side` or `fps` touches hardcoded numbers in `README.md`,
|
||||
`docs/known-limitations.md`, `docs/cli.md`, `docs/python-api.md`,
|
||||
`docs/module-internals.md`, `docs/synthid.md`, and `docs/verification-plan.md`.
|
||||
Drift-prone operational numbers should not live in seven places of prose; their
|
||||
source of truth is the manifest.
|
||||
|
||||
**Silent configuration drift.** `sd-vae-ft-mse/config.json` carries no
|
||||
`scaling_factor`, so 0.18215 is a class default under an upper-unbounded
|
||||
`diffusers>=0.38.0` while `maintain.sh` runs `uv-outdated`. A library bump can move
|
||||
the certified operating point with a green suite.
|
||||
|
||||
**Missing guards that get worse at native resolution.** There is no HDR or >8-bit
|
||||
rejection and no VFR/PTS bridge on the invisible path. A 512 px output plainly
|
||||
reads as a proxy; a native-resolution output reads as a master, and the cost of
|
||||
silently flattening a 10-bit PQ source to 8-bit SDR rises accordingly.
|
||||
|
||||
**Do not bundle axes.** Raising `long_side`, raising `fps`, and lowering crf all
|
||||
reduce total destruction and all require recertification. The metric reference-frame
|
||||
fix is the only exception, because it changes the measurement rather than the
|
||||
pixels. Do not put a crf 18 to 14 change in the same candidate as a resolution
|
||||
change.
|
||||
@@ -10,7 +10,7 @@ Who embeds what, and whether it is locally detectable (so we know which gaps are
|
||||
**Root cause and external confirmation (deep-research 2026-06-19, adversarially verified).** This is the SCHEME's ceiling, not our usage — there is no better decoder to adopt. The imwatermark maintainers state verbatim (both the ShieldMnt and Stability-AI READMEs) that the algorithm "cannot guarantee to decode the original watermarks 100% accurately even though we don't apply any attack." Independent measurement (WMAdapter, arXiv:2406.08337 Table 2) puts dwtDct at only **~0.79 bit accuracy on CLEAN images (~38/48 bits — already below our 44 gate)**, collapsing to ~0.50 (chance) under crop/JPEG. Two code-verified + locally-reproduced mechanisms drive the content-dependent failures: (1) the decoder reads each bit as the **highest-magnitude DCT coefficient per block**, so any content coefficient exceeding the encoded target flips the bit; (2) the default embed is in the **YUV chroma channel, which 8-bit-clamps on white/bright pixels** (a +36 chroma delta survives a white-fill round-trip as only +4, ~89% loss) — this is the mechanism behind the bright-flat / minimalist failures and the all-ones degenerate decode. No maintained fork or detector decodes this scheme reliably: the WAVES benchmark (arXiv:2401.08573) relegates DWT-DCT to supplementary appendix G.5 and targets Stable Signature / Tree-Ring / StegaStamp instead; learned encoder/decoder schemes reach ~0.98-0.99 clean but are a DIFFERENT watermark class (not what SDXL/FLUX stamp). `dwtDctSvd` does not help (SDXL embeds `dwtDct`; dwtDctSvd cannot decode it, and its clean accuracy ~0.72 is lower). **Authoritative conclusion: the open DWT-DCT mark cannot be turned from positive-only into a reliable real-world detector; keep it positive-only and rely on C2PA.** (Refuted along the way: that the library is unmaintained, and that it is robust to JPEG but only fails on geometric attacks — both did not survive verification.)
|
||||
|
||||
Consequence for the FLUX hosted-output question (BFL Playground, FLUX.2 [pro] + FLUX.1 [dev], 2026-06-19): all samples carry the signed C2PA manifest (issuer "Black Forest Labs"); the open DWT-DCT decode returned `None`, but every available FLUX carrier (textured fox AND a minimalist-flat generation) failed the positive control (28/48), so the detector is blind on them and **whether BFL hosted output embeds the open pixel watermark is UNRESOLVED** (an earlier note here wrongly asserted it absent — overstated; a later note blamed "high texture" — also wrong, flat carriers fail too). What IS established: C2PA is the reliable FLUX identifier; the `_BITS_48` pattern is correct (round-trips on chatgpt/firefly/random). Resolving the hosted question needs a hosted FLUX carrier that first passes a >=44/48 positive control, which neither a textured nor a flat prompt produced — low priority (the open mark is only a stripped-metadata fallback).
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Copilot + Designer; Bing Image Creator collected 2026-05-24 still signs as "Microsoft" and now runs **MAI-Image**, NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
|
||||
**Samsung Galaxy AI** signs supported edits with C2PA and may carry the
|
||||
proprietary `genAIType` marker. The registered visible detector covers the
|
||||
@@ -22,8 +22,15 @@ icon-only variants need separate calibrated silhouettes.
|
||||
|
||||
**Black Forest Labs (FLUX)** API output signs C2PA: `claim_generator_info "Black Forest Labs API"` + a `c2pa.ai_generated_content` assertion + `trainedAlgorithmicMedia` (issuer `b"Black Forest Labs"` added to `C2PA_ISSUERS`, platform "Black Forest Labs (FLUX)").
|
||||
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
|
||||
Some applications sign C2PA through an upstream model or infrastructure
|
||||
provider. For AI claims, exact product mappings in `claim_generator` therefore
|
||||
take precedence over issuer attribution. Supported mappings include Higgsfield
|
||||
AI, Topaz Labs Image API, and TikTok Ad Creative Toolbox; an unknown claim
|
||||
generator still falls back to the certificate issuer.
|
||||
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
|
||||
- **EXIF/XMP/PNG-text generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal). Additional verified generator stamps include **NovelAI** (`Software`, `Source`, and `Title` PNG text chunks), **Reve** (`Software` or XMP `CreatorTool` = `reve.com`), and **Aphrodite AI** (`Make` or `Software` = `Aphrodite AI`).
|
||||
- **App-export provenance and AIGC JSON:** supported ByteDance-family exports can place a JSON object in EXIF `ImageDescription` or `UserComment`, independently of C2PA or TC260. Exact `product` values for Doubao, Xinghui, and Dreamina are removable product provenance, but do not alone prove that the pixels were generated. Dreamina additionally requires `exportType=generation` for that verdict. A nested Aweme `aigc_type=1` or private ByteDance `aigc_label_type=1` / `2` is an AIGC disclosure; `0` is inconclusive and can occur on a Dreamina generation export. Plain Aweme, retouch, and `lv` exports are preserved. The lower-case private field is deliberately not interpreted as the normative TC260 `Label`, whose values `1` / `2` / `3` mean generated / possibly generated / suspected generated under [GB 45438-2025](https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf).
|
||||
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
|
||||
|
||||
Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC** — only EXIF `Artist` = a UUID and EXIF `ImageDescription` = `Signature: <base64>` (a crypto signature, unverifiable locally without xAI's public key). This empirically kills the earlier unverified "xAI signs C2PA as xAI" lead — xAI is not even a C2PA member. `exif_generator` misses it (neither field holds an `AI_GENERATOR_TOKENS` token), so a dedicated detector `xai_signature(path)` matches the pair (`ImageDescription ~ ^Signature: [A-Za-z0-9+/=]{64,}` AND UUID `Artist`); wired into `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify` (signal `xai_signature`, platform "xAI (Grok / Aurora)").
|
||||
@@ -36,9 +43,21 @@ AI generator values while retaining unrelated camera and editor EXIF. The
|
||||
shared `xai_signature_pair` helper is the single source of truth for the
|
||||
pair. On the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
|
||||
in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`).
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). The label says only "this is AI", but its `ContentProducer` names the signing entity — `001` + `1` + an 18-char Unified Social Credit Code + a 5-digit product suffix, normalized by `metadata.uscc_of`, or for a few generators a bare product name. `KnownMark.tc260_producer_codes` maps the codes settled per vendor by `scripts/vendor_cohort_harvest.py` to registry mark keys, so an AIGC image relaxes the detector of the vendor it actually carries rather than a guessed pair; an unmapped or absent producer falls back to ByteDance's two products. A code identifies a legal entity, not necessarily one brand, so a hosting or aggregating platform that signs for several apps is a recall bet rather than a proof.
|
||||
|
||||
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item; Doubao's iOS export additionally writes two QuickTime-form variants (a bare `meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list whose `ilst` data items carry the JSON with no key name) — both verified on retained carriers (2026-08-17). The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
|
||||
|
||||
Native MKV/WebM is a sixth serialization. TC260-PG-20257A stores
|
||||
`TagName=AIGC` and the raw JSON `TagString` in
|
||||
`Segment.Tags.Tag.SimpleTag`. The bounded EBML reader skips cluster payloads;
|
||||
the existing ffmpeg stream-copy path removes the tags without transcoding.
|
||||
|
||||
The same [TC260 video guide](https://www.tc260.org.cn/portal/article/303/4061772dcf684d8a96f395a4298e9e53)
|
||||
defines two more native serializations. AVI stores an `AIGC` child in
|
||||
`LIST/INFO`; FLV stores an AMF0 `AIGC` string under `script.onMetaData`. The
|
||||
bounded RIFF and FLV readers validate the JSON field set and skip media
|
||||
payloads. Removal remuxes either container through ffmpeg with stream copy.
|
||||
|
||||
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). All three generic forms (the PNG chunk, the bare `{"AIGC":...}` object, and the bare `AIGC{...}` blob) are gated on at least one TC260 field (`_TC260_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `_TC260_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
|
||||
- **HuggingFace-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** HuggingFace Jobs / Spaces can stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID. It marks the *hosting job*, not a model, so it lifts an Unknown verdict to a tentative AI via `hf_only` but never overrides a hard metadata signal. `_HF_JOB_CAVEAT` states the limit. Removal drops the chunk through the PNG metadata whitelist.
|
||||
- **No detectable signal on some downloads:** Recraft exports and some hosted
|
||||
FLUX surfaces can arrive without a supported local signal. Midjourney samples
|
||||
@@ -46,15 +65,22 @@ in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
|
||||
DWT-DCT decoder only applies when the producing pipeline actually ran its
|
||||
encoder and the carrier remains decodable.
|
||||
- **Invisible but NOT locally detectable (proprietary, API/oracle only — same wall as SynthID):** Amazon Titan Image Generator + Nova Canvas (Bedrock `DetectGeneratedContent` API), Kakao (new SynthID image adopter, May 2026), NVIDIA Cosmos (SynthID video). No local detector possible; treat like SynthID.
|
||||
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest (Digimarc/Imatag/Steg.AI/...) stay name-only (proprietary decoders).
|
||||
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, preserve the structured assertion's signed `value`, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest stay name-only unless a compatible decoder is independently verified.
|
||||
- **Microsoft Paint and Photos InvisMark (reverse-engineered 2026-08-20):** Paint receives a per-generation GUID from remote prompt moderation, embeds it into locally generated pixels, and records the same value in `c2pa.soft-binding` under `com.microsoft.invismark.1`. The C2PA soft-binding registry independently identifies that algorithm as Microsoft Responsible AI InvisMark for image and video. Paint's 144-bit writer framing does not match the public repository's 100-bit pretrained checkpoint interface, so compatibility is not assumed. The parser reports the signed identifier; there is no validated local pixel decoder. Microsoft's external Content Provenance Detection API is the removal oracle because it reports `Watermark` separately from `C2PA`; a metadata-stripped, pixel-identical control must remain watermark-positive before an output-negative result is attributed to pixel removal. Sources: https://xusheng.dev/posts/reversing/mspaint_invisible_watermark/main/, https://github.com/c2pa-org/softbinding-algorithm-list/blob/main/softbinding-algorithm-list.json, and https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection
|
||||
- **Built in the dated batch:** soft-binding vendor detection, IPTC Photo
|
||||
Metadata AI-disclosure fields, C2PA detection and stripping for supported
|
||||
ISOBMFF video, and the optional Adobe TrustMark decoder. Visible video-logo
|
||||
removal and proprietary audio-watermark detection remain outside the package.
|
||||
ISOBMFF video, the optional Adobe TrustMark decoder, and temporally stabilized
|
||||
visible Sora, Veo, Seedance, Dola, Hailuo, and Kling removal. Other visible
|
||||
video logos and proprietary audio-watermark detection remain outside the
|
||||
package.
|
||||
Metadata stripping for supported audio containers is a separate implemented
|
||||
path.
|
||||
|
||||
**Box detection window — now handled (v0.6.8):** detection no longer relies on a fixed first-MB read. `metadata.scan_head(path, size)` reads the first `size` bytes and, for ISOBMFF, appends the payloads of late provenance boxes found by `isobmff.scan_c2pa_region` (a file-seeking top-level box walker that skips past `mdat` by size without reading it), so a C2PA/AIGC/IPTC manifest placed AFTER a large `mdat` in a streaming/non-faststart MP4 is now caught. Every C2PA/marker byte scan (`has_ai_metadata`, `aigc_label`, `iptc_ai_system`, `synthid_source`, `exif_generator` XMP, `get_ai_metadata` soft-binding, and `identify`) goes through `scan_head`; it is behavior-neutral for non-ISOBMFF inputs (exactly `f.read(size)`).
|
||||
**Box detection window — now handled (v0.6.8):** detection no longer relies on a fixed first-MB read. `metadata.scan_head(path, size)` reads the first `size` bytes and, for ISOBMFF, appends the payloads of late provenance boxes found by `isobmff.scan_c2pa_region` (a file-seeking top-level box walker that skips past `mdat` by size without reading it), so a C2PA/AIGC/IPTC manifest placed AFTER a large `mdat` in a streaming/non-faststart MP4 is now caught. Every C2PA/marker byte scan (`has_ai_metadata`, `aigc_label`, `iptc_ai_system`, `synthid_source`, `exif_generator` XMP, `get_ai_metadata` soft-binding, and `identify`) goes through `scan_head`; for PNG it likewise appends the payloads of `tEXt` / `iTXt` / `zTXt` / `eXIf` / `iCCP` chunks that start beyond the window (`_png_late_metadata`, seeking past `IDAT`), which is how a TC260 AIGC label appended after the pixel stream is caught; for WebP it appends the `EXIF` / `XMP ` / `ICCP` / `C2PA` chunks past the window (`_riff_late_metadata`, stepping over the coded image), which is how an IPTC "Made with AI" tag stored after the pixels is caught; and for a file at least `size` bytes long it finally appends the metadata text the decoder reaches but a raw read cannot (`_decoder_visible_text`), which covers a compressed PNG `zTXt` packet no byte scan can spell; for any file that fits inside `size`, it is exactly `f.read(size)`.
|
||||
|
||||
Native TC260 MP4/MOV tags do not live in those top-level provenance boxes.
|
||||
`tc260_aigc_payloads` separately seeks through `moov.udta.meta.keys/ilst`, so
|
||||
the normative tag is also found when a large `mdat` precedes `moov`.
|
||||
|
||||
**Meta-box XMP and EXIF removal are handled in place:** an AI-label XMP packet
|
||||
stored as a meta-box `mime` item is blanked by
|
||||
@@ -69,6 +95,42 @@ against primary sources before adding jurisdiction-specific claims.
|
||||
|
||||
**Google Gemini visible "sparkle" -- tier-dependent, and spec-undocumented by Google.** Google primary sources (the Nano Banana Pro blog and the gemini.google image-generation page, both WebFetch-verified) confirm Gemini images carry BOTH the invisible SynthID (on ALL Google-AI media) AND a visible sparkle, but the visible mark is **tier-gated**: applied for FREE and Google AI **Pro** users, and **REMOVED** for Google AI **Ultra** subscribers, inside **Google AI Studio**, and on **API / dev** output. So a Google-C2PA image with NO visible sparkle is expected (Ultra / API), not evidence it is clean -- this reinforces the `identify` "no visible mark != clean" rule. The ONLY official verifier is the SynthID flow (upload to the Gemini app, ask if it is AI-generated), which reads the INVISIBLE mark; there is **no official visible-sparkle detector**, and Google publishes **no** glyph geometry / size / opacity / color / locale / placement spec. So our capture-based sparkle template is the only source of truth and cannot be validated against a vendor spec -- keep reverse-engineering from real captures (do not expect a published spec).
|
||||
|
||||
**Google Veo video marks use two incompatible visible designs.** Public raw
|
||||
clips verify the current four-point diamond and the legacy bottom-right `Veo`
|
||||
text. The independent
|
||||
[VeoWatermarkRemover](https://github.com/allenk/VeoWatermarkRemover) project
|
||||
reports the same current-versus-legacy split, multiple output layouts, and the
|
||||
need for cross-frame position agreement. Our implementation copies no logo
|
||||
pixels or alpha maps from that project: it uses two synthetic silhouettes,
|
||||
known-layout searches plus a strong relocated-diamond fallback, and a separate
|
||||
temporal arbiter calibrated against raw watermarked clips and clean API exports.
|
||||
|
||||
**ByteDance video surfaces use distinct visible labels.** Public Seedance
|
||||
showcase clips contain a fixed rounded box with `AI`, while the Dola sample in
|
||||
[issue #16](https://github.com/wiltodelta/remove-ai-watermarks/issues/16) uses
|
||||
fixed `Dola AI` text. The independent
|
||||
[Seedance remover](https://github.com/SamurAIGPT/seedance-2.0-watermark-remover)
|
||||
estimates a static corner from a temporal mean frame and edge density. Our
|
||||
implementation instead matches provider-specific synthetic silhouettes on
|
||||
every frame, then requires an anchored temporal run. This extra anchor check
|
||||
was necessary because a moving clean scene detail could retain enough adjacent
|
||||
overlap to pass a recurrence-only gate.
|
||||
|
||||
**Hailuo and Kling use larger fixed composite labels.** Verified Hailuo exports
|
||||
carry a lower-edge waveform, `MINIMAX`, separator, Hailuo ring, and
|
||||
`hailuo AI` text. Verified Kling exports carry a bottom-right swirl,
|
||||
`KLING AI`, a changing version suffix, and sometimes `PRO`. The detectors use
|
||||
only synthetic primitives and fonts. Hailuo expands the matched core to cover
|
||||
the complete composite. Kling combines a version-independent text core with a
|
||||
synthetic ring rescue, then requires the recurring candidate to reach the
|
||||
expected frame edge and contain enough bright low-saturation pixels. Those
|
||||
extra gates were added after clean Luma and PixVerse scene details passed shape
|
||||
and temporal recurrence alone. The generic
|
||||
[WatermarkRemover-AI](https://github.com/D-Ogi/WatermarkRemover-AI) project
|
||||
instead uses Florence-2 to identify arbitrary watermarks before LaMa
|
||||
inpainting. That is broader, but it carries a much heavier model and a less
|
||||
auditable detection boundary than the provider-specific synthetic path here.
|
||||
|
||||
**The faint-visible-mark precision/recall wall is fundamental, not a heuristic artifact.** The visible-watermark-detection literature has moved to LEARNED segmentation / object-detection (WDNet WACV'21 arXiv:2012.07616; SLBR ACM MM'21, open code+weights; the PRCV'18 large-scale detector; Su et al. survey 2025), but three verified findings bound what a learned detector actually buys: (1) a claim that a confidence threshold "cleanly separates" true from false matches even with a learned CNN front-end was **REFUTED** in verification (arXiv:1705.08593) -- the precision/recall wall persists even with learned features. (2) Learned detectors need a LARGE, pattern-diverse labeled dataset trained on synthetic composites (PRCV'18: 60k images / 80 watermark classes; CLWD: 60k / 160 marks), and off-distribution degradation is a documented real axis (models trained on limited-pattern LVW transfer worse; diversity of training patterns drives generalization). (3) Inference is cheap (WDNet ~8 ms at 256x256) -- the cost is the data pipeline, not runtime. Net: a learned detector shifts the frontier but does NOT remove the wall; for a SINGLE mark the cheapest next step is a small patch classifier (real-sparkle vs false-positive) on top of the existing NCC localizer, not a full segmentation model. SLBR is a ready baseline. The current NCC + false-positive gate (core-ring brightness margin + gradient-NCC crispness + white-core saturation) is a sound operating point, and the residual miss is the information-theoretic wall the literature confirms.
|
||||
|
||||
**Visible-mark landscape beyond the registry.** Meta stamps a visible "Imagined with AI" mark (bottom-LEFT, a small symbol) on its OWN Meta AI / "Imagine" output; for third-party images it relies on C2PA / IPTC, not a visible mark. Samsung Galaxy AI additionally uses a **four-star icon** variant in a corner alongside the localized text wordmark `samsung_engine` calibrates (only the Italian text variant is covered) -- the icon is a distinct, uncovered variant. Every source agrees visible + metadata marks are trivially removable (crop / screenshot, ~2 s), which is the tool's premise.
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
schema_version: 1
|
||||
|
||||
context:
|
||||
version: "0.21.2"
|
||||
python_min: "3.10"
|
||||
|
||||
package:
|
||||
name: remove-ai-watermarks
|
||||
version: ${{ version }}
|
||||
|
||||
source:
|
||||
url: https://pypi.org/packages/source/r/remove-ai-watermarks/remove_ai_watermarks-${{ version }}.tar.gz
|
||||
sha256: a013af322d40c137302c500638806c8864d5945699bdc2cdcbdcda3e2f37b3ac
|
||||
|
||||
build:
|
||||
noarch: python
|
||||
number: 0
|
||||
script: python -m pip install . -vv --no-deps --no-build-isolation
|
||||
|
||||
requirements:
|
||||
host:
|
||||
- python ${{ python_min }}.*
|
||||
- pip
|
||||
- hatchling
|
||||
run:
|
||||
- python >=${{ python_min }}
|
||||
- pillow >=10.0.0
|
||||
- piexif >=1.1.3
|
||||
- click >=8.0.0
|
||||
- python-dotenv >=1.0.0
|
||||
# c2pa-python is a default PyPI dependency but is not packaged on conda-forge.
|
||||
# The guarded import falls back to the built-in C2PA byte scanner when it is
|
||||
# absent. Add it here once a c2pa-python feedstock exists.
|
||||
|
||||
tests:
|
||||
- python:
|
||||
imports:
|
||||
- remove_ai_watermarks
|
||||
- remove_ai_watermarks.identify
|
||||
- remove_ai_watermarks.metadata
|
||||
python_version: ${{ python_min }}.*
|
||||
# pip_check defaults to true; disable it explicitly. `pip check` fails on
|
||||
# the conda-forge piexif `py_2` build, whose stale 2019 metadata pip reads
|
||||
# as "piexif 1.1.3 is not supported on this platform" -- even though the
|
||||
# package installs, imports, and works. The imports test verifies loading.
|
||||
pip_check: false
|
||||
|
||||
about:
|
||||
homepage: https://github.com/wiltodelta/remove-ai-watermarks
|
||||
summary: Remove visible and invisible AI watermarks from images
|
||||
description: |
|
||||
Inspect and strip AI-provenance metadata (C2PA, EXIF, IPTC, and PNG text
|
||||
chunks) from images. Optional pip extras add visible watermark removal,
|
||||
SynthID diffusion removal, and additional invisible-watermark detectors.
|
||||
license: Apache-2.0
|
||||
license_file: LICENSE
|
||||
repository: https://github.com/wiltodelta/remove-ai-watermarks
|
||||
|
||||
extra:
|
||||
recipe-maintainers:
|
||||
- wiltodelta
|
||||
+63
-53
@@ -1,9 +1,13 @@
|
||||
[project]
|
||||
# This name is the published PyPI distribution, not a local label: changing it
|
||||
# strands every `pip install remove-ai-watermarks` and every pinned dependency
|
||||
# on it, including the ComfyUI node package. The console script below carries
|
||||
# the same weight, since users have it on PATH.
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.22.0"
|
||||
description = "AI watermark remover: strip visible and invisible AI watermarks (Gemini / Nano Banana sparkle, SynthID) and provenance metadata (C2PA, EXIF) from images"
|
||||
version = "0.30.0"
|
||||
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10.1"
|
||||
requires-python = ">=3.11,<3.15"
|
||||
license = {text = "Apache-2.0"}
|
||||
keywords = [
|
||||
"ai-watermark",
|
||||
@@ -33,10 +37,10 @@ classifiers = [
|
||||
"License :: OSI Approved :: Apache Software License",
|
||||
"Operating System :: OS Independent",
|
||||
"Programming Language :: Python :: 3 :: Only",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Programming Language :: Python :: 3.14",
|
||||
"Topic :: Multimedia :: Graphics",
|
||||
"Topic :: Multimedia :: Graphics :: Graphics Conversion",
|
||||
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
||||
@@ -51,7 +55,7 @@ dependencies = [
|
||||
"python-dotenv>=1.0.0",
|
||||
# Official C2PA reader (Content Authenticity Initiative, MIT/Apache-2.0). The
|
||||
# primary, spec-tracking manifest parser for the identify/metadata path; the
|
||||
# hand-rolled caBX/CBOR scanner in noai/c2pa.py is kept only as a fallback for
|
||||
# hand-rolled caBX/CBOR scanner in _internal/c2pa.py is kept only as a fallback for
|
||||
# synthetic/partial blobs the validator rejects. Binary wheel (Rust), but the
|
||||
# import is light (no torch/numpy) so it fits the dependency-light identify
|
||||
# host. Prebuilt wheels cover the full CI matrix (linux/macos/windows).
|
||||
@@ -60,7 +64,10 @@ dependencies = [
|
||||
|
||||
[project.optional-dependencies]
|
||||
pixels = [
|
||||
"numpy>=1.24.0",
|
||||
"numpy>=1.24.0; python_version < '3.13'",
|
||||
# NumPy 1.x has no CPython 3.13+ wheels. Requiring 2.x prevents installers
|
||||
# from attempting an unsupported source build on the newer Python lines.
|
||||
"numpy>=2; python_version >= '3.13'",
|
||||
"opencv-python-headless>=4.8.0",
|
||||
]
|
||||
# Optional HEIC/AVIF pixel decode. Metadata scanning handles these containers
|
||||
@@ -69,6 +76,12 @@ heif = [
|
||||
"pillow-heif>=0.13.0",
|
||||
]
|
||||
visible = ["remove-ai-watermarks[pixels]"]
|
||||
# Video visible removal uses the shared pixel runtime. PyAV packetizes processed
|
||||
# VFR frames with explicit PTS before system ffmpeg encodes them.
|
||||
video = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
"av>=18,<19",
|
||||
]
|
||||
# Open DWT-DCT watermarks used by Stable Diffusion / SDXL / FLUX. The in-tree
|
||||
# decoder avoids the upstream invisible-watermark package's mandatory torch and
|
||||
# non-headless OpenCV dependencies.
|
||||
@@ -78,16 +91,11 @@ detect = [
|
||||
]
|
||||
diffusion = [
|
||||
"remove-ai-watermarks[pixels]",
|
||||
# A CUDA-enabled torch build is required: invisible-watermark removal has no
|
||||
# CPU, MPS or XPU path. The default PyPI wheel carries CUDA on Linux/Windows;
|
||||
# on macOS there is no CUDA build and this extra installs only for the
|
||||
# non-diffusion imports it shares.
|
||||
"torch>=2.0.0",
|
||||
# The default PyPI torch wheel is a CPU/CUDA build. To drive an Intel GPU
|
||||
# (Arc / Data Center) via ``--device xpu`` you need an XPU-enabled torch
|
||||
# from PyTorch's XPU wheel index (Linux/Windows only -- there is no macOS
|
||||
# XPU build). Install that build first, then this extra (torch is then
|
||||
# already satisfied and won't be re-pulled):
|
||||
# pip install torch --index-url https://download.pytorch.org/whl/xpu
|
||||
# pip install 'remove-ai-watermarks[diffusion]'
|
||||
# uv users can target the ``pytorch-xpu`` index declared under [tool.uv]:
|
||||
# uv pip install torch --index-url https://download.pytorch.org/whl/xpu
|
||||
"diffusers>=0.38.0",
|
||||
# diffusers 0.38's auto-pipeline registry imports ``Qwen3VLForConditional
|
||||
# Generation`` (its ``nucleusmoe_image`` pipeline), which only exists in
|
||||
@@ -112,12 +120,27 @@ qwen-zimage = [
|
||||
"diffsynth>=2.0.17,<3",
|
||||
"torchvision>=0.20.0",
|
||||
]
|
||||
# Opt-in verified-text reconstruction over qwen-zimage. LaMa removes the changed
|
||||
# candidate glyphs before exact Qwen-VAE cores are composited back.
|
||||
text-restoration = [
|
||||
"remove-ai-watermarks[qwen-zimage,lama]",
|
||||
]
|
||||
# Draft-only OCR proposals for verified-text manifests. CPU, no torch:
|
||||
# PaddleOCR detection plus three script-chosen recognition engines. The models
|
||||
# download on first use and are cached by huggingface_hub; never bundled.
|
||||
text-draft = [
|
||||
"paddleocr>=3.3.3",
|
||||
"paddlepaddle>=3.0.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Adobe TrustMark decoder -- the open, keyless watermark behind Adobe Durable
|
||||
# Content Credentials (soft-binding alg ``com.adobe.trustmark.P``). Optional
|
||||
# because it pulls torch and downloads model weights on first use. identify()
|
||||
# guards the import and skips the TrustMark signal when absent.
|
||||
trustmark = [
|
||||
"trustmark>=0.8.0",
|
||||
# TrustMark requires NumPy <2, whose last release has no CPython 3.13+
|
||||
# wheels. Keep this feature installable only on its real runtime range.
|
||||
"trustmark>=0.8.0; python_version < '3.13'",
|
||||
]
|
||||
# Universal region eraser backend -- big-LaMa via onnxruntime (Carve/LaMa-ONNX,
|
||||
# Apache-2.0). CPU, no torch. Model (~200 MB) is downloaded on first use and
|
||||
@@ -125,7 +148,7 @@ trustmark = [
|
||||
# eraser backend needs none of this.
|
||||
lama = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
"onnxruntime>=1.16.0",
|
||||
"onnxruntime>=1.24.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Lightweight inpaint backend -- MI-GAN via onnxruntime (andraniksargsyan/migan,
|
||||
@@ -135,24 +158,11 @@ lama = [
|
||||
# LaMa, the quality-first `auto` choice, is too large. Same runtime as `lama`.
|
||||
migan = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
"onnxruntime>=1.16.0",
|
||||
"onnxruntime>=1.24.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Optional pre-diffusion super-resolution for small inputs (Real-ESRGAN). Loaded via
|
||||
# spandrel (MIT) -- a pure model-loader with NO basicsr dependency (it pulls only
|
||||
# torch / torchvision / safetensors / numpy / einops).
|
||||
# The Real-ESRGAN weights (BSD-3-Clause) download on first use and are cached; they
|
||||
# are never bundled. CPU works but is slow on large inputs -- it is meant for the
|
||||
# pre-diffusion upscale of SMALL inputs (and the GPU worker). Guarded by
|
||||
# upscaler.is_available(); the default upscaler stays Lanczos (cv2, no deps). The
|
||||
# weights are fetched with torch.hub (bundled with spandrel's torch), so no extra
|
||||
# download dependency is needed.
|
||||
esrgan = [
|
||||
"remove-ai-watermarks[pixels]",
|
||||
"spandrel>=0.3.0",
|
||||
]
|
||||
dev = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
"remove-ai-watermarks[video]",
|
||||
"remove-ai-watermarks[detect]",
|
||||
"pytest>=8.0.0",
|
||||
"pytest-cov>=4.1.0",
|
||||
@@ -161,21 +171,10 @@ dev = [
|
||||
"ruff>=0.4.0",
|
||||
"pyright>=1.1.0",
|
||||
"invisible-watermark>=0.2.0",
|
||||
# maintain.sh helpers; they only support newer Pythons, so gate them by
|
||||
# marker to keep the py3.10 resolution (and CI matrix) solvable.
|
||||
"uv-outdated>=0.1.0; python_version >= '3.12'",
|
||||
"uv-secure>=0.12.0; python_version >= '3.12'",
|
||||
]
|
||||
all = ["remove-ai-watermarks[visible,heif,detect,trustmark,diffusion,qwen-zimage,lama,migan,esrgan]"]
|
||||
|
||||
# PyTorch Intel-GPU (XPU) wheel index. ``explicit = true`` keeps it inert for
|
||||
# the default CPU/CUDA install: uv consults it only when a torch install
|
||||
# explicitly targets it (see the ``diffusion`` extra comment), so it does not alter
|
||||
# the locked CPU/CUDA resolution. Linux/Windows only -- no macOS XPU build.
|
||||
[[tool.uv.index]]
|
||||
name = "pytorch-xpu"
|
||||
url = "https://download.pytorch.org/whl/xpu"
|
||||
explicit = true
|
||||
# ``qwen-zimage`` already pulls ``diffusion``; naming both would suggest diffusion is
|
||||
# independently sufficient for a removal, which it is not.
|
||||
all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan]"]
|
||||
|
||||
[project.scripts]
|
||||
remove-ai-watermarks = "remove_ai_watermarks.cli:main"
|
||||
@@ -191,9 +190,20 @@ build-backend = "hatchling.build"
|
||||
packages = ["src/remove_ai_watermarks"]
|
||||
|
||||
[tool.hatch.build.targets.sdist]
|
||||
# Keep the source distribution small: ship the package and metadata, not
|
||||
# calibration or evaluation data under data/. The wheel ships only src/.
|
||||
exclude = ["/data"]
|
||||
# Keep the source distribution small and public-safe: ship tracked source and
|
||||
# metadata, not corpora or local research/session artifacts. The wheel ships
|
||||
# only src/.
|
||||
include = [
|
||||
"/src",
|
||||
"/LICENSE",
|
||||
"/README.md",
|
||||
"/pyproject.toml",
|
||||
]
|
||||
exclude = [
|
||||
"/data",
|
||||
"/tmp",
|
||||
"/.sc",
|
||||
]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
@@ -201,7 +211,7 @@ pythonpath = ["src"]
|
||||
addopts = "-v --tb=short"
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py310"
|
||||
target-version = "py311"
|
||||
line-length = 120
|
||||
exclude = ["_refs"]
|
||||
extend-exclude = ["*.md"]
|
||||
@@ -216,15 +226,15 @@ ignore = [
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
"scripts/*.py" = ["G004", "S108", "S310", "T20"]
|
||||
"tests/*.py" = ["ANN", "S101", "S105", "S106", "S108"]
|
||||
"src/remove_ai_watermarks/noai/watermark_remover.py" = ["S603", "S606", "S607", "T201"] # subprocess calls for auto-install/CUDA fix
|
||||
"src/remove_ai_watermarks/noai/c2pa.py" = ["S110"] # try-except-pass for corrupt file handling
|
||||
"src/remove_ai_watermarks/_internal/watermark_remover.py" = ["S603", "S606", "S607"] # nvidia-smi capability probe
|
||||
"src/remove_ai_watermarks/_internal/c2pa.py" = ["S110"] # try-except-pass for corrupt file handling
|
||||
|
||||
[tool.ruff.format]
|
||||
quote-style = "double"
|
||||
indent-style = "space"
|
||||
|
||||
[tool.pyright]
|
||||
pythonVersion = "3.10"
|
||||
pythonVersion = "3.11"
|
||||
typeCheckingMode = "strict"
|
||||
exclude = ["_refs"]
|
||||
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Pure text-normalization helpers shared by evaluation scripts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unicodedata
|
||||
|
||||
|
||||
def normalize_text(text: str) -> str:
|
||||
"""Normalize text for layout-independent evaluation comparisons."""
|
||||
return "".join(unicodedata.normalize("NFC", text).casefold().split())
|
||||
|
||||
|
||||
def levenshtein_normalized(left: str, right: str) -> float:
|
||||
"""Return normalized Levenshtein distance without changing either input."""
|
||||
if not left and not right:
|
||||
return 0.0
|
||||
previous = list(range(len(right) + 1))
|
||||
for left_index, left_character in enumerate(left, start=1):
|
||||
current = [left_index]
|
||||
for right_index, right_character in enumerate(right, start=1):
|
||||
current.append(
|
||||
min(
|
||||
current[-1] + 1,
|
||||
previous[right_index] + 1,
|
||||
previous[right_index - 1] + (left_character != right_character),
|
||||
)
|
||||
)
|
||||
previous = current
|
||||
return previous[-1] / max(len(left), len(right))
|
||||
|
||||
|
||||
def normalized_edit_distance(left: str, right: str) -> float:
|
||||
"""Normalize text, then return Levenshtein distance over the result."""
|
||||
return levenshtein_normalized(normalize_text(left), normalize_text(right))
|
||||
@@ -1,436 +0,0 @@
|
||||
"""AI-generation scorer for scanned datasets.
|
||||
|
||||
Trains a gradient-boosted classifier on the metadata-derived labels of a
|
||||
scan_dataset.py output and scores every file, WITHOUT needing the
|
||||
metadata to be present at scoring time: the features are pixel and
|
||||
container statistics, so a metadata-stripped file still gets a score.
|
||||
The model is distribution-specific by design; evaluate it again before
|
||||
using it on an unrelated target distribution.
|
||||
|
||||
Modes:
|
||||
uv run --with scikit-learn python scripts/ai_score.py train <scan_glob> <model.pkl> [v1|v2]
|
||||
uv run --with scikit-learn python scripts/ai_score.py score <scan_glob> <model.pkl> <out.jsonl>
|
||||
|
||||
<scan_glob> is a glob of scan_dataset shards, e.g. 'data/scan/part_*.jsonl'.
|
||||
score output: one JSON line per file with file, sha256, ai_score, and the
|
||||
label evidence when a metadata label existed (for monitoring drift).
|
||||
"""
|
||||
|
||||
import glob
|
||||
import json
|
||||
import math
|
||||
import pickle
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
import numpy as np
|
||||
|
||||
# AI-generator names in C2PA claim_generator_info (metadata-level truth).
|
||||
AI_GENERATORS = (
|
||||
"openai",
|
||||
"adobe firefly",
|
||||
"adobe_firefly",
|
||||
"microsoft responsible ai",
|
||||
"microsoft_designer",
|
||||
"black forest labs",
|
||||
"fal-ai",
|
||||
"bria",
|
||||
"chatgpt",
|
||||
"stability",
|
||||
"dreamina",
|
||||
"canva",
|
||||
)
|
||||
# Human-origin software (negative evidence, NOT proof by itself).
|
||||
HUMAN_SOFTWARE = (
|
||||
"photoshop",
|
||||
"lightroom",
|
||||
"picsart",
|
||||
"snapseed",
|
||||
"paint.net",
|
||||
"gimp",
|
||||
"capture one",
|
||||
"meitu",
|
||||
"xingtu",
|
||||
"snow",
|
||||
)
|
||||
|
||||
_FEAT_SCALAR = [
|
||||
("noise", "noise_std"),
|
||||
("noise", "noise_kurtosis"),
|
||||
("fft", "cfa_peak"),
|
||||
("ela", "ela_mean"),
|
||||
("ela", "ela_p95"),
|
||||
("gradient", "laplacian_var"),
|
||||
("color", "saturation_mean"),
|
||||
("color", "value_mean"),
|
||||
("dct", "benford_mad"),
|
||||
]
|
||||
FeatureSchema = Literal["v1", "v2"]
|
||||
_FFT_BANDS = 8
|
||||
_GRADIENT_BINS = 10
|
||||
_COLOR_BINS = 64
|
||||
_DCT_AC_POSITIONS = 8
|
||||
_DCT_AC_BINS = 21
|
||||
_JPEG_QUANT_TABLES = 2
|
||||
_JPEG_QUANT_VALUES = 64
|
||||
_SCORE_BATCH = 4096
|
||||
|
||||
_V1_FEATURE_NAMES = (
|
||||
*(key for _, key in _FEAT_SCALAR),
|
||||
*(f"fft_band_energy_{index}" for index in range(_FFT_BANDS)),
|
||||
*(f"gradient_hist_{index}" for index in range(_GRADIENT_BINS)),
|
||||
*(f"color_hist_4x4x4_{index}" for index in range(_COLOR_BINS)),
|
||||
"jpeg_444",
|
||||
"jpeg_progressive",
|
||||
"megapixels",
|
||||
"aspect_ratio",
|
||||
"format_jpeg",
|
||||
"format_png",
|
||||
)
|
||||
_V2_EXTRA_FEATURE_NAMES = (
|
||||
"cfa_peak_0",
|
||||
"cfa_peak_1",
|
||||
*(f"dct_ac_{position}_{bin_index}" for position in range(_DCT_AC_POSITIONS) for bin_index in range(_DCT_AC_BINS)),
|
||||
*(f"jpeg_quant_{table}_{index}" for table in range(_JPEG_QUANT_TABLES) for index in range(_JPEG_QUANT_VALUES)),
|
||||
"jpeg_quant_table_count",
|
||||
"jpeg_huffman_table_count",
|
||||
"jpeg_huffman_total_bytes",
|
||||
"jpeg_scan_count",
|
||||
"jpeg_restart_interval",
|
||||
"jpeg_precision_bits",
|
||||
"jpeg_adobe_transform",
|
||||
"jpeg_jfif_present",
|
||||
"format_webp",
|
||||
"format_isobmff",
|
||||
"format_other",
|
||||
)
|
||||
_SCHEMA_FEATURE_NAMES: dict[str, tuple[str, ...]] = {
|
||||
"v1": _V1_FEATURE_NAMES,
|
||||
"v2": _V1_FEATURE_NAMES + _V2_EXTRA_FEATURE_NAMES,
|
||||
}
|
||||
|
||||
|
||||
def parse_schema(value: Any) -> FeatureSchema:
|
||||
"""Validate a feature-schema token (CLI argument or model-bundle field)."""
|
||||
if value == "v1" or value == "v2":
|
||||
return value
|
||||
raise ValueError(f"Unknown feature schema: {value!r}")
|
||||
|
||||
|
||||
def feature_names(schema: FeatureSchema = "v1") -> tuple[str, ...]:
|
||||
"""Return the stable ordered feature names for a model schema."""
|
||||
return _SCHEMA_FEATURE_NAMES[parse_schema(schema)]
|
||||
|
||||
|
||||
def _fixed_values(values: Any, count: int, *, normalize: bool = False) -> list[float]:
|
||||
"""Return a fixed-width float vector, padding absent values with NaN."""
|
||||
if not isinstance(values, (list, tuple)):
|
||||
return [float("nan")] * count
|
||||
result = [float(value) for value in values[:count]]
|
||||
if normalize and result:
|
||||
total = sum(value for value in result if math.isfinite(value))
|
||||
if total > 0:
|
||||
result = [value / total for value in result]
|
||||
return result + [float("nan")] * (count - len(result))
|
||||
|
||||
|
||||
def _optional_float(mapping: dict[str, Any], key: str) -> float:
|
||||
value = mapping.get(key)
|
||||
return float(value) if isinstance(value, (int, float)) else float("nan")
|
||||
|
||||
|
||||
def label_of(record: dict[str, Any]) -> int | None:
|
||||
"""1 = strong metadata AI, 0 = human-origin, None = unlabeled."""
|
||||
store = record.get("c2pa_store") or {}
|
||||
for m in (store.get("manifests") or {}).values():
|
||||
for cgi in m.get("claim_generator_info") or []:
|
||||
if any(g in str(cgi.get("name", "")).lower() for g in AI_GENERATORS):
|
||||
return 1
|
||||
for a in m.get("assertions") or []:
|
||||
d = a.get("data") if isinstance(a.get("data"), dict) else {}
|
||||
if "trainedAlgorithmicMedia" in str(d.get("digitalSourceType", "")):
|
||||
return 1
|
||||
for c in record.get("png_chunks", []):
|
||||
t = c.get("text", "")
|
||||
kw = t.split("\x00")[0]
|
||||
if kw in ("prompt", "workflow", "parameters") or "AIGC" in t[:80]:
|
||||
return 1
|
||||
exif = record.get("exif", {})
|
||||
z = exif.get("0th") or {}
|
||||
e = exif.get("Exif") or {}
|
||||
if z.get("Make") and z.get("Model") and (e.get("MakerNote") or e.get("LensModel") or e.get("LensMake")):
|
||||
return 0
|
||||
for c in record.get("png_chunks", []):
|
||||
if c.get("apple_screenshot_marker"):
|
||||
return 0
|
||||
for v in (record.get("iptc") or {}).values():
|
||||
if str(v).strip() == "Screenshot":
|
||||
return 0
|
||||
if any(t in str(z.get("Software", "")).lower() for t in HUMAN_SOFTWARE):
|
||||
return 0
|
||||
return None
|
||||
|
||||
|
||||
def features_of(record: dict[str, Any], *, schema: FeatureSchema = "v1") -> list[float]:
|
||||
"""The structural feature vector (pixel + container stats, no metadata)."""
|
||||
v = [float((record.get(s) or {}).get(k, np.nan)) for s, k in _FEAT_SCALAR]
|
||||
fft = record.get("fft") or {}
|
||||
gradient = record.get("gradient") or {}
|
||||
color = record.get("color") or {}
|
||||
v.extend(_fixed_values(fft.get("fft_band_energy"), _FFT_BANDS))
|
||||
v.extend(_fixed_values(gradient.get("gradient_hist"), _GRADIENT_BINS))
|
||||
v.extend(_fixed_values(color.get("color_hist_4x4x4"), _COLOR_BINS))
|
||||
jf = record.get("jpeg_forensics") or {}
|
||||
v.append(1.0 if jf.get("subsampling") == "4:4:4" else 0.0)
|
||||
v.append(1.0 if jf.get("progressive") else 0.0)
|
||||
pil = record.get("pil") or {}
|
||||
w, h = pil.get("width") or 0, pil.get("height") or 0
|
||||
v += [float(w * h) / 1e6, float(w) / max(h, 1)]
|
||||
fmt = record.get("content_format")
|
||||
v += [1.0 if fmt == "jpeg" else 0.0, 1.0 if fmt == "png" else 0.0]
|
||||
if schema == "v2":
|
||||
v.extend(_fixed_values(fft.get("cfa_peaks"), 2))
|
||||
dct_hist = (record.get("dct") or {}).get("dct_ac_hist")
|
||||
for position in range(_DCT_AC_POSITIONS):
|
||||
values = dct_hist[position] if isinstance(dct_hist, list) and position < len(dct_hist) else None
|
||||
v.extend(_fixed_values(values, _DCT_AC_BINS, normalize=True))
|
||||
quant_tables = jf.get("quant_tables")
|
||||
for table in range(_JPEG_QUANT_TABLES):
|
||||
values = quant_tables.get(str(table)) if isinstance(quant_tables, dict) else None
|
||||
v.extend(_fixed_values(values, _JPEG_QUANT_VALUES))
|
||||
is_jpeg = fmt == "jpeg"
|
||||
huffman_tables = jf.get("huffman_tables_hex")
|
||||
huffman_values = huffman_tables if isinstance(huffman_tables, list) else []
|
||||
v.extend(
|
||||
[
|
||||
float(len(quant_tables)) if isinstance(quant_tables, dict) else (0.0 if is_jpeg else float("nan")),
|
||||
float(len(huffman_values)) if is_jpeg else float("nan"),
|
||||
(
|
||||
float(sum(len(value) // 2 for value in huffman_values if isinstance(value, str)))
|
||||
if is_jpeg
|
||||
else float("nan")
|
||||
),
|
||||
_optional_float(jf, "scan_count"),
|
||||
_optional_float(jf, "restart_interval"),
|
||||
_optional_float(jf, "precision_bits"),
|
||||
_optional_float(jf, "adobe_transform"),
|
||||
1.0 if jf.get("jfif") else (0.0 if is_jpeg else float("nan")),
|
||||
1.0 if fmt == "webp" else 0.0,
|
||||
1.0 if isinstance(fmt, str) and fmt.startswith("isobmff:") else 0.0,
|
||||
(
|
||||
1.0
|
||||
if fmt is not None and fmt not in {"jpeg", "png", "webp"} and not str(fmt).startswith("isobmff:")
|
||||
else 0.0
|
||||
),
|
||||
]
|
||||
)
|
||||
elif schema != "v1":
|
||||
raise ValueError(f"Unknown feature schema: {schema}")
|
||||
expected = len(feature_names(schema))
|
||||
if len(v) != expected:
|
||||
raise ValueError(f"Feature schema {schema} produced {len(v)} values; expected {expected}")
|
||||
return v
|
||||
|
||||
|
||||
def grouped_stratified_split(
|
||||
labels: np.ndarray,
|
||||
hashes: np.ndarray,
|
||||
*,
|
||||
test_size: float = 0.3,
|
||||
random_state: int = 0,
|
||||
) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""Split records by hash while preserving label balance across hash groups."""
|
||||
if len(labels) != len(hashes):
|
||||
raise ValueError("labels and hashes must have equal length")
|
||||
if not 0 < test_size < 1:
|
||||
raise ValueError("test_size must be between 0 and 1")
|
||||
groups: dict[str, list[int]] = defaultdict(list)
|
||||
group_labels: dict[str, int] = {}
|
||||
for index, (label, digest) in enumerate(zip(labels, hashes, strict=True)):
|
||||
key = str(digest) if digest else f"__row_{index}"
|
||||
int_label = int(label)
|
||||
if key in group_labels and group_labels[key] != int_label:
|
||||
raise ValueError(f"Hash {key!r} has conflicting labels")
|
||||
group_labels[key] = int_label
|
||||
groups[key].append(index)
|
||||
keys_by_label: dict[int, list[str]] = defaultdict(list)
|
||||
for key, label in group_labels.items():
|
||||
keys_by_label[label].append(key)
|
||||
rng = np.random.default_rng(random_state)
|
||||
test_set: set[str] = set()
|
||||
for class_keys in keys_by_label.values():
|
||||
if len(class_keys) < 2:
|
||||
raise ValueError("Each label needs at least two distinct hash groups")
|
||||
shuffled = np.asarray(class_keys)
|
||||
rng.shuffle(shuffled)
|
||||
test_count = min(max(round(len(shuffled) * test_size), 1), len(shuffled) - 1)
|
||||
test_set.update(shuffled[:test_count].tolist())
|
||||
train_set = set(groups) - test_set
|
||||
train = np.asarray([index for key, indices in groups.items() if key in train_set for index in indices])
|
||||
test = np.asarray([index for key, indices in groups.items() if key in test_set for index in indices])
|
||||
return np.sort(train), np.sort(test)
|
||||
|
||||
|
||||
def temporal_holdout_split(
|
||||
dates: np.ndarray,
|
||||
hashes: np.ndarray,
|
||||
*,
|
||||
train_fraction: float = 0.7,
|
||||
) -> tuple[np.ndarray, np.ndarray, str]:
|
||||
"""Split chronologically and remove later copies of training hashes."""
|
||||
if len(dates) != len(hashes):
|
||||
raise ValueError("dates and hashes must have equal length")
|
||||
if not 0 < train_fraction < 1:
|
||||
raise ValueError("train_fraction must be between 0 and 1")
|
||||
unique_dates = sorted({str(date) for date in dates})
|
||||
if len(unique_dates) < 2:
|
||||
raise ValueError("Temporal holdout needs at least two distinct dates")
|
||||
cutoff_index = min(int(len(unique_dates) * train_fraction), len(unique_dates) - 1)
|
||||
cutoff = unique_dates[cutoff_index]
|
||||
train = np.flatnonzero(dates < cutoff)
|
||||
test = np.flatnonzero((dates >= cutoff) & ~np.isin(hashes, hashes[train]))
|
||||
return train, test, cutoff
|
||||
|
||||
|
||||
def model_schema(bundle: dict[str, Any]) -> FeatureSchema:
|
||||
"""Read a model's feature schema, defaulting legacy bundles to v1."""
|
||||
return parse_schema(bundle.get("feature_schema", "v1"))
|
||||
|
||||
|
||||
def iter_records(pattern: str) -> Any:
|
||||
for path in sorted(glob.glob(pattern)):
|
||||
with open(path) as fh:
|
||||
for line in fh:
|
||||
yield json.loads(line)
|
||||
|
||||
|
||||
def cmd_train(pattern: str, model_path: str, schema: FeatureSchema = "v2") -> None:
|
||||
from sklearn.ensemble import HistGradientBoostingClassifier
|
||||
from sklearn.impute import SimpleImputer
|
||||
from sklearn.metrics import average_precision_score, roc_auc_score
|
||||
|
||||
# keep the feature vector, not the record: a parsed scan record is an order of
|
||||
# magnitude larger than the row it collapses to.
|
||||
labeled: dict[str, tuple[list[float], int, str]] = {}
|
||||
missing_hash_index = 0
|
||||
for r in iter_records(pattern):
|
||||
if "noise" not in r:
|
||||
continue
|
||||
lab = label_of(r)
|
||||
if lab is None:
|
||||
continue
|
||||
digest = r.get("sha256")
|
||||
if not digest:
|
||||
digest = f"__missing_{missing_hash_index}"
|
||||
missing_hash_index += 1
|
||||
date = Path(r["file"]).parent.name
|
||||
previous = labeled.get(str(digest))
|
||||
if previous is not None and previous[1] != lab:
|
||||
raise ValueError(f"Hash {digest!r} has conflicting labels")
|
||||
if previous is None or date < previous[2]:
|
||||
labeled[str(digest)] = (features_of(r, schema=schema), lab, date)
|
||||
|
||||
rows, labels, row_dates = zip(*labeled.values(), strict=True)
|
||||
X = np.asarray(rows)
|
||||
y = np.asarray(labels)
|
||||
dates = np.asarray(row_dates)
|
||||
hashes = np.asarray(list(labeled))
|
||||
print(
|
||||
f"unique labeled: {len(y)} "
|
||||
f"(pos {int(y.sum())}, neg {int((1 - y).sum())}, features {X.shape[1]}, schema {schema})"
|
||||
)
|
||||
|
||||
def fit(train: np.ndarray) -> tuple[Any, Any]:
|
||||
rows = X[train]
|
||||
imputer = SimpleImputer(strategy="median")
|
||||
classifier = HistGradientBoostingClassifier(
|
||||
random_state=0,
|
||||
max_iter=300,
|
||||
l2_regularization=5.0,
|
||||
early_stopping=False,
|
||||
)
|
||||
classifier.fit(imputer.fit_transform(rows), y[train])
|
||||
return imputer, classifier
|
||||
|
||||
train, test = grouped_stratified_split(y, hashes, test_size=0.3, random_state=0)
|
||||
imp, clf = fit(train)
|
||||
p = clf.predict_proba(imp.transform(X[test]))[:, 1]
|
||||
print(f"SHA-grouped holdout: AUC {roc_auc_score(y[test], p):.4f} | AP {average_precision_score(y[test], p):.4f}")
|
||||
|
||||
temporal_train, temporal_test, cutoff = temporal_holdout_split(dates, hashes)
|
||||
if len(temporal_train) > 100 and len(temporal_test) > 100:
|
||||
imp_t, clf_t = fit(temporal_train)
|
||||
pt = clf_t.predict_proba(imp_t.transform(X[temporal_test]))[:, 1]
|
||||
auc_t = roc_auc_score(y[temporal_test], pt)
|
||||
ap_t = average_precision_score(y[temporal_test], pt)
|
||||
print(f"temporal (<{cutoff}): AUC {auc_t:.4f} | AP {ap_t:.4f}")
|
||||
|
||||
all_indices = np.arange(len(y))
|
||||
imp, clf = fit(all_indices)
|
||||
bundle = {
|
||||
"imputer": imp,
|
||||
"clf": clf,
|
||||
"feature_schema": schema,
|
||||
"feature_names": feature_names(schema),
|
||||
}
|
||||
with open(model_path, "wb") as f:
|
||||
pickle.dump(bundle, f)
|
||||
print(f"model written: {model_path}")
|
||||
|
||||
|
||||
def cmd_score(pattern: str, model_path: str, out_path: str) -> None:
|
||||
# the model file is produced locally by `train`; do not load third-party pickles
|
||||
with open(model_path, "rb") as f:
|
||||
bundle = pickle.load(f) # noqa: S301
|
||||
imp, clf = bundle["imputer"], bundle["clf"]
|
||||
schema = model_schema(bundle)
|
||||
stored_names = bundle.get("feature_names")
|
||||
if stored_names is not None and tuple(stored_names) != feature_names(schema):
|
||||
raise ValueError(f"Model feature names do not match schema {schema}")
|
||||
n = 0
|
||||
batch_rows: list[list[float]] = []
|
||||
batch_meta: list[dict[str, Any]] = []
|
||||
|
||||
def flush(out: Any) -> None:
|
||||
"""Score one batch: per-record predict_proba is dominated by call overhead."""
|
||||
nonlocal n
|
||||
if not batch_rows:
|
||||
return
|
||||
scores = clf.predict_proba(imp.transform(np.asarray(batch_rows)))[:, 1]
|
||||
for meta, score in zip(batch_meta, scores, strict=True):
|
||||
out.write(json.dumps({**meta, "ai_score": round(float(score), 4)}) + "\n")
|
||||
n += len(batch_rows)
|
||||
batch_rows.clear()
|
||||
batch_meta.clear()
|
||||
print(f" {n}", flush=True)
|
||||
|
||||
with open(out_path, "w") as out:
|
||||
for r in iter_records(pattern):
|
||||
if "noise" not in r:
|
||||
continue
|
||||
batch_rows.append(features_of(r, schema=schema))
|
||||
batch_meta.append({"file": r["file"], "sha256": r.get("sha256"), "metadata_label": label_of(r)})
|
||||
if len(batch_rows) >= _SCORE_BATCH:
|
||||
flush(out)
|
||||
flush(out)
|
||||
print(f"scored {n} -> {out_path}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
if len(sys.argv) < 2 or sys.argv[1] not in ("train", "score"):
|
||||
print(__doc__)
|
||||
sys.exit(2)
|
||||
if sys.argv[1] == "train" and len(sys.argv) in (4, 5):
|
||||
schema = parse_schema(sys.argv[4]) if len(sys.argv) == 5 else "v2"
|
||||
cmd_train(sys.argv[2], sys.argv[3], schema)
|
||||
elif sys.argv[1] == "score" and len(sys.argv) == 5:
|
||||
cmd_score(sys.argv[2], sys.argv[3], sys.argv[4])
|
||||
else:
|
||||
print(__doc__)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,241 @@
|
||||
"""Per-method wall time for metadata extraction and the verdict built on it.
|
||||
|
||||
WHY THIS EXISTS
|
||||
The detection path is documented by CAPABILITY -- which signals it reads, in what
|
||||
order, with what confidence -- but not by COST. Batch throughput and the question
|
||||
"which probe dominates, and does it depend on container or on file size" have never
|
||||
been measured.
|
||||
|
||||
WHAT IT MEASURES
|
||||
Only the file-backed half: metadata extraction, then the verdict evaluated on the
|
||||
extracted evidence. The pixel-backed detectors are deliberately out of scope.
|
||||
|
||||
COLD pass: one measurement on a file the process has never touched --
|
||||
``extract_provenance_evidence``. Only the first read of a file is genuinely cold,
|
||||
so it buys exactly one number, and that is the one a single-shot run pays.
|
||||
|
||||
WARM pass, with the filesystem cache now hot:
|
||||
1. ``extract_provenance_evidence`` as a whole.
|
||||
2. Its nine components, timed IN THE ORDER the dataclass constructs them and with
|
||||
the per-file caches cleared once beforehand -- so ``extract_c2pa_info`` carries
|
||||
the Rust manifest reader and the later ``get_ai_metadata`` sees the same warm
|
||||
cache it sees in production. Timing them in any other order moves that cost to
|
||||
a different row and flatters whichever ran second.
|
||||
3. ``identify_from_evidence``: pure verdict logic, the source is never reopened.
|
||||
4. ``identify(check_visible=False, check_invisible=False)`` -- extraction plus
|
||||
verdict as one call, the cross-check that 1 + 3 is the whole metadata path.
|
||||
|
||||
Every ``@lru_cache`` in the metadata and C2PA modules is cleared before each timed
|
||||
unit. Without that the second measurement of a file answers from the memo and
|
||||
reports a cost of zero -- the caches are keyed on (path, mtime, size) and this
|
||||
script reads each file several times.
|
||||
|
||||
READING IT
|
||||
Component times do NOT sum to the ``extract_provenance_evidence`` total for free:
|
||||
they come from a separate cache-cleared run, so the sum is a cross-check. A gap
|
||||
means a component is missing from the list. Both numbers are written.
|
||||
|
||||
DATA SAFETY
|
||||
Read-only over a local dataset. Writes only to the given output prefix, which
|
||||
belongs outside the repository.
|
||||
|
||||
uv run python scripts/detection_timing.py <dataset> <prefix> --limit 200
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable, Iterator
|
||||
|
||||
# The package's OWN tree, not the repository root: from a worktree, an editable
|
||||
# install resolves `remove_ai_watermarks` to the MAIN checkout, so a script measuring
|
||||
# this tree would silently import a different one.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
||||
|
||||
from remove_ai_watermarks import identify as identify_mod
|
||||
from remove_ai_watermarks import metadata as metadata_mod
|
||||
from remove_ai_watermarks._internal import c2pa as c2pa_mod
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SUPPORTED = frozenset({".png", ".jpg", ".jpeg", ".webp", ".heic", ".heif", ".avif"})
|
||||
|
||||
# Timed in the order ``ProvenanceEvidence`` is constructed in ``identify.py``. The
|
||||
# report script imports this to label its columns, so the order is the contract.
|
||||
COMPONENTS: tuple[tuple[str, Callable[[Path], Any]], ...] = (
|
||||
("c2pa_info", c2pa_mod.extract_c2pa_info),
|
||||
("ai_metadata", metadata_mod.get_ai_metadata),
|
||||
("scan_head", lambda p: metadata_mod.scan_head(p, identify_mod._SCAN_BYTES)),
|
||||
("iptc_ai_system", metadata_mod.iptc_ai_system),
|
||||
("aigc_label", metadata_mod.aigc_label),
|
||||
("exif_generator", metadata_mod.exif_generator),
|
||||
("xai_signature", metadata_mod.xai_signature),
|
||||
("huggingface_job", metadata_mod.huggingface_job),
|
||||
("samsung_genai", metadata_mod.samsung_genai),
|
||||
)
|
||||
|
||||
|
||||
def _cache_clearers() -> tuple[Callable[[], None], ...]:
|
||||
"""Every per-file memo in the metadata path, found by attribute, not by name list.
|
||||
|
||||
A hand-written list silently goes stale the next time a probe gains a cache, and a
|
||||
stale entry shows up as a suspiciously fast row rather than as an error.
|
||||
"""
|
||||
found: list[Callable[[], None]] = []
|
||||
for module in (metadata_mod, c2pa_mod):
|
||||
for name in dir(module):
|
||||
clear = getattr(getattr(module, name, None), "cache_clear", None)
|
||||
if callable(clear):
|
||||
found.append(clear)
|
||||
return tuple(found)
|
||||
|
||||
|
||||
_CLEARERS = _cache_clearers()
|
||||
|
||||
|
||||
def _clear() -> None:
|
||||
for clear in _CLEARERS:
|
||||
clear()
|
||||
|
||||
|
||||
def _ms(fn: Callable[[], Any]) -> tuple[float, Any]:
|
||||
"""Wall time in milliseconds plus the call's result."""
|
||||
start = time.perf_counter_ns()
|
||||
value = fn()
|
||||
return (time.perf_counter_ns() - start) / 1e6, value
|
||||
|
||||
|
||||
def _pixel_geometry(path: Path) -> tuple[str | None, int | None, int | None]:
|
||||
"""Container format and pixel dimensions from the header alone."""
|
||||
try:
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(path) as img:
|
||||
return img.format, img.width, img.height
|
||||
except Exception: # unreadable or an unsupported container
|
||||
return None, None, None
|
||||
|
||||
|
||||
def _warm_breakdown(path: Path, row: dict[str, Any]) -> None:
|
||||
"""Fill ``row`` with the warm-cache per-method breakdown."""
|
||||
_clear()
|
||||
row["extract_evidence_ms"], evidence = _ms(lambda: identify_mod.extract_provenance_evidence(path))
|
||||
|
||||
_clear()
|
||||
component_sum = 0.0
|
||||
for name, fn in COMPONENTS:
|
||||
elapsed, _ = _ms(lambda fn=fn: fn(path))
|
||||
row[f"meta_{name}_ms"] = elapsed
|
||||
component_sum += elapsed
|
||||
row["meta_components_sum_ms"] = component_sum
|
||||
|
||||
row["verdict_from_evidence_ms"], report = _ms(lambda: identify_mod.identify_from_evidence(evidence))
|
||||
|
||||
_clear()
|
||||
row["identify_metadata_only_ms"], full = _ms(
|
||||
lambda: identify_mod.identify(path, check_visible=False, check_invisible=False)
|
||||
)
|
||||
|
||||
row["scan_bytes"] = len(evidence.scan)
|
||||
row["has_c2pa"] = bool(evidence.c2pa_info)
|
||||
row["has_ai_metadata"] = bool(evidence.ai_metadata)
|
||||
row["is_ai_generated"] = full.is_ai_generated
|
||||
row["confidence"] = full.confidence
|
||||
row["platform"] = full.platform
|
||||
row["signals"] = [signal.name for signal in full.signals]
|
||||
# The two verdict paths must agree; a mismatch means the breakdown timed a
|
||||
# different code path than the end-to-end call and the rows are not comparable.
|
||||
row["verdict_agrees"] = (report.confidence, report.platform) == (full.confidence, full.platform)
|
||||
|
||||
|
||||
def _measure(path: Path) -> dict[str, Any]:
|
||||
row: dict[str, Any] = {"path": str(path), "ext": path.suffix.lower()}
|
||||
try:
|
||||
row["bytes"] = path.stat().st_size
|
||||
except OSError as exc:
|
||||
return {**row, "error": f"stat: {exc}"}
|
||||
|
||||
# COLD first: this is the only moment the file is untouched by this process.
|
||||
_clear()
|
||||
try:
|
||||
row["cold_extract_evidence_ms"], _ = _ms(lambda: identify_mod.extract_provenance_evidence(path))
|
||||
except Exception as exc:
|
||||
return {**row, "error": f"cold extract: {type(exc).__name__}: {exc}"}
|
||||
|
||||
row["format"], row["width"], row["height"] = _pixel_geometry(path)
|
||||
width, height = row["width"], row["height"]
|
||||
row["megapixels"] = round(width * height / 1e6, 3) if width and height else None
|
||||
|
||||
try:
|
||||
_warm_breakdown(path, row)
|
||||
except Exception as exc:
|
||||
row["error"] = f"warm pass: {type(exc).__name__}: {exc}"
|
||||
return row
|
||||
|
||||
|
||||
def _iter_images(root: Path) -> Iterator[Path]:
|
||||
for path in sorted(root.rglob("*")):
|
||||
if path.is_file() and path.suffix.lower() in SUPPORTED:
|
||||
yield path
|
||||
|
||||
|
||||
def _done_paths(out_path: Path) -> set[str]:
|
||||
"""Paths already recorded, so a long run resumes instead of restarting."""
|
||||
if not out_path.exists():
|
||||
return set()
|
||||
done: set[str] = set()
|
||||
with out_path.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
done.add(json.loads(line)["path"])
|
||||
except (ValueError, KeyError):
|
||||
continue
|
||||
return done
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("dataset", type=Path, help="directory of images, scanned recursively")
|
||||
parser.add_argument("out_prefix", type=Path, help="output prefix; writes <prefix>.jsonl")
|
||||
parser.add_argument("--limit", type=int, default=0, help="stop after N files (0 = all)")
|
||||
parser.add_argument("--progress-every", type=int, default=200)
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
|
||||
out_path = args.out_prefix.with_suffix(".jsonl")
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
done = _done_paths(out_path)
|
||||
if done:
|
||||
log.info("resuming: %d files already recorded", len(done))
|
||||
|
||||
processed = 0
|
||||
started = time.monotonic()
|
||||
with out_path.open("a", encoding="utf-8") as handle:
|
||||
for path in _iter_images(args.dataset):
|
||||
if str(path) in done:
|
||||
continue
|
||||
row = _measure(path)
|
||||
handle.write(json.dumps(row, ensure_ascii=False, default=str) + "\n")
|
||||
handle.flush()
|
||||
processed += 1
|
||||
if processed % args.progress_every == 0:
|
||||
log.info("%d files, %.2f files/s", processed, processed / (time.monotonic() - started))
|
||||
if args.limit and processed >= args.limit:
|
||||
break
|
||||
|
||||
elapsed = time.monotonic() - started
|
||||
rate = processed / max(elapsed, 1e-9)
|
||||
log.info("done: %d files in %.1f s (%.2f files/s) -> %s", processed, elapsed, rate, out_path)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,232 @@
|
||||
"""Aggregate ``detection_timing.py`` records into per-method, per-segment tables.
|
||||
|
||||
WHAT IT PRODUCES
|
||||
Reading it back: the per-file JSONL is one row per image with a millisecond field
|
||||
per method. This collapses it into percentiles per method, then repeats that per
|
||||
segment -- container, megapixels, file size, C2PA presence, verdict confidence --
|
||||
because a single median hides a path whose cost is carried entirely by one
|
||||
container or by the files that actually have a manifest.
|
||||
|
||||
Writes ``<prefix>_summary.csv`` (long form: segment_kind, segment, method, n, p50,
|
||||
p90, p99, mean) and prints a markdown report.
|
||||
|
||||
Percentiles are computed by nearest-rank on the sorted sample, not interpolated:
|
||||
every reported number is a time some real file actually took.
|
||||
|
||||
DATA SAFETY
|
||||
Reads and writes only the given prefix, which belongs outside the repository.
|
||||
|
||||
uv run python scripts/detection_timing_report.py .local-eval/timing/full
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import math
|
||||
import statistics
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable, Iterator
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from detection_timing import COMPONENTS as _TIMED
|
||||
|
||||
# Taken from the script that WROTE the rows, in its order, so a probe added, removed
|
||||
# or reordered there cannot silently leave a column missing or mislabelled here.
|
||||
COMPONENTS = tuple(name for name, _ in _TIMED)
|
||||
METHODS = (
|
||||
("cold_extract_evidence_ms", "extract_provenance_evidence (cold)"),
|
||||
("extract_evidence_ms", "extract_provenance_evidence (warm)"),
|
||||
*((f"meta_{name}_ms", f" {name}") for name in COMPONENTS),
|
||||
("meta_components_sum_ms", " (sum of components)"),
|
||||
("verdict_from_evidence_ms", "identify_from_evidence (verdict)"),
|
||||
("identify_metadata_only_ms", "identify(metadata only), end to end"),
|
||||
)
|
||||
# Median-only columns in the per-segment tables; the full percentile set goes to the CSV.
|
||||
HEADLINE_LABELS = (
|
||||
("extract_evidence_ms", "extract p50"),
|
||||
("verdict_from_evidence_ms", "verdict p50"),
|
||||
("identify_metadata_only_ms", "end-to-end p50"),
|
||||
)
|
||||
HEADLINE = tuple(key for key, _ in HEADLINE_LABELS)
|
||||
|
||||
|
||||
def _bucket(value: float | None, edges: tuple[float, ...], labels: tuple[str, ...]) -> str:
|
||||
if value is None:
|
||||
return "unknown"
|
||||
for edge, label in zip(edges, labels, strict=False):
|
||||
if value < edge:
|
||||
return label
|
||||
return labels[-1]
|
||||
|
||||
|
||||
SEGMENTS: tuple[tuple[str, Callable[[dict[str, Any]], str]], ...] = (
|
||||
("container", lambda r: str(r.get("format") or "unknown")),
|
||||
(
|
||||
"megapixels",
|
||||
lambda r: _bucket(r.get("megapixels"), (1, 4, 12), ("<1 MP", "1-4 MP", "4-12 MP", ">12 MP")),
|
||||
),
|
||||
(
|
||||
"file size",
|
||||
lambda r: _bucket(
|
||||
(r["bytes"] / 1e6) if r.get("bytes") is not None else None,
|
||||
(1, 5, 20),
|
||||
("<1 MB", "1-5 MB", "5-20 MB", ">20 MB"),
|
||||
),
|
||||
),
|
||||
("c2pa", lambda r: "with C2PA" if r.get("has_c2pa") else "no C2PA"),
|
||||
("verdict", lambda r: f"confidence={r.get('confidence') or 'n/a'}"),
|
||||
)
|
||||
|
||||
|
||||
def _numeric(records: list[dict[str, Any]], key: str) -> list[float]:
|
||||
"""The numeric values of ``key``, skipping rows where it is absent or non-numeric.
|
||||
|
||||
One definition of "counts as a measurement", so the printed table and the CSV can
|
||||
never disagree about which rows a method was measured on.
|
||||
"""
|
||||
return [float(r[key]) for r in records if isinstance(r.get(key), (int, float))]
|
||||
|
||||
|
||||
def _percentile(sorted_values: list[float], q: float) -> float:
|
||||
"""Nearest-rank percentile: the reported value is one a real file produced."""
|
||||
index = max(0, math.ceil(q * len(sorted_values)) - 1)
|
||||
return sorted_values[index]
|
||||
|
||||
|
||||
def _stats(values: list[float]) -> dict[str, float | int]:
|
||||
ordered = sorted(values)
|
||||
return {
|
||||
"n": len(ordered),
|
||||
"p50": _percentile(ordered, 0.50),
|
||||
"p90": _percentile(ordered, 0.90),
|
||||
"p99": _percentile(ordered, 0.99),
|
||||
"mean": statistics.fmean(ordered),
|
||||
}
|
||||
|
||||
|
||||
def _rows(path: Path) -> Iterator[dict[str, Any]]:
|
||||
with path.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
yield json.loads(line)
|
||||
except ValueError: # a truncated tail while the run is still writing
|
||||
continue
|
||||
|
||||
|
||||
def _table(rows: list[dict[str, Any]], methods: tuple[tuple[str, str], ...]) -> list[str]:
|
||||
out = ["| method | n | p50 | p90 | p99 | mean |", "|---|---:|---:|---:|---:|---:|"]
|
||||
for key, label in methods:
|
||||
values = _numeric(rows, key)
|
||||
if not values:
|
||||
continue
|
||||
s = _stats(values)
|
||||
out.append(f"| {label} | {s['n']} | {s['p50']:.3f} | {s['p90']:.3f} | {s['p99']:.3f} | {s['mean']:.3f} |")
|
||||
return out
|
||||
|
||||
|
||||
def _correlation(rows: list[dict[str, Any]], x_key: str, y_key: str) -> float | None:
|
||||
both = [r for r in rows if isinstance(r.get(x_key), (int, float)) and isinstance(r.get(y_key), (int, float))]
|
||||
if len(both) < 3:
|
||||
return None
|
||||
xs = _numeric(both, x_key)
|
||||
ys = _numeric(both, y_key)
|
||||
try:
|
||||
return statistics.correlation(xs, ys)
|
||||
except statistics.StatisticsError: # a constant column has no correlation
|
||||
return None
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("prefix", type=Path, help="prefix given to detection_timing.py")
|
||||
args = parser.parse_args()
|
||||
|
||||
jsonl = args.prefix.with_suffix(".jsonl")
|
||||
all_rows = list(_rows(jsonl))
|
||||
failed = [r for r in all_rows if "error" in r]
|
||||
rows = [r for r in all_rows if "error" not in r]
|
||||
if not rows:
|
||||
print(f"no usable records in {jsonl}")
|
||||
return 1
|
||||
|
||||
disagreed = [r for r in rows if r.get("verdict_agrees") is False]
|
||||
|
||||
lines = [
|
||||
f"# Detection timing over {len(rows)} images",
|
||||
"",
|
||||
f"Source: `{jsonl}`. Failed rows: {len(failed)}. Verdict-path mismatches: {len(disagreed)}.",
|
||||
"",
|
||||
"All times in milliseconds. Percentiles are nearest-rank.",
|
||||
"",
|
||||
"## Whole corpus",
|
||||
"",
|
||||
*_table(rows, METHODS),
|
||||
"",
|
||||
]
|
||||
|
||||
csv_rows: list[dict[str, Any]] = []
|
||||
for key, label in METHODS:
|
||||
values = _numeric(rows, key)
|
||||
if values:
|
||||
csv_rows.append({"segment_kind": "all", "segment": "all", "method": label.strip(), **_stats(values)})
|
||||
|
||||
for kind, classify in SEGMENTS:
|
||||
groups: dict[str, list[dict[str, Any]]] = defaultdict(list)
|
||||
for row in rows:
|
||||
groups[classify(row)].append(row)
|
||||
lines += [f"## By {kind}", ""]
|
||||
header = ["| segment | n | " + " | ".join(label for _, label in HEADLINE_LABELS) + " |"]
|
||||
header.append("|---|---:|" + "---:|" * len(HEADLINE))
|
||||
lines += header
|
||||
for segment, group in sorted(groups.items(), key=lambda item: -len(item[1])):
|
||||
cells = []
|
||||
for key in HEADLINE:
|
||||
values = _numeric(group, key)
|
||||
cells.append(f"{_percentile(sorted(values), 0.5):.2f}" if values else "-")
|
||||
lines.append(f"| {segment} | {len(group)} | " + " | ".join(cells) + " |")
|
||||
for key, label in METHODS:
|
||||
values = _numeric(group, key)
|
||||
if values:
|
||||
csv_rows.append(
|
||||
{"segment_kind": kind, "segment": segment, "method": label.strip(), **_stats(values)}
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
corr = _correlation(rows, "scan_bytes", "verdict_from_evidence_ms")
|
||||
corr_size = _correlation(rows, "bytes", "extract_evidence_ms")
|
||||
lines += [
|
||||
"## Scaling",
|
||||
"",
|
||||
f"- verdict time vs scan-buffer size: r = {corr:.3f}" if corr is not None else "- verdict correlation: n/a",
|
||||
(
|
||||
f"- extraction time vs file size: r = {corr_size:.3f}"
|
||||
if corr_size is not None
|
||||
else "- extraction correlation: n/a"
|
||||
),
|
||||
"",
|
||||
]
|
||||
|
||||
summary_csv = args.prefix.with_name(args.prefix.name + "_summary.csv")
|
||||
columns = ["segment_kind", "segment", "method", "n", "p50", "p90", "p99", "mean"]
|
||||
with summary_csv.open("w", encoding="utf-8", newline="") as handle:
|
||||
writer = csv.DictWriter(handle, fieldnames=columns)
|
||||
writer.writeheader()
|
||||
writer.writerows(csv_rows)
|
||||
|
||||
report = "\n".join(lines)
|
||||
args.prefix.with_name(args.prefix.name + "_report.md").write_text(report + "\n", encoding="utf-8")
|
||||
print(report)
|
||||
print(f"\nwrote {summary_csv}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -1,12 +1,11 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "numpy",
|
||||
# "opencv-python-headless",
|
||||
# "opencv-python-headless<5",
|
||||
# "pillow",
|
||||
# "scikit-image",
|
||||
# "rapidfuzz",
|
||||
# "torch",
|
||||
# "lpips",
|
||||
# "paddleocr",
|
||||
@@ -29,7 +28,7 @@ metrics run only where faces are detected, text metrics only where text is.
|
||||
|
||||
Two subcommands:
|
||||
|
||||
ocr -- OCR images (PaddleOCR PP-OCRv6) into a JSON {basename: text} file.
|
||||
ocr -- OCR images (PaddleOCR defaults) into a JSON {basename: text} file.
|
||||
Run this on the ORIGINALS, hand-verify/correct the file, and it
|
||||
becomes the ground truth for ``compare --ground-truth`` -- the clean
|
||||
way to score text, since OCR-vs-OCR is doubly noisy (errors on both
|
||||
@@ -54,6 +53,7 @@ Usage:
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import unicodedata
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
@@ -65,7 +65,10 @@ import numpy as np
|
||||
from _plain_console import Console, Table
|
||||
|
||||
console = Console()
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from scripts._text_eval import levenshtein_normalized # noqa: E402
|
||||
|
||||
# ── helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
@@ -89,7 +92,7 @@ def _norm(text: str) -> str:
|
||||
return "".join(unicodedata.normalize("NFC", text).split())
|
||||
|
||||
|
||||
# ── text: PaddleOCR (PP-OCRv6) ───────────────────────────────────────
|
||||
# ── text: PaddleOCR defaults ─────────────────────────────────────────
|
||||
|
||||
# Our lang codes -> PaddleOCR lang. The 'ch' model also reads Latin; 'ru' reads
|
||||
# Cyrillic + Latin. Multiple langs in one image -> run each model, union detections.
|
||||
@@ -165,9 +168,7 @@ def _ocr_lines(bgr: np.ndarray, langs: list[str], min_score: float = 0.5) -> lis
|
||||
|
||||
|
||||
def _cer(ref: str, hyp: str) -> float:
|
||||
from rapidfuzz.distance import Levenshtein
|
||||
|
||||
return Levenshtein.normalized_distance(_norm(ref), _norm(hyp))
|
||||
return levenshtein_normalized(_norm(ref), _norm(hyp))
|
||||
|
||||
|
||||
# ── face: detection + ArcFace + texture ──────────────────────────────
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "huggingface-hub>=0.20.0",
|
||||
# "numpy",
|
||||
# "onnxruntime>=1.24.0",
|
||||
# "opencv-python-headless<5",
|
||||
# "paddleocr>=3.3.3",
|
||||
# "paddlepaddle",
|
||||
# "pillow",
|
||||
# ]
|
||||
# ///
|
||||
"""Infer stable source-text lines without modifying an image.
|
||||
|
||||
This evaluation-only dry run proposes line annotations for selective text
|
||||
restoration. Every proposal still needs human verification: stable OCR can lose
|
||||
punctuation with high confidence. It separately flags lines whose recognition
|
||||
changes under crop jitter or whose minimum confidence is below the threshold.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
# Dogfoods the packaged draft API (remove_ai_watermarks.text_draft); this script
|
||||
# keeps only the CLI wrapper so the package stays the one home for the logic.
|
||||
from remove_ai_watermarks.text_draft import ( # noqa: E402
|
||||
_detect_line_boxes,
|
||||
_recognize,
|
||||
choose_language,
|
||||
stable_recognition,
|
||||
)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option("--out", required=True, type=click.Path(dir_okay=False, path_type=Path))
|
||||
@click.option("--min-score", default=0.85, show_default=True, type=click.FloatRange(0.0, 1.0))
|
||||
def main(source: Path, out: Path, min_score: float) -> None:
|
||||
"""Write draft line text for SOURCE; manually verify every proposal."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
|
||||
os.environ["PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK"] = "True"
|
||||
from paddleocr import PaddleOCR, TextRecognition
|
||||
|
||||
source_rgb = np.asarray(Image.open(source).convert("RGB"))
|
||||
detector = PaddleOCR(
|
||||
lang="ch",
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
)
|
||||
engines = {
|
||||
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec"),
|
||||
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec"),
|
||||
"ch": TextRecognition(model_name="PP-OCRv5_server_rec"),
|
||||
}
|
||||
boxes = _detect_line_boxes(detector, source_rgb)
|
||||
accepted = []
|
||||
rejected = []
|
||||
for box in boxes:
|
||||
probes = {}
|
||||
for language, engine in engines.items():
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
probes[language] = _recognize(engine, source_rgb, box, script, 0.1)
|
||||
language = choose_language(probes)
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
reads = [_recognize(engines[language], source_rgb, box, script, ratio) for ratio in (0.08, 0.12, 0.2)]
|
||||
text = stable_recognition(reads, min_score)
|
||||
result = {
|
||||
"box": box,
|
||||
"script": script,
|
||||
"language": language,
|
||||
"reads": [{"text": value, "score": score} for value, score in reads],
|
||||
}
|
||||
if text is None:
|
||||
rejected.append(result)
|
||||
else:
|
||||
accepted.append({"box": box, "text": text, "script": script, "min_score": min(score for _, score in reads)})
|
||||
payload = {"source": source.name, "accepted": accepted, "rejected": rejected}
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
out.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
log.info("Accepted %s lines and rejected %s uncertain lines", len(accepted), len(rejected))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,7 +1,7 @@
|
||||
"""How much recall is the coarse scale ladder costing, and what would a denser one cost?
|
||||
|
||||
THE FINDING THIS MEASURES
|
||||
`_tophat_score` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end
|
||||
`_ladder_best` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end
|
||||
sweeps none at all. Measured on stamped marks over controlled backgrounds
|
||||
(`scripts/detector_response.py`), the response is a COMB: doubao scores 0.99 exactly at
|
||||
each rung and collapses to 0.37-0.48 between them, against a 0.50 gate. So a mark whose
|
||||
@@ -93,7 +93,10 @@ def wilson(k: int, n: int, z: float = 1.96) -> tuple[float, float]:
|
||||
|
||||
|
||||
def score_at_scales(engine: Any, image: np.ndarray, scales: tuple[float, ...]) -> dict[float, float]:
|
||||
"""`_tophat_score`'s inner loop, opened up so the ladder is a parameter.
|
||||
"""`_ladder_best`'s inner loop, opened up so the ladder is a parameter.
|
||||
|
||||
A DELIBERATE second copy of the sweep: it exists to measure ladders the shipped
|
||||
engine does not have, so it must not be folded back into `_ladder_best`.
|
||||
|
||||
Deliberately reaches into the engine (`tophat_response`, `_glyph_silhouette`): a
|
||||
measurement script may, product code may not. Kept a faithful copy of the shipped
|
||||
|
||||
@@ -32,7 +32,7 @@ from pathlib import Path
|
||||
import click
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.noai.constants import SUPPORTED_FORMATS
|
||||
from remove_ai_watermarks._internal.constants import SUPPORTED_FORMATS
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "diffusers>=0.35.0",
|
||||
# "transformers>=4.51.0",
|
||||
|
||||
@@ -13,20 +13,20 @@ WHAT IT COVERS
|
||||
metadata real AI-metadata files -> --check detects, --remove strip-and-verifies clean
|
||||
visible real marked images per mark -> the mark is gone on re-detect, output written
|
||||
erase a real image, each fill backend (cv2 / migan / lama) -> output written
|
||||
invisible a real SynthID image on MPS at reduced resolution -> a CHANGED image is written
|
||||
invisible a real SynthID image at reduced resolution -> a CHANGED image is written
|
||||
all a real marked image through the full pipeline -> output written
|
||||
batch a real directory -> every input produces an output
|
||||
|
||||
invisible/all run the diffusion model, so they are gated behind --diffusion and run at a
|
||||
small --max-resolution on MPS (the user's "reduced size on MPS" path). Everything else is
|
||||
cv2/numpy and fast.
|
||||
small --max-resolution. Both profiles are CUDA-only, so that section needs an NVIDIA
|
||||
GPU; everything else is cv2/numpy, fast, and runs anywhere.
|
||||
|
||||
DATA SAFETY
|
||||
Treat input datasets as sensitive and read-only. Output stays in a gitignored temp
|
||||
dir. Records example uids and pass/fail, never image content.
|
||||
|
||||
uv run python scripts/real_examples_e2e.py # fast surface (no diffusion)
|
||||
uv run python scripts/real_examples_e2e.py --diffusion # + invisible/all on MPS
|
||||
uv run python scripts/real_examples_e2e.py --diffusion # + invisible/all (needs CUDA)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -239,12 +239,16 @@ def check_erase(res: Results, tmp: Path) -> None:
|
||||
|
||||
|
||||
def check_diffusion(res: Results, tmp: Path, gemini_src: str, openai_src: str) -> None:
|
||||
"""The GPU path: invisible + all on MPS at reduced resolution -> a CHANGED image."""
|
||||
"""The GPU path: invisible + all at reduced resolution -> a CHANGED image.
|
||||
|
||||
CUDA-only. On a machine without an NVIDIA GPU every row here fails with the
|
||||
library's clean refusal; run this section on a GPU box.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
|
||||
print("\ninvisible / all -- real SynthID image on MPS, reduced resolution")
|
||||
print("\ninvisible / all -- real SynthID image, reduced resolution (CUDA required)")
|
||||
for label, src in (("invisible/gemini", gemini_src), ("invisible/openai", openai_src)):
|
||||
if not src:
|
||||
res.add(label, "-", True, "no real positive found (skipped)")
|
||||
@@ -252,7 +256,7 @@ def check_diffusion(res: Results, tmp: Path, gemini_src: str, openai_src: str) -
|
||||
sp = Path(src)
|
||||
outp = tmp / f"inv_{sp.stem}.png"
|
||||
code, out = run(
|
||||
["invisible", src, "-o", str(outp), "--device", "mps", "--max-resolution", "512", "--seed", "0"],
|
||||
["invisible", src, "-o", str(outp), "--max-resolution", "512", "--seed", "0"],
|
||||
timeout=1200,
|
||||
)
|
||||
if not outp.exists():
|
||||
@@ -276,7 +280,7 @@ def check_diffusion(res: Results, tmp: Path, gemini_src: str, openai_src: str) -
|
||||
sp = Path(gemini_src)
|
||||
outp = tmp / f"all_{sp.stem}.png"
|
||||
code, out = run(
|
||||
["all", gemini_src, "-o", str(outp), "--device", "mps", "--max-resolution", "512", "--seed", "0"],
|
||||
["all", gemini_src, "-o", str(outp), "--max-resolution", "512", "--seed", "0"],
|
||||
timeout=1200,
|
||||
)
|
||||
ok = outp.exists() and outp.stat().st_size > 0
|
||||
@@ -304,7 +308,7 @@ def check_batch(res: Results, tmp: Path) -> None:
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--diffusion", action="store_true", help="also run invisible/all on MPS (slow)")
|
||||
ap.add_argument("--diffusion", action="store_true", help="also run invisible/all (slow, needs CUDA)")
|
||||
ap.add_argument("--gemini", default="")
|
||||
ap.add_argument("--openai", default="")
|
||||
a = ap.parse_args()
|
||||
@@ -320,7 +324,7 @@ def main() -> None:
|
||||
if a.diffusion:
|
||||
check_diffusion(res, tmp, a.gemini, a.openai)
|
||||
else:
|
||||
print("\n(diffusion skipped -- pass --diffusion to run invisible/all on MPS)")
|
||||
print("\n(diffusion skipped -- pass --diffusion to run invisible/all)")
|
||||
raise SystemExit(res.report())
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
"""Audit the record path against the file path over a whole dataset.
|
||||
|
||||
WHY THIS EXISTS
|
||||
|
||||
Two seams reach the same provenance verdict:
|
||||
|
||||
identify(path, check_visible=False, check_invisible=False)
|
||||
|
||||
identify_metadata_record(collect_metadata_record(path), path=path)
|
||||
|
||||
Their equality is the record's entire contract, and it can break from either side --
|
||||
a region the collector stops walking, or a placement the file path learns to read and
|
||||
the record does not. ``tests/test_metadata_record.py`` pins it over the tracked
|
||||
fixtures; those cover the signal families we already know about. This covers the ones
|
||||
we do not: every real placement in a real corpus, which is where all three defects
|
||||
found so far actually came from.
|
||||
|
||||
The record is round-tripped through ``json.dumps``/``loads`` before it is judged, so
|
||||
a value that only survives in memory fails here rather than at a customer.
|
||||
|
||||
WHAT IT REPORTS
|
||||
|
||||
One JSONL row per image: both verdicts, whether they agree, the record size, and any
|
||||
exception from either side. The summary counts disagreements by field and by signal,
|
||||
so "the record lost samsung_genai on 13 files" reads directly off the output instead
|
||||
of being reconstructed.
|
||||
|
||||
Pass ``--baseline`` with an earlier run to also diff against it. That answers the
|
||||
other question a detection change raises: which files changed verdict, and are they
|
||||
exactly the ones that were meant to.
|
||||
|
||||
DATA SAFETY
|
||||
|
||||
Read-only over a local dataset. Writes only the given output path, which belongs
|
||||
outside the repository. Resumable: rerunning skips files already recorded.
|
||||
|
||||
uv run python scripts/record_parity_audit.py data/spaces/originals .local-eval/parity.jsonl
|
||||
uv run python scripts/record_parity_audit.py <dataset> <out> --baseline .local-eval/previous.jsonl
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import collections
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
# The package's OWN tree, not the repository root: from a worktree, an editable
|
||||
# install resolves `remove_ai_watermarks` to the MAIN checkout, so a script measuring
|
||||
# this tree would silently import a different one.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
||||
|
||||
from remove_ai_watermarks.identify import identify, identify_metadata_record
|
||||
from remove_ai_watermarks.metadata_record import collect_metadata_record
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SUPPORTED = frozenset({".png", ".jpg", ".jpeg", ".webp", ".heic", ".heif", ".avif"})
|
||||
# Every field of the verdict a caller can act on. `path` is excluded: it is extraction
|
||||
# context, and the two paths are handed the same one by construction.
|
||||
COMPARED = ("is_ai_generated", "platform", "confidence", "ai_source_kind", "ai_from_metadata")
|
||||
|
||||
|
||||
def _verdict(report: Any) -> dict[str, Any]:
|
||||
return {
|
||||
**{field: getattr(report, field) for field in COMPARED},
|
||||
"signals": sorted(signal.name for signal in report.signals),
|
||||
"watermarks": sorted(report.watermarks),
|
||||
}
|
||||
|
||||
|
||||
def _audit(path: Path) -> dict[str, Any]:
|
||||
row: dict[str, Any] = {"path": str(path)}
|
||||
try:
|
||||
row["bytes"] = path.stat().st_size
|
||||
except OSError as exc:
|
||||
return {**row, "error": f"stat: {exc}"}
|
||||
|
||||
try:
|
||||
started = time.perf_counter()
|
||||
record = json.loads(json.dumps(collect_metadata_record(path)))
|
||||
row["collect_ms"] = (time.perf_counter() - started) * 1000
|
||||
row["record_bytes"] = len(json.dumps(record))
|
||||
row["container"] = record.get("container")
|
||||
via_record = _verdict(identify_metadata_record(record, path=path))
|
||||
except Exception as exc:
|
||||
return {**row, "error": f"record path: {type(exc).__name__}: {exc}"}
|
||||
|
||||
try:
|
||||
via_file = _verdict(identify(path, check_visible=False, check_invisible=False))
|
||||
except Exception as exc:
|
||||
return {**row, "error": f"file path: {type(exc).__name__}: {exc}"}
|
||||
|
||||
row["record"] = via_record
|
||||
row["file"] = via_file
|
||||
row["agree"] = via_record == via_file
|
||||
return row
|
||||
|
||||
|
||||
def _iter_images(root: Path) -> Iterator[Path]:
|
||||
for path in sorted(root.rglob("*")):
|
||||
if path.is_file() and path.suffix.lower() in SUPPORTED:
|
||||
yield path
|
||||
|
||||
|
||||
def _done(out_path: Path) -> set[str]:
|
||||
if not out_path.exists():
|
||||
return set()
|
||||
done: set[str] = set()
|
||||
with out_path.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
done.add(json.loads(line)["path"])
|
||||
except (ValueError, KeyError):
|
||||
continue
|
||||
return done
|
||||
|
||||
|
||||
def _summarize(rows: list[dict[str, Any]], baseline: Path | None) -> None:
|
||||
failed = [r for r in rows if "error" in r]
|
||||
usable = [r for r in rows if "error" not in r]
|
||||
disagreed = [r for r in usable if not r["agree"]]
|
||||
|
||||
print(f"\nimages: {len(rows)} errors: {len(failed)} compared: {len(usable)}")
|
||||
print(f"record path disagrees with file path: {len(disagreed)}")
|
||||
for row in failed[:10]:
|
||||
print(f" ERROR {Path(row['path']).name}: {row['error']}")
|
||||
|
||||
fields: collections.Counter[str] = collections.Counter()
|
||||
for row in disagreed:
|
||||
fields.update(field for field in COMPARED if row["record"][field] != row["file"][field])
|
||||
for name in set(row["file"]["signals"]) - set(row["record"]["signals"]):
|
||||
fields[f"signal missing from record: {name}"] += 1
|
||||
for name in set(row["record"]["signals"]) - set(row["file"]["signals"]):
|
||||
fields[f"signal only in record: {name}"] += 1
|
||||
for label, count in fields.most_common():
|
||||
print(f" {label}: {count}")
|
||||
for row in disagreed[:10]:
|
||||
print(f" {Path(row['path']).name}\n record: {row['record']}\n file: {row['file']}")
|
||||
|
||||
if baseline is None:
|
||||
return
|
||||
previous = {}
|
||||
with baseline.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
item = json.loads(line)
|
||||
except ValueError:
|
||||
continue
|
||||
if "error" not in item:
|
||||
previous[Path(item["path"]).name] = item
|
||||
|
||||
changed = []
|
||||
for row in usable:
|
||||
was = previous.get(Path(row["path"]).name)
|
||||
if was is None:
|
||||
continue
|
||||
# The WHOLE verdict, not a chosen subset. A first version compared confidence
|
||||
# and signals only and reported "0 changed" for a run whose single intended
|
||||
# correction was a watermark line -- the change it existed to show.
|
||||
before = was.get("file") or {}
|
||||
if before and before != row["file"]:
|
||||
changed.append((row["path"], before, row["file"]))
|
||||
print(f"\nverdicts changed against the baseline: {len(changed)}")
|
||||
moved: collections.Counter[str] = collections.Counter()
|
||||
for _, before, after in changed:
|
||||
for name in set(after["signals"]) - set(before.get("signals") or []):
|
||||
moved[f"gained signal {name}"] += 1
|
||||
for name in set(before.get("signals") or []) - set(after["signals"]):
|
||||
moved[f"LOST signal {name}"] += 1
|
||||
for field in (*COMPARED, "watermarks"):
|
||||
if before.get(field) != after.get(field):
|
||||
moved[f"{field} changed"] += 1
|
||||
for label, count in moved.most_common():
|
||||
print(f" {label}: {count}")
|
||||
for path, before, after in changed[:10]:
|
||||
print(f" {Path(path).name}\n before: {before}\n after: {after}")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("dataset", type=Path)
|
||||
parser.add_argument("out", type=Path)
|
||||
parser.add_argument("--baseline", type=Path, default=None, help="an earlier run to diff verdicts against")
|
||||
parser.add_argument("--limit", type=int, default=0)
|
||||
parser.add_argument("--progress-every", type=int, default=2000)
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
|
||||
args.out.parent.mkdir(parents=True, exist_ok=True)
|
||||
done = _done(args.out)
|
||||
if done:
|
||||
log.info("resuming: %d images already audited", len(done))
|
||||
|
||||
processed = 0
|
||||
started = time.monotonic()
|
||||
with args.out.open("a", encoding="utf-8") as handle:
|
||||
for path in _iter_images(args.dataset):
|
||||
if str(path) in done:
|
||||
continue
|
||||
handle.write(json.dumps(_audit(path), ensure_ascii=False, default=str) + "\n")
|
||||
handle.flush()
|
||||
processed += 1
|
||||
if processed % args.progress_every == 0:
|
||||
log.info("%d images, %.1f/s", processed, processed / (time.monotonic() - started))
|
||||
if args.limit and processed >= args.limit:
|
||||
break
|
||||
|
||||
log.info("audited %d images in %.1f s", processed, time.monotonic() - started)
|
||||
with args.out.open(encoding="utf-8") as handle:
|
||||
rows = [json.loads(line) for line in handle]
|
||||
_summarize(rows, args.baseline)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,528 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "huggingface-hub>=0.20.0",
|
||||
# "numpy",
|
||||
# "onnxruntime>=1.24.0",
|
||||
# "opencv-python-headless<5",
|
||||
# "paddleocr>=3.3.3",
|
||||
# "paddlepaddle",
|
||||
# "pillow",
|
||||
# ]
|
||||
# ///
|
||||
"""Evaluation-only text restoration over a scrubbed image.
|
||||
|
||||
``vae-glyphs`` composites only thresholded glyph-core pixels from a separately
|
||||
generated VAE reconstruction over a fresh silhouette edge. ``source-glyphs``
|
||||
can preserve typeface, layout, color, and antialiasing when
|
||||
SOURCE is itself a regenerated layer. It must not be treated as safe when SOURCE
|
||||
is the watermarked original: a provider oracle detected SynthID after that exact
|
||||
paste-back experiment. ``source-silhouette`` instead transfers only a
|
||||
thresholded glyph shape, then synthesizes fresh flat-color pixels and
|
||||
antialiasing. ``rerender`` retains the system-font negative control. This is
|
||||
not a production stage and does not add PaddleOCR to the package graph.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import unicodedata
|
||||
from dataclasses import asdict
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import click
|
||||
import cv2
|
||||
import numpy as np
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
sys.path.insert(0, str(ROOT / "src"))
|
||||
|
||||
from remove_ai_watermarks import region_eraser # noqa: E402
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextLine as TextLine # noqa: E402
|
||||
from remove_ai_watermarks._internal.text_restoration import ( # noqa: E402
|
||||
composite_fresh_text_edges,
|
||||
composite_reconstructed_glyphs,
|
||||
group_text_lines,
|
||||
residual_glyph_mask,
|
||||
source_silhouette_mask,
|
||||
)
|
||||
from scripts._text_eval import normalize_text, normalized_edit_distance # noqa: E402
|
||||
|
||||
if ROOT not in Path(region_eraser.__file__).resolve().parents:
|
||||
raise RuntimeError("selective_text_restoration imported outside the current worktree")
|
||||
|
||||
REGULAR_FONT = Path("/System/Library/Fonts/Supplemental/Arial.ttf")
|
||||
BOLD_FONT = Path("/System/Library/Fonts/Supplemental/Arial Bold.ttf")
|
||||
CJK_FONT = Path("/System/Library/Fonts/STHeiti Medium.ttc")
|
||||
|
||||
|
||||
def should_preserve_line(
|
||||
expected: str,
|
||||
source_text: str,
|
||||
source_score: float,
|
||||
candidate_text: str,
|
||||
candidate_score: float,
|
||||
) -> bool:
|
||||
if min(source_score, candidate_score) < 0.75:
|
||||
return False
|
||||
if normalized_edit_distance(expected, source_text) > 0.25:
|
||||
return False
|
||||
return normalize_text(source_text) == normalize_text(candidate_text)
|
||||
|
||||
|
||||
def composite_source_glyphs(
|
||||
source_rgb: np.ndarray,
|
||||
background_rgb: np.ndarray,
|
||||
glyph_mask: np.ndarray,
|
||||
*,
|
||||
feather: float = 0.7,
|
||||
) -> np.ndarray:
|
||||
"""Composite exact source pixels inside a glyph mask with an outer feather."""
|
||||
if source_rgb.shape != background_rgb.shape or source_rgb.shape[:2] != glyph_mask.shape:
|
||||
raise ValueError("source, background, and glyph mask dimensions must match")
|
||||
blurred = cv2.GaussianBlur(glyph_mask, (0, 0), feather) if feather > 0 else glyph_mask
|
||||
alpha = np.maximum(glyph_mask, blurred).astype(np.float32) / 255.0
|
||||
alpha = alpha[..., None]
|
||||
combined = source_rgb.astype(np.float32) * alpha + background_rgb.astype(np.float32) * (1.0 - alpha)
|
||||
return np.clip(combined, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def source_box_mask(
|
||||
shape: tuple[int, int],
|
||||
boxes: list[tuple[int, int, int, int]],
|
||||
) -> np.ndarray:
|
||||
"""Build a padded text-line mask for aligned regenerated layer compositing."""
|
||||
height, width = shape
|
||||
mask = np.zeros((height, width), dtype=np.uint8)
|
||||
for x1, y1, x2, y2 in boxes:
|
||||
pad = max(8, (y2 - y1) // 4)
|
||||
x1, y1, x2, y2 = _clip_box((x1, y1, x2, y2), width, height, pad=pad)
|
||||
mask[y1:y2, x1:x2] = 255
|
||||
return mask
|
||||
|
||||
|
||||
def _clip_box(box: tuple[int, int, int, int], width: int, height: int, pad: int = 0) -> tuple[int, int, int, int]:
|
||||
x1, y1, x2, y2 = box
|
||||
return max(0, x1 - pad), max(0, y1 - pad), min(width, x2 + pad), min(height, y2 + pad)
|
||||
|
||||
|
||||
def foreground_mask(source_rgb: np.ndarray, box: tuple[int, int, int, int]) -> np.ndarray:
|
||||
height, width = source_rgb.shape[:2]
|
||||
line_height = box[3] - box[1]
|
||||
x1, y1, x2, y2 = _clip_box(box, width, height, pad=max(6, int(line_height * 0.12)))
|
||||
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
|
||||
ring_pad = max(8, min(24, (y2 - y1) // 5))
|
||||
rx1, ry1, rx2, ry2 = _clip_box((x1, y1, x2, y2), width, height, pad=ring_pad)
|
||||
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
|
||||
ring = np.ones(context.shape, dtype=bool)
|
||||
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
|
||||
background_luma = float(np.median(context[ring])) if ring.any() else float(np.median(gray))
|
||||
low, high = float(np.percentile(gray, 4)), float(np.percentile(gray, 96))
|
||||
dark_contrast, light_contrast = background_luma - low, high - background_luma
|
||||
contrast = max(light_contrast, dark_contrast)
|
||||
threshold = max(24.0, min(72.0, contrast * 0.32))
|
||||
if light_contrast > dark_contrast:
|
||||
mask = (gray.astype(np.float32) >= background_luma + threshold).astype(np.uint8) * 255
|
||||
else:
|
||||
mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
|
||||
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((2, 2), np.uint8))
|
||||
dilation = 5 if line_height >= 48 else 3
|
||||
mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * dilation + 1,) * 2))
|
||||
result = np.zeros((height, width), dtype=np.uint8)
|
||||
result[y1:y2, x1:x2] = mask
|
||||
return result
|
||||
|
||||
|
||||
def _line_language(line: TextLine) -> str:
|
||||
if line.script == "cjk":
|
||||
return "ch"
|
||||
return "ru" if any("CYRILLIC" in unicodedata.name(character, "") for character in line.text) else "en"
|
||||
|
||||
|
||||
def _recognition_box(
|
||||
line: TextLine,
|
||||
width: int,
|
||||
height: int,
|
||||
vertical_pad_ratio: float | None = None,
|
||||
) -> tuple[int, int, int, int]:
|
||||
x1, y1, x2, y2 = line.box
|
||||
line_height = y2 - y1
|
||||
if line.script == "cjk":
|
||||
left_pad = max(16, round(line_height * 0.2))
|
||||
right_pad = max(16, round(line_height * 0.6))
|
||||
return max(0, x1 - left_pad), y1, min(width, x2 + right_pad), y2
|
||||
pad_x = max(16, line_height)
|
||||
pad_y = max(8, line_height // 3) if vertical_pad_ratio is None else max(8, round(line_height * vertical_pad_ratio))
|
||||
return max(0, x1 - pad_x), max(0, y1 - pad_y), min(width, x2 + pad_x), min(height, y2 + pad_y)
|
||||
|
||||
|
||||
def _recognize(
|
||||
engine: Any,
|
||||
image: np.ndarray,
|
||||
line: TextLine,
|
||||
vertical_pad_ratio: float | None = None,
|
||||
) -> tuple[str, float]:
|
||||
height, width = image.shape[:2]
|
||||
x1, y1, x2, y2 = _recognition_box(line, width, height, vertical_pad_ratio)
|
||||
crop = image[y1:y2, x1:x2]
|
||||
if crop.shape[0] < 64:
|
||||
scale = 64 / crop.shape[0]
|
||||
crop = cv2.resize(crop, None, fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)
|
||||
result = next(iter(engine.predict(crop)))
|
||||
return str(result.get("rec_text", "")), float(result.get("rec_score", 0.0))
|
||||
|
||||
|
||||
def _sample_text_color(
|
||||
source_rgb: np.ndarray,
|
||||
mask: np.ndarray,
|
||||
box: tuple[int, int, int, int],
|
||||
) -> tuple[int, int, int]:
|
||||
height, width = source_rgb.shape[:2]
|
||||
x1, y1, x2, y2 = _clip_box(box, width, height, pad=2)
|
||||
crop = source_rgb[y1:y2, x1:x2]
|
||||
active = mask[y1:y2, x1:x2] > 0
|
||||
pixels = crop[active]
|
||||
luma = pixels.mean(axis=1)
|
||||
background_luma = float(crop[[0, -1], :, :].reshape(-1, 3).mean(axis=1).mean())
|
||||
if background_luma >= 128:
|
||||
selected = pixels[luma <= np.percentile(luma, 20)]
|
||||
else:
|
||||
selected = pixels[luma >= np.percentile(luma, 80)]
|
||||
return tuple(int(value) for value in np.median(selected, axis=0))
|
||||
|
||||
|
||||
def _render_line(image: Image.Image, line: TextLine, color: tuple[int, int, int]) -> None:
|
||||
font_path = CJK_FONT if line.script == "cjk" else (BOLD_FONT if line.box[3] - line.box[1] >= 55 else REGULAR_FONT)
|
||||
target_width, target_height = line.box[2] - line.box[0], line.box[3] - line.box[1]
|
||||
draw = ImageDraw.Draw(image)
|
||||
low, high = 4, max(8, target_height * 2)
|
||||
font = ImageFont.truetype(str(font_path), low)
|
||||
while low <= high:
|
||||
size = (low + high) // 2
|
||||
candidate = ImageFont.truetype(str(font_path), size)
|
||||
bounds = draw.textbbox((0, 0), line.text, font=candidate)
|
||||
if bounds[2] - bounds[0] <= target_width * 1.03 and bounds[3] - bounds[1] <= target_height * 1.08:
|
||||
font, low = candidate, size + 1
|
||||
else:
|
||||
high = size - 1
|
||||
bounds = draw.textbbox((0, 0), line.text, font=font)
|
||||
y = line.box[1] + math.floor((target_height - (bounds[3] - bounds[1])) / 2) - bounds[1]
|
||||
draw.text((line.box[0], y), line.text, fill=color, font=font)
|
||||
|
||||
|
||||
def _write_manifest(path: Path | None, payload: dict[str, Any]) -> None:
|
||||
if path is None:
|
||||
return
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
|
||||
def _vertical_overlap_ratio(left: tuple[int, int, int, int], right: tuple[int, int, int, int]) -> float:
|
||||
overlap = max(0, min(left[3], right[3]) - max(left[1], right[1]))
|
||||
return overlap / max(1, min(left[3] - left[1], right[3] - right[1]))
|
||||
|
||||
|
||||
def group_word_boxes(boxes: list[tuple[int, int, int, int]]) -> list[tuple[int, int, int, int]]:
|
||||
groups: list[tuple[int, int, int, int]] = []
|
||||
for box in sorted(boxes, key=lambda item: ((item[1] + item[3]) / 2, item[0])):
|
||||
matches = []
|
||||
for index, group in enumerate(groups):
|
||||
if _vertical_overlap_ratio(box, group) < 0.45:
|
||||
continue
|
||||
horizontal_gap = max(0, max(box[0], group[0]) - min(box[2], group[2]))
|
||||
line_height = min(box[3] - box[1], group[3] - group[1])
|
||||
if horizontal_gap <= max(24, line_height * 3):
|
||||
matches.append(index)
|
||||
if not matches:
|
||||
groups.append(box)
|
||||
continue
|
||||
index = max(matches, key=lambda item: _vertical_overlap_ratio(box, groups[item]))
|
||||
x1, y1, x2, y2 = groups[index]
|
||||
groups[index] = min(x1, box[0]), min(y1, box[1]), max(x2, box[2]), max(y2, box[3])
|
||||
return sorted(groups, key=lambda item: ((item[1] + item[3]) / 2, item[0]))
|
||||
|
||||
|
||||
def detect_line_boxes(
|
||||
engine: Any,
|
||||
source_rgb: np.ndarray,
|
||||
expected_count: int | None = None,
|
||||
) -> list[tuple[int, int, int, int]]:
|
||||
boxes: list[tuple[int, int, int, int]] = []
|
||||
for page in engine.predict(source_rgb):
|
||||
detected = page.get("rec_boxes", None)
|
||||
if detected is None or len(detected) == 0:
|
||||
detected = page.get("rec_polys", [])
|
||||
for score, raw_box in zip(page.get("rec_scores", []), detected, strict=False):
|
||||
if float(score) < 0.5:
|
||||
continue
|
||||
points = np.asarray(raw_box, dtype=np.float32).reshape(-1)
|
||||
if points.size == 4:
|
||||
x1, y1, x2, y2 = points
|
||||
else:
|
||||
points = points.reshape(-1, 2)
|
||||
x1, y1 = points.min(axis=0)
|
||||
x2, y2 = points.max(axis=0)
|
||||
boxes.append((round(float(x1)), round(float(y1)), round(float(x2)), round(float(y2))))
|
||||
lines = group_word_boxes(boxes)
|
||||
if expected_count is not None and len(lines) != expected_count:
|
||||
raise click.ClickException(f"detected {len(lines)} source lines; expected exactly {expected_count}")
|
||||
return lines
|
||||
|
||||
|
||||
def _load_lines(path: Path, key: str) -> list[TextLine]:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
return [
|
||||
TextLine(tuple(item["box"]), item["text"], item["script"], float(item.get("angle", 0.0)))
|
||||
for item in payload[key]
|
||||
]
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.argument("candidate", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option("--output", required=True, type=click.Path(dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"--lines-json",
|
||||
default=ROOT / "data/evaluations/fidelity/text-lines.json",
|
||||
type=click.Path(exists=True, dir_okay=False, path_type=Path),
|
||||
)
|
||||
@click.option("--source-key", help="Key in lines JSON; defaults to source basename.")
|
||||
@click.option(
|
||||
"--detect-boxes",
|
||||
is_flag=True,
|
||||
help="Detect source boxes; fail unless their count matches verified lines.",
|
||||
)
|
||||
@click.option(
|
||||
"--restoration",
|
||||
type=click.Choice(("vae-glyphs", "source-glyphs", "source-silhouette", "rerender")),
|
||||
required=True,
|
||||
help="Choose VAE glyph cores, regenerated pixels, fresh source shapes, or the system-font control.",
|
||||
)
|
||||
@click.option(
|
||||
"--glyph-donor",
|
||||
type=click.Path(exists=True, dir_okay=False, path_type=Path),
|
||||
help="VAE reconstruction used only by --restoration vae-glyphs.",
|
||||
)
|
||||
@click.option(
|
||||
"--glyph-feather",
|
||||
type=click.FloatRange(min=0.0),
|
||||
default=0.5,
|
||||
show_default=True,
|
||||
help="Outer donor-edge feather used only by --restoration vae-glyphs.",
|
||||
)
|
||||
@click.option(
|
||||
"--selection",
|
||||
type=click.Choice(("all", "changed")),
|
||||
default="all",
|
||||
show_default=True,
|
||||
help="Restore every verified line or only OCR-confirmed changes.",
|
||||
)
|
||||
@click.option(
|
||||
"--erase-background/--keep-background",
|
||||
default=True,
|
||||
show_default=True,
|
||||
help="Erase candidate glyphs before compositing, or directly blend an aligned regenerated glyph layer.",
|
||||
)
|
||||
@click.option(
|
||||
"--composite-mask",
|
||||
type=click.Choice(("glyphs", "boxes")),
|
||||
default="glyphs",
|
||||
show_default=True,
|
||||
help="Composite isolated glyphs or complete aligned text-line boxes.",
|
||||
)
|
||||
@click.option("--manifest", type=click.Path(dir_okay=False, path_type=Path))
|
||||
def main(
|
||||
source: Path,
|
||||
candidate: Path,
|
||||
output: Path,
|
||||
lines_json: Path,
|
||||
source_key: str | None,
|
||||
detect_boxes: bool,
|
||||
restoration: str,
|
||||
glyph_donor: Path | None,
|
||||
glyph_feather: float,
|
||||
selection: str,
|
||||
erase_background: bool,
|
||||
composite_mask: str,
|
||||
manifest: Path | None,
|
||||
) -> None:
|
||||
"""Restore SOURCE text over the scrubbed CANDIDATE."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
|
||||
if restoration == "rerender":
|
||||
for font in (REGULAR_FONT, BOLD_FONT, CJK_FONT):
|
||||
if not font.exists():
|
||||
raise click.ClickException(f"required evaluation font is unavailable: {font}")
|
||||
if restoration == "vae-glyphs" and glyph_donor is None:
|
||||
raise click.ClickException("--glyph-donor is required for --restoration vae-glyphs")
|
||||
if restoration != "vae-glyphs" and glyph_donor is not None:
|
||||
raise click.ClickException("--glyph-donor is only valid with --restoration vae-glyphs")
|
||||
os.environ["PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK"] = "True"
|
||||
source_rgb = np.asarray(Image.open(source).convert("RGB"))
|
||||
candidate_rgb = np.asarray(Image.open(candidate).convert("RGB"))
|
||||
if candidate_rgb.shape != source_rgb.shape:
|
||||
raise click.ClickException("source and candidate dimensions must match")
|
||||
donor_rgb = np.asarray(Image.open(glyph_donor).convert("RGB")) if glyph_donor else None
|
||||
if donor_rgb is not None and donor_rgb.shape != source_rgb.shape:
|
||||
raise click.ClickException("source and glyph donor dimensions must match")
|
||||
lines = _load_lines(lines_json, source_key or source.name)
|
||||
annotation_boxes = [line.box for line in lines]
|
||||
if detect_boxes:
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
page_engine = PaddleOCR(
|
||||
lang="ch",
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
)
|
||||
boxes = detect_line_boxes(page_engine, source_rgb, len(lines))
|
||||
lines = [TextLine(box, line.text, line.script) for line, box in zip(lines, boxes, strict=True)]
|
||||
vertical_pad_ratio = 0.1 if detect_boxes else None
|
||||
decisions = []
|
||||
selected = list(lines)
|
||||
if selection == "all":
|
||||
decisions = [{"line": asdict(line), "selected": True, "reason": "all-lines"} for line in lines]
|
||||
else:
|
||||
from paddleocr import TextRecognition
|
||||
|
||||
engines = {
|
||||
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec"),
|
||||
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec"),
|
||||
"ch": TextRecognition(model_name="PP-OCRv5_server_rec"),
|
||||
}
|
||||
selected = []
|
||||
for line in lines:
|
||||
language = _line_language(line)
|
||||
source_text, source_score = _recognize(engines[language], source_rgb, line, vertical_pad_ratio)
|
||||
candidate_text, candidate_score = _recognize(engines[language], candidate_rgb, line, vertical_pad_ratio)
|
||||
preserve = should_preserve_line(line.text, source_text, source_score, candidate_text, candidate_score)
|
||||
decisions.append(
|
||||
{
|
||||
"line": asdict(line),
|
||||
"source_text": source_text,
|
||||
"source_score": source_score,
|
||||
"candidate_text": candidate_text,
|
||||
"candidate_score": candidate_score,
|
||||
"preserve": preserve,
|
||||
"selected": not preserve,
|
||||
}
|
||||
)
|
||||
if not preserve:
|
||||
selected.append(line)
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
mask_path = output.with_name(output.stem + "_mask.png")
|
||||
manifest_common = {
|
||||
"source": source.name,
|
||||
"candidate": candidate.name,
|
||||
"output": output.name,
|
||||
"mask": mask_path.name,
|
||||
"glyph_donor": glyph_donor.name if glyph_donor else None,
|
||||
"glyph_feather": glyph_feather if restoration == "vae-glyphs" else None,
|
||||
"restoration": restoration,
|
||||
"selection": selection,
|
||||
"erase_background": erase_background,
|
||||
"composite_mask": composite_mask,
|
||||
"box_source": "detector" if detect_boxes else "verified_annotations",
|
||||
"annotation_boxes": annotation_boxes,
|
||||
"decisions": decisions,
|
||||
}
|
||||
if not selected:
|
||||
combined = np.zeros(source_rgb.shape[:2], dtype=np.uint8)
|
||||
shutil.copyfile(candidate, output)
|
||||
Image.fromarray(combined).save(mask_path)
|
||||
payload = {
|
||||
**manifest_common,
|
||||
"mask_fraction": 0.0,
|
||||
"source_glyph_fraction": 0.0,
|
||||
"source_layer_fraction": 0.0,
|
||||
}
|
||||
_write_manifest(manifest, payload)
|
||||
log.info("Copied %s unchanged because every line passed", output)
|
||||
return
|
||||
if restoration in {"source-silhouette", "vae-glyphs"}:
|
||||
source_masks = [source_silhouette_mask(source_rgb, line.box, line.angle) for line in selected]
|
||||
candidate_masks = [source_silhouette_mask(candidate_rgb, line.box, line.angle) for line in selected]
|
||||
line_masks = []
|
||||
for line, source_mask, candidate_mask in zip(selected, source_masks, candidate_masks, strict=True):
|
||||
radius = 5 if line.box[3] - line.box[1] >= 48 else 3
|
||||
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * radius + 1,) * 2)
|
||||
line_masks.append(cv2.dilate(np.maximum(source_mask, candidate_mask), kernel))
|
||||
masks = line_masks
|
||||
else:
|
||||
source_masks = [foreground_mask(source_rgb, line.box) for line in selected]
|
||||
candidate_masks = [foreground_mask(candidate_rgb, line.box) for line in selected]
|
||||
masks = [np.maximum(left, right) for left, right in zip(source_masks, candidate_masks, strict=True)]
|
||||
del candidate_masks
|
||||
groups = group_text_lines(selected)
|
||||
if erase_background:
|
||||
background = cv2.cvtColor(candidate_rgb, cv2.COLOR_RGB2BGR)
|
||||
for group in groups:
|
||||
background = region_eraser.erase_lama(
|
||||
background,
|
||||
np.maximum.reduce([masks[index] for index in group]),
|
||||
)
|
||||
background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
|
||||
residual_masks = [
|
||||
residual_glyph_mask(background_rgb, mask, line.box) for line, mask in zip(selected, masks, strict=True)
|
||||
]
|
||||
for group in groups:
|
||||
residual = np.maximum.reduce([residual_masks[index] for index in group])
|
||||
if np.any(residual):
|
||||
background = region_eraser.erase_lama(background, residual)
|
||||
background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
|
||||
else:
|
||||
background_rgb = candidate_rgb
|
||||
residual_masks = []
|
||||
source_glyph_mask = np.maximum.reduce(source_masks)
|
||||
source_layer_mask = source_glyph_mask
|
||||
if restoration == "source-glyphs":
|
||||
source_layer_mask = (
|
||||
source_box_mask(source_rgb.shape[:2], [line.box for line in selected])
|
||||
if composite_mask == "boxes"
|
||||
else source_glyph_mask
|
||||
)
|
||||
restored = composite_source_glyphs(source_rgb, background_rgb, source_layer_mask, feather=3.0)
|
||||
Image.fromarray(restored).save(output)
|
||||
elif restoration in {"source-silhouette", "vae-glyphs"}:
|
||||
restored = composite_fresh_text_edges(source_rgb, background_rgb, selected, source_masks)
|
||||
if restoration == "vae-glyphs":
|
||||
if donor_rgb is None:
|
||||
raise RuntimeError("VAE glyph restoration requires a loaded donor")
|
||||
restored = composite_reconstructed_glyphs(
|
||||
donor_rgb,
|
||||
restored,
|
||||
source_layer_mask,
|
||||
feather=glyph_feather,
|
||||
)
|
||||
Image.fromarray(restored).save(output)
|
||||
else:
|
||||
rendered = Image.fromarray(background_rgb)
|
||||
for line, source_mask in zip(selected, source_masks, strict=True):
|
||||
_render_line(rendered, line, _sample_text_color(source_rgb, source_mask, line.box))
|
||||
rendered.save(output)
|
||||
combined = np.maximum.reduce([*masks, *residual_masks])
|
||||
if restoration == "source-glyphs" and not erase_background:
|
||||
combined = source_layer_mask
|
||||
Image.fromarray(combined).save(mask_path)
|
||||
payload = {
|
||||
**manifest_common,
|
||||
"mask_fraction": float((combined > 0).mean()),
|
||||
"source_glyph_fraction": float((source_glyph_mask > 0).mean()),
|
||||
"source_layer_fraction": float((source_layer_mask > 0).mean()),
|
||||
}
|
||||
_write_manifest(manifest, payload)
|
||||
log.info("Wrote %s with %.4f edited fraction", output, payload["mask_fraction"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+61
-104
@@ -17,9 +17,10 @@ WHAT IT COVERS AND WHY THAT SHAPE
|
||||
work" report); a no-op must be byte-identical; `metadata --remove` must actually
|
||||
strip; a JPEG strip must not touch the pixels. Note the pixel-lossless contract is
|
||||
the DEFAULT path's -- `--remove-all` deliberately re-encodes (see metadata.py).
|
||||
* The diffusion bodies under `--diffusion`, at `--max-resolution 512` so they fit MPS
|
||||
(~1 min/image on 32 GB unified memory). Not just exit codes: `invisible` must
|
||||
restore the input resolution and must NOT re-stamp SDXL's own open watermark.
|
||||
* The diffusion bodies under `--diffusion`, at a small `--max-resolution`. Both
|
||||
profiles are CUDA-only, so those rows need an NVIDIA GPU and are reported as
|
||||
skips elsewhere. Not just exit codes: `invisible` must restore the input
|
||||
resolution and must NOT re-stamp SDXL's own open watermark.
|
||||
|
||||
WHAT IT DOES NOT COVER, DELIBERATELY AND LOUDLY
|
||||
* Without `--diffusion`, the model-running bodies are reported as SKIPPED with a
|
||||
@@ -34,7 +35,7 @@ WHAT IT DOES NOT COVER, DELIBERATELY AND LOUDLY
|
||||
|
||||
uv run python scripts/smoke_matrix.py # corpus + fixtures
|
||||
uv run python scripts/smoke_matrix.py --quick # fixtures only, no corpus
|
||||
uv run python scripts/smoke_matrix.py --diffusion # + the SDXL model paths
|
||||
uv run python scripts/smoke_matrix.py --diffusion # + the model paths (needs CUDA)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -113,7 +114,7 @@ def main() -> None:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--quick", action="store_true", help="fixtures only; skip the corpus rows")
|
||||
ap.add_argument(
|
||||
"--diffusion", action="store_true", help="also run the model-running paths (SDXL weights, ~1 min/image)"
|
||||
"--diffusion", action="store_true", help="also run the model-running paths (needs CUDA, ~1 min/image)"
|
||||
)
|
||||
a = ap.parse_args()
|
||||
|
||||
@@ -303,7 +304,7 @@ def main() -> None:
|
||||
_diffusion_rows(r, tmp, doubao)
|
||||
else:
|
||||
for name in ("invisible", "all", "batch --mode invisible"):
|
||||
r.skip(f"{name} (model-running body)", "pass --diffusion to exercise it (needs the SDXL weights)")
|
||||
r.skip(f"{name} (model-running body)", "pass --diffusion to exercise it (needs CUDA and the model weights)")
|
||||
|
||||
# ---- real-data formats and shapes ---------------------------------------
|
||||
if not a.quick:
|
||||
@@ -354,18 +355,18 @@ def main() -> None:
|
||||
def _knob_rows(r: Runner, tmp: Path, img: Path) -> None:
|
||||
"""Every diffusion knob the matrix never touched.
|
||||
|
||||
Deliberately cheap (`--steps 4`, `--max-resolution 384`): these rows answer "is the
|
||||
knob accepted and does the run complete", NOT "is the output good". Quality per knob
|
||||
needs a per-knob oracle and most of them have none (`--humanize` has no oracle at
|
||||
all), so claiming more here would be dishonest.
|
||||
Deliberately cheap (`--max-resolution 384`): these rows answer "is the knob accepted
|
||||
and does the run complete", NOT "is the output good". Quality per knob needs a
|
||||
per-knob oracle and most of them have none (`--humanize` has no oracle at all), so
|
||||
claiming more here would be dishonest.
|
||||
"""
|
||||
from remove_ai_watermarks import upscaler
|
||||
|
||||
# --steps 20 is the floor that WORKS, not an arbitrary choice: effective timesteps
|
||||
# are int(steps * strength), so at the default strength 0.15 anything below
|
||||
# --steps 7 rounds to ZERO and the pipeline dies inside torch. The first version of
|
||||
# these rows used --steps 4 and every single one failed with the same reshape error.
|
||||
fast = ["--max-resolution", "384", "--min-resolution", "0", "--steps", "20", "--force", "--seed", "0"]
|
||||
# Both surviving profiles are CUDA-only, pin a fixed model stack and let each stage
|
||||
# own its schedule and CFG. The knobs that used to contradict that (--model, --steps,
|
||||
# --guidance-scale, --device, --auto) are gone from the parser, so the rows below
|
||||
# assert Click itself refuses them. That is the coverage worth having: an option the
|
||||
# CLI accepts and the library refuses several layers down is what this matrix exists
|
||||
# to catch, and the cheapest way to keep it caught is for the option not to exist.
|
||||
fast = ["--max-resolution", "384", "--force", "--seed", "0"]
|
||||
|
||||
def run(name: str, extra: list[str], *, tag: str, expect: int | None = 0) -> None:
|
||||
r.run(
|
||||
@@ -375,65 +376,53 @@ def _knob_rows(r: Runner, tmp: Path, img: Path) -> None:
|
||||
timeout=2400,
|
||||
)
|
||||
|
||||
run("--pipeline sdxl", ["--pipeline", "sdxl"], tag="sdxl")
|
||||
run("--pipeline controlnet", ["--pipeline", "controlnet"], tag="cnet")
|
||||
run("--strength", ["--strength", "0.2"], tag="strength")
|
||||
run("--guidance-scale", ["--guidance-scale", "5.0"], tag="gs")
|
||||
run("--controlnet-scale", ["--controlnet-scale", "0.5"], tag="cns")
|
||||
run("--humanize", ["--humanize", "0.3"], tag="hum")
|
||||
run("--unsharp", ["--unsharp", "0.5"], tag="uns")
|
||||
run("--no-adaptive-polish", ["--no-adaptive-polish"], tag="nap")
|
||||
run("--tile", ["--tile", "--tile-size", "256", "--tile-overlap", "64"], tag="tile")
|
||||
run("--device mps", ["--device", "mps"], tag="mps")
|
||||
run("--upscaler lanczos", ["--upscaler", "lanczos"], tag="lanczos")
|
||||
run("--auto (deprecated no-op)", ["--auto"], tag="auto")
|
||||
# Click rejects a retired profile at parse time (exit 2) rather than remapping it.
|
||||
for retired in ("sdxl", "controlnet", "qwen", "default"):
|
||||
run(f"--pipeline {retired} is rejected", ["--pipeline", retired], tag=f"retired_{retired}", expect=2)
|
||||
|
||||
if upscaler.is_available():
|
||||
run("--upscaler esrgan", ["--upscaler", "esrgan"], tag="esrgan")
|
||||
else:
|
||||
r.skip("--upscaler esrgan", "the `esrgan` extra is not installed")
|
||||
# Retired options: no longer parsed at all (exit 2, "No such option").
|
||||
for args, tag in (
|
||||
(["--steps", "20"], "steps20"),
|
||||
(["--guidance-scale", "5.0"], "gs5"),
|
||||
(["--model", "org/custom"], "model"),
|
||||
(["--device", "cpu"], "devcpu"),
|
||||
(["--auto"], "auto"),
|
||||
):
|
||||
run(f"{args[0]} is no longer an option", args, tag=tag, expect=2)
|
||||
|
||||
# CPU is correctness-relevant (it is the documented MPS-OOM fallback) but slow, so
|
||||
# it gets the smallest possible run rather than being skipped.
|
||||
r.run(
|
||||
"--device cpu",
|
||||
[
|
||||
"invisible",
|
||||
str(img),
|
||||
"-o",
|
||||
str(tmp / "k_cpu.png"),
|
||||
"--device",
|
||||
"cpu",
|
||||
"--max-resolution",
|
||||
"256",
|
||||
"--min-resolution",
|
||||
"0",
|
||||
"--steps",
|
||||
"20",
|
||||
"--force",
|
||||
"--seed",
|
||||
"0",
|
||||
],
|
||||
timeout=3600,
|
||||
)
|
||||
|
||||
# qwen is CUDA-class by design (bf16 MMDiT, no MPS fallback). On this host the
|
||||
# honest outcome is a CLEAN failure, not a crash -- assert it does not hang or
|
||||
# dump a traceback at the user.
|
||||
try:
|
||||
import torch
|
||||
|
||||
cuda = bool(torch.cuda.is_available())
|
||||
except Exception:
|
||||
cuda = False
|
||||
if cuda:
|
||||
run("--pipeline qwen", ["--pipeline", "qwen"], tag="qwen")
|
||||
else:
|
||||
r.skip("--pipeline qwen", "CUDA-class pipeline; no CUDA device on this host")
|
||||
|
||||
# --model and --hf-token are deliberately not exercised: one would download a second
|
||||
# multi-GB checkpoint, the other needs a real credential. Skipped loudly, not passed.
|
||||
r.skip("--model", "would download a second multi-GB checkpoint")
|
||||
if not cuda:
|
||||
# Without CUDA the honest outcome is a CLEAN refusal naming the reason, not a
|
||||
# traceback from inside a half-built pipeline.
|
||||
run("no CUDA fails cleanly", [], tag="nocuda", expect=1)
|
||||
for label in (
|
||||
"--strength",
|
||||
"--controlnet-scale",
|
||||
"--humanize",
|
||||
"--unsharp",
|
||||
"--no-adaptive-polish",
|
||||
"--tile",
|
||||
"--pipeline sdxl-zimage",
|
||||
):
|
||||
r.skip(label, "accepted-knob rows need a CUDA device")
|
||||
return
|
||||
|
||||
run("--pipeline sdxl-zimage", ["--pipeline", "sdxl-zimage"], tag="sdxlz")
|
||||
run("--strength", ["--strength", "0.2"], tag="strength")
|
||||
run("--controlnet-scale", ["--controlnet-scale", "0.5"], tag="cns")
|
||||
run("--humanize", ["--humanize", "0.3"], tag="hum")
|
||||
run("--unsharp", ["--unsharp", "0.5"], tag="uns")
|
||||
run("--no-adaptive-polish", ["--no-adaptive-polish"], tag="nap")
|
||||
run("--tile", ["--tile", "--tile-size", "256", "--tile-overlap", "64"], tag="tile")
|
||||
|
||||
# --hf-token needs a real credential, so it cannot be exercised meaningfully here.
|
||||
# Skipped loudly rather than silently passed over.
|
||||
r.skip("--hf-token", "needs a real credential; cannot be exercised meaningfully here")
|
||||
|
||||
# CONTRACT, not just execution: the same seed must reproduce the same pixels.
|
||||
@@ -497,7 +486,7 @@ def _sdxl_watermark_bits(img: object) -> float:
|
||||
|
||||
|
||||
def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
"""Exercise the model-running bodies at a reduced resolution (MPS-friendly).
|
||||
"""Exercise the model-running bodies at a reduced resolution (CUDA required).
|
||||
|
||||
Bounded with `--max-resolution 512` and a fixed seed: the point is that the paths
|
||||
RUN and keep their contracts, not to certify removal strength (that needs the
|
||||
@@ -519,7 +508,7 @@ def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
# OUTPUT and the row degrades to a skip rather than a false pass if it goes fragile.
|
||||
mj = SAMPLES / "mj-1.png"
|
||||
inv = tmp / "inv_mj.png"
|
||||
res = r.run("invisible runs (mps, 512px)", ["invisible", str(mj), "-o", str(inv), "--force", *small], timeout=1800)
|
||||
res = r.run("invisible runs (512px)", ["invisible", str(mj), "-o", str(inv), "--force", *small], timeout=1800)
|
||||
if res.status == "pass" and inv.exists():
|
||||
src_img, out_img = imread(str(mj)), imread(str(inv))
|
||||
r.check(
|
||||
@@ -547,7 +536,7 @@ def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
|
||||
# `all`: every stage must land -- the visible mark AND the metadata both gone.
|
||||
allout = tmp / "all_out.png"
|
||||
res = r.run("all runs (mps, 512px)", ["all", str(doubao), "-o", str(allout), *small], timeout=1800)
|
||||
res = r.run("all runs (512px)", ["all", str(doubao), "-o", str(allout), *small], timeout=1800)
|
||||
if res.status == "pass" and allout.exists():
|
||||
rep = json.loads(_capture(["identify", str(allout), "--json"]))
|
||||
r.check(
|
||||
@@ -566,7 +555,7 @@ def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
shutil.copy(f, bd / f.name)
|
||||
bout = tmp / "batch_inv_out"
|
||||
res = r.run(
|
||||
"batch --mode invisible runs (mps, 512px)",
|
||||
"batch --mode invisible runs (512px)",
|
||||
["batch", str(bd), "--mode", "invisible", "-o", str(bout), *small],
|
||||
expect_exit=None,
|
||||
timeout=3600,
|
||||
@@ -578,44 +567,12 @@ def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
# `batch --mode all` -- the only --mode value the matrix never ran.
|
||||
aout = tmp / "batch_all_out"
|
||||
r.run(
|
||||
"batch --mode all runs (mps, 512px)",
|
||||
"batch --mode all runs (512px)",
|
||||
["batch", str(bd), "--mode", "all", "-o", str(aout), *small],
|
||||
expect_exit=None,
|
||||
timeout=3600,
|
||||
)
|
||||
|
||||
# The AI-enhanced composite path: regenerate ONLY a region and feather it back,
|
||||
# leaving everything outside the box pixel-exact. Library-level -- the CLI has no
|
||||
# flag for it, so it would otherwise never be exercised on real data.
|
||||
try:
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
from remove_ai_watermarks.noai.watermark_remover import WatermarkRemover
|
||||
|
||||
src = imread(str(mj))
|
||||
h, w = src.shape[:2]
|
||||
box = (w // 4, h // 4, w // 4, h // 4)
|
||||
rem = WatermarkRemover(pipeline="controlnet")
|
||||
rout = tmp / "region_composite.png"
|
||||
rem.remove_watermark(mj, rout, strength=0.15, num_inference_steps=20, seed=0, region=box)
|
||||
got = imread(str(rout))
|
||||
if got is None or got.shape != src.shape:
|
||||
r.check("region composite keeps the frame outside the box", False, "shape changed or unreadable")
|
||||
else:
|
||||
mask = np.ones(src.shape[:2], dtype=bool)
|
||||
x, y, bw, bh = box
|
||||
# Outside the box PLUS the feather margin must be untouched.
|
||||
pad = 96
|
||||
mask[max(0, y - pad) : y + bh + pad, max(0, x - pad) : x + bw + pad] = False
|
||||
r.check(
|
||||
"region composite keeps the frame outside the box",
|
||||
bool(np.array_equal(src[mask], got[mask])),
|
||||
"pixels changed outside the regenerated region",
|
||||
)
|
||||
except Exception as e:
|
||||
r.skip("region composite (remove_watermark(region=...))", f"{type(e).__name__}: {e}"[:120])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,53 +0,0 @@
|
||||
"""Update the conda recipe to a published package version and sdist hash."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
_VERSION_LINE = re.compile(r'^ version: "[^"]+"$', re.MULTILINE)
|
||||
_SHA256_LINE = re.compile(r"^ sha256: [0-9a-f]{64}$", re.MULTILINE)
|
||||
|
||||
|
||||
def update_recipe(text: str, *, version: str, sha256: str) -> str:
|
||||
"""Return recipe text with exactly one context version and source hash updated."""
|
||||
if not version or any(char.isspace() for char in version):
|
||||
raise ValueError("version must be a non-empty value without whitespace")
|
||||
if re.fullmatch(r"[0-9a-f]{64}", sha256) is None:
|
||||
raise ValueError("sha256 must be 64 lowercase hexadecimal characters")
|
||||
|
||||
updated, version_count = _VERSION_LINE.subn(f' version: "{version}"', text)
|
||||
updated, sha_count = _SHA256_LINE.subn(f" sha256: {sha256}", updated)
|
||||
if version_count != 1 or sha_count != 1:
|
||||
raise ValueError(
|
||||
"expected exactly one context version and one source sha256 "
|
||||
f"(found version={version_count}, sha256={sha_count})"
|
||||
)
|
||||
return updated
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Update the selected recipe in place."""
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--version", required=True)
|
||||
parser.add_argument("--sha256", required=True)
|
||||
parser.add_argument(
|
||||
"--recipe",
|
||||
type=Path,
|
||||
default=Path("packaging/conda/recipe.yaml"),
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
original = args.recipe.read_text()
|
||||
updated = update_recipe(original, version=args.version, sha256=args.sha256)
|
||||
args.recipe.write_text(updated)
|
||||
log.info("Updated %s to version %s", args.recipe, args.version)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
|
||||
main()
|
||||
@@ -22,14 +22,14 @@ import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from collections import Counter
|
||||
from datetime import datetime, timezone
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
from _plain_console import Console, Table
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks.noai.c2pa import extract_c2pa_info
|
||||
from remove_ai_watermarks._internal.c2pa import extract_c2pa_info
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
console = Console()
|
||||
@@ -180,7 +180,7 @@ def ingest(
|
||||
"c2pa_issuer": issuer,
|
||||
"synthid_metadata": "yes" if synthid else "",
|
||||
"verified_via": verified_via,
|
||||
"added": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"added": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"notes": notes,
|
||||
}
|
||||
)
|
||||
|
||||
@@ -62,17 +62,10 @@ OUT = REPO / ".local-eval" / "vendor-cohorts.jsonl"
|
||||
FIRED = REPO / ".local-eval" / "visible-positives.jsonl"
|
||||
SHEET_DIR = REPO / ".local-eval" / "vendor-cohort-sheets"
|
||||
|
||||
# A producer code is `001` + `1` + USCC(18) + a 5-digit app/product suffix, so two
|
||||
# codes sharing the USCC are the same legal entity registering different products.
|
||||
# Slicing is defensive: anything not matching the layout is grouped by its raw value.
|
||||
_USCC_START, _USCC_END = 4, 22
|
||||
|
||||
|
||||
def uscc_of(code: str) -> str:
|
||||
"""The 18-char Unified Social Credit Code embedded in a TC260 producer code."""
|
||||
if len(code) >= _USCC_END and code[:3] == "001":
|
||||
return code[_USCC_START:_USCC_END]
|
||||
return code
|
||||
# `uscc_of` moved into the library (`metadata.uscc_of`) when the USCC -> vendor table
|
||||
# started driving `api.visible_provenance`; this script must group by the same rule the
|
||||
# product uses, so it imports rather than reimplements it.
|
||||
from remove_ai_watermarks.metadata import uscc_of # noqa: E402
|
||||
|
||||
|
||||
def _one(path_str: str) -> dict[str, Any] | None:
|
||||
|
||||
@@ -97,7 +97,7 @@ def _score(args: ScoreArgs) -> dict[str, Any] | None:
|
||||
eng = TextMarkEngine(build_config(asset, name, basis, overrides))
|
||||
loc = eng.locate(img)
|
||||
try:
|
||||
score, box = eng._tophat_best(img, loc)
|
||||
score, box = eng._ladder_best(img, loc)
|
||||
except Exception:
|
||||
return None
|
||||
return {"path": path_str, "score": round(float(score), 4), "box": box}
|
||||
@@ -225,7 +225,7 @@ def sheets(pos: list[dict[str, Any]], name: str, per_sheet: int = 24) -> None:
|
||||
# genuinely unknown, which is the one case a dense ladder earns its cost -- but it also
|
||||
# hands clean corners extra chances to match, so it must never set a gate.
|
||||
_FIT_SCALES = tuple(round(0.4 * (1.03**i), 4) for i in range(80)) # 0.40 .. ~4.1
|
||||
# The ladder the product actually ships (`_text_mark_engine._tophat_best`).
|
||||
# The ladder the product actually ships (`_text_mark_engine._ladder_best`).
|
||||
_SHIPPED_LADDER = (0.8, 1.0, 1.25)
|
||||
|
||||
|
||||
@@ -460,7 +460,7 @@ def _cross_score(args: tuple[str, Any, Any]) -> dict[str, Any] | None:
|
||||
eng = TextMarkEngine(cfg)
|
||||
loc = eng.locate(img)
|
||||
try:
|
||||
score, _ = eng._tophat_best(img, loc)
|
||||
score, _ = eng._ladder_best(img, loc)
|
||||
except Exception:
|
||||
return None
|
||||
out[key] = round(float(score), 4)
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
"""End-to-end fidelity of a DELIVERED video against its source.
|
||||
|
||||
The engine's own ``psnr_db`` is measured against the already-resized frame and
|
||||
before the encoder, so it reports the VAE round trip plus latent noise and
|
||||
nothing else. The downscale, the frame decimation and the H.264 encode -- the
|
||||
three steps that actually cost the user picture -- are invisible to it, and no
|
||||
metric computed inside the streaming loop can see them, because the candidate
|
||||
frame is captured before it is written to the encoder pipe.
|
||||
|
||||
This script measures what the engine cannot: it decodes the delivered file after
|
||||
muxing, upscales each frame back to the source geometry, and scores it against
|
||||
the untouched source frame it came from. That reference is deliberately harsh -
|
||||
detail the downscale destroyed is unrecoverable, so the absolute number is
|
||||
dominated by content the pipeline never had a chance to keep. Use it to RANK
|
||||
configurations against a shared reference, not to attribute loss to one stage.
|
||||
|
||||
It streams, for the same reason the engine does: holding a 1080p clip plus its
|
||||
upscaled candidate and flow maps in memory runs to gigabytes and grows with clip
|
||||
length. Peak here is a handful of frames regardless of duration.
|
||||
|
||||
It is a research tool, not part of the shipped path, and it measures fidelity
|
||||
only. Nothing here is a watermark verdict: only a provider-oracle row is.
|
||||
|
||||
uv run python scripts/video_fidelity_probe.py source.mp4 out-512.mp4 out-768.mp4
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import click
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.video_invisible import _iter_sampled_frames, _probe_video
|
||||
from remove_ai_watermarks.video_temporal import _backward_map, _motion_residual
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
from invisible_quality_audit import _ssim # reuse, do not reimplement a fourth SSIM
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _delivered_geometry(path: Path) -> tuple[int, int, float]:
|
||||
capture = cv2.VideoCapture(str(path))
|
||||
if not capture.isOpened():
|
||||
raise click.ClickException(f"Could not open video: {path}")
|
||||
try:
|
||||
width = round(capture.get(cv2.CAP_PROP_FRAME_WIDTH))
|
||||
height = round(capture.get(cv2.CAP_PROP_FRAME_HEIGHT))
|
||||
fps = float(capture.get(cv2.CAP_PROP_FPS))
|
||||
finally:
|
||||
capture.release()
|
||||
if width <= 0 or height <= 0 or fps <= 0.0:
|
||||
raise click.ClickException(f"Video has no usable geometry or frame rate: {path}")
|
||||
return width, height, fps
|
||||
|
||||
|
||||
def _iter_frames(path: Path) -> Iterator[NDArray[Any]]:
|
||||
capture = cv2.VideoCapture(str(path))
|
||||
if not capture.isOpened():
|
||||
raise click.ClickException(f"Could not open video: {path}")
|
||||
try:
|
||||
while True:
|
||||
ok, frame = capture.read()
|
||||
if not ok:
|
||||
return
|
||||
yield frame
|
||||
finally:
|
||||
capture.release()
|
||||
|
||||
|
||||
def _measure(
|
||||
source: Path,
|
||||
delivered: Path,
|
||||
*,
|
||||
source_geometry: tuple[int, int, float],
|
||||
duration: float | None,
|
||||
) -> dict[str, Any]:
|
||||
source_width, source_height, source_fps = source_geometry
|
||||
width, height, fps = _delivered_geometry(delivered)
|
||||
|
||||
# Drive the source through the engine's own sampler at the source's geometry,
|
||||
# so the frame the probe compares against is the frame the engine regenerated.
|
||||
# Importing the rule is what keeps the pairing correct: a count check cannot
|
||||
# catch a selection rule that reorders frames without changing how many.
|
||||
reference_frames = _iter_sampled_frames(
|
||||
source,
|
||||
source_fps=source_fps,
|
||||
duration=duration,
|
||||
effective_fps=fps,
|
||||
size=(source_width, source_height),
|
||||
)
|
||||
|
||||
squared_error = 0.0
|
||||
pixel_count = 0
|
||||
ssim_scores: list[float] = []
|
||||
temporal_baseline = 0.0
|
||||
temporal_candidate = 0.0
|
||||
previous_gray: NDArray[Any] | None = None
|
||||
previous_reference_f32: NDArray[Any] | None = None
|
||||
previous_candidate_f32: NDArray[Any] | None = None
|
||||
frame_count = 0
|
||||
needs_upscale = (width, height) != (source_width, source_height)
|
||||
|
||||
reference_iter = iter(reference_frames)
|
||||
delivered_iter = _iter_frames(delivered)
|
||||
while True:
|
||||
reference = next(reference_iter, None)
|
||||
delivered_frame = next(delivered_iter, None)
|
||||
if reference is None and delivered_frame is None:
|
||||
break
|
||||
if reference is None or delivered_frame is None:
|
||||
raise click.ClickException(
|
||||
f"{delivered.name} and the sampled source ran out at different points after "
|
||||
f"{frame_count} frames. Pass --duration to match the prefix this output was "
|
||||
f"produced from, or check that it came from {source.name}."
|
||||
)
|
||||
candidate = (
|
||||
cv2.resize(delivered_frame, (source_width, source_height), interpolation=cv2.INTER_LANCZOS4)
|
||||
if needs_upscale
|
||||
else delivered_frame
|
||||
)
|
||||
reference_f32 = reference.astype(np.float32)
|
||||
candidate_f32 = candidate.astype(np.float32)
|
||||
difference = reference_f32 - candidate_f32
|
||||
squared_error += float(np.sum(difference * difference, dtype=np.float64))
|
||||
pixel_count += reference.size
|
||||
ssim_scores.append(
|
||||
_ssim(cv2.cvtColor(reference, cv2.COLOR_BGR2GRAY), cv2.cvtColor(candidate, cv2.COLOR_BGR2GRAY))
|
||||
)
|
||||
|
||||
current_gray = cv2.cvtColor(reference, cv2.COLOR_BGR2GRAY)
|
||||
if previous_gray is not None and previous_reference_f32 is not None and previous_candidate_f32 is not None:
|
||||
frame_maps = _backward_map(current_gray, previous_gray)
|
||||
temporal_baseline += _motion_residual(reference_f32, previous_reference_f32, frame_maps)
|
||||
temporal_candidate += _motion_residual(candidate_f32, previous_candidate_f32, frame_maps)
|
||||
previous_gray = current_gray
|
||||
previous_reference_f32 = reference_f32
|
||||
previous_candidate_f32 = candidate_f32
|
||||
frame_count += 1
|
||||
|
||||
if frame_count < 2:
|
||||
raise click.ClickException(f"{delivered.name} paired fewer than two frames against {source.name}")
|
||||
|
||||
mse = squared_error / pixel_count
|
||||
size_bytes = delivered.stat().st_size
|
||||
return {
|
||||
"file": delivered.name,
|
||||
"source": source.name,
|
||||
"source_width": source_width,
|
||||
"source_height": source_height,
|
||||
"source_fps": round(source_fps, 4),
|
||||
"width": width,
|
||||
"height": height,
|
||||
"fps": round(fps, 4),
|
||||
"frames": frame_count,
|
||||
"pixel_ratio": round((width * height) / (source_width * source_height), 4),
|
||||
"source_psnr_db": math.inf if mse == 0.0 else round(20.0 * math.log10(255.0 / math.sqrt(mse)), 4),
|
||||
"source_ssim": round(float(np.mean(ssim_scores)), 4),
|
||||
"temporal_residual_ratio": round(temporal_candidate / max(temporal_baseline, 1e-6), 4),
|
||||
"size_bytes": size_bytes,
|
||||
# The mux copies the source audio track verbatim, so this is a container
|
||||
# bitrate. It still ranks candidates at fixed crf; it is not a video bitrate.
|
||||
"file_bitrate_kbps": round(size_bytes * 8.0 / (frame_count / fps) / 1000.0, 1),
|
||||
}
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.argument("delivered", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"--duration",
|
||||
type=click.FloatRange(min=0.1),
|
||||
default=None,
|
||||
help="Score only the first N seconds of the source, matching a trimmed sweep candidate.",
|
||||
)
|
||||
@click.option("--json-out", type=click.Path(dir_okay=False, path_type=Path), help="Also write the rows as JSON.")
|
||||
def main(source: Path, delivered: tuple[Path, ...], duration: float | None, json_out: Path | None) -> None:
|
||||
"""Score each DELIVERED file against SOURCE end to end."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(message)s")
|
||||
source_geometry = _probe_video(source)
|
||||
log.info("Source %s: %dx%d at %.4f fps", source.name, *source_geometry)
|
||||
|
||||
rows = [_measure(source, path, source_geometry=source_geometry, duration=duration) for path in delivered]
|
||||
for row in rows:
|
||||
log.info(
|
||||
"%s: %dx%d at %s fps, %.1f%% of source pixels, PSNR %s dB, SSIM %s, temporal %s, %s kbps",
|
||||
row["file"],
|
||||
row["width"],
|
||||
row["height"],
|
||||
row["fps"],
|
||||
row["pixel_ratio"] * 100.0,
|
||||
row["source_psnr_db"],
|
||||
row["source_ssim"],
|
||||
row["temporal_residual_ratio"],
|
||||
row["file_bitrate_kbps"],
|
||||
)
|
||||
if json_out is not None:
|
||||
json_out.write_text(json.dumps(rows, indent=2), encoding="utf-8")
|
||||
log.info("Wrote %s", json_out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,243 @@
|
||||
"""Build oracle-gated video regeneration candidates for SynthID research.
|
||||
|
||||
This is a research harness, not a shipped removal command. Google does not
|
||||
publish a local video SynthID decoder, so the script cannot label a candidate as
|
||||
clean. It produces:
|
||||
|
||||
* a re-encode control with the same duration, frame rate, dimensions, and codec;
|
||||
* one VAE-regenerated video per requested latent-noise level;
|
||||
* paired fidelity and temporal-residual measurements;
|
||||
* a CSV column for the external Gemini SynthID verdict.
|
||||
|
||||
The control is load-bearing. If it reads clean, the experiment is invalid:
|
||||
resize, frame-rate conversion, or H.264 compression already silenced the oracle,
|
||||
so a VAE candidate cannot be credited with removal.
|
||||
|
||||
The regeneration attack follows the general encode, perturb, reconstruct family
|
||||
from WatermarkAttacker (NeurIPS 2024). A single spatial latent-noise sample is
|
||||
shared by every frame. Independent per-frame noise creates avoidable flicker and
|
||||
does not test the video-specific question.
|
||||
|
||||
Run with the project's GPU extra:
|
||||
|
||||
uv run --extra gpu python scripts/video_synthid_sweep.py input.mp4 -o out/
|
||||
|
||||
Then upload ``control.mp4`` and each candidate in separate Gemini chats, invoke
|
||||
the built-in SynthID verifier (``@synthid``), and use the question printed by
|
||||
the script. Do not follow the verdict with an adversarial prompt asking the chat
|
||||
model to reinterpret the detector: that switches back to ordinary reasoning.
|
||||
Only a control-positive, candidate-negative pair from the built-in verifier is
|
||||
removal evidence.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import hashlib
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import click
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.video_invisible import (
|
||||
_decode_frame_latents,
|
||||
_encode_frame_latents,
|
||||
_fit_size,
|
||||
_probe_video,
|
||||
_shared_latent_noise,
|
||||
build_temporal_reference,
|
||||
encode_video_frames,
|
||||
load_video_vae_runtime,
|
||||
paired_psnr,
|
||||
read_sampled_frames,
|
||||
temporal_residual_ratio,
|
||||
)
|
||||
from remove_ai_watermarks.video_synthid import (
|
||||
DEFAULT_VIDEO_SYNTHID_FPS,
|
||||
DEFAULT_VIDEO_SYNTHID_LONG_SIDE,
|
||||
DEFAULT_VIDEO_SYNTHID_VAE,
|
||||
VIDEO_SYNTHID_LATENT_MULTIPLE,
|
||||
VIDEO_SYNTHID_VERIFICATION_PROMPT,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _parse_noise_levels(values: str) -> tuple[float, ...]:
|
||||
levels = tuple(float(value.strip()) for value in values.split(",") if value.strip())
|
||||
if not levels:
|
||||
raise click.BadParameter("At least one noise level is required")
|
||||
if any(not 0.0 <= value <= 1.0 for value in levels):
|
||||
raise click.BadParameter("Noise levels must be between 0 and 1")
|
||||
return levels
|
||||
|
||||
|
||||
def _sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as stream:
|
||||
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _write_manifest(output_dir: Path, rows: Sequence[dict[str, str]]) -> Path:
|
||||
path = output_dir / "sweep.csv"
|
||||
# Mirrors the tracked manifest's run-configuration fields (data/README.md) so a
|
||||
# row carries into data/evaluations/video-synthid-oracle.csv without hand
|
||||
# reconstruction. The two 2026-07-31 rows show the cost of not doing this: their
|
||||
# geometry was never recorded and cannot be recovered from the row.
|
||||
fieldnames = [
|
||||
"variant",
|
||||
"source_sha256",
|
||||
"source_width",
|
||||
"source_height",
|
||||
"source_fps",
|
||||
"duration_seconds",
|
||||
"vae",
|
||||
"noise_std",
|
||||
"long_side",
|
||||
"fps",
|
||||
"seed",
|
||||
"psnr_db",
|
||||
"temporal_residual_ratio",
|
||||
"file",
|
||||
"sha256",
|
||||
"synthid_oracle",
|
||||
]
|
||||
with path.open("w", newline="", encoding="utf-8") as stream:
|
||||
writer = csv.DictWriter(stream, fieldnames=fieldnames)
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
return path
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option("-o", "--output-dir", required=True, type=click.Path(file_okay=False, path_type=Path))
|
||||
@click.option("--noise-levels", default="0,0.05,0.1,0.15", show_default=True)
|
||||
@click.option("--duration", type=click.FloatRange(min=0.1), default=2.0, show_default=True)
|
||||
@click.option("--fps", type=click.FloatRange(min=1.0), default=DEFAULT_VIDEO_SYNTHID_FPS, show_default=True)
|
||||
@click.option(
|
||||
"--long-side",
|
||||
type=click.IntRange(min=VIDEO_SYNTHID_LATENT_MULTIPLE),
|
||||
default=DEFAULT_VIDEO_SYNTHID_LONG_SIDE,
|
||||
show_default=True,
|
||||
)
|
||||
@click.option("--batch-size", type=click.IntRange(min=1), default=4, show_default=True)
|
||||
@click.option("--seed", type=int, default=0, show_default=True)
|
||||
@click.option("--model", default=DEFAULT_VIDEO_SYNTHID_VAE, show_default=True)
|
||||
@click.option("--device", type=click.Choice(["auto", "cuda", "mps", "cpu"]), default="auto", show_default=True)
|
||||
def main(
|
||||
source: Path,
|
||||
output_dir: Path,
|
||||
noise_levels: str,
|
||||
duration: float,
|
||||
fps: float,
|
||||
long_side: int,
|
||||
batch_size: int,
|
||||
seed: int,
|
||||
model: str,
|
||||
device: str,
|
||||
) -> None:
|
||||
"""Generate VAE video candidates from the prefix of SOURCE."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(message)s")
|
||||
levels = _parse_noise_levels(noise_levels)
|
||||
width, height, source_fps = _probe_video(source)
|
||||
size = _fit_size(width, height, long_side)
|
||||
frames, effective_fps = read_sampled_frames(source, duration=duration, output_fps=fps, size=size)
|
||||
run = {
|
||||
"source_sha256": _sha256(source),
|
||||
"source_width": str(width),
|
||||
"source_height": str(height),
|
||||
"source_fps": f"{source_fps:.4f}",
|
||||
"duration_seconds": f"{duration:.4f}",
|
||||
"vae": model,
|
||||
"long_side": str(long_side),
|
||||
"fps": f"{effective_fps:.4f}",
|
||||
"seed": str(seed),
|
||||
}
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
control_path = output_dir / "control.mp4"
|
||||
encode_video_frames(frames, source, control_path, fps=effective_fps)
|
||||
rows: list[dict[str, str]] = [
|
||||
{
|
||||
**run,
|
||||
"variant": "control",
|
||||
"noise_std": "",
|
||||
"psnr_db": "inf",
|
||||
"temporal_residual_ratio": "1",
|
||||
"file": control_path.name,
|
||||
"sha256": _sha256(control_path),
|
||||
"synthid_oracle": "",
|
||||
}
|
||||
]
|
||||
|
||||
# Load through the engine's own loader rather than repeating it here: the
|
||||
# scaling-factor gate lives there, and the harness that produces the certified
|
||||
# rows is the last place that should be exempt from it.
|
||||
runtime = load_video_vae_runtime(model=model, device=device)
|
||||
vae, resolved_device = runtime.vae, runtime.resolved_device
|
||||
|
||||
log.info("Encoding source frames")
|
||||
latent_batches = _encode_frame_latents(
|
||||
frames,
|
||||
vae=vae,
|
||||
device=resolved_device,
|
||||
batch_size=batch_size,
|
||||
scaling_factor=runtime.scaling_factor,
|
||||
)
|
||||
first_latents = latent_batches[0]
|
||||
shared_noise = _shared_latent_noise(
|
||||
first_latents.shape[1:],
|
||||
seed=seed,
|
||||
device=resolved_device,
|
||||
dtype=first_latents.dtype,
|
||||
)
|
||||
reference_stack = np.stack(frames)
|
||||
temporal_maps, temporal_baseline = build_temporal_reference(frames)
|
||||
for level in levels:
|
||||
log.info("Decoding latent noise %.4f", level)
|
||||
regenerated = _decode_frame_latents(
|
||||
latent_batches,
|
||||
vae=vae,
|
||||
noise_std=level,
|
||||
shared_noise=shared_noise,
|
||||
scaling_factor=runtime.scaling_factor,
|
||||
)
|
||||
output_path = output_dir / f"vae-noise-{level:.4f}.mp4"
|
||||
encode_video_frames(
|
||||
regenerated,
|
||||
source,
|
||||
output_path,
|
||||
fps=effective_fps,
|
||||
)
|
||||
psnr = paired_psnr(reference_stack, np.stack(regenerated))
|
||||
temporal_ratio = temporal_residual_ratio(regenerated, temporal_maps, temporal_baseline)
|
||||
rows.append(
|
||||
{
|
||||
**run,
|
||||
"variant": "vae",
|
||||
"noise_std": f"{level:.4f}",
|
||||
"psnr_db": f"{psnr:.4f}",
|
||||
"temporal_residual_ratio": f"{temporal_ratio:.4f}",
|
||||
"file": output_path.name,
|
||||
"sha256": _sha256(output_path),
|
||||
"synthid_oracle": "",
|
||||
}
|
||||
)
|
||||
|
||||
manifest = _write_manifest(output_dir, rows)
|
||||
log.info("Wrote %s", manifest)
|
||||
log.info(
|
||||
"Verify control.mp4 first with Gemini's built-in SynthID verifier and this question: %s",
|
||||
VIDEO_SYNTHID_VERIFICATION_PROMPT,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -38,7 +38,7 @@ sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
# The package's own format set. An inlined copy here silently skipped .heif, which
|
||||
# CLAUDE.md documents as supported.
|
||||
from remove_ai_watermarks.noai.constants import SUPPORTED_FORMATS as _EXTS
|
||||
from remove_ai_watermarks._internal.constants import SUPPORTED_FORMATS as _EXTS
|
||||
|
||||
REPO = Path(__file__).resolve().parents[1]
|
||||
CORPUS = REPO / ".local-eval" / "originals"
|
||||
|
||||
@@ -6,6 +6,13 @@ High-level API (lazy, so ``import remove_ai_watermarks`` stays cheap)::
|
||||
raiw.remove_visible("in.png", "out.png") # clean a file (provenance auto)
|
||||
result, removed = raiw.remove_visible(bgr_array) # array -> array
|
||||
raiw.visible_provenance("in.png") # -> frozenset of confirmed vendors
|
||||
raiw.identify_video("in.mp4") # -> VideoProvenanceReport
|
||||
raiw.inspect_video_metadata("in.mp4") # -> VideoMetadataReport
|
||||
raiw.remove_video_all("in.mp4", "out.mp4") # visible + verified metadata
|
||||
raiw.remove_video_batch("videos", "videos_clean") # complete per-file results
|
||||
raiw.remove_video_metadata("in.mp4", "out.mp4") # verified metadata strip
|
||||
raiw.remove_video_invisible("in.mp4", "out.mp4") # oracle-certified SynthID removal
|
||||
raiw.remove_video_visible("in.mp4", "out.mp4") # stable visible video-mark removal
|
||||
|
||||
For a provenance verdict use the ``identify`` submodule::
|
||||
|
||||
@@ -25,19 +32,75 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
|
||||
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
|
||||
|
||||
|
||||
__version__ = "0.22.0"
|
||||
__version__ = "0.30.0"
|
||||
|
||||
__all__ = ["__version__", "remove_visible", "visible_provenance"]
|
||||
__all__ = [
|
||||
"BatchSummary",
|
||||
"InvisibleOptions",
|
||||
"MetadataStripIncomplete",
|
||||
"RemoveAllResult",
|
||||
"__version__",
|
||||
"identify_video",
|
||||
"inspect_video_metadata",
|
||||
"remove_all",
|
||||
"remove_batch",
|
||||
"remove_video_all",
|
||||
"remove_video_batch",
|
||||
"remove_video_invisible",
|
||||
"remove_video_metadata",
|
||||
"remove_video_visible",
|
||||
"remove_visible",
|
||||
"visible_provenance",
|
||||
]
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from remove_ai_watermarks.api import remove_visible, visible_provenance
|
||||
from remove_ai_watermarks.api import (
|
||||
BatchSummary,
|
||||
InvisibleOptions,
|
||||
MetadataStripIncomplete,
|
||||
RemoveAllResult,
|
||||
remove_all,
|
||||
remove_batch,
|
||||
remove_visible,
|
||||
visible_provenance,
|
||||
)
|
||||
from remove_ai_watermarks.video import (
|
||||
identify_video,
|
||||
inspect_video_metadata,
|
||||
remove_video_all,
|
||||
remove_video_batch,
|
||||
remove_video_invisible,
|
||||
remove_video_metadata,
|
||||
remove_video_visible,
|
||||
)
|
||||
|
||||
|
||||
def __getattr__(name: str) -> object:
|
||||
"""Lazily resolve the high-level API (PEP 562), so the heavy imports (cv2, the
|
||||
metadata/identify stack) load only when a caller actually reaches for them."""
|
||||
if name in ("remove_visible", "visible_provenance"):
|
||||
if name in (
|
||||
"BatchSummary",
|
||||
"InvisibleOptions",
|
||||
"MetadataStripIncomplete",
|
||||
"RemoveAllResult",
|
||||
"remove_all",
|
||||
"remove_batch",
|
||||
"remove_visible",
|
||||
"visible_provenance",
|
||||
):
|
||||
from remove_ai_watermarks import api
|
||||
|
||||
return getattr(api, name)
|
||||
if name in (
|
||||
"identify_video",
|
||||
"inspect_video_metadata",
|
||||
"remove_video_all",
|
||||
"remove_video_batch",
|
||||
"remove_video_invisible",
|
||||
"remove_video_metadata",
|
||||
"remove_video_visible",
|
||||
):
|
||||
from remove_ai_watermarks import video
|
||||
|
||||
return getattr(video, name)
|
||||
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Private namespace for metadata parsing and regeneration internals.
|
||||
|
||||
Deliberately empty. It carried a PEP 562 ``__getattr__`` re-exporting
|
||||
``WatermarkRemover`` and ``remove_ai_metadata`` as a "compatibility namespace",
|
||||
but nothing ever reached for either through this package -- every caller imports
|
||||
the submodule directly. The laziness it defended is real and still enforced, just
|
||||
elsewhere: importing a light submodule (``_internal.c2pa`` / ``_internal.constants``
|
||||
from ``identify``) must not pull ``watermark_remover``, which imports torch at
|
||||
module top. That property comes from those direct submodule imports, not from a
|
||||
shim here; a re-export in this file would be the one thing that could break it.
|
||||
|
||||
Keep this module free of imports. ``import remove_ai_watermarks.identify`` stays
|
||||
around 36 MB even in a full install where torch is present; routing anything heavy
|
||||
through here inflated it to roughly 420 MB and risked OOM on a 512 MB host.
|
||||
"""
|
||||
@@ -0,0 +1,755 @@
|
||||
"""C2PA inspection through the official reader with a bounded PNG fallback."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import functools
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import struct
|
||||
from collections import deque
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any, cast
|
||||
|
||||
from remove_ai_watermarks._internal.constants import (
|
||||
C2PA_ACTIONS,
|
||||
C2PA_AI_TOOLS,
|
||||
C2PA_AI_VENDORS,
|
||||
C2PA_CHUNK_TYPE,
|
||||
C2PA_ISSUERS,
|
||||
C2PA_SIGNATURES,
|
||||
C2PA_SOFT_BINDINGS,
|
||||
PNG_SIGNATURE,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from typing import BinaryIO
|
||||
|
||||
_C2paReader: Any = None
|
||||
_C2paError: Any = None
|
||||
with contextlib.suppress(Exception):
|
||||
from c2pa import C2paError as _C2paError # pyright: ignore[reportMissingTypeStubs]
|
||||
from c2pa import Reader as _C2paReader # pyright: ignore[reportMissingTypeStubs]
|
||||
|
||||
_C2PA_READER_AVAILABLE = _C2paReader is not None
|
||||
_PNG_HEADER = struct.Struct(">I4s")
|
||||
|
||||
_CONTENT_BINDING_MATCHES = (
|
||||
"assertion.dataHash.match",
|
||||
"assertion.bmffHash.match",
|
||||
"assertion.boxesHash.match",
|
||||
"assertion.collectionHash.match",
|
||||
"assertion.multiAssetHash.match",
|
||||
)
|
||||
_CONTENT_BINDING_FAILURE_MARKERS = (
|
||||
"Hash.incorrectFileCount",
|
||||
"Hash.malformed",
|
||||
"Hash.mismatch",
|
||||
"Hash.missingPart",
|
||||
"assertion.hashedURI.mismatch",
|
||||
"claim.hardBindings.missing",
|
||||
"hashedUri.mismatch",
|
||||
"ingredient.manifest.mismatch",
|
||||
)
|
||||
_SIGNATURE_FAILURES = frozenset(
|
||||
{
|
||||
"claimSignature.mismatch",
|
||||
"claimSignature.missing",
|
||||
"claimSignature.outsideValidity",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _PngChunk:
|
||||
payload: bytes
|
||||
serialized: bytes
|
||||
|
||||
|
||||
def reader_available() -> bool:
|
||||
"""Return whether the official C2PA reader loaded successfully."""
|
||||
return _C2PA_READER_AVAILABLE
|
||||
|
||||
|
||||
def _manifest_json_uncached(path: str) -> str | None:
|
||||
"""The manifest store as JSON, or None when this file has no readable manifest.
|
||||
|
||||
Two outcomes are routine and stay at debug: a file with no manifest (``try_create``
|
||||
returns None) and a container the reader does not support. ANY other failure is
|
||||
logged at warning, because the caller cannot tell the difference from the return
|
||||
value and the consequence is severe: the verdict silently falls back to the raw
|
||||
byte scan and can lose a high-confidence signal. The log line preserves the
|
||||
diagnostic context needed to investigate an intermittent reader failure.
|
||||
"""
|
||||
try:
|
||||
reader = _C2paReader.try_create(path)
|
||||
except _C2paError.NotSupported as error:
|
||||
logger.debug("C2PA reader does not support %s: %s", path, error)
|
||||
return None
|
||||
except Exception as error:
|
||||
logger.warning("C2PA reader failed to open %s: %s: %s", path, type(error).__name__, error)
|
||||
return None
|
||||
if reader is None:
|
||||
return None
|
||||
try:
|
||||
with reader:
|
||||
return cast("str", reader.json())
|
||||
except Exception as error:
|
||||
# The reader opened the file, so a manifest is there; failing to serialize it
|
||||
# is never routine.
|
||||
logger.warning("C2PA reader could not serialize %s: %s: %s", path, type(error).__name__, error)
|
||||
return None
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=8)
|
||||
def _manifest_json_cached(path: str, _mtime_ns: int) -> str | None:
|
||||
return _manifest_json_uncached(path)
|
||||
|
||||
|
||||
def read_manifest_store_json(image_path: Path) -> str | None:
|
||||
"""Read the complete manifest-store JSON, caching it until the file changes."""
|
||||
if not reader_available():
|
||||
return None
|
||||
path = str(image_path)
|
||||
try:
|
||||
return _manifest_json_cached(path, image_path.stat().st_mtime_ns)
|
||||
except OSError:
|
||||
return _manifest_json_uncached(path)
|
||||
|
||||
|
||||
def _find_c2pa_chunk(path: Path) -> _PngChunk | None:
|
||||
"""Return the first recognizable C2PA chunk without loading the whole PNG."""
|
||||
try:
|
||||
stream = path.open("rb")
|
||||
except OSError:
|
||||
return None
|
||||
with stream:
|
||||
if stream.read(len(PNG_SIGNATURE)) != PNG_SIGNATURE:
|
||||
return None
|
||||
file_size = stream.seek(0, 2)
|
||||
stream.seek(len(PNG_SIGNATURE))
|
||||
while True:
|
||||
header = stream.read(_PNG_HEADER.size)
|
||||
if len(header) != _PNG_HEADER.size:
|
||||
return None
|
||||
length, kind = _PNG_HEADER.unpack(header)
|
||||
if length + 4 > file_size - stream.tell():
|
||||
return None
|
||||
if kind == C2PA_CHUNK_TYPE:
|
||||
payload = stream.read(length)
|
||||
crc = stream.read(4)
|
||||
if _looks_like_c2pa(payload):
|
||||
return _PngChunk(payload, header + payload + crc)
|
||||
else:
|
||||
stream.seek(length + 4, 1)
|
||||
if kind == b"IEND":
|
||||
return None
|
||||
|
||||
|
||||
def _is_well_formed_png(path: Path) -> bool:
|
||||
"""Validate PNG chunk bounds with seeks rather than payload allocations."""
|
||||
try:
|
||||
stream = path.open("rb")
|
||||
except OSError:
|
||||
return False
|
||||
with stream:
|
||||
if stream.read(len(PNG_SIGNATURE)) != PNG_SIGNATURE:
|
||||
return False
|
||||
file_size = stream.seek(0, 2)
|
||||
stream.seek(len(PNG_SIGNATURE))
|
||||
while True:
|
||||
header = stream.read(_PNG_HEADER.size)
|
||||
if len(header) != _PNG_HEADER.size:
|
||||
return False
|
||||
length, kind = _PNG_HEADER.unpack(header)
|
||||
if length + 4 > file_size - stream.tell():
|
||||
return False
|
||||
stream.seek(length + 4, 1)
|
||||
if kind == b"IEND":
|
||||
return True
|
||||
|
||||
|
||||
def _copy_bytes(source: BinaryIO, target: BinaryIO, byte_count: int) -> None:
|
||||
"""Copy exactly one bounded chunk without allocating its complete payload."""
|
||||
remaining = byte_count
|
||||
while remaining:
|
||||
block = source.read(min(remaining, 1024 * 1024))
|
||||
if not block:
|
||||
raise OSError("PNG changed while it was being copied")
|
||||
target.write(block)
|
||||
remaining -= len(block)
|
||||
|
||||
|
||||
def _looks_like_c2pa(payload: bytes) -> bool:
|
||||
lowered = payload.lower()
|
||||
return any(signature in payload for signature in C2PA_SIGNATURES) or b"c2pa" in lowered or b"jumb" in lowered
|
||||
|
||||
|
||||
def extract_c2pa_chunk(image_path: Path) -> bytes | None:
|
||||
"""Return the first complete C2PA PNG chunk, including header and CRC."""
|
||||
if image_path.suffix.casefold() != ".png":
|
||||
return None
|
||||
chunk = _find_c2pa_chunk(image_path)
|
||||
return None if chunk is None else chunk.serialized
|
||||
|
||||
|
||||
def has_c2pa_metadata(image_path: Path) -> bool:
|
||||
"""Return whether a validly bounded PNG contains a recognizable C2PA chunk."""
|
||||
return extract_c2pa_chunk(Path(image_path)) is not None
|
||||
|
||||
|
||||
def _active_manifest(store: dict[str, Any]) -> dict[str, Any]:
|
||||
manifests = store.get("manifests")
|
||||
if not isinstance(manifests, dict):
|
||||
return {}
|
||||
typed_manifests = cast("dict[object, object]", manifests)
|
||||
active = typed_manifests.get(store.get("active_manifest"))
|
||||
return cast("dict[str, Any]", active) if isinstance(active, dict) else {}
|
||||
|
||||
|
||||
def _manifest_chain(store: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
"""Return the active manifest followed by credential-usable reachable ingredients."""
|
||||
manifests = store.get("manifests")
|
||||
active_label = store.get("active_manifest")
|
||||
if not isinstance(manifests, dict) or not isinstance(active_label, str):
|
||||
return []
|
||||
typed_manifests = cast("dict[object, object]", manifests)
|
||||
pending = deque([active_label])
|
||||
seen: set[str] = set()
|
||||
chain: list[dict[str, Any]] = []
|
||||
while pending:
|
||||
label = pending.popleft()
|
||||
if label in seen:
|
||||
continue
|
||||
seen.add(label)
|
||||
manifest_value = typed_manifests.get(label)
|
||||
if not isinstance(manifest_value, dict):
|
||||
continue
|
||||
manifest = cast("dict[str, Any]", manifest_value)
|
||||
chain.append(manifest)
|
||||
ingredients = manifest.get("ingredients")
|
||||
if not isinstance(ingredients, list):
|
||||
continue
|
||||
for ingredient_value in cast("list[object]", ingredients):
|
||||
if not isinstance(ingredient_value, dict):
|
||||
continue
|
||||
ingredient = cast("dict[object, object]", ingredient_value)
|
||||
child = ingredient.get("active_manifest")
|
||||
if (
|
||||
isinstance(child, str)
|
||||
and child not in seen
|
||||
and not _store_has_invalid_credential(cast("dict[str, Any]", ingredient))
|
||||
):
|
||||
pending.append(child)
|
||||
return chain
|
||||
|
||||
|
||||
def _status_codes(store: dict[str, Any]) -> tuple[list[str], list[str], list[str]]:
|
||||
"""Return ordered status codes for this store's active credential."""
|
||||
successes: list[str] = []
|
||||
informationals: list[str] = []
|
||||
failures: list[str] = []
|
||||
|
||||
def extend(status_set: object) -> None:
|
||||
if not isinstance(status_set, dict):
|
||||
return
|
||||
mapping = cast("dict[object, object]", status_set)
|
||||
for key, target in (
|
||||
("success", successes),
|
||||
("informational", informationals),
|
||||
("failure", failures),
|
||||
):
|
||||
values = mapping.get(key)
|
||||
if not isinstance(values, list):
|
||||
continue
|
||||
for value in cast("list[object]", values):
|
||||
if not isinstance(value, dict):
|
||||
continue
|
||||
code = cast("dict[object, object]", value).get("code")
|
||||
if isinstance(code, str) and code:
|
||||
target.append(code)
|
||||
|
||||
validation_results = store.get("validation_results")
|
||||
if isinstance(validation_results, dict):
|
||||
results = cast("dict[object, object]", validation_results)
|
||||
extend(results.get("activeManifest"))
|
||||
|
||||
if not (successes or informationals or failures):
|
||||
# Older readers expose only the flat non-success list. It still carries the
|
||||
# exact codes needed to reject a mismatched binding or invalid credential.
|
||||
values = store.get("validation_status")
|
||||
if isinstance(values, list):
|
||||
for value in cast("list[object]", values):
|
||||
if not isinstance(value, dict):
|
||||
continue
|
||||
code = cast("dict[object, object]", value).get("code")
|
||||
if isinstance(code, str) and code:
|
||||
failures.append(code)
|
||||
return successes, informationals, failures
|
||||
|
||||
|
||||
def _store_has_invalid_credential(store: dict[str, Any]) -> bool:
|
||||
"""Return whether this store's own active credential failed validation."""
|
||||
_, _, failures = _status_codes(store)
|
||||
return any(marker in code for code in failures for marker in _CONTENT_BINDING_FAILURE_MARKERS) or any(
|
||||
code in _SIGNATURE_FAILURES for code in failures
|
||||
)
|
||||
|
||||
|
||||
def c2pa_info_has_invalid_credential(info: dict[str, Any]) -> bool:
|
||||
"""Return whether parsed C2PA info reports a failed binding or signature."""
|
||||
return info.get("c2pa_integrity") == "invalid" or info.get("c2pa_signature") == "invalid"
|
||||
|
||||
|
||||
def c2pa_info_has_removal_hint(info: dict[str, Any]) -> bool:
|
||||
"""Return whether a C2PA AI or watermark claim should keep removal fail-safe."""
|
||||
soft_bindings = info.get("soft_binding_vendors")
|
||||
has_invismark = isinstance(soft_bindings, list) and "Microsoft InvisMark" in soft_bindings
|
||||
return bool(info.get("ai_source_kind") or info.get("synthid_watermark") or has_invismark)
|
||||
|
||||
|
||||
def _has_suffix(codes: list[str], suffixes: tuple[str, ...]) -> bool:
|
||||
return any(code.endswith(suffix) for code in codes for suffix in suffixes)
|
||||
|
||||
|
||||
def _validation_fields(store: dict[str, Any]) -> dict[str, Any]:
|
||||
successes, informationals, failures = _status_codes(store)
|
||||
all_codes = list(dict.fromkeys([*successes, *informationals, *failures]))
|
||||
|
||||
binding_failed = any(marker in code for code in failures for marker in _CONTENT_BINDING_FAILURE_MARKERS)
|
||||
binding_matched = _has_suffix(successes, _CONTENT_BINDING_MATCHES)
|
||||
signature_failed = any(code in _SIGNATURE_FAILURES for code in failures)
|
||||
signature_validated = "claimSignature.validated" in successes
|
||||
|
||||
if binding_failed:
|
||||
integrity = "invalid"
|
||||
elif binding_matched:
|
||||
integrity = "valid"
|
||||
else:
|
||||
integrity = "unknown"
|
||||
if signature_failed:
|
||||
signature = "invalid"
|
||||
elif signature_validated:
|
||||
signature = "valid"
|
||||
else:
|
||||
signature = "unknown"
|
||||
|
||||
if "signingCredential.trusted" in successes:
|
||||
signer_trust = "trusted"
|
||||
elif "signingCredential.untrusted" in failures:
|
||||
signer_trust = "untrusted"
|
||||
else:
|
||||
signer_trust = "unknown"
|
||||
|
||||
if any(code in failures for code in ("signingCredential.invalid", "signingCredential.ocsp.revoked")):
|
||||
signer_validity = "invalid"
|
||||
elif "signingCredential.expired" in failures:
|
||||
signer_validity = "expired"
|
||||
elif "claimSignature.insideValidity" in successes:
|
||||
signer_validity = "valid"
|
||||
else:
|
||||
signer_validity = "unknown"
|
||||
|
||||
state = store.get("validation_state")
|
||||
return {
|
||||
"c2pa_validation_source": "reader",
|
||||
"c2pa_validation_state": state if isinstance(state, str) and state else "unknown",
|
||||
"c2pa_integrity": integrity,
|
||||
"c2pa_signature": signature,
|
||||
"c2pa_signer_trust": signer_trust,
|
||||
"c2pa_signer_validity": signer_validity,
|
||||
"c2pa_validation_codes": all_codes,
|
||||
}
|
||||
|
||||
|
||||
def _claim_generator_from_store(store: dict[str, Any]) -> str | None:
|
||||
active = _active_manifest(store)
|
||||
direct = active.get("claim_generator")
|
||||
if isinstance(direct, str) and direct.isprintable() and direct:
|
||||
return direct
|
||||
candidates = active.get("claim_generator_info")
|
||||
if isinstance(candidates, list) and candidates and isinstance(candidates[0], dict):
|
||||
candidate = cast("dict[object, object]", candidates[0])
|
||||
name = candidate.get("name")
|
||||
if isinstance(name, str) and name.isprintable() and name:
|
||||
return name
|
||||
return None
|
||||
|
||||
|
||||
def _structured_manifest_fields(store: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Extract provenance only from the active manifest and its ingredient graph."""
|
||||
chain = _manifest_chain(store)
|
||||
if not chain:
|
||||
return {}
|
||||
|
||||
info: dict[str, Any] = {}
|
||||
issuers: list[str] = []
|
||||
tools: list[str] = []
|
||||
actions: list[str] = []
|
||||
source_types: list[str] = []
|
||||
soft_binding_algorithms: list[str] = []
|
||||
soft_binding_values: list[str] = []
|
||||
claim_generator_asserts_ai = False
|
||||
|
||||
def add_tool_matches(value: str, *, asserts_ai: bool = False) -> None:
|
||||
nonlocal claim_generator_asserts_ai
|
||||
matches = _ordered_matches(value.encode(), C2PA_AI_TOOLS)
|
||||
tools.extend(matches)
|
||||
if asserts_ai and matches:
|
||||
claim_generator_asserts_ai = True
|
||||
|
||||
for manifest in chain:
|
||||
signature_value = manifest.get("signature_info")
|
||||
if isinstance(signature_value, dict):
|
||||
signature = cast("dict[object, object]", signature_value)
|
||||
for key in ("issuer", "common_name", "certificate_issuer"):
|
||||
value = signature.get(key)
|
||||
if isinstance(value, str):
|
||||
issuers.extend(_ordered_matches(value.encode(), C2PA_ISSUERS))
|
||||
|
||||
direct_generator = manifest.get("claim_generator")
|
||||
if isinstance(direct_generator, str):
|
||||
add_tool_matches(direct_generator, asserts_ai=True)
|
||||
|
||||
candidates = manifest.get("claim_generator_info")
|
||||
if isinstance(candidates, list):
|
||||
for candidate_value in cast("list[object]", candidates):
|
||||
if not isinstance(candidate_value, dict):
|
||||
continue
|
||||
name = cast("dict[object, object]", candidate_value).get("name")
|
||||
if isinstance(name, str):
|
||||
add_tool_matches(name, asserts_ai=True)
|
||||
|
||||
assertions = manifest.get("assertions")
|
||||
if not isinstance(assertions, list):
|
||||
continue
|
||||
for assertion_value in cast("list[object]", assertions):
|
||||
if not isinstance(assertion_value, dict):
|
||||
continue
|
||||
assertion = cast("dict[object, object]", assertion_value)
|
||||
label = assertion.get("label")
|
||||
data = assertion.get("data")
|
||||
if isinstance(label, str) and label.startswith("c2pa.soft-binding") and isinstance(data, dict):
|
||||
soft_binding = cast("dict[object, object]", data)
|
||||
algorithm = soft_binding.get("alg")
|
||||
if isinstance(algorithm, str):
|
||||
soft_binding_algorithms.append(algorithm)
|
||||
blocks = soft_binding.get("blocks")
|
||||
if isinstance(blocks, list):
|
||||
for block_value in cast("list[object]", blocks):
|
||||
if not isinstance(block_value, dict):
|
||||
continue
|
||||
value = cast("dict[object, object]", block_value).get("value")
|
||||
if isinstance(value, str) and value.isprintable() and len(value) <= 256:
|
||||
soft_binding_values.append(value)
|
||||
if not (isinstance(label, str) and label.startswith("c2pa.actions") and isinstance(data, dict)):
|
||||
continue
|
||||
action_values = cast("dict[object, object]", data).get("actions")
|
||||
if not isinstance(action_values, list):
|
||||
continue
|
||||
for action_value in cast("list[object]", action_values):
|
||||
if not isinstance(action_value, dict):
|
||||
continue
|
||||
action = cast("dict[object, object]", action_value)
|
||||
action_name = action.get("action")
|
||||
if isinstance(action_name, str):
|
||||
actions.append(C2PA_ACTIONS.get(action_name.encode(), action_name.removeprefix("c2pa.")))
|
||||
source_type = action.get("digitalSourceType")
|
||||
if isinstance(source_type, str):
|
||||
source_types.append(source_type)
|
||||
software_agent = action.get("softwareAgent")
|
||||
if isinstance(software_agent, dict):
|
||||
name = cast("dict[object, object]", software_agent).get("name")
|
||||
if isinstance(name, str):
|
||||
add_tool_matches(name)
|
||||
|
||||
issuers = list(dict.fromkeys(issuers))
|
||||
tools = list(dict.fromkeys(tools))
|
||||
actions = list(dict.fromkeys(actions))
|
||||
if issuers:
|
||||
info["issuer"] = ", ".join(issuers)
|
||||
if tools:
|
||||
info["ai_tool"] = ", ".join(tools)
|
||||
if actions:
|
||||
info["actions"] = ", ".join(actions)
|
||||
if claim_generator_asserts_ai:
|
||||
info["c2pa_identity_ai"] = True
|
||||
|
||||
generated = any(
|
||||
"trainedAlgorithmicMedia" in value and "compositeWithTrainedAlgorithmicMedia" not in value
|
||||
for value in source_types
|
||||
)
|
||||
enhanced = any("compositeWithTrainedAlgorithmicMedia" in value for value in source_types)
|
||||
procedural = any("algorithmicMedia" in value for value in source_types)
|
||||
if generated:
|
||||
info.update(source_type="trainedAlgorithmicMedia (AI-generated)", ai_source_kind="generated")
|
||||
elif enhanced:
|
||||
info.update(source_type="compositeWithTrainedAlgorithmicMedia (AI-enhanced)", ai_source_kind="enhanced")
|
||||
elif procedural:
|
||||
info["source_type"] = "algorithmicMedia"
|
||||
|
||||
has_watermark_action = any(action.startswith("watermarked") for action in actions)
|
||||
if has_watermark_action:
|
||||
info["watermarked"] = True
|
||||
if info.get("ai_source_kind"):
|
||||
selected_bytes = json.dumps(chain, ensure_ascii=False).encode()
|
||||
synthid = synthid_evidence_vendors_in(selected_bytes, has_watermark_action=has_watermark_action)
|
||||
if synthid:
|
||||
info["synthid_vendors"] = synthid
|
||||
info["synthid_watermark"] = synthid_verdict(", ".join(synthid))
|
||||
|
||||
if soft_binding_algorithms:
|
||||
soft_binding_algorithms = list(dict.fromkeys(soft_binding_algorithms))
|
||||
algorithms = "\n".join(soft_binding_algorithms).encode()
|
||||
soft_bindings = soft_binding_vendors_in(algorithms)
|
||||
if soft_bindings:
|
||||
info["soft_binding_vendors"] = soft_bindings
|
||||
info["soft_binding"] = ", ".join(soft_bindings)
|
||||
info["soft_binding_algorithm"] = ", ".join(soft_binding_algorithms)
|
||||
if soft_binding_values:
|
||||
info["soft_binding_value"] = ", ".join(dict.fromkeys(soft_binding_values))
|
||||
return info
|
||||
|
||||
|
||||
def synthid_verdict(vendors: str) -> str:
|
||||
"""Describe why supported provenance establishes a SynthID watermark."""
|
||||
return f"present according to {vendors} provenance"
|
||||
|
||||
|
||||
def _names_present(buffer: bytes, registry: dict[bytes, str]) -> list[str]:
|
||||
return sorted({label for token, label in registry.items() if token in buffer})
|
||||
|
||||
|
||||
def synthid_evidence_vendors_in(buffer: bytes, *, has_watermark_action: bool | None = None) -> list[str]:
|
||||
"""List issuers whose provenance establishes SynthID for this asset.
|
||||
|
||||
Google applies SynthID to all media generated by its tools, so its AI C2PA
|
||||
provenance is sufficient. OpenAI C2PA predates OpenAI's SynthID rollout;
|
||||
current manifests distinguish the watermarked generation with the explicit
|
||||
``c2pa.watermarked.*`` action. A legacy OpenAI issuer token alone therefore
|
||||
remains provenance evidence, but not SynthID evidence.
|
||||
"""
|
||||
if has_watermark_action is None:
|
||||
has_watermark_action = b"c2pa.watermarked" in buffer
|
||||
return sorted(
|
||||
{
|
||||
vendor.org
|
||||
for vendor in C2PA_AI_VENDORS
|
||||
if vendor.synthid
|
||||
and vendor.issuer in buffer
|
||||
and (has_watermark_action or not vendor.synthid_requires_watermark_action)
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def soft_binding_vendors_in(buffer: bytes) -> list[str]:
|
||||
"""List the soft-binding algorithms named in manifest bytes."""
|
||||
return _names_present(buffer, C2PA_SOFT_BINDINGS)
|
||||
|
||||
|
||||
def _ordered_matches(buffer: bytes, registry: dict[bytes, str]) -> list[str]:
|
||||
return list(dict.fromkeys(label for token, label in registry.items() if token in buffer))
|
||||
|
||||
|
||||
def _populate_registry_fields(buffer: bytes, info: dict[str, Any]) -> bool:
|
||||
issuers = _ordered_matches(buffer, C2PA_ISSUERS)
|
||||
tools = _ordered_matches(buffer, C2PA_AI_TOOLS)
|
||||
actions = _ordered_matches(buffer, C2PA_ACTIONS)
|
||||
if issuers:
|
||||
info["issuer"] = ", ".join(issuers)
|
||||
if tools:
|
||||
info["ai_tool"] = ", ".join(tools)
|
||||
if actions:
|
||||
info["actions"] = ", ".join(actions)
|
||||
|
||||
ai_source = False
|
||||
if b"trainedAlgorithmicMedia" in buffer:
|
||||
info.update(source_type="trainedAlgorithmicMedia (AI-generated)", ai_source_kind="generated")
|
||||
ai_source = True
|
||||
elif b"compositeWithTrainedAlgorithmicMedia" in buffer:
|
||||
info.update(source_type="compositeWithTrainedAlgorithmicMedia (AI-enhanced)", ai_source_kind="enhanced")
|
||||
ai_source = True
|
||||
elif b"algorithmicMedia" in buffer:
|
||||
info["source_type"] = "algorithmicMedia"
|
||||
|
||||
if b"c2pa.watermarked" in buffer:
|
||||
info["watermarked"] = True
|
||||
synthid = synthid_evidence_vendors_in(buffer, has_watermark_action=info.get("watermarked", False))
|
||||
if ai_source and synthid:
|
||||
info["synthid_vendors"] = synthid
|
||||
info["synthid_watermark"] = synthid_verdict(", ".join(synthid))
|
||||
|
||||
soft_bindings = soft_binding_vendors_in(buffer)
|
||||
if soft_bindings:
|
||||
info["soft_binding_vendors"] = soft_bindings
|
||||
info["soft_binding"] = ", ".join(soft_bindings)
|
||||
return ai_source
|
||||
|
||||
|
||||
def _base_info(byte_count: int, *, fallback: bool = False) -> dict[str, Any]:
|
||||
container = "C2PA manifest" if fallback else "C2PA manifest store"
|
||||
info: dict[str, Any] = {
|
||||
"has_c2pa": True,
|
||||
"type": "C2PA (Coalition for Content Provenance and Authenticity)",
|
||||
"c2pa_manifest": f"{container} ({byte_count} bytes)",
|
||||
}
|
||||
if fallback:
|
||||
info.update(
|
||||
c2pa_validation_source="fallback",
|
||||
c2pa_validation_state="unknown",
|
||||
c2pa_integrity="unknown",
|
||||
c2pa_signature="unknown",
|
||||
c2pa_signer_trust="unknown",
|
||||
c2pa_signer_validity="unknown",
|
||||
c2pa_validation_codes=[],
|
||||
)
|
||||
return info
|
||||
|
||||
|
||||
def _info_from_store(store: dict[str, Any], encoded: bytes) -> dict[str, Any]:
|
||||
info = _base_info(len(encoded))
|
||||
info.update(_structured_manifest_fields(store))
|
||||
info.update(_validation_fields(store))
|
||||
generator = _claim_generator_from_store(store)
|
||||
if generator is not None:
|
||||
info["claim_generator"] = generator
|
||||
signature_value = _active_manifest(store).get("signature_info")
|
||||
if isinstance(signature_value, dict):
|
||||
signature = cast("dict[object, object]", signature_value)
|
||||
timestamp = signature.get("time")
|
||||
if timestamp:
|
||||
info["timestamp"] = str(timestamp)
|
||||
return info
|
||||
|
||||
|
||||
def c2pa_info_from_manifest_store(store: str | dict[str, Any]) -> dict[str, Any]:
|
||||
"""Normalize a manifest store supplied as JSON text or a decoded object."""
|
||||
try:
|
||||
raw_decoded: object = store if isinstance(store, dict) else json.loads(store)
|
||||
decoded = cast("dict[str, Any]", raw_decoded) if isinstance(raw_decoded, dict) else None
|
||||
if not isinstance(decoded, dict) or not decoded or decoded.get("error"):
|
||||
return {}
|
||||
encoded = json.dumps(decoded, ensure_ascii=False).encode() if isinstance(store, dict) else store.encode()
|
||||
except (TypeError, ValueError):
|
||||
return {}
|
||||
return _info_from_store(decoded, encoded)
|
||||
|
||||
|
||||
def cbor_text_after(payload: bytes, key: bytes) -> str | None:
|
||||
"""Decode a definite-length CBOR text value immediately following ``key``."""
|
||||
key_end = payload.find(key)
|
||||
if key_end < 0:
|
||||
return None
|
||||
cursor = key_end + len(key)
|
||||
if cursor >= len(payload):
|
||||
return None
|
||||
initial = payload[cursor]
|
||||
if 0x60 <= initial <= 0x77:
|
||||
length, cursor = initial & 0x1F, cursor + 1
|
||||
elif initial == 0x78 and cursor + 1 < len(payload):
|
||||
length, cursor = payload[cursor + 1], cursor + 2
|
||||
elif initial == 0x79 and cursor + 2 < len(payload):
|
||||
length = int.from_bytes(payload[cursor + 1 : cursor + 3], "big")
|
||||
cursor += 3
|
||||
else:
|
||||
return None
|
||||
raw = payload[cursor : cursor + length]
|
||||
if len(raw) != length:
|
||||
return None
|
||||
try:
|
||||
return raw.decode()
|
||||
except UnicodeDecodeError:
|
||||
return raw.decode("latin1", errors="replace")
|
||||
|
||||
|
||||
def _parse_c2pa_chunk(payload: bytes, info: dict[str, Any]) -> None:
|
||||
info.update(_base_info(len(payload), fallback=True))
|
||||
_populate_registry_fields(payload, info)
|
||||
for key, output_key in ((b"name", "claim_generator"), (b"specVersion", "c2pa_spec")):
|
||||
value = cbor_text_after(payload, key)
|
||||
if value and value.isprintable():
|
||||
info[output_key] = value
|
||||
timestamps = [item.decode() for item in re.findall(rb"\d{14}Z", payload)]
|
||||
if timestamps:
|
||||
info["timestamp"] = timestamps[0]
|
||||
if len(timestamps) > 1:
|
||||
info["timestamps"] = timestamps[:3]
|
||||
|
||||
|
||||
def _extract_c2pa_info_png(image_path: Path) -> dict[str, Any]:
|
||||
if image_path.suffix.casefold() != ".png":
|
||||
return {}
|
||||
chunk = _find_c2pa_chunk(image_path)
|
||||
if chunk is None:
|
||||
return {}
|
||||
info: dict[str, Any] = {}
|
||||
_parse_c2pa_chunk(chunk.payload, info)
|
||||
return info
|
||||
|
||||
|
||||
def _extract_c2pa_info_impl(image_path: Path) -> dict[str, Any]:
|
||||
store = read_manifest_store_json(Path(image_path))
|
||||
if store is not None:
|
||||
return c2pa_info_from_manifest_store(store)
|
||||
return _extract_c2pa_info_png(Path(image_path))
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _extract_c2pa_info_cached(path_str: str, _mtime_ns: int, _size: int, _reader: bool) -> dict[str, Any]:
|
||||
"""Cache shim: every argument after the path is key-only.
|
||||
|
||||
``_reader`` is in the key because the answer genuinely depends on it -- with the
|
||||
official reader the manifest comes back as a store, without it from the hand-rolled
|
||||
PNG chunk parser, and the two produce different ``c2pa_manifest`` labels. Keying on
|
||||
the file alone handed a reader-path result to a caller that had disabled the reader.
|
||||
"""
|
||||
return _extract_c2pa_info_impl(Path(path_str))
|
||||
|
||||
|
||||
def extract_c2pa_info(image_path: Path) -> dict[str, Any]:
|
||||
"""Return normalized C2PA evidence from the official reader or PNG fallback.
|
||||
|
||||
Memoized on ``(path, mtime_ns, size)``: one ``identify`` reaches this twice (once
|
||||
directly, once inside ``get_ai_metadata``) and each call re-runs the Rust manifest
|
||||
reader and re-parses its JSON. Size joins mtime in the key because this package
|
||||
rewrites files in place, and an in-place rewrite can land inside one mtime tick.
|
||||
"""
|
||||
try:
|
||||
stat = image_path.stat()
|
||||
except OSError:
|
||||
# No stat (a pipe, or a race): read uncached rather than fail.
|
||||
return _extract_c2pa_info_impl(image_path)
|
||||
cached = _extract_c2pa_info_cached(str(image_path), stat.st_mtime_ns, stat.st_size, _C2PA_READER_AVAILABLE)
|
||||
# Deep-ish copy: values are scalars plus a few lists, and a caller mutating one of
|
||||
# those lists would otherwise poison every later reader of the same file.
|
||||
return {key: list(cast("list[Any]", value)) if isinstance(value, list) else value for key, value in cached.items()}
|
||||
|
||||
|
||||
def inject_c2pa_chunk(target_path: Path, output_path: Path, c2pa_chunk: bytes) -> None:
|
||||
"""Replace any C2PA chunks in a PNG and insert ``c2pa_chunk`` before IDAT."""
|
||||
if target_path.suffix.casefold() != ".png" or output_path.suffix.casefold() != ".png":
|
||||
raise ValueError("C2PA chunk injection is only supported for PNG files")
|
||||
if not _is_well_formed_png(target_path):
|
||||
raise ValueError("Target is not a well-formed PNG file")
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with target_path.open("rb") as source, output_path.open("wb") as target:
|
||||
target.write(source.read(len(PNG_SIGNATURE)))
|
||||
inserted = False
|
||||
while True:
|
||||
header = source.read(_PNG_HEADER.size)
|
||||
length, kind = _PNG_HEADER.unpack(header)
|
||||
if kind == b"IDAT" and not inserted:
|
||||
target.write(c2pa_chunk)
|
||||
inserted = True
|
||||
if kind == C2PA_CHUNK_TYPE:
|
||||
source.seek(length + 4, 1)
|
||||
else:
|
||||
target.write(header)
|
||||
_copy_bytes(source, target, length + 4)
|
||||
if kind == b"IEND":
|
||||
break
|
||||
@@ -0,0 +1,205 @@
|
||||
"""Registries shared by metadata extraction and provenance classification."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
def _tokens(value: str) -> tuple[str, ...]:
|
||||
return tuple(value.split("|"))
|
||||
|
||||
|
||||
SUPPORTED_FORMATS = frozenset(_tokens(".png|.jpg|.jpeg|.webp|.heic|.heif|.avif"))
|
||||
AI_METADATA_KEYS = _tokens(
|
||||
"parameters|postprocessing|extras|workflow|prompt|Dream|SD:mode|StableDiffusionVersion|"
|
||||
"generation_time|Model|Model hash|Seed"
|
||||
)
|
||||
AI_KEYWORDS = _tokens(
|
||||
"prompt|negative_prompt|sampler|cfg_scale|lora|diffusion|comfy|midjourney|dall-e|dalle|imagen|firefly|c2pa|chatgpt|gpt-4|sora|openai|truepic|stable_diffusion|invokeai"
|
||||
)
|
||||
|
||||
PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
|
||||
C2PA_CHUNK_TYPE = b"caBX"
|
||||
PNG_METADATA_CHUNKS = frozenset({b"tEXt", b"iTXt", b"zTXt", b"eXIf", b"iCCP"})
|
||||
RIFF_METADATA_CHUNKS = frozenset({b"EXIF", b"XMP ", b"ICCP", b"C2PA"})
|
||||
RIFF_CODED_IMAGE_CHUNKS = frozenset({b"VP8 ", b"VP8L", b"ALPH", b"ANMF"})
|
||||
C2PA_SIGNATURES = tuple(
|
||||
token.encode() for token in _tokens("c2pa|C2PA|jumb|jumd|JUMBF|jumbf|cbor|contentcreds|digid|assertions|manifest")
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class C2paAiVendor:
|
||||
"""One issuer signature and its normalized product attribution."""
|
||||
|
||||
issuer: bytes
|
||||
org: str
|
||||
platform: str | None
|
||||
needle: str | None
|
||||
synthid: bool = False
|
||||
asserts_ai: bool = False
|
||||
synthid_requires_watermark_action: bool = False
|
||||
|
||||
|
||||
def _vendor(
|
||||
issuer: bytes | str,
|
||||
org: str,
|
||||
platform: str | None,
|
||||
needle: str | None,
|
||||
*,
|
||||
synthid: bool = False,
|
||||
asserts_ai: bool = False,
|
||||
synthid_requires_watermark_action: bool = False,
|
||||
) -> C2paAiVendor:
|
||||
token = issuer.encode() if isinstance(issuer, str) else issuer
|
||||
return C2paAiVendor(
|
||||
token,
|
||||
org,
|
||||
platform,
|
||||
needle,
|
||||
synthid=synthid,
|
||||
asserts_ai=asserts_ai,
|
||||
synthid_requires_watermark_action=synthid_requires_watermark_action,
|
||||
)
|
||||
|
||||
|
||||
# Order is product priority when a manifest mentions more than one organization.
|
||||
C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
|
||||
_vendor(b"Microsoft", "Microsoft", "Microsoft (Copilot / Designer)", "Microsoft"),
|
||||
_vendor(b"Adobe", "Adobe", "Adobe Firefly", "Adobe"),
|
||||
_vendor(
|
||||
b"OpenAI",
|
||||
"OpenAI",
|
||||
"OpenAI (ChatGPT / gpt-image / DALL-E / Sora)",
|
||||
"OpenAI",
|
||||
synthid=True,
|
||||
synthid_requires_watermark_action=True,
|
||||
),
|
||||
_vendor(b"Google", "Google LLC", "Google (Gemini / Imagen)", "Google", synthid=True),
|
||||
_vendor(b"Stability AI", "Stability AI", "Stability AI (Stable Image / DreamStudio)", "Stability AI"),
|
||||
_vendor(b"Black Forest Labs", "Black Forest Labs", "Black Forest Labs (FLUX)", "Black Forest Labs"),
|
||||
_vendor(
|
||||
b"volcengine",
|
||||
"ByteDance (Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
|
||||
"ByteDance",
|
||||
),
|
||||
_vendor(
|
||||
"北京火山引擎科技有限公司",
|
||||
"ByteDance (Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
|
||||
"ByteDance",
|
||||
),
|
||||
_vendor(
|
||||
b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)", "ByteDance"
|
||||
),
|
||||
_vendor(
|
||||
b"Dreamina",
|
||||
"ByteDance (Dreamina)",
|
||||
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
|
||||
"ByteDance",
|
||||
asserts_ai=True,
|
||||
),
|
||||
_vendor(b"Canva", "Canva", "Canva (Magic Media)", "Canva"),
|
||||
_vendor(b"Eleven Labs", "ElevenLabs", "ElevenLabs", "ElevenLabs"),
|
||||
_vendor(b"fal-ai", "fal.ai", "fal.ai", "fal.ai", asserts_ai=True),
|
||||
_vendor(b"Bria", "Bria Artificial Intelligence", "Bria AI", "Bria", asserts_ai=True),
|
||||
_vendor(b"Truepic", "Truepic", None, None),
|
||||
)
|
||||
|
||||
C2PA_ISSUERS = {vendor.issuer: vendor.org for vendor in C2PA_AI_VENDORS}
|
||||
C2PA_IDENTITY_AI_ORGS = frozenset(vendor.org for vendor in C2PA_AI_VENDORS if vendor.asserts_ai)
|
||||
|
||||
# Product-specific claim generators can sign through a different upstream issuer.
|
||||
# Keep this attribution beside the issuer registry so every C2PA consumer has one
|
||||
# canonical source rather than maintaining a derived product map in identify.py.
|
||||
C2PA_CLAIM_GENERATOR_PLATFORMS: tuple[tuple[str, str], ...] = (
|
||||
("adobe_firefly", "Adobe Firefly"),
|
||||
("firefly", "Adobe Firefly"),
|
||||
("dreamina", "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"),
|
||||
("higgsfield ai", "Higgsfield AI"),
|
||||
("topaz labs image api", "Topaz Labs"),
|
||||
("tiktok ad creative toolbox", "TikTok Ad Creative Toolbox"),
|
||||
)
|
||||
|
||||
C2PA_AI_TOOLS = {
|
||||
token.encode(): label
|
||||
for token, label in (
|
||||
("GPT-4o", "GPT-4o"),
|
||||
("ChatGPT", "ChatGPT"),
|
||||
("Sora", "Sora"),
|
||||
("DALL-E", "DALL-E"),
|
||||
("DALL", "DALL-E"),
|
||||
("Imagen", "Imagen"),
|
||||
("Firefly", "Firefly"),
|
||||
("Dreamina", "Dreamina"),
|
||||
)
|
||||
}
|
||||
|
||||
C2PA_SOFT_BINDINGS = {
|
||||
b"com.adobe.trustmark": "Adobe TrustMark",
|
||||
b"com.adobe.icn": "Adobe (content fingerprint)",
|
||||
b"com.digimarc": "Digimarc",
|
||||
b"com.imatag.lamark": "Imatag (Lamark)",
|
||||
b"ai.steg": "Steg.AI",
|
||||
b"com.microsoft.invismark": "Microsoft InvisMark",
|
||||
b"com.microsoft.wavmark": "Microsoft WavMark",
|
||||
b"com.verimatrix": "Verimatrix",
|
||||
b"com.nagra.nexguard": "NAGRA NexGuard",
|
||||
b"com.aiwatermark": "AIWatermark (Meta PixelSeal)",
|
||||
b"ai.trufo": "Trufo",
|
||||
b"app.overlai": "Overlai",
|
||||
b"com.markany": "MarkAny",
|
||||
b"com.mentaport": "Mentaport",
|
||||
b"es.lumatrace": "LumaTrace",
|
||||
b"ai.verda": "VerdaAI",
|
||||
b"ai.contentlens": "ContentLens",
|
||||
b"io.iscc": "ISCC (content code)",
|
||||
}
|
||||
|
||||
AI_GENERATOR_TOKENS = frozenset(
|
||||
{
|
||||
"firefly",
|
||||
"dall-e",
|
||||
"dalle",
|
||||
"midjourney",
|
||||
"stable diffusion",
|
||||
"stable-diffusion",
|
||||
"stablediffusion",
|
||||
"comfyui",
|
||||
"automatic1111",
|
||||
"invokeai",
|
||||
"imagen",
|
||||
"gpt-image",
|
||||
"nightcafe",
|
||||
"ideogram",
|
||||
"leonardo",
|
||||
"flux",
|
||||
"dreamstudio",
|
||||
"novelai",
|
||||
"reve.com",
|
||||
# Luma AI stamps PNG tEXt Source="Luma AI" / Comment="Generated by
|
||||
# Luma AI's Uni-1 model (https://lumalabs.ai)"; the space-bearing token
|
||||
# avoids matching incidental "luma" runs (luma/chroma key names etc.).
|
||||
"luma ai",
|
||||
"lumalabs",
|
||||
"aphrodite ai",
|
||||
"apple photos clean up",
|
||||
"fal-ai",
|
||||
}
|
||||
)
|
||||
|
||||
_C2PA_ACTION_NAMES = _tokens("created|converted|edited|filtered|cropped|resized|opened|placed|watermarked.unbound")
|
||||
C2PA_ACTIONS = {f"c2pa.{action}".encode(): action for action in _C2PA_ACTION_NAMES}
|
||||
|
||||
|
||||
# TC260 producer identity -> the mark key whose vendor signs with it now lives on the
|
||||
# registry rows (``KnownMark.tc260_producer_codes``, read through
|
||||
# ``watermark_registry.tc260_producer_vendors``). Keeping the codes beside the mark is
|
||||
# what stops a newly registered TC260 vendor from silently falling back to ByteDance.
|
||||
#
|
||||
# What a TC260 label confirms when its producer is absent or unmapped. Historical
|
||||
# behaviour, kept as the fallback so an unrecognized producer never regresses to no
|
||||
# relaxation at all: ByteDance's two products are the ones the relaxed band was
|
||||
# calibrated on (see _text_mark_engine._DEFAULT_PROVENANCE_NCC_FACTOR).
|
||||
TC260_FALLBACK_VENDORS: frozenset[str] = frozenset({"doubao", "jimeng"})
|
||||
@@ -0,0 +1,154 @@
|
||||
"""Bounded Matroska/WebM metadata reader for native TC260 AIGC labels.
|
||||
|
||||
TC260-PG-20257A stores the label as a Matroska ``SimpleTag`` whose
|
||||
``TagName`` is ``AIGC`` and whose ``TagString`` is the normative JSON object.
|
||||
The walker seeks over unrelated elements such as clusters instead of reading
|
||||
their payloads.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, BinaryIO
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks.metadata import MAX_TC260_VALUE_BYTES, parse_tc260_aigc_json
|
||||
|
||||
_EBML_MAGIC = b"\x1aE\xdf\xa3"
|
||||
_SEGMENT_ID = 0x18538067
|
||||
_TAGS_ID = 0x1254C367
|
||||
_TAG_ID = 0x7373
|
||||
_SIMPLE_TAG_ID = 0x67C8
|
||||
_TAG_NAME_ID = 0x45A3
|
||||
_TAG_STRING_ID = 0x4487
|
||||
_MAX_TAG_NAME_BYTES = 256
|
||||
|
||||
|
||||
def _vint_length(first: int, *, maximum: int) -> int | None:
|
||||
"""Return an EBML variable-integer length from its first byte."""
|
||||
mask = 0x80
|
||||
for length in range(1, maximum + 1):
|
||||
if first & mask:
|
||||
return length
|
||||
mask >>= 1
|
||||
return None
|
||||
|
||||
|
||||
def _read_element_header(
|
||||
stream: BinaryIO,
|
||||
pos: int,
|
||||
limit: int,
|
||||
) -> tuple[int, int, int] | None:
|
||||
"""Return ``(element_id, payload_start, element_end)`` inside ``limit``."""
|
||||
if pos < 0 or pos >= limit:
|
||||
return None
|
||||
stream.seek(pos)
|
||||
first_raw = stream.read(1)
|
||||
if not first_raw:
|
||||
return None
|
||||
id_length = _vint_length(first_raw[0], maximum=4)
|
||||
if id_length is None or pos + id_length >= limit:
|
||||
return None
|
||||
element_id_raw = first_raw + stream.read(id_length - 1)
|
||||
if len(element_id_raw) != id_length:
|
||||
return None
|
||||
element_id = int.from_bytes(element_id_raw, "big")
|
||||
|
||||
size_first_raw = stream.read(1)
|
||||
if not size_first_raw:
|
||||
return None
|
||||
size_length = _vint_length(size_first_raw[0], maximum=8)
|
||||
if size_length is None:
|
||||
return None
|
||||
size_rest = stream.read(size_length - 1)
|
||||
if len(size_rest) != size_length - 1:
|
||||
return None
|
||||
marker = 1 << (8 - size_length)
|
||||
size_value = int.from_bytes(bytes([size_first_raw[0] & (marker - 1)]) + size_rest, "big")
|
||||
payload_start = pos + id_length + size_length
|
||||
if payload_start > limit:
|
||||
return None
|
||||
unknown_size = size_value == (1 << (7 * size_length)) - 1
|
||||
element_end = limit if unknown_size else payload_start + size_value
|
||||
if element_end > limit:
|
||||
return None
|
||||
return element_id, payload_start, element_end
|
||||
|
||||
|
||||
def _iter_elements(
|
||||
stream: BinaryIO,
|
||||
start: int,
|
||||
end: int,
|
||||
) -> Iterator[tuple[int, int, int]]:
|
||||
"""Yield valid direct children from one bounded EBML region."""
|
||||
pos = start
|
||||
while pos < end:
|
||||
header = _read_element_header(stream, pos, end)
|
||||
if header is None:
|
||||
return
|
||||
element_id, payload_start, element_end = header
|
||||
yield element_id, payload_start, element_end
|
||||
if element_end <= pos:
|
||||
return
|
||||
pos = element_end
|
||||
|
||||
|
||||
def _read_bounded(
|
||||
stream: BinaryIO,
|
||||
start: int,
|
||||
end: int,
|
||||
maximum: int,
|
||||
) -> bytes | None:
|
||||
size = end - start
|
||||
if size < 0 or size > maximum:
|
||||
return None
|
||||
stream.seek(start)
|
||||
value = stream.read(size)
|
||||
return value if len(value) == size else None
|
||||
|
||||
|
||||
def _simple_tag_payloads(
|
||||
stream: BinaryIO,
|
||||
start: int,
|
||||
end: int,
|
||||
) -> tuple[bytes, ...]:
|
||||
name: bytes | None = None
|
||||
values: list[bytes] = []
|
||||
for element_id, payload_start, element_end in _iter_elements(stream, start, end):
|
||||
if element_id == _TAG_NAME_ID:
|
||||
name = _read_bounded(stream, payload_start, element_end, _MAX_TAG_NAME_BYTES)
|
||||
elif element_id == _TAG_STRING_ID:
|
||||
value = _read_bounded(stream, payload_start, element_end, MAX_TC260_VALUE_BYTES)
|
||||
if value is not None:
|
||||
values.append(value)
|
||||
if name != b"AIGC":
|
||||
return ()
|
||||
return tuple(value for value in values if parse_tc260_aigc_json(value) is not None)
|
||||
|
||||
|
||||
def tc260_aigc_payloads(path: str | Path) -> tuple[bytes, ...]:
|
||||
"""Read validated TC260 values from Matroska/WebM ``SimpleTag`` entries."""
|
||||
found: list[bytes] = []
|
||||
try:
|
||||
with open(path, "rb") as stream:
|
||||
if stream.read(4) != _EBML_MAGIC:
|
||||
return ()
|
||||
stream.seek(0, 2)
|
||||
file_size = stream.tell()
|
||||
for element_id, payload_start, element_end in _iter_elements(stream, 0, file_size):
|
||||
if element_id != _SEGMENT_ID:
|
||||
continue
|
||||
for child_id, child_start, child_end in _iter_elements(stream, payload_start, element_end):
|
||||
if child_id != _TAGS_ID:
|
||||
continue
|
||||
for tag_id, tag_start, tag_end in _iter_elements(stream, child_start, child_end):
|
||||
if tag_id != _TAG_ID:
|
||||
continue
|
||||
for simple_id, simple_start, simple_end in _iter_elements(stream, tag_start, tag_end):
|
||||
if simple_id == _SIMPLE_TAG_ID:
|
||||
found.extend(_simple_tag_payloads(stream, simple_start, simple_end))
|
||||
except OSError:
|
||||
return ()
|
||||
return tuple(found)
|
||||
@@ -0,0 +1,98 @@
|
||||
"""Read image metadata without changing the source container."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any, cast
|
||||
|
||||
import piexif
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks._internal.c2pa import extract_c2pa_chunk, extract_c2pa_info, has_c2pa_metadata
|
||||
from remove_ai_watermarks._internal.constants import AI_KEYWORDS, AI_METADATA_KEYS
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
_EXIF_KEY = "exif"
|
||||
_AI_KEYS_CASEFOLD = frozenset(key.casefold() for key in AI_METADATA_KEYS)
|
||||
|
||||
|
||||
def _read_pillow_info(source_path: Path) -> dict[str, Any]:
|
||||
with Image.open(source_path) as image:
|
||||
return {key: value for key, value in image.info.items() if isinstance(key, str)}
|
||||
|
||||
|
||||
def _decode_exif(raw: object) -> tuple[str, object]:
|
||||
if not isinstance(raw, bytes):
|
||||
return "exif_raw", raw
|
||||
try:
|
||||
return _EXIF_KEY, piexif.load(raw)
|
||||
except Exception:
|
||||
return "exif_raw", raw
|
||||
|
||||
|
||||
def _is_ai_field(key: str) -> bool:
|
||||
folded = key.casefold()
|
||||
return folded in _AI_KEYS_CASEFOLD or any(token in folded for token in AI_KEYWORDS)
|
||||
|
||||
|
||||
def _attach_c2pa(source_path: Path, metadata: dict[str, Any]) -> None:
|
||||
if not has_c2pa_metadata(source_path):
|
||||
return
|
||||
metadata["c2pa"] = extract_c2pa_info(source_path)
|
||||
payload = extract_c2pa_chunk(source_path)
|
||||
if payload is not None:
|
||||
metadata["c2pa_chunk"] = payload
|
||||
|
||||
|
||||
def extract_metadata(source_path: Path) -> dict[str, Any]:
|
||||
"""Return every Pillow-visible field plus decoded EXIF and C2PA data."""
|
||||
raw_info = _read_pillow_info(source_path)
|
||||
metadata = dict(raw_info)
|
||||
if _EXIF_KEY in raw_info:
|
||||
metadata.pop(_EXIF_KEY, None)
|
||||
decoded_key, decoded_value = _decode_exif(raw_info[_EXIF_KEY])
|
||||
metadata[decoded_key] = decoded_value
|
||||
|
||||
_attach_c2pa(source_path, metadata)
|
||||
return metadata
|
||||
|
||||
|
||||
def extract_ai_metadata(source_path: Path) -> dict[str, Any]:
|
||||
"""Return only metadata keys recognized as AI provenance or generation data."""
|
||||
metadata = {key: value for key, value in _read_pillow_info(source_path).items() if _is_ai_field(key)}
|
||||
_attach_c2pa(source_path, metadata)
|
||||
return metadata
|
||||
|
||||
|
||||
def has_ai_metadata(image_path: Path) -> bool:
|
||||
"""Return whether a supported metadata signal is present."""
|
||||
if any(_is_ai_field(key) for key in _read_pillow_info(image_path)):
|
||||
return True
|
||||
return has_c2pa_metadata(image_path)
|
||||
|
||||
|
||||
def _summary_value(value: object) -> str:
|
||||
if isinstance(value, bytes):
|
||||
return f"<binary data ({len(value)} bytes)>"
|
||||
text = str(value)
|
||||
return text if len(text) <= 100 else f"{text[:100]}..."
|
||||
|
||||
|
||||
def get_ai_metadata_summary(source_path: Path) -> str:
|
||||
"""Format the AI-only metadata view for the command-line report."""
|
||||
metadata = extract_ai_metadata(source_path)
|
||||
if not metadata:
|
||||
return "No AI metadata found."
|
||||
|
||||
lines = ["AI Image Metadata:", "-" * 40]
|
||||
for key, value in metadata.items():
|
||||
if key == "c2pa_chunk":
|
||||
continue
|
||||
if key == "c2pa" and isinstance(value, dict):
|
||||
lines.append("C2PA Metadata:")
|
||||
c2pa_fields = cast("dict[str, object]", value)
|
||||
lines.extend(f" {name}: {_summary_value(item)}" for name, item in c2pa_fields.items())
|
||||
continue
|
||||
lines.append(f"{key}: {_summary_value(value)}")
|
||||
return "\n".join(lines)
|
||||
@@ -0,0 +1,160 @@
|
||||
"""Bounded FLV metadata reader for native TC260 AIGC labels.
|
||||
|
||||
TC260-PG-20257A stores the label in the ``onMetaData`` script tag as an AMF0
|
||||
property named ``AIGC`` whose string value is the normative JSON object. Media
|
||||
tag payloads are skipped without being loaded.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks.metadata import MAX_TC260_VALUE_BYTES, parse_tc260_aigc_json
|
||||
|
||||
_SCRIPT_TAG = 18
|
||||
_MAX_SCRIPT_BYTES = 4 * 1024 * 1024
|
||||
|
||||
|
||||
def _u24(value: bytes) -> int:
|
||||
return int.from_bytes(value, "big")
|
||||
|
||||
|
||||
def _amf0_string(data: bytes, position: int, *, long: bool = False) -> tuple[bytes, int] | None:
|
||||
length_size = 4 if long else 2
|
||||
if position + length_size > len(data):
|
||||
return None
|
||||
length = int.from_bytes(data[position : position + length_size], "big")
|
||||
start = position + length_size
|
||||
end = start + length
|
||||
if end > len(data):
|
||||
return None
|
||||
return data[start:end], end
|
||||
|
||||
|
||||
def _skip_amf0(data: bytes, position: int, depth: int = 0) -> int | None:
|
||||
if position >= len(data) or depth > 8:
|
||||
return None
|
||||
value_type = data[position]
|
||||
position += 1
|
||||
if value_type == 0:
|
||||
return position + 8 if position + 8 <= len(data) else None
|
||||
if value_type == 1:
|
||||
return position + 1 if position + 1 <= len(data) else None
|
||||
if value_type == 2:
|
||||
parsed = _amf0_string(data, position)
|
||||
return parsed[1] if parsed is not None else None
|
||||
if value_type in {5, 6}:
|
||||
return position
|
||||
if value_type == 7:
|
||||
return position + 2 if position + 2 <= len(data) else None
|
||||
if value_type == 11:
|
||||
return position + 10 if position + 10 <= len(data) else None
|
||||
if value_type == 12:
|
||||
parsed = _amf0_string(data, position, long=True)
|
||||
return parsed[1] if parsed is not None else None
|
||||
if value_type == 10:
|
||||
if position + 4 > len(data):
|
||||
return None
|
||||
count = int.from_bytes(data[position : position + 4], "big")
|
||||
position += 4
|
||||
for _ in range(count):
|
||||
next_position = _skip_amf0(data, position, depth + 1)
|
||||
if next_position is None:
|
||||
return None
|
||||
position = next_position
|
||||
return position
|
||||
if value_type in {3, 8}:
|
||||
if value_type == 8:
|
||||
if position + 4 > len(data):
|
||||
return None
|
||||
position += 4
|
||||
while position + 3 <= len(data):
|
||||
name_length = int.from_bytes(data[position : position + 2], "big")
|
||||
position += 2
|
||||
if name_length == 0 and data[position] == 9:
|
||||
return position + 1
|
||||
position += name_length
|
||||
if position > len(data):
|
||||
return None
|
||||
next_position = _skip_amf0(data, position, depth + 1)
|
||||
if next_position is None:
|
||||
return None
|
||||
position = next_position
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _script_payloads(data: bytes) -> tuple[bytes, ...]:
|
||||
first = _amf0_string(data, 1) if data[:1] == b"\x02" else None
|
||||
if first is None or first[0] != b"onMetaData":
|
||||
return ()
|
||||
position = first[1]
|
||||
if position >= len(data) or data[position] not in {3, 8}:
|
||||
return ()
|
||||
if data[position] == 8:
|
||||
position += 5
|
||||
else:
|
||||
position += 1
|
||||
found: list[bytes] = []
|
||||
while position + 3 <= len(data):
|
||||
name_length = int.from_bytes(data[position : position + 2], "big")
|
||||
position += 2
|
||||
if name_length == 0 and data[position] == 9:
|
||||
break
|
||||
name_end = position + name_length
|
||||
if name_end > len(data):
|
||||
break
|
||||
name = data[position:name_end]
|
||||
position = name_end
|
||||
if name == b"AIGC" and position < len(data) and data[position] in {2, 12}:
|
||||
long = data[position] == 12
|
||||
parsed = _amf0_string(data, position + 1, long=long)
|
||||
if parsed is None:
|
||||
break
|
||||
value, position = parsed
|
||||
if len(value) <= MAX_TC260_VALUE_BYTES and parse_tc260_aigc_json(value) is not None:
|
||||
found.append(value)
|
||||
continue
|
||||
next_position = _skip_amf0(data, position)
|
||||
if next_position is None:
|
||||
break
|
||||
position = next_position
|
||||
return tuple(found)
|
||||
|
||||
|
||||
def tc260_aigc_payloads(path: str | Path) -> tuple[bytes, ...]:
|
||||
"""Read validated TC260 values from FLV ``script.onMetaData.AIGC``."""
|
||||
found: list[bytes] = []
|
||||
try:
|
||||
with open(path, "rb") as stream:
|
||||
header = stream.read(9)
|
||||
if len(header) != 9 or header[:3] != b"FLV":
|
||||
return ()
|
||||
data_offset = int.from_bytes(header[5:9], "big")
|
||||
stream.seek(0, 2)
|
||||
file_size = stream.tell()
|
||||
position = data_offset + 4
|
||||
while position + 11 <= file_size:
|
||||
stream.seek(position)
|
||||
tag_header = stream.read(11)
|
||||
if len(tag_header) != 11:
|
||||
break
|
||||
tag_type = tag_header[0] & 0x1F
|
||||
data_size = _u24(tag_header[1:4])
|
||||
payload_start = position + 11
|
||||
payload_end = payload_start + data_size
|
||||
if payload_end + 4 > file_size:
|
||||
break
|
||||
if tag_type == _SCRIPT_TAG and data_size <= _MAX_SCRIPT_BYTES:
|
||||
payload = stream.read(data_size)
|
||||
if len(payload) == data_size:
|
||||
found.extend(_script_payloads(payload))
|
||||
if found:
|
||||
return tuple(found)
|
||||
position = payload_end + 4
|
||||
except OSError:
|
||||
return ()
|
||||
return tuple(found)
|
||||
@@ -0,0 +1,679 @@
|
||||
"""Minimal ISOBMFF box walker for AI provenance in AVIF / HEIF / MP4 / JPEG-XL.
|
||||
|
||||
The ISO Base Media File Format wraps content in nested ``[size:4][type:4][...]``
|
||||
boxes. C2PA stores its manifest in a top-level ``uuid`` box keyed by the
|
||||
C2PA UUID; JPEG-XL uses a ``jumb`` box (JUMBF) instead. To strip provenance
|
||||
without re-encoding, the image path drops matching boxes and emits the rest
|
||||
verbatim. The streaming MP4/MOV/M4A path instead preserves all offsets by
|
||||
retyping matching boxes as ``free`` and blanking their payloads in place. The
|
||||
codestream (``mdat`` for ISOBMFF, ``jxlc`` / ``jxlp`` for JPEG-XL) is untouched,
|
||||
so pixel, video, and audio data is preserved bit-for-bit.
|
||||
|
||||
TC260-PG-20257A video metadata is nested instead:
|
||||
``moov.udta.meta.keys/ilst``. Its detector seeks through those boxes without
|
||||
reading media payloads, and its stripper blanks the validated key/value in
|
||||
place so fast-start media offsets remain valid. Two serialization variants are
|
||||
covered: the ISO form (``meta`` as a FullBox under ``udta``) and the QuickTime
|
||||
form Doubao's iOS export writes (a bare ``meta`` box as a direct ``moov`` child,
|
||||
no FullBox header).
|
||||
|
||||
This file intentionally avoids dependencies on format-specific libraries
|
||||
(pillow-heif, pillow-jxl, pymp4) so it works on systems where they aren't
|
||||
installed.
|
||||
|
||||
Reference: ISO/IEC 14496-12 (ISOBMFF) and C2PA 2.1 spec §11.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import struct
|
||||
from typing import TYPE_CHECKING, Any, BinaryIO
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks.metadata import (
|
||||
AIGC_MARKERS,
|
||||
C2PA_UUID,
|
||||
IPTC_AI_FIELD_MARKERS,
|
||||
IPTC_AI_MARKERS,
|
||||
MAX_TC260_VALUE_BYTES,
|
||||
parse_tc260_aigc_json,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Top-level box types that may carry AI provenance. ``uuid`` boxes are checked
|
||||
# against ``C2PA_UUID`` / AI-label markers before being stripped; ``jumb`` boxes
|
||||
# are always stripped (JPEG-XL uses them exclusively for JUMBF).
|
||||
C2PA_BOX_TYPES: frozenset[bytes] = frozenset({b"uuid", b"jumb"})
|
||||
|
||||
# AI-label byte markers (TC260 AIGC, IPTC "Made with AI", IPTC 2025.1 AI fields)
|
||||
# whose presence inside an XMP ``uuid`` box means the box carries an AI label.
|
||||
# Matching the payload rather than a fixed XMP UUID avoids the XMP-box UUID
|
||||
# byte-order ambiguity and stays surgical: only AI-bearing XMP is dropped, plain
|
||||
# XMP (copyright, camera info) is kept.
|
||||
_AI_LABEL_MARKERS: tuple[bytes, ...] = AIGC_MARKERS + IPTC_AI_MARKERS + IPTC_AI_FIELD_MARKERS
|
||||
|
||||
# Adobe XMP packet delimiters (XMP spec part 3). In HEIF/AVIF the XMP packet
|
||||
# sits inside a ``meta``-box ``mime`` item whose bytes live in ``mdat`` / ``idat``,
|
||||
# out of reach of the top-level box stripper, so an AI-label packet there is
|
||||
# blanked in place (see ``blank_ai_xmp_packets``).
|
||||
_XMP_PACKET_RE = re.compile(rb"<\?xpacket begin=.*?<\?xpacket end=[^>]*?\?>", re.DOTALL)
|
||||
_STREAM_COPY_BYTES = 1024 * 1024
|
||||
STREAM_SCAN_BYTES = 4 * 1024 * 1024
|
||||
|
||||
|
||||
# TC260-PG-20257A stores an MP4/MOV label as an ``AIGC`` key in
|
||||
# ``moov.udta.meta.keys`` and its JSON value in the corresponding
|
||||
# ``moov.udta.meta.ilst`` item. The value is intentionally bounded before it is
|
||||
# read: the normative object is tiny, and a corrupt size must not allocate an
|
||||
# arbitrary amount of memory during an inspection.
|
||||
def _iter_top_level_boxes(data: bytes) -> Iterator[tuple[int, int, bytes, int]]:
|
||||
"""Yield ``(start, end, type, payload_offset)`` for each top-level box.
|
||||
|
||||
Handles all three ISOBMFF box-size encodings:
|
||||
- ``size > 1``: 32-bit size field is the total box length.
|
||||
- ``size == 1``: 64-bit ``largesize`` follows after the type field.
|
||||
- ``size == 0``: box runs to end of file.
|
||||
"""
|
||||
pos = 0
|
||||
n = len(data)
|
||||
while pos + 8 <= n:
|
||||
size32 = struct.unpack_from(">I", data, pos)[0]
|
||||
box_type = data[pos + 4 : pos + 8]
|
||||
if size32 == 1:
|
||||
if pos + 16 > n:
|
||||
return
|
||||
size = struct.unpack_from(">Q", data, pos + 8)[0]
|
||||
payload_off = pos + 16
|
||||
elif size32 == 0:
|
||||
size = n - pos
|
||||
payload_off = pos + 8
|
||||
else:
|
||||
size = size32
|
||||
payload_off = pos + 8
|
||||
if size < (payload_off - pos) or pos + size > n:
|
||||
return
|
||||
yield pos, pos + size, box_type, payload_off
|
||||
pos += size
|
||||
|
||||
|
||||
def _read_box_header(
|
||||
stream: BinaryIO,
|
||||
pos: int,
|
||||
limit: int,
|
||||
) -> tuple[int, bytes, int] | None:
|
||||
"""Return ``(end, type, payload_offset)`` for one box inside ``limit``."""
|
||||
if pos < 0 or pos + 8 > limit:
|
||||
return None
|
||||
stream.seek(pos)
|
||||
header = stream.read(8)
|
||||
if len(header) != 8:
|
||||
return None
|
||||
size32 = struct.unpack(">I", header[:4])[0]
|
||||
box_type = header[4:8]
|
||||
payload_off = pos + 8
|
||||
if size32 == 1:
|
||||
extended = stream.read(8)
|
||||
if len(extended) != 8:
|
||||
return None
|
||||
size = struct.unpack(">Q", extended)[0]
|
||||
payload_off = pos + 16
|
||||
elif size32 == 0:
|
||||
size = limit - pos
|
||||
else:
|
||||
size = size32
|
||||
end = pos + size
|
||||
if size < payload_off - pos or end > limit:
|
||||
return None
|
||||
return end, box_type, payload_off
|
||||
|
||||
|
||||
def iter_file_boxes(
|
||||
stream: BinaryIO,
|
||||
start: int,
|
||||
end: int,
|
||||
) -> Iterator[tuple[int, int, bytes, int]]:
|
||||
"""Yield valid boxes from one bounded container region."""
|
||||
pos = start
|
||||
while pos + 8 <= end:
|
||||
header = _read_box_header(stream, pos, end)
|
||||
if header is None:
|
||||
return
|
||||
box_end, box_type, payload_off = header
|
||||
yield pos, box_end, box_type, payload_off
|
||||
pos = box_end
|
||||
|
||||
|
||||
def _tc260_key_indices(
|
||||
stream: BinaryIO,
|
||||
payload_off: int,
|
||||
box_end: int,
|
||||
) -> dict[int, tuple[int, int]]:
|
||||
"""Map every exact ``AIGC`` key index to its byte span."""
|
||||
if payload_off + 8 > box_end:
|
||||
return {}
|
||||
stream.seek(payload_off)
|
||||
prefix = stream.read(8)
|
||||
if len(prefix) != 8:
|
||||
return {}
|
||||
entry_count = struct.unpack(">I", prefix[4:8])[0]
|
||||
pos = payload_off + 8
|
||||
found: dict[int, tuple[int, int]] = {}
|
||||
for index in range(1, entry_count + 1):
|
||||
if pos + 8 > box_end:
|
||||
return {}
|
||||
stream.seek(pos)
|
||||
header = stream.read(8)
|
||||
if len(header) != 8:
|
||||
return {}
|
||||
entry_size = struct.unpack(">I", header[:4])[0]
|
||||
entry_end = pos + entry_size
|
||||
if entry_size < 8 or entry_end > box_end:
|
||||
return {}
|
||||
name_start = pos + 8
|
||||
if entry_end - name_start == 4:
|
||||
stream.seek(name_start)
|
||||
if stream.read(4) == b"AIGC":
|
||||
found[index] = (name_start, entry_end)
|
||||
pos = entry_end
|
||||
return found
|
||||
|
||||
|
||||
# The box types a TC260-bearing ``meta`` box opens with or contains: ISO files
|
||||
# have ``hdlr`` then ``keys``/``ilst``; QuickTime metadata lists have ``hdlr``
|
||||
# and ``ilst`` only. Which payload offset (0 vs 4) yields such children is what
|
||||
# disambiguates the QuickTime form (no FullBox header) from the ISO one.
|
||||
_META_CHILD_TYPES = frozenset({b"hdlr", b"keys", b"ilst"})
|
||||
|
||||
|
||||
def _meta_child_boxes(
|
||||
stream: BinaryIO,
|
||||
meta_payload: int,
|
||||
meta_end: int,
|
||||
) -> Iterator[tuple[int, int, bytes, int]]:
|
||||
"""Yield the child boxes of one ``meta`` box in either serialized form.
|
||||
|
||||
ISO serializes ``meta`` as a FullBox, so its children start 4 bytes into
|
||||
the payload; QuickTime writes a bare box, so they start at 0. Both real
|
||||
forms open with a recognized child (``hdlr``), so the form is picked by
|
||||
which offset's first box type is one of ``_META_CHILD_TYPES``; a wrong
|
||||
probe reads garbage header bytes that match no known type.
|
||||
"""
|
||||
for offset in (0, 4):
|
||||
boxes = iter_file_boxes(stream, meta_payload + offset, meta_end)
|
||||
first = next(boxes, None)
|
||||
if first is not None and first[2] in _META_CHILD_TYPES:
|
||||
yield first
|
||||
yield from boxes
|
||||
return
|
||||
|
||||
|
||||
def _iter_tc260_meta_boxes(
|
||||
stream: BinaryIO,
|
||||
moov_payload: int,
|
||||
moov_end: int,
|
||||
) -> Iterator[tuple[int, int]]:
|
||||
"""Yield ``(payload, end)`` of every ``meta`` box that may hold a TC260 label.
|
||||
|
||||
The normative ISO placement is ``moov.udta.meta``; Doubao's iOS MOV export
|
||||
instead stores the label in a QuickTime-form ``meta`` box that hangs
|
||||
directly off ``moov``. Both are yielded so one consumer covers them.
|
||||
"""
|
||||
for _start, end, box_type, payload in iter_file_boxes(stream, moov_payload, moov_end):
|
||||
if box_type == b"udta":
|
||||
for _udta_start, udta_end, udta_type, udta_payload in iter_file_boxes(stream, payload, end):
|
||||
if udta_type == b"meta":
|
||||
yield udta_payload, udta_end
|
||||
elif box_type == b"meta":
|
||||
yield payload, end
|
||||
|
||||
|
||||
def _tc260_aigc_regions(
|
||||
stream: BinaryIO,
|
||||
file_size: int,
|
||||
) -> list[tuple[tuple[int, int] | None, int, int, bytes]]:
|
||||
"""Locate validated native TC260 entries without reading media payloads.
|
||||
|
||||
Each tuple is ``(key_span, value_start, value_end, value)``; ``key_span`` is
|
||||
the byte span of the ``AIGC`` key when the normative ``keys`` box maps the
|
||||
item, or None for the QuickTime metadata-list form (``hdlr=mdir``) Doubao's
|
||||
iOS export writes, where the JSON sits in a bare ``ilst`` data item with no
|
||||
key name to blank.
|
||||
"""
|
||||
regions: list[tuple[tuple[int, int] | None, int, int, bytes]] = []
|
||||
for _moov_start, moov_end, moov_type, moov_payload in iter_file_boxes(stream, 0, file_size):
|
||||
if moov_type != b"moov":
|
||||
continue
|
||||
for meta_payload, meta_end in _iter_tc260_meta_boxes(stream, moov_payload, moov_end):
|
||||
keys: dict[int, tuple[int, int]] = {}
|
||||
ilst_boxes: list[tuple[int, int]] = []
|
||||
keyed = False
|
||||
for _child_start, child_end, child_type, child_payload in _meta_child_boxes(
|
||||
stream,
|
||||
meta_payload,
|
||||
meta_end,
|
||||
):
|
||||
if child_type == b"keys":
|
||||
keyed = True
|
||||
keys.update(_tc260_key_indices(stream, child_payload, child_end))
|
||||
elif child_type == b"ilst":
|
||||
ilst_boxes.append((child_payload, child_end))
|
||||
if not ilst_boxes:
|
||||
continue
|
||||
for ilst_payload, ilst_end in ilst_boxes:
|
||||
for _item_start, item_end, item_type, item_payload in iter_file_boxes(
|
||||
stream,
|
||||
ilst_payload,
|
||||
ilst_end,
|
||||
):
|
||||
index = int.from_bytes(item_type, "big")
|
||||
key_span = keys.get(index)
|
||||
if keyed and key_span is None:
|
||||
# A keyed (ISO) meta box maps items through ``keys``;
|
||||
# an unmapped index is not an AIGC entry, and reading
|
||||
# its value would pull arbitrary metadata (e.g. cover
|
||||
# art) through the JSON parser on every scan. Only the
|
||||
# keyless QuickTime list falls through to content
|
||||
# validation below.
|
||||
continue
|
||||
for _data_start, data_end, data_type, data_payload in iter_file_boxes(
|
||||
stream,
|
||||
item_payload,
|
||||
item_end,
|
||||
):
|
||||
value_start = data_payload + 8
|
||||
value_size = data_end - value_start
|
||||
if data_type != b"data" or value_size < 0 or value_size > MAX_TC260_VALUE_BYTES:
|
||||
continue
|
||||
stream.seek(value_start)
|
||||
value = stream.read(value_size)
|
||||
if len(value) == value_size and parse_tc260_aigc_json(value) is not None:
|
||||
regions.append((key_span, value_start, data_end, value))
|
||||
return regions
|
||||
|
||||
|
||||
def tc260_aigc_payloads(path: str | Path) -> tuple[bytes, ...]:
|
||||
"""Read native TC260 ``AIGC`` JSON values from an MP4/MOV container."""
|
||||
try:
|
||||
with open(path, "rb") as stream:
|
||||
if not is_isobmff(stream.read(8)):
|
||||
return ()
|
||||
stream.seek(0, 2)
|
||||
file_size = stream.tell()
|
||||
return tuple(region[3] for region in _tc260_aigc_regions(stream, file_size))
|
||||
except OSError:
|
||||
return ()
|
||||
|
||||
|
||||
def blank_tc260_aigc_tags(data: bytes) -> tuple[bytes, int]:
|
||||
"""Blank native TC260 values in place while preserving every box offset.
|
||||
|
||||
Removing a nested ``ilst`` item would shift ``mdat`` in a fast-start MP4 and
|
||||
invalidate its chunk offsets. Replacing the four-byte key with ``free`` and
|
||||
the JSON value with spaces keeps every box size and media offset unchanged.
|
||||
A keyless QuickTime metadata-list entry has no key name, so only its value
|
||||
is blanked.
|
||||
"""
|
||||
if not is_isobmff(data):
|
||||
return data, 0
|
||||
regions = _tc260_aigc_regions(io.BytesIO(data), len(data))
|
||||
if not regions:
|
||||
return data, 0
|
||||
out = bytearray(data)
|
||||
key_spans: set[tuple[int, int]] = set()
|
||||
for key_span, value_start, value_end, _value in regions:
|
||||
if key_span is not None:
|
||||
key_spans.add(key_span)
|
||||
out[key_span[0] : key_span[1]] = b"free"
|
||||
out[value_start:value_end] = b" " * (value_end - value_start)
|
||||
return bytes(out), len(key_spans)
|
||||
|
||||
|
||||
def is_isobmff(data: bytes) -> bool:
|
||||
"""Cheap sniff: ISOBMFF files start with an ``ftyp`` box."""
|
||||
return len(data) >= 8 and data[4:8] == b"ftyp"
|
||||
|
||||
|
||||
def scan_c2pa_region(path: str | Path, *, max_total: int = 4 * 1024 * 1024) -> bytes:
|
||||
"""Concatenated payloads of top-level ``uuid`` / ``jumb`` boxes in an ISOBMFF
|
||||
file, found by seeking past other boxes (``mdat`` etc.) by size.
|
||||
|
||||
C2PA manifests and XMP packets (incl. AI labels) live in top-level ``uuid``
|
||||
boxes; JPEG-XL uses ``jumb``. In a streaming / non-faststart MP4 the manifest
|
||||
sits AFTER a multi-megabyte ``mdat``, so a fixed first-MB read misses it. This
|
||||
walks box headers (8-16 bytes each) and seeks past payloads it does not need,
|
||||
so it never loads ``mdat`` into memory and works on multi-GB files. Returns
|
||||
the relevant box payloads (capped at ``max_total``), or ``b""`` for a
|
||||
non-ISOBMFF file or on any read error.
|
||||
"""
|
||||
collected = bytearray()
|
||||
try:
|
||||
with open(path, "rb") as f:
|
||||
sniff = f.read(8)
|
||||
if len(sniff) < 8 or sniff[4:8] != b"ftyp":
|
||||
return b""
|
||||
f.seek(0, 2)
|
||||
file_size = f.tell()
|
||||
pos = 0
|
||||
while pos + 8 <= file_size and len(collected) < max_total:
|
||||
f.seek(pos)
|
||||
header = f.read(8)
|
||||
if len(header) < 8:
|
||||
break
|
||||
size32 = struct.unpack(">I", header[:4])[0]
|
||||
box_type = header[4:8]
|
||||
payload_off = pos + 8
|
||||
if size32 == 1:
|
||||
ext = f.read(8)
|
||||
if len(ext) < 8:
|
||||
break
|
||||
size = struct.unpack(">Q", ext)[0]
|
||||
payload_off = pos + 16
|
||||
elif size32 == 0:
|
||||
size = file_size - pos
|
||||
else:
|
||||
size = size32
|
||||
if size < (payload_off - pos) or pos + size > file_size:
|
||||
# Detection-only: a malformed box halts the walk, so a manifest
|
||||
# placed after it is missed (best-effort scan; no resync).
|
||||
break
|
||||
if box_type in C2PA_BOX_TYPES:
|
||||
f.seek(payload_off)
|
||||
to_read = min(pos + size - payload_off, max_total - len(collected))
|
||||
if to_read > 0:
|
||||
collected += f.read(to_read)
|
||||
pos += size
|
||||
except OSError:
|
||||
return b""
|
||||
return bytes(collected)
|
||||
|
||||
|
||||
def _payload_has_ai_label(
|
||||
stream: BinaryIO,
|
||||
start: int,
|
||||
end: int,
|
||||
*,
|
||||
max_scan: int,
|
||||
) -> bool:
|
||||
"""Scan a bounded prefix of one metadata payload for an AI-label marker."""
|
||||
longest_marker = max(len(marker) for marker in _AI_LABEL_MARKERS)
|
||||
remaining = min(end - start, max_scan)
|
||||
overlap = b""
|
||||
stream.seek(start)
|
||||
while remaining > 0:
|
||||
chunk = stream.read(min(_STREAM_COPY_BYTES, remaining))
|
||||
if not chunk:
|
||||
return False
|
||||
searchable = overlap + chunk
|
||||
if any(marker in searchable for marker in _AI_LABEL_MARKERS):
|
||||
return True
|
||||
overlap = searchable[-(longest_marker - 1) :]
|
||||
remaining -= len(chunk)
|
||||
return False
|
||||
|
||||
|
||||
def _streaming_provenance_boxes(
|
||||
stream: BinaryIO,
|
||||
file_size: int,
|
||||
*,
|
||||
max_scan: int,
|
||||
) -> list[tuple[int, int, int]] | None:
|
||||
"""Return top-level provenance boxes, or ``None`` for a malformed walk.
|
||||
|
||||
Each result is ``(box_start, payload_start, box_end)``. The walk reads only
|
||||
headers and bounded metadata prefixes, seeking over ``mdat`` payloads.
|
||||
"""
|
||||
stream.seek(0)
|
||||
if not is_isobmff(stream.read(8)):
|
||||
return None
|
||||
targets: list[tuple[int, int, int]] = []
|
||||
pos = 0
|
||||
while pos < file_size:
|
||||
header = _read_box_header(stream, pos, file_size)
|
||||
if header is None:
|
||||
return None
|
||||
box_end, box_type, payload_off = header
|
||||
if box_type == b"uuid":
|
||||
stream.seek(payload_off)
|
||||
is_c2pa = payload_off + 16 <= box_end and stream.read(16) == C2PA_UUID
|
||||
has_ai_label = not is_c2pa and _payload_has_ai_label(
|
||||
stream,
|
||||
payload_off,
|
||||
box_end,
|
||||
max_scan=max_scan,
|
||||
)
|
||||
if is_c2pa or has_ai_label:
|
||||
targets.append((pos, payload_off, box_end))
|
||||
elif box_type == b"jumb":
|
||||
targets.append((pos, payload_off, box_end))
|
||||
pos = box_end
|
||||
return targets
|
||||
|
||||
|
||||
def _overwrite_range(
|
||||
stream: BinaryIO,
|
||||
start: int,
|
||||
end: int,
|
||||
*,
|
||||
byte: bytes,
|
||||
) -> None:
|
||||
"""Overwrite one byte range with bounded allocations."""
|
||||
stream.seek(start)
|
||||
remaining = end - start
|
||||
block = byte * min(_STREAM_COPY_BYTES, max(remaining, 1))
|
||||
while remaining > 0:
|
||||
size = min(len(block), remaining)
|
||||
stream.write(block[:size])
|
||||
remaining -= size
|
||||
|
||||
|
||||
def strip_isobmff_media_file(
|
||||
source: str | Path,
|
||||
output: str | Path,
|
||||
*,
|
||||
max_box_scan: int = STREAM_SCAN_BYTES,
|
||||
) -> tuple[int, int]:
|
||||
"""Stream-copy an MP4/MOV/M4A while removing supported AI metadata.
|
||||
|
||||
The output retains every box size and byte offset. A top-level C2PA/JUMBF or
|
||||
AI-label box is converted to a ``free`` box and its payload is zeroed; native
|
||||
TC260 key/value spans are blanked in place. Keeping the original lengths is
|
||||
required because removing a pre-``mdat`` box would invalidate absolute media
|
||||
offsets in an existing sample table.
|
||||
|
||||
The source is copied in bounded chunks to a sibling temporary file and
|
||||
atomically published only after all patches succeed. A malformed top-level
|
||||
walk is fail-safe: the input is copied unchanged.
|
||||
|
||||
Returns ``(provenance_boxes_blanked, native_tc260_keys_blanked)``.
|
||||
"""
|
||||
from pathlib import Path as _Path
|
||||
|
||||
from remove_ai_watermarks.video_encoding import atomic_video_output
|
||||
|
||||
source_path = _Path(source)
|
||||
output_path = _Path(output)
|
||||
with source_path.open("rb") as stream:
|
||||
stream.seek(0, 2)
|
||||
file_size = stream.tell()
|
||||
targets = _streaming_provenance_boxes(
|
||||
stream,
|
||||
file_size,
|
||||
max_scan=max_box_scan,
|
||||
)
|
||||
tc260_regions = _tc260_aigc_regions(stream, file_size) if targets is not None else []
|
||||
tc260_key_spans = {region[0] for region in tc260_regions if region[0] is not None}
|
||||
|
||||
with atomic_video_output(output_path) as temporary_path:
|
||||
with source_path.open("rb") as source_stream, temporary_path.open("r+b") as temporary:
|
||||
shutil.copyfileobj(source_stream, temporary, length=_STREAM_COPY_BYTES)
|
||||
if targets is not None:
|
||||
for box_start, payload_start, box_end in targets:
|
||||
temporary.seek(box_start + 4)
|
||||
temporary.write(b"free")
|
||||
_overwrite_range(temporary, payload_start, box_end, byte=b"\x00")
|
||||
for key_span, value_start, value_end, _value in tc260_regions:
|
||||
if key_span is not None:
|
||||
temporary.seek(key_span[0])
|
||||
temporary.write(b"free")
|
||||
_overwrite_range(temporary, value_start, value_end, byte=b" ")
|
||||
temporary.flush()
|
||||
os.fsync(temporary.fileno())
|
||||
shutil.copymode(source_path, temporary_path)
|
||||
|
||||
if targets is None:
|
||||
logger.warning(
|
||||
"ISOBMFF box walk failed for %s; copied input unchanged to avoid corrupting media offsets",
|
||||
source_path,
|
||||
)
|
||||
return 0, 0
|
||||
return len(targets), len(tc260_key_spans)
|
||||
|
||||
|
||||
def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
|
||||
"""Return ``(cleaned_bytes, stripped_count)`` with AI-provenance boxes removed.
|
||||
|
||||
Walks top-level boxes and drops:
|
||||
- any ``uuid`` box whose UUID equals ``C2PA_UUID`` (a C2PA manifest);
|
||||
- any ``uuid`` box whose payload carries an AI-label marker (an XMP packet
|
||||
with a TC260 / IPTC / IPTC-2025.1 AI field -- caught by content, not by the
|
||||
XMP UUID, so it works regardless of the UUID's byte order, and leaves plain
|
||||
non-AI XMP intact);
|
||||
- any ``jumb`` box (JPEG-XL JUMBF container).
|
||||
|
||||
All other boxes (incl. ``mdat`` / codestream) are emitted verbatim, so pixel
|
||||
and audio data is preserved bit-for-bit. Non-ISOBMFF input is returned
|
||||
unchanged. Despite the name this also covers MP4/MOV/M4A video and audio
|
||||
(all ISOBMFF). NOTE: this drops only top-level boxes. AI metadata stored as an
|
||||
*item inside the ``meta`` box* (typical for AVIF/HEIF) is handled separately and
|
||||
in place (same length, no offset rewrite): AI-label XMP by
|
||||
:func:`blank_ai_xmp_packets`, and AI-generator tokens in an ``Exif`` item by
|
||||
:func:`blank_ai_exif_tokens`.
|
||||
"""
|
||||
if not is_isobmff(data):
|
||||
return data, 0
|
||||
|
||||
out = bytearray()
|
||||
stripped = 0
|
||||
consumed = 0
|
||||
for start, end, box_type, payload_off in _iter_top_level_boxes(data):
|
||||
consumed = end
|
||||
if box_type == b"uuid":
|
||||
# uuid boxes carry the 16-byte UUID immediately after the type.
|
||||
is_c2pa = payload_off + 16 <= end and data[payload_off : payload_off + 16] == C2PA_UUID
|
||||
has_ai_label = any(marker in data[payload_off:end] for marker in _AI_LABEL_MARKERS)
|
||||
if is_c2pa or has_ai_label:
|
||||
stripped += 1
|
||||
continue
|
||||
elif box_type == b"jumb":
|
||||
stripped += 1
|
||||
continue
|
||||
out.extend(data[start:end])
|
||||
|
||||
# Fail-safe: the walker returns early on a malformed box (bad size, or a box
|
||||
# that runs past EOF), so anything after it was never visited. Emitting `out`
|
||||
# would silently truncate the file from the bad box to EOF -- worse than not
|
||||
# stripping. If the walk did not consume the whole input, return it unchanged.
|
||||
if consumed != len(data):
|
||||
logger.warning(
|
||||
"ISOBMFF box walk stopped at offset %d of %d (malformed box); "
|
||||
"returning input unchanged to avoid truncation",
|
||||
consumed,
|
||||
len(data),
|
||||
)
|
||||
return data, 0
|
||||
|
||||
return bytes(out), stripped
|
||||
|
||||
|
||||
def blank_ai_xmp_packets(data: bytes) -> tuple[bytes, int]:
|
||||
"""Overwrite (with spaces, in place) any XMP packet carrying an AI-label
|
||||
marker; return ``(data, blanked_count)``.
|
||||
|
||||
HEIF/AVIF store XMP as a ``meta``-box ``mime`` item whose bytes live in
|
||||
``mdat`` / ``idat``, which ``strip_c2pa_boxes`` cannot remove without
|
||||
meta-box surgery (``iinf`` / ``iloc`` rewrite). Instead, the XMP packet is
|
||||
located by its ``<?xpacket begin ... end?>`` delimiters and, when it carries
|
||||
an AI-label marker (TC260 AIGC / IPTC / IPTC-2025.1), overwritten with spaces.
|
||||
Because the replacement is the **same length**, every box size and ``iloc``
|
||||
offset stays valid and the coded image data is untouched -- only the AI label
|
||||
content is destroyed. Packets without an AI marker (plain copyright / camera
|
||||
XMP) are left intact, mirroring the top-level XMP-``uuid`` content match.
|
||||
"""
|
||||
blanked = 0
|
||||
|
||||
def _scrub(match: re.Match[bytes]) -> bytes:
|
||||
nonlocal blanked
|
||||
packet = match.group()
|
||||
if any(marker in packet for marker in _AI_LABEL_MARKERS):
|
||||
blanked += 1
|
||||
return b" " * len(packet)
|
||||
return packet
|
||||
|
||||
return _XMP_PACKET_RE.sub(_scrub, data), blanked
|
||||
|
||||
|
||||
# EXIF TIFF byte-order headers: little-endian (II 0x2a 0x00) and big-endian
|
||||
# (MM 0x00 0x2a). A HEIF/AVIF ``Exif`` meta-box item stores its TIFF block in
|
||||
# ``mdat`` / ``idat``, so the block (and these headers) appear in the raw bytes.
|
||||
_TIFF_HEADERS: tuple[bytes, ...] = (b"II\x2a\x00", b"MM\x00\x2a")
|
||||
# How far past a TIFF header an EXIF block plausibly extends; bounds the slice we
|
||||
# hand to piexif and search within (EXIF blocks are small kilobyte-scale).
|
||||
_EXIF_WINDOW = 256 * 1024
|
||||
|
||||
|
||||
def blank_ai_exif_tokens(data: bytes) -> tuple[bytes, int]:
|
||||
"""Overwrite (with spaces, in place) any AI-generator token in an EXIF block
|
||||
stored as an ISOBMFF ``meta``-box ``Exif`` item; return ``(data, blanked_count)``.
|
||||
|
||||
HEIF/AVIF can carry EXIF as a ``meta``-box ``Exif`` item whose TIFF bytes live
|
||||
in ``mdat`` / ``idat`` -- out of reach of the top-level box stripper, and (when
|
||||
no pillow-heif plugin is installed) of the PIL EXIF reader too, so an AI
|
||||
``Software`` / ``Make`` / ``Artist`` / ``ImageDescription`` tag there survived
|
||||
``remove_ai_metadata`` (a documented gap). This locates EXIF TIFF blocks by
|
||||
their byte-order header, **validates each with piexif** (so a coincidental
|
||||
II/MM run in pixel data is ignored -- it will not parse as a TIFF IFD), and
|
||||
overwrites any AI value with spaces of the SAME length. Because the replacement
|
||||
is same-length, every box size and ``iloc`` offset stays valid and the coded image
|
||||
is untouched -- only the AI tag content is destroyed; camera/editor EXIF without an
|
||||
AI token is left intact. This mirrors ``metadata._scrub_ai_exif`` in what it removes
|
||||
-- generator tokens (``Software``/``Make``/``Artist``/``ImageDescription``), the
|
||||
China TC260 ``{"AIGC":{...}}`` block (``ImageDescription``/``UserComment``), and the
|
||||
xAI/Grok ``Signature:`` + UUID-``Artist`` pair -- since on the ISOBMFF path this is
|
||||
the ONLY EXIF scrubber (``_scrub_ai_exif`` never runs there), so without parity a
|
||||
HEIC/AVIF AIGC/xAI tag is detected but not removed.
|
||||
"""
|
||||
import piexif
|
||||
|
||||
# The AI-EXIF rule set is defined ONCE in metadata._ai_exif_targets and shared by both
|
||||
# EXIF scrubbers (the JPEG _scrub_ai_exif pops the tag; here we blank the value bytes),
|
||||
# so their coverage cannot drift. Imported lazily to avoid import-order coupling with
|
||||
# metadata (which imports this module); a deliberate cross-module use, not an API leak.
|
||||
from remove_ai_watermarks.metadata import _ai_exif_targets # pyright: ignore[reportPrivateUsage]
|
||||
|
||||
out = bytearray(data)
|
||||
blanked = 0
|
||||
for header in _TIFF_HEADERS:
|
||||
pos = data.find(header)
|
||||
while pos != -1:
|
||||
window = bytes(out[pos : pos + _EXIF_WINDOW])
|
||||
try:
|
||||
loaded: dict[str, Any] = piexif.load(window)
|
||||
except Exception:
|
||||
loaded = {}
|
||||
for _ifd_key, _tag, value, _name in _ai_exif_targets(loaded):
|
||||
# Blank the value bytes in place, within this EXIF block only.
|
||||
vpos = out.find(value, pos, pos + _EXIF_WINDOW)
|
||||
if vpos != -1:
|
||||
out[vpos : vpos + len(value)] = b" " * len(value)
|
||||
blanked += 1
|
||||
pos = data.find(header, pos + len(header))
|
||||
return bytes(out), blanked
|
||||
+353
-110
@@ -1,17 +1,9 @@
|
||||
"""Qwen 2512 Canny regeneration followed by masked Z-Image face repair.
|
||||
"""Project-native Qwen regeneration with an optional masked face refinement pass.
|
||||
|
||||
This profile ports the two-stage architecture used by cebeuq/Synthid-Bypass:
|
||||
|
||||
1. Qwen-Image-2512 img2img with the 4-step Lightning LoRA and the DiffSynth
|
||||
blockwise Canny ControlNet regenerates the whole image, optionally as
|
||||
overlapping feather-blended tiles for large inputs.
|
||||
2. Faces are detected on the original image, refined to masks with SAM, regenerated
|
||||
from the original face crops with Z-Image Turbo, and feathered into stage 1.
|
||||
|
||||
The runtime intentionally uses permissively licensed YuNet instead of the reference
|
||||
workflow's Ultralytics detector. All diffusion and segmentation models remain the same
|
||||
model families. The adaptive formulas are direct ports, while the face result is scaled
|
||||
for this runtime's different sampler and mask-compositing path.
|
||||
The profile was inspired by public experiments that combine structure-guided global
|
||||
regeneration with a second face-only pass. Its orchestration, sizing rules, adaptive
|
||||
strength policy, detector, masks, prompts, and compositing are implemented here for
|
||||
this library's Pillow and DiffSynth runtime.
|
||||
"""
|
||||
|
||||
# DiffSynth, torch, transformers, and cv2 expose mostly untyped tensor/array APIs.
|
||||
@@ -33,11 +25,13 @@ from typing import TYPE_CHECKING, Any
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks.noai.watermark_profiles import resolve_seed
|
||||
from remove_ai_watermarks._internal.watermark_profiles import resolve_seed
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextManifest
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
QWEN_IMAGE_2512_MODEL_ID = "Qwen/Qwen-Image-2512"
|
||||
@@ -53,10 +47,8 @@ YUNET_MODEL_URL = (
|
||||
)
|
||||
YUNET_MODEL_NAME = "face_detection_yunet_2023mar.onnx"
|
||||
YUNET_MODEL_SHA256 = "8f2383e4dd3cfbb4553ea8718107fc0423210dc964f9f4280604804ed2552fa4"
|
||||
# The upstream graph's 0.2 threshold belongs to YOLO and does not transfer to
|
||||
# YuNet's score calibration. At 0.2 YuNet admitted background and decorative
|
||||
# false positives, multiplying the serial Z-Image face-stage cost. A 0.5 gate
|
||||
# retained all visible faces in the public and upstream comparison fixtures.
|
||||
# This threshold retained the faces in the public validation set without admitting
|
||||
# decorative background regions as faces.
|
||||
YUNET_SCORE_THRESHOLD = 0.5
|
||||
|
||||
GLOBAL_STEPS = 4
|
||||
@@ -65,19 +57,34 @@ GLOBAL_CFG = 1.0
|
||||
FACE_CFG = 1.0
|
||||
GLOBAL_CONTROLNET_SCALE = 1.0
|
||||
RESIDENT_FACE_MODEL_MIN_VRAM_GIB = 64.0
|
||||
# The reference face denoise assumes its ComfyUI detailer sampler, latent
|
||||
# noise-mask feather, and inpaint path. Applying that value unchanged to this
|
||||
# DiffSynth crop-regeneration port over-processes faces. Paired public-fixture
|
||||
# measurements and both provider oracles certified half the reference value.
|
||||
# Below this floor the mandatory Qwen stack streams from disk, which is what makes a
|
||||
# 20B model runnable on a consumer card at all; at or above it, streaming is pure
|
||||
# waste. Set equal to the face floor rather than lower because that is the
|
||||
# configuration actually measured with both stacks resident; a tighter gate is
|
||||
# plausible but unvalidated. Benchmark in docs/module-internals.md, "CPU offload".
|
||||
RESIDENT_GLOBAL_MODEL_MIN_VRAM_GIB = 64.0
|
||||
FACE_DENOISE_SCALE = 0.5
|
||||
|
||||
# The source graph uses normalized Canny thresholds 0.05 and 0.25. OpenCV takes
|
||||
# byte thresholds, so round 255*x to the matching integer values.
|
||||
# Both stages prompt with compile-time constants, and at CFG 1.0 DiffSynth reuses the
|
||||
# positive embedding for the negative side instead of encoding it, so exactly one
|
||||
# embedding per stage is ever needed. Persisting it lets the next container load the
|
||||
# stack without its text encoder at all: 15.45 GiB for Qwen and 7.49 GiB for Z-Image
|
||||
# of a measured 87.6 GiB read per request. The stored tensors are what the encoder
|
||||
# itself produced, so the output stays byte-identical. Bump the version when the
|
||||
# stored shape changes; the model id and prompt are already part of the key, so a
|
||||
# model or prompt change invalidates itself.
|
||||
_PROMPT_CACHE_VERSION = 1
|
||||
_PROMPT_CACHE_DIRNAME = "prompt-embeddings"
|
||||
# DiffSynth identifies a pipeline unit by what it produces, so these tuples are how
|
||||
# the prompt stage is located inside each pipeline and how its cache file is keyed.
|
||||
_QWEN_PROMPT_OUTPUTS = ("prompt_emb", "prompt_emb_mask")
|
||||
_ZIMAGE_PROMPT_OUTPUTS = ("prompt_embeds",)
|
||||
|
||||
_CANNY_LOW = 13
|
||||
_CANNY_HIGH = 64
|
||||
|
||||
# These strings intentionally preserve the reference workflow spelling. They are
|
||||
# model inputs, not user-facing copy, and changing them would change the port.
|
||||
# These short model inputs are retained as calibrated compatibility parameters.
|
||||
# Changing them requires the same provider-oracle and identity evaluation as a model change.
|
||||
_GLOBAL_PROMPT = "ultra clear and smoothe skin, spotless skin"
|
||||
_GLOBAL_NEGATIVE = "moles, freckes, high detail skin"
|
||||
_FACE_PROMPT = ""
|
||||
@@ -95,6 +102,17 @@ def resolve_face_model_residency(
|
||||
return total_memory_gib >= RESIDENT_FACE_MODEL_MIN_VRAM_GIB
|
||||
|
||||
|
||||
def resolve_global_model_residency(
|
||||
requested: bool | None,
|
||||
*,
|
||||
total_memory_gib: float,
|
||||
) -> bool:
|
||||
"""Keep the mandatory Qwen stack resident when explicitly requested or safely sized."""
|
||||
if requested is not None:
|
||||
return requested
|
||||
return total_memory_gib >= RESIDENT_GLOBAL_MODEL_MIN_VRAM_GIB
|
||||
|
||||
|
||||
def _pin_vram_managed_models(pipe: Any) -> None:
|
||||
"""Move the managed Z-Image stack to CUDA once and make offload a no-op."""
|
||||
model_names = ["text_encoder", "dit", "vae_encoder", "vae_decoder"]
|
||||
@@ -126,7 +144,56 @@ def _pin_vram_managed_models(pipe: Any) -> None:
|
||||
pipe.load_models_to_device(model_names)
|
||||
|
||||
|
||||
def _cached_prompt_process(original_process: Any) -> Any:
|
||||
def _cast_prompt_payload(payload: Any, device: Any, dtype: Any) -> Any:
|
||||
"""Move a stored embedding payload onto the runtime device.
|
||||
|
||||
Only floating tensors take the pipeline dtype. The attention masks travel in the
|
||||
same payload and are integer, so casting them would corrupt the prompt.
|
||||
"""
|
||||
import torch
|
||||
|
||||
if isinstance(payload, torch.Tensor):
|
||||
if dtype is not None and payload.is_floating_point():
|
||||
return payload.to(device=device, dtype=dtype)
|
||||
return payload.to(device=device)
|
||||
if isinstance(payload, dict):
|
||||
return {key: _cast_prompt_payload(value, device, dtype) for key, value in payload.items()}
|
||||
if isinstance(payload, (list, tuple)):
|
||||
return type(payload)(_cast_prompt_payload(value, device, dtype) for value in payload)
|
||||
return payload
|
||||
|
||||
|
||||
def _prompt_cache_path(model_id: str, output_params: tuple[str, ...], prompt: str) -> Path:
|
||||
"""Locate the stored embedding for one model and one exact prompt string."""
|
||||
key = "\x1f".join((str(_PROMPT_CACHE_VERSION), model_id, *output_params, prompt))
|
||||
digest = hashlib.sha256(key.encode("utf-8")).hexdigest()[:32]
|
||||
return _model_cache_dir() / _PROMPT_CACHE_DIRNAME / f"{digest}.pt"
|
||||
|
||||
|
||||
def _store_prompt_payload(path: Path, payload: Any) -> None:
|
||||
"""Write the embedding atomically so a torn write can never read back as a hit."""
|
||||
import torch
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.NamedTemporaryFile(dir=path.parent, suffix=".pt", delete=False) as handle:
|
||||
torch.save(_cast_prompt_payload(payload, "cpu", None), handle)
|
||||
temporary = Path(handle.name)
|
||||
temporary.replace(path)
|
||||
|
||||
|
||||
def _load_prompt_payload(path: Path, device: Any, dtype: Any) -> Any:
|
||||
import torch
|
||||
|
||||
return _cast_prompt_payload(torch.load(path, map_location="cpu", weights_only=True), device, dtype)
|
||||
|
||||
|
||||
def _cached_prompt_process(
|
||||
original_process: Any,
|
||||
*,
|
||||
model_id: str | None,
|
||||
output_params: tuple[str, ...],
|
||||
require_cache: bool,
|
||||
) -> Any:
|
||||
cache: dict[str, dict[str, Any]] = {}
|
||||
|
||||
def cached_process(
|
||||
@@ -136,9 +203,30 @@ def _cached_prompt_process(original_process: Any) -> Any:
|
||||
) -> dict[str, Any]:
|
||||
if edit_image is not None:
|
||||
return original_process(runtime_pipe, prompt, edit_image=edit_image)
|
||||
if prompt not in cache:
|
||||
cache[prompt] = original_process(runtime_pipe, prompt, edit_image=None)
|
||||
return cache[prompt]
|
||||
if prompt in cache:
|
||||
return cache[prompt]
|
||||
path = None if model_id is None else _prompt_cache_path(model_id, output_params, prompt)
|
||||
if path is not None and path.exists():
|
||||
try:
|
||||
cache[prompt] = _load_prompt_payload(path, runtime_pipe.device, runtime_pipe.torch_dtype)
|
||||
except Exception:
|
||||
log.warning("Discarding unreadable prompt-embedding cache %s", path, exc_info=True)
|
||||
else:
|
||||
return cache[prompt]
|
||||
if require_cache:
|
||||
# The text encoder was left out of the model stack on the strength of
|
||||
# this file, so there is nothing left to fall back to.
|
||||
raise RuntimeError(
|
||||
f"The cached prompt embedding {path} disappeared after the stack was loaded without its text encoder."
|
||||
)
|
||||
produced = original_process(runtime_pipe, prompt, edit_image=None)
|
||||
if path is not None:
|
||||
try:
|
||||
_store_prompt_payload(path, produced)
|
||||
except OSError:
|
||||
log.warning("Could not persist the prompt embedding to %s", path, exc_info=True)
|
||||
cache[prompt] = produced
|
||||
return produced
|
||||
|
||||
return cached_process
|
||||
|
||||
@@ -146,11 +234,25 @@ def _cached_prompt_process(original_process: Any) -> Any:
|
||||
def _cache_static_prompt_embeddings(
|
||||
pipe: Any,
|
||||
output_params: tuple[str, ...],
|
||||
*,
|
||||
model_id: str | None = None,
|
||||
require_cache: bool = False,
|
||||
) -> bool:
|
||||
"""Memoize a prompt unit when its embedding depends only on static text."""
|
||||
"""Memoize a prompt unit when its embedding depends only on static text.
|
||||
|
||||
With ``model_id`` the memo is also persisted, which is what lets the next
|
||||
container skip the text encoder entirely; without it the memo lives only as long
|
||||
as the pipeline. ``require_cache`` says the stack was already built without a
|
||||
text encoder, so a miss must fail loudly rather than call a model that is absent.
|
||||
"""
|
||||
for unit in pipe.units:
|
||||
if tuple(getattr(unit, "output_params", ())) == output_params:
|
||||
unit.process = _cached_prompt_process(unit.process)
|
||||
unit.process = _cached_prompt_process(
|
||||
unit.process,
|
||||
model_id=model_id,
|
||||
output_params=output_params,
|
||||
require_cache=require_cache,
|
||||
)
|
||||
return True
|
||||
return False
|
||||
|
||||
@@ -169,28 +271,18 @@ def resolution_adaptive_denoise(
|
||||
denoise_min: float = 0.08,
|
||||
denoise_max: float = 0.15,
|
||||
) -> float:
|
||||
"""Port the reference resolution-based adaptive denoise calculation.
|
||||
|
||||
At neutral level 5, 0.30 MP maps to ``denoise_min`` and 3.70 MP maps to
|
||||
``denoise_max``. Levels above or below 5 add the same asymmetric spread as the
|
||||
reference custom node.
|
||||
"""
|
||||
image_mp = max(1.0, float(width) * float(height)) / 1_000_000.0
|
||||
normalized = _clamp((image_mp - 0.30) / (3.70 - 0.30), 0.0, 1.0)
|
||||
|
||||
minimum = float(denoise_min)
|
||||
maximum = float(denoise_max)
|
||||
if maximum < minimum:
|
||||
minimum, maximum = maximum, minimum
|
||||
denoise_range = maximum - minimum
|
||||
base = minimum + denoise_range * normalized
|
||||
|
||||
level = int(adaptive_level)
|
||||
if level >= 5:
|
||||
offset = ((float(level) - 5.0) / 5.0) * denoise_range * 0.285714
|
||||
"""Choose the calibrated global strength from image area and operator level."""
|
||||
low, high = sorted((float(denoise_min), float(denoise_max)))
|
||||
megapixels = max(1.0, float(width) * float(height)) * 1e-6
|
||||
area_fraction = _clamp((megapixels - 0.30) / 3.40, 0.0, 1.0)
|
||||
strength_range = high - low
|
||||
strength = low + strength_range * area_fraction
|
||||
level_delta = float(int(adaptive_level)) - 5.0
|
||||
if level_delta >= 0.0:
|
||||
strength += level_delta * strength_range * (0.285714 / 5.0)
|
||||
else:
|
||||
offset = -((5.0 - float(level)) / 4.0) * denoise_range * 0.257143
|
||||
return _clamp(base + offset, 0.0001, 1.0)
|
||||
strength += level_delta * strength_range * (0.257143 / 4.0)
|
||||
return _clamp(strength, 0.0001, 1.0)
|
||||
|
||||
|
||||
def largest_face_denoise(
|
||||
@@ -202,7 +294,7 @@ def largest_face_denoise(
|
||||
denoise_min: float = 0.05,
|
||||
denoise_max: float = 0.28,
|
||||
) -> float:
|
||||
"""Scale face denoise from the largest face area, matching reference mode."""
|
||||
"""Choose the calibrated face strength from the largest detected face area."""
|
||||
width, height = image_size
|
||||
image_area = max(1.0, float(width) * float(height))
|
||||
largest_ratio = 0.0
|
||||
@@ -211,7 +303,7 @@ def largest_face_denoise(
|
||||
largest_ratio = max(largest_ratio, box_area / image_area)
|
||||
if largest_ratio <= 0.0:
|
||||
return _clamp(base_denoise, denoise_min, denoise_max)
|
||||
scaled = float(base_denoise) * (largest_ratio / max(1e-6, float(adaptive_ratio)))
|
||||
scaled = float(base_denoise) * largest_ratio / max(1e-6, float(adaptive_ratio))
|
||||
return _clamp(scaled, denoise_min, denoise_max)
|
||||
|
||||
|
||||
@@ -229,7 +321,7 @@ def _resize_to_target(image: Image.Image) -> Image.Image:
|
||||
|
||||
|
||||
def build_canny_control_image(image: Image.Image) -> Image.Image:
|
||||
"""Build the three-channel Canny conditioning image used by stage 1."""
|
||||
"""Build the calibrated three-channel Canny conditioning map."""
|
||||
import cv2
|
||||
|
||||
rgb = np.asarray(image.convert("RGB"))
|
||||
@@ -259,8 +351,6 @@ def build_global_kwargs(
|
||||
"seed": seed,
|
||||
"rand_device": "cpu",
|
||||
"num_inference_steps": GLOBAL_STEPS,
|
||||
# The source graph applies ModelSamplingAuraFlow with shift=3. DiffSynth
|
||||
# expresses the same rational sigma shift as exp(mu), so mu=log(3).
|
||||
"exponential_shift_mu": math.log(3.0),
|
||||
"blockwise_controlnet_inputs": [controlnet_input],
|
||||
}
|
||||
@@ -312,7 +402,7 @@ def _expanded_box(
|
||||
*,
|
||||
factor: float = 2.5,
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Expand a face box around its center, matching the reference crop factor."""
|
||||
"""Expand a face box around its center to include local lighting context."""
|
||||
x1, y1, x2, y2 = box
|
||||
image_width, image_height = image_size
|
||||
center_x = (x1 + x2) / 2.0
|
||||
@@ -328,7 +418,13 @@ def _expanded_box(
|
||||
|
||||
|
||||
def _model_cache_dir() -> Path:
|
||||
root = os.environ.get("XDG_CACHE_HOME")
|
||||
"""Where this library persists downloaded and derived model assets.
|
||||
|
||||
``HF_HOME`` comes first because on a scale-to-zero runner it is the one path
|
||||
mounted persistently; anything under a container-local cache is re-derived on
|
||||
every request, which defeats both the YuNet download and the prompt cache.
|
||||
"""
|
||||
root = os.environ.get("HF_HOME") or os.environ.get("XDG_CACHE_HOME")
|
||||
base = Path(root) if root else Path.home() / ".cache"
|
||||
return base / "remove-ai-watermarks"
|
||||
|
||||
@@ -467,11 +563,18 @@ def _clip_sam_masks_to_boxes(
|
||||
image_size: tuple[int, int],
|
||||
) -> list[np.ndarray]:
|
||||
"""Match Impact Pack by intersecting each SAM mask with its detector box."""
|
||||
# Same rectangle primitive the shared fill uses, rather than a private zeros/fill
|
||||
# copy. `dilate=0` because this box CLIPS a SAM mask -- growing it would admit the
|
||||
# pixels the clip exists to exclude. The function-local import keeps region_eraser's
|
||||
# module-scope cv2 off this module's import path.
|
||||
from remove_ai_watermarks.region_eraser import boxes_to_mask
|
||||
|
||||
width, height = image_size
|
||||
clipped: list[np.ndarray] = []
|
||||
for mask, (x1, y1, x2, y2) in zip(masks, boxes, strict=True):
|
||||
box_mask = np.zeros((height, width), dtype=np.uint8)
|
||||
box_mask[max(0, y1) : min(height, y2), max(0, x1) : min(width, x2)] = 255
|
||||
# (x, y, w, h) with w = x2 - x1 keeps x + w == x2, so a negative origin clamps
|
||||
# to the same span the explicit max/min pair produced.
|
||||
box_mask = boxes_to_mask((height, width), [(x1, y1, x2 - x1, y2 - y1)], dilate=0)
|
||||
clipped.append(np.bitwise_and(mask.astype(np.uint8), box_mask))
|
||||
return clipped
|
||||
|
||||
@@ -525,6 +628,7 @@ class QwenZImagePipeline:
|
||||
progress_callback: Callable[[str], None] | None = None
|
||||
controlnet_conditioning_scale: float = GLOBAL_CONTROLNET_SCALE
|
||||
keep_face_models_on_device: bool | None = None
|
||||
keep_global_models_on_device: bool | None = None
|
||||
cache_prompt_embeddings: bool = True
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
@@ -541,8 +645,9 @@ class QwenZImagePipeline:
|
||||
def _require_cuda(self) -> None:
|
||||
if self.device != "cuda":
|
||||
raise RuntimeError(
|
||||
"The qwen-zimage pipeline is CUDA-only. Its Qwen-Image-2512 and "
|
||||
"Z-Image models do not fit the supported MPS path."
|
||||
"The qwen-zimage pipeline is CUDA-only; there is no CPU or MPS path "
|
||||
"for it. WatermarkRemover already refuses a non-CUDA device, so "
|
||||
"reaching this guard means the pipeline was constructed directly."
|
||||
)
|
||||
|
||||
def _vram_limit(self) -> float | None:
|
||||
@@ -552,21 +657,54 @@ class QwenZImagePipeline:
|
||||
return max(1.0, torch.cuda.mem_get_info("cuda")[1] / (1024**3) - 0.5)
|
||||
return None
|
||||
|
||||
def _keep_face_models_resident(self) -> bool:
|
||||
def _total_vram_gib(self) -> float:
|
||||
"""Card capacity for the residency gates, 0.0 when it cannot be read.
|
||||
|
||||
Separate from ``_vram_limit``, which answers a different question (the budget
|
||||
handed to DiffSynth) and must stay ``None`` rather than 0.0 when unknown, so
|
||||
an unreadable device means "no limit" there and "assume small" here.
|
||||
"""
|
||||
import torch
|
||||
|
||||
total_memory_gib = 0.0
|
||||
with contextlib.suppress(Exception):
|
||||
total_memory_gib = torch.cuda.get_device_properties("cuda").total_memory / (1024**3)
|
||||
return torch.cuda.get_device_properties("cuda").total_memory / (1024**3)
|
||||
return 0.0
|
||||
|
||||
def _keep_face_models_resident(self) -> bool:
|
||||
return resolve_face_model_residency(
|
||||
self.keep_face_models_on_device,
|
||||
total_memory_gib=total_memory_gib,
|
||||
total_memory_gib=self._total_vram_gib(),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _qwen_vram_config() -> dict[str, Any]:
|
||||
def _keep_global_models_resident(self) -> bool:
|
||||
return resolve_global_model_residency(
|
||||
self.keep_global_models_on_device,
|
||||
total_memory_gib=self._total_vram_gib(),
|
||||
)
|
||||
|
||||
def _prompt_is_cached(self, model_id: str, output_params: tuple[str, ...], prompt: str) -> bool:
|
||||
"""Whether this stack can be built without its text encoder at all."""
|
||||
return self.cache_prompt_embeddings and _prompt_cache_path(model_id, output_params, prompt).exists()
|
||||
|
||||
def _qwen_vram_config(self) -> dict[str, Any]:
|
||||
import torch
|
||||
|
||||
if self._keep_global_models_resident():
|
||||
# Same fp8 storage and bf16 computation as the streaming config, but the
|
||||
# weights never leave the GPU. Passing no "disk" anywhere is what makes
|
||||
# this work: DiffSynth decides `disk_offload` once from `offload_dtype`,
|
||||
# so a card large enough to hold the stack skips both the `to("meta")`
|
||||
# drop and the DiskMap re-read entirely.
|
||||
return {
|
||||
"offload_dtype": torch.float8_e4m3fn,
|
||||
"offload_device": "cuda",
|
||||
"onload_dtype": torch.float8_e4m3fn,
|
||||
"onload_device": "cuda",
|
||||
"preparing_dtype": torch.float8_e4m3fn,
|
||||
"preparing_device": "cuda",
|
||||
"computation_dtype": torch.bfloat16,
|
||||
"computation_device": "cuda",
|
||||
}
|
||||
return {
|
||||
"offload_dtype": "disk",
|
||||
"offload_device": "disk",
|
||||
@@ -578,6 +716,23 @@ class QwenZImagePipeline:
|
||||
"computation_device": "cuda",
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def _face_stage_dtype(cls) -> Any:
|
||||
"""The dtype every face-stage model loads and computes in.
|
||||
|
||||
Deliberately independent of ``self.torch_dtype``, which belongs to the global
|
||||
stage. ``sdxl-zimage`` runs its global model in fp16, and inheriting that here
|
||||
built the Z-Image modules bf16 (per the VRAM config below) while handing them
|
||||
fp16 latents -- a Half/BFloat16 conv mismatch that crashed every face image
|
||||
while zero-face inputs passed, so no unit test could see it.
|
||||
|
||||
Read by Z-Image and by SAM alike, so the whole stage moves together. SAM never
|
||||
crashed, because it casts its own inputs and leaves through ``.float()``, but it
|
||||
was reading the same wrong field and would re-land the bug for the next profile
|
||||
that changes its global dtype.
|
||||
"""
|
||||
return cls._zimage_vram_config()["computation_dtype"]
|
||||
|
||||
@staticmethod
|
||||
def _zimage_vram_config() -> dict[str, Any]:
|
||||
import torch
|
||||
@@ -610,17 +765,18 @@ class QwenZImagePipeline:
|
||||
|
||||
self._progress("Loading Qwen-Image-2512, Lightning LoRA, and Canny ControlNet...")
|
||||
config = self._qwen_vram_config()
|
||||
text_encoder_config = ModelConfig(
|
||||
model_id=QWEN_IMAGE_2512_MODEL_ID,
|
||||
origin_file_pattern="text_encoder/model*.safetensors",
|
||||
**config,
|
||||
)
|
||||
model_configs = [
|
||||
ModelConfig(
|
||||
model_id=QWEN_IMAGE_2512_MODEL_ID,
|
||||
origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors",
|
||||
**config,
|
||||
),
|
||||
ModelConfig(
|
||||
model_id=QWEN_IMAGE_2512_MODEL_ID,
|
||||
origin_file_pattern="text_encoder/model*.safetensors",
|
||||
**config,
|
||||
),
|
||||
text_encoder_config,
|
||||
ModelConfig(
|
||||
model_id=QWEN_IMAGE_2512_MODEL_ID,
|
||||
origin_file_pattern="vae/diffusion_pytorch_model.safetensors",
|
||||
@@ -632,6 +788,10 @@ class QwenZImagePipeline:
|
||||
**config,
|
||||
),
|
||||
]
|
||||
prompt_cached = self._prompt_is_cached(QWEN_IMAGE_2512_MODEL_ID, _QWEN_PROMPT_OUTPUTS, _GLOBAL_PROMPT)
|
||||
if prompt_cached:
|
||||
model_configs.remove(text_encoder_config)
|
||||
log.info("Qwen prompt embedding is cached; loading the stack without its text encoder")
|
||||
pipe = QwenImagePipeline.from_pretrained(
|
||||
torch_dtype=self.torch_dtype,
|
||||
device=self.device,
|
||||
@@ -656,7 +816,9 @@ class QwenZImagePipeline:
|
||||
if self.cache_prompt_embeddings:
|
||||
_cache_static_prompt_embeddings(
|
||||
pipe,
|
||||
("prompt_emb", "prompt_emb_mask"),
|
||||
_QWEN_PROMPT_OUTPUTS,
|
||||
model_id=QWEN_IMAGE_2512_MODEL_ID,
|
||||
require_cache=prompt_cached,
|
||||
)
|
||||
self._qwen_pipe = (pipe, ControlNetInput)
|
||||
return self._qwen_pipe
|
||||
@@ -679,26 +841,32 @@ class QwenZImagePipeline:
|
||||
self._progress("Loading Z-Image Turbo face-detail model...")
|
||||
keep_on_device = self._keep_face_models_resident()
|
||||
config = self._zimage_vram_config()
|
||||
text_encoder_config = ModelConfig(
|
||||
model_id=ZIMAGE_TURBO_MODEL_ID,
|
||||
origin_file_pattern="text_encoder/*.safetensors",
|
||||
**config,
|
||||
)
|
||||
model_configs = [
|
||||
ModelConfig(
|
||||
model_id=ZIMAGE_TURBO_MODEL_ID,
|
||||
origin_file_pattern="transformer/*.safetensors",
|
||||
**config,
|
||||
),
|
||||
text_encoder_config,
|
||||
ModelConfig(
|
||||
model_id=ZIMAGE_TURBO_MODEL_ID,
|
||||
origin_file_pattern="vae/diffusion_pytorch_model.safetensors",
|
||||
**config,
|
||||
),
|
||||
]
|
||||
prompt_cached = self._prompt_is_cached(ZIMAGE_TURBO_MODEL_ID, _ZIMAGE_PROMPT_OUTPUTS, _FACE_PROMPT)
|
||||
if prompt_cached:
|
||||
model_configs.remove(text_encoder_config)
|
||||
log.info("Z-Image prompt embedding is cached; loading the stack without its text encoder")
|
||||
pipe = ZImagePipeline.from_pretrained(
|
||||
torch_dtype=self.torch_dtype,
|
||||
torch_dtype=self._face_stage_dtype(),
|
||||
device=self.device,
|
||||
model_configs=[
|
||||
ModelConfig(
|
||||
model_id=ZIMAGE_TURBO_MODEL_ID,
|
||||
origin_file_pattern="transformer/*.safetensors",
|
||||
**config,
|
||||
),
|
||||
ModelConfig(
|
||||
model_id=ZIMAGE_TURBO_MODEL_ID,
|
||||
origin_file_pattern="text_encoder/*.safetensors",
|
||||
**config,
|
||||
),
|
||||
ModelConfig(
|
||||
model_id=ZIMAGE_TURBO_MODEL_ID,
|
||||
origin_file_pattern="vae/diffusion_pytorch_model.safetensors",
|
||||
**config,
|
||||
),
|
||||
],
|
||||
model_configs=model_configs,
|
||||
tokenizer_config=ModelConfig(
|
||||
model_id=ZIMAGE_TURBO_MODEL_ID,
|
||||
origin_file_pattern="tokenizer/",
|
||||
@@ -708,7 +876,12 @@ class QwenZImagePipeline:
|
||||
if keep_on_device:
|
||||
_pin_vram_managed_models(pipe)
|
||||
if self.cache_prompt_embeddings:
|
||||
_cache_static_prompt_embeddings(pipe, ("prompt_embeds",))
|
||||
_cache_static_prompt_embeddings(
|
||||
pipe,
|
||||
_ZIMAGE_PROMPT_OUTPUTS,
|
||||
model_id=ZIMAGE_TURBO_MODEL_ID,
|
||||
require_cache=prompt_cached,
|
||||
)
|
||||
self._zimage_pipe = pipe
|
||||
return pipe
|
||||
|
||||
@@ -726,7 +899,7 @@ class QwenZImagePipeline:
|
||||
processor = AutoProcessor.from_pretrained(SAM_MODEL_ID, **kwargs)
|
||||
model = AutoModelForMaskGeneration.from_pretrained(
|
||||
SAM_MODEL_ID,
|
||||
torch_dtype=self.torch_dtype,
|
||||
torch_dtype=self._face_stage_dtype(),
|
||||
**kwargs,
|
||||
).to(self.device)
|
||||
model.eval()
|
||||
@@ -755,7 +928,7 @@ class QwenZImagePipeline:
|
||||
)
|
||||
original_sizes = inputs["original_sizes"].clone()
|
||||
reshaped_sizes = inputs["reshaped_input_sizes"].clone()
|
||||
inputs = _prepare_sam_inputs(inputs, self.device, self.torch_dtype)
|
||||
inputs = _prepare_sam_inputs(inputs, self.device, self._face_stage_dtype())
|
||||
with torch.inference_mode():
|
||||
outputs = model(**inputs, multimask_output=True)
|
||||
processed = processor.post_process_masks(
|
||||
@@ -802,6 +975,30 @@ class QwenZImagePipeline:
|
||||
result = result.resize(image.size, Image.Resampling.LANCZOS)
|
||||
return result.convert("RGB")
|
||||
|
||||
def _qwen_vae_roundtrip(self, image: Image.Image) -> Image.Image:
|
||||
"""Reconstruct source pixels through the already loaded Qwen VAE."""
|
||||
import torch
|
||||
|
||||
pipe, _controlnet_input_cls = self._load_qwen()
|
||||
source_width, source_height = image.size
|
||||
pad_width = (-source_width) % 8
|
||||
pad_height = (-source_height) % 8
|
||||
padded = image.convert("RGB")
|
||||
if pad_width or pad_height:
|
||||
padded = Image.fromarray(
|
||||
np.pad(
|
||||
np.asarray(padded),
|
||||
((0, pad_height), (0, pad_width), (0, 0)),
|
||||
mode="edge",
|
||||
)
|
||||
)
|
||||
pipe.load_models_to_device(["vae"])
|
||||
tensor = pipe.preprocess_image(padded).to(device=self.device, dtype=self.torch_dtype)
|
||||
with torch.inference_mode():
|
||||
latents = pipe.vae.encode(tensor)
|
||||
decoded = pipe.vae.decode(latents)
|
||||
return pipe.vae_output_to_image(decoded).crop((0, 0, source_width, source_height)).convert("RGB")
|
||||
|
||||
@staticmethod
|
||||
def _detail_size(
|
||||
crop_size: tuple[int, int],
|
||||
@@ -813,7 +1010,7 @@ class QwenZImagePipeline:
|
||||
scale_for_face = 768.0 / max(1, max(face_width, face_height))
|
||||
scale_for_crop = 1024.0 / max(1, max(crop_width, crop_height))
|
||||
scale = min(scale_for_face, scale_for_crop)
|
||||
# Never shrink below the crop's current size unless the 1024 cap requires it.
|
||||
# Never shrink below the crop's current size unless the cap requires it.
|
||||
if max(crop_width, crop_height) <= 1024:
|
||||
scale = max(1.0, scale)
|
||||
width = max(16, round(crop_width * scale / 16.0) * 16)
|
||||
@@ -873,15 +1070,38 @@ class QwenZImagePipeline:
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
text_manifest: VerifiedTextManifest | None = None,
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Image.Image:
|
||||
"""Execute global regeneration and masked face repair."""
|
||||
self._require_cuda()
|
||||
seed = resolve_seed(seed, "qwen-zimage")
|
||||
seed = resolve_seed(seed)
|
||||
donor = None
|
||||
if text_manifest is not None:
|
||||
self._progress("Reconstructing the verified text donor with the Qwen VAE...")
|
||||
if tile and max(image.size) > tile_size:
|
||||
from remove_ai_watermarks._internal.tiling import run_tiled
|
||||
|
||||
donor = run_tiled(
|
||||
self._qwen_vae_roundtrip,
|
||||
image,
|
||||
tile_size,
|
||||
tile_overlap,
|
||||
lambda message: self._progress(
|
||||
message.replace(
|
||||
"Tiled diffusion",
|
||||
"Reconstructing the verified text donor",
|
||||
1,
|
||||
)
|
||||
),
|
||||
)
|
||||
else:
|
||||
donor = self._qwen_vae_roundtrip(image)
|
||||
global_strength = (
|
||||
resolution_adaptive_denoise(image.width, image.height) if strength is None else float(strength)
|
||||
)
|
||||
if tile and max(image.size) > tile_size:
|
||||
from remove_ai_watermarks.noai.tiling import run_tiled
|
||||
from remove_ai_watermarks._internal.tiling import run_tiled
|
||||
|
||||
global_result = run_tiled(
|
||||
lambda tile_image: self._run_global(tile_image, global_strength, seed),
|
||||
@@ -897,14 +1117,37 @@ class QwenZImagePipeline:
|
||||
boxes = detect_faces(image)
|
||||
if not boxes:
|
||||
self._progress("No faces detected; keeping the Qwen global result.")
|
||||
return global_result
|
||||
masks = self._sam_masks(image, boxes)
|
||||
face_strength = largest_face_denoise(boxes, image.size) * FACE_DENOISE_SCALE
|
||||
return self._run_faces(
|
||||
image,
|
||||
global_result,
|
||||
boxes,
|
||||
masks,
|
||||
strength=face_strength,
|
||||
seed=seed,
|
||||
result = global_result
|
||||
else:
|
||||
masks = self._sam_masks(image, boxes)
|
||||
face_strength = largest_face_denoise(boxes, image.size) * FACE_DENOISE_SCALE
|
||||
result = self._run_faces(
|
||||
image,
|
||||
global_result,
|
||||
boxes,
|
||||
masks,
|
||||
strength=face_strength,
|
||||
seed=seed,
|
||||
)
|
||||
if text_manifest is None:
|
||||
return result
|
||||
if donor is None:
|
||||
raise RuntimeError("Verified text restoration requires a Qwen-VAE donor")
|
||||
from remove_ai_watermarks._internal.text_restoration import (
|
||||
blend_fidelity_anchor,
|
||||
restore_verified_text,
|
||||
)
|
||||
|
||||
# The anchor is OFF by default since 0.27.1: blending 15% of the Qwen-VAE
|
||||
# donor ACROSS THE WHOLE FRAME returned detector-visible OpenAI SynthID on
|
||||
# poster-scale manifests (official Content Provenance API, 2026-08-19:
|
||||
# detected x6 with the anchor, clean x6 without it, base clean throughout;
|
||||
# see docs/text-protection-research.md). ``fidelity_anchor=True`` keeps the
|
||||
# 0.27.0 research behavior for reproduction.
|
||||
if fidelity_anchor:
|
||||
self._progress("Blending the Qwen-VAE fidelity anchor...")
|
||||
anchor = blend_fidelity_anchor(result, donor)
|
||||
else:
|
||||
anchor = result
|
||||
self._progress(f"Restoring {len(text_manifest.lines)} verified text lines...")
|
||||
return restore_verified_text(image, anchor, donor, text_manifest.lines)
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Bounded AVI metadata reader for native TC260 AIGC labels.
|
||||
|
||||
TC260-PG-20257A stores the label in an AVI ``LIST/INFO`` chunk whose child
|
||||
chunk ID is ``AIGC`` and whose value is the normative JSON object. The walker
|
||||
seeks over media chunks and reads only bounded metadata values.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, BinaryIO
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks.metadata import MAX_TC260_VALUE_BYTES, parse_tc260_aigc_json
|
||||
|
||||
|
||||
def _info_payloads(
|
||||
stream: BinaryIO,
|
||||
start: int,
|
||||
end: int,
|
||||
) -> tuple[bytes, ...]:
|
||||
found: list[bytes] = []
|
||||
position = start
|
||||
while position + 8 <= end:
|
||||
stream.seek(position)
|
||||
chunk_id = stream.read(4)
|
||||
size_raw = stream.read(4)
|
||||
if len(chunk_id) != 4 or len(size_raw) != 4:
|
||||
break
|
||||
size = int.from_bytes(size_raw, "little")
|
||||
payload_start = position + 8
|
||||
payload_end = payload_start + size
|
||||
if payload_end > end:
|
||||
break
|
||||
if chunk_id == b"AIGC" and size <= MAX_TC260_VALUE_BYTES:
|
||||
value = stream.read(size)
|
||||
if len(value) == size and parse_tc260_aigc_json(value) is not None:
|
||||
found.append(value.rstrip(b"\x00 "))
|
||||
position = payload_end + (size & 1)
|
||||
return tuple(found)
|
||||
|
||||
|
||||
def tc260_aigc_payloads(path: str | Path) -> tuple[bytes, ...]:
|
||||
"""Read validated TC260 values from an AVI ``LIST/INFO/AIGC`` chunk."""
|
||||
found: list[bytes] = []
|
||||
try:
|
||||
with open(path, "rb") as stream:
|
||||
header = stream.read(12)
|
||||
if len(header) != 12 or header[:4] != b"RIFF" or header[8:12] != b"AVI ":
|
||||
return ()
|
||||
stream.seek(0, 2)
|
||||
file_size = stream.tell()
|
||||
declared_end = min(8 + int.from_bytes(header[4:8], "little"), file_size)
|
||||
position = 12
|
||||
while position + 8 <= declared_end:
|
||||
stream.seek(position)
|
||||
chunk_id = stream.read(4)
|
||||
size_raw = stream.read(4)
|
||||
if len(chunk_id) != 4 or len(size_raw) != 4:
|
||||
break
|
||||
size = int.from_bytes(size_raw, "little")
|
||||
payload_start = position + 8
|
||||
payload_end = payload_start + size
|
||||
if payload_end > declared_end:
|
||||
break
|
||||
if chunk_id == b"LIST" and size >= 4:
|
||||
list_type = stream.read(4)
|
||||
if list_type == b"INFO":
|
||||
found.extend(_info_payloads(stream, payload_start + 4, payload_end))
|
||||
position = payload_end + (size & 1)
|
||||
except OSError:
|
||||
return ()
|
||||
return tuple(found)
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Shared validation for versioned JSON transport contracts."""
|
||||
|
||||
from collections.abc import Collection
|
||||
|
||||
|
||||
def require_schema_version(
|
||||
value: object,
|
||||
*,
|
||||
contract: str,
|
||||
supported: Collection[int],
|
||||
) -> int:
|
||||
"""Return an explicitly supported integer schema version or raise."""
|
||||
if type(value) is not int or value not in supported:
|
||||
versions = ", ".join(str(version) for version in sorted(supported))
|
||||
raise ValueError(f"Unsupported {contract} schema: {value!r}; supported versions: {versions}")
|
||||
return value
|
||||
@@ -0,0 +1,154 @@
|
||||
"""The qwen-zimage recipe with an SDXL global stage.
|
||||
|
||||
Only the global regeneration model changes. The face stage is inherited verbatim
|
||||
from :class:`QwenZImagePipeline` -- same YuNet detection, same SAM masks, same
|
||||
Z-Image Turbo repair of the original crops, same feathered compositing -- so a
|
||||
change there cannot silently diverge between the two profiles.
|
||||
|
||||
Three pieces cannot be shared, because they are bound to the architecture: the
|
||||
ControlNet, the four-step distillation LoRA, and the sampler. Strength is bound to
|
||||
it too, which is the part that is easy to miss: an SDXL global pass leaves SynthID
|
||||
at the strength Qwen needs. See ``watermark_profiles.SDXL_ZIMAGE_OPENAI_STRENGTH``.
|
||||
"""
|
||||
|
||||
# Diffusers and torch expose mostly untyped tensor APIs. Keep the relaxation local
|
||||
# to this optional ML boundary.
|
||||
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportAttributeAccessIssue=false, reportPrivateUsage=false, reportPrivateImportUsage=false
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import (
|
||||
GLOBAL_STEPS,
|
||||
QwenZImagePipeline,
|
||||
build_canny_control_image,
|
||||
)
|
||||
from remove_ai_watermarks._internal.watermark_profiles import (
|
||||
CONTROLNET_CANNY_MODEL,
|
||||
SDXL_LIGHTNING_MODEL_ID,
|
||||
SDXL_LIGHTNING_PATTERN,
|
||||
SDXL_MODEL_ID,
|
||||
)
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SDXL_VAE_MODEL_ID = "madebyollin/sdxl-vae-fp16-fix"
|
||||
# SDXL aligns to an 8-pixel latent grid, against Qwen's 16.
|
||||
_LATENT_GRID = 8
|
||||
|
||||
|
||||
def sdxl_target_size(width: int, height: int) -> tuple[int, int]:
|
||||
"""Floor dimensions to SDXL's latent grid without changing aspect."""
|
||||
return max(_LATENT_GRID, (width // _LATENT_GRID) * _LATENT_GRID), max(
|
||||
_LATENT_GRID, (height // _LATENT_GRID) * _LATENT_GRID
|
||||
)
|
||||
|
||||
|
||||
def requested_steps(effective_steps: int, strength: float) -> int:
|
||||
"""Translate "spend N denoising steps" into what Diffusers has to be asked for.
|
||||
|
||||
The two runtimes truncate differently and it is easy to port this wrong.
|
||||
DiffSynth sets ``sigma_start = denoising_strength`` and then runs *every*
|
||||
requested step across the shortened sigma range. Diffusers img2img instead
|
||||
truncates the step *count* (``init_timestep = int(steps * strength)``), so
|
||||
asking it for four steps at strength 0.15 runs **zero** and returns nothing but
|
||||
a VAE round-trip. Ask for enough that ``effective_steps`` actually execute.
|
||||
"""
|
||||
return max(1, math.ceil(effective_steps / max(float(strength), 1e-6)))
|
||||
|
||||
|
||||
@dataclass
|
||||
class SdxlZImagePipeline(QwenZImagePipeline):
|
||||
"""Lazy runtime for the SDXL global stage plus the inherited face stage."""
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
super().__post_init__()
|
||||
self._sdxl_pipe: Any = None
|
||||
|
||||
def _load_sdxl(self) -> Any:
|
||||
if self._sdxl_pipe is not None:
|
||||
return self._sdxl_pipe
|
||||
self._require_cuda()
|
||||
import torch
|
||||
from diffusers import (
|
||||
AutoencoderKL,
|
||||
ControlNetModel,
|
||||
EulerDiscreteScheduler,
|
||||
StableDiffusionXLControlNetImg2ImgPipeline,
|
||||
)
|
||||
from huggingface_hub import hf_hub_download
|
||||
|
||||
self._progress("Loading SDXL, Lightning LoRA, and Canny ControlNet...")
|
||||
token = {"token": self.hf_token} if self.hf_token else {}
|
||||
controlnet = ControlNetModel.from_pretrained(CONTROLNET_CANNY_MODEL, torch_dtype=torch.float16, **token)
|
||||
vae = AutoencoderKL.from_pretrained(SDXL_VAE_MODEL_ID, torch_dtype=torch.float16, **token)
|
||||
pipe = StableDiffusionXLControlNetImg2ImgPipeline.from_pretrained(
|
||||
SDXL_MODEL_ID,
|
||||
controlnet=controlnet,
|
||||
vae=vae,
|
||||
torch_dtype=torch.float16,
|
||||
variant="fp16",
|
||||
add_watermarker=False,
|
||||
**token,
|
||||
).to(self.device)
|
||||
# SDXL's own four-step distillation, at the strength its authors document.
|
||||
# The reference graph loads the Qwen LoRA at 0.8; carrying that number to a
|
||||
# different LoRA on a different architecture would be imitation, not parity.
|
||||
pipe.load_lora_weights(hf_hub_download(SDXL_LIGHTNING_MODEL_ID, SDXL_LIGHTNING_PATTERN, **token))
|
||||
pipe.fuse_lora()
|
||||
# SDXL-Lightning is distilled against trailing timestep spacing.
|
||||
pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
|
||||
self._sdxl_pipe = pipe
|
||||
return pipe
|
||||
|
||||
def preload(self, *, global_only: bool = False) -> None:
|
||||
"""Eagerly load the mandatory stage and, by default, the face stack."""
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import _yunet_model_path
|
||||
|
||||
self._load_sdxl()
|
||||
_yunet_model_path()
|
||||
if not global_only:
|
||||
self._load_zimage()
|
||||
self._load_sam()
|
||||
|
||||
def _run_global(self, image: Image.Image, strength: float, seed: int | None) -> Image.Image:
|
||||
import torch
|
||||
|
||||
pipe = self._load_sdxl()
|
||||
target = sdxl_target_size(image.width, image.height)
|
||||
prepared = image if image.size == target else image.resize(target, Image.Resampling.LANCZOS)
|
||||
control = build_canny_control_image(prepared)
|
||||
steps = requested_steps(GLOBAL_STEPS, strength)
|
||||
self._progress(f"Running SDXL Canny pass: strength={strength:.4f}, steps={GLOBAL_STEPS} of {steps}...")
|
||||
generator = torch.Generator(device=self.device).manual_seed(seed) if seed is not None else None
|
||||
result = pipe(
|
||||
prompt=self._global_prompt(),
|
||||
negative_prompt=self._global_negative(),
|
||||
image=prepared,
|
||||
control_image=control,
|
||||
controlnet_conditioning_scale=float(self.controlnet_conditioning_scale),
|
||||
strength=float(strength),
|
||||
num_inference_steps=steps,
|
||||
guidance_scale=1.0,
|
||||
generator=generator,
|
||||
).images[0]
|
||||
if result.size != image.size:
|
||||
result = result.resize(image.size, Image.Resampling.LANCZOS)
|
||||
return result.convert("RGB")
|
||||
|
||||
@staticmethod
|
||||
def _global_prompt() -> str:
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import _GLOBAL_PROMPT
|
||||
|
||||
return _GLOBAL_PROMPT
|
||||
|
||||
@staticmethod
|
||||
def _global_negative() -> str:
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import _GLOBAL_NEGATIVE
|
||||
|
||||
return _GLOBAL_NEGATIVE
|
||||
@@ -0,0 +1,446 @@
|
||||
"""Opt-in restoration of verified text from a Qwen VAE reconstruction."""
|
||||
|
||||
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks._internal.schema import require_schema_version
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
TEXT_MANIFEST_SCHEMA = 2
|
||||
_SUPPORTED_TEXT_MANIFEST_SCHEMAS = frozenset({1, TEXT_MANIFEST_SCHEMA})
|
||||
FIDELITY_BLEND_ALPHA = 0.15
|
||||
GLYPH_FEATHER = 0.5
|
||||
GLYPH_SIDE_PAD_RATIO = 0.12
|
||||
GLYPH_SIDE_PAD_LIMIT = 1.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VerifiedTextLine:
|
||||
"""One operator-verified source region in source-pixel coordinates."""
|
||||
|
||||
box: tuple[int, int, int, int]
|
||||
text: str | None = None
|
||||
script: str | None = None
|
||||
angle: float = 0.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VerifiedTextManifest:
|
||||
"""Verified text regions cryptographically bound to one decoded RGB source."""
|
||||
|
||||
source_pixel_sha256: str
|
||||
width: int
|
||||
height: int
|
||||
lines: tuple[VerifiedTextLine, ...]
|
||||
|
||||
|
||||
def source_pixel_sha256(image: Image.Image) -> str:
|
||||
"""Hash decoded RGB geometry and bytes, independent of container metadata."""
|
||||
rgb = image.convert("RGB")
|
||||
digest = hashlib.sha256()
|
||||
digest.update(rgb.width.to_bytes(8, "big"))
|
||||
digest.update(rgb.height.to_bytes(8, "big"))
|
||||
digest.update(rgb.tobytes())
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def load_verified_text_manifest(path: Path, source: Image.Image) -> VerifiedTextManifest:
|
||||
"""Load and validate an operator-verified manifest for exactly ``source``."""
|
||||
try:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError) as exc:
|
||||
raise ValueError(f"Cannot read text manifest {path}: {exc}") from exc
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("Text manifest must be a JSON object")
|
||||
schema_version = require_schema_version(
|
||||
payload.get("schema_version"),
|
||||
contract="text manifest",
|
||||
supported=_SUPPORTED_TEXT_MANIFEST_SCHEMAS,
|
||||
)
|
||||
if payload.get("verified") is not True:
|
||||
raise ValueError("Text manifest must contain verified=true after operator verification")
|
||||
|
||||
rgb = source.convert("RGB")
|
||||
width = _manifest_integer(payload, "width")
|
||||
height = _manifest_integer(payload, "height")
|
||||
if (width, height) != rgb.size:
|
||||
raise ValueError(f"Text manifest dimensions {width}x{height} do not match source {rgb.width}x{rgb.height}")
|
||||
expected_hash = payload.get("source_pixel_sha256")
|
||||
if not isinstance(expected_hash, str) or len(expected_hash) != 64:
|
||||
raise ValueError("Text manifest source_pixel_sha256 must be a 64-character SHA-256")
|
||||
actual_hash = source_pixel_sha256(rgb)
|
||||
if expected_hash.casefold() != actual_hash:
|
||||
raise ValueError("Text manifest source_pixel_sha256 does not match the decoded source pixels")
|
||||
|
||||
raw_lines = payload.get("lines")
|
||||
if not isinstance(raw_lines, list) or not raw_lines:
|
||||
raise ValueError("Text manifest lines must be a non-empty list")
|
||||
lines = tuple(_load_line(item, width, height, index, schema_version) for index, item in enumerate(raw_lines))
|
||||
if list(lines) != sorted(lines, key=lambda line: (line.box[1], line.box[0])):
|
||||
raise ValueError("Text manifest lines must be in top-to-bottom, left-to-right reading order")
|
||||
return VerifiedTextManifest(actual_hash, width, height, lines)
|
||||
|
||||
|
||||
def _manifest_integer(payload: dict[str, Any], key: str) -> int:
|
||||
value = payload.get(key)
|
||||
if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
|
||||
raise ValueError(f"Text manifest {key} must be a positive integer")
|
||||
return value
|
||||
|
||||
|
||||
def _load_line(item: Any, width: int, height: int, index: int, schema_version: int) -> VerifiedTextLine:
|
||||
if not isinstance(item, dict):
|
||||
raise ValueError(f"Text manifest line {index} must be an object")
|
||||
raw_box = item.get("box")
|
||||
if (
|
||||
not isinstance(raw_box, list)
|
||||
or len(raw_box) != 4
|
||||
or any(isinstance(value, bool) or not isinstance(value, int) for value in raw_box)
|
||||
):
|
||||
raise ValueError(f"Text manifest line {index} box must contain four integers")
|
||||
box = tuple(raw_box)
|
||||
x1, y1, x2, y2 = box
|
||||
if not (0 <= x1 < x2 <= width and 0 <= y1 < y2 <= height):
|
||||
raise ValueError(f"Text manifest line {index} box is outside the source dimensions")
|
||||
text: str | None = None
|
||||
script: str | None = None
|
||||
if schema_version == 1:
|
||||
text = item.get("text")
|
||||
script = item.get("script")
|
||||
if not isinstance(text, str) or not text.strip():
|
||||
raise ValueError(f"Text manifest line {index} text must be non-empty")
|
||||
if not isinstance(script, str) or not script.strip():
|
||||
raise ValueError(f"Text manifest line {index} script must be non-empty")
|
||||
angle_value = item.get("angle", 0.0)
|
||||
if isinstance(angle_value, bool) or not isinstance(angle_value, int | float):
|
||||
raise ValueError(f"Text manifest line {index} angle must be numeric")
|
||||
angle = float(angle_value)
|
||||
if not math.isfinite(angle) or abs(angle) > 30.0:
|
||||
raise ValueError(f"Text manifest line {index} angle must be between -30 and 30 degrees")
|
||||
return VerifiedTextLine(box, text, script, angle)
|
||||
|
||||
|
||||
def blend_fidelity_anchor(clean: Image.Image, donor: Image.Image) -> Image.Image:
|
||||
"""Blend 15% Qwen-VAE reconstruction into the oracle-clean pipeline output."""
|
||||
clean_rgb = np.asarray(clean.convert("RGB"), dtype=np.float32)
|
||||
donor_rgb = np.asarray(donor.convert("RGB"), dtype=np.float32)
|
||||
if clean_rgb.shape != donor_rgb.shape:
|
||||
raise ValueError("Clean result and Qwen-VAE donor dimensions must match")
|
||||
blended = np.rint(clean_rgb * (1.0 - FIDELITY_BLEND_ALPHA) + donor_rgb * FIDELITY_BLEND_ALPHA)
|
||||
return Image.fromarray(np.clip(blended, 0, 255).astype(np.uint8))
|
||||
|
||||
|
||||
def restore_verified_text(
|
||||
source: Image.Image,
|
||||
candidate: Image.Image,
|
||||
donor: Image.Image,
|
||||
lines: tuple[VerifiedTextLine, ...],
|
||||
) -> Image.Image:
|
||||
"""Erase candidate glyphs, then composite verified Qwen-VAE glyph cores."""
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
if not region_eraser.lama_available():
|
||||
raise RuntimeError(
|
||||
"Verified text restoration requires LaMa. Install: pip install 'remove-ai-watermarks[text-restoration]'"
|
||||
)
|
||||
source_rgb = np.asarray(source.convert("RGB"))
|
||||
candidate_rgb = np.asarray(candidate.convert("RGB"))
|
||||
donor_rgb = np.asarray(donor.convert("RGB"))
|
||||
if source_rgb.shape != candidate_rgb.shape or source_rgb.shape != donor_rgb.shape:
|
||||
raise ValueError("Source, candidate, and Qwen-VAE donor dimensions must match")
|
||||
|
||||
source_masks = [source_silhouette_mask(source_rgb, line.box, line.angle) for line in lines]
|
||||
for index, mask in enumerate(source_masks):
|
||||
if not np.any(mask):
|
||||
raise ValueError(f"Verified text line {index} produced no source glyph pixels")
|
||||
candidate_masks = [source_silhouette_mask(candidate_rgb, line.box, line.angle) for line in lines]
|
||||
erase_masks = []
|
||||
for line, source_mask, candidate_mask in zip(lines, source_masks, candidate_masks, strict=True):
|
||||
radius = 5 if line.box[3] - line.box[1] >= 48 else 3
|
||||
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * radius + 1,) * 2)
|
||||
erase_masks.append(cv2.dilate(np.maximum(source_mask, candidate_mask), kernel))
|
||||
del candidate_masks
|
||||
|
||||
groups = group_text_lines(lines)
|
||||
background = cv2.cvtColor(candidate_rgb, cv2.COLOR_RGB2BGR)
|
||||
for group in groups:
|
||||
background = region_eraser.erase_lama(background, np.maximum.reduce([erase_masks[index] for index in group]))
|
||||
background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
|
||||
residual_masks = [
|
||||
residual_glyph_mask(background_rgb, mask, line.box) for line, mask in zip(lines, erase_masks, strict=True)
|
||||
]
|
||||
for group in groups:
|
||||
residual = np.maximum.reduce([residual_masks[index] for index in group])
|
||||
if np.any(residual):
|
||||
background = region_eraser.erase_lama(background, residual)
|
||||
del erase_masks, residual_masks
|
||||
restored = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
|
||||
restored = composite_fresh_text_edges(source_rgb, restored, lines, source_masks)
|
||||
source_glyph_mask = np.maximum.reduce(source_masks)
|
||||
restored = composite_reconstructed_glyphs(donor_rgb, restored, source_glyph_mask)
|
||||
return Image.fromarray(restored)
|
||||
|
||||
|
||||
def _glyph_crop_box(
|
||||
box: tuple[int, int, int, int],
|
||||
width: int,
|
||||
height: int,
|
||||
left_pad: int,
|
||||
right_pad: int,
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Widen the detector box so glyph edges stay inside the silhouette crop.
|
||||
|
||||
Detector boxes can stop inside a leading flourish or trailing punctuation,
|
||||
and Paddle line boxes sit 2-5 px above the true ink bottom on the poster
|
||||
fixtures. The crop is otherwise the box itself, so those pixels never reach
|
||||
the donor composite. Start 12% sideways, expand each side independently
|
||||
while foreground reaches its boundary, and pad 8% up and 25% down.
|
||||
"""
|
||||
x1, y1, x2, y2 = box
|
||||
line_h = max(1, y2 - y1)
|
||||
pad_top = max(1, round(line_h * 0.08))
|
||||
pad_bot = max(2, round(line_h * 0.25))
|
||||
return (
|
||||
max(0, x1 - left_pad),
|
||||
max(0, y1 - pad_top),
|
||||
min(width, x2 + right_pad),
|
||||
min(height, y2 + pad_bot),
|
||||
)
|
||||
|
||||
|
||||
def _silhouette_crop_mask(
|
||||
source_rgb: NDArray[Any],
|
||||
crop_box: tuple[int, int, int, int],
|
||||
angle: float,
|
||||
) -> NDArray[Any]:
|
||||
"""Threshold one candidate crop so its horizontal boundaries can be tested."""
|
||||
height, width = source_rgb.shape[:2]
|
||||
x1, y1, x2, y2 = crop_box
|
||||
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
|
||||
support = np.ones(gray.shape, dtype=np.uint8)
|
||||
if angle:
|
||||
box_width, box_height = x2 - x1, y2 - y1
|
||||
theta = math.radians(abs(angle))
|
||||
cosine, sine = math.cos(theta), math.sin(theta)
|
||||
denominator = cosine * cosine - sine * sine
|
||||
rect_width = (box_width * cosine - box_height * sine) / denominator
|
||||
rect_height = (box_height * cosine - box_width * sine) / denominator
|
||||
rotated = cv2.boxPoints(
|
||||
((box_width / 2, box_height / 2), (max(1.0, rect_width * 0.92), max(1.0, rect_height * 0.62)), -angle)
|
||||
)
|
||||
support.fill(0)
|
||||
cv2.fillConvexPoly(support, np.rint(rotated).astype(np.int32), 1)
|
||||
values = gray[support > 0]
|
||||
background_luma = float(np.median(values))
|
||||
else:
|
||||
ring_pad = max(6, min(20, (y2 - y1) // 4))
|
||||
rx1, ry1, rx2, ry2 = _clip_box(crop_box, width, height, pad=ring_pad)
|
||||
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
|
||||
ring = np.ones(context.shape, dtype=bool)
|
||||
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
|
||||
background_luma = float(np.median(context[ring])) if ring.any() else float(np.median(gray))
|
||||
values = gray.reshape(-1)
|
||||
low, high = float(np.percentile(values, 2)), float(np.percentile(values, 98))
|
||||
dark_contrast, light_contrast = background_luma - low, high - background_luma
|
||||
threshold = max(16.0, min(56.0, max(light_contrast, dark_contrast) * 0.22))
|
||||
if light_contrast > dark_contrast:
|
||||
crop_mask = (gray.astype(np.float32) >= background_luma + threshold).astype(np.uint8) * 255
|
||||
else:
|
||||
crop_mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
|
||||
crop_mask[support == 0] = 0
|
||||
return crop_mask
|
||||
|
||||
|
||||
def _anchored_components_reach_sides(
|
||||
crop_mask: NDArray[Any],
|
||||
anchor_box: tuple[int, int, int, int],
|
||||
) -> tuple[bool, bool]:
|
||||
"""Whether anchored foreground reaches the left and right crop sides."""
|
||||
count, labels, stats, _centroids = cv2.connectedComponentsWithStats(crop_mask, 8)
|
||||
x1, y1, x2, y2 = anchor_box
|
||||
anchor_counts = np.bincount(labels[y1:y2, x1:x2].reshape(-1), minlength=count)
|
||||
anchor_counts[0] = 0
|
||||
|
||||
def reaches(edge_labels: NDArray[Any]) -> bool:
|
||||
return any(
|
||||
label != 0 and anchor_counts[label] >= max(3, round(stats[label, cv2.CC_STAT_AREA] * 0.1))
|
||||
for label in np.unique(edge_labels)
|
||||
)
|
||||
|
||||
return reaches(labels[y1:y2, :2]), reaches(labels[y1:y2, -2:])
|
||||
|
||||
|
||||
def source_silhouette_mask(
|
||||
source_rgb: NDArray[Any],
|
||||
box: tuple[int, int, int, int],
|
||||
angle: float = 0.0,
|
||||
) -> NDArray[Any]:
|
||||
"""Recover a thresholded glyph shape without retaining source amplitudes."""
|
||||
height, width = source_rgb.shape[:2]
|
||||
box_x1, box_y1, box_x2, box_y2 = box
|
||||
line_h = max(1, box_y2 - box_y1)
|
||||
step = max(2, round(line_h * GLYPH_SIDE_PAD_RATIO))
|
||||
limit = max(step, round(line_h * GLYPH_SIDE_PAD_LIMIT))
|
||||
left_pad = right_pad = step
|
||||
while True:
|
||||
x1, y1, x2, y2 = _glyph_crop_box(box, width, height, left_pad, right_pad)
|
||||
crop_mask = _silhouette_crop_mask(source_rgb, (x1, y1, x2, y2), angle)
|
||||
core_y1 = max(0, box_y1 - y1)
|
||||
core_y2 = min(y2 - y1, box_y2 - y1)
|
||||
anchor_box = (max(0, box_x1 - x1), core_y1, min(x2 - x1, box_x2 - x1), core_y2)
|
||||
can_expand_left = x1 > 0 and left_pad < limit
|
||||
can_expand_right = x2 < width and right_pad < limit
|
||||
left_anchored, right_anchored = (
|
||||
_anchored_components_reach_sides(crop_mask, anchor_box)
|
||||
if can_expand_left or can_expand_right
|
||||
else (False, False)
|
||||
)
|
||||
left_touches = can_expand_left and left_anchored
|
||||
right_touches = can_expand_right and right_anchored
|
||||
if not left_touches and not right_touches:
|
||||
break
|
||||
if left_touches:
|
||||
left_pad = min(limit, left_pad + step)
|
||||
if right_touches:
|
||||
right_pad = min(limit, right_pad + step)
|
||||
result = np.zeros((height, width), dtype=np.uint8)
|
||||
result[y1:y2, x1:x2] = crop_mask
|
||||
return result
|
||||
|
||||
|
||||
def residual_glyph_mask(
|
||||
background_rgb: NDArray[Any],
|
||||
original_mask: NDArray[Any],
|
||||
box: tuple[int, int, int, int],
|
||||
) -> NDArray[Any]:
|
||||
"""Find glyph-like contrast left after the first inpaint pass."""
|
||||
residual = _foreground_mask(background_rgb, box)
|
||||
residual = cv2.bitwise_and(residual, original_mask)
|
||||
return cv2.dilate(residual, np.ones((5, 5), np.uint8), iterations=1)
|
||||
|
||||
|
||||
def composite_fresh_text_edges(
|
||||
source_rgb: NDArray[Any],
|
||||
background_rgb: NDArray[Any],
|
||||
lines: tuple[VerifiedTextLine, ...],
|
||||
masks: list[NDArray[Any]],
|
||||
) -> NDArray[Any]:
|
||||
"""Render fresh antialiased edges for source-derived glyph masks."""
|
||||
restored = background_rgb
|
||||
for line, mask in zip(lines, masks, strict=True):
|
||||
color = _sample_text_color(source_rgb, mask, line.box)
|
||||
restored = composite_fresh_silhouette(restored, mask, color)
|
||||
return restored
|
||||
|
||||
|
||||
def composite_reconstructed_glyphs(
|
||||
donor_rgb: NDArray[Any],
|
||||
background_rgb: NDArray[Any],
|
||||
glyph_mask: NDArray[Any],
|
||||
*,
|
||||
feather: float = GLYPH_FEATHER,
|
||||
) -> NDArray[Any]:
|
||||
"""Composite an exact reconstructed core with a narrow donor edge."""
|
||||
if donor_rgb.shape != background_rgb.shape or donor_rgb.shape[:2] != glyph_mask.shape:
|
||||
raise ValueError("donor, background, and glyph mask dimensions must match")
|
||||
blurred = cv2.GaussianBlur(glyph_mask, (0, 0), feather) if feather > 0 else glyph_mask
|
||||
alpha = np.maximum(glyph_mask, blurred).astype(np.float32) / 255.0
|
||||
combined = donor_rgb.astype(np.float32) * alpha[..., None] + background_rgb.astype(np.float32) * (
|
||||
1.0 - alpha[..., None]
|
||||
)
|
||||
return np.clip(np.rint(combined), 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def composite_fresh_silhouette(
|
||||
background_rgb: NDArray[Any],
|
||||
glyph_mask: NDArray[Any],
|
||||
color: tuple[int, int, int],
|
||||
*,
|
||||
feather: float = 0.35,
|
||||
) -> NDArray[Any]:
|
||||
"""Render a binary source shape with fresh color and antialiasing."""
|
||||
if background_rgb.shape[:2] != glyph_mask.shape:
|
||||
raise ValueError("background and glyph mask dimensions must match")
|
||||
antialiased = cv2.GaussianBlur(glyph_mask, (0, 0), feather) if feather > 0 else glyph_mask
|
||||
alpha = antialiased.astype(np.float32)[..., None] / 255.0
|
||||
foreground = np.empty_like(background_rgb)
|
||||
foreground[:, :] = color
|
||||
combined = foreground.astype(np.float32) * alpha + background_rgb.astype(np.float32) * (1.0 - alpha)
|
||||
return np.clip(combined, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def _clip_box(box: tuple[int, int, int, int], width: int, height: int, pad: int = 0) -> tuple[int, int, int, int]:
|
||||
x1, y1, x2, y2 = box
|
||||
return max(0, x1 - pad), max(0, y1 - pad), min(width, x2 + pad), min(height, y2 + pad)
|
||||
|
||||
|
||||
def _foreground_mask(source_rgb: NDArray[Any], box: tuple[int, int, int, int]) -> NDArray[Any]:
|
||||
height, width = source_rgb.shape[:2]
|
||||
line_height = box[3] - box[1]
|
||||
x1, y1, x2, y2 = _clip_box(box, width, height, pad=max(6, int(line_height * 0.12)))
|
||||
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
|
||||
ring_pad = max(8, min(24, (y2 - y1) // 5))
|
||||
rx1, ry1, rx2, ry2 = _clip_box((x1, y1, x2, y2), width, height, pad=ring_pad)
|
||||
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
|
||||
ring = np.ones(context.shape, dtype=bool)
|
||||
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
|
||||
background_luma = float(np.median(context[ring])) if ring.any() else float(np.median(gray))
|
||||
low, high = float(np.percentile(gray, 4)), float(np.percentile(gray, 96))
|
||||
dark_contrast, light_contrast = background_luma - low, high - background_luma
|
||||
threshold = max(24.0, min(72.0, max(light_contrast, dark_contrast) * 0.32))
|
||||
if light_contrast > dark_contrast:
|
||||
mask = (gray.astype(np.float32) >= background_luma + threshold).astype(np.uint8) * 255
|
||||
else:
|
||||
mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
|
||||
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((2, 2), np.uint8))
|
||||
dilation = 5 if line_height >= 48 else 3
|
||||
mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * dilation + 1,) * 2))
|
||||
result = np.zeros((height, width), dtype=np.uint8)
|
||||
result[y1:y2, x1:x2] = mask
|
||||
return result
|
||||
|
||||
|
||||
def _sample_text_color(
|
||||
source_rgb: NDArray[Any], mask: NDArray[Any], box: tuple[int, int, int, int]
|
||||
) -> tuple[int, int, int]:
|
||||
height, width = source_rgb.shape[:2]
|
||||
x1, y1, x2, y2 = _clip_box(box, width, height, pad=2)
|
||||
crop = source_rgb[y1:y2, x1:x2]
|
||||
pixels = crop[mask[y1:y2, x1:x2] > 0]
|
||||
luma = pixels.mean(axis=1)
|
||||
background_luma = float(crop[[0, -1], :, :].reshape(-1, 3).mean(axis=1).mean())
|
||||
selected = (
|
||||
pixels[luma <= np.percentile(luma, 20)] if background_luma >= 128 else pixels[luma >= np.percentile(luma, 80)]
|
||||
)
|
||||
return tuple(int(value) for value in np.median(selected, axis=0))
|
||||
|
||||
|
||||
def group_text_lines(lines: Sequence[VerifiedTextLine]) -> list[list[int]]:
|
||||
"""Group nearby same-script lines for a shared LaMa erase pass."""
|
||||
groups: list[list[int]] = []
|
||||
for index, line in enumerate(lines):
|
||||
if not groups:
|
||||
groups.append([index])
|
||||
continue
|
||||
previous = lines[groups[-1][-1]]
|
||||
gap = line.box[1] - previous.box[3]
|
||||
if line.script != previous.script or gap > max(60, int((previous.box[3] - previous.box[1]) * 1.1)):
|
||||
groups.append([index])
|
||||
else:
|
||||
groups[-1].append(index)
|
||||
return groups
|
||||
+6
-59
@@ -1,11 +1,11 @@
|
||||
"""Sliding-window tiled diffusion for large images.
|
||||
|
||||
The img2img / ControlNet pipeline denoises the WHOLE image in one forward pass,
|
||||
so it OOMs on MPS/GPU above ~2K (issue #10). Tiling splits the image into
|
||||
overlapping tiles -- each kept near SDXL's ~1024 training size -- regenerates
|
||||
each tile independently, and feather-blends the overlaps. The result retains the
|
||||
input's native dimensions without an explicit ``--max-resolution`` downscale, but
|
||||
it is not pixel-lossless because every tile is regenerated.
|
||||
The global stage denoises the WHOLE image in one forward pass, so it OOMs on a
|
||||
GPU above ~2K (issue #10). Tiling splits the image into overlapping tiles -- each
|
||||
kept near the ~1024 training size -- regenerates each tile independently, and
|
||||
feather-blends the overlaps. The result retains the input's native dimensions
|
||||
without an explicit ``--max-resolution`` downscale, but it is not pixel-lossless
|
||||
because every tile is regenerated.
|
||||
|
||||
The geometry (``plan_tiles``) and the blend weighting (``feather_weights``) are
|
||||
pure functions, unit-tested without the diffusion model. ``run_tiled`` is the
|
||||
@@ -100,59 +100,6 @@ def feather_weights(width: int, height: int, overlap: int) -> NDArray[Any]:
|
||||
return weights
|
||||
|
||||
|
||||
def feather_region_composite(
|
||||
base: NDArray[Any],
|
||||
regenerated: NDArray[Any],
|
||||
box: tuple[int, int, int, int],
|
||||
*,
|
||||
feather: int = 64,
|
||||
) -> NDArray[Any]:
|
||||
"""Composite ``regenerated`` over ``base`` inside ``box`` only, feathering the seam.
|
||||
|
||||
For AI-ENHANCED composites (digitalSourceType ``compositeWithTrainedAlgorithmicMedia``):
|
||||
the diffusion remover regenerates the whole frame, but only the AI-composited
|
||||
REGION should change -- the rest is a real photo that must be preserved. This
|
||||
blends the regenerated pixels in over ``box = (x, y, w, h)`` with a separable
|
||||
linear taper of ``feather`` px at the box edges, so the result equals ``base``
|
||||
EXACTLY outside the box and ramps smoothly (no hard seam) at the boundary.
|
||||
|
||||
Pure and model-free (unit-tested): ``base`` and ``regenerated`` must be the same
|
||||
shape (H x W, or H x W x C). The output preserves ``base``'s dtype. ``feather`` is
|
||||
clamped to half the box on each axis, so a small region still tapers symmetrically;
|
||||
``feather=0`` is a hard-edged paste.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
if base.shape != regenerated.shape:
|
||||
raise ValueError(f"shape mismatch: base {base.shape} vs regenerated {regenerated.shape}")
|
||||
h, w = base.shape[:2]
|
||||
x, y, bw, bh = box
|
||||
x0, y0 = max(0, x), max(0, y)
|
||||
x1, y1 = min(w, x + bw), min(h, y + bh)
|
||||
out = base.copy()
|
||||
if x1 <= x0 or y1 <= y0:
|
||||
return out # empty / off-image box -> nothing regenerated
|
||||
|
||||
def taper(n: int) -> NDArray[Any]:
|
||||
win = np.ones(n, dtype=np.float32)
|
||||
f = min(max(feather, 0), n // 2)
|
||||
if f > 0:
|
||||
ramp = (np.arange(f, dtype=np.float32) + 1.0) / (f + 1.0) # in (0, 1), 0 at the edge
|
||||
win[:f] = ramp
|
||||
win[n - f :] = ramp[::-1]
|
||||
return win
|
||||
|
||||
rh, rw = y1 - y0, x1 - x0
|
||||
wmap = np.outer(taper(rh), taper(rw)) # ~0 at the box edge, 1 in the interior
|
||||
if base.ndim == 3:
|
||||
wmap = wmap[:, :, None]
|
||||
roi_base = base[y0:y1, x0:x1].astype(np.float32)
|
||||
roi_gen = regenerated[y0:y1, x0:x1].astype(np.float32)
|
||||
blended = roi_base * (1.0 - wmap) + roi_gen * wmap
|
||||
out[y0:y1, x0:x1] = np.clip(blended, 0, 255).astype(base.dtype)
|
||||
return out
|
||||
|
||||
|
||||
def run_tiled(
|
||||
generate_tile: Callable[[PILImage.Image], PILImage.Image],
|
||||
image: PILImage.Image,
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Small path helpers shared by optional image pipelines."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from remove_ai_watermarks._internal.constants import SUPPORTED_FORMATS
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
_PIL_FORMAT_BY_SUFFIX = {
|
||||
".jpg": "JPEG",
|
||||
".jpeg": "JPEG",
|
||||
".png": "PNG",
|
||||
}
|
||||
|
||||
|
||||
def is_supported_format(file_path: Path) -> bool:
|
||||
"""Return whether ``file_path`` has a supported raster suffix."""
|
||||
return file_path.suffix.casefold() in SUPPORTED_FORMATS
|
||||
|
||||
|
||||
def get_image_format(file_path: Path) -> str:
|
||||
"""Return the Pillow save format used by the legacy metadata API.
|
||||
|
||||
The metadata writer only has specialized PNG and JPEG paths. Other accepted
|
||||
inputs therefore use its PNG fallback, matching the established API contract.
|
||||
"""
|
||||
return _PIL_FORMAT_BY_SUFFIX.get(file_path.suffix.casefold(), "PNG")
|
||||
@@ -0,0 +1,164 @@
|
||||
"""Project-owned configuration for invisible-watermark regeneration profiles.
|
||||
|
||||
Two profiles remain, and both are CUDA-only: ``qwen-zimage`` (the default) and
|
||||
``sdxl-zimage``. The older ``controlnet``, ``sdxl``, ``qwen`` and ``default`` profiles
|
||||
were removed rather than kept as a CPU path, because none of them matched the two-stage
|
||||
recipe's face preservation and keeping them implied a quality this library no longer
|
||||
offers. Removing invisible watermarks therefore needs a CUDA device; the visible-mark
|
||||
registry and every identify path still run anywhere.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
# SDXL base is no longer a profile of its own, but it is still the global stage of
|
||||
# sdxl-zimage, so the checkpoint id stays. Named for what it is rather than
|
||||
# ``DEFAULT_MODEL_ID``: there is no user-selectable model any more, so "default"
|
||||
# implied an override that both profiles reject.
|
||||
SDXL_MODEL_ID = "stabilityai/stable-diffusion-xl-base-1.0"
|
||||
CONTROLNET_CANNY_MODEL = "xinsir/controlnet-canny-sdxl-1.0"
|
||||
|
||||
QWEN_ZIMAGE_PROFILE = "qwen-zimage"
|
||||
SDXL_ZIMAGE_PROFILE = "sdxl-zimage"
|
||||
DEFAULT_PROFILE = QWEN_ZIMAGE_PROFILE
|
||||
PROFILE_CHOICES = (QWEN_ZIMAGE_PROFILE, SDXL_ZIMAGE_PROFILE)
|
||||
|
||||
# The modules a real removal run needs, and the extra that installs them. Both live
|
||||
# here, in the only profile module that imports nothing heavy, because the CLI's
|
||||
# availability gate and the remover's own precondition must agree: when they drifted,
|
||||
# the CLI passed on a torch+diffusers environment and the run then died at the
|
||||
# DiffSynth face stage, telling the user to install an extra that does not contain it.
|
||||
REMOVAL_MODULES = ("torch", "diffusers", "diffsynth")
|
||||
# Shell-quoted, because this string is printed as a command the user copy-pastes.
|
||||
# Bare brackets are a glob in zsh (the macOS default shell): an unquoted
|
||||
# ``pip install remove-ai-watermarks[qwen-zimage]`` dies with "no matches found"
|
||||
# before pip ever runs -- another install hint that does not install.
|
||||
INVISIBLE_EXTRA = "'remove-ai-watermarks[qwen-zimage]'"
|
||||
|
||||
# qwen-zimage's output already matches the input's detail level, so polishing it is a
|
||||
# no-op at best. sdxl-zimage's global pass leaves the softer output the polish exists
|
||||
# for. This is per-profile data, not a CLI concern: the flag defaults to None so that
|
||||
# "the user did not choose" stays a value rather than an inference from Click state.
|
||||
PROFILE_ADAPTIVE_POLISH = {QWEN_ZIMAGE_PROFILE: False, SDXL_ZIMAGE_PROFILE: True}
|
||||
|
||||
SDXL_LIGHTNING_MODEL_ID = "ByteDance/SDXL-Lightning"
|
||||
SDXL_LIGHTNING_PATTERN = "sdxl_lightning_4step_lora.safetensors"
|
||||
|
||||
# Both profiles are certified at a fixed seed because SynthID removal near the
|
||||
# strength floor is seed-dependent. The step count and CFG are not settable at all --
|
||||
# each stage owns them (``GLOBAL_STEPS`` / ``FACE_STEPS`` in qwen_zimage_pipeline).
|
||||
PROFILE_SEED = 0
|
||||
|
||||
# sdxl-zimage runs the qwen-zimage recipe on an SDXL global stage, and strength is
|
||||
# architecture-bound: at Qwen's 0.154 an SDXL global pass leaves SynthID on a native
|
||||
# 2816x1536 Gemini original, while 0.20, 0.25 and 0.30 all read clean in the Gemini
|
||||
# app. 0.25 keeps a rung of margin over that boundary, which the historical SDXL
|
||||
# certification argues for -- it recorded 0.20 as DETECTED against Gemini on an
|
||||
# older SDXL pipeline. OpenAI is the easier oracle: the profile already cleared
|
||||
# openai.com/verify at 0.1102, so 0.15 sits above what was verified rather than on
|
||||
# it. Unknown follows Gemini, the stricter of the two.
|
||||
#
|
||||
# Unlike qwen-zimage this is a flat vendor policy rather than a resolution curve,
|
||||
# because flat values are what was measured. Every verdict above comes from a fixed
|
||||
# strength at one size; no size dependence has been established for this stage.
|
||||
SDXL_ZIMAGE_OPENAI_STRENGTH = 0.15
|
||||
SDXL_ZIMAGE_GEMINI_STRENGTH = 0.25
|
||||
SDXL_ZIMAGE_UNKNOWN_STRENGTH = SDXL_ZIMAGE_GEMINI_STRENGTH
|
||||
|
||||
# qwen-zimage keeps its resolution curve for openai/unknown content, but Google
|
||||
# content takes this flat oracle-measured floor instead of the curve (whose top,
|
||||
# 0.154, left a 4.33 MP CJK-sign fixture detected x3 on the full production path
|
||||
# with a valid pixel control, 2026-08-18). 0.30 anchors measured clean in Gemini
|
||||
# across two fixtures and two work accounts (CJK sign + 18-face, 3/3 checks each,
|
||||
# 2026-08 research) and stayed clean under the vae-glyphs donor layer on top.
|
||||
QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.30
|
||||
|
||||
|
||||
# sdxl-zimage picks its strength from the VENDOR (unlike qwen-zimage, which derives it
|
||||
# from image area). An unlisted or unknown vendor falls back to the Gemini value.
|
||||
_SDXL_ZIMAGE_STRENGTH_BY_VENDOR: dict[str, float] = {
|
||||
"openai": SDXL_ZIMAGE_OPENAI_STRENGTH,
|
||||
"google": SDXL_ZIMAGE_GEMINI_STRENGTH,
|
||||
}
|
||||
_ALIASES = {
|
||||
"qwen_zimage": QWEN_ZIMAGE_PROFILE,
|
||||
"sdxl_zimage": SDXL_ZIMAGE_PROFILE,
|
||||
}
|
||||
|
||||
|
||||
def normalize_profile(profile: str) -> str:
|
||||
"""Normalize spelling and resolve the underscore spellings."""
|
||||
value = profile.strip().casefold()
|
||||
return _ALIASES.get(value, value)
|
||||
|
||||
|
||||
def resolve_seed(seed: int | None) -> int:
|
||||
"""Keep both profiles reproducible by default."""
|
||||
return PROFILE_SEED if seed is None else seed
|
||||
|
||||
|
||||
def resolve_adaptive_polish(adaptive_polish: bool | None, pipeline: str) -> bool:
|
||||
"""Return an explicit polish choice, or the profile's calibrated default."""
|
||||
if adaptive_polish is not None:
|
||||
return adaptive_polish
|
||||
return PROFILE_ADAPTIVE_POLISH.get(normalize_profile(pipeline), True)
|
||||
|
||||
|
||||
def strength_default_help() -> str:
|
||||
"""Describe the live default policy without duplicating its values."""
|
||||
return (
|
||||
"profile-adaptive (qwen-zimage uses resolution-adaptive denoise, with a "
|
||||
f"flat oracle-measured Google floor of {QWEN_ZIMAGE_GOOGLE_STRENGTH}; sdxl-zimage "
|
||||
f"uses OpenAI {SDXL_ZIMAGE_OPENAI_STRENGTH} / Google {SDXL_ZIMAGE_GEMINI_STRENGTH} / "
|
||||
f"unknown {SDXL_ZIMAGE_UNKNOWN_STRENGTH}, from the C2PA issuer)"
|
||||
)
|
||||
|
||||
|
||||
def resolve_strength(
|
||||
strength: float | None,
|
||||
vendor: str | None = None,
|
||||
pipeline: str | None = None,
|
||||
*,
|
||||
size: tuple[int, int] | None = None,
|
||||
) -> float:
|
||||
"""Resolve a user override or the calibrated policy for a profile and vendor.
|
||||
|
||||
Total by design. qwen-zimage picks its strength from image area rather than
|
||||
from the vendor, so it needs ``size``; returning ``None`` for it instead would push that
|
||||
branch onto every caller and move one of the two strength policies outside this
|
||||
module. ``size`` is required for qwen-zimage without an explicit strength.
|
||||
One measured exception since 0.27.2: Google-provenance content on qwen-zimage
|
||||
takes the flat ``QWEN_ZIMAGE_GOOGLE_STRENGTH`` floor instead of the area curve
|
||||
(the curve's 0.154 top left a 4.33 MP fixture oracle-detected on the full
|
||||
production path; see the constant's comment).
|
||||
"""
|
||||
if strength is not None:
|
||||
return strength
|
||||
if normalize_profile(pipeline or "") == SDXL_ZIMAGE_PROFILE:
|
||||
return _SDXL_ZIMAGE_STRENGTH_BY_VENDOR.get((vendor or "").casefold(), SDXL_ZIMAGE_UNKNOWN_STRENGTH)
|
||||
if size is None:
|
||||
raise ValueError("qwen-zimage resolves strength from image area, so size is required")
|
||||
if (vendor or "").casefold() == "google":
|
||||
return QWEN_ZIMAGE_GOOGLE_STRENGTH
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import resolution_adaptive_denoise
|
||||
|
||||
return resolution_adaptive_denoise(*size)
|
||||
|
||||
|
||||
def vendor_for_strength(image_path: Path) -> Literal["openai", "google"] | None:
|
||||
"""Select the strength cohort using the input's SynthID provenance evidence."""
|
||||
try:
|
||||
from remove_ai_watermarks.metadata import synthid_source
|
||||
|
||||
evidence = (synthid_source(image_path) or "").casefold()
|
||||
except Exception:
|
||||
return None
|
||||
if "google" in evidence:
|
||||
return "google"
|
||||
if "openai" in evidence:
|
||||
return "openai"
|
||||
return None
|
||||
@@ -0,0 +1,226 @@
|
||||
"""Project-native orchestration for diffusion-based pixel regeneration."""
|
||||
|
||||
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false, reportOptionalMemberAccess=false, reportOptionalCall=false, reportOptionalSubscript=false, reportOptionalOperand=false, reportAttributeAccessIssue=false, reportPrivateImportUsage=false, reportPrivateUsage=false, reportInvalidTypeForm=false, reportConstantRedefinition=false, reportUnnecessaryComparison=false
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks._internal.watermark_profiles import (
|
||||
DEFAULT_PROFILE,
|
||||
INVISIBLE_EXTRA,
|
||||
PROFILE_CHOICES,
|
||||
REMOVAL_MODULES,
|
||||
SDXL_ZIMAGE_PROFILE,
|
||||
normalize_profile,
|
||||
resolve_seed,
|
||||
resolve_strength,
|
||||
)
|
||||
from remove_ai_watermarks.optional_deps import module_available
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextManifest
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
try:
|
||||
import torch
|
||||
|
||||
_HAS_TORCH = True
|
||||
except ImportError:
|
||||
torch = None # type: ignore[assignment]
|
||||
_HAS_TORCH = False
|
||||
|
||||
# Probed once at import. ``torch`` is imported above rather than probed because this
|
||||
# module needs the object, not just the answer.
|
||||
_HAS_REMOVAL_MODULES = module_available(*(name for name in REMOVAL_MODULES if name != "torch"))
|
||||
|
||||
|
||||
def is_watermark_removal_available() -> bool:
|
||||
"""Return whether the full removal runtime can be imported."""
|
||||
return _HAS_TORCH and _HAS_REMOVAL_MODULES
|
||||
|
||||
|
||||
def _ensure_watermark_deps() -> None:
|
||||
if not is_watermark_removal_available():
|
||||
raise ImportError(
|
||||
f"Invisible watermark regeneration requires the 'qwen-zimage' extra: pip install {INVISIBLE_EXTRA}."
|
||||
)
|
||||
|
||||
|
||||
def _has_nvidia_gpu() -> bool:
|
||||
try:
|
||||
subprocess.run(
|
||||
["nvidia-smi"],
|
||||
check=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
except (FileNotFoundError, subprocess.CalledProcessError):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _cuda_works() -> bool:
|
||||
try:
|
||||
probe = torch.tensor([1.0], device="cuda") # type: ignore[union-attr]
|
||||
_ = probe + probe
|
||||
except (AssertionError, RuntimeError):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def get_device() -> str:
|
||||
"""Return ``"cuda"`` when a usable CUDA backend is present, else ``"cpu"``.
|
||||
|
||||
Deliberately binary. Both profiles are CUDA-only, so an XPU or MPS answer would
|
||||
only travel one frame further to the same refusal in :class:`WatermarkRemover`,
|
||||
while costing a probe on each. ``"cpu"`` here means "no CUDA", which is exactly
|
||||
what that refusal reports.
|
||||
"""
|
||||
if not _HAS_TORCH:
|
||||
return "cpu"
|
||||
if torch.cuda.is_available() and _cuda_works(): # type: ignore[union-attr]
|
||||
return "cuda"
|
||||
if _has_nvidia_gpu():
|
||||
logger.warning("NVIDIA GPU detected, but the installed PyTorch build has no working CUDA backend")
|
||||
return "cpu"
|
||||
|
||||
|
||||
class WatermarkRemover:
|
||||
"""Load one regeneration profile and write a metadata-clean raster output."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
device: str | None = None,
|
||||
progress_callback: Callable[[str], None] | None = None,
|
||||
hf_token: str | None = None,
|
||||
pipeline: str = DEFAULT_PROFILE,
|
||||
controlnet_conditioning_scale: float = 1.0,
|
||||
cpu_offload: bool = False,
|
||||
) -> None:
|
||||
self.model_profile = normalize_profile(pipeline)
|
||||
if self.model_profile not in PROFILE_CHOICES:
|
||||
raise ValueError(f"Unsupported pipeline '{pipeline}'. Use one of: {', '.join(PROFILE_CHOICES)}.")
|
||||
# There is no ``model_id`` parameter and no ``model_id`` attribute: each
|
||||
# profile pins a fixed model stack, and the dtype below is bound to that
|
||||
# stack's weights. Both used to be constructor overrides that existed only to
|
||||
# be rejected or to break the run, and the attribute only existed to echo the
|
||||
# rejected value back.
|
||||
_ensure_watermark_deps()
|
||||
selected_device = (device or get_device()).casefold()
|
||||
self.device = get_device() if selected_device == "auto" else selected_device
|
||||
# CUDA is a precondition of the object, not of the run. Both profiles raise on
|
||||
# any other device, so accepting one here only defers a guaranteed failure to
|
||||
# model-load time, several layers down and under the wrong profile's name.
|
||||
if self.device != "cuda":
|
||||
raise ValueError(
|
||||
f"Invisible-watermark removal is CUDA-only, so '{self.device}' cannot run it. "
|
||||
"Both remaining profiles need an NVIDIA GPU. Visible-mark removal and "
|
||||
"every identify command still run on CPU."
|
||||
)
|
||||
|
||||
if self.model_profile == SDXL_ZIMAGE_PROFILE:
|
||||
# SDXL ships fp16 weights and an fp16-safe VAE; bf16 would give up the
|
||||
# variant without buying anything on this architecture.
|
||||
self.torch_dtype = torch.float16 # type: ignore[union-attr]
|
||||
else:
|
||||
self.torch_dtype = torch.bfloat16 # type: ignore[union-attr]
|
||||
|
||||
self.cpu_offload = cpu_offload
|
||||
self.controlnet_conditioning_scale = controlnet_conditioning_scale
|
||||
self.hf_token = hf_token or os.environ.get("HF_TOKEN")
|
||||
self._progress_callback = progress_callback
|
||||
self._qwen_zimage_pipeline: Any = None
|
||||
|
||||
def preload(self, *, global_only: bool = False) -> None:
|
||||
"""Materialize the selected model stack before the first request."""
|
||||
self._load_qwen_zimage_pipeline().preload(global_only=global_only)
|
||||
|
||||
def _load_qwen_zimage_pipeline(self) -> Any:
|
||||
if self._qwen_zimage_pipeline is None:
|
||||
if self.model_profile == SDXL_ZIMAGE_PROFILE:
|
||||
from remove_ai_watermarks._internal.sdxl_zimage_pipeline import (
|
||||
SdxlZImagePipeline as _Pipeline,
|
||||
)
|
||||
else:
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import (
|
||||
QwenZImagePipeline as _Pipeline,
|
||||
)
|
||||
|
||||
self._qwen_zimage_pipeline = _Pipeline(
|
||||
device=self.device,
|
||||
torch_dtype=self.torch_dtype,
|
||||
hf_token=self.hf_token,
|
||||
progress_callback=self._progress_callback,
|
||||
controlnet_conditioning_scale=self.controlnet_conditioning_scale,
|
||||
keep_face_models_on_device=False if self.cpu_offload else None,
|
||||
keep_global_models_on_device=False if self.cpu_offload else None,
|
||||
)
|
||||
return self._qwen_zimage_pipeline
|
||||
|
||||
def _write_output(self, image: Image.Image, output_path: Path) -> None:
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
bgr = np.ascontiguousarray(np.asarray(image.convert("RGB"))[:, :, ::-1])
|
||||
if not image_io.imwrite(str(output_path), bgr):
|
||||
image.save(output_path)
|
||||
from remove_ai_watermarks.metadata import remove_ai_metadata
|
||||
|
||||
remove_ai_metadata(output_path, output_path, keep_standard=True)
|
||||
|
||||
def remove_watermark(
|
||||
self,
|
||||
image_path: Path,
|
||||
output_path: Path | None = None,
|
||||
strength: float | None = None,
|
||||
seed: int | None = None,
|
||||
vendor: str | None = None,
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
text_manifest: VerifiedTextManifest | None = None,
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Path:
|
||||
"""Regenerate image pixels and write the result without AI metadata.
|
||||
|
||||
Step count and CFG are not parameters. Each stage of both profiles is a
|
||||
distilled schedule that owns its own, so the only thing a caller-supplied
|
||||
value could do is break the run or be rejected.
|
||||
"""
|
||||
if not image_path.exists():
|
||||
raise FileNotFoundError(f"Image not found: {image_path}")
|
||||
destination = output_path or image_path
|
||||
with Image.open(image_path) as opened:
|
||||
source = opened.convert("RGB")
|
||||
|
||||
resolved_strength = resolve_strength(strength, vendor, self.model_profile, size=source.size)
|
||||
if not 0.0 <= resolved_strength <= 1.0:
|
||||
raise ValueError(f"Strength must be between 0.0 and 1.0, got {resolved_strength}")
|
||||
if text_manifest is not None and self.model_profile == SDXL_ZIMAGE_PROFILE:
|
||||
raise ValueError("Verified text restoration is supported only by the qwen-zimage profile")
|
||||
if text_manifest is not None and tile:
|
||||
raise ValueError("Verified text restoration is not calibrated with tiled diffusion")
|
||||
|
||||
result = self._load_qwen_zimage_pipeline().run(
|
||||
source,
|
||||
strength=resolved_strength,
|
||||
seed=resolve_seed(seed),
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
text_manifest=text_manifest,
|
||||
fidelity_anchor=fidelity_anchor,
|
||||
)
|
||||
self._write_output(result, destination)
|
||||
return destination
|
||||
@@ -132,7 +132,7 @@ class TextMarkConfig:
|
||||
# Which image dimension the mark's size and margins scale with. VENDOR-SPECIFIC,
|
||||
# measured, not assumed -- see TextMarkEngine.scale_base. "short" = min(h, w), "width" = w.
|
||||
scale_basis: Literal["short", "width"] = "width"
|
||||
# Scale rungs ``_tophat_best`` sweeps (the detection comb). PER-MARK: a vendor
|
||||
# Scale rungs ``_ladder_best`` sweeps (the detection comb). PER-MARK: a vendor
|
||||
# whose stamp sizes do not land on the shared 3-rung comb carries its own ladder
|
||||
# (measured for 千问, whose marks sit in two size modes ~1.6x apart -- one fraction
|
||||
# on 3 rungs covers only ~75% of them). Densifying the SHARED ladder for everyone
|
||||
@@ -156,7 +156,6 @@ class TextMarkLocation:
|
||||
y: int
|
||||
w: int
|
||||
h: int
|
||||
is_fallback: bool = True # geometry anchor (no template match) -> always True for now
|
||||
|
||||
@property
|
||||
def bbox(self) -> tuple[int, int, int, int]:
|
||||
@@ -171,6 +170,33 @@ class TextMarkDetection:
|
||||
confidence: float = 0.0
|
||||
region: tuple[int, int, int, int] = (0, 0, 0, 0)
|
||||
coverage: float = 0.0 # fraction of the box occupied by glyph pixels
|
||||
# ROI-local (x0, y0, x1, y1) of the ladder sweep's best match, in the LOCATED BOX's
|
||||
# coordinates. None for the ``binary`` front-end (it runs no sweep) and whenever no
|
||||
# rung matched. ``footprint_mask`` bounds the fill with it, so carrying it here is
|
||||
# what stops the mask path re-running a sweep the detector already ran.
|
||||
match_box: tuple[int, int, int, int] | None = None
|
||||
# The trust level this detection was taken at, mirroring detect()'s ``provenance``.
|
||||
# ``footprint_mask`` reuses a threaded detection only when it matches the STRICT
|
||||
# level its own re-detect would have used -- see TextMarkEngine._strict_detection.
|
||||
provenance: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TextMarkScan:
|
||||
"""The trust-level-BLIND half of text-mark detection, reusable across both levels.
|
||||
|
||||
``loc is None`` means detection stopped before any scan (empty or too-small image).
|
||||
``score is None`` means the binary front-end fell below its coverage gate, which is
|
||||
a verdict of "not detected, confidence 0.0" without consulting the rival margin.
|
||||
"""
|
||||
|
||||
loc: TextMarkLocation | None
|
||||
box: NDArray[Any] | None # box-sized binary glyph mask
|
||||
base: int # scale_base(image)
|
||||
frame: tuple[int, int] = (0, 0) # (h, w) of the scanned image
|
||||
coverage: float = 0.0
|
||||
score: float | None = None
|
||||
match_box: tuple[int, int, int, int] | None = None
|
||||
|
||||
|
||||
# Alpha / silhouette templates, cached per asset name. This shared cache lets every
|
||||
@@ -307,6 +333,28 @@ class TextMarkEngine:
|
||||
|
||||
# ── Locate ──────────────────────────────────────────────────────────
|
||||
|
||||
def _roi_fields(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[NDArray[Any], NDArray[Any], NDArray[Any]]:
|
||||
"""``(luma, saturation, local_background)`` for the located box, all float32.
|
||||
|
||||
The ROI is normalized to 3-channel BGR first (grayscale / BGRA would break
|
||||
``axis=2``).
|
||||
|
||||
Local background model: a strong Gaussian blur (sigma ~ box height); the white
|
||||
top-hat (``luma - local_bg``) lights up bright thin strokes regardless of the
|
||||
absolute background level.
|
||||
|
||||
The callers each keep their own ``bh < 16 or bw < 16`` guard: they return three
|
||||
different sentinels for a degenerate ROI, so the check cannot move in here.
|
||||
"""
|
||||
x, y, bw, bh = loc.bbox
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
luma = roi.mean(axis=2)
|
||||
sat = roi.max(axis=2) - roi.min(axis=2)
|
||||
sigma = max(4.0, bh * 0.4) # 0.4 factor and 4.0 floor are calibrated; do not retune
|
||||
return luma, sat, cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma)
|
||||
|
||||
def tophat_response(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any] | None:
|
||||
"""The CONTINUOUS white top-hat in the located box -- the glyph signal, unbinarized.
|
||||
|
||||
@@ -329,26 +377,48 @@ class TextMarkEngine:
|
||||
Kept per-mark (``detect_frontend``) rather than switched globally, because a
|
||||
front-end change must be measured per mark before it ships.
|
||||
"""
|
||||
return self._residual_response(image, loc, absolute=False)
|
||||
|
||||
def _residual_response(self, image: NDArray[Any], loc: TextMarkLocation, *, absolute: bool) -> NDArray[Any] | None:
|
||||
"""Max-normalized uint8 local-luma residual in the located box, saturation-weighted.
|
||||
|
||||
``absolute=False`` keeps only the POSITIVE side -- the white top-hat, for a mark
|
||||
always rendered brighter than its background. ``absolute=True`` takes the
|
||||
magnitude, for a renderer that switches between light-on-dark and dark-on-light
|
||||
while preserving one silhouette; a one-polarity top-hat misses the latter.
|
||||
"""
|
||||
c = self.config
|
||||
x, y, bw, bh = loc.bbox
|
||||
_x, _y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return None
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
luma = roi.mean(axis=2)
|
||||
sat = roi.max(axis=2) - roi.min(axis=2)
|
||||
sigma = max(4.0, bh * 0.4)
|
||||
tophat = luma - cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma)
|
||||
resp = np.clip(tophat, 0, None) * (sat < c.max_saturation)
|
||||
luma, sat, local_bg = self._roi_fields(image, loc)
|
||||
residual = luma - local_bg
|
||||
resp = (np.abs(residual) if absolute else np.clip(residual, 0, None)) * (sat < c.max_saturation)
|
||||
peak = float(resp.max())
|
||||
if peak <= 1e-6:
|
||||
return None
|
||||
return (resp / peak * 255).astype(np.uint8)
|
||||
|
||||
def _tophat_best(
|
||||
def _detect_response(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any] | None:
|
||||
"""The uint8 image the ladder sweep correlates the silhouette against, chosen by
|
||||
``TextMarkConfig.detect_frontend``. ``binary`` runs no sweep and never reaches here."""
|
||||
frontend = self.config.detect_frontend
|
||||
if frontend == "gray":
|
||||
x, y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return None
|
||||
return cv2.cvtColor(image_io.to_bgr(image[y : y + bh, x : x + bw]), cv2.COLOR_BGR2GRAY)
|
||||
if frontend == "contrast":
|
||||
return self._residual_response(image, loc, absolute=True)
|
||||
if frontend == "tophat":
|
||||
return self._residual_response(image, loc, absolute=False)
|
||||
raise ValueError(f"{frontend!r} has no ladder response (binary runs no sweep)")
|
||||
|
||||
def _ladder_best(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Best TM_CCOEFF_NORMED of a soft template against the continuous response, and
|
||||
the ROI-local box (x0, y0, x1, y1) where that best match sits.
|
||||
"""Best TM_CCOEFF_NORMED of the mark's silhouette against its front-end response,
|
||||
and the ROI-local box (x0, y0, x1, y1) where that best match sits.
|
||||
|
||||
Sweeps the mark's scale ladder: the nominal glyph size is derived from the mark's
|
||||
geometry, but a vendor re-rasterization shifts it by a few percent and the
|
||||
@@ -359,10 +429,15 @@ class TextMarkEngine:
|
||||
detection, the box bounds the fill. Sharing it is deliberate: the standing rule is
|
||||
that detection and the mask use the same front-end, and the way that rule was last
|
||||
broken was a drift between two separate implementations. One method makes the drift
|
||||
impossible instead of merely discouraged.
|
||||
impossible instead of merely discouraged -- which is why the three continuous
|
||||
front-ends (tophat / contrast / gray) sweep here rather than in a copy each.
|
||||
|
||||
Tie-breaking is load-bearing: the ``0.0`` seed plus the STRICT ``>`` means the
|
||||
EARLIEST ladder rung wins a tie, and a sweep whose maximum is not above 0.0
|
||||
returns no box at all.
|
||||
"""
|
||||
c = self.config
|
||||
resp = self.tophat_response(image, loc)
|
||||
resp = self._detect_response(image, loc)
|
||||
sil = self._glyph_silhouette()
|
||||
if resp is None or sil is None:
|
||||
return (0.0, None)
|
||||
@@ -374,90 +449,12 @@ class TextMarkEngine:
|
||||
gh = max(4, int(c.alpha_height_frac * base * scale))
|
||||
if gw >= resp.shape[1] or gh >= resp.shape[0]:
|
||||
continue
|
||||
tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32)
|
||||
if c.template_blur > 0:
|
||||
tmpl = cv2.GaussianBlur(tmpl, (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur)
|
||||
result = cv2.matchTemplate(resp, tmpl.astype(np.uint8), cv2.TM_CCOEFF_NORMED)
|
||||
_, score, _, top_left = cv2.minMaxLoc(result)
|
||||
if score > best_score:
|
||||
tx, ty = int(top_left[0]), int(top_left[1])
|
||||
best_score, best_box = float(score), (tx, ty, tx + gw - 1, ty + gh - 1)
|
||||
return (best_score, best_box)
|
||||
|
||||
def _tophat_score(self, image: NDArray[Any], loc: TextMarkLocation) -> float:
|
||||
"""The detection score alone -- the box the removal mask needs is discarded here."""
|
||||
return self._tophat_best(image, loc)[0]
|
||||
|
||||
def _contrast_best(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Best silhouette match against the absolute local-luma residual.
|
||||
|
||||
Unlike the white top-hat, this response is polarity-independent: the same
|
||||
watermark can be lighter or darker than its local background. Detection and
|
||||
removal share the returned box, preserving the front-end parity contract.
|
||||
"""
|
||||
c = self.config
|
||||
x, y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return (0.0, None)
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
luma = roi.mean(axis=2)
|
||||
sat = roi.max(axis=2) - roi.min(axis=2)
|
||||
sigma = max(4.0, bh * 0.4)
|
||||
response = np.abs(luma - cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma))
|
||||
response *= sat < c.max_saturation
|
||||
peak = float(response.max())
|
||||
sil = self._glyph_silhouette()
|
||||
if peak <= 1e-6 or sil is None:
|
||||
return (0.0, None)
|
||||
response = (response / peak * 255).astype(np.uint8)
|
||||
base = self.scale_base(image)
|
||||
best_score = 0.0
|
||||
best_box: tuple[int, int, int, int] | None = None
|
||||
for scale in c.ladder:
|
||||
gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
|
||||
gh = max(4, int(c.alpha_height_frac * base * scale))
|
||||
if gw >= response.shape[1] or gh >= response.shape[0]:
|
||||
continue
|
||||
template = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA)
|
||||
result = cv2.matchTemplate(response, template, cv2.TM_CCOEFF_NORMED)
|
||||
_, score, _, top_left = cv2.minMaxLoc(result)
|
||||
if score > best_score:
|
||||
tx, ty = int(top_left[0]), int(top_left[1])
|
||||
best_score, best_box = float(score), (tx, ty, tx + gw - 1, ty + gh - 1)
|
||||
return (best_score, best_box)
|
||||
|
||||
def _gray_best(self, image: NDArray[Any], loc: TextMarkLocation) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Best TM_CCOEFF_NORMED of the silhouette against the raw GRAYSCALE ROI, and
|
||||
the ROI-local box (x0, y0, x1, y1) of that best match.
|
||||
|
||||
Mirrors :meth:`_tophat_best` (same ladder sweep, same one-method contract so
|
||||
detection and the removal mask can never drift), but skips the top-hat
|
||||
entirely: the RunningHub mark is a faint mid-gray text the top-hat's
|
||||
background subtraction suppresses to clean-arm levels, while raw gray NCC
|
||||
separates (see ``TextMarkConfig.detect_frontend``). Contrast-DEPENDENT by
|
||||
construction, so the gate must be picked against the clean arm, which is
|
||||
what ``scripts/vendor_mark_calibrate.py`` does.
|
||||
"""
|
||||
c = self.config
|
||||
x, y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return (0.0, None)
|
||||
roi = cv2.cvtColor(image_io.to_bgr(image[y : y + bh, x : x + bw]), cv2.COLOR_BGR2GRAY)
|
||||
sil = self._glyph_silhouette()
|
||||
if sil is None:
|
||||
return (0.0, None)
|
||||
base = self.scale_base(image)
|
||||
best_score = 0.0
|
||||
best_box: tuple[int, int, int, int] | None = None
|
||||
for scale in c.ladder:
|
||||
gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
|
||||
gh = max(4, int(c.alpha_height_frac * base * scale))
|
||||
if gw >= roi.shape[1] or gh >= roi.shape[0]:
|
||||
continue
|
||||
tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA)
|
||||
result = cv2.matchTemplate(roi, tmpl, cv2.TM_CCOEFF_NORMED)
|
||||
if c.detect_frontend == "tophat" and c.template_blur > 0:
|
||||
tmpl = cv2.GaussianBlur(
|
||||
tmpl.astype(np.float32), (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur
|
||||
).astype(np.uint8)
|
||||
result = cv2.matchTemplate(resp, tmpl, cv2.TM_CCOEFF_NORMED)
|
||||
_, score, _, top_left = cv2.minMaxLoc(result)
|
||||
if score > best_score:
|
||||
tx, ty = int(top_left[0]), int(top_left[1])
|
||||
@@ -512,7 +509,7 @@ class TextMarkEngine:
|
||||
y = min(margin_b, max(0, h - wm_h)) if c.corner == "tl" else max(0, h - margin_b - wm_h)
|
||||
wm_w = min(wm_w, w - x)
|
||||
wm_h = min(wm_h, h - y)
|
||||
return TextMarkLocation(x=x, y=y, w=wm_w, h=wm_h, is_fallback=True)
|
||||
return TextMarkLocation(x=x, y=y, w=wm_w, h=wm_h)
|
||||
|
||||
# ── Mask ────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -571,10 +568,33 @@ class TextMarkEngine:
|
||||
the mark's own ``provenance_ncc_factor`` to recover a faint or slightly
|
||||
re-rendered mark (per-mark, not shared -- see _DEFAULT_PROVENANCE_NCC_FACTOR).
|
||||
"""
|
||||
scan = self._scan(image)
|
||||
return self._verdict(scan, provenance=provenance)
|
||||
|
||||
def detect_both(self, image: NDArray[Any] | None) -> tuple[TextMarkDetection, TextMarkDetection]:
|
||||
"""``(strict, relaxed)`` from ONE scan of the image.
|
||||
|
||||
``provenance`` scales the acceptance THRESHOLD and nothing else -- the locate
|
||||
box, the glyph mask, the coverage and the front-end ladder score are computed
|
||||
identically at either trust level. Two ``detect`` calls therefore ran the same
|
||||
expensive sweep twice to reach two verdicts, which is what the arbiter's
|
||||
perception pass did for every mark on every image.
|
||||
|
||||
Returns two DISTINCT objects: subclasses demote a verdict by mutating it.
|
||||
"""
|
||||
scan = self._scan(image)
|
||||
return self._verdict(scan, provenance=False), self._verdict(scan, provenance=True)
|
||||
|
||||
def _scan(self, image: NDArray[Any] | None) -> TextMarkScan:
|
||||
"""Everything in detection that does not depend on the trust level.
|
||||
|
||||
Per-CALL only, never memoized on ``self``: ``remove_auto_marks`` re-invokes each
|
||||
engine on a progressively cleaned frame inside one process, so a cached scan
|
||||
would answer for the wrong pixels.
|
||||
"""
|
||||
c = self.config
|
||||
det = TextMarkDetection()
|
||||
if image is None or image.size == 0:
|
||||
return det
|
||||
return TextMarkScan(None, None, 0)
|
||||
# Guard against the small-image NCC-noise false positive (see
|
||||
# _MIN_DETECT_SHORT_SIDE): an icon/thumbnail is too small to carry a real
|
||||
# text label, and the degraded few-pixel template spuriously correlates.
|
||||
@@ -585,57 +605,57 @@ class TextMarkEngine:
|
||||
min(image.shape[:2]),
|
||||
_MIN_DETECT_SHORT_SIDE,
|
||||
)
|
||||
return det
|
||||
return TextMarkScan(None, None, 0)
|
||||
loc = self.locate(image)
|
||||
box = self.extract_mask(image, loc) # box-sized mask (== old full-frame cropped to bbox)
|
||||
_x, _y, bw, bh = loc.bbox
|
||||
coverage = float((box > 0).sum()) / float(max(1, bw * bh))
|
||||
det.region = loc.bbox
|
||||
det.coverage = coverage
|
||||
if c.detect_frontend == "tophat":
|
||||
# The continuous front-end does not depend on the binarized blob, so the
|
||||
# coverage gate (a blob-area heuristic) does not apply to it.
|
||||
score = self._tophat_score(image, loc)
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug("%s detect (tophat): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
|
||||
base = self.scale_base(image)
|
||||
match_box: tuple[int, int, int, int] | None = None
|
||||
if c.detect_frontend == "binary":
|
||||
# The coverage gate is a blob-AREA heuristic, so it applies only to the
|
||||
# front-end that binarizes; the continuous ones never build a blob. Below
|
||||
# the gate the detection stays at confidence 0.0 and the rival margin is
|
||||
# never consulted, so the score stays None here.
|
||||
score = self._template_match_score(box, base) if coverage >= c.detect_min_coverage else None
|
||||
else:
|
||||
score, match_box = self._ladder_best(image, loc)
|
||||
return TextMarkScan(loc, box, base, frame=image.shape[:2], coverage=coverage, score=score, match_box=match_box)
|
||||
|
||||
def _verdict(self, scan: TextMarkScan, *, provenance: bool) -> TextMarkDetection:
|
||||
"""Apply the trust-level-dependent tail to a scan, as a fresh result object."""
|
||||
c = self.config
|
||||
det = TextMarkDetection(provenance=provenance)
|
||||
if scan.loc is None or scan.box is None:
|
||||
return det
|
||||
if c.detect_frontend == "gray":
|
||||
# Same no-coverage-gate reasoning as tophat: the gray front-end never
|
||||
# binarizes, so a blob-area heuristic does not apply to it either.
|
||||
score = self._gray_best(image, loc)[0]
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug("%s detect (gray): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
|
||||
det.region = scan.loc.bbox
|
||||
det.coverage = scan.coverage
|
||||
det.match_box = scan.match_box
|
||||
if scan.score is None: # binary front-end below the coverage gate
|
||||
return det
|
||||
if c.detect_frontend == "contrast":
|
||||
score = self._contrast_best(image, loc)[0]
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug(
|
||||
"%s detect (contrast): ncc=%.2f thr=%.2f detected=%s",
|
||||
c.name,
|
||||
score,
|
||||
threshold,
|
||||
det.detected,
|
||||
)
|
||||
return det
|
||||
if coverage >= c.detect_min_coverage:
|
||||
score = self._template_match_score(box, self.scale_base(image))
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug(
|
||||
"%s detect: coverage=%.3f ncc=%.2f thr=%.2f detected=%s",
|
||||
c.name,
|
||||
coverage,
|
||||
score,
|
||||
threshold,
|
||||
det.detected,
|
||||
)
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = scan.score
|
||||
# Short-circuit is load-bearing: _rival_margin_ok scores every rival template
|
||||
# and logs its own rejection line, so it must stay unevaluated below threshold.
|
||||
det.detected = scan.score >= threshold and self._rival_margin_ok(scan.score, scan.box, scan.base)
|
||||
logger.debug(
|
||||
"%s detect (%s): coverage=%.3f ncc=%.2f thr=%.2f detected=%s",
|
||||
c.name,
|
||||
c.detect_frontend,
|
||||
scan.coverage,
|
||||
scan.score,
|
||||
threshold,
|
||||
det.detected,
|
||||
)
|
||||
return self._post_gate(det, scan)
|
||||
|
||||
def _post_gate(self, det: TextMarkDetection, scan: TextMarkScan) -> TextMarkDetection:
|
||||
"""Per-mark demotion applied after the shared threshold, for both trust levels.
|
||||
|
||||
OVERRIDABLE. It lives here rather than in a ``detect`` override so a mark's gate
|
||||
cannot be silently skipped by the single-pass ``detect_both`` path -- which is
|
||||
exactly what happened while the anchor demotions were ``detect`` overrides.
|
||||
"""
|
||||
return det
|
||||
|
||||
# ── Inpaint footprint (for the inpaint-fallback removal path) ────────
|
||||
@@ -645,8 +665,132 @@ class TextMarkEngine:
|
||||
# to mask. A real strip covers hundreds of pixels.
|
||||
_MIN_GLYPH_PIXELS = 20
|
||||
|
||||
def _strict_detection(self, image: NDArray[Any], detection: TextMarkDetection | None) -> TextMarkDetection:
|
||||
"""The STRICT detection the footprint is bounded by.
|
||||
|
||||
A threaded detection is reused only when it was taken at the same strict level
|
||||
this method would have used itself. A provenance-RELAXED detection is NOT
|
||||
reused: a strict re-detect can demote a mark the relaxed gate accepted, which
|
||||
for a continuous front-end means no mask at all. That is a MEASURED difference,
|
||||
not a refactor, so the strict semantics stay.
|
||||
|
||||
Reuse is safe against ``remove_auto_marks`` chaining marks on a progressively
|
||||
cleaned frame: the registry re-detects on that same cleaned array before
|
||||
threading (``KnownMark.localize``), so a threaded detection is never stale.
|
||||
"""
|
||||
if detection is not None and not detection.provenance:
|
||||
return detection
|
||||
return self.detect(image) # polymorphic: a subclass gate must still apply
|
||||
|
||||
def _geometry_rect(self, loc: TextMarkLocation, frame: tuple[int, int]) -> tuple[int, int, int, int]:
|
||||
"""The whole locate box, clamped to the frame -- the ``force`` footprint."""
|
||||
bx, by, bw, bh = loc.bbox
|
||||
h, w = frame
|
||||
return (bx, by, min(w, bx + bw), min(h, by + bh))
|
||||
|
||||
def _extend_match_box(
|
||||
self, box: tuple[int, int, int, int], loc: TextMarkLocation, frame: tuple[int, int]
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Grow an ROI-local box into the absolute fill rectangle by a symmetric pad.
|
||||
|
||||
OVERRIDABLE, and the override contract is specifically the DETECTOR'S MATCH BOX:
|
||||
a mark whose removable footprint reaches beyond what the NCC localizes -- Baidu's
|
||||
flat white tag right of the text run, LibLibAI's triangle logo left of the
|
||||
wordmark -- supplies its own extension here and inherits the rest of the
|
||||
footprint path. The blob-bbox branch never routes through an override.
|
||||
"""
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
bx, by, _bw, bh = loc.bbox
|
||||
h, w = frame
|
||||
pad = max(4, int(0.10 * bh))
|
||||
return (
|
||||
max(0, bx + gx0 - pad),
|
||||
max(0, by + gy0 - pad),
|
||||
min(w, bx + gx1 + 1 + pad),
|
||||
min(h, by + gy1 + 1 + pad),
|
||||
)
|
||||
|
||||
def _footprint_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Default footprint policy: the binary glyph blob's bbox, else the detector's
|
||||
own match box for the front-ends that under-segment, else the geometry box
|
||||
under ``force``.
|
||||
|
||||
The gray front-end exists for marks the top-hat under-segments, so the binary
|
||||
blob is NOT authoritative there: trusting it first bounded the fill by a PARTIAL
|
||||
blob (the faint head glyphs dropped out) and left the leftmost "Runni" of
|
||||
"RunningHub AI生成" unremoved (2026-07-22).
|
||||
|
||||
A dark-on-light Yuanbao mark has no WHITE top-hat blob at all, so the contrast
|
||||
front-end is bounded by the polarity-independent detector's match box too.
|
||||
|
||||
A tophat mark found only by the CONTINUOUS front-end has no binary glyph blob to
|
||||
bound, so the mask came back empty and removal was a silent no-op while
|
||||
``identify`` still reported the mark. Use the DETECTOR'S OWN best-match box: the
|
||||
correlation already located the mark at a position and scale, and thresholding
|
||||
the response was a strictly worse proxy for that. An earlier fix thresholded the
|
||||
max-normalized uint8 response at 0.5 -- which selects every non-zero pixel, not
|
||||
"half the peak" as its comment claimed -- and filled ~120% of the corner box on
|
||||
textured frames (measured: whole corner vs 58.7% for the match box, both
|
||||
detector-clean). Gated on an actual detection: on a clean corner the box would
|
||||
be spurious.
|
||||
"""
|
||||
ys, xs = np.where(self.extract_mask(image, loc) > 0)
|
||||
blob = (
|
||||
(int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max())) if xs.size >= self._MIN_GLYPH_PIXELS else None
|
||||
)
|
||||
frontend = self.config.detect_frontend
|
||||
if frontend in ("gray", "contrast"):
|
||||
det = self._strict_detection(image, detection)
|
||||
box = det.match_box if det.detected else blob
|
||||
elif blob is not None:
|
||||
box = blob
|
||||
elif frontend == "tophat":
|
||||
det = self._strict_detection(image, detection)
|
||||
box = det.match_box if det.detected else None
|
||||
else:
|
||||
box = None
|
||||
if box is not None:
|
||||
return self._extend_match_box(box, loc, image.shape[:2])
|
||||
return self._geometry_rect(loc, image.shape[:2]) if force else None
|
||||
|
||||
def _match_box_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Footprint policy for a mark whose fill must be bounded by the DETECTOR's match
|
||||
box and never by the binary glyph blob.
|
||||
|
||||
Baidu's white tag has a flat interior a top-hat cannot answer, and LibLibAI's
|
||||
blob bleeds up into background structure; in both cases the blob bbox is
|
||||
measurably wrong and the NCC match box is right. ``force`` takes priority here,
|
||||
unlike the default policy: a ``--no-detect`` caller named the mark, so the whole
|
||||
geometry box is the honest footprint.
|
||||
"""
|
||||
if force:
|
||||
return self._geometry_rect(loc, image.shape[:2])
|
||||
det = self._strict_detection(image, detection)
|
||||
if not det.detected or det.match_box is None:
|
||||
return None
|
||||
return self._extend_match_box(det.match_box, loc, image.shape[:2])
|
||||
|
||||
def footprint_mask(
|
||||
self, image: NDArray[Any], *, force: bool = False, dilate: int | None = None
|
||||
self,
|
||||
image: NDArray[Any] | None,
|
||||
*,
|
||||
force: bool = False,
|
||||
dilate: int | None = None,
|
||||
detection: TextMarkDetection | None = None,
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame uint8 mask (255 = mark) of the mark footprint, for the shared
|
||||
fill removal path (cv2 / MI-GAN / LaMa), or None if no glyph is found.
|
||||
@@ -663,6 +807,10 @@ class TextMarkEngine:
|
||||
|
||||
With ``force`` and no glyph found, falls back to the whole geometry box (the
|
||||
``--no-detect`` path). The caller gates on detection.
|
||||
|
||||
``detection`` is the caller's already-computed detection, threaded in so the
|
||||
footprint does not re-run a sweep the detector already ran. See
|
||||
:meth:`_strict_detection` for when it is reused.
|
||||
"""
|
||||
if image is None or image.size == 0:
|
||||
return None # guard before to_bgr (cvtColor raises on an empty Mat); mirror detect()
|
||||
@@ -671,46 +819,10 @@ class TextMarkEngine:
|
||||
if h < 32 or w < 64:
|
||||
return None
|
||||
loc = self.locate(image)
|
||||
bx, by, bw, bh = loc.bbox
|
||||
glyph = self.extract_mask(image, loc) # box-sized, 255 = glyph
|
||||
ys, xs = np.where(glyph > 0)
|
||||
box: tuple[int, int, int, int] | None = None
|
||||
if self.config.detect_frontend == "gray" and self.detect(image).detected:
|
||||
# The gray front-end exists for marks the top-hat under-segments, so the
|
||||
# binary blob is NOT authoritative here: trusting it first bounded the
|
||||
# fill by a PARTIAL blob (the faint head glyphs dropped out) and left the
|
||||
# leftmost "Runni" of "RunningHub AI生成" unremoved (2026-07-22). Use the
|
||||
# detector's own best-match box, same as the tophat faint path below.
|
||||
_, box = self._gray_best(image, loc)
|
||||
elif self.config.detect_frontend == "contrast" and self.detect(image).detected:
|
||||
# A dark-on-light Yuanbao mark has no WHITE top-hat blob at all. Bound
|
||||
# the fill by the polarity-independent detector's own match box.
|
||||
_, box = self._contrast_best(image, loc)
|
||||
elif xs.size >= self._MIN_GLYPH_PIXELS:
|
||||
box = (int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max()))
|
||||
elif self.config.detect_frontend == "tophat" and self.detect(image).detected:
|
||||
# A mark found only by the CONTINUOUS front-end has no binary glyph blob to
|
||||
# bound, so the mask came back empty and removal was a silent no-op while
|
||||
# `identify` still reported the mark while removal left it untouched.
|
||||
# Use the DETECTOR'S OWN best-match box: the correlation already located the
|
||||
# mark at a position and scale, and thresholding the response was a strictly
|
||||
# worse proxy for that. An earlier fix thresholded the max-normalized uint8
|
||||
# response at 0.5 -- which selects every non-zero pixel, not "half the peak" as
|
||||
# its comment claimed -- and filled ~120% of the corner box on textured frames
|
||||
# (measured: whole corner vs 58.7% for the match box, both detector-clean).
|
||||
# Gated on an actual detection: on a clean corner the box would be spurious.
|
||||
_, box = self._tophat_best(image, loc)
|
||||
if box is not None:
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
pad = max(4, int(0.10 * bh))
|
||||
rx1 = max(0, bx + gx0 - pad)
|
||||
rx2 = min(w, bx + gx1 + 1 + pad)
|
||||
ry1 = max(0, by + gy0 - pad)
|
||||
ry2 = min(h, by + gy1 + 1 + pad)
|
||||
elif force:
|
||||
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
|
||||
else:
|
||||
rect = self._footprint_rect(image, loc, force=force, detection=detection)
|
||||
if rect is None:
|
||||
return None
|
||||
rx1, ry1, rx2, ry2 = rect
|
||||
if rx1 >= rx2 or ry1 >= ry2:
|
||||
return None
|
||||
# Rectangular footprint + dilation is exactly region_eraser.boxes_to_mask (the
|
||||
@@ -718,5 +830,5 @@ class TextMarkEngine:
|
||||
# zeros/fill/MORPH_ELLIPSE-dilate here.
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * bw))
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * loc.w))
|
||||
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
|
||||
+518
-13
@@ -16,11 +16,14 @@ Imports stay lazy (inside the functions), so ``import remove_ai_watermarks`` is
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import suppress
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import TYPE_CHECKING, Any, Literal
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
from remove_ai_watermarks.watermark_registry import Backend, Sensitivity
|
||||
@@ -40,8 +43,17 @@ def visible_provenance(source: str | Path) -> frozenset[str]:
|
||||
"""Vendor keys that the file's local metadata confirms, the evidence that drives
|
||||
the ``auto`` sensitivity (relaxing a corroborated mark's detection trust gate).
|
||||
|
||||
Mapping: a Google/Gemini C2PA issuer -> ``"gemini"``; a China-AIGC (TC260) label
|
||||
-> ``"doubao"``/``"jimeng"``; a ``samsung_genai`` marker -> ``"samsung"``.
|
||||
Mapping: a Google/Gemini C2PA issuer -> ``"gemini"``; a ``samsung_genai`` marker ->
|
||||
``"samsung"``; a China-AIGC (TC260) label -> the vendor its ``ContentProducer``
|
||||
names (``KnownMark.tc260_producer_codes``), falling back to ByteDance's two products
|
||||
when the producer is absent or unmapped.
|
||||
|
||||
The TC260 label itself says only "this is AI", so it used to relax Doubao and
|
||||
Jimeng on EVERY China-AIGC image -- including one carrying a Qwen or Kling mark,
|
||||
where relaxing the wrong pair is pure false-positive risk and the mark actually
|
||||
present never reached the relaxed gate its own ``provenance_ncc_factor`` was
|
||||
calibrated for. The producer code identifies the signing entity, so it can.
|
||||
|
||||
Best-effort: any read error yields an empty set (no relaxation). Metadata-only, so
|
||||
it never loads cv2/torch.
|
||||
"""
|
||||
@@ -52,19 +64,33 @@ def visible_provenance(source: str | Path) -> frozenset[str]:
|
||||
from remove_ai_watermarks import identify
|
||||
|
||||
rep = identify.identify(path, check_visible=False, check_invisible=False)
|
||||
signal_names = {signal.name for signal in rep.signals}
|
||||
keys: set[str] = set()
|
||||
platform = (rep.platform or "").lower()
|
||||
if "google" in platform or "gemini" in platform:
|
||||
keys.add("gemini")
|
||||
if "aigc" in signal_names:
|
||||
keys |= {"doubao", "jimeng"}
|
||||
if "samsung_genai" in signal_names:
|
||||
keys.add("samsung")
|
||||
return frozenset(keys)
|
||||
return _provenance_from_report(rep, path)
|
||||
return frozenset()
|
||||
|
||||
|
||||
def _tc260_vendors(path: Path) -> frozenset[str]:
|
||||
"""Vendor keys a TC260 label confirms, from its ``ContentProducer`` identity.
|
||||
|
||||
An absent, unreadable or unmapped producer falls back to the historical pair rather
|
||||
than to nothing: the caller has already established that the AIGC signal fired, so
|
||||
the image IS China-AIGC labelled, and dropping to no relaxation would lose the
|
||||
detections the fallback recovers today. The re-read is deliberately isolated -- a
|
||||
failure here must narrow the answer, never discard the rest of the provenance.
|
||||
"""
|
||||
import contextlib
|
||||
|
||||
from remove_ai_watermarks._internal.constants import TC260_FALLBACK_VENDORS
|
||||
|
||||
with contextlib.suppress(Exception):
|
||||
from remove_ai_watermarks.metadata import aigc_label, uscc_of
|
||||
from remove_ai_watermarks.watermark_registry import tc260_producer_vendors
|
||||
|
||||
producer = (aigc_label(path) or {}).get("ContentProducer", "")
|
||||
if producer and (vendor := tc260_producer_vendors().get(uscc_of(producer))):
|
||||
return frozenset({vendor})
|
||||
return TC260_FALLBACK_VENDORS
|
||||
|
||||
|
||||
def _load_visible_input(source: str | Path | NDArray[Any]) -> _VisibleInput:
|
||||
"""Normalize a path/array source without making the public operation stateful."""
|
||||
if not isinstance(source, (str, Path)):
|
||||
@@ -177,3 +203,482 @@ def remove_visible(
|
||||
write_noop=write_noop,
|
||||
)
|
||||
return result, removed
|
||||
|
||||
|
||||
# ── The three-stage image pipeline (visible -> invisible -> metadata) ──
|
||||
# This is the `all` / `batch` pipeline. It lived only in cli.py, written twice with
|
||||
# divergences, so a library caller could not run the flagship path at all. The CLI is
|
||||
# now a thin wrapper: it builds the options, prints the stage lines through
|
||||
# `progress`, and turns the outcome into console text and an exit code.
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class InvisibleOptions:
|
||||
"""The invisible stage's knobs, as one value instead of a dozen parameters.
|
||||
|
||||
ENGINE KNOBS ONLY, under the engine's own names and defaults, so a bare
|
||||
``InvisibleOptions()`` behaves exactly like calling the engine with no arguments.
|
||||
The engine takes them across TWO callables -- ``__init__`` for the ones that shape
|
||||
the loaded stack, ``remove_watermark`` for the per-image ones -- so this is not a
|
||||
splat-through bag; ``_run_invisible`` forwards each field to the right one.
|
||||
``TestInvisibleOptionsMirrorTheEngine`` compares the signatures field by field with
|
||||
no exception table to maintain: a decision made before the engine runs, like
|
||||
``force``, is a parameter of ``remove_all`` next to ``backend`` and ``sensitivity``,
|
||||
not a knob smuggled in here.
|
||||
|
||||
Immutable so a batch can build it once and reuse it across every image while the
|
||||
engine itself is cached separately.
|
||||
"""
|
||||
|
||||
strength: float | None = None
|
||||
pipeline: str = "qwen-zimage"
|
||||
seed: int | None = None
|
||||
hf_token: str | None = None
|
||||
humanize: float = 0.0
|
||||
unsharp: float = 0.0
|
||||
adaptive_polish: bool | None = None
|
||||
max_resolution: int = 0
|
||||
controlnet_conditioning_scale: float = 1.0
|
||||
cpu_offload: bool = False
|
||||
tile: bool = False
|
||||
tile_size: int = 1024
|
||||
tile_overlap: int = 128
|
||||
text_manifest: Path | None = None
|
||||
fidelity_anchor: bool = False
|
||||
|
||||
|
||||
# What the invisible stage did. "unavailable" is the one outcome the caller must
|
||||
# surface loudly: the file looks processed but still carries the watermark.
|
||||
InvisibleOutcome = Literal["removed", "no-signal", "unavailable"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RemoveAllResult:
|
||||
"""Outcome of :func:`remove_all` -- what each stage actually did."""
|
||||
|
||||
output: Path
|
||||
visible_label: str | None # the removed mark(s), or None when nothing fired
|
||||
invisible: InvisibleOutcome
|
||||
|
||||
|
||||
class MetadataStripIncomplete(RuntimeError):
|
||||
"""AI metadata survived the strip, so no output was produced.
|
||||
|
||||
Raised BEFORE the final write, deliberately: the contract is that an incomplete
|
||||
strip leaves nothing on disk rather than an AI-readable file the caller might ship.
|
||||
"""
|
||||
|
||||
def __init__(self, surviving: set[str]) -> None:
|
||||
self.surviving = surviving
|
||||
super().__init__(f"AI metadata survived the strip: {', '.join(sorted(surviving))}")
|
||||
|
||||
|
||||
class _SourceEvidence:
|
||||
"""One metadata extraction per source file, serving every stage of one call.
|
||||
|
||||
``remove_all`` asks the same file two provenance questions -- which vendor the
|
||||
metadata confirms (for the visible pass) and whether an invisible target exists
|
||||
(for the scrub gate). Both start from the same file-backed extraction, and running
|
||||
them independently paid for it twice.
|
||||
|
||||
Per-CALL only, never module-level: the pipeline REWRITES its input in place on some
|
||||
paths (``batch`` with the output directory equal to the input), so a holder that
|
||||
outlived one call would answer from pre-write evidence. The individual metadata
|
||||
probes are memoized on content, which is what makes that safe -- this holder only
|
||||
removes the remaining assembly work.
|
||||
|
||||
Every accessor fails safe exactly as the function it replaces does: an extraction or
|
||||
verdict error yields no provenance (no relaxation) and an invisible target of True
|
||||
(scrub rather than skip).
|
||||
"""
|
||||
|
||||
__slots__ = ("_evidence", "_extracted", "_path")
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
self._path = path
|
||||
self._extracted = False
|
||||
self._evidence: Any | None = None
|
||||
|
||||
def _extract(self) -> Any | None:
|
||||
if not self._extracted:
|
||||
self._extracted = True
|
||||
with suppress(Exception):
|
||||
from remove_ai_watermarks.identify import extract_provenance_evidence
|
||||
|
||||
self._evidence = extract_provenance_evidence(self._path)
|
||||
return self._evidence
|
||||
|
||||
def visible_provenance(self) -> frozenset[str]:
|
||||
"""Vendor keys the metadata confirms; empty on any failure (no relaxation)."""
|
||||
evidence = self._extract()
|
||||
if evidence is None:
|
||||
return frozenset()
|
||||
# The suppress spans the VERDICT and the mapping, not just the extraction --
|
||||
# `visible_provenance` fails safe as a whole, and a raise here would escape as a
|
||||
# traceback where the old code returned an empty set.
|
||||
with suppress(Exception):
|
||||
from remove_ai_watermarks.identify import identify_from_evidence
|
||||
|
||||
report = identify_from_evidence(evidence)
|
||||
return _provenance_from_report(report, self._path)
|
||||
return frozenset()
|
||||
|
||||
def has_invisible_target(self) -> bool:
|
||||
"""Whether a diffusion target exists. True on any failure -- see the fail-safe
|
||||
note on ``identify.has_invisible_target``: leaving a watermark on a paid removal
|
||||
is worse than over-regenerating a clean image."""
|
||||
evidence = self._extract()
|
||||
if evidence is None:
|
||||
return True
|
||||
try:
|
||||
from remove_ai_watermarks._internal.c2pa import c2pa_info_has_removal_hint
|
||||
from remove_ai_watermarks.identify import identify_from_evidence
|
||||
|
||||
report = identify_from_evidence(evidence, image_path=self._path, check_invisible=True)
|
||||
except Exception:
|
||||
return True
|
||||
return report.ai_from_metadata or c2pa_info_has_removal_hint(evidence.c2pa_info)
|
||||
|
||||
|
||||
def _provenance_from_report(report: Any, path: Path) -> frozenset[str]:
|
||||
"""Map an already-built report to the vendor keys it confirms.
|
||||
|
||||
Shared by :func:`visible_provenance` and the evidence holder so the two cannot
|
||||
drift; which evidence confirms which mark is data on the registry row.
|
||||
"""
|
||||
from remove_ai_watermarks.watermark_registry import known_marks
|
||||
|
||||
signal_names = {signal.name for signal in report.signals}
|
||||
platform = (report.platform or "").lower()
|
||||
keys: set[str] = set()
|
||||
aigc = False
|
||||
for mark in known_marks():
|
||||
if any(token in platform for token in mark.provenance_platform_tokens):
|
||||
keys.add(mark.key)
|
||||
if "aigc" in mark.provenance_signals and "aigc" in signal_names:
|
||||
aigc = True
|
||||
elif any(name in signal_names for name in mark.provenance_signals):
|
||||
keys.add(mark.key)
|
||||
if aigc:
|
||||
# The TC260 label is vendor-agnostic: which of its marks it confirms comes from
|
||||
# the producer identity, not from the signal firing.
|
||||
keys |= _tc260_vendors(path)
|
||||
return frozenset(keys)
|
||||
|
||||
|
||||
def remove_all(
|
||||
source: str | Path,
|
||||
output: str | Path,
|
||||
*,
|
||||
backend: Backend = "auto",
|
||||
sensitivity: Sensitivity = "auto",
|
||||
invisible: InvisibleOptions | None = None,
|
||||
force: bool = False,
|
||||
engine: Any | None = None,
|
||||
progress: Callable[[str, str], None] | None = None,
|
||||
) -> RemoveAllResult:
|
||||
"""Remove visible marks, the invisible watermark, and AI metadata, in that order.
|
||||
|
||||
Stages are chained through a file in the SYSTEM temp dir, not next to ``output``:
|
||||
the point of staging is that the user never sees a partial output file during a long
|
||||
model download, and writing the partial next to the final defeats that.
|
||||
|
||||
``force`` scrubs even when no invisible watermark is locally detectable. It sits
|
||||
here rather than in ``InvisibleOptions`` because it decides WHETHER the engine runs,
|
||||
which is settled before the engine is built; the options carry only what the engine
|
||||
itself takes.
|
||||
|
||||
``engine`` accepts an already-constructed ``InvisibleEngine`` so a batch can build
|
||||
the model once; leave it None to construct one per call.
|
||||
|
||||
``progress`` receives ``(stage, detail)`` per step -- ``stage`` is one of
|
||||
``visible`` / ``invisible`` / ``metadata`` and ``detail`` is a stable token, not
|
||||
prose: the caller owns the wording. The invisible stage reports its
|
||||
:data:`InvisibleOutcome`, plus a ``strength=<value>`` line before it runs.
|
||||
|
||||
Raises :class:`MetadataStripIncomplete` before writing anything when AI metadata
|
||||
survives, and ``OSError`` when the output cannot be written.
|
||||
"""
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
from remove_ai_watermarks import image_io, watermark_registry
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
|
||||
def say(stage: str, detail: str) -> None:
|
||||
if progress is not None:
|
||||
progress(stage, detail)
|
||||
|
||||
opts = invisible if invisible is not None else InvisibleOptions()
|
||||
src, out = Path(source), Path(output)
|
||||
watermark_registry.validate_sensitivity(sensitivity)
|
||||
|
||||
image, alpha = image_io.read_bgr_and_alpha(src)
|
||||
if image is None:
|
||||
raise ValueError(f"Could not read image: {src}")
|
||||
|
||||
# One metadata extraction for the whole pipeline: the visible pass asks which
|
||||
# vendor is confirmed and the scrub gate asks whether a target exists, and both
|
||||
# start from the same evidence.
|
||||
evidence = _SourceEvidence(src)
|
||||
|
||||
tmp_fd, tmp_name = tempfile.mkstemp(suffix=src.suffix)
|
||||
os.close(tmp_fd)
|
||||
staged = Path(tmp_name)
|
||||
try:
|
||||
# ── 1. Visible marks ──
|
||||
result, removed = watermark_registry.remove_auto_marks(
|
||||
image,
|
||||
sensitivity=sensitivity,
|
||||
provenance=evidence.visible_provenance(),
|
||||
backend=backend,
|
||||
)
|
||||
visible_label = ", ".join(removed) if removed else None
|
||||
say("visible", visible_label or "")
|
||||
if not image_io.write_bgr_with_alpha(staged, result, alpha):
|
||||
raise OSError(f"failed to write the staged intermediate: {staged}")
|
||||
|
||||
# ── 2. Invisible watermark ──
|
||||
outcome = _run_invisible(src, staged, staged, opts, engine, say, evidence, force)
|
||||
|
||||
# ── 3. AI metadata ──
|
||||
# Read the pristine ORIGINAL for provenance above and the STAGED file here:
|
||||
# the visible pass has already dropped this file's C2PA.
|
||||
_, leftover = strip_and_verify(staged, staged)
|
||||
if leftover:
|
||||
# Before the write, on purpose -- see MetadataStripIncomplete.
|
||||
raise MetadataStripIncomplete(set(leftover))
|
||||
say("metadata", "stripped")
|
||||
|
||||
# The invisible stage (and the cv2.IMREAD_COLOR paths under it) drops alpha, so
|
||||
# re-attach the ORIGINAL alpha plane unchanged for transparent formats.
|
||||
final_bgr, _ = image_io.read_bgr_and_alpha(staged)
|
||||
if final_bgr is None:
|
||||
raise OSError(f"failed to read back the staged intermediate: {staged}")
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not image_io.write_bgr_with_alpha(out, final_bgr, alpha):
|
||||
raise OSError(f"failed to write output (is the destination writable?): {out}")
|
||||
finally:
|
||||
if staged.exists():
|
||||
staged.unlink()
|
||||
return RemoveAllResult(out, visible_label, outcome)
|
||||
|
||||
|
||||
def _run_invisible(
|
||||
vendor_source: Path,
|
||||
in_path: Path,
|
||||
out_path: Path,
|
||||
opts: InvisibleOptions,
|
||||
engine: Any | None,
|
||||
say: Callable[[str, str], None],
|
||||
evidence: _SourceEvidence,
|
||||
force: bool,
|
||||
) -> InvisibleOutcome:
|
||||
"""Run, or deliberately skip, the diffusion scrub.
|
||||
|
||||
``vendor_source`` is the PRISTINE original: the staged/output file has already lost
|
||||
its C2PA to the visible pass, so gating or resolving the vendor from it would always
|
||||
read as "no signal" and "unknown vendor". ``in_path``/``out_path`` are what the
|
||||
engine reads and writes (the same staged file for ``remove_all``, input->output for
|
||||
an invisible-only batch).
|
||||
"""
|
||||
from remove_ai_watermarks.invisible_engine import is_available
|
||||
|
||||
if not is_available():
|
||||
say("invisible", "unavailable")
|
||||
return "unavailable"
|
||||
if not (force or evidence.has_invisible_target()):
|
||||
say("invisible", "no-signal")
|
||||
return "no-signal"
|
||||
|
||||
from remove_ai_watermarks._internal.watermark_profiles import resolve_strength, vendor_for_strength
|
||||
|
||||
vendor = vendor_for_strength(vendor_source)
|
||||
# Report the strength the engine will actually execute, resolved the same way it
|
||||
# resolves it, so the reported value cannot drift from the executed one.
|
||||
with suppress(Exception):
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(vendor_source) as probe:
|
||||
say("invisible", f"strength={resolve_strength(opts.strength, vendor, opts.pipeline, size=probe.size)}")
|
||||
|
||||
if engine is None:
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine
|
||||
|
||||
engine = InvisibleEngine(
|
||||
pipeline=opts.pipeline,
|
||||
hf_token=opts.hf_token,
|
||||
progress_callback=lambda message: say("invisible", message),
|
||||
controlnet_conditioning_scale=opts.controlnet_conditioning_scale,
|
||||
cpu_offload=opts.cpu_offload,
|
||||
)
|
||||
engine.remove_watermark(
|
||||
image_path=in_path,
|
||||
output_path=out_path,
|
||||
strength=opts.strength,
|
||||
seed=opts.seed,
|
||||
humanize=opts.humanize,
|
||||
unsharp=opts.unsharp,
|
||||
adaptive_polish=opts.adaptive_polish,
|
||||
max_resolution=opts.max_resolution,
|
||||
vendor=vendor,
|
||||
tile=opts.tile,
|
||||
tile_size=opts.tile_size,
|
||||
tile_overlap=opts.tile_overlap,
|
||||
text_manifest=opts.text_manifest,
|
||||
fidelity_anchor=opts.fidelity_anchor,
|
||||
)
|
||||
say("invisible", "removed")
|
||||
return "removed"
|
||||
|
||||
|
||||
BatchMode = Literal["all", "visible", "metadata", "invisible"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BatchSummary:
|
||||
"""Per-directory outcome of :func:`remove_batch`."""
|
||||
|
||||
processed: int
|
||||
failed: int
|
||||
# Files whose invisible watermark was left in place because the GPU extra is
|
||||
# missing. Non-empty means the outputs LOOK processed but still carry it.
|
||||
invisible_unavailable: list[Path]
|
||||
errors: list[tuple[Path, str]]
|
||||
|
||||
|
||||
def remove_batch(
|
||||
directory: str | Path,
|
||||
output_dir: str | Path,
|
||||
*,
|
||||
mode: BatchMode = "all",
|
||||
backend: Backend = "auto",
|
||||
sensitivity: Sensitivity = "auto",
|
||||
invisible: InvisibleOptions | None = None,
|
||||
force: bool = False,
|
||||
engine: Any | None = None,
|
||||
progress: Callable[[Path, str, str], None] | None = None,
|
||||
) -> BatchSummary:
|
||||
"""Run one removal ``mode`` over every supported image in ``directory``.
|
||||
|
||||
Never raises for a single bad image: a per-file failure is counted and recorded in
|
||||
``BatchSummary.errors`` so one unreadable file cannot abandon the rest of the
|
||||
directory. ``force`` and ``engine`` are threaded straight through, so a caller that
|
||||
passes a constructed ``InvisibleEngine`` loads the model once for the whole run.
|
||||
|
||||
``progress`` receives ``(path, stage, detail)``. Every image ends with exactly one
|
||||
terminal stage -- ``done`` or ``failed`` -- whatever the mode does in between, so a
|
||||
caller driving a progress bar can advance on that alone.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.utils import is_supported_format
|
||||
|
||||
src_dir, out_dir = Path(directory), Path(output_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def say(path: Path, stage: str, detail: str) -> None:
|
||||
if progress is not None:
|
||||
progress(path, stage, detail)
|
||||
|
||||
processed = failed = 0
|
||||
unavailable: list[Path] = []
|
||||
errors: list[tuple[Path, str]] = []
|
||||
for img_path in sorted(p for p in src_dir.iterdir() if is_supported_format(p)):
|
||||
out_path = out_dir / img_path.name
|
||||
try:
|
||||
outcome = _run_batch_one(img_path, out_path, mode, backend, sensitivity, invisible, force, engine, say)
|
||||
except Exception as exc:
|
||||
failed += 1
|
||||
errors.append((img_path, str(exc)))
|
||||
say(img_path, "failed", str(exc))
|
||||
continue
|
||||
processed += 1
|
||||
if outcome == "unavailable":
|
||||
unavailable.append(img_path)
|
||||
# Exactly one terminal event per image, in every mode: a caller's progress bar
|
||||
# advances on this and nothing else. Keying it off a mode-specific stage line
|
||||
# left `visible` and `metadata` runs sitting at 0% for the whole batch.
|
||||
say(img_path, "done", outcome or "")
|
||||
return BatchSummary(processed, failed, unavailable, errors)
|
||||
|
||||
|
||||
def _run_batch_one(
|
||||
img_path: Path,
|
||||
out_path: Path,
|
||||
mode: BatchMode,
|
||||
backend: Backend,
|
||||
sensitivity: Sensitivity,
|
||||
invisible: InvisibleOptions | None,
|
||||
force: bool,
|
||||
engine: Any | None,
|
||||
say: Callable[[Path, str, str], None],
|
||||
) -> InvisibleOutcome | None:
|
||||
"""One image, one mode. Returns the invisible outcome when that stage ran."""
|
||||
from remove_ai_watermarks import image_io
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
|
||||
if mode == "all":
|
||||
result = remove_all(
|
||||
img_path,
|
||||
out_path,
|
||||
backend=backend,
|
||||
sensitivity=sensitivity,
|
||||
invisible=invisible,
|
||||
force=force,
|
||||
engine=engine,
|
||||
progress=lambda stage, detail: say(img_path, stage, detail),
|
||||
)
|
||||
return result.invisible
|
||||
|
||||
# NOTE on holders in batch: `out_path` may EQUAL `img_path` (nothing forbids
|
||||
# `-o <the input directory>`), and the visible stage rewrites it. So each stage that
|
||||
# asks the file a provenance question builds its OWN holder, after any write that
|
||||
# precedes it. One holder spanning the write would answer the invisible gate from
|
||||
# pre-write evidence and scrub a file that today is correctly skipped.
|
||||
if mode == "visible":
|
||||
# Deliberately NOT `remove_visible`: its no-op branch copies the original bytes
|
||||
# through when nothing was removed, which is right for a single lossless call
|
||||
# and wrong here. A batch must produce every output through the one writer, so a
|
||||
# failed write RAISES and the run is counted and exits non-zero -- a read-only
|
||||
# output directory once produced zero files and still exited 0 (Tier E).
|
||||
# Always read the ORIGINAL: a stale out_path from a previous run must not be
|
||||
# re-processed as if it were the input.
|
||||
from remove_ai_watermarks import watermark_registry
|
||||
|
||||
image, alpha = image_io.read_bgr_and_alpha(img_path)
|
||||
if image is None:
|
||||
raise ValueError(f"Could not read image: {img_path}")
|
||||
result, _ = watermark_registry.remove_auto_marks(
|
||||
image,
|
||||
sensitivity=sensitivity,
|
||||
provenance=visible_provenance(img_path),
|
||||
backend=backend,
|
||||
)
|
||||
if not image_io.write_bgr_with_alpha(out_path, result, alpha):
|
||||
raise OSError(f"failed to write output (is the destination writable?): {out_path}")
|
||||
return None
|
||||
|
||||
if mode == "metadata":
|
||||
_, leftover = strip_and_verify(img_path, out_path)
|
||||
if leftover:
|
||||
raise MetadataStripIncomplete(set(leftover))
|
||||
return None
|
||||
|
||||
# invisible-only: no preceding visible pass, so out_path does not exist yet, and
|
||||
# the input IS the pristine original for both the gate and the vendor probe.
|
||||
outcome = _run_invisible(
|
||||
img_path,
|
||||
img_path,
|
||||
out_path,
|
||||
invisible if invisible is not None else InvisibleOptions(),
|
||||
engine,
|
||||
lambda stage, detail: say(img_path, stage, detail),
|
||||
_SourceEvidence(img_path),
|
||||
force,
|
||||
)
|
||||
if not out_path.exists():
|
||||
# Keep the output directory COMPLETE even when the pixels are deliberately
|
||||
# left alone; a hole the caller cannot see is worse than an unchanged copy.
|
||||
src_bgr, src_alpha = image_io.read_bgr_and_alpha(img_path)
|
||||
if src_bgr is None or not image_io.write_bgr_with_alpha(out_path, src_bgr, src_alpha):
|
||||
raise OSError(f"failed to copy input through to output: {out_path}")
|
||||
return outcome
|
||||
@@ -30,11 +30,14 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkLocation,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image SHORT side (measured basis). The
|
||||
@@ -100,79 +103,51 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
BaiduDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Baidu alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
"""Binary "百度" silhouette (255 = glyph) from the alpha map, or None."""
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the Baidu glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class BaiduEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible Baidu "百度 AI生成" mark (bottom-right; localize -> fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
def footprint_mask(
|
||||
self, image: NDArray[Any] | None, *, force: bool = False, dilate: int | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame mask of the WHOLE mark (text run + the pill tag to its right).
|
||||
def _footprint_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Bound the fill by the detector's match box, never by the binary glyph blob.
|
||||
|
||||
The base class's blob-bbox footprint UNDERCOVERS this mark: the white tag's
|
||||
flat interior gives no top-hat response (a top-hat answers edges, not flats),
|
||||
so the blob ends at the text run and the fill leaves the tag's right half as
|
||||
a ghost (measured 2026-07-22 on the 768x1024 cohort frame: blob bbox x
|
||||
632..746 vs the tag ending ~758). The layout is measured and fixed -- the
|
||||
text run is at the left of the locate box, the tag runs to the corner -- so
|
||||
the footprint is the detector's match box extended RIGHT to the corner.
|
||||
632..746 vs the tag ending ~758).
|
||||
"""
|
||||
if image is None or image.size == 0:
|
||||
return None
|
||||
return self._match_box_rect(image, loc, force=force, detection=detection)
|
||||
|
||||
from remove_ai_watermarks import image_io, region_eraser
|
||||
def _extend_match_box(
|
||||
self, box: tuple[int, int, int, int], loc: TextMarkLocation, frame: tuple[int, int]
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Extend the match box RIGHT to the corner end of the locate box.
|
||||
|
||||
image = image_io.to_bgr(image)
|
||||
h, w = image.shape[:2]
|
||||
if h < 32 or w < 64:
|
||||
return None
|
||||
loc = self.locate(image)
|
||||
The layout is measured and fixed: the text run is at the left of the locate
|
||||
box and the tag runs to the corner, so the mark's right edge is the box's.
|
||||
"""
|
||||
gx0, gy0, _gx1, gy1 = box
|
||||
bx, by, bw, bh = loc.bbox
|
||||
if force:
|
||||
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
|
||||
else:
|
||||
if not self.detect(image).detected:
|
||||
return None
|
||||
_, box = self._tophat_best(image, loc)
|
||||
if box is None:
|
||||
return None
|
||||
gx0, gy0, _gx1, gy1 = box
|
||||
pad = max(4, int(0.15 * bh))
|
||||
rx1 = max(0, bx + gx0 - pad)
|
||||
ry1 = max(0, by + gy0 - pad)
|
||||
rx2 = min(w, bx + bw) # the tag runs to the corner end of the box
|
||||
ry2 = min(h, by + gy1 + 1 + pad)
|
||||
if rx1 >= rx2 or ry1 >= ry2:
|
||||
return None
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * bw))
|
||||
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
h, w = frame
|
||||
pad = max(4, int(0.15 * bh))
|
||||
return (
|
||||
max(0, bx + gx0 - pad),
|
||||
max(0, by + gy0 - pad),
|
||||
min(w, bx + bw), # the tag runs to the corner end of the box
|
||||
min(h, by + gy1 + 1 + pad),
|
||||
)
|
||||
+698
-704
File diff suppressed because it is too large.
Load diff
@@ -23,11 +23,9 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of image WIDTH (the mark scales with width, anchored
|
||||
@@ -96,9 +94,6 @@ _CONFIG = TextMarkConfig(
|
||||
min_gw=8,
|
||||
)
|
||||
|
||||
# Doubao-specific aliases for the shared detection result/engine.
|
||||
DoubaoDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Doubao alpha template (float [0,1]), or None."""
|
||||
@@ -120,13 +115,3 @@ class DoubaoEngine(TextMarkEngine):
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
@@ -1,7 +1,11 @@
|
||||
"""DWT-DCT decoder compatible with invisible-watermark's ``dwtDct`` path.
|
||||
|
||||
Derived from ShieldMnt/invisible-watermark ``imwatermark/maxDct.py`` (MIT),
|
||||
trimmed to the matrix path used by Stable Diffusion, SDXL, and FLUX.
|
||||
trimmed to the matrix path used by Stable Diffusion, SDXL, and FLUX. The block
|
||||
scan is vectorized rather than transcribed, so the file no longer reads line by
|
||||
line against upstream; what it preserves is the output, bit for bit. See
|
||||
[`docs/module-internals.md`](../../docs/module-internals.md) for the
|
||||
measurements and for why a faster hand-rolled transform is not available.
|
||||
|
||||
Copyright (c) 2021 ShieldMnt
|
||||
|
||||
@@ -23,7 +27,39 @@ if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
|
||||
_DEFAULT_SCALES = (0, 36, 36)
|
||||
_DEFAULT_BLOCK = 4
|
||||
# Fixed by the format being decoded, and the 4-way fold chains in `_frame_bits`
|
||||
# are written for it. It was a constructor parameter while the block scan was a
|
||||
# generic Python loop; nothing ever passed another value, and a knob that now
|
||||
# silently returns wrong bits is worse than no knob.
|
||||
_BLOCK = 4
|
||||
|
||||
# Block-rows of the approximation band handled per strip. Not a tuned value:
|
||||
# every height measured landed inside the others' noise. What matters is strips
|
||||
# at all rather than a full-plane intermediate, not this number.
|
||||
_STRIP = 16
|
||||
|
||||
|
||||
def _approximation(rows: NDArray[Any]) -> NDArray[Any]:
|
||||
"""One Haar pass along the last axis, approximation band only.
|
||||
|
||||
``pywt.dwt(x, "haar", axis=1)[0]`` computes and allocates the detail band as
|
||||
well, and dispatches per row. Flattening lets one ``downcoef`` call do the
|
||||
whole plane, and it is the same numbers in the same order **only while the
|
||||
last axis is even**: Haar's filter is length 2, so an even row length makes
|
||||
every pair fall inside its own row with no boundary extension. On an odd
|
||||
width the pairs walk across row boundaries and the reshape below still
|
||||
succeeds whenever the total is even -- wrong bits, no exception, and the
|
||||
upstream-parity test is `skipif`-gated. Hence the explicit check.
|
||||
|
||||
The transform itself stays inside pywt however slow that looks. Its C
|
||||
convolution contracts into an FMA that no numpy expression reproduces, and
|
||||
the caller's threshold is ``peak % 36 > 18.0`` against values that are exact
|
||||
multiples of 0.5, so a 1-ulp difference flips real bits.
|
||||
"""
|
||||
height, width = rows.shape
|
||||
if width % 2:
|
||||
raise RuntimeError(f"row length {width} is odd; the raveled Haar pass requires an even last axis")
|
||||
return pywt.downcoef("a", rows.ravel(), "haar").reshape(height, width // 2)
|
||||
|
||||
|
||||
class _DecodeMaxDct:
|
||||
@@ -33,52 +69,85 @@ class _DecodeMaxDct:
|
||||
self,
|
||||
wm_lengths: tuple[int, ...],
|
||||
scales: tuple[int, int, int] = _DEFAULT_SCALES,
|
||||
block: int = _DEFAULT_BLOCK,
|
||||
) -> None:
|
||||
self._wm_lengths = wm_lengths
|
||||
self._scales = scales
|
||||
self._block = block
|
||||
|
||||
def decode(self, bgr: NDArray[Any]) -> dict[int, NDArray[Any]]:
|
||||
row, col, _channels = bgr.shape
|
||||
yuv = cv2.cvtColor(bgr, cv2.COLOR_BGR2YUV)
|
||||
trimmed = yuv[: row // 4 * 4, : col // 4 * 4]
|
||||
|
||||
scores_by_length = {wm_len: [[] for _ in range(wm_len)] for wm_len in self._wm_lengths}
|
||||
for channel in range(2):
|
||||
if self._scales[channel] <= 0:
|
||||
continue
|
||||
ca1, _detail = pywt.dwt2(yuv[: row // 4 * 4, : col // 4 * 4, channel], "haar")
|
||||
self._decode_frame(ca1, self._scales[channel], scores_by_length)
|
||||
per_channel = [
|
||||
self._plane_bits(trimmed, channel, self._scales[channel])
|
||||
for channel in range(2)
|
||||
if self._scales[channel] > 0
|
||||
]
|
||||
# Each channel restarts the bit index at 0, so the buckets come from a
|
||||
# per-channel arange rather than one running counter.
|
||||
index = np.concatenate([np.arange(bits.size) for bits in per_channel] or [np.zeros(0, dtype=np.int64)])
|
||||
weights = np.concatenate(per_channel or [np.zeros(0)])
|
||||
|
||||
return {
|
||||
wm_len: np.asarray([float(np.asarray(score).mean()) if score else 0.0 for score in scores]) * 255 > 127
|
||||
for wm_len, scores in scores_by_length.items()
|
||||
}
|
||||
decoded: dict[int, NDArray[Any]] = {}
|
||||
for wm_len in self._wm_lengths:
|
||||
bucket = index % wm_len
|
||||
sums = np.bincount(bucket, weights=weights, minlength=wm_len)
|
||||
counts = np.bincount(bucket, minlength=wm_len)
|
||||
decoded[wm_len] = sums * 255 > counts * 127
|
||||
return decoded
|
||||
|
||||
def _decode_frame(
|
||||
self,
|
||||
frame: NDArray[Any],
|
||||
scale: int,
|
||||
scores_by_length: dict[int, list[list[int]]],
|
||||
) -> None:
|
||||
row, col = frame.shape
|
||||
bit_index = 0
|
||||
for i in range(row // self._block):
|
||||
for j in range(col // self._block):
|
||||
block = frame[
|
||||
i * self._block : i * self._block + self._block,
|
||||
j * self._block : j * self._block + self._block,
|
||||
]
|
||||
inferred = self._infer_bit(block, scale)
|
||||
for wm_len, scores in scores_by_length.items():
|
||||
scores[bit_index % wm_len].append(inferred)
|
||||
bit_index += 1
|
||||
def _plane_bits(self, trimmed: NDArray[Any], channel: int, scale: int) -> NDArray[Any]:
|
||||
"""Block bits for one colour plane, a strip of block-rows at a time.
|
||||
|
||||
def _infer_bit(self, block: NDArray[Any], scale: int) -> int:
|
||||
position = int(np.argmax(np.abs(block.flatten()[1:]))) + 1
|
||||
i, j = position // self._block, position % self._block
|
||||
value = abs(float(block[i][j]))
|
||||
return int((value % scale) > 0.5 * scale)
|
||||
``dwt2`` is ``dwtn``: it transforms along axis 0, then along axis 1 over
|
||||
both halves, and three of the four bands it returns are discarded here.
|
||||
Only the approximation is ever asked for, and transposing between the
|
||||
two passes lets pywt walk a contiguous axis instead of a column -- that
|
||||
access pattern, not the arithmetic saved, is where the time goes.
|
||||
|
||||
Strips mean no full-plane float64 intermediate is ever materialized, and
|
||||
they are seam-free for the reason ``_approximation`` documents: output
|
||||
row ``k`` reads input rows ``2k`` and ``2k + 1`` only. Strips start on
|
||||
multiples of ``2 * _BLOCK``, so neither a pair nor a 4x4 block straddles
|
||||
one.
|
||||
"""
|
||||
if trimmed.shape[0] == 0 or trimmed.shape[1] == 0:
|
||||
# Reachable: a 1x65536 image clears the caller's area check and
|
||||
# trims to an empty plane. Left to dwt2 so the exception stays the
|
||||
# one this module has always raised -- returning empty bits here
|
||||
# instead would silently turn a raise into an all-false verdict.
|
||||
return pywt.dwt2(trimmed[:, :, channel], "haar")[0]
|
||||
rows = trimmed.shape[0] // (2 * _BLOCK)
|
||||
cols = trimmed.shape[1] // (2 * _BLOCK)
|
||||
if rows == 0 or cols == 0:
|
||||
return np.zeros(0, dtype=np.float64)
|
||||
|
||||
width = cols * 2 * _BLOCK
|
||||
pieces: list[NDArray[Any]] = []
|
||||
for start in range(0, rows, _STRIP):
|
||||
stop = min(start + _STRIP, rows)
|
||||
strip = trimmed[start * 2 * _BLOCK : stop * 2 * _BLOCK, :width]
|
||||
columns = cv2.transpose(cv2.extractChannel(strip, channel))
|
||||
band = _approximation(cv2.transpose(_approximation(columns)))
|
||||
pieces.append(self._frame_bits(band, scale))
|
||||
return np.concatenate(pieces)
|
||||
|
||||
def _frame_bits(self, frame: NDArray[Any], scale: int) -> NDArray[Any]:
|
||||
"""One bit per 4x4 block, in row-major block order.
|
||||
|
||||
Upstream's per-block loop, said to numpy once instead of to the
|
||||
interpreter ~135k times per image. Zeroing the DC term in the absolute
|
||||
band says "ignore index 0" without materializing a ``(nblocks, 16)``
|
||||
copy, and the 4-way ``np.maximum`` chains reduce over contiguous rows.
|
||||
"""
|
||||
rows = frame.shape[0] // _BLOCK
|
||||
cols = frame.shape[1] // _BLOCK
|
||||
band = np.abs(frame[: rows * _BLOCK, : cols * _BLOCK])
|
||||
band[::_BLOCK, ::_BLOCK] = 0.0
|
||||
folded = np.maximum(np.maximum(band[0::4], band[1::4]), np.maximum(band[2::4], band[3::4]))
|
||||
folded = folded.reshape(rows * cols, _BLOCK)
|
||||
peak = np.maximum(np.maximum(folded[:, 0], folded[:, 1]), np.maximum(folded[:, 2], folded[:, 3]))
|
||||
return ((peak % scale) > 0.5 * scale).astype(np.float64)
|
||||
|
||||
|
||||
def decode_dwt_dct(bgr: NDArray[Any], wm_len: int) -> NDArray[Any]:
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -1,7 +1,7 @@
|
||||
"""Post-processing filters for the cleaned output.
|
||||
|
||||
``apply_analog_humanizer`` injects film grain and chromatic aberration to defeat
|
||||
digital AI-perfection classifiers (ported from NeuralBleach); ``unsharp_mask``
|
||||
``apply_analog_humanizer`` injects film grain and chromatic aberration to reduce
|
||||
overly uniform digital surfaces; ``unsharp_mask``
|
||||
counters the soft, over-smoothed look that the diffusion pass leaves behind
|
||||
(itself a common "this is AI" tell).
|
||||
"""
|
||||
@@ -16,10 +16,7 @@ from numpy.typing import NDArray
|
||||
|
||||
def apply_analog_humanizer(image: NDArray, grain_intensity: float = 4.0, chromatic_shift: int = 1) -> NDArray:
|
||||
"""
|
||||
Apply Analog Humanizer (film grain and chromatic aberration) to an image.
|
||||
This simulates analog film imperfections to defeat digital AI perfection classifiers.
|
||||
|
||||
Ported from NeuralBleach.
|
||||
Apply shared-luminance grain and a small lateral color offset.
|
||||
|
||||
Args:
|
||||
image: BGR image as numpy array (uint8).
|
||||
@@ -33,26 +30,22 @@ def apply_analog_humanizer(image: NDArray, grain_intensity: float = 4.0, chromat
|
||||
if len(image.shape) != 3 or image.shape[2] != 3:
|
||||
return image.copy()
|
||||
|
||||
# Split channels (OpenCV uses BGR)
|
||||
# B = 0, G = 1, R = 2
|
||||
# Translate the outer color channels without circular edge wrapping.
|
||||
b, g, r = cv2.split(image)
|
||||
|
||||
# 1. Chromatic Aberration
|
||||
# Shift R channel left, B channel right. np.roll is circular, so it wraps
|
||||
# the opposite edge into a thin colored fringe at the L/R borders; replicate
|
||||
# the original edge columns there to keep the intended offset interior-only.
|
||||
# Clamp so the edge-replication slices below always have a source column: a shift
|
||||
# >= width would leave them empty and crash the broadcast (r[:, -shift:] = (H, 0)).
|
||||
shift = min(chromatic_shift, image.shape[1] - 1)
|
||||
shift = min(max(0, chromatic_shift), max(0, image.shape[1] - 1))
|
||||
if shift > 0:
|
||||
r = np.roll(r, -shift, axis=1)
|
||||
r[:, -shift:] = r[:, -shift - 1 : -shift]
|
||||
b = np.roll(b, shift, axis=1)
|
||||
b[:, :shift] = b[:, shift : shift + 1]
|
||||
shifted_b = np.empty_like(b)
|
||||
shifted_b[:, :shift] = b[:, :1]
|
||||
shifted_b[:, shift:] = b[:, :-shift]
|
||||
b = shifted_b
|
||||
|
||||
shifted_r = np.empty_like(r)
|
||||
shifted_r[:, :-shift] = r[:, shift:]
|
||||
shifted_r[:, -shift:] = r[:, -1:]
|
||||
r = shifted_r
|
||||
|
||||
merged = cv2.merge((b, g, r))
|
||||
|
||||
# 2. Film Grain (Gaussian Noise)
|
||||
if grain_intensity > 0:
|
||||
img_f = merged.astype(np.float32)
|
||||
noise = np.random.normal(0, grain_intensity, img_f.shape).astype(np.float32)
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -89,6 +89,19 @@ def _pil_read(path: str | Path, flags: int) -> NDArray[Any] | None:
|
||||
return None
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read ``path`` as a BGR ndarray, raising instead of returning ``None``.
|
||||
|
||||
:func:`imread` keeps cv2's ``None``-on-failure contract because the removal paths
|
||||
branch on it. Scripts and tests want the opposite -- fail loudly at the read -- so
|
||||
they call this. Each vendor engine used to carry its own verbatim copy.
|
||||
"""
|
||||
image = imread(path)
|
||||
if image is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return image
|
||||
|
||||
|
||||
def to_bgr(image: NDArray[Any]) -> NDArray[Any]:
|
||||
"""Return a 3-channel BGR view of ``image``, promoting grayscale and BGRA.
|
||||
|
||||
|
||||
@@ -1,10 +1,7 @@
|
||||
"""Invisible watermark removal engine.
|
||||
"""Diffusion engine for regenerating images that carry invisible AI watermarks.
|
||||
|
||||
Wraps the vendored noai-watermark code for removing invisible AI watermarks
|
||||
(SynthID, StableSignature, TreeRing) via diffusion-based regeneration.
|
||||
|
||||
This module requires the 'gpu' extra dependencies:
|
||||
uv pip install 'remove-ai-watermarks[diffusion]'
|
||||
Requires the 'qwen-zimage' extra and a CUDA device:
|
||||
uv pip install 'remove-ai-watermarks[qwen-zimage]'
|
||||
"""
|
||||
|
||||
# cv2/torch boundary: this engine wraps cv2 (resize/imwrite/cvtColor) and the
|
||||
@@ -17,12 +14,13 @@ import logging
|
||||
import os
|
||||
import warnings
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from .noai.watermark_profiles import (
|
||||
DEFAULT_MODEL_ID as DEFAULT_SDXL_MODEL_ID,
|
||||
)
|
||||
from .noai.watermark_profiles import (
|
||||
from ._internal.watermark_profiles import (
|
||||
DEFAULT_PROFILE,
|
||||
QWEN_ZIMAGE_PROFILE,
|
||||
REMOVAL_MODULES,
|
||||
resolve_adaptive_polish,
|
||||
resolve_seed,
|
||||
)
|
||||
|
||||
@@ -43,27 +41,30 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
"""Check if invisible watermark removal dependencies are installed."""
|
||||
"""Whether the dependencies for a real removal run are installed.
|
||||
|
||||
Shares :data:`REMOVAL_MODULES` with the remover's own precondition so the two
|
||||
cannot drift. When they did, a torch+diffusers-only environment passed this gate
|
||||
and then died at the DiffSynth face stage.
|
||||
"""
|
||||
from .optional_deps import module_available
|
||||
|
||||
return module_available("diffusers", "torch")
|
||||
return module_available(*REMOVAL_MODULES)
|
||||
|
||||
|
||||
def _target_size(width: int, height: int, max_resolution: int, min_resolution: int = 0) -> tuple[int, int] | None:
|
||||
def _target_size(width: int, height: int, max_resolution: int) -> tuple[int, int] | None:
|
||||
"""Compute the (width, height) to process at, or None for native.
|
||||
|
||||
Two opposite long-side adjustments, in precedence order:
|
||||
One long-side adjustment: if it exceeds ``max_resolution``, scale DOWN to it
|
||||
(integer-truncated, matching the PIL ``resize`` call site). 0/negative = no cap.
|
||||
Set only to bound GPU/MPS memory on very large inputs (issue #10).
|
||||
|
||||
- ``max_resolution`` (cap): if the long side exceeds it, scale DOWN to it
|
||||
(integer-truncated, matching the PIL ``resize`` call site). 0/negative = no
|
||||
cap. Set only to bound GPU/MPS memory on very large inputs (issue #10).
|
||||
- ``min_resolution`` (floor): else if the long side is below it, scale UP to it
|
||||
(rounded) so SDXL img2img runs near its ~1024 training resolution instead of
|
||||
degrading on a tiny latent (a 381x512 portrait distorts badly at native).
|
||||
The output is restored to the original size by the caller, so the floor is a
|
||||
transparent quality boost. 0 = no floor. Skipped on a ``min > max`` misconfig.
|
||||
There was also a ``min_resolution`` floor that scaled small inputs UP toward
|
||||
SDXL's ~1024 training size. It went with the SDXL profiles: both surviving
|
||||
profiles run at native geometry, so the floor was forced to 0 on every path and
|
||||
could not fire.
|
||||
|
||||
Returns None when neither applies (native resolution). Pure function so the
|
||||
Returns None when the cap does not apply (native resolution). Pure function so the
|
||||
resolution decision is unit-testable without loading the diffusion model.
|
||||
"""
|
||||
long_side = max(width, height)
|
||||
@@ -72,31 +73,20 @@ def _target_size(width: int, height: int, max_resolution: int, min_resolution: i
|
||||
# Clamp the short side to >=1: extreme aspect ratios (e.g. 5000x3 capped
|
||||
# at 1024) would otherwise truncate it to 0 and crash image.resize().
|
||||
return (max(1, int(width * ratio)), max(1, int(height * ratio)))
|
||||
if min_resolution > 0 and long_side < min_resolution and (max_resolution <= 0 or min_resolution <= max_resolution):
|
||||
ratio = min_resolution / long_side
|
||||
return (max(1, round(width * ratio)), max(1, round(height * ratio)))
|
||||
return None
|
||||
|
||||
|
||||
class InvisibleEngine:
|
||||
"""Remove invisible AI watermarks using diffusion model regeneration.
|
||||
|
||||
Based on noai-watermark by mertizci:
|
||||
https://github.com/mertizci/noai-watermark
|
||||
|
||||
The approach encodes the image into latent space, injects controlled noise
|
||||
to break watermark patterns, and reconstructs via reverse diffusion.
|
||||
"""
|
||||
|
||||
# SDXL base is the default since May 2026; the vendor-adaptive strength
|
||||
# removes the current SynthID (see watermark_profiles + docs/synthid.md).
|
||||
DEFAULT_MODEL_ID = DEFAULT_SDXL_MODEL_ID
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
model_id: str | None = None,
|
||||
device: str | None = None,
|
||||
pipeline: str = "controlnet",
|
||||
pipeline: str = DEFAULT_PROFILE,
|
||||
hf_token: str | None = None,
|
||||
progress_callback: Callable[[str], None] | None = None,
|
||||
controlnet_conditioning_scale: float = 1.0,
|
||||
@@ -105,31 +95,28 @@ class InvisibleEngine:
|
||||
"""Initialize the invisible watermark removal engine.
|
||||
|
||||
Args:
|
||||
model_id: HuggingFace model ID. None = use the SDXL base default.
|
||||
device: Device for inference (auto/cpu/mps/cuda/xpu). None = auto.
|
||||
pipeline: Pipeline profile. "controlnet" (DEFAULT; SDXL + canny ControlNet
|
||||
that preserves text/face structure via edge conditioning while removing
|
||||
SynthID), "sdxl" (plain SDXL img2img, lighter but leaves SynthID on
|
||||
flat-graphic content), or "qwen" (Qwen-Image 20B img2img, best text/
|
||||
structure preservation but CUDA/cloud-class), or "qwen-zimage"
|
||||
(Qwen-Image-2512 Lightning + Canny, then SAM-masked Z-Image face
|
||||
repair; CUDA-only). "default" aliases "sdxl".
|
||||
device: Device for inference. Both profiles are CUDA-only, so the
|
||||
usable values are "cuda" and None/"auto" (which detects it);
|
||||
anything else raises rather than falling back.
|
||||
pipeline: Pipeline profile, one of "qwen-zimage" (DEFAULT;
|
||||
Qwen-Image-2512 Lightning + Canny, then SAM-masked Z-Image face repair)
|
||||
or "sdxl-zimage" (the same recipe and the same face stage on an SDXL
|
||||
global pass, vendor-adaptive strength because an SDXL global stage
|
||||
needs more of it). BOTH ARE CUDA-ONLY -- there is no CPU or MPS path
|
||||
for invisible-watermark removal.
|
||||
hf_token: HuggingFace API token.
|
||||
progress_callback: Optional callback for progress messages.
|
||||
controlnet_conditioning_scale: ControlNet structure-preservation
|
||||
strength (controlnet pipeline only).
|
||||
controlnet_conditioning_scale: Canny ControlNet structure-preservation
|
||||
strength on the global stage of both profiles.
|
||||
cpu_offload: Offload model components to CPU between CUDA calls instead
|
||||
of keeping the whole pipeline in VRAM, at the cost of speed. For
|
||||
qwen-zimage, force the face stack to offload instead of using automatic
|
||||
residency. CUDA only.
|
||||
"""
|
||||
|
||||
from remove_ai_watermarks.noai.watermark_remover import WatermarkRemover
|
||||
|
||||
effective_model = model_id or self.DEFAULT_MODEL_ID
|
||||
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
|
||||
|
||||
self._remover = WatermarkRemover(
|
||||
model_id=effective_model,
|
||||
device=device,
|
||||
progress_callback=progress_callback,
|
||||
hf_token=hf_token,
|
||||
@@ -148,113 +135,98 @@ class InvisibleEngine:
|
||||
"""
|
||||
self._remover.preload(global_only=global_only)
|
||||
|
||||
def _esrgan_upscale(self, image: Any, target: tuple[int, int]) -> Any:
|
||||
"""Upscale a PIL image to ``target`` with Real-ESRGAN, else Lanczos.
|
||||
|
||||
Runs Real-ESRGAN at its native factor (on the remover's device, CPU fallback),
|
||||
then resizes to the exact ``target`` with Lanczos. Falls back to a plain Lanczos
|
||||
resize when the ``esrgan`` extra is absent or the model errors.
|
||||
"""
|
||||
import cv2
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks import upscaler
|
||||
|
||||
if not upscaler.is_available():
|
||||
logger.debug("esrgan upscaler requested but the extra is absent; using Lanczos")
|
||||
return image.resize(target, Image.Resampling.LANCZOS)
|
||||
try:
|
||||
bgr = cv2.cvtColor(np.array(image.convert("RGB")), cv2.COLOR_RGB2BGR)
|
||||
big = upscaler.upscale(bgr, device=self._remover.device)
|
||||
if (big.shape[1], big.shape[0]) != target:
|
||||
big = cv2.resize(big, target, interpolation=cv2.INTER_LANCZOS4)
|
||||
return Image.fromarray(cv2.cvtColor(big, cv2.COLOR_BGR2RGB))
|
||||
except Exception as e: # never let an optional upscaler break removal
|
||||
logger.warning("Real-ESRGAN upscale failed (%s); using Lanczos", e)
|
||||
return image.resize(target, Image.Resampling.LANCZOS)
|
||||
|
||||
def remove_watermark(
|
||||
self,
|
||||
image_path: Path,
|
||||
output_path: Path | None = None,
|
||||
strength: float | None = None,
|
||||
num_inference_steps: int | None = None,
|
||||
guidance_scale: float | None = None,
|
||||
seed: int | None = None,
|
||||
humanize: float = 0.0,
|
||||
max_resolution: int = 0,
|
||||
min_resolution: int = 1024,
|
||||
vendor: str | None = None,
|
||||
unsharp: float = 0.0,
|
||||
adaptive_polish: bool = False,
|
||||
upscaler: str = "lanczos",
|
||||
adaptive_polish: bool | None = None,
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
text_manifest: Path | None = None,
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Path:
|
||||
"""Remove invisible watermark from an image.
|
||||
|
||||
Args:
|
||||
image_path: Path to the watermarked image.
|
||||
output_path: Output path (None = overwrite source).
|
||||
strength: Denoising strength (0.0-1.0). None -> the vendor-adaptive
|
||||
default.
|
||||
num_inference_steps: Number of denoising steps. None keeps the existing
|
||||
100-step library default, except qwen-zimage uses its required
|
||||
four-step Lightning schedule.
|
||||
guidance_scale: Classifier-free guidance scale.
|
||||
seed: Random seed for reproducibility. None resolves to 0 for
|
||||
qwen-zimage and stays random for the other profiles.
|
||||
strength: Denoising strength (0.0-1.0). None -> the profile's calibrated
|
||||
default (resolution-adaptive for qwen-zimage, vendor-adaptive for
|
||||
sdxl-zimage).
|
||||
seed: Random seed for reproducibility. None resolves to 0, because both
|
||||
profiles are certified at a fixed seed.
|
||||
humanize: Intensity of Analog Humanizer film grain (0 = off).
|
||||
unsharp: Final unsharp-mask sharpening strength (0 = off, default).
|
||||
Applied last to counter the soft / over-smoothed look of the
|
||||
diffusion pass; ~0.5-0.8 is a safe range, higher risks edge halos.
|
||||
adaptive_polish: When True (the CLI default), restore the input's detail
|
||||
level in the softened output: a capped unsharp + edge-masked grain
|
||||
targeting the input's Laplacian variance. Self-limiting -- a no-op when
|
||||
the output already meets the input's detail level (text/flat graphics),
|
||||
so it only acts on over-smoothed photo/face texture. Runs LAST.
|
||||
adaptive_polish: Restore the input's detail level in the softened
|
||||
output: a capped unsharp + edge-masked grain targeting the input's
|
||||
Laplacian variance. Self-limiting -- a no-op when the output already
|
||||
meets the input's detail level (text/flat graphics), so it only acts on
|
||||
over-smoothed photo/face texture. Runs LAST. None (the default) follows
|
||||
the profile: off for qwen-zimage, on for sdxl-zimage. This resolves
|
||||
through the same ``resolve_adaptive_polish`` the CLI uses, so a library
|
||||
caller and a CLI caller on one profile get the same output.
|
||||
max_resolution: Cap the long side (px) before diffusion. 0 (default)
|
||||
= no cap. Set a positive value only to bound GPU/MPS memory on
|
||||
very large inputs (it reintroduces a lossy downscale->upscale
|
||||
round-trip).
|
||||
min_resolution: Upscale the long side UP to this (px) before diffusion
|
||||
when the input is smaller, so SDXL runs near its ~1024 training
|
||||
resolution (small inputs degrade/distort badly at native). 1024
|
||||
(default) = on; 0 = off. The output is restored to the original
|
||||
input size, so this is a transparent quality boost; it adds time
|
||||
and memory on small inputs. Ignored on a min > max misconfig.
|
||||
upscaler: How to upscale a small input to the ``min_resolution`` floor:
|
||||
``"lanczos"`` (default, cv2, no model download) or ``"esrgan"`` (Real-ESRGAN
|
||||
via the ``esrgan`` extra). Only applies when UPscaling (the floor
|
||||
case); a ``max_resolution`` downscale always uses Lanczos. Falls back
|
||||
to Lanczos if the extra is absent.
|
||||
= no cap. Set a positive value only to bound GPU memory on very large
|
||||
inputs (it reintroduces a lossy downscale->upscale round-trip).
|
||||
tile: Process the diffusion pass in overlapping tiles instead of one
|
||||
forward pass. This retains the input's native dimensions instead
|
||||
of applying ``max_resolution``, but each tile is still regenerated.
|
||||
Engages only when the long side exceeds ``tile_size``.
|
||||
tile_size: Tile dimension in px (default 1024).
|
||||
tile_overlap: Overlap between adjacent tiles in px (default 128).
|
||||
text_manifest: Operator-verified text lines bound to the decoded source
|
||||
pixels. Enables the experimental Qwen-VAE ``vae-glyphs`` post-pass.
|
||||
Requires the ``text-restoration`` extra and the ``qwen-zimage``
|
||||
profile. Incompatible with downscaling, humanize, unsharp, and
|
||||
adaptive polish. Tiling is supported: the VAE donor uses the same
|
||||
overlapping tiles as the global pass, then glyph restore runs on
|
||||
the blended full frame.
|
||||
fidelity_anchor: Blend 15% of the Qwen-VAE donor across the whole frame
|
||||
before glyph restoration. OFF by default since 0.27.1: that global
|
||||
blend was measured to return detector-visible OpenAI SynthID on
|
||||
poster-scale manifests (detected x6 with the anchor vs clean x6
|
||||
without it, base clean; official Content Provenance API,
|
||||
2026-08-19 - docs/text-protection-research.md). ``True`` reproduces
|
||||
the 0.27.0 research behavior. Requires ``text_manifest``.
|
||||
|
||||
Returns:
|
||||
Path to the cleaned image.
|
||||
"""
|
||||
import tempfile
|
||||
|
||||
if num_inference_steps is None:
|
||||
profile = getattr(self._remover, "model_profile", None)
|
||||
num_inference_steps = 4 if profile == "qwen-zimage" else 100
|
||||
profile = getattr(self._remover, "model_profile", "controlnet")
|
||||
seed = resolve_seed(seed, profile)
|
||||
seed = resolve_seed(seed)
|
||||
adaptive_polish = resolve_adaptive_polish(adaptive_polish, self._remover.model_profile)
|
||||
|
||||
if fidelity_anchor and text_manifest is None:
|
||||
raise ValueError("fidelity_anchor requires a text manifest")
|
||||
if text_manifest is not None:
|
||||
if self._remover.model_profile != QWEN_ZIMAGE_PROFILE:
|
||||
raise ValueError("--text-manifest is supported only by the qwen-zimage profile")
|
||||
if max_resolution != 0:
|
||||
raise ValueError("--text-manifest requires --max-resolution 0")
|
||||
if humanize > 0.0 or unsharp > 0.0 or adaptive_polish:
|
||||
raise ValueError("--text-manifest requires humanize=0, unsharp=0, and adaptive polish disabled")
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
if not region_eraser.lama_available():
|
||||
raise RuntimeError(
|
||||
"Verified text restoration requires LaMa. Install: "
|
||||
"pip install 'remove-ai-watermarks[text-restoration]'"
|
||||
)
|
||||
|
||||
from PIL import Image, ImageOps
|
||||
|
||||
# Resolution policy: a max_resolution cap (0 = none) bounds memory on huge
|
||||
# inputs, and a min_resolution floor (1024 = default) upscales tiny inputs so
|
||||
# SDXL img2img runs near its ~1024 training size instead of distorting on a
|
||||
# tiny latent (a 381x512 portrait wrecks at native -- issue #36 follow-up).
|
||||
# The output is restored to orig_size below, so the floor is transparent.
|
||||
# inputs. See _target_size for why it is the only lever left.
|
||||
# Register the HEIF/AVIF opener so a .heic/.avif input (now a SUPPORTED_FORMAT)
|
||||
# decodes here too. The --force skip path bypasses image_io.imread, which is
|
||||
# what would otherwise register it, so a bare Image.open would fail on HEIC.
|
||||
@@ -267,35 +239,22 @@ class InvisibleEngine:
|
||||
# Full-res original, kept for the adaptive-polish detail target (image is
|
||||
# reassigned to the resized copy below; PIL resize returns a new object).
|
||||
reference_pil = image
|
||||
verified_text = None
|
||||
if text_manifest is not None:
|
||||
from remove_ai_watermarks._internal.text_restoration import load_verified_text_manifest
|
||||
|
||||
# qwen-zimage operates at the input's native geometry in its reference graph.
|
||||
# Keep an explicit max cap available for callers, but do not apply the SDXL
|
||||
# 1024px minimum-resolution floor to this profile.
|
||||
effective_min_resolution = (
|
||||
0 if getattr(self._remover, "model_profile", None) == "qwen-zimage" else min_resolution
|
||||
)
|
||||
target = _target_size(
|
||||
image.width,
|
||||
image.height,
|
||||
max_resolution,
|
||||
effective_min_resolution,
|
||||
)
|
||||
verified_text = load_verified_text_manifest(text_manifest, reference_pil)
|
||||
|
||||
# Both profiles run at the input's native geometry, so only the explicit max
|
||||
# cap can move it, and it can only ever scale down.
|
||||
target = _target_size(image.width, image.height, max_resolution)
|
||||
if target is not None:
|
||||
upscaling = max(target) > max(image.width, image.height)
|
||||
if self._progress_callback:
|
||||
reason = (
|
||||
f"min-resolution floor {min_resolution}px"
|
||||
if upscaling
|
||||
else f"max-resolution cap {max_resolution}px"
|
||||
self._progress_callback(
|
||||
f"Downscaling {image.width}x{image.height} to {target[0]}x{target[1]} "
|
||||
f"(max-resolution cap {max_resolution}px)..."
|
||||
)
|
||||
verb = "Upscaling" if upscaling else "Downscaling"
|
||||
self._progress_callback(f"{verb} {image.width}x{image.height} to {target[0]}x{target[1]} ({reason})...")
|
||||
# Real-ESRGAN only helps when UPscaling (the floor case); a downscale cap
|
||||
# always uses Lanczos. _esrgan_upscale falls back to Lanczos if the extra is absent.
|
||||
if upscaling and upscaler == "esrgan":
|
||||
image = self._esrgan_upscale(image, target)
|
||||
else:
|
||||
image = image.resize(target, Image.Resampling.LANCZOS)
|
||||
image = image.resize(target, Image.Resampling.LANCZOS)
|
||||
|
||||
# Always persist to a temp file, even without downscaling: WatermarkRemover
|
||||
# reloads by path, so the EXIF-transposed pixels must be saved or rotation
|
||||
@@ -315,13 +274,13 @@ class InvisibleEngine:
|
||||
image_path=image_path,
|
||||
output_path=output_path,
|
||||
strength=strength,
|
||||
num_inference_steps=num_inference_steps,
|
||||
guidance_scale=guidance_scale,
|
||||
seed=seed,
|
||||
vendor=vendor,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
text_manifest=verified_text,
|
||||
fidelity_anchor=fidelity_anchor,
|
||||
)
|
||||
|
||||
# Post-processing chain: decode the diffusion output ONCE, apply the
|
||||
@@ -387,21 +346,3 @@ class InvisibleEngine:
|
||||
# _tmp_path is always set above (we persist the image unconditionally).
|
||||
if _tmp_path.exists():
|
||||
_tmp_path.unlink()
|
||||
|
||||
def remove_watermark_batch(
|
||||
self,
|
||||
input_dir: Path,
|
||||
output_dir: Path,
|
||||
strength: float | None = None,
|
||||
steps: int | None = None,
|
||||
) -> list[Path]:
|
||||
"""Remove invisible watermarks from all images in a directory."""
|
||||
if steps is None:
|
||||
profile = getattr(self._remover, "model_profile", None)
|
||||
steps = 4 if profile == "qwen-zimage" else 50
|
||||
return self._remover.remove_watermark_batch(
|
||||
input_dir=input_dir,
|
||||
output_dir=output_dir,
|
||||
strength=strength,
|
||||
num_inference_steps=steps,
|
||||
)
|
||||
@@ -29,6 +29,9 @@ from typing import TYPE_CHECKING
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -81,7 +84,7 @@ def _bits_to_bytes(bits: Iterable[object], nbytes: int) -> bytes:
|
||||
return bytes(int(value) for value in packed[:nbytes])
|
||||
|
||||
|
||||
def detect_invisible_watermark(image_path: Path) -> str | None:
|
||||
def detect_invisible_watermark(image_path: Path, *, image: NDArray[Any] | None = None) -> str | None:
|
||||
"""Return the embedding scheme name if a known open watermark is decoded.
|
||||
|
||||
Returns e.g. ``"Stable Diffusion XL"`` / ``"FLUX.2 (Black Forest Labs)"`` /
|
||||
@@ -94,7 +97,10 @@ def detect_invisible_watermark(image_path: Path) -> str | None:
|
||||
from remove_ai_watermarks import image_io
|
||||
from remove_ai_watermarks.dwt_dct import decode_dwt_dct_lengths
|
||||
|
||||
img = image_io.imread(image_path)
|
||||
# ``image`` lets a caller that has already decoded these pixels hand them in
|
||||
# (mirrors gemini_engine.detect_sparkle_confidence). The decoder only reads the
|
||||
# array -- it converts colour spaces into fresh buffers -- so no copy is needed.
|
||||
img = image if image is not None else image_io.imread(image_path)
|
||||
if img is None:
|
||||
return None
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -85,8 +85,6 @@ _CONFIG = TextMarkConfig(
|
||||
min_gw=8,
|
||||
)
|
||||
|
||||
JimengDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Jimeng alpha template (float [0,1]), or None."""
|
||||
|
||||
@@ -52,11 +52,9 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image SHORT side (measured basis -- see
|
||||
@@ -114,8 +112,6 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
KlingDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Kling alpha template (float [0,1]), or None."""
|
||||
@@ -127,23 +123,8 @@ def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the Kling glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class KlingEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible Kling "可灵AI 3.0" watermark (locate -> mask; mask feeds the fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
@@ -25,11 +25,15 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkLocation,
|
||||
TextMarkScan,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image WIDTH (measured basis). The box is
|
||||
@@ -84,24 +88,12 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
LibLibDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled LibLibAI alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
"""Binary "LibLibAI" silhouette (255 = glyph) from the alpha map, or None."""
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the LibLibAI glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class LibLibEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible LibLibAI wordmark (bottom-center; localize -> fill)."""
|
||||
|
||||
@@ -111,60 +103,51 @@ class LibLibEngine(TextMarkEngine):
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
def detect(self, image: NDArray[Any] | None, *, provenance: bool = False) -> TextMarkDetection:
|
||||
if image is None or not image.size or min(image.shape[:2]) < self._MIN_SHORT_SIDE:
|
||||
return TextMarkDetection()
|
||||
return super().detect(image, provenance=provenance)
|
||||
def _scan(self, image: NDArray[Any] | None) -> TextMarkScan:
|
||||
"""Skip the scan entirely below the size floor.
|
||||
|
||||
def footprint_mask(
|
||||
self, image: NDArray[Any] | None, *, force: bool = False, dilate: int | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame mask of the logo + wordmark, bounded by the detector's match box.
|
||||
Gating the SCAN rather than overriding ``detect`` is what keeps the floor on the
|
||||
single-pass perception path too, and it means a small image costs nothing.
|
||||
"""
|
||||
if image is None or not image.size or min(image.shape[:2]) < self._MIN_SHORT_SIDE:
|
||||
return TextMarkScan(None, None, 0)
|
||||
return super()._scan(image)
|
||||
|
||||
def _footprint_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Bound the fill by the detector's match box, never by the binary glyph blob.
|
||||
|
||||
The base class's blob-bbox footprint is wrong in both directions here: the
|
||||
blob bleeds UP into bright background structure (on the 768x1024 cohort
|
||||
frame it reached y 931 and the fill ate the shirt's own print) and it does
|
||||
not own the triangle logo anyway. The match box bounds the wordmark exactly
|
||||
(that is what the NCC localized); the logo sits its own height to the LEFT
|
||||
of the text (measured on the cohort zoom: logo ~1.0x the glyph height, gap
|
||||
~0.3x), so the footprint is the match box extended left by ~1.3 heights.
|
||||
not own the triangle logo anyway.
|
||||
"""
|
||||
if image is None or image.size == 0:
|
||||
return None
|
||||
from remove_ai_watermarks import image_io, region_eraser
|
||||
return self._match_box_rect(image, loc, force=force, detection=detection)
|
||||
|
||||
image = image_io.to_bgr(image)
|
||||
h, w = image.shape[:2]
|
||||
if h < 32 or w < 64:
|
||||
return None
|
||||
loc = self.locate(image)
|
||||
bx, by, bw, bh = loc.bbox
|
||||
if force:
|
||||
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
|
||||
else:
|
||||
if not self.detect(image).detected:
|
||||
return None
|
||||
_, box = self._tophat_best(image, loc)
|
||||
if box is None:
|
||||
return None
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
gh = gy1 - gy0 + 1
|
||||
pad = max(3, int(0.25 * gh))
|
||||
rx1 = max(0, bx + gx0 - int(1.3 * gh)) # the triangle logo, left of the text
|
||||
ry1 = max(0, by + gy0 - pad)
|
||||
rx2 = min(w, bx + gx1 + 1 + pad)
|
||||
ry2 = min(h, by + gy1 + 1 + pad)
|
||||
if rx1 >= rx2 or ry1 >= ry2:
|
||||
return None
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * bw))
|
||||
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
|
||||
def _extend_match_box(
|
||||
self, box: tuple[int, int, int, int], loc: TextMarkLocation, frame: tuple[int, int]
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Extend the match box LEFT to take in the triangle logo.
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
The match box bounds the wordmark exactly (that is what the NCC localized);
|
||||
the logo sits its own height to the LEFT of the text (measured on the cohort
|
||||
zoom: logo ~1.0x the glyph height, gap ~0.3x), so the footprint is the match
|
||||
box extended left by ~1.3 heights.
|
||||
"""
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
bx, by, _bw, _bh = loc.bbox
|
||||
h, w = frame
|
||||
gh = gy1 - gy0 + 1
|
||||
pad = max(3, int(0.25 * gh))
|
||||
return (
|
||||
max(0, bx + gx0 - int(1.3 * gh)), # the triangle logo, left of the text
|
||||
max(0, by + gy0 - pad),
|
||||
min(w, bx + gx1 + 1 + pad),
|
||||
min(h, by + gy1 + 1 + pad),
|
||||
)
|
||||
@@ -1,7 +1,4 @@
|
||||
"""AI metadata detection and removal.
|
||||
|
||||
Wraps the noai-watermark metadata handling for stripping AI-generation
|
||||
metadata (EXIF, PNG text chunks, C2PA provenance) from images.
|
||||
"""Detect and remove AI provenance metadata from image containers.
|
||||
|
||||
For metadata-only operations, the heavy ML dependencies are NOT required.
|
||||
"""
|
||||
@@ -10,14 +7,22 @@ from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import functools
|
||||
import itertools
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import struct
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import TYPE_CHECKING, Any, cast
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable, Iterable
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks._internal.constants import (
|
||||
PNG_METADATA_CHUNKS,
|
||||
RIFF_METADATA_CHUNKS,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Smaller scan_head window for the cheap marker checks (has_ai_metadata,
|
||||
@@ -101,7 +106,7 @@ IPTC_AI_MARKERS: tuple[bytes, ...] = (
|
||||
# (Meta / Instagram / MidJourney) use ``trainedAlgorithmicMedia``. Including the bare
|
||||
# token flagged clean procedural images as AI (is_ai=high + has_invisible_target=True ->
|
||||
# a diffusion scrub of clean content), contradicting the c2pa layer, which sets
|
||||
# source_type without ai_source for it (tests/test_noai.py::test_plain_algorithmic_media_not_flagged_ai).
|
||||
# source_type without ai_source for it (tests/test_metadata_internals.py::test_plain_algorithmic_media_not_flagged_ai).
|
||||
# It is not a substring of the trained/composite tokens, so its removal does not affect
|
||||
# their detection.
|
||||
|
||||
@@ -120,12 +125,13 @@ IPTC_AI_FIELD_MARKERS: tuple[bytes, ...] = (
|
||||
# the container level (image, video, audio -- all ISOBMFF). A content sniff
|
||||
# (``ftyp``) is also accepted, so this is a fast-path hint, not the sole gate.
|
||||
_ISOBMFF_EXTS: frozenset[str] = frozenset({".avif", ".heif", ".heic", ".jxl", ".mp4", ".mov", ".m4v", ".m4a"})
|
||||
_STREAMING_ISOBMFF_EXTS: frozenset[str] = frozenset({".mp4", ".mov", ".m4v", ".m4a"})
|
||||
|
||||
# Non-ISOBMFF audio/video the ISOBMFF box walker can't reach (EBML / framed /
|
||||
# RIFF / Vorbis). remove_ai_metadata strips their container metadata losslessly
|
||||
# via ffmpeg (`-c copy`), so it needs ffmpeg on PATH for these.
|
||||
_FFMPEG_STRIP_EXTS: frozenset[str] = frozenset(
|
||||
{".webm", ".mkv", ".mka", ".mp3", ".wav", ".flac", ".ogg", ".oga", ".opus", ".aac"}
|
||||
{".webm", ".mkv", ".mka", ".avi", ".flv", ".mp3", ".wav", ".flac", ".ogg", ".oga", ".opus", ".aac"}
|
||||
)
|
||||
|
||||
# China's mandatory AI-content labeling (TC260, the national cybersecurity
|
||||
@@ -143,7 +149,7 @@ AIGC_MARKERS: tuple[bytes, ...] = (
|
||||
# the same object as a PNG ``tEXt`` chunk keyed ``AIGC`` (raw JSON, not XMP), so
|
||||
# a JSON object carrying at least one of these is accepted as a valid TC260
|
||||
# label even when the namespaced XMP element is absent.
|
||||
_TC260_FIELDS: frozenset[str] = frozenset(
|
||||
TC260_AIGC_FIELDS: frozenset[str] = frozenset(
|
||||
{
|
||||
# Producer-side schema (Doubao and most China-served generators).
|
||||
"Label",
|
||||
@@ -161,6 +167,37 @@ _TC260_FIELDS: frozenset[str] = frozenset(
|
||||
"ServiceUser",
|
||||
}
|
||||
)
|
||||
MAX_TC260_VALUE_BYTES = 1024 * 1024
|
||||
|
||||
|
||||
# A TC260 producer code is ``001`` + ``1`` + USCC(18) + a 5-digit app/product suffix,
|
||||
# so two codes sharing the USCC are the same legal entity registering different
|
||||
# products. Slicing is defensive: anything not matching the layout is returned as-is,
|
||||
# which also passes through the bare-name forms some generators write ("doubao",
|
||||
# "picwish").
|
||||
_USCC_START, _USCC_END = 4, 22
|
||||
|
||||
|
||||
def uscc_of(code: str) -> str:
|
||||
"""The 18-char Unified Social Credit Code embedded in a TC260 producer code."""
|
||||
if len(code) >= _USCC_END and code[:3] == "001":
|
||||
return code[_USCC_START:_USCC_END]
|
||||
return code
|
||||
|
||||
|
||||
def parse_tc260_aigc_json(value: bytes) -> dict[str, str] | None:
|
||||
"""Parse a bounded JSON object carrying at least one normative TC260 field."""
|
||||
if len(value) > MAX_TC260_VALUE_BYTES:
|
||||
return None
|
||||
try:
|
||||
parsed = json.loads(value.rstrip(b"\x00 ").decode("utf-8"))
|
||||
except (UnicodeDecodeError, ValueError):
|
||||
return None
|
||||
if not isinstance(parsed, dict):
|
||||
return None
|
||||
fields = {str(key): str(item) for key, item in cast("dict[object, object]", parsed).items()}
|
||||
return fields if TC260_AIGC_FIELDS & fields.keys() else None
|
||||
|
||||
|
||||
# HuggingFace-hosted GPU jobs (Jobs / Spaces) stamp generated PNGs with this
|
||||
# ``tEXt`` chunk key holding the job UUID. It marks the hosting job, not a
|
||||
@@ -198,15 +235,11 @@ def _is_ai_value(value: str) -> bool:
|
||||
detection: NovelAI stamps a generic ``Title``/``Source`` text chunk (an
|
||||
AI-shaped value under a non-AI key) that ``_is_ai_key`` alone would keep.
|
||||
"""
|
||||
from remove_ai_watermarks.noai.constants import AI_GENERATOR_TOKENS
|
||||
from remove_ai_watermarks._internal.constants import AI_GENERATOR_TOKENS
|
||||
|
||||
value_lower = value.lower()
|
||||
return any(token in value_lower for token in AI_GENERATOR_TOKENS)
|
||||
|
||||
|
||||
# PNG ancillary chunks that can carry provenance metadata (XMP, EXIF, text).
|
||||
# Never IDAT -- that is the compressed pixel stream.
|
||||
_PNG_META_CHUNKS: frozenset[bytes] = frozenset({b"tEXt", b"iTXt", b"zTXt", b"eXIf", b"iCCP"})
|
||||
provenance, generator = _app_metadata_evidence(value)
|
||||
return provenance is not None or generator is not None or any(token in value_lower for token in AI_GENERATOR_TOKENS)
|
||||
|
||||
|
||||
def _png_late_metadata(image_path: Path, window: int) -> bytes:
|
||||
@@ -240,7 +273,7 @@ def _png_late_metadata(image_path: Path, window: int) -> bytes:
|
||||
# Clamp the attacker-controlled 32-bit length to the bytes that
|
||||
# actually remain, so a malformed huge length can't allocate GBs.
|
||||
safe_length = max(0, min(length, file_size - data_start))
|
||||
if chunk_type in _PNG_META_CHUNKS and data_start >= window:
|
||||
if chunk_type in PNG_METADATA_CHUNKS and data_start >= window:
|
||||
f.seek(data_start)
|
||||
out += f.read(safe_length)
|
||||
# Advance by the CLAMPED length: a malformed/inflated `length` that
|
||||
@@ -253,17 +286,86 @@ def _png_late_metadata(image_path: Path, window: int) -> bytes:
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _riff_late_metadata(image_path: Path, window: int, *, max_total: int = 4 * 1024 * 1024) -> bytes:
|
||||
"""Payloads of RIFF metadata chunks that start *beyond* the first ``window``
|
||||
bytes, found by stepping over the (large) coded-image chunk.
|
||||
|
||||
The WebP layout puts ``XMP ``/``EXIF`` AFTER the pixels, so a fixed read can stop
|
||||
before an IPTC or C2PA AI label. This is the RIFF analogue of
|
||||
:func:`_png_late_metadata`; it returns only chunks past ``window`` so bytes
|
||||
already in the head are not duplicated, and empty when there are none.
|
||||
|
||||
``max_total`` caps what a metadata scan can pull into memory, the same ceiling
|
||||
``isobmff.scan_c2pa_region`` applies. Clamping each chunk to the bytes that remain
|
||||
is not enough on its own: a corrupt or crafted file can declare one ``XMP `` chunk
|
||||
spanning most of itself, and this runs on the memoized verdict path for images from
|
||||
arbitrary sources. A label that needs more than 4 MB of XMP does not exist.
|
||||
"""
|
||||
out = bytearray()
|
||||
try:
|
||||
with open(image_path, "rb") as f:
|
||||
if f.read(4) != b"RIFF":
|
||||
return b""
|
||||
f.seek(0, 2)
|
||||
file_size = f.tell()
|
||||
f.seek(4)
|
||||
declared_size = f.read(4)
|
||||
if len(declared_size) < 4:
|
||||
return b""
|
||||
container_end = min(file_size, 8 + struct.unpack("<I", declared_size)[0])
|
||||
position = 12 # 'RIFF' + size + form type
|
||||
while position + 8 <= container_end and len(out) < max_total:
|
||||
f.seek(position)
|
||||
header = f.read(8)
|
||||
if len(header) < 8:
|
||||
break
|
||||
chunk_type = header[:4]
|
||||
(length,) = struct.unpack("<I", header[4:8])
|
||||
start = position + 8
|
||||
# Clamp to what remains: a malformed 32-bit length must not push the
|
||||
# walk past EOF and abandon a genuine label chunk after it.
|
||||
safe_length = max(0, min(length, container_end - start))
|
||||
if chunk_type in RIFF_METADATA_CHUNKS and start >= window:
|
||||
f.seek(start)
|
||||
out += f.read(min(safe_length, max_total - len(out)))
|
||||
position = start + safe_length + (safe_length & 1) # chunks are word-aligned
|
||||
except OSError as exc:
|
||||
logger.debug("RIFF late-metadata scan failed on %s: %s", image_path, exc)
|
||||
return b""
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _stat_key(image_path: Path) -> tuple[str, int, int] | None:
|
||||
"""Cache key identifying this file's exact CONTENT, or None when it cannot stat.
|
||||
|
||||
``(path, mtime_ns, size)`` -- size as well as mtime because an in-place rewrite
|
||||
can land inside the same mtime tick on a coarse filesystem, and this package does
|
||||
rewrite in place (``remove_ai_metadata(p, p)``, the batch output-equals-input
|
||||
case). A file it cannot stat is read uncached rather than failing.
|
||||
"""
|
||||
try:
|
||||
st = image_path.stat()
|
||||
except OSError:
|
||||
return None
|
||||
return (str(image_path), st.st_mtime_ns, st.st_size)
|
||||
|
||||
|
||||
def scan_head(image_path: Path, size: int = 1024 * 1024) -> bytes:
|
||||
"""First ``size`` bytes of the file, plus the payloads of any provenance
|
||||
metadata found beyond that window: ISOBMFF ``uuid`` / ``jumb`` boxes (seeking
|
||||
past large boxes like ``mdat``) and PNG ``tEXt`` / ``iTXt`` / ``eXIf`` chunks
|
||||
(seeking past ``IDAT``).
|
||||
|
||||
A file at least ``size`` bytes long additionally gets the metadata text its
|
||||
decoder can reach but a raw read cannot (:func:`_decoder_visible_text`): a
|
||||
compressed PNG ``zTXt`` packet, or a chunk past the window in a container with no
|
||||
late-chunk reader here. A file that fits inside ``size`` is exactly
|
||||
``f.read(size)``, since the raw read already holds every byte.
|
||||
|
||||
This is the shared input for every C2PA / AIGC / IPTC byte scan. The
|
||||
extensions catch a manifest or XMP packet placed AFTER the media data -- a
|
||||
non-faststart MP4 manifest, or a PNG XMP packet appended after the pixels --
|
||||
which a fixed first-MB read would miss. For other inputs, and for files that
|
||||
fit within ``size``, it is exactly ``f.read(size)`` -- behavior-neutral.
|
||||
which a fixed first-MB read would miss.
|
||||
|
||||
The result is memoized per (path, size, mtime): one ``identify``/``get_ai_metadata``
|
||||
call fans out to ~8 byte-scan detectors that each call this on the same file, so
|
||||
@@ -290,7 +392,7 @@ def _scan_head_impl(image_path: Path, size: int) -> bytes:
|
||||
with open(image_path, "rb") as f:
|
||||
head = f.read(size)
|
||||
# Lazy import: isobmff imports this module's constants at top level.
|
||||
from remove_ai_watermarks.noai import isobmff
|
||||
from remove_ai_watermarks._internal import isobmff
|
||||
|
||||
if isobmff.is_isobmff(head):
|
||||
region = isobmff.scan_c2pa_region(image_path)
|
||||
@@ -300,9 +402,63 @@ def _scan_head_impl(image_path: Path, size: int) -> bytes:
|
||||
# len(head) == size means the file is at least `size` bytes, so metadata
|
||||
# chunks may lie beyond the window; otherwise the whole PNG is in `head`.
|
||||
head += _png_late_metadata(image_path, size)
|
||||
elif head[:4] == b"RIFF" and head[8:12] == b"WEBP" and len(head) == size:
|
||||
head += _riff_late_metadata(image_path, size)
|
||||
if len(head) >= size:
|
||||
head += _decoder_visible_text(image_path, head)
|
||||
return head
|
||||
|
||||
|
||||
# Text values the image decoder can reach that a raw byte read cannot. Bounded: a
|
||||
# packet larger than this is not a provenance label.
|
||||
_DECODED_TEXT_LIMIT = 512 * 1024
|
||||
# Decoder values that are binary payloads with their own readers, not metadata text.
|
||||
# An ICC profile is colour data and can run to hundreds of kilobytes; appending it
|
||||
# would bloat the buffer every later detector re-scans, for no signal.
|
||||
_DECODER_BINARY_KEYS = frozenset({"icc_profile"})
|
||||
|
||||
|
||||
def _decoder_visible_text(image_path: Path, head: bytes) -> bytes:
|
||||
"""Metadata text PIL can decode but the raw window does not contain.
|
||||
|
||||
This is the last of two layers, not the first. Metadata placed BEYOND the window
|
||||
is the structural readers' job (``_png_late_metadata``, ``_riff_late_metadata``,
|
||||
the ISOBMFF box walk), and they work on a file no decoder can open. What is left
|
||||
for this one is metadata the bytes do not spell at all:
|
||||
|
||||
* COMPRESSED -- a PNG ``zTXt`` chunk is zlib-deflated, so an XMP packet carrying
|
||||
a TC260 AIGC label is unreadable as bytes while PIL inflates it on open.
|
||||
|
||||
It stays container-agnostic on purpose: it is the net under a placement no
|
||||
structural reader here knows about yet.
|
||||
|
||||
Only text ALREADY MISSING from ``head`` is appended, so the common case adds
|
||||
nothing and no detector sees a value twice. Skipped entirely when the file fits
|
||||
inside the window, since then the raw read already holds every byte.
|
||||
"""
|
||||
try:
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(image_path) as img:
|
||||
values = [value for key, value in img.info.items() if key not in _DECODER_BINARY_KEYS]
|
||||
except Exception as exc: # a container PIL cannot open: the raw scan stands alone
|
||||
logger.debug("decoder-visible text unavailable for %s: %s", image_path, exc)
|
||||
return b""
|
||||
|
||||
out = bytearray()
|
||||
for value in values:
|
||||
if isinstance(value, str):
|
||||
encoded = value.encode("utf-8", "replace")
|
||||
elif isinstance(value, bytes):
|
||||
encoded = value
|
||||
else:
|
||||
continue
|
||||
if len(encoded) > _DECODED_TEXT_LIMIT or not encoded or encoded in head:
|
||||
continue
|
||||
out += b"\x00" + encoded
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def has_ai_metadata(image_path: Path) -> bool:
|
||||
"""Check if an image contains AI-generation metadata.
|
||||
|
||||
@@ -322,13 +478,16 @@ def has_ai_metadata(image_path: Path) -> bool:
|
||||
for key in img.info:
|
||||
if isinstance(key, str) and _is_ai_key(key):
|
||||
return True
|
||||
exif_bytes = img.info.get("exif")
|
||||
if exif_bytes and any(_app_metadata_evidence(exif_bytes)):
|
||||
return True
|
||||
except Exception as exc:
|
||||
logger.debug("PIL could not open %s for metadata scan: %s", image_path, exc)
|
||||
|
||||
# Check C2PA — via the official c2pa-python reader first (spec-tracking, every
|
||||
# container it supports), then a binary scan that also catches AVIF/HEIF/JPEG-XL
|
||||
# containers and synthetic/partial blobs the validator rejects.
|
||||
from remove_ai_watermarks.noai.c2pa import read_manifest_store_json
|
||||
from remove_ai_watermarks._internal.c2pa import read_manifest_store_json
|
||||
|
||||
if read_manifest_store_json(image_path) is not None:
|
||||
return True
|
||||
@@ -345,6 +504,8 @@ def has_ai_metadata(image_path: Path) -> bool:
|
||||
# IPTC 2025.1 AI-disclosure XMP properties (their presence flags AI content).
|
||||
if any(marker in data for marker in IPTC_AI_FIELD_MARKERS):
|
||||
return True
|
||||
if any(_app_metadata_evidence(data)):
|
||||
return True
|
||||
# China TC260 AIGC label as a PNG text chunk (the byte scan above catches
|
||||
# only the XMP form; the raw-JSON tEXt chunk needs the PIL-based parse).
|
||||
if aigc_label(image_path) is not None:
|
||||
@@ -363,16 +524,15 @@ def aigc_label_from_metadata(data: bytes, candidates: tuple[str, ...] = ()) -> d
|
||||
from typing import cast
|
||||
|
||||
def _parse(text: str, *, require_tc260_field: bool) -> dict[str, str] | None:
|
||||
if require_tc260_field:
|
||||
return parse_tc260_aigc_json(text.encode("utf-8"))
|
||||
try:
|
||||
parsed = json.loads(text)
|
||||
except ValueError:
|
||||
return None
|
||||
if not isinstance(parsed, dict):
|
||||
return None
|
||||
fields = {str(k): str(v) for k, v in cast("dict[object, object]", parsed).items()}
|
||||
if require_tc260_field and not (_TC260_FIELDS & fields.keys()):
|
||||
return None
|
||||
return fields
|
||||
return {str(k): str(v) for k, v in cast("dict[object, object]", parsed).items()}
|
||||
|
||||
for candidate in candidates:
|
||||
if result := _parse(candidate, require_tc260_field=True):
|
||||
@@ -404,13 +564,18 @@ def aigc_label_from_metadata(data: bytes, candidates: tuple[str, ...] = ()) -> d
|
||||
return None
|
||||
|
||||
|
||||
def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
def _aigc_label_impl(image_path: Path) -> dict[str, str] | None:
|
||||
"""Parse a China TC260 AI-labeling block, if present.
|
||||
|
||||
Three serializations are recognized:
|
||||
Supported serializations are:
|
||||
|
||||
- a PNG ``tEXt``/``iTXt`` chunk keyed ``AIGC`` carrying the raw JSON object
|
||||
(as written by Doubao / ByteDance), read via PIL;
|
||||
- a native MP4/MOV ``AIGC`` key in ``moov.udta.meta.keys`` whose matching
|
||||
``ilst`` item carries the raw JSON object;
|
||||
- a native MKV/WebM ``AIGC`` simple tag carrying the raw JSON object;
|
||||
- a native AVI ``LIST/INFO/AIGC`` chunk or FLV
|
||||
``script.onMetaData.AIGC`` string carrying the raw JSON object;
|
||||
- an XMP ``<TC260:AIGC>{...}</TC260:AIGC>`` block (HTML-entity encoded text),
|
||||
found by a container-agnostic raw-byte scan (PNG/JPEG/WebP alike); and
|
||||
- a raw-JSON ``{"AIGC":{...}}`` block with no namespace, as embedded in JPEG
|
||||
@@ -423,7 +588,7 @@ def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
Returns the decoded JSON (e.g. ``{"Label": "1", "ContentProducer": ...}``)
|
||||
or None. The generic forms (the PNG-chunk key ``AIGC``, the bare
|
||||
``{"AIGC":...}`` object, and the bare ``AIGC{...}`` blob) are accepted only
|
||||
if they carry at least one known TC260 field (``_TC260_FIELDS``); the
|
||||
if they carry at least one known TC260 field (``TC260_AIGC_FIELDS``); the
|
||||
namespaced XMP element is unambiguous, so any JSON object is accepted.
|
||||
"""
|
||||
try:
|
||||
@@ -434,9 +599,46 @@ def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
except Exception as exc:
|
||||
logger.debug("PIL could not open %s for AIGC chunk scan: %s", image_path, exc)
|
||||
value = None
|
||||
|
||||
if isinstance(value, str) and (result := aigc_label_from_metadata(b"", (value,))):
|
||||
return result
|
||||
|
||||
# Native container TC260 metadata. Every reader walks its own container structure
|
||||
# and skips media payloads instead of relying on a raw substring that could collide
|
||||
# inside compressed video, and every one of them SELF-GATES on its magic bytes --
|
||||
# returning () after a 4-12 byte read on anything else. So the route is content, not
|
||||
# extension: a correctly formatted AVI or FLV served under the wrong suffix used to
|
||||
# be missed, which contradicts this module's own rule elsewhere ("route on the
|
||||
# actual content format, not the extension").
|
||||
for reader in _tc260_container_readers():
|
||||
candidates = tuple(payload.decode("utf-8", "replace") for payload in reader(image_path))
|
||||
if result := aigc_label_from_metadata(b"", candidates):
|
||||
return result
|
||||
|
||||
data = scan_head(image_path)
|
||||
candidates = (value,) if isinstance(value, str) else ()
|
||||
return aigc_label_from_metadata(data, candidates)
|
||||
return aigc_label_from_metadata(data)
|
||||
|
||||
|
||||
def _tc260_container_readers() -> tuple[Callable[[Path], tuple[bytes, ...]], ...]:
|
||||
"""The native-container TC260 readers, most common first.
|
||||
|
||||
Static imports rather than ``importlib``: this module carries no pyright pragma, so
|
||||
a dynamically resolved callable would be ``Any`` and fail the strict gate. They stay
|
||||
function-local because ``isobmff`` imports this module's constants at import time.
|
||||
|
||||
* MP4/MOV -- the ``AIGC`` key in ``moov.udta.meta.keys`` points at raw JSON in
|
||||
``ilst``; the bounded box walker finds a tail ``moov`` after a large ``mdat``
|
||||
without loading the media payload.
|
||||
* MKV/WebM -- ``Segment.Tags.Tag.SimpleTag`` carries ``TagName=AIGC``.
|
||||
* AVI -- a ``LIST/INFO/AIGC`` chunk.
|
||||
* FLV -- ``script.onMetaData.AIGC``.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.ebml import tc260_aigc_payloads as ebml_payloads
|
||||
from remove_ai_watermarks._internal.flv import tc260_aigc_payloads as flv_payloads
|
||||
from remove_ai_watermarks._internal.isobmff import tc260_aigc_payloads as isobmff_payloads
|
||||
from remove_ai_watermarks._internal.riff import tc260_aigc_payloads as riff_payloads
|
||||
|
||||
return (isobmff_payloads, ebml_payloads, riff_payloads, flv_payloads)
|
||||
|
||||
|
||||
# C2PA "Durable Content Credentials" manifest repositories (C2PA 2.4). When the
|
||||
@@ -479,7 +681,7 @@ def c2pa_cloud_manifest(image_path: Path) -> str | None:
|
||||
return c2pa_cloud_manifest_in(scan_head(image_path, _QUICK_SCAN_BYTES))
|
||||
|
||||
|
||||
def huggingface_job(image_path: Path) -> str | None:
|
||||
def _huggingface_job_impl(image_path: Path) -> str | None:
|
||||
"""Return the HuggingFace job id if the image carries an ``hf-job-id`` PNG
|
||||
text chunk, else None.
|
||||
|
||||
@@ -540,7 +742,7 @@ def samsung_genai_in(data: bytes) -> int | None:
|
||||
return int(match.group(1)) or None
|
||||
|
||||
|
||||
def samsung_genai(image_path: Path) -> int | None:
|
||||
def _samsung_genai_impl(image_path: Path) -> int | None:
|
||||
"""Return Samsung's non-zero ``genAIType`` value if the image carries the
|
||||
Galaxy AI editing marker, else None.
|
||||
|
||||
@@ -577,7 +779,7 @@ def iptc_ai_system_in(data: bytes) -> str | None:
|
||||
return "fields present"
|
||||
|
||||
|
||||
def iptc_ai_system(image_path: Path) -> str | None:
|
||||
def _iptc_ai_system_impl(image_path: Path) -> str | None:
|
||||
"""Return an IPTC 2025.1 AI-disclosure note if the file carries those XMP
|
||||
properties, else None.
|
||||
|
||||
@@ -591,18 +793,18 @@ def iptc_ai_system(image_path: Path) -> str | None:
|
||||
|
||||
|
||||
def synthid_source(image_path: Path) -> str | None:
|
||||
"""Return the vendor name(s) if the image carries a SynthID pixel watermark.
|
||||
"""Return the vendor name(s) when provenance establishes SynthID.
|
||||
|
||||
This is a *metadata-based* proxy: Google (Imagen/Gemini) and OpenAI
|
||||
(ChatGPT/DALL-E/gpt-image) embed an invisible SynthID watermark alongside
|
||||
a C2PA manifest, so a C2PA manifest signed by one of them on AI-generated
|
||||
content implies SynthID in the pixels. Adobe Firefly / Microsoft Designer
|
||||
sign C2PA but do not use SynthID, so they return None.
|
||||
This is provenance-based, not a local pixel decode. Google states that all
|
||||
media generated by its tools carries SynthID, so Google AI C2PA establishes
|
||||
the mark. OpenAI C2PA existed before OpenAI adopted SynthID, so OpenAI also
|
||||
requires the explicit ``c2pa.watermarked.*`` action used by current manifests.
|
||||
Adobe Firefly and Microsoft sign C2PA but do not use SynthID, so they return
|
||||
None.
|
||||
|
||||
The verdict is reliable only while the C2PA manifest is intact -- absence
|
||||
is not proof, because C2PA can be stripped while the pixel watermark
|
||||
survives, and the pixel watermark itself is not locally detectable
|
||||
(proprietary decoder).
|
||||
The evidence is readable only while the C2PA manifest is intact. Absence is
|
||||
not proof: C2PA can be stripped while the pixel watermark survives, and the
|
||||
pixel watermark itself is not locally detectable (proprietary decoder).
|
||||
|
||||
Args:
|
||||
image_path: Path to the image (PNG, JPEG, WebP, or ISOBMFF container).
|
||||
@@ -610,10 +812,19 @@ def synthid_source(image_path: Path) -> str | None:
|
||||
Returns:
|
||||
Comma-joined vendor name(s) (e.g. ``"OpenAI"``) or None.
|
||||
"""
|
||||
from remove_ai_watermarks.noai.c2pa import extract_c2pa_info, synthid_vendors_in
|
||||
from remove_ai_watermarks._internal.c2pa import (
|
||||
c2pa_info_has_invalid_credential,
|
||||
extract_c2pa_info,
|
||||
synthid_evidence_vendors_in,
|
||||
)
|
||||
|
||||
# PNG: the caBX chunk parser gives a clean, structured issuer.
|
||||
vendors = extract_c2pa_info(image_path).get("synthid_vendors")
|
||||
# Prefer the official reader's structured result. A failed asset binding or
|
||||
# signature cannot establish the claim, and the raw fallback below must not
|
||||
# resurrect it from the same damaged manifest bytes.
|
||||
c2pa = extract_c2pa_info(image_path)
|
||||
if c2pa_info_has_invalid_credential(c2pa):
|
||||
return None
|
||||
vendors = c2pa.get("synthid_vendors")
|
||||
if vendors:
|
||||
return ", ".join(vendors)
|
||||
|
||||
@@ -626,24 +837,87 @@ def synthid_source(image_path: Path) -> str | None:
|
||||
ai_source = b"trainedAlgorithmicMedia" in data or b"TrainedAlgorithmicMedia" in data
|
||||
if not (has_c2pa and ai_source):
|
||||
return None
|
||||
matched = synthid_vendors_in(data)
|
||||
matched = synthid_evidence_vendors_in(data)
|
||||
return ", ".join(matched) if matched else None
|
||||
|
||||
|
||||
def generator_from_metadata(candidates: list[str], scan: bytes = b"") -> str | None:
|
||||
def generator_from_metadata(candidates: Iterable[str], scan: bytes = b"") -> str | None:
|
||||
"""Return a known AI generator from collected EXIF, PNG, or XMP values."""
|
||||
from remove_ai_watermarks.noai.constants import AI_GENERATOR_TOKENS
|
||||
from remove_ai_watermarks._internal.constants import AI_GENERATOR_TOKENS
|
||||
|
||||
candidates.extend(
|
||||
if app_generator := app_generator_from_metadata(scan):
|
||||
return app_generator
|
||||
|
||||
creator_tools = (
|
||||
match.group(1).decode("latin1", "replace")
|
||||
for match in re.finditer(rb"CreatorTool[>\"'=\s]{1,4}([^<\"']{1,80})", scan)
|
||||
)
|
||||
for value in candidates:
|
||||
for value in itertools.chain(candidates, creator_tools):
|
||||
if app_generator := app_generator_from_metadata(value):
|
||||
return app_generator
|
||||
if any(token in value.lower() for token in AI_GENERATOR_TOKENS):
|
||||
return value.strip()
|
||||
return None
|
||||
|
||||
|
||||
_APP_PROVENANCE_PRODUCTS: dict[str, str] = {
|
||||
"doubao": "ByteDance Doubao",
|
||||
"xinghui": "ByteDance Xinghui",
|
||||
"dreamina": "ByteDance Dreamina",
|
||||
"dreamina_oversea": "ByteDance Dreamina",
|
||||
}
|
||||
_APP_PRODUCT_RE = re.compile(r'"product"\s*:\s*"([a-z0-9_]+)"', re.IGNORECASE)
|
||||
_APP_EXPORT_TYPE_RE = re.compile(r'"exportType"\s*:\s*"([a-z0-9_]+)"', re.IGNORECASE)
|
||||
_APP_AIGC_LABEL_RE = re.compile(r'"aigc_label_type"\s*:\s*[12](?=\s*[,}])', re.IGNORECASE)
|
||||
_APP_AIGC_TYPE_RE = re.compile(r'"aigc_type"\s*:\s*1(?=\s*[,}])', re.IGNORECASE)
|
||||
|
||||
|
||||
def _normalized_app_metadata(value: str | bytes) -> str:
|
||||
text = value.decode("latin-1", "ignore") if isinstance(value, bytes) else value
|
||||
return text.replace('\\"', '"')
|
||||
|
||||
|
||||
def _app_metadata_evidence(value: str | bytes) -> tuple[str | None, str | None]:
|
||||
"""Return removable product provenance and stronger AI-origin evidence."""
|
||||
normalized = _normalized_app_metadata(value)
|
||||
products = tuple(match.group(1).lower() for match in _APP_PRODUCT_RE.finditer(normalized))
|
||||
provenance = next(
|
||||
(_APP_PROVENANCE_PRODUCTS[product] for product in products if product in _APP_PROVENANCE_PRODUCTS), None
|
||||
)
|
||||
export_types = {match.group(1).lower() for match in _APP_EXPORT_TYPE_RE.finditer(normalized)}
|
||||
|
||||
generator = None
|
||||
if set(products).intersection({"dreamina", "dreamina_oversea"}) and "generation" in export_types:
|
||||
generator = "ByteDance Dreamina"
|
||||
elif '"aigc_info"' in normalized.lower():
|
||||
if "aweme" in products and _APP_AIGC_TYPE_RE.search(normalized):
|
||||
generator = "ByteDance Aweme AI"
|
||||
elif _APP_AIGC_LABEL_RE.search(normalized):
|
||||
generator = "Embedded app AIGC disclosure"
|
||||
return provenance, generator
|
||||
|
||||
|
||||
def app_provenance_from_metadata(value: str | bytes) -> str | None:
|
||||
"""Return exact AI-product provenance that is safe to scrub.
|
||||
|
||||
An exporting product does not by itself prove that the pixels were generated.
|
||||
That stronger interpretation stays in :func:`app_generator_from_metadata`.
|
||||
"""
|
||||
return _app_metadata_evidence(value)[0]
|
||||
|
||||
|
||||
def app_generator_from_metadata(value: str | bytes) -> str | None:
|
||||
"""Identify exact app-export AI disclosures embedded in JSON-shaped metadata.
|
||||
|
||||
ByteDance-family apps write a second provenance object beside C2PA or TC260.
|
||||
Its keys are ordinary EXIF fields, so generic key matching misses it and a
|
||||
metadata-preserving JPEG scrub keeps it. Only explicit AIGC discriminators or
|
||||
a Dreamina generation export assert AI origin. Product provenance alone remains
|
||||
removable without changing the image's origin verdict.
|
||||
"""
|
||||
return _app_metadata_evidence(value)[1]
|
||||
|
||||
|
||||
def exif_generator(image_path: Path) -> str | None:
|
||||
"""Return an AI-generator name from the EXIF ``Software`` / XMP ``CreatorTool``
|
||||
field (or a PNG text chunk), if it matches a known generator (see
|
||||
@@ -674,7 +948,9 @@ def exif_generator(image_path: Path) -> str | None:
|
||||
if isinstance(value, str) and value:
|
||||
candidates.append(value)
|
||||
if exif_bytes:
|
||||
tags = piexif.load(exif_bytes).get("0th", {})
|
||||
loaded = piexif.load(exif_bytes)
|
||||
tags = loaded.get("0th", {})
|
||||
exif_tags: dict[int, Any] = loaded.get("Exif") or {}
|
||||
# Make catches camera-style tags AI tools reuse (Ideogram writes
|
||||
# Make="Ideogram AI"); real cameras put "Apple"/"Canon" there, which
|
||||
# carry no AI token, so this stays low-false-positive.
|
||||
@@ -687,6 +963,9 @@ def exif_generator(image_path: Path) -> str | None:
|
||||
value = tags.get(tag)
|
||||
if isinstance(value, bytes):
|
||||
candidates.append(value.decode("latin1", "replace"))
|
||||
user_comment = exif_tags.get(piexif.ExifIFD.UserComment)
|
||||
if isinstance(user_comment, bytes):
|
||||
candidates.append(user_comment.decode("latin1", "replace"))
|
||||
except Exception as exc: # unopenable format / malformed EXIF
|
||||
logger.debug("EXIF generator read failed for %s: %s", image_path, exc)
|
||||
|
||||
@@ -716,7 +995,7 @@ def _exif_text(ifd: dict[int, Any], tag: int) -> str:
|
||||
return value.decode("latin1", "replace").strip() if isinstance(value, bytes) else ""
|
||||
|
||||
|
||||
def xai_signature(image_path: Path) -> bool:
|
||||
def _xai_signature_impl(image_path: Path) -> bool:
|
||||
"""Detect xAI / Grok's EXIF provenance signature scheme.
|
||||
|
||||
Grok image downloads (Aurora model) carry no C2PA, XMP, SynthID, or IPTC --
|
||||
@@ -752,7 +1031,7 @@ def _is_aigc_exif_value(raw: object) -> bool:
|
||||
Mirrors ``aigc_label``'s EXIF path: the ``{"AIGC":{...}}`` wrapper embedded in
|
||||
``UserComment`` / ``ImageDescription`` by China-served generators (Doubao's
|
||||
producer schema AND Tencent Cloud's service-provider schema, both keyed under
|
||||
``_TC260_FIELDS``). Gated on both the ``AIGC`` marker and a TC260 field so a
|
||||
``TC260_AIGC_FIELDS``). Gated on both the ``AIGC`` marker and a TC260 field so a
|
||||
coincidental token cannot false-drop a genuine caption/comment. Accepts a ``str``
|
||||
too (a PNG ``tEXt``/``iTXt`` value), not only EXIF bytes.
|
||||
"""
|
||||
@@ -763,7 +1042,7 @@ def _is_aigc_exif_value(raw: object) -> bool:
|
||||
if b"AIGC" not in raw:
|
||||
return False
|
||||
text = bytes(raw).decode("latin-1", "ignore")
|
||||
return any(field in text for field in _TC260_FIELDS)
|
||||
return any(field in text for field in TC260_AIGC_FIELDS)
|
||||
|
||||
|
||||
def _ai_exif_targets(loaded: dict[str, Any]) -> list[tuple[str, int, bytes, str]]:
|
||||
@@ -782,7 +1061,7 @@ def _ai_exif_targets(loaded: dict[str, Any]) -> list[tuple[str, int, bytes, str]
|
||||
"""
|
||||
import piexif
|
||||
|
||||
from remove_ai_watermarks.noai.constants import AI_GENERATOR_TOKENS
|
||||
from remove_ai_watermarks._internal.constants import AI_GENERATOR_TOKENS
|
||||
|
||||
ifd0: dict[int, Any] = loaded.get("0th") or {}
|
||||
ifde: dict[int, Any] = loaded.get("Exif") or {}
|
||||
@@ -813,6 +1092,15 @@ def _ai_exif_targets(loaded: dict[str, Any]) -> list[tuple[str, int, bytes, str]
|
||||
add("0th", ifd0, piexif.ImageIFD.ImageDescription, "ImageDescription")
|
||||
if _is_aigc_exif_value(ifde.get(piexif.ExifIFD.UserComment)):
|
||||
add("Exif", ifde, piexif.ExifIFD.UserComment, "UserComment")
|
||||
# (d) ByteDance-family app JSON. Exact AI-product provenance is removable even
|
||||
# when it does not by itself assert generated pixels. Ordinary Aweme/retouch/lv
|
||||
# exports remain untouched.
|
||||
for ifd_key, ifd, tag, name in (
|
||||
("0th", ifd0, piexif.ImageIFD.ImageDescription, "ImageDescription"),
|
||||
("Exif", ifde, piexif.ExifIFD.UserComment, "UserComment"),
|
||||
):
|
||||
if any(_app_metadata_evidence(ifd.get(tag, b""))):
|
||||
add(ifd_key, ifd, tag, name)
|
||||
|
||||
return targets
|
||||
|
||||
@@ -841,7 +1129,7 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
"""
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks.noai.c2pa import extract_c2pa_info, soft_binding_vendors_in, synthid_verdict
|
||||
from remove_ai_watermarks._internal.c2pa import extract_c2pa_info, soft_binding_vendors_in, synthid_verdict
|
||||
|
||||
result: dict[str, str] = {}
|
||||
|
||||
@@ -861,7 +1149,7 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
except Exception as exc:
|
||||
logger.debug("PIL could not open %s for AI-metadata scan: %s", image_path, exc)
|
||||
|
||||
# C2PA manifest fields from the single canonical parser (noai/c2pa.py).
|
||||
# C2PA manifest fields from the single canonical parser (_internal/c2pa.py).
|
||||
c2pa = extract_c2pa_info(image_path)
|
||||
for key in (
|
||||
"c2pa_manifest",
|
||||
@@ -872,6 +1160,14 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
"actions",
|
||||
"synthid_watermark",
|
||||
"soft_binding",
|
||||
"soft_binding_algorithm",
|
||||
"soft_binding_value",
|
||||
"c2pa_validation_source",
|
||||
"c2pa_validation_state",
|
||||
"c2pa_integrity",
|
||||
"c2pa_signature",
|
||||
"c2pa_signer_trust",
|
||||
"c2pa_signer_validity",
|
||||
):
|
||||
if key in c2pa:
|
||||
result.setdefault(key, str(c2pa[key]))
|
||||
@@ -890,6 +1186,18 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
if (aigc := aigc_label(image_path)) is not None:
|
||||
producer = aigc.get("ContentProducer", "")
|
||||
result["aigc_label"] = f"China AIGC label (TC260){f'; producer {producer}' if producer else ''}"
|
||||
# The structural producer beside its display rendering: a machine
|
||||
# consumer (video-visible provenance) must not parse the formatted
|
||||
# sentence above, whose wording can change without notice.
|
||||
if producer:
|
||||
result["aigc_producer"] = producer
|
||||
|
||||
app_scan = scan_head(image_path)
|
||||
app_provenance, app_generator = _app_metadata_evidence(app_scan)
|
||||
if app_provenance:
|
||||
result.setdefault("app_provenance", f"App export provenance ({app_provenance})")
|
||||
if app_generator:
|
||||
result.setdefault("app_aigc", f"App AIGC disclosure ({app_generator})")
|
||||
|
||||
# xAI / Grok EXIF signature scheme (its only provenance signal).
|
||||
if xai_signature(image_path):
|
||||
@@ -985,7 +1293,7 @@ def _jpeg_app_carries_ai(marker: int, payload: bytes) -> bool:
|
||||
# (detection<->removal parity). Skip APP1-EXIF (0xE1 ``Exif``): its camera tags are
|
||||
# scrubbed tag-by-tag via piexif, not dropped wholesale.
|
||||
if not (marker == 0xE1 and payload.startswith(b"Exif")):
|
||||
return _is_aigc_exif_value(payload)
|
||||
return _is_aigc_exif_value(payload) or any(_app_metadata_evidence(payload))
|
||||
return False
|
||||
|
||||
|
||||
@@ -1149,39 +1457,56 @@ def remove_ai_metadata(
|
||||
# strip C2PA + AI-label boxes at the container level without re-encoding.
|
||||
# Avoids needing PIL plugins (pillow-heif / pillow-jxl) and preserves the
|
||||
# codestream bit-for-bit. MP4/MOV/M4A are ISOBMFF too, so the same top-level
|
||||
# uuid/jumb box walker applies. Route by suffix OR by an ``ftyp`` content
|
||||
# sniff, so a correctly-shaped container is handled whatever its extension.
|
||||
from remove_ai_watermarks.noai.isobmff import (
|
||||
# uuid/jumb box walker applies. Known media suffixes take the bounded,
|
||||
# offset-preserving streaming path; images retain the in-memory item scrub
|
||||
# needed for XMP/EXIF inside mdat/idat. Route the remaining formats by suffix
|
||||
# OR by an ``ftyp`` content sniff.
|
||||
from remove_ai_watermarks._internal.isobmff import (
|
||||
blank_ai_exif_tokens,
|
||||
blank_ai_xmp_packets,
|
||||
blank_tc260_aigc_tags,
|
||||
is_isobmff,
|
||||
strip_c2pa_boxes,
|
||||
strip_isobmff_media_file,
|
||||
)
|
||||
|
||||
with open(source_path, "rb") as f:
|
||||
head = f.read(12)
|
||||
if source_path.suffix.lower() in _STREAMING_ISOBMFF_EXTS and is_isobmff(head):
|
||||
stripped, tc260_blanked = strip_isobmff_media_file(source_path, output_path)
|
||||
logger.info(
|
||||
"Stream-blanked %d AI-provenance box(es) and %d native TC260 tag(s) → %s",
|
||||
stripped,
|
||||
tc260_blanked,
|
||||
output_path,
|
||||
)
|
||||
return output_path
|
||||
if source_path.suffix.lower() in _ISOBMFF_EXTS or is_isobmff(head):
|
||||
data = source_path.read_bytes()
|
||||
# Top-level uuid/jumb boxes (C2PA + AI-label XMP), then the meta-box items
|
||||
# the top-level stripper can't reach (HEIF/AVIF store them in mdat/idat):
|
||||
# AI-label XMP packets and AI-generator tokens in an Exif item -- both
|
||||
# blanked in place (same length) so box sizes and iloc offsets stay valid
|
||||
# and the coded image is untouched.
|
||||
# Native TC260 tags, AI-label XMP packets, and AI-generator tokens in an
|
||||
# Exif item are blanked in place (same length) so box sizes and iloc /
|
||||
# media offsets stay valid and the coded content is untouched.
|
||||
cleaned, stripped = strip_c2pa_boxes(data)
|
||||
cleaned, tc260_blanked = blank_tc260_aigc_tags(cleaned)
|
||||
cleaned, blanked = blank_ai_xmp_packets(cleaned)
|
||||
cleaned, exif_blanked = blank_ai_exif_tokens(cleaned)
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_bytes(cleaned)
|
||||
logger.info(
|
||||
"Stripped %d AI-provenance box(es), blanked %d meta-box XMP packet(s) + %d EXIF token(s) → %s",
|
||||
"Stripped %d AI-provenance box(es), blanked %d native TC260 tag(s) + "
|
||||
"%d meta-box XMP packet(s) + %d EXIF token(s) → %s",
|
||||
stripped,
|
||||
tc260_blanked,
|
||||
blanked,
|
||||
exif_blanked,
|
||||
output_path,
|
||||
)
|
||||
return output_path
|
||||
|
||||
# Non-ISOBMFF audio/video (WebM/Matroska EBML, MP3 ID3, WAV/FLAC/OGG): the
|
||||
# Non-ISOBMFF audio/video (WebM/Matroska EBML, AVI/FLV, MP3 ID3,
|
||||
# WAV/FLAC/OGG): the
|
||||
# box walker can't reach these, so strip container metadata losslessly via
|
||||
# ffmpeg (-c copy -- codec data untouched, only tags/chapters dropped).
|
||||
if source_path.suffix.lower() in _FFMPEG_STRIP_EXTS:
|
||||
@@ -1296,3 +1621,100 @@ def remove_ai_metadata(
|
||||
|
||||
logger.info("Stripped AI metadata → %s", output_path)
|
||||
return output_path
|
||||
|
||||
|
||||
# ── Per-file probe memoization ──────────────────────────────────────────────
|
||||
# One ``identify`` reaches each of these twice (``get_ai_metadata`` internally, then
|
||||
# ``extract_provenance_evidence``), and each call re-walks the container -- the
|
||||
# ISOBMFF/EBML walks in ``aigc_label`` and the file-tail read in ``samsung_genai``
|
||||
# are the expensive ones. Keyed on (path, mtime_ns, size) so an in-place rewrite
|
||||
# invalidates; ``maxsize`` bounds memory to a handful of entries.
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _aigc_label_cached(path_str: str, _mtime_ns: int, _size: int) -> dict[str, str] | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _aigc_label_impl(_Path(path_str))
|
||||
|
||||
|
||||
def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
"""See :func:`_aigc_label_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _aigc_label_impl(image_path)
|
||||
result = _aigc_label_cached(*key)
|
||||
return dict(result) if result is not None else None
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _huggingface_job_cached(path_str: str, _mtime_ns: int, _size: int) -> str | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _huggingface_job_impl(_Path(path_str))
|
||||
|
||||
|
||||
def huggingface_job(image_path: Path) -> str | None:
|
||||
"""See :func:`_huggingface_job_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _huggingface_job_impl(image_path)
|
||||
return _huggingface_job_cached(*key)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _samsung_genai_cached(path_str: str, _mtime_ns: int, _size: int) -> int | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _samsung_genai_impl(_Path(path_str))
|
||||
|
||||
|
||||
def samsung_genai(image_path: Path) -> int | None:
|
||||
"""See :func:`_samsung_genai_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _samsung_genai_impl(image_path)
|
||||
return _samsung_genai_cached(*key)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _iptc_ai_system_cached(path_str: str, _mtime_ns: int, _size: int) -> str | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _iptc_ai_system_impl(_Path(path_str))
|
||||
|
||||
|
||||
def iptc_ai_system(image_path: Path) -> str | None:
|
||||
"""See :func:`_iptc_ai_system_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _iptc_ai_system_impl(image_path)
|
||||
return _iptc_ai_system_cached(*key)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _xai_signature_cached(path_str: str, _mtime_ns: int, _size: int) -> bool:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _xai_signature_impl(_Path(path_str))
|
||||
|
||||
|
||||
def xai_signature(image_path: Path) -> bool:
|
||||
"""See :func:`_xai_signature_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _xai_signature_impl(image_path)
|
||||
return _xai_signature_cached(*key)
|
||||
|
||||
|
||||
# ── Shared with the portable metadata record ────────────────────────
|
||||
# `metadata_record` must read exactly the windows and markers the file path reads: a
|
||||
# record built from a different window is a record whose verdict can disagree with
|
||||
# `identify` on the same image. Aliased rather than renamed because the private names
|
||||
# are load-bearing in this module's own tests and in a corpus script.
|
||||
QUICK_SCAN_BYTES = _QUICK_SCAN_BYTES
|
||||
SAMSUNG_EDITOR_MARKER = _SAMSUNG_EDITOR_MARKER
|
||||
read_file_tail = _read_file_tail
|
||||
png_late_metadata = _png_late_metadata
|
||||
riff_late_metadata = _riff_late_metadata
|
||||
exif_text = _exif_text
|
||||
@@ -0,0 +1,409 @@
|
||||
"""Collect one image's provenance metadata into a portable, JSON-safe record.
|
||||
|
||||
WHY THIS EXISTS
|
||||
|
||||
``extract_provenance_evidence`` reads a file and hands back evidence in memory, so
|
||||
collection and verdict must happen in the same process, on the machine holding the
|
||||
image. This module splits them: collect here, judge anywhere, from a record that
|
||||
survives JSON.
|
||||
|
||||
record = collect_metadata_record(path) # touches the file
|
||||
evidence = evidence_from_metadata_record(record, path=path)
|
||||
report = identify_from_evidence(evidence) # touches nothing
|
||||
|
||||
WHAT GOES IN, AND WHY NOT SIMPLY THE FILE HEAD
|
||||
|
||||
The verdict reads a scan buffer that ``scan_head`` fills with the first mebibyte of
|
||||
the file. Shipping that verbatim would make a record larger than a phone photo's
|
||||
worth of metadata by two orders of magnitude, because for a PNG almost all of that
|
||||
mebibyte is compressed pixel data in ``IDAT`` -- bytes no provenance token can ever
|
||||
live in. A record carries the metadata REGIONS instead, walked per container: the
|
||||
JPEG marker segments before the coded scan, every PNG chunk but ``IDAT``, the RIFF
|
||||
chunks that are not coded image, the ISOBMFF provenance boxes, and in every case the
|
||||
container's trailer.
|
||||
|
||||
COMPLETENESS IS A MEASURED PROPERTY, NOT A CLAIM
|
||||
|
||||
A region walker is only correct if nothing the verdict reads falls outside the
|
||||
regions it keeps, and no test over fixtures can establish that: the failure mode is
|
||||
a container placement nobody thought of. The contract is therefore ALSO verified
|
||||
against the file path over a real corpus -- same image, both paths, identical
|
||||
``ProvenanceReport``.
|
||||
|
||||
The placements that defeated an earlier draft of this collector, and the reason each
|
||||
rule below exists, are recorded in ``docs/module-internals.md`` under "Portable
|
||||
metadata record".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import logging
|
||||
import struct
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks._internal.constants import PNG_SIGNATURE, RIFF_CODED_IMAGE_CHUNKS
|
||||
from remove_ai_watermarks._internal.schema import require_schema_version
|
||||
from remove_ai_watermarks.metadata import (
|
||||
QUICK_SCAN_BYTES,
|
||||
SAMSUNG_EDITOR_MARKER,
|
||||
exif_text,
|
||||
read_file_tail,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# The structural walk covers the same window the file path reads raw, so the two
|
||||
# cannot disagree about a chunk type inside it. A smaller window would be cheaper but
|
||||
# opens a blind spot: past the window only ``png_late_metadata``'s ALLOWLIST is
|
||||
# collected, while the file path still sees every chunk type up to its own window --
|
||||
# and a C2PA ``caBX`` chunk is in neither that allowlist nor ``IDAT``. Walking here
|
||||
# costs little because the payload of the pixel stream is skipped, not copied.
|
||||
HEAD_WINDOW = 1024 * 1024
|
||||
# The window searched for the container's end marker. Matches the quick-scan window
|
||||
# the file path uses when it goes looking for a Samsung trailer, so a trailer visible
|
||||
# to one path is visible to the other.
|
||||
TAIL_WINDOW = QUICK_SCAN_BYTES
|
||||
# Kept from the tail when no end marker is found, so an unrecognized container still
|
||||
# contributes its last bytes without carrying half a photo.
|
||||
UNKNOWN_TRAILER_WINDOW = 64 * 1024
|
||||
|
||||
# PNG text keys the file path reads for a generator tag, in ITS order. NovelAI stamps
|
||||
# Software/Source/Title rather than EXIF, and the first match wins, so order matters.
|
||||
_GENERATOR_TEXT_KEYS = ("Software", "Source", "Title", "Description")
|
||||
# Stable transport contract for records produced by this module. The version is
|
||||
# deliberately separate from the verdict version: collection and interpretation can
|
||||
# evolve independently as long as old records remain readable.
|
||||
METADATA_RECORD_SCHEMA_VERSION = 1
|
||||
METADATA_RECORD_TYPE = "provenance_metadata"
|
||||
|
||||
|
||||
def _jpeg_regions(data: bytes) -> bytes:
|
||||
"""Every marker segment up to the entropy-coded scan, plus the trailer after EOI.
|
||||
|
||||
The scan itself is skipped by walking to SOS and then jumping to the trailing
|
||||
EOI, so a 20 MB photo contributes only its markers.
|
||||
|
||||
TWIN: ``metadata._strip_jpeg_metadata_lossless`` walks the same marker chain. The
|
||||
two were left separate on purpose -- that one couples the walk to "return False and
|
||||
fall back to a PIL re-encode", a decision the lossless strip path owns and this one
|
||||
must not inherit -- so a fix to marker handling belongs in BOTH.
|
||||
"""
|
||||
out = bytearray()
|
||||
index, size = 2, len(data)
|
||||
while index + 1 < size:
|
||||
if data[index] != 0xFF:
|
||||
break # malformed boundary: keep what was collected, the tail still follows
|
||||
marker = data[index + 1]
|
||||
if marker in (0xDA, 0xD9): # SOS / EOI: the coded scan follows
|
||||
break
|
||||
if 0xD0 <= marker <= 0xD7 or marker == 0x01: # standalone, no length
|
||||
index += 2
|
||||
continue
|
||||
if index + 4 > size:
|
||||
break
|
||||
segment_length = int.from_bytes(data[index + 2 : index + 4], "big")
|
||||
end = index + 2 + segment_length
|
||||
if segment_length < 2 or end > size:
|
||||
break
|
||||
out += data[index:end]
|
||||
index = end
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _png_regions(data: bytes) -> bytes:
|
||||
"""Every chunk except the ``IDAT`` payloads, plus whatever follows IEND.
|
||||
|
||||
TWIN: ``metadata._png_late_metadata`` walks the same chunk chain by SEEKING over
|
||||
the file rather than over a buffer, and keeps an allowlist rather than skipping
|
||||
``IDAT``. Both filters are deliberate: inside the window the file path sees every
|
||||
chunk type raw, past it only the allowlist survives.
|
||||
"""
|
||||
out = bytearray()
|
||||
size = len(data)
|
||||
position = len(PNG_SIGNATURE)
|
||||
while position + 8 <= size:
|
||||
(length,) = struct.unpack(">I", data[position : position + 4])
|
||||
chunk_type = data[position + 4 : position + 8]
|
||||
start = position + 8
|
||||
# Clamp the length to the bytes that remain: a malformed 32-bit length must
|
||||
# not push the walk past EOF and abandon a genuine label chunk after it.
|
||||
safe_length = max(0, min(length, size - start))
|
||||
if chunk_type != b"IDAT":
|
||||
out += chunk_type + data[start : start + safe_length]
|
||||
position = start + safe_length + 4 # payload + CRC
|
||||
if chunk_type == b"IEND":
|
||||
out += data[position:] # a trailer past IEND is metadata too
|
||||
break
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _riff_regions(data: bytes) -> bytes:
|
||||
"""Every RIFF chunk except the coded image payloads.
|
||||
|
||||
TWIN: ``metadata._riff_late_metadata`` (seek-based, past the scan window) and
|
||||
``_internal.riff`` (AVI ``LIST/INFO``). Same chunk-stepping arithmetic, three
|
||||
input models.
|
||||
"""
|
||||
out = bytearray(data[:12]) # 'RIFF' + size + 'WEBP'
|
||||
declared_end = 8 + struct.unpack("<I", data[4:8])[0] if len(data) >= 12 else len(data)
|
||||
size = min(len(data), declared_end)
|
||||
position = 12
|
||||
while position + 8 <= size:
|
||||
chunk_type = data[position : position + 4]
|
||||
(length,) = struct.unpack("<I", data[position + 4 : position + 8])
|
||||
start = position + 8
|
||||
safe_length = max(0, min(length, size - start))
|
||||
if chunk_type not in RIFF_CODED_IMAGE_CHUNKS:
|
||||
out += chunk_type + data[start : start + safe_length]
|
||||
position = start + safe_length + (safe_length & 1) # chunks are word-aligned
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _isobmff_regions(image_path: Path, head: bytes) -> bytes:
|
||||
"""Header window plus the provenance regions the bounded box walkers find.
|
||||
|
||||
ISOBMFF hides a manifest in a ``uuid``/``jumb`` box that can sit after a
|
||||
multi-megabyte ``mdat``, and a TC260 label in ``moov.udta``. Both walkers seek
|
||||
rather than read the media, so neither pulls the payload in.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.isobmff import scan_c2pa_region, tc260_aigc_payloads
|
||||
|
||||
out = bytearray(head[:HEAD_WINDOW])
|
||||
try:
|
||||
out += scan_c2pa_region(image_path)
|
||||
except Exception as exc:
|
||||
logger.debug("ISOBMFF C2PA region scan failed on %s: %s", image_path, exc)
|
||||
try:
|
||||
for payload in tc260_aigc_payloads(image_path):
|
||||
out += payload
|
||||
except Exception as exc:
|
||||
logger.debug("ISOBMFF TC260 scan failed on %s: %s", image_path, exc)
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _container_regions(image_path: Path, head: bytes) -> tuple[str, bytes]:
|
||||
"""(container label, metadata bytes) for the container ``head`` starts with.
|
||||
|
||||
``head`` must be the file's raw first bytes. Handing this the ``scan_head``
|
||||
buffer instead is a trap: that buffer is the head CONCATENATED with late metadata
|
||||
payloads, so a structural walk runs off the end of the real head and parses the
|
||||
appended bytes as chunks, inflating the record and creating false signals.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.isobmff import is_isobmff
|
||||
from remove_ai_watermarks.metadata import png_late_metadata, riff_late_metadata
|
||||
|
||||
if head.startswith(b"\xff\xd8"):
|
||||
return "jpeg", _jpeg_regions(head)
|
||||
if head.startswith(PNG_SIGNATURE):
|
||||
# Chunks placed after the pixel stream (an XMP packet at 2.7 MB, say) are
|
||||
# past the window; the same seek-past-IDAT reader the file path uses gets them.
|
||||
return "png", _png_regions(head) + png_late_metadata(image_path, HEAD_WINDOW)
|
||||
if head.startswith(b"RIFF") and head[8:12] == b"WEBP":
|
||||
return "webp", _riff_regions(head) + riff_late_metadata(image_path, HEAD_WINDOW)
|
||||
if is_isobmff(head):
|
||||
return "isobmff", _isobmff_regions(image_path, head)
|
||||
return "unknown", head
|
||||
|
||||
|
||||
def _raw_head(image_path: Path) -> bytes:
|
||||
"""The file's first bytes, unmodified -- the input every structural walk needs."""
|
||||
try:
|
||||
with open(image_path, "rb") as handle:
|
||||
return handle.read(HEAD_WINDOW)
|
||||
except OSError as exc:
|
||||
logger.debug("head read failed for %s: %s", image_path, exc)
|
||||
return b""
|
||||
|
||||
|
||||
def _trailer(image_path: Path, container: str) -> bytes:
|
||||
"""The bytes that follow the container's end marker, and nothing else.
|
||||
|
||||
A fixed-size tail read would be almost entirely pixels: the trailer of a 20 MB
|
||||
photo is a few kilobytes at most. So the end marker is located in the tail window
|
||||
and only what follows it is kept. When no marker is found (an unknown container,
|
||||
or one whose end lies before the window) the window is kept as-is, bounded --
|
||||
that is what a byte scan of the same file would have seen anyway.
|
||||
"""
|
||||
if container == "webp":
|
||||
# RIFF declares its structural end in bytes 4..8. A fixed tail window is
|
||||
# normally the last animation/frame payload, not a trailer, so preserve
|
||||
# only bytes appended after the declared RIFF container.
|
||||
try:
|
||||
with open(image_path, "rb") as handle:
|
||||
header = handle.read(12)
|
||||
if len(header) < 12 or not header.startswith(b"RIFF"):
|
||||
return b""
|
||||
declared_end = 8 + struct.unpack("<I", header[4:8])[0]
|
||||
handle.seek(0, 2)
|
||||
file_size = handle.tell()
|
||||
if declared_end < 12 or declared_end >= file_size:
|
||||
return b""
|
||||
handle.seek(declared_end)
|
||||
return handle.read(min(file_size - declared_end, UNKNOWN_TRAILER_WINDOW))
|
||||
except OSError as exc:
|
||||
logger.debug("RIFF trailer read failed for %s: %s", image_path, exc)
|
||||
return b""
|
||||
if container == "isobmff":
|
||||
# ISOBMFF has no out-of-container trailer convention. Its bounded box
|
||||
# walkers already collect late provenance while skipping ``mdat``; keeping
|
||||
# a blind tail here would carry coded media bytes.
|
||||
return b""
|
||||
|
||||
tail = read_file_tail(image_path, TAIL_WINDOW)
|
||||
if SAMSUNG_EDITOR_MARKER in tail:
|
||||
# Galaxy AI splits its evidence: the marker sits in the post-EOI trailer, but
|
||||
# the `genAIType` value it is gated on can sit INSIDE the entropy-coded scan.
|
||||
# Keeping only the trailer therefore carries the marker without the value and the
|
||||
# verdict silently drops the Samsung signal, so a marked file keeps the whole
|
||||
# window. Only Samsung-marked files pay for it.
|
||||
return tail
|
||||
marker = {"jpeg": b"\xff\xd9", "png": b"IEND\xae\x42\x60\x82"}.get(container)
|
||||
if marker is None:
|
||||
return tail[-UNKNOWN_TRAILER_WINDOW:]
|
||||
index = tail.rfind(marker)
|
||||
return tail[index + len(marker) :] if index >= 0 else tail[-UNKNOWN_TRAILER_WINDOW:]
|
||||
|
||||
|
||||
def _decoder_info(image_path: Path) -> dict[str, Any]:
|
||||
"""PIL's ``info`` mapping, read once.
|
||||
|
||||
One open for both consumers below. They want different parts of the same mapping
|
||||
(the text keys, and the raw EXIF blob), and opening twice repeats the container
|
||||
header parse and, for a PNG carrying ``zTXt``, the zlib inflate with it.
|
||||
"""
|
||||
try:
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(image_path) as img:
|
||||
# PIL types this mapping with a non-string key union (a DPI tuple key
|
||||
# exists), so the keys are normalized here rather than assumed.
|
||||
return {str(key): value for key, value in img.info.items()}
|
||||
except Exception as exc: # a container PIL cannot open
|
||||
logger.debug("PIL info unavailable for %s: %s", image_path, exc)
|
||||
return {}
|
||||
|
||||
|
||||
def _exif_pairs(info: dict[str, Any]) -> dict[str, str]:
|
||||
"""The 0th-IFD tags the verdict reads, under their tag NAMES.
|
||||
|
||||
Not a convenience: two probes key on names rather than on the raw bytes already
|
||||
in the regions. ``xai_signature_pair`` wants an (ImageDescription, Artist) pair,
|
||||
and ``_external_exif_generator`` looks for Software / Make / Artist /
|
||||
ImageDescription. Ship the bytes alone and both silently return nothing, which
|
||||
is how a collector can silently lose Grok and NovelAI verdicts.
|
||||
"""
|
||||
exif_bytes = info.get("exif")
|
||||
if not exif_bytes:
|
||||
return {}
|
||||
try:
|
||||
import piexif
|
||||
|
||||
loaded = piexif.load(exif_bytes)
|
||||
tags = loaded.get("0th", {})
|
||||
exif_tags = loaded.get("Exif", {})
|
||||
except Exception as exc: # malformed EXIF
|
||||
logger.debug("EXIF parse failed: %s", exc)
|
||||
return {}
|
||||
|
||||
pairs = {
|
||||
name: text
|
||||
for name, tag in (
|
||||
("Software", piexif.ImageIFD.Software),
|
||||
("Make", piexif.ImageIFD.Make),
|
||||
("Artist", piexif.ImageIFD.Artist),
|
||||
("ImageDescription", piexif.ImageIFD.ImageDescription),
|
||||
)
|
||||
if (text := exif_text(tags, tag))
|
||||
}
|
||||
if text := exif_text(exif_tags, piexif.ExifIFD.UserComment):
|
||||
pairs["UserComment"] = text
|
||||
return pairs
|
||||
|
||||
|
||||
def _pil_info(info: dict[str, Any]) -> dict[str, str]:
|
||||
"""PIL's ``info`` mapping as strings, the source of PNG text keys and ``hf-job-id``."""
|
||||
|
||||
def text_of(value: Any) -> str:
|
||||
return value.decode("utf-8", "replace") if isinstance(value, bytes) else str(value)
|
||||
|
||||
# Emitted in the file path's own candidate order. ``generator_from_metadata``
|
||||
# returns the FIRST candidate carrying a known token. A record using PIL's natural
|
||||
# dict order can therefore choose a different platform string than the file path,
|
||||
# even though the two paths are supposed to be indistinguishable.
|
||||
out: dict[str, str] = {}
|
||||
for key in _GENERATOR_TEXT_KEYS:
|
||||
value = info.get(key)
|
||||
if value is not None and not isinstance(value, (dict, list, tuple)):
|
||||
out[f"info:{key}"] = text_of(value)
|
||||
for key, value in info.items():
|
||||
if key in _GENERATOR_TEXT_KEYS or isinstance(value, (dict, list, tuple)):
|
||||
continue
|
||||
out[f"info:{key}"] = text_of(value)
|
||||
return out
|
||||
|
||||
|
||||
def collect_metadata_record(
|
||||
image_path: Path,
|
||||
*,
|
||||
schema_version: int = METADATA_RECORD_SCHEMA_VERSION,
|
||||
) -> dict[str, Any]:
|
||||
"""Collect everything the provenance verdict reads, as a JSON-safe record.
|
||||
|
||||
The record is the transport format for
|
||||
:func:`identify.evidence_from_metadata_record`: it carries the metadata regions
|
||||
(base64), the C2PA manifest store, and PIL's info mapping without carrying the
|
||||
primary coded-pixel stream. Schema and collection status are explicit so a
|
||||
consumer cannot mistake a failed read for an unknown provenance verdict.
|
||||
|
||||
Args:
|
||||
image_path: Path to the image.
|
||||
schema_version: Output schema implemented by the consumer.
|
||||
|
||||
Returns:
|
||||
A versioned JSON-serializable dict. ``metadata_base64`` holds the
|
||||
concatenated container regions, ``tail_base64`` the file trailer.
|
||||
"""
|
||||
schema_version = require_schema_version(
|
||||
schema_version,
|
||||
contract="provenance metadata",
|
||||
supported=(1,),
|
||||
)
|
||||
|
||||
from remove_ai_watermarks._internal.c2pa import read_manifest_store_json
|
||||
|
||||
try:
|
||||
image_path.stat()
|
||||
status = "complete"
|
||||
issues: list[dict[str, str]] = []
|
||||
except OSError as exc:
|
||||
logger.debug("metadata source unavailable for %s: %s", image_path, exc)
|
||||
status = "error"
|
||||
issues = [{"stage": "source", "code": "unavailable"}]
|
||||
|
||||
container, regions = _container_regions(image_path, _raw_head(image_path))
|
||||
|
||||
info = _decoder_info(image_path)
|
||||
record: dict[str, Any] = {
|
||||
"schema_version": schema_version,
|
||||
"record_type": METADATA_RECORD_TYPE,
|
||||
"status": status,
|
||||
"issues": issues,
|
||||
"container": container,
|
||||
"name": image_path.name,
|
||||
"metadata_base64": base64.b64encode(regions).decode("ascii"),
|
||||
# Always collected: Samsung's Galaxy AI marker is a post-EOI trailer, and a
|
||||
# record without it loses that verdict outright.
|
||||
"tail_base64": base64.b64encode(_trailer(image_path, container)).decode("ascii"),
|
||||
# PIL info BEFORE exif: the file path prefers a PNG text tag over an EXIF
|
||||
# one, and the normalizer walks the record in insertion order.
|
||||
"pil": _pil_info(info),
|
||||
"exif": _exif_pairs(info),
|
||||
}
|
||||
store = read_manifest_store_json(image_path)
|
||||
if store is not None:
|
||||
record["c2pa_store"] = store
|
||||
return record
|
||||
Loaded 100 of 170 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user