mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-18 23:37:14 +02:00
Compare commits
No files matched your search
@@ -1,6 +1,6 @@
|
||||
---
|
||||
globs: ["src/**/*.py", "tests/**/*.py", "scripts/**/*.py", "pyproject.toml", "uv.lock", "maintain.sh", ".github/workflows/*.yml"]
|
||||
description: Command contracts, project gate, typing boundaries, and model-adjacent test invariants.
|
||||
description: Command contracts, project gate, typing boundaries, model-adjacent test invariants, and the detection-path measurement rule.
|
||||
---
|
||||
|
||||
# Development invariants
|
||||
@@ -11,23 +11,129 @@ Every single-image command declares `source` with `dir_okay=False`; `batch` decl
|
||||
|
||||
Exit-code and no-signal behavior is a public contract. Read the command-line section of [`../../docs/module-internals.md`](../../docs/module-internals.md) before changing it.
|
||||
|
||||
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
|
||||
|
||||
`device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. It is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds.
|
||||
|
||||
The same rule applies to install hints: name the extra that actually makes the command work (`qwen-zimage`, not `diffusion`), and keep the printed command shell-quoted -- bare `pkg[extra]` is a glob in zsh.
|
||||
|
||||
## Local gate
|
||||
|
||||
Run `bash maintain.sh` from the repository root. The authoritative type gate is scoped to `src/`; full-project Pyright can exhaust Node memory on the ML dependency graph.
|
||||
|
||||
Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.
|
||||
|
||||
From a worktree, `uv run` imports the package from the MAIN checkout -- that is where the editable install points. A script measuring a worktree's edit must insert that worktree's `src` at `sys.path[0]` and assert `module.__file__` resolves inside it, or it silently compares unmodified code against itself.
|
||||
|
||||
## Model-adjacent tests
|
||||
|
||||
Do not classify an entire module as untestable because its main path downloads a model. Keep pure behavior covered without downloads, including:
|
||||
|
||||
- target-size selection in `test_invisible_engine.py`;
|
||||
- unsharp and adaptive-polish helpers in `test_humanizer.py`;
|
||||
- mocked device fallback in `test_img2img_runner.py`;
|
||||
- tiling geometry and blending in `test_tiling.py`;
|
||||
- prompt-embedding cache keying, storage round-trip, and the cross-pipeline reuse
|
||||
that lets a stack load without its text encoder, in `test_qwen_zimage_pipeline.py`.
|
||||
that lets a stack load without its text encoder, in `test_qwen_zimage_pipeline.py`;
|
||||
- the face stack's dtype, in `test_qwen_zimage_pipeline.py`. A subclass that changes
|
||||
the pipeline dtype for its own global model must not change the inherited face
|
||||
stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a
|
||||
face. When one profile inherits another's stage, guard the invariants that stage
|
||||
relies on, not just the code path.
|
||||
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
|
||||
mirror another, compare them field by field rather than pinning the values you happen
|
||||
to know about, so the next field added on one side and not the other fails at the
|
||||
seam. Two of these defaults drifted in practice and neither needed a GPU to catch;
|
||||
the incident is recorded in `docs/module-internals.md`. Keep the comparison free of
|
||||
an exception table: a field that needs one is a field that belongs elsewhere, which
|
||||
is what `force` turned out to be.
|
||||
|
||||
A defaults comparison is not a forwarding test, and the two fail differently. Pin the
|
||||
VALUE at the seam, not just the name -- `_run_invisible` passed the whole suite with
|
||||
`controlnet_conditioning_scale` hardcoded, because nothing asserted the caller's value
|
||||
arrived. `test_every_field_arrives_at_the_engine_with_the_caller_s_value` drives the
|
||||
real seam with every field set off its default, so one test covers the whole bag
|
||||
instead of one assertion per knob.
|
||||
|
||||
Count the seams before believing a knob is covered. Each of `force` and
|
||||
`controlnet_conditioning_scale` reaches the engine through TWO paths -- `remove_all`
|
||||
versus `remove_batch(mode="all")` for the first, `_run_invisible` versus `_batch_engine`
|
||||
for the second -- and in both cases guarding one path left the other free to hardcode a
|
||||
constant with a green suite. The mode-parametrized guards in
|
||||
`TestRemoveBatchLibrary::test_force_reaches_the_scrub_gate_in_every_scrubbing_mode` and
|
||||
`TestBatchCommand::test_batch_controlnet_scale_flows_to_the_cached_engine` exist because
|
||||
that is what actually happened.
|
||||
|
||||
Use availability checks only for paths that actually load large models.
|
||||
|
||||
## One measurement, one gate seam
|
||||
|
||||
A detector is split into a trust-level-blind scan and a verdict that applies the
|
||||
threshold, so `detect` and `detect_both` reach the same numbers by construction. Two
|
||||
rules follow, and both were broken in practice before they were written down:
|
||||
|
||||
- A per-mark demotion goes in the `_post_gate` hook (or, for a whole-scan precondition
|
||||
like LibLibAI's size floor, in `_scan`) -- never in a `detect` override. An override
|
||||
is invisible to `detect_both`, so the RunningHub and Yuanbao anchor gates silently
|
||||
stopped applying on the arbiter's perception path. `TestSinglePassPerception` is the
|
||||
guard: it asserts `detect_both` equals two `detect` calls field for field.
|
||||
- Detection and the removal mask must read ONE sweep. The winning box travels on
|
||||
`TextMarkDetection.match_box` and the registry threads the detection into the mask
|
||||
builder; a mask path that re-runs its own sweep is how the two drift apart.
|
||||
|
||||
The C2PA manifest-store JSON is NOT stable across reads: the reader regenerates manifest
|
||||
URNs and instance ids. Compare the derived `c2pa_info`, never the raw store.
|
||||
|
||||
A third seam reaches the same verdict: `collect_metadata_record` ->
|
||||
`evidence_from_metadata_record` -> `identify_from_evidence`, the path a caller uses when
|
||||
collection and verdict run on different machines. Its contract is equality with
|
||||
`identify(path, check_visible=False, check_invisible=False)` on the same image, and it
|
||||
can break from EITHER side -- a region the collector stops walking, or a placement the
|
||||
file path learns to read and the record does not. `tests/test_metadata_record.py` pins
|
||||
it over the tracked fixtures; a separate local evaluation corpus catches placements the
|
||||
fixtures do not cover. Change either side and re-run both.
|
||||
|
||||
Before changing anything in the detection path, record the detectors' exact verdicts
|
||||
over a local sample first and diff them after. A refactor here is only correct if that
|
||||
record is byte-identical, and a green test suite does not establish that on its own. A
|
||||
change that is meant to FIX detection is the exception that proves the rule: the diff
|
||||
must then be exactly the files you intended to change, named in advance.
|
||||
|
||||
Two corollaries in `dwt_dct.py`, where "close enough" has an exact meaning. The bit test
|
||||
is `peak % 36 > 18.0` and for uint8 input the exact Haar value is a multiple of 0.5, so it
|
||||
lands ON the threshold once per 72 blocks and a 1-ulp difference flips real bits: leave the
|
||||
transform to `pywt` however slow it looks, because its C convolution contracts into an FMA
|
||||
that numpy has no ufunc for (`docs/module-internals.md` carries what that costs). And
|
||||
`tests/test_invisible_watermark.py` is `skipif(not is_available())` -- without the `detect`
|
||||
extra the upstream-parity test never runs, so a green suite is not evidence here and the
|
||||
verdict record is not optional.
|
||||
|
||||
## A certified operating point is data, not a constant
|
||||
|
||||
The video SynthID default is only meaningful as a row in
|
||||
`data/evaluations/video-synthid-oracle.csv`, so
|
||||
`test_shipped_defaults_match_a_certified_manifest_row` derives the pin from that
|
||||
manifest instead of restating literals. Pinning `noise_std` alone had let `long_side`
|
||||
and `fps` -- two thirds of what the oracle was actually shown -- move with a green
|
||||
suite.
|
||||
|
||||
The certified profile is a perturbation-to-signal ratio, not a bare `noise_std`, so
|
||||
the latent scaling factor is gated in `load_video_vae_runtime`, carried on
|
||||
`VideoVaeRuntime`, and passed into encode and decode: the validated value and the
|
||||
applied value are one measurement. Anything that produces oracle evidence loads
|
||||
through that function. `video_synthid_sweep.py` hand-rolled the load and was the one
|
||||
path exempt from the gate it exists to feed, which is exactly backwards.
|
||||
|
||||
Prove a video-path refactor the same way the detection path is proven, and without
|
||||
needing an oracle carrier: build a clip from a tracked fixture with ffmpeg, run the
|
||||
engine before and after, and require an identical output sha256. Keep the generated
|
||||
media outside the repository.
|
||||
|
||||
Frame sampling is compared as `timestamp + 1e-9 >= next_sample_time`, so mutating
|
||||
that `>=` to `>` changes nothing and a green suite proves nothing. Mutate the phase
|
||||
or the period instead -- starting the accumulator half a period late shifts the
|
||||
selection by one frame while keeping the count identical, which is the drift a
|
||||
frame-count check cannot see and what
|
||||
`test_pairing_follows_the_engine_sampling_rule_not_just_the_frame_count` exists to
|
||||
catch.
|
||||
|
||||
Environment setup, dependency recovery, CI behavior, and fixture policy: [`../../docs/development.md`](../../docs/development.md).
|
||||
@@ -1,14 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"WebSearch",
|
||||
"WebFetch"
|
||||
]
|
||||
},
|
||||
"enabledPlugins": {
|
||||
"pyright-lsp@claude-plugins-official": true,
|
||||
"context7@claude-plugins-official": true,
|
||||
"code-simplifier@claude-plugins-official": true,
|
||||
"claude-md-management@claude-plugins-official": true
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,8 @@
|
||||
name: Distribute on release
|
||||
|
||||
# Fans a published GitHub Release out to the channels that need a nudge.
|
||||
# PyPI is handled by publish.yml; the conda-forge channel is handled by its
|
||||
# autotick bot. This workflow updates the repository recipe and event-drives
|
||||
# the three channels that would otherwise be manual:
|
||||
# PyPI is handled by publish.yml. This workflow event-drives the three channels
|
||||
# that would otherwise be manual:
|
||||
# - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
|
||||
# - HF Space: factory-rebuild so it reinstalls the latest sdist from PyPI.
|
||||
# - ComfyUI: sync, test, and publish the node against the exact library release.
|
||||
@@ -89,34 +88,6 @@ jobs:
|
||||
git commit -m "remove-ai-watermarks $VERSION" || { echo "Formula already current"; exit 0; }
|
||||
git push
|
||||
|
||||
conda-recipe:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- name: Checkout main
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
ref: main
|
||||
- name: Bump the repository conda recipe
|
||||
env:
|
||||
SHA: ${{ needs.resolve.outputs.sha }}
|
||||
VERSION: ${{ needs.resolve.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
python scripts/sync_conda_recipe.py --version "$VERSION" --sha256 "$SHA"
|
||||
git diff --check
|
||||
if git diff --quiet -- packaging/conda/recipe.yaml; then
|
||||
echo "Conda recipe already current"
|
||||
exit 0
|
||||
fi
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git add packaging/conda/recipe.yaml
|
||||
git commit -m "Sync conda recipe with v$VERSION"
|
||||
git push
|
||||
|
||||
hf-space:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -28,13 +28,26 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Cross-OS x Python-floor/recent. The default-install surface only
|
||||
# needs the core deps plus dev tooling (no GPU extra): the model-running
|
||||
# paths are availability-gated and skip when torch/diffusers are absent,
|
||||
# so this matrix exercises metadata, identify, visible, and the cv2
|
||||
# eraser on every OS without pulling the heavy ML stack.
|
||||
os: [ubuntu-latest, macos-latest, windows-latest]
|
||||
python-version: ["3.10", "3.12"]
|
||||
# Exercise every supported minor. The floor and latest minor run on all
|
||||
# operating systems; intermediate minors run on Linux to bound the
|
||||
# matrix without leaving a declared Python version untested.
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.11"
|
||||
- os: macos-latest
|
||||
python-version: "3.11"
|
||||
- os: windows-latest
|
||||
python-version: "3.11"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.12"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.13"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.14"
|
||||
- os: macos-latest
|
||||
python-version: "3.14"
|
||||
- os: windows-latest
|
||||
python-version: "3.14"
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
@@ -45,6 +58,27 @@ jobs:
|
||||
- name: Run tests
|
||||
run: uv run pytest -q
|
||||
|
||||
transport-contract-python-314:
|
||||
name: versioned transport contracts py3.14
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
python-version: "3.14"
|
||||
- name: Create clean runtime contract environment
|
||||
run: uv venv --python 3.14 .venv-contract
|
||||
- name: Install runtime contract dependencies
|
||||
run: >-
|
||||
uv pip install --python .venv-contract/bin/python
|
||||
".[pixels,heif]" "numpy>=2,<3" "pytest>=8"
|
||||
- name: Run versioned transport contract tests
|
||||
run: >-
|
||||
.venv-contract/bin/python -m pytest -q
|
||||
tests/test_forensic_metadata.py
|
||||
tests/test_metadata_record.py
|
||||
tests/test_pixel_evidence.py
|
||||
|
||||
video-e2e:
|
||||
name: video full-clip end-to-end
|
||||
runs-on: ubuntu-latest
|
||||
@@ -62,3 +96,4 @@ jobs:
|
||||
run: >-
|
||||
uv run pytest -vv -o faulthandler_timeout=60
|
||||
tests/test_video.py::TestVideoVisibleFullClip
|
||||
tests/test_video_fidelity_probe.py
|
||||
+14
-14
@@ -20,8 +20,7 @@ Thumbs.db
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# SynthID corpus reference fills (synthetic black/white calibration tiles,
|
||||
# regenerable; the labeled pos/neg/cleaned images ARE tracked, see README)
|
||||
# Generated calibration inputs
|
||||
data/synthid_corpus/refs/
|
||||
|
||||
# Reference materials
|
||||
@@ -35,11 +34,7 @@ yolov8n.pt
|
||||
.claude/settings.local.json
|
||||
.claude/scheduled_tasks.lock
|
||||
|
||||
# Visible-watermark alpha calibration. The solid black/gray/white CAPTURES are
|
||||
# committed (content-free: a solid colour + the watermark; the source for
|
||||
# scripts/visible_alpha_solve.py so the alpha assets are reproducible). The
|
||||
# synthetic seeds (regenerable) and any real-content validation download (a real
|
||||
# generated scene, kept local for privacy) are NOT committed.
|
||||
# Generated calibration inputs and local evaluation artifacts
|
||||
data/doubao_capture/seeds/
|
||||
data/jimeng_capture/seeds/
|
||||
data/jimeng_capture/captures/jimeng_content_*.png
|
||||
@@ -48,13 +43,18 @@ data/gemini_capture/captures/gemini_content_*.png
|
||||
data/samsung_capture/seeds/
|
||||
data/samsung_capture/captures/samsung_content_*
|
||||
|
||||
# Leftover GFPGAN weights dir from the retired face-restore experiments
|
||||
# (GFPGAN wrote RetinaFace/parsing weights to a CWD ./gfpgan/weights/ working
|
||||
# dir on first use). Runtime artifact, never committed.
|
||||
# Runtime model artifacts
|
||||
gfpgan/
|
||||
|
||||
# Qwen ControlNet experiment outputs (throwaway eval; never the committed corpus)
|
||||
scripts/_qwen_exp_out/
|
||||
|
||||
# Local-only working data for analysis (not a committed corpus; never tracked)
|
||||
# Local evaluation data
|
||||
.local-eval/
|
||||
|
||||
# Other harnesses' generated config. Claude's sync owns these; a committed copy
|
||||
# is a second source nobody reconciles. Anchored so a same-named source file
|
||||
# deeper in the tree is not caught.
|
||||
/.agents/
|
||||
/.codex/
|
||||
/sch_*.yaml
|
||||
/.opencode/
|
||||
/opencode.json
|
||||
/opencode.jsonc
|
||||
@@ -33,7 +33,7 @@ Optional features and installation groups are documented in [`docs/installation.
|
||||
|
||||
Command, gate, typing, and model-test invariants auto-load from [`.claude/rules/development.md`](.claude/rules/development.md). Environment recovery, CI behavior, and fixture policy live in [`docs/development.md`](docs/development.md).
|
||||
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Keep the source-distribution public allowlist.
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Treat the release as complete only after PyPI, Homebrew, the Hugging Face Space, and the ComfyUI Registry are verified; conda is not published. Keep the source-distribution public allowlist.
|
||||
|
||||
## Module architecture
|
||||
|
||||
@@ -51,4 +51,4 @@ Topic-specific rules live in `.claude/rules/*.md` and are auto-loaded when match
|
||||
|
||||
| File | Covers |
|
||||
|---|---|
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, and model-adjacent tests |
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, and the detection-path measurement rule |
|
||||
@@ -11,8 +11,9 @@ cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo, and
|
||||
Kling mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
removal.
|
||||
|
||||
> Try it online at [raiw.cc](https://raiw.cc) if you do not want to install Python
|
||||
> or run diffusion models locally.
|
||||
> [raiw.cc](https://raiw.cc) runs this library as a hosted service, with the GPU
|
||||
> included and nothing to install. Visible mark and metadata removal are free
|
||||
> there; invisible watermark removal is paid.
|
||||
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
@@ -39,7 +40,7 @@ removal.
|
||||
| Remove a registered visible AI mark from video | `video visible` | No |
|
||||
| Process a directory of videos | `video batch` | Depends on mode |
|
||||
| Remove video SynthID with the certified VAE profile | `video invisible` | Recommended |
|
||||
| Regenerate an image to disrupt invisible watermarks | `invisible` | Recommended |
|
||||
| Regenerate an image to disrupt invisible watermarks | `invisible` | Required (CUDA) |
|
||||
| Run visible, invisible, and metadata removal | `all` | Recommended |
|
||||
| Process a directory | `batch` | Depends on mode |
|
||||
|
||||
@@ -52,13 +53,13 @@ removal.
|
||||
| Visible video processing | `remove-ai-watermarks[video]` |
|
||||
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
|
||||
| Torch-free DWT-DCT detection | `remove-ai-watermarks[detect]` |
|
||||
| Diffusion removal | `remove-ai-watermarks[diffusion]` |
|
||||
| Every production feature | `remove-ai-watermarks[all]` |
|
||||
| Invisible image removal (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| Every production feature available on the active Python | `remove-ai-watermarks[all]` |
|
||||
|
||||
Lower-level and specialized extras include `pixels`, `heif`, `trustmark`,
|
||||
`migan`, `lama`, `esrgan`, and `qwen-zimage`. The
|
||||
`migan`, `lama`, and `diffusion`. The
|
||||
[installation guide](docs/installation.md#feature-extras) documents their exact
|
||||
dependency composition and model requirements.
|
||||
dependency composition, Python compatibility, and model requirements.
|
||||
|
||||
## Quick start
|
||||
|
||||
@@ -92,6 +93,8 @@ Strip metadata without running visible inpainting or diffusion:
|
||||
remove-ai-watermarks metadata image.png --remove -o clean.png
|
||||
```
|
||||
|
||||
Without `-o` this command overwrites the source in place.
|
||||
|
||||
Inspect or remove AI metadata from an MP4, MOV, M4V, WebM, MKV, AVI, or FLV
|
||||
file:
|
||||
|
||||
@@ -100,10 +103,13 @@ remove-ai-watermarks video metadata input.mp4 --check
|
||||
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
|
||||
```
|
||||
|
||||
The metadata command does not transcode video or audio streams. When `-o` is
|
||||
omitted it writes `<source>_clean` and preserves the original. MP4 and MOV
|
||||
The `video metadata` command does not transcode video or audio streams. Unlike
|
||||
the image command above, when `-o` is omitted it writes `<source>_clean` and
|
||||
preserves the original. MP4 and MOV
|
||||
inspection includes the native TC260 `AIGC` tag in
|
||||
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload.
|
||||
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload,
|
||||
plus the QuickTime-form `meta` variants Doubao's iOS export writes (a bare
|
||||
`meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list).
|
||||
MKV and WebM inspection reads the normative
|
||||
`Segment.Tags.Tag.SimpleTag` placement. AVI uses `LIST/INFO/AIGC`, while FLV
|
||||
uses `script.onMetaData.AIGC`. The non-ISOBMFF formats are remuxed with stream
|
||||
@@ -180,10 +186,11 @@ check. Google does not publish a local decoder, so a fresh provider check
|
||||
remains useful for unusually important files or after provider changes, but it
|
||||
is not a product result state.
|
||||
|
||||
For invisible watermark removal, install the diffusion dependencies:
|
||||
For invisible watermark removal, install the `qwen-zimage` extra. **An NVIDIA GPU
|
||||
is required**: both profiles are CUDA-only, and there is no CPU or MPS fallback.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[diffusion]"
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png
|
||||
```
|
||||
|
||||
@@ -194,6 +201,20 @@ image came from an AI generator, add `--force`:
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
Typography-heavy images can opt into the experimental verified-text post-pass.
|
||||
It accepts manually reviewed strings and line boxes or an operator-verified
|
||||
geometry-only manifest; it never treats raw OCR output as ground truth or runs
|
||||
automatically:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
See the [CLI guide](docs/cli.md#restore-operator-verified-text) for the manifest
|
||||
schema, compatibility restrictions, and oracle caveats.
|
||||
|
||||
See the [installation guide](docs/installation.md) for Homebrew, uv, optional
|
||||
features, and development setup.
|
||||
|
||||
@@ -207,14 +228,14 @@ features, and development setup.
|
||||
|
||||
### High quality invisible removal
|
||||
|
||||
The `qwen-zimage` profile is the highest fidelity option for face heavy images.
|
||||
It is CUDA only and uses a much larger model stack than the default ControlNet
|
||||
profile.
|
||||
`qwen-zimage` is the default profile: a Qwen-Image-2512 Lightning pass under Canny
|
||||
ControlNet, followed by SAM-masked Z-Image repair of any detected face. The
|
||||
alternative, `sdxl-zimage`, swaps the global stage for SDXL and keeps the same face
|
||||
stage. Both are CUDA only.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --force
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
| OpenAI example before | OpenAI example after |
|
||||
@@ -273,7 +294,7 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
```
|
||||
|
||||
CPU offload lowers CUDA memory pressure by moving model components between CPU
|
||||
and GPU. It is slower and has no effect on CPU or MPS.
|
||||
and GPU, at the cost of speed.
|
||||
|
||||
### Process a directory
|
||||
|
||||
@@ -287,7 +308,7 @@ remove-ai-watermarks batch ./images --mode all
|
||||
Visible mark support includes:
|
||||
|
||||
- Google Gemini and Nano Banana sparkle;
|
||||
- Doubao, Jimeng, Qwen, Kling, Baidu, LibLibAI, and RunningHub labels;
|
||||
- Doubao, Jimeng, Qwen, Kling, Yuanbao, Baidu, LibLibAI, and RunningHub labels;
|
||||
- one calibrated Samsung Galaxy AI label variant.
|
||||
|
||||
Metadata and provenance inspection covers C2PA, EXIF, XMP, IPTC, common
|
||||
@@ -350,8 +371,9 @@ The high level API accepts a file path or a BGR NumPy array. For path inputs it
|
||||
also reads provenance metadata, preserves alpha, and can strip AI metadata from
|
||||
the written result.
|
||||
|
||||
See the [Python API guide](docs/python-api.md) for visible removal, provenance
|
||||
inspection, metadata stripping, and diffusion usage.
|
||||
See the [Python API guide](docs/python-api.md) for visible removal, the full
|
||||
`remove_all` and `remove_batch` pipeline, provenance inspection, metadata
|
||||
stripping, and diffusion usage.
|
||||
|
||||
## ComfyUI
|
||||
|
||||
@@ -378,8 +400,9 @@ invisible removal.
|
||||
The shipped profile is oracle-certified, but no public local decoder can
|
||||
certify an arbitrary output at runtime. Recheck unusually important outputs
|
||||
after provider changes.
|
||||
- `qwen-zimage` requires CUDA. The other diffusion profiles also support the
|
||||
devices listed by `remove-ai-watermarks invisible --help`.
|
||||
- Invisible-watermark removal requires CUDA. Both profiles refuse any other
|
||||
device at construction rather than falling back to one that cannot run them.
|
||||
Visible removal, metadata stripping and `identify` still run anywhere.
|
||||
- Provider watermark systems can change. Validate important outputs with the
|
||||
provider's own verifier when one is available.
|
||||
|
||||
@@ -392,6 +415,14 @@ verifier verdict blank:
|
||||
uv run --extra video --extra diffusion python scripts/video_synthid_sweep.py input.mp4 -o sweep/
|
||||
```
|
||||
|
||||
To score what an output actually cost, use `scripts/video_fidelity_probe.py`:
|
||||
the engine's own PSNR is measured before the resize and the encode, so only the
|
||||
probe sees the delivered picture.
|
||||
|
||||
```bash
|
||||
uv run --extra video python scripts/video_fidelity_probe.py input.mp4 input_clean.mp4
|
||||
```
|
||||
|
||||
The control must still be SynthID-positive before a negative candidate can
|
||||
count as removal evidence. In the 2026-07-29 two-clip calibration, both matched
|
||||
controls were positive in Gemini's built-in SynthID verifier; the stronger
|
||||
|
||||
+41
-1
@@ -6,6 +6,7 @@ Tracked data is organized by purpose:
|
||||
data/
|
||||
fixtures/
|
||||
provenance/ Real format and provenance fixtures used by tests
|
||||
(source records live in fixtures/README.md)
|
||||
calibration/
|
||||
<vendor>/ Minimal controlled inputs needed to rebuild detector assets
|
||||
synthid/
|
||||
@@ -31,10 +32,49 @@ data/
|
||||
commands, hashes, and oracle verdicts instead of committing another corpus
|
||||
copy. A small curated before-and-after example may live in `docs/images/`
|
||||
when it is part of the public documentation.
|
||||
6. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
6. Keep third-party fixture license notices in `licenses/`, outside directories
|
||||
that tests enumerate as media inputs.
|
||||
7. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
Unregistered research candidates belong in
|
||||
`scripts/assets/visible-mark-candidates/` so they are not shipped in the
|
||||
wheel.
|
||||
|
||||
The source distribution excludes `data/`; the wheel contains only package
|
||||
runtime assets.
|
||||
|
||||
## Video SynthID oracle manifest
|
||||
|
||||
`evaluations/video-synthid-oracle.csv` is the only evidence that the shipped
|
||||
video removal profile works, so it is also the source of truth for three shipped
|
||||
defaults: `tests/test_video_invisible.py` asserts that `noise_std`, `long_side`,
|
||||
and `fps` together match a row this manifest records as certified. Changing one
|
||||
of those three without adding the row that certifies it fails the suite. `vae` is
|
||||
deliberately outside that check because neither tracked row records one; add it
|
||||
to the assertion in the same commit as the first row that does.
|
||||
|
||||
| Column | Meaning |
|
||||
| --- | --- |
|
||||
| `date`, `source_url`, `source_sha256` | Identify the carrier. |
|
||||
| `source_width`, `source_height`, `source_fps` | Carrier geometry. Without it the actual downscale factor of a row cannot be recovered later. |
|
||||
| `duration_seconds`, `source_verdict` | Clip length submitted and the verifier's reading of the untouched carrier. |
|
||||
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The full run configuration. |
|
||||
| `output_sha256` | Identifies the exact submitted file. |
|
||||
| `output_verdict` | One of `detected`, `not_detected`, `indeterminate`, `refused`. |
|
||||
| `output_verdict_text` | The verifier's wording, verbatim. |
|
||||
| `output_detected_range` | Time range the verifier reported for the output. |
|
||||
| `track` | `visual`, `audio`, `both`, or empty. The verifier scores tracks separately and this path copies source audio unchanged. |
|
||||
| `session_id` | Groups rows submitted in one oracle session, so per-session drift stays visible. |
|
||||
| `stratum` | Content class of the carrier, for stratified certification. |
|
||||
| `psnr_db`, `temporal_residual_ratio` | Fidelity measurements. Neither is a watermark verdict. |
|
||||
|
||||
Record `indeterminate` when the verifier answers with its unclear state rather
|
||||
than a negative: an unclear reading logged as `not_detected` is exactly the
|
||||
silent regression this manifest exists to prevent. Leave a field empty when it
|
||||
was not recorded, and never backfill it with a plausible value.
|
||||
|
||||
`psnr_db` is measured against the already-resized frame and before the encode,
|
||||
so it excludes the downscale, the decimation, and the codec. Rows stay
|
||||
comparable to each other only while that definition holds.
|
||||
|
||||
The two 2026-07-31 rows predate this schema; their empty fields were never
|
||||
recorded and are not recoverable from the row.
|
||||
@@ -7,31 +7,394 @@ ground truth and instructions.
|
||||
|
||||
| Original | Provider | Content | Exercises |
|
||||
| --- | --- | --- | --- |
|
||||
| `ChatGPT Image May 31, 2026, 02_02_23 PM.png` | OpenAI | Light multilingual typography | Text preservation |
|
||||
| `ChatGPT Image May 31, 2026, 02_03_55 PM.png` | OpenAI | Multilingual typography | Text preservation |
|
||||
| `Gemini_Generated_Image_633uuy633uuy633u.png` | Google | Landscape with a Chinese sign | CJK text preservation |
|
||||
| `Gemini_Generated_Image_y48j3cy48j3cy48j.png` | Google | Portrait grid | Face identity and skin texture |
|
||||
|
||||
## Text ground truth
|
||||
|
||||
`ground-truth.json` contains hand-verified OCR for the two text-bearing
|
||||
`ground-truth.json` contains hand-verified OCR for the three text-bearing
|
||||
originals. To regenerate an OCR seed:
|
||||
|
||||
```bash
|
||||
uv run scripts/fidelity_metrics.py ocr \
|
||||
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png" \
|
||||
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png" \
|
||||
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
|
||||
--langs en,ru,ch \
|
||||
--out data/evaluations/fidelity/ground-truth.json
|
||||
```
|
||||
|
||||
Verify and correct the generated text by hand before using it as ground truth.
|
||||
`text-lines.json` contains the verified per-line strings and source-space boxes
|
||||
used by the evaluation-only selective-restoration experiment. It is not an
|
||||
automatic scene-text annotation set.
|
||||
|
||||
`scripts/infer_text_lines.py` can generate a draft from source pixels without
|
||||
existing annotations. On the two posters it proposed 20 and 18 lines at the
|
||||
default threshold, but exact-text precision was only 90.0% and 94.4% because
|
||||
high-confidence OCR still dropped an English comma and replaced a Chinese comma
|
||||
with ASCII. Its output therefore requires manual verification of every line;
|
||||
`accepted` means crop-stable, not ground-truth-correct.
|
||||
|
||||
## Text-preservation benchmark
|
||||
|
||||
`text-preservation-2026-08-13.csv` records a fixed-seed comparison of the two
|
||||
current profiles and a global Z-Image Turbo prototype on all three text
|
||||
fixtures. Each candidate ran through the
|
||||
complete `visible -> invisible -> metadata` route with its profile default
|
||||
strength and adaptive-polish setting. The Z-Image prototype instead sweeps
|
||||
0.08, 0.10, 0.15, 0.20, and 0.25 without polish because its provider-specific
|
||||
removal floor is not known. The output hash identifies the exact bytes
|
||||
measured; generated outputs remain outside the repository.
|
||||
|
||||
The character-weighted added CER is 0.262 for `qwen-zimage` and 0.256 for
|
||||
`sdxl-zimage`. That 0.006 absolute difference is not a
|
||||
stable ordering: SDXL wins the light poster, Qwen wins the dark poster, and the
|
||||
Chinese sign is tied. A paired sign test on the two non-ties is 1-1 (`p=1.0`).
|
||||
The measured sample therefore does not support a general text-preservation
|
||||
winner. Both profiles substantially degrade the smallest multilingual poster
|
||||
text and preserve the larger Chinese sign.
|
||||
|
||||
Z-Image Turbo is the clear fidelity lead. At strength 0.10 its
|
||||
character-weighted CER is 0.093, against the unchanged sources' 0.124 OCR
|
||||
floor, while whole-image LPIPS stays between 0.047 and 0.086. Visual inspection
|
||||
still finds substitutions in the smallest Cyrillic and Chinese poster lines,
|
||||
so a favorable OCR score does not mean pixel-exact text. Quality drops quickly
|
||||
above 0.15; at 0.25 the weighted CER is 0.428.
|
||||
|
||||
Provider-oracle checks on 2026-08-13 bracket the OpenAI requirement at no more
|
||||
than 0.10 for this sample. Both original OpenAI controls were detected by
|
||||
`openai.com/verify`; the light poster was clean at 0.08, while the dark poster
|
||||
was detected at 0.08 and clean at 0.10. The light 0.10 output was not separately
|
||||
checked. Google fails the entire tested sweep. The original Gemini image was
|
||||
detected through C2PA, and a pixel-identical copy with AI metadata stripped was
|
||||
separately detected by Gemini's built-in SynthID verifier. Z-Image outputs at
|
||||
0.08, 0.10, 0.15, 0.20, and 0.25 were all still detected. At 0.25 the weighted
|
||||
CER has already risen to 0.428, worse than the raw weighted CER of both current
|
||||
profiles (Qwen 0.387, SDXL 0.381). Increasing strength beyond the measured grid
|
||||
would therefore no longer serve the text-preservation objective without a new
|
||||
mechanism or hypothesis. Z-Image is not a viable global replacement on this
|
||||
evidence: no clean Google operating point was found before it lost its fidelity
|
||||
advantage.
|
||||
|
||||
Qwen-Image-2.0 was not added to the numeric comparison. Its weights are not
|
||||
published, and its hosted edit API exposes an editing instruction and seed but
|
||||
no low-strength denoise control. It can be evaluated as a separate hosted edit
|
||||
strategy when credentials are available, but it is not a drop-in replacement
|
||||
for the partial-regeneration mechanism measured here.
|
||||
|
||||
The OCR floor is the source image scored against the hand-verified text. Use
|
||||
`added_cer = text_cer - ocr_floor` when interpreting pipeline damage, because the
|
||||
unchanged poster sources already score 0.127 CER. `oracle_rechecked=true` marks
|
||||
the exact Z-Image bytes checked above; the remaining rows were not rechecked.
|
||||
The table does not certify other seeds, content classes, or strengths beyond
|
||||
the recorded provider verdicts.
|
||||
|
||||
## Text-restoration prototype
|
||||
|
||||
`text-restoration-2026-08-13.csv` evaluates an OCR-driven post-pass on the exact
|
||||
Qwen outputs above. The prototype recognizes English and Russian with macOS
|
||||
Vision and CJK with PaddleOCR, derives glyph masks independently from the
|
||||
source and Qwen output, removes both sets of glyphs with block-wise LaMa, and
|
||||
draws the recognized strings with new system-font pixels. It never composites
|
||||
source pixels back into the result.
|
||||
|
||||
On the two multilingual posters, character-weighted CER fell from 0.338 and
|
||||
0.305 to 0.007 on both. OpenAI Verify reported no OpenAI signals for both Qwen
|
||||
controls and both restored outputs in the same run. The improvement comes with
|
||||
a substantial whole-image fidelity cost: LPIPS rose from 0.107 to 0.174 and
|
||||
from 0.095 to 0.162, while PSNR fell by about 10.5-11.4 dB. Visual inspection
|
||||
found one residual shadow in the smallest English line of the light poster;
|
||||
the dark poster was clean but the substitute fonts visibly changed typography.
|
||||
|
||||
The Chinese sign did not improve: CER rose from 0.074 to 0.111 because the OCR
|
||||
and renderer changed punctuation. The Gemini verifier returned detected for
|
||||
both the restored output and its byte-identical Qwen control on a third work
|
||||
account, although that Qwen hash had previously returned clean on another work
|
||||
account. This run therefore does not isolate a restoration-stage regression;
|
||||
its Google verdict is inconclusive until a source-positive, Qwen-negative
|
||||
control can be reproduced in the same available account.
|
||||
|
||||
This prototype is not ready to ship. Its strongest result establishes that
|
||||
fresh-glyph reconstruction can recover literal text without reintroducing an
|
||||
OpenAI signal, but portable OCR, font/style reconstruction, a tighter mask, and
|
||||
a reproducible Google oracle control remain prerequisites.
|
||||
|
||||
### Selective restoration follow-up
|
||||
|
||||
`selective-text-restoration-2026-08-13.csv` compares that full compositor with
|
||||
a selective prototype on the same two posters. The prototype leaves a Qwen line
|
||||
unchanged when padded source and output recognition agree, and applies the same
|
||||
LaMa plus fresh-system-font reconstruction only to lines whose recognized text
|
||||
changed. This reduced the edited area from 15.4% and 17.0% to 5.5% on both
|
||||
posters.
|
||||
|
||||
Under one consistent Paddle `en+ru+ch` measurement route, selective restoration
|
||||
reduced Qwen CER from 0.378 to 0.101 on the light poster and from 0.413 to 0.112
|
||||
on the dark poster. Its image LPIPS was 0.120 and 0.103, substantially closer to
|
||||
Qwen than the full compositor's 0.174 and 0.162. These CER values must not be
|
||||
mixed with the preceding table's Vision/Paddle hybrid values: the comparison
|
||||
file remeasures all three variants through Paddle so their relative result is
|
||||
valid on one OCR route.
|
||||
|
||||
OpenAI Verify returned `No OpenAI signals detected` for both selective outputs,
|
||||
then detected the original light poster as `Generated with OpenAI tools` in the
|
||||
same Chrome sequence. Visual inspection found the dark output clean, but the
|
||||
light output still retained a local shadow around one replaced fine-text line.
|
||||
Selective restoration is therefore the strongest current direction, not a
|
||||
production-ready default. The next implementation needs automatic line matching
|
||||
and a tighter source-plus-candidate glyph mask before it can be proposed for the
|
||||
pipeline.
|
||||
|
||||
A mask-only follow-up added two pixels of dilation around every selected glyph.
|
||||
It visually removed the light poster's shadow and improved its CER from 0.101 to
|
||||
0.081 with effectively unchanged LPIPS, but increased the edited area from 5.5%
|
||||
to 7.0%. Applying the same rule to the dark poster worsened CER from 0.112 to
|
||||
0.147. These expanded-mask rows were not resubmitted to the provider oracle.
|
||||
The opposing result rules out one global dilation constant and makes per-line
|
||||
residual detection the next fidelity gate.
|
||||
|
||||
An automatic follow-up removed the hand-selected line list. For each known
|
||||
source line, it runs a recognition-only model on padded, upscaled source and
|
||||
Qwen crops. It preserves a line only when both confidence scores are at least
|
||||
0.75, the source recognition is within 0.25 normalized edit distance of the
|
||||
verified line, and normalized source and candidate recognition match exactly.
|
||||
This reproduced the manual dark-poster line selection and additionally kept one
|
||||
correct large Russian line on the light poster.
|
||||
|
||||
The automatic prototype also replaced uniform dilation with a second measured
|
||||
LaMa pass. After the first erase, it extracts contrast remaining at the original
|
||||
glyph positions, dilates only that residual, and inpaints it before drawing fresh
|
||||
text. Visual inspection found no remaining double glyphs on either output. The
|
||||
effective edited fractions were 4.1% and 5.7%. Image LPIPS was 0.113 and 0.104,
|
||||
against 0.120 and 0.103 for the manual selective version. Detector-based page CER
|
||||
was 0.123 and 0.119; the light value is inflated by the detector splitting the
|
||||
preserved Cyrillic line into Latin-like fragments even though the padded Russian
|
||||
recognizer reads it exactly. This is another reason that line decisions use
|
||||
recognition-only crops rather than the page detector's aggregate CER.
|
||||
|
||||
Both exact automatic output hashes returned `No OpenAI signals detected`, and
|
||||
the original light poster returned `Generated with OpenAI tools` immediately
|
||||
afterward in the same Chrome sequence. The tracked script reproduced the dark
|
||||
hash exactly. Its light output differed from the initial prototype in only 386
|
||||
pixels by at most one channel value because separate ONNX environments rounded
|
||||
differently; the script's exact `900def5a...` hash was therefore submitted and
|
||||
also returned clean before the positive source control.
|
||||
|
||||
The script and its required verified line annotations are tracked as
|
||||
`scripts/selective_text_restoration.py` and `text-lines.json`. For example:
|
||||
|
||||
```bash
|
||||
PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK=True \
|
||||
uv run scripts/selective_text_restoration.py \
|
||||
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
|
||||
path/to/qwen-candidate.png \
|
||||
--restoration vae-glyphs \
|
||||
--glyph-donor path/to/qwen-vae-reconstruction.png \
|
||||
--glyph-feather 0.5 \
|
||||
--output path/to/selective-output.png \
|
||||
--manifest path/to/manifest.json
|
||||
```
|
||||
|
||||
The restoration mode is deliberately explicit. `vae-glyphs` requires a
|
||||
separately generated VAE reconstruction in `--glyph-donor`; it takes exact
|
||||
thresholded glyph cores from that donor, uses a fresh silhouette beneath them,
|
||||
and defaults to a narrow 0.5-pixel donor edge. The donor and the resulting exact bytes still require
|
||||
full-pipeline oracle evaluation. `rerender` reproduces the tracked
|
||||
fresh-system-font experiment. `source-glyphs` is only for compositing a
|
||||
separately regenerated, oracle-evaluated layer; feeding it the watermarked
|
||||
original would paste provenance-bearing pixels back into the result. The
|
||||
`--keep-background --composite-mask boxes` combination is an aligned-layer
|
||||
experiment and is not a production text restorer.
|
||||
|
||||
The `source-silhouette` follow-up keeps the source glyph geometry but discards
|
||||
its pixel amplitudes: it thresholds each source line to a binary shape, samples
|
||||
one foreground color, and synthesizes new antialiasing over the scrubbed image.
|
||||
On a 24-image matrix spanning serif, Latin-diacritic, Cyrillic, CJK, tiny-UI,
|
||||
and rotated text, the original outer-feather compositor preserved all 34
|
||||
source-readable lines exactly under crop OCR, with median text-box SSIM 0.859.
|
||||
An inner-antialias variant raised median text-box SSIM to 0.902 and left
|
||||
whole-image SSIM unchanged at 0.672, but preserved 33/34 lines: OCR read one
|
||||
middle dot as a colon. The higher-fidelity antialiasing is retained in the
|
||||
evaluation script, with that punctuation miss recorded as a caveat. Its exact
|
||||
outputs were not submitted to the provider oracle because the
|
||||
public verifier reached its request limit; a clean verdict from the earlier
|
||||
outer-feather bytes does not transfer to the new hashes.
|
||||
|
||||
Follow-up visual review on a typography-rich control rejected both silhouette
|
||||
compositors despite their OCR and SSIM scores. They preserved literal content
|
||||
and approximate glyph geometry, but changed stroke weight, color variation,
|
||||
edge antialiasing, and small decorative details enough to be plainly different
|
||||
from the source. OCR exactness and text-box SSIM are therefore screening metrics,
|
||||
not acceptance gates for source-typography preservation. A candidate must also
|
||||
retain source-like edge pixels and pass direct visual comparison at native size.
|
||||
|
||||
A narrower Qwen-VAE donor follow-up keeps the scrubbed background, then copies
|
||||
only VAE-reconstructed pixels through the source-silhouette mask with a
|
||||
0.5-pixel feather. Across the 48-case typography matrix (548 annotated lines),
|
||||
text-box SSIM improved in 47/48 cases and glyph-edge error improved in 48/48.
|
||||
Median text-box SSIM rose from 0.854 to 0.914, while median glyph-edge MAE fell
|
||||
from 37.59 to 32.42. The median nonzero alpha area was 3.62%; on dense or large
|
||||
text this is still substantially wider than the target control. These are
|
||||
fidelity results only; the 48 matrix outputs were not submitted to the provider
|
||||
oracle. On a separate untracked dense-typography control, the exact core-only
|
||||
Qwen-VAE donor with a 1.43% source-silhouette area returned `No OpenAI signals
|
||||
detected`. Its 0.5-pixel feathered sibling, which raises the nonzero donor area
|
||||
to 2.75%, returned the same verdict and improved mean text-box SSIM from 0.957
|
||||
to 0.960. Crop OCR recovered from 7/15 exact lines on the raw pass to 14/15,
|
||||
matching the source OCR floor. Whole-image LPIPS was 0.082, but only 0.108% of
|
||||
pixels were exactly unchanged and the detected face retained 0.670 of source
|
||||
Laplacian variance. The tracked script reproduced the feathered file byte for
|
||||
byte. These two exact-byte verdicts do not certify other images or the larger
|
||||
matrix masks, and the global smoothing fails a strict unchanged-image criterion.
|
||||
|
||||
The opt-in production port was rechecked separately on 2026-08-15. Its current
|
||||
LaMa runtime did not reproduce the earlier evaluation PNG byte for byte, but all
|
||||
changed pixels were confined to the erased background outside the donor glyph
|
||||
core. The exact production artifact returned `No OpenAI signals detected` in
|
||||
3/3 OpenAI Verify runs, while the matched source control returned `Generated
|
||||
with OpenAI tools` in 2/2 runs in the same Chrome session; expanded details
|
||||
identified SynthID and no C2PA manifest on the control. The private control and
|
||||
artifact hashes remain outside the public repository. This certifies only that
|
||||
runtime, verified manifest, and output, not arbitrary text masks or images.
|
||||
|
||||
The Google result is negative. On the synthetic CJK sign case, two separate
|
||||
work-account runs both detected SynthID in the resaved source control and in the
|
||||
exact Qwen-VAE donor output. The candidate improved mean text-box SSIM from
|
||||
0.791 to 0.835 and glyph-edge MAE from 35.48 to 22.49 with a 3.67% donor area,
|
||||
but Gemini still detected it. The intermediate Qwen silhouette base and the
|
||||
earlier original-sign Qwen baseline were also detected in the same account.
|
||||
This isolates the current blocker upstream of text restoration: the global
|
||||
Google pass did not reach the SynthID removal floor, so the OpenAI-clean
|
||||
`vae-glyphs` configuration cannot be used as a Google operating point.
|
||||
|
||||
On the large Chinese-sign control, source and candidate recognition agreed on
|
||||
all three lines. The script selected no changed lines, emitted a zero mask, and
|
||||
copied the candidate byte for byte instead of re-encoding it. This corrects the
|
||||
earlier unnecessary CJK rerender and proves the no-edit branch. It does not add
|
||||
a Google-negative oracle result: the available Google account still cannot
|
||||
reproduce a source-positive, Qwen-negative control sequence.
|
||||
|
||||
The earlier automatic rerender was the first variant in the experiment to pass
|
||||
the recorded visual, fidelity, and OpenAI-oracle gates on its two poster
|
||||
fixtures. The later typography-rich control shows that result does not
|
||||
generalize to source-typography preservation. It remains evaluation-only: it
|
||||
depends on verified source text and source line boxes, uses macOS system fonts,
|
||||
and has not been validated on natural scene text, rotated text, or automatic
|
||||
line-box discovery.
|
||||
|
||||
An opt-in `--detect-boxes` follow-up tested automatic geometry. Grouping Paddle
|
||||
word detections by vertical overlap found exactly 20/20, 20/20, and 3/3 lines;
|
||||
mean IoU with verified boxes was 0.857, 0.847, and 1.000. Reusing the annotation
|
||||
crop padding was unstable and preserved only 4/20 dark-poster lines. Reducing
|
||||
vertical recognition padding to 10% restored the exact 8/20 and 7/20 selection
|
||||
decisions, but detector CER was 0.127 and 0.154 rather than 0.123 and 0.119. The
|
||||
dark regression failed the fidelity gate, so these hashes were not submitted to
|
||||
the provider oracle. The flag is retained only to reproduce that negative
|
||||
evaluation and still requires verified strings and an exact line-count match.
|
||||
|
||||
### AnyText2 glyph-conditioned follow-up
|
||||
|
||||
`anytext2-restoration-2026-08-13.csv` tests the official Apache-2.0
|
||||
AnyText2 checkpoint as a local text-editing pass over the exact Qwen Chinese-sign
|
||||
output. The checkpoint hash was verified against ModelScope. Its own edit example
|
||||
successfully replaced a masked blackboard line with the requested `DADDY`, so the
|
||||
runtime and checkpoint were functional before the tracked fixture was measured.
|
||||
|
||||
AnyText2 failed the fidelity gate. The standard full-image detector scored the
|
||||
default edit at CER 0.185 and the source-font mimic at 0.222, against 0.074 for
|
||||
Qwen; image LPIPS moved from 0.289 to 0.338 and 0.345. A padded crop-recognition
|
||||
cross-check removed the detector's punctuation misses: Qwen scored 0.000, the
|
||||
default edit still scored 0.074 with two substituted characters, and the mimic
|
||||
edit stayed at 0.222. Visual inspection agreed, with additional line-shape and
|
||||
punctuation errors in the mimic output. No provider-oracle submission was made
|
||||
because neither candidate preserved the requested content better than its
|
||||
already-clean Qwen input.
|
||||
|
||||
This result covers the published SD1.5 checkpoint, three Chinese lines of no more
|
||||
than 20 characters, one seed, and one fixture. The repository still describes
|
||||
AnyText2XL as unreleased, and the inference wrapper truncates each requested line
|
||||
after 20 characters, so the published model cannot directly cover the longer
|
||||
English and Russian poster lines.
|
||||
|
||||
The padded cross-check also changes the interpretation of this fixture's earlier
|
||||
restoration result. Its Qwen image already contains all three correct Chinese
|
||||
lines; the reported 0.074 came from the detector cropping the final punctuation
|
||||
from two lines. A deterministic rerender with horizontally padded recognition
|
||||
and a mask-matched Hiragino Sans GB W6 font reached detector CER 0.000, but image
|
||||
LPIPS worsened from 0.289 to 0.327 and the font was visibly heavier. The correct
|
||||
policy on this large-text fixture is therefore to leave a Qwen line unchanged
|
||||
when padded recognition confirms the source and candidate strings agree.
|
||||
|
||||
## Face-stage isolation
|
||||
|
||||
`face-stage-isolation-2026-08-13.csv` separates the two stages on the 18-face
|
||||
Gemini portrait-grid fixture. All variants use the same visible-cleaned,
|
||||
metadata-stripped input and seed 0. The prepared input remained positive in the
|
||||
Gemini SynthID verifier, so the surrounding visible and metadata stages did not
|
||||
create the clean verdicts.
|
||||
|
||||
| Route | Gemini oracle | ID cosine | Face LPIPS | Laplacian ratio | Image LPIPS |
|
||||
| --- | --- | ---: | ---: | ---: | ---: |
|
||||
| Prepared control | detected | - | - | - | - |
|
||||
| Z-Image face stage only | detected | 0.927 | 0.028 | 0.612 | 0.013 |
|
||||
| Qwen global stage only | clean | 0.589 | 0.217 | 0.331 | 0.288 |
|
||||
| Full Qwen then Z-Image | clean | 0.852 | 0.050 | 0.492 | 0.252 |
|
||||
|
||||
This isolates the roles: the Qwen global pass removes the watermark, while the
|
||||
masked Z-Image pass restores face identity and detail without reintroducing a
|
||||
detectable whole-image signal. Z-Image face-only does not remove SynthID. The
|
||||
result explains why the face stage succeeded while global Z-Image failed: the
|
||||
earlier clean verdict belonged to the two-stage pipeline, not to Z-Image as a
|
||||
standalone remover. This is one fixture, one seed, and one oracle pass per
|
||||
variant; it does not establish a general reintroduction threshold for mask size.
|
||||
|
||||
## Upstream Synthid-Bypass v2 reproduction
|
||||
|
||||
`upstream-v2-reproduction-2026-08-13.csv` records a source-level audit and a
|
||||
close reproduction of the public
|
||||
[`Synthid-Bypass-v2.0.json`](https://github.com/cebeuq/Synthid-Bypass/blob/3007d0351596ae0a78b7074dae7ad179710b1e48/Synthid-Bypass-v2.0.json).
|
||||
The audited workflow is SHA-256
|
||||
`41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6` at
|
||||
upstream commit `3007d0351596ae0a78b7074dae7ad179710b1e48`.
|
||||
|
||||
Tracing links backward from `SaveImage` confirms that the active global stage is
|
||||
Qwen-Image-2512 Q4 with the Lightning four-step LoRA at 0.8, DiffSynth Canny at
|
||||
1.0, AuraFlow shift 3, `dpmpp_2m` plus `sgm_uniform`, CFG 1, and four steps. Its
|
||||
resolution helper uses adaptive level 6 over 0.08..0.15, which resolves to the
|
||||
0.154 ceiling for the 2816x1536 portrait fixture. Z-Image Turbo is used only by
|
||||
the face detailer: eight steps, CFG 1, `res_2s` plus `bong_tangent`, a 768 px
|
||||
guide, 1024 px cap, and direct adaptive denoise
|
||||
`clamp(0.10 * largest_face_ratio / 0.03, 0.05, 0.28)`. The active face regions
|
||||
come from YOLOv8-face plus SAM. Although the README describes MediaPipe as part
|
||||
of the strict path, the MediaPipe nodes do not reach the saved output. The
|
||||
1.2-megapixel scaler is also bypassed (`mode=4`).
|
||||
|
||||
The published upstream pair 12 changed from SynthID-positive to clean in the
|
||||
Gemini verifier and scored 0.975 face identity. On the project portrait fixture,
|
||||
the prepared control was positive, while both the close global reproduction and
|
||||
the close full reproduction were clean. The global stage scored 0.589 identity
|
||||
and 0.217 face LPIPS; the upstream-strength face stage improved those to 0.783
|
||||
and 0.083. The maintained profile's weaker face pass scored 0.852 and 0.050 on
|
||||
the same control, so copying upstream's roughly doubled face strength would be a
|
||||
quality regression on this fixture.
|
||||
|
||||
The close reproduction is intentionally labeled rather than presented as an
|
||||
exact ComfyUI run. It uses full-precision DiffSynth bf16 weights instead of the
|
||||
Q4 GGUF files, the maintained DiffSynth Lightning scheduler approximation
|
||||
instead of the ComfyUI sampler pair, YuNet plus SAM instead of YOLOv8-face plus
|
||||
SAM, and fixed seed 0 instead of randomized seeds. The two independent oracle
|
||||
controls and the published upstream pair establish the direction of the result;
|
||||
they do not make the local output byte-equivalent to upstream.
|
||||
|
||||
## Compare
|
||||
|
||||
```bash
|
||||
uv run scripts/fidelity_metrics.py compare \
|
||||
--original data/synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png \
|
||||
--variant controlnet=<out>.png \
|
||||
--variant qwen=<out>.png \
|
||||
--variant qwen-zimage=<qwen-out>.png \
|
||||
--variant sdxl-zimage=<sdxl-out>.png \
|
||||
--ocr-langs ""
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
date,source_commit,fixture,variant,model,model_revision,model_sha256,ocr_route,text_cer,img_lpips,ssim,psnr_db,output_sha256,provider_oracle,notes
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,,,,"Paddle ch",0.074,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621,not_run,Exact Qwen baseline reused from text-restoration benchmark
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-default,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.185,0.338,0.642,18.2,87729fe7e0482d2cead6d3b7c7dd386ecc75ab3d8658131c54e67528b8f9feeb,not_run,Fidelity gate failed; padded crop recognition CER 0.074 still shows two substituted characters; locally composited three-line edit; 20 DDIM steps; seed 0
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-mimic,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.222,0.345,0.644,17.9,f5198d8f2365b79880aba8ea6b0c3885142b9de21c9f7f4f6712ac4a0db8dd85,not_run,Fidelity gate failed; source glyph masks supplied as font hints; 20 DDIM steps; seed 0
|
||||
|
@@ -0,0 +1,5 @@
|
||||
date,source_commit,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,zimage-face-only,0,,0.06604512532552083,18,0.927,0.028,0.612,0.013,0.959,34.7,c388e9bbf737bb9b22a7cb2216ffb8026321ca632f9db29e1e456b7c14a8553f,synthid_detected
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-global-only,0,0.153999996,,18,0.589,0.217,0.331,0.288,0.646,24.2,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-zimage-full,0,0.153999996,0.06604512532552083,18,0.852,0.050,0.492,0.252,0.671,24.7,69de191bddbfbf53348c7638bdba661b6d45d257c00a8a2646b2558670c5a4c2,clean
|
||||
|
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
|
||||
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png": "每天都是一个新的机会。\n用微笑开始它。\n世界也会向你微笑。"
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Paddle en+ru+ch",0.378,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.048,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.101,0.120,0.796,19.5,0.054799,15b40b8cdf9e97558c66d4acdcb97d46769e64f04f143f8061746bc731fbbc21,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.081,0.119,0.796,19.5,0.070267,5a8a0bcbe5763d1b1af0d64149cec7e35b16c5751e995d3bacf17581d64177ec,not_checked,fidelity-only follow-up
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.123,0.113,0.808,21.2,0.041279,900def5a433c039aa474789eddc4dc4c63b8126ba1aff6829d2e8d600b9f9c71,no_openai_signals_detected,source detected immediately afterward in the same Chrome sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Paddle en+ru+ch",0.413,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.086,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.112,0.103,0.780,20.8,0.054912,846f39db0ad4a48a734a5019f0b0ff5d110eecb3210223407e72529dfa4fcf54,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.147,0.104,0.779,20.8,0.072556,8ae76e1dedeec26bebb8d35e71fd92d77f142dbb0f3440b330cdf33b79558afe,not_checked,fidelity-only follow-up
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.119,0.104,0.779,20.8,0.057110,ac4201bd2e342b1a37737319438cedb2203f6bb58cd9113f825cf9b80d913755,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-selective-automatic-passthrough,"Paddle detector ch; horizontal-only padded recognizer selection",0.074,0.289,0.696,23.3,0.000000,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_checked,all three source and candidate recognitions agreed exactly; output is byte-identical to candidate
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.127,0.113,0.805,20.9,0.041280,70c52fed8b8a1d98f4cda8558ae08825313e510dfb372e448370d8dcf1da73b0,not_checked,fidelity-only automatic-box follow-up; same 8/20 preserve decisions
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.154,0.104,0.778,20.8,0.057146,1fa01ce706042c003a61429d5d97fc6c928ec32278071a91926d588f1e760b00,not_checked,fidelity-only automatic-box follow-up; same 7/20 preserve decisions but worse CER
|
||||
|
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": [
|
||||
{"box": [156, 206, 709, 296], "text": "This is a longer", "script": "alphabetic"},
|
||||
{"box": [157, 293, 939, 379], "text": "sample text in English.", "script": "alphabetic"},
|
||||
{"box": [165, 393, 774, 434], "text": "Typography can flow smoothly from", "script": "alphabetic"},
|
||||
{"box": [161, 434, 521, 477], "text": "large to medium size.", "script": "alphabetic"},
|
||||
{"box": [165, 495, 566, 522], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
|
||||
{"box": [165, 524, 354, 550], "text": "rhythm, and clarity.", "script": "alphabetic"},
|
||||
{"box": [165, 564, 448, 585], "text": "Even the finest text should remain", "script": "alphabetic"},
|
||||
{"box": [165, 593, 330, 613], "text": "clean and readable.", "script": "alphabetic"},
|
||||
{"box": [158, 673, 903, 740], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
|
||||
{"box": [162, 753, 697, 793], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
|
||||
{"box": [164, 807, 524, 829], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
|
||||
{"box": [163, 845, 483, 868], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
|
||||
{"box": [162, 951, 780, 1009], "text": "Это более длинный пример", "script": "alphabetic"},
|
||||
{"box": [161, 1019, 723, 1062], "text": "текста на русском языке.", "script": "alphabetic"},
|
||||
{"box": [165, 1076, 639, 1104], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
|
||||
{"box": [165, 1111, 505, 1133], "text": "крупного размера к среднему.", "script": "alphabetic"},
|
||||
{"box": [165, 1151, 513, 1172], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
|
||||
{"box": [167, 1178, 287, 1192], "text": "ритм и ясность.", "script": "alphabetic"},
|
||||
{"box": [165, 1210, 417, 1227], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
|
||||
{"box": [165, 1235, 381, 1247], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
|
||||
],
|
||||
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": [
|
||||
{"box": [165, 204, 754, 299], "text": "This is a longer", "script": "alphabetic"},
|
||||
{"box": [167, 295, 1006, 385], "text": "sample text in English.", "script": "alphabetic"},
|
||||
{"box": [171, 401, 831, 446], "text": "Typography can flow smoothly from", "script": "alphabetic"},
|
||||
{"box": [169, 446, 560, 491], "text": "large to medium size.", "script": "alphabetic"},
|
||||
{"box": [173, 507, 605, 536], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
|
||||
{"box": [173, 538, 379, 567], "text": "rhythm, and clarity.", "script": "alphabetic"},
|
||||
{"box": [175, 583, 481, 603], "text": "Even the finest text should remain", "script": "alphabetic"},
|
||||
{"box": [173, 611, 352, 632], "text": "clean and readable.", "script": "alphabetic"},
|
||||
{"box": [170, 693, 956, 757], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
|
||||
{"box": [174, 775, 740, 812], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
|
||||
{"box": [175, 829, 561, 852], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
|
||||
{"box": [174, 869, 534, 892], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
|
||||
{"box": [169, 958, 824, 1022], "text": "Это более длинный пример", "script": "alphabetic"},
|
||||
{"box": [171, 1029, 764, 1074], "text": "текста на русском языке.", "script": "alphabetic"},
|
||||
{"box": [175, 1090, 678, 1119], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
|
||||
{"box": [175, 1125, 546, 1151], "text": "крупного размера к среднему.", "script": "alphabetic"},
|
||||
{"box": [175, 1169, 552, 1192], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
|
||||
{"box": [175, 1196, 312, 1217], "text": "ритм и ясность.", "script": "alphabetic"},
|
||||
{"box": [175, 1233, 479, 1253], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
|
||||
{"box": [177, 1259, 440, 1272], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
|
||||
],
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png": [
|
||||
{"box": [1281, 650, 2357, 818], "text": "每天都是一个新的机会。", "script": "cjk"},
|
||||
{"box": [1466, 788, 2167, 957], "text": "用微笑开始它。", "script": "cjk"},
|
||||
{"box": [1367, 910, 2264, 1112], "text": "世界也会向你微笑。", "script": "cjk"}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
date,source_commit,fixture,profile,route,seed,strength,adaptive_polish,text_cer,ocr_floor,added_cer,img_lpips,ssim,psnr_db,output_sha256,oracle_rechecked
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.378,0.127,0.251,0.107,0.832,24.4,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.356,0.127,0.229,0.125,0.707,25.3,9bb932d6e66b52f34ff07c3fcdbdb9c33eb601ae2cd84bec6b8c7b5e30df9926,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.413,0.127,0.286,0.095,0.818,26.0,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.422,0.127,0.295,0.099,0.754,25.9,7605b1175b0c99f481ae6929acc488f34d98bc0f2571d42a1d2972b671d9b8cb,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,original,reference,,,,0.037,0.037,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,visible-invisible-metadata,0,0.153999996,false,0.074,0.037,0.037,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,sdxl-zimage,visible-invisible-metadata,0,0.25,true,0.074,0.037,0.037,0.176,0.769,26.1,d9f89465c31bd2d38ffb111775b9bbea4577f40713fcbebf421e05d344ab27ad,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.068,0.127,-0.059,0.042,0.874,31.4,534de6422a3a03cfde44377aa9bf824a8267db3b73511240f8f07fb8dcb95eff,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.086,0.127,-0.041,0.047,0.862,30.8,4dba8211e8d84ebbfff72de00f54d4365d0d202394acd412d1419ae93d1e9f71,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.177,0.127,0.050,0.065,0.838,29.1,fc45c5fff6d33f15d30e05dd02580fe0addd97098b735b2894015f9abdaa43a7,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.248,0.127,0.121,0.093,0.813,27.3,1e55e11f0e289a752098370e9f1d53fed6cd4fab7ceff46fe86dd20c927f9ffb,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.343,0.127,0.216,0.120,0.787,25.7,f5a2d2829e88a790cff5bedc50fca1f878a92cdb392d530afb49d0d95c23b884,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.143,0.127,0.016,0.042,0.877,31.5,5cfa466e276ab37cf532f7458d6dd86ea4f0807e2cbdb6788b7c01ca6ee35a1a,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.101,0.127,-0.026,0.047,0.864,30.9,ade869ec7c5b08098495d9b9d67ee900773f189f94bbb411b043f242f5c2fa37,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.208,0.127,0.081,0.060,0.839,29.3,a11976963e3f2ca856ce66278e22064d51345d449fa6d672029039a79517d60a,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.341,0.127,0.214,0.070,0.819,27.6,1ab1ae075565e9d39960a80cf5d61e5b0c50d7e23283493ea0277056b3e964ab,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.538,0.127,0.411,0.084,0.799,25.6,5ff5a364d84f738de4c56ead7d4950a2f9a1aaf4bec16e97bda6a65915744a57,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.08,false,0.074,0.037,0.037,0.068,0.865,30.8,81419eddc3e5150956859dd10c26dd841bb93d9c8b34c01f344e6d69bca6edbf,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.10,false,0.074,0.037,0.037,0.086,0.844,29.8,4d1f32bd0dac0e05885ff943e7c10230b06de7d8547e6215f8e2941f732a1df5,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.15,false,0.000,0.037,-0.037,0.135,0.793,27.6,526c7987b34896f9513f7439b63b8a87573b59d15930fa307f3bb924435f3cec,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.20,false,0.000,0.037,-0.037,0.179,0.746,25.9,325364cf7a3b5d39ac0c44af82181753ca4d6fde0f1b522c896414191725e8fc,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.25,false,0.000,0.037,-0.037,0.218,0.703,24.5,f01d64dc0cabd45d66a93c1be39b819b988f29cda1f0a7bc0d5a1954571d807e,true
|
||||
|
@@ -0,0 +1,7 @@
|
||||
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.338,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.305,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,"Paddle ch",0.074,0.289,0.696,23.3,,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,synthid_detected,inconclusive account: clean baseline not reproduced
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-restored,"Paddle ch",0.111,0.328,0.653,18.1,0.051,2ab7666a4161ff0e12c53d9412b23d013970c7613a8a55a1b1ae8a2744e3290d,synthid_detected,inconclusive account: qwen-zimage control also detected
|
||||
|
@@ -0,0 +1,6 @@
|
||||
date,upstream_commit,workflow_sha256,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle,implementation
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/before/12.png,official-before,,,,10,,,,,,,056276464387c883f5fd89bd2f1415faeb425abb3153cd10efb25d1b9053bdc2,synthid_detected,upstream-published-bytes
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/after/12.png,official-after,,,,10,0.975,0.093,0.354,0.172,0.820,23.8,a822d805666224adb3e360b3ee89e14f20a341f1e7c4d3a5d3d5251e9286aefe,clean,upstream-published-bytes
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected,project-runtime-control
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-global-close-reproduction,0,0.153999996,,18,0.589,0.217,0.331,0.286,0.647,24.3,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean,diffsynth-bf16-yunet-sam
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-full-close-reproduction,0,0.153999996,0.13209025065104166,18,0.783,0.083,0.470,0.260,0.659,24.7,2d7a8f76f98b905f14d69e5f9e11979610c72c6be7b6ac298dec4659f17bd819,clean,diffsynth-bf16-yunet-sam
|
||||
|
@@ -1,3 +1,3 @@
|
||||
date,source_url,source_sha256,duration_seconds,source_verdict,noise_std,long_side,fps,seed,output_sha256,output_verdict,psnr_db,temporal_residual_ratio
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,8,detected_all_frames,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,26.2932,1.0072
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,8,detected_all_frames,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,25.3911,1.0578
|
||||
date,source_url,source_sha256,source_width,source_height,source_fps,duration_seconds,source_verdict,vae,noise_std,long_side,fps,seed,output_sha256,output_verdict,output_verdict_text,output_detected_range,track,session_id,stratum,psnr_db,temporal_residual_ratio
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
|
||||
|
@@ -0,0 +1,15 @@
|
||||
# Provenance fixtures
|
||||
|
||||
These files exercise real container, metadata, and pixel-provenance formats.
|
||||
They are test inputs, not evaluation corpora.
|
||||
|
||||
## Adobe TrustMark
|
||||
|
||||
`adobe-trustmark-p.png` is Adobe's official `images/ghost_P.png` Variant P
|
||||
example from [`adobe/trustmark`](https://github.com/adobe/trustmark) at commit
|
||||
`0ed40cbe8188f664fd9cbbeacd969807de27440a`.
|
||||
|
||||
- Source: `https://raw.githubusercontent.com/adobe/trustmark/0ed40cbe8188f664fd9cbbeacd969807de27440a/images/ghost_P.png`
|
||||
- SHA-256: `e58c5825ed7e5d9fb04710ea541b61bd55879cad65554c0d46260aa24b3d0755`
|
||||
- Expected signal: TrustMark Variant P, schema 1
|
||||
- License: MIT, reproduced in `../licenses/adobe-trustmark-MIT.txt`
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 360 KiB |
@@ -0,0 +1,25 @@
|
||||
Copyright 2023 Adobe
|
||||
All Rights Reserved.
|
||||
|
||||
NOTICE: Adobe permits you to use, modify, and distribute this file in
|
||||
accordance with the terms of the license agreement accompanying it.
|
||||
|
||||
MIT License
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -56,7 +56,9 @@ Ground-truth quality, strongest first:
|
||||
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
|
||||
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
|
||||
- `synthid-portal` — checked via Google's SynthID Detector portal.
|
||||
- `c2pa-metadata` — issuer-only proxy (Google/OpenAI C2PA manifest present). Weaker: the C2PA can be stripped while the pixel watermark remains.
|
||||
- `c2pa-metadata` — supported provenance evidence (Google AI C2PA, or OpenAI
|
||||
C2PA with an explicit `c2pa.watermarked.*` action). Weaker than a provider
|
||||
oracle: C2PA can be stripped while the pixel watermark remains.
|
||||
- `third-party` — label asserted by an external dataset, not independently verified.
|
||||
- `none` — unverified.
|
||||
|
||||
|
||||
+107
-23
@@ -16,17 +16,17 @@ defaults. This page focuses on choosing the right command.
|
||||
| `metadata` and metadata-only `identify` | Default package |
|
||||
| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` |
|
||||
| Adobe TrustMark signals in `identify` | `remove-ai-watermarks[trustmark]` |
|
||||
| Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` |
|
||||
| `visible` and `erase` with OpenCV | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| `visible` or `erase` with MI-GAN | `remove-ai-watermarks[migan]` |
|
||||
| `visible` or `erase` with big-LaMa | `remove-ai-watermarks[lama]` |
|
||||
| `invisible` | `remove-ai-watermarks[diffusion]` |
|
||||
| `invisible --pipeline qwen-zimage` | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| `invisible` and `all` (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| `video metadata` and `video identify --no-visible` | Default package |
|
||||
| `video identify`, `video visible`, and visible/all batch modes | `remove-ai-watermarks[video]` |
|
||||
| `video invisible` and `video all --invisible` | `remove-ai-watermarks[video,diffusion]` |
|
||||
| `video identify` | `remove-ai-watermarks[video]` |
|
||||
| `video visible`, `video all`, and visible/all batch modes | `remove-ai-watermarks[video]` plus ffmpeg on PATH |
|
||||
| `video invisible` and `video all --invisible` | `remove-ai-watermarks[video,diffusion]` plus ffmpeg on PATH |
|
||||
| HEIC/HEIF/AVIF pixel input | Add `remove-ai-watermarks[heif]` |
|
||||
| Every production command and backend | `remove-ai-watermarks[all]` |
|
||||
| Every production command and Python-compatible backend | `remove-ai-watermarks[all]` |
|
||||
|
||||
`batch` requires the same extra as its selected mode. Extras can be combined in
|
||||
one installation, for example `remove-ai-watermarks[visible,detect,heif]`.
|
||||
@@ -40,7 +40,13 @@ remove-ai-watermarks identify image.png
|
||||
`identify` always inspects supported metadata. When pixel extras are installed,
|
||||
it also evaluates supported visible and invisible pixel signals. When no signal
|
||||
is found, it reports the origin as unknown. It does not claim the image is
|
||||
clean.
|
||||
clean. For C2PA files, the text report shows asset integrity, claim-signature,
|
||||
signer-trust, and signer-validity results separately. An intact claim from an
|
||||
untrusted or expired signer is reported at medium confidence; a failed asset
|
||||
binding or signature does not confirm the claimed origin. When a structured
|
||||
C2PA soft binding is present, the report also names its exact algorithm and
|
||||
signed value; removing the manifest does not remove the referenced pixel
|
||||
watermark or content fingerprint.
|
||||
|
||||
Machine readable output:
|
||||
|
||||
@@ -68,7 +74,8 @@ remove-ai-watermarks visible image.png -o clean.png
|
||||
The default behavior:
|
||||
|
||||
- checks every registered visible mark;
|
||||
- removes every detected match;
|
||||
- removes every detected match, except the weakly detected Jimeng label pill,
|
||||
which needs corroboration (see [supported signals](supported-signals.md));
|
||||
- selects the best installed fill backend;
|
||||
- strips AI metadata from the output.
|
||||
|
||||
@@ -129,6 +136,13 @@ remove-ai-watermarks erase image.png \
|
||||
`erase` accepts `cv2`, `migan`, and `lama`. The corresponding optional extra
|
||||
must be installed for a learned backend.
|
||||
|
||||
Two more knobs tune the fill. `--dilate N` (default 3) grows every box by `N`
|
||||
pixels before inpainting, which helps when a mark has a soft edge or a drop
|
||||
shadow just outside the box you measured; it applies to every backend because it
|
||||
shapes the mask. `--inpaint-method telea|ns` selects the classical algorithm and
|
||||
only affects the `cv2` backend. Like `visible`, `erase` strips AI metadata from
|
||||
the output by default; pass `--keep-metadata` to retain it.
|
||||
|
||||
## Strip AI metadata
|
||||
|
||||
Inspect metadata:
|
||||
@@ -146,6 +160,11 @@ remove-ai-watermarks metadata image.png --remove -o clean.png
|
||||
When `-o` is omitted, removal overwrites the source. Standard metadata is kept
|
||||
unless you pass `--remove-all`.
|
||||
|
||||
A quiet `--check` or a successful `--remove` is not a clean verdict. The command
|
||||
only inspects and strips embedded AI metadata; a pixel watermark such as SynthID
|
||||
has no local decoder once that metadata proxy is gone. `identify` reports the
|
||||
same limit.
|
||||
|
||||
The command also supports the audio and video containers listed in
|
||||
[supported signals](supported-signals.md). ffmpeg must be available for the
|
||||
non-ISOBMFF audio and video path.
|
||||
@@ -214,11 +233,17 @@ remove-ai-watermarks video metadata input.mp4 --check
|
||||
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
|
||||
```
|
||||
|
||||
As with the generic `metadata` command, a quiet check or a successful strip is
|
||||
not a clean verdict: video SynthID is not decoded locally after the metadata
|
||||
proxy is gone.
|
||||
|
||||
Supported containers are MP4, MOV, M4V, WebM, MKV, AVI, and FLV. The operation
|
||||
delegates to the same verified metadata scanner and stripper as the generic
|
||||
`metadata` command, so detection and removal stay in parity. Video and audio
|
||||
streams are not transcoded. For MP4 and MOV, this includes the native TC260
|
||||
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`. The inspector
|
||||
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`, plus the
|
||||
QuickTime-form `meta` variants Doubao's iOS export writes (a bare `meta` box
|
||||
as a direct `moov` child, and a keyless `hdlr=mdir` metadata list). The inspector
|
||||
seeks past a large `mdat` to find a tail `moov`. Removal stream-copies the
|
||||
container in bounded chunks, converts supported top-level provenance boxes to
|
||||
same-size `free` boxes, and blanks the TC260 key/value in place. Box sizes,
|
||||
@@ -340,10 +365,11 @@ failed encode does not overwrite an existing result.
|
||||
|
||||
## Remove invisible watermarks
|
||||
|
||||
Install the diffusion dependencies first:
|
||||
Install the removal dependencies first. Both profiles are CUDA-only and both
|
||||
run the DiffSynth Z-Image face stage, so this is the extra either one needs:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[diffusion]"
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
```
|
||||
|
||||
Then run:
|
||||
@@ -363,10 +389,15 @@ remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
|
||||
| Pipeline | When to use it |
|
||||
| --- | --- |
|
||||
| `controlnet` | Default compatibility profile with structural conditioning |
|
||||
| `sdxl` | Lighter plain SDXL regeneration |
|
||||
| `qwen` | Large CUDA oriented Qwen Image profile |
|
||||
| `qwen-zimage` | CUDA only high fidelity profile with a separate face stage |
|
||||
| `qwen-zimage` | Default. Qwen-Image-2512 global pass plus a SAM-masked Z-Image face stage |
|
||||
| `sdxl-zimage` | The same recipe and face stage on an SDXL global pass, at a higher denoise |
|
||||
|
||||
**Both are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
removal. The former `controlnet`, `sdxl`, `qwen` and `default` profiles were removed
|
||||
rather than kept as a CPU path: none of them matched this recipe's face preservation,
|
||||
so offering them implied a quality the library no longer delivers. Passing a retired
|
||||
name is rejected at parse time rather than remapped. Visible-mark removal and every
|
||||
identify path still run anywhere.
|
||||
|
||||
Example:
|
||||
|
||||
@@ -375,8 +406,55 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --force
|
||||
```
|
||||
|
||||
The legacy `default` value is an alias for `sdxl`. The `--auto` option is
|
||||
deprecated, emits a warning, and changes nothing.
|
||||
There is no `--model`, `--steps`, `--guidance-scale` or `--device` option, and the
|
||||
deprecated `--auto` is gone. Each profile pins its model stack, its per-stage
|
||||
schedule, CFG 1.0 and CUDA, so every one of those flags existed only to be refused
|
||||
several layers down. They are not parsed at all now, which fails at the point the
|
||||
user can act on rather than after a model load.
|
||||
|
||||
### Restore operator-verified text
|
||||
|
||||
`--text-manifest` enables the experimental `vae-glyphs` post-pass. It reconstructs
|
||||
the source with the Qwen VAE, blends 15% of that reconstruction into the normal
|
||||
`qwen-zimage` result, erases the annotated candidate glyphs with LaMa, and composites
|
||||
only the reconstructed glyph cores through source-derived silhouettes. It does not
|
||||
run OCR or choose which strings are correct.
|
||||
|
||||
Install the combined extra and run only with an operator-verified manifest:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
``verified: true`` may also be set by an automated operator that attests
|
||||
machine-verified geometry: stability-gated detector boxes inside sane caps. Such
|
||||
operators should use the geometry-only schema 2, which carries no transcription
|
||||
or script metadata.
|
||||
|
||||
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default: it
|
||||
was measured to return detector-visible OpenAI SynthID on poster-scale manifests
|
||||
(official Content Provenance API, 2026-08-19). `--fidelity-anchor` restores the
|
||||
0.27.0 research behavior; text-box fidelity lost by the default is well under one
|
||||
MAE point on the measured fixtures.
|
||||
|
||||
The manifest is a JSON object with `verified: true`, decoded RGB dimensions,
|
||||
`source_pixel_sha256`, and a non-empty `lines` array. Schema 1 is retained for
|
||||
manually reviewed annotations: each line has an integer `[x1, y1, x2, y2]` box,
|
||||
exact `text`, a non-empty `script`, and an optional angle from -30 to 30 degrees.
|
||||
Schema 2 is geometry-only: each line has the box and optional angle, with no
|
||||
required `text` or `script`. Lines must be in top-to-bottom, left-to-right order.
|
||||
The hash binds the annotations to decoded RGB geometry and pixels, so metadata-only
|
||||
container changes remain valid while a resized or edited source fails closed. The
|
||||
experimental helper
|
||||
`remove_ai_watermarks._internal.text_restoration.source_pixel_sha256` computes it.
|
||||
|
||||
This mode is supported only by `qwen-zimage` at native geometry with
|
||||
`humanize=0`, `unsharp=0`, and adaptive polish disabled. `all` also accepts the flag,
|
||||
but its manifest must match the pixels entering the invisible stage; if visible-mark
|
||||
removal changes those pixels, the hash check rejects the run. One oracle verdict does
|
||||
not certify another manifest, seed, model/runtime version, or output hash.
|
||||
|
||||
### Work with limited memory
|
||||
|
||||
@@ -409,8 +487,8 @@ It is a memory strategy, not a guarantee of better quality.
|
||||
|
||||
The `all` command and the `all` installation extra are separate concepts. The
|
||||
command runs every applicable stage. Installing `remove-ai-watermarks[all]`
|
||||
makes every production backend available; a smaller installation such as
|
||||
`remove-ai-watermarks[visible,diffusion]` can also run the command with fewer
|
||||
makes every backend compatible with the active Python available; a smaller installation such as
|
||||
`remove-ai-watermarks[visible,qwen-zimage]` can also run the command with fewer
|
||||
optional backends.
|
||||
|
||||
```bash
|
||||
@@ -425,10 +503,16 @@ The command runs:
|
||||
|
||||
The visible options and diffusion options are also available on `all`.
|
||||
|
||||
If diffusion is required but the `diffusion` extra is unavailable, `all` still
|
||||
writes the result of the visible and metadata stages, prints a prominent
|
||||
warning, and exits with code 1. This prevents a partial result from being
|
||||
reported as complete.
|
||||
When the `qwen-zimage` extra is unavailable, `all` still writes the result of
|
||||
the visible and metadata stages, prints a prominent warning, and exits with code
|
||||
1. That happens on every run without the extra: the skipped stage is what would
|
||||
have decided whether a signal was there. This prevents a partial result from
|
||||
being reported as complete.
|
||||
|
||||
If the extra is installed but the machine has no CUDA, which is the usual macOS
|
||||
case, the run fails at engine construction instead: `all` prints
|
||||
`Error: Invisible-watermark removal is CUDA-only ...`, writes no output at all,
|
||||
and exits with code 1.
|
||||
|
||||
## Process a directory
|
||||
|
||||
|
||||
@@ -7,3 +7,15 @@ This page records notices required by source dependencies and licensed derivativ
|
||||
- The DWT-DCT implementation derives from ShieldMnt's
|
||||
[`invisible-watermark`](https://github.com/ShieldMnt/invisible-watermark), licensed
|
||||
under MIT. Its notice ships in `src/remove_ai_watermarks/licenses/invisible-watermark-MIT.txt`.
|
||||
The decode path is a vectorized reformulation, not a transcription: it produces
|
||||
the same bits and is checked against upstream's own decoder, but it no longer
|
||||
corresponds line by line to `imwatermark/maxDct.py`. Diffing the two files will
|
||||
show structurally different code, which is expected and does not mean the
|
||||
derivation notice is stale.
|
||||
|
||||
## Licensed test fixtures
|
||||
|
||||
- `data/fixtures/provenance/adobe-trustmark-p.png` is Adobe's official
|
||||
TrustMark Variant P example, licensed under MIT. Its source commit, digest,
|
||||
and reproduced license are recorded beside the fixture in
|
||||
`data/fixtures/README.md` and `data/licenses/adobe-trustmark-MIT.txt`.
|
||||
@@ -3,7 +3,10 @@
|
||||
> Research archive. This document records experiments, superseded defaults, and
|
||||
> deployment considerations from the time of the study. It does not define the
|
||||
> current CLI or Python API. See `README.md`, `docs/cli.md`, and
|
||||
> `docs/known-limitations.md` for current behavior.
|
||||
> `docs/known-limitations.md` for current behavior. Everything it names on the
|
||||
> face-restoration path was removed after this study: the `--restore-faces` and
|
||||
> `--face-id` flags, the `instantid`, `photomaker` and `faceid` extras, and their
|
||||
> modules. So were the `controlnet`, `sdxl`, `qwen` and `default` profiles.
|
||||
|
||||
Date: 2026-06-02. Source: a manual primary-source pass (WebSearch + WebFetch over the
|
||||
watermark-removal-attack and SDXL-ControlNet literature). Prompted by issue #35
|
||||
@@ -55,8 +58,7 @@ content it protects. Controlnet is the text/structure PRESERVATION pipeline; rem
|
||||
is set by STRENGTH, separately calibrated, not by the pipeline choice.
|
||||
|
||||
This section is the single consolidated reference for the controlnet pipeline's
|
||||
removal behavior. (Mirrored briefly in `docs/synthid.md` §5.5 and the CLAUDE.md
|
||||
controlnet / face_restore bullets, which point here.)
|
||||
removal behavior. (Mirrored briefly in `docs/synthid.md` §5.5, which points here.)
|
||||
|
||||
### What we measured (real gpt-image + Gemini originals)
|
||||
|
||||
@@ -288,8 +290,8 @@ text-vs-scrub tension as an empirical question to measure, not assume.
|
||||
**Prototype (runs locally on 32 GB MPS — no dedicated GPU required):**
|
||||
|
||||
Compute is NOT the bottleneck. On a 32 GB Apple-silicon machine (M5 here) native SDXL already
|
||||
runs entirely on MPS with no CPU fallback (~155 s at 1122x1402, verified — see `synthid.md` /
|
||||
CLAUDE.md). The prototype runs at **1024** (fewer pixels than that) with SDXL base + an SDXL
|
||||
runs entirely on MPS with no CPU fallback (~155 s at 1122x1402, verified 2026-05-31).
|
||||
The prototype runs at **1024** (fewer pixels than that) with SDXL base + an SDXL
|
||||
ControlNet + activations in **fp32** (MPS fp16 decodes to all-black NaN — issue #29 — confirmed
|
||||
on run 1 below; fp32 is the required default on mps/cpu) — fits the 32 GB budget with vae-tiling +
|
||||
attention-slicing; ~1-2 min/image, so a coarse sweep is a sub-hour background run. A dedicated GPU
|
||||
@@ -534,7 +536,8 @@ https://instantid.github.io/ · https://arxiv.org/abs/2308.06721 (IP-Adapter)
|
||||
|
||||
### FaceID prototype run 1 -- 2026-06-03 (NEGATIVE on dense small-face groups)
|
||||
|
||||
Built and shipped the masked multi-face FaceID layer (`--face-id`, `face_id.py`, `faceid` extra).
|
||||
Built and shipped the masked multi-face FaceID layer (`--face-id`, `face_id.py`, `faceid` extra;
|
||||
all three were removed after this run and never reached the tracked tree).
|
||||
First real run on the gemini_3 group photo (Google, s015, scale 0.6, native 2816 via cap 1536):
|
||||
insightface detected **17 faces**, the masked multi-face pass composed and ran end-to-end (non-black
|
||||
output), so the API is correct. **At s015 the result is a clear FAILURE: every face corrupted --
|
||||
|
||||
+10
-1
@@ -12,9 +12,18 @@ Read this reference for environment setup, dependency recovery, CI behavior, and
|
||||
|
||||
The optional TrustMark decoder downloads weights into its installed package directory. After pruning that extra, a leftover weights directory can make availability checks see an empty namespace package. If Pyright reports an unknown `TrustMark` import and `find_spec("trustmark")` returns a loader-less spec, remove that regenerable remnant from the active virtual environment and resync.
|
||||
|
||||
### Known security-gate blocks
|
||||
|
||||
`maintain.sh` fails while PyPI ships no fixed release for a transitive CVE. Current case (triaged 2026-08-17): `lightning` 2.6.5, pulled only by the optional `trustmark` extra, carries PYSEC-2026-3624 (RCE via `load_from_checkpoint` on an attacker-crafted checkpoint). The vulnerable API is unreachable here, because the TrustMark decoder loads only its own pinned weights downloaded from the TrustMark release, never a user-supplied checkpoint. The upstream fix is merged but unreleased, and ignores are never added, so run and report the remaining core checks (Ruff, Pyright, tests) separately until a fixed `lightning` release lands; bump it with `uv lock --upgrade-package lightning` as soon as one does.
|
||||
|
||||
## CI
|
||||
|
||||
`.github/workflows/test.yml` runs Ruff and a cross-platform supported-Python test matrix with default plus development dependencies. Diffusion and model-running tests skip in that matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the OpenCV eraser remain covered across operating systems.
|
||||
`.github/workflows/test.yml` runs Ruff, a test matrix over every supported Python
|
||||
minor with default plus development dependencies, cross-platform coverage at the
|
||||
oldest and newest minors, and a separate job that installs ffmpeg on Ubuntu to
|
||||
run the full-clip video test. Diffusion and model-running tests skip in that
|
||||
matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the
|
||||
OpenCV eraser remain covered across operating systems.
|
||||
|
||||
Keep `uv.lock` compatible with `uv sync --frozen`. Dependency pull-request checks use GitHub's merge result against current `main`; if `main` moves, merge it locally and rerun the full gate because a newer linter can expose stale directives in later code.
|
||||
|
||||
|
||||
+3
-1
@@ -19,9 +19,10 @@ to run the tool. Use the maintainer references only when changing the code.
|
||||
| Page | Purpose |
|
||||
| --- | --- |
|
||||
| [Module internals](module-internals.md) | Current architecture, invariants, and regression guards by module. |
|
||||
| [Development](development.md) | Environment setup, dependency recovery, CI behavior, and fixture policy. |
|
||||
| [Code provenance](code-provenance.md) | Required notices for licensed derivative work. |
|
||||
| [Verification plan](verification-plan.md) | Verification methods, completed measurements, and remaining validation gaps. |
|
||||
| [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, and release workflow. |
|
||||
| [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, ComfyUI Registry, and the release workflow. |
|
||||
| [Watermarking landscape](watermarking-landscape.md) | Vendor signals and detection approaches. |
|
||||
| [SynthID technical reference](synthid.md) | Mechanism, detector access, robustness, and implications for this project. |
|
||||
|
||||
@@ -36,4 +37,5 @@ The current behavior is defined by the code, tests, README, and user guides.
|
||||
- [Doubao reverse-alpha research](research-doubao-distillation.md)
|
||||
- [SynthID identity research](synthid-robust-identity-research.md)
|
||||
- [SynthID identity follow-up](synthid-robust-identity-research-2026-06-08.md)
|
||||
- [Video SynthID quality research](video-synthid-quality-research.md)
|
||||
- [Text protection research](text-protection-research.md)
|
||||
+48
-27
@@ -1,6 +1,6 @@
|
||||
# Installation
|
||||
|
||||
Python 3.10.1 or newer is required.
|
||||
Python 3.11 through 3.14 are supported.
|
||||
|
||||
## Default metadata mode
|
||||
|
||||
@@ -48,7 +48,10 @@ uv tool install --force "remove-ai-watermarks[visible,heif]"
|
||||
|
||||
## Video processing
|
||||
|
||||
Video metadata inspection and stripping work with the default package. Stable
|
||||
Video metadata inspection works with the default package, and MP4 and MOV
|
||||
stripping uses the in-tree ISOBMFF box walker. Stripping the non-ISOBMFF
|
||||
containers (MKV, WebM, AVI, FLV, and the audio formats) and writing any cleaned
|
||||
video need ffmpeg on PATH, for example `brew install ffmpeg`. Stable
|
||||
visible-mark identification and removal, full video cleaning, and visible/all
|
||||
batch modes need the `video` extra:
|
||||
|
||||
@@ -65,22 +68,33 @@ uv tool install --force "remove-ai-watermarks[video,diffusion]"
|
||||
|
||||
## Invisible watermark removal
|
||||
|
||||
Diffusion based removal needs the `diffusion` extra:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[diffusion]"
|
||||
```
|
||||
|
||||
The code supports CUDA, XPU, MPS, and CPU devices. A GPU is recommended because
|
||||
CPU inference is slow.
|
||||
|
||||
For the CUDA only Qwen Image plus Z-Image profile:
|
||||
Install the `qwen-zimage` extra:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
```
|
||||
|
||||
The `qwen-zimage` extra includes the normal `diffusion` dependencies.
|
||||
Both remaining profiles run a Z-Image face stage on the DiffSynth runtime, so
|
||||
both need this extra. It includes the `diffusion` dependencies; `diffusion` on its
|
||||
own covers the torch and diffusers imports but not the face stage, so it is not
|
||||
enough to run a removal.
|
||||
|
||||
**An NVIDIA GPU is required.** `qwen-zimage` and `sdxl-zimage` are CUDA-only, and
|
||||
construction refuses any other device rather than falling back to a slow or broken
|
||||
one. There is no CPU, MPS or XPU path for invisible-watermark removal. Visible-mark
|
||||
removal, metadata stripping and every `identify` command still run anywhere.
|
||||
|
||||
Video SynthID regeneration is a separate VAE path and does still run on CPU or MPS;
|
||||
it needs the `diffusion` extra, not this one.
|
||||
|
||||
The experimental verified-text post-pass additionally needs LaMa:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
```
|
||||
|
||||
That extra includes `qwen-zimage` and `lama`; it does not add OCR. Text strings and
|
||||
line boxes must be reviewed before the run.
|
||||
|
||||
## Feature extras
|
||||
|
||||
@@ -94,14 +108,14 @@ application actually uses:
|
||||
| `visible` | Visible mark detection, OpenCV inpainting, and manual erasing | `pixels` | No |
|
||||
| `video` | Visible video identification/removal and timestamp preservation | `visible`, PyAV | No |
|
||||
| `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No |
|
||||
| `trustmark` | Adobe TrustMark detection | trustmark | Yes |
|
||||
| `diffusion` | Diffusion-based invisible watermark removal | `pixels`, Torch, Diffusers | Yes |
|
||||
| `trustmark` | Adobe TrustMark detection on Python 3.11-3.12 | trustmark | Yes |
|
||||
| `diffusion` | Torch and Diffusers runtime; video SynthID regeneration | `pixels`, Torch, Diffusers | Yes |
|
||||
| `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `esrgan` | Real-ESRGAN upscaling before diffusion | `pixels`, spandrel | Yes |
|
||||
| `qwen-zimage` | CUDA-only Qwen Image plus Z-Image pipeline | `diffusion`, DiffSynth | Yes |
|
||||
| `all` | Every production feature | All rows above | Yes |
|
||||
| `dev` | Tests, linting, typing, and upstream parity checks | `visible`, `detect`, upstream invisible-watermark | Yes, for parity tests |
|
||||
| `qwen-zimage` | Invisible image-watermark removal, both CUDA-only profiles | `diffusion`, DiffSynth | Yes |
|
||||
| `text-restoration` | Opt-in verified Qwen-VAE glyph restoration | `qwen-zimage`, `lama` | Yes |
|
||||
| `all` | Every production feature available on the active Python | All compatible rows above | Yes |
|
||||
| `dev` | Tests, linting, typing, and upstream parity checks | `video`, `detect`, upstream invisible-watermark | Yes, for parity tests |
|
||||
|
||||
Dependency composition:
|
||||
|
||||
@@ -113,15 +127,18 @@ flowchart LR
|
||||
diffusion --> pixels
|
||||
migan --> visible
|
||||
lama --> visible
|
||||
esrgan --> pixels
|
||||
qwen["qwen-zimage"] --> diffusion
|
||||
text["text-restoration"] --> qwen
|
||||
text --> lama
|
||||
heif
|
||||
trustmark
|
||||
```
|
||||
|
||||
`heif` and `trustmark` are independent branches. Combine them explicitly with
|
||||
another feature when required. The `all` bundle contains every production
|
||||
branch but never includes `dev`.
|
||||
another feature when required. TrustMark requires NumPy 1.x, which has no
|
||||
CPython 3.13 or 3.14 wheels, so that branch is available only on Python
|
||||
3.11-3.12. The `all` bundle contains every production branch compatible with
|
||||
the active Python and never includes `dev`.
|
||||
|
||||
Examples:
|
||||
|
||||
@@ -138,7 +155,7 @@ uv tool install --force "remove-ai-watermarks[video]"
|
||||
# DWT-DCT and TrustMark detection without diffusion removal
|
||||
uv tool install --force "remove-ai-watermarks[detect,trustmark]"
|
||||
|
||||
# Every production capability
|
||||
# Every production capability compatible with this Python
|
||||
uv tool install --force "remove-ai-watermarks[all]"
|
||||
|
||||
# An arbitrary minimal combination
|
||||
@@ -188,8 +205,10 @@ Run the complete project gate:
|
||||
bash maintain.sh
|
||||
```
|
||||
|
||||
The script runs dependency checks, linting, formatting checks, type checking,
|
||||
and the test suite.
|
||||
The script syncs every optional backend on top of the `dev` environment above,
|
||||
then runs dependency checks, linting, formatting, type checking, and the test
|
||||
suite. It applies Ruff fixes and formatting in place rather than only reporting
|
||||
them.
|
||||
|
||||
## Hugging Face authentication
|
||||
|
||||
@@ -216,5 +235,7 @@ The normal behavior is to skip diffusion when no supported local signal is
|
||||
found. A missing signal does not prove that the image is clean. If you know the
|
||||
image came from a relevant generator, use `--force`.
|
||||
|
||||
If the CLI reports that diffusion dependencies are unavailable, install the
|
||||
`diffusion` extra. Video SynthID removal needs both `video` and `diffusion`.
|
||||
If the CLI reports that the removal dependencies are unavailable, install the
|
||||
`qwen-zimage` extra. `diffusion` alone covers Torch and Diffusers but not the
|
||||
DiffSynth face stage that both profiles run. Video SynthID removal is a separate
|
||||
path and needs `video` and `diffusion`.
|
||||
+132
-44
@@ -57,18 +57,49 @@ Invisible removal does not decode and delete a payload. It regenerates the
|
||||
image through a diffusion pipeline. Faces, text, colors, and fine detail can
|
||||
change even when the watermark is successfully disrupted.
|
||||
|
||||
ControlNet is the default compatibility profile. It conditions on edges to
|
||||
preserve structure, but edges do not preserve identity or exact texture.
|
||||
Text, tables, and UI screenshots are the worst case for a forced scrub: the
|
||||
model redraws glyphs as plausible-but-wrong shapes. Do not run the invisible
|
||||
stage on such content unless a locally-detectable invisible signal exists.
|
||||
The default `all`, `invisible`, and `batch` commands already skip it then, and
|
||||
`visible` (which strips AI metadata by default) or `metadata` alone never
|
||||
redraws a glyph outside the filled watermark box. `--force` is for content you
|
||||
know carries a pixel watermark despite no local signal; on a clean screenshot
|
||||
it only buys distortion. Preserving or pasting back the original text pixels
|
||||
during a real scrub is deliberately not offered: an invisible watermark such
|
||||
as SynthID lives in the pixels everywhere, including inside the glyphs, so
|
||||
frozen text regions would keep the watermark
|
||||
([text protection research](text-protection-research.md)).
|
||||
|
||||
The CUDA only `qwen-zimage` profile adds a separate face stage and is the
|
||||
highest fidelity option in the current implementation. It is larger, slower,
|
||||
and still may alter small text or difficult faces.
|
||||
`qwen-zimage` is the default profile and `sdxl-zimage` the only alternative.
|
||||
Both are CUDA only and differ only in the global regeneration model: each
|
||||
conditions that stage on a canny edge map, which preserves structure but not
|
||||
identity or exact texture, and each then runs the same face stage.
|
||||
Existing face evaluations favor `qwen-zimage`, but there is no blanket fidelity
|
||||
ordering across content types. A fixed-seed, three-scene text comparison at the
|
||||
profile defaults found no stable winner: SDXL won one poster, Qwen won one, and
|
||||
the Chinese sign tied. Both are large, slow, and may still alter small text or
|
||||
difficult faces. The measurements and their OCR and oracle caveats are tracked
|
||||
in [`data/evaluations/fidelity/`](../data/evaluations/fidelity/README.md).
|
||||
A global Z-Image Turbo prototype preserved text substantially better at low
|
||||
strength, but it has no useful cross-provider operating point and is not a
|
||||
supported profile. Automatic text restorers also remain research-only:
|
||||
fresh-font and silhouette variants visibly changed typography. The higher-fidelity
|
||||
`vae-glyphs` route is available only as an experimental opt-in with verified strings
|
||||
and line geometry. It builds its donor internally but still requires an independently
|
||||
clean global anchor. Automatic OCR and line-box proposals are not reliable enough to
|
||||
remove those requirements, and exact oracle results do not establish a general mask,
|
||||
seed, runtime, or provider operating range. Qwen-Image-2.0 is hosted-only and exposes
|
||||
no equivalent low-strength denoise control. Exact experiments, controls, and pass
|
||||
rates are kept in
|
||||
[`text-protection-research.md`](text-protection-research.md) and the
|
||||
[`fidelity` evaluation record](../data/evaluations/fidelity/README.md).
|
||||
|
||||
### Removal cannot be verified locally for proprietary SynthID
|
||||
|
||||
The project has no public local SynthID pixel decoder. It can infer likely
|
||||
presence from supported provenance metadata, but a missing metadata proxy is
|
||||
not a negative pixel verdict.
|
||||
The project has no public local SynthID pixel decoder. It recognizes presence
|
||||
from supported provenance metadata: Google AI C2PA under Google's all-media
|
||||
policy, or current OpenAI C2PA with an explicit watermark action. A missing
|
||||
provenance signal is not a negative pixel verdict.
|
||||
|
||||
For important outputs:
|
||||
|
||||
@@ -111,6 +142,21 @@ measured 25.39 dB paired PSNR and a 1.058 motion-compensated temporal-residual
|
||||
ratio. This is one carrier, not a universal guarantee; hashes and exact verdicts
|
||||
are tracked in `data/evaluations/video-synthid-oracle.csv`.
|
||||
|
||||
### The invisible video path does not touch the audio track
|
||||
|
||||
`remove_video_invisible` regenerates the video stream and copies the source audio
|
||||
verbatim: the extracted audio bitstream of input and output has an identical
|
||||
sha256, measured on two carriers. It also strips every metadata marker, so the
|
||||
output can report clean from `get_ai_metadata` and `identify_video` while carrying
|
||||
an untouched generated audio track. Google's verifier scores audio and visual
|
||||
tracks separately, and a track this path never modifies is a track it cannot have
|
||||
cleaned.
|
||||
|
||||
Whether a given carrier's audio actually holds a mark the verifier reads has not
|
||||
been established -- that needs a provider verdict, which has not been obtained.
|
||||
Treat a clean local report on a clip with generated audio as unproven, not as a
|
||||
guarantee, and check the audio separately when it matters.
|
||||
|
||||
The shipped engine streams sampled frames in bounded batches, computes its
|
||||
fidelity metrics incrementally, and pipes regenerated pixels directly to
|
||||
ffmpeg. Its frame and latent memory is therefore bounded by `--batch-size`
|
||||
@@ -118,45 +164,76 @@ rather than clip duration. Runtime still grows linearly with duration, and the
|
||||
separate multi-candidate research sweep deliberately retains its short sampled
|
||||
prefix so it can reuse identical latents across candidate strengths.
|
||||
|
||||
The reported `psnr_db` is measured against the already-resized frame and before
|
||||
the H.264 encode, so it excludes the downscale, the frame decimation, and the
|
||||
encoder. It measures the VAE round trip plus latent noise at the working geometry.
|
||||
[`video-synthid-quality-research.md`](video-synthid-quality-research.md) records
|
||||
what the manifest rows constrain, what a higher-resolution or higher-frame-rate
|
||||
profile would need in order to be certified, and the audio-track question this
|
||||
path has not yet answered.
|
||||
|
||||
### Strength is content and seed dependent
|
||||
|
||||
For SDXL and ControlNet, the CLI resolves an unset strength from the detected
|
||||
vendor:
|
||||
The two profiles resolve an unset strength differently, because different things
|
||||
were measured for each.
|
||||
|
||||
- OpenAI: `0.10`;
|
||||
- Google: `0.15`;
|
||||
- unknown: `0.15`.
|
||||
`qwen-zimage` reads it from image area, through the resolution-adaptive denoise
|
||||
curve. The vendor is deliberately ignored for openai/unknown content: the curve,
|
||||
not the issuer, is what was calibrated. One measured exception (0.27.2):
|
||||
Google-provenance content takes the flat `QWEN_ZIMAGE_GOOGLE_STRENGTH` 0.30 floor
|
||||
- the curve's 0.154 top left a 4.33 MP fixture oracle-detected on the full
|
||||
production path (2026-08-18), while 0.30 anchors measured clean on two fixtures
|
||||
in two accounts.
|
||||
|
||||
An explicit `--strength` overrides these defaults. The defaults are operating
|
||||
points, not universal guarantees. Near a removal threshold, different content
|
||||
or a different random seed may change the verifier result.
|
||||
`sdxl-zimage` reads it from the C2PA issuer, on a flat ladder:
|
||||
|
||||
The base Qwen and `qwen-zimage` profiles have profile specific strength
|
||||
behavior. Consult `remove-ai-watermarks invisible --help` and the source of
|
||||
[`watermark_profiles.py`](../src/remove_ai_watermarks/_internal/watermark_profiles.py)
|
||||
for the current resolver.
|
||||
- OpenAI: `0.15`;
|
||||
- Google: `0.25`;
|
||||
- unknown: `0.25`, following the stricter of the two.
|
||||
|
||||
An SDXL global pass needs more denoise than Qwen at the same fidelity, and the
|
||||
values are flat rather than a curve because flat values are what was measured: each
|
||||
verdict came from a fixed strength at one size, and no size dependence has been
|
||||
established for that stage.
|
||||
|
||||
An explicit `--strength` overrides both. The defaults are operating points, not
|
||||
universal guarantees. Near a removal threshold, different content or a different
|
||||
random seed may change the verifier result, which is why both profiles are
|
||||
certified at a fixed seed. The live resolver is
|
||||
[`watermark_profiles.py`](../src/remove_ai_watermarks/_internal/watermark_profiles.py).
|
||||
|
||||
### Pipelines have different quality tradeoffs
|
||||
|
||||
| Pipeline | Main limit |
|
||||
| --- | --- |
|
||||
| `controlnet` | Edge conditioning can preserve a watermark carrying region too closely, and faces may drift. |
|
||||
| `sdxl` | Flat graphics and precise structure may receive too little or unhelpful change. |
|
||||
| `qwen` | Large CUDA oriented model; face smoothing can still be significant. |
|
||||
| `qwen-zimage` | CUDA only, large model stack, and limited broad certification across seeds and content. |
|
||||
| `sdxl-zimage` | CUDA only. Its strength ladder is flat per vendor, not a resolution curve, because flat values are what was measured. |
|
||||
|
||||
The legacy `default` profile name maps to `sdxl`. The `--auto` flag is
|
||||
deprecated, emits a warning, and changes nothing.
|
||||
Only manually verified `vae-glyphs` is an optional production stage, and it is
|
||||
experimental rather than a default.
|
||||
OCR plus LaMa recovered literal poster text but changed fonts and worsened whole-image
|
||||
fidelity. Restricting it to OCR-mismatched lines improved the tradeoff but still
|
||||
left a local shadow on one poster. The published AnyText2 SD1.5 checkpoint
|
||||
substituted Chinese characters and increased CER on the sign fixture. AnyText2XL
|
||||
is not published, and the released wrapper truncates individual text lines after
|
||||
20 characters.
|
||||
|
||||
The `controlnet`, `sdxl`, `qwen` and `default` profiles were removed, not aliased
|
||||
onward: a retired name is rejected at parse time rather than routed into a profile
|
||||
the caller never chose. There is no `--model`, `--steps`, `--guidance-scale`,
|
||||
`--device` or `--auto` option either; each profile pins its model stack, its
|
||||
per-stage schedule, CFG 1.0 and CUDA.
|
||||
|
||||
## Resolution and memory
|
||||
|
||||
### Small images are enlarged before SDXL based diffusion
|
||||
### Small images are processed at their native size
|
||||
|
||||
The SDXL, ControlNet, and base Qwen paths use a default minimum long side of
|
||||
`1024`. Smaller inputs are enlarged before diffusion and restored to their
|
||||
original dimensions afterward. Set `--min-resolution 0` to disable the floor.
|
||||
There is no minimum-resolution floor. It existed to enlarge small inputs toward
|
||||
SDXL's ~1024 training resolution and was removed with the SDXL profiles, which
|
||||
never applied it anyway. Both surviving profiles run at native geometry, so a
|
||||
small input is neither enlarged before diffusion nor restored afterward.
|
||||
|
||||
`qwen-zimage` does not apply this SDXL minimum resolution floor.
|
||||
`--max-resolution` still caps very large inputs, and only ever scales down.
|
||||
|
||||
### Large images stay at native resolution unless capped
|
||||
|
||||
@@ -169,23 +246,25 @@ overlapping tiles. It avoids the explicit downscale, but it is not pixel
|
||||
lossless: each tile is independently regenerated. With `qwen-zimage`, only the
|
||||
global Qwen stage is tiled; the face stage runs after tile blending.
|
||||
|
||||
### CPU offload is CUDA only
|
||||
### CPU offload trades speed for VRAM
|
||||
|
||||
`--cpu-offload` moves Diffusers model components between CPU and CUDA instead of
|
||||
keeping the complete standard pipeline in GPU memory. For `qwen-zimage`, it
|
||||
forces the face stack to use its offload path.
|
||||
`--cpu-offload` forces both stacks of the two-stage profile out of automatic
|
||||
device residency, streaming weights instead of pinning them. It reduces CUDA
|
||||
memory pressure at the cost of speed.
|
||||
|
||||
The option reduces CUDA memory pressure at the cost of speed. It has no effect
|
||||
on CPU or MPS and fails loudly when a CUDA Diffusers pipeline does not expose
|
||||
the required offload method.
|
||||
Residency is otherwise chosen from the card's total VRAM. On a card large enough
|
||||
to hold a stack, offloading is pure waste: DiffSynth drops weights to the meta
|
||||
device and re-reads every parameter from disk on each stage transition.
|
||||
|
||||
### MPS may fall back to CPU
|
||||
### There is no CPU or MPS fallback
|
||||
|
||||
The SDXL paths include an MPS out-of-memory fallback that reloads on CPU. A run
|
||||
that appears much slower after an MPS failure may be continuing on CPU.
|
||||
Invisible-watermark removal refuses any device but CUDA at construction. There is
|
||||
no MPS out-of-memory fallback that continues on CPU, and no lighter profile to
|
||||
drop to: both remaining profiles need an NVIDIA GPU.
|
||||
|
||||
Memory needs depend on the pipeline, input size, dtype, and machine. Use tiling,
|
||||
a resolution cap, or a lighter pipeline when necessary.
|
||||
When a run does not fit, the levers are `--tile` (native geometry, tiled
|
||||
diffusion), `--max-resolution` (an explicit downscale), and `--cpu-offload`.
|
||||
Memory needs depend on the profile, input size, dtype, and card.
|
||||
|
||||
## Metadata and formats
|
||||
|
||||
@@ -323,8 +402,17 @@ Diffusion, SDXL, and FLUX workflows. That decoder is sensitive to the carrier
|
||||
and transformations. A negative result is not a universal negative.
|
||||
|
||||
The `trustmark` extra adds Adobe TrustMark decoding. The implementation retains
|
||||
an additional JPEG re-encode gate because isolated decoder hits can otherwise
|
||||
be content noise.
|
||||
an additional JPEG re-encode gate and requires the binary payload and schema to
|
||||
remain identical because isolated decoder hits can otherwise be content noise.
|
||||
It accepts Variant P schemas 0-2. Variant Q requires a different model, and
|
||||
schema 3 is rejected at the measured precision threshold. Its NumPy 1.x runtime
|
||||
limits the extra to Python 3.11-3.12; the rest of the package remains supported
|
||||
through Python 3.14. Through 0.29.0 the extra also resolves lightning 2.6.5
|
||||
(PYSEC-2026-3624, no fixed release yet); the vulnerable `load_from_checkpoint`
|
||||
path is unreachable here because TrustMark loads its checksummed checkpoints
|
||||
with plain `torch.load`. Bump lightning and cut a patch release when a fix
|
||||
ships. The calibration history is in
|
||||
[module internals](module-internals.md#metadata-and-provenance).
|
||||
|
||||
External AI versus real image classifiers are out of scope. The project
|
||||
identifies concrete local provenance signals instead of shipping a generic
|
||||
|
||||
+531
-58
@@ -42,6 +42,10 @@ Important contracts:
|
||||
- `invisible` writes no output when no supported local signal is found, unless
|
||||
`--force` is supplied.
|
||||
- The two no-signal conditions currently share exit code `2`.
|
||||
- A quiet `metadata --check` and a successful `metadata --remove` (same for
|
||||
`video metadata`) end by repeating the `identify` limit: the pixel channel is
|
||||
untouched and a watermark such as SynthID has no local decoder once its
|
||||
metadata proxy is gone, so neither outcome is a clean verdict.
|
||||
- Hard processing and write failures exit with code `1`.
|
||||
- `all` can still write the completed visible and metadata stages when the
|
||||
diffusion dependencies are unavailable, but exits with code `1` so the
|
||||
@@ -53,9 +57,19 @@ The decorators for diffusion options are shared by `invisible`, `all`, and
|
||||
`batch`. The runtime help generated by Click is the source of truth for option
|
||||
names and defaults.
|
||||
|
||||
The deprecated `--auto` option does not select a pipeline or change adaptive
|
||||
polishing. [`_resolve_auto_polish`](../src/remove_ai_watermarks/cli.py) emits a
|
||||
warning and returns the explicit polish value unchanged.
|
||||
`--adaptive-polish` is tri-state: it declares `default=None`, so "the user did not
|
||||
choose" is a value the CLI passes through rather than a default it has to invent.
|
||||
`resolve_adaptive_polish` in `watermark_profiles.py` turns that `None` into the
|
||||
profile's answer (off for `qwen-zimage`, whose output already matches the input's
|
||||
detail level; on for `sdxl-zimage`). The same call runs inside
|
||||
`InvisibleEngine.remove_watermark`, so a library caller and a CLI caller on one
|
||||
profile get the same output.
|
||||
|
||||
It used to read Click's parameter source in the CLI instead. That put per-profile
|
||||
data in the argument-parsing layer, left the engine declaring the opposite default,
|
||||
and silently lost the polish for anything supplying the flag non-interactively (an
|
||||
envvar default or a wrapper calling `main()` with a defaulted list is classified
|
||||
`DEFAULT`). The seed follows the same rule: the CLI does not pre-resolve it either.
|
||||
|
||||
Regression coverage:
|
||||
|
||||
@@ -65,12 +79,39 @@ Regression coverage:
|
||||
|
||||
## High-level Python API
|
||||
|
||||
[`api.py`](../src/remove_ai_watermarks/api.py) provides:
|
||||
[`api.py`](../src/remove_ai_watermarks/api.py) provides the visible-mark entry
|
||||
points:
|
||||
|
||||
- `remove_visible`
|
||||
- `visible_provenance`
|
||||
|
||||
The package root exposes both lazily through
|
||||
and the image pipeline that the `all` and `batch` commands are thin wrappers
|
||||
over:
|
||||
|
||||
- `remove_all`, returning a `RemoveAllResult` after the visible, invisible, and
|
||||
metadata stages
|
||||
- `remove_batch`, returning a `BatchSummary` for one directory and one mode
|
||||
- `InvisibleOptions`, the invisible stage's knobs as one immutable value. Engine
|
||||
knobs only, under the engine's own names and defaults, so a bare
|
||||
`InvisibleOptions()` behaves exactly like calling the engine with no arguments.
|
||||
The engine takes them across two callables, `__init__` for what shapes the
|
||||
loaded stack and `remove_watermark` for the per-image ones, so `_run_invisible`
|
||||
forwards each field to the right one rather than splatting the whole bag. Two
|
||||
defaults silently stopped
|
||||
mirroring: `max_resolution=None` reached `_target_size`'s `max_resolution > 0`
|
||||
and raised `TypeError` on every library call, and `cpu_offload=True` made a
|
||||
library run slower than the identical CLI run.
|
||||
`TestInvisibleOptionsMirrorTheEngine` compares the two signatures field by
|
||||
field, and deliberately keeps no exception table: a field needing one is a
|
||||
field that belongs elsewhere. `force` was such a field, and it decides whether
|
||||
the engine runs rather than how, so it is a parameter of `remove_all` and
|
||||
`remove_batch` next to `backend` and `sensitivity`
|
||||
- `MetadataStripIncomplete`, raised before any write when AI metadata survives
|
||||
|
||||
`remove_all` reports progress as `(stage, detail)` pairs of stable tokens, not
|
||||
prose the caller has to parse back.
|
||||
|
||||
The package root exposes all of them lazily through
|
||||
[`__getattr__`](../src/remove_ai_watermarks/__init__.py), keeping a plain package
|
||||
import free of the heavier image and model imports.
|
||||
|
||||
@@ -121,7 +162,14 @@ Native MP4/MOV TC260 labels follow TC260-PG-20257A:
|
||||
`moov.udta.meta.keys` maps an `AIGC` key to a raw JSON value in `ilst`.
|
||||
[`_internal/isobmff.py`](../src/remove_ai_watermarks/_internal/isobmff.py) walks those
|
||||
nested boxes by seeking, so detection reaches a tail `moov` without reading the
|
||||
preceding `mdat`. The MP4/MOV/M4V/M4A removal path first validates the top-level
|
||||
preceding `mdat`. Two Doubao iOS variants sit outside that normative placement
|
||||
and are covered by the same walker (2026-08-17 corpus findings, both previously
|
||||
undetected): a QuickTime-form `meta` box as a *direct* `moov` child (no FullBox
|
||||
header, disambiguated by probing the child-box offset), and a QuickTime
|
||||
`hdlr=mdir` metadata list under `udta.meta` whose `ilst` data items carry the
|
||||
validated JSON with no `keys` box at all (content-validated, so only genuine
|
||||
TC260 JSON matches; the keyless entry has no key name to blank, so its value
|
||||
alone is spaced out). The MP4/MOV/M4V/M4A removal path first validates the top-level
|
||||
box walk, then copies the source to a sibling temporary file in bounded chunks.
|
||||
Supported C2PA/JUMBF/AI-label boxes become same-size `free` boxes with blank
|
||||
payloads; TC260 removal changes the four-byte key to `free` and blanks only the
|
||||
@@ -215,7 +263,35 @@ quality measurements. Neither is a watermark detector. The high-level result
|
||||
reports completed removal without a separate verification-status flag. The companion
|
||||
`scripts/video_synthid_sweep.py` imports the same engine helpers to build a
|
||||
matched control and candidate grid, preventing research and shipped
|
||||
regeneration paths from drifting. The full-clip oracle floor is
|
||||
regeneration paths from drifting.
|
||||
|
||||
The engine's `psnr_db` is measured against the already-resized frame and before
|
||||
the encoder, so it scores the VAE round trip plus latent noise and cannot see
|
||||
the downscale, the decimation, or the codec. No in-loop metric can: the
|
||||
candidate frame is captured before it reaches the encoder pipe.
|
||||
`scripts/video_fidelity_probe.py` covers the rest by decoding the delivered file
|
||||
after muxing, upscaling it back to the source geometry, and scoring it against
|
||||
the untouched source frames. It also reports the delivered file's bitrate, so a
|
||||
fixed-crf bitrate rise cannot read as unchanged quality; because the mux copies
|
||||
source audio verbatim, that figure is a container bitrate, not a video one. The
|
||||
probe streams and accumulates the same way the engine does, so its peak memory
|
||||
does not grow with clip length. It drives the source through the engine's own
|
||||
`_iter_sampled_frames` at the source geometry rather than repeating the
|
||||
selection rule: a frame-count check cannot catch a rule that reorders frames
|
||||
without changing how many, so the rule itself has to be shared.
|
||||
|
||||
`load_video_vae_runtime` asserts the default model's latent scaling factor
|
||||
against `VIDEO_SYNTHID_VAE_SCALING_FACTOR` and warns that no certified profile
|
||||
exists for any other model. The published `sd-vae-ft-mse` config carries no
|
||||
`scaling_factor` key, so the value is a `diffusers` class default under an
|
||||
upper-unbounded pin, and the certified profile is a perturbation-to-signal ratio
|
||||
rather than a bare `noise_std`. A library bump that moved that default would
|
||||
otherwise rescale every perturbation with a green suite. The validated factor is
|
||||
carried on `VideoVaeRuntime` and passed into encode and decode, so the gated
|
||||
value and the applied value are one measurement rather than three independent
|
||||
reads. `scripts/video_synthid_sweep.py` loads through the same function: the
|
||||
harness that produces the certified rows is the last place that should be exempt
|
||||
from the gate. The full-clip oracle floor is
|
||||
`noise_std=0.15`: on the public eight-second Veo carrier, `0.10` remained
|
||||
detected while `0.15` did not.
|
||||
|
||||
@@ -243,13 +319,17 @@ takes the first stable result in specificity order (`sora`, `veo`, `seedance`,
|
||||
candidate. Removal also collects authoritative per-frame timestamps for the
|
||||
encoder, while identification omits that unused ffprobe pass.
|
||||
|
||||
Every per-frame result is untrusted. The provider-specific stabilization
|
||||
wrappers share one recurrence implementation, while retaining separate visual
|
||||
floors and minimum-run policy. Provenance can relax a low-contrast run only
|
||||
Every per-frame result is untrusted. Each provider's floors, minimum-run policy,
|
||||
fill padding and mask style are one row in `VISIBLE_MARK_POLICIES`, and every mark
|
||||
enters the same `stabilize_localizations` entry point; the recurrence
|
||||
implementation underneath knows nothing about providers. That policy row also
|
||||
carries `accepts_provenance`, which forces `provenance=False` for Hailuo and Kling
|
||||
— they have no metadata that could confirm them, and the guarantee used to be
|
||||
structural (their wrappers took no `provenance` parameter at all). Provenance can relax a low-contrast run only
|
||||
after recurring visual evidence exists. Sora transition frames follow the
|
||||
nearest confirmed moving position only with Sora provenance. Veo, Seedance,
|
||||
Dola, Hailuo, and Kling additionally require candidates to remain anchored to
|
||||
the start of a run. This rejects slowly drifting scene details that still have
|
||||
nearest confirmed moving position only with Sora provenance. Seedance, Dola,
|
||||
Hailuo, and Kling additionally require candidates to remain anchored to the
|
||||
start of a run. This rejects slowly drifting scene details that still have
|
||||
high frame-to-frame overlap. Hailuo and Kling do not infer provenance from
|
||||
technical encoder tags; their confirmed public samples carried no provider
|
||||
metadata.
|
||||
@@ -279,6 +359,13 @@ Regression coverage:
|
||||
properties, metadata stripping, and a large-`mdat` metadata case that rejects
|
||||
any full-source `read_bytes()` call. CI installs ffmpeg explicitly for this
|
||||
test so the integration gate cannot silently skip.
|
||||
- [`test_video_fidelity_probe.py`](../tests/test_video_fidelity_probe.py), which
|
||||
builds its clips from solid-color frames whose index is recoverable from the
|
||||
pixels, so a decimation that keeps the frame count but shifts the phase scores
|
||||
far worse instead of passing unnoticed. It also asserts that the probe binds the
|
||||
engine's sampler rather than a copy of it, and it holds the suite's only
|
||||
constraint on that sampler's phase. Nothing here needs a model, but it does need
|
||||
ffmpeg, so CI runs this file in the same job that installs it.
|
||||
|
||||
## Metadata and provenance
|
||||
|
||||
@@ -288,9 +375,47 @@ Regression coverage:
|
||||
official `c2pa-python` reader first. Its byte-level PNG parser remains a fallback
|
||||
for partial and synthetic fixtures that the official reader rejects.
|
||||
|
||||
Structured extraction is limited to the active manifest and the ingredient
|
||||
manifests reachable from it. Validation is preserved as separate dimensions:
|
||||
asset binding integrity, claim signature, signer trust, and signer certificate
|
||||
validity. A matching asset hash and valid claim signature do not make an
|
||||
untrusted or expired signer trusted. `identify` therefore assigns high confidence
|
||||
only when all four dimensions validate, medium confidence to an intact but
|
||||
untrusted or unvalidated claim, and no origin verdict to a hash/signature failure.
|
||||
The failed claim remains in the marker inventory and keeps the removal gate
|
||||
fail-safe because a post-signing container edit can invalidate C2PA without
|
||||
removing a declared pixel watermark.
|
||||
|
||||
The structured walk also treats an exact known AI product in a reachable
|
||||
`claim_generator` as an AI assertion. This covers update chains where the active
|
||||
manifest names only `c2pa-tool` while a validated ingredient names Dreamina, and
|
||||
Firefly chains that identify `Adobe_Firefly` without repeating a digital source
|
||||
type. Unreachable manifests remain excluded.
|
||||
|
||||
Reachable `c2pa.soft-binding*` assertions retain their exact `alg` and bounded,
|
||||
printable block `value` in addition to the normalized vendor label. A block value
|
||||
without its algorithm is not surfaced because it cannot be attributed to a
|
||||
decoder or registry entry. `com.microsoft.invismark.1` uses that value as the
|
||||
pixel-watermark identifier in Microsoft Paint output. An InvisMark soft binding
|
||||
keeps the invisible-removal gate fail-safe even when the C2PA asset binding has
|
||||
since become invalid, because metadata damage does not prove the pixel carrier
|
||||
disappeared. Content-fingerprint soft bindings do not trigger pixel regeneration.
|
||||
|
||||
The SDK default enables trust verification but supplies no production trust
|
||||
anchors. Consequently, an installation without an explicitly maintained C2PA
|
||||
trust bundle reports otherwise valid signer chains as untrusted and keeps their
|
||||
attribution at medium confidence. Shipping or fetching an official trust bundle
|
||||
requires a separate update, provenance, and availability policy; do not silently
|
||||
convert `signingCredential.untrusted` into trusted based on a vendor-name match.
|
||||
|
||||
Vendor attribution comes from the registry in
|
||||
[`_internal/constants.py`](../src/remove_ai_watermarks/_internal/constants.py). Derived
|
||||
issuer and platform maps should not be maintained separately.
|
||||
For an AI C2PA claim, a recognized product in `claim_generator` takes precedence
|
||||
over the certificate issuer: an application can sign through an upstream model
|
||||
provider without becoming that provider's product. Only exact product mappings
|
||||
receive this precedence; an unknown claim generator still falls back to issuer
|
||||
attribution.
|
||||
|
||||
### Metadata scanning and stripping
|
||||
|
||||
@@ -299,9 +424,23 @@ metadata scanners and `remove_ai_metadata`.
|
||||
|
||||
Key contracts:
|
||||
|
||||
- `scan_head` is the shared cached input for bounded byte scans.
|
||||
- `scan_head` is the shared cached input for bounded byte scans. It fills the buffer
|
||||
in two layers. Structural readers first, one per container, each seeking past the
|
||||
pixel payload to reach metadata placed beyond the window: `isobmff.scan_c2pa_region`,
|
||||
`_png_late_metadata`, `_riff_late_metadata`. A decoder-backed fallback last,
|
||||
`_decoder_visible_text`, for metadata the raw bytes do not spell at all — a
|
||||
zlib-compressed PNG `zTXt` packet is readable only after inflation. The layers are
|
||||
ordered that way because the structural readers work on files no decoder can open.
|
||||
- A C2PA reader failure is logged at warning, not debug. It returns the same `None` as
|
||||
a file with no manifest, so nothing downstream can distinguish "no credentials" from
|
||||
"the credentials could not be read", and the second silently downgrades a verdict.
|
||||
- JPEG stripping walks metadata segments and preserves the entropy-coded image
|
||||
scan.
|
||||
- Exact app-export JSON disclosures in EXIF `ImageDescription` or `UserComment`
|
||||
share one parser. Known AI-product provenance is removable without by itself
|
||||
asserting that the pixels were generated; explicit `aigc_info` discriminator
|
||||
values and Dreamina `exportType=generation` do assert AI origin. Ordinary
|
||||
Aweme, retouch, and `lv` editor exports are preserved.
|
||||
- ISOBMFF containers use
|
||||
[`_internal/isobmff.py`](../src/remove_ai_watermarks/_internal/isobmff.py).
|
||||
- Native MP4/MOV TC260 `AIGC` entries are read from
|
||||
@@ -338,30 +477,204 @@ metadata extraction from verdict logic:
|
||||
- `extract_provenance_evidence` reads the supported metadata signals into
|
||||
`ProvenanceEvidence`.
|
||||
- `evidence_from_metadata_record` normalizes an externally collected nested
|
||||
metadata record into the same evidence type without file access. Diagnostic
|
||||
values under `error` and `kind` are excluded from evidence while nested raw
|
||||
bytes remain available through encoded binary fields.
|
||||
- `identify_from_evidence` evaluates that evidence without reopening the source.
|
||||
metadata record into the same evidence type without file access. Versioned native
|
||||
records accept only source-derived fields; filenames, hashes, timings, errors,
|
||||
prior verdicts, and pixel results cannot become evidence. Unknown native schema
|
||||
versions and other record types are rejected.
|
||||
- The vendor registries are matched over `_metadata_region(head)`, not the whole scan
|
||||
buffer: they see the container's metadata and not its coded pixels. The tokens are
|
||||
raw substrings and the shortest are four and five bytes, so over a megabyte of
|
||||
compressed data one turns up by chance, and the entry it hits may assert AI. The
|
||||
trim happens only when the container parses -- a malformed or unknown one is left
|
||||
whole, because dropping real evidence to avoid a chance match is the wrong trade.
|
||||
- `identify_from_evidence` evaluates that evidence without reopening the source. Rules
|
||||
that decide a verdict live here, not in extraction: extraction has two
|
||||
implementations, and a rule in only one of them is a rule the other lacks. The
|
||||
SynthID provenance evidence is the worked example — its structured form comes from the manifest,
|
||||
and the byte-scan fallback for containers no parser reaches runs in the verdict, so
|
||||
both extractors reach the same answer. It did not, and the record path silently
|
||||
reported no SynthID for images the file path flagged.
|
||||
- `identify` preserves the path-based API and adds the optional registered
|
||||
visible-mark and open invisible-watermark decoders after extraction.
|
||||
|
||||
### Portable metadata record
|
||||
|
||||
[`metadata_record.py`](../src/remove_ai_watermarks/metadata_record.py) produces the
|
||||
record `evidence_from_metadata_record` consumes, so collection and verdict can run
|
||||
on different machines. Its contract is equality with the file path, and the three
|
||||
defects found while establishing that equality are the reason each rule exists:
|
||||
|
||||
- It walks the file's RAW head, never the `scan_head` buffer. That buffer is the
|
||||
head concatenated with late metadata payloads, so a structural walk runs off the
|
||||
end of the real head and parses appended bytes as chunks, inflating the record and
|
||||
creating false signals.
|
||||
- Samsung Galaxy AI splits its evidence: the `PhotoEditor_Re_Edit_Data` marker sits
|
||||
in the post-EOI trailer while the `genAIType` value it is gated on can sit inside
|
||||
the entropy-coded scan. A marked file therefore keeps the whole tail window, not
|
||||
just the trailer.
|
||||
- PIL's info keys are emitted in the file path's own candidate order
|
||||
(`Software`, `Source`, `Title`, `Description`, then EXIF). `generator_from_metadata`
|
||||
returns the FIRST candidate carrying a known token, so preserving candidate order
|
||||
is part of verdict equivalence.
|
||||
|
||||
The transport is independently versioned as `provenance_metadata` schema 1. Native
|
||||
records require the exact integer schema version and a `complete` status. Source
|
||||
read failures are explicit error records and cannot be judged. WebP walks the full
|
||||
declared RIFF container by seeking over `VP8`, `VP8L`, `ALPH`, and `ANMF`, so late
|
||||
XMP/C2PA remains visible without shipping coded frames or parsing appended trailer
|
||||
bytes as chunks.
|
||||
|
||||
Pixel forensics are deliberately absent: the provenance path does not read them.
|
||||
Verdict equivalence is checked over tracked fixtures and a separate local evaluation
|
||||
corpus.
|
||||
|
||||
### Broad forensic metadata
|
||||
|
||||
[`forensic_metadata.py`](../src/remove_ai_watermarks/forensic_metadata.py) owns the
|
||||
wide metadata-only inspection record: hashes and timestamps, full EXIF/IPTC, C2PA,
|
||||
container inventories, bounded binary metadata, and embedded-thumbnail forensics.
|
||||
It is a separate `forensic_metadata` record type and is deliberately rejected by the
|
||||
provenance normalizer. Integration code publishes the strict
|
||||
`ProvenanceReport.to_dict()` alongside it rather than letting operational fields or
|
||||
derived results influence detection.
|
||||
|
||||
### Pixel forensics
|
||||
|
||||
[`pixel_evidence.py`](../src/remove_ai_watermarks/pixel_evidence.py) measures six
|
||||
families of scale-robust pixel statistics (block-DCT histograms and Benford
|
||||
deviation, FFT band energies and CFA peaks, high-pass residual, error level,
|
||||
gradient, color) in a single decode, sharing the intermediate maps between them.
|
||||
|
||||
It remains independent of verdict and removal. `PixelEvidence.to_dict()` is the
|
||||
versioned service boundary: it omits the local path, exposes complete/partial/error
|
||||
status, keeps exception details in logs, and can include opt-in per-stage timings.
|
||||
|
||||
The provenance metadata collector, broad forensic collector, provenance report,
|
||||
and pixel report all accept an explicit output `schema_version`. Package releases
|
||||
may add an output schema while retaining older serializers, so a rolling consumer
|
||||
can keep requesting the version it already understands. Within one schema, changes
|
||||
are additive; existing fields, types, meanings, signal names, and watermark labels
|
||||
remain stable. Unsupported selections raise before a different shape is returned.
|
||||
|
||||
`artifacts=True` additionally returns the spatial layer: a perceptual hash, a 128px
|
||||
JPEG thumbnail, and coarse ELA, residual and phase maps. Those identify the source
|
||||
image rather than describe it, which is why they are opt-in and a separate field: a
|
||||
caller storing them is handling image content, not statistics about it.
|
||||
|
||||
The DWT-DCT detector and the visible-mark stage share a single decode of the
|
||||
source, held by
|
||||
a per-call `_SharedDecode`. It exposes two accessors because the two arms need
|
||||
opposite failure handling: the visible arm swallows a decode failure (no cv2, no
|
||||
visible marks, metadata verdict untouched), while the invisible arm re-raises it
|
||||
so `has_invisible_target` reaches its documented fail-safe `True`. Swallowing it
|
||||
there would skip a diffusion scrub on a file that used to get one. TrustMark
|
||||
deliberately keeps its own Pillow decode: cv2 and Pillow disagree on EXIF
|
||||
orientation and on 16-bit PNG, so substituting one for the other is not
|
||||
behavior-preserving.
|
||||
|
||||
The metadata probes (`aigc_label`, `xai_signature`, `iptc_ai_system`,
|
||||
`huggingface_job`, `samsung_genai`) and `extract_c2pa_info` are memoized on
|
||||
`(path, mtime_ns, size)`. One `identify` reaches each of them twice, and each
|
||||
re-walks the container or re-runs the manifest reader. Size is in the key as well
|
||||
as mtime because this package rewrites files in place, and an in-place rewrite
|
||||
can land inside one mtime tick. The C2PA key additionally carries the
|
||||
reader-availability flag: with the official reader the manifest comes back as a
|
||||
store and without it from the PNG chunk parser, so the answer depends on process
|
||||
state and not on the file alone.
|
||||
|
||||
Native-container TC260 readers (`isobmff`, `ebml`, `riff`, `flv`) all run, in that
|
||||
order, on every file. Each self-gates on its own magic bytes after a 4-12 byte
|
||||
read, so gating the AVI and FLV ones on the file extension as well was redundant
|
||||
and made a correctly formatted container served under the wrong name invisible.
|
||||
WebP is the one input class the now-unconditional RIFF reader newly touches; its
|
||||
`AVI ` form check is what rejects it.
|
||||
|
||||
`api._SourceEvidence` extracts that metadata once per `remove_all` call and serves
|
||||
both the visible pass (which vendor is confirmed) and the scrub gate (is there an
|
||||
invisible target). It is per-call, never module-level: `batch` may write its
|
||||
output over its input, and a holder that outlived one call would answer the scrub
|
||||
gate from pre-write evidence. In `batch` each stage builds its own holder after
|
||||
any write that precedes it, for the same reason. Every accessor fails safe the way
|
||||
the function it replaces does — no provenance means no relaxation, and an unknown
|
||||
invisible target means scrub rather than skip.
|
||||
|
||||
The `detect` extra composes the shared `pixels` runtime with PyWavelets. Its
|
||||
in-tree [`dwt_dct.py`](../src/remove_ai_watermarks/dwt_dct.py) decoder preserves
|
||||
the upstream matrix algorithm without installing Torch or non-headless OpenCV.
|
||||
The upstream MIT notice ships inside the wheel under `licenses/`.
|
||||
in-tree [`dwt_dct.py`](../src/remove_ai_watermarks/dwt_dct.py) decoder reproduces
|
||||
the upstream algorithm's output bit for bit without installing Torch or
|
||||
non-headless OpenCV; the block scan is vectorized rather than transcribed, so
|
||||
the file no longer reads line by line against `maxDct.py`. The upstream MIT
|
||||
notice ships inside the wheel under `licenses/`.
|
||||
|
||||
`is_ai_generated` is `True` or `None`; absence of evidence is not reported as a
|
||||
human-made verdict. `ai_source_kind` distinguishes fully generated content from
|
||||
AI-enhanced composites when the source metadata provides that distinction.
|
||||
|
||||
TrustMark is reported as a watermark signal but does not by itself assert AI
|
||||
origin because it can also protect human-authored content.
|
||||
origin because it can also protect human-authored content. The decoder requests
|
||||
binary mode, matching Adobe's Durable Content Credentials example, then requires
|
||||
the same payload and schema after a quality-95 JPEG round-trip. Only Variant P
|
||||
schemas 0-2 count as positives. Schema 3 is below the precision threshold: all
|
||||
38 measured historical false-positive candidates used it, and six retained the
|
||||
same false payload after re-encoding. The official Adobe Variant P schema-1
|
||||
fixture in `data/fixtures/provenance/` is the positive regression control.
|
||||
|
||||
#### Why the DWT-DCT decoder looks the way it does
|
||||
|
||||
Two things in `dwt_dct.py` are load-bearing and neither is obvious from the code.
|
||||
|
||||
`_approximation` calls `pywt.dwt` twice instead of `pywt.dwt2`, and transposes
|
||||
before each pass. A factorial ablation over both axes separates the two:
|
||||
skipping the three detail bands `dwt2` computes and this code discards is worth
|
||||
**4%**, while the transposes are worth **2.8x**, because pywt walks the axis it
|
||||
transforms and on axis 0 of a C-contiguous plane that is a column walk. The
|
||||
arithmetic saving is the intuitive explanation and it is the small term; four
|
||||
independent profiles named it as the mechanism before the ablation contradicted
|
||||
them.
|
||||
|
||||
Bit-identity is a hard requirement, not a preference. For uint8 input the exact
|
||||
Haar LL value is a multiple of 0.5 and the bit test is `peak % 36 > 18.0`, a
|
||||
threshold sitting exactly on a representable value that ~1 block in 72 lands on,
|
||||
so a 1-ulp difference deterministically flips real bits. That is why the ~16x
|
||||
available from a hand-rolled numpy Haar is unreachable rather than merely
|
||||
untaken: pywt's C convolution contracts into an FMA that numpy has no ufunc for,
|
||||
and `np.longdouble` is 64-bit on arm64 macOS.
|
||||
|
||||
Each pass is one flat `pywt.downcoef` call over a raveled strip rather than
|
||||
`pywt.dwt(..., axis=1)[0]`, and the plane is processed in strips of `_STRIP`
|
||||
block-rows so no full-plane float64 intermediate is ever materialized. The strip
|
||||
height is not a tuned value: 8 through 64 all scored inside each other's noise
|
||||
with unstable ordering, and only "strips at all" versus whole-plane matters.
|
||||
|
||||
`_approximation`'s even-last-axis check is load-bearing, not defensive. Haar's
|
||||
filter is length 2, so an even row length keeps every pair inside its own row;
|
||||
on an odd width the pairs walk across row boundaries and the reshape still
|
||||
succeeds whenever the total is even, which would be wrong bits with no
|
||||
exception. Both call sites are even by construction today, and
|
||||
`TestRaveledHaarPass` pins both halves -- the `downcoef`/`dwt` equivalence,
|
||||
which a pywt upgrade could take away, and the raise on an odd width.
|
||||
|
||||
Measured on a 1536x2816 image, all arms timed in one process: the decoder went
|
||||
0.112 s (the original per-block Python loop) to 0.011 s vectorized to 0.007 s
|
||||
with strips, and a warm `identify()` 1.757 s to 1.365 s on the first step and a
|
||||
further 0.4% on the second. That last figure is the point at which this target
|
||||
is finished: the decoder is now under 2% of `identify()`, so speed here has
|
||||
stopped buying anything. What the strips buy is peak RSS in the stage, 111 MB to
|
||||
21 MB on a 4.3 MP image, which is what matters on the memory-limited Space.
|
||||
|
||||
Both steps were verified by recording decoder output and detector verdict over
|
||||
200 sampled `data/` images plus two synthesized carriers before and after: the
|
||||
record is byte-identical, as are seven degenerate shapes (`1x65536` through
|
||||
`8x8192`, plus an odd width) that clear the caller's area check.
|
||||
|
||||
Regression coverage:
|
||||
|
||||
- [`test_identify.py`](../tests/test_identify.py)
|
||||
- [`test_trustmark_detector.py`](../tests/test_trustmark_detector.py)
|
||||
- [`test_invisible_watermark.py`](../tests/test_invisible_watermark.py)
|
||||
- [`test_invisible_watermark.py`](../tests/test_invisible_watermark.py) --
|
||||
note `test_in_tree_decoder_matches_upstream` is the parity guard against
|
||||
upstream's own decoder, and the whole module is `skipif(not is_available())`.
|
||||
A green run without the `detect` extra installed has not checked parity at
|
||||
all, so a decoder change still owes the before/after verdict record.
|
||||
|
||||
## Visible mark removal
|
||||
|
||||
@@ -387,6 +700,30 @@ The Jimeng pill has an additional decision gate because its visual detector is
|
||||
weaker than the other registered marks. Keep that policy in the registry, not
|
||||
inside unrelated detector engines.
|
||||
|
||||
Everything about a mark is one registry row: its product family, its label regime,
|
||||
the platform sentence `identify` reports for it, and the metadata signals that
|
||||
confirm its vendor. `identify._VISIBLE_MARK_PLATFORM` and the signal mapping in
|
||||
`api.visible_provenance` are derived from those rows rather than hand-maintained
|
||||
beside them, so registering a mark is one edit. Two marks carry no platform of
|
||||
their own: the Gemini sparkle has its own higher-confidence path, and the
|
||||
capture-less pill is too weak to attribute.
|
||||
|
||||
The set of marks that veto the pill is DERIVED from the registry rows: every mark
|
||||
under the same label regime (`tc260`) belonging to a different product. It used to
|
||||
be a hand-written list of keys, and that list drifted -- LibLibAI was registered
|
||||
alongside RunningHub and Baidu, both of which were added to it, and LibLibAI was
|
||||
not, so a confident LibLibAI detection did not suppress the pill the way its two
|
||||
siblings did. Marks outside the TC260 regime (Gemini, Samsung) are deliberately
|
||||
not vetoers: neither can put `jimeng` into `provenance`, so neither can enable the
|
||||
arm it would be vetoing.
|
||||
|
||||
A TC260 label relaxes the vendor its `ContentProducer` names, resolved through
|
||||
`KnownMark.tc260_producer_codes`. The label itself is vendor-agnostic, so this used to
|
||||
relax ByteDance's two products on every China-AIGC image -- which both risked a
|
||||
false fill on an image carrying some other vendor's mark and denied that vendor's
|
||||
own mark the relaxed gate its `provenance_ncc_factor` was calibrated for. An
|
||||
absent or unmapped producer still falls back to the ByteDance pair.
|
||||
|
||||
`remove_auto_marks` removes every selected mark, not only the strongest one.
|
||||
This matters for images that carry marks in more than one corner.
|
||||
|
||||
@@ -430,7 +767,23 @@ be represented by the shared base:
|
||||
|
||||
The detector and removal mask must use compatible geometry. A detector that
|
||||
fires while producing an empty or misplaced mask is a removal failure even if
|
||||
the detection test passes.
|
||||
the detection test passes. That parity is now structural rather than a
|
||||
convention: the three continuous front ends share one ladder sweep
|
||||
(`_ladder_best`), and the winning box travels to the mask on
|
||||
`TextMarkDetection.match_box` instead of being swept a second time.
|
||||
|
||||
Detection is split into a trust-level-blind `_scan` and a `_verdict` that applies
|
||||
the threshold. `detect_both` returns the strict and relaxed verdicts from one
|
||||
scan, which is what the arbiter's perception stage calls. A per-mark demotion
|
||||
belongs in the `_post_gate` hook, never in a `detect` override: an override is
|
||||
invisible to the single-pass path, and the RunningHub and Yuanbao anchor gates
|
||||
were briefly skipped there for exactly that reason.
|
||||
|
||||
A mark whose removable footprint differs from what the detector localizes
|
||||
overrides `_footprint_rect` (which policy) and `_extend_match_box` (how far the
|
||||
box grows), not the whole `footprint_mask`. Baidu extends right to the corner tag
|
||||
and LibLibAI extends left to the triangle logo; both inherit every guard around
|
||||
that arithmetic.
|
||||
|
||||
Yuanbao uses the polarity-independent `contrast` front end because its standard
|
||||
two-line mark can be light on dark scenes or dark on light scenes. Its detector
|
||||
@@ -477,48 +830,75 @@ Regression coverage:
|
||||
[`_internal/watermark_profiles.py`](../src/remove_ai_watermarks/_internal/watermark_profiles.py)
|
||||
is the source of truth for:
|
||||
|
||||
- profile aliases;
|
||||
- default model identifiers;
|
||||
- default steps and seeds;
|
||||
- vendor-adaptive strength resolution;
|
||||
- the minimum viable step calculation.
|
||||
- profile names and their underscore spellings;
|
||||
- the fixed seed;
|
||||
- the SDXL global-stage checkpoint id (`SDXL_MODEL_ID`) and the Canny ControlNet id;
|
||||
- strength resolution for both profiles.
|
||||
|
||||
The current profiles are `qwen-zimage` (the default) and `sdxl-zimage`, and both
|
||||
are CUDA-only. `controlnet`, `sdxl`, `qwen` and `default` were removed rather than
|
||||
kept as a CPU path, and are rejected rather than aliased onward. There is no
|
||||
content-dependent automatic router.
|
||||
|
||||
The current profiles are `controlnet`, `sdxl`, `qwen`, and `qwen-zimage`.
|
||||
For serverless cold starts, `InvisibleEngine.preload(global_only=True)` loads the
|
||||
mandatory Qwen stage and YuNet while leaving the optional Z-Image and SAM face
|
||||
mandatory global stage and YuNet while leaving the optional Z-Image and SAM face
|
||||
stack lazy until a face is detected. The default `preload()` still loads every
|
||||
stage.
|
||||
`default` is a legacy alias for `sdxl`. There is no content-dependent automatic
|
||||
router.
|
||||
|
||||
**What is deliberately not a parameter.** Model id, step count and CFG are fixed
|
||||
by the profile, so none of them appears in `WatermarkRemover.__init__`,
|
||||
`remove_watermark`, `InvisibleEngine`, or the CLI. They used to be accepted and
|
||||
then rejected several frames down; a signature that refuses the argument outright
|
||||
fails where the caller can act on it, and stops a wrapper from threading a value
|
||||
that would silently do nothing. The step count and CFG live with the stage that
|
||||
runs them (`GLOBAL_STEPS`, `FACE_STEPS`, `GLOBAL_CFG`, `FACE_CFG` in
|
||||
`qwen_zimage_pipeline.py`). The dtype is likewise profile-owned: see "Face-stage
|
||||
dtype" for what an override cost the last time one existed.
|
||||
|
||||
`device` is the exception and remains a library parameter: `None` or `"auto"`
|
||||
detect, `"cuda"` pins without detecting (which is what a container that knows its
|
||||
hardware wants), and any other value raises at construction. It is not a CLI
|
||||
option, because the only useful value a user could type is the one detection
|
||||
already returns.
|
||||
|
||||
[`invisible_engine.py`](../src/remove_ai_watermarks/invisible_engine.py) handles
|
||||
image sizing, optional pre-upscaling, postprocessing, and the public engine
|
||||
image sizing, postprocessing, and the public engine
|
||||
interface. It delegates model execution to
|
||||
[`_internal/watermark_remover.py`](../src/remove_ai_watermarks/_internal/watermark_remover.py).
|
||||
|
||||
The Python engine and CLI do not have identical defaults for every optional
|
||||
postprocessing argument. Integrations that require reproducibility should pass
|
||||
the relevant values explicitly.
|
||||
`get_device` in that module answers only `cuda` or `cpu`. An `mps` or `xpu` answer
|
||||
would travel one frame to the same CUDA-only refusal while costing a device probe,
|
||||
and reporting it implied an Apple-silicon or Intel-GPU path that does not exist.
|
||||
The refusal names the *resolved* device, so `device=None` on a CUDA-less host says
|
||||
`'cpu'` rather than `'None'`.
|
||||
|
||||
The standard Qwen and ControlNet prompts are calibrated model inputs, and the
|
||||
ControlNet edge map uses fixed Canny thresholds of 100 and 200. Treat those
|
||||
values as behavioral compatibility contracts: a refactor must preserve them,
|
||||
and any deliberate change requires image-quality evaluation rather than only a
|
||||
unit-test pass. Exact prompt and edge-map regression guards live in
|
||||
`test_platform.py` and `test_invisible_engine.py`.
|
||||
The Python engine and the CLI now resolve the same defaults: the CLI forwards an
|
||||
unset `--adaptive-polish` and `--seed` as `None` and the engine applies the
|
||||
profile's answer, so a library caller and a CLI caller on one profile produce the
|
||||
same pixels. They diverged before, in opposite directions, for exactly this knob.
|
||||
|
||||
The global and face prompts are calibrated model inputs, and the Canny edge map
|
||||
uses fixed thresholds of `_CANNY_LOW = 13` / `_CANNY_HIGH = 64`
|
||||
(`qwen_zimage_pipeline.py`). Treat those values as behavioral compatibility
|
||||
contracts: a refactor must preserve them, and any deliberate change requires
|
||||
image-quality evaluation rather than only a unit-test pass. The prompt and
|
||||
edge-map regression guards are
|
||||
`test_qwen_zimage_pipeline.py::test_global_kwargs_use_lightning_and_diffsynth_controlnet_shape`,
|
||||
`::test_face_kwargs_use_project_zimage_settings` and
|
||||
`::test_canny_control_image_is_three_channel_and_detects_an_edge`.
|
||||
|
||||
Regression coverage:
|
||||
|
||||
- [`test_watermark_profiles.py`](../tests/test_watermark_profiles.py)
|
||||
- [`test_invisible_engine.py`](../tests/test_invisible_engine.py)
|
||||
- [`test_img2img_runner.py`](../tests/test_img2img_runner.py)
|
||||
- [`test_qwen_zimage_pipeline.py`](../tests/test_qwen_zimage_pipeline.py)
|
||||
- [`test_platform.py`](../tests/test_platform.py)
|
||||
|
||||
### CPU offload
|
||||
|
||||
CPU offload is enabled only when requested on CUDA. The standard Diffusers
|
||||
profiles call `enable_model_cpu_offload`. The `qwen-zimage` profile uses the
|
||||
same flag to force **both** its stacks out of automatic device residency.
|
||||
CPU offload is enabled only when requested. Nothing calls Diffusers'
|
||||
`enable_model_cpu_offload` any more -- that belonged to the deleted single-stage
|
||||
profiles. `--cpu-offload` now forces **both** stacks of the two-stage profiles out
|
||||
of automatic device residency.
|
||||
|
||||
Residency is otherwise chosen from the card's total VRAM, once per stack:
|
||||
`resolve_global_model_residency` gates the mandatory Qwen stack at
|
||||
@@ -589,6 +969,63 @@ orchestration, YuNet integration, SAM selection, masks, sizing helpers, and pixe
|
||||
compositing are implemented for this runtime. Changing a calibrated model input
|
||||
requires the same provider-oracle and identity evaluation as a model change.
|
||||
|
||||
#### Verified text restoration
|
||||
|
||||
[`_internal/text_restoration.py`](../src/remove_ai_watermarks/_internal/text_restoration.py)
|
||||
implements the opt-in `vae-glyphs` stage. A versioned manifest carries manually
|
||||
reviewed strings and source-space line boxes in schema 1, or verified source-space
|
||||
geometry alone in schema 2, plus a SHA-256 over decoded RGB width, height, and pixels.
|
||||
Validation happens before model loading. The library never treats OCR confidence as
|
||||
verification, and geometry-only operators do not need to invent text or script fields.
|
||||
|
||||
When enabled, `QwenZImagePipeline` reconstructs the source once through its already
|
||||
loaded Qwen VAE, runs the ordinary global and face stages, blends 15% of the VAE
|
||||
reconstruction into that clean result, and calls the shared restoration compositor.
|
||||
The compositor derives binary source and candidate silhouettes, groups nearby lines,
|
||||
uses LaMa for the initial and residual-glyph erase passes, paints fresh silhouette
|
||||
edges, then copies the Qwen-VAE core with a 0.5-pixel feather. The evaluation script
|
||||
imports these same mask and compositing helpers so the two implementations cannot
|
||||
silently drift. Silhouette crops start 12% of line height beyond each horizontal
|
||||
side, then expand each side independently while a foreground component anchored
|
||||
inside the detector box still reaches that boundary, up to one line height. They
|
||||
extend 8% above and 25% below. The anchored-component gate covers clipped leading
|
||||
flourishes, trailing punctuation, icons, and descenders without walking into
|
||||
disconnected decoration or background texture.
|
||||
|
||||
The stage is deliberately narrower than the engine: it rejects `sdxl-zimage`, tiles,
|
||||
resolution caps, humanize, unsharp, and adaptive polish. Those combinations change
|
||||
geometry or final pixels after the verified layer and have no measured oracle result.
|
||||
It remains opt-in because annotations are manual and provider verdicts apply only to
|
||||
the exact tested output hashes, not to the mechanism in general.
|
||||
|
||||
A matched stage-isolation check on the 18-face Gemini portrait grid confirms the
|
||||
division of responsibility. The visible-cleaned, metadata-stripped control and the
|
||||
Z-Image face-only output were both SynthID-positive; Qwen global-only and the full
|
||||
Qwen-then-Z-Image output were both clean. The face stage raised identity cosine from
|
||||
0.589 to 0.852 and reduced face LPIPS from 0.217 to 0.050 without reintroducing a
|
||||
detectable whole-image signal. Thus Z-Image is a masked fidelity repair stage here,
|
||||
not the watermark-removal stage. Exact hashes, metrics, strengths, and the one-fixture,
|
||||
one-seed caveat are recorded in
|
||||
[`data/evaluations/fidelity/face-stage-isolation-2026-08-13.csv`](../data/evaluations/fidelity/face-stage-isolation-2026-08-13.csv).
|
||||
|
||||
The public Synthid-Bypass v2 graph was subsequently audited at upstream commit
|
||||
`3007d035`. Its saved-output path confirms the same division: Qwen-Image-2512
|
||||
Lightning plus Canny is global, and Z-Image Turbo exists only inside the masked
|
||||
face detailer. The connected face path is YOLOv8-face plus SAM; the MediaPipe nodes
|
||||
described by the upstream README and the 1.2-megapixel normalization node do not
|
||||
reach `SaveImage`. Upstream also applies its adaptive face strength directly,
|
||||
whereas this implementation multiplies it by `FACE_DENOISE_SCALE = 0.5`.
|
||||
|
||||
A close reproduction on the same portrait fixture kept the control positive and
|
||||
made both Qwen global-only and full Qwen-then-Z-Image outputs clean. Applying the
|
||||
upstream-strength face pass raised identity from 0.589 to 0.783 and reduced face
|
||||
LPIPS from 0.217 to 0.083, but remained worse than this profile's 0.852 and 0.050.
|
||||
The published upstream pair 12 was also independently checked positive before and
|
||||
clean after, with 0.975 identity. Exact workflow provenance, hashes, metrics,
|
||||
oracle outcomes, and the DiffSynth/GGUF, scheduler, detector, and seed caveats are
|
||||
recorded in
|
||||
[`data/evaluations/fidelity/upstream-v2-reproduction-2026-08-13.csv`](../data/evaluations/fidelity/upstream-v2-reproduction-2026-08-13.csv).
|
||||
|
||||
### SDXL plus Z-Image
|
||||
|
||||
[`_internal/sdxl_zimage_pipeline.py`](../src/remove_ai_watermarks/_internal/sdxl_zimage_pipeline.py)
|
||||
@@ -617,6 +1054,38 @@ sigma range; Diffusers img2img truncates the step *count*
|
||||
(`init_timestep = int(steps * strength)`), so asking it for four steps at 0.15 executes
|
||||
**zero** and returns a bare VAE round-trip.
|
||||
|
||||
**The face stage keeps its own dtype, and 0.23.0 shipped without that.** The remover
|
||||
gives this profile `torch.float16`, because SDXL ships fp16 weights and an fp16-safe
|
||||
VAE. That dtype reached the inherited `_load_zimage`, while `_zimage_vram_config()`
|
||||
hardcodes bfloat16 for its offload, onload and computation dtypes -- so the Z-Image
|
||||
modules were built bf16 and handed fp16 latents, and every image containing a face
|
||||
died in the VAE with `Input type (c10::Half) and bias type (c10::BFloat16) should be
|
||||
the same`. Every face-stage loader now reads `_face_stage_dtype()`, which returns the
|
||||
computation dtype of the VRAM config it is paired with, so the two cannot drift again.
|
||||
|
||||
Two things hid this. Zero-face inputs never enter `_run_faces`, so the profile looked
|
||||
healthy on exactly the images used to time it; and the profile's tests deliberately
|
||||
avoid model downloads, so nothing exercised the loader. The lesson is narrower than
|
||||
"add a GPU test": inheriting a stage means inheriting its *invariants*, and this one
|
||||
was a dtype the subclass silently changed out from under it.
|
||||
|
||||
Note what the seam is, because it decides where the fix belongs.
|
||||
`SdxlZImagePipeline._load_sdxl` hardcodes fp16 for its own ControlNet, VAE and
|
||||
pipeline, so `self.torch_dtype` was never actually the global stage's dtype on this
|
||||
profile -- its only remaining readers were face-stage code. SAM was the second one:
|
||||
it never crashed, because it casts its own inputs and leaves through `.float()`, but it
|
||||
was reading the same wrong field and would have re-landed the bug for the next profile
|
||||
with a different global dtype. It is routed through the same accessor, which for
|
||||
`qwen-zimage` is the bfloat16 it already used.
|
||||
|
||||
The guard is `test_face_stage_loads_in_its_own_dtype_when_the_global_stage_differs`.
|
||||
It asserts the dtype the Z-Image and SAM loaders actually receive, not the accessor
|
||||
against the config it is derived from -- that comparison would restate the
|
||||
implementation and pass for any consistently wrong value. Both assertions were
|
||||
mutation-tested against the pre-fix line. For `qwen-zimage` the whole change is a
|
||||
strict no-op: the remover already handed it bfloat16, the same value
|
||||
`_face_stage_dtype()` returns.
|
||||
|
||||
This profile is not deployed. Before it could be, it needs the other three Gemini
|
||||
originals, OpenAI re-verified at 0.15, a flat-graphic content class, and a low
|
||||
resolution case -- every verdict so far comes from one fixture and one seed.
|
||||
@@ -634,7 +1103,7 @@ everything it has been tested at:
|
||||
| Gemini app | 1.40 MP | -- | 0.1066 (the curve's own value) |
|
||||
|
||||
Read the last two rows before concluding the curve's low end is under-driven. Against
|
||||
the 4.33 MP Gemini boundary the sub-1 MP rungs of 0.084-0.094 look short, but at those
|
||||
the 4.33 MP Gemini boundary the sub-1 MP rungs of 0.084-0.098 look short, but at those
|
||||
sizes the curve's own values verify clean, which is what a resolution-scaled
|
||||
requirement would predict. There is no measured size at which the shipped curve fails,
|
||||
so it is left alone.
|
||||
@@ -680,31 +1149,35 @@ Regression coverage:
|
||||
### Tiling
|
||||
|
||||
[`_internal/tiling.py`](../src/remove_ai_watermarks/_internal/tiling.py) contains pure
|
||||
tile planning, feather weights, tile orchestration, and region compositing.
|
||||
tile planning, feather weights, and tile orchestration.
|
||||
|
||||
Tiling engages only when requested and the long side exceeds the tile size.
|
||||
It avoids an explicit full-image downscale but does not make diffusion
|
||||
pixel-preserving. Each tile is still regenerated.
|
||||
|
||||
`feather_region_composite` changes only the requested box and leaves pixels
|
||||
outside it unchanged.
|
||||
It also held a `feather_region_composite` for AI-*enhanced* composites, where only
|
||||
the edited region should change. Nothing ever reached it: the `erase` command
|
||||
inpaints through `region_eraser`, and the remover's `region` argument was only
|
||||
reachable from a module-level convenience wrapper with no callers. Both went.
|
||||
|
||||
Regression coverage:
|
||||
|
||||
- [`test_tiling.py`](../tests/test_tiling.py)
|
||||
|
||||
### Upscaling and postprocessing
|
||||
|
||||
[`upscaler.py`](../src/remove_ai_watermarks/upscaler.py) is the optional
|
||||
Real-ESRGAN path used only when enlarging a small image to the minimum
|
||||
resolution floor. Failure or an absent extra falls back to Lanczos.
|
||||
### Postprocessing
|
||||
|
||||
[`humanizer.py`](../src/remove_ai_watermarks/humanizer.py) contains explicit
|
||||
grain, unsharp masking, and adaptive polish helpers.
|
||||
|
||||
`upscaler.py` held an optional Real-ESRGAN path, reachable only when enlarging a
|
||||
small image to the minimum-resolution floor. That floor existed to lift small
|
||||
inputs toward SDXL's ~1024 training size; when the SDXL profiles were removed it
|
||||
was forced to 0 on every path, so the module, the `--min-resolution` and
|
||||
`--upscaler` options and the `esrgan` extra were all unreachable and went with
|
||||
it. Only the `max_resolution` cap can move geometry now, and it only scales down.
|
||||
|
||||
Regression coverage:
|
||||
|
||||
- [`test_upscaler.py`](../tests/test_upscaler.py)
|
||||
- [`test_humanizer.py`](../tests/test_humanizer.py)
|
||||
|
||||
## Image input and output
|
||||
|
||||
+255
-24
@@ -6,8 +6,9 @@ and pipeline modules are intended for maintainers and specialized workflows.
|
||||
Dependency groups are identical for the CLI and Python API. The default install
|
||||
covers metadata extraction, normalization, verdict logic, and stripping.
|
||||
Array/pixel APIs use `pixels`; visible removal uses `visible`; DWT-DCT detection
|
||||
uses `detect`; diffusion removal uses `diffusion`; and visible video processing
|
||||
uses `video`. Video SynthID removal combines `video` and `diffusion`. Add `heif`
|
||||
uses `detect`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and
|
||||
visible video processing uses `video`. Video SynthID removal is a separate VAE
|
||||
path that still runs on CPU and combines `video` and `diffusion`. Add `heif`
|
||||
independently when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See
|
||||
the complete [feature-extra matrix](installation.md#feature-extras).
|
||||
|
||||
@@ -76,6 +77,61 @@ image = cv2.imread("input.png")
|
||||
result, removed = raiw.remove_visible(image, backend="cv2")
|
||||
```
|
||||
|
||||
## Run the full pipeline
|
||||
|
||||
`remove_all` is the library form of the `all` command: visible marks, then the
|
||||
invisible watermark, then AI metadata. Stages are chained through a file in the
|
||||
system temp directory, so a partial result never appears at the output path.
|
||||
|
||||
```python
|
||||
import remove_ai_watermarks as raiw
|
||||
|
||||
result = raiw.remove_all("input.png", "clean.png") # -> RemoveAllResult
|
||||
print(result.output) # the path written
|
||||
print(result.visible_label) # the marks removed, or None
|
||||
print(result.invisible) # "removed" | "no-signal" | "unavailable"
|
||||
```
|
||||
|
||||
`invisible` is the field to check. `"unavailable"` means the GPU extra is not
|
||||
installed, so the output *looks* processed but still carries the watermark;
|
||||
`"no-signal"` means the scrub was deliberately skipped because nothing was
|
||||
locally detectable, which is a successful run.
|
||||
|
||||
Pass `InvisibleOptions` to tune the diffusion stage, and `engine` to reuse one
|
||||
loaded model across many calls:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks import InvisibleOptions
|
||||
|
||||
raiw.remove_all(
|
||||
"input.png",
|
||||
"clean.png",
|
||||
invisible=InvisibleOptions(strength=0.35),
|
||||
force=True,
|
||||
progress=print,
|
||||
)
|
||||
```
|
||||
|
||||
`InvisibleOptions` carries only what `InvisibleEngine` itself takes, and uses the
|
||||
engine's own parameter names and defaults. `force`, which decides whether the
|
||||
engine runs at all, is a parameter of `remove_all` and `remove_batch` alongside
|
||||
`backend` and `sensitivity`.
|
||||
|
||||
If AI metadata survives the strip, `remove_all` raises `MetadataStripIncomplete`
|
||||
**before** writing anything: an AI-readable output is worse than no output.
|
||||
|
||||
`remove_batch` runs one mode over a directory and never lets a single bad file
|
||||
end the run:
|
||||
|
||||
```python
|
||||
summary = raiw.remove_batch("in_dir", "out_dir", mode="visible") # -> BatchSummary
|
||||
print(summary.processed, summary.failed, summary.errors)
|
||||
print(summary.invisible_unavailable) # outputs that still carry the watermark
|
||||
```
|
||||
|
||||
`mode` is `all`, `visible`, `invisible`, or `metadata`. Pass a constructed
|
||||
`InvisibleEngine` as `engine` to load the model once for the whole directory.
|
||||
|
||||
## Inspect provenance
|
||||
|
||||
The default installation evaluates file metadata. Add `visible`, `detect`, or
|
||||
@@ -99,8 +155,17 @@ from remove_ai_watermarks.identify import identify
|
||||
report = identify(Path("input.png"))
|
||||
print(report.platform)
|
||||
print(report.signals)
|
||||
print(report.c2pa_validation)
|
||||
```
|
||||
|
||||
`c2pa_validation`, when present, reports `integrity`, `signature`,
|
||||
`signer_trust`, and `signer_validity` independently, plus the reader status
|
||||
codes. A valid hash and signature with an untrusted or expired signer is a
|
||||
medium-confidence signed claim. A hash or signature failure does not confirm the
|
||||
claimed platform or AI origin. Fallback parsing reports unknown validation
|
||||
dimensions, while a raw marker in an unsupported or malformed container can
|
||||
leave `c2pa_validation` as `None`.
|
||||
|
||||
Use `check_visible=False` and `check_invisible=False` for metadata-only
|
||||
inspection through the compatible path-based API:
|
||||
|
||||
@@ -126,8 +191,61 @@ evidence = extract_provenance_evidence(Path("input.png"))
|
||||
report = identify_from_evidence(evidence)
|
||||
```
|
||||
|
||||
If metadata was collected by another component, normalize its nested record
|
||||
without reopening the original file:
|
||||
### Collect once, judge elsewhere
|
||||
|
||||
`collect_metadata_record` splits the two halves apart: it is the only step that
|
||||
touches the file, and it returns a JSON-serializable record the verdict can be
|
||||
built from on another machine, in another process, or later.
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
from remove_ai_watermarks.identify import identify_metadata_record
|
||||
from remove_ai_watermarks.metadata_record import collect_metadata_record
|
||||
|
||||
record = collect_metadata_record(Path("input.png"), schema_version=1) # reads the file
|
||||
blob = json.dumps(record) # ship it anywhere
|
||||
|
||||
report = identify_metadata_record(json.loads(blob), path=Path("input.png")) # reads nothing
|
||||
payload = report.to_dict(schema_version=1) # versioned JSON contract
|
||||
```
|
||||
|
||||
The collection record has `record_type="provenance_metadata"`,
|
||||
`schema_version=1`, and a `status`. A vanished or unreadable source produces an
|
||||
`error` record with structured `issues`; `identify_metadata_record` rejects that
|
||||
record instead of turning a collection failure into an unknown-image verdict.
|
||||
Unknown schema versions, non-integer aliases, and native records without a
|
||||
`complete` collection status are rejected explicitly.
|
||||
|
||||
The verdict is the same one `identify(path, check_visible=False,
|
||||
check_invisible=False)` returns for that file. That equality is the record's whole
|
||||
contract and is verified over the tracked provenance fixtures and a separate local
|
||||
evaluation corpus. `ProvenanceReport.to_dict()` is the stable service boundary: it
|
||||
adds a `schema_version`, contains only JSON-safe values, and deliberately omits the
|
||||
local source path.
|
||||
|
||||
Package and transport versions evolve independently. Long-lived consumers should
|
||||
request the schema they implement, as above, instead of assuming the installed
|
||||
package's latest schema. Within schema 1, existing fields, types, meanings,
|
||||
`signals[].name` values, and `watermarks[]` labels remain compatible; releases may
|
||||
add fields that consumers must ignore. A breaking change requires a new schema while
|
||||
the schema 1 serializer remains available for rolling upgrades. Asking a release for
|
||||
an unsupported schema raises `ValueError` rather than silently returning another
|
||||
shape.
|
||||
|
||||
A record carries metadata regions, not the primary coded-pixel stream: marker
|
||||
segments before the JPEG scan, every PNG chunk except `IDAT`, RIFF chunks except the
|
||||
coded image, the ISOBMFF provenance boxes, the container's trailer, the parsed EXIF tags the
|
||||
verdict reads by name, PIL's info mapping, and the C2PA manifest store. Record size
|
||||
is bounded by those metadata regions and trailers; images with large embedded
|
||||
manifests naturally produce larger records.
|
||||
|
||||
The `path` argument is metadata: it labels the report and is never opened by
|
||||
either function, so a record collected elsewhere can be judged against a path that
|
||||
does not exist locally.
|
||||
|
||||
If metadata was collected by another component instead, normalize its nested
|
||||
record the same way:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.identify import (
|
||||
@@ -143,16 +261,70 @@ evidence = evidence_from_metadata_record(record, path=Path("input.png"))
|
||||
report = identify_from_evidence(evidence)
|
||||
```
|
||||
|
||||
The normalizer recursively preserves text and byte values. It also decodes
|
||||
Unversioned third-party records are normalized recursively for compatibility. The
|
||||
normalizer preserves text and byte values. It also decodes
|
||||
strings prefixed with `hex:` and fields named `base64` or ending in
|
||||
`_base64`. Diagnostic values under `error` and `kind` are ignored because they
|
||||
describe the collector rather than the source file. Pass a C2PA manifest-store
|
||||
dictionary in `record["c2pa_store"]`, or through the explicit
|
||||
`_base64`. Diagnostic, transport, timing, hash, provenance-result, and pixel-result
|
||||
subtrees are ignored because they describe the collector or a derived result rather
|
||||
than the source file. A versioned portable record is stricter still: only
|
||||
`metadata_base64`, `tail_base64`, `pil`, `exif`, and `c2pa_store` are accepted as
|
||||
source evidence. Other `record_type` values are rejected, so do not pass a broad
|
||||
forensic inspection record to this API. Pass a C2PA manifest-store dictionary in
|
||||
`record["c2pa_store"]`, or through the explicit
|
||||
`c2pa_manifest_store` argument.
|
||||
|
||||
`identify_from_evidence` does not reopen the source file. It evaluates metadata
|
||||
only; registered visible marks and pixel-backed invisible watermarks remain in
|
||||
the path-based `identify` call.
|
||||
### Broad metadata inspection
|
||||
|
||||
`collect_forensic_metadata` provides the wide metadata-only record used by forensic
|
||||
inspection and migration adapters. It preserves hashes and timestamps, full EXIF and
|
||||
IPTC, C2PA, container inventories, bounded raw metadata payloads, and embedded
|
||||
thumbnail forensics. It does not calculate a provenance verdict or pixel statistics.
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.forensic_metadata import collect_forensic_metadata
|
||||
|
||||
record = collect_forensic_metadata(Path("input.png"), schema_version=1)
|
||||
assert record["record_type"] == "forensic_metadata"
|
||||
```
|
||||
|
||||
This record is intentionally not accepted by `identify_metadata_record`. Collect the
|
||||
small strict provenance record separately and publish the resulting
|
||||
`ProvenanceReport.to_dict()` as the detector contract.
|
||||
|
||||
### Pixel evidence
|
||||
|
||||
`extract_pixel_evidence` decodes once and calculates the DCT, FFT, residual, ELA,
|
||||
gradient, and color families. Its versioned `to_dict()` result has a semantic
|
||||
`status`: `complete`, `partial` when an individual family failed, or `error` when the
|
||||
source could not be decoded. Transported errors contain only the exception class, so
|
||||
local paths stay in the caller's logs rather than crossing the service boundary.
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.pixel_evidence import extract_pixel_evidence
|
||||
|
||||
pixels = extract_pixel_evidence(Path("input.png"), artifacts=False, timings=True)
|
||||
payload = pixels.to_dict(schema_version=1)
|
||||
```
|
||||
|
||||
Timings and spatial artifacts are opt-in. Artifacts include image-identifying data
|
||||
such as a thumbnail and perceptual hash; aggregate feature families do not.
|
||||
|
||||
`identify_from_evidence` does not reopen the source file by default: it evaluates
|
||||
metadata only, and registered visible marks and pixel-backed invisible watermarks
|
||||
remain in the path-based `identify` call.
|
||||
|
||||
Pass `image_path` together with `check_visible` or `check_invisible` to add those
|
||||
pixel detectors on top of the SAME evidence. That is how a caller asking one file
|
||||
two provenance questions — which vendor is confirmed, and is there an invisible
|
||||
target — pays for the metadata extraction once:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.identify import extract_provenance_evidence, identify_from_evidence
|
||||
|
||||
evidence = extract_provenance_evidence(source)
|
||||
metadata_only = identify_from_evidence(evidence)
|
||||
with_pixels = identify_from_evidence(evidence, image_path=source, check_invisible=True)
|
||||
```
|
||||
|
||||
## Strip metadata
|
||||
|
||||
@@ -259,7 +431,8 @@ must use the same container extension as the source.
|
||||
The returned `VideoMetadataResult` records the source, output, metadata detected
|
||||
before removal, and any markers remaining after the verified strip. MP4/MOV
|
||||
inspection recognizes the native TC260 `AIGC` entry in
|
||||
`moov.udta.meta.keys/ilst`; its removal preserves container size and encoded
|
||||
`moov.udta.meta.keys/ilst` and the QuickTime-form `meta` variants Doubao's iOS
|
||||
export writes; its removal preserves container size and encoded
|
||||
stream bytes. MP4/MOV/M4V are copied in bounded chunks, so a large `mdat` is not
|
||||
loaded into memory; publication is atomic. MKV/WebM inspection recognizes the corresponding
|
||||
`Segment.Tags.Tag.SimpleTag` representation; its removal requires ffmpeg for a
|
||||
@@ -380,8 +553,8 @@ to 8-bit SDR.
|
||||
|
||||
## Remove invisible watermarks
|
||||
|
||||
Install `remove-ai-watermarks[diffusion]` for the standard pipelines or
|
||||
`remove-ai-watermarks[qwen-zimage]` for the CUDA-only high-fidelity profile.
|
||||
Install `remove-ai-watermarks[qwen-zimage]`. Both profiles need it, and both
|
||||
need an NVIDIA GPU.
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
@@ -389,7 +562,7 @@ from pathlib import Path
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine
|
||||
|
||||
engine = InvisibleEngine(
|
||||
pipeline="controlnet",
|
||||
pipeline="qwen-zimage", # the default; the only other value is "sdxl-zimage"
|
||||
device=None,
|
||||
cpu_offload=False,
|
||||
)
|
||||
@@ -400,29 +573,87 @@ engine.remove_watermark(
|
||||
)
|
||||
```
|
||||
|
||||
`device=None` selects the device automatically. Supported explicit values are
|
||||
defined by the CLI and runtime device resolver.
|
||||
`device=None` and `device="auto"` both run detection. `"cuda"` pins it without
|
||||
detecting. Every other value raises at construction rather than deferring a
|
||||
guaranteed failure to model-load time.
|
||||
|
||||
For limited CUDA memory:
|
||||
|
||||
```python
|
||||
engine = InvisibleEngine(
|
||||
pipeline="controlnet",
|
||||
pipeline="qwen-zimage",
|
||||
cpu_offload=True,
|
||||
)
|
||||
```
|
||||
|
||||
For the CUDA only high fidelity profile:
|
||||
Both profiles are CUDA-only, so on a machine without an NVIDIA GPU `device=None`
|
||||
resolves to `cpu` and construction raises. For the SDXL global stage instead of
|
||||
Qwen:
|
||||
|
||||
```python
|
||||
engine = InvisibleEngine(pipeline="qwen-zimage")
|
||||
engine = InvisibleEngine(pipeline="sdxl-zimage")
|
||||
```
|
||||
|
||||
The `qwen-zimage` extra must be installed for that profile.
|
||||
The `qwen-zimage` extra is required for both profiles: each runs the same
|
||||
DiffSynth Z-Image face stage.
|
||||
|
||||
The full `remove_watermark` signature includes strength, steps, guidance,
|
||||
seeding, tiling, resolution, upscaling, and postprocessing controls. Read the
|
||||
method signature in
|
||||
The opt-in verified-text stage uses the same `text_manifest` argument as the CLI:
|
||||
|
||||
```python
|
||||
engine.remove_watermark(
|
||||
Path("watermarked.png"),
|
||||
Path("clean.png"),
|
||||
text_manifest=Path("verified-lines.json"),
|
||||
)
|
||||
```
|
||||
|
||||
Install `remove-ai-watermarks[text-restoration]`. The manifest schema and safety
|
||||
constraints are documented in the CLI guide. The engine verifies its decoded RGB
|
||||
hash before loading the diffusion models and rejects SDXL, downscaling, and
|
||||
postprocessing combinations that were not evaluated. Tiling is allowed: the VAE
|
||||
donor uses the same overlapping tiles as the global pass. `InvisibleOptions` exposes the
|
||||
same field for `remove_all`; after a visible-stage edit, the manifest must be built
|
||||
against the staged pixels rather than the pristine source.
|
||||
|
||||
Use manifest schema 1 for manually reviewed text plus script metadata. Automated
|
||||
operators that verify only text-region geometry should emit schema 2 lines with a
|
||||
`box` and optional `angle`; no placeholder transcription or script is required.
|
||||
|
||||
Since 0.27.1 the mode's global 15% Qwen-VAE fidelity-anchor blend is **off by
|
||||
default** (`fidelity_anchor=False`): that whole-frame blend was measured to
|
||||
return detector-visible OpenAI SynthID on poster-scale manifests (official
|
||||
Content Provenance API, 2026-08-19 - detected x6 with the anchor, clean x6
|
||||
without it, controls and base outputs validated in the same sessions). Pass
|
||||
`fidelity_anchor=True` to reproduce the 0.27.0 research behavior.
|
||||
|
||||
### Drafting manifest lines
|
||||
|
||||
`remove_ai_watermarks.text_draft` proposes lines for a manifest; it never
|
||||
produces verified ones:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.text_draft import draft_text_lines
|
||||
|
||||
draft = draft_text_lines(Path("watermarked.png"))
|
||||
for line in draft.accepted:
|
||||
print(line.box, line.script, line.min_score, line.text)
|
||||
```
|
||||
|
||||
Install `remove-ai-watermarks[text-draft]` (CPU, no torch: PaddleOCR detection
|
||||
plus three script-chosen recognition engines). A line lands in `accepted` only
|
||||
when three crop paddings normalize identically and every confidence clears
|
||||
`min_score` (default 0.85); `accepted` means crop-stable, NOT ground-truth-
|
||||
correct - on the reference posters the draft's exact-text precision was 90.0%
|
||||
and 94.4% because high-confidence OCR still lost punctuation. Every accepted
|
||||
line needs a human yes/no before a manifest may claim `verified: true`.
|
||||
`source_pixel_sha256` is re-exported here for building the manifest's
|
||||
pixel-binding hash against the exact source the engine will decode.
|
||||
|
||||
`remove_watermark` takes strength, seed, tiling, resolution, and postprocessing
|
||||
controls. It takes no model id, step count or guidance scale, and neither does the
|
||||
constructor: each profile pins its model stack, its per-stage schedule and CFG
|
||||
1.0, so passing one raises `TypeError` at the call rather than being accepted and
|
||||
refused several layers down. Read the method signature in
|
||||
[`invisible_engine.py`](../src/remove_ai_watermarks/invisible_engine.py) or use
|
||||
the CLI guide for the concepts.
|
||||
Defaults can differ between the Python method and CLI profile resolution, so
|
||||
|
||||
@@ -3,6 +3,21 @@
|
||||
This page describes the release behavior defined in this repository. External
|
||||
registry state can change independently, so verify it during a release.
|
||||
|
||||
## Published surfaces
|
||||
|
||||
A release is complete only after all four published surfaces are verified:
|
||||
|
||||
| Surface | Published result | Automation |
|
||||
|---|---|---|
|
||||
| PyPI | The `remove-ai-watermarks` wheel and source distribution | `publish.yml` |
|
||||
| Homebrew | The `remove-ai-watermarks` formula in `wiltodelta/homebrew-tap` | `distribute.yml` |
|
||||
| Hugging Face Space | The Space demo deployed on the new release through a requirements-pin bump in `wiltodelta/raiw-hf-space` | `raiw-hf-space` sync workflow; the pin bump is manual, `distribute.yml` only re-installs the pinned version |
|
||||
| ComfyUI Registry | A compatible release of `wiltodelta/ComfyUI-remove-ai-watermarks` with its own node version | `distribute.yml` and the node repository's workflows |
|
||||
|
||||
The GitHub Release is the trigger and release record for this flow. Conda is
|
||||
not a supported publishing surface: this repository has no conda recipe or
|
||||
conda publication job.
|
||||
|
||||
## Release sources of truth
|
||||
|
||||
The package version appears in:
|
||||
@@ -43,25 +58,23 @@ PyPI API token from the repository.
|
||||
waits for the matching source distribution to appear on PyPI, then:
|
||||
|
||||
- updates the Homebrew tap formula URL and SHA-256;
|
||||
- updates the repository conda recipe version and source-distribution SHA-256;
|
||||
- triggers a factory rebuild of the Hugging Face Space.
|
||||
- triggers a factory rebuild of the Hugging Face Space: this re-installs the
|
||||
version pinned in the Space repo, it does NOT upgrade the demo;
|
||||
- synchronizes, tests, versions, and publishes the ComfyUI nodes.
|
||||
|
||||
Upgrading the Hugging Face Space is a separate, manual step: bump
|
||||
`remove-ai-watermarks[visible,heif]` in `wiltodelta/raiw-hf-space` (`pyproject.toml`,
|
||||
`uv lock`, and the re-exported `requirements.txt`), then push. That repository's
|
||||
`sync-to-hf.yml` mirrors the files onto the Space, which rebuilds on the new
|
||||
pin. Its callback smoke tests run in that repository's CI on every push; they
|
||||
are the scenario check for the demo surface.
|
||||
|
||||
The workflow can also be started manually with an optional version input.
|
||||
Conda-forge updates are outside this workflow.
|
||||
|
||||
If a distribution job fails because a repository or Hugging Face credential is
|
||||
invalid, rotate the corresponding GitHub secret and rerun the failed job. A
|
||||
manual Homebrew formula update is the fallback when its automation is blocked.
|
||||
|
||||
The conda job uses the published artifact rather than a locally built archive
|
||||
as the hash source and commits the resulting recipe change to `main`. Runtime
|
||||
dependency mapping remains review-controlled: keep it aligned with the default
|
||||
metadata dependencies in `pyproject.toml`, do not copy optional pixel extras
|
||||
into the default recipe, and document any conda-forge package that is
|
||||
unavailable and must be omitted.
|
||||
The optional `video` extra carries PyAV with Python-version-specific bounds; it
|
||||
does not belong in the default metadata-focused conda recipe.
|
||||
|
||||
## Source distribution boundary
|
||||
|
||||
The wheel includes the package under `src/`.
|
||||
@@ -71,9 +84,10 @@ The source distribution uses an explicit allowlist for `/src`, `/LICENSE`,
|
||||
`[tool.hatch.build.targets.sdist]` in `pyproject.toml`. It also defensively
|
||||
excludes `/data`, `/tmp`, and `/.sc`. Keep both controls: calibration captures,
|
||||
test corpora, generated research outputs, and local session state do not belong
|
||||
in the published package archive. The matching local-root entries in
|
||||
`.gitignore` prevent accidental commits, but are not a substitute for the build
|
||||
boundary because hatchling may include untracked files.
|
||||
in the published package archive. Hatchling always adds the root `.gitignore` to
|
||||
the sdist, so keep its comments generic and free of local operational context.
|
||||
Ignore rules are not the build boundary: `data/` is deliberately tracked, while
|
||||
the sdist configuration keeps it and the other excluded paths out of the archive.
|
||||
|
||||
## Build backend
|
||||
|
||||
@@ -82,8 +96,6 @@ The package uses hatchling through the unpinned `hatchling` build requirement in
|
||||
|
||||
## Other channels
|
||||
|
||||
The repository includes a conda recipe under `packaging/conda/recipe.yaml`.
|
||||
|
||||
The ComfyUI nodes are maintained and versioned in their own repository. After
|
||||
the matching source distribution appears on PyPI, `distribute.yml` dispatches
|
||||
that repository's sync workflow with the exact library version and waits for it
|
||||
@@ -96,13 +108,30 @@ write access.
|
||||
|
||||
## Release verification
|
||||
|
||||
Forensic transports are versioned independently from the package. Before publishing
|
||||
a change to provenance metadata, provenance reports, broad forensic metadata, or
|
||||
pixel evidence, run their schema 1 contract tests. Additive fields are compatible;
|
||||
renaming a field, changing its type or meaning, changing a signal name or watermark
|
||||
label, or removing a field requires a new output schema. Add the new serializer
|
||||
without removing schema 1 so long-lived consumers can update separately. A package
|
||||
release must never silently substitute its latest schema when a caller explicitly
|
||||
requests an older supported one.
|
||||
|
||||
After publication, verify:
|
||||
|
||||
- both wheel and source distribution exist on PyPI;
|
||||
- the package version matches the tag;
|
||||
- the Homebrew formula points to the new source distribution;
|
||||
- the distribution workflow completed successfully;
|
||||
- the repository's conda recipe matches the published version and source
|
||||
distribution;
|
||||
- the ComfyUI Registry node requires the new library version;
|
||||
- the Hugging Face Space serves the new release: its `requirements.txt` pin
|
||||
matches (raw file via the HF API), the runtime stage is `RUNNING` after the
|
||||
rebuild, and one live `identify` and one live `visible` API call succeed
|
||||
against the running Space;
|
||||
- a clean install can run `remove-ai-watermarks --version`.
|
||||
|
||||
A clean-install check run immediately after publication can fail with "no
|
||||
version of remove-ai-watermarks==X.Y.Z" even though the simple index already
|
||||
lists it: uv serves its cached pre-release view of the index. Re-run with a
|
||||
fresh cache (`UV_CACHE_DIR=$(mktemp -d)`) before treating it as a release
|
||||
failure.
|
||||
+40
-14
@@ -37,7 +37,7 @@ when you can select the affected area yourself.
|
||||
| `veo` | Current four-point diamond and legacy `Veo` text | Fixed bottom-right corner | Uses separate silhouettes and requires a recurring match; learned fill is preferable on structured backgrounds. |
|
||||
| `seedance` | Boxed `AI` label | Fixed bottom-right corner | Requires an anchored recurring match; the full localized box is filled because a thinner synthetic shape mask leaves the real translucent rim behind. |
|
||||
| `dola` | `Dola AI` text | Fixed bottom-right corner | Requires an anchored recurring match; ByteDance or BytePlus provenance can relax only an existing visual run. |
|
||||
| `hailuo` | `MINIMAX | hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. |
|
||||
| `hailuo` | `MINIMAX \| hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. A TC260 label naming MiniMax as producer can relax only an existing stable run. |
|
||||
| `kling` | Kling swirl, `KLING AI`, version, and optional `PRO` suffix | Fixed bottom-right edge | Combines a synthetic logo rescue with font variants, an edge gate, a white-label gate, and anchored temporal recurrence. |
|
||||
|
||||
`video identify`, `video visible`, and `video all` share this registry and the
|
||||
@@ -67,6 +67,9 @@ The inspection and stripping code handles signals in these groups:
|
||||
|
||||
- C2PA Content Credentials and supported cloud manifest references;
|
||||
- EXIF and XMP generator fields;
|
||||
- exact app-export provenance and AIGC disclosures from supported
|
||||
ByteDance-family products, with product-only provenance excluded from the
|
||||
generated-image verdict;
|
||||
- IPTC AI disclosure fields;
|
||||
- PNG text chunks and embedded generation parameters;
|
||||
- China TC260 AIGC labels in supported image placements and the normative
|
||||
@@ -77,11 +80,17 @@ The inspection and stripping code handles signals in these groups:
|
||||
- Samsung AI editing markers;
|
||||
- Hugging Face job metadata;
|
||||
- open Stable Diffusion style DWT-DCT watermarks with the `detect` extra;
|
||||
- Adobe TrustMark with the `trustmark` extra.
|
||||
- Adobe TrustMark Variant P schemas 0-2 with the `trustmark` extra. Variant Q
|
||||
needs a different model, while schema 3 is deliberately rejected because it
|
||||
produced persistent false positives on unrelated generators.
|
||||
|
||||
`identify` combines detected signals into a `ProvenanceReport`. It reports
|
||||
unknown when evidence is absent. It never treats missing metadata as proof that
|
||||
an image is human made.
|
||||
an image is human made. C2PA presence alone is not a verified identity: the
|
||||
report distinguishes asset binding, claim signature, signer trust, and signer
|
||||
validity. High-confidence C2PA attribution requires all four; intact but
|
||||
untrusted or fallback claims are medium-confidence, while a failed binding or
|
||||
signature contributes no origin verdict.
|
||||
|
||||
## File and container formats
|
||||
|
||||
@@ -114,17 +123,32 @@ The `invisible` command uses diffusion regeneration. It targets watermark
|
||||
patterns by changing the image rather than decoding and deleting a known
|
||||
payload.
|
||||
|
||||
Current pipeline values:
|
||||
Current pipeline values, both CUDA-only:
|
||||
|
||||
- `controlnet`;
|
||||
- `sdxl`;
|
||||
- `qwen`;
|
||||
- `qwen-zimage`;
|
||||
- legacy alias `default`, which resolves to `sdxl`.
|
||||
- `qwen-zimage`, the default;
|
||||
- `sdxl-zimage`, the same recipe and the same face stage on an SDXL global pass.
|
||||
|
||||
SynthID does not have a public local pixel decoder in this project. The tool can
|
||||
infer likely presence from supported provenance metadata, but after that
|
||||
metadata is removed a local negative result is inconclusive.
|
||||
The `controlnet`, `sdxl`, `qwen` and `default` values were removed. A retired name
|
||||
is rejected at parse time rather than remapped onto a surviving profile.
|
||||
|
||||
SynthID does not have a public local pixel decoder in this project. The tool
|
||||
recognizes presence from supported provenance: Google AI C2PA under Google's
|
||||
all-media watermark policy, and current OpenAI C2PA carrying an explicit
|
||||
`c2pa.watermarked.*` action. Legacy OpenAI C2PA without that action does not
|
||||
assert SynthID. After provenance metadata is removed, a local negative result
|
||||
is still inconclusive.
|
||||
|
||||
Microsoft Paint can name `com.microsoft.invismark.1` in a C2PA soft-binding
|
||||
assertion. Inspection reports both that exact algorithm and its signed `value`,
|
||||
which Paint uses as the identifier carried by the pixel watermark. Photos has a
|
||||
parallel local writer path, but no public output sample was available for this
|
||||
work. Metadata stripping removes the embedded manifest, not the pixel carrier.
|
||||
The project has no validated local InvisMark decoder. Microsoft's official
|
||||
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
|
||||
is the external oracle: it reports pixel `Watermark` and embedded `C2PA` results
|
||||
separately. The diffusion profiles have not been broadly certified against that
|
||||
oracle, so a control-positive, output-negative pair is still a per-file result,
|
||||
not a universal guarantee.
|
||||
|
||||
For MP4, MOV, and M4V, `video invisible` or the explicit
|
||||
`video all --invisible` option can regenerate the video through a VAE and strip
|
||||
@@ -146,11 +170,13 @@ not a universal clean verdict.
|
||||
| Google Gemini | Sparkle | Diffusion regeneration for SynthID | C2PA and related source signals |
|
||||
| Google Veo video | Veo diamond and legacy text | Oracle-certified VAE removal for SynthID | C2PA and related source signals |
|
||||
| OpenAI image generators | None registered | Diffusion regeneration for supported invisible signals | C2PA and generator provenance |
|
||||
| Microsoft Paint and Photos | None registered | External Microsoft oracle for InvisMark; no validated local decoder | Paint C2PA soft-binding algorithm and identifier |
|
||||
| Stable Diffusion and SDXL | None registered | Diffusion regeneration; optional open decoder | Embedded parameters and text metadata |
|
||||
| FLUX | None registered | Diffusion regeneration; optional open decoder | C2PA for supported sources |
|
||||
| Adobe Firefly | None registered | No proprietary local decoder | C2PA; optional TrustMark decoder |
|
||||
| Adobe Firefly | None registered | Optional TrustMark Variant P decoder | C2PA |
|
||||
| Midjourney | None registered | No registered pixel decoder | EXIF, XMP, and IPTC signals |
|
||||
| ByteDance generators | Doubao and Jimeng marks | No registered pixel decoder | TC260 AIGC and supported C2PA signals |
|
||||
| Luma AI | None registered | No registered pixel decoder | PNG text generator tags (Uni-1) |
|
||||
| ByteDance generators | Doubao and Jimeng marks | No registered pixel decoder | TC260 AIGC, supported C2PA, and exact app-export AIGC disclosures |
|
||||
| Qwen | Qwen mark | No registered pixel decoder | TC260 AIGC |
|
||||
| Kling | Kling image and video marks | No registered pixel decoder | TC260 AIGC |
|
||||
| Hailuo / MiniMax video | Hailuo composite video label | No registered pixel decoder | TC260 AIGC where present |
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# Deep research: SynthID-safe face-identity recovery for SDXL (2026-06-08)
|
||||
|
||||
> Research archive. This dated follow-up records evidence available during the
|
||||
> study. It is not the current command reference.
|
||||
> study. It is not the current command reference. The `instantid` and `photomaker`
|
||||
> extras it discusses, the `--restore-faces` flag, and their modules were all removed
|
||||
> from the library afterwards.
|
||||
|
||||
**Stats:** {"angles": 6, "sourcesFetched": 28, "claimsExtracted": 104, "claimsVerified": 25, "confirmed": 19, "killed": 6, "afterSynthesis": 6, "urlDupes": 1, "budgetDropped": 7, "agentCalls": 111}
|
||||
|
||||
@@ -149,8 +151,9 @@ on Modal A100 in two phases:
|
||||
- v3 (tighter ellipse 0.32*bw x 0.42*bh + per-channel mean color match to
|
||||
local cleaned canvas + softer feather): patchwork visually softened; faces
|
||||
still read as studio portraits inserted into the scene, not as people
|
||||
shot in the scene. Single portrait identity drifted (tatsunari -> "round
|
||||
Asian male" vs original's thin face).
|
||||
shot in the scene. Single portrait identity drifted (the single-portrait
|
||||
fixture regenerated as a generic "round Asian male" vs the original's thin
|
||||
face).
|
||||
2. **Phase 2: InstantID img2img on cleaned crop.** Switched to the upstream
|
||||
`pipeline_stable_diffusion_xl_instantid_img2img` (downloaded at first use
|
||||
from raw.githubusercontent.com; requires `trust_remote_code=True`). Same
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
> Research archive. This document evaluates approaches considered during the
|
||||
> study. It does not describe the current public interface. The current
|
||||
> pipelines and limits are documented in `docs/cli.md` and
|
||||
> `docs/known-limitations.md`.
|
||||
> `docs/known-limitations.md`. The path it calls shipped -- `photomaker_restore.py`
|
||||
> and the `photomaker` extra -- was removed from the library afterwards.
|
||||
|
||||
> **Status (2026-06-08): retired.** Every approach described below was empirically
|
||||
> tested and rejected -- see `docs/synthid-robust-identity-research-2026-06-08.md`
|
||||
@@ -246,7 +247,9 @@ from the test set + this doc).
|
||||
- If no -> the assumption is wrong; PhotoMaker would re-introduce the
|
||||
watermark. Stop and reconsider.
|
||||
2. **PhotoMaker-V1 prototype** in the existing `controlnet` pipeline:
|
||||
- Mirror the `_load_controlnet_pipeline` path: add a PhotoMaker variant that
|
||||
- Mirror the `_load_controlnet_pipeline` path (removed in 0.24.0 with the
|
||||
controlnet profile; the equivalent seam is now `SdxlZImagePipeline._load_sdxl`):
|
||||
add a PhotoMaker variant that
|
||||
loads SDXL + canny ControlNet + PhotoMaker adapter on the same engine.
|
||||
- Extract the OpenCLIP face embedding from the watermarked face crops (use
|
||||
OpenCV YuNet, already bundled for `auto`, to find the face boxes).
|
||||
|
||||
+66
-27
@@ -289,14 +289,19 @@ The external variant SynthID-O is available "through partnerships" only. Our
|
||||
tool cannot locally detect SynthID presence or absence -- this is by design,
|
||||
not a gap we can fill.
|
||||
|
||||
### 3.2 How our tool detects SynthID (metadata proxy)
|
||||
### 3.2 How our tool recognizes SynthID from provenance
|
||||
|
||||
We detect SynthID indirectly: if the image's C2PA manifest is signed by a
|
||||
known SynthID-using issuer (Google, OpenAI), we infer SynthID is present. This
|
||||
is a **metadata proxy**, not a pixel watermark decode. It works while the C2PA
|
||||
manifest is intact, and is silent once the manifest is stripped or the image
|
||||
is re-encoded without C2PA (e.g., a screenshot, a social-media re-upload, or
|
||||
after `metadata --remove`).
|
||||
We recognize SynthID indirectly from supported C2PA evidence; this is not a
|
||||
pixel watermark decode. Google states that all media generated by its tools is
|
||||
watermarked, so Google AI C2PA establishes SynthID. OpenAI C2PA predates its
|
||||
SynthID rollout, but current manifests add the signed
|
||||
`c2pa.watermarked.unbound` action; OpenAI provenance establishes SynthID only
|
||||
when that action is present. Legacy OpenAI C2PA without it remains valid origin
|
||||
evidence but does not assert a pixel watermark.
|
||||
|
||||
This works while the C2PA manifest is intact and is silent once the manifest is
|
||||
stripped or the image is re-encoded without C2PA (e.g., a screenshot, a
|
||||
social-media re-upload, or after `metadata --remove`).
|
||||
|
||||
This is why:
|
||||
- `identify` on a GitHub-recompressed issue attachment returns Unknown (C2PA is
|
||||
@@ -372,6 +377,11 @@ itself. `0.15` is therefore the shipped default. The tracked manifest
|
||||
`data/evaluations/video-synthid-oracle.csv` records the public source URL,
|
||||
hashes, fidelity metrics, and verdicts without committing generated videos.
|
||||
|
||||
What that calibration does and does not constrain, the ranked experiment program
|
||||
for trading less quality for the same removal, and the change candidates that were
|
||||
refuted along the way are recorded in
|
||||
[`video-synthid-quality-research.md`](video-synthid-quality-research.md).
|
||||
|
||||
The VAE perturbation follows the general regeneration-attack construction from
|
||||
Zhao et al. The video-specific control and temporal metric are local additions.
|
||||
VideoMarkBench motivates testing frame aggregation and matched perturbations,
|
||||
@@ -404,8 +414,9 @@ OpenAI confirmed SynthID adoption (Help Center, updated 2026-05-21):
|
||||
> "ChatGPT images include both C2PA metadata and SynthID watermarks."
|
||||
|
||||
This is time-gated: pre-rollout ChatGPT/gpt-image images carry C2PA without
|
||||
SynthID. Our C2PA proxy therefore over-reports SynthID presence on old images
|
||||
(hence the `_OPENAI_CAVEAT` hedging flag in the codebase).
|
||||
SynthID. Current OpenAI manifests distinguish the watermarked output with the
|
||||
signed `c2pa.watermarked.unbound` action. The detector requires that action, so
|
||||
old OpenAI C2PA remains an origin signal without becoming a SynthID claim.
|
||||
|
||||
### 4.3 Other vendors
|
||||
|
||||
@@ -448,6 +459,14 @@ the payload, reconstituting SynthID in text. The lesson held and shaped the
|
||||
current design: **content is preserved by REGENERATING it under structural
|
||||
conditioning, never by copying original pixels.**
|
||||
|
||||
> **Superseded in 0.24.0.** The `controlnet`, `sdxl`, `qwen` and `default` profiles
|
||||
> were removed, and a retired name is now rejected at parse time rather than routed
|
||||
> onward. `--pipeline` accepts only `qwen-zimage` (the default) and `sdxl-zimage`;
|
||||
> both are CUDA-only, both condition their global stage on a canny edge map, and both
|
||||
> run the same Z-Image face stage. The bullets below are a record of what was measured
|
||||
> on the removed profiles, kept because the oracle verdicts are still the evidence the
|
||||
> current design rests on.
|
||||
|
||||
- **Text + structure:** `--pipeline controlnet` (SDXL img2img + a canny ControlNet) is
|
||||
**THE DEFAULT pipeline since 2026-06-09** (`--pipeline default` opts down to plain
|
||||
SDXL img2img for inputs without text/faces). It conditions the regeneration on the
|
||||
@@ -480,21 +499,21 @@ conditioning, never by copying original pixels.**
|
||||
measured at. See §5.2.**
|
||||
- **Highest-fidelity CUDA option:** `--pipeline qwen-zimage` is the recommended
|
||||
quality mode when preserving face identity matters more than latency, model size,
|
||||
and GPU cost. ControlNet remains the default because it is much cheaper and supports
|
||||
and GPU cost. ControlNet was then the default, because it was much cheaper and ran on
|
||||
CUDA, XPU, MPS, and CPU, but canny conditioning preserves edges rather than identity.
|
||||
On two direct upstream comparisons, `qwen-zimage` retained substantially more
|
||||
ArcFace identity than polished ControlNet. On 2026-07-25 the exact six-output
|
||||
`visible -> qwen-zimage -> metadata` candidate was negative in the corresponding
|
||||
OpenAI and Gemini oracles. This is a quality recommendation for the measured content,
|
||||
not broad removal certification; very small text can still degrade.
|
||||
See `docs/known-limitations.md` for the metrics, runtime, and validation scope.
|
||||
- **Face identity:** canny holds face *structure* but not *identity*. The standard
|
||||
SDXL and ControlNet profiles do not run a separate face-restoration option.
|
||||
Earlier GFPGAN, PhotoMaker, and FaceID experiments were removed after they
|
||||
degraded identity or risked reintroducing source pixels. The separate
|
||||
`qwen-zimage` profile now provides the only shipped face-specific stage:
|
||||
YuNet and SAM locate faces, then Z-Image regenerates the selected original
|
||||
face crops before a feathered composite. See
|
||||
See `docs/qwen-improvement-research.md` for the identity and text metrics and the
|
||||
validation scope of those comparisons.
|
||||
- **Face identity:** canny holds face *structure* but not *identity*. The removed
|
||||
SDXL and ControlNet profiles did not run a separate face-restoration stage, and
|
||||
earlier GFPGAN, PhotoMaker, and FaceID experiments were dropped after they
|
||||
degraded identity or risked reintroducing source pixels. Both shipped profiles
|
||||
now run the same face-specific stage: YuNet and SAM locate faces, then Z-Image
|
||||
regenerates the selected original face crops before a feathered composite. See
|
||||
`docs/controlnet-removal-pipeline-research.md` for the historical experiments.
|
||||
|
||||
### 5.2 Strength setting
|
||||
@@ -514,10 +533,17 @@ study (section 2.2) gives empirical floors:
|
||||
- **Google native 2816**: 0.15 clears (n=2, deployed controlnet worker, 2026-06-14) --
|
||||
the same rung as capped 1536, so no resolution penalty was observed.
|
||||
|
||||
The default is **vendor-adaptive** (`watermark_profiles.resolve_strength` +
|
||||
`vendor_for_strength`): the tool reads the C2PA issuer on the original input and picks
|
||||
> **Superseded in 0.24.0.** The `sdxl`, `controlnet`, `qwen` and `default` profiles
|
||||
> were removed, and `OPENAI_STRENGTH` / `GEMINI_STRENGTH` / `UNKNOWN_STRENGTH` went
|
||||
> with them. Everything from here to the end of this section is a record of what was
|
||||
> measured on those profiles, kept because the oracle verdicts are still the evidence
|
||||
> base. For the strength policy in force now see `module-internals.md`: `qwen-zimage`
|
||||
> uses `resolution_adaptive_denoise`, `sdxl-zimage` a flat vendor ladder.
|
||||
|
||||
The default was **vendor-adaptive** (`watermark_profiles.resolve_strength` +
|
||||
`vendor_for_strength`): the tool read the C2PA issuer on the original input and picked
|
||||
`OPENAI_STRENGTH` 0.10 / `GEMINI_STRENGTH` 0.15 / `UNKNOWN_STRENGTH` 0.15 **(LOWERED
|
||||
2026-06-14 from the 2026-06-04 cert floors 0.20/0.30/0.30)**. **The SAME ladder applies
|
||||
2026-06-14 from the 2026-06-04 cert floors 0.20/0.30/0.30)**. **The SAME ladder applied
|
||||
to both pipelines** (`sdxl` and `controlnet`). The 2026-06-14 re-test on the deployed
|
||||
Modal controlnet worker (v0.10.0) cleared SynthID on the oracle at OpenAI 0.10 (2
|
||||
photoreal) and Google 0.15 (2 NATIVE 2816x1536, contradicting the "native >= 0.30" guess
|
||||
@@ -538,7 +564,7 @@ for `sdxl`, not a separate certification (no local SynthID detector to self-veri
|
||||
The higher strength costs little quality where it matters, because `controlnet` is now
|
||||
the default pipeline, so `sdxl` is reached only via an explicit `--pipeline sdxl` (a
|
||||
deliberate opt-down), where over-regeneration has no faces/text to damage.
|
||||
This uses the vendor signal we DO have locally (the C2PA SynthID proxy) to avoid the
|
||||
This uses the vendor signal we DO have locally (the C2PA SynthID provenance) to avoid the
|
||||
overkill of a single high default on OpenAI images, without needing a local pixel
|
||||
detector. An explicit `--strength` always wins. If the watermark still survives (e.g. a
|
||||
large native Gemini beyond the capped-1536 validation), raise toward 0.35-0.40 (0.40
|
||||
@@ -597,7 +623,7 @@ Two constraints on reading this:
|
||||
- **The correct oracle for OpenAI images is openai.com/verify**, not the Gemini
|
||||
app. The two oracles detect different payloads.
|
||||
- **A quiet `identify` output after processing is not proof of removal.** It
|
||||
means the metadata proxy is gone. The pixel watermark state is unknown without
|
||||
means the provenance evidence is gone. The pixel watermark state is unknown without
|
||||
an oracle check.
|
||||
- **After removal, the output may carry forensic artifacts** detectable by an
|
||||
independent classifier even if the vendor oracle reads negative. Defeating the
|
||||
@@ -707,10 +733,11 @@ end has simply never been through the Gemini oracle on any pipeline. Do not reas
|
||||
a resolution trend here; measure it.
|
||||
|
||||
**Current implication:** the old floor table remains evidence about the dated
|
||||
test set, not the current resolver. The shipped SDXL and ControlNet defaults are
|
||||
defined in `watermark_profiles.py`, and face restoration is available only
|
||||
through the separate `qwen-zimage` profile. Removal near a threshold remains
|
||||
seed dependent, so reproducible verification requires a fixed seed.
|
||||
test set, not the current resolver. The SDXL and ControlNet profiles it measured
|
||||
no longer exist; the shipped defaults are defined in `watermark_profiles.py`, and
|
||||
both surviving profiles run face repair as a built-in second stage rather than as
|
||||
an optional restore. Removal near a threshold remains seed dependent, so
|
||||
reproducible verification requires a fixed seed.
|
||||
|
||||
---
|
||||
|
||||
@@ -742,3 +769,15 @@ seed dependent, so reproducible verification requires a fixed seed.
|
||||
8. Jiang et al. (2025). **VideoMarkBench: Benchmarking Robustness of Video
|
||||
Watermarking.** arXiv:2505.21620.
|
||||
https://arxiv.org/abs/2505.21620
|
||||
|
||||
**Google floor on qwen-zimage (0.27.2, 2026-08-19).** The bracket above did not
|
||||
survive the full production path: on the 4.33 MP CJK-sign fixture (visible-stage
|
||||
sparkle removal -> qwen-zimage seed 0 at the curve's 0.154 top -> resize-back ->
|
||||
metadata strip), the Gemini verifier detected SynthID x3 with a valid
|
||||
pixel-identical stripped control in the same session. Google-provenance content
|
||||
now takes a flat `QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.30` floor instead of the area
|
||||
curve - 0.30 anchors measured clean in Gemini on two fixtures (CJK sign +
|
||||
18-face) at 3/3 checks across two work accounts, and stayed clean with the
|
||||
vae-glyphs donor layer on top; openai/unknown content keeps the resolution
|
||||
curve. An explicit strength still wins. Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-18/` (the failing re-baseline).
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
> Research archive. This page records evaluated ideas, including rejected
|
||||
> prototypes. Current behavior is documented in the user guides and source.
|
||||
> The `text_protector.py` module it describes was removed from the library
|
||||
> afterwards.
|
||||
|
||||
Date: 2026-05-29. Source: a deep-research run (104 agents, 5 search angles, sources
|
||||
fetched and 3-vote adversarially verified). Not committed automatically — saved as a
|
||||
@@ -141,3 +143,229 @@ Deep-research workflow run `wf_118b9a03-3eb` (2026-05-29). Findings adversariall
|
||||
(2/3 refutes required to kill a claim). This note records research only; no code change is
|
||||
implied until a prototype validates fidelity and the SynthID-scrub guarantee on the
|
||||
restored output.
|
||||
|
||||
## 2026-08-13 OCR plus LaMa prototype
|
||||
|
||||
A lightweight version of the recommended two-stage architecture was tested on
|
||||
the three tracked text fixtures. It used the already-clean Qwen result as the
|
||||
base, recognized the source text, removed source and Qwen glyph contours with
|
||||
LaMa, and rendered the recognized strings as fresh pixels. No source pixels
|
||||
were pasted back.
|
||||
|
||||
The result validates only part of the hypothesis. Character-weighted CER on the
|
||||
two multilingual posters fell from 0.338/0.305 to 0.007/0.007, and OpenAI Verify
|
||||
remained clean for both matched Qwen and restored pairs. However, replacement
|
||||
fonts changed the design, whole-image LPIPS worsened by 0.067 on each poster,
|
||||
and the light poster retained a shadow in one small English line. On the Chinese
|
||||
sign, punctuation errors moved CER from 0.074 to 0.111. The Google verifier run
|
||||
was inconclusive because the available account detected both the restored image
|
||||
and the byte-identical Qwen control that a previous account had called clean.
|
||||
|
||||
The experiment supports a proper glyph-conditioned restorer, not shipping the
|
||||
plain OCR/system-font compositor. Exact hashes, metrics, masks, and provider
|
||||
verdicts are tracked in
|
||||
[`data/evaluations/fidelity/text-restoration-2026-08-13.csv`](../data/evaluations/fidelity/text-restoration-2026-08-13.csv).
|
||||
|
||||
## 2026-08-13 AnyText2 follow-up
|
||||
|
||||
The official AnyText2 SD1.5 checkpoint was then tested as the glyph-conditioned
|
||||
restorer. Its ModelScope entry and code are Apache 2.0, and the downloaded
|
||||
checkpoint matched the published SHA-256. An official blackboard edit completed
|
||||
successfully before the fixture run, establishing that the runtime reproduced
|
||||
the model rather than silently exercising a fallback.
|
||||
|
||||
On the Chinese sign, a three-line local edit over the exact Qwen output scored
|
||||
CER 0.185 under the standard detector. Font mimic from the source glyph masks
|
||||
scored 0.222. Whole-image LPIPS changed from 0.289 to 0.338 and 0.345 respectively.
|
||||
A padded crop-recognition check showed why the comparison matters: Qwen contained
|
||||
the full correct text and scored 0.000, while default AnyText2 still scored 0.074
|
||||
with two substituted characters and mimic remained at 0.222. The provider oracle
|
||||
was deliberately not called because these variants had already failed the
|
||||
content-fidelity gate.
|
||||
|
||||
AnyText2 is therefore not the missing production restorer in its published form.
|
||||
The negative result is narrower than the model family: only the SD1.5 checkpoint
|
||||
is public, while AnyText2XL remains listed as future work. The current wrapper
|
||||
also truncates each quoted text line to 20 characters, which prevents a direct
|
||||
test of several tracked English and Russian poster lines. Exact model provenance,
|
||||
output hashes, and metrics are recorded in
|
||||
[`data/evaluations/fidelity/anytext2-restoration-2026-08-13.csv`](../data/evaluations/fidelity/anytext2-restoration-2026-08-13.csv).
|
||||
|
||||
The same cross-check exposes a measurement bug in the earlier lightweight
|
||||
restoration result. Paddle detection made tight boxes that omitted the final
|
||||
Chinese full stop from two lines, producing Qwen CER 0.074 although the pixels
|
||||
and padded recognition were correct. Adding 16-32 pixels of horizontal context
|
||||
restored every punctuation mark. A deterministic rerender using the full OCR
|
||||
strings and the closest of nine mask-scored CJK fonts also reached detector CER
|
||||
0.000, but its heavier Hiragino Sans GB W6 glyphs raised LPIPS from 0.289 to 0.327.
|
||||
The actionable design is selective restoration: compare padded source and output
|
||||
recognition first, and preserve the Qwen output when they already match.
|
||||
|
||||
That policy was tested on the two multilingual posters. A manual prototype kept
|
||||
the lines whose padded source and Qwen recognition agreed and rerendered only the
|
||||
changed lines. The edited mask fell from 15.4%/17.0% in the full compositor to
|
||||
5.5%/5.5%. On a single Paddle `en+ru+ch` route, CER changed from 0.378/0.413 for
|
||||
Qwen to 0.101/0.112, while LPIPS was only 0.120/0.103 rather than the full
|
||||
compositor's 0.174/0.162. The different OCR route is intentional and all three
|
||||
variants were remeasured through it; these CER values are not directly
|
||||
comparable with the earlier Vision/Paddle hybrid results.
|
||||
|
||||
Both selective outputs returned `No OpenAI signals detected` in OpenAI Verify,
|
||||
and the original light poster returned `Generated with OpenAI tools` immediately
|
||||
afterward as the positive control. The dark poster was visually clean. The light
|
||||
poster still showed one local old-text shadow, so mask construction remains the
|
||||
blocking defect. Exact hashes, metrics, mask fractions, and oracle controls are
|
||||
in
|
||||
[`data/evaluations/fidelity/selective-text-restoration-2026-08-13.csv`](../data/evaluations/fidelity/selective-text-restoration-2026-08-13.csv).
|
||||
|
||||
Uniformly expanding every selected glyph mask by two pixels removed that light
|
||||
poster shadow and moved CER from 0.101 to 0.081 without a meaningful LPIPS cost.
|
||||
It also expanded the edited area from 5.5% to 7.0%, and the identical rule made
|
||||
the dark poster worse at CER 0.147 rather than 0.112. Those follow-up bytes have
|
||||
not been oracle-checked. The next prototype should detect residual source glyphs
|
||||
per line and expand only the failing component, rather than introducing another
|
||||
poster-wide tuning constant.
|
||||
|
||||
The manual selection was then replaced with per-line padded recognition. A line
|
||||
is left untouched only when the source recognizer is sufficiently consistent
|
||||
with the verified line and normalized source/candidate recognition agree
|
||||
exactly. The automatic rule reproduced the manual selection on the dark poster
|
||||
and correctly kept one additional large Russian line on the light poster. It
|
||||
reduced that poster's effective edited area from 5.5% to 4.1%.
|
||||
|
||||
Mask cleanup became a measured second pass rather than a global dilation. After
|
||||
the first LaMa erase, the prototype finds contrast that remains specifically at
|
||||
the original glyph positions, dilates only that residual, and erases it before
|
||||
fresh text is drawn. This removed the visible double glyphs from both posters.
|
||||
The automatic outputs scored LPIPS 0.113/0.104 and detector CER 0.123/0.119, with
|
||||
effective edited fractions of 4.1%/5.7%. The light aggregate CER is pessimistic:
|
||||
the page detector fragments its untouched Cyrillic line into Latin-like pieces,
|
||||
while the padded Russian recognizer returns the exact expected text.
|
||||
|
||||
OpenAI Verify returned `No OpenAI signals detected` for both new hashes and then
|
||||
`Generated with OpenAI tools` for the original light poster in the same browser
|
||||
sequence. Reproducing the prototype from the tracked script exposed harmless
|
||||
ONNX rounding on the light image: 386 pixels differed by at most one channel
|
||||
value from the initially checked hash. The exact tracked output hash
|
||||
`900def5a...` was therefore submitted separately, returned clean, and was
|
||||
followed immediately by another positive source result. The dark tracked output
|
||||
was byte-identical to the initially checked hash.
|
||||
|
||||
The Chinese sign provides a no-edit control for the selector. Detector boxes for
|
||||
its three large lines overlap vertically, so Latin-style vertical padding
|
||||
incorrectly mixed neighboring lines. Horizontal-only CJK padding captured the
|
||||
terminal punctuation without mixing neighbors and made source and candidate
|
||||
recognition agree exactly on all three lines.
|
||||
The script then produced a zero mask and copied the Qwen candidate byte for byte.
|
||||
This prevents the heavier-font regression seen in the earlier CJK compositor,
|
||||
but does not resolve the unavailable Google-negative oracle control.
|
||||
|
||||
The result clears the measured OpenAI gate without a manual line selection, but
|
||||
is not a production design yet. It still assumes verified source strings and
|
||||
line boxes, relies on macOS system fonts, and needs evaluation on uncurated
|
||||
layouts, rotated text, false OCR matches, automatic line-box discovery, and a
|
||||
reproducible Google-negative control.
|
||||
|
||||
The later source-silhouette experiment removed the font lookup but did not meet
|
||||
the actual visual requirement. Native-size review found changed stroke weight,
|
||||
color variation, edge antialiasing, and decorative details even where crop OCR
|
||||
was exact. A narrower `vae-glyphs` experiment uses the silhouette only as a
|
||||
support mask: exact pixels come from a Qwen-VAE reconstruction, while a fresh
|
||||
flat-color silhouette supplies only the outer edge beneath that core. Across 48
|
||||
typography cases (548 annotated lines), this improved glyph-edge MAE in 48/48
|
||||
and text-box SSIM in 47/48. The median values moved from 37.59 to 32.42 and from
|
||||
0.854 to 0.914 respectively. One separate untracked core-only output with a
|
||||
1.43% donor area returned `No OpenAI signals detected`; the tracked script then
|
||||
reproduced those exact bytes. A 0.5-pixel feathered sibling with 2.75% nonzero
|
||||
donor area improved mean text-box SSIM from 0.957 to 0.960, with a 0.918 minimum
|
||||
across 15 verified lines, while full-image RGB SSIM reached 0.864. Crop OCR
|
||||
recovered from 7/15 exact lines on the raw pass to 14/15, equal to the source's
|
||||
own recognizer floor. Whole-image LPIPS was 0.082, but only 0.108% of pixels
|
||||
remained exactly equal to the source and the detected face's Laplacian-variance
|
||||
ratio was 0.670, confirming residual global smoothing. A same-session
|
||||
OpenAI Verify sequence detected the exact source control in 1/1 check, returned
|
||||
clean for the raw global pass and clean-fidelity anchor in 1/1 each, and returned
|
||||
clean for the feathered output in 3/3 repeats. The tracked script reproduced
|
||||
that feathered file byte for byte. This proves two materially better exact-output
|
||||
Pareto points, not a general operating range. The 48 matrix outputs and other
|
||||
mask sizes remain oracle-unverified.
|
||||
|
||||
The first Google oracle sample did not transfer. For the synthetic CJK sign,
|
||||
two separate work accounts detected SynthID in both the resaved positive source
|
||||
control and the exact Qwen-VAE donor candidate. The candidate's text-box SSIM
|
||||
improved from 0.791 to 0.835 and its glyph-edge MAE from 35.48 to 22.49, but its
|
||||
3.67% donor layer did not create a clean operating point. The Qwen silhouette
|
||||
base was already detected, as was the earlier original-sign Qwen baseline in the
|
||||
same account. The measured cause is therefore insufficient global Google
|
||||
regeneration, not a demonstrated watermark regression from the text donor.
|
||||
Google needs a stronger oracle-clean global anchor before the narrow donor can
|
||||
be evaluated there.
|
||||
|
||||
A targeted follow-up supplied that missing anchor for one CJK case. An explicit
|
||||
Qwen global pass at strength 0.30 returned no reliable SynthID signal in Gemini,
|
||||
while the matched source control was detected. Applying the 0.5-pixel
|
||||
`vae-glyphs` layer to that clean anchor changed 1.17% of the image and raised
|
||||
mean text-box SSIM from 0.765 to 0.964 (minimum 0.963), with mean text-box MAE
|
||||
falling from 20.74 to 5.38. The post-donor output then returned clean in 3/3
|
||||
Gemini checks across two work accounts. A separate 18-face fixture with four
|
||||
tiny UI-text lines also had a clean strength-0.30 anchor; its donor layer changed
|
||||
0.70% of the image, raised mean text-box SSIM from 0.617 to 0.976 (minimum
|
||||
0.972), and returned clean in 3/3 checks across the same two accounts. One
|
||||
quota-exceeded response is excluded from both pass rates. These are two exact
|
||||
oracle-certified outputs, not a general operating range: other layouts, masks,
|
||||
strengths, seeds, and donor areas remain unverified.
|
||||
|
||||
An automatic-box follow-up merged Paddle word detections by vertical overlap.
|
||||
It found exactly 20/20 poster lines and 3/3 sign lines, with mean IoU 0.857,
|
||||
0.847, and 1.000 against the verified boxes. That structural match was not
|
||||
sufficient: reusing the annotation crop padding changed recognition decisions,
|
||||
reduced preserved dark-poster lines from 7 to 4, and expanded its edited
|
||||
fraction from 5.7% to 11.2%.
|
||||
|
||||
A recognition-only sweep isolated the crop sensitivity. Limiting vertical
|
||||
padding around detector boxes to 8-12% of line height reproduced the verified-
|
||||
box decision vectors on both posters; 10% was used for a full follow-up. It kept
|
||||
8/20 and 7/20 lines with edited fractions of 4.1% and 5.7%, and visual inspection
|
||||
found no double glyphs. Whole-image LPIPS remained 0.113/0.104, but detector CER
|
||||
was 0.127/0.154 instead of 0.123/0.119. The dark regression failed the fidelity
|
||||
gate, so these hashes were not submitted to the provider oracle. The opt-in flag
|
||||
remains only to reproduce the negative result. Count, IoU, and matching decision
|
||||
vectors are therefore insufficient gates for automatic boxes; the next design
|
||||
needs box rectification or recognition stability under crop jitter before it
|
||||
can replace verified geometry.
|
||||
|
||||
Removing verified strings was tested separately with an annotation-seed dry
|
||||
run. It detects boxes, chooses `en`, `ru`, or `ch` from Unicode script, and
|
||||
accepts a draft only when three crop paddings normalize identically and every
|
||||
confidence is at least 0.85. One execution proposed 20 and 18 poster lines, but
|
||||
exact-text precision against the held-out annotations was only 90.0% and 94.4%.
|
||||
The stable errors were punctuation: one lost English comma and an ideographic
|
||||
comma consistently replaced by ASCII. Thus crop agreement and confidence do
|
||||
not verify literal text. The helper is retained only to seed annotations; every
|
||||
proposal needs manual review before it can enter the restoration compositor.
|
||||
|
||||
## Production-path anchor leak and fix (2026-08-19)
|
||||
|
||||
The engine `text_manifest=` path (unlike the research compositor above) blends a
|
||||
15% Qwen-VAE fidelity anchor across the WHOLE frame before glyph restoration.
|
||||
Certified through the official OpenAI Content Provenance API on the exact
|
||||
production path (qwen-zimage, seed 0, no strength override, untiled, LANCZOS
|
||||
resize-back, metadata strip; library 0.27.0 @ 50d9f448):
|
||||
|
||||
- controls (pixel-identical metadata-stripped sources): `synthid: detected` -
|
||||
the pixel channel was proven live before every session;
|
||||
- base outputs: `clean` x6 (two 20-line posters, 3 attempts each);
|
||||
- text-restored WITH the anchor: `synthid: detected` x6, deterministic;
|
||||
- text-restored with `FIDELITY_BLEND_ALPHA = 0` (runtime ablation, base bytes
|
||||
identical between runs): `clean` x6.
|
||||
|
||||
The anchor was therefore the leak channel: a 15% global donor admixture over a
|
||||
scrubbed frame returns enough OpenAI SynthID for the provider detector. Fidelity
|
||||
cost of dropping it (source-referenced text-box MAE): 11.60->11.72, 7.79->7.86,
|
||||
and 7.57->8.13 across the two posters and the CJK sign - well under one point
|
||||
against base MAEs of 24.7/16.8/10.2. Since 0.27.1 the anchor is off by default
|
||||
(`fidelity_anchor` parameter on `remove_watermark` / `InvisibleOptions` /
|
||||
`--fidelity-anchor`); Google rows were not re-queried because the default
|
||||
strength leaves that fixture's base detected (the vendor-strength lane tracks
|
||||
that separately). Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-19-A1/` (sha256-stamped rows).
|
||||
+39
-16
@@ -39,6 +39,15 @@ confidence), reviewed once, then re-baselined. Lost detections are the alarm.
|
||||
|
||||
Implemented as `scripts/sidecar_regression.py` (resumable, ~1.5 h at 8 workers).
|
||||
|
||||
A 2026-08-15 metadata-only C2PA regression audit over the local historical
|
||||
corpus caught two structured-parser gaps before release: an exact Firefly claim
|
||||
generator without a repeated source type, and a Dreamina generator carried by a
|
||||
reachable ingredient under a generic update manifest. Both now have focused
|
||||
tests. The same audit confirmed that invalid hash/signature claims lose origin
|
||||
attribution without losing the C2PA inventory, and exposed the absence of
|
||||
production trust anchors in the SDK defaults. Dataset-derived counts and
|
||||
identifiers remain in the gitignored audit output.
|
||||
|
||||
#### Local run protocol
|
||||
|
||||
Re-run `identify` against locally recorded sidecars, classify losses separately from intended new detections, and keep generated reports under `.local-eval/`. Do not commit dataset-derived counts or identifiers.
|
||||
@@ -72,8 +81,10 @@ Run detection, removal, and re-detection over a representative local set. Confir
|
||||
|
||||
### A5. Contract sweep across every parameter choice
|
||||
|
||||
`scripts/smoke_matrix.py` (exists, 68 rows, 0 skipped with `--diffusion`) covers every
|
||||
choice-valued flag on fixtures. Extend from fixtures to a stratified corpus slice
|
||||
`scripts/smoke_matrix.py` covers every choice-valued flag on fixtures. Its knob rows
|
||||
were rewritten when the CPU/MPS profiles and the ESRGAN chain were removed: most now
|
||||
assert a knob is REJECTED, and the accepted-knob rows skip without a CUDA device, so
|
||||
the row count is host-dependent rather than the fixed 68 recorded here before. Extend from fixtures to a stratified corpus slice
|
||||
(~500 images spanning format x provenance x aspect ratio), asserting exit-code semantics
|
||||
rather than just absence of crash.
|
||||
|
||||
@@ -84,25 +95,33 @@ sweep asserts on stderr, or the codes get split -- the latter is the better fix.
|
||||
#### Coverage method
|
||||
|
||||
Compare the flags and values exercised by the matrix against the options declared by
|
||||
the CLI. Include optional backends, batch modes, tiling, region-targeted composition,
|
||||
the ESRGAN upscaler, and the ffmpeg audio/video strip. The gap to find is not only
|
||||
the CLI. Include optional backends, batch modes, tiling, and the ffmpeg audio/video
|
||||
strip. (Region-targeted diffusion composition was on this list until the parameter
|
||||
was deleted: nothing but a caller-less convenience wrapper ever reached it, and the
|
||||
user-facing region path is `erase`, which inpaints instead.) The gap to find is not only
|
||||
"logic untested" but
|
||||
"never executed on real data", which is precisely what this campaign is for.
|
||||
|
||||
#### Bug found by the extension: `--steps` below ~7 crashes inside torch
|
||||
#### Bug found by the extension: `--steps` below ~7 crashed inside torch
|
||||
|
||||
Effective timesteps are `int(steps * strength)`. At the vendor-adaptive default strength
|
||||
(0.15, or 0.10 for OpenAI) any `--steps` under 7 rounds to **zero**, and the pipeline dies
|
||||
with a raw traceback:
|
||||
**Fixed by deletion.** `--steps` no longer exists, on the CLI or in the Python API,
|
||||
so this class of failure is unreachable. Kept as a record of why.
|
||||
|
||||
Effective timesteps were `int(steps * strength)`. At the vendor-adaptive default
|
||||
strength (0.15, or 0.10 for OpenAI) any `--steps` under 7 rounded to **zero**, and the
|
||||
pipeline died with a raw traceback:
|
||||
|
||||
```
|
||||
$ remove-ai-watermarks invisible img.png --steps 5
|
||||
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1, 1, 512]
|
||||
```
|
||||
|
||||
Fully valid CLI arguments, no special flags, no `--force`. The value is accepted, the
|
||||
crash is a torch internal, and nothing tells the user that steps and strength interact.
|
||||
Fix is either a clamp to >=1 effective step or an up-front validation naming both values.
|
||||
Fully valid CLI arguments, no special flags, no `--force`. The value was accepted, the
|
||||
crash was a torch internal, and nothing told the user that steps and strength interact.
|
||||
The considered fixes were a clamp to >=1 effective step or an up-front validation
|
||||
naming both values; what shipped instead is that each stage owns its own distilled
|
||||
schedule and no caller can set it. The general lesson stands: a knob whose valid range
|
||||
depends on another knob's value needs the interaction validated where both are known,
|
||||
or it needs to not be a knob.
|
||||
|
||||
Method note: the first run of the knob rows failed 12 times with this identical error,
|
||||
which read like twelve broken features. It was one bad harness parameter (`--steps 4`)
|
||||
@@ -167,10 +186,14 @@ report recall from the detector-sampled set.
|
||||
|
||||
## Tier D -- external oracles (manual, not automatable here)
|
||||
|
||||
SynthID removal cannot be verified locally by design -- no public decoder exists. Each
|
||||
vendor has its own oracle and it covers only that vendor's content: `openai.com/verify` for
|
||||
OpenAI (more accessible, the automation candidate), the Gemini app for Google (manual,
|
||||
rate-limited). A quiet metadata proxy is **not** proof the pixel watermark is gone.
|
||||
Proprietary watermark removal cannot be verified locally by design -- no public decoder
|
||||
exists. Each vendor has its own oracle and it covers only that vendor's content:
|
||||
`openai.com/verify` for OpenAI, the Gemini app for Google, and Microsoft's
|
||||
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
|
||||
for InvisMark. The Microsoft API reports `Watermark` and `C2PA` separately. It therefore
|
||||
needs a pixel-identical metadata-stripped control: the control must lose `C2PA` while
|
||||
remaining `Watermark`-positive before a candidate's negative result can be attributed to
|
||||
pixel regeneration. A quiet metadata proxy is **not** proof the pixel watermark is gone.
|
||||
|
||||
Scope honestly: this tier certifies strength floors on a handful of images per vendor, and
|
||||
that is all it can do. See `docs/synthid.md`.
|
||||
|
||||
@@ -0,0 +1,607 @@
|
||||
# Video SynthID quality research (2026-08-05)
|
||||
|
||||
> Research archive. This page records experiments and decisions from the date
|
||||
> above. It may mention prototypes or defaults that were later changed. Use the
|
||||
> user guides and current source code for the supported interface.
|
||||
|
||||
Cited research behind the question **"can the video SynthID path keep removing the
|
||||
mark while giving up far less quality than the shipped 512 px / 12 fps profile?"**
|
||||
Produced by a 13-agent workflow: 6 parallel scouts (pipeline ablation, SynthID
|
||||
internals, attack literature, autoencoder landscape, perceptual masking, experiment
|
||||
design), 11 change proposals from 3 independent design angles, and 3 adversarial
|
||||
critics (signal theory, repository engineering, verifiability). No experiment was
|
||||
run against the provider oracle for this page; every claim is labeled MEASURED,
|
||||
REPORTED, or INFERRED.
|
||||
|
||||
Repository claims below were re-verified against source after the workflow
|
||||
returned: the PSNR reference frame, the audio stream copy, the missing `_fit_size`
|
||||
clamp, the absent `enable_tiling` call, the crf asymmetry, the test pins, and the
|
||||
absent `scaling_factor` key in the cached checkpoint config.
|
||||
|
||||
## Context
|
||||
|
||||
`remove_video_invisible` regenerates video pixels through `sd-vae-ft-mse`: frames
|
||||
are decimated to 12 fps, resized to a 512 px long side, encoded to latents with
|
||||
`latent_dist.mode()`, perturbed by one seeded spatial noise field shared across
|
||||
every frame, decoded, and streamed to an H.264 encoder at crf 18. A separate
|
||||
stream-copy mux re-adds source audio and strips metadata. The shipped
|
||||
`noise_std=0.15` is certified by one oracle row on one carrier.
|
||||
|
||||
The quality complaint is real: a 1080p source is delivered at roughly a quarter of
|
||||
its linear resolution and half its frame rate. This page asks what part of that
|
||||
cost is buying removal and what part is buying nothing.
|
||||
|
||||
## What the two oracle rows do and do not prove
|
||||
|
||||
`data/evaluations/video-synthid-oracle.csv`, one carrier (Veo 3 off-road,
|
||||
sha256 `79a552b9...`), one seed, one geometry:
|
||||
|
||||
| noise_std | long_side | fps | verdict | psnr_db | temporal_residual_ratio |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 0.10 | 512 | 12 | detected | 26.2932 | 1.0072 |
|
||||
| 0.15 | 512 | 12 | not_detected | 25.3911 | 1.0578 |
|
||||
|
||||
They prove exactly one thing: the envelope of 512 px, 12 fps, crf 18, and a full
|
||||
VAE round trip does **not** silence the oracle on its own. The verdict flip is
|
||||
bought by the last 0.05 of latent noise.
|
||||
|
||||
They cannot decompose that conjunction. No row varies `long_side` or `fps`, so the
|
||||
contribution of the downscale is unmeasured. The frequent reading "the 512 px
|
||||
downscale was probably doing the removal work" is not supported, and neither is its
|
||||
opposite.
|
||||
|
||||
## Measured facts from the repository
|
||||
|
||||
- **The fidelity metric is blind to the expensive steps.** `_iter_sampled_frames`
|
||||
applies `cv2.resize` at [`video_invisible.py:236`](../src/remove_ai_watermarks/video_invisible.py);
|
||||
the same generator feeds the loop at `:401-407`; the accumulator at `:429-435`
|
||||
zips those already-resized frames against `regenerated` captured at `:423`, which
|
||||
is before `frame_pipe.write` at `:430`. `psnr_db` therefore excludes the
|
||||
downscale, the decimation, and the encoder. It measures the VAE round trip plus
|
||||
latent noise at the working geometry, nothing else.
|
||||
- **Frame decimation cannot destroy the carrier.** `_iter_sampled_frames`
|
||||
(`:224-238`) is pure subset selection: the only statement inside the threshold
|
||||
test is `yield cv2.resize(...)`, with no arithmetic across frames. The
|
||||
perturbation is equally time-blind: `shared_noise.expand(latents.shape[0], -1, -1, -1)`
|
||||
at `:295` broadcasts one CHW field across the batch and never mixes across time.
|
||||
Any fps effect on the verdict is a detection-probability effect, not carrier
|
||||
destruction.
|
||||
- **`_fit_size` has no clamp at 1.0.** `scale = long_side / max(width, height)` at
|
||||
`:88` upscales any source whose long side is below 512, and both axes are floored
|
||||
independently to a multiple of 8 at `:89-96`, which introduces an anamorphic
|
||||
shift at scale 1.0 (1366x768 becomes 1360x768).
|
||||
- **`enable_tiling` is never called** anywhere in `src/`, `scripts/`, or `tests/`.
|
||||
`enable_slicing()` at `:135` already splits both encode and decode to single
|
||||
frames, so activation VRAM is set by one frame and does not scale with
|
||||
`--batch-size`.
|
||||
- **crf asymmetry:** 18 on the invisible path (`:321`, `:386`) against 14 on the
|
||||
visible path ([`video_visible.py:1343`](../src/remove_ai_watermarks/video_visible.py))
|
||||
through the same encoder.
|
||||
- **No HDR guard on the invisible path.** `_HDR_TRANSFERS` (`video_visible.py:97`)
|
||||
and the `component_depth > 8` rejection (`:1322`) exist only for visible removal.
|
||||
- **Audio is byte-copied.** `-map 0:v:0`, `-map 1:a?`, `-c copy` at
|
||||
[`video_encoding.py:413`](../src/remove_ai_watermarks/video_encoding.py).
|
||||
- **Only `noise_std` is pinned.** `tests/test_video_invisible.py:285` asserts
|
||||
`DEFAULT_VIDEO_SYNTHID_NOISE_STD == 0.15`. Neither `long_side` nor `fps` is
|
||||
pinned anywhere, so changing either breaks no test.
|
||||
- **`scaling_factor` 0.18215 is a class default, not a checkpoint fact.** The
|
||||
cached `config.json` for `stabilityai/sd-vae-ft-mse` has no `scaling_factor` key
|
||||
at all (verified locally: `_class_name`, `latent_channels`, `block_out_channels`,
|
||||
`sample_size`, and block types only). The value comes from the `AutoencoderKL`
|
||||
class default under `diffusers>=0.38.0` with no upper bound (`pyproject.toml:94`,
|
||||
`uv.lock` resolves 0.39.0), while `maintain.sh` runs `uv-outdated`. A dependency
|
||||
bump can move the certified operating point with a fully green test suite.
|
||||
- **The manifest schema cannot record what a real program needs:** no source
|
||||
geometry, no control row, no track column, no verbatim verdict, no indeterminate
|
||||
state.
|
||||
|
||||
## Measured facts from external sources
|
||||
|
||||
- Gemini video verification quota: 10 checks per rolling 24 hours, up to 5 minutes
|
||||
of video total, under 90 seconds and 100 MB per file. The verifier reports which
|
||||
parts of the video carry the mark, and it has a **third** state beyond detected
|
||||
and not detected: unclear, with documented causes including "not enough details
|
||||
to watermark".
|
||||
- The verifier scores **audio and visual tracks separately**. Google's published
|
||||
example verdict reads as SynthID detected in the audio over a time range with no
|
||||
SynthID detected in the visuals.
|
||||
- SynthID-Image (arXiv:2510.09263) is a post-hoc, model-independent pixel-space
|
||||
watermark: a separate encoder network stamps an already-decoded image rather than
|
||||
being injected into the generator's latents. Its published payload figure is 136
|
||||
bits within a 512x512 image, and its product setup runs at 1536x1536.
|
||||
|
||||
## Inferred, with the reasoning that makes them weak
|
||||
|
||||
- **The noise axis is nearly exhausted.** Fitting `MSE = A + B * noise_std^2` to the
|
||||
two measured rows gives A = 124.5 and B = 2815, so a pure round trip
|
||||
(`noise_std = 0`) lands near 27.2 dB. The entire noise budget is worth at most
|
||||
**+1.79 dB**; everything else is the autoencoder. This is a two-parameter fit to
|
||||
two points with zero degrees of freedom, and the assumptions that MSE is
|
||||
quadratic in `noise_std` and that reconstruction and noise errors are orthogonal
|
||||
are untested. One local run at `noise_std=0` replaces it with a measurement.
|
||||
- **Statistical power of the current certification.** With zero failures at n = 1,
|
||||
the exact one-sided Clopper-Pearson bound `1 - 0.05^(1/n)` is 95%: the data are
|
||||
compatible with removal failing almost always. n = 15 gives 18.1%, n = 30 gives
|
||||
9.5%.
|
||||
- **Size of the geometry prize.** At a 1280x720 source, `_fit_size(1280, 720, 512)`
|
||||
is `(512, 288)` (pinned in `tests/test_video_synthid_sweep.py:30`), so 6.25x of
|
||||
the pixels are discarded by the downscale and another 2x by decimation from a
|
||||
24 fps source. Ladder rungs: 768 gives 2.25x the current area, 1024 gives 4.0x.
|
||||
The carrier's own geometry is recorded nowhere, so this is conditional.
|
||||
- **The direction of the resolution effect is disputed, and both sides are
|
||||
inference.** Against raising it: the absolute frequency ceiling below which an f8
|
||||
VAE reconstructs faithfully is tied to the latent pitch, so 512x288 (a 64x36
|
||||
latent) preserves roughly up to 32 cycles per frame width while 1920 (a 240x135
|
||||
latent) preserves up to about 120. Raising resolution moves the carrier band out
|
||||
of the regime the decoder synthesizes and into the regime it reproduces
|
||||
faithfully, handing the detector more evidence. For raising it: Google's product
|
||||
operating point is 1536x1536, so native is closer to the distribution the
|
||||
watermark encoder targets. Note that the second argument cuts against the
|
||||
proposal rather than for it.
|
||||
|
||||
## First local measurements (2026-08-05)
|
||||
|
||||
Run without the oracle on a locally built carrier: a 6-second 1280x720 24 fps
|
||||
clip panning across `data/fixtures/provenance/doubao-1.png`, processed by the
|
||||
shipped path at 512 px / 12 fps on MPS. Generated media stayed outside the
|
||||
repository.
|
||||
|
||||
| noise_std | engine `psnr_db` | end-to-end PSNR | end-to-end SSIM | bitrate |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 0.00 | 27.8049 | 27.6935 | 0.6955 | 2564 kbps |
|
||||
| 0.15 | 25.8853 | 25.8541 | 0.6703 | 2716 kbps |
|
||||
|
||||
Three readings, all MEASURED, none of them about SynthID:
|
||||
|
||||
1. **The two-point fit's shape survives contact with independent content.** The
|
||||
pure round trip lands at 27.80 dB against the 27.2 dB the carrier fit
|
||||
predicted, and the whole noise budget costs 1.92 dB against the predicted
|
||||
1.79 dB. These are not the carrier's numbers, but the decomposition holds:
|
||||
the autoencoder is the floor and the entire `noise_std` axis is worth about
|
||||
2 dB.
|
||||
2. **On this content the downscale is nearly free in PSNR terms.** End-to-end
|
||||
PSNR sits within 0.11 dB of the in-loop number, meaning the 512 px geometry
|
||||
cost almost nothing next to the VAE damage. This clip is a pan over a smooth
|
||||
generated image with little high-frequency detail, so it is the friendly
|
||||
case: real camera texture should widen that gap. Run the probe across content
|
||||
types before trusting any estimate of the geometry prize.
|
||||
3. **`temporal_residual_ratio` is not meaningful on a near-static shot.** It read
|
||||
1.82 at `noise_std=0` and 2.47 at 0.15, far outside the [1.0072, 1.0578] band
|
||||
ever observed before. A slow pan gives the source almost no motion residual,
|
||||
so the `max(temporal_baseline, 1e-6)` denominator collapses and the ratio
|
||||
inflates. This is the predicted defect, now observed rather than argued.
|
||||
|
||||
## Measurements on the real carrier (2026-08-06)
|
||||
|
||||
The public Veo 3 off-road sample was downloaded and its sha256 matched the manifest
|
||||
exactly, so everything below is measured on the same file the 2026-07-31
|
||||
calibration used. Its properties, none of which the manifest recorded at the time:
|
||||
H.264 1920x1080 at 24 fps, 8.000 s, 5,155,063 bps video, plus an **AAC 48 kHz
|
||||
stereo 192,241 bps audio track of 7.914 s**.
|
||||
|
||||
The shipped profile therefore delivers 512x288 at 12 fps: **7.1% of the frame area
|
||||
and 3.6% of the pixels per second.**
|
||||
|
||||
### The geometry prize, measured end to end
|
||||
|
||||
Scored with `scripts/video_fidelity_probe.py` against the untouched 1920x1080
|
||||
source, all at `noise_std=0.15`:
|
||||
|
||||
| Geometry | Frame area | PSNR | SSIM | Temporal | Bitrate |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 512x288 @12 (shipped) | 7.1% | 25.3971 | 0.8215 | 1.0407 | 2187 kbps |
|
||||
| 768x432 @12 | 16.0% | 27.3796 | 0.8556 | 1.0323 | 3782 kbps |
|
||||
| 1024x576 @12 | 28.4% | 28.8542 | 0.8819 | 1.0262 | 5604 kbps |
|
||||
| 512x288 @24 | 7.1% | 25.6142 | 0.8270 | 1.0995 | 3181 kbps |
|
||||
|
||||
**512 to 1024 is worth +3.46 dB, against 1.92 dB for the entire `noise_std` axis.**
|
||||
The geometry lever is roughly twice the noise lever, which is the first hard number
|
||||
this investigation has for the size of the prize. What it still costs in removal is
|
||||
unknown: none of these was ever submitted.
|
||||
|
||||
Two readings that are not obvious from the table:
|
||||
|
||||
- **Temporal residual improves as resolution rises** (1.0407 to 1.0262). The shared
|
||||
noise field lives on the latent grid, so at 1920 it is four times finer relative
|
||||
to the frame than at 512, where it stretches into low-frequency blotches. Higher
|
||||
resolution plausibly makes the perturbation *less* visible, not more.
|
||||
- **The frame-rate arm looks bad here and the metric is the wrong judge.** 24 fps
|
||||
buys +0.22 dB and worsens the temporal ratio to 1.0995, but what 24 fps actually
|
||||
buys is smoothness, which neither PSNR nor SSIM measures at all. The metric sees
|
||||
only the cost. Do not decide the fps question from this table.
|
||||
|
||||
Encoder cost on an M-series MPS device, 8 seconds of source: 51 s at 512/12, 180 s
|
||||
at 768/12, 380 s at 1024/12, 190 s at 512/24. Native 1920x1080 at 24 fps
|
||||
extrapolates to roughly 46 minutes for the same 8 seconds, around 350x real time. A
|
||||
native run was started and killed before producing anything, so that figure remains
|
||||
an extrapolation.
|
||||
|
||||
### The certified row does not reproduce
|
||||
|
||||
Re-running the certified configuration (512/12/0.15, seed 0) on this machine gave
|
||||
25.5002 dB and 1.0675 against the manifest's 25.3911 and 1.0578, and a different
|
||||
output sha256. The manifest records neither device nor dtype, and the original was
|
||||
almost certainly CUDA fp16 while this was MPS fp32. **The oracle certified a
|
||||
specific file that cannot be rebuilt**, so any re-run produces an artifact the
|
||||
verifier never saw. Device and dtype belong in the schema.
|
||||
|
||||
### Audio passes through untouched, and the output still reads clean
|
||||
|
||||
Established on two carriers, by comparing the extracted audio bitstream sha256 of
|
||||
input and output:
|
||||
|
||||
- Veo carrier: `674063df...` in, `674063df...` out.
|
||||
- A Lyria 3 track generated for this purpose: `6ee4c041...` in, `6ee4c041...` out.
|
||||
|
||||
The Lyria case is the sharper one because its provenance is known. Google's own
|
||||
generator stamps it: `get_ai_metadata` reads a C2PA manifest naming
|
||||
`Google C2PA Core Generator Library`, issuer `Google LLC`, `trainedAlgorithmicMedia`.
|
||||
After the shipped path runs, that output reports `{}` -- every metadata marker
|
||||
stripped, `remaining_metadata` empty, visuals regenerated -- while the audio the
|
||||
generator produced is bit-for-bit intact.
|
||||
|
||||
So the pipeline can produce a file that **our own detector calls clean while an
|
||||
untouched Google-generated audio track sits inside it**. The mechanism is proven;
|
||||
whether that audio carries an audible-to-the-verifier SynthID mark is not, and only
|
||||
the oracle can say. If it does, this is a shipped defect rather than a quality
|
||||
question, and it outranks everything else in this document.
|
||||
|
||||
Worth recording alongside it: the Veo carrier itself returns `{}` from
|
||||
`get_ai_metadata`. The public sample carries no C2PA at all, which is exactly why it
|
||||
needed an oracle verdict in the first place.
|
||||
|
||||
## State when this work was parked (2026-08-06)
|
||||
|
||||
Landed in `main` through PR #78: the manifest-backed default pin, the
|
||||
scaling-factor gate and its single seam, the sweep loading through the engine's
|
||||
loader, `scripts/video_fidelity_probe.py` with its tests, the extended manifest
|
||||
schema, and this document. **The shipped operating point did not move**: it is
|
||||
still 512 px / 12 fps / `noise_std=0.15`.
|
||||
|
||||
A product requirement arrived late and reframes the target: output resolution,
|
||||
frame rate and bitrate must not be reduced. Resolution, frame rate and crf are
|
||||
parameters and can be preserved, but zero quality loss is not achievable by this
|
||||
mechanism at all -- removal works by rebuilding pixels, so a regeneration can never
|
||||
return its input. The honest goal is to preserve container properties and make the
|
||||
perceptual loss imperceptible, judged by DISTS and VMAF rather than PSNR. Two
|
||||
consequences follow: crf 18 on this path against crf 14 on the visible path is an
|
||||
unexplained asymmetry and a free quality win, and the oracle's 100 MB limit means a
|
||||
higher-quality profile is *harder* to certify than the one it replaces.
|
||||
|
||||
Nothing is blocked on analysis. Six oracle submissions were prepared and never run,
|
||||
because file upload to the verifier failed at the tool level:
|
||||
|
||||
1. A Lyria carrier and the pipeline's output from it, answering the audio question.
|
||||
2. A multiplexed control-plus-candidate file at the shipped geometry, validating
|
||||
whether the verifier reads a concatenated pair by time range. Verified locally
|
||||
frame by frame: half one is bit-identical to the control, half two to the
|
||||
candidate.
|
||||
3. Three more multiplexed pairs for 768, 1024 and 24 fps, worth three queries
|
||||
instead of six if the instrument validates.
|
||||
|
||||
Every generated artifact for those submissions has since been lost with its
|
||||
scratch directory, which is the intended lifecycle for generated media. Rebuilding
|
||||
costs one download and roughly 25 minutes of encoding; the Lyria track is the
|
||||
exception, since regenerating it would produce a different track, and the copies
|
||||
that survive are the only ones.
|
||||
|
||||
## The single most important unknown
|
||||
|
||||
**How much the 512 px downscale contributes to removal.** Every quality gain routes
|
||||
through this question, and the critics moved it from "probably free" to "direction
|
||||
unknown", which is exactly what makes it the highest-information experiment
|
||||
available.
|
||||
|
||||
## Ranked experiment program
|
||||
|
||||
Ranked by information per oracle query. The budget is roughly 10 checks per 24
|
||||
hours (MEASURED), so the program is paced by calendar, not by money.
|
||||
|
||||
### E0. The zero-oracle tier (0 submissions) - do this first
|
||||
|
||||
Not an experiment on the oracle; the precondition that makes everything else
|
||||
interpretable.
|
||||
|
||||
- `ffprobe` the carrier and record `source_width`, `source_height`, `source_fps`.
|
||||
The actual downscale factor of the existing rows is currently unrecoverable.
|
||||
- Extend the manifest schema: variant/control, source geometry, `vae`, `track`
|
||||
(audio|visual), verdict state in {DETECTED, NOT_DETECTED, INDETERMINATE,
|
||||
REFUSED}, verbatim verdict text, detected time range for the output,
|
||||
`session_id`, content stratum.
|
||||
- **Measure the `noise_std = 0.0` round trip locally** on the same carrier. This
|
||||
converts the inferred 27.2 dB ceiling into a measurement and bounds the whole
|
||||
noise axis for one GPU pass and zero oracle cost.
|
||||
- Freeze the current defaults' metrics on a fixed clip set, matching the
|
||||
record-then-diff rule in `.claude/rules/development.md`.
|
||||
- Determine whether the carrier has an audio track and whether it carries SynthID.
|
||||
|
||||
### E1. Instrument validation: multiplexing and a session anchor (1-2 submissions)
|
||||
|
||||
The verifier reports time ranges, so one file can carry several doses. A 24-second
|
||||
file of `[control 8 s | certified 0.15 8 s | control 8 s]` should read detected on
|
||||
the outer segments and not detected on the middle one, both halves already known
|
||||
from the manifest.
|
||||
|
||||
`encode_video_frames` requires matching dimensions and one frame rate, so
|
||||
multiplexing works **only along the `noise_std` axis**, not across geometries: one
|
||||
multiplexed file per geometric envelope.
|
||||
|
||||
Separately, submitting the existing detected file first in each session costs 10%
|
||||
of the quota, turns "the oracle may have changed" into a per-session gate, and
|
||||
measures the flip rate on a byte-identical file, a quantity every plan silently
|
||||
assumes is zero.
|
||||
|
||||
### E2. Resolution ladder at fixed dose - the decisive axis (2 submissions per rung)
|
||||
|
||||
A fixed value, not a ceiling. Rung 1 is `long_side = 768` with fps held at 12 so
|
||||
exactly one destruction axis moves. Jumping straight to 1920 is a bad first rung:
|
||||
a 3.75x jump makes a detected verdict uninformative about where the boundary lies.
|
||||
|
||||
- **Matched control:** 768 / 12 / `noise_std = 0.10`, expected DETECTED. It is
|
||||
strictly stronger than a plain re-encode control because it discharges the
|
||||
envelope, the VAE round trip, and a nonzero dose at once, and it diffs directly
|
||||
against the existing 512 row.
|
||||
- **Candidate:** 768 / 12 / 0.15.
|
||||
- **Positive:** the resolution axis is open; next rungs 1024 and then native with a
|
||||
clamp, 2 submissions each.
|
||||
- **Negative:** the downscale contributes to removal. This is the most valuable
|
||||
available negative: it kills the native-resolution proposals outright and turns
|
||||
the question into how much dose must be spent to buy resolution back, at the
|
||||
known exchange rate B = 2815.
|
||||
- Watch for a monotonicity violation: 768 / 0.10 reading NOT_DETECTED would
|
||||
overturn the critics' spectral argument.
|
||||
|
||||
### E3. Frame rate at fixed geometry and dose (2 submissions)
|
||||
|
||||
Control 512 / 24 / 0.10 expected DETECTED; candidate 512 / 24 / 0.15 expected
|
||||
not detected. The mechanism is provable from source, so the prior is high, but the
|
||||
only counter-mechanism - per-frame count aggregation - is probabilistic and n = 1
|
||||
on an 8-second clip does not measure it. Certify on the longest clip that fits the
|
||||
90 s / 100 MB limits, because the risk compounds with frame count.
|
||||
|
||||
Ranked below E2 because the outcome is nearly predetermined and the prize (judder)
|
||||
is smaller than the geometry prize. It is the safest bet if a guaranteed win is
|
||||
wanted.
|
||||
|
||||
### E4. `noise_std = 0.0` as an oracle probe (1 submission) - deliberately low rank
|
||||
|
||||
The local half of this probe is the cheapest high-information measurement in the
|
||||
program and already sits in E0 at zero oracle cost. The **oracle** half ranks low:
|
||||
it buys no quality by construction, and its likely DETECTED outcome is nearly
|
||||
deducible from the 0.10 row under monotonicity. Its one real value is that
|
||||
NOT_DETECTED here would be non-monotone against 0.10 and would refute the
|
||||
assumption underneath every ladder and bisection in this program. Worth one query,
|
||||
after E2 and E3.
|
||||
|
||||
### E5. Stratified certification of the surviving operating point (15-30 submissions)
|
||||
|
||||
Only after E2 and E3 produce a winner. Zero detections across at least 15
|
||||
stratified carriers (at least 3 per stratum: face closeup, fast motion, flat sky,
|
||||
dark night, text overlay) bounds the failure rate at 18.1%; 30 carriers bound it at
|
||||
9.5%. Add the project's 1.5x margin convention, at least 2 seeds, at least 2
|
||||
sessions, and per-track verdict recording.
|
||||
|
||||
For the flat-sky stratum, check the source reads detected first: Google documents a
|
||||
"not enough details to watermark" state, so a clean reading there may say nothing
|
||||
about removal.
|
||||
|
||||
## Surviving change candidates
|
||||
|
||||
**S1. Fix the metric's reference frame (0 oracle submissions).** Add
|
||||
`source_psnr_db` (candidate upscaled back to native against the untouched source
|
||||
frame) **and** a separate post-mux pass that decodes the delivered file. The
|
||||
critics killed the original claim that this would expose crf 18: the candidate is
|
||||
captured at `:423`, before `frame_pipe.write` at `:430`, so no in-loop metric can
|
||||
see the codec. Only the post-mux pass covers resize, decimation, crf, and mux
|
||||
together. Use `INTER_AREA` or a fixed analytic size for the upscale so the CPU
|
||||
resize does not dominate the loop, and update both other consumers of the
|
||||
generator in the same change (`read_sampled_frames` at `:188-209`, and
|
||||
`scripts/video_synthid_sweep.py:147` where `np.stack(frames)` must keep receiving
|
||||
resized frames). Do not redefine the existing `psnr_db`: the two manifest rows stay
|
||||
comparable to each other only while that field keeps its exact current meaning.
|
||||
|
||||
**S2. Frame rate as a fixed certified value (after E3).**
|
||||
`DEFAULT_VIDEO_SYNTHID_FPS` 12.0 to 24.0, keeping `min(fps, source_fps)`. A
|
||||
ceiling of 60.0 was **rejected**: it makes the delivered operating point a function
|
||||
of the user's source, so one sample from a family gets certified while the rest
|
||||
ship uncertified, and the risk direction is unfavorable. Port the VFR/PTS bridge
|
||||
(`probe_video_timestamps`, `timestamped_input`, currently only in
|
||||
`video_visible.py`): at native frame rate the output stops reading as a proxy, and
|
||||
silent CFR-ification becomes a master-quality defect. Add `fps` and `long_side`
|
||||
pins next to `tests/test_video_invisible.py:285`, tied to the certifying manifest
|
||||
row.
|
||||
|
||||
**S3. Clamp and align `_fit_size` (0 oracle submissions, with a caveat).**
|
||||
`scale = min(1.0, long_side / max(width, height))` at `:88`; align the long side
|
||||
and derive the short side from the true aspect, rounding to the nearest multiple of
|
||||
8. The pinned `_fit_size(1280, 720, 512) == (512, 288)` survives. The caveat: the
|
||||
clamp is an **uncertified operator change for a whole class of users**, since a
|
||||
320x240 source is upscaled to 512x384 today and would run at 320x240 afterwards,
|
||||
and neither has been tested. Ship it as its own documented change, not as a
|
||||
drive-by fix.
|
||||
|
||||
**S4. Extend the manifest schema and oracle protocol (0 submissions).** A
|
||||
precondition for the whole program. Record the verbatim verdict text, because an
|
||||
unclear state logged as not detected is exactly the silent regression the protocol
|
||||
exists to prevent.
|
||||
|
||||
**S5. A sigma normalization contract (0 submissions, applies to the current path).**
|
||||
No VAE swap survived, but two elements are real risks today: `vae.config.scaling_factor`
|
||||
cannot be trusted, so log `latents.std()` at `:276` and assert the effective
|
||||
scaling factor at load; and any cross-model sigma transfer must match on the **RMS
|
||||
of the decoded pixel perturbation**, not on latent units.
|
||||
|
||||
**S6. Additive texture masking - contingency on an E2 negative only.** Not energy
|
||||
preserving: the certified sigma stays as a floor everywhere and the mask only adds
|
||||
on top in textured regions. The original energy-preserving form was refuted - the
|
||||
mechanism normalizes and then clips, which breaks the claimed invariant by a
|
||||
content-dependent amount, and the proposed unit test asserted the invariant before
|
||||
the clip and would have stayed green. Energy preservation is also unsafe in
|
||||
principle: the detector does not need a full frame, and the verifier reports
|
||||
per-segment, so a flat region perturbed only at the floor is a crop carrying nearly
|
||||
the full carrier. The salvaged version has **no standalone PSNR win**; its only
|
||||
value is freeing distortion budget to spend on resolution if E2 shows the dose must
|
||||
rise. It additionally needs motion compensation for the mask (flow is computed at
|
||||
`:442`, after the decode at `:423`, so the loop must be restructured) and one
|
||||
deliberately bad candidate so `temporal_residual_ratio` acquires a known failing
|
||||
value: it has only ever been observed in [1.0072, 1.0578], so "the ratio looks
|
||||
fine" is currently an unfalsified claim.
|
||||
|
||||
## Refuted proposals - do not resurrect
|
||||
|
||||
- **Carrier-scale, resolution-invariant noise field.** "Carrier scale" is set by
|
||||
our VAE's latent pitch, not the carrier's. The motivating arithmetic treats a
|
||||
field on the latent grid as white at pixel resolution and is wrong by exactly the
|
||||
square of the scale factor: white noise on a 240-wide latent already spans
|
||||
0-120 cycles per frame, entirely inside the band the decoder reproduces. The
|
||||
construction actually band-limits the perturbation from 0-120 to 0-32 cycles,
|
||||
making it smoother - the first thing any spread-spectrum extractor's content
|
||||
suppression removes - while moving its energy toward the peak of the contrast
|
||||
sensitivity function, making it more visible. Bilinear upsampling is also
|
||||
heteroscedastic, and global renormalization by `noise.std()` stamps a visible
|
||||
amplitude lattice rather than fixing it.
|
||||
- **Native resolution plus tiling plus a carrier-scale field.** Inherits the
|
||||
arithmetic above, concedes that native processing destroys less carrier without
|
||||
proposing a replacement mechanism, and bundles four destruction axes into one
|
||||
candidate on a 4-query budget so a detected verdict is undecomposable. Its cost
|
||||
model was also wrong by roughly 10x. The salvageable parts moved into S1 and S3.
|
||||
- **`AutoencoderKLTemporalDecoder`.** Refuted independently by three critics.
|
||||
Removal is a property of the encode-decode composition, so a strictly more
|
||||
faithful decoder preserves more carrier at the same sigma. The class implements
|
||||
neither slicing nor tiling, so `enable_slicing()` at `:135` raises
|
||||
`NotImplementedError` and the documented bounded-memory property dies. Its
|
||||
temporal receptive field needs windows of tens of frames, its license carries a
|
||||
revenue restriction incompatible with an Apache-2.0 default, and its own
|
||||
published table shows a **worse** FID (9.17 against 7.61).
|
||||
- **Wan or another temporal video VAE.** Hard refusal in code: `AutoencoderKLWan._encode`
|
||||
uses `iter_ = 1 + (num_frame - 1) // 4`, so at the shipped `batch_size = 4` only
|
||||
frame 0 is encoded and a 4-frame batch decodes back to 1, which makes
|
||||
`zip(..., strict=True)` at `:429` raise. The proposed remedy of carrying the
|
||||
causal feature cache across chunks is impossible through the public API, since
|
||||
`clear_cache()` runs on both entry and exit of `_encode` and `_decode`.
|
||||
`float(vae.config.scaling_factor)` at `:270` and `:292` also raises, because Wan
|
||||
exposes `latents_mean`/`latents_std` instead. Logically, feeding 4:1 temporal
|
||||
compression frames 83 ms apart either reconstructs well (carrier preserved, no
|
||||
removal gain) or hallucinates (no quality prize); the two claims are mutually
|
||||
exclusive.
|
||||
- **A 16-channel f8 VAE.** Self-refuted and confirmed by the critics. Quadrupling
|
||||
latent channels roughly doubles L in Zhao's theorem, so sigma must roughly double
|
||||
to hold removal, and the VAE frontier in the UnMarker results is monotone with no
|
||||
published point where a more faithful autoencoder removes **more**. Predicted net
|
||||
loss of 2-7.5 dB by its own arithmetic. Its config also carries
|
||||
`scaling_factor = 0.2614`, so an unchanged `noise_std = 0.15` is about 30% weaker
|
||||
in raw latent units. Only the normalization contract survived, as S5.
|
||||
- **A per-channel luma/chroma probe.** The arms match in absolute latent units but
|
||||
not in relative dose per channel, since `scaling_factor` normalizes the aggregate
|
||||
latent, so the very reading the probe exists for is confounded with dose. The
|
||||
chroma arm is additionally subsampled by the yuv420p encode, which no current
|
||||
metric sees. Maximum prize under 1 dB by its own fit.
|
||||
- **Post-processing with unsharp plus grain, as a shipped stage.** The
|
||||
carrier-reimport argument is sound and was verified line by line: `unsharp_mask`
|
||||
reads only its own argument ([`humanizer.py:79`](../src/remove_ai_watermarks/humanizer.py)),
|
||||
and `adaptive_polish` touches the source only through one float. But reimport was
|
||||
never the binding risk:
|
||||
`cv2.addWeighted(img_f, 1.0 + amount, blurred, -amount, 0.0)` at
|
||||
`_ADAPTIVE_MAX_UNSHARP = 1.0` multiplies the **surviving carrier residue** by up
|
||||
to 2x as the terminal operation before encoding, and whether that re-arms a given
|
||||
file depends on that file's unobservable margin. With no local decoder and only a
|
||||
sampled oracle the stage is structurally uncertifiable, not merely expensive to
|
||||
certify. The only rescue is moving sharpening **above** the VAE stage so the
|
||||
removal operator stays terminal. Grain, which monotonically lowers detector SNR,
|
||||
is the safe half and can be separated.
|
||||
|
||||
## Measuring quality properly
|
||||
|
||||
The current `psnr_db` cannot show the improvement this work exists to produce.
|
||||
Minimum upgrade:
|
||||
|
||||
1. **`source_psnr_db`** - candidate upscaled back to native against the untouched
|
||||
source frame. Good for ranking configurations against a common reference,
|
||||
dominated by unrecoverable high-frequency content, so not a measure of what the
|
||||
VAE costs.
|
||||
2. **A post-mux end-to-end pass** - decode the delivered file after
|
||||
`mux_encoded_video` (`:456`) and compare against the source. The only measure
|
||||
covering resize, decimation, crf, and mux together. Keep it out of the streaming
|
||||
loop so `test_stream_batches_consumes_only_one_batch_ahead` stays valid.
|
||||
Implemented as `scripts/video_fidelity_probe.py`, which streams, reports the
|
||||
delivered file's bitrate, and shares the engine's frame-selection rule rather
|
||||
than copying it.
|
||||
3. **DISTS** (arXiv:2004.07728), built to tolerate texture resampling - exactly
|
||||
what the VAE does to foliage, skin, and grass, and exactly what PSNR and LPIPS
|
||||
punish even when the result is perceptually equivalent. It separates "the VAE
|
||||
resampled the grass" from "the VAE destroyed an edge".
|
||||
4. **VMAF** through `ffmpeg` `libvmaf`, cheapest to add, whose ADM/DLM feature
|
||||
names this exact complaint. Its temporal term is only a mean absolute luma
|
||||
difference between neighboring frames, so it is not a flicker detector: keep the
|
||||
motion-compensated ratio.
|
||||
5. **Encoded file size as a third axis.** Every current metric is taken before the
|
||||
pipe, so a bitrate explosion currently reads as "quality did not suffer". This
|
||||
matters most for any perturbation that varies frame to frame: at fixed crf it
|
||||
raises bitrate rather than lowering quality.
|
||||
6. **`temporal_residual_ratio` repairs**: add a p95 across frames next to the mean
|
||||
so localized flicker stops being averaged away, add a long-horizon term against
|
||||
the first frame, and replace `max(temporal_baseline, 1e-6)` with an explicit
|
||||
undefined result on static shots.
|
||||
7. **A no-reference metric** (DOVER or FAST-VQA) for the case where no
|
||||
full-reference measure can compare a 512/12 output against a 1024/24 output on
|
||||
one scale. Relative ranking within a sweep only; absolute values are
|
||||
uncalibrated for this artifact class.
|
||||
|
||||
Nothing local measures removal. `remove_video_invisible` checks `get_ai_metadata`
|
||||
on the output, which is metadata, not pixels. Only a manifest row counts.
|
||||
|
||||
## Risks
|
||||
|
||||
**Error asymmetry is the governing constraint.** Shipping a leak means a user
|
||||
receives a watermarked file believing it is clean, with no local pixel decoder to
|
||||
catch it and no feedback path that would surface it. Staying conservative means a
|
||||
user receives 512 px / 12 fps, a cost that is visible, bounded, and reversible with
|
||||
an explicit flag. A symmetric test is therefore inadmissible, and the burden of
|
||||
proof sits entirely on the new default.
|
||||
|
||||
**n = 1.** The current default is certified by one success on one carrier with one
|
||||
seed. Any quality change inherits that weakness and must not deepen it.
|
||||
|
||||
**Content and seed dependence** is MEASURED on this project's image branch:
|
||||
survivors switch by content type, and near the threshold the same input flipped
|
||||
between runs on seed alone.
|
||||
|
||||
**Oracle instability.** Three verdict states, not two. Per-segment reporting.
|
||||
Separate audio and visual tracks - and the pipeline byte-copies audio, so a Veo 3
|
||||
clip with generated sound leaves with its audio SynthID intact, and the manifest
|
||||
cannot even record which track the 2026-07-31 negative referred to. That is
|
||||
potentially a shipped product hole, not only an experimental confound. Session
|
||||
drift is currently unfalsifiable.
|
||||
|
||||
**Observability narrows exactly where quality rises.** At fixed crf the bitrate
|
||||
grows with pixels per second, so the maximum duration fitting under 100 MB falls as
|
||||
fps and resolution rise. The configuration users actually receive becomes **less**
|
||||
observable than the one it replaces. That is a permanent property, not an
|
||||
inconvenience.
|
||||
|
||||
**Ceilings instead of fixed values.** Any ceiling makes the delivered operating
|
||||
point a function of the user's source, and with it the perturbation's cycles per
|
||||
frame - the quantity carrying the certified margin. Ship fixed values.
|
||||
|
||||
**Documentation and test surface.** Only `DEFAULT_VIDEO_SYNTHID_NOISE_STD` is
|
||||
pinned. Moving `long_side` or `fps` touches hardcoded numbers in `README.md`,
|
||||
`docs/known-limitations.md`, `docs/cli.md`, `docs/python-api.md`,
|
||||
`docs/module-internals.md`, `docs/synthid.md`, and `docs/verification-plan.md`.
|
||||
Drift-prone operational numbers should not live in seven places of prose; their
|
||||
source of truth is the manifest.
|
||||
|
||||
**Silent configuration drift.** `sd-vae-ft-mse/config.json` carries no
|
||||
`scaling_factor`, so 0.18215 is a class default under an upper-unbounded
|
||||
`diffusers>=0.38.0` while `maintain.sh` runs `uv-outdated`. A library bump can move
|
||||
the certified operating point with a green suite.
|
||||
|
||||
**Missing guards that get worse at native resolution.** There is no HDR or >8-bit
|
||||
rejection and no VFR/PTS bridge on the invisible path. A 512 px output plainly
|
||||
reads as a proxy; a native-resolution output reads as a master, and the cost of
|
||||
silently flattening a 10-bit PQ source to 8-bit SDR rises accordingly.
|
||||
|
||||
**Do not bundle axes.** Raising `long_side`, raising `fps`, and lowering crf all
|
||||
reduce total destruction and all require recertification. The metric reference-frame
|
||||
fix is the only exception, because it changes the measurement rather than the
|
||||
pixels. Do not put a crf 18 to 14 change in the same candidate as a resolution
|
||||
change.
|
||||
@@ -10,7 +10,7 @@ Who embeds what, and whether it is locally detectable (so we know which gaps are
|
||||
**Root cause and external confirmation (deep-research 2026-06-19, adversarially verified).** This is the SCHEME's ceiling, not our usage — there is no better decoder to adopt. The imwatermark maintainers state verbatim (both the ShieldMnt and Stability-AI READMEs) that the algorithm "cannot guarantee to decode the original watermarks 100% accurately even though we don't apply any attack." Independent measurement (WMAdapter, arXiv:2406.08337 Table 2) puts dwtDct at only **~0.79 bit accuracy on CLEAN images (~38/48 bits — already below our 44 gate)**, collapsing to ~0.50 (chance) under crop/JPEG. Two code-verified + locally-reproduced mechanisms drive the content-dependent failures: (1) the decoder reads each bit as the **highest-magnitude DCT coefficient per block**, so any content coefficient exceeding the encoded target flips the bit; (2) the default embed is in the **YUV chroma channel, which 8-bit-clamps on white/bright pixels** (a +36 chroma delta survives a white-fill round-trip as only +4, ~89% loss) — this is the mechanism behind the bright-flat / minimalist failures and the all-ones degenerate decode. No maintained fork or detector decodes this scheme reliably: the WAVES benchmark (arXiv:2401.08573) relegates DWT-DCT to supplementary appendix G.5 and targets Stable Signature / Tree-Ring / StegaStamp instead; learned encoder/decoder schemes reach ~0.98-0.99 clean but are a DIFFERENT watermark class (not what SDXL/FLUX stamp). `dwtDctSvd` does not help (SDXL embeds `dwtDct`; dwtDctSvd cannot decode it, and its clean accuracy ~0.72 is lower). **Authoritative conclusion: the open DWT-DCT mark cannot be turned from positive-only into a reliable real-world detector; keep it positive-only and rely on C2PA.** (Refuted along the way: that the library is unmaintained, and that it is robust to JPEG but only fails on geometric attacks — both did not survive verification.)
|
||||
|
||||
Consequence for the FLUX hosted-output question (BFL Playground, FLUX.2 [pro] + FLUX.1 [dev], 2026-06-19): all samples carry the signed C2PA manifest (issuer "Black Forest Labs"); the open DWT-DCT decode returned `None`, but every available FLUX carrier (textured fox AND a minimalist-flat generation) failed the positive control (28/48), so the detector is blind on them and **whether BFL hosted output embeds the open pixel watermark is UNRESOLVED** (an earlier note here wrongly asserted it absent — overstated; a later note blamed "high texture" — also wrong, flat carriers fail too). What IS established: C2PA is the reliable FLUX identifier; the `_BITS_48` pattern is correct (round-trips on chatgpt/firefly/random). Resolving the hosted question needs a hosted FLUX carrier that first passes a >=44/48 positive control, which neither a textured nor a flat prompt produced — low priority (the open mark is only a stripped-metadata fallback).
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Copilot + Designer; Bing Image Creator collected 2026-05-24 still signs as "Microsoft" and now runs **MAI-Image**, NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
|
||||
**Samsung Galaxy AI** signs supported edits with C2PA and may carry the
|
||||
proprietary `genAIType` marker. The registered visible detector covers the
|
||||
@@ -22,8 +22,15 @@ icon-only variants need separate calibrated silhouettes.
|
||||
|
||||
**Black Forest Labs (FLUX)** API output signs C2PA: `claim_generator_info "Black Forest Labs API"` + a `c2pa.ai_generated_content` assertion + `trainedAlgorithmicMedia` (issuer `b"Black Forest Labs"` added to `C2PA_ISSUERS`, platform "Black Forest Labs (FLUX)").
|
||||
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
|
||||
Some applications sign C2PA through an upstream model or infrastructure
|
||||
provider. For AI claims, exact product mappings in `claim_generator` therefore
|
||||
take precedence over issuer attribution. Supported mappings include Higgsfield
|
||||
AI, Topaz Labs Image API, and TikTok Ad Creative Toolbox; an unknown claim
|
||||
generator still falls back to the certificate issuer.
|
||||
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
|
||||
- **EXIF/XMP/PNG-text generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal). Additional verified generator stamps include **NovelAI** (`Software`, `Source`, and `Title` PNG text chunks), **Reve** (`Software` or XMP `CreatorTool` = `reve.com`), and **Aphrodite AI** (`Make` or `Software` = `Aphrodite AI`).
|
||||
- **App-export provenance and AIGC JSON:** supported ByteDance-family exports can place a JSON object in EXIF `ImageDescription` or `UserComment`, independently of C2PA or TC260. Exact `product` values for Doubao, Xinghui, and Dreamina are removable product provenance, but do not alone prove that the pixels were generated. Dreamina additionally requires `exportType=generation` for that verdict. A nested Aweme `aigc_type=1` or private ByteDance `aigc_label_type=1` / `2` is an AIGC disclosure; `0` is inconclusive and can occur on a Dreamina generation export. Plain Aweme, retouch, and `lv` exports are preserved. The lower-case private field is deliberately not interpreted as the normative TC260 `Label`, whose values `1` / `2` / `3` mean generated / possibly generated / suspected generated under [GB 45438-2025](https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf).
|
||||
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
|
||||
|
||||
Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC** — only EXIF `Artist` = a UUID and EXIF `ImageDescription` = `Signature: <base64>` (a crypto signature, unverifiable locally without xAI's public key). This empirically kills the earlier unverified "xAI signs C2PA as xAI" lead — xAI is not even a C2PA member. `exif_generator` misses it (neither field holds an `AI_GENERATOR_TOKENS` token), so a dedicated detector `xai_signature(path)` matches the pair (`ImageDescription ~ ^Signature: [A-Za-z0-9+/=]{64,}` AND UUID `Artist`); wired into `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify` (signal `xai_signature`, platform "xAI (Grok / Aurora)").
|
||||
@@ -36,9 +43,9 @@ AI generator values while retaining unrelated camera and editor EXIF. The
|
||||
shared `xai_signature_pair` helper is the single source of truth for the
|
||||
pair. On the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
|
||||
in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`).
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). The label says only "this is AI", but its `ContentProducer` names the signing entity — `001` + `1` + an 18-char Unified Social Credit Code + a 5-digit product suffix, normalized by `metadata.uscc_of`, or for a few generators a bare product name. `KnownMark.tc260_producer_codes` maps the codes settled per vendor by `scripts/vendor_cohort_harvest.py` to registry mark keys, so an AIGC image relaxes the detector of the vendor it actually carries rather than a guessed pair; an unmapped or absent producer falls back to ByteDance's two products. A code identifies a legal entity, not necessarily one brand, so a hosting or aggregating platform that signs for several apps is a recall bet rather than a proof.
|
||||
|
||||
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item. The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
|
||||
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item; Doubao's iOS export additionally writes two QuickTime-form variants (a bare `meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list whose `ilst` data items carry the JSON with no key name) — both verified on retained carriers (2026-08-17). The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
|
||||
|
||||
Native MKV/WebM is a sixth serialization. TC260-PG-20257A stores
|
||||
`TagName=AIGC` and the raw JSON `TagString` in
|
||||
@@ -58,7 +65,8 @@ payloads. Removal remuxes either container through ffmpeg with stream copy.
|
||||
DWT-DCT decoder only applies when the producing pipeline actually ran its
|
||||
encoder and the carrier remains decodable.
|
||||
- **Invisible but NOT locally detectable (proprietary, API/oracle only — same wall as SynthID):** Amazon Titan Image Generator + Nova Canvas (Bedrock `DetectGeneratedContent` API), Kakao (new SynthID image adopter, May 2026), NVIDIA Cosmos (SynthID video). No local detector possible; treat like SynthID.
|
||||
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest (Digimarc/Imatag/Steg.AI/...) stay name-only (proprietary decoders).
|
||||
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, preserve the structured assertion's signed `value`, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest stay name-only unless a compatible decoder is independently verified.
|
||||
- **Microsoft Paint and Photos InvisMark (reverse-engineered 2026-08-20):** Paint receives a per-generation GUID from remote prompt moderation, embeds it into locally generated pixels, and records the same value in `c2pa.soft-binding` under `com.microsoft.invismark.1`. The C2PA soft-binding registry independently identifies that algorithm as Microsoft Responsible AI InvisMark for image and video. Paint's 144-bit writer framing does not match the public repository's 100-bit pretrained checkpoint interface, so compatibility is not assumed. The parser reports the signed identifier; there is no validated local pixel decoder. Microsoft's external Content Provenance Detection API is the removal oracle because it reports `Watermark` separately from `C2PA`; a metadata-stripped, pixel-identical control must remain watermark-positive before an output-negative result is attributed to pixel removal. Sources: https://xusheng.dev/posts/reversing/mspaint_invisible_watermark/main/, https://github.com/c2pa-org/softbinding-algorithm-list/blob/main/softbinding-algorithm-list.json, and https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection
|
||||
- **Built in the dated batch:** soft-binding vendor detection, IPTC Photo
|
||||
Metadata AI-disclosure fields, C2PA detection and stripping for supported
|
||||
ISOBMFF video, the optional Adobe TrustMark decoder, and temporally stabilized
|
||||
@@ -68,7 +76,7 @@ payloads. Removal remuxes either container through ffmpeg with stream copy.
|
||||
Metadata stripping for supported audio containers is a separate implemented
|
||||
path.
|
||||
|
||||
**Box detection window — now handled (v0.6.8):** detection no longer relies on a fixed first-MB read. `metadata.scan_head(path, size)` reads the first `size` bytes and, for ISOBMFF, appends the payloads of late provenance boxes found by `isobmff.scan_c2pa_region` (a file-seeking top-level box walker that skips past `mdat` by size without reading it), so a C2PA/AIGC/IPTC manifest placed AFTER a large `mdat` in a streaming/non-faststart MP4 is now caught. Every C2PA/marker byte scan (`has_ai_metadata`, `aigc_label`, `iptc_ai_system`, `synthid_source`, `exif_generator` XMP, `get_ai_metadata` soft-binding, and `identify`) goes through `scan_head`; it is behavior-neutral for non-ISOBMFF inputs (exactly `f.read(size)`).
|
||||
**Box detection window — now handled (v0.6.8):** detection no longer relies on a fixed first-MB read. `metadata.scan_head(path, size)` reads the first `size` bytes and, for ISOBMFF, appends the payloads of late provenance boxes found by `isobmff.scan_c2pa_region` (a file-seeking top-level box walker that skips past `mdat` by size without reading it), so a C2PA/AIGC/IPTC manifest placed AFTER a large `mdat` in a streaming/non-faststart MP4 is now caught. Every C2PA/marker byte scan (`has_ai_metadata`, `aigc_label`, `iptc_ai_system`, `synthid_source`, `exif_generator` XMP, `get_ai_metadata` soft-binding, and `identify`) goes through `scan_head`; for PNG it likewise appends the payloads of `tEXt` / `iTXt` / `zTXt` / `eXIf` / `iCCP` chunks that start beyond the window (`_png_late_metadata`, seeking past `IDAT`), which is how a TC260 AIGC label appended after the pixel stream is caught; for WebP it appends the `EXIF` / `XMP ` / `ICCP` / `C2PA` chunks past the window (`_riff_late_metadata`, stepping over the coded image), which is how an IPTC "Made with AI" tag stored after the pixels is caught; and for a file at least `size` bytes long it finally appends the metadata text the decoder reaches but a raw read cannot (`_decoder_visible_text`), which covers a compressed PNG `zTXt` packet no byte scan can spell; for any file that fits inside `size`, it is exactly `f.read(size)`.
|
||||
|
||||
Native TC260 MP4/MOV tags do not live in those top-level provenance boxes.
|
||||
`tc260_aigc_payloads` separately seeks through `moov.udta.meta.keys/ilst`, so
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
schema_version: 1
|
||||
|
||||
context:
|
||||
version: "0.22.0"
|
||||
python_min: "3.10"
|
||||
|
||||
package:
|
||||
name: remove-ai-watermarks
|
||||
version: ${{ version }}
|
||||
|
||||
source:
|
||||
url: https://pypi.org/packages/source/r/remove-ai-watermarks/remove_ai_watermarks-${{ version }}.tar.gz
|
||||
sha256: 00549927083c2b60444dd8404830c7333899a9345aacc41d43b21a77c63b8dd0
|
||||
|
||||
build:
|
||||
noarch: python
|
||||
number: 0
|
||||
script: python -m pip install . -vv --no-deps --no-build-isolation
|
||||
|
||||
requirements:
|
||||
host:
|
||||
- python ${{ python_min }}.*
|
||||
- pip
|
||||
- hatchling
|
||||
run:
|
||||
- python >=${{ python_min }}
|
||||
- pillow >=10.0.0
|
||||
- piexif >=1.1.3
|
||||
- click >=8.0.0
|
||||
- python-dotenv >=1.0.0
|
||||
# c2pa-python is a default PyPI dependency but is not packaged on conda-forge.
|
||||
# The guarded import falls back to the built-in C2PA byte scanner when it is
|
||||
# absent. Add it here once a c2pa-python feedstock exists.
|
||||
|
||||
tests:
|
||||
- python:
|
||||
imports:
|
||||
- remove_ai_watermarks
|
||||
- remove_ai_watermarks.identify
|
||||
- remove_ai_watermarks.metadata
|
||||
python_version: ${{ python_min }}.*
|
||||
# pip_check defaults to true; disable it explicitly. `pip check` fails on
|
||||
# the conda-forge piexif `py_2` build, whose stale 2019 metadata pip reads
|
||||
# as "piexif 1.1.3 is not supported on this platform" -- even though the
|
||||
# package installs, imports, and works. The imports test verifies loading.
|
||||
pip_check: false
|
||||
|
||||
about:
|
||||
homepage: https://github.com/wiltodelta/remove-ai-watermarks
|
||||
summary: Inspect and strip AI provenance metadata from media
|
||||
description: |
|
||||
Inspect and strip AI-provenance metadata (C2PA, EXIF, IPTC, and PNG text
|
||||
chunks) from images and supported video containers. Optional pip extras add
|
||||
visible watermark removal, video processing, SynthID diffusion removal, and
|
||||
additional invisible-watermark detectors.
|
||||
license: Apache-2.0
|
||||
license_file: LICENSE
|
||||
repository: https://github.com/wiltodelta/remove-ai-watermarks
|
||||
|
||||
extra:
|
||||
recipe-maintainers:
|
||||
- wiltodelta
|
||||
+40
-53
@@ -1,9 +1,13 @@
|
||||
[project]
|
||||
# This name is the published PyPI distribution, not a local label: changing it
|
||||
# strands every `pip install remove-ai-watermarks` and every pinned dependency
|
||||
# on it, including the ComfyUI node package. The console script below carries
|
||||
# the same weight, since users have it on PATH.
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.23.0"
|
||||
version = "0.30.0"
|
||||
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10.1"
|
||||
requires-python = ">=3.11,<3.15"
|
||||
license = {text = "Apache-2.0"}
|
||||
keywords = [
|
||||
"ai-watermark",
|
||||
@@ -33,10 +37,10 @@ classifiers = [
|
||||
"License :: OSI Approved :: Apache Software License",
|
||||
"Operating System :: OS Independent",
|
||||
"Programming Language :: Python :: 3 :: Only",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Programming Language :: Python :: 3.14",
|
||||
"Topic :: Multimedia :: Graphics",
|
||||
"Topic :: Multimedia :: Graphics :: Graphics Conversion",
|
||||
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
||||
@@ -60,7 +64,10 @@ dependencies = [
|
||||
|
||||
[project.optional-dependencies]
|
||||
pixels = [
|
||||
"numpy>=1.24.0",
|
||||
"numpy>=1.24.0; python_version < '3.13'",
|
||||
# NumPy 1.x has no CPython 3.13+ wheels. Requiring 2.x prevents installers
|
||||
# from attempting an unsupported source build on the newer Python lines.
|
||||
"numpy>=2; python_version >= '3.13'",
|
||||
"opencv-python-headless>=4.8.0",
|
||||
]
|
||||
# Optional HEIC/AVIF pixel decode. Metadata scanning handles these containers
|
||||
@@ -70,12 +77,10 @@ heif = [
|
||||
]
|
||||
visible = ["remove-ai-watermarks[pixels]"]
|
||||
# Video visible removal uses the shared pixel runtime. PyAV packetizes processed
|
||||
# VFR frames with explicit PTS before system ffmpeg encodes them. PyAV 18 requires
|
||||
# Python 3.11; the 16.x wheel line still covers Python 3.10.
|
||||
# VFR frames with explicit PTS before system ffmpeg encodes them.
|
||||
video = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
"av>=16,<17; python_version < '3.11'",
|
||||
"av>=18,<19; python_version >= '3.11'",
|
||||
"av>=18,<19",
|
||||
]
|
||||
# Open DWT-DCT watermarks used by Stable Diffusion / SDXL / FLUX. The in-tree
|
||||
# decoder avoids the upstream invisible-watermark package's mandatory torch and
|
||||
@@ -86,16 +91,11 @@ detect = [
|
||||
]
|
||||
diffusion = [
|
||||
"remove-ai-watermarks[pixels]",
|
||||
# A CUDA-enabled torch build is required: invisible-watermark removal has no
|
||||
# CPU, MPS or XPU path. The default PyPI wheel carries CUDA on Linux/Windows;
|
||||
# on macOS there is no CUDA build and this extra installs only for the
|
||||
# non-diffusion imports it shares.
|
||||
"torch>=2.0.0",
|
||||
# The default PyPI torch wheel is a CPU/CUDA build. To drive an Intel GPU
|
||||
# (Arc / Data Center) via ``--device xpu`` you need an XPU-enabled torch
|
||||
# from PyTorch's XPU wheel index (Linux/Windows only -- there is no macOS
|
||||
# XPU build). Install that build first, then this extra (torch is then
|
||||
# already satisfied and won't be re-pulled):
|
||||
# pip install torch --index-url https://download.pytorch.org/whl/xpu
|
||||
# pip install 'remove-ai-watermarks[diffusion]'
|
||||
# uv users can target the ``pytorch-xpu`` index declared under [tool.uv]:
|
||||
# uv pip install torch --index-url https://download.pytorch.org/whl/xpu
|
||||
"diffusers>=0.38.0",
|
||||
# diffusers 0.38's auto-pipeline registry imports ``Qwen3VLForConditional
|
||||
# Generation`` (its ``nucleusmoe_image`` pipeline), which only exists in
|
||||
@@ -120,12 +120,27 @@ qwen-zimage = [
|
||||
"diffsynth>=2.0.17,<3",
|
||||
"torchvision>=0.20.0",
|
||||
]
|
||||
# Opt-in verified-text reconstruction over qwen-zimage. LaMa removes the changed
|
||||
# candidate glyphs before exact Qwen-VAE cores are composited back.
|
||||
text-restoration = [
|
||||
"remove-ai-watermarks[qwen-zimage,lama]",
|
||||
]
|
||||
# Draft-only OCR proposals for verified-text manifests. CPU, no torch:
|
||||
# PaddleOCR detection plus three script-chosen recognition engines. The models
|
||||
# download on first use and are cached by huggingface_hub; never bundled.
|
||||
text-draft = [
|
||||
"paddleocr>=3.3.3",
|
||||
"paddlepaddle>=3.0.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Adobe TrustMark decoder -- the open, keyless watermark behind Adobe Durable
|
||||
# Content Credentials (soft-binding alg ``com.adobe.trustmark.P``). Optional
|
||||
# because it pulls torch and downloads model weights on first use. identify()
|
||||
# guards the import and skips the TrustMark signal when absent.
|
||||
trustmark = [
|
||||
"trustmark>=0.8.0",
|
||||
# TrustMark requires NumPy <2, whose last release has no CPython 3.13+
|
||||
# wheels. Keep this feature installable only on its real runtime range.
|
||||
"trustmark>=0.8.0; python_version < '3.13'",
|
||||
]
|
||||
# Universal region eraser backend -- big-LaMa via onnxruntime (Carve/LaMa-ONNX,
|
||||
# Apache-2.0). CPU, no torch. Model (~200 MB) is downloaded on first use and
|
||||
@@ -133,10 +148,7 @@ trustmark = [
|
||||
# eraser backend needs none of this.
|
||||
lama = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
# ONNX Runtime 1.24 dropped CPython 3.10 wheels; keep the project's
|
||||
# supported 3.10 line on the last compatible release series.
|
||||
"onnxruntime>=1.16.0,<1.24; python_version < '3.11'",
|
||||
"onnxruntime>=1.16.0; python_version >= '3.11'",
|
||||
"onnxruntime>=1.24.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Lightweight inpaint backend -- MI-GAN via onnxruntime (andraniksargsyan/migan,
|
||||
@@ -146,23 +158,9 @@ lama = [
|
||||
# LaMa, the quality-first `auto` choice, is too large. Same runtime as `lama`.
|
||||
migan = [
|
||||
"remove-ai-watermarks[visible]",
|
||||
"onnxruntime>=1.16.0,<1.24; python_version < '3.11'",
|
||||
"onnxruntime>=1.16.0; python_version >= '3.11'",
|
||||
"onnxruntime>=1.24.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Optional pre-diffusion super-resolution for small inputs (Real-ESRGAN). Loaded via
|
||||
# spandrel (MIT) -- a pure model-loader with NO basicsr dependency (it pulls only
|
||||
# torch / torchvision / safetensors / numpy / einops).
|
||||
# The Real-ESRGAN weights (BSD-3-Clause) download on first use and are cached; they
|
||||
# are never bundled. CPU works but is slow on large inputs -- it is meant for the
|
||||
# pre-diffusion upscale of SMALL inputs (and the GPU worker). Guarded by
|
||||
# upscaler.is_available(); the default upscaler stays Lanczos (cv2, no deps). The
|
||||
# weights are fetched with torch.hub (bundled with spandrel's torch), so no extra
|
||||
# download dependency is needed.
|
||||
esrgan = [
|
||||
"remove-ai-watermarks[pixels]",
|
||||
"spandrel>=0.3.0",
|
||||
]
|
||||
dev = [
|
||||
"remove-ai-watermarks[video]",
|
||||
"remove-ai-watermarks[detect]",
|
||||
@@ -173,21 +171,10 @@ dev = [
|
||||
"ruff>=0.4.0",
|
||||
"pyright>=1.1.0",
|
||||
"invisible-watermark>=0.2.0",
|
||||
# maintain.sh helpers; they only support newer Pythons, so gate them by
|
||||
# marker to keep the py3.10 resolution (and CI matrix) solvable.
|
||||
"uv-outdated>=0.1.0; python_version >= '3.12'",
|
||||
"uv-secure>=0.12.0; python_version >= '3.12'",
|
||||
]
|
||||
all = ["remove-ai-watermarks[video,heif,detect,trustmark,diffusion,qwen-zimage,lama,migan,esrgan]"]
|
||||
|
||||
# PyTorch Intel-GPU (XPU) wheel index. ``explicit = true`` keeps it inert for
|
||||
# the default CPU/CUDA install: uv consults it only when a torch install
|
||||
# explicitly targets it (see the ``diffusion`` extra comment), so it does not alter
|
||||
# the locked CPU/CUDA resolution. Linux/Windows only -- no macOS XPU build.
|
||||
[[tool.uv.index]]
|
||||
name = "pytorch-xpu"
|
||||
url = "https://download.pytorch.org/whl/xpu"
|
||||
explicit = true
|
||||
# ``qwen-zimage`` already pulls ``diffusion``; naming both would suggest diffusion is
|
||||
# independently sufficient for a removal, which it is not.
|
||||
all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan]"]
|
||||
|
||||
[project.scripts]
|
||||
remove-ai-watermarks = "remove_ai_watermarks.cli:main"
|
||||
@@ -224,7 +211,7 @@ pythonpath = ["src"]
|
||||
addopts = "-v --tb=short"
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py310"
|
||||
target-version = "py311"
|
||||
line-length = 120
|
||||
exclude = ["_refs"]
|
||||
extend-exclude = ["*.md"]
|
||||
@@ -247,7 +234,7 @@ quote-style = "double"
|
||||
indent-style = "space"
|
||||
|
||||
[tool.pyright]
|
||||
pythonVersion = "3.10"
|
||||
pythonVersion = "3.11"
|
||||
typeCheckingMode = "strict"
|
||||
exclude = ["_refs"]
|
||||
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Pure text-normalization helpers shared by evaluation scripts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unicodedata
|
||||
|
||||
|
||||
def normalize_text(text: str) -> str:
|
||||
"""Normalize text for layout-independent evaluation comparisons."""
|
||||
return "".join(unicodedata.normalize("NFC", text).casefold().split())
|
||||
|
||||
|
||||
def levenshtein_normalized(left: str, right: str) -> float:
|
||||
"""Return normalized Levenshtein distance without changing either input."""
|
||||
if not left and not right:
|
||||
return 0.0
|
||||
previous = list(range(len(right) + 1))
|
||||
for left_index, left_character in enumerate(left, start=1):
|
||||
current = [left_index]
|
||||
for right_index, right_character in enumerate(right, start=1):
|
||||
current.append(
|
||||
min(
|
||||
current[-1] + 1,
|
||||
previous[right_index] + 1,
|
||||
previous[right_index - 1] + (left_character != right_character),
|
||||
)
|
||||
)
|
||||
previous = current
|
||||
return previous[-1] / max(len(left), len(right))
|
||||
|
||||
|
||||
def normalized_edit_distance(left: str, right: str) -> float:
|
||||
"""Normalize text, then return Levenshtein distance over the result."""
|
||||
return levenshtein_normalized(normalize_text(left), normalize_text(right))
|
||||
@@ -1,436 +0,0 @@
|
||||
"""AI-generation scorer for scanned datasets.
|
||||
|
||||
Trains a gradient-boosted classifier on the metadata-derived labels of a
|
||||
scan_dataset.py output and scores every file, WITHOUT needing the
|
||||
metadata to be present at scoring time: the features are pixel and
|
||||
container statistics, so a metadata-stripped file still gets a score.
|
||||
The model is distribution-specific by design; evaluate it again before
|
||||
using it on an unrelated target distribution.
|
||||
|
||||
Modes:
|
||||
uv run --with scikit-learn python scripts/ai_score.py train <scan_glob> <model.pkl> [v1|v2]
|
||||
uv run --with scikit-learn python scripts/ai_score.py score <scan_glob> <model.pkl> <out.jsonl>
|
||||
|
||||
<scan_glob> is a glob of scan_dataset shards, e.g. 'data/scan/part_*.jsonl'.
|
||||
score output: one JSON line per file with file, sha256, ai_score, and the
|
||||
label evidence when a metadata label existed (for monitoring drift).
|
||||
"""
|
||||
|
||||
import glob
|
||||
import json
|
||||
import math
|
||||
import pickle
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
import numpy as np
|
||||
|
||||
# AI-generator names in C2PA claim_generator_info (metadata-level truth).
|
||||
AI_GENERATORS = (
|
||||
"openai",
|
||||
"adobe firefly",
|
||||
"adobe_firefly",
|
||||
"microsoft responsible ai",
|
||||
"microsoft_designer",
|
||||
"black forest labs",
|
||||
"fal-ai",
|
||||
"bria",
|
||||
"chatgpt",
|
||||
"stability",
|
||||
"dreamina",
|
||||
"canva",
|
||||
)
|
||||
# Human-origin software (negative evidence, NOT proof by itself).
|
||||
HUMAN_SOFTWARE = (
|
||||
"photoshop",
|
||||
"lightroom",
|
||||
"picsart",
|
||||
"snapseed",
|
||||
"paint.net",
|
||||
"gimp",
|
||||
"capture one",
|
||||
"meitu",
|
||||
"xingtu",
|
||||
"snow",
|
||||
)
|
||||
|
||||
_FEAT_SCALAR = [
|
||||
("noise", "noise_std"),
|
||||
("noise", "noise_kurtosis"),
|
||||
("fft", "cfa_peak"),
|
||||
("ela", "ela_mean"),
|
||||
("ela", "ela_p95"),
|
||||
("gradient", "laplacian_var"),
|
||||
("color", "saturation_mean"),
|
||||
("color", "value_mean"),
|
||||
("dct", "benford_mad"),
|
||||
]
|
||||
FeatureSchema = Literal["v1", "v2"]
|
||||
_FFT_BANDS = 8
|
||||
_GRADIENT_BINS = 10
|
||||
_COLOR_BINS = 64
|
||||
_DCT_AC_POSITIONS = 8
|
||||
_DCT_AC_BINS = 21
|
||||
_JPEG_QUANT_TABLES = 2
|
||||
_JPEG_QUANT_VALUES = 64
|
||||
_SCORE_BATCH = 4096
|
||||
|
||||
_V1_FEATURE_NAMES = (
|
||||
*(key for _, key in _FEAT_SCALAR),
|
||||
*(f"fft_band_energy_{index}" for index in range(_FFT_BANDS)),
|
||||
*(f"gradient_hist_{index}" for index in range(_GRADIENT_BINS)),
|
||||
*(f"color_hist_4x4x4_{index}" for index in range(_COLOR_BINS)),
|
||||
"jpeg_444",
|
||||
"jpeg_progressive",
|
||||
"megapixels",
|
||||
"aspect_ratio",
|
||||
"format_jpeg",
|
||||
"format_png",
|
||||
)
|
||||
_V2_EXTRA_FEATURE_NAMES = (
|
||||
"cfa_peak_0",
|
||||
"cfa_peak_1",
|
||||
*(f"dct_ac_{position}_{bin_index}" for position in range(_DCT_AC_POSITIONS) for bin_index in range(_DCT_AC_BINS)),
|
||||
*(f"jpeg_quant_{table}_{index}" for table in range(_JPEG_QUANT_TABLES) for index in range(_JPEG_QUANT_VALUES)),
|
||||
"jpeg_quant_table_count",
|
||||
"jpeg_huffman_table_count",
|
||||
"jpeg_huffman_total_bytes",
|
||||
"jpeg_scan_count",
|
||||
"jpeg_restart_interval",
|
||||
"jpeg_precision_bits",
|
||||
"jpeg_adobe_transform",
|
||||
"jpeg_jfif_present",
|
||||
"format_webp",
|
||||
"format_isobmff",
|
||||
"format_other",
|
||||
)
|
||||
_SCHEMA_FEATURE_NAMES: dict[str, tuple[str, ...]] = {
|
||||
"v1": _V1_FEATURE_NAMES,
|
||||
"v2": _V1_FEATURE_NAMES + _V2_EXTRA_FEATURE_NAMES,
|
||||
}
|
||||
|
||||
|
||||
def parse_schema(value: Any) -> FeatureSchema:
|
||||
"""Validate a feature-schema token (CLI argument or model-bundle field)."""
|
||||
if value == "v1" or value == "v2":
|
||||
return value
|
||||
raise ValueError(f"Unknown feature schema: {value!r}")
|
||||
|
||||
|
||||
def feature_names(schema: FeatureSchema = "v1") -> tuple[str, ...]:
|
||||
"""Return the stable ordered feature names for a model schema."""
|
||||
return _SCHEMA_FEATURE_NAMES[parse_schema(schema)]
|
||||
|
||||
|
||||
def _fixed_values(values: Any, count: int, *, normalize: bool = False) -> list[float]:
|
||||
"""Return a fixed-width float vector, padding absent values with NaN."""
|
||||
if not isinstance(values, (list, tuple)):
|
||||
return [float("nan")] * count
|
||||
result = [float(value) for value in values[:count]]
|
||||
if normalize and result:
|
||||
total = sum(value for value in result if math.isfinite(value))
|
||||
if total > 0:
|
||||
result = [value / total for value in result]
|
||||
return result + [float("nan")] * (count - len(result))
|
||||
|
||||
|
||||
def _optional_float(mapping: dict[str, Any], key: str) -> float:
|
||||
value = mapping.get(key)
|
||||
return float(value) if isinstance(value, (int, float)) else float("nan")
|
||||
|
||||
|
||||
def label_of(record: dict[str, Any]) -> int | None:
|
||||
"""1 = strong metadata AI, 0 = human-origin, None = unlabeled."""
|
||||
store = record.get("c2pa_store") or {}
|
||||
for m in (store.get("manifests") or {}).values():
|
||||
for cgi in m.get("claim_generator_info") or []:
|
||||
if any(g in str(cgi.get("name", "")).lower() for g in AI_GENERATORS):
|
||||
return 1
|
||||
for a in m.get("assertions") or []:
|
||||
d = a.get("data") if isinstance(a.get("data"), dict) else {}
|
||||
if "trainedAlgorithmicMedia" in str(d.get("digitalSourceType", "")):
|
||||
return 1
|
||||
for c in record.get("png_chunks", []):
|
||||
t = c.get("text", "")
|
||||
kw = t.split("\x00")[0]
|
||||
if kw in ("prompt", "workflow", "parameters") or "AIGC" in t[:80]:
|
||||
return 1
|
||||
exif = record.get("exif", {})
|
||||
z = exif.get("0th") or {}
|
||||
e = exif.get("Exif") or {}
|
||||
if z.get("Make") and z.get("Model") and (e.get("MakerNote") or e.get("LensModel") or e.get("LensMake")):
|
||||
return 0
|
||||
for c in record.get("png_chunks", []):
|
||||
if c.get("apple_screenshot_marker"):
|
||||
return 0
|
||||
for v in (record.get("iptc") or {}).values():
|
||||
if str(v).strip() == "Screenshot":
|
||||
return 0
|
||||
if any(t in str(z.get("Software", "")).lower() for t in HUMAN_SOFTWARE):
|
||||
return 0
|
||||
return None
|
||||
|
||||
|
||||
def features_of(record: dict[str, Any], *, schema: FeatureSchema = "v1") -> list[float]:
|
||||
"""The structural feature vector (pixel + container stats, no metadata)."""
|
||||
v = [float((record.get(s) or {}).get(k, np.nan)) for s, k in _FEAT_SCALAR]
|
||||
fft = record.get("fft") or {}
|
||||
gradient = record.get("gradient") or {}
|
||||
color = record.get("color") or {}
|
||||
v.extend(_fixed_values(fft.get("fft_band_energy"), _FFT_BANDS))
|
||||
v.extend(_fixed_values(gradient.get("gradient_hist"), _GRADIENT_BINS))
|
||||
v.extend(_fixed_values(color.get("color_hist_4x4x4"), _COLOR_BINS))
|
||||
jf = record.get("jpeg_forensics") or {}
|
||||
v.append(1.0 if jf.get("subsampling") == "4:4:4" else 0.0)
|
||||
v.append(1.0 if jf.get("progressive") else 0.0)
|
||||
pil = record.get("pil") or {}
|
||||
w, h = pil.get("width") or 0, pil.get("height") or 0
|
||||
v += [float(w * h) / 1e6, float(w) / max(h, 1)]
|
||||
fmt = record.get("content_format")
|
||||
v += [1.0 if fmt == "jpeg" else 0.0, 1.0 if fmt == "png" else 0.0]
|
||||
if schema == "v2":
|
||||
v.extend(_fixed_values(fft.get("cfa_peaks"), 2))
|
||||
dct_hist = (record.get("dct") or {}).get("dct_ac_hist")
|
||||
for position in range(_DCT_AC_POSITIONS):
|
||||
values = dct_hist[position] if isinstance(dct_hist, list) and position < len(dct_hist) else None
|
||||
v.extend(_fixed_values(values, _DCT_AC_BINS, normalize=True))
|
||||
quant_tables = jf.get("quant_tables")
|
||||
for table in range(_JPEG_QUANT_TABLES):
|
||||
values = quant_tables.get(str(table)) if isinstance(quant_tables, dict) else None
|
||||
v.extend(_fixed_values(values, _JPEG_QUANT_VALUES))
|
||||
is_jpeg = fmt == "jpeg"
|
||||
huffman_tables = jf.get("huffman_tables_hex")
|
||||
huffman_values = huffman_tables if isinstance(huffman_tables, list) else []
|
||||
v.extend(
|
||||
[
|
||||
float(len(quant_tables)) if isinstance(quant_tables, dict) else (0.0 if is_jpeg else float("nan")),
|
||||
float(len(huffman_values)) if is_jpeg else float("nan"),
|
||||
(
|
||||
float(sum(len(value) // 2 for value in huffman_values if isinstance(value, str)))
|
||||
if is_jpeg
|
||||
else float("nan")
|
||||
),
|
||||
_optional_float(jf, "scan_count"),
|
||||
_optional_float(jf, "restart_interval"),
|
||||
_optional_float(jf, "precision_bits"),
|
||||
_optional_float(jf, "adobe_transform"),
|
||||
1.0 if jf.get("jfif") else (0.0 if is_jpeg else float("nan")),
|
||||
1.0 if fmt == "webp" else 0.0,
|
||||
1.0 if isinstance(fmt, str) and fmt.startswith("isobmff:") else 0.0,
|
||||
(
|
||||
1.0
|
||||
if fmt is not None and fmt not in {"jpeg", "png", "webp"} and not str(fmt).startswith("isobmff:")
|
||||
else 0.0
|
||||
),
|
||||
]
|
||||
)
|
||||
elif schema != "v1":
|
||||
raise ValueError(f"Unknown feature schema: {schema}")
|
||||
expected = len(feature_names(schema))
|
||||
if len(v) != expected:
|
||||
raise ValueError(f"Feature schema {schema} produced {len(v)} values; expected {expected}")
|
||||
return v
|
||||
|
||||
|
||||
def grouped_stratified_split(
|
||||
labels: np.ndarray,
|
||||
hashes: np.ndarray,
|
||||
*,
|
||||
test_size: float = 0.3,
|
||||
random_state: int = 0,
|
||||
) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""Split records by hash while preserving label balance across hash groups."""
|
||||
if len(labels) != len(hashes):
|
||||
raise ValueError("labels and hashes must have equal length")
|
||||
if not 0 < test_size < 1:
|
||||
raise ValueError("test_size must be between 0 and 1")
|
||||
groups: dict[str, list[int]] = defaultdict(list)
|
||||
group_labels: dict[str, int] = {}
|
||||
for index, (label, digest) in enumerate(zip(labels, hashes, strict=True)):
|
||||
key = str(digest) if digest else f"__row_{index}"
|
||||
int_label = int(label)
|
||||
if key in group_labels and group_labels[key] != int_label:
|
||||
raise ValueError(f"Hash {key!r} has conflicting labels")
|
||||
group_labels[key] = int_label
|
||||
groups[key].append(index)
|
||||
keys_by_label: dict[int, list[str]] = defaultdict(list)
|
||||
for key, label in group_labels.items():
|
||||
keys_by_label[label].append(key)
|
||||
rng = np.random.default_rng(random_state)
|
||||
test_set: set[str] = set()
|
||||
for class_keys in keys_by_label.values():
|
||||
if len(class_keys) < 2:
|
||||
raise ValueError("Each label needs at least two distinct hash groups")
|
||||
shuffled = np.asarray(class_keys)
|
||||
rng.shuffle(shuffled)
|
||||
test_count = min(max(round(len(shuffled) * test_size), 1), len(shuffled) - 1)
|
||||
test_set.update(shuffled[:test_count].tolist())
|
||||
train_set = set(groups) - test_set
|
||||
train = np.asarray([index for key, indices in groups.items() if key in train_set for index in indices])
|
||||
test = np.asarray([index for key, indices in groups.items() if key in test_set for index in indices])
|
||||
return np.sort(train), np.sort(test)
|
||||
|
||||
|
||||
def temporal_holdout_split(
|
||||
dates: np.ndarray,
|
||||
hashes: np.ndarray,
|
||||
*,
|
||||
train_fraction: float = 0.7,
|
||||
) -> tuple[np.ndarray, np.ndarray, str]:
|
||||
"""Split chronologically and remove later copies of training hashes."""
|
||||
if len(dates) != len(hashes):
|
||||
raise ValueError("dates and hashes must have equal length")
|
||||
if not 0 < train_fraction < 1:
|
||||
raise ValueError("train_fraction must be between 0 and 1")
|
||||
unique_dates = sorted({str(date) for date in dates})
|
||||
if len(unique_dates) < 2:
|
||||
raise ValueError("Temporal holdout needs at least two distinct dates")
|
||||
cutoff_index = min(int(len(unique_dates) * train_fraction), len(unique_dates) - 1)
|
||||
cutoff = unique_dates[cutoff_index]
|
||||
train = np.flatnonzero(dates < cutoff)
|
||||
test = np.flatnonzero((dates >= cutoff) & ~np.isin(hashes, hashes[train]))
|
||||
return train, test, cutoff
|
||||
|
||||
|
||||
def model_schema(bundle: dict[str, Any]) -> FeatureSchema:
|
||||
"""Read a model's feature schema, defaulting legacy bundles to v1."""
|
||||
return parse_schema(bundle.get("feature_schema", "v1"))
|
||||
|
||||
|
||||
def iter_records(pattern: str) -> Any:
|
||||
for path in sorted(glob.glob(pattern)):
|
||||
with open(path) as fh:
|
||||
for line in fh:
|
||||
yield json.loads(line)
|
||||
|
||||
|
||||
def cmd_train(pattern: str, model_path: str, schema: FeatureSchema = "v2") -> None:
|
||||
from sklearn.ensemble import HistGradientBoostingClassifier
|
||||
from sklearn.impute import SimpleImputer
|
||||
from sklearn.metrics import average_precision_score, roc_auc_score
|
||||
|
||||
# keep the feature vector, not the record: a parsed scan record is an order of
|
||||
# magnitude larger than the row it collapses to.
|
||||
labeled: dict[str, tuple[list[float], int, str]] = {}
|
||||
missing_hash_index = 0
|
||||
for r in iter_records(pattern):
|
||||
if "noise" not in r:
|
||||
continue
|
||||
lab = label_of(r)
|
||||
if lab is None:
|
||||
continue
|
||||
digest = r.get("sha256")
|
||||
if not digest:
|
||||
digest = f"__missing_{missing_hash_index}"
|
||||
missing_hash_index += 1
|
||||
date = Path(r["file"]).parent.name
|
||||
previous = labeled.get(str(digest))
|
||||
if previous is not None and previous[1] != lab:
|
||||
raise ValueError(f"Hash {digest!r} has conflicting labels")
|
||||
if previous is None or date < previous[2]:
|
||||
labeled[str(digest)] = (features_of(r, schema=schema), lab, date)
|
||||
|
||||
rows, labels, row_dates = zip(*labeled.values(), strict=True)
|
||||
X = np.asarray(rows)
|
||||
y = np.asarray(labels)
|
||||
dates = np.asarray(row_dates)
|
||||
hashes = np.asarray(list(labeled))
|
||||
print(
|
||||
f"unique labeled: {len(y)} "
|
||||
f"(pos {int(y.sum())}, neg {int((1 - y).sum())}, features {X.shape[1]}, schema {schema})"
|
||||
)
|
||||
|
||||
def fit(train: np.ndarray) -> tuple[Any, Any]:
|
||||
rows = X[train]
|
||||
imputer = SimpleImputer(strategy="median")
|
||||
classifier = HistGradientBoostingClassifier(
|
||||
random_state=0,
|
||||
max_iter=300,
|
||||
l2_regularization=5.0,
|
||||
early_stopping=False,
|
||||
)
|
||||
classifier.fit(imputer.fit_transform(rows), y[train])
|
||||
return imputer, classifier
|
||||
|
||||
train, test = grouped_stratified_split(y, hashes, test_size=0.3, random_state=0)
|
||||
imp, clf = fit(train)
|
||||
p = clf.predict_proba(imp.transform(X[test]))[:, 1]
|
||||
print(f"SHA-grouped holdout: AUC {roc_auc_score(y[test], p):.4f} | AP {average_precision_score(y[test], p):.4f}")
|
||||
|
||||
temporal_train, temporal_test, cutoff = temporal_holdout_split(dates, hashes)
|
||||
if len(temporal_train) > 100 and len(temporal_test) > 100:
|
||||
imp_t, clf_t = fit(temporal_train)
|
||||
pt = clf_t.predict_proba(imp_t.transform(X[temporal_test]))[:, 1]
|
||||
auc_t = roc_auc_score(y[temporal_test], pt)
|
||||
ap_t = average_precision_score(y[temporal_test], pt)
|
||||
print(f"temporal (<{cutoff}): AUC {auc_t:.4f} | AP {ap_t:.4f}")
|
||||
|
||||
all_indices = np.arange(len(y))
|
||||
imp, clf = fit(all_indices)
|
||||
bundle = {
|
||||
"imputer": imp,
|
||||
"clf": clf,
|
||||
"feature_schema": schema,
|
||||
"feature_names": feature_names(schema),
|
||||
}
|
||||
with open(model_path, "wb") as f:
|
||||
pickle.dump(bundle, f)
|
||||
print(f"model written: {model_path}")
|
||||
|
||||
|
||||
def cmd_score(pattern: str, model_path: str, out_path: str) -> None:
|
||||
# the model file is produced locally by `train`; do not load third-party pickles
|
||||
with open(model_path, "rb") as f:
|
||||
bundle = pickle.load(f) # noqa: S301
|
||||
imp, clf = bundle["imputer"], bundle["clf"]
|
||||
schema = model_schema(bundle)
|
||||
stored_names = bundle.get("feature_names")
|
||||
if stored_names is not None and tuple(stored_names) != feature_names(schema):
|
||||
raise ValueError(f"Model feature names do not match schema {schema}")
|
||||
n = 0
|
||||
batch_rows: list[list[float]] = []
|
||||
batch_meta: list[dict[str, Any]] = []
|
||||
|
||||
def flush(out: Any) -> None:
|
||||
"""Score one batch: per-record predict_proba is dominated by call overhead."""
|
||||
nonlocal n
|
||||
if not batch_rows:
|
||||
return
|
||||
scores = clf.predict_proba(imp.transform(np.asarray(batch_rows)))[:, 1]
|
||||
for meta, score in zip(batch_meta, scores, strict=True):
|
||||
out.write(json.dumps({**meta, "ai_score": round(float(score), 4)}) + "\n")
|
||||
n += len(batch_rows)
|
||||
batch_rows.clear()
|
||||
batch_meta.clear()
|
||||
print(f" {n}", flush=True)
|
||||
|
||||
with open(out_path, "w") as out:
|
||||
for r in iter_records(pattern):
|
||||
if "noise" not in r:
|
||||
continue
|
||||
batch_rows.append(features_of(r, schema=schema))
|
||||
batch_meta.append({"file": r["file"], "sha256": r.get("sha256"), "metadata_label": label_of(r)})
|
||||
if len(batch_rows) >= _SCORE_BATCH:
|
||||
flush(out)
|
||||
flush(out)
|
||||
print(f"scored {n} -> {out_path}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
if len(sys.argv) < 2 or sys.argv[1] not in ("train", "score"):
|
||||
print(__doc__)
|
||||
sys.exit(2)
|
||||
if sys.argv[1] == "train" and len(sys.argv) in (4, 5):
|
||||
schema = parse_schema(sys.argv[4]) if len(sys.argv) == 5 else "v2"
|
||||
cmd_train(sys.argv[2], sys.argv[3], schema)
|
||||
elif sys.argv[1] == "score" and len(sys.argv) == 5:
|
||||
cmd_score(sys.argv[2], sys.argv[3], sys.argv[4])
|
||||
else:
|
||||
print(__doc__)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,241 @@
|
||||
"""Per-method wall time for metadata extraction and the verdict built on it.
|
||||
|
||||
WHY THIS EXISTS
|
||||
The detection path is documented by CAPABILITY -- which signals it reads, in what
|
||||
order, with what confidence -- but not by COST. Batch throughput and the question
|
||||
"which probe dominates, and does it depend on container or on file size" have never
|
||||
been measured.
|
||||
|
||||
WHAT IT MEASURES
|
||||
Only the file-backed half: metadata extraction, then the verdict evaluated on the
|
||||
extracted evidence. The pixel-backed detectors are deliberately out of scope.
|
||||
|
||||
COLD pass: one measurement on a file the process has never touched --
|
||||
``extract_provenance_evidence``. Only the first read of a file is genuinely cold,
|
||||
so it buys exactly one number, and that is the one a single-shot run pays.
|
||||
|
||||
WARM pass, with the filesystem cache now hot:
|
||||
1. ``extract_provenance_evidence`` as a whole.
|
||||
2. Its nine components, timed IN THE ORDER the dataclass constructs them and with
|
||||
the per-file caches cleared once beforehand -- so ``extract_c2pa_info`` carries
|
||||
the Rust manifest reader and the later ``get_ai_metadata`` sees the same warm
|
||||
cache it sees in production. Timing them in any other order moves that cost to
|
||||
a different row and flatters whichever ran second.
|
||||
3. ``identify_from_evidence``: pure verdict logic, the source is never reopened.
|
||||
4. ``identify(check_visible=False, check_invisible=False)`` -- extraction plus
|
||||
verdict as one call, the cross-check that 1 + 3 is the whole metadata path.
|
||||
|
||||
Every ``@lru_cache`` in the metadata and C2PA modules is cleared before each timed
|
||||
unit. Without that the second measurement of a file answers from the memo and
|
||||
reports a cost of zero -- the caches are keyed on (path, mtime, size) and this
|
||||
script reads each file several times.
|
||||
|
||||
READING IT
|
||||
Component times do NOT sum to the ``extract_provenance_evidence`` total for free:
|
||||
they come from a separate cache-cleared run, so the sum is a cross-check. A gap
|
||||
means a component is missing from the list. Both numbers are written.
|
||||
|
||||
DATA SAFETY
|
||||
Read-only over a local dataset. Writes only to the given output prefix, which
|
||||
belongs outside the repository.
|
||||
|
||||
uv run python scripts/detection_timing.py <dataset> <prefix> --limit 200
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable, Iterator
|
||||
|
||||
# The package's OWN tree, not the repository root: from a worktree, an editable
|
||||
# install resolves `remove_ai_watermarks` to the MAIN checkout, so a script measuring
|
||||
# this tree would silently import a different one.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
||||
|
||||
from remove_ai_watermarks import identify as identify_mod
|
||||
from remove_ai_watermarks import metadata as metadata_mod
|
||||
from remove_ai_watermarks._internal import c2pa as c2pa_mod
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SUPPORTED = frozenset({".png", ".jpg", ".jpeg", ".webp", ".heic", ".heif", ".avif"})
|
||||
|
||||
# Timed in the order ``ProvenanceEvidence`` is constructed in ``identify.py``. The
|
||||
# report script imports this to label its columns, so the order is the contract.
|
||||
COMPONENTS: tuple[tuple[str, Callable[[Path], Any]], ...] = (
|
||||
("c2pa_info", c2pa_mod.extract_c2pa_info),
|
||||
("ai_metadata", metadata_mod.get_ai_metadata),
|
||||
("scan_head", lambda p: metadata_mod.scan_head(p, identify_mod._SCAN_BYTES)),
|
||||
("iptc_ai_system", metadata_mod.iptc_ai_system),
|
||||
("aigc_label", metadata_mod.aigc_label),
|
||||
("exif_generator", metadata_mod.exif_generator),
|
||||
("xai_signature", metadata_mod.xai_signature),
|
||||
("huggingface_job", metadata_mod.huggingface_job),
|
||||
("samsung_genai", metadata_mod.samsung_genai),
|
||||
)
|
||||
|
||||
|
||||
def _cache_clearers() -> tuple[Callable[[], None], ...]:
|
||||
"""Every per-file memo in the metadata path, found by attribute, not by name list.
|
||||
|
||||
A hand-written list silently goes stale the next time a probe gains a cache, and a
|
||||
stale entry shows up as a suspiciously fast row rather than as an error.
|
||||
"""
|
||||
found: list[Callable[[], None]] = []
|
||||
for module in (metadata_mod, c2pa_mod):
|
||||
for name in dir(module):
|
||||
clear = getattr(getattr(module, name, None), "cache_clear", None)
|
||||
if callable(clear):
|
||||
found.append(clear)
|
||||
return tuple(found)
|
||||
|
||||
|
||||
_CLEARERS = _cache_clearers()
|
||||
|
||||
|
||||
def _clear() -> None:
|
||||
for clear in _CLEARERS:
|
||||
clear()
|
||||
|
||||
|
||||
def _ms(fn: Callable[[], Any]) -> tuple[float, Any]:
|
||||
"""Wall time in milliseconds plus the call's result."""
|
||||
start = time.perf_counter_ns()
|
||||
value = fn()
|
||||
return (time.perf_counter_ns() - start) / 1e6, value
|
||||
|
||||
|
||||
def _pixel_geometry(path: Path) -> tuple[str | None, int | None, int | None]:
|
||||
"""Container format and pixel dimensions from the header alone."""
|
||||
try:
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(path) as img:
|
||||
return img.format, img.width, img.height
|
||||
except Exception: # unreadable or an unsupported container
|
||||
return None, None, None
|
||||
|
||||
|
||||
def _warm_breakdown(path: Path, row: dict[str, Any]) -> None:
|
||||
"""Fill ``row`` with the warm-cache per-method breakdown."""
|
||||
_clear()
|
||||
row["extract_evidence_ms"], evidence = _ms(lambda: identify_mod.extract_provenance_evidence(path))
|
||||
|
||||
_clear()
|
||||
component_sum = 0.0
|
||||
for name, fn in COMPONENTS:
|
||||
elapsed, _ = _ms(lambda fn=fn: fn(path))
|
||||
row[f"meta_{name}_ms"] = elapsed
|
||||
component_sum += elapsed
|
||||
row["meta_components_sum_ms"] = component_sum
|
||||
|
||||
row["verdict_from_evidence_ms"], report = _ms(lambda: identify_mod.identify_from_evidence(evidence))
|
||||
|
||||
_clear()
|
||||
row["identify_metadata_only_ms"], full = _ms(
|
||||
lambda: identify_mod.identify(path, check_visible=False, check_invisible=False)
|
||||
)
|
||||
|
||||
row["scan_bytes"] = len(evidence.scan)
|
||||
row["has_c2pa"] = bool(evidence.c2pa_info)
|
||||
row["has_ai_metadata"] = bool(evidence.ai_metadata)
|
||||
row["is_ai_generated"] = full.is_ai_generated
|
||||
row["confidence"] = full.confidence
|
||||
row["platform"] = full.platform
|
||||
row["signals"] = [signal.name for signal in full.signals]
|
||||
# The two verdict paths must agree; a mismatch means the breakdown timed a
|
||||
# different code path than the end-to-end call and the rows are not comparable.
|
||||
row["verdict_agrees"] = (report.confidence, report.platform) == (full.confidence, full.platform)
|
||||
|
||||
|
||||
def _measure(path: Path) -> dict[str, Any]:
|
||||
row: dict[str, Any] = {"path": str(path), "ext": path.suffix.lower()}
|
||||
try:
|
||||
row["bytes"] = path.stat().st_size
|
||||
except OSError as exc:
|
||||
return {**row, "error": f"stat: {exc}"}
|
||||
|
||||
# COLD first: this is the only moment the file is untouched by this process.
|
||||
_clear()
|
||||
try:
|
||||
row["cold_extract_evidence_ms"], _ = _ms(lambda: identify_mod.extract_provenance_evidence(path))
|
||||
except Exception as exc:
|
||||
return {**row, "error": f"cold extract: {type(exc).__name__}: {exc}"}
|
||||
|
||||
row["format"], row["width"], row["height"] = _pixel_geometry(path)
|
||||
width, height = row["width"], row["height"]
|
||||
row["megapixels"] = round(width * height / 1e6, 3) if width and height else None
|
||||
|
||||
try:
|
||||
_warm_breakdown(path, row)
|
||||
except Exception as exc:
|
||||
row["error"] = f"warm pass: {type(exc).__name__}: {exc}"
|
||||
return row
|
||||
|
||||
|
||||
def _iter_images(root: Path) -> Iterator[Path]:
|
||||
for path in sorted(root.rglob("*")):
|
||||
if path.is_file() and path.suffix.lower() in SUPPORTED:
|
||||
yield path
|
||||
|
||||
|
||||
def _done_paths(out_path: Path) -> set[str]:
|
||||
"""Paths already recorded, so a long run resumes instead of restarting."""
|
||||
if not out_path.exists():
|
||||
return set()
|
||||
done: set[str] = set()
|
||||
with out_path.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
done.add(json.loads(line)["path"])
|
||||
except (ValueError, KeyError):
|
||||
continue
|
||||
return done
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("dataset", type=Path, help="directory of images, scanned recursively")
|
||||
parser.add_argument("out_prefix", type=Path, help="output prefix; writes <prefix>.jsonl")
|
||||
parser.add_argument("--limit", type=int, default=0, help="stop after N files (0 = all)")
|
||||
parser.add_argument("--progress-every", type=int, default=200)
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
|
||||
out_path = args.out_prefix.with_suffix(".jsonl")
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
done = _done_paths(out_path)
|
||||
if done:
|
||||
log.info("resuming: %d files already recorded", len(done))
|
||||
|
||||
processed = 0
|
||||
started = time.monotonic()
|
||||
with out_path.open("a", encoding="utf-8") as handle:
|
||||
for path in _iter_images(args.dataset):
|
||||
if str(path) in done:
|
||||
continue
|
||||
row = _measure(path)
|
||||
handle.write(json.dumps(row, ensure_ascii=False, default=str) + "\n")
|
||||
handle.flush()
|
||||
processed += 1
|
||||
if processed % args.progress_every == 0:
|
||||
log.info("%d files, %.2f files/s", processed, processed / (time.monotonic() - started))
|
||||
if args.limit and processed >= args.limit:
|
||||
break
|
||||
|
||||
elapsed = time.monotonic() - started
|
||||
rate = processed / max(elapsed, 1e-9)
|
||||
log.info("done: %d files in %.1f s (%.2f files/s) -> %s", processed, elapsed, rate, out_path)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,232 @@
|
||||
"""Aggregate ``detection_timing.py`` records into per-method, per-segment tables.
|
||||
|
||||
WHAT IT PRODUCES
|
||||
Reading it back: the per-file JSONL is one row per image with a millisecond field
|
||||
per method. This collapses it into percentiles per method, then repeats that per
|
||||
segment -- container, megapixels, file size, C2PA presence, verdict confidence --
|
||||
because a single median hides a path whose cost is carried entirely by one
|
||||
container or by the files that actually have a manifest.
|
||||
|
||||
Writes ``<prefix>_summary.csv`` (long form: segment_kind, segment, method, n, p50,
|
||||
p90, p99, mean) and prints a markdown report.
|
||||
|
||||
Percentiles are computed by nearest-rank on the sorted sample, not interpolated:
|
||||
every reported number is a time some real file actually took.
|
||||
|
||||
DATA SAFETY
|
||||
Reads and writes only the given prefix, which belongs outside the repository.
|
||||
|
||||
uv run python scripts/detection_timing_report.py .local-eval/timing/full
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import math
|
||||
import statistics
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable, Iterator
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from detection_timing import COMPONENTS as _TIMED
|
||||
|
||||
# Taken from the script that WROTE the rows, in its order, so a probe added, removed
|
||||
# or reordered there cannot silently leave a column missing or mislabelled here.
|
||||
COMPONENTS = tuple(name for name, _ in _TIMED)
|
||||
METHODS = (
|
||||
("cold_extract_evidence_ms", "extract_provenance_evidence (cold)"),
|
||||
("extract_evidence_ms", "extract_provenance_evidence (warm)"),
|
||||
*((f"meta_{name}_ms", f" {name}") for name in COMPONENTS),
|
||||
("meta_components_sum_ms", " (sum of components)"),
|
||||
("verdict_from_evidence_ms", "identify_from_evidence (verdict)"),
|
||||
("identify_metadata_only_ms", "identify(metadata only), end to end"),
|
||||
)
|
||||
# Median-only columns in the per-segment tables; the full percentile set goes to the CSV.
|
||||
HEADLINE_LABELS = (
|
||||
("extract_evidence_ms", "extract p50"),
|
||||
("verdict_from_evidence_ms", "verdict p50"),
|
||||
("identify_metadata_only_ms", "end-to-end p50"),
|
||||
)
|
||||
HEADLINE = tuple(key for key, _ in HEADLINE_LABELS)
|
||||
|
||||
|
||||
def _bucket(value: float | None, edges: tuple[float, ...], labels: tuple[str, ...]) -> str:
|
||||
if value is None:
|
||||
return "unknown"
|
||||
for edge, label in zip(edges, labels, strict=False):
|
||||
if value < edge:
|
||||
return label
|
||||
return labels[-1]
|
||||
|
||||
|
||||
SEGMENTS: tuple[tuple[str, Callable[[dict[str, Any]], str]], ...] = (
|
||||
("container", lambda r: str(r.get("format") or "unknown")),
|
||||
(
|
||||
"megapixels",
|
||||
lambda r: _bucket(r.get("megapixels"), (1, 4, 12), ("<1 MP", "1-4 MP", "4-12 MP", ">12 MP")),
|
||||
),
|
||||
(
|
||||
"file size",
|
||||
lambda r: _bucket(
|
||||
(r["bytes"] / 1e6) if r.get("bytes") is not None else None,
|
||||
(1, 5, 20),
|
||||
("<1 MB", "1-5 MB", "5-20 MB", ">20 MB"),
|
||||
),
|
||||
),
|
||||
("c2pa", lambda r: "with C2PA" if r.get("has_c2pa") else "no C2PA"),
|
||||
("verdict", lambda r: f"confidence={r.get('confidence') or 'n/a'}"),
|
||||
)
|
||||
|
||||
|
||||
def _numeric(records: list[dict[str, Any]], key: str) -> list[float]:
|
||||
"""The numeric values of ``key``, skipping rows where it is absent or non-numeric.
|
||||
|
||||
One definition of "counts as a measurement", so the printed table and the CSV can
|
||||
never disagree about which rows a method was measured on.
|
||||
"""
|
||||
return [float(r[key]) for r in records if isinstance(r.get(key), (int, float))]
|
||||
|
||||
|
||||
def _percentile(sorted_values: list[float], q: float) -> float:
|
||||
"""Nearest-rank percentile: the reported value is one a real file produced."""
|
||||
index = max(0, math.ceil(q * len(sorted_values)) - 1)
|
||||
return sorted_values[index]
|
||||
|
||||
|
||||
def _stats(values: list[float]) -> dict[str, float | int]:
|
||||
ordered = sorted(values)
|
||||
return {
|
||||
"n": len(ordered),
|
||||
"p50": _percentile(ordered, 0.50),
|
||||
"p90": _percentile(ordered, 0.90),
|
||||
"p99": _percentile(ordered, 0.99),
|
||||
"mean": statistics.fmean(ordered),
|
||||
}
|
||||
|
||||
|
||||
def _rows(path: Path) -> Iterator[dict[str, Any]]:
|
||||
with path.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
yield json.loads(line)
|
||||
except ValueError: # a truncated tail while the run is still writing
|
||||
continue
|
||||
|
||||
|
||||
def _table(rows: list[dict[str, Any]], methods: tuple[tuple[str, str], ...]) -> list[str]:
|
||||
out = ["| method | n | p50 | p90 | p99 | mean |", "|---|---:|---:|---:|---:|---:|"]
|
||||
for key, label in methods:
|
||||
values = _numeric(rows, key)
|
||||
if not values:
|
||||
continue
|
||||
s = _stats(values)
|
||||
out.append(f"| {label} | {s['n']} | {s['p50']:.3f} | {s['p90']:.3f} | {s['p99']:.3f} | {s['mean']:.3f} |")
|
||||
return out
|
||||
|
||||
|
||||
def _correlation(rows: list[dict[str, Any]], x_key: str, y_key: str) -> float | None:
|
||||
both = [r for r in rows if isinstance(r.get(x_key), (int, float)) and isinstance(r.get(y_key), (int, float))]
|
||||
if len(both) < 3:
|
||||
return None
|
||||
xs = _numeric(both, x_key)
|
||||
ys = _numeric(both, y_key)
|
||||
try:
|
||||
return statistics.correlation(xs, ys)
|
||||
except statistics.StatisticsError: # a constant column has no correlation
|
||||
return None
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("prefix", type=Path, help="prefix given to detection_timing.py")
|
||||
args = parser.parse_args()
|
||||
|
||||
jsonl = args.prefix.with_suffix(".jsonl")
|
||||
all_rows = list(_rows(jsonl))
|
||||
failed = [r for r in all_rows if "error" in r]
|
||||
rows = [r for r in all_rows if "error" not in r]
|
||||
if not rows:
|
||||
print(f"no usable records in {jsonl}")
|
||||
return 1
|
||||
|
||||
disagreed = [r for r in rows if r.get("verdict_agrees") is False]
|
||||
|
||||
lines = [
|
||||
f"# Detection timing over {len(rows)} images",
|
||||
"",
|
||||
f"Source: `{jsonl}`. Failed rows: {len(failed)}. Verdict-path mismatches: {len(disagreed)}.",
|
||||
"",
|
||||
"All times in milliseconds. Percentiles are nearest-rank.",
|
||||
"",
|
||||
"## Whole corpus",
|
||||
"",
|
||||
*_table(rows, METHODS),
|
||||
"",
|
||||
]
|
||||
|
||||
csv_rows: list[dict[str, Any]] = []
|
||||
for key, label in METHODS:
|
||||
values = _numeric(rows, key)
|
||||
if values:
|
||||
csv_rows.append({"segment_kind": "all", "segment": "all", "method": label.strip(), **_stats(values)})
|
||||
|
||||
for kind, classify in SEGMENTS:
|
||||
groups: dict[str, list[dict[str, Any]]] = defaultdict(list)
|
||||
for row in rows:
|
||||
groups[classify(row)].append(row)
|
||||
lines += [f"## By {kind}", ""]
|
||||
header = ["| segment | n | " + " | ".join(label for _, label in HEADLINE_LABELS) + " |"]
|
||||
header.append("|---|---:|" + "---:|" * len(HEADLINE))
|
||||
lines += header
|
||||
for segment, group in sorted(groups.items(), key=lambda item: -len(item[1])):
|
||||
cells = []
|
||||
for key in HEADLINE:
|
||||
values = _numeric(group, key)
|
||||
cells.append(f"{_percentile(sorted(values), 0.5):.2f}" if values else "-")
|
||||
lines.append(f"| {segment} | {len(group)} | " + " | ".join(cells) + " |")
|
||||
for key, label in METHODS:
|
||||
values = _numeric(group, key)
|
||||
if values:
|
||||
csv_rows.append(
|
||||
{"segment_kind": kind, "segment": segment, "method": label.strip(), **_stats(values)}
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
corr = _correlation(rows, "scan_bytes", "verdict_from_evidence_ms")
|
||||
corr_size = _correlation(rows, "bytes", "extract_evidence_ms")
|
||||
lines += [
|
||||
"## Scaling",
|
||||
"",
|
||||
f"- verdict time vs scan-buffer size: r = {corr:.3f}" if corr is not None else "- verdict correlation: n/a",
|
||||
(
|
||||
f"- extraction time vs file size: r = {corr_size:.3f}"
|
||||
if corr_size is not None
|
||||
else "- extraction correlation: n/a"
|
||||
),
|
||||
"",
|
||||
]
|
||||
|
||||
summary_csv = args.prefix.with_name(args.prefix.name + "_summary.csv")
|
||||
columns = ["segment_kind", "segment", "method", "n", "p50", "p90", "p99", "mean"]
|
||||
with summary_csv.open("w", encoding="utf-8", newline="") as handle:
|
||||
writer = csv.DictWriter(handle, fieldnames=columns)
|
||||
writer.writeheader()
|
||||
writer.writerows(csv_rows)
|
||||
|
||||
report = "\n".join(lines)
|
||||
args.prefix.with_name(args.prefix.name + "_report.md").write_text(report + "\n", encoding="utf-8")
|
||||
print(report)
|
||||
print(f"\nwrote {summary_csv}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -1,12 +1,11 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "numpy",
|
||||
# "opencv-python-headless",
|
||||
# "opencv-python-headless<5",
|
||||
# "pillow",
|
||||
# "scikit-image",
|
||||
# "rapidfuzz",
|
||||
# "torch",
|
||||
# "lpips",
|
||||
# "paddleocr",
|
||||
@@ -29,7 +28,7 @@ metrics run only where faces are detected, text metrics only where text is.
|
||||
|
||||
Two subcommands:
|
||||
|
||||
ocr -- OCR images (PaddleOCR PP-OCRv6) into a JSON {basename: text} file.
|
||||
ocr -- OCR images (PaddleOCR defaults) into a JSON {basename: text} file.
|
||||
Run this on the ORIGINALS, hand-verify/correct the file, and it
|
||||
becomes the ground truth for ``compare --ground-truth`` -- the clean
|
||||
way to score text, since OCR-vs-OCR is doubly noisy (errors on both
|
||||
@@ -54,6 +53,7 @@ Usage:
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import unicodedata
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
@@ -65,7 +65,10 @@ import numpy as np
|
||||
from _plain_console import Console, Table
|
||||
|
||||
console = Console()
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from scripts._text_eval import levenshtein_normalized # noqa: E402
|
||||
|
||||
# ── helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
@@ -89,7 +92,7 @@ def _norm(text: str) -> str:
|
||||
return "".join(unicodedata.normalize("NFC", text).split())
|
||||
|
||||
|
||||
# ── text: PaddleOCR (PP-OCRv6) ───────────────────────────────────────
|
||||
# ── text: PaddleOCR defaults ─────────────────────────────────────────
|
||||
|
||||
# Our lang codes -> PaddleOCR lang. The 'ch' model also reads Latin; 'ru' reads
|
||||
# Cyrillic + Latin. Multiple langs in one image -> run each model, union detections.
|
||||
@@ -165,9 +168,7 @@ def _ocr_lines(bgr: np.ndarray, langs: list[str], min_score: float = 0.5) -> lis
|
||||
|
||||
|
||||
def _cer(ref: str, hyp: str) -> float:
|
||||
from rapidfuzz.distance import Levenshtein
|
||||
|
||||
return Levenshtein.normalized_distance(_norm(ref), _norm(hyp))
|
||||
return levenshtein_normalized(_norm(ref), _norm(hyp))
|
||||
|
||||
|
||||
# ── face: detection + ArcFace + texture ──────────────────────────────
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "huggingface-hub>=0.20.0",
|
||||
# "numpy",
|
||||
# "onnxruntime>=1.24.0",
|
||||
# "opencv-python-headless<5",
|
||||
# "paddleocr>=3.3.3",
|
||||
# "paddlepaddle",
|
||||
# "pillow",
|
||||
# ]
|
||||
# ///
|
||||
"""Infer stable source-text lines without modifying an image.
|
||||
|
||||
This evaluation-only dry run proposes line annotations for selective text
|
||||
restoration. Every proposal still needs human verification: stable OCR can lose
|
||||
punctuation with high confidence. It separately flags lines whose recognition
|
||||
changes under crop jitter or whose minimum confidence is below the threshold.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
# Dogfoods the packaged draft API (remove_ai_watermarks.text_draft); this script
|
||||
# keeps only the CLI wrapper so the package stays the one home for the logic.
|
||||
from remove_ai_watermarks.text_draft import ( # noqa: E402
|
||||
_detect_line_boxes,
|
||||
_recognize,
|
||||
choose_language,
|
||||
stable_recognition,
|
||||
)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option("--out", required=True, type=click.Path(dir_okay=False, path_type=Path))
|
||||
@click.option("--min-score", default=0.85, show_default=True, type=click.FloatRange(0.0, 1.0))
|
||||
def main(source: Path, out: Path, min_score: float) -> None:
|
||||
"""Write draft line text for SOURCE; manually verify every proposal."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
|
||||
os.environ["PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK"] = "True"
|
||||
from paddleocr import PaddleOCR, TextRecognition
|
||||
|
||||
source_rgb = np.asarray(Image.open(source).convert("RGB"))
|
||||
detector = PaddleOCR(
|
||||
lang="ch",
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
)
|
||||
engines = {
|
||||
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec"),
|
||||
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec"),
|
||||
"ch": TextRecognition(model_name="PP-OCRv5_server_rec"),
|
||||
}
|
||||
boxes = _detect_line_boxes(detector, source_rgb)
|
||||
accepted = []
|
||||
rejected = []
|
||||
for box in boxes:
|
||||
probes = {}
|
||||
for language, engine in engines.items():
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
probes[language] = _recognize(engine, source_rgb, box, script, 0.1)
|
||||
language = choose_language(probes)
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
reads = [_recognize(engines[language], source_rgb, box, script, ratio) for ratio in (0.08, 0.12, 0.2)]
|
||||
text = stable_recognition(reads, min_score)
|
||||
result = {
|
||||
"box": box,
|
||||
"script": script,
|
||||
"language": language,
|
||||
"reads": [{"text": value, "score": score} for value, score in reads],
|
||||
}
|
||||
if text is None:
|
||||
rejected.append(result)
|
||||
else:
|
||||
accepted.append({"box": box, "text": text, "script": script, "min_score": min(score for _, score in reads)})
|
||||
payload = {"source": source.name, "accepted": accepted, "rejected": rejected}
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
out.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
log.info("Accepted %s lines and rejected %s uncertain lines", len(accepted), len(rejected))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,7 +1,7 @@
|
||||
"""How much recall is the coarse scale ladder costing, and what would a denser one cost?
|
||||
|
||||
THE FINDING THIS MEASURES
|
||||
`_tophat_score` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end
|
||||
`_ladder_best` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end
|
||||
sweeps none at all. Measured on stamped marks over controlled backgrounds
|
||||
(`scripts/detector_response.py`), the response is a COMB: doubao scores 0.99 exactly at
|
||||
each rung and collapses to 0.37-0.48 between them, against a 0.50 gate. So a mark whose
|
||||
@@ -93,7 +93,10 @@ def wilson(k: int, n: int, z: float = 1.96) -> tuple[float, float]:
|
||||
|
||||
|
||||
def score_at_scales(engine: Any, image: np.ndarray, scales: tuple[float, ...]) -> dict[float, float]:
|
||||
"""`_tophat_score`'s inner loop, opened up so the ladder is a parameter.
|
||||
"""`_ladder_best`'s inner loop, opened up so the ladder is a parameter.
|
||||
|
||||
A DELIBERATE second copy of the sweep: it exists to measure ladders the shipped
|
||||
engine does not have, so it must not be folded back into `_ladder_best`.
|
||||
|
||||
Deliberately reaches into the engine (`tophat_response`, `_glyph_silhouette`): a
|
||||
measurement script may, product code may not. Kept a faithful copy of the shipped
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "diffusers>=0.35.0",
|
||||
# "transformers>=4.51.0",
|
||||
|
||||
@@ -13,20 +13,20 @@ WHAT IT COVERS
|
||||
metadata real AI-metadata files -> --check detects, --remove strip-and-verifies clean
|
||||
visible real marked images per mark -> the mark is gone on re-detect, output written
|
||||
erase a real image, each fill backend (cv2 / migan / lama) -> output written
|
||||
invisible a real SynthID image on MPS at reduced resolution -> a CHANGED image is written
|
||||
invisible a real SynthID image at reduced resolution -> a CHANGED image is written
|
||||
all a real marked image through the full pipeline -> output written
|
||||
batch a real directory -> every input produces an output
|
||||
|
||||
invisible/all run the diffusion model, so they are gated behind --diffusion and run at a
|
||||
small --max-resolution on MPS (the user's "reduced size on MPS" path). Everything else is
|
||||
cv2/numpy and fast.
|
||||
small --max-resolution. Both profiles are CUDA-only, so that section needs an NVIDIA
|
||||
GPU; everything else is cv2/numpy, fast, and runs anywhere.
|
||||
|
||||
DATA SAFETY
|
||||
Treat input datasets as sensitive and read-only. Output stays in a gitignored temp
|
||||
dir. Records example uids and pass/fail, never image content.
|
||||
|
||||
uv run python scripts/real_examples_e2e.py # fast surface (no diffusion)
|
||||
uv run python scripts/real_examples_e2e.py --diffusion # + invisible/all on MPS
|
||||
uv run python scripts/real_examples_e2e.py --diffusion # + invisible/all (needs CUDA)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -239,12 +239,16 @@ def check_erase(res: Results, tmp: Path) -> None:
|
||||
|
||||
|
||||
def check_diffusion(res: Results, tmp: Path, gemini_src: str, openai_src: str) -> None:
|
||||
"""The GPU path: invisible + all on MPS at reduced resolution -> a CHANGED image."""
|
||||
"""The GPU path: invisible + all at reduced resolution -> a CHANGED image.
|
||||
|
||||
CUDA-only. On a machine without an NVIDIA GPU every row here fails with the
|
||||
library's clean refusal; run this section on a GPU box.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
|
||||
print("\ninvisible / all -- real SynthID image on MPS, reduced resolution")
|
||||
print("\ninvisible / all -- real SynthID image, reduced resolution (CUDA required)")
|
||||
for label, src in (("invisible/gemini", gemini_src), ("invisible/openai", openai_src)):
|
||||
if not src:
|
||||
res.add(label, "-", True, "no real positive found (skipped)")
|
||||
@@ -252,7 +256,7 @@ def check_diffusion(res: Results, tmp: Path, gemini_src: str, openai_src: str) -
|
||||
sp = Path(src)
|
||||
outp = tmp / f"inv_{sp.stem}.png"
|
||||
code, out = run(
|
||||
["invisible", src, "-o", str(outp), "--device", "mps", "--max-resolution", "512", "--seed", "0"],
|
||||
["invisible", src, "-o", str(outp), "--max-resolution", "512", "--seed", "0"],
|
||||
timeout=1200,
|
||||
)
|
||||
if not outp.exists():
|
||||
@@ -276,7 +280,7 @@ def check_diffusion(res: Results, tmp: Path, gemini_src: str, openai_src: str) -
|
||||
sp = Path(gemini_src)
|
||||
outp = tmp / f"all_{sp.stem}.png"
|
||||
code, out = run(
|
||||
["all", gemini_src, "-o", str(outp), "--device", "mps", "--max-resolution", "512", "--seed", "0"],
|
||||
["all", gemini_src, "-o", str(outp), "--max-resolution", "512", "--seed", "0"],
|
||||
timeout=1200,
|
||||
)
|
||||
ok = outp.exists() and outp.stat().st_size > 0
|
||||
@@ -304,7 +308,7 @@ def check_batch(res: Results, tmp: Path) -> None:
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--diffusion", action="store_true", help="also run invisible/all on MPS (slow)")
|
||||
ap.add_argument("--diffusion", action="store_true", help="also run invisible/all (slow, needs CUDA)")
|
||||
ap.add_argument("--gemini", default="")
|
||||
ap.add_argument("--openai", default="")
|
||||
a = ap.parse_args()
|
||||
@@ -320,7 +324,7 @@ def main() -> None:
|
||||
if a.diffusion:
|
||||
check_diffusion(res, tmp, a.gemini, a.openai)
|
||||
else:
|
||||
print("\n(diffusion skipped -- pass --diffusion to run invisible/all on MPS)")
|
||||
print("\n(diffusion skipped -- pass --diffusion to run invisible/all)")
|
||||
raise SystemExit(res.report())
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
"""Audit the record path against the file path over a whole dataset.
|
||||
|
||||
WHY THIS EXISTS
|
||||
|
||||
Two seams reach the same provenance verdict:
|
||||
|
||||
identify(path, check_visible=False, check_invisible=False)
|
||||
|
||||
identify_metadata_record(collect_metadata_record(path), path=path)
|
||||
|
||||
Their equality is the record's entire contract, and it can break from either side --
|
||||
a region the collector stops walking, or a placement the file path learns to read and
|
||||
the record does not. ``tests/test_metadata_record.py`` pins it over the tracked
|
||||
fixtures; those cover the signal families we already know about. This covers the ones
|
||||
we do not: every real placement in a real corpus, which is where all three defects
|
||||
found so far actually came from.
|
||||
|
||||
The record is round-tripped through ``json.dumps``/``loads`` before it is judged, so
|
||||
a value that only survives in memory fails here rather than at a customer.
|
||||
|
||||
WHAT IT REPORTS
|
||||
|
||||
One JSONL row per image: both verdicts, whether they agree, the record size, and any
|
||||
exception from either side. The summary counts disagreements by field and by signal,
|
||||
so "the record lost samsung_genai on 13 files" reads directly off the output instead
|
||||
of being reconstructed.
|
||||
|
||||
Pass ``--baseline`` with an earlier run to also diff against it. That answers the
|
||||
other question a detection change raises: which files changed verdict, and are they
|
||||
exactly the ones that were meant to.
|
||||
|
||||
DATA SAFETY
|
||||
|
||||
Read-only over a local dataset. Writes only the given output path, which belongs
|
||||
outside the repository. Resumable: rerunning skips files already recorded.
|
||||
|
||||
uv run python scripts/record_parity_audit.py data/spaces/originals .local-eval/parity.jsonl
|
||||
uv run python scripts/record_parity_audit.py <dataset> <out> --baseline .local-eval/previous.jsonl
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import collections
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
# The package's OWN tree, not the repository root: from a worktree, an editable
|
||||
# install resolves `remove_ai_watermarks` to the MAIN checkout, so a script measuring
|
||||
# this tree would silently import a different one.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
||||
|
||||
from remove_ai_watermarks.identify import identify, identify_metadata_record
|
||||
from remove_ai_watermarks.metadata_record import collect_metadata_record
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SUPPORTED = frozenset({".png", ".jpg", ".jpeg", ".webp", ".heic", ".heif", ".avif"})
|
||||
# Every field of the verdict a caller can act on. `path` is excluded: it is extraction
|
||||
# context, and the two paths are handed the same one by construction.
|
||||
COMPARED = ("is_ai_generated", "platform", "confidence", "ai_source_kind", "ai_from_metadata")
|
||||
|
||||
|
||||
def _verdict(report: Any) -> dict[str, Any]:
|
||||
return {
|
||||
**{field: getattr(report, field) for field in COMPARED},
|
||||
"signals": sorted(signal.name for signal in report.signals),
|
||||
"watermarks": sorted(report.watermarks),
|
||||
}
|
||||
|
||||
|
||||
def _audit(path: Path) -> dict[str, Any]:
|
||||
row: dict[str, Any] = {"path": str(path)}
|
||||
try:
|
||||
row["bytes"] = path.stat().st_size
|
||||
except OSError as exc:
|
||||
return {**row, "error": f"stat: {exc}"}
|
||||
|
||||
try:
|
||||
started = time.perf_counter()
|
||||
record = json.loads(json.dumps(collect_metadata_record(path)))
|
||||
row["collect_ms"] = (time.perf_counter() - started) * 1000
|
||||
row["record_bytes"] = len(json.dumps(record))
|
||||
row["container"] = record.get("container")
|
||||
via_record = _verdict(identify_metadata_record(record, path=path))
|
||||
except Exception as exc:
|
||||
return {**row, "error": f"record path: {type(exc).__name__}: {exc}"}
|
||||
|
||||
try:
|
||||
via_file = _verdict(identify(path, check_visible=False, check_invisible=False))
|
||||
except Exception as exc:
|
||||
return {**row, "error": f"file path: {type(exc).__name__}: {exc}"}
|
||||
|
||||
row["record"] = via_record
|
||||
row["file"] = via_file
|
||||
row["agree"] = via_record == via_file
|
||||
return row
|
||||
|
||||
|
||||
def _iter_images(root: Path) -> Iterator[Path]:
|
||||
for path in sorted(root.rglob("*")):
|
||||
if path.is_file() and path.suffix.lower() in SUPPORTED:
|
||||
yield path
|
||||
|
||||
|
||||
def _done(out_path: Path) -> set[str]:
|
||||
if not out_path.exists():
|
||||
return set()
|
||||
done: set[str] = set()
|
||||
with out_path.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
done.add(json.loads(line)["path"])
|
||||
except (ValueError, KeyError):
|
||||
continue
|
||||
return done
|
||||
|
||||
|
||||
def _summarize(rows: list[dict[str, Any]], baseline: Path | None) -> None:
|
||||
failed = [r for r in rows if "error" in r]
|
||||
usable = [r for r in rows if "error" not in r]
|
||||
disagreed = [r for r in usable if not r["agree"]]
|
||||
|
||||
print(f"\nimages: {len(rows)} errors: {len(failed)} compared: {len(usable)}")
|
||||
print(f"record path disagrees with file path: {len(disagreed)}")
|
||||
for row in failed[:10]:
|
||||
print(f" ERROR {Path(row['path']).name}: {row['error']}")
|
||||
|
||||
fields: collections.Counter[str] = collections.Counter()
|
||||
for row in disagreed:
|
||||
fields.update(field for field in COMPARED if row["record"][field] != row["file"][field])
|
||||
for name in set(row["file"]["signals"]) - set(row["record"]["signals"]):
|
||||
fields[f"signal missing from record: {name}"] += 1
|
||||
for name in set(row["record"]["signals"]) - set(row["file"]["signals"]):
|
||||
fields[f"signal only in record: {name}"] += 1
|
||||
for label, count in fields.most_common():
|
||||
print(f" {label}: {count}")
|
||||
for row in disagreed[:10]:
|
||||
print(f" {Path(row['path']).name}\n record: {row['record']}\n file: {row['file']}")
|
||||
|
||||
if baseline is None:
|
||||
return
|
||||
previous = {}
|
||||
with baseline.open(encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
try:
|
||||
item = json.loads(line)
|
||||
except ValueError:
|
||||
continue
|
||||
if "error" not in item:
|
||||
previous[Path(item["path"]).name] = item
|
||||
|
||||
changed = []
|
||||
for row in usable:
|
||||
was = previous.get(Path(row["path"]).name)
|
||||
if was is None:
|
||||
continue
|
||||
# The WHOLE verdict, not a chosen subset. A first version compared confidence
|
||||
# and signals only and reported "0 changed" for a run whose single intended
|
||||
# correction was a watermark line -- the change it existed to show.
|
||||
before = was.get("file") or {}
|
||||
if before and before != row["file"]:
|
||||
changed.append((row["path"], before, row["file"]))
|
||||
print(f"\nverdicts changed against the baseline: {len(changed)}")
|
||||
moved: collections.Counter[str] = collections.Counter()
|
||||
for _, before, after in changed:
|
||||
for name in set(after["signals"]) - set(before.get("signals") or []):
|
||||
moved[f"gained signal {name}"] += 1
|
||||
for name in set(before.get("signals") or []) - set(after["signals"]):
|
||||
moved[f"LOST signal {name}"] += 1
|
||||
for field in (*COMPARED, "watermarks"):
|
||||
if before.get(field) != after.get(field):
|
||||
moved[f"{field} changed"] += 1
|
||||
for label, count in moved.most_common():
|
||||
print(f" {label}: {count}")
|
||||
for path, before, after in changed[:10]:
|
||||
print(f" {Path(path).name}\n before: {before}\n after: {after}")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("dataset", type=Path)
|
||||
parser.add_argument("out", type=Path)
|
||||
parser.add_argument("--baseline", type=Path, default=None, help="an earlier run to diff verdicts against")
|
||||
parser.add_argument("--limit", type=int, default=0)
|
||||
parser.add_argument("--progress-every", type=int, default=2000)
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
|
||||
args.out.parent.mkdir(parents=True, exist_ok=True)
|
||||
done = _done(args.out)
|
||||
if done:
|
||||
log.info("resuming: %d images already audited", len(done))
|
||||
|
||||
processed = 0
|
||||
started = time.monotonic()
|
||||
with args.out.open("a", encoding="utf-8") as handle:
|
||||
for path in _iter_images(args.dataset):
|
||||
if str(path) in done:
|
||||
continue
|
||||
handle.write(json.dumps(_audit(path), ensure_ascii=False, default=str) + "\n")
|
||||
handle.flush()
|
||||
processed += 1
|
||||
if processed % args.progress_every == 0:
|
||||
log.info("%d images, %.1f/s", processed, processed / (time.monotonic() - started))
|
||||
if args.limit and processed >= args.limit:
|
||||
break
|
||||
|
||||
log.info("audited %d images in %.1f s", processed, time.monotonic() - started)
|
||||
with args.out.open(encoding="utf-8") as handle:
|
||||
rows = [json.loads(line) for line in handle]
|
||||
_summarize(rows, args.baseline)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,528 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "huggingface-hub>=0.20.0",
|
||||
# "numpy",
|
||||
# "onnxruntime>=1.24.0",
|
||||
# "opencv-python-headless<5",
|
||||
# "paddleocr>=3.3.3",
|
||||
# "paddlepaddle",
|
||||
# "pillow",
|
||||
# ]
|
||||
# ///
|
||||
"""Evaluation-only text restoration over a scrubbed image.
|
||||
|
||||
``vae-glyphs`` composites only thresholded glyph-core pixels from a separately
|
||||
generated VAE reconstruction over a fresh silhouette edge. ``source-glyphs``
|
||||
can preserve typeface, layout, color, and antialiasing when
|
||||
SOURCE is itself a regenerated layer. It must not be treated as safe when SOURCE
|
||||
is the watermarked original: a provider oracle detected SynthID after that exact
|
||||
paste-back experiment. ``source-silhouette`` instead transfers only a
|
||||
thresholded glyph shape, then synthesizes fresh flat-color pixels and
|
||||
antialiasing. ``rerender`` retains the system-font negative control. This is
|
||||
not a production stage and does not add PaddleOCR to the package graph.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import unicodedata
|
||||
from dataclasses import asdict
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import click
|
||||
import cv2
|
||||
import numpy as np
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
sys.path.insert(0, str(ROOT / "src"))
|
||||
|
||||
from remove_ai_watermarks import region_eraser # noqa: E402
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextLine as TextLine # noqa: E402
|
||||
from remove_ai_watermarks._internal.text_restoration import ( # noqa: E402
|
||||
composite_fresh_text_edges,
|
||||
composite_reconstructed_glyphs,
|
||||
group_text_lines,
|
||||
residual_glyph_mask,
|
||||
source_silhouette_mask,
|
||||
)
|
||||
from scripts._text_eval import normalize_text, normalized_edit_distance # noqa: E402
|
||||
|
||||
if ROOT not in Path(region_eraser.__file__).resolve().parents:
|
||||
raise RuntimeError("selective_text_restoration imported outside the current worktree")
|
||||
|
||||
REGULAR_FONT = Path("/System/Library/Fonts/Supplemental/Arial.ttf")
|
||||
BOLD_FONT = Path("/System/Library/Fonts/Supplemental/Arial Bold.ttf")
|
||||
CJK_FONT = Path("/System/Library/Fonts/STHeiti Medium.ttc")
|
||||
|
||||
|
||||
def should_preserve_line(
|
||||
expected: str,
|
||||
source_text: str,
|
||||
source_score: float,
|
||||
candidate_text: str,
|
||||
candidate_score: float,
|
||||
) -> bool:
|
||||
if min(source_score, candidate_score) < 0.75:
|
||||
return False
|
||||
if normalized_edit_distance(expected, source_text) > 0.25:
|
||||
return False
|
||||
return normalize_text(source_text) == normalize_text(candidate_text)
|
||||
|
||||
|
||||
def composite_source_glyphs(
|
||||
source_rgb: np.ndarray,
|
||||
background_rgb: np.ndarray,
|
||||
glyph_mask: np.ndarray,
|
||||
*,
|
||||
feather: float = 0.7,
|
||||
) -> np.ndarray:
|
||||
"""Composite exact source pixels inside a glyph mask with an outer feather."""
|
||||
if source_rgb.shape != background_rgb.shape or source_rgb.shape[:2] != glyph_mask.shape:
|
||||
raise ValueError("source, background, and glyph mask dimensions must match")
|
||||
blurred = cv2.GaussianBlur(glyph_mask, (0, 0), feather) if feather > 0 else glyph_mask
|
||||
alpha = np.maximum(glyph_mask, blurred).astype(np.float32) / 255.0
|
||||
alpha = alpha[..., None]
|
||||
combined = source_rgb.astype(np.float32) * alpha + background_rgb.astype(np.float32) * (1.0 - alpha)
|
||||
return np.clip(combined, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def source_box_mask(
|
||||
shape: tuple[int, int],
|
||||
boxes: list[tuple[int, int, int, int]],
|
||||
) -> np.ndarray:
|
||||
"""Build a padded text-line mask for aligned regenerated layer compositing."""
|
||||
height, width = shape
|
||||
mask = np.zeros((height, width), dtype=np.uint8)
|
||||
for x1, y1, x2, y2 in boxes:
|
||||
pad = max(8, (y2 - y1) // 4)
|
||||
x1, y1, x2, y2 = _clip_box((x1, y1, x2, y2), width, height, pad=pad)
|
||||
mask[y1:y2, x1:x2] = 255
|
||||
return mask
|
||||
|
||||
|
||||
def _clip_box(box: tuple[int, int, int, int], width: int, height: int, pad: int = 0) -> tuple[int, int, int, int]:
|
||||
x1, y1, x2, y2 = box
|
||||
return max(0, x1 - pad), max(0, y1 - pad), min(width, x2 + pad), min(height, y2 + pad)
|
||||
|
||||
|
||||
def foreground_mask(source_rgb: np.ndarray, box: tuple[int, int, int, int]) -> np.ndarray:
|
||||
height, width = source_rgb.shape[:2]
|
||||
line_height = box[3] - box[1]
|
||||
x1, y1, x2, y2 = _clip_box(box, width, height, pad=max(6, int(line_height * 0.12)))
|
||||
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
|
||||
ring_pad = max(8, min(24, (y2 - y1) // 5))
|
||||
rx1, ry1, rx2, ry2 = _clip_box((x1, y1, x2, y2), width, height, pad=ring_pad)
|
||||
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
|
||||
ring = np.ones(context.shape, dtype=bool)
|
||||
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
|
||||
background_luma = float(np.median(context[ring])) if ring.any() else float(np.median(gray))
|
||||
low, high = float(np.percentile(gray, 4)), float(np.percentile(gray, 96))
|
||||
dark_contrast, light_contrast = background_luma - low, high - background_luma
|
||||
contrast = max(light_contrast, dark_contrast)
|
||||
threshold = max(24.0, min(72.0, contrast * 0.32))
|
||||
if light_contrast > dark_contrast:
|
||||
mask = (gray.astype(np.float32) >= background_luma + threshold).astype(np.uint8) * 255
|
||||
else:
|
||||
mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
|
||||
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((2, 2), np.uint8))
|
||||
dilation = 5 if line_height >= 48 else 3
|
||||
mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * dilation + 1,) * 2))
|
||||
result = np.zeros((height, width), dtype=np.uint8)
|
||||
result[y1:y2, x1:x2] = mask
|
||||
return result
|
||||
|
||||
|
||||
def _line_language(line: TextLine) -> str:
|
||||
if line.script == "cjk":
|
||||
return "ch"
|
||||
return "ru" if any("CYRILLIC" in unicodedata.name(character, "") for character in line.text) else "en"
|
||||
|
||||
|
||||
def _recognition_box(
|
||||
line: TextLine,
|
||||
width: int,
|
||||
height: int,
|
||||
vertical_pad_ratio: float | None = None,
|
||||
) -> tuple[int, int, int, int]:
|
||||
x1, y1, x2, y2 = line.box
|
||||
line_height = y2 - y1
|
||||
if line.script == "cjk":
|
||||
left_pad = max(16, round(line_height * 0.2))
|
||||
right_pad = max(16, round(line_height * 0.6))
|
||||
return max(0, x1 - left_pad), y1, min(width, x2 + right_pad), y2
|
||||
pad_x = max(16, line_height)
|
||||
pad_y = max(8, line_height // 3) if vertical_pad_ratio is None else max(8, round(line_height * vertical_pad_ratio))
|
||||
return max(0, x1 - pad_x), max(0, y1 - pad_y), min(width, x2 + pad_x), min(height, y2 + pad_y)
|
||||
|
||||
|
||||
def _recognize(
|
||||
engine: Any,
|
||||
image: np.ndarray,
|
||||
line: TextLine,
|
||||
vertical_pad_ratio: float | None = None,
|
||||
) -> tuple[str, float]:
|
||||
height, width = image.shape[:2]
|
||||
x1, y1, x2, y2 = _recognition_box(line, width, height, vertical_pad_ratio)
|
||||
crop = image[y1:y2, x1:x2]
|
||||
if crop.shape[0] < 64:
|
||||
scale = 64 / crop.shape[0]
|
||||
crop = cv2.resize(crop, None, fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)
|
||||
result = next(iter(engine.predict(crop)))
|
||||
return str(result.get("rec_text", "")), float(result.get("rec_score", 0.0))
|
||||
|
||||
|
||||
def _sample_text_color(
|
||||
source_rgb: np.ndarray,
|
||||
mask: np.ndarray,
|
||||
box: tuple[int, int, int, int],
|
||||
) -> tuple[int, int, int]:
|
||||
height, width = source_rgb.shape[:2]
|
||||
x1, y1, x2, y2 = _clip_box(box, width, height, pad=2)
|
||||
crop = source_rgb[y1:y2, x1:x2]
|
||||
active = mask[y1:y2, x1:x2] > 0
|
||||
pixels = crop[active]
|
||||
luma = pixels.mean(axis=1)
|
||||
background_luma = float(crop[[0, -1], :, :].reshape(-1, 3).mean(axis=1).mean())
|
||||
if background_luma >= 128:
|
||||
selected = pixels[luma <= np.percentile(luma, 20)]
|
||||
else:
|
||||
selected = pixels[luma >= np.percentile(luma, 80)]
|
||||
return tuple(int(value) for value in np.median(selected, axis=0))
|
||||
|
||||
|
||||
def _render_line(image: Image.Image, line: TextLine, color: tuple[int, int, int]) -> None:
|
||||
font_path = CJK_FONT if line.script == "cjk" else (BOLD_FONT if line.box[3] - line.box[1] >= 55 else REGULAR_FONT)
|
||||
target_width, target_height = line.box[2] - line.box[0], line.box[3] - line.box[1]
|
||||
draw = ImageDraw.Draw(image)
|
||||
low, high = 4, max(8, target_height * 2)
|
||||
font = ImageFont.truetype(str(font_path), low)
|
||||
while low <= high:
|
||||
size = (low + high) // 2
|
||||
candidate = ImageFont.truetype(str(font_path), size)
|
||||
bounds = draw.textbbox((0, 0), line.text, font=candidate)
|
||||
if bounds[2] - bounds[0] <= target_width * 1.03 and bounds[3] - bounds[1] <= target_height * 1.08:
|
||||
font, low = candidate, size + 1
|
||||
else:
|
||||
high = size - 1
|
||||
bounds = draw.textbbox((0, 0), line.text, font=font)
|
||||
y = line.box[1] + math.floor((target_height - (bounds[3] - bounds[1])) / 2) - bounds[1]
|
||||
draw.text((line.box[0], y), line.text, fill=color, font=font)
|
||||
|
||||
|
||||
def _write_manifest(path: Path | None, payload: dict[str, Any]) -> None:
|
||||
if path is None:
|
||||
return
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
|
||||
def _vertical_overlap_ratio(left: tuple[int, int, int, int], right: tuple[int, int, int, int]) -> float:
|
||||
overlap = max(0, min(left[3], right[3]) - max(left[1], right[1]))
|
||||
return overlap / max(1, min(left[3] - left[1], right[3] - right[1]))
|
||||
|
||||
|
||||
def group_word_boxes(boxes: list[tuple[int, int, int, int]]) -> list[tuple[int, int, int, int]]:
|
||||
groups: list[tuple[int, int, int, int]] = []
|
||||
for box in sorted(boxes, key=lambda item: ((item[1] + item[3]) / 2, item[0])):
|
||||
matches = []
|
||||
for index, group in enumerate(groups):
|
||||
if _vertical_overlap_ratio(box, group) < 0.45:
|
||||
continue
|
||||
horizontal_gap = max(0, max(box[0], group[0]) - min(box[2], group[2]))
|
||||
line_height = min(box[3] - box[1], group[3] - group[1])
|
||||
if horizontal_gap <= max(24, line_height * 3):
|
||||
matches.append(index)
|
||||
if not matches:
|
||||
groups.append(box)
|
||||
continue
|
||||
index = max(matches, key=lambda item: _vertical_overlap_ratio(box, groups[item]))
|
||||
x1, y1, x2, y2 = groups[index]
|
||||
groups[index] = min(x1, box[0]), min(y1, box[1]), max(x2, box[2]), max(y2, box[3])
|
||||
return sorted(groups, key=lambda item: ((item[1] + item[3]) / 2, item[0]))
|
||||
|
||||
|
||||
def detect_line_boxes(
|
||||
engine: Any,
|
||||
source_rgb: np.ndarray,
|
||||
expected_count: int | None = None,
|
||||
) -> list[tuple[int, int, int, int]]:
|
||||
boxes: list[tuple[int, int, int, int]] = []
|
||||
for page in engine.predict(source_rgb):
|
||||
detected = page.get("rec_boxes", None)
|
||||
if detected is None or len(detected) == 0:
|
||||
detected = page.get("rec_polys", [])
|
||||
for score, raw_box in zip(page.get("rec_scores", []), detected, strict=False):
|
||||
if float(score) < 0.5:
|
||||
continue
|
||||
points = np.asarray(raw_box, dtype=np.float32).reshape(-1)
|
||||
if points.size == 4:
|
||||
x1, y1, x2, y2 = points
|
||||
else:
|
||||
points = points.reshape(-1, 2)
|
||||
x1, y1 = points.min(axis=0)
|
||||
x2, y2 = points.max(axis=0)
|
||||
boxes.append((round(float(x1)), round(float(y1)), round(float(x2)), round(float(y2))))
|
||||
lines = group_word_boxes(boxes)
|
||||
if expected_count is not None and len(lines) != expected_count:
|
||||
raise click.ClickException(f"detected {len(lines)} source lines; expected exactly {expected_count}")
|
||||
return lines
|
||||
|
||||
|
||||
def _load_lines(path: Path, key: str) -> list[TextLine]:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
return [
|
||||
TextLine(tuple(item["box"]), item["text"], item["script"], float(item.get("angle", 0.0)))
|
||||
for item in payload[key]
|
||||
]
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.argument("candidate", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option("--output", required=True, type=click.Path(dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"--lines-json",
|
||||
default=ROOT / "data/evaluations/fidelity/text-lines.json",
|
||||
type=click.Path(exists=True, dir_okay=False, path_type=Path),
|
||||
)
|
||||
@click.option("--source-key", help="Key in lines JSON; defaults to source basename.")
|
||||
@click.option(
|
||||
"--detect-boxes",
|
||||
is_flag=True,
|
||||
help="Detect source boxes; fail unless their count matches verified lines.",
|
||||
)
|
||||
@click.option(
|
||||
"--restoration",
|
||||
type=click.Choice(("vae-glyphs", "source-glyphs", "source-silhouette", "rerender")),
|
||||
required=True,
|
||||
help="Choose VAE glyph cores, regenerated pixels, fresh source shapes, or the system-font control.",
|
||||
)
|
||||
@click.option(
|
||||
"--glyph-donor",
|
||||
type=click.Path(exists=True, dir_okay=False, path_type=Path),
|
||||
help="VAE reconstruction used only by --restoration vae-glyphs.",
|
||||
)
|
||||
@click.option(
|
||||
"--glyph-feather",
|
||||
type=click.FloatRange(min=0.0),
|
||||
default=0.5,
|
||||
show_default=True,
|
||||
help="Outer donor-edge feather used only by --restoration vae-glyphs.",
|
||||
)
|
||||
@click.option(
|
||||
"--selection",
|
||||
type=click.Choice(("all", "changed")),
|
||||
default="all",
|
||||
show_default=True,
|
||||
help="Restore every verified line or only OCR-confirmed changes.",
|
||||
)
|
||||
@click.option(
|
||||
"--erase-background/--keep-background",
|
||||
default=True,
|
||||
show_default=True,
|
||||
help="Erase candidate glyphs before compositing, or directly blend an aligned regenerated glyph layer.",
|
||||
)
|
||||
@click.option(
|
||||
"--composite-mask",
|
||||
type=click.Choice(("glyphs", "boxes")),
|
||||
default="glyphs",
|
||||
show_default=True,
|
||||
help="Composite isolated glyphs or complete aligned text-line boxes.",
|
||||
)
|
||||
@click.option("--manifest", type=click.Path(dir_okay=False, path_type=Path))
|
||||
def main(
|
||||
source: Path,
|
||||
candidate: Path,
|
||||
output: Path,
|
||||
lines_json: Path,
|
||||
source_key: str | None,
|
||||
detect_boxes: bool,
|
||||
restoration: str,
|
||||
glyph_donor: Path | None,
|
||||
glyph_feather: float,
|
||||
selection: str,
|
||||
erase_background: bool,
|
||||
composite_mask: str,
|
||||
manifest: Path | None,
|
||||
) -> None:
|
||||
"""Restore SOURCE text over the scrubbed CANDIDATE."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
|
||||
if restoration == "rerender":
|
||||
for font in (REGULAR_FONT, BOLD_FONT, CJK_FONT):
|
||||
if not font.exists():
|
||||
raise click.ClickException(f"required evaluation font is unavailable: {font}")
|
||||
if restoration == "vae-glyphs" and glyph_donor is None:
|
||||
raise click.ClickException("--glyph-donor is required for --restoration vae-glyphs")
|
||||
if restoration != "vae-glyphs" and glyph_donor is not None:
|
||||
raise click.ClickException("--glyph-donor is only valid with --restoration vae-glyphs")
|
||||
os.environ["PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK"] = "True"
|
||||
source_rgb = np.asarray(Image.open(source).convert("RGB"))
|
||||
candidate_rgb = np.asarray(Image.open(candidate).convert("RGB"))
|
||||
if candidate_rgb.shape != source_rgb.shape:
|
||||
raise click.ClickException("source and candidate dimensions must match")
|
||||
donor_rgb = np.asarray(Image.open(glyph_donor).convert("RGB")) if glyph_donor else None
|
||||
if donor_rgb is not None and donor_rgb.shape != source_rgb.shape:
|
||||
raise click.ClickException("source and glyph donor dimensions must match")
|
||||
lines = _load_lines(lines_json, source_key or source.name)
|
||||
annotation_boxes = [line.box for line in lines]
|
||||
if detect_boxes:
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
page_engine = PaddleOCR(
|
||||
lang="ch",
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
)
|
||||
boxes = detect_line_boxes(page_engine, source_rgb, len(lines))
|
||||
lines = [TextLine(box, line.text, line.script) for line, box in zip(lines, boxes, strict=True)]
|
||||
vertical_pad_ratio = 0.1 if detect_boxes else None
|
||||
decisions = []
|
||||
selected = list(lines)
|
||||
if selection == "all":
|
||||
decisions = [{"line": asdict(line), "selected": True, "reason": "all-lines"} for line in lines]
|
||||
else:
|
||||
from paddleocr import TextRecognition
|
||||
|
||||
engines = {
|
||||
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec"),
|
||||
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec"),
|
||||
"ch": TextRecognition(model_name="PP-OCRv5_server_rec"),
|
||||
}
|
||||
selected = []
|
||||
for line in lines:
|
||||
language = _line_language(line)
|
||||
source_text, source_score = _recognize(engines[language], source_rgb, line, vertical_pad_ratio)
|
||||
candidate_text, candidate_score = _recognize(engines[language], candidate_rgb, line, vertical_pad_ratio)
|
||||
preserve = should_preserve_line(line.text, source_text, source_score, candidate_text, candidate_score)
|
||||
decisions.append(
|
||||
{
|
||||
"line": asdict(line),
|
||||
"source_text": source_text,
|
||||
"source_score": source_score,
|
||||
"candidate_text": candidate_text,
|
||||
"candidate_score": candidate_score,
|
||||
"preserve": preserve,
|
||||
"selected": not preserve,
|
||||
}
|
||||
)
|
||||
if not preserve:
|
||||
selected.append(line)
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
mask_path = output.with_name(output.stem + "_mask.png")
|
||||
manifest_common = {
|
||||
"source": source.name,
|
||||
"candidate": candidate.name,
|
||||
"output": output.name,
|
||||
"mask": mask_path.name,
|
||||
"glyph_donor": glyph_donor.name if glyph_donor else None,
|
||||
"glyph_feather": glyph_feather if restoration == "vae-glyphs" else None,
|
||||
"restoration": restoration,
|
||||
"selection": selection,
|
||||
"erase_background": erase_background,
|
||||
"composite_mask": composite_mask,
|
||||
"box_source": "detector" if detect_boxes else "verified_annotations",
|
||||
"annotation_boxes": annotation_boxes,
|
||||
"decisions": decisions,
|
||||
}
|
||||
if not selected:
|
||||
combined = np.zeros(source_rgb.shape[:2], dtype=np.uint8)
|
||||
shutil.copyfile(candidate, output)
|
||||
Image.fromarray(combined).save(mask_path)
|
||||
payload = {
|
||||
**manifest_common,
|
||||
"mask_fraction": 0.0,
|
||||
"source_glyph_fraction": 0.0,
|
||||
"source_layer_fraction": 0.0,
|
||||
}
|
||||
_write_manifest(manifest, payload)
|
||||
log.info("Copied %s unchanged because every line passed", output)
|
||||
return
|
||||
if restoration in {"source-silhouette", "vae-glyphs"}:
|
||||
source_masks = [source_silhouette_mask(source_rgb, line.box, line.angle) for line in selected]
|
||||
candidate_masks = [source_silhouette_mask(candidate_rgb, line.box, line.angle) for line in selected]
|
||||
line_masks = []
|
||||
for line, source_mask, candidate_mask in zip(selected, source_masks, candidate_masks, strict=True):
|
||||
radius = 5 if line.box[3] - line.box[1] >= 48 else 3
|
||||
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * radius + 1,) * 2)
|
||||
line_masks.append(cv2.dilate(np.maximum(source_mask, candidate_mask), kernel))
|
||||
masks = line_masks
|
||||
else:
|
||||
source_masks = [foreground_mask(source_rgb, line.box) for line in selected]
|
||||
candidate_masks = [foreground_mask(candidate_rgb, line.box) for line in selected]
|
||||
masks = [np.maximum(left, right) for left, right in zip(source_masks, candidate_masks, strict=True)]
|
||||
del candidate_masks
|
||||
groups = group_text_lines(selected)
|
||||
if erase_background:
|
||||
background = cv2.cvtColor(candidate_rgb, cv2.COLOR_RGB2BGR)
|
||||
for group in groups:
|
||||
background = region_eraser.erase_lama(
|
||||
background,
|
||||
np.maximum.reduce([masks[index] for index in group]),
|
||||
)
|
||||
background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
|
||||
residual_masks = [
|
||||
residual_glyph_mask(background_rgb, mask, line.box) for line, mask in zip(selected, masks, strict=True)
|
||||
]
|
||||
for group in groups:
|
||||
residual = np.maximum.reduce([residual_masks[index] for index in group])
|
||||
if np.any(residual):
|
||||
background = region_eraser.erase_lama(background, residual)
|
||||
background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
|
||||
else:
|
||||
background_rgb = candidate_rgb
|
||||
residual_masks = []
|
||||
source_glyph_mask = np.maximum.reduce(source_masks)
|
||||
source_layer_mask = source_glyph_mask
|
||||
if restoration == "source-glyphs":
|
||||
source_layer_mask = (
|
||||
source_box_mask(source_rgb.shape[:2], [line.box for line in selected])
|
||||
if composite_mask == "boxes"
|
||||
else source_glyph_mask
|
||||
)
|
||||
restored = composite_source_glyphs(source_rgb, background_rgb, source_layer_mask, feather=3.0)
|
||||
Image.fromarray(restored).save(output)
|
||||
elif restoration in {"source-silhouette", "vae-glyphs"}:
|
||||
restored = composite_fresh_text_edges(source_rgb, background_rgb, selected, source_masks)
|
||||
if restoration == "vae-glyphs":
|
||||
if donor_rgb is None:
|
||||
raise RuntimeError("VAE glyph restoration requires a loaded donor")
|
||||
restored = composite_reconstructed_glyphs(
|
||||
donor_rgb,
|
||||
restored,
|
||||
source_layer_mask,
|
||||
feather=glyph_feather,
|
||||
)
|
||||
Image.fromarray(restored).save(output)
|
||||
else:
|
||||
rendered = Image.fromarray(background_rgb)
|
||||
for line, source_mask in zip(selected, source_masks, strict=True):
|
||||
_render_line(rendered, line, _sample_text_color(source_rgb, source_mask, line.box))
|
||||
rendered.save(output)
|
||||
combined = np.maximum.reduce([*masks, *residual_masks])
|
||||
if restoration == "source-glyphs" and not erase_background:
|
||||
combined = source_layer_mask
|
||||
Image.fromarray(combined).save(mask_path)
|
||||
payload = {
|
||||
**manifest_common,
|
||||
"mask_fraction": float((combined > 0).mean()),
|
||||
"source_glyph_fraction": float((source_glyph_mask > 0).mean()),
|
||||
"source_layer_fraction": float((source_layer_mask > 0).mean()),
|
||||
}
|
||||
_write_manifest(manifest, payload)
|
||||
log.info("Wrote %s with %.4f edited fraction", output, payload["mask_fraction"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+61
-104
@@ -17,9 +17,10 @@ WHAT IT COVERS AND WHY THAT SHAPE
|
||||
work" report); a no-op must be byte-identical; `metadata --remove` must actually
|
||||
strip; a JPEG strip must not touch the pixels. Note the pixel-lossless contract is
|
||||
the DEFAULT path's -- `--remove-all` deliberately re-encodes (see metadata.py).
|
||||
* The diffusion bodies under `--diffusion`, at `--max-resolution 512` so they fit MPS
|
||||
(~1 min/image on 32 GB unified memory). Not just exit codes: `invisible` must
|
||||
restore the input resolution and must NOT re-stamp SDXL's own open watermark.
|
||||
* The diffusion bodies under `--diffusion`, at a small `--max-resolution`. Both
|
||||
profiles are CUDA-only, so those rows need an NVIDIA GPU and are reported as
|
||||
skips elsewhere. Not just exit codes: `invisible` must restore the input
|
||||
resolution and must NOT re-stamp SDXL's own open watermark.
|
||||
|
||||
WHAT IT DOES NOT COVER, DELIBERATELY AND LOUDLY
|
||||
* Without `--diffusion`, the model-running bodies are reported as SKIPPED with a
|
||||
@@ -34,7 +35,7 @@ WHAT IT DOES NOT COVER, DELIBERATELY AND LOUDLY
|
||||
|
||||
uv run python scripts/smoke_matrix.py # corpus + fixtures
|
||||
uv run python scripts/smoke_matrix.py --quick # fixtures only, no corpus
|
||||
uv run python scripts/smoke_matrix.py --diffusion # + the SDXL model paths
|
||||
uv run python scripts/smoke_matrix.py --diffusion # + the model paths (needs CUDA)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -113,7 +114,7 @@ def main() -> None:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--quick", action="store_true", help="fixtures only; skip the corpus rows")
|
||||
ap.add_argument(
|
||||
"--diffusion", action="store_true", help="also run the model-running paths (SDXL weights, ~1 min/image)"
|
||||
"--diffusion", action="store_true", help="also run the model-running paths (needs CUDA, ~1 min/image)"
|
||||
)
|
||||
a = ap.parse_args()
|
||||
|
||||
@@ -303,7 +304,7 @@ def main() -> None:
|
||||
_diffusion_rows(r, tmp, doubao)
|
||||
else:
|
||||
for name in ("invisible", "all", "batch --mode invisible"):
|
||||
r.skip(f"{name} (model-running body)", "pass --diffusion to exercise it (needs the SDXL weights)")
|
||||
r.skip(f"{name} (model-running body)", "pass --diffusion to exercise it (needs CUDA and the model weights)")
|
||||
|
||||
# ---- real-data formats and shapes ---------------------------------------
|
||||
if not a.quick:
|
||||
@@ -354,18 +355,18 @@ def main() -> None:
|
||||
def _knob_rows(r: Runner, tmp: Path, img: Path) -> None:
|
||||
"""Every diffusion knob the matrix never touched.
|
||||
|
||||
Deliberately cheap (`--steps 4`, `--max-resolution 384`): these rows answer "is the
|
||||
knob accepted and does the run complete", NOT "is the output good". Quality per knob
|
||||
needs a per-knob oracle and most of them have none (`--humanize` has no oracle at
|
||||
all), so claiming more here would be dishonest.
|
||||
Deliberately cheap (`--max-resolution 384`): these rows answer "is the knob accepted
|
||||
and does the run complete", NOT "is the output good". Quality per knob needs a
|
||||
per-knob oracle and most of them have none (`--humanize` has no oracle at all), so
|
||||
claiming more here would be dishonest.
|
||||
"""
|
||||
from remove_ai_watermarks import upscaler
|
||||
|
||||
# --steps 20 is the floor that WORKS, not an arbitrary choice: effective timesteps
|
||||
# are int(steps * strength), so at the default strength 0.15 anything below
|
||||
# --steps 7 rounds to ZERO and the pipeline dies inside torch. The first version of
|
||||
# these rows used --steps 4 and every single one failed with the same reshape error.
|
||||
fast = ["--max-resolution", "384", "--min-resolution", "0", "--steps", "20", "--force", "--seed", "0"]
|
||||
# Both surviving profiles are CUDA-only, pin a fixed model stack and let each stage
|
||||
# own its schedule and CFG. The knobs that used to contradict that (--model, --steps,
|
||||
# --guidance-scale, --device, --auto) are gone from the parser, so the rows below
|
||||
# assert Click itself refuses them. That is the coverage worth having: an option the
|
||||
# CLI accepts and the library refuses several layers down is what this matrix exists
|
||||
# to catch, and the cheapest way to keep it caught is for the option not to exist.
|
||||
fast = ["--max-resolution", "384", "--force", "--seed", "0"]
|
||||
|
||||
def run(name: str, extra: list[str], *, tag: str, expect: int | None = 0) -> None:
|
||||
r.run(
|
||||
@@ -375,65 +376,53 @@ def _knob_rows(r: Runner, tmp: Path, img: Path) -> None:
|
||||
timeout=2400,
|
||||
)
|
||||
|
||||
run("--pipeline sdxl", ["--pipeline", "sdxl"], tag="sdxl")
|
||||
run("--pipeline controlnet", ["--pipeline", "controlnet"], tag="cnet")
|
||||
run("--strength", ["--strength", "0.2"], tag="strength")
|
||||
run("--guidance-scale", ["--guidance-scale", "5.0"], tag="gs")
|
||||
run("--controlnet-scale", ["--controlnet-scale", "0.5"], tag="cns")
|
||||
run("--humanize", ["--humanize", "0.3"], tag="hum")
|
||||
run("--unsharp", ["--unsharp", "0.5"], tag="uns")
|
||||
run("--no-adaptive-polish", ["--no-adaptive-polish"], tag="nap")
|
||||
run("--tile", ["--tile", "--tile-size", "256", "--tile-overlap", "64"], tag="tile")
|
||||
run("--device mps", ["--device", "mps"], tag="mps")
|
||||
run("--upscaler lanczos", ["--upscaler", "lanczos"], tag="lanczos")
|
||||
run("--auto (deprecated no-op)", ["--auto"], tag="auto")
|
||||
# Click rejects a retired profile at parse time (exit 2) rather than remapping it.
|
||||
for retired in ("sdxl", "controlnet", "qwen", "default"):
|
||||
run(f"--pipeline {retired} is rejected", ["--pipeline", retired], tag=f"retired_{retired}", expect=2)
|
||||
|
||||
if upscaler.is_available():
|
||||
run("--upscaler esrgan", ["--upscaler", "esrgan"], tag="esrgan")
|
||||
else:
|
||||
r.skip("--upscaler esrgan", "the `esrgan` extra is not installed")
|
||||
# Retired options: no longer parsed at all (exit 2, "No such option").
|
||||
for args, tag in (
|
||||
(["--steps", "20"], "steps20"),
|
||||
(["--guidance-scale", "5.0"], "gs5"),
|
||||
(["--model", "org/custom"], "model"),
|
||||
(["--device", "cpu"], "devcpu"),
|
||||
(["--auto"], "auto"),
|
||||
):
|
||||
run(f"{args[0]} is no longer an option", args, tag=tag, expect=2)
|
||||
|
||||
# CPU is correctness-relevant (it is the documented MPS-OOM fallback) but slow, so
|
||||
# it gets the smallest possible run rather than being skipped.
|
||||
r.run(
|
||||
"--device cpu",
|
||||
[
|
||||
"invisible",
|
||||
str(img),
|
||||
"-o",
|
||||
str(tmp / "k_cpu.png"),
|
||||
"--device",
|
||||
"cpu",
|
||||
"--max-resolution",
|
||||
"256",
|
||||
"--min-resolution",
|
||||
"0",
|
||||
"--steps",
|
||||
"20",
|
||||
"--force",
|
||||
"--seed",
|
||||
"0",
|
||||
],
|
||||
timeout=3600,
|
||||
)
|
||||
|
||||
# qwen is CUDA-class by design (bf16 MMDiT, no MPS fallback). On this host the
|
||||
# honest outcome is a CLEAN failure, not a crash -- assert it does not hang or
|
||||
# dump a traceback at the user.
|
||||
try:
|
||||
import torch
|
||||
|
||||
cuda = bool(torch.cuda.is_available())
|
||||
except Exception:
|
||||
cuda = False
|
||||
if cuda:
|
||||
run("--pipeline qwen", ["--pipeline", "qwen"], tag="qwen")
|
||||
else:
|
||||
r.skip("--pipeline qwen", "CUDA-class pipeline; no CUDA device on this host")
|
||||
|
||||
# --model and --hf-token are deliberately not exercised: one would download a second
|
||||
# multi-GB checkpoint, the other needs a real credential. Skipped loudly, not passed.
|
||||
r.skip("--model", "would download a second multi-GB checkpoint")
|
||||
if not cuda:
|
||||
# Without CUDA the honest outcome is a CLEAN refusal naming the reason, not a
|
||||
# traceback from inside a half-built pipeline.
|
||||
run("no CUDA fails cleanly", [], tag="nocuda", expect=1)
|
||||
for label in (
|
||||
"--strength",
|
||||
"--controlnet-scale",
|
||||
"--humanize",
|
||||
"--unsharp",
|
||||
"--no-adaptive-polish",
|
||||
"--tile",
|
||||
"--pipeline sdxl-zimage",
|
||||
):
|
||||
r.skip(label, "accepted-knob rows need a CUDA device")
|
||||
return
|
||||
|
||||
run("--pipeline sdxl-zimage", ["--pipeline", "sdxl-zimage"], tag="sdxlz")
|
||||
run("--strength", ["--strength", "0.2"], tag="strength")
|
||||
run("--controlnet-scale", ["--controlnet-scale", "0.5"], tag="cns")
|
||||
run("--humanize", ["--humanize", "0.3"], tag="hum")
|
||||
run("--unsharp", ["--unsharp", "0.5"], tag="uns")
|
||||
run("--no-adaptive-polish", ["--no-adaptive-polish"], tag="nap")
|
||||
run("--tile", ["--tile", "--tile-size", "256", "--tile-overlap", "64"], tag="tile")
|
||||
|
||||
# --hf-token needs a real credential, so it cannot be exercised meaningfully here.
|
||||
# Skipped loudly rather than silently passed over.
|
||||
r.skip("--hf-token", "needs a real credential; cannot be exercised meaningfully here")
|
||||
|
||||
# CONTRACT, not just execution: the same seed must reproduce the same pixels.
|
||||
@@ -497,7 +486,7 @@ def _sdxl_watermark_bits(img: object) -> float:
|
||||
|
||||
|
||||
def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
"""Exercise the model-running bodies at a reduced resolution (MPS-friendly).
|
||||
"""Exercise the model-running bodies at a reduced resolution (CUDA required).
|
||||
|
||||
Bounded with `--max-resolution 512` and a fixed seed: the point is that the paths
|
||||
RUN and keep their contracts, not to certify removal strength (that needs the
|
||||
@@ -519,7 +508,7 @@ def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
# OUTPUT and the row degrades to a skip rather than a false pass if it goes fragile.
|
||||
mj = SAMPLES / "mj-1.png"
|
||||
inv = tmp / "inv_mj.png"
|
||||
res = r.run("invisible runs (mps, 512px)", ["invisible", str(mj), "-o", str(inv), "--force", *small], timeout=1800)
|
||||
res = r.run("invisible runs (512px)", ["invisible", str(mj), "-o", str(inv), "--force", *small], timeout=1800)
|
||||
if res.status == "pass" and inv.exists():
|
||||
src_img, out_img = imread(str(mj)), imread(str(inv))
|
||||
r.check(
|
||||
@@ -547,7 +536,7 @@ def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
|
||||
# `all`: every stage must land -- the visible mark AND the metadata both gone.
|
||||
allout = tmp / "all_out.png"
|
||||
res = r.run("all runs (mps, 512px)", ["all", str(doubao), "-o", str(allout), *small], timeout=1800)
|
||||
res = r.run("all runs (512px)", ["all", str(doubao), "-o", str(allout), *small], timeout=1800)
|
||||
if res.status == "pass" and allout.exists():
|
||||
rep = json.loads(_capture(["identify", str(allout), "--json"]))
|
||||
r.check(
|
||||
@@ -566,7 +555,7 @@ def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
shutil.copy(f, bd / f.name)
|
||||
bout = tmp / "batch_inv_out"
|
||||
res = r.run(
|
||||
"batch --mode invisible runs (mps, 512px)",
|
||||
"batch --mode invisible runs (512px)",
|
||||
["batch", str(bd), "--mode", "invisible", "-o", str(bout), *small],
|
||||
expect_exit=None,
|
||||
timeout=3600,
|
||||
@@ -578,44 +567,12 @@ def _diffusion_rows(r: Runner, tmp: Path, doubao: Path) -> None:
|
||||
# `batch --mode all` -- the only --mode value the matrix never ran.
|
||||
aout = tmp / "batch_all_out"
|
||||
r.run(
|
||||
"batch --mode all runs (mps, 512px)",
|
||||
"batch --mode all runs (512px)",
|
||||
["batch", str(bd), "--mode", "all", "-o", str(aout), *small],
|
||||
expect_exit=None,
|
||||
timeout=3600,
|
||||
)
|
||||
|
||||
# The AI-enhanced composite path: regenerate ONLY a region and feather it back,
|
||||
# leaving everything outside the box pixel-exact. Library-level -- the CLI has no
|
||||
# flag for it, so it would otherwise never be exercised on real data.
|
||||
try:
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
|
||||
src = imread(str(mj))
|
||||
h, w = src.shape[:2]
|
||||
box = (w // 4, h // 4, w // 4, h // 4)
|
||||
rem = WatermarkRemover(pipeline="controlnet")
|
||||
rout = tmp / "region_composite.png"
|
||||
rem.remove_watermark(mj, rout, strength=0.15, num_inference_steps=20, seed=0, region=box)
|
||||
got = imread(str(rout))
|
||||
if got is None or got.shape != src.shape:
|
||||
r.check("region composite keeps the frame outside the box", False, "shape changed or unreadable")
|
||||
else:
|
||||
mask = np.ones(src.shape[:2], dtype=bool)
|
||||
x, y, bw, bh = box
|
||||
# Outside the box PLUS the feather margin must be untouched.
|
||||
pad = 96
|
||||
mask[max(0, y - pad) : y + bh + pad, max(0, x - pad) : x + bw + pad] = False
|
||||
r.check(
|
||||
"region composite keeps the frame outside the box",
|
||||
bool(np.array_equal(src[mask], got[mask])),
|
||||
"pixels changed outside the regenerated region",
|
||||
)
|
||||
except Exception as e:
|
||||
r.skip("region composite (remove_watermark(region=...))", f"{type(e).__name__}: {e}"[:120])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,53 +0,0 @@
|
||||
"""Update the conda recipe to a published package version and sdist hash."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
_VERSION_LINE = re.compile(r'^ version: "[^"]+"$', re.MULTILINE)
|
||||
_SHA256_LINE = re.compile(r"^ sha256: [0-9a-f]{64}$", re.MULTILINE)
|
||||
|
||||
|
||||
def update_recipe(text: str, *, version: str, sha256: str) -> str:
|
||||
"""Return recipe text with exactly one context version and source hash updated."""
|
||||
if not version or any(char.isspace() for char in version):
|
||||
raise ValueError("version must be a non-empty value without whitespace")
|
||||
if re.fullmatch(r"[0-9a-f]{64}", sha256) is None:
|
||||
raise ValueError("sha256 must be 64 lowercase hexadecimal characters")
|
||||
|
||||
updated, version_count = _VERSION_LINE.subn(f' version: "{version}"', text)
|
||||
updated, sha_count = _SHA256_LINE.subn(f" sha256: {sha256}", updated)
|
||||
if version_count != 1 or sha_count != 1:
|
||||
raise ValueError(
|
||||
"expected exactly one context version and one source sha256 "
|
||||
f"(found version={version_count}, sha256={sha_count})"
|
||||
)
|
||||
return updated
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Update the selected recipe in place."""
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--version", required=True)
|
||||
parser.add_argument("--sha256", required=True)
|
||||
parser.add_argument(
|
||||
"--recipe",
|
||||
type=Path,
|
||||
default=Path("packaging/conda/recipe.yaml"),
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
original = args.recipe.read_text()
|
||||
updated = update_recipe(original, version=args.version, sha256=args.sha256)
|
||||
args.recipe.write_text(updated)
|
||||
log.info("Updated %s to version %s", args.recipe, args.version)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
|
||||
main()
|
||||
@@ -22,7 +22,7 @@ import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from collections import Counter
|
||||
from datetime import datetime, timezone
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
@@ -180,7 +180,7 @@ def ingest(
|
||||
"c2pa_issuer": issuer,
|
||||
"synthid_metadata": "yes" if synthid else "",
|
||||
"verified_via": verified_via,
|
||||
"added": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"added": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"notes": notes,
|
||||
}
|
||||
)
|
||||
|
||||
@@ -62,17 +62,10 @@ OUT = REPO / ".local-eval" / "vendor-cohorts.jsonl"
|
||||
FIRED = REPO / ".local-eval" / "visible-positives.jsonl"
|
||||
SHEET_DIR = REPO / ".local-eval" / "vendor-cohort-sheets"
|
||||
|
||||
# A producer code is `001` + `1` + USCC(18) + a 5-digit app/product suffix, so two
|
||||
# codes sharing the USCC are the same legal entity registering different products.
|
||||
# Slicing is defensive: anything not matching the layout is grouped by its raw value.
|
||||
_USCC_START, _USCC_END = 4, 22
|
||||
|
||||
|
||||
def uscc_of(code: str) -> str:
|
||||
"""The 18-char Unified Social Credit Code embedded in a TC260 producer code."""
|
||||
if len(code) >= _USCC_END and code[:3] == "001":
|
||||
return code[_USCC_START:_USCC_END]
|
||||
return code
|
||||
# `uscc_of` moved into the library (`metadata.uscc_of`) when the USCC -> vendor table
|
||||
# started driving `api.visible_provenance`; this script must group by the same rule the
|
||||
# product uses, so it imports rather than reimplements it.
|
||||
from remove_ai_watermarks.metadata import uscc_of # noqa: E402
|
||||
|
||||
|
||||
def _one(path_str: str) -> dict[str, Any] | None:
|
||||
|
||||
@@ -97,7 +97,7 @@ def _score(args: ScoreArgs) -> dict[str, Any] | None:
|
||||
eng = TextMarkEngine(build_config(asset, name, basis, overrides))
|
||||
loc = eng.locate(img)
|
||||
try:
|
||||
score, box = eng._tophat_best(img, loc)
|
||||
score, box = eng._ladder_best(img, loc)
|
||||
except Exception:
|
||||
return None
|
||||
return {"path": path_str, "score": round(float(score), 4), "box": box}
|
||||
@@ -225,7 +225,7 @@ def sheets(pos: list[dict[str, Any]], name: str, per_sheet: int = 24) -> None:
|
||||
# genuinely unknown, which is the one case a dense ladder earns its cost -- but it also
|
||||
# hands clean corners extra chances to match, so it must never set a gate.
|
||||
_FIT_SCALES = tuple(round(0.4 * (1.03**i), 4) for i in range(80)) # 0.40 .. ~4.1
|
||||
# The ladder the product actually ships (`_text_mark_engine._tophat_best`).
|
||||
# The ladder the product actually ships (`_text_mark_engine._ladder_best`).
|
||||
_SHIPPED_LADDER = (0.8, 1.0, 1.25)
|
||||
|
||||
|
||||
@@ -460,7 +460,7 @@ def _cross_score(args: tuple[str, Any, Any]) -> dict[str, Any] | None:
|
||||
eng = TextMarkEngine(cfg)
|
||||
loc = eng.locate(img)
|
||||
try:
|
||||
score, _ = eng._tophat_best(img, loc)
|
||||
score, _ = eng._ladder_best(img, loc)
|
||||
except Exception:
|
||||
return None
|
||||
out[key] = round(float(score), 4)
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
"""End-to-end fidelity of a DELIVERED video against its source.
|
||||
|
||||
The engine's own ``psnr_db`` is measured against the already-resized frame and
|
||||
before the encoder, so it reports the VAE round trip plus latent noise and
|
||||
nothing else. The downscale, the frame decimation and the H.264 encode -- the
|
||||
three steps that actually cost the user picture -- are invisible to it, and no
|
||||
metric computed inside the streaming loop can see them, because the candidate
|
||||
frame is captured before it is written to the encoder pipe.
|
||||
|
||||
This script measures what the engine cannot: it decodes the delivered file after
|
||||
muxing, upscales each frame back to the source geometry, and scores it against
|
||||
the untouched source frame it came from. That reference is deliberately harsh -
|
||||
detail the downscale destroyed is unrecoverable, so the absolute number is
|
||||
dominated by content the pipeline never had a chance to keep. Use it to RANK
|
||||
configurations against a shared reference, not to attribute loss to one stage.
|
||||
|
||||
It streams, for the same reason the engine does: holding a 1080p clip plus its
|
||||
upscaled candidate and flow maps in memory runs to gigabytes and grows with clip
|
||||
length. Peak here is a handful of frames regardless of duration.
|
||||
|
||||
It is a research tool, not part of the shipped path, and it measures fidelity
|
||||
only. Nothing here is a watermark verdict: only a provider-oracle row is.
|
||||
|
||||
uv run python scripts/video_fidelity_probe.py source.mp4 out-512.mp4 out-768.mp4
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import click
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.video_invisible import _iter_sampled_frames, _probe_video
|
||||
from remove_ai_watermarks.video_temporal import _backward_map, _motion_residual
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
from invisible_quality_audit import _ssim # reuse, do not reimplement a fourth SSIM
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _delivered_geometry(path: Path) -> tuple[int, int, float]:
|
||||
capture = cv2.VideoCapture(str(path))
|
||||
if not capture.isOpened():
|
||||
raise click.ClickException(f"Could not open video: {path}")
|
||||
try:
|
||||
width = round(capture.get(cv2.CAP_PROP_FRAME_WIDTH))
|
||||
height = round(capture.get(cv2.CAP_PROP_FRAME_HEIGHT))
|
||||
fps = float(capture.get(cv2.CAP_PROP_FPS))
|
||||
finally:
|
||||
capture.release()
|
||||
if width <= 0 or height <= 0 or fps <= 0.0:
|
||||
raise click.ClickException(f"Video has no usable geometry or frame rate: {path}")
|
||||
return width, height, fps
|
||||
|
||||
|
||||
def _iter_frames(path: Path) -> Iterator[NDArray[Any]]:
|
||||
capture = cv2.VideoCapture(str(path))
|
||||
if not capture.isOpened():
|
||||
raise click.ClickException(f"Could not open video: {path}")
|
||||
try:
|
||||
while True:
|
||||
ok, frame = capture.read()
|
||||
if not ok:
|
||||
return
|
||||
yield frame
|
||||
finally:
|
||||
capture.release()
|
||||
|
||||
|
||||
def _measure(
|
||||
source: Path,
|
||||
delivered: Path,
|
||||
*,
|
||||
source_geometry: tuple[int, int, float],
|
||||
duration: float | None,
|
||||
) -> dict[str, Any]:
|
||||
source_width, source_height, source_fps = source_geometry
|
||||
width, height, fps = _delivered_geometry(delivered)
|
||||
|
||||
# Drive the source through the engine's own sampler at the source's geometry,
|
||||
# so the frame the probe compares against is the frame the engine regenerated.
|
||||
# Importing the rule is what keeps the pairing correct: a count check cannot
|
||||
# catch a selection rule that reorders frames without changing how many.
|
||||
reference_frames = _iter_sampled_frames(
|
||||
source,
|
||||
source_fps=source_fps,
|
||||
duration=duration,
|
||||
effective_fps=fps,
|
||||
size=(source_width, source_height),
|
||||
)
|
||||
|
||||
squared_error = 0.0
|
||||
pixel_count = 0
|
||||
ssim_scores: list[float] = []
|
||||
temporal_baseline = 0.0
|
||||
temporal_candidate = 0.0
|
||||
previous_gray: NDArray[Any] | None = None
|
||||
previous_reference_f32: NDArray[Any] | None = None
|
||||
previous_candidate_f32: NDArray[Any] | None = None
|
||||
frame_count = 0
|
||||
needs_upscale = (width, height) != (source_width, source_height)
|
||||
|
||||
reference_iter = iter(reference_frames)
|
||||
delivered_iter = _iter_frames(delivered)
|
||||
while True:
|
||||
reference = next(reference_iter, None)
|
||||
delivered_frame = next(delivered_iter, None)
|
||||
if reference is None and delivered_frame is None:
|
||||
break
|
||||
if reference is None or delivered_frame is None:
|
||||
raise click.ClickException(
|
||||
f"{delivered.name} and the sampled source ran out at different points after "
|
||||
f"{frame_count} frames. Pass --duration to match the prefix this output was "
|
||||
f"produced from, or check that it came from {source.name}."
|
||||
)
|
||||
candidate = (
|
||||
cv2.resize(delivered_frame, (source_width, source_height), interpolation=cv2.INTER_LANCZOS4)
|
||||
if needs_upscale
|
||||
else delivered_frame
|
||||
)
|
||||
reference_f32 = reference.astype(np.float32)
|
||||
candidate_f32 = candidate.astype(np.float32)
|
||||
difference = reference_f32 - candidate_f32
|
||||
squared_error += float(np.sum(difference * difference, dtype=np.float64))
|
||||
pixel_count += reference.size
|
||||
ssim_scores.append(
|
||||
_ssim(cv2.cvtColor(reference, cv2.COLOR_BGR2GRAY), cv2.cvtColor(candidate, cv2.COLOR_BGR2GRAY))
|
||||
)
|
||||
|
||||
current_gray = cv2.cvtColor(reference, cv2.COLOR_BGR2GRAY)
|
||||
if previous_gray is not None and previous_reference_f32 is not None and previous_candidate_f32 is not None:
|
||||
frame_maps = _backward_map(current_gray, previous_gray)
|
||||
temporal_baseline += _motion_residual(reference_f32, previous_reference_f32, frame_maps)
|
||||
temporal_candidate += _motion_residual(candidate_f32, previous_candidate_f32, frame_maps)
|
||||
previous_gray = current_gray
|
||||
previous_reference_f32 = reference_f32
|
||||
previous_candidate_f32 = candidate_f32
|
||||
frame_count += 1
|
||||
|
||||
if frame_count < 2:
|
||||
raise click.ClickException(f"{delivered.name} paired fewer than two frames against {source.name}")
|
||||
|
||||
mse = squared_error / pixel_count
|
||||
size_bytes = delivered.stat().st_size
|
||||
return {
|
||||
"file": delivered.name,
|
||||
"source": source.name,
|
||||
"source_width": source_width,
|
||||
"source_height": source_height,
|
||||
"source_fps": round(source_fps, 4),
|
||||
"width": width,
|
||||
"height": height,
|
||||
"fps": round(fps, 4),
|
||||
"frames": frame_count,
|
||||
"pixel_ratio": round((width * height) / (source_width * source_height), 4),
|
||||
"source_psnr_db": math.inf if mse == 0.0 else round(20.0 * math.log10(255.0 / math.sqrt(mse)), 4),
|
||||
"source_ssim": round(float(np.mean(ssim_scores)), 4),
|
||||
"temporal_residual_ratio": round(temporal_candidate / max(temporal_baseline, 1e-6), 4),
|
||||
"size_bytes": size_bytes,
|
||||
# The mux copies the source audio track verbatim, so this is a container
|
||||
# bitrate. It still ranks candidates at fixed crf; it is not a video bitrate.
|
||||
"file_bitrate_kbps": round(size_bytes * 8.0 / (frame_count / fps) / 1000.0, 1),
|
||||
}
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.argument("delivered", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"--duration",
|
||||
type=click.FloatRange(min=0.1),
|
||||
default=None,
|
||||
help="Score only the first N seconds of the source, matching a trimmed sweep candidate.",
|
||||
)
|
||||
@click.option("--json-out", type=click.Path(dir_okay=False, path_type=Path), help="Also write the rows as JSON.")
|
||||
def main(source: Path, delivered: tuple[Path, ...], duration: float | None, json_out: Path | None) -> None:
|
||||
"""Score each DELIVERED file against SOURCE end to end."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(message)s")
|
||||
source_geometry = _probe_video(source)
|
||||
log.info("Source %s: %dx%d at %.4f fps", source.name, *source_geometry)
|
||||
|
||||
rows = [_measure(source, path, source_geometry=source_geometry, duration=duration) for path in delivered]
|
||||
for row in rows:
|
||||
log.info(
|
||||
"%s: %dx%d at %s fps, %.1f%% of source pixels, PSNR %s dB, SSIM %s, temporal %s, %s kbps",
|
||||
row["file"],
|
||||
row["width"],
|
||||
row["height"],
|
||||
row["fps"],
|
||||
row["pixel_ratio"] * 100.0,
|
||||
row["source_psnr_db"],
|
||||
row["source_ssim"],
|
||||
row["temporal_residual_ratio"],
|
||||
row["file_bitrate_kbps"],
|
||||
)
|
||||
if json_out is not None:
|
||||
json_out.write_text(json.dumps(rows, indent=2), encoding="utf-8")
|
||||
log.info("Wrote %s", json_out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -39,17 +39,17 @@ from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import click
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.video_invisible import (
|
||||
_decode_frame_latents,
|
||||
_encode_frame_latents,
|
||||
_fit_size,
|
||||
_pick_device,
|
||||
_probe_video,
|
||||
_shared_latent_noise,
|
||||
build_temporal_reference,
|
||||
encode_video_frames,
|
||||
load_video_vae_runtime,
|
||||
paired_psnr,
|
||||
read_sampled_frames,
|
||||
temporal_residual_ratio,
|
||||
@@ -87,9 +87,22 @@ def _sha256(path: Path) -> str:
|
||||
|
||||
def _write_manifest(output_dir: Path, rows: Sequence[dict[str, str]]) -> Path:
|
||||
path = output_dir / "sweep.csv"
|
||||
# Mirrors the tracked manifest's run-configuration fields (data/README.md) so a
|
||||
# row carries into data/evaluations/video-synthid-oracle.csv without hand
|
||||
# reconstruction. The two 2026-07-31 rows show the cost of not doing this: their
|
||||
# geometry was never recorded and cannot be recovered from the row.
|
||||
fieldnames = [
|
||||
"variant",
|
||||
"source_sha256",
|
||||
"source_width",
|
||||
"source_height",
|
||||
"source_fps",
|
||||
"duration_seconds",
|
||||
"vae",
|
||||
"noise_std",
|
||||
"long_side",
|
||||
"fps",
|
||||
"seed",
|
||||
"psnr_db",
|
||||
"temporal_residual_ratio",
|
||||
"file",
|
||||
@@ -133,23 +146,27 @@ def main(
|
||||
) -> None:
|
||||
"""Generate VAE video candidates from the prefix of SOURCE."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(message)s")
|
||||
import torch
|
||||
from diffusers import AutoencoderKL
|
||||
|
||||
levels = _parse_noise_levels(noise_levels)
|
||||
capture = cv2.VideoCapture(str(source))
|
||||
if not capture.isOpened():
|
||||
raise click.ClickException(f"Could not open video: {source}")
|
||||
width = round(capture.get(cv2.CAP_PROP_FRAME_WIDTH))
|
||||
height = round(capture.get(cv2.CAP_PROP_FRAME_HEIGHT))
|
||||
capture.release()
|
||||
width, height, source_fps = _probe_video(source)
|
||||
size = _fit_size(width, height, long_side)
|
||||
frames, effective_fps = read_sampled_frames(source, duration=duration, output_fps=fps, size=size)
|
||||
run = {
|
||||
"source_sha256": _sha256(source),
|
||||
"source_width": str(width),
|
||||
"source_height": str(height),
|
||||
"source_fps": f"{source_fps:.4f}",
|
||||
"duration_seconds": f"{duration:.4f}",
|
||||
"vae": model,
|
||||
"long_side": str(long_side),
|
||||
"fps": f"{effective_fps:.4f}",
|
||||
"seed": str(seed),
|
||||
}
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
control_path = output_dir / "control.mp4"
|
||||
encode_video_frames(frames, source, control_path, fps=effective_fps)
|
||||
rows: list[dict[str, str]] = [
|
||||
{
|
||||
**run,
|
||||
"variant": "control",
|
||||
"noise_std": "",
|
||||
"psnr_db": "inf",
|
||||
@@ -160,12 +177,11 @@ def main(
|
||||
}
|
||||
]
|
||||
|
||||
resolved_device = _pick_device(device)
|
||||
dtype = torch.float16 if resolved_device == "cuda" else torch.float32
|
||||
log.info("Loading %s on %s", model, resolved_device)
|
||||
vae = AutoencoderKL.from_pretrained(model, torch_dtype=dtype).to(resolved_device)
|
||||
vae.eval()
|
||||
vae.enable_slicing()
|
||||
# Load through the engine's own loader rather than repeating it here: the
|
||||
# scaling-factor gate lives there, and the harness that produces the certified
|
||||
# rows is the last place that should be exempt from it.
|
||||
runtime = load_video_vae_runtime(model=model, device=device)
|
||||
vae, resolved_device = runtime.vae, runtime.resolved_device
|
||||
|
||||
log.info("Encoding source frames")
|
||||
latent_batches = _encode_frame_latents(
|
||||
@@ -173,6 +189,7 @@ def main(
|
||||
vae=vae,
|
||||
device=resolved_device,
|
||||
batch_size=batch_size,
|
||||
scaling_factor=runtime.scaling_factor,
|
||||
)
|
||||
first_latents = latent_batches[0]
|
||||
shared_noise = _shared_latent_noise(
|
||||
@@ -190,6 +207,7 @@ def main(
|
||||
vae=vae,
|
||||
noise_std=level,
|
||||
shared_noise=shared_noise,
|
||||
scaling_factor=runtime.scaling_factor,
|
||||
)
|
||||
output_path = output_dir / f"vae-noise-{level:.4f}.mp4"
|
||||
encode_video_frames(
|
||||
@@ -202,6 +220,7 @@ def main(
|
||||
temporal_ratio = temporal_residual_ratio(regenerated, temporal_maps, temporal_baseline)
|
||||
rows.append(
|
||||
{
|
||||
**run,
|
||||
"variant": "vae",
|
||||
"noise_std": f"{level:.4f}",
|
||||
"psnr_db": f"{psnr:.4f}",
|
||||
|
||||
@@ -32,12 +32,18 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
|
||||
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
|
||||
|
||||
|
||||
__version__ = "0.23.0"
|
||||
__version__ = "0.30.0"
|
||||
|
||||
__all__ = [
|
||||
"BatchSummary",
|
||||
"InvisibleOptions",
|
||||
"MetadataStripIncomplete",
|
||||
"RemoveAllResult",
|
||||
"__version__",
|
||||
"identify_video",
|
||||
"inspect_video_metadata",
|
||||
"remove_all",
|
||||
"remove_batch",
|
||||
"remove_video_all",
|
||||
"remove_video_batch",
|
||||
"remove_video_invisible",
|
||||
@@ -48,7 +54,16 @@ __all__ = [
|
||||
]
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from remove_ai_watermarks.api import remove_visible, visible_provenance
|
||||
from remove_ai_watermarks.api import (
|
||||
BatchSummary,
|
||||
InvisibleOptions,
|
||||
MetadataStripIncomplete,
|
||||
RemoveAllResult,
|
||||
remove_all,
|
||||
remove_batch,
|
||||
remove_visible,
|
||||
visible_provenance,
|
||||
)
|
||||
from remove_ai_watermarks.video import (
|
||||
identify_video,
|
||||
inspect_video_metadata,
|
||||
@@ -63,7 +78,16 @@ if TYPE_CHECKING:
|
||||
def __getattr__(name: str) -> object:
|
||||
"""Lazily resolve the high-level API (PEP 562), so the heavy imports (cv2, the
|
||||
metadata/identify stack) load only when a caller actually reaches for them."""
|
||||
if name in ("remove_visible", "visible_provenance"):
|
||||
if name in (
|
||||
"BatchSummary",
|
||||
"InvisibleOptions",
|
||||
"MetadataStripIncomplete",
|
||||
"RemoveAllResult",
|
||||
"remove_all",
|
||||
"remove_batch",
|
||||
"remove_visible",
|
||||
"visible_provenance",
|
||||
):
|
||||
from remove_ai_watermarks import api
|
||||
|
||||
return getattr(api, name)
|
||||
|
||||
@@ -1,36 +1,15 @@
|
||||
"""Compatibility namespace for metadata and regeneration helpers.
|
||||
"""Private namespace for metadata parsing and regeneration internals.
|
||||
|
||||
The public API (``WatermarkRemover`` / ``remove_watermark`` / ``remove_ai_metadata``)
|
||||
is exposed **lazily** via PEP 562 ``__getattr__``: importing a light submodule
|
||||
(e.g. ``_internal.c2pa`` / ``_internal.constants`` from ``identify``) must NOT eagerly pull
|
||||
``watermark_remover``, which imports torch + diffusers at module top. Keeping this
|
||||
lazy is what lets ``import remove_ai_watermarks.identify`` stay cheap (~36 MB, no
|
||||
torch) even in a full install where the ``diffusion`` extra is present --
|
||||
otherwise the mere presence of torch in the env inflated identify to ~420 MB and
|
||||
risked OOM on a 512 MB host.
|
||||
Deliberately empty. It carried a PEP 562 ``__getattr__`` re-exporting
|
||||
``WatermarkRemover`` and ``remove_ai_metadata`` as a "compatibility namespace",
|
||||
but nothing ever reached for either through this package -- every caller imports
|
||||
the submodule directly. The laziness it defended is real and still enforced, just
|
||||
elsewhere: importing a light submodule (``_internal.c2pa`` / ``_internal.constants``
|
||||
from ``identify``) must not pull ``watermark_remover``, which imports torch at
|
||||
module top. That property comes from those direct submodule imports, not from a
|
||||
shim here; a re-export in this file would be the one thing that could break it.
|
||||
|
||||
Keep this module free of imports. ``import remove_ai_watermarks.identify`` stays
|
||||
around 36 MB even in a full install where torch is present; routing anything heavy
|
||||
through here inflated it to roughly 420 MB and risked OOM on a 512 MB host.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover, remove_watermark
|
||||
from remove_ai_watermarks.metadata import remove_ai_metadata
|
||||
|
||||
__all__ = ["WatermarkRemover", "remove_ai_metadata", "remove_watermark"]
|
||||
|
||||
|
||||
def __getattr__(name: str) -> object:
|
||||
"""Resolve the public API on first access (PEP 562), not at package import."""
|
||||
if name == "remove_ai_metadata":
|
||||
# Re-export the single, robust stripper (byte-level, lossless-for-JPEG, all
|
||||
# containers); the old legacy metadata helper implementation is retired.
|
||||
from remove_ai_watermarks.metadata import remove_ai_metadata
|
||||
|
||||
return remove_ai_metadata
|
||||
if name in ("WatermarkRemover", "remove_watermark"):
|
||||
from remove_ai_watermarks._internal import watermark_remover
|
||||
|
||||
return getattr(watermark_remover, name)
|
||||
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
||||
@@ -8,6 +8,7 @@ import json
|
||||
import logging
|
||||
import re
|
||||
import struct
|
||||
from collections import deque
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any, cast
|
||||
@@ -15,12 +16,12 @@ from typing import TYPE_CHECKING, Any, cast
|
||||
from remove_ai_watermarks._internal.constants import (
|
||||
C2PA_ACTIONS,
|
||||
C2PA_AI_TOOLS,
|
||||
C2PA_AI_VENDORS,
|
||||
C2PA_CHUNK_TYPE,
|
||||
C2PA_ISSUERS,
|
||||
C2PA_SIGNATURES,
|
||||
C2PA_SOFT_BINDINGS,
|
||||
PNG_SIGNATURE,
|
||||
SYNTHID_C2PA_ISSUERS,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -29,12 +30,39 @@ if TYPE_CHECKING:
|
||||
from typing import BinaryIO
|
||||
|
||||
_C2paReader: Any = None
|
||||
_C2paError: Any = None
|
||||
with contextlib.suppress(Exception):
|
||||
from c2pa import C2paError as _C2paError # pyright: ignore[reportMissingTypeStubs]
|
||||
from c2pa import Reader as _C2paReader # pyright: ignore[reportMissingTypeStubs]
|
||||
|
||||
_C2PA_READER_AVAILABLE = _C2paReader is not None
|
||||
_PNG_HEADER = struct.Struct(">I4s")
|
||||
|
||||
_CONTENT_BINDING_MATCHES = (
|
||||
"assertion.dataHash.match",
|
||||
"assertion.bmffHash.match",
|
||||
"assertion.boxesHash.match",
|
||||
"assertion.collectionHash.match",
|
||||
"assertion.multiAssetHash.match",
|
||||
)
|
||||
_CONTENT_BINDING_FAILURE_MARKERS = (
|
||||
"Hash.incorrectFileCount",
|
||||
"Hash.malformed",
|
||||
"Hash.mismatch",
|
||||
"Hash.missingPart",
|
||||
"assertion.hashedURI.mismatch",
|
||||
"claim.hardBindings.missing",
|
||||
"hashedUri.mismatch",
|
||||
"ingredient.manifest.mismatch",
|
||||
)
|
||||
_SIGNATURE_FAILURES = frozenset(
|
||||
{
|
||||
"claimSignature.mismatch",
|
||||
"claimSignature.missing",
|
||||
"claimSignature.outsideValidity",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _PngChunk:
|
||||
@@ -48,10 +76,22 @@ def reader_available() -> bool:
|
||||
|
||||
|
||||
def _manifest_json_uncached(path: str) -> str | None:
|
||||
"""The manifest store as JSON, or None when this file has no readable manifest.
|
||||
|
||||
Two outcomes are routine and stay at debug: a file with no manifest (``try_create``
|
||||
returns None) and a container the reader does not support. ANY other failure is
|
||||
logged at warning, because the caller cannot tell the difference from the return
|
||||
value and the consequence is severe: the verdict silently falls back to the raw
|
||||
byte scan and can lose a high-confidence signal. The log line preserves the
|
||||
diagnostic context needed to investigate an intermittent reader failure.
|
||||
"""
|
||||
try:
|
||||
reader = _C2paReader.try_create(path)
|
||||
except _C2paError.NotSupported as error:
|
||||
logger.debug("C2PA reader does not support %s: %s", path, error)
|
||||
return None
|
||||
except Exception as error:
|
||||
logger.debug("C2PA reader rejected %s: %s", path, error)
|
||||
logger.warning("C2PA reader failed to open %s: %s: %s", path, type(error).__name__, error)
|
||||
return None
|
||||
if reader is None:
|
||||
return None
|
||||
@@ -59,7 +99,9 @@ def _manifest_json_uncached(path: str) -> str | None:
|
||||
with reader:
|
||||
return cast("str", reader.json())
|
||||
except Exception as error:
|
||||
logger.debug("C2PA reader could not serialize %s: %s", path, error)
|
||||
# The reader opened the file, so a manifest is there; failing to serialize it
|
||||
# is never routine.
|
||||
logger.warning("C2PA reader could not serialize %s: %s: %s", path, type(error).__name__, error)
|
||||
return None
|
||||
|
||||
|
||||
@@ -169,6 +211,161 @@ def _active_manifest(store: dict[str, Any]) -> dict[str, Any]:
|
||||
return cast("dict[str, Any]", active) if isinstance(active, dict) else {}
|
||||
|
||||
|
||||
def _manifest_chain(store: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
"""Return the active manifest followed by credential-usable reachable ingredients."""
|
||||
manifests = store.get("manifests")
|
||||
active_label = store.get("active_manifest")
|
||||
if not isinstance(manifests, dict) or not isinstance(active_label, str):
|
||||
return []
|
||||
typed_manifests = cast("dict[object, object]", manifests)
|
||||
pending = deque([active_label])
|
||||
seen: set[str] = set()
|
||||
chain: list[dict[str, Any]] = []
|
||||
while pending:
|
||||
label = pending.popleft()
|
||||
if label in seen:
|
||||
continue
|
||||
seen.add(label)
|
||||
manifest_value = typed_manifests.get(label)
|
||||
if not isinstance(manifest_value, dict):
|
||||
continue
|
||||
manifest = cast("dict[str, Any]", manifest_value)
|
||||
chain.append(manifest)
|
||||
ingredients = manifest.get("ingredients")
|
||||
if not isinstance(ingredients, list):
|
||||
continue
|
||||
for ingredient_value in cast("list[object]", ingredients):
|
||||
if not isinstance(ingredient_value, dict):
|
||||
continue
|
||||
ingredient = cast("dict[object, object]", ingredient_value)
|
||||
child = ingredient.get("active_manifest")
|
||||
if (
|
||||
isinstance(child, str)
|
||||
and child not in seen
|
||||
and not _store_has_invalid_credential(cast("dict[str, Any]", ingredient))
|
||||
):
|
||||
pending.append(child)
|
||||
return chain
|
||||
|
||||
|
||||
def _status_codes(store: dict[str, Any]) -> tuple[list[str], list[str], list[str]]:
|
||||
"""Return ordered status codes for this store's active credential."""
|
||||
successes: list[str] = []
|
||||
informationals: list[str] = []
|
||||
failures: list[str] = []
|
||||
|
||||
def extend(status_set: object) -> None:
|
||||
if not isinstance(status_set, dict):
|
||||
return
|
||||
mapping = cast("dict[object, object]", status_set)
|
||||
for key, target in (
|
||||
("success", successes),
|
||||
("informational", informationals),
|
||||
("failure", failures),
|
||||
):
|
||||
values = mapping.get(key)
|
||||
if not isinstance(values, list):
|
||||
continue
|
||||
for value in cast("list[object]", values):
|
||||
if not isinstance(value, dict):
|
||||
continue
|
||||
code = cast("dict[object, object]", value).get("code")
|
||||
if isinstance(code, str) and code:
|
||||
target.append(code)
|
||||
|
||||
validation_results = store.get("validation_results")
|
||||
if isinstance(validation_results, dict):
|
||||
results = cast("dict[object, object]", validation_results)
|
||||
extend(results.get("activeManifest"))
|
||||
|
||||
if not (successes or informationals or failures):
|
||||
# Older readers expose only the flat non-success list. It still carries the
|
||||
# exact codes needed to reject a mismatched binding or invalid credential.
|
||||
values = store.get("validation_status")
|
||||
if isinstance(values, list):
|
||||
for value in cast("list[object]", values):
|
||||
if not isinstance(value, dict):
|
||||
continue
|
||||
code = cast("dict[object, object]", value).get("code")
|
||||
if isinstance(code, str) and code:
|
||||
failures.append(code)
|
||||
return successes, informationals, failures
|
||||
|
||||
|
||||
def _store_has_invalid_credential(store: dict[str, Any]) -> bool:
|
||||
"""Return whether this store's own active credential failed validation."""
|
||||
_, _, failures = _status_codes(store)
|
||||
return any(marker in code for code in failures for marker in _CONTENT_BINDING_FAILURE_MARKERS) or any(
|
||||
code in _SIGNATURE_FAILURES for code in failures
|
||||
)
|
||||
|
||||
|
||||
def c2pa_info_has_invalid_credential(info: dict[str, Any]) -> bool:
|
||||
"""Return whether parsed C2PA info reports a failed binding or signature."""
|
||||
return info.get("c2pa_integrity") == "invalid" or info.get("c2pa_signature") == "invalid"
|
||||
|
||||
|
||||
def c2pa_info_has_removal_hint(info: dict[str, Any]) -> bool:
|
||||
"""Return whether a C2PA AI or watermark claim should keep removal fail-safe."""
|
||||
soft_bindings = info.get("soft_binding_vendors")
|
||||
has_invismark = isinstance(soft_bindings, list) and "Microsoft InvisMark" in soft_bindings
|
||||
return bool(info.get("ai_source_kind") or info.get("synthid_watermark") or has_invismark)
|
||||
|
||||
|
||||
def _has_suffix(codes: list[str], suffixes: tuple[str, ...]) -> bool:
|
||||
return any(code.endswith(suffix) for code in codes for suffix in suffixes)
|
||||
|
||||
|
||||
def _validation_fields(store: dict[str, Any]) -> dict[str, Any]:
|
||||
successes, informationals, failures = _status_codes(store)
|
||||
all_codes = list(dict.fromkeys([*successes, *informationals, *failures]))
|
||||
|
||||
binding_failed = any(marker in code for code in failures for marker in _CONTENT_BINDING_FAILURE_MARKERS)
|
||||
binding_matched = _has_suffix(successes, _CONTENT_BINDING_MATCHES)
|
||||
signature_failed = any(code in _SIGNATURE_FAILURES for code in failures)
|
||||
signature_validated = "claimSignature.validated" in successes
|
||||
|
||||
if binding_failed:
|
||||
integrity = "invalid"
|
||||
elif binding_matched:
|
||||
integrity = "valid"
|
||||
else:
|
||||
integrity = "unknown"
|
||||
if signature_failed:
|
||||
signature = "invalid"
|
||||
elif signature_validated:
|
||||
signature = "valid"
|
||||
else:
|
||||
signature = "unknown"
|
||||
|
||||
if "signingCredential.trusted" in successes:
|
||||
signer_trust = "trusted"
|
||||
elif "signingCredential.untrusted" in failures:
|
||||
signer_trust = "untrusted"
|
||||
else:
|
||||
signer_trust = "unknown"
|
||||
|
||||
if any(code in failures for code in ("signingCredential.invalid", "signingCredential.ocsp.revoked")):
|
||||
signer_validity = "invalid"
|
||||
elif "signingCredential.expired" in failures:
|
||||
signer_validity = "expired"
|
||||
elif "claimSignature.insideValidity" in successes:
|
||||
signer_validity = "valid"
|
||||
else:
|
||||
signer_validity = "unknown"
|
||||
|
||||
state = store.get("validation_state")
|
||||
return {
|
||||
"c2pa_validation_source": "reader",
|
||||
"c2pa_validation_state": state if isinstance(state, str) and state else "unknown",
|
||||
"c2pa_integrity": integrity,
|
||||
"c2pa_signature": signature,
|
||||
"c2pa_signer_trust": signer_trust,
|
||||
"c2pa_signer_validity": signer_validity,
|
||||
"c2pa_validation_codes": all_codes,
|
||||
}
|
||||
|
||||
|
||||
def _claim_generator_from_store(store: dict[str, Any]) -> str | None:
|
||||
active = _active_manifest(store)
|
||||
direct = active.get("claim_generator")
|
||||
@@ -183,19 +380,170 @@ def _claim_generator_from_store(store: dict[str, Any]) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _structured_manifest_fields(store: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Extract provenance only from the active manifest and its ingredient graph."""
|
||||
chain = _manifest_chain(store)
|
||||
if not chain:
|
||||
return {}
|
||||
|
||||
info: dict[str, Any] = {}
|
||||
issuers: list[str] = []
|
||||
tools: list[str] = []
|
||||
actions: list[str] = []
|
||||
source_types: list[str] = []
|
||||
soft_binding_algorithms: list[str] = []
|
||||
soft_binding_values: list[str] = []
|
||||
claim_generator_asserts_ai = False
|
||||
|
||||
def add_tool_matches(value: str, *, asserts_ai: bool = False) -> None:
|
||||
nonlocal claim_generator_asserts_ai
|
||||
matches = _ordered_matches(value.encode(), C2PA_AI_TOOLS)
|
||||
tools.extend(matches)
|
||||
if asserts_ai and matches:
|
||||
claim_generator_asserts_ai = True
|
||||
|
||||
for manifest in chain:
|
||||
signature_value = manifest.get("signature_info")
|
||||
if isinstance(signature_value, dict):
|
||||
signature = cast("dict[object, object]", signature_value)
|
||||
for key in ("issuer", "common_name", "certificate_issuer"):
|
||||
value = signature.get(key)
|
||||
if isinstance(value, str):
|
||||
issuers.extend(_ordered_matches(value.encode(), C2PA_ISSUERS))
|
||||
|
||||
direct_generator = manifest.get("claim_generator")
|
||||
if isinstance(direct_generator, str):
|
||||
add_tool_matches(direct_generator, asserts_ai=True)
|
||||
|
||||
candidates = manifest.get("claim_generator_info")
|
||||
if isinstance(candidates, list):
|
||||
for candidate_value in cast("list[object]", candidates):
|
||||
if not isinstance(candidate_value, dict):
|
||||
continue
|
||||
name = cast("dict[object, object]", candidate_value).get("name")
|
||||
if isinstance(name, str):
|
||||
add_tool_matches(name, asserts_ai=True)
|
||||
|
||||
assertions = manifest.get("assertions")
|
||||
if not isinstance(assertions, list):
|
||||
continue
|
||||
for assertion_value in cast("list[object]", assertions):
|
||||
if not isinstance(assertion_value, dict):
|
||||
continue
|
||||
assertion = cast("dict[object, object]", assertion_value)
|
||||
label = assertion.get("label")
|
||||
data = assertion.get("data")
|
||||
if isinstance(label, str) and label.startswith("c2pa.soft-binding") and isinstance(data, dict):
|
||||
soft_binding = cast("dict[object, object]", data)
|
||||
algorithm = soft_binding.get("alg")
|
||||
if isinstance(algorithm, str):
|
||||
soft_binding_algorithms.append(algorithm)
|
||||
blocks = soft_binding.get("blocks")
|
||||
if isinstance(blocks, list):
|
||||
for block_value in cast("list[object]", blocks):
|
||||
if not isinstance(block_value, dict):
|
||||
continue
|
||||
value = cast("dict[object, object]", block_value).get("value")
|
||||
if isinstance(value, str) and value.isprintable() and len(value) <= 256:
|
||||
soft_binding_values.append(value)
|
||||
if not (isinstance(label, str) and label.startswith("c2pa.actions") and isinstance(data, dict)):
|
||||
continue
|
||||
action_values = cast("dict[object, object]", data).get("actions")
|
||||
if not isinstance(action_values, list):
|
||||
continue
|
||||
for action_value in cast("list[object]", action_values):
|
||||
if not isinstance(action_value, dict):
|
||||
continue
|
||||
action = cast("dict[object, object]", action_value)
|
||||
action_name = action.get("action")
|
||||
if isinstance(action_name, str):
|
||||
actions.append(C2PA_ACTIONS.get(action_name.encode(), action_name.removeprefix("c2pa.")))
|
||||
source_type = action.get("digitalSourceType")
|
||||
if isinstance(source_type, str):
|
||||
source_types.append(source_type)
|
||||
software_agent = action.get("softwareAgent")
|
||||
if isinstance(software_agent, dict):
|
||||
name = cast("dict[object, object]", software_agent).get("name")
|
||||
if isinstance(name, str):
|
||||
add_tool_matches(name)
|
||||
|
||||
issuers = list(dict.fromkeys(issuers))
|
||||
tools = list(dict.fromkeys(tools))
|
||||
actions = list(dict.fromkeys(actions))
|
||||
if issuers:
|
||||
info["issuer"] = ", ".join(issuers)
|
||||
if tools:
|
||||
info["ai_tool"] = ", ".join(tools)
|
||||
if actions:
|
||||
info["actions"] = ", ".join(actions)
|
||||
if claim_generator_asserts_ai:
|
||||
info["c2pa_identity_ai"] = True
|
||||
|
||||
generated = any(
|
||||
"trainedAlgorithmicMedia" in value and "compositeWithTrainedAlgorithmicMedia" not in value
|
||||
for value in source_types
|
||||
)
|
||||
enhanced = any("compositeWithTrainedAlgorithmicMedia" in value for value in source_types)
|
||||
procedural = any("algorithmicMedia" in value for value in source_types)
|
||||
if generated:
|
||||
info.update(source_type="trainedAlgorithmicMedia (AI-generated)", ai_source_kind="generated")
|
||||
elif enhanced:
|
||||
info.update(source_type="compositeWithTrainedAlgorithmicMedia (AI-enhanced)", ai_source_kind="enhanced")
|
||||
elif procedural:
|
||||
info["source_type"] = "algorithmicMedia"
|
||||
|
||||
has_watermark_action = any(action.startswith("watermarked") for action in actions)
|
||||
if has_watermark_action:
|
||||
info["watermarked"] = True
|
||||
if info.get("ai_source_kind"):
|
||||
selected_bytes = json.dumps(chain, ensure_ascii=False).encode()
|
||||
synthid = synthid_evidence_vendors_in(selected_bytes, has_watermark_action=has_watermark_action)
|
||||
if synthid:
|
||||
info["synthid_vendors"] = synthid
|
||||
info["synthid_watermark"] = synthid_verdict(", ".join(synthid))
|
||||
|
||||
if soft_binding_algorithms:
|
||||
soft_binding_algorithms = list(dict.fromkeys(soft_binding_algorithms))
|
||||
algorithms = "\n".join(soft_binding_algorithms).encode()
|
||||
soft_bindings = soft_binding_vendors_in(algorithms)
|
||||
if soft_bindings:
|
||||
info["soft_binding_vendors"] = soft_bindings
|
||||
info["soft_binding"] = ", ".join(soft_bindings)
|
||||
info["soft_binding_algorithm"] = ", ".join(soft_binding_algorithms)
|
||||
if soft_binding_values:
|
||||
info["soft_binding_value"] = ", ".join(dict.fromkeys(soft_binding_values))
|
||||
return info
|
||||
|
||||
|
||||
def synthid_verdict(vendors: str) -> str:
|
||||
"""Describe why metadata implies a likely pixel-level SynthID watermark."""
|
||||
return f"likely present ({vendors} embeds SynthID with C2PA)"
|
||||
"""Describe why supported provenance establishes a SynthID watermark."""
|
||||
return f"present according to {vendors} provenance"
|
||||
|
||||
|
||||
def _names_present(buffer: bytes, registry: dict[bytes, str]) -> list[str]:
|
||||
return sorted({label for token, label in registry.items() if token in buffer})
|
||||
|
||||
|
||||
def synthid_vendors_in(buffer: bytes) -> list[str]:
|
||||
"""List matching C2PA issuers known to pair their manifests with SynthID."""
|
||||
registry = {token: label for token, label in C2PA_ISSUERS.items() if token in SYNTHID_C2PA_ISSUERS}
|
||||
return _names_present(buffer, registry)
|
||||
def synthid_evidence_vendors_in(buffer: bytes, *, has_watermark_action: bool | None = None) -> list[str]:
|
||||
"""List issuers whose provenance establishes SynthID for this asset.
|
||||
|
||||
Google applies SynthID to all media generated by its tools, so its AI C2PA
|
||||
provenance is sufficient. OpenAI C2PA predates OpenAI's SynthID rollout;
|
||||
current manifests distinguish the watermarked generation with the explicit
|
||||
``c2pa.watermarked.*`` action. A legacy OpenAI issuer token alone therefore
|
||||
remains provenance evidence, but not SynthID evidence.
|
||||
"""
|
||||
if has_watermark_action is None:
|
||||
has_watermark_action = b"c2pa.watermarked" in buffer
|
||||
return sorted(
|
||||
{
|
||||
vendor.org
|
||||
for vendor in C2PA_AI_VENDORS
|
||||
if vendor.synthid
|
||||
and vendor.issuer in buffer
|
||||
and (has_watermark_action or not vendor.synthid_requires_watermark_action)
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def soft_binding_vendors_in(buffer: bytes) -> list[str]:
|
||||
@@ -228,7 +576,9 @@ def _populate_registry_fields(buffer: bytes, info: dict[str, Any]) -> bool:
|
||||
elif b"algorithmicMedia" in buffer:
|
||||
info["source_type"] = "algorithmicMedia"
|
||||
|
||||
synthid = synthid_vendors_in(buffer)
|
||||
if b"c2pa.watermarked" in buffer:
|
||||
info["watermarked"] = True
|
||||
synthid = synthid_evidence_vendors_in(buffer, has_watermark_action=info.get("watermarked", False))
|
||||
if ai_source and synthid:
|
||||
info["synthid_vendors"] = synthid
|
||||
info["synthid_watermark"] = synthid_verdict(", ".join(synthid))
|
||||
@@ -242,16 +592,28 @@ def _populate_registry_fields(buffer: bytes, info: dict[str, Any]) -> bool:
|
||||
|
||||
def _base_info(byte_count: int, *, fallback: bool = False) -> dict[str, Any]:
|
||||
container = "C2PA manifest" if fallback else "C2PA manifest store"
|
||||
return {
|
||||
info: dict[str, Any] = {
|
||||
"has_c2pa": True,
|
||||
"type": "C2PA (Coalition for Content Provenance and Authenticity)",
|
||||
"c2pa_manifest": f"{container} ({byte_count} bytes)",
|
||||
}
|
||||
if fallback:
|
||||
info.update(
|
||||
c2pa_validation_source="fallback",
|
||||
c2pa_validation_state="unknown",
|
||||
c2pa_integrity="unknown",
|
||||
c2pa_signature="unknown",
|
||||
c2pa_signer_trust="unknown",
|
||||
c2pa_signer_validity="unknown",
|
||||
c2pa_validation_codes=[],
|
||||
)
|
||||
return info
|
||||
|
||||
|
||||
def _info_from_store(store: dict[str, Any], encoded: bytes) -> dict[str, Any]:
|
||||
info = _base_info(len(encoded))
|
||||
_populate_registry_fields(encoded, info)
|
||||
info.update(_structured_manifest_fields(store))
|
||||
info.update(_validation_fields(store))
|
||||
generator = _claim_generator_from_store(store)
|
||||
if generator is not None:
|
||||
info["claim_generator"] = generator
|
||||
@@ -329,14 +691,44 @@ def _extract_c2pa_info_png(image_path: Path) -> dict[str, Any]:
|
||||
return info
|
||||
|
||||
|
||||
def extract_c2pa_info(image_path: Path) -> dict[str, Any]:
|
||||
"""Return normalized C2PA evidence from the official reader or PNG fallback."""
|
||||
def _extract_c2pa_info_impl(image_path: Path) -> dict[str, Any]:
|
||||
store = read_manifest_store_json(Path(image_path))
|
||||
if store is not None:
|
||||
return c2pa_info_from_manifest_store(store)
|
||||
return _extract_c2pa_info_png(Path(image_path))
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _extract_c2pa_info_cached(path_str: str, _mtime_ns: int, _size: int, _reader: bool) -> dict[str, Any]:
|
||||
"""Cache shim: every argument after the path is key-only.
|
||||
|
||||
``_reader`` is in the key because the answer genuinely depends on it -- with the
|
||||
official reader the manifest comes back as a store, without it from the hand-rolled
|
||||
PNG chunk parser, and the two produce different ``c2pa_manifest`` labels. Keying on
|
||||
the file alone handed a reader-path result to a caller that had disabled the reader.
|
||||
"""
|
||||
return _extract_c2pa_info_impl(Path(path_str))
|
||||
|
||||
|
||||
def extract_c2pa_info(image_path: Path) -> dict[str, Any]:
|
||||
"""Return normalized C2PA evidence from the official reader or PNG fallback.
|
||||
|
||||
Memoized on ``(path, mtime_ns, size)``: one ``identify`` reaches this twice (once
|
||||
directly, once inside ``get_ai_metadata``) and each call re-runs the Rust manifest
|
||||
reader and re-parses its JSON. Size joins mtime in the key because this package
|
||||
rewrites files in place, and an in-place rewrite can land inside one mtime tick.
|
||||
"""
|
||||
try:
|
||||
stat = image_path.stat()
|
||||
except OSError:
|
||||
# No stat (a pipe, or a race): read uncached rather than fail.
|
||||
return _extract_c2pa_info_impl(image_path)
|
||||
cached = _extract_c2pa_info_cached(str(image_path), stat.st_mtime_ns, stat.st_size, _C2PA_READER_AVAILABLE)
|
||||
# Deep-ish copy: values are scalars plus a few lists, and a caller mutating one of
|
||||
# those lists would otherwise poison every later reader of the same file.
|
||||
return {key: list(cast("list[Any]", value)) if isinstance(value, list) else value for key, value in cached.items()}
|
||||
|
||||
|
||||
def inject_c2pa_chunk(target_path: Path, output_path: Path, c2pa_chunk: bytes) -> None:
|
||||
"""Replace any C2PA chunks in a PNG and insert ``c2pa_chunk`` before IDAT."""
|
||||
if target_path.suffix.casefold() != ".png" or output_path.suffix.casefold() != ".png":
|
||||
|
||||
@@ -14,15 +14,15 @@ AI_METADATA_KEYS = _tokens(
|
||||
"parameters|postprocessing|extras|workflow|prompt|Dream|SD:mode|StableDiffusionVersion|"
|
||||
"generation_time|Model|Model hash|Seed"
|
||||
)
|
||||
PNG_METADATA_KEYS = _tokens(
|
||||
"Author|Title|Description|Copyright|Creation Time|Software|Disclaimer|Warning|Source|Comment"
|
||||
)
|
||||
AI_KEYWORDS = _tokens(
|
||||
"prompt|negative_prompt|sampler|cfg_scale|lora|diffusion|comfy|midjourney|dall-e|dalle|imagen|firefly|c2pa|chatgpt|gpt-4|sora|openai|truepic|stable_diffusion|invokeai"
|
||||
)
|
||||
|
||||
PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
|
||||
C2PA_CHUNK_TYPE = b"caBX"
|
||||
PNG_METADATA_CHUNKS = frozenset({b"tEXt", b"iTXt", b"zTXt", b"eXIf", b"iCCP"})
|
||||
RIFF_METADATA_CHUNKS = frozenset({b"EXIF", b"XMP ", b"ICCP", b"C2PA"})
|
||||
RIFF_CODED_IMAGE_CHUNKS = frozenset({b"VP8 ", b"VP8L", b"ALPH", b"ANMF"})
|
||||
C2PA_SIGNATURES = tuple(
|
||||
token.encode() for token in _tokens("c2pa|C2PA|jumb|jumd|JUMBF|jumbf|cbor|contentcreds|digid|assertions|manifest")
|
||||
)
|
||||
@@ -38,6 +38,7 @@ class C2paAiVendor:
|
||||
needle: str | None
|
||||
synthid: bool = False
|
||||
asserts_ai: bool = False
|
||||
synthid_requires_watermark_action: bool = False
|
||||
|
||||
|
||||
def _vendor(
|
||||
@@ -48,31 +49,54 @@ def _vendor(
|
||||
*,
|
||||
synthid: bool = False,
|
||||
asserts_ai: bool = False,
|
||||
synthid_requires_watermark_action: bool = False,
|
||||
) -> C2paAiVendor:
|
||||
token = issuer.encode() if isinstance(issuer, str) else issuer
|
||||
return C2paAiVendor(token, org, platform, needle, synthid, asserts_ai)
|
||||
return C2paAiVendor(
|
||||
token,
|
||||
org,
|
||||
platform,
|
||||
needle,
|
||||
synthid=synthid,
|
||||
asserts_ai=asserts_ai,
|
||||
synthid_requires_watermark_action=synthid_requires_watermark_action,
|
||||
)
|
||||
|
||||
|
||||
# Order is product priority when a manifest mentions more than one organization.
|
||||
C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
|
||||
_vendor(b"Microsoft", "Microsoft", "Microsoft (Bing Image Creator / Designer)", "Microsoft"),
|
||||
_vendor(b"Microsoft", "Microsoft", "Microsoft (Copilot / Designer)", "Microsoft"),
|
||||
_vendor(b"Adobe", "Adobe", "Adobe Firefly", "Adobe"),
|
||||
_vendor(b"OpenAI", "OpenAI", "OpenAI (ChatGPT / gpt-image / DALL-E / Sora)", "OpenAI", synthid=True),
|
||||
_vendor(
|
||||
b"OpenAI",
|
||||
"OpenAI",
|
||||
"OpenAI (ChatGPT / gpt-image / DALL-E / Sora)",
|
||||
"OpenAI",
|
||||
synthid=True,
|
||||
synthid_requires_watermark_action=True,
|
||||
),
|
||||
_vendor(b"Google", "Google LLC", "Google (Gemini / Imagen)", "Google", synthid=True),
|
||||
_vendor(b"Stability AI", "Stability AI", "Stability AI (Stable Image / DreamStudio)", "Stability AI"),
|
||||
_vendor(b"Black Forest Labs", "Black Forest Labs", "Black Forest Labs (FLUX)", "Black Forest Labs"),
|
||||
_vendor(b"volcengine", "ByteDance (Volcano Engine)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
|
||||
_vendor(
|
||||
b"volcengine",
|
||||
"ByteDance (Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
|
||||
"ByteDance",
|
||||
),
|
||||
_vendor(
|
||||
"北京火山引擎科技有限公司",
|
||||
"ByteDance (Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
|
||||
"ByteDance",
|
||||
),
|
||||
_vendor(b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
|
||||
_vendor(
|
||||
b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)", "ByteDance"
|
||||
),
|
||||
_vendor(
|
||||
b"Dreamina",
|
||||
"ByteDance (Dreamina)",
|
||||
"ByteDance (Doubao / Jimeng / Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
|
||||
"ByteDance",
|
||||
asserts_ai=True,
|
||||
),
|
||||
@@ -85,7 +109,18 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
|
||||
|
||||
C2PA_ISSUERS = {vendor.issuer: vendor.org for vendor in C2PA_AI_VENDORS}
|
||||
C2PA_IDENTITY_AI_ORGS = frozenset(vendor.org for vendor in C2PA_AI_VENDORS if vendor.asserts_ai)
|
||||
SYNTHID_C2PA_ISSUERS = frozenset(vendor.issuer for vendor in C2PA_AI_VENDORS if vendor.synthid)
|
||||
|
||||
# Product-specific claim generators can sign through a different upstream issuer.
|
||||
# Keep this attribution beside the issuer registry so every C2PA consumer has one
|
||||
# canonical source rather than maintaining a derived product map in identify.py.
|
||||
C2PA_CLAIM_GENERATOR_PLATFORMS: tuple[tuple[str, str], ...] = (
|
||||
("adobe_firefly", "Adobe Firefly"),
|
||||
("firefly", "Adobe Firefly"),
|
||||
("dreamina", "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"),
|
||||
("higgsfield ai", "Higgsfield AI"),
|
||||
("topaz labs image api", "Topaz Labs"),
|
||||
("tiktok ad creative toolbox", "TikTok Ad Creative Toolbox"),
|
||||
)
|
||||
|
||||
C2PA_AI_TOOLS = {
|
||||
token.encode(): label
|
||||
@@ -97,6 +132,7 @@ C2PA_AI_TOOLS = {
|
||||
("DALL", "DALL-E"),
|
||||
("Imagen", "Imagen"),
|
||||
("Firefly", "Firefly"),
|
||||
("Dreamina", "Dreamina"),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -142,11 +178,28 @@ AI_GENERATOR_TOKENS = frozenset(
|
||||
"dreamstudio",
|
||||
"novelai",
|
||||
"reve.com",
|
||||
# Luma AI stamps PNG tEXt Source="Luma AI" / Comment="Generated by
|
||||
# Luma AI's Uni-1 model (https://lumalabs.ai)"; the space-bearing token
|
||||
# avoids matching incidental "luma" runs (luma/chroma key names etc.).
|
||||
"luma ai",
|
||||
"lumalabs",
|
||||
"aphrodite ai",
|
||||
"apple photos clean up",
|
||||
"fal-ai",
|
||||
}
|
||||
)
|
||||
|
||||
_C2PA_ACTION_NAMES = _tokens("created|converted|edited|filtered|cropped|resized|opened|placed")
|
||||
_C2PA_ACTION_NAMES = _tokens("created|converted|edited|filtered|cropped|resized|opened|placed|watermarked.unbound")
|
||||
C2PA_ACTIONS = {f"c2pa.{action}".encode(): action for action in _C2PA_ACTION_NAMES}
|
||||
|
||||
|
||||
# TC260 producer identity -> the mark key whose vendor signs with it now lives on the
|
||||
# registry rows (``KnownMark.tc260_producer_codes``, read through
|
||||
# ``watermark_registry.tc260_producer_vendors``). Keeping the codes beside the mark is
|
||||
# what stops a newly registered TC260 vendor from silently falling back to ByteDance.
|
||||
#
|
||||
# What a TC260 label confirms when its producer is absent or unmapped. Historical
|
||||
# behaviour, kept as the fallback so an unrecognized producer never regresses to no
|
||||
# relaxation at all: ByteDance's two products are the ones the relaxed band was
|
||||
# calibrated on (see _text_mark_engine._DEFAULT_PROVENANCE_NCC_FACTOR).
|
||||
TC260_FALLBACK_VENDORS: frozenset[str] = frozenset({"doubao", "jimeng"})
|
||||
@@ -1,136 +0,0 @@
|
||||
"""Execute Diffusers img2img calls and recover from an MPS runtime failure."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks._internal.progress import is_mps_error, make_pipeline_progress
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from PIL import Image
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _pipeline_arguments(
|
||||
image: Image.Image,
|
||||
strength: float,
|
||||
num_inference_steps: int,
|
||||
guidance_scale: float,
|
||||
generator: Any,
|
||||
step_callback: Any,
|
||||
overrides: dict[str, Any] | None,
|
||||
) -> dict[str, Any]:
|
||||
arguments: dict[str, Any] = {
|
||||
"prompt": "",
|
||||
"image": image,
|
||||
"strength": strength,
|
||||
"num_inference_steps": num_inference_steps,
|
||||
"guidance_scale": guidance_scale,
|
||||
"generator": generator,
|
||||
}
|
||||
arguments.update(overrides or {})
|
||||
if step_callback is not None:
|
||||
arguments.update(callback=step_callback, callback_steps=1)
|
||||
return arguments
|
||||
|
||||
|
||||
def _invoke(pipeline: Any, arguments: dict[str, Any]) -> Image.Image:
|
||||
response = pipeline(**arguments)
|
||||
return response.images[0]
|
||||
|
||||
|
||||
def run_img2img(
|
||||
pipeline: Any,
|
||||
image: Image.Image,
|
||||
strength: float,
|
||||
num_inference_steps: int,
|
||||
guidance_scale: float,
|
||||
generator: Any,
|
||||
device: str,
|
||||
set_progress: Callable[[str], None],
|
||||
extra_kwargs: dict[str, Any] | None = None,
|
||||
) -> Image.Image:
|
||||
"""Run one img2img request and report denoising progress when supported."""
|
||||
callback, started, finished, launch_monitor = make_pipeline_progress(
|
||||
max(1, int(num_inference_steps * strength)), device, set_progress
|
||||
)
|
||||
launch_monitor()
|
||||
arguments = _pipeline_arguments(
|
||||
image, strength, num_inference_steps, guidance_scale, generator, callback, extra_kwargs
|
||||
)
|
||||
try:
|
||||
try:
|
||||
return _invoke(pipeline, arguments)
|
||||
except TypeError as error:
|
||||
if "callback" not in str(error):
|
||||
raise
|
||||
started.set()
|
||||
arguments.pop("callback", None)
|
||||
arguments.pop("callback_steps", None)
|
||||
return _invoke(pipeline, arguments)
|
||||
finally:
|
||||
started.set()
|
||||
finished.set()
|
||||
|
||||
|
||||
def run_img2img_with_mps_fallback(
|
||||
load_pipeline: Callable[[], Any],
|
||||
image: Image.Image,
|
||||
strength: float,
|
||||
num_inference_steps: int,
|
||||
guidance_scale: float,
|
||||
generator: Any,
|
||||
device: str,
|
||||
set_progress: Callable[[str], None],
|
||||
*,
|
||||
reload_on_cpu: Callable[[], Any],
|
||||
extra_kwargs: dict[str, Any] | None = None,
|
||||
) -> tuple[Image.Image, str]:
|
||||
"""Retry an MPS-specific failure once with a freshly loaded CPU pipeline."""
|
||||
try:
|
||||
output = run_img2img(
|
||||
load_pipeline(),
|
||||
image,
|
||||
strength,
|
||||
num_inference_steps,
|
||||
guidance_scale,
|
||||
generator,
|
||||
device,
|
||||
set_progress,
|
||||
extra_kwargs,
|
||||
)
|
||||
return output, device
|
||||
except RuntimeError as error:
|
||||
if device != "mps" or not is_mps_error(error):
|
||||
raise
|
||||
logger.warning("MPS execution failed (%s); retrying on CPU", error)
|
||||
set_progress("MPS execution failed; retrying on CPU...")
|
||||
try_empty_device_cache("mps")
|
||||
output = run_img2img(
|
||||
reload_on_cpu(),
|
||||
image,
|
||||
strength,
|
||||
num_inference_steps,
|
||||
guidance_scale,
|
||||
None,
|
||||
"cpu",
|
||||
set_progress,
|
||||
extra_kwargs,
|
||||
)
|
||||
return output, "cpu"
|
||||
|
||||
|
||||
def try_empty_device_cache(device: str) -> None:
|
||||
"""Ask Torch to release cached accelerator memory when the backend supports it."""
|
||||
with contextlib.suppress(Exception):
|
||||
import torch
|
||||
|
||||
backend = getattr(torch, device, None)
|
||||
empty_cache = getattr(backend, "empty_cache", None)
|
||||
if callable(empty_cache):
|
||||
empty_cache()
|
||||
@@ -12,7 +12,10 @@ so pixel, video, and audio data is preserved bit-for-bit.
|
||||
TC260-PG-20257A video metadata is nested instead:
|
||||
``moov.udta.meta.keys/ilst``. Its detector seeks through those boxes without
|
||||
reading media payloads, and its stripper blanks the validated key/value in
|
||||
place so fast-start media offsets remain valid.
|
||||
place so fast-start media offsets remain valid. Two serialization variants are
|
||||
covered: the ISO form (``meta`` as a FullBox under ``udta``) and the QuickTime
|
||||
form Doubao's iOS export writes (a bare ``meta`` box as a direct ``moov`` child,
|
||||
no FullBox header).
|
||||
|
||||
This file intentionally avoids dependencies on format-specific libraries
|
||||
(pillow-heif, pillow-jxl, pymp4) so it works on systems where they aren't
|
||||
@@ -64,7 +67,7 @@ _AI_LABEL_MARKERS: tuple[bytes, ...] = AIGC_MARKERS + IPTC_AI_MARKERS + IPTC_AI_
|
||||
# blanked in place (see ``blank_ai_xmp_packets``).
|
||||
_XMP_PACKET_RE = re.compile(rb"<\?xpacket begin=.*?<\?xpacket end=[^>]*?\?>", re.DOTALL)
|
||||
_STREAM_COPY_BYTES = 1024 * 1024
|
||||
_STREAM_SCAN_BYTES = 4 * 1024 * 1024
|
||||
STREAM_SCAN_BYTES = 4 * 1024 * 1024
|
||||
|
||||
|
||||
# TC260-PG-20257A stores an MP4/MOV label as an ``AIGC`` key in
|
||||
@@ -133,7 +136,7 @@ def _read_box_header(
|
||||
return end, box_type, payload_off
|
||||
|
||||
|
||||
def _iter_file_boxes(
|
||||
def iter_file_boxes(
|
||||
stream: BinaryIO,
|
||||
start: int,
|
||||
end: int,
|
||||
@@ -184,68 +187,116 @@ def _tc260_key_indices(
|
||||
return found
|
||||
|
||||
|
||||
# The box types a TC260-bearing ``meta`` box opens with or contains: ISO files
|
||||
# have ``hdlr`` then ``keys``/``ilst``; QuickTime metadata lists have ``hdlr``
|
||||
# and ``ilst`` only. Which payload offset (0 vs 4) yields such children is what
|
||||
# disambiguates the QuickTime form (no FullBox header) from the ISO one.
|
||||
_META_CHILD_TYPES = frozenset({b"hdlr", b"keys", b"ilst"})
|
||||
|
||||
|
||||
def _meta_child_boxes(
|
||||
stream: BinaryIO,
|
||||
meta_payload: int,
|
||||
meta_end: int,
|
||||
) -> Iterator[tuple[int, int, bytes, int]]:
|
||||
"""Yield the child boxes of one ``meta`` box in either serialized form.
|
||||
|
||||
ISO serializes ``meta`` as a FullBox, so its children start 4 bytes into
|
||||
the payload; QuickTime writes a bare box, so they start at 0. Both real
|
||||
forms open with a recognized child (``hdlr``), so the form is picked by
|
||||
which offset's first box type is one of ``_META_CHILD_TYPES``; a wrong
|
||||
probe reads garbage header bytes that match no known type.
|
||||
"""
|
||||
for offset in (0, 4):
|
||||
boxes = iter_file_boxes(stream, meta_payload + offset, meta_end)
|
||||
first = next(boxes, None)
|
||||
if first is not None and first[2] in _META_CHILD_TYPES:
|
||||
yield first
|
||||
yield from boxes
|
||||
return
|
||||
|
||||
|
||||
def _iter_tc260_meta_boxes(
|
||||
stream: BinaryIO,
|
||||
moov_payload: int,
|
||||
moov_end: int,
|
||||
) -> Iterator[tuple[int, int]]:
|
||||
"""Yield ``(payload, end)`` of every ``meta`` box that may hold a TC260 label.
|
||||
|
||||
The normative ISO placement is ``moov.udta.meta``; Doubao's iOS MOV export
|
||||
instead stores the label in a QuickTime-form ``meta`` box that hangs
|
||||
directly off ``moov``. Both are yielded so one consumer covers them.
|
||||
"""
|
||||
for _start, end, box_type, payload in iter_file_boxes(stream, moov_payload, moov_end):
|
||||
if box_type == b"udta":
|
||||
for _udta_start, udta_end, udta_type, udta_payload in iter_file_boxes(stream, payload, end):
|
||||
if udta_type == b"meta":
|
||||
yield udta_payload, udta_end
|
||||
elif box_type == b"meta":
|
||||
yield payload, end
|
||||
|
||||
|
||||
def _tc260_aigc_regions(
|
||||
stream: BinaryIO,
|
||||
file_size: int,
|
||||
) -> list[tuple[int, int, int, int, bytes]]:
|
||||
) -> list[tuple[tuple[int, int] | None, int, int, bytes]]:
|
||||
"""Locate validated native TC260 entries without reading media payloads.
|
||||
|
||||
Each tuple is ``(key_start, key_end, value_start, value_end, value)``.
|
||||
Each tuple is ``(key_span, value_start, value_end, value)``; ``key_span`` is
|
||||
the byte span of the ``AIGC`` key when the normative ``keys`` box maps the
|
||||
item, or None for the QuickTime metadata-list form (``hdlr=mdir``) Doubao's
|
||||
iOS export writes, where the JSON sits in a bare ``ilst`` data item with no
|
||||
key name to blank.
|
||||
"""
|
||||
regions: list[tuple[int, int, int, int, bytes]] = []
|
||||
for _moov_start, moov_end, moov_type, moov_payload in _iter_file_boxes(stream, 0, file_size):
|
||||
regions: list[tuple[tuple[int, int] | None, int, int, bytes]] = []
|
||||
for _moov_start, moov_end, moov_type, moov_payload in iter_file_boxes(stream, 0, file_size):
|
||||
if moov_type != b"moov":
|
||||
continue
|
||||
for _udta_start, udta_end, udta_type, udta_payload in _iter_file_boxes(
|
||||
stream,
|
||||
moov_payload,
|
||||
moov_end,
|
||||
):
|
||||
if udta_type != b"udta":
|
||||
continue
|
||||
for _meta_start, meta_end, meta_type, meta_payload in _iter_file_boxes(
|
||||
for meta_payload, meta_end in _iter_tc260_meta_boxes(stream, moov_payload, moov_end):
|
||||
keys: dict[int, tuple[int, int]] = {}
|
||||
ilst_boxes: list[tuple[int, int]] = []
|
||||
keyed = False
|
||||
for _child_start, child_end, child_type, child_payload in _meta_child_boxes(
|
||||
stream,
|
||||
udta_payload,
|
||||
udta_end,
|
||||
meta_payload,
|
||||
meta_end,
|
||||
):
|
||||
if meta_type != b"meta" or meta_payload + 4 > meta_end:
|
||||
continue
|
||||
keys: dict[int, tuple[int, int]] = {}
|
||||
ilst_boxes: list[tuple[int, int]] = []
|
||||
for _child_start, child_end, child_type, child_payload in _iter_file_boxes(
|
||||
if child_type == b"keys":
|
||||
keyed = True
|
||||
keys.update(_tc260_key_indices(stream, child_payload, child_end))
|
||||
elif child_type == b"ilst":
|
||||
ilst_boxes.append((child_payload, child_end))
|
||||
if not ilst_boxes:
|
||||
continue
|
||||
for ilst_payload, ilst_end in ilst_boxes:
|
||||
for _item_start, item_end, item_type, item_payload in iter_file_boxes(
|
||||
stream,
|
||||
meta_payload + 4,
|
||||
meta_end,
|
||||
ilst_payload,
|
||||
ilst_end,
|
||||
):
|
||||
if child_type == b"keys":
|
||||
keys.update(_tc260_key_indices(stream, child_payload, child_end))
|
||||
elif child_type == b"ilst":
|
||||
ilst_boxes.append((child_payload, child_end))
|
||||
if not keys:
|
||||
continue
|
||||
for ilst_payload, ilst_end in ilst_boxes:
|
||||
for _item_start, item_end, item_type, item_payload in _iter_file_boxes(
|
||||
index = int.from_bytes(item_type, "big")
|
||||
key_span = keys.get(index)
|
||||
if keyed and key_span is None:
|
||||
# A keyed (ISO) meta box maps items through ``keys``;
|
||||
# an unmapped index is not an AIGC entry, and reading
|
||||
# its value would pull arbitrary metadata (e.g. cover
|
||||
# art) through the JSON parser on every scan. Only the
|
||||
# keyless QuickTime list falls through to content
|
||||
# validation below.
|
||||
continue
|
||||
for _data_start, data_end, data_type, data_payload in iter_file_boxes(
|
||||
stream,
|
||||
ilst_payload,
|
||||
ilst_end,
|
||||
item_payload,
|
||||
item_end,
|
||||
):
|
||||
index = int.from_bytes(item_type, "big")
|
||||
key_span = keys.get(index)
|
||||
if key_span is None:
|
||||
value_start = data_payload + 8
|
||||
value_size = data_end - value_start
|
||||
if data_type != b"data" or value_size < 0 or value_size > MAX_TC260_VALUE_BYTES:
|
||||
continue
|
||||
for _data_start, data_end, data_type, data_payload in _iter_file_boxes(
|
||||
stream,
|
||||
item_payload,
|
||||
item_end,
|
||||
):
|
||||
value_start = data_payload + 8
|
||||
value_size = data_end - value_start
|
||||
if data_type != b"data" or value_size < 0 or value_size > MAX_TC260_VALUE_BYTES:
|
||||
continue
|
||||
stream.seek(value_start)
|
||||
value = stream.read(value_size)
|
||||
if len(value) == value_size and parse_tc260_aigc_json(value) is not None:
|
||||
regions.append((*key_span, value_start, data_end, value))
|
||||
stream.seek(value_start)
|
||||
value = stream.read(value_size)
|
||||
if len(value) == value_size and parse_tc260_aigc_json(value) is not None:
|
||||
regions.append((key_span, value_start, data_end, value))
|
||||
return regions
|
||||
|
||||
|
||||
@@ -257,7 +308,7 @@ def tc260_aigc_payloads(path: str | Path) -> tuple[bytes, ...]:
|
||||
return ()
|
||||
stream.seek(0, 2)
|
||||
file_size = stream.tell()
|
||||
return tuple(region[4] for region in _tc260_aigc_regions(stream, file_size))
|
||||
return tuple(region[3] for region in _tc260_aigc_regions(stream, file_size))
|
||||
except OSError:
|
||||
return ()
|
||||
|
||||
@@ -268,6 +319,8 @@ def blank_tc260_aigc_tags(data: bytes) -> tuple[bytes, int]:
|
||||
Removing a nested ``ilst`` item would shift ``mdat`` in a fast-start MP4 and
|
||||
invalidate its chunk offsets. Replacing the four-byte key with ``free`` and
|
||||
the JSON value with spaces keeps every box size and media offset unchanged.
|
||||
A keyless QuickTime metadata-list entry has no key name, so only its value
|
||||
is blanked.
|
||||
"""
|
||||
if not is_isobmff(data):
|
||||
return data, 0
|
||||
@@ -276,9 +329,10 @@ def blank_tc260_aigc_tags(data: bytes) -> tuple[bytes, int]:
|
||||
return data, 0
|
||||
out = bytearray(data)
|
||||
key_spans: set[tuple[int, int]] = set()
|
||||
for key_start, key_end, value_start, value_end, _value in regions:
|
||||
key_spans.add((key_start, key_end))
|
||||
out[key_start:key_end] = b"free"
|
||||
for key_span, value_start, value_end, _value in regions:
|
||||
if key_span is not None:
|
||||
key_spans.add(key_span)
|
||||
out[key_span[0] : key_span[1]] = b"free"
|
||||
out[value_start:value_end] = b" " * (value_end - value_start)
|
||||
return bytes(out), len(key_spans)
|
||||
|
||||
@@ -425,7 +479,7 @@ def strip_isobmff_media_file(
|
||||
source: str | Path,
|
||||
output: str | Path,
|
||||
*,
|
||||
max_box_scan: int = _STREAM_SCAN_BYTES,
|
||||
max_box_scan: int = STREAM_SCAN_BYTES,
|
||||
) -> tuple[int, int]:
|
||||
"""Stream-copy an MP4/MOV/M4A while removing supported AI metadata.
|
||||
|
||||
@@ -456,7 +510,7 @@ def strip_isobmff_media_file(
|
||||
max_scan=max_box_scan,
|
||||
)
|
||||
tc260_regions = _tc260_aigc_regions(stream, file_size) if targets is not None else []
|
||||
tc260_key_spans = {(region[0], region[1]) for region in tc260_regions}
|
||||
tc260_key_spans = {region[0] for region in tc260_regions if region[0] is not None}
|
||||
|
||||
with atomic_video_output(output_path) as temporary_path:
|
||||
with source_path.open("rb") as source_stream, temporary_path.open("r+b") as temporary:
|
||||
@@ -466,9 +520,10 @@ def strip_isobmff_media_file(
|
||||
temporary.seek(box_start + 4)
|
||||
temporary.write(b"free")
|
||||
_overwrite_range(temporary, payload_start, box_end, byte=b"\x00")
|
||||
for key_start, _key_end, value_start, value_end, _value in tc260_regions:
|
||||
temporary.seek(key_start)
|
||||
temporary.write(b"free")
|
||||
for key_span, value_start, value_end, _value in tc260_regions:
|
||||
if key_span is not None:
|
||||
temporary.seek(key_span[0])
|
||||
temporary.write(b"free")
|
||||
_overwrite_range(temporary, value_start, value_end, byte=b" ")
|
||||
temporary.flush()
|
||||
os.fsync(temporary.fileno())
|
||||
|
||||
@@ -1,212 +0,0 @@
|
||||
"""Progress reporting utilities for long-running optional model operations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import io
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import warnings
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
_BAR_WIDTH = 28
|
||||
_SPINNER = ("|", "/", "-", "\\")
|
||||
|
||||
|
||||
def _truncate(text: str, max_len: int = 72) -> str:
|
||||
if len(text) <= max_len:
|
||||
return text
|
||||
return f"{text[: max(0, max_len - 3)]}..."
|
||||
|
||||
|
||||
def _build_bar(step: int) -> str:
|
||||
position = step % (2 * _BAR_WIDTH - 2)
|
||||
if position >= _BAR_WIDTH:
|
||||
position = 2 * _BAR_WIDTH - 2 - position
|
||||
cells = ["-"] * _BAR_WIDTH
|
||||
cells[position] = "="
|
||||
return "".join(cells)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _TaskResult:
|
||||
value: Any = None
|
||||
error: BaseException | None = None
|
||||
complete: threading.Event = field(default_factory=threading.Event)
|
||||
|
||||
|
||||
def run_with_progress(task: Callable[[], Any], progress_state: dict[str, str] | None = None) -> Any:
|
||||
"""Run ``task`` on a worker thread and render a compact terminal heartbeat."""
|
||||
outcome = _TaskResult()
|
||||
|
||||
def invoke() -> None:
|
||||
try:
|
||||
outcome.value = task()
|
||||
except BaseException as error: # re-raised on the caller thread
|
||||
outcome.error = error
|
||||
finally:
|
||||
outcome.complete.set()
|
||||
|
||||
worker = threading.Thread(target=invoke, name="raiw-progress-task", daemon=True)
|
||||
worker.start()
|
||||
started_at = time.monotonic()
|
||||
frame = 0
|
||||
terminal = sys.__stderr__
|
||||
while not outcome.complete.wait(0.1):
|
||||
message = _truncate((progress_state or {}).get("message", "Processing..."))
|
||||
elapsed = int(time.monotonic() - started_at)
|
||||
if terminal is not None:
|
||||
terminal.write(
|
||||
f"\r\033[2K {_SPINNER[frame % len(_SPINNER)]} [{_build_bar(frame)}] {elapsed:>3}s {message}"
|
||||
)
|
||||
terminal.flush()
|
||||
frame += 1
|
||||
|
||||
worker.join()
|
||||
elapsed = int(time.monotonic() - started_at)
|
||||
message = _truncate((progress_state or {}).get("message", "Processing..."))
|
||||
if terminal is not None:
|
||||
terminal.write(f"\r\033[2K Completed in {elapsed}s {message}\n")
|
||||
terminal.flush()
|
||||
if outcome.error is not None:
|
||||
raise outcome.error
|
||||
return outcome.value
|
||||
|
||||
|
||||
def _silence_diffusers() -> None:
|
||||
from diffusers.utils import logging as diffusers_logging
|
||||
|
||||
diffusers_logging.set_verbosity_error()
|
||||
disable = getattr(diffusers_logging, "disable_progress_bar", None)
|
||||
if callable(disable):
|
||||
disable()
|
||||
|
||||
|
||||
def _configure_quiet_libraries() -> None:
|
||||
os.environ.setdefault("HF_HUB_DISABLE_PROGRESS_BARS", "1")
|
||||
operations = (
|
||||
lambda: __import__("transformers").logging.set_verbosity_error(),
|
||||
_silence_diffusers,
|
||||
lambda: __import__("huggingface_hub").logging.set_verbosity_error(),
|
||||
)
|
||||
for operation in operations:
|
||||
with contextlib.suppress(Exception):
|
||||
operation()
|
||||
|
||||
|
||||
def silence_library_output(
|
||||
run_func: Callable[[], Any],
|
||||
set_progress: Callable[[str], None] | None = None,
|
||||
) -> Callable[[], Any]:
|
||||
"""Wrap a model call so third-party progress bars do not corrupt our CLI UI."""
|
||||
|
||||
def quiet_call() -> Any:
|
||||
if set_progress is not None:
|
||||
set_progress("Preparing model runtime...")
|
||||
_configure_quiet_libraries()
|
||||
with (
|
||||
warnings.catch_warnings(),
|
||||
contextlib.redirect_stdout(io.StringIO()),
|
||||
contextlib.redirect_stderr(io.StringIO()),
|
||||
):
|
||||
warnings.simplefilter("ignore")
|
||||
if set_progress is not None:
|
||||
set_progress("Running watermark regeneration...")
|
||||
return run_func()
|
||||
|
||||
return quiet_call
|
||||
|
||||
|
||||
@dataclass
|
||||
class _PipelineMonitor:
|
||||
total_steps: int
|
||||
device: str
|
||||
update: Callable[[str], None]
|
||||
bar_len: int
|
||||
label: str
|
||||
pre_phases: list[tuple[int, str]]
|
||||
post_phases: list[tuple[int, str]]
|
||||
first_step: threading.Event = field(default_factory=threading.Event)
|
||||
done: threading.Event = field(default_factory=threading.Event)
|
||||
started_at: float = field(default_factory=time.monotonic)
|
||||
last_step_at: float = field(default_factory=time.monotonic)
|
||||
|
||||
def callback(self, step: int, _timestep: int, _latents: Any) -> None:
|
||||
self.first_step.set()
|
||||
now = time.monotonic()
|
||||
self.last_step_at = now
|
||||
current = min(self.total_steps, step + 1)
|
||||
filled = round(self.bar_len * current / self.total_steps)
|
||||
elapsed = now - self.started_at
|
||||
eta = elapsed * max(0, self.total_steps - current) / max(1, current)
|
||||
bar = "#" * filled + "." * (self.bar_len - filled)
|
||||
self.update(
|
||||
f"{self.label} [{bar}] {current}/{self.total_steps}, "
|
||||
f"{elapsed:.0f}s elapsed, ~{eta:.0f}s left, {self.device}"
|
||||
)
|
||||
|
||||
def _phase_message(self, phases: list[tuple[int, str]], elapsed: float) -> str:
|
||||
message = phases[0][1]
|
||||
for threshold, candidate in phases:
|
||||
if elapsed < threshold:
|
||||
break
|
||||
message = candidate
|
||||
return message
|
||||
|
||||
def monitor(self) -> None:
|
||||
while not self.first_step.wait(0.4):
|
||||
elapsed = time.monotonic() - self.started_at
|
||||
self.update(self._phase_message(self.pre_phases, elapsed))
|
||||
decode_started: float | None = None
|
||||
while not self.done.wait(0.4):
|
||||
if time.monotonic() - self.last_step_at < 1.5:
|
||||
decode_started = None
|
||||
continue
|
||||
decode_started = decode_started or time.monotonic()
|
||||
self.update(self._phase_message(self.post_phases, time.monotonic() - decode_started))
|
||||
|
||||
def start(self) -> threading.Thread:
|
||||
self.started_at = self.last_step_at = time.monotonic()
|
||||
self.first_step.clear()
|
||||
self.done.clear()
|
||||
thread = threading.Thread(target=self.monitor, name="raiw-pipeline-progress", daemon=True)
|
||||
thread.start()
|
||||
return thread
|
||||
|
||||
|
||||
def make_pipeline_progress(
|
||||
effective_steps: int,
|
||||
device: str,
|
||||
set_progress: Callable[[str], None],
|
||||
*,
|
||||
bar_len: int = 20,
|
||||
label: str = "Denoising",
|
||||
pre_phases: list[tuple[int, str]] | None = None,
|
||||
post_phases: list[tuple[int, str]] | None = None,
|
||||
) -> tuple[Callable[..., None], threading.Event, threading.Event, Callable[[], threading.Thread]]:
|
||||
"""Build a callback and monitor for the legacy Diffusers callback interface."""
|
||||
|
||||
def qualify(entries: list[tuple[int, str]]) -> list[tuple[int, str]]:
|
||||
return [(second, f"{text} on {device}") for second, text in entries]
|
||||
|
||||
monitor = _PipelineMonitor(
|
||||
total_steps=max(1, effective_steps),
|
||||
device=device,
|
||||
update=set_progress,
|
||||
bar_len=bar_len,
|
||||
label=label,
|
||||
pre_phases=pre_phases or qualify([(0, "Encoding image"), (8, "Preparing denoiser"), (20, "Starting sampler")]),
|
||||
post_phases=post_phases or qualify([(0, "Decoding image"), (10, "Finalizing pixels"), (45, "Still decoding")]),
|
||||
)
|
||||
return monitor.callback, monitor.first_step, monitor.done, monitor.start
|
||||
|
||||
|
||||
def is_mps_error(error: Exception) -> bool:
|
||||
"""Return whether an error message identifies Apple's MPS backend."""
|
||||
return "mps" in str(error).casefold()
|
||||
@@ -30,6 +30,8 @@ from remove_ai_watermarks._internal.watermark_profiles import resolve_seed
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextManifest
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
QWEN_IMAGE_2512_MODEL_ID = "Qwen/Qwen-Image-2512"
|
||||
@@ -561,11 +563,18 @@ def _clip_sam_masks_to_boxes(
|
||||
image_size: tuple[int, int],
|
||||
) -> list[np.ndarray]:
|
||||
"""Match Impact Pack by intersecting each SAM mask with its detector box."""
|
||||
# Same rectangle primitive the shared fill uses, rather than a private zeros/fill
|
||||
# copy. `dilate=0` because this box CLIPS a SAM mask -- growing it would admit the
|
||||
# pixels the clip exists to exclude. The function-local import keeps region_eraser's
|
||||
# module-scope cv2 off this module's import path.
|
||||
from remove_ai_watermarks.region_eraser import boxes_to_mask
|
||||
|
||||
width, height = image_size
|
||||
clipped: list[np.ndarray] = []
|
||||
for mask, (x1, y1, x2, y2) in zip(masks, boxes, strict=True):
|
||||
box_mask = np.zeros((height, width), dtype=np.uint8)
|
||||
box_mask[max(0, y1) : min(height, y2), max(0, x1) : min(width, x2)] = 255
|
||||
# (x, y, w, h) with w = x2 - x1 keeps x + w == x2, so a negative origin clamps
|
||||
# to the same span the explicit max/min pair produced.
|
||||
box_mask = boxes_to_mask((height, width), [(x1, y1, x2 - x1, y2 - y1)], dilate=0)
|
||||
clipped.append(np.bitwise_and(mask.astype(np.uint8), box_mask))
|
||||
return clipped
|
||||
|
||||
@@ -636,8 +645,9 @@ class QwenZImagePipeline:
|
||||
def _require_cuda(self) -> None:
|
||||
if self.device != "cuda":
|
||||
raise RuntimeError(
|
||||
"The qwen-zimage pipeline is CUDA-only. Its Qwen-Image-2512 and "
|
||||
"Z-Image models do not fit the supported MPS path."
|
||||
"The qwen-zimage pipeline is CUDA-only; there is no CPU or MPS path "
|
||||
"for it. WatermarkRemover already refuses a non-CUDA device, so "
|
||||
"reaching this guard means the pipeline was constructed directly."
|
||||
)
|
||||
|
||||
def _vram_limit(self) -> float | None:
|
||||
@@ -706,6 +716,23 @@ class QwenZImagePipeline:
|
||||
"computation_device": "cuda",
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def _face_stage_dtype(cls) -> Any:
|
||||
"""The dtype every face-stage model loads and computes in.
|
||||
|
||||
Deliberately independent of ``self.torch_dtype``, which belongs to the global
|
||||
stage. ``sdxl-zimage`` runs its global model in fp16, and inheriting that here
|
||||
built the Z-Image modules bf16 (per the VRAM config below) while handing them
|
||||
fp16 latents -- a Half/BFloat16 conv mismatch that crashed every face image
|
||||
while zero-face inputs passed, so no unit test could see it.
|
||||
|
||||
Read by Z-Image and by SAM alike, so the whole stage moves together. SAM never
|
||||
crashed, because it casts its own inputs and leaves through ``.float()``, but it
|
||||
was reading the same wrong field and would re-land the bug for the next profile
|
||||
that changes its global dtype.
|
||||
"""
|
||||
return cls._zimage_vram_config()["computation_dtype"]
|
||||
|
||||
@staticmethod
|
||||
def _zimage_vram_config() -> dict[str, Any]:
|
||||
import torch
|
||||
@@ -837,7 +864,7 @@ class QwenZImagePipeline:
|
||||
model_configs.remove(text_encoder_config)
|
||||
log.info("Z-Image prompt embedding is cached; loading the stack without its text encoder")
|
||||
pipe = ZImagePipeline.from_pretrained(
|
||||
torch_dtype=self.torch_dtype,
|
||||
torch_dtype=self._face_stage_dtype(),
|
||||
device=self.device,
|
||||
model_configs=model_configs,
|
||||
tokenizer_config=ModelConfig(
|
||||
@@ -872,7 +899,7 @@ class QwenZImagePipeline:
|
||||
processor = AutoProcessor.from_pretrained(SAM_MODEL_ID, **kwargs)
|
||||
model = AutoModelForMaskGeneration.from_pretrained(
|
||||
SAM_MODEL_ID,
|
||||
torch_dtype=self.torch_dtype,
|
||||
torch_dtype=self._face_stage_dtype(),
|
||||
**kwargs,
|
||||
).to(self.device)
|
||||
model.eval()
|
||||
@@ -901,7 +928,7 @@ class QwenZImagePipeline:
|
||||
)
|
||||
original_sizes = inputs["original_sizes"].clone()
|
||||
reshaped_sizes = inputs["reshaped_input_sizes"].clone()
|
||||
inputs = _prepare_sam_inputs(inputs, self.device, self.torch_dtype)
|
||||
inputs = _prepare_sam_inputs(inputs, self.device, self._face_stage_dtype())
|
||||
with torch.inference_mode():
|
||||
outputs = model(**inputs, multimask_output=True)
|
||||
processed = processor.post_process_masks(
|
||||
@@ -948,6 +975,30 @@ class QwenZImagePipeline:
|
||||
result = result.resize(image.size, Image.Resampling.LANCZOS)
|
||||
return result.convert("RGB")
|
||||
|
||||
def _qwen_vae_roundtrip(self, image: Image.Image) -> Image.Image:
|
||||
"""Reconstruct source pixels through the already loaded Qwen VAE."""
|
||||
import torch
|
||||
|
||||
pipe, _controlnet_input_cls = self._load_qwen()
|
||||
source_width, source_height = image.size
|
||||
pad_width = (-source_width) % 8
|
||||
pad_height = (-source_height) % 8
|
||||
padded = image.convert("RGB")
|
||||
if pad_width or pad_height:
|
||||
padded = Image.fromarray(
|
||||
np.pad(
|
||||
np.asarray(padded),
|
||||
((0, pad_height), (0, pad_width), (0, 0)),
|
||||
mode="edge",
|
||||
)
|
||||
)
|
||||
pipe.load_models_to_device(["vae"])
|
||||
tensor = pipe.preprocess_image(padded).to(device=self.device, dtype=self.torch_dtype)
|
||||
with torch.inference_mode():
|
||||
latents = pipe.vae.encode(tensor)
|
||||
decoded = pipe.vae.decode(latents)
|
||||
return pipe.vae_output_to_image(decoded).crop((0, 0, source_width, source_height)).convert("RGB")
|
||||
|
||||
@staticmethod
|
||||
def _detail_size(
|
||||
crop_size: tuple[int, int],
|
||||
@@ -1019,10 +1070,33 @@ class QwenZImagePipeline:
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
text_manifest: VerifiedTextManifest | None = None,
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Image.Image:
|
||||
"""Execute global regeneration and masked face repair."""
|
||||
self._require_cuda()
|
||||
seed = resolve_seed(seed, "qwen-zimage")
|
||||
seed = resolve_seed(seed)
|
||||
donor = None
|
||||
if text_manifest is not None:
|
||||
self._progress("Reconstructing the verified text donor with the Qwen VAE...")
|
||||
if tile and max(image.size) > tile_size:
|
||||
from remove_ai_watermarks._internal.tiling import run_tiled
|
||||
|
||||
donor = run_tiled(
|
||||
self._qwen_vae_roundtrip,
|
||||
image,
|
||||
tile_size,
|
||||
tile_overlap,
|
||||
lambda message: self._progress(
|
||||
message.replace(
|
||||
"Tiled diffusion",
|
||||
"Reconstructing the verified text donor",
|
||||
1,
|
||||
)
|
||||
),
|
||||
)
|
||||
else:
|
||||
donor = self._qwen_vae_roundtrip(image)
|
||||
global_strength = (
|
||||
resolution_adaptive_denoise(image.width, image.height) if strength is None else float(strength)
|
||||
)
|
||||
@@ -1043,14 +1117,37 @@ class QwenZImagePipeline:
|
||||
boxes = detect_faces(image)
|
||||
if not boxes:
|
||||
self._progress("No faces detected; keeping the Qwen global result.")
|
||||
return global_result
|
||||
masks = self._sam_masks(image, boxes)
|
||||
face_strength = largest_face_denoise(boxes, image.size) * FACE_DENOISE_SCALE
|
||||
return self._run_faces(
|
||||
image,
|
||||
global_result,
|
||||
boxes,
|
||||
masks,
|
||||
strength=face_strength,
|
||||
seed=seed,
|
||||
result = global_result
|
||||
else:
|
||||
masks = self._sam_masks(image, boxes)
|
||||
face_strength = largest_face_denoise(boxes, image.size) * FACE_DENOISE_SCALE
|
||||
result = self._run_faces(
|
||||
image,
|
||||
global_result,
|
||||
boxes,
|
||||
masks,
|
||||
strength=face_strength,
|
||||
seed=seed,
|
||||
)
|
||||
if text_manifest is None:
|
||||
return result
|
||||
if donor is None:
|
||||
raise RuntimeError("Verified text restoration requires a Qwen-VAE donor")
|
||||
from remove_ai_watermarks._internal.text_restoration import (
|
||||
blend_fidelity_anchor,
|
||||
restore_verified_text,
|
||||
)
|
||||
|
||||
# The anchor is OFF by default since 0.27.1: blending 15% of the Qwen-VAE
|
||||
# donor ACROSS THE WHOLE FRAME returned detector-visible OpenAI SynthID on
|
||||
# poster-scale manifests (official Content Provenance API, 2026-08-19:
|
||||
# detected x6 with the anchor, clean x6 without it, base clean throughout;
|
||||
# see docs/text-protection-research.md). ``fidelity_anchor=True`` keeps the
|
||||
# 0.27.0 research behavior for reproduction.
|
||||
if fidelity_anchor:
|
||||
self._progress("Blending the Qwen-VAE fidelity anchor...")
|
||||
anchor = blend_fidelity_anchor(result, donor)
|
||||
else:
|
||||
anchor = result
|
||||
self._progress(f"Restoring {len(text_manifest.lines)} verified text lines...")
|
||||
return restore_verified_text(image, anchor, donor, text_manifest.lines)
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Shared validation for versioned JSON transport contracts."""
|
||||
|
||||
from collections.abc import Collection
|
||||
|
||||
|
||||
def require_schema_version(
|
||||
value: object,
|
||||
*,
|
||||
contract: str,
|
||||
supported: Collection[int],
|
||||
) -> int:
|
||||
"""Return an explicitly supported integer schema version or raise."""
|
||||
if type(value) is not int or value not in supported:
|
||||
versions = ", ".join(str(version) for version in sorted(supported))
|
||||
raise ValueError(f"Unsupported {contract} schema: {value!r}; supported versions: {versions}")
|
||||
return value
|
||||
@@ -30,9 +30,9 @@ from remove_ai_watermarks._internal.qwen_zimage_pipeline import (
|
||||
)
|
||||
from remove_ai_watermarks._internal.watermark_profiles import (
|
||||
CONTROLNET_CANNY_MODEL,
|
||||
DEFAULT_MODEL_ID,
|
||||
SDXL_LIGHTNING_MODEL_ID,
|
||||
SDXL_LIGHTNING_PATTERN,
|
||||
SDXL_MODEL_ID,
|
||||
)
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -88,7 +88,7 @@ class SdxlZImagePipeline(QwenZImagePipeline):
|
||||
controlnet = ControlNetModel.from_pretrained(CONTROLNET_CANNY_MODEL, torch_dtype=torch.float16, **token)
|
||||
vae = AutoencoderKL.from_pretrained(SDXL_VAE_MODEL_ID, torch_dtype=torch.float16, **token)
|
||||
pipe = StableDiffusionXLControlNetImg2ImgPipeline.from_pretrained(
|
||||
DEFAULT_MODEL_ID,
|
||||
SDXL_MODEL_ID,
|
||||
controlnet=controlnet,
|
||||
vae=vae,
|
||||
torch_dtype=torch.float16,
|
||||
|
||||
@@ -0,0 +1,446 @@
|
||||
"""Opt-in restoration of verified text from a Qwen VAE reconstruction."""
|
||||
|
||||
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks._internal.schema import require_schema_version
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
TEXT_MANIFEST_SCHEMA = 2
|
||||
_SUPPORTED_TEXT_MANIFEST_SCHEMAS = frozenset({1, TEXT_MANIFEST_SCHEMA})
|
||||
FIDELITY_BLEND_ALPHA = 0.15
|
||||
GLYPH_FEATHER = 0.5
|
||||
GLYPH_SIDE_PAD_RATIO = 0.12
|
||||
GLYPH_SIDE_PAD_LIMIT = 1.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VerifiedTextLine:
|
||||
"""One operator-verified source region in source-pixel coordinates."""
|
||||
|
||||
box: tuple[int, int, int, int]
|
||||
text: str | None = None
|
||||
script: str | None = None
|
||||
angle: float = 0.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VerifiedTextManifest:
|
||||
"""Verified text regions cryptographically bound to one decoded RGB source."""
|
||||
|
||||
source_pixel_sha256: str
|
||||
width: int
|
||||
height: int
|
||||
lines: tuple[VerifiedTextLine, ...]
|
||||
|
||||
|
||||
def source_pixel_sha256(image: Image.Image) -> str:
|
||||
"""Hash decoded RGB geometry and bytes, independent of container metadata."""
|
||||
rgb = image.convert("RGB")
|
||||
digest = hashlib.sha256()
|
||||
digest.update(rgb.width.to_bytes(8, "big"))
|
||||
digest.update(rgb.height.to_bytes(8, "big"))
|
||||
digest.update(rgb.tobytes())
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def load_verified_text_manifest(path: Path, source: Image.Image) -> VerifiedTextManifest:
|
||||
"""Load and validate an operator-verified manifest for exactly ``source``."""
|
||||
try:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError) as exc:
|
||||
raise ValueError(f"Cannot read text manifest {path}: {exc}") from exc
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("Text manifest must be a JSON object")
|
||||
schema_version = require_schema_version(
|
||||
payload.get("schema_version"),
|
||||
contract="text manifest",
|
||||
supported=_SUPPORTED_TEXT_MANIFEST_SCHEMAS,
|
||||
)
|
||||
if payload.get("verified") is not True:
|
||||
raise ValueError("Text manifest must contain verified=true after operator verification")
|
||||
|
||||
rgb = source.convert("RGB")
|
||||
width = _manifest_integer(payload, "width")
|
||||
height = _manifest_integer(payload, "height")
|
||||
if (width, height) != rgb.size:
|
||||
raise ValueError(f"Text manifest dimensions {width}x{height} do not match source {rgb.width}x{rgb.height}")
|
||||
expected_hash = payload.get("source_pixel_sha256")
|
||||
if not isinstance(expected_hash, str) or len(expected_hash) != 64:
|
||||
raise ValueError("Text manifest source_pixel_sha256 must be a 64-character SHA-256")
|
||||
actual_hash = source_pixel_sha256(rgb)
|
||||
if expected_hash.casefold() != actual_hash:
|
||||
raise ValueError("Text manifest source_pixel_sha256 does not match the decoded source pixels")
|
||||
|
||||
raw_lines = payload.get("lines")
|
||||
if not isinstance(raw_lines, list) or not raw_lines:
|
||||
raise ValueError("Text manifest lines must be a non-empty list")
|
||||
lines = tuple(_load_line(item, width, height, index, schema_version) for index, item in enumerate(raw_lines))
|
||||
if list(lines) != sorted(lines, key=lambda line: (line.box[1], line.box[0])):
|
||||
raise ValueError("Text manifest lines must be in top-to-bottom, left-to-right reading order")
|
||||
return VerifiedTextManifest(actual_hash, width, height, lines)
|
||||
|
||||
|
||||
def _manifest_integer(payload: dict[str, Any], key: str) -> int:
|
||||
value = payload.get(key)
|
||||
if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
|
||||
raise ValueError(f"Text manifest {key} must be a positive integer")
|
||||
return value
|
||||
|
||||
|
||||
def _load_line(item: Any, width: int, height: int, index: int, schema_version: int) -> VerifiedTextLine:
|
||||
if not isinstance(item, dict):
|
||||
raise ValueError(f"Text manifest line {index} must be an object")
|
||||
raw_box = item.get("box")
|
||||
if (
|
||||
not isinstance(raw_box, list)
|
||||
or len(raw_box) != 4
|
||||
or any(isinstance(value, bool) or not isinstance(value, int) for value in raw_box)
|
||||
):
|
||||
raise ValueError(f"Text manifest line {index} box must contain four integers")
|
||||
box = tuple(raw_box)
|
||||
x1, y1, x2, y2 = box
|
||||
if not (0 <= x1 < x2 <= width and 0 <= y1 < y2 <= height):
|
||||
raise ValueError(f"Text manifest line {index} box is outside the source dimensions")
|
||||
text: str | None = None
|
||||
script: str | None = None
|
||||
if schema_version == 1:
|
||||
text = item.get("text")
|
||||
script = item.get("script")
|
||||
if not isinstance(text, str) or not text.strip():
|
||||
raise ValueError(f"Text manifest line {index} text must be non-empty")
|
||||
if not isinstance(script, str) or not script.strip():
|
||||
raise ValueError(f"Text manifest line {index} script must be non-empty")
|
||||
angle_value = item.get("angle", 0.0)
|
||||
if isinstance(angle_value, bool) or not isinstance(angle_value, int | float):
|
||||
raise ValueError(f"Text manifest line {index} angle must be numeric")
|
||||
angle = float(angle_value)
|
||||
if not math.isfinite(angle) or abs(angle) > 30.0:
|
||||
raise ValueError(f"Text manifest line {index} angle must be between -30 and 30 degrees")
|
||||
return VerifiedTextLine(box, text, script, angle)
|
||||
|
||||
|
||||
def blend_fidelity_anchor(clean: Image.Image, donor: Image.Image) -> Image.Image:
|
||||
"""Blend 15% Qwen-VAE reconstruction into the oracle-clean pipeline output."""
|
||||
clean_rgb = np.asarray(clean.convert("RGB"), dtype=np.float32)
|
||||
donor_rgb = np.asarray(donor.convert("RGB"), dtype=np.float32)
|
||||
if clean_rgb.shape != donor_rgb.shape:
|
||||
raise ValueError("Clean result and Qwen-VAE donor dimensions must match")
|
||||
blended = np.rint(clean_rgb * (1.0 - FIDELITY_BLEND_ALPHA) + donor_rgb * FIDELITY_BLEND_ALPHA)
|
||||
return Image.fromarray(np.clip(blended, 0, 255).astype(np.uint8))
|
||||
|
||||
|
||||
def restore_verified_text(
|
||||
source: Image.Image,
|
||||
candidate: Image.Image,
|
||||
donor: Image.Image,
|
||||
lines: tuple[VerifiedTextLine, ...],
|
||||
) -> Image.Image:
|
||||
"""Erase candidate glyphs, then composite verified Qwen-VAE glyph cores."""
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
if not region_eraser.lama_available():
|
||||
raise RuntimeError(
|
||||
"Verified text restoration requires LaMa. Install: pip install 'remove-ai-watermarks[text-restoration]'"
|
||||
)
|
||||
source_rgb = np.asarray(source.convert("RGB"))
|
||||
candidate_rgb = np.asarray(candidate.convert("RGB"))
|
||||
donor_rgb = np.asarray(donor.convert("RGB"))
|
||||
if source_rgb.shape != candidate_rgb.shape or source_rgb.shape != donor_rgb.shape:
|
||||
raise ValueError("Source, candidate, and Qwen-VAE donor dimensions must match")
|
||||
|
||||
source_masks = [source_silhouette_mask(source_rgb, line.box, line.angle) for line in lines]
|
||||
for index, mask in enumerate(source_masks):
|
||||
if not np.any(mask):
|
||||
raise ValueError(f"Verified text line {index} produced no source glyph pixels")
|
||||
candidate_masks = [source_silhouette_mask(candidate_rgb, line.box, line.angle) for line in lines]
|
||||
erase_masks = []
|
||||
for line, source_mask, candidate_mask in zip(lines, source_masks, candidate_masks, strict=True):
|
||||
radius = 5 if line.box[3] - line.box[1] >= 48 else 3
|
||||
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * radius + 1,) * 2)
|
||||
erase_masks.append(cv2.dilate(np.maximum(source_mask, candidate_mask), kernel))
|
||||
del candidate_masks
|
||||
|
||||
groups = group_text_lines(lines)
|
||||
background = cv2.cvtColor(candidate_rgb, cv2.COLOR_RGB2BGR)
|
||||
for group in groups:
|
||||
background = region_eraser.erase_lama(background, np.maximum.reduce([erase_masks[index] for index in group]))
|
||||
background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
|
||||
residual_masks = [
|
||||
residual_glyph_mask(background_rgb, mask, line.box) for line, mask in zip(lines, erase_masks, strict=True)
|
||||
]
|
||||
for group in groups:
|
||||
residual = np.maximum.reduce([residual_masks[index] for index in group])
|
||||
if np.any(residual):
|
||||
background = region_eraser.erase_lama(background, residual)
|
||||
del erase_masks, residual_masks
|
||||
restored = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
|
||||
restored = composite_fresh_text_edges(source_rgb, restored, lines, source_masks)
|
||||
source_glyph_mask = np.maximum.reduce(source_masks)
|
||||
restored = composite_reconstructed_glyphs(donor_rgb, restored, source_glyph_mask)
|
||||
return Image.fromarray(restored)
|
||||
|
||||
|
||||
def _glyph_crop_box(
|
||||
box: tuple[int, int, int, int],
|
||||
width: int,
|
||||
height: int,
|
||||
left_pad: int,
|
||||
right_pad: int,
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Widen the detector box so glyph edges stay inside the silhouette crop.
|
||||
|
||||
Detector boxes can stop inside a leading flourish or trailing punctuation,
|
||||
and Paddle line boxes sit 2-5 px above the true ink bottom on the poster
|
||||
fixtures. The crop is otherwise the box itself, so those pixels never reach
|
||||
the donor composite. Start 12% sideways, expand each side independently
|
||||
while foreground reaches its boundary, and pad 8% up and 25% down.
|
||||
"""
|
||||
x1, y1, x2, y2 = box
|
||||
line_h = max(1, y2 - y1)
|
||||
pad_top = max(1, round(line_h * 0.08))
|
||||
pad_bot = max(2, round(line_h * 0.25))
|
||||
return (
|
||||
max(0, x1 - left_pad),
|
||||
max(0, y1 - pad_top),
|
||||
min(width, x2 + right_pad),
|
||||
min(height, y2 + pad_bot),
|
||||
)
|
||||
|
||||
|
||||
def _silhouette_crop_mask(
|
||||
source_rgb: NDArray[Any],
|
||||
crop_box: tuple[int, int, int, int],
|
||||
angle: float,
|
||||
) -> NDArray[Any]:
|
||||
"""Threshold one candidate crop so its horizontal boundaries can be tested."""
|
||||
height, width = source_rgb.shape[:2]
|
||||
x1, y1, x2, y2 = crop_box
|
||||
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
|
||||
support = np.ones(gray.shape, dtype=np.uint8)
|
||||
if angle:
|
||||
box_width, box_height = x2 - x1, y2 - y1
|
||||
theta = math.radians(abs(angle))
|
||||
cosine, sine = math.cos(theta), math.sin(theta)
|
||||
denominator = cosine * cosine - sine * sine
|
||||
rect_width = (box_width * cosine - box_height * sine) / denominator
|
||||
rect_height = (box_height * cosine - box_width * sine) / denominator
|
||||
rotated = cv2.boxPoints(
|
||||
((box_width / 2, box_height / 2), (max(1.0, rect_width * 0.92), max(1.0, rect_height * 0.62)), -angle)
|
||||
)
|
||||
support.fill(0)
|
||||
cv2.fillConvexPoly(support, np.rint(rotated).astype(np.int32), 1)
|
||||
values = gray[support > 0]
|
||||
background_luma = float(np.median(values))
|
||||
else:
|
||||
ring_pad = max(6, min(20, (y2 - y1) // 4))
|
||||
rx1, ry1, rx2, ry2 = _clip_box(crop_box, width, height, pad=ring_pad)
|
||||
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
|
||||
ring = np.ones(context.shape, dtype=bool)
|
||||
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
|
||||
background_luma = float(np.median(context[ring])) if ring.any() else float(np.median(gray))
|
||||
values = gray.reshape(-1)
|
||||
low, high = float(np.percentile(values, 2)), float(np.percentile(values, 98))
|
||||
dark_contrast, light_contrast = background_luma - low, high - background_luma
|
||||
threshold = max(16.0, min(56.0, max(light_contrast, dark_contrast) * 0.22))
|
||||
if light_contrast > dark_contrast:
|
||||
crop_mask = (gray.astype(np.float32) >= background_luma + threshold).astype(np.uint8) * 255
|
||||
else:
|
||||
crop_mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
|
||||
crop_mask[support == 0] = 0
|
||||
return crop_mask
|
||||
|
||||
|
||||
def _anchored_components_reach_sides(
|
||||
crop_mask: NDArray[Any],
|
||||
anchor_box: tuple[int, int, int, int],
|
||||
) -> tuple[bool, bool]:
|
||||
"""Whether anchored foreground reaches the left and right crop sides."""
|
||||
count, labels, stats, _centroids = cv2.connectedComponentsWithStats(crop_mask, 8)
|
||||
x1, y1, x2, y2 = anchor_box
|
||||
anchor_counts = np.bincount(labels[y1:y2, x1:x2].reshape(-1), minlength=count)
|
||||
anchor_counts[0] = 0
|
||||
|
||||
def reaches(edge_labels: NDArray[Any]) -> bool:
|
||||
return any(
|
||||
label != 0 and anchor_counts[label] >= max(3, round(stats[label, cv2.CC_STAT_AREA] * 0.1))
|
||||
for label in np.unique(edge_labels)
|
||||
)
|
||||
|
||||
return reaches(labels[y1:y2, :2]), reaches(labels[y1:y2, -2:])
|
||||
|
||||
|
||||
def source_silhouette_mask(
|
||||
source_rgb: NDArray[Any],
|
||||
box: tuple[int, int, int, int],
|
||||
angle: float = 0.0,
|
||||
) -> NDArray[Any]:
|
||||
"""Recover a thresholded glyph shape without retaining source amplitudes."""
|
||||
height, width = source_rgb.shape[:2]
|
||||
box_x1, box_y1, box_x2, box_y2 = box
|
||||
line_h = max(1, box_y2 - box_y1)
|
||||
step = max(2, round(line_h * GLYPH_SIDE_PAD_RATIO))
|
||||
limit = max(step, round(line_h * GLYPH_SIDE_PAD_LIMIT))
|
||||
left_pad = right_pad = step
|
||||
while True:
|
||||
x1, y1, x2, y2 = _glyph_crop_box(box, width, height, left_pad, right_pad)
|
||||
crop_mask = _silhouette_crop_mask(source_rgb, (x1, y1, x2, y2), angle)
|
||||
core_y1 = max(0, box_y1 - y1)
|
||||
core_y2 = min(y2 - y1, box_y2 - y1)
|
||||
anchor_box = (max(0, box_x1 - x1), core_y1, min(x2 - x1, box_x2 - x1), core_y2)
|
||||
can_expand_left = x1 > 0 and left_pad < limit
|
||||
can_expand_right = x2 < width and right_pad < limit
|
||||
left_anchored, right_anchored = (
|
||||
_anchored_components_reach_sides(crop_mask, anchor_box)
|
||||
if can_expand_left or can_expand_right
|
||||
else (False, False)
|
||||
)
|
||||
left_touches = can_expand_left and left_anchored
|
||||
right_touches = can_expand_right and right_anchored
|
||||
if not left_touches and not right_touches:
|
||||
break
|
||||
if left_touches:
|
||||
left_pad = min(limit, left_pad + step)
|
||||
if right_touches:
|
||||
right_pad = min(limit, right_pad + step)
|
||||
result = np.zeros((height, width), dtype=np.uint8)
|
||||
result[y1:y2, x1:x2] = crop_mask
|
||||
return result
|
||||
|
||||
|
||||
def residual_glyph_mask(
|
||||
background_rgb: NDArray[Any],
|
||||
original_mask: NDArray[Any],
|
||||
box: tuple[int, int, int, int],
|
||||
) -> NDArray[Any]:
|
||||
"""Find glyph-like contrast left after the first inpaint pass."""
|
||||
residual = _foreground_mask(background_rgb, box)
|
||||
residual = cv2.bitwise_and(residual, original_mask)
|
||||
return cv2.dilate(residual, np.ones((5, 5), np.uint8), iterations=1)
|
||||
|
||||
|
||||
def composite_fresh_text_edges(
|
||||
source_rgb: NDArray[Any],
|
||||
background_rgb: NDArray[Any],
|
||||
lines: tuple[VerifiedTextLine, ...],
|
||||
masks: list[NDArray[Any]],
|
||||
) -> NDArray[Any]:
|
||||
"""Render fresh antialiased edges for source-derived glyph masks."""
|
||||
restored = background_rgb
|
||||
for line, mask in zip(lines, masks, strict=True):
|
||||
color = _sample_text_color(source_rgb, mask, line.box)
|
||||
restored = composite_fresh_silhouette(restored, mask, color)
|
||||
return restored
|
||||
|
||||
|
||||
def composite_reconstructed_glyphs(
|
||||
donor_rgb: NDArray[Any],
|
||||
background_rgb: NDArray[Any],
|
||||
glyph_mask: NDArray[Any],
|
||||
*,
|
||||
feather: float = GLYPH_FEATHER,
|
||||
) -> NDArray[Any]:
|
||||
"""Composite an exact reconstructed core with a narrow donor edge."""
|
||||
if donor_rgb.shape != background_rgb.shape or donor_rgb.shape[:2] != glyph_mask.shape:
|
||||
raise ValueError("donor, background, and glyph mask dimensions must match")
|
||||
blurred = cv2.GaussianBlur(glyph_mask, (0, 0), feather) if feather > 0 else glyph_mask
|
||||
alpha = np.maximum(glyph_mask, blurred).astype(np.float32) / 255.0
|
||||
combined = donor_rgb.astype(np.float32) * alpha[..., None] + background_rgb.astype(np.float32) * (
|
||||
1.0 - alpha[..., None]
|
||||
)
|
||||
return np.clip(np.rint(combined), 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def composite_fresh_silhouette(
|
||||
background_rgb: NDArray[Any],
|
||||
glyph_mask: NDArray[Any],
|
||||
color: tuple[int, int, int],
|
||||
*,
|
||||
feather: float = 0.35,
|
||||
) -> NDArray[Any]:
|
||||
"""Render a binary source shape with fresh color and antialiasing."""
|
||||
if background_rgb.shape[:2] != glyph_mask.shape:
|
||||
raise ValueError("background and glyph mask dimensions must match")
|
||||
antialiased = cv2.GaussianBlur(glyph_mask, (0, 0), feather) if feather > 0 else glyph_mask
|
||||
alpha = antialiased.astype(np.float32)[..., None] / 255.0
|
||||
foreground = np.empty_like(background_rgb)
|
||||
foreground[:, :] = color
|
||||
combined = foreground.astype(np.float32) * alpha + background_rgb.astype(np.float32) * (1.0 - alpha)
|
||||
return np.clip(combined, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def _clip_box(box: tuple[int, int, int, int], width: int, height: int, pad: int = 0) -> tuple[int, int, int, int]:
|
||||
x1, y1, x2, y2 = box
|
||||
return max(0, x1 - pad), max(0, y1 - pad), min(width, x2 + pad), min(height, y2 + pad)
|
||||
|
||||
|
||||
def _foreground_mask(source_rgb: NDArray[Any], box: tuple[int, int, int, int]) -> NDArray[Any]:
|
||||
height, width = source_rgb.shape[:2]
|
||||
line_height = box[3] - box[1]
|
||||
x1, y1, x2, y2 = _clip_box(box, width, height, pad=max(6, int(line_height * 0.12)))
|
||||
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
|
||||
ring_pad = max(8, min(24, (y2 - y1) // 5))
|
||||
rx1, ry1, rx2, ry2 = _clip_box((x1, y1, x2, y2), width, height, pad=ring_pad)
|
||||
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
|
||||
ring = np.ones(context.shape, dtype=bool)
|
||||
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
|
||||
background_luma = float(np.median(context[ring])) if ring.any() else float(np.median(gray))
|
||||
low, high = float(np.percentile(gray, 4)), float(np.percentile(gray, 96))
|
||||
dark_contrast, light_contrast = background_luma - low, high - background_luma
|
||||
threshold = max(24.0, min(72.0, max(light_contrast, dark_contrast) * 0.32))
|
||||
if light_contrast > dark_contrast:
|
||||
mask = (gray.astype(np.float32) >= background_luma + threshold).astype(np.uint8) * 255
|
||||
else:
|
||||
mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
|
||||
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((2, 2), np.uint8))
|
||||
dilation = 5 if line_height >= 48 else 3
|
||||
mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * dilation + 1,) * 2))
|
||||
result = np.zeros((height, width), dtype=np.uint8)
|
||||
result[y1:y2, x1:x2] = mask
|
||||
return result
|
||||
|
||||
|
||||
def _sample_text_color(
|
||||
source_rgb: NDArray[Any], mask: NDArray[Any], box: tuple[int, int, int, int]
|
||||
) -> tuple[int, int, int]:
|
||||
height, width = source_rgb.shape[:2]
|
||||
x1, y1, x2, y2 = _clip_box(box, width, height, pad=2)
|
||||
crop = source_rgb[y1:y2, x1:x2]
|
||||
pixels = crop[mask[y1:y2, x1:x2] > 0]
|
||||
luma = pixels.mean(axis=1)
|
||||
background_luma = float(crop[[0, -1], :, :].reshape(-1, 3).mean(axis=1).mean())
|
||||
selected = (
|
||||
pixels[luma <= np.percentile(luma, 20)] if background_luma >= 128 else pixels[luma >= np.percentile(luma, 80)]
|
||||
)
|
||||
return tuple(int(value) for value in np.median(selected, axis=0))
|
||||
|
||||
|
||||
def group_text_lines(lines: Sequence[VerifiedTextLine]) -> list[list[int]]:
|
||||
"""Group nearby same-script lines for a shared LaMa erase pass."""
|
||||
groups: list[list[int]] = []
|
||||
for index, line in enumerate(lines):
|
||||
if not groups:
|
||||
groups.append([index])
|
||||
continue
|
||||
previous = lines[groups[-1][-1]]
|
||||
gap = line.box[1] - previous.box[3]
|
||||
if line.script != previous.script or gap > max(60, int((previous.box[3] - previous.box[1]) * 1.1)):
|
||||
groups.append([index])
|
||||
else:
|
||||
groups[-1].append(index)
|
||||
return groups
|
||||
@@ -1,11 +1,11 @@
|
||||
"""Sliding-window tiled diffusion for large images.
|
||||
|
||||
The img2img / ControlNet pipeline denoises the WHOLE image in one forward pass,
|
||||
so it OOMs on MPS/GPU above ~2K (issue #10). Tiling splits the image into
|
||||
overlapping tiles -- each kept near SDXL's ~1024 training size -- regenerates
|
||||
each tile independently, and feather-blends the overlaps. The result retains the
|
||||
input's native dimensions without an explicit ``--max-resolution`` downscale, but
|
||||
it is not pixel-lossless because every tile is regenerated.
|
||||
The global stage denoises the WHOLE image in one forward pass, so it OOMs on a
|
||||
GPU above ~2K (issue #10). Tiling splits the image into overlapping tiles -- each
|
||||
kept near the ~1024 training size -- regenerates each tile independently, and
|
||||
feather-blends the overlaps. The result retains the input's native dimensions
|
||||
without an explicit ``--max-resolution`` downscale, but it is not pixel-lossless
|
||||
because every tile is regenerated.
|
||||
|
||||
The geometry (``plan_tiles``) and the blend weighting (``feather_weights``) are
|
||||
pure functions, unit-tested without the diffusion model. ``run_tiled`` is the
|
||||
@@ -100,59 +100,6 @@ def feather_weights(width: int, height: int, overlap: int) -> NDArray[Any]:
|
||||
return weights
|
||||
|
||||
|
||||
def feather_region_composite(
|
||||
base: NDArray[Any],
|
||||
regenerated: NDArray[Any],
|
||||
box: tuple[int, int, int, int],
|
||||
*,
|
||||
feather: int = 64,
|
||||
) -> NDArray[Any]:
|
||||
"""Composite ``regenerated`` over ``base`` inside ``box`` only, feathering the seam.
|
||||
|
||||
For AI-ENHANCED composites (digitalSourceType ``compositeWithTrainedAlgorithmicMedia``):
|
||||
the diffusion remover regenerates the whole frame, but only the AI-composited
|
||||
REGION should change -- the rest is a real photo that must be preserved. This
|
||||
blends the regenerated pixels in over ``box = (x, y, w, h)`` with a separable
|
||||
linear taper of ``feather`` px at the box edges, so the result equals ``base``
|
||||
EXACTLY outside the box and ramps smoothly (no hard seam) at the boundary.
|
||||
|
||||
Pure and model-free (unit-tested): ``base`` and ``regenerated`` must be the same
|
||||
shape (H x W, or H x W x C). The output preserves ``base``'s dtype. ``feather`` is
|
||||
clamped to half the box on each axis, so a small region still tapers symmetrically;
|
||||
``feather=0`` is a hard-edged paste.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
if base.shape != regenerated.shape:
|
||||
raise ValueError(f"shape mismatch: base {base.shape} vs regenerated {regenerated.shape}")
|
||||
h, w = base.shape[:2]
|
||||
x, y, bw, bh = box
|
||||
x0, y0 = max(0, x), max(0, y)
|
||||
x1, y1 = min(w, x + bw), min(h, y + bh)
|
||||
out = base.copy()
|
||||
if x1 <= x0 or y1 <= y0:
|
||||
return out # empty / off-image box -> nothing regenerated
|
||||
|
||||
def taper(n: int) -> NDArray[Any]:
|
||||
win = np.ones(n, dtype=np.float32)
|
||||
f = min(max(feather, 0), n // 2)
|
||||
if f > 0:
|
||||
ramp = (np.arange(f, dtype=np.float32) + 1.0) / (f + 1.0) # in (0, 1), 0 at the edge
|
||||
win[:f] = ramp
|
||||
win[n - f :] = ramp[::-1]
|
||||
return win
|
||||
|
||||
rh, rw = y1 - y0, x1 - x0
|
||||
wmap = np.outer(taper(rh), taper(rw)) # ~0 at the box edge, 1 in the interior
|
||||
if base.ndim == 3:
|
||||
wmap = wmap[:, :, None]
|
||||
roi_base = base[y0:y1, x0:x1].astype(np.float32)
|
||||
roi_gen = regenerated[y0:y1, x0:x1].astype(np.float32)
|
||||
blended = roi_base * (1.0 - wmap) + roi_gen * wmap
|
||||
out[y0:y1, x0:x1] = np.clip(blended, 0, 255).astype(base.dtype)
|
||||
return out
|
||||
|
||||
|
||||
def run_tiled(
|
||||
generate_tile: Callable[[PILImage.Image], PILImage.Image],
|
||||
image: PILImage.Image,
|
||||
|
||||
@@ -1,33 +1,57 @@
|
||||
"""Project-owned configuration for invisible-watermark regeneration profiles."""
|
||||
"""Project-owned configuration for invisible-watermark regeneration profiles.
|
||||
|
||||
Two profiles remain, and both are CUDA-only: ``qwen-zimage`` (the default) and
|
||||
``sdxl-zimage``. The older ``controlnet``, ``sdxl``, ``qwen`` and ``default`` profiles
|
||||
were removed rather than kept as a CPU path, because none of them matched the two-stage
|
||||
recipe's face preservation and keeping them implied a quality this library no longer
|
||||
offers. Removing invisible watermarks therefore needs a CUDA device; the visible-mark
|
||||
registry and every identify path still run anywhere.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
DEFAULT_MODEL_ID = "stabilityai/stable-diffusion-xl-base-1.0"
|
||||
QWEN_MODEL_ID = "Qwen/Qwen-Image"
|
||||
# SDXL base is no longer a profile of its own, but it is still the global stage of
|
||||
# sdxl-zimage, so the checkpoint id stays. Named for what it is rather than
|
||||
# ``DEFAULT_MODEL_ID``: there is no user-selectable model any more, so "default"
|
||||
# implied an override that both profiles reject.
|
||||
SDXL_MODEL_ID = "stabilityai/stable-diffusion-xl-base-1.0"
|
||||
CONTROLNET_CANNY_MODEL = "xinsir/controlnet-canny-sdxl-1.0"
|
||||
|
||||
SDXL_PROFILE = "sdxl"
|
||||
QWEN_ZIMAGE_PROFILE = "qwen-zimage"
|
||||
SDXL_ZIMAGE_PROFILE = "sdxl-zimage"
|
||||
DEFAULT_PROFILE = QWEN_ZIMAGE_PROFILE
|
||||
PROFILE_CHOICES = (QWEN_ZIMAGE_PROFILE, SDXL_ZIMAGE_PROFILE)
|
||||
|
||||
# The modules a real removal run needs, and the extra that installs them. Both live
|
||||
# here, in the only profile module that imports nothing heavy, because the CLI's
|
||||
# availability gate and the remover's own precondition must agree: when they drifted,
|
||||
# the CLI passed on a torch+diffusers environment and the run then died at the
|
||||
# DiffSynth face stage, telling the user to install an extra that does not contain it.
|
||||
REMOVAL_MODULES = ("torch", "diffusers", "diffsynth")
|
||||
# Shell-quoted, because this string is printed as a command the user copy-pastes.
|
||||
# Bare brackets are a glob in zsh (the macOS default shell): an unquoted
|
||||
# ``pip install remove-ai-watermarks[qwen-zimage]`` dies with "no matches found"
|
||||
# before pip ever runs -- another install hint that does not install.
|
||||
INVISIBLE_EXTRA = "'remove-ai-watermarks[qwen-zimage]'"
|
||||
|
||||
# qwen-zimage's output already matches the input's detail level, so polishing it is a
|
||||
# no-op at best. sdxl-zimage's global pass leaves the softer output the polish exists
|
||||
# for. This is per-profile data, not a CLI concern: the flag defaults to None so that
|
||||
# "the user did not choose" stays a value rather than an inference from Click state.
|
||||
PROFILE_ADAPTIVE_POLISH = {QWEN_ZIMAGE_PROFILE: False, SDXL_ZIMAGE_PROFILE: True}
|
||||
|
||||
SDXL_LIGHTNING_MODEL_ID = "ByteDance/SDXL-Lightning"
|
||||
SDXL_LIGHTNING_PATTERN = "sdxl_lightning_4step_lora.safetensors"
|
||||
|
||||
OPENAI_STRENGTH = 0.10
|
||||
GEMINI_STRENGTH = 0.15
|
||||
UNKNOWN_STRENGTH = GEMINI_STRENGTH
|
||||
DEFAULT_STRENGTH = UNKNOWN_STRENGTH
|
||||
|
||||
QWEN_OPENAI_STRENGTH = 0.10
|
||||
QWEN_GEMINI_STRENGTH = 0.25
|
||||
QWEN_UNKNOWN_STRENGTH = QWEN_GEMINI_STRENGTH
|
||||
# Both profiles are certified at a fixed seed because SynthID removal near the
|
||||
# strength floor is seed-dependent. The step count and CFG are not settable at all --
|
||||
# each stage owns them (``GLOBAL_STEPS`` / ``FACE_STEPS`` in qwen_zimage_pipeline).
|
||||
PROFILE_SEED = 0
|
||||
|
||||
# sdxl-zimage runs the qwen-zimage recipe on an SDXL global stage, and strength is
|
||||
# architecture-bound: at Qwen's 0.154 an SDXL global pass leaves SynthID on a native
|
||||
@@ -45,88 +69,88 @@ SDXL_ZIMAGE_OPENAI_STRENGTH = 0.15
|
||||
SDXL_ZIMAGE_GEMINI_STRENGTH = 0.25
|
||||
SDXL_ZIMAGE_UNKNOWN_STRENGTH = SDXL_ZIMAGE_GEMINI_STRENGTH
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _StrengthPolicy:
|
||||
unknown: float
|
||||
by_vendor: dict[str, float]
|
||||
|
||||
def choose(self, vendor: str | None) -> float:
|
||||
return self.by_vendor.get((vendor or "").casefold(), self.unknown)
|
||||
# qwen-zimage keeps its resolution curve for openai/unknown content, but Google
|
||||
# content takes this flat oracle-measured floor instead of the curve (whose top,
|
||||
# 0.154, left a 4.33 MP CJK-sign fixture detected x3 on the full production path
|
||||
# with a valid pixel control, 2026-08-18). 0.30 anchors measured clean in Gemini
|
||||
# across two fixtures and two work accounts (CJK sign + 18-face, 3/3 checks each,
|
||||
# 2026-08 research) and stayed clean under the vae-glyphs donor layer on top.
|
||||
QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.30
|
||||
|
||||
|
||||
_STANDARD_POLICY = _StrengthPolicy(
|
||||
unknown=UNKNOWN_STRENGTH,
|
||||
by_vendor={"openai": OPENAI_STRENGTH, "google": GEMINI_STRENGTH},
|
||||
)
|
||||
_QWEN_POLICY = _StrengthPolicy(
|
||||
unknown=QWEN_UNKNOWN_STRENGTH,
|
||||
by_vendor={"openai": QWEN_OPENAI_STRENGTH, "google": QWEN_GEMINI_STRENGTH},
|
||||
)
|
||||
_SDXL_ZIMAGE_POLICY = _StrengthPolicy(
|
||||
unknown=SDXL_ZIMAGE_UNKNOWN_STRENGTH,
|
||||
by_vendor={"openai": SDXL_ZIMAGE_OPENAI_STRENGTH, "google": SDXL_ZIMAGE_GEMINI_STRENGTH},
|
||||
)
|
||||
# sdxl-zimage picks its strength from the VENDOR (unlike qwen-zimage, which derives it
|
||||
# from image area). An unlisted or unknown vendor falls back to the Gemini value.
|
||||
_SDXL_ZIMAGE_STRENGTH_BY_VENDOR: dict[str, float] = {
|
||||
"openai": SDXL_ZIMAGE_OPENAI_STRENGTH,
|
||||
"google": SDXL_ZIMAGE_GEMINI_STRENGTH,
|
||||
}
|
||||
_ALIASES = {
|
||||
"default": SDXL_PROFILE,
|
||||
"qwen_zimage": QWEN_ZIMAGE_PROFILE,
|
||||
"sdxl_zimage": SDXL_ZIMAGE_PROFILE,
|
||||
}
|
||||
_FOUR_STEP_PROFILES = frozenset({QWEN_ZIMAGE_PROFILE, SDXL_ZIMAGE_PROFILE})
|
||||
|
||||
|
||||
def normalize_profile(profile: str) -> str:
|
||||
"""Normalize spelling and resolve compatibility aliases."""
|
||||
"""Normalize spelling and resolve the underscore spellings."""
|
||||
value = profile.strip().casefold()
|
||||
return _ALIASES.get(value, value)
|
||||
|
||||
|
||||
def resolve_steps(num_inference_steps: int | None, pipeline: str) -> int:
|
||||
"""Return an explicit step count or the selected profile's default."""
|
||||
if num_inference_steps is not None:
|
||||
return num_inference_steps
|
||||
return 4 if normalize_profile(pipeline) in _FOUR_STEP_PROFILES else 50
|
||||
def resolve_seed(seed: int | None) -> int:
|
||||
"""Keep both profiles reproducible by default."""
|
||||
return PROFILE_SEED if seed is None else seed
|
||||
|
||||
|
||||
def resolve_seed(seed: int | None, pipeline: str) -> int | None:
|
||||
"""Keep the fixed four-step Z-Image profiles reproducible by default."""
|
||||
if seed is not None:
|
||||
return seed
|
||||
return 0 if normalize_profile(pipeline) in _FOUR_STEP_PROFILES else None
|
||||
def resolve_adaptive_polish(adaptive_polish: bool | None, pipeline: str) -> bool:
|
||||
"""Return an explicit polish choice, or the profile's calibrated default."""
|
||||
if adaptive_polish is not None:
|
||||
return adaptive_polish
|
||||
return PROFILE_ADAPTIVE_POLISH.get(normalize_profile(pipeline), True)
|
||||
|
||||
|
||||
def strength_default_help() -> str:
|
||||
"""Describe the live default policy without duplicating its values."""
|
||||
return (
|
||||
f"vendor-adaptive (OpenAI {OPENAI_STRENGTH} / Google {GEMINI_STRENGTH} / "
|
||||
f"unknown {UNKNOWN_STRENGTH}, from the C2PA issuer; qwen-zimage instead uses "
|
||||
"resolution-adaptive denoise)"
|
||||
"profile-adaptive (qwen-zimage uses resolution-adaptive denoise, with a "
|
||||
f"flat oracle-measured Google floor of {QWEN_ZIMAGE_GOOGLE_STRENGTH}; sdxl-zimage "
|
||||
f"uses OpenAI {SDXL_ZIMAGE_OPENAI_STRENGTH} / Google {SDXL_ZIMAGE_GEMINI_STRENGTH} / "
|
||||
f"unknown {SDXL_ZIMAGE_UNKNOWN_STRENGTH}, from the C2PA issuer)"
|
||||
)
|
||||
|
||||
|
||||
def resolve_strength(strength: float | None, vendor: str | None = None, pipeline: str | None = None) -> float:
|
||||
"""Resolve a user override or the calibrated policy for a profile and vendor."""
|
||||
def resolve_strength(
|
||||
strength: float | None,
|
||||
vendor: str | None = None,
|
||||
pipeline: str | None = None,
|
||||
*,
|
||||
size: tuple[int, int] | None = None,
|
||||
) -> float:
|
||||
"""Resolve a user override or the calibrated policy for a profile and vendor.
|
||||
|
||||
Total by design. qwen-zimage picks its strength from image area rather than
|
||||
from the vendor, so it needs ``size``; returning ``None`` for it instead would push that
|
||||
branch onto every caller and move one of the two strength policies outside this
|
||||
module. ``size`` is required for qwen-zimage without an explicit strength.
|
||||
One measured exception since 0.27.2: Google-provenance content on qwen-zimage
|
||||
takes the flat ``QWEN_ZIMAGE_GOOGLE_STRENGTH`` floor instead of the area curve
|
||||
(the curve's 0.154 top left a 4.33 MP fixture oracle-detected on the full
|
||||
production path; see the constant's comment).
|
||||
"""
|
||||
if strength is not None:
|
||||
return strength
|
||||
profile = normalize_profile(pipeline) if pipeline is not None else ""
|
||||
if profile == "qwen":
|
||||
policy = _QWEN_POLICY
|
||||
elif profile == SDXL_ZIMAGE_PROFILE:
|
||||
policy = _SDXL_ZIMAGE_POLICY
|
||||
else:
|
||||
policy = _STANDARD_POLICY
|
||||
return policy.choose(vendor)
|
||||
if normalize_profile(pipeline or "") == SDXL_ZIMAGE_PROFILE:
|
||||
return _SDXL_ZIMAGE_STRENGTH_BY_VENDOR.get((vendor or "").casefold(), SDXL_ZIMAGE_UNKNOWN_STRENGTH)
|
||||
if size is None:
|
||||
raise ValueError("qwen-zimage resolves strength from image area, so size is required")
|
||||
if (vendor or "").casefold() == "google":
|
||||
return QWEN_ZIMAGE_GOOGLE_STRENGTH
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import resolution_adaptive_denoise
|
||||
|
||||
|
||||
def viable_steps(num_inference_steps: int, strength: float) -> int:
|
||||
"""Ensure Diffusers receives at least one effective img2img denoising step."""
|
||||
if strength <= 0 or int(num_inference_steps * strength) >= 1:
|
||||
return num_inference_steps
|
||||
return math.ceil(1.0 / strength)
|
||||
return resolution_adaptive_denoise(*size)
|
||||
|
||||
|
||||
def vendor_for_strength(image_path: Path) -> Literal["openai", "google"] | None:
|
||||
"""Select the strength cohort using the input's SynthID provenance proxy."""
|
||||
"""Select the strength cohort using the input's SynthID provenance evidence."""
|
||||
try:
|
||||
from remove_ai_watermarks.metadata import synthid_source
|
||||
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false, reportOptionalMemberAccess=false, reportOptionalCall=false, reportOptionalSubscript=false, reportOptionalOperand=false, reportAttributeAccessIssue=false, reportPrivateImportUsage=false, reportPrivateUsage=false, reportInvalidTypeForm=false, reportConstantRedefinition=false, reportUnnecessaryComparison=false
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
@@ -12,32 +11,24 @@ from typing import TYPE_CHECKING, Any
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks._internal.watermark_profiles import (
|
||||
CONTROLNET_CANNY_MODEL,
|
||||
DEFAULT_MODEL_ID,
|
||||
DEFAULT_STRENGTH,
|
||||
QWEN_MODEL_ID,
|
||||
QWEN_ZIMAGE_PROFILE,
|
||||
DEFAULT_PROFILE,
|
||||
INVISIBLE_EXTRA,
|
||||
PROFILE_CHOICES,
|
||||
REMOVAL_MODULES,
|
||||
SDXL_ZIMAGE_PROFILE,
|
||||
normalize_profile,
|
||||
resolve_seed,
|
||||
resolve_steps,
|
||||
resolve_strength,
|
||||
viable_steps,
|
||||
)
|
||||
from remove_ai_watermarks.optional_deps import module_available
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextManifest
|
||||
|
||||
# Both two-stage profiles share the face stage, the four-step schedule, CFG 1.0, the
|
||||
# fixed model stack and the native-resolution contract; only the global model differs.
|
||||
_ZIMAGE_PROFILES = frozenset({QWEN_ZIMAGE_PROFILE, SDXL_ZIMAGE_PROFILE})
|
||||
_ZIMAGE_STACKS = {
|
||||
QWEN_ZIMAGE_PROFILE: "Qwen-Image-2512 and Z-Image",
|
||||
SDXL_ZIMAGE_PROFILE: "SDXL and Z-Image",
|
||||
}
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
try:
|
||||
import torch
|
||||
@@ -47,49 +38,23 @@ except ImportError:
|
||||
torch = None # type: ignore[assignment]
|
||||
_HAS_TORCH = False
|
||||
|
||||
try:
|
||||
from diffusers import AutoPipelineForImage2Image as AutoImg2ImgPipeline
|
||||
|
||||
_HAS_DIFFUSERS = True
|
||||
except ImportError:
|
||||
AutoImg2ImgPipeline = None # type: ignore[assignment,misc]
|
||||
_HAS_DIFFUSERS = False
|
||||
|
||||
_SDXL_FP16_VAE_ID = "madebyollin/sdxl-vae-fp16-fix"
|
||||
_DEGENERATE_THRESHOLD = 1.0
|
||||
_CANNY_LOW = 100
|
||||
_CANNY_HIGH = 200
|
||||
_CONTROLNET_PROMPT = "best quality, high quality, sharp, detailed, photographic"
|
||||
_CONTROLNET_NEGATIVE = "blurry, lowres, deformed, distorted text, garbled text, watermark, jpeg artifacts"
|
||||
_QWEN_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_QWEN_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
# Probed once at import. ``torch`` is imported above rather than probed because this
|
||||
# module needs the object, not just the answer.
|
||||
_HAS_REMOVAL_MODULES = module_available(*(name for name in REMOVAL_MODULES if name != "torch"))
|
||||
|
||||
|
||||
def is_watermark_removal_available() -> bool:
|
||||
"""Return whether the standard diffusion runtime can be imported."""
|
||||
return _HAS_TORCH and _HAS_DIFFUSERS
|
||||
"""Return whether the full removal runtime can be imported."""
|
||||
return _HAS_TORCH and _HAS_REMOVAL_MODULES
|
||||
|
||||
|
||||
def _ensure_watermark_deps() -> None:
|
||||
if not is_watermark_removal_available():
|
||||
raise ImportError(
|
||||
"Invisible watermark regeneration requires the 'diffusion' extra. Install remove-ai-watermarks[diffusion]."
|
||||
f"Invisible watermark regeneration requires the 'qwen-zimage' extra: pip install {INVISIBLE_EXTRA}."
|
||||
)
|
||||
|
||||
|
||||
def _needs_fp16_vae_fix(model_id: str, default_model_id: str, is_fp16: bool) -> bool:
|
||||
"""Return whether the default SDXL pipeline needs the overflow-safe VAE."""
|
||||
return is_fp16 and model_id == default_model_id
|
||||
|
||||
|
||||
def _is_degenerate_image(image: Image.Image) -> bool:
|
||||
"""Detect the uniform near-black output produced by an fp16 decode collapse."""
|
||||
import numpy as np
|
||||
|
||||
pixels = np.asarray(image.convert("RGB"), dtype=np.float32)
|
||||
return float(pixels.mean()) < _DEGENERATE_THRESHOLD and float(pixels.std()) < _DEGENERATE_THRESHOLD
|
||||
|
||||
|
||||
def _has_nvidia_gpu() -> bool:
|
||||
try:
|
||||
subprocess.run(
|
||||
@@ -103,28 +68,9 @@ def _has_nvidia_gpu() -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _detect_cuda_index_url() -> str:
|
||||
"""Return a PyTorch wheel index compatible with the reported CUDA runtime."""
|
||||
def _cuda_works() -> bool:
|
||||
try:
|
||||
report = subprocess.run(
|
||||
["nvidia-smi"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout
|
||||
except (FileNotFoundError, subprocess.CalledProcessError):
|
||||
return "https://download.pytorch.org/whl/cu121"
|
||||
import re
|
||||
|
||||
match = re.search(r"CUDA Version:\s*(\d+)\.(\d+)", report)
|
||||
if match is None:
|
||||
return "https://download.pytorch.org/whl/cu121"
|
||||
return f"https://download.pytorch.org/whl/cu{match.group(1)}{match.group(2)}"
|
||||
|
||||
|
||||
def _backend_works(device: str) -> bool:
|
||||
try:
|
||||
probe = torch.tensor([1.0], device=device) # type: ignore[union-attr]
|
||||
probe = torch.tensor([1.0], device="cuda") # type: ignore[union-attr]
|
||||
_ = probe + probe
|
||||
except (AssertionError, RuntimeError):
|
||||
return False
|
||||
@@ -132,234 +78,75 @@ def _backend_works(device: str) -> bool:
|
||||
|
||||
|
||||
def get_device() -> str:
|
||||
"""Select CUDA, XPU, MPS, or CPU in that order when each backend is usable."""
|
||||
"""Return ``"cuda"`` when a usable CUDA backend is present, else ``"cpu"``.
|
||||
|
||||
Deliberately binary. Both profiles are CUDA-only, so an XPU or MPS answer would
|
||||
only travel one frame further to the same refusal in :class:`WatermarkRemover`,
|
||||
while costing a probe on each. ``"cpu"`` here means "no CUDA", which is exactly
|
||||
what that refusal reports.
|
||||
"""
|
||||
if not _HAS_TORCH:
|
||||
return "cpu"
|
||||
if torch.cuda.is_available() and _backend_works("cuda"): # type: ignore[union-attr]
|
||||
if torch.cuda.is_available() and _cuda_works(): # type: ignore[union-attr]
|
||||
return "cuda"
|
||||
xpu = getattr(torch, "xpu", None)
|
||||
if xpu is not None and xpu.is_available() and _backend_works("xpu"):
|
||||
return "xpu"
|
||||
if _has_nvidia_gpu():
|
||||
logger.warning("NVIDIA GPU detected, but the installed PyTorch build has no working CUDA backend")
|
||||
mps = getattr(getattr(torch, "backends", None), "mps", None)
|
||||
if mps is not None and mps.is_available():
|
||||
return "mps"
|
||||
return "cpu"
|
||||
|
||||
|
||||
def _make_seed_generator(device: str, seed: int) -> Any:
|
||||
"""Create a deterministic generator, using CPU when device RNG is unavailable."""
|
||||
try:
|
||||
return torch.Generator(device=device).manual_seed(seed) # type: ignore[union-attr]
|
||||
except (RuntimeError, TypeError):
|
||||
return torch.Generator().manual_seed(seed) # type: ignore[union-attr]
|
||||
|
||||
|
||||
def _qwen_target_size(width: int, height: int) -> tuple[int, int]:
|
||||
"""Floor dimensions to Qwen's 16-pixel latent grid."""
|
||||
return max(16, width - width % 16), max(16, height - height % 16)
|
||||
|
||||
|
||||
def _build_qwen_kwargs(
|
||||
image: Image.Image,
|
||||
strength: float,
|
||||
num_inference_steps: int,
|
||||
true_cfg_scale: float,
|
||||
generator: Any,
|
||||
) -> dict[str, Any]:
|
||||
"""Build the Qwen img2img call without importing its optional pipeline class."""
|
||||
width, height = _qwen_target_size(image.width, image.height)
|
||||
return {
|
||||
"prompt": _QWEN_PROMPT,
|
||||
"negative_prompt": _QWEN_NEGATIVE,
|
||||
"image": image,
|
||||
"strength": strength,
|
||||
"num_inference_steps": num_inference_steps,
|
||||
"true_cfg_scale": true_cfg_scale,
|
||||
"generator": generator,
|
||||
"width": width,
|
||||
"height": height,
|
||||
}
|
||||
|
||||
|
||||
class WatermarkRemover:
|
||||
"""Load one regeneration profile and write a metadata-clean raster output."""
|
||||
|
||||
DEFAULT_MODEL_ID = DEFAULT_MODEL_ID
|
||||
DEFAULT_STRENGTH = DEFAULT_STRENGTH
|
||||
CONTROLNET_CANNY_MODEL = CONTROLNET_CANNY_MODEL
|
||||
_DEVICES = frozenset({"cpu", "mps", "cuda", "xpu"})
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
model_id: str | None = None,
|
||||
device: str | None = None,
|
||||
torch_dtype: Any = None,
|
||||
progress_callback: Callable[[str], None] | None = None,
|
||||
hf_token: str | None = None,
|
||||
pipeline: str = "controlnet",
|
||||
pipeline: str = DEFAULT_PROFILE,
|
||||
controlnet_conditioning_scale: float = 1.0,
|
||||
cpu_offload: bool = False,
|
||||
) -> None:
|
||||
requested_model = model_id or self.DEFAULT_MODEL_ID
|
||||
self.model_profile = normalize_profile(pipeline)
|
||||
if self.model_profile in _ZIMAGE_PROFILES and model_id not in {None, self.DEFAULT_MODEL_ID}:
|
||||
raise ValueError(
|
||||
f"The {self.model_profile} profile uses a fixed {_ZIMAGE_STACKS[self.model_profile]} model stack."
|
||||
)
|
||||
self.model_id = (
|
||||
"Qwen/Qwen-Image-2512 + Tongyi-MAI/Z-Image-Turbo"
|
||||
if self.model_profile == QWEN_ZIMAGE_PROFILE
|
||||
else requested_model
|
||||
)
|
||||
if self.model_profile not in PROFILE_CHOICES:
|
||||
raise ValueError(f"Unsupported pipeline '{pipeline}'. Use one of: {', '.join(PROFILE_CHOICES)}.")
|
||||
# There is no ``model_id`` parameter and no ``model_id`` attribute: each
|
||||
# profile pins a fixed model stack, and the dtype below is bound to that
|
||||
# stack's weights. Both used to be constructor overrides that existed only to
|
||||
# be rejected or to break the run, and the attribute only existed to echo the
|
||||
# rejected value back.
|
||||
_ensure_watermark_deps()
|
||||
selected_device = (device or get_device()).casefold()
|
||||
self.device = get_device() if selected_device == "auto" else selected_device
|
||||
if self.device not in self._DEVICES:
|
||||
raise ValueError(f"Unsupported device '{device}'. Use one of: auto, cpu, mps, cuda, xpu.")
|
||||
# CUDA is a precondition of the object, not of the run. Both profiles raise on
|
||||
# any other device, so accepting one here only defers a guaranteed failure to
|
||||
# model-load time, several layers down and under the wrong profile's name.
|
||||
if self.device != "cuda":
|
||||
raise ValueError(
|
||||
f"Invisible-watermark removal is CUDA-only, so '{self.device}' cannot run it. "
|
||||
"Both remaining profiles need an NVIDIA GPU. Visible-mark removal and "
|
||||
"every identify command still run on CPU."
|
||||
)
|
||||
|
||||
if torch_dtype is not None:
|
||||
self.torch_dtype = torch_dtype
|
||||
elif self.device in {"cpu", "mps"}:
|
||||
self.torch_dtype = torch.float32 # type: ignore[union-attr]
|
||||
elif self.model_profile == SDXL_ZIMAGE_PROFILE:
|
||||
if self.model_profile == SDXL_ZIMAGE_PROFILE:
|
||||
# SDXL ships fp16 weights and an fp16-safe VAE; bf16 would give up the
|
||||
# variant without buying anything on this architecture.
|
||||
self.torch_dtype = torch.float16 # type: ignore[union-attr]
|
||||
elif self.model_profile in {"qwen", QWEN_ZIMAGE_PROFILE}:
|
||||
self.torch_dtype = torch.bfloat16 # type: ignore[union-attr]
|
||||
else:
|
||||
self.torch_dtype = torch.float16 # type: ignore[union-attr]
|
||||
self.torch_dtype = torch.bfloat16 # type: ignore[union-attr]
|
||||
|
||||
self.cpu_offload = cpu_offload
|
||||
self.controlnet_conditioning_scale = controlnet_conditioning_scale
|
||||
self.hf_token = hf_token or os.environ.get("HF_TOKEN")
|
||||
self._progress_callback = progress_callback
|
||||
self._pipeline: Any = None
|
||||
self._controlnet_pipeline: Any = None
|
||||
self._qwen_pipeline: Any = None
|
||||
self._qwen_zimage_pipeline: Any = None
|
||||
|
||||
def _set_progress(self, message: str) -> None:
|
||||
if self._progress_callback is not None:
|
||||
with contextlib.suppress(Exception):
|
||||
self._progress_callback(message)
|
||||
|
||||
def preload(self, *, global_only: bool = False) -> None:
|
||||
"""Materialize the selected model stack before the first request."""
|
||||
if self.model_profile in _ZIMAGE_PROFILES:
|
||||
self._load_qwen_zimage_pipeline().preload(global_only=global_only)
|
||||
elif self.model_profile == "qwen":
|
||||
self._load_qwen_pipeline()
|
||||
elif self.model_profile == "controlnet":
|
||||
self._load_controlnet_pipeline()
|
||||
else:
|
||||
self._load_pipeline()
|
||||
|
||||
def _base_load_kwargs(self) -> dict[str, Any]:
|
||||
options: dict[str, Any] = {"torch_dtype": self.torch_dtype}
|
||||
if self.hf_token:
|
||||
options["token"] = self.hf_token
|
||||
return options
|
||||
|
||||
def _load_from_pretrained(self, cls: Any, model_id: str, **kwargs: Any) -> Any:
|
||||
if self.torch_dtype == torch.float16: # type: ignore[union-attr]
|
||||
try:
|
||||
return cls.from_pretrained(model_id, variant="fp16", **kwargs)
|
||||
except Exception as error:
|
||||
logger.info("Model %s has no usable fp16 variant (%s); using default weights", model_id, error)
|
||||
return cls.from_pretrained(model_id, **kwargs)
|
||||
|
||||
def _maybe_add_fp16_vae(self, options: dict[str, Any]) -> None:
|
||||
if not _needs_fp16_vae_fix(
|
||||
self.model_id,
|
||||
self.DEFAULT_MODEL_ID,
|
||||
self.torch_dtype == torch.float16, # type: ignore[union-attr]
|
||||
):
|
||||
return
|
||||
from diffusers import AutoencoderKL
|
||||
|
||||
options["vae"] = AutoencoderKL.from_pretrained(_SDXL_FP16_VAE_ID, torch_dtype=torch.float16)
|
||||
|
||||
@staticmethod
|
||||
def _disable_sdxl_watermarker(options: dict[str, Any]) -> None:
|
||||
options["add_watermarker"] = False
|
||||
|
||||
def _move_to_device_and_optimize(self, pipeline: Any) -> Any:
|
||||
if self.cpu_offload and self.device == "cuda":
|
||||
offload = getattr(pipeline, "enable_model_cpu_offload", None)
|
||||
if not callable(offload):
|
||||
raise RuntimeError("CPU offload was requested, but this pipeline does not support it.")
|
||||
offload(device="cuda")
|
||||
else:
|
||||
try:
|
||||
pipeline = pipeline.to(self.device)
|
||||
except (RuntimeError, AssertionError) as error:
|
||||
if self.device == "cuda":
|
||||
raise RuntimeError(
|
||||
f"Failed to move model to CUDA ({error}). Install a compatible PyTorch wheel from "
|
||||
f"{_detect_cuda_index_url()}."
|
||||
) from error
|
||||
raise
|
||||
optimize = getattr(pipeline, "enable_xformers_memory_efficient_attention", None)
|
||||
if callable(optimize):
|
||||
with contextlib.suppress(Exception):
|
||||
optimize()
|
||||
if self.device == "mps":
|
||||
slice_attention = getattr(pipeline, "enable_attention_slicing", None)
|
||||
if callable(slice_attention):
|
||||
with contextlib.suppress(Exception):
|
||||
slice_attention("max")
|
||||
return pipeline
|
||||
|
||||
def _sdxl_options(self) -> dict[str, Any]:
|
||||
options = self._base_load_kwargs()
|
||||
self._disable_sdxl_watermarker(options)
|
||||
self._maybe_add_fp16_vae(options)
|
||||
return options
|
||||
|
||||
def _load_pipeline(self) -> Any:
|
||||
if self._pipeline is None:
|
||||
options = self._sdxl_options()
|
||||
options.update(safety_checker=None, requires_safety_checker=False)
|
||||
loaded = self._load_from_pretrained(AutoImg2ImgPipeline, self.model_id, **options)
|
||||
self._pipeline = self._move_to_device_and_optimize(loaded)
|
||||
return self._pipeline
|
||||
|
||||
def _load_controlnet_pipeline(self) -> Any:
|
||||
if self._controlnet_pipeline is None:
|
||||
from diffusers import ControlNetModel, StableDiffusionXLControlNetImg2ImgPipeline
|
||||
|
||||
controlnet = self._load_from_pretrained(
|
||||
ControlNetModel,
|
||||
CONTROLNET_CANNY_MODEL,
|
||||
torch_dtype=self.torch_dtype,
|
||||
)
|
||||
options = self._sdxl_options()
|
||||
options["controlnet"] = controlnet
|
||||
loaded = self._load_from_pretrained(
|
||||
StableDiffusionXLControlNetImg2ImgPipeline,
|
||||
self.model_id,
|
||||
**options,
|
||||
)
|
||||
self._controlnet_pipeline = self._move_to_device_and_optimize(loaded)
|
||||
return self._controlnet_pipeline
|
||||
|
||||
def _load_qwen_pipeline(self) -> Any:
|
||||
if self._qwen_pipeline is None:
|
||||
try:
|
||||
from diffusers import QwenImageImg2ImgPipeline
|
||||
except ImportError as error:
|
||||
raise ImportError("The qwen profile requires Diffusers with QwenImageImg2ImgPipeline.") from error
|
||||
model_id = QWEN_MODEL_ID if self.model_id == self.DEFAULT_MODEL_ID else self.model_id
|
||||
loaded = QwenImageImg2ImgPipeline.from_pretrained(model_id, **self._base_load_kwargs())
|
||||
self._qwen_pipeline = self._move_to_device_and_optimize(loaded)
|
||||
return self._qwen_pipeline
|
||||
self._load_qwen_zimage_pipeline().preload(global_only=global_only)
|
||||
|
||||
def _load_qwen_zimage_pipeline(self) -> Any:
|
||||
if self._qwen_zimage_pipeline is None:
|
||||
if getattr(self, "model_profile", QWEN_ZIMAGE_PROFILE) == SDXL_ZIMAGE_PROFILE:
|
||||
if self.model_profile == SDXL_ZIMAGE_PROFILE:
|
||||
from remove_ai_watermarks._internal.sdxl_zimage_pipeline import (
|
||||
SdxlZImagePipeline as _Pipeline,
|
||||
)
|
||||
@@ -379,146 +166,6 @@ class WatermarkRemover:
|
||||
)
|
||||
return self._qwen_zimage_pipeline
|
||||
|
||||
def _reload_on_cpu(self, cache_name: str, loader: Callable[[], Any]) -> Any:
|
||||
self.device = "cpu"
|
||||
self.torch_dtype = torch.float32 # type: ignore[union-attr]
|
||||
setattr(self, cache_name, None)
|
||||
return loader()
|
||||
|
||||
def _run_img2img(
|
||||
self,
|
||||
init_image: Image.Image,
|
||||
strength: float,
|
||||
num_inference_steps: int,
|
||||
guidance_scale: float,
|
||||
generator: Any,
|
||||
) -> Image.Image:
|
||||
from remove_ai_watermarks._internal.img2img_runner import run_img2img_with_mps_fallback
|
||||
|
||||
output, device = run_img2img_with_mps_fallback(
|
||||
self._load_pipeline,
|
||||
init_image,
|
||||
strength,
|
||||
num_inference_steps,
|
||||
guidance_scale,
|
||||
generator,
|
||||
self.device,
|
||||
self._set_progress,
|
||||
reload_on_cpu=lambda: self._reload_on_cpu("_pipeline", self._load_pipeline),
|
||||
)
|
||||
self.device = device
|
||||
return output
|
||||
|
||||
def _build_canny_control_image(self, init_image: Image.Image) -> Image.Image:
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
gray = cv2.cvtColor(np.asarray(init_image.convert("RGB")), cv2.COLOR_RGB2GRAY)
|
||||
edges = cv2.Canny(gray, _CANNY_LOW, _CANNY_HIGH)
|
||||
return Image.fromarray(np.repeat(edges[:, :, None], 3, axis=2))
|
||||
|
||||
def _run_controlnet(
|
||||
self,
|
||||
init_image: Image.Image,
|
||||
strength: float,
|
||||
num_inference_steps: int,
|
||||
guidance_scale: float,
|
||||
generator: Any,
|
||||
) -> Image.Image:
|
||||
from remove_ai_watermarks._internal.img2img_runner import run_img2img_with_mps_fallback
|
||||
|
||||
extras = {
|
||||
"prompt": _CONTROLNET_PROMPT,
|
||||
"negative_prompt": _CONTROLNET_NEGATIVE,
|
||||
"control_image": self._build_canny_control_image(init_image),
|
||||
"controlnet_conditioning_scale": float(self.controlnet_conditioning_scale),
|
||||
}
|
||||
output, device = run_img2img_with_mps_fallback(
|
||||
self._load_controlnet_pipeline,
|
||||
init_image,
|
||||
strength,
|
||||
num_inference_steps,
|
||||
guidance_scale,
|
||||
generator,
|
||||
self.device,
|
||||
self._set_progress,
|
||||
reload_on_cpu=lambda: self._reload_on_cpu("_controlnet_pipeline", self._load_controlnet_pipeline),
|
||||
extra_kwargs=extras,
|
||||
)
|
||||
self.device = device
|
||||
return output
|
||||
|
||||
def _run_qwen(
|
||||
self,
|
||||
init_image: Image.Image,
|
||||
strength: float,
|
||||
num_inference_steps: int,
|
||||
guidance_scale: float,
|
||||
generator: Any,
|
||||
) -> Image.Image:
|
||||
response = self._load_qwen_pipeline()(
|
||||
**_build_qwen_kwargs(init_image, strength, num_inference_steps, guidance_scale, generator)
|
||||
)
|
||||
return response.images[0]
|
||||
|
||||
def _run_qwen_zimage(
|
||||
self,
|
||||
init_image: Image.Image,
|
||||
strength: float,
|
||||
seed: int | None,
|
||||
*,
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
) -> Image.Image:
|
||||
return self._load_qwen_zimage_pipeline().run(
|
||||
init_image,
|
||||
strength=strength,
|
||||
seed=seed,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
)
|
||||
|
||||
def _generate(
|
||||
self,
|
||||
image: Image.Image,
|
||||
strength: float,
|
||||
steps: int,
|
||||
guidance: float,
|
||||
generator: Any,
|
||||
seed: int | None,
|
||||
*,
|
||||
tile: bool,
|
||||
tile_size: int,
|
||||
tile_overlap: int,
|
||||
) -> Image.Image:
|
||||
if self.model_profile in _ZIMAGE_PROFILES:
|
||||
return self._run_qwen_zimage(
|
||||
image,
|
||||
strength,
|
||||
seed,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
)
|
||||
|
||||
runner = {
|
||||
"qwen": self._run_qwen,
|
||||
"controlnet": self._run_controlnet,
|
||||
}.get(self.model_profile, self._run_img2img)
|
||||
if tile and max(image.size) > tile_size:
|
||||
from remove_ai_watermarks._internal.tiling import run_tiled
|
||||
|
||||
return run_tiled(
|
||||
lambda crop: runner(crop, strength, steps, guidance, generator),
|
||||
image,
|
||||
tile_size,
|
||||
tile_overlap,
|
||||
self._set_progress,
|
||||
)
|
||||
return runner(image, strength, steps, guidance, generator)
|
||||
|
||||
def _write_output(self, image: Image.Image, output_path: Path) -> None:
|
||||
import numpy as np
|
||||
|
||||
@@ -537,134 +184,43 @@ class WatermarkRemover:
|
||||
image_path: Path,
|
||||
output_path: Path | None = None,
|
||||
strength: float | None = None,
|
||||
num_inference_steps: int | None = None,
|
||||
guidance_scale: float | None = None,
|
||||
seed: int | None = None,
|
||||
vendor: str | None = None,
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
region: tuple[int, int, int, int] | None = None,
|
||||
region_feather: int = 64,
|
||||
text_manifest: VerifiedTextManifest | None = None,
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Path:
|
||||
"""Regenerate image pixels and write the result without AI metadata."""
|
||||
"""Regenerate image pixels and write the result without AI metadata.
|
||||
|
||||
Step count and CFG are not parameters. Each stage of both profiles is a
|
||||
distilled schedule that owns its own, so the only thing a caller-supplied
|
||||
value could do is break the run or be rejected.
|
||||
"""
|
||||
if not image_path.exists():
|
||||
raise FileNotFoundError(f"Image not found: {image_path}")
|
||||
destination = output_path or image_path
|
||||
with Image.open(image_path) as opened:
|
||||
source = opened.convert("RGB")
|
||||
|
||||
if self.model_profile == QWEN_ZIMAGE_PROFILE:
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import resolution_adaptive_denoise
|
||||
|
||||
resolved_strength = strength if strength is not None else resolution_adaptive_denoise(*source.size)
|
||||
else:
|
||||
resolved_strength = resolve_strength(strength, vendor, self.model_profile)
|
||||
resolved_strength = resolve_strength(strength, vendor, self.model_profile, size=source.size)
|
||||
if not 0.0 <= resolved_strength <= 1.0:
|
||||
raise ValueError(f"Strength must be between 0.0 and 1.0, got {resolved_strength}")
|
||||
if text_manifest is not None and self.model_profile == SDXL_ZIMAGE_PROFILE:
|
||||
raise ValueError("Verified text restoration is supported only by the qwen-zimage profile")
|
||||
if text_manifest is not None and tile:
|
||||
raise ValueError("Verified text restoration is not calibrated with tiled diffusion")
|
||||
|
||||
resolved_seed = resolve_seed(seed, self.model_profile)
|
||||
steps = resolve_steps(num_inference_steps, self.model_profile)
|
||||
guidance = 1.0 if guidance_scale is None and self.model_profile in _ZIMAGE_PROFILES else guidance_scale or 7.5
|
||||
if self.model_profile in _ZIMAGE_PROFILES:
|
||||
if steps != 4:
|
||||
raise ValueError(f"The {self.model_profile} profile requires 4 steps.")
|
||||
if guidance != 1.0:
|
||||
raise ValueError(f"The {self.model_profile} profile requires CFG 1.0.")
|
||||
else:
|
||||
steps = viable_steps(steps, resolved_strength)
|
||||
|
||||
generator = None
|
||||
if resolved_seed is not None and _HAS_TORCH:
|
||||
generator = _make_seed_generator(self.device, resolved_seed)
|
||||
result = self._generate(
|
||||
result = self._load_qwen_zimage_pipeline().run(
|
||||
source,
|
||||
resolved_strength,
|
||||
steps,
|
||||
guidance,
|
||||
generator,
|
||||
resolved_seed,
|
||||
strength=resolved_strength,
|
||||
seed=resolve_seed(seed),
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
text_manifest=text_manifest,
|
||||
fidelity_anchor=fidelity_anchor,
|
||||
)
|
||||
|
||||
if self.torch_dtype == torch.float16 and _is_degenerate_image(result): # type: ignore[union-attr]
|
||||
self.torch_dtype = torch.float32 # type: ignore[union-attr]
|
||||
self._pipeline = self._controlnet_pipeline = self._qwen_pipeline = self._qwen_zimage_pipeline = None
|
||||
result = self._generate(
|
||||
source,
|
||||
resolved_strength,
|
||||
steps,
|
||||
guidance,
|
||||
generator,
|
||||
resolved_seed,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
)
|
||||
|
||||
if region is not None:
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks._internal.tiling import feather_region_composite
|
||||
|
||||
if result.size != source.size:
|
||||
result = result.resize(source.size, Image.Resampling.LANCZOS)
|
||||
merged = feather_region_composite(
|
||||
np.asarray(source),
|
||||
np.asarray(result.convert("RGB")),
|
||||
region,
|
||||
feather=region_feather,
|
||||
)
|
||||
result = Image.fromarray(merged)
|
||||
|
||||
self._write_output(result, destination)
|
||||
return destination
|
||||
|
||||
def remove_watermark_batch(
|
||||
self,
|
||||
input_dir: Path,
|
||||
output_dir: Path,
|
||||
strength: float | None = None,
|
||||
num_inference_steps: int | None = None,
|
||||
extensions: tuple[str, ...] = (".png", ".jpg", ".jpeg", ".webp"),
|
||||
) -> list[Path]:
|
||||
"""Process matching files in a directory, logging and continuing on failures."""
|
||||
if not input_dir.exists():
|
||||
raise FileNotFoundError(f"Input directory not found: {input_dir}")
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
from remove_ai_watermarks._internal.img2img_runner import try_empty_device_cache
|
||||
|
||||
outputs: list[Path] = []
|
||||
candidates = sorted(path for path in input_dir.iterdir() if path.suffix.casefold() in extensions)
|
||||
for source in candidates:
|
||||
try:
|
||||
outputs.append(self.remove_watermark(source, output_dir / source.name, strength, num_inference_steps))
|
||||
except Exception as error:
|
||||
logger.error("Failed to process %s: %s", source, error)
|
||||
finally:
|
||||
try_empty_device_cache(self.device)
|
||||
return outputs
|
||||
|
||||
|
||||
def remove_watermark(
|
||||
image_path: Path,
|
||||
output_path: Path | None = None,
|
||||
strength: float | None = None,
|
||||
model_id: str | None = None,
|
||||
device: str | None = None,
|
||||
hf_token: str | None = None,
|
||||
region: tuple[int, int, int, int] | None = None,
|
||||
) -> Path:
|
||||
"""Convenience wrapper using the default ControlNet profile."""
|
||||
from remove_ai_watermarks._internal.watermark_profiles import vendor_for_strength
|
||||
|
||||
remover = WatermarkRemover(model_id=model_id, device=device, hf_token=hf_token)
|
||||
return remover.remove_watermark(
|
||||
image_path,
|
||||
output_path,
|
||||
strength,
|
||||
vendor=vendor_for_strength(image_path),
|
||||
region=region,
|
||||
)
|
||||
@@ -132,7 +132,7 @@ class TextMarkConfig:
|
||||
# Which image dimension the mark's size and margins scale with. VENDOR-SPECIFIC,
|
||||
# measured, not assumed -- see TextMarkEngine.scale_base. "short" = min(h, w), "width" = w.
|
||||
scale_basis: Literal["short", "width"] = "width"
|
||||
# Scale rungs ``_tophat_best`` sweeps (the detection comb). PER-MARK: a vendor
|
||||
# Scale rungs ``_ladder_best`` sweeps (the detection comb). PER-MARK: a vendor
|
||||
# whose stamp sizes do not land on the shared 3-rung comb carries its own ladder
|
||||
# (measured for 千问, whose marks sit in two size modes ~1.6x apart -- one fraction
|
||||
# on 3 rungs covers only ~75% of them). Densifying the SHARED ladder for everyone
|
||||
@@ -170,6 +170,33 @@ class TextMarkDetection:
|
||||
confidence: float = 0.0
|
||||
region: tuple[int, int, int, int] = (0, 0, 0, 0)
|
||||
coverage: float = 0.0 # fraction of the box occupied by glyph pixels
|
||||
# ROI-local (x0, y0, x1, y1) of the ladder sweep's best match, in the LOCATED BOX's
|
||||
# coordinates. None for the ``binary`` front-end (it runs no sweep) and whenever no
|
||||
# rung matched. ``footprint_mask`` bounds the fill with it, so carrying it here is
|
||||
# what stops the mask path re-running a sweep the detector already ran.
|
||||
match_box: tuple[int, int, int, int] | None = None
|
||||
# The trust level this detection was taken at, mirroring detect()'s ``provenance``.
|
||||
# ``footprint_mask`` reuses a threaded detection only when it matches the STRICT
|
||||
# level its own re-detect would have used -- see TextMarkEngine._strict_detection.
|
||||
provenance: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TextMarkScan:
|
||||
"""The trust-level-BLIND half of text-mark detection, reusable across both levels.
|
||||
|
||||
``loc is None`` means detection stopped before any scan (empty or too-small image).
|
||||
``score is None`` means the binary front-end fell below its coverage gate, which is
|
||||
a verdict of "not detected, confidence 0.0" without consulting the rival margin.
|
||||
"""
|
||||
|
||||
loc: TextMarkLocation | None
|
||||
box: NDArray[Any] | None # box-sized binary glyph mask
|
||||
base: int # scale_base(image)
|
||||
frame: tuple[int, int] = (0, 0) # (h, w) of the scanned image
|
||||
coverage: float = 0.0
|
||||
score: float | None = None
|
||||
match_box: tuple[int, int, int, int] | None = None
|
||||
|
||||
|
||||
# Alpha / silhouette templates, cached per asset name. This shared cache lets every
|
||||
@@ -306,6 +333,28 @@ class TextMarkEngine:
|
||||
|
||||
# ── Locate ──────────────────────────────────────────────────────────
|
||||
|
||||
def _roi_fields(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[NDArray[Any], NDArray[Any], NDArray[Any]]:
|
||||
"""``(luma, saturation, local_background)`` for the located box, all float32.
|
||||
|
||||
The ROI is normalized to 3-channel BGR first (grayscale / BGRA would break
|
||||
``axis=2``).
|
||||
|
||||
Local background model: a strong Gaussian blur (sigma ~ box height); the white
|
||||
top-hat (``luma - local_bg``) lights up bright thin strokes regardless of the
|
||||
absolute background level.
|
||||
|
||||
The callers each keep their own ``bh < 16 or bw < 16`` guard: they return three
|
||||
different sentinels for a degenerate ROI, so the check cannot move in here.
|
||||
"""
|
||||
x, y, bw, bh = loc.bbox
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
luma = roi.mean(axis=2)
|
||||
sat = roi.max(axis=2) - roi.min(axis=2)
|
||||
sigma = max(4.0, bh * 0.4) # 0.4 factor and 4.0 floor are calibrated; do not retune
|
||||
return luma, sat, cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma)
|
||||
|
||||
def tophat_response(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any] | None:
|
||||
"""The CONTINUOUS white top-hat in the located box -- the glyph signal, unbinarized.
|
||||
|
||||
@@ -328,26 +377,48 @@ class TextMarkEngine:
|
||||
Kept per-mark (``detect_frontend``) rather than switched globally, because a
|
||||
front-end change must be measured per mark before it ships.
|
||||
"""
|
||||
return self._residual_response(image, loc, absolute=False)
|
||||
|
||||
def _residual_response(self, image: NDArray[Any], loc: TextMarkLocation, *, absolute: bool) -> NDArray[Any] | None:
|
||||
"""Max-normalized uint8 local-luma residual in the located box, saturation-weighted.
|
||||
|
||||
``absolute=False`` keeps only the POSITIVE side -- the white top-hat, for a mark
|
||||
always rendered brighter than its background. ``absolute=True`` takes the
|
||||
magnitude, for a renderer that switches between light-on-dark and dark-on-light
|
||||
while preserving one silhouette; a one-polarity top-hat misses the latter.
|
||||
"""
|
||||
c = self.config
|
||||
x, y, bw, bh = loc.bbox
|
||||
_x, _y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return None
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
luma = roi.mean(axis=2)
|
||||
sat = roi.max(axis=2) - roi.min(axis=2)
|
||||
sigma = max(4.0, bh * 0.4)
|
||||
tophat = luma - cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma)
|
||||
resp = np.clip(tophat, 0, None) * (sat < c.max_saturation)
|
||||
luma, sat, local_bg = self._roi_fields(image, loc)
|
||||
residual = luma - local_bg
|
||||
resp = (np.abs(residual) if absolute else np.clip(residual, 0, None)) * (sat < c.max_saturation)
|
||||
peak = float(resp.max())
|
||||
if peak <= 1e-6:
|
||||
return None
|
||||
return (resp / peak * 255).astype(np.uint8)
|
||||
|
||||
def _tophat_best(
|
||||
def _detect_response(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any] | None:
|
||||
"""The uint8 image the ladder sweep correlates the silhouette against, chosen by
|
||||
``TextMarkConfig.detect_frontend``. ``binary`` runs no sweep and never reaches here."""
|
||||
frontend = self.config.detect_frontend
|
||||
if frontend == "gray":
|
||||
x, y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return None
|
||||
return cv2.cvtColor(image_io.to_bgr(image[y : y + bh, x : x + bw]), cv2.COLOR_BGR2GRAY)
|
||||
if frontend == "contrast":
|
||||
return self._residual_response(image, loc, absolute=True)
|
||||
if frontend == "tophat":
|
||||
return self._residual_response(image, loc, absolute=False)
|
||||
raise ValueError(f"{frontend!r} has no ladder response (binary runs no sweep)")
|
||||
|
||||
def _ladder_best(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Best TM_CCOEFF_NORMED of a soft template against the continuous response, and
|
||||
the ROI-local box (x0, y0, x1, y1) where that best match sits.
|
||||
"""Best TM_CCOEFF_NORMED of the mark's silhouette against its front-end response,
|
||||
and the ROI-local box (x0, y0, x1, y1) where that best match sits.
|
||||
|
||||
Sweeps the mark's scale ladder: the nominal glyph size is derived from the mark's
|
||||
geometry, but a vendor re-rasterization shifts it by a few percent and the
|
||||
@@ -358,10 +429,15 @@ class TextMarkEngine:
|
||||
detection, the box bounds the fill. Sharing it is deliberate: the standing rule is
|
||||
that detection and the mask use the same front-end, and the way that rule was last
|
||||
broken was a drift between two separate implementations. One method makes the drift
|
||||
impossible instead of merely discouraged.
|
||||
impossible instead of merely discouraged -- which is why the three continuous
|
||||
front-ends (tophat / contrast / gray) sweep here rather than in a copy each.
|
||||
|
||||
Tie-breaking is load-bearing: the ``0.0`` seed plus the STRICT ``>`` means the
|
||||
EARLIEST ladder rung wins a tie, and a sweep whose maximum is not above 0.0
|
||||
returns no box at all.
|
||||
"""
|
||||
c = self.config
|
||||
resp = self.tophat_response(image, loc)
|
||||
resp = self._detect_response(image, loc)
|
||||
sil = self._glyph_silhouette()
|
||||
if resp is None or sil is None:
|
||||
return (0.0, None)
|
||||
@@ -373,90 +449,12 @@ class TextMarkEngine:
|
||||
gh = max(4, int(c.alpha_height_frac * base * scale))
|
||||
if gw >= resp.shape[1] or gh >= resp.shape[0]:
|
||||
continue
|
||||
tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32)
|
||||
if c.template_blur > 0:
|
||||
tmpl = cv2.GaussianBlur(tmpl, (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur)
|
||||
result = cv2.matchTemplate(resp, tmpl.astype(np.uint8), cv2.TM_CCOEFF_NORMED)
|
||||
_, score, _, top_left = cv2.minMaxLoc(result)
|
||||
if score > best_score:
|
||||
tx, ty = int(top_left[0]), int(top_left[1])
|
||||
best_score, best_box = float(score), (tx, ty, tx + gw - 1, ty + gh - 1)
|
||||
return (best_score, best_box)
|
||||
|
||||
def _tophat_score(self, image: NDArray[Any], loc: TextMarkLocation) -> float:
|
||||
"""The detection score alone -- the box the removal mask needs is discarded here."""
|
||||
return self._tophat_best(image, loc)[0]
|
||||
|
||||
def _contrast_best(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Best silhouette match against the absolute local-luma residual.
|
||||
|
||||
Unlike the white top-hat, this response is polarity-independent: the same
|
||||
watermark can be lighter or darker than its local background. Detection and
|
||||
removal share the returned box, preserving the front-end parity contract.
|
||||
"""
|
||||
c = self.config
|
||||
x, y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return (0.0, None)
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
luma = roi.mean(axis=2)
|
||||
sat = roi.max(axis=2) - roi.min(axis=2)
|
||||
sigma = max(4.0, bh * 0.4)
|
||||
response = np.abs(luma - cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma))
|
||||
response *= sat < c.max_saturation
|
||||
peak = float(response.max())
|
||||
sil = self._glyph_silhouette()
|
||||
if peak <= 1e-6 or sil is None:
|
||||
return (0.0, None)
|
||||
response = (response / peak * 255).astype(np.uint8)
|
||||
base = self.scale_base(image)
|
||||
best_score = 0.0
|
||||
best_box: tuple[int, int, int, int] | None = None
|
||||
for scale in c.ladder:
|
||||
gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
|
||||
gh = max(4, int(c.alpha_height_frac * base * scale))
|
||||
if gw >= response.shape[1] or gh >= response.shape[0]:
|
||||
continue
|
||||
template = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA)
|
||||
result = cv2.matchTemplate(response, template, cv2.TM_CCOEFF_NORMED)
|
||||
_, score, _, top_left = cv2.minMaxLoc(result)
|
||||
if score > best_score:
|
||||
tx, ty = int(top_left[0]), int(top_left[1])
|
||||
best_score, best_box = float(score), (tx, ty, tx + gw - 1, ty + gh - 1)
|
||||
return (best_score, best_box)
|
||||
|
||||
def _gray_best(self, image: NDArray[Any], loc: TextMarkLocation) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Best TM_CCOEFF_NORMED of the silhouette against the raw GRAYSCALE ROI, and
|
||||
the ROI-local box (x0, y0, x1, y1) of that best match.
|
||||
|
||||
Mirrors :meth:`_tophat_best` (same ladder sweep, same one-method contract so
|
||||
detection and the removal mask can never drift), but skips the top-hat
|
||||
entirely: the RunningHub mark is a faint mid-gray text the top-hat's
|
||||
background subtraction suppresses to clean-arm levels, while raw gray NCC
|
||||
separates (see ``TextMarkConfig.detect_frontend``). Contrast-DEPENDENT by
|
||||
construction, so the gate must be picked against the clean arm, which is
|
||||
what ``scripts/vendor_mark_calibrate.py`` does.
|
||||
"""
|
||||
c = self.config
|
||||
x, y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return (0.0, None)
|
||||
roi = cv2.cvtColor(image_io.to_bgr(image[y : y + bh, x : x + bw]), cv2.COLOR_BGR2GRAY)
|
||||
sil = self._glyph_silhouette()
|
||||
if sil is None:
|
||||
return (0.0, None)
|
||||
base = self.scale_base(image)
|
||||
best_score = 0.0
|
||||
best_box: tuple[int, int, int, int] | None = None
|
||||
for scale in c.ladder:
|
||||
gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
|
||||
gh = max(4, int(c.alpha_height_frac * base * scale))
|
||||
if gw >= roi.shape[1] or gh >= roi.shape[0]:
|
||||
continue
|
||||
tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA)
|
||||
result = cv2.matchTemplate(roi, tmpl, cv2.TM_CCOEFF_NORMED)
|
||||
if c.detect_frontend == "tophat" and c.template_blur > 0:
|
||||
tmpl = cv2.GaussianBlur(
|
||||
tmpl.astype(np.float32), (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur
|
||||
).astype(np.uint8)
|
||||
result = cv2.matchTemplate(resp, tmpl, cv2.TM_CCOEFF_NORMED)
|
||||
_, score, _, top_left = cv2.minMaxLoc(result)
|
||||
if score > best_score:
|
||||
tx, ty = int(top_left[0]), int(top_left[1])
|
||||
@@ -570,10 +568,33 @@ class TextMarkEngine:
|
||||
the mark's own ``provenance_ncc_factor`` to recover a faint or slightly
|
||||
re-rendered mark (per-mark, not shared -- see _DEFAULT_PROVENANCE_NCC_FACTOR).
|
||||
"""
|
||||
scan = self._scan(image)
|
||||
return self._verdict(scan, provenance=provenance)
|
||||
|
||||
def detect_both(self, image: NDArray[Any] | None) -> tuple[TextMarkDetection, TextMarkDetection]:
|
||||
"""``(strict, relaxed)`` from ONE scan of the image.
|
||||
|
||||
``provenance`` scales the acceptance THRESHOLD and nothing else -- the locate
|
||||
box, the glyph mask, the coverage and the front-end ladder score are computed
|
||||
identically at either trust level. Two ``detect`` calls therefore ran the same
|
||||
expensive sweep twice to reach two verdicts, which is what the arbiter's
|
||||
perception pass did for every mark on every image.
|
||||
|
||||
Returns two DISTINCT objects: subclasses demote a verdict by mutating it.
|
||||
"""
|
||||
scan = self._scan(image)
|
||||
return self._verdict(scan, provenance=False), self._verdict(scan, provenance=True)
|
||||
|
||||
def _scan(self, image: NDArray[Any] | None) -> TextMarkScan:
|
||||
"""Everything in detection that does not depend on the trust level.
|
||||
|
||||
Per-CALL only, never memoized on ``self``: ``remove_auto_marks`` re-invokes each
|
||||
engine on a progressively cleaned frame inside one process, so a cached scan
|
||||
would answer for the wrong pixels.
|
||||
"""
|
||||
c = self.config
|
||||
det = TextMarkDetection()
|
||||
if image is None or image.size == 0:
|
||||
return det
|
||||
return TextMarkScan(None, None, 0)
|
||||
# Guard against the small-image NCC-noise false positive (see
|
||||
# _MIN_DETECT_SHORT_SIDE): an icon/thumbnail is too small to carry a real
|
||||
# text label, and the degraded few-pixel template spuriously correlates.
|
||||
@@ -584,57 +605,57 @@ class TextMarkEngine:
|
||||
min(image.shape[:2]),
|
||||
_MIN_DETECT_SHORT_SIDE,
|
||||
)
|
||||
return det
|
||||
return TextMarkScan(None, None, 0)
|
||||
loc = self.locate(image)
|
||||
box = self.extract_mask(image, loc) # box-sized mask (== old full-frame cropped to bbox)
|
||||
_x, _y, bw, bh = loc.bbox
|
||||
coverage = float((box > 0).sum()) / float(max(1, bw * bh))
|
||||
det.region = loc.bbox
|
||||
det.coverage = coverage
|
||||
if c.detect_frontend == "tophat":
|
||||
# The continuous front-end does not depend on the binarized blob, so the
|
||||
# coverage gate (a blob-area heuristic) does not apply to it.
|
||||
score = self._tophat_score(image, loc)
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug("%s detect (tophat): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
|
||||
base = self.scale_base(image)
|
||||
match_box: tuple[int, int, int, int] | None = None
|
||||
if c.detect_frontend == "binary":
|
||||
# The coverage gate is a blob-AREA heuristic, so it applies only to the
|
||||
# front-end that binarizes; the continuous ones never build a blob. Below
|
||||
# the gate the detection stays at confidence 0.0 and the rival margin is
|
||||
# never consulted, so the score stays None here.
|
||||
score = self._template_match_score(box, base) if coverage >= c.detect_min_coverage else None
|
||||
else:
|
||||
score, match_box = self._ladder_best(image, loc)
|
||||
return TextMarkScan(loc, box, base, frame=image.shape[:2], coverage=coverage, score=score, match_box=match_box)
|
||||
|
||||
def _verdict(self, scan: TextMarkScan, *, provenance: bool) -> TextMarkDetection:
|
||||
"""Apply the trust-level-dependent tail to a scan, as a fresh result object."""
|
||||
c = self.config
|
||||
det = TextMarkDetection(provenance=provenance)
|
||||
if scan.loc is None or scan.box is None:
|
||||
return det
|
||||
if c.detect_frontend == "gray":
|
||||
# Same no-coverage-gate reasoning as tophat: the gray front-end never
|
||||
# binarizes, so a blob-area heuristic does not apply to it either.
|
||||
score = self._gray_best(image, loc)[0]
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug("%s detect (gray): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
|
||||
det.region = scan.loc.bbox
|
||||
det.coverage = scan.coverage
|
||||
det.match_box = scan.match_box
|
||||
if scan.score is None: # binary front-end below the coverage gate
|
||||
return det
|
||||
if c.detect_frontend == "contrast":
|
||||
score = self._contrast_best(image, loc)[0]
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug(
|
||||
"%s detect (contrast): ncc=%.2f thr=%.2f detected=%s",
|
||||
c.name,
|
||||
score,
|
||||
threshold,
|
||||
det.detected,
|
||||
)
|
||||
return det
|
||||
if coverage >= c.detect_min_coverage:
|
||||
score = self._template_match_score(box, self.scale_base(image))
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug(
|
||||
"%s detect: coverage=%.3f ncc=%.2f thr=%.2f detected=%s",
|
||||
c.name,
|
||||
coverage,
|
||||
score,
|
||||
threshold,
|
||||
det.detected,
|
||||
)
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = scan.score
|
||||
# Short-circuit is load-bearing: _rival_margin_ok scores every rival template
|
||||
# and logs its own rejection line, so it must stay unevaluated below threshold.
|
||||
det.detected = scan.score >= threshold and self._rival_margin_ok(scan.score, scan.box, scan.base)
|
||||
logger.debug(
|
||||
"%s detect (%s): coverage=%.3f ncc=%.2f thr=%.2f detected=%s",
|
||||
c.name,
|
||||
c.detect_frontend,
|
||||
scan.coverage,
|
||||
scan.score,
|
||||
threshold,
|
||||
det.detected,
|
||||
)
|
||||
return self._post_gate(det, scan)
|
||||
|
||||
def _post_gate(self, det: TextMarkDetection, scan: TextMarkScan) -> TextMarkDetection:
|
||||
"""Per-mark demotion applied after the shared threshold, for both trust levels.
|
||||
|
||||
OVERRIDABLE. It lives here rather than in a ``detect`` override so a mark's gate
|
||||
cannot be silently skipped by the single-pass ``detect_both`` path -- which is
|
||||
exactly what happened while the anchor demotions were ``detect`` overrides.
|
||||
"""
|
||||
return det
|
||||
|
||||
# ── Inpaint footprint (for the inpaint-fallback removal path) ────────
|
||||
@@ -644,8 +665,132 @@ class TextMarkEngine:
|
||||
# to mask. A real strip covers hundreds of pixels.
|
||||
_MIN_GLYPH_PIXELS = 20
|
||||
|
||||
def _strict_detection(self, image: NDArray[Any], detection: TextMarkDetection | None) -> TextMarkDetection:
|
||||
"""The STRICT detection the footprint is bounded by.
|
||||
|
||||
A threaded detection is reused only when it was taken at the same strict level
|
||||
this method would have used itself. A provenance-RELAXED detection is NOT
|
||||
reused: a strict re-detect can demote a mark the relaxed gate accepted, which
|
||||
for a continuous front-end means no mask at all. That is a MEASURED difference,
|
||||
not a refactor, so the strict semantics stay.
|
||||
|
||||
Reuse is safe against ``remove_auto_marks`` chaining marks on a progressively
|
||||
cleaned frame: the registry re-detects on that same cleaned array before
|
||||
threading (``KnownMark.localize``), so a threaded detection is never stale.
|
||||
"""
|
||||
if detection is not None and not detection.provenance:
|
||||
return detection
|
||||
return self.detect(image) # polymorphic: a subclass gate must still apply
|
||||
|
||||
def _geometry_rect(self, loc: TextMarkLocation, frame: tuple[int, int]) -> tuple[int, int, int, int]:
|
||||
"""The whole locate box, clamped to the frame -- the ``force`` footprint."""
|
||||
bx, by, bw, bh = loc.bbox
|
||||
h, w = frame
|
||||
return (bx, by, min(w, bx + bw), min(h, by + bh))
|
||||
|
||||
def _extend_match_box(
|
||||
self, box: tuple[int, int, int, int], loc: TextMarkLocation, frame: tuple[int, int]
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Grow an ROI-local box into the absolute fill rectangle by a symmetric pad.
|
||||
|
||||
OVERRIDABLE, and the override contract is specifically the DETECTOR'S MATCH BOX:
|
||||
a mark whose removable footprint reaches beyond what the NCC localizes -- Baidu's
|
||||
flat white tag right of the text run, LibLibAI's triangle logo left of the
|
||||
wordmark -- supplies its own extension here and inherits the rest of the
|
||||
footprint path. The blob-bbox branch never routes through an override.
|
||||
"""
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
bx, by, _bw, bh = loc.bbox
|
||||
h, w = frame
|
||||
pad = max(4, int(0.10 * bh))
|
||||
return (
|
||||
max(0, bx + gx0 - pad),
|
||||
max(0, by + gy0 - pad),
|
||||
min(w, bx + gx1 + 1 + pad),
|
||||
min(h, by + gy1 + 1 + pad),
|
||||
)
|
||||
|
||||
def _footprint_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Default footprint policy: the binary glyph blob's bbox, else the detector's
|
||||
own match box for the front-ends that under-segment, else the geometry box
|
||||
under ``force``.
|
||||
|
||||
The gray front-end exists for marks the top-hat under-segments, so the binary
|
||||
blob is NOT authoritative there: trusting it first bounded the fill by a PARTIAL
|
||||
blob (the faint head glyphs dropped out) and left the leftmost "Runni" of
|
||||
"RunningHub AI生成" unremoved (2026-07-22).
|
||||
|
||||
A dark-on-light Yuanbao mark has no WHITE top-hat blob at all, so the contrast
|
||||
front-end is bounded by the polarity-independent detector's match box too.
|
||||
|
||||
A tophat mark found only by the CONTINUOUS front-end has no binary glyph blob to
|
||||
bound, so the mask came back empty and removal was a silent no-op while
|
||||
``identify`` still reported the mark. Use the DETECTOR'S OWN best-match box: the
|
||||
correlation already located the mark at a position and scale, and thresholding
|
||||
the response was a strictly worse proxy for that. An earlier fix thresholded the
|
||||
max-normalized uint8 response at 0.5 -- which selects every non-zero pixel, not
|
||||
"half the peak" as its comment claimed -- and filled ~120% of the corner box on
|
||||
textured frames (measured: whole corner vs 58.7% for the match box, both
|
||||
detector-clean). Gated on an actual detection: on a clean corner the box would
|
||||
be spurious.
|
||||
"""
|
||||
ys, xs = np.where(self.extract_mask(image, loc) > 0)
|
||||
blob = (
|
||||
(int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max())) if xs.size >= self._MIN_GLYPH_PIXELS else None
|
||||
)
|
||||
frontend = self.config.detect_frontend
|
||||
if frontend in ("gray", "contrast"):
|
||||
det = self._strict_detection(image, detection)
|
||||
box = det.match_box if det.detected else blob
|
||||
elif blob is not None:
|
||||
box = blob
|
||||
elif frontend == "tophat":
|
||||
det = self._strict_detection(image, detection)
|
||||
box = det.match_box if det.detected else None
|
||||
else:
|
||||
box = None
|
||||
if box is not None:
|
||||
return self._extend_match_box(box, loc, image.shape[:2])
|
||||
return self._geometry_rect(loc, image.shape[:2]) if force else None
|
||||
|
||||
def _match_box_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Footprint policy for a mark whose fill must be bounded by the DETECTOR's match
|
||||
box and never by the binary glyph blob.
|
||||
|
||||
Baidu's white tag has a flat interior a top-hat cannot answer, and LibLibAI's
|
||||
blob bleeds up into background structure; in both cases the blob bbox is
|
||||
measurably wrong and the NCC match box is right. ``force`` takes priority here,
|
||||
unlike the default policy: a ``--no-detect`` caller named the mark, so the whole
|
||||
geometry box is the honest footprint.
|
||||
"""
|
||||
if force:
|
||||
return self._geometry_rect(loc, image.shape[:2])
|
||||
det = self._strict_detection(image, detection)
|
||||
if not det.detected or det.match_box is None:
|
||||
return None
|
||||
return self._extend_match_box(det.match_box, loc, image.shape[:2])
|
||||
|
||||
def footprint_mask(
|
||||
self, image: NDArray[Any], *, force: bool = False, dilate: int | None = None
|
||||
self,
|
||||
image: NDArray[Any] | None,
|
||||
*,
|
||||
force: bool = False,
|
||||
dilate: int | None = None,
|
||||
detection: TextMarkDetection | None = None,
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame uint8 mask (255 = mark) of the mark footprint, for the shared
|
||||
fill removal path (cv2 / MI-GAN / LaMa), or None if no glyph is found.
|
||||
@@ -662,6 +807,10 @@ class TextMarkEngine:
|
||||
|
||||
With ``force`` and no glyph found, falls back to the whole geometry box (the
|
||||
``--no-detect`` path). The caller gates on detection.
|
||||
|
||||
``detection`` is the caller's already-computed detection, threaded in so the
|
||||
footprint does not re-run a sweep the detector already ran. See
|
||||
:meth:`_strict_detection` for when it is reused.
|
||||
"""
|
||||
if image is None or image.size == 0:
|
||||
return None # guard before to_bgr (cvtColor raises on an empty Mat); mirror detect()
|
||||
@@ -670,46 +819,10 @@ class TextMarkEngine:
|
||||
if h < 32 or w < 64:
|
||||
return None
|
||||
loc = self.locate(image)
|
||||
bx, by, bw, bh = loc.bbox
|
||||
glyph = self.extract_mask(image, loc) # box-sized, 255 = glyph
|
||||
ys, xs = np.where(glyph > 0)
|
||||
box: tuple[int, int, int, int] | None = None
|
||||
if self.config.detect_frontend == "gray" and self.detect(image).detected:
|
||||
# The gray front-end exists for marks the top-hat under-segments, so the
|
||||
# binary blob is NOT authoritative here: trusting it first bounded the
|
||||
# fill by a PARTIAL blob (the faint head glyphs dropped out) and left the
|
||||
# leftmost "Runni" of "RunningHub AI生成" unremoved (2026-07-22). Use the
|
||||
# detector's own best-match box, same as the tophat faint path below.
|
||||
_, box = self._gray_best(image, loc)
|
||||
elif self.config.detect_frontend == "contrast" and self.detect(image).detected:
|
||||
# A dark-on-light Yuanbao mark has no WHITE top-hat blob at all. Bound
|
||||
# the fill by the polarity-independent detector's own match box.
|
||||
_, box = self._contrast_best(image, loc)
|
||||
elif xs.size >= self._MIN_GLYPH_PIXELS:
|
||||
box = (int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max()))
|
||||
elif self.config.detect_frontend == "tophat" and self.detect(image).detected:
|
||||
# A mark found only by the CONTINUOUS front-end has no binary glyph blob to
|
||||
# bound, so the mask came back empty and removal was a silent no-op while
|
||||
# `identify` still reported the mark while removal left it untouched.
|
||||
# Use the DETECTOR'S OWN best-match box: the correlation already located the
|
||||
# mark at a position and scale, and thresholding the response was a strictly
|
||||
# worse proxy for that. An earlier fix thresholded the max-normalized uint8
|
||||
# response at 0.5 -- which selects every non-zero pixel, not "half the peak" as
|
||||
# its comment claimed -- and filled ~120% of the corner box on textured frames
|
||||
# (measured: whole corner vs 58.7% for the match box, both detector-clean).
|
||||
# Gated on an actual detection: on a clean corner the box would be spurious.
|
||||
_, box = self._tophat_best(image, loc)
|
||||
if box is not None:
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
pad = max(4, int(0.10 * bh))
|
||||
rx1 = max(0, bx + gx0 - pad)
|
||||
rx2 = min(w, bx + gx1 + 1 + pad)
|
||||
ry1 = max(0, by + gy0 - pad)
|
||||
ry2 = min(h, by + gy1 + 1 + pad)
|
||||
elif force:
|
||||
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
|
||||
else:
|
||||
rect = self._footprint_rect(image, loc, force=force, detection=detection)
|
||||
if rect is None:
|
||||
return None
|
||||
rx1, ry1, rx2, ry2 = rect
|
||||
if rx1 >= rx2 or ry1 >= ry2:
|
||||
return None
|
||||
# Rectangular footprint + dilation is exactly region_eraser.boxes_to_mask (the
|
||||
@@ -717,5 +830,5 @@ class TextMarkEngine:
|
||||
# zeros/fill/MORPH_ELLIPSE-dilate here.
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * bw))
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * loc.w))
|
||||
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
|
||||
+518
-13
@@ -16,11 +16,14 @@ Imports stay lazy (inside the functions), so ``import remove_ai_watermarks`` is
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import suppress
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import TYPE_CHECKING, Any, Literal
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
from remove_ai_watermarks.watermark_registry import Backend, Sensitivity
|
||||
@@ -40,8 +43,17 @@ def visible_provenance(source: str | Path) -> frozenset[str]:
|
||||
"""Vendor keys that the file's local metadata confirms, the evidence that drives
|
||||
the ``auto`` sensitivity (relaxing a corroborated mark's detection trust gate).
|
||||
|
||||
Mapping: a Google/Gemini C2PA issuer -> ``"gemini"``; a China-AIGC (TC260) label
|
||||
-> ``"doubao"``/``"jimeng"``; a ``samsung_genai`` marker -> ``"samsung"``.
|
||||
Mapping: a Google/Gemini C2PA issuer -> ``"gemini"``; a ``samsung_genai`` marker ->
|
||||
``"samsung"``; a China-AIGC (TC260) label -> the vendor its ``ContentProducer``
|
||||
names (``KnownMark.tc260_producer_codes``), falling back to ByteDance's two products
|
||||
when the producer is absent or unmapped.
|
||||
|
||||
The TC260 label itself says only "this is AI", so it used to relax Doubao and
|
||||
Jimeng on EVERY China-AIGC image -- including one carrying a Qwen or Kling mark,
|
||||
where relaxing the wrong pair is pure false-positive risk and the mark actually
|
||||
present never reached the relaxed gate its own ``provenance_ncc_factor`` was
|
||||
calibrated for. The producer code identifies the signing entity, so it can.
|
||||
|
||||
Best-effort: any read error yields an empty set (no relaxation). Metadata-only, so
|
||||
it never loads cv2/torch.
|
||||
"""
|
||||
@@ -52,19 +64,33 @@ def visible_provenance(source: str | Path) -> frozenset[str]:
|
||||
from remove_ai_watermarks import identify
|
||||
|
||||
rep = identify.identify(path, check_visible=False, check_invisible=False)
|
||||
signal_names = {signal.name for signal in rep.signals}
|
||||
keys: set[str] = set()
|
||||
platform = (rep.platform or "").lower()
|
||||
if "google" in platform or "gemini" in platform:
|
||||
keys.add("gemini")
|
||||
if "aigc" in signal_names:
|
||||
keys |= {"doubao", "jimeng"}
|
||||
if "samsung_genai" in signal_names:
|
||||
keys.add("samsung")
|
||||
return frozenset(keys)
|
||||
return _provenance_from_report(rep, path)
|
||||
return frozenset()
|
||||
|
||||
|
||||
def _tc260_vendors(path: Path) -> frozenset[str]:
|
||||
"""Vendor keys a TC260 label confirms, from its ``ContentProducer`` identity.
|
||||
|
||||
An absent, unreadable or unmapped producer falls back to the historical pair rather
|
||||
than to nothing: the caller has already established that the AIGC signal fired, so
|
||||
the image IS China-AIGC labelled, and dropping to no relaxation would lose the
|
||||
detections the fallback recovers today. The re-read is deliberately isolated -- a
|
||||
failure here must narrow the answer, never discard the rest of the provenance.
|
||||
"""
|
||||
import contextlib
|
||||
|
||||
from remove_ai_watermarks._internal.constants import TC260_FALLBACK_VENDORS
|
||||
|
||||
with contextlib.suppress(Exception):
|
||||
from remove_ai_watermarks.metadata import aigc_label, uscc_of
|
||||
from remove_ai_watermarks.watermark_registry import tc260_producer_vendors
|
||||
|
||||
producer = (aigc_label(path) or {}).get("ContentProducer", "")
|
||||
if producer and (vendor := tc260_producer_vendors().get(uscc_of(producer))):
|
||||
return frozenset({vendor})
|
||||
return TC260_FALLBACK_VENDORS
|
||||
|
||||
|
||||
def _load_visible_input(source: str | Path | NDArray[Any]) -> _VisibleInput:
|
||||
"""Normalize a path/array source without making the public operation stateful."""
|
||||
if not isinstance(source, (str, Path)):
|
||||
@@ -177,3 +203,482 @@ def remove_visible(
|
||||
write_noop=write_noop,
|
||||
)
|
||||
return result, removed
|
||||
|
||||
|
||||
# ── The three-stage image pipeline (visible -> invisible -> metadata) ──
|
||||
# This is the `all` / `batch` pipeline. It lived only in cli.py, written twice with
|
||||
# divergences, so a library caller could not run the flagship path at all. The CLI is
|
||||
# now a thin wrapper: it builds the options, prints the stage lines through
|
||||
# `progress`, and turns the outcome into console text and an exit code.
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class InvisibleOptions:
|
||||
"""The invisible stage's knobs, as one value instead of a dozen parameters.
|
||||
|
||||
ENGINE KNOBS ONLY, under the engine's own names and defaults, so a bare
|
||||
``InvisibleOptions()`` behaves exactly like calling the engine with no arguments.
|
||||
The engine takes them across TWO callables -- ``__init__`` for the ones that shape
|
||||
the loaded stack, ``remove_watermark`` for the per-image ones -- so this is not a
|
||||
splat-through bag; ``_run_invisible`` forwards each field to the right one.
|
||||
``TestInvisibleOptionsMirrorTheEngine`` compares the signatures field by field with
|
||||
no exception table to maintain: a decision made before the engine runs, like
|
||||
``force``, is a parameter of ``remove_all`` next to ``backend`` and ``sensitivity``,
|
||||
not a knob smuggled in here.
|
||||
|
||||
Immutable so a batch can build it once and reuse it across every image while the
|
||||
engine itself is cached separately.
|
||||
"""
|
||||
|
||||
strength: float | None = None
|
||||
pipeline: str = "qwen-zimage"
|
||||
seed: int | None = None
|
||||
hf_token: str | None = None
|
||||
humanize: float = 0.0
|
||||
unsharp: float = 0.0
|
||||
adaptive_polish: bool | None = None
|
||||
max_resolution: int = 0
|
||||
controlnet_conditioning_scale: float = 1.0
|
||||
cpu_offload: bool = False
|
||||
tile: bool = False
|
||||
tile_size: int = 1024
|
||||
tile_overlap: int = 128
|
||||
text_manifest: Path | None = None
|
||||
fidelity_anchor: bool = False
|
||||
|
||||
|
||||
# What the invisible stage did. "unavailable" is the one outcome the caller must
|
||||
# surface loudly: the file looks processed but still carries the watermark.
|
||||
InvisibleOutcome = Literal["removed", "no-signal", "unavailable"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RemoveAllResult:
|
||||
"""Outcome of :func:`remove_all` -- what each stage actually did."""
|
||||
|
||||
output: Path
|
||||
visible_label: str | None # the removed mark(s), or None when nothing fired
|
||||
invisible: InvisibleOutcome
|
||||
|
||||
|
||||
class MetadataStripIncomplete(RuntimeError):
|
||||
"""AI metadata survived the strip, so no output was produced.
|
||||
|
||||
Raised BEFORE the final write, deliberately: the contract is that an incomplete
|
||||
strip leaves nothing on disk rather than an AI-readable file the caller might ship.
|
||||
"""
|
||||
|
||||
def __init__(self, surviving: set[str]) -> None:
|
||||
self.surviving = surviving
|
||||
super().__init__(f"AI metadata survived the strip: {', '.join(sorted(surviving))}")
|
||||
|
||||
|
||||
class _SourceEvidence:
|
||||
"""One metadata extraction per source file, serving every stage of one call.
|
||||
|
||||
``remove_all`` asks the same file two provenance questions -- which vendor the
|
||||
metadata confirms (for the visible pass) and whether an invisible target exists
|
||||
(for the scrub gate). Both start from the same file-backed extraction, and running
|
||||
them independently paid for it twice.
|
||||
|
||||
Per-CALL only, never module-level: the pipeline REWRITES its input in place on some
|
||||
paths (``batch`` with the output directory equal to the input), so a holder that
|
||||
outlived one call would answer from pre-write evidence. The individual metadata
|
||||
probes are memoized on content, which is what makes that safe -- this holder only
|
||||
removes the remaining assembly work.
|
||||
|
||||
Every accessor fails safe exactly as the function it replaces does: an extraction or
|
||||
verdict error yields no provenance (no relaxation) and an invisible target of True
|
||||
(scrub rather than skip).
|
||||
"""
|
||||
|
||||
__slots__ = ("_evidence", "_extracted", "_path")
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
self._path = path
|
||||
self._extracted = False
|
||||
self._evidence: Any | None = None
|
||||
|
||||
def _extract(self) -> Any | None:
|
||||
if not self._extracted:
|
||||
self._extracted = True
|
||||
with suppress(Exception):
|
||||
from remove_ai_watermarks.identify import extract_provenance_evidence
|
||||
|
||||
self._evidence = extract_provenance_evidence(self._path)
|
||||
return self._evidence
|
||||
|
||||
def visible_provenance(self) -> frozenset[str]:
|
||||
"""Vendor keys the metadata confirms; empty on any failure (no relaxation)."""
|
||||
evidence = self._extract()
|
||||
if evidence is None:
|
||||
return frozenset()
|
||||
# The suppress spans the VERDICT and the mapping, not just the extraction --
|
||||
# `visible_provenance` fails safe as a whole, and a raise here would escape as a
|
||||
# traceback where the old code returned an empty set.
|
||||
with suppress(Exception):
|
||||
from remove_ai_watermarks.identify import identify_from_evidence
|
||||
|
||||
report = identify_from_evidence(evidence)
|
||||
return _provenance_from_report(report, self._path)
|
||||
return frozenset()
|
||||
|
||||
def has_invisible_target(self) -> bool:
|
||||
"""Whether a diffusion target exists. True on any failure -- see the fail-safe
|
||||
note on ``identify.has_invisible_target``: leaving a watermark on a paid removal
|
||||
is worse than over-regenerating a clean image."""
|
||||
evidence = self._extract()
|
||||
if evidence is None:
|
||||
return True
|
||||
try:
|
||||
from remove_ai_watermarks._internal.c2pa import c2pa_info_has_removal_hint
|
||||
from remove_ai_watermarks.identify import identify_from_evidence
|
||||
|
||||
report = identify_from_evidence(evidence, image_path=self._path, check_invisible=True)
|
||||
except Exception:
|
||||
return True
|
||||
return report.ai_from_metadata or c2pa_info_has_removal_hint(evidence.c2pa_info)
|
||||
|
||||
|
||||
def _provenance_from_report(report: Any, path: Path) -> frozenset[str]:
|
||||
"""Map an already-built report to the vendor keys it confirms.
|
||||
|
||||
Shared by :func:`visible_provenance` and the evidence holder so the two cannot
|
||||
drift; which evidence confirms which mark is data on the registry row.
|
||||
"""
|
||||
from remove_ai_watermarks.watermark_registry import known_marks
|
||||
|
||||
signal_names = {signal.name for signal in report.signals}
|
||||
platform = (report.platform or "").lower()
|
||||
keys: set[str] = set()
|
||||
aigc = False
|
||||
for mark in known_marks():
|
||||
if any(token in platform for token in mark.provenance_platform_tokens):
|
||||
keys.add(mark.key)
|
||||
if "aigc" in mark.provenance_signals and "aigc" in signal_names:
|
||||
aigc = True
|
||||
elif any(name in signal_names for name in mark.provenance_signals):
|
||||
keys.add(mark.key)
|
||||
if aigc:
|
||||
# The TC260 label is vendor-agnostic: which of its marks it confirms comes from
|
||||
# the producer identity, not from the signal firing.
|
||||
keys |= _tc260_vendors(path)
|
||||
return frozenset(keys)
|
||||
|
||||
|
||||
def remove_all(
|
||||
source: str | Path,
|
||||
output: str | Path,
|
||||
*,
|
||||
backend: Backend = "auto",
|
||||
sensitivity: Sensitivity = "auto",
|
||||
invisible: InvisibleOptions | None = None,
|
||||
force: bool = False,
|
||||
engine: Any | None = None,
|
||||
progress: Callable[[str, str], None] | None = None,
|
||||
) -> RemoveAllResult:
|
||||
"""Remove visible marks, the invisible watermark, and AI metadata, in that order.
|
||||
|
||||
Stages are chained through a file in the SYSTEM temp dir, not next to ``output``:
|
||||
the point of staging is that the user never sees a partial output file during a long
|
||||
model download, and writing the partial next to the final defeats that.
|
||||
|
||||
``force`` scrubs even when no invisible watermark is locally detectable. It sits
|
||||
here rather than in ``InvisibleOptions`` because it decides WHETHER the engine runs,
|
||||
which is settled before the engine is built; the options carry only what the engine
|
||||
itself takes.
|
||||
|
||||
``engine`` accepts an already-constructed ``InvisibleEngine`` so a batch can build
|
||||
the model once; leave it None to construct one per call.
|
||||
|
||||
``progress`` receives ``(stage, detail)`` per step -- ``stage`` is one of
|
||||
``visible`` / ``invisible`` / ``metadata`` and ``detail`` is a stable token, not
|
||||
prose: the caller owns the wording. The invisible stage reports its
|
||||
:data:`InvisibleOutcome`, plus a ``strength=<value>`` line before it runs.
|
||||
|
||||
Raises :class:`MetadataStripIncomplete` before writing anything when AI metadata
|
||||
survives, and ``OSError`` when the output cannot be written.
|
||||
"""
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
from remove_ai_watermarks import image_io, watermark_registry
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
|
||||
def say(stage: str, detail: str) -> None:
|
||||
if progress is not None:
|
||||
progress(stage, detail)
|
||||
|
||||
opts = invisible if invisible is not None else InvisibleOptions()
|
||||
src, out = Path(source), Path(output)
|
||||
watermark_registry.validate_sensitivity(sensitivity)
|
||||
|
||||
image, alpha = image_io.read_bgr_and_alpha(src)
|
||||
if image is None:
|
||||
raise ValueError(f"Could not read image: {src}")
|
||||
|
||||
# One metadata extraction for the whole pipeline: the visible pass asks which
|
||||
# vendor is confirmed and the scrub gate asks whether a target exists, and both
|
||||
# start from the same evidence.
|
||||
evidence = _SourceEvidence(src)
|
||||
|
||||
tmp_fd, tmp_name = tempfile.mkstemp(suffix=src.suffix)
|
||||
os.close(tmp_fd)
|
||||
staged = Path(tmp_name)
|
||||
try:
|
||||
# ── 1. Visible marks ──
|
||||
result, removed = watermark_registry.remove_auto_marks(
|
||||
image,
|
||||
sensitivity=sensitivity,
|
||||
provenance=evidence.visible_provenance(),
|
||||
backend=backend,
|
||||
)
|
||||
visible_label = ", ".join(removed) if removed else None
|
||||
say("visible", visible_label or "")
|
||||
if not image_io.write_bgr_with_alpha(staged, result, alpha):
|
||||
raise OSError(f"failed to write the staged intermediate: {staged}")
|
||||
|
||||
# ── 2. Invisible watermark ──
|
||||
outcome = _run_invisible(src, staged, staged, opts, engine, say, evidence, force)
|
||||
|
||||
# ── 3. AI metadata ──
|
||||
# Read the pristine ORIGINAL for provenance above and the STAGED file here:
|
||||
# the visible pass has already dropped this file's C2PA.
|
||||
_, leftover = strip_and_verify(staged, staged)
|
||||
if leftover:
|
||||
# Before the write, on purpose -- see MetadataStripIncomplete.
|
||||
raise MetadataStripIncomplete(set(leftover))
|
||||
say("metadata", "stripped")
|
||||
|
||||
# The invisible stage (and the cv2.IMREAD_COLOR paths under it) drops alpha, so
|
||||
# re-attach the ORIGINAL alpha plane unchanged for transparent formats.
|
||||
final_bgr, _ = image_io.read_bgr_and_alpha(staged)
|
||||
if final_bgr is None:
|
||||
raise OSError(f"failed to read back the staged intermediate: {staged}")
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not image_io.write_bgr_with_alpha(out, final_bgr, alpha):
|
||||
raise OSError(f"failed to write output (is the destination writable?): {out}")
|
||||
finally:
|
||||
if staged.exists():
|
||||
staged.unlink()
|
||||
return RemoveAllResult(out, visible_label, outcome)
|
||||
|
||||
|
||||
def _run_invisible(
|
||||
vendor_source: Path,
|
||||
in_path: Path,
|
||||
out_path: Path,
|
||||
opts: InvisibleOptions,
|
||||
engine: Any | None,
|
||||
say: Callable[[str, str], None],
|
||||
evidence: _SourceEvidence,
|
||||
force: bool,
|
||||
) -> InvisibleOutcome:
|
||||
"""Run, or deliberately skip, the diffusion scrub.
|
||||
|
||||
``vendor_source`` is the PRISTINE original: the staged/output file has already lost
|
||||
its C2PA to the visible pass, so gating or resolving the vendor from it would always
|
||||
read as "no signal" and "unknown vendor". ``in_path``/``out_path`` are what the
|
||||
engine reads and writes (the same staged file for ``remove_all``, input->output for
|
||||
an invisible-only batch).
|
||||
"""
|
||||
from remove_ai_watermarks.invisible_engine import is_available
|
||||
|
||||
if not is_available():
|
||||
say("invisible", "unavailable")
|
||||
return "unavailable"
|
||||
if not (force or evidence.has_invisible_target()):
|
||||
say("invisible", "no-signal")
|
||||
return "no-signal"
|
||||
|
||||
from remove_ai_watermarks._internal.watermark_profiles import resolve_strength, vendor_for_strength
|
||||
|
||||
vendor = vendor_for_strength(vendor_source)
|
||||
# Report the strength the engine will actually execute, resolved the same way it
|
||||
# resolves it, so the reported value cannot drift from the executed one.
|
||||
with suppress(Exception):
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(vendor_source) as probe:
|
||||
say("invisible", f"strength={resolve_strength(opts.strength, vendor, opts.pipeline, size=probe.size)}")
|
||||
|
||||
if engine is None:
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine
|
||||
|
||||
engine = InvisibleEngine(
|
||||
pipeline=opts.pipeline,
|
||||
hf_token=opts.hf_token,
|
||||
progress_callback=lambda message: say("invisible", message),
|
||||
controlnet_conditioning_scale=opts.controlnet_conditioning_scale,
|
||||
cpu_offload=opts.cpu_offload,
|
||||
)
|
||||
engine.remove_watermark(
|
||||
image_path=in_path,
|
||||
output_path=out_path,
|
||||
strength=opts.strength,
|
||||
seed=opts.seed,
|
||||
humanize=opts.humanize,
|
||||
unsharp=opts.unsharp,
|
||||
adaptive_polish=opts.adaptive_polish,
|
||||
max_resolution=opts.max_resolution,
|
||||
vendor=vendor,
|
||||
tile=opts.tile,
|
||||
tile_size=opts.tile_size,
|
||||
tile_overlap=opts.tile_overlap,
|
||||
text_manifest=opts.text_manifest,
|
||||
fidelity_anchor=opts.fidelity_anchor,
|
||||
)
|
||||
say("invisible", "removed")
|
||||
return "removed"
|
||||
|
||||
|
||||
BatchMode = Literal["all", "visible", "metadata", "invisible"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BatchSummary:
|
||||
"""Per-directory outcome of :func:`remove_batch`."""
|
||||
|
||||
processed: int
|
||||
failed: int
|
||||
# Files whose invisible watermark was left in place because the GPU extra is
|
||||
# missing. Non-empty means the outputs LOOK processed but still carry it.
|
||||
invisible_unavailable: list[Path]
|
||||
errors: list[tuple[Path, str]]
|
||||
|
||||
|
||||
def remove_batch(
|
||||
directory: str | Path,
|
||||
output_dir: str | Path,
|
||||
*,
|
||||
mode: BatchMode = "all",
|
||||
backend: Backend = "auto",
|
||||
sensitivity: Sensitivity = "auto",
|
||||
invisible: InvisibleOptions | None = None,
|
||||
force: bool = False,
|
||||
engine: Any | None = None,
|
||||
progress: Callable[[Path, str, str], None] | None = None,
|
||||
) -> BatchSummary:
|
||||
"""Run one removal ``mode`` over every supported image in ``directory``.
|
||||
|
||||
Never raises for a single bad image: a per-file failure is counted and recorded in
|
||||
``BatchSummary.errors`` so one unreadable file cannot abandon the rest of the
|
||||
directory. ``force`` and ``engine`` are threaded straight through, so a caller that
|
||||
passes a constructed ``InvisibleEngine`` loads the model once for the whole run.
|
||||
|
||||
``progress`` receives ``(path, stage, detail)``. Every image ends with exactly one
|
||||
terminal stage -- ``done`` or ``failed`` -- whatever the mode does in between, so a
|
||||
caller driving a progress bar can advance on that alone.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.utils import is_supported_format
|
||||
|
||||
src_dir, out_dir = Path(directory), Path(output_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def say(path: Path, stage: str, detail: str) -> None:
|
||||
if progress is not None:
|
||||
progress(path, stage, detail)
|
||||
|
||||
processed = failed = 0
|
||||
unavailable: list[Path] = []
|
||||
errors: list[tuple[Path, str]] = []
|
||||
for img_path in sorted(p for p in src_dir.iterdir() if is_supported_format(p)):
|
||||
out_path = out_dir / img_path.name
|
||||
try:
|
||||
outcome = _run_batch_one(img_path, out_path, mode, backend, sensitivity, invisible, force, engine, say)
|
||||
except Exception as exc:
|
||||
failed += 1
|
||||
errors.append((img_path, str(exc)))
|
||||
say(img_path, "failed", str(exc))
|
||||
continue
|
||||
processed += 1
|
||||
if outcome == "unavailable":
|
||||
unavailable.append(img_path)
|
||||
# Exactly one terminal event per image, in every mode: a caller's progress bar
|
||||
# advances on this and nothing else. Keying it off a mode-specific stage line
|
||||
# left `visible` and `metadata` runs sitting at 0% for the whole batch.
|
||||
say(img_path, "done", outcome or "")
|
||||
return BatchSummary(processed, failed, unavailable, errors)
|
||||
|
||||
|
||||
def _run_batch_one(
|
||||
img_path: Path,
|
||||
out_path: Path,
|
||||
mode: BatchMode,
|
||||
backend: Backend,
|
||||
sensitivity: Sensitivity,
|
||||
invisible: InvisibleOptions | None,
|
||||
force: bool,
|
||||
engine: Any | None,
|
||||
say: Callable[[Path, str, str], None],
|
||||
) -> InvisibleOutcome | None:
|
||||
"""One image, one mode. Returns the invisible outcome when that stage ran."""
|
||||
from remove_ai_watermarks import image_io
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
|
||||
if mode == "all":
|
||||
result = remove_all(
|
||||
img_path,
|
||||
out_path,
|
||||
backend=backend,
|
||||
sensitivity=sensitivity,
|
||||
invisible=invisible,
|
||||
force=force,
|
||||
engine=engine,
|
||||
progress=lambda stage, detail: say(img_path, stage, detail),
|
||||
)
|
||||
return result.invisible
|
||||
|
||||
# NOTE on holders in batch: `out_path` may EQUAL `img_path` (nothing forbids
|
||||
# `-o <the input directory>`), and the visible stage rewrites it. So each stage that
|
||||
# asks the file a provenance question builds its OWN holder, after any write that
|
||||
# precedes it. One holder spanning the write would answer the invisible gate from
|
||||
# pre-write evidence and scrub a file that today is correctly skipped.
|
||||
if mode == "visible":
|
||||
# Deliberately NOT `remove_visible`: its no-op branch copies the original bytes
|
||||
# through when nothing was removed, which is right for a single lossless call
|
||||
# and wrong here. A batch must produce every output through the one writer, so a
|
||||
# failed write RAISES and the run is counted and exits non-zero -- a read-only
|
||||
# output directory once produced zero files and still exited 0 (Tier E).
|
||||
# Always read the ORIGINAL: a stale out_path from a previous run must not be
|
||||
# re-processed as if it were the input.
|
||||
from remove_ai_watermarks import watermark_registry
|
||||
|
||||
image, alpha = image_io.read_bgr_and_alpha(img_path)
|
||||
if image is None:
|
||||
raise ValueError(f"Could not read image: {img_path}")
|
||||
result, _ = watermark_registry.remove_auto_marks(
|
||||
image,
|
||||
sensitivity=sensitivity,
|
||||
provenance=visible_provenance(img_path),
|
||||
backend=backend,
|
||||
)
|
||||
if not image_io.write_bgr_with_alpha(out_path, result, alpha):
|
||||
raise OSError(f"failed to write output (is the destination writable?): {out_path}")
|
||||
return None
|
||||
|
||||
if mode == "metadata":
|
||||
_, leftover = strip_and_verify(img_path, out_path)
|
||||
if leftover:
|
||||
raise MetadataStripIncomplete(set(leftover))
|
||||
return None
|
||||
|
||||
# invisible-only: no preceding visible pass, so out_path does not exist yet, and
|
||||
# the input IS the pristine original for both the gate and the vendor probe.
|
||||
outcome = _run_invisible(
|
||||
img_path,
|
||||
img_path,
|
||||
out_path,
|
||||
invisible if invisible is not None else InvisibleOptions(),
|
||||
engine,
|
||||
lambda stage, detail: say(img_path, stage, detail),
|
||||
_SourceEvidence(img_path),
|
||||
force,
|
||||
)
|
||||
if not out_path.exists():
|
||||
# Keep the output directory COMPLETE even when the pixels are deliberately
|
||||
# left alone; a hole the caller cannot see is worse than an unchanged copy.
|
||||
src_bgr, src_alpha = image_io.read_bgr_and_alpha(img_path)
|
||||
if src_bgr is None or not image_io.write_bgr_with_alpha(out_path, src_bgr, src_alpha):
|
||||
raise OSError(f"failed to copy input through to output: {out_path}")
|
||||
return outcome
|
||||
@@ -30,11 +30,14 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkLocation,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image SHORT side (measured basis). The
|
||||
@@ -100,79 +103,51 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
BaiduDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Baidu alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
"""Binary "百度" silhouette (255 = glyph) from the alpha map, or None."""
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the Baidu glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class BaiduEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible Baidu "百度 AI生成" mark (bottom-right; localize -> fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
def footprint_mask(
|
||||
self, image: NDArray[Any] | None, *, force: bool = False, dilate: int | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame mask of the WHOLE mark (text run + the pill tag to its right).
|
||||
def _footprint_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Bound the fill by the detector's match box, never by the binary glyph blob.
|
||||
|
||||
The base class's blob-bbox footprint UNDERCOVERS this mark: the white tag's
|
||||
flat interior gives no top-hat response (a top-hat answers edges, not flats),
|
||||
so the blob ends at the text run and the fill leaves the tag's right half as
|
||||
a ghost (measured 2026-07-22 on the 768x1024 cohort frame: blob bbox x
|
||||
632..746 vs the tag ending ~758). The layout is measured and fixed -- the
|
||||
text run is at the left of the locate box, the tag runs to the corner -- so
|
||||
the footprint is the detector's match box extended RIGHT to the corner.
|
||||
632..746 vs the tag ending ~758).
|
||||
"""
|
||||
if image is None or image.size == 0:
|
||||
return None
|
||||
return self._match_box_rect(image, loc, force=force, detection=detection)
|
||||
|
||||
from remove_ai_watermarks import image_io, region_eraser
|
||||
def _extend_match_box(
|
||||
self, box: tuple[int, int, int, int], loc: TextMarkLocation, frame: tuple[int, int]
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Extend the match box RIGHT to the corner end of the locate box.
|
||||
|
||||
image = image_io.to_bgr(image)
|
||||
h, w = image.shape[:2]
|
||||
if h < 32 or w < 64:
|
||||
return None
|
||||
loc = self.locate(image)
|
||||
The layout is measured and fixed: the text run is at the left of the locate
|
||||
box and the tag runs to the corner, so the mark's right edge is the box's.
|
||||
"""
|
||||
gx0, gy0, _gx1, gy1 = box
|
||||
bx, by, bw, bh = loc.bbox
|
||||
if force:
|
||||
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
|
||||
else:
|
||||
if not self.detect(image).detected:
|
||||
return None
|
||||
_, box = self._tophat_best(image, loc)
|
||||
if box is None:
|
||||
return None
|
||||
gx0, gy0, _gx1, gy1 = box
|
||||
pad = max(4, int(0.15 * bh))
|
||||
rx1 = max(0, bx + gx0 - pad)
|
||||
ry1 = max(0, by + gy0 - pad)
|
||||
rx2 = min(w, bx + bw) # the tag runs to the corner end of the box
|
||||
ry2 = min(h, by + gy1 + 1 + pad)
|
||||
if rx1 >= rx2 or ry1 >= ry2:
|
||||
return None
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * bw))
|
||||
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
h, w = frame
|
||||
pad = max(4, int(0.15 * bh))
|
||||
return (
|
||||
max(0, bx + gx0 - pad),
|
||||
max(0, by + gy0 - pad),
|
||||
min(w, bx + bw), # the tag runs to the corner end of the box
|
||||
min(h, by + gy1 + 1 + pad),
|
||||
)
|
||||
+297
-713
File diff suppressed because it is too large.
Load diff
@@ -23,11 +23,9 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of image WIDTH (the mark scales with width, anchored
|
||||
@@ -96,9 +94,6 @@ _CONFIG = TextMarkConfig(
|
||||
min_gw=8,
|
||||
)
|
||||
|
||||
# Doubao-specific aliases for the shared detection result/engine.
|
||||
DoubaoDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Doubao alpha template (float [0,1]), or None."""
|
||||
@@ -120,13 +115,3 @@ class DoubaoEngine(TextMarkEngine):
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
@@ -1,7 +1,11 @@
|
||||
"""DWT-DCT decoder compatible with invisible-watermark's ``dwtDct`` path.
|
||||
|
||||
Derived from ShieldMnt/invisible-watermark ``imwatermark/maxDct.py`` (MIT),
|
||||
trimmed to the matrix path used by Stable Diffusion, SDXL, and FLUX.
|
||||
trimmed to the matrix path used by Stable Diffusion, SDXL, and FLUX. The block
|
||||
scan is vectorized rather than transcribed, so the file no longer reads line by
|
||||
line against upstream; what it preserves is the output, bit for bit. See
|
||||
[`docs/module-internals.md`](../../docs/module-internals.md) for the
|
||||
measurements and for why a faster hand-rolled transform is not available.
|
||||
|
||||
Copyright (c) 2021 ShieldMnt
|
||||
|
||||
@@ -23,7 +27,39 @@ if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
|
||||
_DEFAULT_SCALES = (0, 36, 36)
|
||||
_DEFAULT_BLOCK = 4
|
||||
# Fixed by the format being decoded, and the 4-way fold chains in `_frame_bits`
|
||||
# are written for it. It was a constructor parameter while the block scan was a
|
||||
# generic Python loop; nothing ever passed another value, and a knob that now
|
||||
# silently returns wrong bits is worse than no knob.
|
||||
_BLOCK = 4
|
||||
|
||||
# Block-rows of the approximation band handled per strip. Not a tuned value:
|
||||
# every height measured landed inside the others' noise. What matters is strips
|
||||
# at all rather than a full-plane intermediate, not this number.
|
||||
_STRIP = 16
|
||||
|
||||
|
||||
def _approximation(rows: NDArray[Any]) -> NDArray[Any]:
|
||||
"""One Haar pass along the last axis, approximation band only.
|
||||
|
||||
``pywt.dwt(x, "haar", axis=1)[0]`` computes and allocates the detail band as
|
||||
well, and dispatches per row. Flattening lets one ``downcoef`` call do the
|
||||
whole plane, and it is the same numbers in the same order **only while the
|
||||
last axis is even**: Haar's filter is length 2, so an even row length makes
|
||||
every pair fall inside its own row with no boundary extension. On an odd
|
||||
width the pairs walk across row boundaries and the reshape below still
|
||||
succeeds whenever the total is even -- wrong bits, no exception, and the
|
||||
upstream-parity test is `skipif`-gated. Hence the explicit check.
|
||||
|
||||
The transform itself stays inside pywt however slow that looks. Its C
|
||||
convolution contracts into an FMA that no numpy expression reproduces, and
|
||||
the caller's threshold is ``peak % 36 > 18.0`` against values that are exact
|
||||
multiples of 0.5, so a 1-ulp difference flips real bits.
|
||||
"""
|
||||
height, width = rows.shape
|
||||
if width % 2:
|
||||
raise RuntimeError(f"row length {width} is odd; the raveled Haar pass requires an even last axis")
|
||||
return pywt.downcoef("a", rows.ravel(), "haar").reshape(height, width // 2)
|
||||
|
||||
|
||||
class _DecodeMaxDct:
|
||||
@@ -33,54 +69,85 @@ class _DecodeMaxDct:
|
||||
self,
|
||||
wm_lengths: tuple[int, ...],
|
||||
scales: tuple[int, int, int] = _DEFAULT_SCALES,
|
||||
block: int = _DEFAULT_BLOCK,
|
||||
) -> None:
|
||||
self._wm_lengths = wm_lengths
|
||||
self._scales = scales
|
||||
self._block = block
|
||||
|
||||
def decode(self, bgr: NDArray[Any]) -> dict[int, NDArray[Any]]:
|
||||
row, col, _channels = bgr.shape
|
||||
yuv = cv2.cvtColor(bgr, cv2.COLOR_BGR2YUV)
|
||||
trimmed = yuv[: row // 4 * 4, : col // 4 * 4]
|
||||
|
||||
scores_by_length = {wm_len: ([0] * wm_len, [0] * wm_len) for wm_len in self._wm_lengths}
|
||||
for channel in range(2):
|
||||
if self._scales[channel] <= 0:
|
||||
continue
|
||||
ca1, _detail = pywt.dwt2(yuv[: row // 4 * 4, : col // 4 * 4, channel], "haar")
|
||||
self._decode_frame(ca1, self._scales[channel], scores_by_length)
|
||||
per_channel = [
|
||||
self._plane_bits(trimmed, channel, self._scales[channel])
|
||||
for channel in range(2)
|
||||
if self._scales[channel] > 0
|
||||
]
|
||||
# Each channel restarts the bit index at 0, so the buckets come from a
|
||||
# per-channel arange rather than one running counter.
|
||||
index = np.concatenate([np.arange(bits.size) for bits in per_channel] or [np.zeros(0, dtype=np.int64)])
|
||||
weights = np.concatenate(per_channel or [np.zeros(0)])
|
||||
|
||||
return {
|
||||
wm_len: np.asarray(sums) * 255 > np.asarray(counts) * 127
|
||||
for wm_len, (sums, counts) in scores_by_length.items()
|
||||
}
|
||||
decoded: dict[int, NDArray[Any]] = {}
|
||||
for wm_len in self._wm_lengths:
|
||||
bucket = index % wm_len
|
||||
sums = np.bincount(bucket, weights=weights, minlength=wm_len)
|
||||
counts = np.bincount(bucket, minlength=wm_len)
|
||||
decoded[wm_len] = sums * 255 > counts * 127
|
||||
return decoded
|
||||
|
||||
def _decode_frame(
|
||||
self,
|
||||
frame: NDArray[Any],
|
||||
scale: int,
|
||||
scores_by_length: dict[int, tuple[list[int], list[int]]],
|
||||
) -> None:
|
||||
row, col = frame.shape
|
||||
bit_index = 0
|
||||
for i in range(row // self._block):
|
||||
for j in range(col // self._block):
|
||||
block = frame[
|
||||
i * self._block : i * self._block + self._block,
|
||||
j * self._block : j * self._block + self._block,
|
||||
]
|
||||
inferred = self._infer_bit(block, scale)
|
||||
for wm_len, (sums, counts) in scores_by_length.items():
|
||||
bucket = bit_index % wm_len
|
||||
sums[bucket] += inferred
|
||||
counts[bucket] += 1
|
||||
bit_index += 1
|
||||
def _plane_bits(self, trimmed: NDArray[Any], channel: int, scale: int) -> NDArray[Any]:
|
||||
"""Block bits for one colour plane, a strip of block-rows at a time.
|
||||
|
||||
def _infer_bit(self, block: NDArray[Any], scale: int) -> int:
|
||||
position = int(np.argmax(np.abs(block.flatten()[1:]))) + 1
|
||||
i, j = position // self._block, position % self._block
|
||||
value = abs(float(block[i][j]))
|
||||
return int((value % scale) > 0.5 * scale)
|
||||
``dwt2`` is ``dwtn``: it transforms along axis 0, then along axis 1 over
|
||||
both halves, and three of the four bands it returns are discarded here.
|
||||
Only the approximation is ever asked for, and transposing between the
|
||||
two passes lets pywt walk a contiguous axis instead of a column -- that
|
||||
access pattern, not the arithmetic saved, is where the time goes.
|
||||
|
||||
Strips mean no full-plane float64 intermediate is ever materialized, and
|
||||
they are seam-free for the reason ``_approximation`` documents: output
|
||||
row ``k`` reads input rows ``2k`` and ``2k + 1`` only. Strips start on
|
||||
multiples of ``2 * _BLOCK``, so neither a pair nor a 4x4 block straddles
|
||||
one.
|
||||
"""
|
||||
if trimmed.shape[0] == 0 or trimmed.shape[1] == 0:
|
||||
# Reachable: a 1x65536 image clears the caller's area check and
|
||||
# trims to an empty plane. Left to dwt2 so the exception stays the
|
||||
# one this module has always raised -- returning empty bits here
|
||||
# instead would silently turn a raise into an all-false verdict.
|
||||
return pywt.dwt2(trimmed[:, :, channel], "haar")[0]
|
||||
rows = trimmed.shape[0] // (2 * _BLOCK)
|
||||
cols = trimmed.shape[1] // (2 * _BLOCK)
|
||||
if rows == 0 or cols == 0:
|
||||
return np.zeros(0, dtype=np.float64)
|
||||
|
||||
width = cols * 2 * _BLOCK
|
||||
pieces: list[NDArray[Any]] = []
|
||||
for start in range(0, rows, _STRIP):
|
||||
stop = min(start + _STRIP, rows)
|
||||
strip = trimmed[start * 2 * _BLOCK : stop * 2 * _BLOCK, :width]
|
||||
columns = cv2.transpose(cv2.extractChannel(strip, channel))
|
||||
band = _approximation(cv2.transpose(_approximation(columns)))
|
||||
pieces.append(self._frame_bits(band, scale))
|
||||
return np.concatenate(pieces)
|
||||
|
||||
def _frame_bits(self, frame: NDArray[Any], scale: int) -> NDArray[Any]:
|
||||
"""One bit per 4x4 block, in row-major block order.
|
||||
|
||||
Upstream's per-block loop, said to numpy once instead of to the
|
||||
interpreter ~135k times per image. Zeroing the DC term in the absolute
|
||||
band says "ignore index 0" without materializing a ``(nblocks, 16)``
|
||||
copy, and the 4-way ``np.maximum`` chains reduce over contiguous rows.
|
||||
"""
|
||||
rows = frame.shape[0] // _BLOCK
|
||||
cols = frame.shape[1] // _BLOCK
|
||||
band = np.abs(frame[: rows * _BLOCK, : cols * _BLOCK])
|
||||
band[::_BLOCK, ::_BLOCK] = 0.0
|
||||
folded = np.maximum(np.maximum(band[0::4], band[1::4]), np.maximum(band[2::4], band[3::4]))
|
||||
folded = folded.reshape(rows * cols, _BLOCK)
|
||||
peak = np.maximum(np.maximum(folded[:, 0], folded[:, 1]), np.maximum(folded[:, 2], folded[:, 3]))
|
||||
return ((peak % scale) > 0.5 * scale).astype(np.float64)
|
||||
|
||||
|
||||
def decode_dwt_dct(bgr: NDArray[Any], wm_len: int) -> NDArray[Any]:
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -6,7 +6,7 @@ from __future__ import annotations
|
||||
|
||||
import functools
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, replace
|
||||
from enum import Enum
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
@@ -72,6 +72,26 @@ class _Candidate:
|
||||
return self.spatial * 0.50 + self.gradient * 0.30 + self.variance * 0.20
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _SparkleScan:
|
||||
"""The provenance-BLIND half of sparkle detection, reusable across trust levels.
|
||||
|
||||
``source`` is the BGR-normalized image the false-positive gate re-reads, ``best``
|
||||
the winning candidate, and ``base`` a template result carrying everything the scan
|
||||
already resolved (``size`` and, when a candidate won, ``region`` and the component
|
||||
scores). ``best is None`` covers both no-candidate cases; ``base`` distinguishes
|
||||
them, since an empty image never resolves a ``size`` and a candidate-less one does.
|
||||
|
||||
Per-CALL only, never cached on the engine: ``remove_auto_marks`` re-invokes each
|
||||
engine on a progressively cleaned frame within one process, so a memo on ``self``
|
||||
would hand back a pre-fill scan of a different image.
|
||||
"""
|
||||
|
||||
source: NDArray[Any] | None
|
||||
best: _Candidate | None
|
||||
base: DetectionResult
|
||||
|
||||
|
||||
def get_watermark_size(width: int, height: int) -> WatermarkSize:
|
||||
"""Return the provider's large tier only when both axes exceed 1024."""
|
||||
return WatermarkSize.LARGE if width > 1024 and height > 1024 else WatermarkSize.SMALL
|
||||
@@ -200,29 +220,62 @@ class GeminiEngine:
|
||||
trust_provenance: bool = False,
|
||||
) -> DetectionResult:
|
||||
"""Return the strongest sparkle-shaped bottom-right candidate."""
|
||||
result = DetectionResult()
|
||||
scan = self._sparkle_scan(image, force_size)
|
||||
return self._verdict(scan, trust_provenance=trust_provenance)
|
||||
|
||||
def detect_watermark_both(
|
||||
self, image: NDArray[Any], force_size: WatermarkSize | None = None
|
||||
) -> tuple[DetectionResult, DetectionResult]:
|
||||
"""``(strict, relaxed)`` from ONE scan of the image.
|
||||
|
||||
The scan -- global candidate search, corner promotion and fused scoring -- is
|
||||
provenance-blind; ``trust_provenance`` only decides whether the false-positive
|
||||
gate demotes the confidence afterwards. Two calls therefore repeated the whole
|
||||
sweep to reach two verdicts.
|
||||
|
||||
The two results are DISTINCT objects with genuinely different confidences (the
|
||||
gate rewrites one of them), and callers mutate them.
|
||||
"""
|
||||
scan = self._sparkle_scan(image, force_size)
|
||||
return (
|
||||
self._verdict(scan, trust_provenance=False),
|
||||
self._verdict(scan, trust_provenance=True),
|
||||
)
|
||||
|
||||
def _sparkle_scan(self, image: NDArray[Any], force_size: WatermarkSize | None) -> _SparkleScan:
|
||||
"""Everything in detection that does not depend on the trust level."""
|
||||
if image is None or image.size == 0:
|
||||
return result
|
||||
return _SparkleScan(None, None, DetectionResult())
|
||||
|
||||
source = image_io.to_bgr(image)
|
||||
height, width = source.shape[:2]
|
||||
result.size = force_size or get_watermark_size(width, height)
|
||||
size = force_size or get_watermark_size(width, height)
|
||||
candidates = self._global_candidates(source)
|
||||
promoted = self._corner_promote(source, candidates[0].spatial if candidates else -1.0)
|
||||
if promoted is not None:
|
||||
candidates.append(_Candidate(promoted[0], promoted[1], promoted[2], promoted[3]))
|
||||
# The no-candidate result is NOT the empty-image one: `size` is already resolved
|
||||
# here, and it is a public field the caller can force.
|
||||
base = DetectionResult(size=size)
|
||||
if not candidates:
|
||||
return result
|
||||
return _SparkleScan(None, None, base)
|
||||
|
||||
best = max((self._score_candidate(source, candidate) for candidate in candidates), key=lambda item: item.fused)
|
||||
result.region = (best.x, best.y, best.scale, best.scale)
|
||||
result.spatial_score = float(best.spatial)
|
||||
result.gradient_score = float(best.gradient)
|
||||
result.variance_score = float(best.variance)
|
||||
base.region = (best.x, best.y, best.scale, best.scale)
|
||||
base.spatial_score = float(best.spatial)
|
||||
base.gradient_score = float(best.gradient)
|
||||
base.variance_score = float(best.variance)
|
||||
return _SparkleScan(source, best, base)
|
||||
|
||||
def _verdict(self, scan: _SparkleScan, *, trust_provenance: bool) -> DetectionResult:
|
||||
"""Apply the trust-level-dependent tail to a scan, as a fresh result object."""
|
||||
result = replace(scan.base)
|
||||
if scan.best is None or scan.source is None:
|
||||
return result
|
||||
best = scan.best
|
||||
confidence = best.fused
|
||||
if best.spatial >= 0.25 and confidence < self._SPARKLE_FP_CONF and not trust_provenance:
|
||||
confidence = self._apply_false_positive_gate(source, best, confidence)
|
||||
confidence = self._apply_false_positive_gate(scan.source, best, confidence)
|
||||
result.confidence = float(np.clip(confidence, 0.0, 1.0))
|
||||
result.detected = result.confidence >= 0.35
|
||||
return result
|
||||
|
||||
@@ -6,8 +6,8 @@ Aggregates every locally-readable signal into a single :class:`ProvenanceReport`
|
||||
the signing platform (OpenAI, Google, Adobe, Microsoft).
|
||||
- **IPTC ``digitalSourceType``** "Made with AI" marker (Meta, X, others).
|
||||
- **PNG text / EXIF generation parameters** (Stable Diffusion, ComfyUI, InvokeAI).
|
||||
- **SynthID metadata proxy** -- a C2PA companion from a SynthID-using vendor
|
||||
(Google / OpenAI) implies the invisible pixel watermark.
|
||||
- **SynthID provenance evidence** -- Google AI C2PA follows Google's all-media
|
||||
policy; current OpenAI C2PA explicitly declares a watermark action.
|
||||
- **Registered visible marks** (optional; needs cv2/numpy, no GPU) through the
|
||||
shared watermark registry.
|
||||
|
||||
@@ -22,21 +22,28 @@ from __future__ import annotations
|
||||
import base64
|
||||
import itertools
|
||||
import logging
|
||||
import struct
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any, cast
|
||||
|
||||
from remove_ai_watermarks._internal.c2pa import (
|
||||
c2pa_info_from_manifest_store,
|
||||
c2pa_info_has_invalid_credential,
|
||||
c2pa_info_has_removal_hint,
|
||||
cbor_text_after,
|
||||
extract_c2pa_info,
|
||||
soft_binding_vendors_in,
|
||||
synthid_evidence_vendors_in,
|
||||
synthid_verdict,
|
||||
)
|
||||
from remove_ai_watermarks._internal.constants import (
|
||||
C2PA_AI_TOOLS,
|
||||
C2PA_AI_VENDORS,
|
||||
C2PA_CLAIM_GENERATOR_PLATFORMS,
|
||||
C2PA_IDENTITY_AI_ORGS,
|
||||
C2PA_ISSUERS,
|
||||
)
|
||||
from remove_ai_watermarks._internal.schema import require_schema_version
|
||||
from remove_ai_watermarks.metadata import (
|
||||
AI_METADATA_KEYS,
|
||||
AIGC_MARKERS,
|
||||
@@ -59,6 +66,7 @@ from remove_ai_watermarks.metadata import (
|
||||
xai_signature_pair,
|
||||
)
|
||||
from remove_ai_watermarks.watermark_registry import GEMINI_SPARKLE_TRUST_CONF
|
||||
from remove_ai_watermarks.watermark_registry import known_marks as _known_marks
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
@@ -69,6 +77,10 @@ if TYPE_CHECKING:
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Stable JSON contract for callers that pass a verdict between services. Bump this
|
||||
# only for a breaking shape or semantic change; adding optional fields is compatible.
|
||||
PROVENANCE_REPORT_SCHEMA_VERSION = 1
|
||||
|
||||
# How much of a non-PNG container to binary-scan for the C2PA issuer.
|
||||
_SCAN_BYTES = 1024 * 1024
|
||||
|
||||
@@ -101,12 +113,20 @@ _STRIP_CAVEAT = (
|
||||
"text chunks are stripped by re-encoding, screenshots, or social-media upload."
|
||||
)
|
||||
_SYNTHID_CAVEAT = (
|
||||
"SynthID is a metadata proxy here; the pixel watermark is not locally "
|
||||
"verifiable (proprietary decoder). Confirm via the Gemini app or openai.com/verify."
|
||||
"SynthID presence comes from supported provenance here; the pixel watermark is not locally "
|
||||
"decoded (proprietary decoder). Confirm via the Gemini app or openai.com/verify."
|
||||
)
|
||||
_OPENAI_CAVEAT = (
|
||||
"OpenAI began pairing SynthID with C2PA around 2026-05; OpenAI images from "
|
||||
"before the rollout carry C2PA without SynthID, so the SynthID verdict is 'likely'."
|
||||
_C2PA_UNTRUSTED_CAVEAT = (
|
||||
"The C2PA claim signature and asset binding validate, but the signer identity is not anchored "
|
||||
"to a trusted credential here; treat the named platform as a signed claim, not a verified identity."
|
||||
)
|
||||
_C2PA_UNVALIDATED_CAVEAT = (
|
||||
"The C2PA marker was parsed without cryptographic validation; treat its origin and watermark "
|
||||
"assertions as unverified claims."
|
||||
)
|
||||
_C2PA_INVALID_CAVEAT = (
|
||||
"The embedded C2PA claim no longer validates against this asset. Its origin and watermark assertions "
|
||||
"are retained only as removal hints, not as verified provenance."
|
||||
)
|
||||
_IPTC_ONLY_CAVEAT = "The IPTC 'Made with AI' tag flags AI provenance but does not identify the specific platform."
|
||||
_INVISIBLE_WM_CAVEAT = (
|
||||
@@ -125,6 +145,10 @@ _C2PA_CLOUD_CAVEAT = (
|
||||
"It marks Content Credentials, not AI origin: the cloud manifest may describe a "
|
||||
"human edit, and reading it needs a network fetch this tool does not make."
|
||||
)
|
||||
_SOFT_BINDING_CAVEAT = (
|
||||
"Removing the embedded C2PA manifest does not remove its soft binding: the named "
|
||||
"watermark or fingerprint may remain in the pixels and re-link the asset to provenance."
|
||||
)
|
||||
_SAMSUNG_GENAI_CAVEAT = (
|
||||
"Samsung's genAIType marker shows a Galaxy AI editing tool (Generative Edit, "
|
||||
"Sketch to Image, ...) touched the image; it is an undocumented proprietary "
|
||||
@@ -168,7 +192,32 @@ def _external_metadata(value: Any) -> tuple[list[tuple[str, Any]], bytes]:
|
||||
"""Index nested metadata and recover common encoded binary values in one pass."""
|
||||
pairs: list[tuple[str, Any]] = []
|
||||
parts: list[bytes] = []
|
||||
diagnostic_keys = {"error", "kind"}
|
||||
diagnostic_keys = {
|
||||
"artifacts",
|
||||
"birthtime",
|
||||
"color",
|
||||
"content_format",
|
||||
"dct",
|
||||
"ela",
|
||||
"error",
|
||||
"extension",
|
||||
"fft",
|
||||
"file",
|
||||
"filename",
|
||||
"full",
|
||||
"gradient",
|
||||
"kind",
|
||||
"mtime",
|
||||
"name",
|
||||
"noise",
|
||||
"path",
|
||||
"pixel",
|
||||
"provenance",
|
||||
"sha256",
|
||||
"signals",
|
||||
"size_bytes",
|
||||
"timing_ms",
|
||||
}
|
||||
|
||||
def visit(item: Any) -> None:
|
||||
if isinstance(item, dict):
|
||||
@@ -176,7 +225,6 @@ def _external_metadata(value: Any) -> tuple[list[tuple[str, Any]], bytes]:
|
||||
for key, nested in mapping.items():
|
||||
key_text = str(key)
|
||||
pairs.append((key_text, nested))
|
||||
parts.append(key_text.encode("utf-8", "replace"))
|
||||
if key_text.lower() in diagnostic_keys:
|
||||
continue
|
||||
if isinstance(nested, str) and (key_text == "base64" or key_text.endswith("_base64")):
|
||||
@@ -231,6 +279,7 @@ def _external_exif_generator(pairs: list[tuple[str, Any]], scan: bytes) -> str |
|
||||
"title",
|
||||
"description",
|
||||
"creatortool",
|
||||
"usercomment",
|
||||
}
|
||||
candidates = [
|
||||
_external_text(value)
|
||||
@@ -240,17 +289,69 @@ def _external_exif_generator(pairs: list[tuple[str, Any]], scan: bytes) -> str |
|
||||
return generator_from_metadata(candidates, scan)
|
||||
|
||||
|
||||
def _metadata_source_kind(info: dict[str, Any], scan: bytes) -> str | None:
|
||||
"""Normalize the source type wherever it is carried: C2PA or IPTC/XMP.
|
||||
|
||||
A composite marker contains ``TrainedAlgorithmicMedia`` as a substring, so it
|
||||
is removed before looking for a standalone full-generation marker. When a file
|
||||
genuinely carries both kinds, full generation wins.
|
||||
"""
|
||||
structured = info.get("ai_source_kind")
|
||||
without_composites = scan.replace(b"compositeWithTrainedAlgorithmicMedia", b"").replace(b"compositeSynthetic", b"")
|
||||
generated = structured == "generated" or any(
|
||||
marker in without_composites for marker in (b"trainedAlgorithmicMedia", b"TrainedAlgorithmicMedia")
|
||||
)
|
||||
if generated:
|
||||
return "generated"
|
||||
if structured == "enhanced" or any(
|
||||
marker in scan for marker in (b"compositeWithTrainedAlgorithmicMedia", b"compositeSynthetic")
|
||||
):
|
||||
return "enhanced"
|
||||
return None
|
||||
|
||||
|
||||
def evidence_from_metadata_record(
|
||||
record: dict[str, Any], *, path: Path, c2pa_manifest_store: str | dict[str, Any] | None = None
|
||||
) -> ProvenanceEvidence:
|
||||
"""Normalize an externally collected metadata record into provenance evidence.
|
||||
|
||||
The record may contain arbitrary nested dictionaries and lists. Text, bytes,
|
||||
hexadecimal values prefixed with ``hex:``, and fields named ``base64`` or
|
||||
ending in ``_base64`` are included in the shared byte scan. No source file is
|
||||
opened.
|
||||
Unversioned external records may contain arbitrary nested dictionaries and
|
||||
lists. Versioned native records accept only the source-derived fields emitted by
|
||||
``collect_metadata_record``; other native record types and unknown schema
|
||||
versions are rejected. No source file is opened.
|
||||
"""
|
||||
pairs, scan = _external_metadata(record)
|
||||
from remove_ai_watermarks.metadata_record import METADATA_RECORD_SCHEMA_VERSION, METADATA_RECORD_TYPE
|
||||
|
||||
# Records produced by ``collect_metadata_record`` are a versioned transport
|
||||
# contract. Only their source-derived fields are evidence: the filename,
|
||||
# container label and schema bookkeeping describe the collector and must never
|
||||
# become detector input. Shape-detect the pre-versioned form as well so records
|
||||
# emitted by 0.26 remain safe and readable.
|
||||
record_type = record.get("record_type")
|
||||
if record_type not in (None, METADATA_RECORD_TYPE):
|
||||
raise ValueError(f"Unsupported metadata record type: {record_type!r}")
|
||||
if record_type == METADATA_RECORD_TYPE:
|
||||
require_schema_version(
|
||||
record.get("schema_version"),
|
||||
contract="provenance metadata",
|
||||
supported=(METADATA_RECORD_SCHEMA_VERSION,),
|
||||
)
|
||||
status = record.get("status")
|
||||
if status == "error":
|
||||
raise ValueError("Provenance metadata collection failed")
|
||||
if status != "complete":
|
||||
raise ValueError(f"Unsupported provenance metadata collection status: {status!r}")
|
||||
is_portable_record = record_type == METADATA_RECORD_TYPE or {
|
||||
"container",
|
||||
"metadata_base64",
|
||||
"tail_base64",
|
||||
}.issubset(record)
|
||||
evidence_record = (
|
||||
{key: record[key] for key in ("metadata_base64", "tail_base64", "pil", "exif") if key in record}
|
||||
if is_portable_record
|
||||
else record
|
||||
)
|
||||
pairs, scan = _external_metadata(evidence_record)
|
||||
store = c2pa_manifest_store
|
||||
if store is None:
|
||||
candidate = record.get("c2pa_store")
|
||||
@@ -291,6 +392,8 @@ def evidence_from_metadata_record(
|
||||
"actions",
|
||||
"synthid_watermark",
|
||||
"soft_binding",
|
||||
"soft_binding_algorithm",
|
||||
"soft_binding_value",
|
||||
):
|
||||
if key in c2pa_info:
|
||||
ai_metadata.setdefault(key, str(c2pa_info[key]))
|
||||
@@ -357,21 +460,22 @@ class ProvenanceReport:
|
||||
is_ai_generated: bool | None # True / False is never asserted; None = unknown
|
||||
platform: str | None
|
||||
confidence: str # "high" | "medium" | "none"
|
||||
# Coarse AI-origin kind from the C2PA digital-source-type, so a caller can
|
||||
# branch on full generation vs an AI-touched real photo:
|
||||
# Coarse AI-origin kind from a C2PA or standalone IPTC/XMP digital-source-type,
|
||||
# so a caller can branch on full generation vs an AI-touched real photo:
|
||||
# "generated" -- digitalSourceType trainedAlgorithmicMedia (fully AI).
|
||||
# "enhanced" -- compositeWithTrainedAlgorithmicMedia (real content with an
|
||||
# AI-composited region; scrub the AI region, keep the photo).
|
||||
# None -- no C2PA AI source-type (verdict, if AI, came from another
|
||||
# signal: IPTC, AIGC, local gen params, xAI, ...).
|
||||
# None -- no AI digital-source-type (verdict, if AI, came from another
|
||||
# signal: AIGC, local gen params, xAI, ...).
|
||||
ai_source_kind: str | None = None
|
||||
# True when the AI verdict rests on a metadata or embedded-invisible signal
|
||||
# (C2PA AI issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, or
|
||||
# an open DWT-DCT / TrustMark decode) -- as opposed to a visible mark or a
|
||||
# weak medium-confidence hint (hf-job, Samsung genAIType). It is exactly the
|
||||
# set of signals an invisible/diffusion scrub targets: a visible-only or
|
||||
# no-signal image has it False. Equivalent to ``confidence == "high"``;
|
||||
# surfaced as a field so callers gate on intent, not on the string.
|
||||
# (C2PA AI issuer / SynthID provenance, IPTC, AIGC, local gen params, EXIF/xAI, or
|
||||
# an open DWT-DCT decode) -- as opposed to a visible mark, provenance-only
|
||||
# TrustMark, or a weak medium-confidence hint (hf-job, Samsung genAIType). This
|
||||
# is the set of signals an invisible/diffusion scrub targets: a visible-only
|
||||
# or no-signal image has it False. An intact C2PA AI claim from an untrusted
|
||||
# signer is deliberately medium-confidence while this remains True; callers
|
||||
# should gate on intent, not on the confidence string.
|
||||
ai_from_metadata: bool = False
|
||||
watermarks: list[str] = field(default_factory=list[str])
|
||||
signals: list[Signal] = field(default_factory=list["Signal"])
|
||||
@@ -381,6 +485,78 @@ class ProvenanceReport:
|
||||
# AI-generation markers). Non-empty means the provenance is internally
|
||||
# inconsistent -- a strong tell of spoofed, transplanted, or laundered metadata.
|
||||
integrity_clashes: list[str] = field(default_factory=list[str])
|
||||
# Orthogonal C2PA checks. A valid content binding does not make an untrusted
|
||||
# signer identity trusted, and an expired credential does not by itself mean the
|
||||
# signed bytes changed. Fallback parsing reports each dimension as unknown;
|
||||
# None means no C2PA result exists.
|
||||
c2pa_validation: dict[str, Any] | None = None
|
||||
|
||||
def to_dict(
|
||||
self,
|
||||
*,
|
||||
schema_version: int = PROVENANCE_REPORT_SCHEMA_VERSION,
|
||||
) -> dict[str, Any]:
|
||||
"""Return the versioned, JSON-safe verdict contract.
|
||||
|
||||
``path`` is deliberately omitted. It is extraction context, not part of the
|
||||
verdict, and local filesystem paths should not cross a service boundary.
|
||||
Request an explicit schema for a long-lived transport consumer.
|
||||
"""
|
||||
schema_version = require_schema_version(
|
||||
schema_version,
|
||||
contract="provenance report",
|
||||
supported=(1,),
|
||||
)
|
||||
return {
|
||||
"schema_version": schema_version,
|
||||
"is_ai_generated": self.is_ai_generated,
|
||||
"platform": self.platform,
|
||||
"confidence": self.confidence,
|
||||
"ai_source_kind": self.ai_source_kind,
|
||||
"ai_from_metadata": self.ai_from_metadata,
|
||||
"watermarks": list(self.watermarks),
|
||||
"signals": [
|
||||
{
|
||||
"name": signal.name,
|
||||
"detail": signal.detail,
|
||||
"confidence": signal.confidence,
|
||||
}
|
||||
for signal in self.signals
|
||||
],
|
||||
"caveats": list(self.caveats),
|
||||
"integrity_clashes": list(self.integrity_clashes),
|
||||
"c2pa_validation": self.c2pa_validation,
|
||||
}
|
||||
|
||||
|
||||
def _c2pa_validation(info: dict[str, Any]) -> dict[str, Any] | None:
|
||||
source = info.get("c2pa_validation_source")
|
||||
if not isinstance(source, str):
|
||||
return None
|
||||
codes = info.get("c2pa_validation_codes")
|
||||
return {
|
||||
"source": source,
|
||||
"state": str(info.get("c2pa_validation_state", "unknown")),
|
||||
"integrity": str(info.get("c2pa_integrity", "unknown")),
|
||||
"signature": str(info.get("c2pa_signature", "unknown")),
|
||||
"signer_trust": str(info.get("c2pa_signer_trust", "unknown")),
|
||||
"signer_validity": str(info.get("c2pa_signer_validity", "unknown")),
|
||||
"codes": list(cast("list[object]", codes)) if isinstance(codes, list) else [],
|
||||
}
|
||||
|
||||
|
||||
def _c2pa_credential_level(info: dict[str, Any]) -> str:
|
||||
"""Return invalid, verified, or unverified for provenance attribution."""
|
||||
if c2pa_info_has_invalid_credential(info):
|
||||
return "invalid"
|
||||
if (
|
||||
info.get("c2pa_integrity") == "valid"
|
||||
and info.get("c2pa_signature") == "valid"
|
||||
and info.get("c2pa_signer_trust") == "trusted"
|
||||
and info.get("c2pa_signer_validity") == "valid"
|
||||
):
|
||||
return "verified"
|
||||
return "unverified"
|
||||
|
||||
|
||||
def extract_provenance_evidence(image_path: Path) -> ProvenanceEvidence:
|
||||
@@ -446,14 +622,88 @@ _DEVICE_C2PA_PLATFORM: tuple[tuple[bytes, str], ...] = (
|
||||
)
|
||||
|
||||
|
||||
def _device_platform(head: bytes) -> str | None:
|
||||
"""Map a distinctive C2PA device/camera token in the manifest bytes to a platform."""
|
||||
for token, platform in _DEVICE_C2PA_PLATFORM:
|
||||
def _metadata_region(head: bytes) -> bytes:
|
||||
"""The part of the scan buffer that can hold metadata, with the coded pixels cut out.
|
||||
|
||||
The vendor registries are matched as raw substrings, and the shortest tokens are
|
||||
four and five bytes (``Bria``, ``Adobe``, ``Canva``). Over a megabyte of compressed
|
||||
pixel data a four-byte sequence appears by chance about once in three thousand
|
||||
images -- measured: ``Bria`` matched inside the entropy-coded scan of 4 of 14,707
|
||||
corpus JPEGs, in none of which the manifest names Bria. That is not a cosmetic
|
||||
mislabel, because the Bria entry carries ``asserts_ai``: a chance match can declare
|
||||
an image AI-generated.
|
||||
|
||||
``c2pa_marker_in`` already refuses a bare ``c2pa`` substring for the same reason.
|
||||
This is the same defence for the registries: they see the container's metadata and
|
||||
not its pixels.
|
||||
|
||||
JPEG keeps the marker segments before the entropy-coded scan, PNG every chunk but
|
||||
``IDAT``, and both keep the trailer past the end marker. Anything ``scan_head``
|
||||
APPENDED past the window is metadata by construction (late chunks, boxes, decoder
|
||||
text), so it is always kept and never walked -- walking it is what produced 11 MB
|
||||
records and a phantom AIGC signal in the record collector.
|
||||
|
||||
Trimming happens only when the container actually parses: a JPEG whose marker walk
|
||||
reaches the coded scan, a PNG whose chunk walk reaches ``IDAT``. Anything else --
|
||||
a malformed container, a synthetic blob, a format with no walker here -- is
|
||||
returned whole. Cutting a buffer this function did not understand would drop real
|
||||
evidence to avoid a chance match, which is the wrong way round.
|
||||
"""
|
||||
raw, appended = head[:_SCAN_BYTES], head[_SCAN_BYTES:]
|
||||
if raw[:2] == b"\xff\xd8":
|
||||
index, size = 2, len(raw)
|
||||
while index + 1 < size:
|
||||
if raw[index] != 0xFF:
|
||||
return head # not a marker boundary: the walk is lost, keep everything
|
||||
marker = raw[index + 1]
|
||||
if marker in (0xDA, 0xD9): # SOS / EOI: the coded scan follows
|
||||
end = raw.rfind(b"\xff\xd9")
|
||||
return raw[:index] + (raw[end + 2 :] if end >= index else b"") + appended
|
||||
if 0xD0 <= marker <= 0xD7 or marker == 0x01:
|
||||
index += 2
|
||||
continue
|
||||
if index + 4 > size:
|
||||
break
|
||||
length = int.from_bytes(raw[index + 2 : index + 4], "big")
|
||||
if length < 2 or index + 2 + length > size:
|
||||
break
|
||||
index += 2 + length
|
||||
return head # ran out of buffer before the scan: nothing was skipped anyway
|
||||
if raw[:8] == b"\x89PNG\r\n\x1a\n":
|
||||
out = bytearray()
|
||||
position, size, saw_idat = 8, len(raw), False
|
||||
while position + 8 <= size:
|
||||
(length,) = struct.unpack(">I", raw[position : position + 4])
|
||||
chunk_type = raw[position + 4 : position + 8]
|
||||
start = position + 8
|
||||
if chunk_type == b"IDAT":
|
||||
saw_idat = True
|
||||
else:
|
||||
out += chunk_type + raw[start : start + min(length, size - start)]
|
||||
position = start + length + 4
|
||||
if chunk_type == b"IEND":
|
||||
out += raw[position:]
|
||||
break
|
||||
return bytes(out) + appended if saw_idat else head
|
||||
return head
|
||||
|
||||
|
||||
def _first_token_match(head: bytes, table: tuple[tuple[bytes, str], ...]) -> str | None:
|
||||
"""First platform in ``table`` whose token appears in ``head``, else None.
|
||||
|
||||
Table order is priority: the more specific token must be listed first.
|
||||
"""
|
||||
for token, platform in table:
|
||||
if token in head:
|
||||
return platform
|
||||
return None
|
||||
|
||||
|
||||
def _device_platform(head: bytes) -> str | None:
|
||||
"""Map a distinctive C2PA device/camera token in the manifest bytes to a platform."""
|
||||
return _first_token_match(head, _DEVICE_C2PA_PLATFORM)
|
||||
|
||||
|
||||
# C2PA signers that are an editing app or AI-capable device rather than a
|
||||
# verified-capture camera. Unlike `_DEVICE_C2PA_PLATFORM`, these do NOT feed the
|
||||
# camera-vs-AI integrity clash (rule 2 in `_integrity_clashes`): a Galaxy phone
|
||||
@@ -474,10 +724,7 @@ _SIGNER_C2PA_PLATFORM: tuple[tuple[bytes, str], ...] = (
|
||||
|
||||
def _signer_platform(head: bytes) -> str | None:
|
||||
"""Map a C2PA editing-app / AI-capable-device signer token to a platform."""
|
||||
for token, platform in _SIGNER_C2PA_PLATFORM:
|
||||
if token in head:
|
||||
return platform
|
||||
return None
|
||||
return _first_token_match(head, _SIGNER_C2PA_PLATFORM)
|
||||
|
||||
|
||||
def _attribute_platform(issuers: list[str], *, is_ai: bool = True) -> str | None:
|
||||
@@ -503,9 +750,17 @@ def _attribute_platform(issuers: list[str], *, is_ai: bool = True) -> str | None
|
||||
return None
|
||||
|
||||
|
||||
def _claim_generator_platform(generator: str | None) -> str | None:
|
||||
"""Resolve a distinctive C2PA claim generator to its user-facing product."""
|
||||
if not generator:
|
||||
return None
|
||||
lowered = generator.lower()
|
||||
return next((platform for token, platform in C2PA_CLAIM_GENERATOR_PLATFORMS if token in lowered), None)
|
||||
|
||||
|
||||
# Coarse origin-vendor normalization for integrity-clash detection. Two signals
|
||||
# that resolve to the SAME key are consistent (a C2PA "Google (Gemini)" issuer
|
||||
# and a SynthID-Google proxy, or Adobe Firefly + its Adobe TrustMark soft
|
||||
# and Google SynthID provenance, or Adobe Firefly + its Adobe TrustMark soft
|
||||
# binding); two DIFFERENT keys from independent generator stamps are a
|
||||
# contradiction (a C2PA OpenAI manifest on an image whose EXIF says "Ideogram
|
||||
# AI"). Substring match on the lowercased platform/detail string; first hit wins,
|
||||
@@ -521,6 +776,7 @@ _AI_VENDOR_TOKENS: tuple[tuple[str, str], ...] = (
|
||||
("google", "Google"),
|
||||
("firefly", "Adobe"),
|
||||
("adobe", "Adobe"),
|
||||
("copilot", "Microsoft"),
|
||||
("bing", "Microsoft"),
|
||||
("designer", "Microsoft"),
|
||||
("microsoft", "Microsoft"),
|
||||
@@ -547,6 +803,8 @@ _AI_VENDOR_TOKENS: tuple[tuple[str, str], ...] = (
|
||||
("fal-ai", "fal.ai"),
|
||||
("bria", "Bria"),
|
||||
("apple photos clean up", "Apple"),
|
||||
("luma ai", "Luma AI"),
|
||||
("lumalabs", "Luma AI"),
|
||||
)
|
||||
|
||||
|
||||
@@ -563,14 +821,14 @@ def _vendor_of(text: str | None) -> str | None:
|
||||
|
||||
# Clash-detection provenance sources. Rule 1 (below) flags two AI vendors only
|
||||
# when they come from *independent* signals. The C2PA issuer attribution and the
|
||||
# SynthID proxy are NOT independent -- the proxy is inferred from the same C2PA
|
||||
# SynthID evidence are NOT independent -- both are read from the same C2PA
|
||||
# manifest -- so they share one source. A multi-actor manifest (a product wrapping
|
||||
# another vendor's engine, e.g. Microsoft+OpenAI or Microsoft+Google; or an edit
|
||||
# chain like Adobe over a Gemini original) legitimately names several vendors in
|
||||
# one valid chain and must not read as spoofing. Families not listed here are each
|
||||
# their own independent source (EXIF/XMP generator, IPTC AISystemUsed, AIGC, ...).
|
||||
# The single C2PA-manifest source shared by the issuer attribution and the SynthID
|
||||
# proxy (both inferred from the same embedded manifest). Rule 2 keys off it too:
|
||||
# evidence (both read from the same embedded manifest). Rule 2 keys off it too:
|
||||
# the camera device label is read from this manifest, so an AI marker is a clash
|
||||
# only when its source differs from this (i.e. it is genuinely independent).
|
||||
_C2PA_MANIFEST_SOURCE = "c2pa_manifest"
|
||||
@@ -596,7 +854,7 @@ def _integrity_clashes(
|
||||
Args:
|
||||
ai_vendors: family name -> normalized AI-origin vendor, one entry per
|
||||
generator-stamped signal (C2PA issuer when the source is AI, SynthID
|
||||
proxy, EXIF/XMP generator tag, IPTC AISystemUsed, xAI, AIGC label).
|
||||
provenance, EXIF/XMP generator tag, IPTC AISystemUsed, xAI, AIGC label).
|
||||
camera_label: a camera/verified-capture C2PA device platform, if one was
|
||||
identified (Pixel, Leica, Sony, Nikon, Truepic), else None.
|
||||
camera_has_ai_marker: True when an AI-generation stamp coexists with the
|
||||
@@ -610,7 +868,7 @@ def _integrity_clashes(
|
||||
# Rule 1: two genuinely INDEPENDENT signals naming different AI vendors. Two
|
||||
# families clash only when they belong to different provenance sources (see
|
||||
# _CLASH_SOURCE) AND name different vendors -- so multiple vendors named within
|
||||
# one C2PA manifest (c2pa issuer + synthid proxy) do not flag.
|
||||
# one C2PA manifest (C2PA issuer + SynthID provenance) do not flag.
|
||||
# The generic TC260 AIGC label is a Chinese regulatory "this is AI" stamp. When a
|
||||
# Chinese TC260-applying vendor (ByteDance) is ALSO attributed, the label is that
|
||||
# vendor's own stamp on its own output, so attribute it to that vendor -- a legit
|
||||
@@ -643,7 +901,7 @@ def _integrity_clashes(
|
||||
# a contradiction. A device that both captures and runs on-device generative
|
||||
# AI (Google Pixel Magic Editor / Pixel Studio) records the capture and the
|
||||
# AI edit in ONE manifest, so the AI vendor is named only from that same
|
||||
# manifest (c2pa issuer + synthid proxy) -- a legitimate edit chain, not a
|
||||
# manifest (C2PA issuer + SynthID provenance) -- a legitimate edit chain, not a
|
||||
# spoof. An EXIF/XMP generator, IPTC field, TC260 AIGC label, or second
|
||||
# manifest naming AI on a camera capture is the real laundering tell.
|
||||
independent_ai_marker = any(grp != _C2PA_MANIFEST_SOURCE for grp in source.values())
|
||||
@@ -678,16 +936,18 @@ def _visible_sparkle(image_path: Path, *, image: NDArray[Any] | None = None) ->
|
||||
# Gemini-sparkle phrasing. These are the stripped-metadata visual fallback for
|
||||
# the China-served ByteDance generators (normally also caught by the TC260 AIGC
|
||||
# metadata label); the per-engine detection thresholds live in the registry.
|
||||
_VISIBLE_MARK_PLATFORM = {
|
||||
"doubao": "ByteDance Doubao (visible 豆包AI生成 mark detected)",
|
||||
"jimeng": "ByteDance Jimeng / Dreamina (visible 即梦AI mark detected)",
|
||||
"qwen": "Alibaba Tongyi Qianwen (visible 千问AI生成 mark detected)",
|
||||
"kling": "Kuaishou Kling (visible 可灵AI 3.0 mark detected)",
|
||||
"yuanbao": "Tencent Yuanbao (visible 元宝 / AI生成 mark detected)",
|
||||
"samsung": "Samsung Galaxy AI (visible 'Contenuti generati dall'AI' mark detected)",
|
||||
"runninghub": "RunningHub (visible RunningHub AI生成 mark detected)",
|
||||
"baidu": "Baidu (visible 百度 AI生成 mark detected)",
|
||||
"liblib": "LibLibAI (visible LibLibAI mark detected)",
|
||||
# Text mark -> the platform sentence this report prints when that mark is the strongest
|
||||
# evidence, DERIVED from the registry rows so registering a mark is one edit. It was a
|
||||
# hand-maintained copy, and that class of copy is how LibLibAI ended up registered but
|
||||
# missing from the pill veto. Insertion order is the registry's, which is what fixes the
|
||||
# scan order below. The Gemini sparkle and the capture-less pill carry no platform of
|
||||
# their own (`KnownMark.platform is None`) and are excluded here: the sparkle has its
|
||||
# own higher-confidence `_visible_sparkle` path.
|
||||
#
|
||||
# Safe at module scope: `watermark_registry` is already imported above for
|
||||
# GEMINI_SPARKLE_TRUST_CONF, and it is deliberately cv2-free at import time.
|
||||
_VISIBLE_MARK_PLATFORM: dict[str, str] = {
|
||||
mark.key: mark.platform for mark in _known_marks() if mark.platform is not None
|
||||
}
|
||||
|
||||
|
||||
@@ -724,15 +984,20 @@ def _visible_text_marks(image_path: Path, *, image: NDArray[Any] | None = None)
|
||||
return detections
|
||||
|
||||
|
||||
def _invisible_watermark(image_path: Path) -> str | None:
|
||||
def _invisible_watermark(image_path: Path, decode: _SharedDecode) -> str | None:
|
||||
"""Open invisible-watermark scheme name (SD/SDXL/FLUX) or None.
|
||||
|
||||
Optional: needs the torch-free DWT-DCT decoder (extra ``detect``). Returns
|
||||
None if it is not installed or no known watermark decodes.
|
||||
"""
|
||||
from remove_ai_watermarks.invisible_watermark import detect_invisible_watermark
|
||||
from remove_ai_watermarks.invisible_watermark import detect_invisible_watermark, is_available
|
||||
|
||||
return detect_invisible_watermark(image_path)
|
||||
if not is_available():
|
||||
return None
|
||||
# `decode.get()` re-raises a decode failure exactly as the old unguarded
|
||||
# `imread` inside the detector did -- `has_invisible_target` needs that to reach
|
||||
# its fail-safe rather than silently reporting "no signal".
|
||||
return detect_invisible_watermark(image_path, image=decode.get())
|
||||
|
||||
|
||||
def _trustmark(image_path: Path) -> str | None:
|
||||
@@ -746,27 +1011,79 @@ def _trustmark(image_path: Path) -> str | None:
|
||||
return detect_trustmark(image_path)
|
||||
|
||||
|
||||
class _SharedDecode:
|
||||
"""One decode of the source pixels, shared by every detector in a single report.
|
||||
|
||||
``identify`` used to decode the file three times. This holder unifies TWO of
|
||||
them -- the DWT-DCT detector and the visible-mark stage, whose own docstring
|
||||
already promised a single shared array. TrustMark keeps its own Pillow decode
|
||||
on purpose and is NOT served from here: cv2 and Pillow disagree on EXIF
|
||||
orientation and on 16-bit PNG, so feeding it this array would change what it
|
||||
decodes. An install carrying the optional ``trustmark`` extra therefore still
|
||||
pays two decodes, not one.
|
||||
|
||||
Two accessors, because the two arms need OPPOSITE failure handling:
|
||||
|
||||
* :meth:`get_or_none` swallows a decode failure and logs it. That is the visible
|
||||
arm's historical behavior -- no cv2, no visible marks, verdict unchanged.
|
||||
* :meth:`get` RE-RAISES it. The invisible arm never caught a decode error, and
|
||||
``has_invisible_target`` converts that exception into its documented fail-safe
|
||||
``True``. Swallowing it here would silently skip a diffusion scrub on a file
|
||||
that used to get one -- leaving a watermark on a paid removal.
|
||||
|
||||
Per-CALL only: constructed inside ``_identify_from_evidence`` and discarded with
|
||||
it, so an in-place rewrite between calls can never be answered from a stale array.
|
||||
"""
|
||||
|
||||
__slots__ = ("_done", "_error", "_image", "_path")
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
self._path = path
|
||||
self._done = False
|
||||
self._image: NDArray[Any] | None = None
|
||||
self._error: Exception | None = None
|
||||
|
||||
def _decode(self) -> None:
|
||||
if self._done:
|
||||
return
|
||||
self._done = True
|
||||
try:
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
|
||||
self._image = imread(self._path)
|
||||
except Exception as exc: # cv2 missing / unreadable container
|
||||
self._error = exc
|
||||
|
||||
def get(self) -> NDArray[Any] | None:
|
||||
"""The decoded array; re-raises a decode failure, None only on a clean miss."""
|
||||
self._decode()
|
||||
if self._error is not None:
|
||||
raise self._error
|
||||
return self._image
|
||||
|
||||
def get_or_none(self) -> NDArray[Any] | None:
|
||||
"""The decoded array, or None when it could not be decoded at all."""
|
||||
self._decode()
|
||||
if self._error is not None:
|
||||
logger.debug("visible-mark decode unavailable: %s", self._error)
|
||||
return None
|
||||
return self._image
|
||||
|
||||
|
||||
def _collect_visible_signals(
|
||||
image_path: Path,
|
||||
signals: list[Signal],
|
||||
watermarks: list[str],
|
||||
platform: str | None,
|
||||
decode: _SharedDecode,
|
||||
) -> str | None:
|
||||
"""Decode once, append every trusted visible-mark signal, and return platform.
|
||||
"""Append every trusted visible-mark signal and return platform.
|
||||
|
||||
Keeping this stage separate from metadata aggregation makes the optional cv2
|
||||
boundary explicit and guarantees that all visible detectors share one decoded
|
||||
BGR array. A decode failure preserves the detectors' historical fallback/no-op
|
||||
behavior.
|
||||
All visible detectors share the one decoded BGR array held by ``decode`` (which
|
||||
the invisible detectors have usually already paid for). A decode failure
|
||||
preserves the detectors' historical fallback/no-op behavior.
|
||||
"""
|
||||
image: NDArray[Any] | None = None
|
||||
try:
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
|
||||
image = imread(image_path)
|
||||
except Exception as exc: # cv2 missing - detectors fall back / no-op
|
||||
logger.debug("visible-mark decode unavailable: %s", exc)
|
||||
return platform
|
||||
image = decode.get_or_none()
|
||||
if image is None:
|
||||
return platform
|
||||
|
||||
@@ -800,6 +1117,10 @@ def _identify_from_evidence(
|
||||
if (check_visible or check_invisible) and image_path is None:
|
||||
raise ValueError("Pixel-backed checks require image_path")
|
||||
pixel_path = image_path
|
||||
# One decode for every pixel detector in this report. Built here, per call, so it
|
||||
# dies with the report -- an in-place rewrite between two calls cannot be answered
|
||||
# from a stale array. Lazy inside, so a metadata-only report never decodes at all.
|
||||
decode = _SharedDecode(pixel_path) if pixel_path is not None else _SharedDecode(evidence.path)
|
||||
|
||||
info = evidence.c2pa_info
|
||||
meta = evidence.ai_metadata
|
||||
@@ -814,31 +1135,41 @@ def _identify_from_evidence(
|
||||
# score, the latter can be a by-product of our own SDXL removal pass, so
|
||||
# neither is a trustworthy "the generator stamped its identity" claim.
|
||||
ai_vendor_claims: dict[str, str] = {}
|
||||
camera_label = _device_platform(head)
|
||||
signer_label = _signer_platform(head)
|
||||
# The vendor registries match short raw substrings, so they read the container's
|
||||
# metadata rather than its pixels -- see `_metadata_region`. Every other check
|
||||
# below keeps the full buffer: their markers are long and distinctive.
|
||||
region = _metadata_region(head)
|
||||
camera_label = _device_platform(region)
|
||||
signer_label = _signer_platform(region)
|
||||
|
||||
# ── C2PA Content Credentials ────────────────────────────────────
|
||||
has_c2pa = bool(info) or c2pa_marker_in(head)
|
||||
issuers = [info["issuer"]] if info.get("issuer") else _issuers_in(head)
|
||||
c2pa_validation = _c2pa_validation(info)
|
||||
c2pa_level = _c2pa_credential_level(info)
|
||||
c2pa_usable = c2pa_level != "invalid"
|
||||
failed_c2pa_codes = [
|
||||
str(code)
|
||||
for code in cast("list[object]", info.get("c2pa_validation_codes", []))
|
||||
if "mismatch" in str(code) or "invalid" in str(code)
|
||||
]
|
||||
issuers = [info["issuer"]] if info.get("issuer") else _issuers_in(region)
|
||||
# Full AI generation (trainedAlgorithmicMedia) vs an AI-enhanced real photo
|
||||
# (compositeWithTrainedAlgorithmicMedia). The structured kind is parsed once in
|
||||
# _internal.c2pa._populate_registry_fields (covers PNG + any container the c2pa-python
|
||||
# _internal.c2pa._structured_manifest_fields (covers PNG + any container the c2pa-python
|
||||
# reader handles); fall back to a raw head scan for the non-PNG raw-blob path
|
||||
# where extract_c2pa_info returns {}. Full generation wins when both appear.
|
||||
c2pa_source_kind = info.get("ai_source_kind")
|
||||
if c2pa_source_kind is None:
|
||||
if b"trainedAlgorithmicMedia" in head:
|
||||
c2pa_source_kind = "generated"
|
||||
elif b"compositeWithTrainedAlgorithmicMedia" in head:
|
||||
c2pa_source_kind = "enhanced"
|
||||
source_kind = _metadata_source_kind(info, head)
|
||||
# An identity-AI issuer (a pure-generator brand like Dreamina) asserts AI even
|
||||
# without a digitalSourceType -- some ByteDance/Dreamina manifests ship no
|
||||
# trainedAlgorithmicMedia, so the registered generator name is the only signal.
|
||||
# Restricted to the ``asserts_ai`` vendors (distinctive brand strings), so it
|
||||
# does not reopen the incidental-mention problem the common-word issuers have.
|
||||
issuer_blob = " ".join(issuers)
|
||||
c2pa_identity_ai = has_c2pa and any(org in issuer_blob for org in C2PA_IDENTITY_AI_ORGS)
|
||||
c2pa_is_ai = c2pa_source_kind is not None or c2pa_identity_ai
|
||||
c2pa_identity_ai = has_c2pa and (
|
||||
bool(info.get("c2pa_identity_ai")) or any(org in issuer_blob for org in C2PA_IDENTITY_AI_ORGS)
|
||||
)
|
||||
c2pa_claims_ai = source_kind is not None or c2pa_identity_ai
|
||||
c2pa_is_ai = c2pa_usable and c2pa_claims_ai
|
||||
# Generator string (for the signal detail): structured for PNG, CBOR-scanned
|
||||
# for other containers. Best-effort -- some manifests key it as
|
||||
# `claim_generator_info` (Pixel), so this can be None even when a device is
|
||||
@@ -846,23 +1177,53 @@ def _identify_from_evidence(
|
||||
generator = (
|
||||
info.get("claim_generator")
|
||||
or cbor_text_after(head, b"claim_generator")
|
||||
or (", ".join(tools) if (tools := _ai_tools_in(head)) else None)
|
||||
or (", ".join(tools) if (tools := _ai_tools_in(region)) else None)
|
||||
)
|
||||
# Platform: a distinctive device/camera token in the manifest wins (it is the
|
||||
# signer/producer), then an editing-app/AI-device signer (Samsung Galaxy,
|
||||
# ASUS Gallery), with the issuer byte-scan only as fallback. The issuer scan
|
||||
# alone mis-attributed real samples (Leica->Truepic timestamp authority,
|
||||
# Nikon->Adobe namespace, Pixel->Google Gemini) -- the token scans fix that.
|
||||
platform = (camera_label or signer_label or _attribute_platform(issuers, is_ai=c2pa_is_ai)) if has_c2pa else None
|
||||
platform = (
|
||||
(
|
||||
camera_label
|
||||
or signer_label
|
||||
or (_claim_generator_platform(generator) if c2pa_is_ai else None)
|
||||
or (_claim_generator_platform(str(info.get("ai_tool"))) if c2pa_is_ai and info.get("ai_tool") else None)
|
||||
or _attribute_platform(issuers, is_ai=c2pa_is_ai)
|
||||
)
|
||||
if has_c2pa and c2pa_usable
|
||||
else None
|
||||
)
|
||||
if has_c2pa:
|
||||
detail = ", ".join(filter(None, [", ".join(issuers), generator, info.get("source_type")]))
|
||||
signals.append(Signal("c2pa", detail or "C2PA manifest present", "high"))
|
||||
watermarks.append(f"C2PA Content Credentials ({', '.join(issuers) or 'unknown signer'})")
|
||||
if c2pa_level == "invalid":
|
||||
suffix = f"; {', '.join(failed_c2pa_codes)}" if failed_c2pa_codes else ""
|
||||
signals.append(Signal("c2pa", f"C2PA manifest present, credential integrity invalid{suffix}", "medium"))
|
||||
watermarks.append("C2PA Content Credentials (invalid asset binding or signature)")
|
||||
caveats.append(_C2PA_INVALID_CAVEAT)
|
||||
else:
|
||||
signals.append(
|
||||
Signal("c2pa", detail or "C2PA manifest present", "high" if c2pa_level == "verified" else "medium")
|
||||
)
|
||||
watermarks.append(f"C2PA Content Credentials ({', '.join(issuers) or 'unknown signer'})")
|
||||
if c2pa_level == "unverified":
|
||||
caveats.append(
|
||||
_C2PA_UNTRUSTED_CAVEAT
|
||||
if info.get("c2pa_integrity") == "valid" and info.get("c2pa_signature") == "valid"
|
||||
else _C2PA_UNVALIDATED_CAVEAT
|
||||
)
|
||||
# Record the AI-origin vendor for clash detection only when the source is
|
||||
# actually AI -- classify the issuer attribution / generator, NOT the
|
||||
# resolved `platform` (which may be a camera device token whose label,
|
||||
# e.g. "Google Pixel", would mis-normalize to an AI vendor).
|
||||
if c2pa_is_ai and (v := (_vendor_of(_attribute_platform(issuers, is_ai=True)) or _vendor_of(generator))):
|
||||
if c2pa_is_ai and (
|
||||
v := (
|
||||
_vendor_of(_attribute_platform(issuers, is_ai=True))
|
||||
or _vendor_of(generator)
|
||||
or _vendor_of(str(info.get("ai_tool", "")))
|
||||
)
|
||||
):
|
||||
ai_vendor_claims["c2pa"] = v
|
||||
|
||||
# ── C2PA cloud-manifest reference (Durable Content Credentials) ─
|
||||
@@ -880,29 +1241,59 @@ def _identify_from_evidence(
|
||||
if platform is None:
|
||||
platform = f"C2PA signer: {cloud_vendor} (cloud manifest)"
|
||||
|
||||
# ── SynthID metadata proxy ──────────────────────────────────────
|
||||
# get_ai_metadata already sets synthid_watermark for both PNG (caBX parser)
|
||||
# and non-PNG (its own synthid_source fallback), so no extra scan is needed.
|
||||
# ── SynthID provenance evidence ─────────────────────────────────
|
||||
# Structured first (the PNG caBX parser and the manifest store both fill
|
||||
# `synthid_watermark`), then the byte scan for the containers that keep the
|
||||
# manifest where no parser reaches it.
|
||||
#
|
||||
# The scan lives HERE, in the verdict, and not in extraction, for the same reason
|
||||
# `soft_binding` below does: extraction has two implementations -- one reading a
|
||||
# file, one reading a portable record -- and a rule that lives in only one of them
|
||||
# is a rule the other silently lacks. It did: 74 corpus images reported SynthID
|
||||
# through `identify` and not through the record, because `get_ai_metadata`'s own
|
||||
# fallback has no counterpart on the record side. `get_ai_metadata` keeps its copy
|
||||
# for its own callers; the verdict no longer depends on which extractor ran.
|
||||
synthid = meta.get("synthid_watermark")
|
||||
# The literal byte checks mirror `metadata.synthid_source` exactly rather than
|
||||
# reusing the derived `has_c2pa` / `source_kind` above, which are broader:
|
||||
# the file path's answer must not move.
|
||||
trained_source = b"trainedAlgorithmicMedia" in head or b"TrainedAlgorithmicMedia" in head
|
||||
if not synthid and trained_source and c2pa_marker_in(head) and (vendors := synthid_evidence_vendors_in(region)):
|
||||
synthid = synthid_verdict(", ".join(vendors))
|
||||
if synthid:
|
||||
watermarks.append(f"SynthID watermark, inferred from C2PA metadata ({synthid})")
|
||||
watermarks.append(
|
||||
f"SynthID watermark ({synthid})"
|
||||
if c2pa_usable
|
||||
else f"SynthID watermark claimed by invalid C2PA credentials ({synthid})"
|
||||
)
|
||||
caveats.append(_SYNTHID_CAVEAT)
|
||||
if _vendor_of(synthid) == "OpenAI":
|
||||
caveats.append(_OPENAI_CAVEAT)
|
||||
if v := _vendor_of(synthid):
|
||||
if c2pa_usable and (v := _vendor_of(synthid)):
|
||||
ai_vendor_claims["synthid"] = v
|
||||
|
||||
# ── C2PA soft-binding: a named forensic/third-party watermark vendor ─
|
||||
# (Adobe TrustMark, Digimarc, Imatag, ...). Present in the manifest even when
|
||||
# the watermark itself can't be decoded; names whose watermark stamped the pixels.
|
||||
soft_binding = meta.get("soft_binding") or (", ".join(v) if (v := soft_binding_vendors_in(head)) else None)
|
||||
soft_binding = meta.get("soft_binding") or (", ".join(v) if (v := soft_binding_vendors_in(region)) else None)
|
||||
if soft_binding:
|
||||
signals.append(Signal("soft_binding", f"C2PA soft binding: {soft_binding}", "high"))
|
||||
watermarks.append(f"Forensic watermark soft binding ({soft_binding})")
|
||||
soft_binding_algorithm = meta.get("soft_binding_algorithm") or info.get("soft_binding_algorithm")
|
||||
soft_binding_value = meta.get("soft_binding_value") or info.get("soft_binding_value")
|
||||
soft_binding_details = "; ".join(
|
||||
str(value) for value in (soft_binding, soft_binding_algorithm, soft_binding_value) if value
|
||||
)
|
||||
signals.append(
|
||||
Signal(
|
||||
"soft_binding",
|
||||
f"C2PA soft binding: {soft_binding_details}",
|
||||
"high" if c2pa_level == "verified" else "medium",
|
||||
)
|
||||
)
|
||||
watermarks.append(f"Forensic watermark soft binding ({soft_binding_details})")
|
||||
caveats.append(_SOFT_BINDING_CAVEAT)
|
||||
|
||||
# ── IPTC "Made with AI" (Meta etc.), only meaningful without C2PA ─
|
||||
iptc = any(m in head for m in IPTC_AI_MARKERS)
|
||||
if iptc and not has_c2pa:
|
||||
standalone_iptc = iptc and not has_c2pa
|
||||
if standalone_iptc:
|
||||
signals.append(Signal("iptc", "digitalSourceType (Made with AI)", "high"))
|
||||
watermarks.append("IPTC digitalSourceType (Made with AI)")
|
||||
caveats.append(_IPTC_ONLY_CAVEAT)
|
||||
@@ -1006,7 +1397,7 @@ def _identify_from_evidence(
|
||||
|
||||
# ── Open invisible watermark (SD / SDXL / FLUX, dwtDct) ──────────
|
||||
# Public decoder, no key -- a definitive embedded signal on pristine files.
|
||||
if check_invisible and pixel_path is not None and (scheme := _invisible_watermark(pixel_path)) is not None:
|
||||
if check_invisible and pixel_path is not None and (scheme := _invisible_watermark(pixel_path, decode)) is not None:
|
||||
signals.append(Signal("invisible_watermark", scheme, "high"))
|
||||
watermarks.append(f"Open invisible watermark: {scheme}")
|
||||
caveats.append(_INVISIBLE_WM_CAVEAT)
|
||||
@@ -1028,8 +1419,18 @@ def _identify_from_evidence(
|
||||
exif_gen = any(s.name == "exif_generator" for s in signals)
|
||||
xai_sig = any(s.name == "xai_signature" for s in signals)
|
||||
ai_from_metadata = bool(
|
||||
(has_c2pa and (c2pa_is_ai or synthid))
|
||||
or iptc
|
||||
(has_c2pa and c2pa_usable and (c2pa_is_ai or synthid))
|
||||
or standalone_iptc
|
||||
or iptc_ai
|
||||
or aigc
|
||||
or local_keys
|
||||
or invisible_wm
|
||||
or exif_gen
|
||||
or xai_sig
|
||||
)
|
||||
high_ai_from_metadata = bool(
|
||||
(has_c2pa and c2pa_level == "verified" and (c2pa_is_ai or synthid))
|
||||
or standalone_iptc
|
||||
or iptc_ai
|
||||
or aigc
|
||||
or local_keys
|
||||
@@ -1039,7 +1440,7 @@ def _identify_from_evidence(
|
||||
)
|
||||
|
||||
if check_visible and pixel_path is not None:
|
||||
platform = _collect_visible_signals(pixel_path, signals, watermarks, platform)
|
||||
platform = _collect_visible_signals(pixel_path, signals, watermarks, platform, decode)
|
||||
|
||||
visible_only = any(s.name.startswith("visible_") for s in signals) and not ai_from_metadata
|
||||
hf_only = bool(hf_job) and not ai_from_metadata
|
||||
@@ -1047,7 +1448,7 @@ def _identify_from_evidence(
|
||||
|
||||
if ai_from_metadata:
|
||||
is_ai: bool | None = True
|
||||
confidence = "high"
|
||||
confidence = "high" if high_ai_from_metadata else "medium"
|
||||
elif visible_only or hf_only or samsung_only:
|
||||
is_ai = True
|
||||
confidence = "medium"
|
||||
@@ -1056,7 +1457,17 @@ def _identify_from_evidence(
|
||||
confidence = "none"
|
||||
|
||||
# ── Integrity clashes: contradictions between independent signals ─
|
||||
clashes = _integrity_clashes(ai_vendor_claims, camera_label, camera_has_ai_marker=bool(ai_vendor_claims))
|
||||
clashes = _integrity_clashes(
|
||||
ai_vendor_claims,
|
||||
camera_label if c2pa_usable else None,
|
||||
camera_has_ai_marker=bool(ai_vendor_claims),
|
||||
)
|
||||
if c2pa_level == "invalid":
|
||||
clashes.insert(
|
||||
0,
|
||||
"C2PA credentials failed integrity validation"
|
||||
+ (f": {', '.join(failed_c2pa_codes)}" if failed_c2pa_codes else "."),
|
||||
)
|
||||
|
||||
caveats.append(_STRIP_CAVEAT)
|
||||
# De-duplicate while preserving order.
|
||||
@@ -1067,20 +1478,49 @@ def _identify_from_evidence(
|
||||
is_ai_generated=is_ai,
|
||||
platform=platform,
|
||||
confidence=confidence,
|
||||
# Only meaningful when the AI verdict actually came from the C2PA source
|
||||
# type; a non-C2PA AI signal (IPTC/AIGC/local gen) leaves it None.
|
||||
ai_source_kind=c2pa_source_kind if (is_ai and has_c2pa) else None,
|
||||
# Meaningful for the same digitalSourceType whether carried by C2PA or a
|
||||
# standalone IPTC/XMP label. Other AI signals leave it None.
|
||||
ai_source_kind=(source_kind if (is_ai and ((has_c2pa and c2pa_usable) or standalone_iptc)) else None),
|
||||
ai_from_metadata=ai_from_metadata,
|
||||
watermarks=watermarks,
|
||||
signals=signals,
|
||||
caveats=caveats,
|
||||
integrity_clashes=clashes,
|
||||
c2pa_validation=c2pa_validation,
|
||||
)
|
||||
|
||||
|
||||
def identify_from_evidence(evidence: ProvenanceEvidence) -> ProvenanceReport:
|
||||
"""Build a metadata-only provenance verdict without reopening the source."""
|
||||
return _identify_from_evidence(evidence)
|
||||
def identify_from_evidence(
|
||||
evidence: ProvenanceEvidence,
|
||||
*,
|
||||
image_path: Path | None = None,
|
||||
check_visible: bool = False,
|
||||
check_invisible: bool = False,
|
||||
) -> ProvenanceReport:
|
||||
"""Build a provenance verdict from already-extracted evidence.
|
||||
|
||||
Metadata-only by default -- the source is never reopened. Pass ``image_path`` with
|
||||
``check_visible`` / ``check_invisible`` to add the pixel-backed detectors on top of
|
||||
the SAME evidence, which is how a caller that asks the file two provenance questions
|
||||
(which vendor is confirmed, and is there an invisible target) pays for the metadata
|
||||
extraction once.
|
||||
"""
|
||||
return _identify_from_evidence(
|
||||
evidence,
|
||||
image_path=image_path,
|
||||
check_visible=check_visible,
|
||||
check_invisible=check_invisible,
|
||||
)
|
||||
|
||||
|
||||
def identify_metadata_record(record: dict[str, Any], *, path: Path) -> ProvenanceReport:
|
||||
"""Build a metadata-only verdict from a portable metadata record.
|
||||
|
||||
This is the service-integration entry point: the source file is never opened,
|
||||
and callers receive the same verdict as the explicit
|
||||
``evidence_from_metadata_record`` / ``identify_from_evidence`` sequence.
|
||||
"""
|
||||
return identify_from_evidence(evidence_from_metadata_record(record, path=path))
|
||||
|
||||
|
||||
def identify(
|
||||
@@ -1121,13 +1561,16 @@ def has_invisible_target(image_path: Path) -> bool:
|
||||
"""True when a locally-detectable invisible/metadata AI signal is present.
|
||||
|
||||
The decision gate for the diffusion scrub (``invisible`` / ``all`` / ``batch``):
|
||||
regenerating pixels removes an invisible watermark (SynthID, open DWT-DCT,
|
||||
TrustMark) but degrades a real photo, so it must not run when there is nothing
|
||||
to remove. Runs :func:`identify` with ``check_visible=False`` -- a visible mark
|
||||
is handled by the separate visible pass and is NOT a diffusion target -- and
|
||||
``check_invisible=True`` so an open watermark counts. Returns
|
||||
``report.ai_from_metadata`` (C2PA AI issuer / SynthID proxy, IPTC, AIGC, local
|
||||
gen params, EXIF/xAI, open DWT-DCT / TrustMark).
|
||||
regenerating pixels removes an AI-specific invisible watermark (SynthID,
|
||||
open DWT-DCT) but degrades a real photo, so it must not run when there is
|
||||
nothing to remove. It runs the same evidence pipeline as :func:`identify`
|
||||
with visible checks disabled and invisible checks enabled, so a visible mark
|
||||
is handled by the separate visible pass and is NOT a diffusion target. Returns
|
||||
True for ``report.ai_from_metadata`` (C2PA AI issuer / SynthID provenance,
|
||||
IPTC, AIGC, local gen params, EXIF/xAI, or open DWT-DCT), and also when an
|
||||
invalid C2PA claim retains an AI or watermark removal hint. TrustMark alone
|
||||
does not trigger the scrub because it also protects human-authored work and
|
||||
therefore is not an AI signal by itself.
|
||||
|
||||
IMPORTANT -- this cannot prove a pixel SynthID is absent: SynthID is detectable
|
||||
only through its C2PA proxy, so a metadata-stripped AI image reads as no signal
|
||||
@@ -1138,8 +1581,17 @@ def has_invisible_target(image_path: Path) -> bool:
|
||||
watermark on a paid removal is worse than over-regenerating a clean image.
|
||||
"""
|
||||
try:
|
||||
report = identify(image_path, check_visible=False, check_invisible=True)
|
||||
evidence = extract_provenance_evidence(image_path)
|
||||
report = _identify_from_evidence(
|
||||
evidence,
|
||||
image_path=image_path,
|
||||
check_visible=False,
|
||||
check_invisible=True,
|
||||
)
|
||||
except Exception: # unreadable / detector error -> do not skip the removal
|
||||
logger.debug("has_invisible_target: identify failed, defaulting to run", exc_info=True)
|
||||
return True
|
||||
return report.ai_from_metadata
|
||||
# An asset edit can invalidate the C2PA binding while leaving the declared pixel
|
||||
# watermark intact. Keep the removal gate fail-safe without promoting that broken
|
||||
# claim back into the provenance verdict.
|
||||
return report.ai_from_metadata or c2pa_info_has_removal_hint(evidence.c2pa_info)
|
||||
@@ -89,6 +89,19 @@ def _pil_read(path: str | Path, flags: int) -> NDArray[Any] | None:
|
||||
return None
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read ``path`` as a BGR ndarray, raising instead of returning ``None``.
|
||||
|
||||
:func:`imread` keeps cv2's ``None``-on-failure contract because the removal paths
|
||||
branch on it. Scripts and tests want the opposite -- fail loudly at the read -- so
|
||||
they call this. Each vendor engine used to carry its own verbatim copy.
|
||||
"""
|
||||
image = imread(path)
|
||||
if image is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return image
|
||||
|
||||
|
||||
def to_bgr(image: NDArray[Any]) -> NDArray[Any]:
|
||||
"""Return a 3-channel BGR view of ``image``, promoting grayscale and BGRA.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Diffusion engine for regenerating images that carry invisible AI watermarks.
|
||||
|
||||
This module requires the 'gpu' extra dependencies:
|
||||
uv pip install 'remove-ai-watermarks[diffusion]'
|
||||
Requires the 'qwen-zimage' extra and a CUDA device:
|
||||
uv pip install 'remove-ai-watermarks[qwen-zimage]'
|
||||
"""
|
||||
|
||||
# cv2/torch boundary: this engine wraps cv2 (resize/imwrite/cvtColor) and the
|
||||
@@ -14,12 +14,13 @@ import logging
|
||||
import os
|
||||
import warnings
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from ._internal.watermark_profiles import (
|
||||
DEFAULT_MODEL_ID as DEFAULT_SDXL_MODEL_ID,
|
||||
)
|
||||
from ._internal.watermark_profiles import (
|
||||
DEFAULT_PROFILE,
|
||||
QWEN_ZIMAGE_PROFILE,
|
||||
REMOVAL_MODULES,
|
||||
resolve_adaptive_polish,
|
||||
resolve_seed,
|
||||
)
|
||||
|
||||
@@ -40,27 +41,30 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
"""Check if invisible watermark removal dependencies are installed."""
|
||||
"""Whether the dependencies for a real removal run are installed.
|
||||
|
||||
Shares :data:`REMOVAL_MODULES` with the remover's own precondition so the two
|
||||
cannot drift. When they did, a torch+diffusers-only environment passed this gate
|
||||
and then died at the DiffSynth face stage.
|
||||
"""
|
||||
from .optional_deps import module_available
|
||||
|
||||
return module_available("diffusers", "torch")
|
||||
return module_available(*REMOVAL_MODULES)
|
||||
|
||||
|
||||
def _target_size(width: int, height: int, max_resolution: int, min_resolution: int = 0) -> tuple[int, int] | None:
|
||||
def _target_size(width: int, height: int, max_resolution: int) -> tuple[int, int] | None:
|
||||
"""Compute the (width, height) to process at, or None for native.
|
||||
|
||||
Two opposite long-side adjustments, in precedence order:
|
||||
One long-side adjustment: if it exceeds ``max_resolution``, scale DOWN to it
|
||||
(integer-truncated, matching the PIL ``resize`` call site). 0/negative = no cap.
|
||||
Set only to bound GPU/MPS memory on very large inputs (issue #10).
|
||||
|
||||
- ``max_resolution`` (cap): if the long side exceeds it, scale DOWN to it
|
||||
(integer-truncated, matching the PIL ``resize`` call site). 0/negative = no
|
||||
cap. Set only to bound GPU/MPS memory on very large inputs (issue #10).
|
||||
- ``min_resolution`` (floor): else if the long side is below it, scale UP to it
|
||||
(rounded) so SDXL img2img runs near its ~1024 training resolution instead of
|
||||
degrading on a tiny latent (a 381x512 portrait distorts badly at native).
|
||||
The output is restored to the original size by the caller, so the floor is a
|
||||
transparent quality boost. 0 = no floor. Skipped on a ``min > max`` misconfig.
|
||||
There was also a ``min_resolution`` floor that scaled small inputs UP toward
|
||||
SDXL's ~1024 training size. It went with the SDXL profiles: both surviving
|
||||
profiles run at native geometry, so the floor was forced to 0 on every path and
|
||||
could not fire.
|
||||
|
||||
Returns None when neither applies (native resolution). Pure function so the
|
||||
Returns None when the cap does not apply (native resolution). Pure function so the
|
||||
resolution decision is unit-testable without loading the diffusion model.
|
||||
"""
|
||||
long_side = max(width, height)
|
||||
@@ -69,9 +73,6 @@ def _target_size(width: int, height: int, max_resolution: int, min_resolution: i
|
||||
# Clamp the short side to >=1: extreme aspect ratios (e.g. 5000x3 capped
|
||||
# at 1024) would otherwise truncate it to 0 and crash image.resize().
|
||||
return (max(1, int(width * ratio)), max(1, int(height * ratio)))
|
||||
if min_resolution > 0 and long_side < min_resolution and (max_resolution <= 0 or min_resolution <= max_resolution):
|
||||
ratio = min_resolution / long_side
|
||||
return (max(1, round(width * ratio)), max(1, round(height * ratio)))
|
||||
return None
|
||||
|
||||
|
||||
@@ -82,15 +83,10 @@ class InvisibleEngine:
|
||||
to break watermark patterns, and reconstructs via reverse diffusion.
|
||||
"""
|
||||
|
||||
# SDXL base is the default since May 2026; the vendor-adaptive strength
|
||||
# removes the current SynthID (see watermark_profiles + docs/synthid.md).
|
||||
DEFAULT_MODEL_ID = DEFAULT_SDXL_MODEL_ID
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
model_id: str | None = None,
|
||||
device: str | None = None,
|
||||
pipeline: str = "controlnet",
|
||||
pipeline: str = DEFAULT_PROFILE,
|
||||
hf_token: str | None = None,
|
||||
progress_callback: Callable[[str], None] | None = None,
|
||||
controlnet_conditioning_scale: float = 1.0,
|
||||
@@ -99,21 +95,19 @@ class InvisibleEngine:
|
||||
"""Initialize the invisible watermark removal engine.
|
||||
|
||||
Args:
|
||||
model_id: HuggingFace model ID. None = use the SDXL base default.
|
||||
device: Device for inference (auto/cpu/mps/cuda/xpu). None = auto.
|
||||
pipeline: Pipeline profile. "controlnet" (DEFAULT; SDXL + canny ControlNet
|
||||
that preserves text/face structure via edge conditioning while removing
|
||||
SynthID), "sdxl" (plain SDXL img2img, lighter but leaves SynthID on
|
||||
flat-graphic content), or "qwen" (Qwen-Image 20B img2img, best text/
|
||||
structure preservation but CUDA/cloud-class), or "qwen-zimage"
|
||||
(Qwen-Image-2512 Lightning + Canny, then SAM-masked Z-Image face
|
||||
repair; CUDA-only), or "sdxl-zimage" (the same recipe and the same face
|
||||
stage on an SDXL global pass, vendor-adaptive strength because an SDXL
|
||||
global stage needs more of it; CUDA-only). "default" aliases "sdxl".
|
||||
device: Device for inference. Both profiles are CUDA-only, so the
|
||||
usable values are "cuda" and None/"auto" (which detects it);
|
||||
anything else raises rather than falling back.
|
||||
pipeline: Pipeline profile, one of "qwen-zimage" (DEFAULT;
|
||||
Qwen-Image-2512 Lightning + Canny, then SAM-masked Z-Image face repair)
|
||||
or "sdxl-zimage" (the same recipe and the same face stage on an SDXL
|
||||
global pass, vendor-adaptive strength because an SDXL global stage
|
||||
needs more of it). BOTH ARE CUDA-ONLY -- there is no CPU or MPS path
|
||||
for invisible-watermark removal.
|
||||
hf_token: HuggingFace API token.
|
||||
progress_callback: Optional callback for progress messages.
|
||||
controlnet_conditioning_scale: ControlNet structure-preservation
|
||||
strength (controlnet pipeline only).
|
||||
controlnet_conditioning_scale: Canny ControlNet structure-preservation
|
||||
strength on the global stage of both profiles.
|
||||
cpu_offload: Offload model components to CPU between CUDA calls instead
|
||||
of keeping the whole pipeline in VRAM, at the cost of speed. For
|
||||
qwen-zimage, force the face stack to offload instead of using automatic
|
||||
@@ -122,10 +116,7 @@ class InvisibleEngine:
|
||||
|
||||
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
|
||||
|
||||
effective_model = model_id or self.DEFAULT_MODEL_ID
|
||||
|
||||
self._remover = WatermarkRemover(
|
||||
model_id=effective_model,
|
||||
device=device,
|
||||
progress_callback=progress_callback,
|
||||
hf_token=hf_token,
|
||||
@@ -144,113 +135,98 @@ class InvisibleEngine:
|
||||
"""
|
||||
self._remover.preload(global_only=global_only)
|
||||
|
||||
def _esrgan_upscale(self, image: Any, target: tuple[int, int]) -> Any:
|
||||
"""Upscale a PIL image to ``target`` with Real-ESRGAN, else Lanczos.
|
||||
|
||||
Runs Real-ESRGAN at its native factor (on the remover's device, CPU fallback),
|
||||
then resizes to the exact ``target`` with Lanczos. Falls back to a plain Lanczos
|
||||
resize when the ``esrgan`` extra is absent or the model errors.
|
||||
"""
|
||||
import cv2
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks import upscaler
|
||||
|
||||
if not upscaler.is_available():
|
||||
logger.debug("esrgan upscaler requested but the extra is absent; using Lanczos")
|
||||
return image.resize(target, Image.Resampling.LANCZOS)
|
||||
try:
|
||||
bgr = cv2.cvtColor(np.array(image.convert("RGB")), cv2.COLOR_RGB2BGR)
|
||||
big = upscaler.upscale(bgr, device=self._remover.device)
|
||||
if (big.shape[1], big.shape[0]) != target:
|
||||
big = cv2.resize(big, target, interpolation=cv2.INTER_LANCZOS4)
|
||||
return Image.fromarray(cv2.cvtColor(big, cv2.COLOR_BGR2RGB))
|
||||
except Exception as e: # never let an optional upscaler break removal
|
||||
logger.warning("Real-ESRGAN upscale failed (%s); using Lanczos", e)
|
||||
return image.resize(target, Image.Resampling.LANCZOS)
|
||||
|
||||
def remove_watermark(
|
||||
self,
|
||||
image_path: Path,
|
||||
output_path: Path | None = None,
|
||||
strength: float | None = None,
|
||||
num_inference_steps: int | None = None,
|
||||
guidance_scale: float | None = None,
|
||||
seed: int | None = None,
|
||||
humanize: float = 0.0,
|
||||
max_resolution: int = 0,
|
||||
min_resolution: int = 1024,
|
||||
vendor: str | None = None,
|
||||
unsharp: float = 0.0,
|
||||
adaptive_polish: bool = False,
|
||||
upscaler: str = "lanczos",
|
||||
adaptive_polish: bool | None = None,
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
text_manifest: Path | None = None,
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Path:
|
||||
"""Remove invisible watermark from an image.
|
||||
|
||||
Args:
|
||||
image_path: Path to the watermarked image.
|
||||
output_path: Output path (None = overwrite source).
|
||||
strength: Denoising strength (0.0-1.0). None -> the vendor-adaptive
|
||||
default.
|
||||
num_inference_steps: Number of denoising steps. None keeps the existing
|
||||
100-step library default, except qwen-zimage uses its required
|
||||
four-step Lightning schedule.
|
||||
guidance_scale: Classifier-free guidance scale.
|
||||
seed: Random seed for reproducibility. None resolves to 0 for
|
||||
qwen-zimage and stays random for the other profiles.
|
||||
strength: Denoising strength (0.0-1.0). None -> the profile's calibrated
|
||||
default (resolution-adaptive for qwen-zimage, vendor-adaptive for
|
||||
sdxl-zimage).
|
||||
seed: Random seed for reproducibility. None resolves to 0, because both
|
||||
profiles are certified at a fixed seed.
|
||||
humanize: Intensity of Analog Humanizer film grain (0 = off).
|
||||
unsharp: Final unsharp-mask sharpening strength (0 = off, default).
|
||||
Applied last to counter the soft / over-smoothed look of the
|
||||
diffusion pass; ~0.5-0.8 is a safe range, higher risks edge halos.
|
||||
adaptive_polish: When True (the CLI default), restore the input's detail
|
||||
level in the softened output: a capped unsharp + edge-masked grain
|
||||
targeting the input's Laplacian variance. Self-limiting -- a no-op when
|
||||
the output already meets the input's detail level (text/flat graphics),
|
||||
so it only acts on over-smoothed photo/face texture. Runs LAST.
|
||||
adaptive_polish: Restore the input's detail level in the softened
|
||||
output: a capped unsharp + edge-masked grain targeting the input's
|
||||
Laplacian variance. Self-limiting -- a no-op when the output already
|
||||
meets the input's detail level (text/flat graphics), so it only acts on
|
||||
over-smoothed photo/face texture. Runs LAST. None (the default) follows
|
||||
the profile: off for qwen-zimage, on for sdxl-zimage. This resolves
|
||||
through the same ``resolve_adaptive_polish`` the CLI uses, so a library
|
||||
caller and a CLI caller on one profile get the same output.
|
||||
max_resolution: Cap the long side (px) before diffusion. 0 (default)
|
||||
= no cap. Set a positive value only to bound GPU/MPS memory on
|
||||
very large inputs (it reintroduces a lossy downscale->upscale
|
||||
round-trip).
|
||||
min_resolution: Upscale the long side UP to this (px) before diffusion
|
||||
when the input is smaller, so SDXL runs near its ~1024 training
|
||||
resolution (small inputs degrade/distort badly at native). 1024
|
||||
(default) = on; 0 = off. The output is restored to the original
|
||||
input size, so this is a transparent quality boost; it adds time
|
||||
and memory on small inputs. Ignored on a min > max misconfig.
|
||||
upscaler: How to upscale a small input to the ``min_resolution`` floor:
|
||||
``"lanczos"`` (default, cv2, no model download) or ``"esrgan"`` (Real-ESRGAN
|
||||
via the ``esrgan`` extra). Only applies when UPscaling (the floor
|
||||
case); a ``max_resolution`` downscale always uses Lanczos. Falls back
|
||||
to Lanczos if the extra is absent.
|
||||
= no cap. Set a positive value only to bound GPU memory on very large
|
||||
inputs (it reintroduces a lossy downscale->upscale round-trip).
|
||||
tile: Process the diffusion pass in overlapping tiles instead of one
|
||||
forward pass. This retains the input's native dimensions instead
|
||||
of applying ``max_resolution``, but each tile is still regenerated.
|
||||
Engages only when the long side exceeds ``tile_size``.
|
||||
tile_size: Tile dimension in px (default 1024).
|
||||
tile_overlap: Overlap between adjacent tiles in px (default 128).
|
||||
text_manifest: Operator-verified text lines bound to the decoded source
|
||||
pixels. Enables the experimental Qwen-VAE ``vae-glyphs`` post-pass.
|
||||
Requires the ``text-restoration`` extra and the ``qwen-zimage``
|
||||
profile. Incompatible with downscaling, humanize, unsharp, and
|
||||
adaptive polish. Tiling is supported: the VAE donor uses the same
|
||||
overlapping tiles as the global pass, then glyph restore runs on
|
||||
the blended full frame.
|
||||
fidelity_anchor: Blend 15% of the Qwen-VAE donor across the whole frame
|
||||
before glyph restoration. OFF by default since 0.27.1: that global
|
||||
blend was measured to return detector-visible OpenAI SynthID on
|
||||
poster-scale manifests (detected x6 with the anchor vs clean x6
|
||||
without it, base clean; official Content Provenance API,
|
||||
2026-08-19 - docs/text-protection-research.md). ``True`` reproduces
|
||||
the 0.27.0 research behavior. Requires ``text_manifest``.
|
||||
|
||||
Returns:
|
||||
Path to the cleaned image.
|
||||
"""
|
||||
import tempfile
|
||||
|
||||
if num_inference_steps is None:
|
||||
profile = getattr(self._remover, "model_profile", None)
|
||||
num_inference_steps = 4 if profile in {"qwen-zimage", "sdxl-zimage"} else 100
|
||||
profile = getattr(self._remover, "model_profile", "controlnet")
|
||||
seed = resolve_seed(seed, profile)
|
||||
seed = resolve_seed(seed)
|
||||
adaptive_polish = resolve_adaptive_polish(adaptive_polish, self._remover.model_profile)
|
||||
|
||||
if fidelity_anchor and text_manifest is None:
|
||||
raise ValueError("fidelity_anchor requires a text manifest")
|
||||
if text_manifest is not None:
|
||||
if self._remover.model_profile != QWEN_ZIMAGE_PROFILE:
|
||||
raise ValueError("--text-manifest is supported only by the qwen-zimage profile")
|
||||
if max_resolution != 0:
|
||||
raise ValueError("--text-manifest requires --max-resolution 0")
|
||||
if humanize > 0.0 or unsharp > 0.0 or adaptive_polish:
|
||||
raise ValueError("--text-manifest requires humanize=0, unsharp=0, and adaptive polish disabled")
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
if not region_eraser.lama_available():
|
||||
raise RuntimeError(
|
||||
"Verified text restoration requires LaMa. Install: "
|
||||
"pip install 'remove-ai-watermarks[text-restoration]'"
|
||||
)
|
||||
|
||||
from PIL import Image, ImageOps
|
||||
|
||||
# Resolution policy: a max_resolution cap (0 = none) bounds memory on huge
|
||||
# inputs, and a min_resolution floor (1024 = default) upscales tiny inputs so
|
||||
# SDXL img2img runs near its ~1024 training size instead of distorting on a
|
||||
# tiny latent (a 381x512 portrait wrecks at native -- issue #36 follow-up).
|
||||
# The output is restored to orig_size below, so the floor is transparent.
|
||||
# inputs. See _target_size for why it is the only lever left.
|
||||
# Register the HEIF/AVIF opener so a .heic/.avif input (now a SUPPORTED_FORMAT)
|
||||
# decodes here too. The --force skip path bypasses image_io.imread, which is
|
||||
# what would otherwise register it, so a bare Image.open would fail on HEIC.
|
||||
@@ -263,35 +239,22 @@ class InvisibleEngine:
|
||||
# Full-res original, kept for the adaptive-polish detail target (image is
|
||||
# reassigned to the resized copy below; PIL resize returns a new object).
|
||||
reference_pil = image
|
||||
verified_text = None
|
||||
if text_manifest is not None:
|
||||
from remove_ai_watermarks._internal.text_restoration import load_verified_text_manifest
|
||||
|
||||
# qwen-zimage operates at the input's native geometry in its reference graph.
|
||||
# Keep an explicit max cap available for callers, but do not apply the SDXL
|
||||
# 1024px minimum-resolution floor to this profile.
|
||||
effective_min_resolution = (
|
||||
0 if getattr(self._remover, "model_profile", None) in {"qwen-zimage", "sdxl-zimage"} else min_resolution
|
||||
)
|
||||
target = _target_size(
|
||||
image.width,
|
||||
image.height,
|
||||
max_resolution,
|
||||
effective_min_resolution,
|
||||
)
|
||||
verified_text = load_verified_text_manifest(text_manifest, reference_pil)
|
||||
|
||||
# Both profiles run at the input's native geometry, so only the explicit max
|
||||
# cap can move it, and it can only ever scale down.
|
||||
target = _target_size(image.width, image.height, max_resolution)
|
||||
if target is not None:
|
||||
upscaling = max(target) > max(image.width, image.height)
|
||||
if self._progress_callback:
|
||||
reason = (
|
||||
f"min-resolution floor {min_resolution}px"
|
||||
if upscaling
|
||||
else f"max-resolution cap {max_resolution}px"
|
||||
self._progress_callback(
|
||||
f"Downscaling {image.width}x{image.height} to {target[0]}x{target[1]} "
|
||||
f"(max-resolution cap {max_resolution}px)..."
|
||||
)
|
||||
verb = "Upscaling" if upscaling else "Downscaling"
|
||||
self._progress_callback(f"{verb} {image.width}x{image.height} to {target[0]}x{target[1]} ({reason})...")
|
||||
# Real-ESRGAN only helps when UPscaling (the floor case); a downscale cap
|
||||
# always uses Lanczos. _esrgan_upscale falls back to Lanczos if the extra is absent.
|
||||
if upscaling and upscaler == "esrgan":
|
||||
image = self._esrgan_upscale(image, target)
|
||||
else:
|
||||
image = image.resize(target, Image.Resampling.LANCZOS)
|
||||
image = image.resize(target, Image.Resampling.LANCZOS)
|
||||
|
||||
# Always persist to a temp file, even without downscaling: WatermarkRemover
|
||||
# reloads by path, so the EXIF-transposed pixels must be saved or rotation
|
||||
@@ -311,13 +274,13 @@ class InvisibleEngine:
|
||||
image_path=image_path,
|
||||
output_path=output_path,
|
||||
strength=strength,
|
||||
num_inference_steps=num_inference_steps,
|
||||
guidance_scale=guidance_scale,
|
||||
seed=seed,
|
||||
vendor=vendor,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
text_manifest=verified_text,
|
||||
fidelity_anchor=fidelity_anchor,
|
||||
)
|
||||
|
||||
# Post-processing chain: decode the diffusion output ONCE, apply the
|
||||
@@ -383,21 +346,3 @@ class InvisibleEngine:
|
||||
# _tmp_path is always set above (we persist the image unconditionally).
|
||||
if _tmp_path.exists():
|
||||
_tmp_path.unlink()
|
||||
|
||||
def remove_watermark_batch(
|
||||
self,
|
||||
input_dir: Path,
|
||||
output_dir: Path,
|
||||
strength: float | None = None,
|
||||
steps: int | None = None,
|
||||
) -> list[Path]:
|
||||
"""Remove invisible watermarks from all images in a directory."""
|
||||
if steps is None:
|
||||
profile = getattr(self._remover, "model_profile", None)
|
||||
steps = 4 if profile in {"qwen-zimage", "sdxl-zimage"} else 50
|
||||
return self._remover.remove_watermark_batch(
|
||||
input_dir=input_dir,
|
||||
output_dir=output_dir,
|
||||
strength=strength,
|
||||
num_inference_steps=steps,
|
||||
)
|
||||
@@ -29,6 +29,9 @@ from typing import TYPE_CHECKING
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -81,7 +84,7 @@ def _bits_to_bytes(bits: Iterable[object], nbytes: int) -> bytes:
|
||||
return bytes(int(value) for value in packed[:nbytes])
|
||||
|
||||
|
||||
def detect_invisible_watermark(image_path: Path) -> str | None:
|
||||
def detect_invisible_watermark(image_path: Path, *, image: NDArray[Any] | None = None) -> str | None:
|
||||
"""Return the embedding scheme name if a known open watermark is decoded.
|
||||
|
||||
Returns e.g. ``"Stable Diffusion XL"`` / ``"FLUX.2 (Black Forest Labs)"`` /
|
||||
@@ -94,7 +97,10 @@ def detect_invisible_watermark(image_path: Path) -> str | None:
|
||||
from remove_ai_watermarks import image_io
|
||||
from remove_ai_watermarks.dwt_dct import decode_dwt_dct_lengths
|
||||
|
||||
img = image_io.imread(image_path)
|
||||
# ``image`` lets a caller that has already decoded these pixels hand them in
|
||||
# (mirrors gemini_engine.detect_sparkle_confidence). The decoder only reads the
|
||||
# array -- it converts colour spaces into fresh buffers -- so no copy is needed.
|
||||
img = image if image is not None else image_io.imread(image_path)
|
||||
if img is None:
|
||||
return None
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -85,8 +85,6 @@ _CONFIG = TextMarkConfig(
|
||||
min_gw=8,
|
||||
)
|
||||
|
||||
JimengDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Jimeng alpha template (float [0,1]), or None."""
|
||||
|
||||
@@ -52,11 +52,9 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image SHORT side (measured basis -- see
|
||||
@@ -114,8 +112,6 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
KlingDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Kling alpha template (float [0,1]), or None."""
|
||||
@@ -127,23 +123,8 @@ def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the Kling glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class KlingEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible Kling "可灵AI 3.0" watermark (locate -> mask; mask feeds the fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
@@ -25,11 +25,15 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkLocation,
|
||||
TextMarkScan,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image WIDTH (measured basis). The box is
|
||||
@@ -84,24 +88,12 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
LibLibDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled LibLibAI alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
"""Binary "LibLibAI" silhouette (255 = glyph) from the alpha map, or None."""
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the LibLibAI glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class LibLibEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible LibLibAI wordmark (bottom-center; localize -> fill)."""
|
||||
|
||||
@@ -111,60 +103,51 @@ class LibLibEngine(TextMarkEngine):
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
def detect(self, image: NDArray[Any] | None, *, provenance: bool = False) -> TextMarkDetection:
|
||||
if image is None or not image.size or min(image.shape[:2]) < self._MIN_SHORT_SIDE:
|
||||
return TextMarkDetection()
|
||||
return super().detect(image, provenance=provenance)
|
||||
def _scan(self, image: NDArray[Any] | None) -> TextMarkScan:
|
||||
"""Skip the scan entirely below the size floor.
|
||||
|
||||
def footprint_mask(
|
||||
self, image: NDArray[Any] | None, *, force: bool = False, dilate: int | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame mask of the logo + wordmark, bounded by the detector's match box.
|
||||
Gating the SCAN rather than overriding ``detect`` is what keeps the floor on the
|
||||
single-pass perception path too, and it means a small image costs nothing.
|
||||
"""
|
||||
if image is None or not image.size or min(image.shape[:2]) < self._MIN_SHORT_SIDE:
|
||||
return TextMarkScan(None, None, 0)
|
||||
return super()._scan(image)
|
||||
|
||||
def _footprint_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Bound the fill by the detector's match box, never by the binary glyph blob.
|
||||
|
||||
The base class's blob-bbox footprint is wrong in both directions here: the
|
||||
blob bleeds UP into bright background structure (on the 768x1024 cohort
|
||||
frame it reached y 931 and the fill ate the shirt's own print) and it does
|
||||
not own the triangle logo anyway. The match box bounds the wordmark exactly
|
||||
(that is what the NCC localized); the logo sits its own height to the LEFT
|
||||
of the text (measured on the cohort zoom: logo ~1.0x the glyph height, gap
|
||||
~0.3x), so the footprint is the match box extended left by ~1.3 heights.
|
||||
not own the triangle logo anyway.
|
||||
"""
|
||||
if image is None or image.size == 0:
|
||||
return None
|
||||
from remove_ai_watermarks import image_io, region_eraser
|
||||
return self._match_box_rect(image, loc, force=force, detection=detection)
|
||||
|
||||
image = image_io.to_bgr(image)
|
||||
h, w = image.shape[:2]
|
||||
if h < 32 or w < 64:
|
||||
return None
|
||||
loc = self.locate(image)
|
||||
bx, by, bw, bh = loc.bbox
|
||||
if force:
|
||||
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
|
||||
else:
|
||||
if not self.detect(image).detected:
|
||||
return None
|
||||
_, box = self._tophat_best(image, loc)
|
||||
if box is None:
|
||||
return None
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
gh = gy1 - gy0 + 1
|
||||
pad = max(3, int(0.25 * gh))
|
||||
rx1 = max(0, bx + gx0 - int(1.3 * gh)) # the triangle logo, left of the text
|
||||
ry1 = max(0, by + gy0 - pad)
|
||||
rx2 = min(w, bx + gx1 + 1 + pad)
|
||||
ry2 = min(h, by + gy1 + 1 + pad)
|
||||
if rx1 >= rx2 or ry1 >= ry2:
|
||||
return None
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * bw))
|
||||
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
|
||||
def _extend_match_box(
|
||||
self, box: tuple[int, int, int, int], loc: TextMarkLocation, frame: tuple[int, int]
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Extend the match box LEFT to take in the triangle logo.
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
The match box bounds the wordmark exactly (that is what the NCC localized);
|
||||
the logo sits its own height to the LEFT of the text (measured on the cohort
|
||||
zoom: logo ~1.0x the glyph height, gap ~0.3x), so the footprint is the match
|
||||
box extended left by ~1.3 heights.
|
||||
"""
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
bx, by, _bw, _bh = loc.bbox
|
||||
h, w = frame
|
||||
gh = gy1 - gy0 + 1
|
||||
pad = max(3, int(0.25 * gh))
|
||||
return (
|
||||
max(0, bx + gx0 - int(1.3 * gh)), # the triangle logo, left of the text
|
||||
max(0, by + gy0 - pad),
|
||||
min(w, bx + gx1 + 1 + pad),
|
||||
min(h, by + gy1 + 1 + pad),
|
||||
)
|
||||
@@ -15,9 +15,14 @@ import struct
|
||||
from typing import TYPE_CHECKING, Any, cast
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable
|
||||
from collections.abc import Callable, Iterable
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks._internal.constants import (
|
||||
PNG_METADATA_CHUNKS,
|
||||
RIFF_METADATA_CHUNKS,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Smaller scan_head window for the cheap marker checks (has_ai_metadata,
|
||||
@@ -165,6 +170,21 @@ TC260_AIGC_FIELDS: frozenset[str] = frozenset(
|
||||
MAX_TC260_VALUE_BYTES = 1024 * 1024
|
||||
|
||||
|
||||
# A TC260 producer code is ``001`` + ``1`` + USCC(18) + a 5-digit app/product suffix,
|
||||
# so two codes sharing the USCC are the same legal entity registering different
|
||||
# products. Slicing is defensive: anything not matching the layout is returned as-is,
|
||||
# which also passes through the bare-name forms some generators write ("doubao",
|
||||
# "picwish").
|
||||
_USCC_START, _USCC_END = 4, 22
|
||||
|
||||
|
||||
def uscc_of(code: str) -> str:
|
||||
"""The 18-char Unified Social Credit Code embedded in a TC260 producer code."""
|
||||
if len(code) >= _USCC_END and code[:3] == "001":
|
||||
return code[_USCC_START:_USCC_END]
|
||||
return code
|
||||
|
||||
|
||||
def parse_tc260_aigc_json(value: bytes) -> dict[str, str] | None:
|
||||
"""Parse a bounded JSON object carrying at least one normative TC260 field."""
|
||||
if len(value) > MAX_TC260_VALUE_BYTES:
|
||||
@@ -218,12 +238,8 @@ def _is_ai_value(value: str) -> bool:
|
||||
from remove_ai_watermarks._internal.constants import AI_GENERATOR_TOKENS
|
||||
|
||||
value_lower = value.lower()
|
||||
return any(token in value_lower for token in AI_GENERATOR_TOKENS)
|
||||
|
||||
|
||||
# PNG ancillary chunks that can carry provenance metadata (XMP, EXIF, text).
|
||||
# Never IDAT -- that is the compressed pixel stream.
|
||||
_PNG_META_CHUNKS: frozenset[bytes] = frozenset({b"tEXt", b"iTXt", b"zTXt", b"eXIf", b"iCCP"})
|
||||
provenance, generator = _app_metadata_evidence(value)
|
||||
return provenance is not None or generator is not None or any(token in value_lower for token in AI_GENERATOR_TOKENS)
|
||||
|
||||
|
||||
def _png_late_metadata(image_path: Path, window: int) -> bytes:
|
||||
@@ -257,7 +273,7 @@ def _png_late_metadata(image_path: Path, window: int) -> bytes:
|
||||
# Clamp the attacker-controlled 32-bit length to the bytes that
|
||||
# actually remain, so a malformed huge length can't allocate GBs.
|
||||
safe_length = max(0, min(length, file_size - data_start))
|
||||
if chunk_type in _PNG_META_CHUNKS and data_start >= window:
|
||||
if chunk_type in PNG_METADATA_CHUNKS and data_start >= window:
|
||||
f.seek(data_start)
|
||||
out += f.read(safe_length)
|
||||
# Advance by the CLAMPED length: a malformed/inflated `length` that
|
||||
@@ -270,17 +286,86 @@ def _png_late_metadata(image_path: Path, window: int) -> bytes:
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _riff_late_metadata(image_path: Path, window: int, *, max_total: int = 4 * 1024 * 1024) -> bytes:
|
||||
"""Payloads of RIFF metadata chunks that start *beyond* the first ``window``
|
||||
bytes, found by stepping over the (large) coded-image chunk.
|
||||
|
||||
The WebP layout puts ``XMP ``/``EXIF`` AFTER the pixels, so a fixed read can stop
|
||||
before an IPTC or C2PA AI label. This is the RIFF analogue of
|
||||
:func:`_png_late_metadata`; it returns only chunks past ``window`` so bytes
|
||||
already in the head are not duplicated, and empty when there are none.
|
||||
|
||||
``max_total`` caps what a metadata scan can pull into memory, the same ceiling
|
||||
``isobmff.scan_c2pa_region`` applies. Clamping each chunk to the bytes that remain
|
||||
is not enough on its own: a corrupt or crafted file can declare one ``XMP `` chunk
|
||||
spanning most of itself, and this runs on the memoized verdict path for images from
|
||||
arbitrary sources. A label that needs more than 4 MB of XMP does not exist.
|
||||
"""
|
||||
out = bytearray()
|
||||
try:
|
||||
with open(image_path, "rb") as f:
|
||||
if f.read(4) != b"RIFF":
|
||||
return b""
|
||||
f.seek(0, 2)
|
||||
file_size = f.tell()
|
||||
f.seek(4)
|
||||
declared_size = f.read(4)
|
||||
if len(declared_size) < 4:
|
||||
return b""
|
||||
container_end = min(file_size, 8 + struct.unpack("<I", declared_size)[0])
|
||||
position = 12 # 'RIFF' + size + form type
|
||||
while position + 8 <= container_end and len(out) < max_total:
|
||||
f.seek(position)
|
||||
header = f.read(8)
|
||||
if len(header) < 8:
|
||||
break
|
||||
chunk_type = header[:4]
|
||||
(length,) = struct.unpack("<I", header[4:8])
|
||||
start = position + 8
|
||||
# Clamp to what remains: a malformed 32-bit length must not push the
|
||||
# walk past EOF and abandon a genuine label chunk after it.
|
||||
safe_length = max(0, min(length, container_end - start))
|
||||
if chunk_type in RIFF_METADATA_CHUNKS and start >= window:
|
||||
f.seek(start)
|
||||
out += f.read(min(safe_length, max_total - len(out)))
|
||||
position = start + safe_length + (safe_length & 1) # chunks are word-aligned
|
||||
except OSError as exc:
|
||||
logger.debug("RIFF late-metadata scan failed on %s: %s", image_path, exc)
|
||||
return b""
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _stat_key(image_path: Path) -> tuple[str, int, int] | None:
|
||||
"""Cache key identifying this file's exact CONTENT, or None when it cannot stat.
|
||||
|
||||
``(path, mtime_ns, size)`` -- size as well as mtime because an in-place rewrite
|
||||
can land inside the same mtime tick on a coarse filesystem, and this package does
|
||||
rewrite in place (``remove_ai_metadata(p, p)``, the batch output-equals-input
|
||||
case). A file it cannot stat is read uncached rather than failing.
|
||||
"""
|
||||
try:
|
||||
st = image_path.stat()
|
||||
except OSError:
|
||||
return None
|
||||
return (str(image_path), st.st_mtime_ns, st.st_size)
|
||||
|
||||
|
||||
def scan_head(image_path: Path, size: int = 1024 * 1024) -> bytes:
|
||||
"""First ``size`` bytes of the file, plus the payloads of any provenance
|
||||
metadata found beyond that window: ISOBMFF ``uuid`` / ``jumb`` boxes (seeking
|
||||
past large boxes like ``mdat``) and PNG ``tEXt`` / ``iTXt`` / ``eXIf`` chunks
|
||||
(seeking past ``IDAT``).
|
||||
|
||||
A file at least ``size`` bytes long additionally gets the metadata text its
|
||||
decoder can reach but a raw read cannot (:func:`_decoder_visible_text`): a
|
||||
compressed PNG ``zTXt`` packet, or a chunk past the window in a container with no
|
||||
late-chunk reader here. A file that fits inside ``size`` is exactly
|
||||
``f.read(size)``, since the raw read already holds every byte.
|
||||
|
||||
This is the shared input for every C2PA / AIGC / IPTC byte scan. The
|
||||
extensions catch a manifest or XMP packet placed AFTER the media data -- a
|
||||
non-faststart MP4 manifest, or a PNG XMP packet appended after the pixels --
|
||||
which a fixed first-MB read would miss. For other inputs, and for files that
|
||||
fit within ``size``, it is exactly ``f.read(size)`` -- behavior-neutral.
|
||||
which a fixed first-MB read would miss.
|
||||
|
||||
The result is memoized per (path, size, mtime): one ``identify``/``get_ai_metadata``
|
||||
call fans out to ~8 byte-scan detectors that each call this on the same file, so
|
||||
@@ -317,9 +402,63 @@ def _scan_head_impl(image_path: Path, size: int) -> bytes:
|
||||
# len(head) == size means the file is at least `size` bytes, so metadata
|
||||
# chunks may lie beyond the window; otherwise the whole PNG is in `head`.
|
||||
head += _png_late_metadata(image_path, size)
|
||||
elif head[:4] == b"RIFF" and head[8:12] == b"WEBP" and len(head) == size:
|
||||
head += _riff_late_metadata(image_path, size)
|
||||
if len(head) >= size:
|
||||
head += _decoder_visible_text(image_path, head)
|
||||
return head
|
||||
|
||||
|
||||
# Text values the image decoder can reach that a raw byte read cannot. Bounded: a
|
||||
# packet larger than this is not a provenance label.
|
||||
_DECODED_TEXT_LIMIT = 512 * 1024
|
||||
# Decoder values that are binary payloads with their own readers, not metadata text.
|
||||
# An ICC profile is colour data and can run to hundreds of kilobytes; appending it
|
||||
# would bloat the buffer every later detector re-scans, for no signal.
|
||||
_DECODER_BINARY_KEYS = frozenset({"icc_profile"})
|
||||
|
||||
|
||||
def _decoder_visible_text(image_path: Path, head: bytes) -> bytes:
|
||||
"""Metadata text PIL can decode but the raw window does not contain.
|
||||
|
||||
This is the last of two layers, not the first. Metadata placed BEYOND the window
|
||||
is the structural readers' job (``_png_late_metadata``, ``_riff_late_metadata``,
|
||||
the ISOBMFF box walk), and they work on a file no decoder can open. What is left
|
||||
for this one is metadata the bytes do not spell at all:
|
||||
|
||||
* COMPRESSED -- a PNG ``zTXt`` chunk is zlib-deflated, so an XMP packet carrying
|
||||
a TC260 AIGC label is unreadable as bytes while PIL inflates it on open.
|
||||
|
||||
It stays container-agnostic on purpose: it is the net under a placement no
|
||||
structural reader here knows about yet.
|
||||
|
||||
Only text ALREADY MISSING from ``head`` is appended, so the common case adds
|
||||
nothing and no detector sees a value twice. Skipped entirely when the file fits
|
||||
inside the window, since then the raw read already holds every byte.
|
||||
"""
|
||||
try:
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(image_path) as img:
|
||||
values = [value for key, value in img.info.items() if key not in _DECODER_BINARY_KEYS]
|
||||
except Exception as exc: # a container PIL cannot open: the raw scan stands alone
|
||||
logger.debug("decoder-visible text unavailable for %s: %s", image_path, exc)
|
||||
return b""
|
||||
|
||||
out = bytearray()
|
||||
for value in values:
|
||||
if isinstance(value, str):
|
||||
encoded = value.encode("utf-8", "replace")
|
||||
elif isinstance(value, bytes):
|
||||
encoded = value
|
||||
else:
|
||||
continue
|
||||
if len(encoded) > _DECODED_TEXT_LIMIT or not encoded or encoded in head:
|
||||
continue
|
||||
out += b"\x00" + encoded
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def has_ai_metadata(image_path: Path) -> bool:
|
||||
"""Check if an image contains AI-generation metadata.
|
||||
|
||||
@@ -339,6 +478,9 @@ def has_ai_metadata(image_path: Path) -> bool:
|
||||
for key in img.info:
|
||||
if isinstance(key, str) and _is_ai_key(key):
|
||||
return True
|
||||
exif_bytes = img.info.get("exif")
|
||||
if exif_bytes and any(_app_metadata_evidence(exif_bytes)):
|
||||
return True
|
||||
except Exception as exc:
|
||||
logger.debug("PIL could not open %s for metadata scan: %s", image_path, exc)
|
||||
|
||||
@@ -362,6 +504,8 @@ def has_ai_metadata(image_path: Path) -> bool:
|
||||
# IPTC 2025.1 AI-disclosure XMP properties (their presence flags AI content).
|
||||
if any(marker in data for marker in IPTC_AI_FIELD_MARKERS):
|
||||
return True
|
||||
if any(_app_metadata_evidence(data)):
|
||||
return True
|
||||
# China TC260 AIGC label as a PNG text chunk (the byte scan above catches
|
||||
# only the XMP form; the raw-JSON tEXt chunk needs the PIL-based parse).
|
||||
if aigc_label(image_path) is not None:
|
||||
@@ -420,7 +564,7 @@ def aigc_label_from_metadata(data: bytes, candidates: tuple[str, ...] = ()) -> d
|
||||
return None
|
||||
|
||||
|
||||
def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
def _aigc_label_impl(image_path: Path) -> dict[str, str] | None:
|
||||
"""Parse a China TC260 AI-labeling block, if present.
|
||||
|
||||
Supported serializations are:
|
||||
@@ -459,45 +603,44 @@ def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
if isinstance(value, str) and (result := aigc_label_from_metadata(b"", (value,))):
|
||||
return result
|
||||
|
||||
# Native MP4/MOV TC260 metadata (TC260-PG-20257A): the ``AIGC`` key lives
|
||||
# in ``moov.udta.meta.keys`` and points to a raw JSON value in ``ilst``.
|
||||
# Read it through the bounded box walker so a tail ``moov`` after a large
|
||||
# ``mdat`` is found without loading or scanning the media payload.
|
||||
from remove_ai_watermarks._internal.isobmff import tc260_aigc_payloads
|
||||
|
||||
isobmff_candidates = tuple(payload.decode("utf-8", "replace") for payload in tc260_aigc_payloads(image_path))
|
||||
if result := aigc_label_from_metadata(b"", isobmff_candidates):
|
||||
return result
|
||||
|
||||
# Native MKV/WebM TC260 metadata: ``Segment.Tags.Tag.SimpleTag`` carries
|
||||
# ``TagName=AIGC`` and the raw JSON in ``TagString``. The EBML walker seeks
|
||||
# over clusters and reads only bounded metadata values.
|
||||
from remove_ai_watermarks._internal.ebml import tc260_aigc_payloads as ebml_tc260_aigc_payloads
|
||||
|
||||
ebml_candidates = tuple(payload.decode("utf-8", "replace") for payload in ebml_tc260_aigc_payloads(image_path))
|
||||
if result := aigc_label_from_metadata(b"", ebml_candidates):
|
||||
return result
|
||||
|
||||
# Native AVI and FLV TC260 metadata. Both readers walk their container
|
||||
# structures and skip media payloads instead of relying on a raw substring
|
||||
# that could collide inside compressed video.
|
||||
legacy_payloads: tuple[bytes, ...] = ()
|
||||
if image_path.suffix.lower() == ".avi":
|
||||
from remove_ai_watermarks._internal.riff import tc260_aigc_payloads as riff_tc260_aigc_payloads
|
||||
|
||||
legacy_payloads = riff_tc260_aigc_payloads(image_path)
|
||||
elif image_path.suffix.lower() == ".flv":
|
||||
from remove_ai_watermarks._internal.flv import tc260_aigc_payloads as flv_tc260_aigc_payloads
|
||||
|
||||
legacy_payloads = flv_tc260_aigc_payloads(image_path)
|
||||
legacy_candidates = tuple(payload.decode("utf-8", "replace") for payload in legacy_payloads)
|
||||
if result := aigc_label_from_metadata(b"", legacy_candidates):
|
||||
return result
|
||||
# Native container TC260 metadata. Every reader walks its own container structure
|
||||
# and skips media payloads instead of relying on a raw substring that could collide
|
||||
# inside compressed video, and every one of them SELF-GATES on its magic bytes --
|
||||
# returning () after a 4-12 byte read on anything else. So the route is content, not
|
||||
# extension: a correctly formatted AVI or FLV served under the wrong suffix used to
|
||||
# be missed, which contradicts this module's own rule elsewhere ("route on the
|
||||
# actual content format, not the extension").
|
||||
for reader in _tc260_container_readers():
|
||||
candidates = tuple(payload.decode("utf-8", "replace") for payload in reader(image_path))
|
||||
if result := aigc_label_from_metadata(b"", candidates):
|
||||
return result
|
||||
|
||||
data = scan_head(image_path)
|
||||
return aigc_label_from_metadata(data)
|
||||
|
||||
|
||||
def _tc260_container_readers() -> tuple[Callable[[Path], tuple[bytes, ...]], ...]:
|
||||
"""The native-container TC260 readers, most common first.
|
||||
|
||||
Static imports rather than ``importlib``: this module carries no pyright pragma, so
|
||||
a dynamically resolved callable would be ``Any`` and fail the strict gate. They stay
|
||||
function-local because ``isobmff`` imports this module's constants at import time.
|
||||
|
||||
* MP4/MOV -- the ``AIGC`` key in ``moov.udta.meta.keys`` points at raw JSON in
|
||||
``ilst``; the bounded box walker finds a tail ``moov`` after a large ``mdat``
|
||||
without loading the media payload.
|
||||
* MKV/WebM -- ``Segment.Tags.Tag.SimpleTag`` carries ``TagName=AIGC``.
|
||||
* AVI -- a ``LIST/INFO/AIGC`` chunk.
|
||||
* FLV -- ``script.onMetaData.AIGC``.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.ebml import tc260_aigc_payloads as ebml_payloads
|
||||
from remove_ai_watermarks._internal.flv import tc260_aigc_payloads as flv_payloads
|
||||
from remove_ai_watermarks._internal.isobmff import tc260_aigc_payloads as isobmff_payloads
|
||||
from remove_ai_watermarks._internal.riff import tc260_aigc_payloads as riff_payloads
|
||||
|
||||
return (isobmff_payloads, ebml_payloads, riff_payloads, flv_payloads)
|
||||
|
||||
|
||||
# C2PA "Durable Content Credentials" manifest repositories (C2PA 2.4). When the
|
||||
# embedded manifest is stripped, an XMP ``dcterms:provenance`` URL can still point
|
||||
# at the vendor's cloud manifest store, from which the credentials are recoverable
|
||||
@@ -538,7 +681,7 @@ def c2pa_cloud_manifest(image_path: Path) -> str | None:
|
||||
return c2pa_cloud_manifest_in(scan_head(image_path, _QUICK_SCAN_BYTES))
|
||||
|
||||
|
||||
def huggingface_job(image_path: Path) -> str | None:
|
||||
def _huggingface_job_impl(image_path: Path) -> str | None:
|
||||
"""Return the HuggingFace job id if the image carries an ``hf-job-id`` PNG
|
||||
text chunk, else None.
|
||||
|
||||
@@ -599,7 +742,7 @@ def samsung_genai_in(data: bytes) -> int | None:
|
||||
return int(match.group(1)) or None
|
||||
|
||||
|
||||
def samsung_genai(image_path: Path) -> int | None:
|
||||
def _samsung_genai_impl(image_path: Path) -> int | None:
|
||||
"""Return Samsung's non-zero ``genAIType`` value if the image carries the
|
||||
Galaxy AI editing marker, else None.
|
||||
|
||||
@@ -636,7 +779,7 @@ def iptc_ai_system_in(data: bytes) -> str | None:
|
||||
return "fields present"
|
||||
|
||||
|
||||
def iptc_ai_system(image_path: Path) -> str | None:
|
||||
def _iptc_ai_system_impl(image_path: Path) -> str | None:
|
||||
"""Return an IPTC 2025.1 AI-disclosure note if the file carries those XMP
|
||||
properties, else None.
|
||||
|
||||
@@ -650,18 +793,18 @@ def iptc_ai_system(image_path: Path) -> str | None:
|
||||
|
||||
|
||||
def synthid_source(image_path: Path) -> str | None:
|
||||
"""Return the vendor name(s) if the image carries a SynthID pixel watermark.
|
||||
"""Return the vendor name(s) when provenance establishes SynthID.
|
||||
|
||||
This is a *metadata-based* proxy: Google (Imagen/Gemini) and OpenAI
|
||||
(ChatGPT/DALL-E/gpt-image) embed an invisible SynthID watermark alongside
|
||||
a C2PA manifest, so a C2PA manifest signed by one of them on AI-generated
|
||||
content implies SynthID in the pixels. Adobe Firefly / Microsoft Designer
|
||||
sign C2PA but do not use SynthID, so they return None.
|
||||
This is provenance-based, not a local pixel decode. Google states that all
|
||||
media generated by its tools carries SynthID, so Google AI C2PA establishes
|
||||
the mark. OpenAI C2PA existed before OpenAI adopted SynthID, so OpenAI also
|
||||
requires the explicit ``c2pa.watermarked.*`` action used by current manifests.
|
||||
Adobe Firefly and Microsoft sign C2PA but do not use SynthID, so they return
|
||||
None.
|
||||
|
||||
The verdict is reliable only while the C2PA manifest is intact -- absence
|
||||
is not proof, because C2PA can be stripped while the pixel watermark
|
||||
survives, and the pixel watermark itself is not locally detectable
|
||||
(proprietary decoder).
|
||||
The evidence is readable only while the C2PA manifest is intact. Absence is
|
||||
not proof: C2PA can be stripped while the pixel watermark survives, and the
|
||||
pixel watermark itself is not locally detectable (proprietary decoder).
|
||||
|
||||
Args:
|
||||
image_path: Path to the image (PNG, JPEG, WebP, or ISOBMFF container).
|
||||
@@ -669,10 +812,19 @@ def synthid_source(image_path: Path) -> str | None:
|
||||
Returns:
|
||||
Comma-joined vendor name(s) (e.g. ``"OpenAI"``) or None.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.c2pa import extract_c2pa_info, synthid_vendors_in
|
||||
from remove_ai_watermarks._internal.c2pa import (
|
||||
c2pa_info_has_invalid_credential,
|
||||
extract_c2pa_info,
|
||||
synthid_evidence_vendors_in,
|
||||
)
|
||||
|
||||
# PNG: the caBX chunk parser gives a clean, structured issuer.
|
||||
vendors = extract_c2pa_info(image_path).get("synthid_vendors")
|
||||
# Prefer the official reader's structured result. A failed asset binding or
|
||||
# signature cannot establish the claim, and the raw fallback below must not
|
||||
# resurrect it from the same damaged manifest bytes.
|
||||
c2pa = extract_c2pa_info(image_path)
|
||||
if c2pa_info_has_invalid_credential(c2pa):
|
||||
return None
|
||||
vendors = c2pa.get("synthid_vendors")
|
||||
if vendors:
|
||||
return ", ".join(vendors)
|
||||
|
||||
@@ -685,7 +837,7 @@ def synthid_source(image_path: Path) -> str | None:
|
||||
ai_source = b"trainedAlgorithmicMedia" in data or b"TrainedAlgorithmicMedia" in data
|
||||
if not (has_c2pa and ai_source):
|
||||
return None
|
||||
matched = synthid_vendors_in(data)
|
||||
matched = synthid_evidence_vendors_in(data)
|
||||
return ", ".join(matched) if matched else None
|
||||
|
||||
|
||||
@@ -693,16 +845,79 @@ def generator_from_metadata(candidates: Iterable[str], scan: bytes = b"") -> str
|
||||
"""Return a known AI generator from collected EXIF, PNG, or XMP values."""
|
||||
from remove_ai_watermarks._internal.constants import AI_GENERATOR_TOKENS
|
||||
|
||||
if app_generator := app_generator_from_metadata(scan):
|
||||
return app_generator
|
||||
|
||||
creator_tools = (
|
||||
match.group(1).decode("latin1", "replace")
|
||||
for match in re.finditer(rb"CreatorTool[>\"'=\s]{1,4}([^<\"']{1,80})", scan)
|
||||
)
|
||||
for value in itertools.chain(candidates, creator_tools):
|
||||
if app_generator := app_generator_from_metadata(value):
|
||||
return app_generator
|
||||
if any(token in value.lower() for token in AI_GENERATOR_TOKENS):
|
||||
return value.strip()
|
||||
return None
|
||||
|
||||
|
||||
_APP_PROVENANCE_PRODUCTS: dict[str, str] = {
|
||||
"doubao": "ByteDance Doubao",
|
||||
"xinghui": "ByteDance Xinghui",
|
||||
"dreamina": "ByteDance Dreamina",
|
||||
"dreamina_oversea": "ByteDance Dreamina",
|
||||
}
|
||||
_APP_PRODUCT_RE = re.compile(r'"product"\s*:\s*"([a-z0-9_]+)"', re.IGNORECASE)
|
||||
_APP_EXPORT_TYPE_RE = re.compile(r'"exportType"\s*:\s*"([a-z0-9_]+)"', re.IGNORECASE)
|
||||
_APP_AIGC_LABEL_RE = re.compile(r'"aigc_label_type"\s*:\s*[12](?=\s*[,}])', re.IGNORECASE)
|
||||
_APP_AIGC_TYPE_RE = re.compile(r'"aigc_type"\s*:\s*1(?=\s*[,}])', re.IGNORECASE)
|
||||
|
||||
|
||||
def _normalized_app_metadata(value: str | bytes) -> str:
|
||||
text = value.decode("latin-1", "ignore") if isinstance(value, bytes) else value
|
||||
return text.replace('\\"', '"')
|
||||
|
||||
|
||||
def _app_metadata_evidence(value: str | bytes) -> tuple[str | None, str | None]:
|
||||
"""Return removable product provenance and stronger AI-origin evidence."""
|
||||
normalized = _normalized_app_metadata(value)
|
||||
products = tuple(match.group(1).lower() for match in _APP_PRODUCT_RE.finditer(normalized))
|
||||
provenance = next(
|
||||
(_APP_PROVENANCE_PRODUCTS[product] for product in products if product in _APP_PROVENANCE_PRODUCTS), None
|
||||
)
|
||||
export_types = {match.group(1).lower() for match in _APP_EXPORT_TYPE_RE.finditer(normalized)}
|
||||
|
||||
generator = None
|
||||
if set(products).intersection({"dreamina", "dreamina_oversea"}) and "generation" in export_types:
|
||||
generator = "ByteDance Dreamina"
|
||||
elif '"aigc_info"' in normalized.lower():
|
||||
if "aweme" in products and _APP_AIGC_TYPE_RE.search(normalized):
|
||||
generator = "ByteDance Aweme AI"
|
||||
elif _APP_AIGC_LABEL_RE.search(normalized):
|
||||
generator = "Embedded app AIGC disclosure"
|
||||
return provenance, generator
|
||||
|
||||
|
||||
def app_provenance_from_metadata(value: str | bytes) -> str | None:
|
||||
"""Return exact AI-product provenance that is safe to scrub.
|
||||
|
||||
An exporting product does not by itself prove that the pixels were generated.
|
||||
That stronger interpretation stays in :func:`app_generator_from_metadata`.
|
||||
"""
|
||||
return _app_metadata_evidence(value)[0]
|
||||
|
||||
|
||||
def app_generator_from_metadata(value: str | bytes) -> str | None:
|
||||
"""Identify exact app-export AI disclosures embedded in JSON-shaped metadata.
|
||||
|
||||
ByteDance-family apps write a second provenance object beside C2PA or TC260.
|
||||
Its keys are ordinary EXIF fields, so generic key matching misses it and a
|
||||
metadata-preserving JPEG scrub keeps it. Only explicit AIGC discriminators or
|
||||
a Dreamina generation export assert AI origin. Product provenance alone remains
|
||||
removable without changing the image's origin verdict.
|
||||
"""
|
||||
return _app_metadata_evidence(value)[1]
|
||||
|
||||
|
||||
def exif_generator(image_path: Path) -> str | None:
|
||||
"""Return an AI-generator name from the EXIF ``Software`` / XMP ``CreatorTool``
|
||||
field (or a PNG text chunk), if it matches a known generator (see
|
||||
@@ -733,7 +948,9 @@ def exif_generator(image_path: Path) -> str | None:
|
||||
if isinstance(value, str) and value:
|
||||
candidates.append(value)
|
||||
if exif_bytes:
|
||||
tags = piexif.load(exif_bytes).get("0th", {})
|
||||
loaded = piexif.load(exif_bytes)
|
||||
tags = loaded.get("0th", {})
|
||||
exif_tags: dict[int, Any] = loaded.get("Exif") or {}
|
||||
# Make catches camera-style tags AI tools reuse (Ideogram writes
|
||||
# Make="Ideogram AI"); real cameras put "Apple"/"Canon" there, which
|
||||
# carry no AI token, so this stays low-false-positive.
|
||||
@@ -746,6 +963,9 @@ def exif_generator(image_path: Path) -> str | None:
|
||||
value = tags.get(tag)
|
||||
if isinstance(value, bytes):
|
||||
candidates.append(value.decode("latin1", "replace"))
|
||||
user_comment = exif_tags.get(piexif.ExifIFD.UserComment)
|
||||
if isinstance(user_comment, bytes):
|
||||
candidates.append(user_comment.decode("latin1", "replace"))
|
||||
except Exception as exc: # unopenable format / malformed EXIF
|
||||
logger.debug("EXIF generator read failed for %s: %s", image_path, exc)
|
||||
|
||||
@@ -775,7 +995,7 @@ def _exif_text(ifd: dict[int, Any], tag: int) -> str:
|
||||
return value.decode("latin1", "replace").strip() if isinstance(value, bytes) else ""
|
||||
|
||||
|
||||
def xai_signature(image_path: Path) -> bool:
|
||||
def _xai_signature_impl(image_path: Path) -> bool:
|
||||
"""Detect xAI / Grok's EXIF provenance signature scheme.
|
||||
|
||||
Grok image downloads (Aurora model) carry no C2PA, XMP, SynthID, or IPTC --
|
||||
@@ -872,6 +1092,15 @@ def _ai_exif_targets(loaded: dict[str, Any]) -> list[tuple[str, int, bytes, str]
|
||||
add("0th", ifd0, piexif.ImageIFD.ImageDescription, "ImageDescription")
|
||||
if _is_aigc_exif_value(ifde.get(piexif.ExifIFD.UserComment)):
|
||||
add("Exif", ifde, piexif.ExifIFD.UserComment, "UserComment")
|
||||
# (d) ByteDance-family app JSON. Exact AI-product provenance is removable even
|
||||
# when it does not by itself assert generated pixels. Ordinary Aweme/retouch/lv
|
||||
# exports remain untouched.
|
||||
for ifd_key, ifd, tag, name in (
|
||||
("0th", ifd0, piexif.ImageIFD.ImageDescription, "ImageDescription"),
|
||||
("Exif", ifde, piexif.ExifIFD.UserComment, "UserComment"),
|
||||
):
|
||||
if any(_app_metadata_evidence(ifd.get(tag, b""))):
|
||||
add(ifd_key, ifd, tag, name)
|
||||
|
||||
return targets
|
||||
|
||||
@@ -931,6 +1160,14 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
"actions",
|
||||
"synthid_watermark",
|
||||
"soft_binding",
|
||||
"soft_binding_algorithm",
|
||||
"soft_binding_value",
|
||||
"c2pa_validation_source",
|
||||
"c2pa_validation_state",
|
||||
"c2pa_integrity",
|
||||
"c2pa_signature",
|
||||
"c2pa_signer_trust",
|
||||
"c2pa_signer_validity",
|
||||
):
|
||||
if key in c2pa:
|
||||
result.setdefault(key, str(c2pa[key]))
|
||||
@@ -949,6 +1186,18 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
if (aigc := aigc_label(image_path)) is not None:
|
||||
producer = aigc.get("ContentProducer", "")
|
||||
result["aigc_label"] = f"China AIGC label (TC260){f'; producer {producer}' if producer else ''}"
|
||||
# The structural producer beside its display rendering: a machine
|
||||
# consumer (video-visible provenance) must not parse the formatted
|
||||
# sentence above, whose wording can change without notice.
|
||||
if producer:
|
||||
result["aigc_producer"] = producer
|
||||
|
||||
app_scan = scan_head(image_path)
|
||||
app_provenance, app_generator = _app_metadata_evidence(app_scan)
|
||||
if app_provenance:
|
||||
result.setdefault("app_provenance", f"App export provenance ({app_provenance})")
|
||||
if app_generator:
|
||||
result.setdefault("app_aigc", f"App AIGC disclosure ({app_generator})")
|
||||
|
||||
# xAI / Grok EXIF signature scheme (its only provenance signal).
|
||||
if xai_signature(image_path):
|
||||
@@ -1044,7 +1293,7 @@ def _jpeg_app_carries_ai(marker: int, payload: bytes) -> bool:
|
||||
# (detection<->removal parity). Skip APP1-EXIF (0xE1 ``Exif``): its camera tags are
|
||||
# scrubbed tag-by-tag via piexif, not dropped wholesale.
|
||||
if not (marker == 0xE1 and payload.startswith(b"Exif")):
|
||||
return _is_aigc_exif_value(payload)
|
||||
return _is_aigc_exif_value(payload) or any(_app_metadata_evidence(payload))
|
||||
return False
|
||||
|
||||
|
||||
@@ -1372,3 +1621,100 @@ def remove_ai_metadata(
|
||||
|
||||
logger.info("Stripped AI metadata → %s", output_path)
|
||||
return output_path
|
||||
|
||||
|
||||
# ── Per-file probe memoization ──────────────────────────────────────────────
|
||||
# One ``identify`` reaches each of these twice (``get_ai_metadata`` internally, then
|
||||
# ``extract_provenance_evidence``), and each call re-walks the container -- the
|
||||
# ISOBMFF/EBML walks in ``aigc_label`` and the file-tail read in ``samsung_genai``
|
||||
# are the expensive ones. Keyed on (path, mtime_ns, size) so an in-place rewrite
|
||||
# invalidates; ``maxsize`` bounds memory to a handful of entries.
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _aigc_label_cached(path_str: str, _mtime_ns: int, _size: int) -> dict[str, str] | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _aigc_label_impl(_Path(path_str))
|
||||
|
||||
|
||||
def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
"""See :func:`_aigc_label_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _aigc_label_impl(image_path)
|
||||
result = _aigc_label_cached(*key)
|
||||
return dict(result) if result is not None else None
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _huggingface_job_cached(path_str: str, _mtime_ns: int, _size: int) -> str | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _huggingface_job_impl(_Path(path_str))
|
||||
|
||||
|
||||
def huggingface_job(image_path: Path) -> str | None:
|
||||
"""See :func:`_huggingface_job_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _huggingface_job_impl(image_path)
|
||||
return _huggingface_job_cached(*key)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _samsung_genai_cached(path_str: str, _mtime_ns: int, _size: int) -> int | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _samsung_genai_impl(_Path(path_str))
|
||||
|
||||
|
||||
def samsung_genai(image_path: Path) -> int | None:
|
||||
"""See :func:`_samsung_genai_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _samsung_genai_impl(image_path)
|
||||
return _samsung_genai_cached(*key)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _iptc_ai_system_cached(path_str: str, _mtime_ns: int, _size: int) -> str | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _iptc_ai_system_impl(_Path(path_str))
|
||||
|
||||
|
||||
def iptc_ai_system(image_path: Path) -> str | None:
|
||||
"""See :func:`_iptc_ai_system_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _iptc_ai_system_impl(image_path)
|
||||
return _iptc_ai_system_cached(*key)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _xai_signature_cached(path_str: str, _mtime_ns: int, _size: int) -> bool:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _xai_signature_impl(_Path(path_str))
|
||||
|
||||
|
||||
def xai_signature(image_path: Path) -> bool:
|
||||
"""See :func:`_xai_signature_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _xai_signature_impl(image_path)
|
||||
return _xai_signature_cached(*key)
|
||||
|
||||
|
||||
# ── Shared with the portable metadata record ────────────────────────
|
||||
# `metadata_record` must read exactly the windows and markers the file path reads: a
|
||||
# record built from a different window is a record whose verdict can disagree with
|
||||
# `identify` on the same image. Aliased rather than renamed because the private names
|
||||
# are load-bearing in this module's own tests and in a corpus script.
|
||||
QUICK_SCAN_BYTES = _QUICK_SCAN_BYTES
|
||||
SAMSUNG_EDITOR_MARKER = _SAMSUNG_EDITOR_MARKER
|
||||
read_file_tail = _read_file_tail
|
||||
png_late_metadata = _png_late_metadata
|
||||
riff_late_metadata = _riff_late_metadata
|
||||
exif_text = _exif_text
|
||||
@@ -0,0 +1,409 @@
|
||||
"""Collect one image's provenance metadata into a portable, JSON-safe record.
|
||||
|
||||
WHY THIS EXISTS
|
||||
|
||||
``extract_provenance_evidence`` reads a file and hands back evidence in memory, so
|
||||
collection and verdict must happen in the same process, on the machine holding the
|
||||
image. This module splits them: collect here, judge anywhere, from a record that
|
||||
survives JSON.
|
||||
|
||||
record = collect_metadata_record(path) # touches the file
|
||||
evidence = evidence_from_metadata_record(record, path=path)
|
||||
report = identify_from_evidence(evidence) # touches nothing
|
||||
|
||||
WHAT GOES IN, AND WHY NOT SIMPLY THE FILE HEAD
|
||||
|
||||
The verdict reads a scan buffer that ``scan_head`` fills with the first mebibyte of
|
||||
the file. Shipping that verbatim would make a record larger than a phone photo's
|
||||
worth of metadata by two orders of magnitude, because for a PNG almost all of that
|
||||
mebibyte is compressed pixel data in ``IDAT`` -- bytes no provenance token can ever
|
||||
live in. A record carries the metadata REGIONS instead, walked per container: the
|
||||
JPEG marker segments before the coded scan, every PNG chunk but ``IDAT``, the RIFF
|
||||
chunks that are not coded image, the ISOBMFF provenance boxes, and in every case the
|
||||
container's trailer.
|
||||
|
||||
COMPLETENESS IS A MEASURED PROPERTY, NOT A CLAIM
|
||||
|
||||
A region walker is only correct if nothing the verdict reads falls outside the
|
||||
regions it keeps, and no test over fixtures can establish that: the failure mode is
|
||||
a container placement nobody thought of. The contract is therefore ALSO verified
|
||||
against the file path over a real corpus -- same image, both paths, identical
|
||||
``ProvenanceReport``.
|
||||
|
||||
The placements that defeated an earlier draft of this collector, and the reason each
|
||||
rule below exists, are recorded in ``docs/module-internals.md`` under "Portable
|
||||
metadata record".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import logging
|
||||
import struct
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks._internal.constants import PNG_SIGNATURE, RIFF_CODED_IMAGE_CHUNKS
|
||||
from remove_ai_watermarks._internal.schema import require_schema_version
|
||||
from remove_ai_watermarks.metadata import (
|
||||
QUICK_SCAN_BYTES,
|
||||
SAMSUNG_EDITOR_MARKER,
|
||||
exif_text,
|
||||
read_file_tail,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# The structural walk covers the same window the file path reads raw, so the two
|
||||
# cannot disagree about a chunk type inside it. A smaller window would be cheaper but
|
||||
# opens a blind spot: past the window only ``png_late_metadata``'s ALLOWLIST is
|
||||
# collected, while the file path still sees every chunk type up to its own window --
|
||||
# and a C2PA ``caBX`` chunk is in neither that allowlist nor ``IDAT``. Walking here
|
||||
# costs little because the payload of the pixel stream is skipped, not copied.
|
||||
HEAD_WINDOW = 1024 * 1024
|
||||
# The window searched for the container's end marker. Matches the quick-scan window
|
||||
# the file path uses when it goes looking for a Samsung trailer, so a trailer visible
|
||||
# to one path is visible to the other.
|
||||
TAIL_WINDOW = QUICK_SCAN_BYTES
|
||||
# Kept from the tail when no end marker is found, so an unrecognized container still
|
||||
# contributes its last bytes without carrying half a photo.
|
||||
UNKNOWN_TRAILER_WINDOW = 64 * 1024
|
||||
|
||||
# PNG text keys the file path reads for a generator tag, in ITS order. NovelAI stamps
|
||||
# Software/Source/Title rather than EXIF, and the first match wins, so order matters.
|
||||
_GENERATOR_TEXT_KEYS = ("Software", "Source", "Title", "Description")
|
||||
# Stable transport contract for records produced by this module. The version is
|
||||
# deliberately separate from the verdict version: collection and interpretation can
|
||||
# evolve independently as long as old records remain readable.
|
||||
METADATA_RECORD_SCHEMA_VERSION = 1
|
||||
METADATA_RECORD_TYPE = "provenance_metadata"
|
||||
|
||||
|
||||
def _jpeg_regions(data: bytes) -> bytes:
|
||||
"""Every marker segment up to the entropy-coded scan, plus the trailer after EOI.
|
||||
|
||||
The scan itself is skipped by walking to SOS and then jumping to the trailing
|
||||
EOI, so a 20 MB photo contributes only its markers.
|
||||
|
||||
TWIN: ``metadata._strip_jpeg_metadata_lossless`` walks the same marker chain. The
|
||||
two were left separate on purpose -- that one couples the walk to "return False and
|
||||
fall back to a PIL re-encode", a decision the lossless strip path owns and this one
|
||||
must not inherit -- so a fix to marker handling belongs in BOTH.
|
||||
"""
|
||||
out = bytearray()
|
||||
index, size = 2, len(data)
|
||||
while index + 1 < size:
|
||||
if data[index] != 0xFF:
|
||||
break # malformed boundary: keep what was collected, the tail still follows
|
||||
marker = data[index + 1]
|
||||
if marker in (0xDA, 0xD9): # SOS / EOI: the coded scan follows
|
||||
break
|
||||
if 0xD0 <= marker <= 0xD7 or marker == 0x01: # standalone, no length
|
||||
index += 2
|
||||
continue
|
||||
if index + 4 > size:
|
||||
break
|
||||
segment_length = int.from_bytes(data[index + 2 : index + 4], "big")
|
||||
end = index + 2 + segment_length
|
||||
if segment_length < 2 or end > size:
|
||||
break
|
||||
out += data[index:end]
|
||||
index = end
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _png_regions(data: bytes) -> bytes:
|
||||
"""Every chunk except the ``IDAT`` payloads, plus whatever follows IEND.
|
||||
|
||||
TWIN: ``metadata._png_late_metadata`` walks the same chunk chain by SEEKING over
|
||||
the file rather than over a buffer, and keeps an allowlist rather than skipping
|
||||
``IDAT``. Both filters are deliberate: inside the window the file path sees every
|
||||
chunk type raw, past it only the allowlist survives.
|
||||
"""
|
||||
out = bytearray()
|
||||
size = len(data)
|
||||
position = len(PNG_SIGNATURE)
|
||||
while position + 8 <= size:
|
||||
(length,) = struct.unpack(">I", data[position : position + 4])
|
||||
chunk_type = data[position + 4 : position + 8]
|
||||
start = position + 8
|
||||
# Clamp the length to the bytes that remain: a malformed 32-bit length must
|
||||
# not push the walk past EOF and abandon a genuine label chunk after it.
|
||||
safe_length = max(0, min(length, size - start))
|
||||
if chunk_type != b"IDAT":
|
||||
out += chunk_type + data[start : start + safe_length]
|
||||
position = start + safe_length + 4 # payload + CRC
|
||||
if chunk_type == b"IEND":
|
||||
out += data[position:] # a trailer past IEND is metadata too
|
||||
break
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _riff_regions(data: bytes) -> bytes:
|
||||
"""Every RIFF chunk except the coded image payloads.
|
||||
|
||||
TWIN: ``metadata._riff_late_metadata`` (seek-based, past the scan window) and
|
||||
``_internal.riff`` (AVI ``LIST/INFO``). Same chunk-stepping arithmetic, three
|
||||
input models.
|
||||
"""
|
||||
out = bytearray(data[:12]) # 'RIFF' + size + 'WEBP'
|
||||
declared_end = 8 + struct.unpack("<I", data[4:8])[0] if len(data) >= 12 else len(data)
|
||||
size = min(len(data), declared_end)
|
||||
position = 12
|
||||
while position + 8 <= size:
|
||||
chunk_type = data[position : position + 4]
|
||||
(length,) = struct.unpack("<I", data[position + 4 : position + 8])
|
||||
start = position + 8
|
||||
safe_length = max(0, min(length, size - start))
|
||||
if chunk_type not in RIFF_CODED_IMAGE_CHUNKS:
|
||||
out += chunk_type + data[start : start + safe_length]
|
||||
position = start + safe_length + (safe_length & 1) # chunks are word-aligned
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _isobmff_regions(image_path: Path, head: bytes) -> bytes:
|
||||
"""Header window plus the provenance regions the bounded box walkers find.
|
||||
|
||||
ISOBMFF hides a manifest in a ``uuid``/``jumb`` box that can sit after a
|
||||
multi-megabyte ``mdat``, and a TC260 label in ``moov.udta``. Both walkers seek
|
||||
rather than read the media, so neither pulls the payload in.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.isobmff import scan_c2pa_region, tc260_aigc_payloads
|
||||
|
||||
out = bytearray(head[:HEAD_WINDOW])
|
||||
try:
|
||||
out += scan_c2pa_region(image_path)
|
||||
except Exception as exc:
|
||||
logger.debug("ISOBMFF C2PA region scan failed on %s: %s", image_path, exc)
|
||||
try:
|
||||
for payload in tc260_aigc_payloads(image_path):
|
||||
out += payload
|
||||
except Exception as exc:
|
||||
logger.debug("ISOBMFF TC260 scan failed on %s: %s", image_path, exc)
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _container_regions(image_path: Path, head: bytes) -> tuple[str, bytes]:
|
||||
"""(container label, metadata bytes) for the container ``head`` starts with.
|
||||
|
||||
``head`` must be the file's raw first bytes. Handing this the ``scan_head``
|
||||
buffer instead is a trap: that buffer is the head CONCATENATED with late metadata
|
||||
payloads, so a structural walk runs off the end of the real head and parses the
|
||||
appended bytes as chunks, inflating the record and creating false signals.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.isobmff import is_isobmff
|
||||
from remove_ai_watermarks.metadata import png_late_metadata, riff_late_metadata
|
||||
|
||||
if head.startswith(b"\xff\xd8"):
|
||||
return "jpeg", _jpeg_regions(head)
|
||||
if head.startswith(PNG_SIGNATURE):
|
||||
# Chunks placed after the pixel stream (an XMP packet at 2.7 MB, say) are
|
||||
# past the window; the same seek-past-IDAT reader the file path uses gets them.
|
||||
return "png", _png_regions(head) + png_late_metadata(image_path, HEAD_WINDOW)
|
||||
if head.startswith(b"RIFF") and head[8:12] == b"WEBP":
|
||||
return "webp", _riff_regions(head) + riff_late_metadata(image_path, HEAD_WINDOW)
|
||||
if is_isobmff(head):
|
||||
return "isobmff", _isobmff_regions(image_path, head)
|
||||
return "unknown", head
|
||||
|
||||
|
||||
def _raw_head(image_path: Path) -> bytes:
|
||||
"""The file's first bytes, unmodified -- the input every structural walk needs."""
|
||||
try:
|
||||
with open(image_path, "rb") as handle:
|
||||
return handle.read(HEAD_WINDOW)
|
||||
except OSError as exc:
|
||||
logger.debug("head read failed for %s: %s", image_path, exc)
|
||||
return b""
|
||||
|
||||
|
||||
def _trailer(image_path: Path, container: str) -> bytes:
|
||||
"""The bytes that follow the container's end marker, and nothing else.
|
||||
|
||||
A fixed-size tail read would be almost entirely pixels: the trailer of a 20 MB
|
||||
photo is a few kilobytes at most. So the end marker is located in the tail window
|
||||
and only what follows it is kept. When no marker is found (an unknown container,
|
||||
or one whose end lies before the window) the window is kept as-is, bounded --
|
||||
that is what a byte scan of the same file would have seen anyway.
|
||||
"""
|
||||
if container == "webp":
|
||||
# RIFF declares its structural end in bytes 4..8. A fixed tail window is
|
||||
# normally the last animation/frame payload, not a trailer, so preserve
|
||||
# only bytes appended after the declared RIFF container.
|
||||
try:
|
||||
with open(image_path, "rb") as handle:
|
||||
header = handle.read(12)
|
||||
if len(header) < 12 or not header.startswith(b"RIFF"):
|
||||
return b""
|
||||
declared_end = 8 + struct.unpack("<I", header[4:8])[0]
|
||||
handle.seek(0, 2)
|
||||
file_size = handle.tell()
|
||||
if declared_end < 12 or declared_end >= file_size:
|
||||
return b""
|
||||
handle.seek(declared_end)
|
||||
return handle.read(min(file_size - declared_end, UNKNOWN_TRAILER_WINDOW))
|
||||
except OSError as exc:
|
||||
logger.debug("RIFF trailer read failed for %s: %s", image_path, exc)
|
||||
return b""
|
||||
if container == "isobmff":
|
||||
# ISOBMFF has no out-of-container trailer convention. Its bounded box
|
||||
# walkers already collect late provenance while skipping ``mdat``; keeping
|
||||
# a blind tail here would carry coded media bytes.
|
||||
return b""
|
||||
|
||||
tail = read_file_tail(image_path, TAIL_WINDOW)
|
||||
if SAMSUNG_EDITOR_MARKER in tail:
|
||||
# Galaxy AI splits its evidence: the marker sits in the post-EOI trailer, but
|
||||
# the `genAIType` value it is gated on can sit INSIDE the entropy-coded scan.
|
||||
# Keeping only the trailer therefore carries the marker without the value and the
|
||||
# verdict silently drops the Samsung signal, so a marked file keeps the whole
|
||||
# window. Only Samsung-marked files pay for it.
|
||||
return tail
|
||||
marker = {"jpeg": b"\xff\xd9", "png": b"IEND\xae\x42\x60\x82"}.get(container)
|
||||
if marker is None:
|
||||
return tail[-UNKNOWN_TRAILER_WINDOW:]
|
||||
index = tail.rfind(marker)
|
||||
return tail[index + len(marker) :] if index >= 0 else tail[-UNKNOWN_TRAILER_WINDOW:]
|
||||
|
||||
|
||||
def _decoder_info(image_path: Path) -> dict[str, Any]:
|
||||
"""PIL's ``info`` mapping, read once.
|
||||
|
||||
One open for both consumers below. They want different parts of the same mapping
|
||||
(the text keys, and the raw EXIF blob), and opening twice repeats the container
|
||||
header parse and, for a PNG carrying ``zTXt``, the zlib inflate with it.
|
||||
"""
|
||||
try:
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(image_path) as img:
|
||||
# PIL types this mapping with a non-string key union (a DPI tuple key
|
||||
# exists), so the keys are normalized here rather than assumed.
|
||||
return {str(key): value for key, value in img.info.items()}
|
||||
except Exception as exc: # a container PIL cannot open
|
||||
logger.debug("PIL info unavailable for %s: %s", image_path, exc)
|
||||
return {}
|
||||
|
||||
|
||||
def _exif_pairs(info: dict[str, Any]) -> dict[str, str]:
|
||||
"""The 0th-IFD tags the verdict reads, under their tag NAMES.
|
||||
|
||||
Not a convenience: two probes key on names rather than on the raw bytes already
|
||||
in the regions. ``xai_signature_pair`` wants an (ImageDescription, Artist) pair,
|
||||
and ``_external_exif_generator`` looks for Software / Make / Artist /
|
||||
ImageDescription. Ship the bytes alone and both silently return nothing, which
|
||||
is how a collector can silently lose Grok and NovelAI verdicts.
|
||||
"""
|
||||
exif_bytes = info.get("exif")
|
||||
if not exif_bytes:
|
||||
return {}
|
||||
try:
|
||||
import piexif
|
||||
|
||||
loaded = piexif.load(exif_bytes)
|
||||
tags = loaded.get("0th", {})
|
||||
exif_tags = loaded.get("Exif", {})
|
||||
except Exception as exc: # malformed EXIF
|
||||
logger.debug("EXIF parse failed: %s", exc)
|
||||
return {}
|
||||
|
||||
pairs = {
|
||||
name: text
|
||||
for name, tag in (
|
||||
("Software", piexif.ImageIFD.Software),
|
||||
("Make", piexif.ImageIFD.Make),
|
||||
("Artist", piexif.ImageIFD.Artist),
|
||||
("ImageDescription", piexif.ImageIFD.ImageDescription),
|
||||
)
|
||||
if (text := exif_text(tags, tag))
|
||||
}
|
||||
if text := exif_text(exif_tags, piexif.ExifIFD.UserComment):
|
||||
pairs["UserComment"] = text
|
||||
return pairs
|
||||
|
||||
|
||||
def _pil_info(info: dict[str, Any]) -> dict[str, str]:
|
||||
"""PIL's ``info`` mapping as strings, the source of PNG text keys and ``hf-job-id``."""
|
||||
|
||||
def text_of(value: Any) -> str:
|
||||
return value.decode("utf-8", "replace") if isinstance(value, bytes) else str(value)
|
||||
|
||||
# Emitted in the file path's own candidate order. ``generator_from_metadata``
|
||||
# returns the FIRST candidate carrying a known token. A record using PIL's natural
|
||||
# dict order can therefore choose a different platform string than the file path,
|
||||
# even though the two paths are supposed to be indistinguishable.
|
||||
out: dict[str, str] = {}
|
||||
for key in _GENERATOR_TEXT_KEYS:
|
||||
value = info.get(key)
|
||||
if value is not None and not isinstance(value, (dict, list, tuple)):
|
||||
out[f"info:{key}"] = text_of(value)
|
||||
for key, value in info.items():
|
||||
if key in _GENERATOR_TEXT_KEYS or isinstance(value, (dict, list, tuple)):
|
||||
continue
|
||||
out[f"info:{key}"] = text_of(value)
|
||||
return out
|
||||
|
||||
|
||||
def collect_metadata_record(
|
||||
image_path: Path,
|
||||
*,
|
||||
schema_version: int = METADATA_RECORD_SCHEMA_VERSION,
|
||||
) -> dict[str, Any]:
|
||||
"""Collect everything the provenance verdict reads, as a JSON-safe record.
|
||||
|
||||
The record is the transport format for
|
||||
:func:`identify.evidence_from_metadata_record`: it carries the metadata regions
|
||||
(base64), the C2PA manifest store, and PIL's info mapping without carrying the
|
||||
primary coded-pixel stream. Schema and collection status are explicit so a
|
||||
consumer cannot mistake a failed read for an unknown provenance verdict.
|
||||
|
||||
Args:
|
||||
image_path: Path to the image.
|
||||
schema_version: Output schema implemented by the consumer.
|
||||
|
||||
Returns:
|
||||
A versioned JSON-serializable dict. ``metadata_base64`` holds the
|
||||
concatenated container regions, ``tail_base64`` the file trailer.
|
||||
"""
|
||||
schema_version = require_schema_version(
|
||||
schema_version,
|
||||
contract="provenance metadata",
|
||||
supported=(1,),
|
||||
)
|
||||
|
||||
from remove_ai_watermarks._internal.c2pa import read_manifest_store_json
|
||||
|
||||
try:
|
||||
image_path.stat()
|
||||
status = "complete"
|
||||
issues: list[dict[str, str]] = []
|
||||
except OSError as exc:
|
||||
logger.debug("metadata source unavailable for %s: %s", image_path, exc)
|
||||
status = "error"
|
||||
issues = [{"stage": "source", "code": "unavailable"}]
|
||||
|
||||
container, regions = _container_regions(image_path, _raw_head(image_path))
|
||||
|
||||
info = _decoder_info(image_path)
|
||||
record: dict[str, Any] = {
|
||||
"schema_version": schema_version,
|
||||
"record_type": METADATA_RECORD_TYPE,
|
||||
"status": status,
|
||||
"issues": issues,
|
||||
"container": container,
|
||||
"name": image_path.name,
|
||||
"metadata_base64": base64.b64encode(regions).decode("ascii"),
|
||||
# Always collected: Samsung's Galaxy AI marker is a post-EOI trailer, and a
|
||||
# record without it loses that verdict outright.
|
||||
"tail_base64": base64.b64encode(_trailer(image_path, container)).decode("ascii"),
|
||||
# PIL info BEFORE exif: the file path prefers a PNG text tag over an EXIF
|
||||
# one, and the normalizer walks the record in insertion order.
|
||||
"pil": _pil_info(info),
|
||||
"exif": _exif_pairs(info),
|
||||
}
|
||||
store = read_manifest_store_json(image_path)
|
||||
if store is not None:
|
||||
record["c2pa_store"] = store
|
||||
return record
|
||||
@@ -61,6 +61,10 @@ _MASK_W, _MASK_H = 0.205, 0.115 # width of W, height of W
|
||||
# invisible inpaint. Threshold = median Sobel magnitude over the footprint box at a
|
||||
# normalized width. The reliable bottom-right wordmark arm is NOT texture-gated:
|
||||
# a wordmark-confirmed pill is removed regardless.
|
||||
#
|
||||
# Measured through the PRODUCT path (the `_keep_pill` gate), not the raw detector, by
|
||||
# ``scripts/pill_gate_audit.py`` -- the raw path bypasses the gate and reads as a
|
||||
# disaster that the shipped behaviour does not have. Re-run it when the gate changes.
|
||||
_FLAT_TEXTURE_MAX = 6.0
|
||||
|
||||
_silhouette: NDArray[Any] | None = None
|
||||
@@ -163,8 +167,11 @@ class PillEngine:
|
||||
box = self._footprint_box(image)
|
||||
if box is None:
|
||||
return None
|
||||
# Same primitive the shared fill uses, rather than a private zeros/fill copy.
|
||||
# `dilate=0` because this footprint is already generous by construction; the
|
||||
# box is clamped to the frame in _footprint_box and both origins are positive
|
||||
# fractions, so boxes_to_mask's own clamping is a no-op here.
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
x0, y0, x1, y1 = box
|
||||
h, w = image.shape[:2]
|
||||
mask = np.zeros((h, w), np.uint8)
|
||||
mask[y0:y1, x0:x1] = 255
|
||||
return mask
|
||||
return region_eraser.boxes_to_mask(image.shape[:2], [(x0, y0, x1 - x0, y1 - y0)], dilate=0)
|
||||
@@ -0,0 +1,484 @@
|
||||
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportMissingTypeStubs=false
|
||||
"""The complete pixel-forensics layer for one image.
|
||||
|
||||
STATUS
|
||||
|
||||
Independent from provenance verdicts, removal, and the CLI. Consumers use the
|
||||
versioned :meth:`PixelEvidence.to_dict` boundary; feature extraction failures are
|
||||
reported per family without discarding successful measurements.
|
||||
|
||||
WHAT IS MEASURED
|
||||
|
||||
One decode, then six families of scale-robust statistics over it:
|
||||
|
||||
* ``dct`` -- AC coefficient histograms over the 8x8 block DCT, plus the deviation of
|
||||
leading digits from Benford's law.
|
||||
* ``fft`` -- radial band energies of the log-magnitude spectrum, plus the
|
||||
color-filter-array periodicity peaks a demosaiced camera capture leaves.
|
||||
* ``noise`` -- standard deviation and kurtosis of a high-pass residual.
|
||||
* ``ela`` -- error level after a quality-90 JPEG re-save.
|
||||
* ``gradient`` -- gradient-magnitude histogram and Laplacian variance.
|
||||
* ``color`` -- 4x4x4 RGB histogram, mean saturation, mean value.
|
||||
|
||||
and, in ``artifacts``, the spatial layer those statistics are computed from: a
|
||||
64-bit perceptual hash, a 128px JPEG thumbnail, and coarse ELA, noise-residual and
|
||||
FFT-phase maps.
|
||||
|
||||
THE ARTIFACTS ARE NOT AGGREGATES
|
||||
|
||||
Everything above ``artifacts`` is a scalar or a fixed-length histogram, and an image
|
||||
cannot be reconstructed from those. ``artifacts`` is different in kind: a thumbnail
|
||||
is a picture, a perceptual hash identifies one, and the coarse maps carry layout.
|
||||
Collecting them makes a record that identifies the source image, so a caller storing
|
||||
or forwarding them is handling image content, not statistics about it. That is why
|
||||
they are a separate field and not merged into the families.
|
||||
|
||||
REQUIREMENTS
|
||||
|
||||
Needs the ``pixels`` extra (numpy). Guard a call with :func:`is_available` when the
|
||||
caller must not hard-depend on it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import io
|
||||
import logging
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks._internal.schema import require_schema_version
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Analysis resolution. Every statistic here is scale-robust, and a 2048px cap keeps
|
||||
# the FFT and the sliding-window residual bounded on a 100 MP input.
|
||||
MAX_SIDE = 2048
|
||||
# The eight lowest-frequency AC positions of the 8x8 block DCT, zig-zag order.
|
||||
AC_POSITIONS = ((0, 1), (1, 0), (1, 1), (0, 2), (2, 0), (2, 1), (1, 2), (0, 3))
|
||||
FFT_BANDS = 8
|
||||
# A Bayer CFA shows as symmetric peaks at half the Nyquist on the diagonals.
|
||||
BAYER_OFFSETS = ((1, 1), (1, -1))
|
||||
INSTALL_HINT = "install the pixel extra: uv add 'remove-ai-watermarks[pixels]'"
|
||||
PIXEL_EVIDENCE_SCHEMA_VERSION = 1
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PixelEvidence:
|
||||
"""Pixel statistics for one image, and the spatial artifacts behind them.
|
||||
|
||||
``decode`` carries the source dimensions, or ``{"error": ...}`` when the image
|
||||
could not be decoded -- in which case every other field is empty. A family is also
|
||||
empty when the image is too small for it (the block DCT needs 8x8, the FFT 32x32,
|
||||
the residual 3x3), so a caller must treat every field as optional rather than
|
||||
assume a fixed feature width.
|
||||
"""
|
||||
|
||||
path: Path
|
||||
decode: dict[str, Any]
|
||||
dct: dict[str, Any] = field(default_factory=dict[str, Any])
|
||||
fft: dict[str, Any] = field(default_factory=dict[str, Any])
|
||||
noise: dict[str, Any] = field(default_factory=dict[str, Any])
|
||||
ela: dict[str, Any] = field(default_factory=dict[str, Any])
|
||||
gradient: dict[str, Any] = field(default_factory=dict[str, Any])
|
||||
color: dict[str, Any] = field(default_factory=dict[str, Any])
|
||||
# Identifies the source image; see the module note. Empty unless asked for.
|
||||
artifacts: dict[str, Any] = field(default_factory=dict[str, Any])
|
||||
# Opt-in timings for callers measuring pipeline latency. Empty by default so
|
||||
# repeated evidence collection remains value-deterministic.
|
||||
timing_ms: dict[str, float] = field(default_factory=dict[str, float])
|
||||
|
||||
@property
|
||||
def decoded(self) -> bool:
|
||||
"""False when the source could not be decoded at all."""
|
||||
return "error" not in self.decode
|
||||
|
||||
@property
|
||||
def status(self) -> str:
|
||||
"""``complete``, ``partial`` for a failed family, or ``error`` on decode."""
|
||||
if not self.decoded:
|
||||
return "error"
|
||||
sections = (self.dct, self.fft, self.noise, self.ela, self.gradient, self.color, self.artifacts)
|
||||
return "partial" if any("error" in section for section in sections) else "complete"
|
||||
|
||||
def to_dict(
|
||||
self,
|
||||
*,
|
||||
schema_version: int = PIXEL_EVIDENCE_SCHEMA_VERSION,
|
||||
) -> dict[str, Any]:
|
||||
"""Return the selected JSON-safe transport schema without a local path."""
|
||||
schema_version = require_schema_version(
|
||||
schema_version,
|
||||
contract="pixel evidence",
|
||||
supported=(1,),
|
||||
)
|
||||
return {
|
||||
"schema_version": schema_version,
|
||||
"status": self.status,
|
||||
"decode": dict(self.decode),
|
||||
"dct": dict(self.dct),
|
||||
"fft": dict(self.fft),
|
||||
"noise": dict(self.noise),
|
||||
"ela": dict(self.ela),
|
||||
"gradient": dict(self.gradient),
|
||||
"color": dict(self.color),
|
||||
"artifacts": dict(self.artifacts),
|
||||
"timing_ms": dict(self.timing_ms),
|
||||
}
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
"""True when the optional pixel dependencies are installed."""
|
||||
from remove_ai_watermarks.optional_deps import module_available
|
||||
|
||||
return module_available("numpy")
|
||||
|
||||
|
||||
def _numpy() -> Any:
|
||||
from remove_ai_watermarks.optional_deps import module_available
|
||||
|
||||
if not module_available("numpy"):
|
||||
raise RuntimeError(f"Pixel evidence needs numpy -- {INSTALL_HINT}")
|
||||
import numpy as np
|
||||
|
||||
return np
|
||||
|
||||
|
||||
def _dct_matrix(np: Any, n: int = 8) -> Any:
|
||||
"""Orthonormal n x n DCT-II basis: M[i, j] = cos(pi (2j + 1) i / 2n)."""
|
||||
i = np.arange(n)[:, None]
|
||||
j = np.arange(n)[None, :]
|
||||
m = np.cos(np.pi * (2 * j + 1) * i / (2 * n))
|
||||
m[0, :] *= 1 / np.sqrt(2)
|
||||
return m * np.sqrt(2 / n)
|
||||
|
||||
|
||||
def read_gray(image_path: Path) -> tuple[Any, Any, dict[str, Any]]:
|
||||
"""Decode to float32 grayscale (and RGB for color stats), downscaled.
|
||||
|
||||
Pillow, not cv2, and the source dimensions are recorded BEFORE the downscale.
|
||||
"""
|
||||
np = _numpy()
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
try:
|
||||
image_io._register_heif() # pyright: ignore[reportPrivateUsage]
|
||||
with Image.open(image_path) as img:
|
||||
info: dict[str, Any] = {"width": img.width, "height": img.height}
|
||||
if max(img.size) > MAX_SIDE:
|
||||
img.thumbnail((MAX_SIDE, MAX_SIDE), Image.Resampling.LANCZOS)
|
||||
rgb = np.asarray(img.convert("RGB"), dtype=np.float32)
|
||||
gray = np.asarray(img.convert("L"), dtype=np.float32)
|
||||
except Exception as exc:
|
||||
logger.debug("pixel decode failed for %s: %s", image_path, exc)
|
||||
# Exception text from Pillow commonly embeds the absolute source path.
|
||||
# Keep that detail in the log, not in the pathless transport contract.
|
||||
return None, None, {"error": type(exc).__name__}
|
||||
return gray, rgb, info
|
||||
|
||||
|
||||
def dct_features(gray: Any) -> dict[str, Any]:
|
||||
"""AC coefficient histograms over the 8x8 block DCT + Benford deviation."""
|
||||
np = _numpy()
|
||||
height, width = gray.shape
|
||||
h8, w8 = height // 8 * 8, width // 8 * 8
|
||||
if h8 < 8 or w8 < 8:
|
||||
return {}
|
||||
basis = _dct_matrix(np)
|
||||
bins = np.linspace(-20.5, 20.5, 22)
|
||||
blocks = gray[:h8, :w8].reshape(h8 // 8, 8, w8 // 8, 8).swapaxes(1, 2)
|
||||
rows = basis[[row for row, _ in AC_POSITIONS]]
|
||||
columns = basis[[column for _, column in AC_POSITIONS]]
|
||||
coeff = np.einsum("ki,abij,kj->abk", rows, blocks, columns)
|
||||
hists = []
|
||||
lead_vals: list[Any] = []
|
||||
for index in range(len(AC_POSITIONS)):
|
||||
values = coeff[:, :, index].ravel()
|
||||
hists.append(np.histogram(values, bins=bins)[0].tolist())
|
||||
lead_vals.append(np.abs(values))
|
||||
out: dict[str, Any] = {"dct_ac_hist": hists}
|
||||
flat = np.abs(np.concatenate(lead_vals))
|
||||
flat = flat[flat >= 1]
|
||||
if flat.size > 100:
|
||||
leading = (flat / 10 ** np.floor(np.log10(flat))).astype(int)
|
||||
leading = leading[(leading >= 1) & (leading <= 9)]
|
||||
if leading.size > 100:
|
||||
observed = np.bincount(leading, minlength=10)[1:10] / leading.size
|
||||
benford = np.log10(1 + 1 / np.arange(1, 10))
|
||||
out["benford_mad"] = float(np.abs(observed - benford).mean())
|
||||
return out
|
||||
|
||||
|
||||
def noise_residual_map(gray: Any) -> Any:
|
||||
"""High-pass residual, the map the noise statistics are computed from."""
|
||||
np = _numpy()
|
||||
from numpy.lib.stride_tricks import sliding_window_view
|
||||
|
||||
if gray.shape[0] < 3 or gray.shape[1] < 3:
|
||||
return None
|
||||
kernel = np.array([[-1.0, -1.0, -1.0], [-1.0, 8.0, -1.0], [-1.0, -1.0, -1.0]])
|
||||
height, width = gray.shape
|
||||
# kernel is float64, so the residual is float64 like the unchunked form
|
||||
out = np.empty((height - 2, width - 2), dtype=np.float64)
|
||||
# Row-chunked: the (window * kernel) temporary is ~150 MB at 2048px if
|
||||
# materialized whole. Per-element 9-tap sums are computed in the same order,
|
||||
# so the result is bit-identical to the unchunked form.
|
||||
for y0 in range(0, height - 2, 256):
|
||||
y1 = min(y0 + 256, height - 2)
|
||||
window = sliding_window_view(gray[y0 : y1 + 2], (3, 3))
|
||||
out[y0:y1] = (window * kernel).sum(axis=(-1, -2))
|
||||
return out
|
||||
|
||||
|
||||
def noise_features(residual: Any) -> dict[str, Any]:
|
||||
"""High-pass residual std and kurtosis."""
|
||||
flat = residual.ravel()
|
||||
std = float(flat.std())
|
||||
if std < 1e-9:
|
||||
return {"noise_std": 0.0, "noise_kurtosis": 0.0}
|
||||
z = (flat - flat.mean()) / std
|
||||
return {"noise_std": std, "noise_kurtosis": float((z**4).mean() - 3.0)}
|
||||
|
||||
|
||||
def fft_decompose(gray: Any) -> tuple[Any, Any] | None:
|
||||
"""Log-magnitude (fftshifted) and phase of the image spectrum."""
|
||||
np = _numpy()
|
||||
if min(gray.shape) < 32:
|
||||
return None
|
||||
spectrum = np.fft.fftshift(np.fft.fft2(gray - gray.mean()))
|
||||
return np.log1p(np.abs(spectrum)), np.angle(spectrum)
|
||||
|
||||
|
||||
def fft_features(mag: Any) -> dict[str, Any]:
|
||||
"""Radial magnitude band energies (no phase) + CFA periodicity peaks."""
|
||||
np = _numpy()
|
||||
height, width = mag.shape
|
||||
cy, cx = height // 2, width // 2
|
||||
# 1D broadcast instead of an mgrid: saves ~160 MB of int64 temporaries at
|
||||
# 2048px. The squares are exact in float64 (values < 2^53), so band means
|
||||
# are identical to the mgrid form.
|
||||
r2y = (np.arange(height, dtype=np.float64) - cy) ** 2
|
||||
r2x = (np.arange(width, dtype=np.float64) - cx) ** 2
|
||||
radius = np.sqrt(r2y[:, None] + r2x[None, :])
|
||||
r_max = radius.max()
|
||||
bands = []
|
||||
for index in range(FFT_BANDS):
|
||||
mask = (radius >= r_max * index / FFT_BANDS) & (radius < r_max * (index + 1) / FFT_BANDS)
|
||||
bands.append(float(mag[mask].mean()) if mask.any() else 0.0)
|
||||
peaks = []
|
||||
for dy, dx in BAYER_OFFSETS:
|
||||
y, x = cy + dy * (height // 4), cx + dx * (width // 4)
|
||||
neighborhood = mag[y - 2 : y + 3, x - 2 : x + 3]
|
||||
peaks.append(float(neighborhood.max() - mag.mean()))
|
||||
return {"fft_band_energy": bands, "cfa_peaks": peaks, "cfa_peak": max(peaks)}
|
||||
|
||||
|
||||
def ela_map(rgb: Any) -> Any:
|
||||
"""Absolute per-pixel error after a quality-90 JPEG re-save."""
|
||||
np = _numpy()
|
||||
from PIL import Image
|
||||
|
||||
try:
|
||||
buffer = io.BytesIO()
|
||||
Image.fromarray(rgb.astype(np.uint8)).save(buffer, "JPEG", quality=90)
|
||||
buffer.seek(0)
|
||||
resaved = np.asarray(Image.open(buffer).convert("RGB"), dtype=np.float32)
|
||||
except Exception as exc:
|
||||
logger.debug("ELA re-save failed: %s", exc)
|
||||
return None
|
||||
if resaved.shape != rgb.shape:
|
||||
return None
|
||||
return np.abs(rgb - resaved).mean(axis=-1)
|
||||
|
||||
|
||||
def ela_features(err: Any) -> dict[str, Any]:
|
||||
"""Error-level stats after a quality-90 JPEG re-save."""
|
||||
np = _numpy()
|
||||
return {"ela_mean": float(err.mean()), "ela_p95": float(np.percentile(err, 95))}
|
||||
|
||||
|
||||
def gradient_features(gray: Any) -> dict[str, Any]:
|
||||
np = _numpy()
|
||||
gy, gx = np.gradient(gray)
|
||||
mag = np.sqrt(gx**2 + gy**2)
|
||||
hist = np.histogram(mag, bins=10, range=(0, 255))[0].tolist()
|
||||
laplacian = np.gradient(gy, axis=0) + np.gradient(gx, axis=1)
|
||||
return {"gradient_hist": hist, "laplacian_var": float(laplacian.var())}
|
||||
|
||||
|
||||
def color_features(rgb: Any) -> dict[str, Any]:
|
||||
np = _numpy()
|
||||
small = rgb[::4, ::4] # decimate; the histogram is position-blind anyway
|
||||
bins = (small / 256 * 4).astype(int).clip(0, 3)
|
||||
index = bins[..., 0] * 16 + bins[..., 1] * 4 + bins[..., 2]
|
||||
hist = np.bincount(index.ravel(), minlength=64).tolist()
|
||||
mx = small.max(axis=-1)
|
||||
mn = small.min(axis=-1)
|
||||
saturation = np.where(mx > 0, (mx - mn) / np.maximum(mx, 1e-6), 0)
|
||||
return {
|
||||
"color_hist_4x4x4": hist,
|
||||
"saturation_mean": float(saturation.mean()),
|
||||
"value_mean": float(mx.mean() / 255),
|
||||
}
|
||||
|
||||
|
||||
def perceptual_hash(gray: Any) -> str:
|
||||
"""64-bit DCT perceptual hash. Identifies an image; see the module note."""
|
||||
np = _numpy()
|
||||
from PIL import Image
|
||||
|
||||
small = np.asarray(Image.fromarray(gray.astype(np.float32), mode="F").resize((32, 32), Image.Resampling.LANCZOS))
|
||||
basis = _dct_matrix(np, 32)
|
||||
low_basis = basis[:8]
|
||||
low = (low_basis @ small @ low_basis.T).ravel()[1:] # drop DC
|
||||
bits = low > np.median(low)
|
||||
return f"{int(''.join('1' if bit else '0' for bit in bits), 2):016x}"
|
||||
|
||||
|
||||
def _coarse(np: Any, arr: Any, side: int = 64) -> Any:
|
||||
"""Downscale a 2D map to at most ``side`` on the long edge."""
|
||||
from PIL import Image
|
||||
|
||||
height, width = arr.shape
|
||||
if max(height, width) <= side:
|
||||
return arr
|
||||
img = Image.fromarray(arr.astype(np.float32), mode="F")
|
||||
img.thumbnail((side, side), Image.Resampling.BILINEAR)
|
||||
return np.asarray(img)
|
||||
|
||||
|
||||
def _array_payload(arr: Any) -> dict[str, Any]:
|
||||
return {
|
||||
"shape": list(arr.shape),
|
||||
"dtype": str(arr.dtype),
|
||||
"base64": base64.b64encode(arr.tobytes()).decode("ascii"),
|
||||
}
|
||||
|
||||
|
||||
def spatial_artifacts(gray: Any, rgb: Any, *, ela: Any, residual: Any, phase: Any) -> dict[str, Any]:
|
||||
"""Perceptual hash, thumbnail, and coarse ELA / residual / phase maps.
|
||||
|
||||
These identify the source image rather than describe it -- see the module note.
|
||||
The maps are the ones the statistics were computed from, passed in rather than
|
||||
recomputed.
|
||||
"""
|
||||
np = _numpy()
|
||||
from PIL import Image
|
||||
|
||||
out: dict[str, Any] = {"phash": perceptual_hash(gray)}
|
||||
thumbnail = Image.fromarray(rgb.astype(np.uint8))
|
||||
thumbnail.thumbnail((128, 128), Image.Resampling.LANCZOS)
|
||||
buffer = io.BytesIO()
|
||||
thumbnail.save(buffer, "JPEG", quality=70)
|
||||
out["thumbnail_jpeg_b64"] = base64.b64encode(buffer.getvalue()).decode("ascii")
|
||||
|
||||
if ela is not None:
|
||||
out["ela_map"] = _array_payload(_coarse(np, ela))
|
||||
if residual is not None:
|
||||
clipped = np.clip(residual / 4.0, -1, 1)
|
||||
out["noise_residual"] = _array_payload(_coarse(np, (clipped * 127).astype(np.int8)))
|
||||
if phase is not None:
|
||||
out["fft_phase"] = _array_payload(_coarse(np, phase.astype(np.float32), 32))
|
||||
return out
|
||||
|
||||
|
||||
def extract_pixel_evidence(image_path: Path, *, artifacts: bool = False, timings: bool = False) -> PixelEvidence:
|
||||
"""Measure every pixel-statistic family for one image in a single decode.
|
||||
|
||||
The image is decoded ONCE and the intermediate maps (high-pass residual, ELA
|
||||
error, FFT magnitude and phase) are computed once and shared, because the
|
||||
residual's sliding window and the ELA re-save are the two expensive steps and
|
||||
each family would otherwise redo them.
|
||||
|
||||
A family that fails or does not apply is left empty rather than raising: an
|
||||
undecodable file, or one too small for the block DCT, still returns a
|
||||
:class:`PixelEvidence` whose ``decoded`` / empty fields say so. Missing numpy is
|
||||
the one hard error, since then nothing can be measured at all.
|
||||
|
||||
Args:
|
||||
image_path: Path to the image. Any container Pillow can open.
|
||||
artifacts: Also return the spatial layer -- perceptual hash, thumbnail and
|
||||
coarse maps. Off by default: those identify the source image, so asking
|
||||
for them is a decision the caller makes explicitly.
|
||||
timings: Measure each stage and include rounded milliseconds in
|
||||
:attr:`PixelEvidence.timing_ms`.
|
||||
|
||||
Returns:
|
||||
A :class:`PixelEvidence`.
|
||||
"""
|
||||
started = time.perf_counter()
|
||||
stage_started = started
|
||||
measured: dict[str, float] = {}
|
||||
|
||||
gray, rgb, info = read_gray(image_path)
|
||||
measured["decode"] = time.perf_counter() - stage_started
|
||||
if gray is None or rgb is None:
|
||||
measured["total"] = time.perf_counter() - started
|
||||
timing_ms = {name: round(seconds * 1000, 1) for name, seconds in measured.items()} if timings else {}
|
||||
return PixelEvidence(path=image_path, decode=info, timing_ms=timing_ms)
|
||||
|
||||
families: dict[str, dict[str, Any]] = {}
|
||||
|
||||
residual = None
|
||||
stage_started = time.perf_counter()
|
||||
try:
|
||||
residual = noise_residual_map(gray)
|
||||
families["noise"] = noise_features(residual) if residual is not None else {}
|
||||
except Exception as exc:
|
||||
logger.debug("pixel family noise failed for %s: %s", image_path, exc)
|
||||
families["noise"] = {"error": type(exc).__name__}
|
||||
measured["noise"] = time.perf_counter() - stage_started
|
||||
|
||||
spectrum = None
|
||||
stage_started = time.perf_counter()
|
||||
try:
|
||||
spectrum = fft_decompose(gray)
|
||||
families["fft"] = fft_features(spectrum[0]) if spectrum is not None else {}
|
||||
except Exception as exc:
|
||||
logger.debug("pixel family fft failed for %s: %s", image_path, exc)
|
||||
families["fft"] = {"error": type(exc).__name__}
|
||||
measured["fft"] = time.perf_counter() - stage_started
|
||||
|
||||
error = None
|
||||
stage_started = time.perf_counter()
|
||||
try:
|
||||
error = ela_map(rgb)
|
||||
families["ela"] = ela_features(error) if error is not None else {}
|
||||
except Exception as exc:
|
||||
logger.debug("pixel family ela failed for %s: %s", image_path, exc)
|
||||
families["ela"] = {"error": type(exc).__name__}
|
||||
measured["ela"] = time.perf_counter() - stage_started
|
||||
|
||||
for name, compute in (
|
||||
("dct", lambda: dct_features(gray)),
|
||||
("gradient", lambda: gradient_features(gray)),
|
||||
("color", lambda: color_features(rgb)),
|
||||
):
|
||||
stage_started = time.perf_counter()
|
||||
try:
|
||||
families[name] = compute()
|
||||
except Exception as exc: # one bad family must not lose the other five
|
||||
logger.debug("pixel family %s failed for %s: %s", name, image_path, exc)
|
||||
families[name] = {"error": type(exc).__name__}
|
||||
measured[name] = time.perf_counter() - stage_started
|
||||
|
||||
if artifacts:
|
||||
stage_started = time.perf_counter()
|
||||
try:
|
||||
families["artifacts"] = spatial_artifacts(
|
||||
gray, rgb, ela=error, residual=residual, phase=spectrum[1] if spectrum is not None else None
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.debug("pixel artifacts failed for %s: %s", image_path, exc)
|
||||
families["artifacts"] = {"error": type(exc).__name__}
|
||||
measured["full_artifacts"] = time.perf_counter() - stage_started
|
||||
|
||||
measured["total"] = time.perf_counter() - started
|
||||
timing_ms = {name: round(seconds * 1000, 1) for name, seconds in measured.items()} if timings else {}
|
||||
return PixelEvidence(path=image_path, decode=info, timing_ms=timing_ms, **families)
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Qwen (Tongyi Qianwen, Alibaba) visible watermark detector/localizer.
|
||||
"""Qwen (Alibaba) visible watermark detector/localizer.
|
||||
|
||||
Qwen stamps its generations with a visible "千问AI生成" text strip in the
|
||||
bottom-right corner -- the explicit AIGC label mandated by China's GB 45438-2025
|
||||
@@ -52,11 +52,9 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image SHORT side (measured basis -- see
|
||||
@@ -122,8 +120,6 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
QwenDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Qwen alpha template (float [0,1]), or None."""
|
||||
@@ -135,23 +131,8 @@ def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the Qwen glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class QwenEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible Qwen "千问AI生成" watermark (locate -> mask; mask feeds the fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
@@ -21,6 +21,10 @@ Backends:
|
||||
texture but ~200 MB model and ~4.7 GB peak RAM (too heavy for a small host).
|
||||
The model is downloaded on first use and cached by huggingface_hub; it is
|
||||
never bundled in this repo.
|
||||
|
||||
The per-backend RAM and wall-time figures above are reproduced by
|
||||
``scripts/resource_ceilings.py`` (fresh subprocess per measurement, synthetic inputs).
|
||||
Re-run it before changing any of them.
|
||||
"""
|
||||
|
||||
# cv2/numpy boundary: cv2 ships no usable type info, so strict pyright cannot know
|
||||
@@ -30,6 +34,7 @@ Backends:
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any, Literal
|
||||
|
||||
import cv2
|
||||
@@ -48,9 +53,38 @@ _LAMA_FILE = "lama_fp32.onnx"
|
||||
_MIGAN_REPO = "andraniksargsyan/migan"
|
||||
_MIGAN_FILE = "migan.onnx"
|
||||
|
||||
# Cached onnxruntime sessions (loading is expensive; reuse across calls).
|
||||
_lama_session: object | None = None
|
||||
_migan_session: object | None = None
|
||||
# Cached onnxruntime sessions, keyed by backend name (loading is expensive; reuse
|
||||
# across calls).
|
||||
_sessions: dict[str, object] = {}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _LearnedBackend:
|
||||
"""One optional model-backed fill: its human label and the module-level names of
|
||||
its availability probe and erase function.
|
||||
|
||||
NAMES, not function objects: the fallback tests monkeypatch these by attribute on
|
||||
the module, and a table holding bound references would not see the patch.
|
||||
"""
|
||||
|
||||
label: str
|
||||
available: str
|
||||
erase: str
|
||||
|
||||
|
||||
# The learned tier, best-quality first. `resolve_backend`'s preference order and the
|
||||
# CLI's choices are separate literals by design (see FILL_BACKENDS below), but the
|
||||
# availability probe, install hint and dispatch all read this one table.
|
||||
_LEARNED_BACKENDS: dict[str, _LearnedBackend] = {
|
||||
"lama": _LearnedBackend("LaMa", "lama_available", "erase_lama"),
|
||||
"migan": _LearnedBackend("MI-GAN", "migan_available", "erase_migan"),
|
||||
}
|
||||
|
||||
# Every fill backend this module can execute, plus the caller-facing `auto`. Kept as a
|
||||
# literal tuple (not derived through an import) so `watermark_registry` and the CLI can
|
||||
# state their choices without importing this cv2-loading module at their import time --
|
||||
# a test pins the two in sync.
|
||||
FILL_BACKENDS: tuple[str, ...] = ("auto", "cv2", "migan", "lama")
|
||||
|
||||
|
||||
def boxes_to_mask(
|
||||
@@ -119,19 +153,25 @@ def lama_available() -> bool:
|
||||
return module_available("onnxruntime")
|
||||
|
||||
|
||||
def _get_lama_session() -> object:
|
||||
"""Load (once) the big-LaMa ONNX session, downloading the model on first use."""
|
||||
global _lama_session
|
||||
if _lama_session is not None:
|
||||
return _lama_session
|
||||
def _get_session(name: str, repo_id: str, filename: str, label: str) -> object:
|
||||
"""Load (once) an ONNX session, downloading the model on first use."""
|
||||
cached = _sessions.get(name)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
import onnxruntime as ort
|
||||
from huggingface_hub import hf_hub_download
|
||||
|
||||
model_path = hf_hub_download(repo_id=_LAMA_REPO, filename=_LAMA_FILE)
|
||||
logger.info("Loading LaMa-ONNX model: %s", model_path)
|
||||
_lama_session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
|
||||
return _lama_session
|
||||
model_path = hf_hub_download(repo_id=repo_id, filename=filename)
|
||||
logger.info("Loading %s model: %s", label, model_path)
|
||||
session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
|
||||
_sessions[name] = session
|
||||
return session
|
||||
|
||||
|
||||
def _get_lama_session() -> object:
|
||||
"""The big-LaMa ONNX session (kept as a named seam the tests monkeypatch)."""
|
||||
return _get_session("lama", _LAMA_REPO, _LAMA_FILE, "LaMa-ONNX")
|
||||
|
||||
|
||||
def erase_lama(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]:
|
||||
@@ -196,25 +236,21 @@ def erase_lama(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]:
|
||||
|
||||
|
||||
def migan_available() -> bool:
|
||||
"""True when the optional MI-GAN backend can run (onnxruntime installed)."""
|
||||
"""True when the optional MI-GAN backend can run (onnxruntime installed).
|
||||
|
||||
Deliberately a separate function from :func:`lama_available` even though both
|
||||
currently reduce to the same onnxruntime probe: they are independent capability
|
||||
questions and `auto` resolves them separately (see
|
||||
``watermark_registry.preferred_inpaint_backend``).
|
||||
"""
|
||||
from .optional_deps import module_available
|
||||
|
||||
return module_available("onnxruntime")
|
||||
|
||||
|
||||
def _get_migan_session() -> object:
|
||||
"""Load (once) the MI-GAN ONNX session, downloading the model on first use."""
|
||||
global _migan_session
|
||||
if _migan_session is not None:
|
||||
return _migan_session
|
||||
|
||||
import onnxruntime as ort
|
||||
from huggingface_hub import hf_hub_download
|
||||
|
||||
model_path = hf_hub_download(repo_id=_MIGAN_REPO, filename=_MIGAN_FILE)
|
||||
logger.info("Loading MI-GAN ONNX model: %s", model_path)
|
||||
_migan_session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
|
||||
return _migan_session
|
||||
"""The MI-GAN ONNX session (kept as a named seam the tests monkeypatch)."""
|
||||
return _get_session("migan", _MIGAN_REPO, _MIGAN_FILE, "MI-GAN ONNX")
|
||||
|
||||
|
||||
def erase_migan(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]:
|
||||
@@ -308,16 +344,17 @@ def erase(
|
||||
if not mask.any():
|
||||
return image_bgr.copy()
|
||||
|
||||
if backend == "migan":
|
||||
if not migan_available():
|
||||
learned = _LEARNED_BACKENDS.get(backend)
|
||||
if learned is not None:
|
||||
# Probe and erase by MODULE NAME, not by the captured function object: the
|
||||
# availability probes and the erase functions are monkeypatched by name in the
|
||||
# fallback tests, and a table of bound references would not see those patches.
|
||||
if not globals()[learned.available]():
|
||||
raise RuntimeError(
|
||||
"MI-GAN backend requires onnxruntime. Install the extra: pip install 'remove-ai-watermarks[migan]'"
|
||||
f"{learned.label} backend requires onnxruntime. "
|
||||
f"Install the extra: pip install 'remove-ai-watermarks[{backend}]'"
|
||||
)
|
||||
return erase_migan(image_bgr, mask)
|
||||
if backend == "lama":
|
||||
if not lama_available():
|
||||
raise RuntimeError(
|
||||
"LaMa backend requires onnxruntime. Install the extra: pip install 'remove-ai-watermarks[lama]'"
|
||||
)
|
||||
return erase_lama(image_bgr, mask)
|
||||
return globals()[learned.erase](image_bgr, mask)
|
||||
# cv2 and anything unrecognized (including "auto", which a library caller may pass
|
||||
# straight through) degrade to the classical fill rather than raising.
|
||||
return erase_cv2(image_bgr, mask, method=cv2_method, radius=cv2_radius)
|
||||
@@ -48,13 +48,16 @@ import logging
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkScan,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image WIDTH (the measured basis: every
|
||||
@@ -111,24 +114,12 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
RunningHubDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled RunningHub alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
"""Binary "RunningHub AI生成" silhouette (255 = glyph) from the alpha map, or None."""
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the RunningHub glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class RunningHubEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible RunningHub "RunningHub AI生成" mark (top-left; localize -> fill)."""
|
||||
|
||||
@@ -140,18 +131,21 @@ class RunningHubEngine(TextMarkEngine):
|
||||
_ANCHOR_MAX_X = 0.025
|
||||
_ANCHOR_MAX_Y = 0.015
|
||||
|
||||
def detect(self, image: NDArray[Any], *, provenance: bool = False) -> TextMarkDetection:
|
||||
det = super().detect(image, provenance=provenance)
|
||||
if not det.detected:
|
||||
def _post_gate(self, det: TextMarkDetection, scan: TextMarkScan) -> TextMarkDetection:
|
||||
"""Demote a match that does not hug the top-left corner.
|
||||
|
||||
A shared post-gate rather than a ``detect`` override, so the single-pass
|
||||
perception path (``detect_both``) cannot skip it.
|
||||
"""
|
||||
if not det.detected or scan.loc is None:
|
||||
return det
|
||||
loc = self.locate(image)
|
||||
_, box = self._gray_best(image, loc)
|
||||
box = det.match_box # the sweep the scan already ran on this same loc
|
||||
if box is None:
|
||||
det.detected = False
|
||||
return det
|
||||
h, w = image.shape[:2]
|
||||
ax = (loc.x + box[0]) / w
|
||||
ay = (loc.y + box[1]) / h
|
||||
h, w = scan.frame
|
||||
ax = (scan.loc.x + box[0]) / w
|
||||
ay = (scan.loc.y + box[1]) / h
|
||||
if ax > self._ANCHOR_MAX_X or ay > self._ANCHOR_MAX_Y:
|
||||
logger.debug(
|
||||
"RunningHub detect: score %.3f but match off-anchor (x=%.3f y=%.3f); demoting.",
|
||||
@@ -161,13 +155,3 @@ class RunningHubEngine(TextMarkEngine):
|
||||
)
|
||||
det.detected = False
|
||||
return det
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
@@ -28,7 +28,7 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -79,8 +79,6 @@ _CONFIG = TextMarkConfig(
|
||||
min_gw=16,
|
||||
)
|
||||
|
||||
SamsungDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Samsung alpha template (float [0,1]), or None."""
|
||||
|
||||
@@ -0,0 +1,299 @@
|
||||
"""Draft operator-verifiable text lines for verified-text manifests.
|
||||
|
||||
Proposal-only OCR: PaddleOCR detection plus three script-chosen recognition
|
||||
engines (Latin / Cyrillic / CJK), accepted only when three crop paddings
|
||||
normalize identically and every confidence clears the floor. ``accepted``
|
||||
means crop-stable, NEVER ground-truth-correct: on the reference posters the
|
||||
draft's exact-text precision was 90.0% and 94.4% because high-confidence OCR
|
||||
still lost punctuation (one English comma dropped, one ideographic comma
|
||||
replaced with ASCII). Every accepted line needs a human yes/no before it may
|
||||
enter a manifest with ``verified: true``.
|
||||
|
||||
Heavy imports (paddle, and the numpy/cv2/PIL pixel stack) stay inside the
|
||||
call so importing this module costs nothing without the ``text-draft`` extra;
|
||||
``draft_available()`` reports whether the extra is installed.
|
||||
"""
|
||||
|
||||
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unicodedata
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks._internal.text_restoration import source_pixel_sha256
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
__all__ = [
|
||||
"DraftLine",
|
||||
"RejectedLine",
|
||||
"TextDraft",
|
||||
"choose_language",
|
||||
"draft_available",
|
||||
"draft_text_lines",
|
||||
"group_word_boxes",
|
||||
"source_pixel_sha256",
|
||||
"stable_recognition",
|
||||
]
|
||||
|
||||
# Crop paddings probed per line: recognition must be invariant across them.
|
||||
JITTER_RATIOS: tuple[float, ...] = (0.08, 0.12, 0.2)
|
||||
DETECT_SCORE_FLOOR = 0.5
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DraftLine:
|
||||
"""One crop-stable proposal. ``accepted`` means stable, not correct."""
|
||||
|
||||
box: tuple[int, int, int, int]
|
||||
text: str
|
||||
script: str
|
||||
language: str
|
||||
min_score: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RejectedLine:
|
||||
"""A detected line whose recognition was unstable or low-confidence."""
|
||||
|
||||
box: tuple[int, int, int, int]
|
||||
script: str
|
||||
language: str
|
||||
reads: tuple[tuple[str, float], ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TextDraft:
|
||||
"""The full proposal set for one image."""
|
||||
|
||||
accepted: tuple[DraftLine, ...] = ()
|
||||
rejected: tuple[RejectedLine, ...] = ()
|
||||
|
||||
|
||||
def normalize_text(text: str) -> str:
|
||||
"""Normalize text for layout-independent comparison (casefold + no spaces)."""
|
||||
return "".join(unicodedata.normalize("NFC", text).casefold().split())
|
||||
|
||||
|
||||
def _has_script(text: str, script: str) -> bool:
|
||||
return any(script in unicodedata.name(character, "") for character in text)
|
||||
|
||||
|
||||
def choose_language(probes: dict[str, tuple[str, float]]) -> str:
|
||||
"""Pick ``ch``/``ru``/``en`` from what each probe engine actually read."""
|
||||
if _has_script(probes["ch"][0], "CJK"):
|
||||
return "ch"
|
||||
if _has_script(probes["ru"][0], "CYRILLIC"):
|
||||
return "ru"
|
||||
return "en"
|
||||
|
||||
|
||||
def stable_recognition(reads: list[tuple[str, float]], min_score: float = 0.85) -> str | None:
|
||||
"""The proposal text when every read normalizes identically and clears the floor."""
|
||||
normalized = {normalize_text(text) for text, _score in reads}
|
||||
if len(normalized) != 1 or min(score for _text, score in reads) < min_score:
|
||||
return None
|
||||
return reads[0][0]
|
||||
|
||||
|
||||
def _vertical_overlap_ratio(left: tuple[int, int, int, int], right: tuple[int, int, int, int]) -> float:
|
||||
overlap = max(0, min(left[3], right[3]) - max(left[1], right[1]))
|
||||
return overlap / max(1, min(left[3] - left[1], right[3] - right[1]))
|
||||
|
||||
|
||||
def _row_center(box: tuple[int, int, int, int]) -> float:
|
||||
return (box[1] + box[3]) / 2
|
||||
|
||||
|
||||
def group_word_boxes(boxes: list[tuple[int, int, int, int]]) -> list[tuple[int, int, int, int]]:
|
||||
"""Merge word detections into line boxes by vertical overlap and gap."""
|
||||
groups: list[tuple[int, int, int, int]] = []
|
||||
for box in sorted(boxes, key=lambda item: (_row_center(item), item[0])):
|
||||
matches: list[int] = []
|
||||
for index, group in enumerate(groups):
|
||||
if _vertical_overlap_ratio(box, group) < 0.45:
|
||||
continue
|
||||
horizontal_gap = max(0, max(box[0], group[0]) - min(box[2], group[2]))
|
||||
line_height = min(box[3] - box[1], group[3] - group[1])
|
||||
if horizontal_gap <= max(24, line_height * 3):
|
||||
matches.append(index)
|
||||
if not matches:
|
||||
groups.append(box)
|
||||
continue
|
||||
index = max(matches, key=lambda item: _vertical_overlap_ratio(box, groups[item]))
|
||||
x1, y1, x2, y2 = groups[index]
|
||||
groups[index] = min(x1, box[0]), min(y1, box[1]), max(x2, box[2]), max(y2, box[3])
|
||||
return sorted(groups, key=_row_center)
|
||||
|
||||
|
||||
def _recognition_box(
|
||||
box: tuple[int, int, int, int],
|
||||
script: str,
|
||||
width: int,
|
||||
height: int,
|
||||
vertical_pad_ratio: float | None = None,
|
||||
) -> tuple[int, int, int, int]:
|
||||
x1, y1, x2, y2 = box
|
||||
line_height = y2 - y1
|
||||
if script == "cjk":
|
||||
left_pad = max(16, round(line_height * 0.2))
|
||||
right_pad = max(16, round(line_height * 0.6))
|
||||
return max(0, x1 - left_pad), y1, min(width, x2 + right_pad), y2
|
||||
pad_x = max(16, line_height)
|
||||
pad_y = max(8, line_height // 3) if vertical_pad_ratio is None else max(8, round(line_height * vertical_pad_ratio))
|
||||
return max(0, x1 - pad_x), max(0, y1 - pad_y), min(width, x2 + pad_x), min(height, y2 + pad_y)
|
||||
|
||||
|
||||
def _recognize(
|
||||
engine: Any,
|
||||
image: Any,
|
||||
box: tuple[int, int, int, int],
|
||||
script: str,
|
||||
vertical_pad_ratio: float | None = None,
|
||||
) -> tuple[str, float]:
|
||||
import cv2
|
||||
|
||||
height, width = image.shape[:2]
|
||||
x1, y1, x2, y2 = _recognition_box(box, script, width, height, vertical_pad_ratio)
|
||||
crop = image[y1:y2, x1:x2]
|
||||
if crop.shape[0] < 64:
|
||||
scale = 64 / crop.shape[0]
|
||||
crop = cv2.resize(crop, None, fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)
|
||||
result = next(iter(engine.predict(crop)))
|
||||
return str(result.get("rec_text", "")), float(result.get("rec_score", 0.0))
|
||||
|
||||
|
||||
def _detect_line_boxes(engine: Any, source_rgb: Any) -> list[tuple[int, int, int, int]]:
|
||||
import numpy as np
|
||||
|
||||
boxes: list[tuple[int, int, int, int]] = []
|
||||
for page in engine.predict(source_rgb):
|
||||
detected = page.get("rec_boxes", None)
|
||||
if detected is None or len(detected) == 0:
|
||||
detected = page.get("rec_polys", [])
|
||||
for score, raw_box in zip(page.get("rec_scores", []), detected, strict=False):
|
||||
if float(score) < DETECT_SCORE_FLOOR:
|
||||
continue
|
||||
points = np.asarray(raw_box, dtype=np.float32).reshape(-1)
|
||||
if points.size == 4:
|
||||
x1, y1, x2, y2 = points
|
||||
else:
|
||||
points = points.reshape(-1, 2)
|
||||
x1, y1 = points.min(axis=0)
|
||||
x2, y2 = points.max(axis=0)
|
||||
boxes.append((round(float(x1)), round(float(y1)), round(float(x2)), round(float(y2))))
|
||||
return group_word_boxes(boxes)
|
||||
|
||||
|
||||
def draft_available() -> bool:
|
||||
"""True when the ``text-draft`` extra (paddleocr + paddle) can run."""
|
||||
from remove_ai_watermarks.optional_deps import module_available
|
||||
|
||||
return module_available("paddleocr") and module_available("paddle")
|
||||
|
||||
|
||||
def _build_engines() -> tuple[Any, dict[str, Any]]:
|
||||
from paddleocr import PaddleOCR, TextRecognition
|
||||
|
||||
# enable_mkldnn=False is load-bearing on linux: paddle's oneDNN PIR executor
|
||||
# raises NotImplementedError on the text-detection graph there (seen on the
|
||||
# Modal draft container, 2026-08-19), while the reference CPU path runs. The
|
||||
# eval scripts never hit it because macOS pads take a different path.
|
||||
detector = PaddleOCR(
|
||||
lang="ch",
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
enable_mkldnn=False,
|
||||
)
|
||||
engines = {
|
||||
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec", enable_mkldnn=False),
|
||||
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec", enable_mkldnn=False),
|
||||
"ch": TextRecognition(model_name="PP-OCRv5_server_rec", enable_mkldnn=False),
|
||||
}
|
||||
return detector, engines
|
||||
|
||||
|
||||
def draft_text_lines(
|
||||
image: Path,
|
||||
*,
|
||||
min_score: float = 0.85,
|
||||
detector: Any | None = None,
|
||||
engines: dict[str, Any] | None = None,
|
||||
stable: bool = True,
|
||||
) -> TextDraft:
|
||||
"""Propose verified-text manifest lines for ``image``; never verified ones.
|
||||
|
||||
Args:
|
||||
image: path of the source image (draft boxes are in ITS pixel space).
|
||||
min_score: recognition confidence floor. With ``stable=True`` every
|
||||
jittered read must clear it; with ``stable=False`` the single
|
||||
probe for the chosen language must.
|
||||
detector/engines: injectable Paddle objects (tests use fakes); when
|
||||
None they are built from the ``text-draft`` extra, which must be
|
||||
installed (``draft_available()`` reports it).
|
||||
stable: when True (default), accept a line only if three crop paddings
|
||||
normalize identically. When False, take one recognition trio and
|
||||
accept on score alone. Automatic restoration uses box/script only,
|
||||
so the jitter gate is optional there.
|
||||
|
||||
Returns:
|
||||
``TextDraft`` with crop-stable ``accepted`` proposals and unstable
|
||||
``rejected`` lines. Both lists need human review before any manifest
|
||||
may claim ``verified: true``; accepted means crop-stable, NOT
|
||||
ground-truth-correct.
|
||||
"""
|
||||
import os
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
if detector is None or engines is None:
|
||||
if not draft_available():
|
||||
raise RuntimeError(
|
||||
"Text drafting requires PaddleOCR. Install: pip install 'remove-ai-watermarks[text-draft]'"
|
||||
)
|
||||
os.environ.setdefault("PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK", "True")
|
||||
detector, engines = _build_engines()
|
||||
|
||||
source_rgb = np.asarray(Image.open(image).convert("RGB"))
|
||||
accepted: list[DraftLine] = []
|
||||
rejected: list[RejectedLine] = []
|
||||
for box in _detect_line_boxes(detector, source_rgb):
|
||||
probes: dict[str, tuple[str, float]] = {}
|
||||
for language, engine in engines.items():
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
probes[language] = _recognize(engine, source_rgb, box, script, 0.1)
|
||||
language = choose_language(probes)
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
if stable:
|
||||
reads = [_recognize(engines[language], source_rgb, box, script, ratio) for ratio in JITTER_RATIOS]
|
||||
text = stable_recognition(reads, min_score)
|
||||
else:
|
||||
text, score = probes[language]
|
||||
reads = [(text, score)]
|
||||
if not text.strip() or score < min_score:
|
||||
text = None
|
||||
if text is None:
|
||||
rejected.append(
|
||||
RejectedLine(
|
||||
box=box,
|
||||
script=script,
|
||||
language=language,
|
||||
reads=tuple((value, score) for value, score in reads),
|
||||
)
|
||||
)
|
||||
else:
|
||||
accepted.append(
|
||||
DraftLine(
|
||||
box=box,
|
||||
text=text,
|
||||
script=script,
|
||||
language=language,
|
||||
min_score=min(score for _value, score in reads),
|
||||
)
|
||||
)
|
||||
return TextDraft(accepted=tuple(accepted), rejected=tuple(rejected))
|
||||
@@ -32,6 +32,9 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
# Adobe ships Variant P in production (com.adobe.trustmark.P).
|
||||
_MODEL_TYPE = "P"
|
||||
# Schema 3 (BCH_3) is below the detector's measured precision threshold; the
|
||||
# calibration history is canonical in docs/module-internals.md.
|
||||
_SUPPORTED_SCHEMAS = frozenset({0, 1, 2})
|
||||
# Lazily constructed singleton -- model load + first-use download is expensive.
|
||||
# Guarded by a lock so concurrent callers don't double-construct/double-download.
|
||||
_tm: Any = None
|
||||
@@ -57,8 +60,9 @@ def _decoder() -> Any:
|
||||
|
||||
|
||||
# JPEG quality for the false-positive durability gate (see detect_trustmark).
|
||||
# Deliberately mild: a genuine TrustMark survives far harsher, while every
|
||||
# observed false positive collapsed even at this quality.
|
||||
# Deliberately mild: a genuine TrustMark survives far harsher. The round-trip
|
||||
# still needs payload and schema validation because content-correlated false
|
||||
# positives can survive this compression level.
|
||||
_REENCODE_QUALITY = 95
|
||||
|
||||
|
||||
@@ -79,8 +83,8 @@ def detect_trustmark(image_path: Path) -> str | None:
|
||||
cannot carry Adobe's watermark, and decoded a random-bytes secret). A genuine
|
||||
TrustMark is a *durable* soft binding engineered to survive re-encoding (that
|
||||
is its entire purpose once C2PA is stripped), so we re-decode after a mild
|
||||
JPEG round-trip and require the same schema both times. Every observed false
|
||||
positive collapsed under this gate.
|
||||
JPEG round-trip and require the same binary payload and schema both times.
|
||||
Only the calibrated schemas 0-2 count as high-precision positives.
|
||||
"""
|
||||
if not is_available():
|
||||
return None
|
||||
@@ -90,10 +94,17 @@ def detect_trustmark(image_path: Path) -> str | None:
|
||||
with Image.open(image_path) as img:
|
||||
cover = img.convert("RGB")
|
||||
decoder = _decoder()
|
||||
_wm_secret, wm_present, wm_schema = decoder.decode(cover)
|
||||
wm_secret, wm_present, wm_schema = decoder.decode(cover, "binary")
|
||||
if not wm_present:
|
||||
return None
|
||||
if not _survives_reencode(decoder, cover, wm_schema):
|
||||
if wm_schema not in _SUPPORTED_SCHEMAS:
|
||||
logger.debug(
|
||||
"TrustMark decode for %s used weak schema %s; treating as false positive",
|
||||
image_path,
|
||||
wm_schema,
|
||||
)
|
||||
return None
|
||||
if not _survives_reencode(decoder, cover, wm_secret, wm_schema):
|
||||
logger.debug("TrustMark decode for %s did not survive re-encode; treating as false positive", image_path)
|
||||
return None
|
||||
except Exception as exc: # model download / decode failure / unreadable image
|
||||
@@ -102,10 +113,8 @@ def detect_trustmark(image_path: Path) -> str | None:
|
||||
return f"Adobe TrustMark (variant {_MODEL_TYPE}, schema {wm_schema})"
|
||||
|
||||
|
||||
def _survives_reencode(decoder: Any, cover: Any, schema: int) -> bool:
|
||||
"""True if the watermark re-decodes with the same schema after a mild JPEG
|
||||
round-trip -- the durability a genuine TrustMark guarantees, which a BCH
|
||||
false positive (content noise) does not."""
|
||||
def _survives_reencode(decoder: Any, cover: Any, payload: str, schema: int) -> bool:
|
||||
"""True if the same watermark re-decodes after a mild JPEG round-trip."""
|
||||
import io
|
||||
|
||||
from PIL import Image
|
||||
@@ -114,5 +123,5 @@ def _survives_reencode(decoder: Any, cover: Any, schema: int) -> bool:
|
||||
cover.save(buffer, "JPEG", quality=_REENCODE_QUALITY)
|
||||
buffer.seek(0)
|
||||
with Image.open(buffer) as reencoded:
|
||||
_secret, present, reencoded_schema = decoder.decode(reencoded.convert("RGB"))
|
||||
return bool(present) and reencoded_schema == schema
|
||||
reencoded_payload, present, reencoded_schema = decoder.decode(reencoded.convert("RGB"), "binary")
|
||||
return bool(present) and reencoded_schema == schema and reencoded_payload == payload
|
||||
@@ -1,126 +0,0 @@
|
||||
"""Optional pre-diffusion super-resolution for small inputs (Real-ESRGAN via spandrel).
|
||||
|
||||
Mirrors ``region_eraser``'s optional-backend pattern: ``is_available()`` guards the
|
||||
``spandrel`` import, a lazy singleton (double-checked lock) holds the loaded model, and
|
||||
the weights download on first use (cached by ``torch.hub``) -- they are never bundled.
|
||||
|
||||
The DEFAULT upscaler stays Lanczos (cv2, no model download); this is opt-in via the ``esrgan``
|
||||
extra and feeds the ``--upscaler esrgan`` path. ``spandrel`` is a pure model-loader
|
||||
(MIT) with NO basicsr dependency -- it pulls only torch/torchvision/safetensors/numpy/
|
||||
einops -- so it sidesteps the basicsr / ``torchvision.transforms.functional_tensor``
|
||||
breakage that the retired ``restore`` (GFPGAN) extra had to shim. Real-ESRGAN weights
|
||||
are BSD-3-Clause.
|
||||
|
||||
CPU works but is slow on large inputs, so this is meant for the pre-diffusion upscale of
|
||||
SMALL inputs (and the GPU worker). On a memory-constrained host it is a no-op (the extra
|
||||
is absent), and the caller falls back to Lanczos.
|
||||
"""
|
||||
|
||||
# torch/spandrel boundary: these libs ship no usable element types; relax the
|
||||
# unknown-type rules for this file only.
|
||||
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false, reportAttributeAccessIssue=false, reportPrivateImportUsage=false
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Real-ESRGAN x2plus (BSD-3-Clause), official release. x2 is the right native factor for
|
||||
# the pre-diffusion floor upscale (small inputs ~512 -> ~1024); spandrel infers the
|
||||
# architecture and scale from the checkpoint, so swapping the URL is enough to change it.
|
||||
_MODEL_URL = "https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth"
|
||||
_MODEL_FILENAME = "RealESRGAN_x2plus.pth"
|
||||
|
||||
_model: Any = None # lazy singleton (spandrel ImageModelDescriptor)
|
||||
_model_device: str = "cpu"
|
||||
_lock = threading.Lock()
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
"""True if the ``esrgan`` extra (spandrel + torch) is importable."""
|
||||
from .optional_deps import module_available
|
||||
|
||||
return module_available("spandrel", "torch")
|
||||
|
||||
|
||||
def _model_cache_path() -> Path:
|
||||
"""Path the weights are cached at (the torch.hub checkpoints dir)."""
|
||||
import torch
|
||||
|
||||
cache_dir = Path(torch.hub.get_dir()) / "checkpoints"
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
return cache_dir / _MODEL_FILENAME
|
||||
|
||||
|
||||
def _get_model(device: str) -> Any:
|
||||
"""Load the Real-ESRGAN model once (downloading the weights on first use)."""
|
||||
global _model, _model_device
|
||||
if _model is not None and _model_device == device:
|
||||
return _model
|
||||
with _lock:
|
||||
if _model is None:
|
||||
import torch
|
||||
from spandrel import ImageModelDescriptor, ModelLoader
|
||||
|
||||
dst = _model_cache_path()
|
||||
if not dst.exists():
|
||||
logger.info("Downloading Real-ESRGAN weights to %s", dst)
|
||||
torch.hub.download_url_to_file(_MODEL_URL, str(dst), progress=False)
|
||||
model = ModelLoader().load_from_file(str(dst))
|
||||
if not isinstance(model, ImageModelDescriptor):
|
||||
raise RuntimeError(f"Unexpected spandrel model type: {type(model).__name__}")
|
||||
_model = model.eval()
|
||||
if _model_device != device:
|
||||
_model.to(device)
|
||||
_model_device = device
|
||||
return _model
|
||||
|
||||
|
||||
def scale() -> int:
|
||||
"""The model's native upscale factor (e.g. 2 for x2plus). Loads the model if needed."""
|
||||
return int(_get_model("cpu").scale)
|
||||
|
||||
|
||||
def upscale(image: NDArray[Any], device: str | None = None) -> NDArray[Any]:
|
||||
"""Upscale a BGR uint8 image by the model's native factor with Real-ESRGAN.
|
||||
|
||||
Returns a BGR uint8 array. Falls back to CPU if the requested device errors (an
|
||||
MPS/CUDA OOM or unsupported-op on the small pre-diffusion input), mirroring the
|
||||
diffusion engine's MPS->CPU fallback.
|
||||
|
||||
Raises:
|
||||
RuntimeError: if the ``esrgan`` extra is not installed (guard with
|
||||
``is_available()`` first).
|
||||
"""
|
||||
if not is_available():
|
||||
raise RuntimeError("Real-ESRGAN upscaler needs the 'esrgan' extra (spandrel). Install it or use Lanczos.")
|
||||
import cv2
|
||||
import numpy as np
|
||||
import torch
|
||||
|
||||
target_device = (device or "cpu").lower()
|
||||
if target_device not in {"cpu", "mps", "cuda", "xpu"}:
|
||||
target_device = "cpu"
|
||||
rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
|
||||
tensor = torch.from_numpy(rgb).permute(2, 0, 1).float().div(255.0).unsqueeze(0)
|
||||
|
||||
def _run(dev: str) -> NDArray[Any]:
|
||||
model = _get_model(dev)
|
||||
with torch.no_grad():
|
||||
out = model(tensor.to(dev))
|
||||
arr = out.clamp(0.0, 1.0).squeeze(0).permute(1, 2, 0).cpu().numpy() * 255.0
|
||||
return cv2.cvtColor(arr.round().astype(np.uint8), cv2.COLOR_RGB2BGR)
|
||||
|
||||
try:
|
||||
return _run(target_device)
|
||||
except Exception as e: # GPU OOM / unsupported op: fall back to CPU
|
||||
if target_device == "cpu":
|
||||
raise
|
||||
logger.warning("Real-ESRGAN on %s failed (%s); retrying on CPU", target_device, e)
|
||||
return _run("cpu")
|
||||
Loaded 100 of 146 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user