Compare commits

...
18 Commits
Author SHA1 Message Date
Victor Kuznetsov 8871eb3f67 Release 0.29.0
pre-commit: 1) maintain.sh - exit 1 on uv-secure, lightning PYSEC-2026-3624 unchanged from 0.28.1, no fixed release exists, vulnerable API unreachable (no load_from_checkpoint in project or trustmark); ruff, pyright src/, and 1386 tests passed separately; 2) /simplify - version bump only; 3) docs sync - no version refs outside pyproject, __init__.py, uv.lock; 4) CLAUDE.md - no change
2026-08-25 13:25:33 -07:00
Victor Kuznetsov 44ec203c22 Drop the stale untiled restriction from text restoration docs 2026-08-25 13:22:49 -07:00
Victor Kuznetsov 4beb61463a Say metadata check and strip results are not a clean verdict 2026-08-25 13:22:38 -07:00
Victor Kuznetsov 0d11e4099c Fix adaptive text restoration crop bounds 2026-08-25 12:44:05 -07:00
Victor Kuznetsov fa83c1444f Add geometry-only text manifest schema 2026-08-25 11:33:21 -07:00
Victor Kuznetsov f52e40ae24 Name Copilot, Dreamina, and Qwen in C2PA source labels 2026-08-21 14:43:39 -07:00
Victor Kuznetsov 0aa03c4e89 Release 0.28.1
pre-commit: 1 maintain.sh - uv-secure lightning PYSEC-2026-3624 known, no fix, same as 0.28.0; ruff/pyright/1378 tests passed; 2) /simplify - version bump only; 3) docs sync - no refs; 4) CLAUDE.md - no change
EOF
)
2026-08-19 21:46:14 -07:00
Victor Kuznetsov 91aa49b03a Allow verified-text restoration on tiled images
Tile the Qwen VAE donor the same way as the global pass. Glyph restore already runs on the blended full frame, so the old tile ban was an artificial gate.
2026-08-19 21:33:22 -07:00
Victor Kuznetsov ee865e95e4 Add geometry draft mode and pad silhouette descenders
pre-commit: 1 maintain.sh - not rerun full, ruff + 34 targeted tests passed; 2) /simplify - pad lives in silhouette crop, stable=False reuses probes; 3) docs sync - no README/cli refs to update; 4) CLAUDE.md - no change
EOF
)
2026-08-19 19:53:38 -07:00
Victor Kuznetsov d84fd1a125 Merge pull request #86 from wiltodelta/dependabot/uv/minor-and-patch-267694dd16
Bump the minor-and-patch group across 1 directory with 7 updates
2026-08-19 12:22:54 -07:00
Victor Kuznetsov e315eacb14 Document automated-operator verified manifests
verified:true may attest machine-verified geometry (stability-gated
boxes inside caps) set by a service; the pipeline consumes box/script
only, text is advisory, so the verification burden is on the geometry.
2026-08-19 12:21:32 -07:00
Victor Kuznetsov 3ba4ace7e0 Disable mkldnn in the draft engines (linux PIR crash) 2026-08-19 10:22:58 -07:00
dependabot[bot] 13888704f8 Bump the minor-and-patch group across 1 directory with 7 updates
Bumps the minor-and-patch group with 7 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [python-dotenv](https://github.com/theskumar/python-dotenv) | `1.2.2` | `1.2.3` |
| [c2pa-python](https://github.com/contentauth/c2pa-python) | `0.37.5` | `0.37.7` |
| [av](https://github.com/PyAV-Org/PyAV) | `18.0.0` | `18.1.0` |
| [transformers](https://github.com/huggingface/transformers) | `5.14.1` | `5.15.0` |
| diffsynth | `2.1.0` | `2.1.2` |
| [paddleocr](https://github.com/PaddlePaddle/PaddleOCR) | `3.4.1` | `3.7.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.2` | `0.16.3` |



Updates `python-dotenv` from 1.2.2 to 1.2.3
- [Release notes](https://github.com/theskumar/python-dotenv/releases)
- [Changelog](https://github.com/theskumar/python-dotenv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/theskumar/python-dotenv/compare/v1.2.2...v1.2.3)

Updates `c2pa-python` from 0.37.5 to 0.37.7
- [Release notes](https://github.com/contentauth/c2pa-python/releases)
- [Changelog](https://github.com/contentauth/c2pa-python/blob/main/docs/release-notes.md)
- [Commits](https://github.com/contentauth/c2pa-python/compare/v0.37.5...v0.37.7)

Updates `av` from 18.0.0 to 18.1.0
- [Release notes](https://github.com/PyAV-Org/PyAV/releases)
- [Changelog](https://github.com/PyAV-Org/PyAV/blob/v18.1.0/CHANGELOG.rst)
- [Commits](https://github.com/PyAV-Org/PyAV/compare/v18.0.0...v18.1.0)

Updates `transformers` from 5.14.1 to 5.15.0
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.14.1...v5.15.0)

Updates `diffsynth` from 2.1.0 to 2.1.2

Updates `paddleocr` from 3.4.1 to 3.7.0
- [Release notes](https://github.com/PaddlePaddle/PaddleOCR/releases)
- [Commits](https://github.com/PaddlePaddle/PaddleOCR/compare/v3.4.1...v3.7.0)

Updates `ruff` from 0.16.2 to 0.16.3
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.2...0.16.3)

---
updated-dependencies:
- dependency-name: python-dotenv
  dependency-version: 1.2.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: c2pa-python
  dependency-version: 0.37.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: av
  dependency-version: 18.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: transformers
  dependency-version: 5.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: diffsynth
  dependency-version: 2.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: paddleocr
  dependency-version: 3.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-19 17:19:11 +00:00
Victor Kuznetsov e938b57f8c Package the verified-text draft annotator as text_draft
Move the draft-annotation logic (PaddleOCR line detection, word-box
grouping, three script-chosen recognition engines, crop-jitter
stability gate) from the evaluation-only scripts into the installable
package, with lazy paddle imports and a text-draft extra (CPU, no
torch). draft_text_lines() returns accepted (crop-stable, NEVER
ground-truth-correct - precision on the reference posters was 90.0% /
94.4%) and rejected proposals; source_pixel_sha256 is re-exported for
manifest building. scripts/infer_text_lines.py now dogfoods the
package module instead of loading the eval script by path.
2026-08-19 10:13:22 -07:00
Victor Kuznetsov 3cd1e47935 Add the oracle-measured Google strength floor to qwen-zimage
The resolution curve's 0.154 top left the 4.33 MP CJK-sign fixture
SynthID-detected x3 in Gemini on the full production path (visible
stage -> qwen-zimage seed 0 -> resize-back -> metadata strip), with a
valid pixel-identical stripped control in the same session
(2026-08-18). Google-provenance content now resolves to the flat
QWEN_ZIMAGE_GOOGLE_STRENGTH 0.30 floor instead of the area curve -
anchors at 0.30 measured clean in Gemini on two fixtures (CJK sign +
18-face) at 3/3 checks across two work accounts, and stayed clean
under the vae-glyphs donor layer. openai/unknown content keeps the
curve; an explicit strength still wins.
2026-08-18 22:34:41 -07:00
Victor Kuznetsov 2298282b8b Thread fidelity_anchor through WatermarkRemover
The engine forwards the new kwarg into WatermarkRemover.remove_watermark,
which rejected it with TypeError on the real (non-fake) path - caught by
the deployed Modal smoke test, not by the unit fakes.
2026-08-18 21:32:57 -07:00
Victor Kuznetsov 42d427731f Turn the verified-text fidelity anchor off by default
The whole-frame 15% Qwen-VAE blend returned detector-visible OpenAI
SynthID on poster-scale manifests through the engine text-manifest
path (official Content Provenance API, 2026-08-19: restored detected
x6 with the anchor, clean x6 without it; base outputs clean x6;
pixel-identical stripped controls detected, proving the pixel channel).
Add fidelity_anchor=False to remove_watermark and InvisibleOptions and
--fidelity-anchor on the CLI to reproduce the 0.27.0 research
behavior. Text-box MAE cost of the new default is under one point on
all three fixtures (11.60->11.72, 7.79->7.86, 7.57->8.13).
2026-08-18 21:23:25 -07:00
Victor Kuznetsov 50d9f44829 Guide text-heavy content away from forced diffusion scrubs
Issue #84: text, tables, and UI screenshots are the worst case for
--force regeneration. Document that the default pipeline skips the
invisible stage without a detectable signal, that visible/metadata
stages never redraw glyphs, and why paste-back is not offered.
2026-08-18 11:03:53 -07:00
32 changed files with 2006 additions and 291 deletions

No files matched your search

+3 -2
View File
@@ -202,8 +202,9 @@ remove-ai-watermarks invisible image.png -o clean.png --force
```
Typography-heavy images can opt into the experimental verified-text post-pass.
It requires manually reviewed strings and line boxes; it never trusts OCR as ground
truth or runs automatically:
It accepts manually reviewed strings and line boxes or an operator-verified
geometry-only manifest; it never treats raw OCR output as ground truth or runs
automatically:
```bash
uv tool install --force "remove-ai-watermarks[text-restoration]"
+28 -6
View File
@@ -157,6 +157,11 @@ remove-ai-watermarks metadata image.png --remove -o clean.png
When `-o` is omitted, removal overwrites the source. Standard metadata is kept
unless you pass `--remove-all`.
A quiet `--check` or a successful `--remove` is not a clean verdict. The command
only inspects and strips embedded AI metadata; a pixel watermark such as SynthID
has no local decoder once that metadata proxy is gone. `identify` reports the
same limit.
The command also supports the audio and video containers listed in
[supported signals](supported-signals.md). ffmpeg must be available for the
non-ISOBMFF audio and video path.
@@ -225,6 +230,10 @@ remove-ai-watermarks video metadata input.mp4 --check
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
```
As with the generic `metadata` command, a quiet check or a successful strip is
not a clean verdict: video SynthID is not decoded locally after the metadata
proxy is gone.
Supported containers are MP4, MOV, M4V, WebM, MKV, AVI, and FLV. The operation
delegates to the same verified metadata scanner and stripper as the generic
`metadata` command, so detection and removal stay in parity. Video and audio
@@ -408,7 +417,7 @@ the source with the Qwen VAE, blends 15% of that reconstruction into the normal
only the reconstructed glyph cores through source-derived silhouettes. It does not
run OCR or choose which strings are correct.
Install the combined extra and run only with a manually reviewed manifest:
Install the combined extra and run only with an operator-verified manifest:
```bash
uv tool install --force "remove-ai-watermarks[text-restoration]"
@@ -416,16 +425,29 @@ remove-ai-watermarks invisible image.png -o clean.png \
--pipeline qwen-zimage --text-manifest verified-lines.json --force
```
The manifest is a JSON object with `schema_version: 1`, `verified: true`, decoded
RGB dimensions, `source_pixel_sha256`, and a non-empty `lines` array. Each line has
an integer `[x1, y1, x2, y2]` box, exact `text`, a non-empty `script`, and an optional
angle from -30 to 30 degrees. Lines must be in top-to-bottom, left-to-right order.
``verified: true`` may also be set by an automated operator that attests
machine-verified geometry: stability-gated detector boxes inside sane caps. Such
operators should use the geometry-only schema 2, which carries no transcription
or script metadata.
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default: it
was measured to return detector-visible OpenAI SynthID on poster-scale manifests
(official Content Provenance API, 2026-08-19). `--fidelity-anchor` restores the
0.27.0 research behavior; text-box fidelity lost by the default is well under one
MAE point on the measured fixtures.
The manifest is a JSON object with `verified: true`, decoded RGB dimensions,
`source_pixel_sha256`, and a non-empty `lines` array. Schema 1 is retained for
manually reviewed annotations: each line has an integer `[x1, y1, x2, y2]` box,
exact `text`, a non-empty `script`, and an optional angle from -30 to 30 degrees.
Schema 2 is geometry-only: each line has the box and optional angle, with no
required `text` or `script`. Lines must be in top-to-bottom, left-to-right order.
The hash binds the annotations to decoded RGB geometry and pixels, so metadata-only
container changes remain valid while a resized or edited source fails closed. The
experimental helper
`remove_ai_watermarks._internal.text_restoration.source_pixel_sha256` computes it.
This mode is supported only by `qwen-zimage` at native untiled geometry with
This mode is supported only by `qwen-zimage` at native geometry with
`humanize=0`, `unsharp=0`, and adaptive polish disabled. `all` also accepts the flag,
but its manifest must match the pixels entering the invisible stage; if visible-mark
removal changes those pixels, the hash check rejects the run. One oracle verdict does
+19 -2
View File
@@ -57,6 +57,19 @@ Invisible removal does not decode and delete a payload. It regenerates the
image through a diffusion pipeline. Faces, text, colors, and fine detail can
change even when the watermark is successfully disrupted.
Text, tables, and UI screenshots are the worst case for a forced scrub: the
model redraws glyphs as plausible-but-wrong shapes. Do not run the invisible
stage on such content unless a locally-detectable invisible signal exists.
The default `all`, `invisible`, and `batch` commands already skip it then, and
`visible` (which strips AI metadata by default) or `metadata` alone never
redraws a glyph outside the filled watermark box. `--force` is for content you
know carries a pixel watermark despite no local signal; on a clean screenshot
it only buys distortion. Preserving or pasting back the original text pixels
during a real scrub is deliberately not offered: an invisible watermark such
as SynthID lives in the pixels everywhere, including inside the glyphs, so
frozen text regions would keep the watermark
([text protection research](text-protection-research.md)).
`qwen-zimage` is the default profile and `sdxl-zimage` the only alternative.
Both are CUDA only and differ only in the global regeneration model: each
conditions that stage on a canny edge map, which preserves structure but not
@@ -165,8 +178,12 @@ The two profiles resolve an unset strength differently, because different things
were measured for each.
`qwen-zimage` reads it from image area, through the resolution-adaptive denoise
curve. The vendor is deliberately ignored: the curve, not the issuer, is what was
calibrated.
curve. The vendor is deliberately ignored for openai/unknown content: the curve,
not the issuer, is what was calibrated. One measured exception (0.27.2):
Google-provenance content takes the flat `QWEN_ZIMAGE_GOOGLE_STRENGTH` 0.30 floor
- the curve's 0.154 top left a 4.33 MP fixture oracle-detected on the full
production path (2026-08-18), while 0.30 anchors measured clean on two fixtures
in two accounts.
`sdxl-zimage` reads it from the C2PA issuer, on a flat ladder:
+14 -4
View File
@@ -42,6 +42,10 @@ Important contracts:
- `invisible` writes no output when no supported local signal is found, unless
`--force` is supplied.
- The two no-signal conditions currently share exit code `2`.
- A quiet `metadata --check` and a successful `metadata --remove` (same for
`video metadata`) end by repeating the `identify` limit: the pixel channel is
untouched and a watermark such as SynthID has no local decoder once its
metadata proxy is gone, so neither outcome is a clean verdict.
- Hard processing and write failures exit with code `1`.
- `all` can still write the completed visible and metadata stages when the
diffusion dependencies are unavailable, but exits with code `1` so the
@@ -960,9 +964,10 @@ requires the same provider-oracle and identity evaluation as a model change.
[`_internal/text_restoration.py`](../src/remove_ai_watermarks/_internal/text_restoration.py)
implements the opt-in `vae-glyphs` stage. A versioned manifest carries manually
reviewed strings and source-space line boxes, plus a SHA-256 over decoded RGB width,
height, and pixels. Validation happens before model loading. The product never treats
OCR confidence as verification.
reviewed strings and source-space line boxes in schema 1, or verified source-space
geometry alone in schema 2, plus a SHA-256 over decoded RGB width, height, and pixels.
Validation happens before model loading. The library never treats OCR confidence as
verification, and geometry-only operators do not need to invent text or script fields.
When enabled, `QwenZImagePipeline` reconstructs the source once through its already
loaded Qwen VAE, runs the ordinary global and face stages, blends 15% of the VAE
@@ -971,7 +976,12 @@ The compositor derives binary source and candidate silhouettes, groups nearby li
uses LaMa for the initial and residual-glyph erase passes, paints fresh silhouette
edges, then copies the Qwen-VAE core with a 0.5-pixel feather. The evaluation script
imports these same mask and compositing helpers so the two implementations cannot
silently drift.
silently drift. Silhouette crops start 12% of line height beyond each horizontal
side, then expand each side independently while a foreground component anchored
inside the detector box still reaches that boundary, up to one line height. They
extend 8% above and 25% below. The anchored-component gate covers clipped leading
flourishes, trailing punctuation, icons, and descenders without walking into
disconnected decoration or background texture.
The stage is deliberately narrower than the engine: it rejects `sdxl-zimage`, tiles,
resolution caps, humanize, unsharp, and adaptive polish. Those combinations change
+37 -2
View File
@@ -609,11 +609,46 @@ engine.remove_watermark(
Install `remove-ai-watermarks[text-restoration]`. The manifest schema and safety
constraints are documented in the CLI guide. The engine verifies its decoded RGB
hash before loading the diffusion models and rejects SDXL, tiling, downscaling, and
postprocessing combinations that were not evaluated. `InvisibleOptions` exposes the
hash before loading the diffusion models and rejects SDXL, downscaling, and
postprocessing combinations that were not evaluated. Tiling is allowed: the VAE
donor uses the same overlapping tiles as the global pass. `InvisibleOptions` exposes the
same field for `remove_all`; after a visible-stage edit, the manifest must be built
against the staged pixels rather than the pristine source.
Use manifest schema 1 for manually reviewed text plus script metadata. Automated
operators that verify only text-region geometry should emit schema 2 lines with a
`box` and optional `angle`; no placeholder transcription or script is required.
Since 0.27.1 the mode's global 15% Qwen-VAE fidelity-anchor blend is **off by
default** (`fidelity_anchor=False`): that whole-frame blend was measured to
return detector-visible OpenAI SynthID on poster-scale manifests (official
Content Provenance API, 2026-08-19 - detected x6 with the anchor, clean x6
without it, controls and base outputs validated in the same sessions). Pass
`fidelity_anchor=True` to reproduce the 0.27.0 research behavior.
### Drafting manifest lines
`remove_ai_watermarks.text_draft` proposes lines for a manifest; it never
produces verified ones:
```python
from remove_ai_watermarks.text_draft import draft_text_lines
draft = draft_text_lines(Path("watermarked.png"))
for line in draft.accepted:
print(line.box, line.script, line.min_score, line.text)
```
Install `remove-ai-watermarks[text-draft]` (CPU, no torch: PaddleOCR detection
plus three script-chosen recognition engines). A line lands in `accepted` only
when three crop paddings normalize identically and every confidence clears
`min_score` (default 0.85); `accepted` means crop-stable, NOT ground-truth-
correct - on the reference posters the draft's exact-text precision was 90.0%
and 94.4% because high-confidence OCR still lost punctuation. Every accepted
line needs a human yes/no before a manifest may claim `verified: true`.
`source_pixel_sha256` is re-exported here for building the manifest's
pixel-binding hash against the exact source the engine will decode.
`remove_watermark` takes strength, seed, tiling, resolution, and postprocessing
controls. It takes no model id, step count or guidance scale, and neither does the
constructor: each profile pins its model stack, its per-stage schedule and CFG
+12
View File
@@ -769,3 +769,15 @@ reproducible verification requires a fixed seed.
8. Jiang et al. (2025). **VideoMarkBench: Benchmarking Robustness of Video
Watermarking.** arXiv:2505.21620.
https://arxiv.org/abs/2505.21620
**Google floor on qwen-zimage (0.27.2, 2026-08-19).** The bracket above did not
survive the full production path: on the 4.33 MP CJK-sign fixture (visible-stage
sparkle removal -> qwen-zimage seed 0 at the curve's 0.154 top -> resize-back ->
metadata strip), the Gemini verifier detected SynthID x3 with a valid
pixel-identical stripped control in the same session. Google-provenance content
now takes a flat `QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.30` floor instead of the area
curve - 0.30 anchors measured clean in Gemini on two fixtures (CJK sign +
18-face) at 3/3 checks across two work accounts, and stayed clean with the
vae-glyphs donor layer on top; openai/unknown content keeps the resolution
curve. An explicit strength still wins. Certification artifacts: raiw-app
`data/certification/text-restoration-2026-08-18/` (the failing re-baseline).
+26
View File
@@ -343,3 +343,29 @@ The stable errors were punctuation: one lost English comma and an ideographic
comma consistently replaced by ASCII. Thus crop agreement and confidence do
not verify literal text. The helper is retained only to seed annotations; every
proposal needs manual review before it can enter the restoration compositor.
## Production-path anchor leak and fix (2026-08-19)
The engine `text_manifest=` path (unlike the research compositor above) blends a
15% Qwen-VAE fidelity anchor across the WHOLE frame before glyph restoration.
Certified through the official OpenAI Content Provenance API on the exact
production path (qwen-zimage, seed 0, no strength override, untiled, LANCZOS
resize-back, metadata strip; library 0.27.0 @ 50d9f448):
- controls (pixel-identical metadata-stripped sources): `synthid: detected` -
the pixel channel was proven live before every session;
- base outputs: `clean` x6 (two 20-line posters, 3 attempts each);
- text-restored WITH the anchor: `synthid: detected` x6, deterministic;
- text-restored with `FIDELITY_BLEND_ALPHA = 0` (runtime ablation, base bytes
identical between runs): `clean` x6.
The anchor was therefore the leak channel: a 15% global donor admixture over a
scrubbed frame returns enough OpenAI SynthID for the provider detector. Fidelity
cost of dropping it (source-referenced text-box MAE): 11.60->11.72, 7.79->7.86,
and 7.57->8.13 across the two posters and the CJK sign - well under one point
against base MAEs of 24.7/16.8/10.2. Since 0.27.1 the anchor is off by default
(`fidelity_anchor` parameter on `remove_watermark` / `InvisibleOptions` /
`--fidelity-anchor`); Google rows were not re-queried because the default
strength leaves that fixture's base detected (the vendor-strength lane tracks
that separately). Certification artifacts: raiw-app
`data/certification/text-restoration-2026-08-19-A1/` (sha256-stamped rows).
+2 -2
View File
@@ -10,7 +10,7 @@ Who embeds what, and whether it is locally detectable (so we know which gaps are
**Root cause and external confirmation (deep-research 2026-06-19, adversarially verified).** This is the SCHEME's ceiling, not our usage — there is no better decoder to adopt. The imwatermark maintainers state verbatim (both the ShieldMnt and Stability-AI READMEs) that the algorithm "cannot guarantee to decode the original watermarks 100% accurately even though we don't apply any attack." Independent measurement (WMAdapter, arXiv:2406.08337 Table 2) puts dwtDct at only **~0.79 bit accuracy on CLEAN images (~38/48 bits — already below our 44 gate)**, collapsing to ~0.50 (chance) under crop/JPEG. Two code-verified + locally-reproduced mechanisms drive the content-dependent failures: (1) the decoder reads each bit as the **highest-magnitude DCT coefficient per block**, so any content coefficient exceeding the encoded target flips the bit; (2) the default embed is in the **YUV chroma channel, which 8-bit-clamps on white/bright pixels** (a +36 chroma delta survives a white-fill round-trip as only +4, ~89% loss) — this is the mechanism behind the bright-flat / minimalist failures and the all-ones degenerate decode. No maintained fork or detector decodes this scheme reliably: the WAVES benchmark (arXiv:2401.08573) relegates DWT-DCT to supplementary appendix G.5 and targets Stable Signature / Tree-Ring / StegaStamp instead; learned encoder/decoder schemes reach ~0.98-0.99 clean but are a DIFFERENT watermark class (not what SDXL/FLUX stamp). `dwtDctSvd` does not help (SDXL embeds `dwtDct`; dwtDctSvd cannot decode it, and its clean accuracy ~0.72 is lower). **Authoritative conclusion: the open DWT-DCT mark cannot be turned from positive-only into a reliable real-world detector; keep it positive-only and rely on C2PA.** (Refuted along the way: that the library is unmaintained, and that it is robust to JPEG but only fails on geometric attacks — both did not survive verification.)
Consequence for the FLUX hosted-output question (BFL Playground, FLUX.2 [pro] + FLUX.1 [dev], 2026-06-19): all samples carry the signed C2PA manifest (issuer "Black Forest Labs"); the open DWT-DCT decode returned `None`, but every available FLUX carrier (textured fox AND a minimalist-flat generation) failed the positive control (28/48), so the detector is blind on them and **whether BFL hosted output embeds the open pixel watermark is UNRESOLVED** (an earlier note here wrongly asserted it absent — overstated; a later note blamed "high texture" — also wrong, flat carriers fail too). What IS established: C2PA is the reliable FLUX identifier; the `_BITS_48` pattern is correct (round-trips on chatgpt/firefly/random). Resolving the hosted question needs a hosted FLUX carrier that first passes a >=44/48 positive control, which neither a textured nor a flat prompt produced — low priority (the open mark is only a stripped-metadata fallback).
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Copilot + Designer; Bing Image Creator collected 2026-05-24 still signs as "Microsoft" and now runs **MAI-Image**, NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
**Samsung Galaxy AI** signs supported edits with C2PA and may carry the
proprietary `genAIType` marker. The registered visible detector covers the
@@ -28,7 +28,7 @@ take precedence over issuer attribution. Supported mappings include Higgsfield
AI, Topaz Labs Image API, and TikTok Ad Creative Toolbox; an unknown claim
generator still falls back to the certificate issuer.
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
- **EXIF/XMP/PNG-text generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal). Additional verified generator stamps include **NovelAI** (`Software`, `Source`, and `Title` PNG text chunks), **Reve** (`Software` or XMP `CreatorTool` = `reve.com`), and **Aphrodite AI** (`Make` or `Software` = `Aphrodite AI`).
- **App-export provenance and AIGC JSON:** supported ByteDance-family exports can place a JSON object in EXIF `ImageDescription` or `UserComment`, independently of C2PA or TC260. Exact `product` values for Doubao, Xinghui, and Dreamina are removable product provenance, but do not alone prove that the pixels were generated. Dreamina additionally requires `exportType=generation` for that verdict. A nested Aweme `aigc_type=1` or private ByteDance `aigc_label_type=1` / `2` is an AIGC disclosure; `0` is inconclusive and can occur on a Dreamina generation export. Plain Aweme, retouch, and `lv` exports are preserved. The lower-case private field is deliberately not interpreted as the normative TC260 `Label`, whose values `1` / `2` / `3` mean generated / possibly generated / suspected generated under [GB 45438-2025](https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf).
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
+9 -1
View File
@@ -4,7 +4,7 @@
# on it, including the ComfyUI node package. The console script below carries
# the same weight, since users have it on PATH.
name = "remove-ai-watermarks"
version = "0.27.0"
version = "0.29.0"
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
readme = "README.md"
requires-python = ">=3.11,<3.15"
@@ -125,6 +125,14 @@ qwen-zimage = [
text-restoration = [
"remove-ai-watermarks[qwen-zimage,lama]",
]
# Draft-only OCR proposals for verified-text manifests. CPU, no torch:
# PaddleOCR detection plus three script-chosen recognition engines. The models
# download on first use and are cached by huggingface_hub; never bundled.
text-draft = [
"paddleocr>=3.3.3",
"paddlepaddle>=3.0.0",
"huggingface-hub>=0.20.0",
]
# Adobe TrustMark decoder -- the open, keyless watermark behind Adobe Durable
# Content Credentials (soft-binding alg ``com.adobe.trustmark.P``). Optional
# because it pulls torch and downloads model weights on first use. identify()
+11 -40
View File
@@ -21,14 +21,11 @@ changes under crop jitter or whose minimum confidence is below the threshold.
from __future__ import annotations
import importlib.util
import json
import logging
import os
import sys
import unicodedata
from pathlib import Path
from typing import Any
import click
import numpy as np
@@ -37,38 +34,15 @@ from PIL import Image
log = logging.getLogger(__name__)
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
RESTORATION_SCRIPT = ROOT / "scripts/selective_text_restoration.py"
from scripts._text_eval import normalize_text # noqa: E402
def _load_restoration_module() -> Any:
spec = importlib.util.spec_from_file_location("selective_text_restoration_for_inference", RESTORATION_SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"unable to load {RESTORATION_SCRIPT}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
def _has_script(text: str, script: str) -> bool:
return any(script in unicodedata.name(character, "") for character in text)
def choose_language(probes: dict[str, tuple[str, float]]) -> str:
if _has_script(probes["ch"][0], "CJK"):
return "ch"
if _has_script(probes["ru"][0], "CYRILLIC"):
return "ru"
return "en"
def stable_recognition(reads: list[tuple[str, float]], min_score: float = 0.85) -> str | None:
normalized = {normalize_text(text) for text, _score in reads}
if len(normalized) != 1 or min(score for _text, score in reads) < min_score:
return None
return reads[0][0]
# Dogfoods the packaged draft API (remove_ai_watermarks.text_draft); this script
# keeps only the CLI wrapper so the package stays the one home for the logic.
from remove_ai_watermarks.text_draft import ( # noqa: E402
_detect_line_boxes,
_recognize,
choose_language,
stable_recognition,
)
@click.command()
@@ -81,7 +55,6 @@ def main(source: Path, out: Path, min_score: float) -> None:
os.environ["PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK"] = "True"
from paddleocr import PaddleOCR, TextRecognition
restoration = _load_restoration_module()
source_rgb = np.asarray(Image.open(source).convert("RGB"))
detector = PaddleOCR(
lang="ch",
@@ -94,19 +67,17 @@ def main(source: Path, out: Path, min_score: float) -> None:
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec"),
"ch": TextRecognition(model_name="PP-OCRv5_server_rec"),
}
boxes = restoration.detect_line_boxes(detector, source_rgb)
boxes = _detect_line_boxes(detector, source_rgb)
accepted = []
rejected = []
for box in boxes:
probes = {}
for language, engine in engines.items():
script = "cjk" if language == "ch" else "alphabetic"
line = restoration.TextLine(box, "", script)
probes[language] = restoration._recognize(engine, source_rgb, line, 0.1)
probes[language] = _recognize(engine, source_rgb, box, script, 0.1)
language = choose_language(probes)
script = "cjk" if language == "ch" else "alphabetic"
line = restoration.TextLine(box, "", script)
reads = [restoration._recognize(engines[language], source_rgb, line, ratio) for ratio in (0.08, 0.12, 0.2)]
reads = [_recognize(engines[language], source_rgb, box, script, ratio) for ratio in (0.08, 0.12, 0.2)]
text = stable_recognition(reads, min_score)
result = {
"box": box,
+1 -1
View File
@@ -32,7 +32,7 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
__version__ = "0.27.0"
__version__ = "0.29.0"
__all__ = [
"BatchSummary",
@@ -65,7 +65,7 @@ def _vendor(
# Order is product priority when a manifest mentions more than one organization.
C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
_vendor(b"Microsoft", "Microsoft", "Microsoft (Bing Image Creator / Designer)", "Microsoft"),
_vendor(b"Microsoft", "Microsoft", "Microsoft (Copilot / Designer)", "Microsoft"),
_vendor(b"Adobe", "Adobe", "Adobe Firefly", "Adobe"),
_vendor(
b"OpenAI",
@@ -78,18 +78,25 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
_vendor(b"Google", "Google LLC", "Google (Gemini / Imagen)", "Google", synthid=True),
_vendor(b"Stability AI", "Stability AI", "Stability AI (Stable Image / DreamStudio)", "Stability AI"),
_vendor(b"Black Forest Labs", "Black Forest Labs", "Black Forest Labs (FLUX)", "Black Forest Labs"),
_vendor(b"volcengine", "ByteDance (Volcano Engine)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
_vendor(
b"volcengine",
"ByteDance (Volcano Engine)",
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
"ByteDance",
),
_vendor(
"北京火山引擎科技有限公司",
"ByteDance (Volcano Engine)",
"ByteDance (Doubao / Jimeng / Volcano Engine)",
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
"ByteDance",
),
_vendor(b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
_vendor(
b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)", "ByteDance"
),
_vendor(
b"Dreamina",
"ByteDance (Dreamina)",
"ByteDance (Doubao / Jimeng / Volcano Engine)",
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
"ByteDance",
asserts_ai=True,
),
@@ -109,7 +116,7 @@ C2PA_IDENTITY_AI_ORGS = frozenset(vendor.org for vendor in C2PA_AI_VENDORS if ve
C2PA_CLAIM_GENERATOR_PLATFORMS: tuple[tuple[str, str], ...] = (
("adobe_firefly", "Adobe Firefly"),
("firefly", "Adobe Firefly"),
("dreamina", "ByteDance (Doubao / Jimeng / Volcano Engine)"),
("dreamina", "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"),
("higgsfield ai", "Higgsfield AI"),
("topaz labs image api", "Topaz Labs"),
("tiktok ad creative toolbox", "TikTok Ad Creative Toolbox"),
@@ -1071,6 +1071,7 @@ class QwenZImagePipeline:
tile_size: int = 1024,
tile_overlap: int = 128,
text_manifest: VerifiedTextManifest | None = None,
fidelity_anchor: bool = False,
) -> Image.Image:
"""Execute global regeneration and masked face repair."""
self._require_cuda()
@@ -1078,7 +1079,24 @@ class QwenZImagePipeline:
donor = None
if text_manifest is not None:
self._progress("Reconstructing the verified text donor with the Qwen VAE...")
donor = self._qwen_vae_roundtrip(image)
if tile and max(image.size) > tile_size:
from remove_ai_watermarks._internal.tiling import run_tiled
donor = run_tiled(
self._qwen_vae_roundtrip,
image,
tile_size,
tile_overlap,
lambda message: self._progress(
message.replace(
"Tiled diffusion",
"Reconstructing the verified text donor",
1,
)
),
)
else:
donor = self._qwen_vae_roundtrip(image)
global_strength = (
resolution_adaptive_denoise(image.width, image.height) if strength is None else float(strength)
)
@@ -1120,7 +1138,16 @@ class QwenZImagePipeline:
restore_verified_text,
)
self._progress("Blending the Qwen-VAE fidelity anchor...")
anchor = blend_fidelity_anchor(result, donor)
# The anchor is OFF by default since 0.27.1: blending 15% of the Qwen-VAE
# donor ACROSS THE WHOLE FRAME returned detector-visible OpenAI SynthID on
# poster-scale manifests (official Content Provenance API, 2026-08-19:
# detected x6 with the anchor, clean x6 without it, base clean throughout;
# see docs/text-protection-research.md). ``fidelity_anchor=True`` keeps the
# 0.27.0 research behavior for reproduction.
if fidelity_anchor:
self._progress("Blending the Qwen-VAE fidelity anchor...")
anchor = blend_fidelity_anchor(result, donor)
else:
anchor = result
self._progress(f"Restoring {len(text_manifest.lines)} verified text lines...")
return restore_verified_text(image, anchor, donor, text_manifest.lines)
@@ -13,30 +13,35 @@ import cv2
import numpy as np
from PIL import Image
from remove_ai_watermarks._internal.schema import require_schema_version
if TYPE_CHECKING:
from collections.abc import Sequence
from pathlib import Path
from numpy.typing import NDArray
TEXT_MANIFEST_SCHEMA = 1
TEXT_MANIFEST_SCHEMA = 2
_SUPPORTED_TEXT_MANIFEST_SCHEMAS = frozenset({1, TEXT_MANIFEST_SCHEMA})
FIDELITY_BLEND_ALPHA = 0.15
GLYPH_FEATHER = 0.5
GLYPH_SIDE_PAD_RATIO = 0.12
GLYPH_SIDE_PAD_LIMIT = 1.0
@dataclass(frozen=True)
class VerifiedTextLine:
"""One operator-verified source line in source-pixel coordinates."""
"""One operator-verified source region in source-pixel coordinates."""
box: tuple[int, int, int, int]
text: str
script: str
text: str | None = None
script: str | None = None
angle: float = 0.0
@dataclass(frozen=True)
class VerifiedTextManifest:
"""Text annotations cryptographically bound to one decoded RGB source."""
"""Verified text regions cryptographically bound to one decoded RGB source."""
source_pixel_sha256: str
width: int
@@ -55,17 +60,20 @@ def source_pixel_sha256(image: Image.Image) -> str:
def load_verified_text_manifest(path: Path, source: Image.Image) -> VerifiedTextManifest:
"""Load and validate a manually verified manifest for exactly ``source``."""
"""Load and validate an operator-verified manifest for exactly ``source``."""
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise ValueError(f"Cannot read text manifest {path}: {exc}") from exc
if not isinstance(payload, dict):
raise ValueError("Text manifest must be a JSON object")
if payload.get("schema_version") != TEXT_MANIFEST_SCHEMA:
raise ValueError(f"Text manifest schema_version must be {TEXT_MANIFEST_SCHEMA}")
schema_version = require_schema_version(
payload.get("schema_version"),
contract="text manifest",
supported=_SUPPORTED_TEXT_MANIFEST_SCHEMAS,
)
if payload.get("verified") is not True:
raise ValueError("Text manifest must contain verified=true after manual review")
raise ValueError("Text manifest must contain verified=true after operator verification")
rgb = source.convert("RGB")
width = _manifest_integer(payload, "width")
@@ -82,7 +90,7 @@ def load_verified_text_manifest(path: Path, source: Image.Image) -> VerifiedText
raw_lines = payload.get("lines")
if not isinstance(raw_lines, list) or not raw_lines:
raise ValueError("Text manifest lines must be a non-empty list")
lines = tuple(_load_line(item, width, height, index) for index, item in enumerate(raw_lines))
lines = tuple(_load_line(item, width, height, index, schema_version) for index, item in enumerate(raw_lines))
if list(lines) != sorted(lines, key=lambda line: (line.box[1], line.box[0])):
raise ValueError("Text manifest lines must be in top-to-bottom, left-to-right reading order")
return VerifiedTextManifest(actual_hash, width, height, lines)
@@ -95,7 +103,7 @@ def _manifest_integer(payload: dict[str, Any], key: str) -> int:
return value
def _load_line(item: Any, width: int, height: int, index: int) -> VerifiedTextLine:
def _load_line(item: Any, width: int, height: int, index: int, schema_version: int) -> VerifiedTextLine:
if not isinstance(item, dict):
raise ValueError(f"Text manifest line {index} must be an object")
raw_box = item.get("box")
@@ -109,12 +117,15 @@ def _load_line(item: Any, width: int, height: int, index: int) -> VerifiedTextLi
x1, y1, x2, y2 = box
if not (0 <= x1 < x2 <= width and 0 <= y1 < y2 <= height):
raise ValueError(f"Text manifest line {index} box is outside the source dimensions")
text = item.get("text")
script = item.get("script")
if not isinstance(text, str) or not text.strip():
raise ValueError(f"Text manifest line {index} text must be non-empty")
if not isinstance(script, str) or not script.strip():
raise ValueError(f"Text manifest line {index} script must be non-empty")
text: str | None = None
script: str | None = None
if schema_version == 1:
text = item.get("text")
script = item.get("script")
if not isinstance(text, str) or not text.strip():
raise ValueError(f"Text manifest line {index} text must be non-empty")
if not isinstance(script, str) or not script.strip():
raise ValueError(f"Text manifest line {index} script must be non-empty")
angle_value = item.get("angle", 0.0)
if isinstance(angle_value, bool) or not isinstance(angle_value, int | float):
raise ValueError(f"Text manifest line {index} angle must be numeric")
@@ -185,14 +196,41 @@ def restore_verified_text(
return Image.fromarray(restored)
def source_silhouette_mask(
source_rgb: NDArray[Any],
def _glyph_crop_box(
box: tuple[int, int, int, int],
angle: float = 0.0,
width: int,
height: int,
left_pad: int,
right_pad: int,
) -> tuple[int, int, int, int]:
"""Widen the detector box so glyph edges stay inside the silhouette crop.
Detector boxes can stop inside a leading flourish or trailing punctuation,
and Paddle line boxes sit 2-5 px above the true ink bottom on the poster
fixtures. The crop is otherwise the box itself, so those pixels never reach
the donor composite. Start 12% sideways, expand each side independently
while foreground reaches its boundary, and pad 8% up and 25% down.
"""
x1, y1, x2, y2 = box
line_h = max(1, y2 - y1)
pad_top = max(1, round(line_h * 0.08))
pad_bot = max(2, round(line_h * 0.25))
return (
max(0, x1 - left_pad),
max(0, y1 - pad_top),
min(width, x2 + right_pad),
min(height, y2 + pad_bot),
)
def _silhouette_crop_mask(
source_rgb: NDArray[Any],
crop_box: tuple[int, int, int, int],
angle: float,
) -> NDArray[Any]:
"""Recover a thresholded glyph shape without retaining source amplitudes."""
"""Threshold one candidate crop so its horizontal boundaries can be tested."""
height, width = source_rgb.shape[:2]
x1, y1, x2, y2 = _clip_box(box, width, height)
x1, y1, x2, y2 = crop_box
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
support = np.ones(gray.shape, dtype=np.uint8)
if angle:
@@ -211,7 +249,7 @@ def source_silhouette_mask(
background_luma = float(np.median(values))
else:
ring_pad = max(6, min(20, (y2 - y1) // 4))
rx1, ry1, rx2, ry2 = _clip_box((x1, y1, x2, y2), width, height, pad=ring_pad)
rx1, ry1, rx2, ry2 = _clip_box(crop_box, width, height, pad=ring_pad)
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
ring = np.ones(context.shape, dtype=bool)
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
@@ -225,6 +263,61 @@ def source_silhouette_mask(
else:
crop_mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
crop_mask[support == 0] = 0
return crop_mask
def _anchored_components_reach_sides(
crop_mask: NDArray[Any],
anchor_box: tuple[int, int, int, int],
) -> tuple[bool, bool]:
"""Whether anchored foreground reaches the left and right crop sides."""
count, labels, stats, _centroids = cv2.connectedComponentsWithStats(crop_mask, 8)
x1, y1, x2, y2 = anchor_box
anchor_counts = np.bincount(labels[y1:y2, x1:x2].reshape(-1), minlength=count)
anchor_counts[0] = 0
def reaches(edge_labels: NDArray[Any]) -> bool:
return any(
label != 0 and anchor_counts[label] >= max(3, round(stats[label, cv2.CC_STAT_AREA] * 0.1))
for label in np.unique(edge_labels)
)
return reaches(labels[y1:y2, :2]), reaches(labels[y1:y2, -2:])
def source_silhouette_mask(
source_rgb: NDArray[Any],
box: tuple[int, int, int, int],
angle: float = 0.0,
) -> NDArray[Any]:
"""Recover a thresholded glyph shape without retaining source amplitudes."""
height, width = source_rgb.shape[:2]
box_x1, box_y1, box_x2, box_y2 = box
line_h = max(1, box_y2 - box_y1)
step = max(2, round(line_h * GLYPH_SIDE_PAD_RATIO))
limit = max(step, round(line_h * GLYPH_SIDE_PAD_LIMIT))
left_pad = right_pad = step
while True:
x1, y1, x2, y2 = _glyph_crop_box(box, width, height, left_pad, right_pad)
crop_mask = _silhouette_crop_mask(source_rgb, (x1, y1, x2, y2), angle)
core_y1 = max(0, box_y1 - y1)
core_y2 = min(y2 - y1, box_y2 - y1)
anchor_box = (max(0, box_x1 - x1), core_y1, min(x2 - x1, box_x2 - x1), core_y2)
can_expand_left = x1 > 0 and left_pad < limit
can_expand_right = x2 < width and right_pad < limit
left_anchored, right_anchored = (
_anchored_components_reach_sides(crop_mask, anchor_box)
if can_expand_left or can_expand_right
else (False, False)
)
left_touches = can_expand_left and left_anchored
right_touches = can_expand_right and right_anchored
if not left_touches and not right_touches:
break
if left_touches:
left_pad = min(limit, left_pad + step)
if right_touches:
right_pad = min(limit, right_pad + step)
result = np.zeros((height, width), dtype=np.uint8)
result[y1:y2, x1:x2] = crop_mask
return result
@@ -69,6 +69,14 @@ SDXL_ZIMAGE_OPENAI_STRENGTH = 0.15
SDXL_ZIMAGE_GEMINI_STRENGTH = 0.25
SDXL_ZIMAGE_UNKNOWN_STRENGTH = SDXL_ZIMAGE_GEMINI_STRENGTH
# qwen-zimage keeps its resolution curve for openai/unknown content, but Google
# content takes this flat oracle-measured floor instead of the curve (whose top,
# 0.154, left a 4.33 MP CJK-sign fixture detected x3 on the full production path
# with a valid pixel control, 2026-08-18). 0.30 anchors measured clean in Gemini
# across two fixtures and two work accounts (CJK sign + 18-face, 3/3 checks each,
# 2026-08 research) and stayed clean under the vae-glyphs donor layer on top.
QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.30
# sdxl-zimage picks its strength from the VENDOR (unlike qwen-zimage, which derives it
# from image area). An unlisted or unknown vendor falls back to the Gemini value.
@@ -103,7 +111,8 @@ def resolve_adaptive_polish(adaptive_polish: bool | None, pipeline: str) -> bool
def strength_default_help() -> str:
"""Describe the live default policy without duplicating its values."""
return (
"profile-adaptive (qwen-zimage uses resolution-adaptive denoise; sdxl-zimage "
"profile-adaptive (qwen-zimage uses resolution-adaptive denoise, with a "
f"flat oracle-measured Google floor of {QWEN_ZIMAGE_GOOGLE_STRENGTH}; sdxl-zimage "
f"uses OpenAI {SDXL_ZIMAGE_OPENAI_STRENGTH} / Google {SDXL_ZIMAGE_GEMINI_STRENGTH} / "
f"unknown {SDXL_ZIMAGE_UNKNOWN_STRENGTH}, from the C2PA issuer)"
)
@@ -118,10 +127,14 @@ def resolve_strength(
) -> float:
"""Resolve a user override or the calibrated policy for a profile and vendor.
Total by design. qwen-zimage picks its strength from image area rather than from
the vendor, so it needs ``size``; returning ``None`` for it instead would push that
Total by design. qwen-zimage picks its strength from image area rather than
from the vendor, so it needs ``size``; returning ``None`` for it instead would push that
branch onto every caller and move one of the two strength policies outside this
module. ``size`` is required for qwen-zimage without an explicit strength.
One measured exception since 0.27.2: Google-provenance content on qwen-zimage
takes the flat ``QWEN_ZIMAGE_GOOGLE_STRENGTH`` floor instead of the area curve
(the curve's 0.154 top left a 4.33 MP fixture oracle-detected on the full
production path; see the constant's comment).
"""
if strength is not None:
return strength
@@ -129,6 +142,8 @@ def resolve_strength(
return _SDXL_ZIMAGE_STRENGTH_BY_VENDOR.get((vendor or "").casefold(), SDXL_ZIMAGE_UNKNOWN_STRENGTH)
if size is None:
raise ValueError("qwen-zimage resolves strength from image area, so size is required")
if (vendor or "").casefold() == "google":
return QWEN_ZIMAGE_GOOGLE_STRENGTH
from remove_ai_watermarks._internal.qwen_zimage_pipeline import resolution_adaptive_denoise
return resolution_adaptive_denoise(*size)
@@ -190,6 +190,7 @@ class WatermarkRemover:
tile_size: int = 1024,
tile_overlap: int = 128,
text_manifest: VerifiedTextManifest | None = None,
fidelity_anchor: bool = False,
) -> Path:
"""Regenerate image pixels and write the result without AI metadata.
@@ -219,6 +220,7 @@ class WatermarkRemover:
tile_size=tile_size,
tile_overlap=tile_overlap,
text_manifest=text_manifest,
fidelity_anchor=fidelity_anchor,
)
self._write_output(result, destination)
return destination
+2
View File
@@ -244,6 +244,7 @@ class InvisibleOptions:
tile_size: int = 1024
tile_overlap: int = 128
text_manifest: Path | None = None
fidelity_anchor: bool = False
# What the invisible stage did. "unavailable" is the one outcome the caller must
@@ -525,6 +526,7 @@ def _run_invisible(
tile_size=opts.tile_size,
tile_overlap=opts.tile_overlap,
text_manifest=opts.text_manifest,
fidelity_anchor=opts.fidelity_anchor,
)
say("invisible", "removed")
return "removed"
+35
View File
@@ -321,6 +321,16 @@ _text_manifest_option = click.option(
),
)
_fidelity_anchor_option = click.option(
"--fidelity-anchor/--no-fidelity-anchor",
default=False,
help=(
"With --text-manifest: blend 15% of the Qwen-VAE donor across the whole "
"frame. Off by default - the global blend was measured to return "
"detector-visible OpenAI SynthID on poster-scale manifests."
),
)
_visible_backend_option = click.option(
"--backend",
@@ -798,6 +808,7 @@ def cmd_erase(
@_force_option
@_cpu_offload_option
@_text_manifest_option
@_fidelity_anchor_option
@click.pass_context
def cmd_invisible(
ctx: click.Context,
@@ -818,6 +829,7 @@ def cmd_invisible(
force: bool,
cpu_offload: bool,
text_manifest: Path | None,
fidelity_anchor: bool,
) -> None:
"""Remove invisible AI watermarks (SynthID, StableSignature, TreeRing).
@@ -880,6 +892,7 @@ def cmd_invisible(
tile_size=tile_size,
tile_overlap=tile_overlap,
text_manifest=text_manifest,
fidelity_anchor=fidelity_anchor,
)
except (OSError, RuntimeError, ValueError) as exc:
console.print(f" Error: {exc}")
@@ -891,10 +904,27 @@ def cmd_invisible(
# ── Metadata operations ──
def _print_metadata_not_a_clean_verdict() -> None:
"""Repeat the identify empty-scan limit on metadata check and strip success.
``metadata --check`` and ``metadata --remove`` answer a narrower question than
``identify``: they report embedded AI metadata only. A quiet result used to
stop at "No AI metadata found" / "AI metadata stripped", which readers treat
as a clean-image verdict. The pixel channel is unchanged, and this project has
no local SynthID decoder, so the command must say so in the same words
``identify`` already uses.
"""
console.print(
" This is not the same as 'clean': a pixel watermark such as SynthID cannot be\n"
" detected here once its metadata proxy is absent."
)
def _print_metadata_report(source: Path, has_ai: bool, metadata: dict[str, str]) -> None:
"""Render one metadata inspection result for the generic and video commands."""
if not has_ai:
console.print(f" No AI metadata found in {source.name}")
_print_metadata_not_a_clean_verdict()
return
console.print(f" Warning: AI metadata detected in {source.name}:")
@@ -961,6 +991,7 @@ def cmd_metadata(
console.print(" the file could not be decoded, so it was copied through unchanged")
raise SystemExit(1)
console.print(f" AI metadata stripped -> {out}")
_print_metadata_not_a_clean_verdict()
# ── Video pipeline ──
@@ -1102,6 +1133,7 @@ def cmd_video_metadata(
console.print(f" still present: {', '.join(sorted(result.remaining))}")
raise SystemExit(1)
console.print(f" AI metadata stripped -> {result.output}")
_print_metadata_not_a_clean_verdict()
@cmd_video.command("invisible")
@@ -1428,6 +1460,7 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
@_force_option
@_cpu_offload_option
@_text_manifest_option
@_fidelity_anchor_option
@click.pass_context
def cmd_all(
ctx: click.Context,
@@ -1450,6 +1483,7 @@ def cmd_all(
force: bool,
cpu_offload: bool,
text_manifest: Path | None,
fidelity_anchor: bool,
) -> None:
"""Remove ALL watermarks: visible + invisible + metadata.
@@ -1528,6 +1562,7 @@ def cmd_all(
tile_size=tile_size,
tile_overlap=tile_overlap,
text_manifest=text_manifest,
fidelity_anchor=fidelity_anchor,
),
force=force,
progress=progress,
+1
View File
@@ -770,6 +770,7 @@ _AI_VENDOR_TOKENS: tuple[tuple[str, str], ...] = (
("google", "Google"),
("firefly", "Adobe"),
("adobe", "Adobe"),
("copilot", "Microsoft"),
("bing", "Microsoft"),
("designer", "Microsoft"),
("microsoft", "Microsoft"),
+15 -4
View File
@@ -150,6 +150,7 @@ class InvisibleEngine:
tile_size: int = 1024,
tile_overlap: int = 128,
text_manifest: Path | None = None,
fidelity_anchor: bool = False,
) -> Path:
"""Remove invisible watermark from an image.
@@ -185,8 +186,17 @@ class InvisibleEngine:
text_manifest: Operator-verified text lines bound to the decoded source
pixels. Enables the experimental Qwen-VAE ``vae-glyphs`` post-pass.
Requires the ``text-restoration`` extra and the ``qwen-zimage``
profile. Incompatible with tiling, downscaling, humanize, unsharp,
and adaptive polish because those combinations are not calibrated.
profile. Incompatible with downscaling, humanize, unsharp, and
adaptive polish. Tiling is supported: the VAE donor uses the same
overlapping tiles as the global pass, then glyph restore runs on
the blended full frame.
fidelity_anchor: Blend 15% of the Qwen-VAE donor across the whole frame
before glyph restoration. OFF by default since 0.27.1: that global
blend was measured to return detector-visible OpenAI SynthID on
poster-scale manifests (detected x6 with the anchor vs clean x6
without it, base clean; official Content Provenance API,
2026-08-19 - docs/text-protection-research.md). ``True`` reproduces
the 0.27.0 research behavior. Requires ``text_manifest``.
Returns:
Path to the cleaned image.
@@ -196,13 +206,13 @@ class InvisibleEngine:
seed = resolve_seed(seed)
adaptive_polish = resolve_adaptive_polish(adaptive_polish, self._remover.model_profile)
if fidelity_anchor and text_manifest is None:
raise ValueError("fidelity_anchor requires a text manifest")
if text_manifest is not None:
if self._remover.model_profile != QWEN_ZIMAGE_PROFILE:
raise ValueError("--text-manifest is supported only by the qwen-zimage profile")
if max_resolution != 0:
raise ValueError("--text-manifest requires --max-resolution 0")
if tile:
raise ValueError("--text-manifest is not calibrated with --tile")
if humanize > 0.0 or unsharp > 0.0 or adaptive_polish:
raise ValueError("--text-manifest requires humanize=0, unsharp=0, and adaptive polish disabled")
from remove_ai_watermarks import region_eraser
@@ -270,6 +280,7 @@ class InvisibleEngine:
tile_size=tile_size,
tile_overlap=tile_overlap,
text_manifest=verified_text,
fidelity_anchor=fidelity_anchor,
)
# Post-processing chain: decode the diffusion output ONCE, apply the
+1 -1
View File
@@ -1,4 +1,4 @@
"""Qwen (Tongyi Qianwen, Alibaba) visible watermark detector/localizer.
"""Qwen (Alibaba) visible watermark detector/localizer.
Qwen stamps its generations with a visible "千问AI生成" text strip in the
bottom-right corner -- the explicit AIGC label mandated by China's GB 45438-2025
+299
View File
@@ -0,0 +1,299 @@
"""Draft operator-verifiable text lines for verified-text manifests.
Proposal-only OCR: PaddleOCR detection plus three script-chosen recognition
engines (Latin / Cyrillic / CJK), accepted only when three crop paddings
normalize identically and every confidence clears the floor. ``accepted``
means crop-stable, NEVER ground-truth-correct: on the reference posters the
draft's exact-text precision was 90.0% and 94.4% because high-confidence OCR
still lost punctuation (one English comma dropped, one ideographic comma
replaced with ASCII). Every accepted line needs a human yes/no before it may
enter a manifest with ``verified: true``.
Heavy imports (paddle, and the numpy/cv2/PIL pixel stack) stay inside the
call so importing this module costs nothing without the ``text-draft`` extra;
``draft_available()`` reports whether the extra is installed.
"""
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false
from __future__ import annotations
import unicodedata
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
from remove_ai_watermarks._internal.text_restoration import source_pixel_sha256
if TYPE_CHECKING:
from pathlib import Path
__all__ = [
"DraftLine",
"RejectedLine",
"TextDraft",
"choose_language",
"draft_available",
"draft_text_lines",
"group_word_boxes",
"source_pixel_sha256",
"stable_recognition",
]
# Crop paddings probed per line: recognition must be invariant across them.
JITTER_RATIOS: tuple[float, ...] = (0.08, 0.12, 0.2)
DETECT_SCORE_FLOOR = 0.5
@dataclass(frozen=True)
class DraftLine:
"""One crop-stable proposal. ``accepted`` means stable, not correct."""
box: tuple[int, int, int, int]
text: str
script: str
language: str
min_score: float
@dataclass(frozen=True)
class RejectedLine:
"""A detected line whose recognition was unstable or low-confidence."""
box: tuple[int, int, int, int]
script: str
language: str
reads: tuple[tuple[str, float], ...]
@dataclass(frozen=True)
class TextDraft:
"""The full proposal set for one image."""
accepted: tuple[DraftLine, ...] = ()
rejected: tuple[RejectedLine, ...] = ()
def normalize_text(text: str) -> str:
"""Normalize text for layout-independent comparison (casefold + no spaces)."""
return "".join(unicodedata.normalize("NFC", text).casefold().split())
def _has_script(text: str, script: str) -> bool:
return any(script in unicodedata.name(character, "") for character in text)
def choose_language(probes: dict[str, tuple[str, float]]) -> str:
"""Pick ``ch``/``ru``/``en`` from what each probe engine actually read."""
if _has_script(probes["ch"][0], "CJK"):
return "ch"
if _has_script(probes["ru"][0], "CYRILLIC"):
return "ru"
return "en"
def stable_recognition(reads: list[tuple[str, float]], min_score: float = 0.85) -> str | None:
"""The proposal text when every read normalizes identically and clears the floor."""
normalized = {normalize_text(text) for text, _score in reads}
if len(normalized) != 1 or min(score for _text, score in reads) < min_score:
return None
return reads[0][0]
def _vertical_overlap_ratio(left: tuple[int, int, int, int], right: tuple[int, int, int, int]) -> float:
overlap = max(0, min(left[3], right[3]) - max(left[1], right[1]))
return overlap / max(1, min(left[3] - left[1], right[3] - right[1]))
def _row_center(box: tuple[int, int, int, int]) -> float:
return (box[1] + box[3]) / 2
def group_word_boxes(boxes: list[tuple[int, int, int, int]]) -> list[tuple[int, int, int, int]]:
"""Merge word detections into line boxes by vertical overlap and gap."""
groups: list[tuple[int, int, int, int]] = []
for box in sorted(boxes, key=lambda item: (_row_center(item), item[0])):
matches: list[int] = []
for index, group in enumerate(groups):
if _vertical_overlap_ratio(box, group) < 0.45:
continue
horizontal_gap = max(0, max(box[0], group[0]) - min(box[2], group[2]))
line_height = min(box[3] - box[1], group[3] - group[1])
if horizontal_gap <= max(24, line_height * 3):
matches.append(index)
if not matches:
groups.append(box)
continue
index = max(matches, key=lambda item: _vertical_overlap_ratio(box, groups[item]))
x1, y1, x2, y2 = groups[index]
groups[index] = min(x1, box[0]), min(y1, box[1]), max(x2, box[2]), max(y2, box[3])
return sorted(groups, key=_row_center)
def _recognition_box(
box: tuple[int, int, int, int],
script: str,
width: int,
height: int,
vertical_pad_ratio: float | None = None,
) -> tuple[int, int, int, int]:
x1, y1, x2, y2 = box
line_height = y2 - y1
if script == "cjk":
left_pad = max(16, round(line_height * 0.2))
right_pad = max(16, round(line_height * 0.6))
return max(0, x1 - left_pad), y1, min(width, x2 + right_pad), y2
pad_x = max(16, line_height)
pad_y = max(8, line_height // 3) if vertical_pad_ratio is None else max(8, round(line_height * vertical_pad_ratio))
return max(0, x1 - pad_x), max(0, y1 - pad_y), min(width, x2 + pad_x), min(height, y2 + pad_y)
def _recognize(
engine: Any,
image: Any,
box: tuple[int, int, int, int],
script: str,
vertical_pad_ratio: float | None = None,
) -> tuple[str, float]:
import cv2
height, width = image.shape[:2]
x1, y1, x2, y2 = _recognition_box(box, script, width, height, vertical_pad_ratio)
crop = image[y1:y2, x1:x2]
if crop.shape[0] < 64:
scale = 64 / crop.shape[0]
crop = cv2.resize(crop, None, fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)
result = next(iter(engine.predict(crop)))
return str(result.get("rec_text", "")), float(result.get("rec_score", 0.0))
def _detect_line_boxes(engine: Any, source_rgb: Any) -> list[tuple[int, int, int, int]]:
import numpy as np
boxes: list[tuple[int, int, int, int]] = []
for page in engine.predict(source_rgb):
detected = page.get("rec_boxes", None)
if detected is None or len(detected) == 0:
detected = page.get("rec_polys", [])
for score, raw_box in zip(page.get("rec_scores", []), detected, strict=False):
if float(score) < DETECT_SCORE_FLOOR:
continue
points = np.asarray(raw_box, dtype=np.float32).reshape(-1)
if points.size == 4:
x1, y1, x2, y2 = points
else:
points = points.reshape(-1, 2)
x1, y1 = points.min(axis=0)
x2, y2 = points.max(axis=0)
boxes.append((round(float(x1)), round(float(y1)), round(float(x2)), round(float(y2))))
return group_word_boxes(boxes)
def draft_available() -> bool:
"""True when the ``text-draft`` extra (paddleocr + paddle) can run."""
from remove_ai_watermarks.optional_deps import module_available
return module_available("paddleocr") and module_available("paddle")
def _build_engines() -> tuple[Any, dict[str, Any]]:
from paddleocr import PaddleOCR, TextRecognition
# enable_mkldnn=False is load-bearing on linux: paddle's oneDNN PIR executor
# raises NotImplementedError on the text-detection graph there (seen on the
# Modal draft container, 2026-08-19), while the reference CPU path runs. The
# eval scripts never hit it because macOS pads take a different path.
detector = PaddleOCR(
lang="ch",
use_doc_orientation_classify=False,
use_doc_unwarping=False,
use_textline_orientation=False,
enable_mkldnn=False,
)
engines = {
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec", enable_mkldnn=False),
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec", enable_mkldnn=False),
"ch": TextRecognition(model_name="PP-OCRv5_server_rec", enable_mkldnn=False),
}
return detector, engines
def draft_text_lines(
image: Path,
*,
min_score: float = 0.85,
detector: Any | None = None,
engines: dict[str, Any] | None = None,
stable: bool = True,
) -> TextDraft:
"""Propose verified-text manifest lines for ``image``; never verified ones.
Args:
image: path of the source image (draft boxes are in ITS pixel space).
min_score: recognition confidence floor. With ``stable=True`` every
jittered read must clear it; with ``stable=False`` the single
probe for the chosen language must.
detector/engines: injectable Paddle objects (tests use fakes); when
None they are built from the ``text-draft`` extra, which must be
installed (``draft_available()`` reports it).
stable: when True (default), accept a line only if three crop paddings
normalize identically. When False, take one recognition trio and
accept on score alone. Automatic restoration uses box/script only,
so the jitter gate is optional there.
Returns:
``TextDraft`` with crop-stable ``accepted`` proposals and unstable
``rejected`` lines. Both lists need human review before any manifest
may claim ``verified: true``; accepted means crop-stable, NOT
ground-truth-correct.
"""
import os
import numpy as np
from PIL import Image
if detector is None or engines is None:
if not draft_available():
raise RuntimeError(
"Text drafting requires PaddleOCR. Install: pip install 'remove-ai-watermarks[text-draft]'"
)
os.environ.setdefault("PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK", "True")
detector, engines = _build_engines()
source_rgb = np.asarray(Image.open(image).convert("RGB"))
accepted: list[DraftLine] = []
rejected: list[RejectedLine] = []
for box in _detect_line_boxes(detector, source_rgb):
probes: dict[str, tuple[str, float]] = {}
for language, engine in engines.items():
script = "cjk" if language == "ch" else "alphabetic"
probes[language] = _recognize(engine, source_rgb, box, script, 0.1)
language = choose_language(probes)
script = "cjk" if language == "ch" else "alphabetic"
if stable:
reads = [_recognize(engines[language], source_rgb, box, script, ratio) for ratio in JITTER_RATIOS]
text = stable_recognition(reads, min_score)
else:
text, score = probes[language]
reads = [(text, score)]
if not text.strip() or score < min_score:
text = None
if text is None:
rejected.append(
RejectedLine(
box=box,
script=script,
language=language,
reads=tuple((value, score) for value, score in reads),
)
)
else:
accepted.append(
DraftLine(
box=box,
text=text,
script=script,
language=language,
min_score=min(score for _value, score in reads),
)
)
return TextDraft(accepted=tuple(accepted), rejected=tuple(rejected))
@@ -20,7 +20,7 @@ Entries:
- ``gemini`` -- Google Gemini / Nano Banana sparkle, bottom-right.
- ``doubao`` -- ByteDance Doubao "豆包AI生成" text strip, bottom-right.
- ``jimeng`` -- ByteDance Jimeng / Dreamina "★ 即梦AI" wordmark, bottom-right.
- ``qwen`` -- Alibaba Tongyi Qianwen "千问AI生成" text strip, bottom-right.
- ``qwen`` -- Alibaba Qwen "千问AI生成" text strip, bottom-right.
- ``kling`` -- Kuaishou Kling "可灵AI 3.0" text strip, bottom-right.
- ``yuanbao`` -- Tencent Yuanbao "元宝 / AI生成" two-line mark, bottom-right.
- ``samsung`` -- Samsung Galaxy AI "Contenuti generati dall'AI" strip, bottom-left.
@@ -651,7 +651,7 @@ _REGISTRY: tuple[KnownMark, ...] = (
"qwen",
"Qwen 千问AI生成 text",
"bottom-right",
platform="Alibaba Tongyi Qianwen (visible 千问AI生成 mark detected)",
platform="Alibaba Qwen (visible 千问AI生成 mark detected)",
tc260_producer_codes=("91440101MA9Y9T4H7A",),
),
_text_mark(
+3
View File
@@ -629,11 +629,13 @@ class TestMetadataCommand:
result = runner.invoke(main, ["metadata", str(tmp_clean_png), "--check"])
assert result.exit_code == 0
assert "No AI metadata" in result.output
assert "not the same as 'clean'" in result.output
def test_metadata_check_ai(self, runner, tmp_png_with_ai_metadata):
result = runner.invoke(main, ["metadata", str(tmp_png_with_ai_metadata), "--check"])
assert result.exit_code == 0
assert "AI metadata detected" in result.output
assert "not the same as 'clean'" not in result.output
def test_metadata_remove(self, runner, tmp_png_with_ai_metadata, tmp_path):
output = tmp_path / "stripped.png"
@@ -649,6 +651,7 @@ class TestMetadataCommand:
)
assert result.exit_code == 0
assert "stripped" in result.output
assert "not the same as 'clean'" in result.output
def test_metadata_remove_reports_failure_when_the_strip_was_a_no_op(self, runner, tmp_path):
"""A file PIL cannot decode is copied through UNCHANGED by the fail-safe.
+10 -1
View File
@@ -16,6 +16,7 @@ from unittest.mock import patch
import pytest
from remove_ai_watermarks._internal.constants import C2PA_AI_VENDORS, C2PA_CLAIM_GENERATOR_PLATFORMS
from remove_ai_watermarks.identify import (
ProvenanceEvidence,
ProvenanceReport,
@@ -1300,12 +1301,20 @@ class TestVendorOf:
def test_registered_vendors_normalize(self):
# Regression: these registered C2PA vendors returned None, so their claims never
# entered clash detection (a coverage hole). They now normalize to one origin.
assert _vendor_of("ByteDance (Doubao / Jimeng / Volcano Engine)") == "ByteDance"
assert _vendor_of("Microsoft (Copilot / Designer)") == "Microsoft"
assert _vendor_of("Copilot") == "Microsoft"
assert _vendor_of("ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)") == "ByteDance"
assert _vendor_of("Dreamina/1.2") == "ByteDance"
assert _vendor_of("Canva (Magic Media)") == "Canva"
assert _vendor_of("Black Forest Labs (FLUX)") == "Black Forest Labs"
assert _vendor_of("Eleven Labs Inc.") == "ElevenLabs"
def test_bytedance_issuers_share_one_platform(self):
expected = "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"
platforms = {vendor.platform for vendor in C2PA_AI_VENDORS if vendor.needle == "ByteDance"}
assert platforms == {expected}
assert ("dreamina", expected) in C2PA_CLAIM_GENERATOR_PLATFORMS
class TestIntegrityClashesHelper:
def test_two_ai_vendors_clash(self):
+21 -1
View File
@@ -61,7 +61,6 @@ class TestVerifiedTextMode:
cases = (
("sdxl-zimage", {}, "qwen-zimage"),
("qwen-zimage", {"max_resolution": 1024}, "max-resolution 0"),
("qwen-zimage", {"tile": True}, "not calibrated"),
("qwen-zimage", {"humanize": 1.0}, "humanize=0"),
("qwen-zimage", {"adaptive_polish": True}, "polish disabled"),
)
@@ -73,6 +72,15 @@ class TestVerifiedTextMode:
**kwargs,
)
def test_rejects_fidelity_anchor_without_manifest(self, tmp_path):
import pytest
with pytest.raises(ValueError, match="fidelity_anchor requires a text manifest"):
self._engine().remove_watermark(
tmp_path / "unused.png",
fidelity_anchor=True,
)
def test_loads_and_forwards_verified_manifest(self, tmp_path, monkeypatch):
import json
@@ -111,6 +119,18 @@ class TestVerifiedTextMode:
engine.remove_watermark(source, output, text_manifest=manifest)
assert seen["text_manifest"].lines[0].text == "Exact"
# Leak-safe default since 0.27.1: the global 15% donor blend is OFF unless
# explicitly requested (measured to return detector-visible OpenAI SynthID
# on poster-scale manifests; see docs/text-protection-research.md).
assert seen["fidelity_anchor"] is False
engine.remove_watermark(source, output, text_manifest=manifest, fidelity_anchor=True)
assert seen["fidelity_anchor"] is True
engine.remove_watermark(source, output, text_manifest=manifest, tile=True)
assert seen["tile"] is True
class TestNativeOutputSize:
+10 -4
View File
@@ -14,6 +14,7 @@ import pytest
from remove_ai_watermarks._internal.utils import get_image_format, is_supported_format
from remove_ai_watermarks._internal.watermark_profiles import (
PROFILE_CHOICES,
QWEN_ZIMAGE_GOOGLE_STRENGTH,
REMOVAL_MODULES,
SDXL_ZIMAGE_GEMINI_STRENGTH,
SDXL_ZIMAGE_OPENAI_STRENGTH,
@@ -204,16 +205,21 @@ class TestResolveStrength:
qwen-zimage picks strength from image area, so it takes the size. Returning
None for it would push that branch onto every caller and leave one of the two
strength policies living outside this module. The vendor is ignored here on
purpose - the curve, not the issuer, is what was calibrated.
strength policies living outside this module. Google content is the one
measured exception since 0.27.2: it takes the flat oracle-measured floor
instead of the curve (the curve's 0.154 top left a 4.33 MP fixture
oracle-detected on the full production path).
"""
assert resolve_strength(None, "google", "qwen-zimage", size=(2000, 1850)) == pytest.approx(0.154)
assert resolve_strength(None, "openai", "qwen-zimage", size=(2000, 1850)) == pytest.approx(0.154)
assert resolve_strength(None, None, "qwen-zimage", size=(600, 500)) == pytest.approx(0.084)
assert resolve_strength(None, "google", "qwen-zimage", size=(600, 500)) == QWEN_ZIMAGE_GOOGLE_STRENGTH
# The floor holds at every size, not only above the curve's top rung.
assert resolve_strength(None, "google", "qwen-zimage", size=(2000, 1850)) == QWEN_ZIMAGE_GOOGLE_STRENGTH
def test_qwen_zimage_without_a_size_fails_loudly(self):
"""A missing size must not silently fall back to some vendor value."""
with pytest.raises(ValueError, match="size is required"):
resolve_strength(None, "google", "qwen-zimage")
resolve_strength(None, "openai", "qwen-zimage")
def test_sdxl_zimage_uses_its_flat_vendor_ladder(self):
+57 -4
View File
@@ -646,9 +646,58 @@ def test_no_face_path_still_runs_verified_text_restoration(monkeypatch):
result = pipeline.run(source, strength=0.1, seed=0, text_manifest=manifest)
assert result is restored
# Off by default since 0.27.1 (leak finding, docs/text-protection-research.md):
# no whole-frame donor blend; the raw global result feeds restoration.
blend.assert_not_called()
restore.assert_called_once_with(source, global_result, donor, manifest.lines)
result = pipeline.run(source, strength=0.1, seed=0, text_manifest=manifest, fidelity_anchor=True)
assert result is restored
blend.assert_called_once_with(global_result, donor)
restore.assert_called_once_with(source, anchor, donor, manifest.lines)
restore.assert_called_with(source, anchor, donor, manifest.lines)
def test_tiled_verified_text_runs_vae_donor_per_tile(monkeypatch):
from remove_ai_watermarks._internal import qwen_zimage_pipeline, text_restoration
from remove_ai_watermarks._internal.qwen_zimage_pipeline import QwenZImagePipeline
from remove_ai_watermarks._internal.text_restoration import VerifiedTextLine, VerifiedTextManifest
pipeline = object.__new__(QwenZImagePipeline)
pipeline.device = "cuda"
pipeline.progress_callback = None
source = Image.new("RGB", (96, 80), (10, 20, 30))
restored = Image.new("RGB", (96, 80), (130, 140, 150))
pipeline._qwen_vae_roundtrip = MagicMock(side_effect=lambda tile: tile)
pipeline._run_global = MagicMock(side_effect=lambda tile, _strength, _seed: tile)
monkeypatch.setattr(qwen_zimage_pipeline, "detect_faces", lambda _image: [])
restore = MagicMock(return_value=restored)
monkeypatch.setattr(text_restoration, "restore_verified_text", restore)
manifest = VerifiedTextManifest(
"0" * 64,
96,
80,
(VerifiedTextLine((4, 4, 20, 16), "Exact", "alphabetic"),),
)
result = pipeline.run(
source,
strength=0.1,
seed=0,
tile=True,
tile_size=64,
tile_overlap=16,
text_manifest=manifest,
)
assert result is restored
assert pipeline._qwen_vae_roundtrip.call_count > 1
assert pipeline._run_global.call_count > 1
restore.assert_called_once()
assert restore.call_args.args[0].size == (96, 80)
assert restore.call_args.args[1].size == (96, 80)
assert restore.call_args.args[2].size == (96, 80)
def test_watermark_remover_dispatches_to_full_pipeline(tmp_path, monkeypatch):
@@ -858,6 +907,7 @@ def test_invisible_engine_passes_the_seed_but_never_a_step_count(tmp_image_path,
def test_sdxl_zimage_strength_is_vendor_adaptive_and_leaves_other_profiles_alone():
"""An SDXL global pass needs more strength than Qwen, so it gets its own policy."""
from remove_ai_watermarks._internal.watermark_profiles import (
QWEN_ZIMAGE_GOOGLE_STRENGTH,
SDXL_ZIMAGE_GEMINI_STRENGTH,
SDXL_ZIMAGE_OPENAI_STRENGTH,
resolve_strength,
@@ -867,11 +917,14 @@ def test_sdxl_zimage_strength_is_vendor_adaptive_and_leaves_other_profiles_alone
assert resolve_strength(None, "google", "sdxl-zimage") == pytest.approx(SDXL_ZIMAGE_GEMINI_STRENGTH)
# Unknown provenance takes the stricter of the two.
assert resolve_strength(None, None, "sdxl-zimage") == pytest.approx(SDXL_ZIMAGE_GEMINI_STRENGTH)
# An explicit value still wins, and qwen-zimage is untouched by this ladder: it
# defers to its resolution curve rather than to a vendor value.
# An explicit value still wins, and qwen-zimage's curve is untouched by this
# ladder - with the one measured exception: Google content takes the flat
# oracle floor (0.27.2), not the area curve.
assert resolve_strength(0.4, "google", "sdxl-zimage") == pytest.approx(0.4)
assert resolve_strength(None, "openai", "qwen-zimage", size=(2000, 1850)) == pytest.approx(0.154)
assert resolve_strength(None, "google", "qwen-zimage", size=(2000, 1850)) == pytest.approx(0.154)
assert resolve_strength(None, "google", "qwen-zimage", size=(2000, 1850)) == pytest.approx(
QWEN_ZIMAGE_GOOGLE_STRENGTH
)
def test_sdxl_zimage_shares_the_fixed_seed_contract():
+158
View File
@@ -0,0 +1,158 @@
"""text_draft: proposal-only OCR for verified-text manifests (no paddle needed)."""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
from PIL import Image
from remove_ai_watermarks.text_draft import (
TextDraft,
choose_language,
draft_available,
draft_text_lines,
group_word_boxes,
stable_recognition,
)
if TYPE_CHECKING:
from pathlib import Path
class TestPureHelpers:
def test_choose_language_follows_unicode_script(self):
assert choose_language({"ch": ("你好", 0.9), "ru": ("xxx", 0.1), "en": ("yyy", 0.1)}) == "ch"
assert choose_language({"ch": ("??", 0.1), "ru": ("привет", 0.9), "en": ("yyy", 0.1)}) == "ru"
assert choose_language({"ch": ("??", 0.1), "ru": ("xxx", 0.1), "en": ("hello", 0.9)}) == "en"
def test_stable_recognition_requires_identical_normalizations(self):
# Same text modulo spacing/case (comma kept): stable.
assert (
stable_recognition([("Hello, World", 0.97), ("hello,world", 0.96), ("HELLO, WORLD", 0.95)])
== "Hello, World"
)
# Different texts: rejected (None).
assert stable_recognition([("Hello", 0.97), ("Hallo", 0.96), ("Hello", 0.95)]) is None
# Identical text but one read under the floor: rejected.
assert stable_recognition([("Hello", 0.97), ("Hello", 0.60), ("Hello", 0.95)]) is None
assert stable_recognition([("Hello", 0.97), ("Hello", 0.96), ("Hello", 0.95)], min_score=0.99) is None
def test_group_word_boxes_merges_a_line_and_keeps_rows_apart(self):
# Two words on one baseline merge; a distant lower line stays separate.
merged = group_word_boxes([(10, 100, 60, 140), (70, 100, 120, 140)])
assert len(merged) == 1
assert merged[0] == (10, 100, 120, 140)
apart = group_word_boxes([(10, 100, 60, 140), (10, 300, 60, 340)])
assert len(apart) == 2
class TestDraftAvailable:
def test_returns_a_bool_without_importing_paddle(self):
assert isinstance(draft_available(), bool)
class _FakeDetector:
"""Emits Paddle-style pages for one synthetic poster."""
def __init__(self, boxes: list[tuple[int, int, int, int]]) -> None:
self._boxes = boxes
def predict(self, _image: Any) -> Any:
import numpy as np
yield {
"rec_boxes": np.asarray(self._boxes, dtype=np.float32),
"rec_scores": [0.99] * len(self._boxes),
}
class _FakeEngine:
"""Returns a fixed text at high score regardless of the crop."""
def __init__(self, text: str) -> None:
self._text = text
def predict(self, _crop: Any) -> Any:
yield {"rec_text": self._text, "rec_score": 0.97}
class _JitterEngine:
"""Changes its answer with the crop height - exactly what the gate rejects."""
def __init__(self, texts: list[str]) -> None:
self._texts = texts
self._calls = 0
def predict(self, crop: Any) -> Any:
text = self._texts[self._calls % len(self._texts)]
self._calls += 1
yield {"rec_text": text, "rec_score": 0.97}
class TestDraftTextLines:
@staticmethod
def _poster(tmp_path: Path) -> Path:
path = tmp_path / "poster.png"
Image.new("RGB", (400, 300), (255, 255, 255)).save(path)
return path
def test_accepts_crop_stable_and_rejects_jittered(self, tmp_path: Path):
path = self._poster(tmp_path)
stable = _FakeEngine("Invoice 42")
jitter = _JitterEngine(["Hello", "Hallo", "Hullo"])
# All three probe engines read the same string, so language = en; the
# jitter engine then serves as the en engine and flips across crops.
draft = draft_text_lines(
path,
detector=_FakeDetector([(20, 20, 200, 60), (20, 80, 200, 120)]),
engines={"en": jitter, "ru": stable, "ch": stable},
)
assert isinstance(draft, TextDraft)
assert len(draft.rejected) == 2
assert all(line.language == "en" for line in draft.rejected)
assert draft.accepted == ()
def test_accepted_line_carries_box_text_script_and_floor(self, tmp_path: Path):
path = self._poster(tmp_path)
engine = _FakeEngine("Total: 1,234.56")
draft = draft_text_lines(
path,
detector=_FakeDetector([(20, 20, 260, 60)]),
engines={"en": engine, "ru": engine, "ch": engine},
)
(line,) = draft.accepted
assert line.box == (20, 20, 260, 60)
assert line.text == "Total: 1,234.56"
assert line.script == "alphabetic"
assert line.language == "en"
assert line.min_score >= 0.85
def test_cjk_probe_switches_script_and_language(self, tmp_path: Path):
path = self._poster(tmp_path)
cjk = _FakeEngine("每天都是一个新的机会。")
latin = _FakeEngine("hello")
draft = draft_text_lines(
path,
detector=_FakeDetector([(30, 30, 300, 90)]),
engines={"en": latin, "ru": latin, "ch": cjk},
)
(line,) = draft.accepted
assert line.language == "ch"
assert line.script == "cjk"
def test_unstable_geometry_draft_accepts_a_single_high_score_read(self, tmp_path: Path):
path = self._poster(tmp_path)
jitter = _JitterEngine(["Hello", "Hallo", "Hullo"])
stable = _FakeEngine("Hello")
# Probes disagree across engines so language is en; jitter would fail
# the stable gate. Geometry mode uses the one en probe and accepts.
draft = draft_text_lines(
path,
min_score=0.75,
stable=False,
detector=_FakeDetector([(20, 20, 200, 60)]),
engines={"en": jitter, "ru": stable, "ch": stable},
)
(line,) = draft.accepted
assert line.text == "Hello"
assert line.min_score >= 0.75
+72 -1
View File
@@ -15,6 +15,7 @@ from remove_ai_watermarks._internal.text_restoration import (
load_verified_text_manifest,
restore_verified_text,
source_pixel_sha256,
source_silhouette_mask,
)
@@ -36,6 +37,17 @@ def _manifest(image: Image.Image) -> dict[str, object]:
}
def _geometry_manifest(image: Image.Image) -> dict[str, object]:
return {
"schema_version": 2,
"verified": True,
"source_pixel_sha256": source_pixel_sha256(image),
"width": image.width,
"height": image.height,
"lines": [{"box": [8, 8, 40, 24], "angle": 0.0}],
}
def test_pixel_hash_ignores_container_metadata(tmp_path) -> None:
image = Image.new("RGB", (48, 32), (10, 20, 30))
plain = tmp_path / "plain.png"
@@ -62,6 +74,40 @@ def test_verified_manifest_is_bound_to_source_pixels(tmp_path) -> None:
assert loaded.lines == (VerifiedTextLine((8, 8, 40, 24), "Exact text", "alphabetic", 0.0),)
def test_geometry_manifest_needs_no_transcription_or_script(tmp_path) -> None:
source = Image.new("RGB", (48, 32), (10, 20, 30))
path = tmp_path / "regions.json"
path.write_text(json.dumps(_geometry_manifest(source)), encoding="utf-8")
loaded = load_verified_text_manifest(path, source)
assert loaded.lines == (VerifiedTextLine((8, 8, 40, 24), angle=0.0),)
@pytest.mark.parametrize("field", ["text", "script"])
def test_schema_one_still_requires_text_metadata(tmp_path, field) -> None:
source = Image.new("RGB", (48, 32), (10, 20, 30))
payload = _manifest(source)
del payload["lines"][0][field]
path = tmp_path / "lines.json"
path.write_text(json.dumps(payload), encoding="utf-8")
with pytest.raises(ValueError, match=field):
load_verified_text_manifest(path, source)
@pytest.mark.parametrize("schema_version", [True, 0, 3])
def test_manifest_rejects_unsupported_schema_versions(tmp_path, schema_version) -> None:
source = Image.new("RGB", (48, 32), (10, 20, 30))
payload = _geometry_manifest(source)
payload["schema_version"] = schema_version
path = tmp_path / "lines.json"
path.write_text(json.dumps(payload), encoding="utf-8")
with pytest.raises(ValueError, match="Unsupported text manifest schema"):
load_verified_text_manifest(path, source)
@pytest.mark.parametrize(
("mutation", "message"),
[
@@ -119,10 +165,35 @@ def test_restoration_uses_lama_and_qwen_vae_core(monkeypatch) -> None:
Image.fromarray(source),
Image.fromarray(candidate),
Image.fromarray(donor),
(VerifiedTextLine((8, 8, 48, 28), "Exact text", "alphabetic"),),
(VerifiedTextLine((8, 8, 48, 28)),),
)
restored = np.asarray(result)
assert calls
assert np.all(restored[16, 20] == donor[16, 20])
assert np.all(restored[0, 0] == candidate[0, 0])
def test_silhouette_includes_descender_below_the_detector_box() -> None:
source = np.full((40, 50, 3), 240, dtype=np.uint8)
source[10:22, 18:24] = 20
source[22:27, 18:22] = 20 # tail of a y / Cyrillic u, under the box
mask = source_silhouette_mask(source, (10, 10, 40, 22))
assert mask[24, 20] == 255
assert mask[16, 20] == 255
def test_silhouette_includes_glyph_edges_beside_the_detector_box() -> None:
source = np.full((40, 70, 3), 240, dtype=np.uint8)
source[10:22, 20:50] = 20
source[14:18, 10:20] = 20 # leading flourish, left of the detector box
source[14:18, 50:58] = 20 # punctuation or icon edge, right of the box
source[14:18, 2:5] = 20 # separate decoration must stay outside the crop
mask = source_silhouette_mask(source, (20, 10, 50, 22))
assert mask[16, 12] == 255
assert mask[16, 56] == 255
assert mask[16, 3] == 0
+2
View File
@@ -951,6 +951,7 @@ class TestVideoMetadataCli:
assert result.exit_code == 0, result.output
assert "AI metadata detected" in result.output
assert "not the same as 'clean'" not in result.output
def test_remove_reports_output(self, tmp_path: Path):
runner = CliRunner()
@@ -961,6 +962,7 @@ class TestVideoMetadataCli:
assert result.exit_code == 0, result.output
assert "AI metadata stripped" in result.output
assert "not the same as 'clean'" in result.output
assert C2PA_UUID not in output.read_bytes()
def test_rejects_image_input(self, tmp_clean_png: Path):
Generated
+977 -178
View File
File diff suppressed because it is too large. Load diff