mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-15 22:10:45 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8871eb3f67 | ||
|
|
44ec203c22 | ||
|
|
4beb61463a | ||
|
|
0d11e4099c | ||
|
|
fa83c1444f | ||
|
|
f52e40ae24 | ||
|
|
0aa03c4e89 | ||
|
|
91aa49b03a | ||
|
|
ee865e95e4 | ||
|
|
d84fd1a125 | ||
|
|
e315eacb14 | ||
|
|
3ba4ace7e0 | ||
|
|
13888704f8 | ||
|
|
e938b57f8c | ||
|
|
3cd1e47935 | ||
|
|
2298282b8b | ||
|
|
42d427731f | ||
|
|
50d9f44829 |
No files matched your search
@@ -202,8 +202,9 @@ remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
Typography-heavy images can opt into the experimental verified-text post-pass.
|
||||
It requires manually reviewed strings and line boxes; it never trusts OCR as ground
|
||||
truth or runs automatically:
|
||||
It accepts manually reviewed strings and line boxes or an operator-verified
|
||||
geometry-only manifest; it never treats raw OCR output as ground truth or runs
|
||||
automatically:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
|
||||
+28
-6
@@ -157,6 +157,11 @@ remove-ai-watermarks metadata image.png --remove -o clean.png
|
||||
When `-o` is omitted, removal overwrites the source. Standard metadata is kept
|
||||
unless you pass `--remove-all`.
|
||||
|
||||
A quiet `--check` or a successful `--remove` is not a clean verdict. The command
|
||||
only inspects and strips embedded AI metadata; a pixel watermark such as SynthID
|
||||
has no local decoder once that metadata proxy is gone. `identify` reports the
|
||||
same limit.
|
||||
|
||||
The command also supports the audio and video containers listed in
|
||||
[supported signals](supported-signals.md). ffmpeg must be available for the
|
||||
non-ISOBMFF audio and video path.
|
||||
@@ -225,6 +230,10 @@ remove-ai-watermarks video metadata input.mp4 --check
|
||||
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
|
||||
```
|
||||
|
||||
As with the generic `metadata` command, a quiet check or a successful strip is
|
||||
not a clean verdict: video SynthID is not decoded locally after the metadata
|
||||
proxy is gone.
|
||||
|
||||
Supported containers are MP4, MOV, M4V, WebM, MKV, AVI, and FLV. The operation
|
||||
delegates to the same verified metadata scanner and stripper as the generic
|
||||
`metadata` command, so detection and removal stay in parity. Video and audio
|
||||
@@ -408,7 +417,7 @@ the source with the Qwen VAE, blends 15% of that reconstruction into the normal
|
||||
only the reconstructed glyph cores through source-derived silhouettes. It does not
|
||||
run OCR or choose which strings are correct.
|
||||
|
||||
Install the combined extra and run only with a manually reviewed manifest:
|
||||
Install the combined extra and run only with an operator-verified manifest:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
@@ -416,16 +425,29 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
The manifest is a JSON object with `schema_version: 1`, `verified: true`, decoded
|
||||
RGB dimensions, `source_pixel_sha256`, and a non-empty `lines` array. Each line has
|
||||
an integer `[x1, y1, x2, y2]` box, exact `text`, a non-empty `script`, and an optional
|
||||
angle from -30 to 30 degrees. Lines must be in top-to-bottom, left-to-right order.
|
||||
``verified: true`` may also be set by an automated operator that attests
|
||||
machine-verified geometry: stability-gated detector boxes inside sane caps. Such
|
||||
operators should use the geometry-only schema 2, which carries no transcription
|
||||
or script metadata.
|
||||
|
||||
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default: it
|
||||
was measured to return detector-visible OpenAI SynthID on poster-scale manifests
|
||||
(official Content Provenance API, 2026-08-19). `--fidelity-anchor` restores the
|
||||
0.27.0 research behavior; text-box fidelity lost by the default is well under one
|
||||
MAE point on the measured fixtures.
|
||||
|
||||
The manifest is a JSON object with `verified: true`, decoded RGB dimensions,
|
||||
`source_pixel_sha256`, and a non-empty `lines` array. Schema 1 is retained for
|
||||
manually reviewed annotations: each line has an integer `[x1, y1, x2, y2]` box,
|
||||
exact `text`, a non-empty `script`, and an optional angle from -30 to 30 degrees.
|
||||
Schema 2 is geometry-only: each line has the box and optional angle, with no
|
||||
required `text` or `script`. Lines must be in top-to-bottom, left-to-right order.
|
||||
The hash binds the annotations to decoded RGB geometry and pixels, so metadata-only
|
||||
container changes remain valid while a resized or edited source fails closed. The
|
||||
experimental helper
|
||||
`remove_ai_watermarks._internal.text_restoration.source_pixel_sha256` computes it.
|
||||
|
||||
This mode is supported only by `qwen-zimage` at native untiled geometry with
|
||||
This mode is supported only by `qwen-zimage` at native geometry with
|
||||
`humanize=0`, `unsharp=0`, and adaptive polish disabled. `all` also accepts the flag,
|
||||
but its manifest must match the pixels entering the invisible stage; if visible-mark
|
||||
removal changes those pixels, the hash check rejects the run. One oracle verdict does
|
||||
|
||||
@@ -57,6 +57,19 @@ Invisible removal does not decode and delete a payload. It regenerates the
|
||||
image through a diffusion pipeline. Faces, text, colors, and fine detail can
|
||||
change even when the watermark is successfully disrupted.
|
||||
|
||||
Text, tables, and UI screenshots are the worst case for a forced scrub: the
|
||||
model redraws glyphs as plausible-but-wrong shapes. Do not run the invisible
|
||||
stage on such content unless a locally-detectable invisible signal exists.
|
||||
The default `all`, `invisible`, and `batch` commands already skip it then, and
|
||||
`visible` (which strips AI metadata by default) or `metadata` alone never
|
||||
redraws a glyph outside the filled watermark box. `--force` is for content you
|
||||
know carries a pixel watermark despite no local signal; on a clean screenshot
|
||||
it only buys distortion. Preserving or pasting back the original text pixels
|
||||
during a real scrub is deliberately not offered: an invisible watermark such
|
||||
as SynthID lives in the pixels everywhere, including inside the glyphs, so
|
||||
frozen text regions would keep the watermark
|
||||
([text protection research](text-protection-research.md)).
|
||||
|
||||
`qwen-zimage` is the default profile and `sdxl-zimage` the only alternative.
|
||||
Both are CUDA only and differ only in the global regeneration model: each
|
||||
conditions that stage on a canny edge map, which preserves structure but not
|
||||
@@ -165,8 +178,12 @@ The two profiles resolve an unset strength differently, because different things
|
||||
were measured for each.
|
||||
|
||||
`qwen-zimage` reads it from image area, through the resolution-adaptive denoise
|
||||
curve. The vendor is deliberately ignored: the curve, not the issuer, is what was
|
||||
calibrated.
|
||||
curve. The vendor is deliberately ignored for openai/unknown content: the curve,
|
||||
not the issuer, is what was calibrated. One measured exception (0.27.2):
|
||||
Google-provenance content takes the flat `QWEN_ZIMAGE_GOOGLE_STRENGTH` 0.30 floor
|
||||
- the curve's 0.154 top left a 4.33 MP fixture oracle-detected on the full
|
||||
production path (2026-08-18), while 0.30 anchors measured clean on two fixtures
|
||||
in two accounts.
|
||||
|
||||
`sdxl-zimage` reads it from the C2PA issuer, on a flat ladder:
|
||||
|
||||
|
||||
@@ -42,6 +42,10 @@ Important contracts:
|
||||
- `invisible` writes no output when no supported local signal is found, unless
|
||||
`--force` is supplied.
|
||||
- The two no-signal conditions currently share exit code `2`.
|
||||
- A quiet `metadata --check` and a successful `metadata --remove` (same for
|
||||
`video metadata`) end by repeating the `identify` limit: the pixel channel is
|
||||
untouched and a watermark such as SynthID has no local decoder once its
|
||||
metadata proxy is gone, so neither outcome is a clean verdict.
|
||||
- Hard processing and write failures exit with code `1`.
|
||||
- `all` can still write the completed visible and metadata stages when the
|
||||
diffusion dependencies are unavailable, but exits with code `1` so the
|
||||
@@ -960,9 +964,10 @@ requires the same provider-oracle and identity evaluation as a model change.
|
||||
|
||||
[`_internal/text_restoration.py`](../src/remove_ai_watermarks/_internal/text_restoration.py)
|
||||
implements the opt-in `vae-glyphs` stage. A versioned manifest carries manually
|
||||
reviewed strings and source-space line boxes, plus a SHA-256 over decoded RGB width,
|
||||
height, and pixels. Validation happens before model loading. The product never treats
|
||||
OCR confidence as verification.
|
||||
reviewed strings and source-space line boxes in schema 1, or verified source-space
|
||||
geometry alone in schema 2, plus a SHA-256 over decoded RGB width, height, and pixels.
|
||||
Validation happens before model loading. The library never treats OCR confidence as
|
||||
verification, and geometry-only operators do not need to invent text or script fields.
|
||||
|
||||
When enabled, `QwenZImagePipeline` reconstructs the source once through its already
|
||||
loaded Qwen VAE, runs the ordinary global and face stages, blends 15% of the VAE
|
||||
@@ -971,7 +976,12 @@ The compositor derives binary source and candidate silhouettes, groups nearby li
|
||||
uses LaMa for the initial and residual-glyph erase passes, paints fresh silhouette
|
||||
edges, then copies the Qwen-VAE core with a 0.5-pixel feather. The evaluation script
|
||||
imports these same mask and compositing helpers so the two implementations cannot
|
||||
silently drift.
|
||||
silently drift. Silhouette crops start 12% of line height beyond each horizontal
|
||||
side, then expand each side independently while a foreground component anchored
|
||||
inside the detector box still reaches that boundary, up to one line height. They
|
||||
extend 8% above and 25% below. The anchored-component gate covers clipped leading
|
||||
flourishes, trailing punctuation, icons, and descenders without walking into
|
||||
disconnected decoration or background texture.
|
||||
|
||||
The stage is deliberately narrower than the engine: it rejects `sdxl-zimage`, tiles,
|
||||
resolution caps, humanize, unsharp, and adaptive polish. Those combinations change
|
||||
|
||||
+37
-2
@@ -609,11 +609,46 @@ engine.remove_watermark(
|
||||
|
||||
Install `remove-ai-watermarks[text-restoration]`. The manifest schema and safety
|
||||
constraints are documented in the CLI guide. The engine verifies its decoded RGB
|
||||
hash before loading the diffusion models and rejects SDXL, tiling, downscaling, and
|
||||
postprocessing combinations that were not evaluated. `InvisibleOptions` exposes the
|
||||
hash before loading the diffusion models and rejects SDXL, downscaling, and
|
||||
postprocessing combinations that were not evaluated. Tiling is allowed: the VAE
|
||||
donor uses the same overlapping tiles as the global pass. `InvisibleOptions` exposes the
|
||||
same field for `remove_all`; after a visible-stage edit, the manifest must be built
|
||||
against the staged pixels rather than the pristine source.
|
||||
|
||||
Use manifest schema 1 for manually reviewed text plus script metadata. Automated
|
||||
operators that verify only text-region geometry should emit schema 2 lines with a
|
||||
`box` and optional `angle`; no placeholder transcription or script is required.
|
||||
|
||||
Since 0.27.1 the mode's global 15% Qwen-VAE fidelity-anchor blend is **off by
|
||||
default** (`fidelity_anchor=False`): that whole-frame blend was measured to
|
||||
return detector-visible OpenAI SynthID on poster-scale manifests (official
|
||||
Content Provenance API, 2026-08-19 - detected x6 with the anchor, clean x6
|
||||
without it, controls and base outputs validated in the same sessions). Pass
|
||||
`fidelity_anchor=True` to reproduce the 0.27.0 research behavior.
|
||||
|
||||
### Drafting manifest lines
|
||||
|
||||
`remove_ai_watermarks.text_draft` proposes lines for a manifest; it never
|
||||
produces verified ones:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.text_draft import draft_text_lines
|
||||
|
||||
draft = draft_text_lines(Path("watermarked.png"))
|
||||
for line in draft.accepted:
|
||||
print(line.box, line.script, line.min_score, line.text)
|
||||
```
|
||||
|
||||
Install `remove-ai-watermarks[text-draft]` (CPU, no torch: PaddleOCR detection
|
||||
plus three script-chosen recognition engines). A line lands in `accepted` only
|
||||
when three crop paddings normalize identically and every confidence clears
|
||||
`min_score` (default 0.85); `accepted` means crop-stable, NOT ground-truth-
|
||||
correct - on the reference posters the draft's exact-text precision was 90.0%
|
||||
and 94.4% because high-confidence OCR still lost punctuation. Every accepted
|
||||
line needs a human yes/no before a manifest may claim `verified: true`.
|
||||
`source_pixel_sha256` is re-exported here for building the manifest's
|
||||
pixel-binding hash against the exact source the engine will decode.
|
||||
|
||||
`remove_watermark` takes strength, seed, tiling, resolution, and postprocessing
|
||||
controls. It takes no model id, step count or guidance scale, and neither does the
|
||||
constructor: each profile pins its model stack, its per-stage schedule and CFG
|
||||
|
||||
@@ -769,3 +769,15 @@ reproducible verification requires a fixed seed.
|
||||
8. Jiang et al. (2025). **VideoMarkBench: Benchmarking Robustness of Video
|
||||
Watermarking.** arXiv:2505.21620.
|
||||
https://arxiv.org/abs/2505.21620
|
||||
|
||||
**Google floor on qwen-zimage (0.27.2, 2026-08-19).** The bracket above did not
|
||||
survive the full production path: on the 4.33 MP CJK-sign fixture (visible-stage
|
||||
sparkle removal -> qwen-zimage seed 0 at the curve's 0.154 top -> resize-back ->
|
||||
metadata strip), the Gemini verifier detected SynthID x3 with a valid
|
||||
pixel-identical stripped control in the same session. Google-provenance content
|
||||
now takes a flat `QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.30` floor instead of the area
|
||||
curve - 0.30 anchors measured clean in Gemini on two fixtures (CJK sign +
|
||||
18-face) at 3/3 checks across two work accounts, and stayed clean with the
|
||||
vae-glyphs donor layer on top; openai/unknown content keeps the resolution
|
||||
curve. An explicit strength still wins. Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-18/` (the failing re-baseline).
|
||||
@@ -343,3 +343,29 @@ The stable errors were punctuation: one lost English comma and an ideographic
|
||||
comma consistently replaced by ASCII. Thus crop agreement and confidence do
|
||||
not verify literal text. The helper is retained only to seed annotations; every
|
||||
proposal needs manual review before it can enter the restoration compositor.
|
||||
|
||||
## Production-path anchor leak and fix (2026-08-19)
|
||||
|
||||
The engine `text_manifest=` path (unlike the research compositor above) blends a
|
||||
15% Qwen-VAE fidelity anchor across the WHOLE frame before glyph restoration.
|
||||
Certified through the official OpenAI Content Provenance API on the exact
|
||||
production path (qwen-zimage, seed 0, no strength override, untiled, LANCZOS
|
||||
resize-back, metadata strip; library 0.27.0 @ 50d9f448):
|
||||
|
||||
- controls (pixel-identical metadata-stripped sources): `synthid: detected` -
|
||||
the pixel channel was proven live before every session;
|
||||
- base outputs: `clean` x6 (two 20-line posters, 3 attempts each);
|
||||
- text-restored WITH the anchor: `synthid: detected` x6, deterministic;
|
||||
- text-restored with `FIDELITY_BLEND_ALPHA = 0` (runtime ablation, base bytes
|
||||
identical between runs): `clean` x6.
|
||||
|
||||
The anchor was therefore the leak channel: a 15% global donor admixture over a
|
||||
scrubbed frame returns enough OpenAI SynthID for the provider detector. Fidelity
|
||||
cost of dropping it (source-referenced text-box MAE): 11.60->11.72, 7.79->7.86,
|
||||
and 7.57->8.13 across the two posters and the CJK sign - well under one point
|
||||
against base MAEs of 24.7/16.8/10.2. Since 0.27.1 the anchor is off by default
|
||||
(`fidelity_anchor` parameter on `remove_watermark` / `InvisibleOptions` /
|
||||
`--fidelity-anchor`); Google rows were not re-queried because the default
|
||||
strength leaves that fixture's base detected (the vendor-strength lane tracks
|
||||
that separately). Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-19-A1/` (sha256-stamped rows).
|
||||
@@ -10,7 +10,7 @@ Who embeds what, and whether it is locally detectable (so we know which gaps are
|
||||
**Root cause and external confirmation (deep-research 2026-06-19, adversarially verified).** This is the SCHEME's ceiling, not our usage — there is no better decoder to adopt. The imwatermark maintainers state verbatim (both the ShieldMnt and Stability-AI READMEs) that the algorithm "cannot guarantee to decode the original watermarks 100% accurately even though we don't apply any attack." Independent measurement (WMAdapter, arXiv:2406.08337 Table 2) puts dwtDct at only **~0.79 bit accuracy on CLEAN images (~38/48 bits — already below our 44 gate)**, collapsing to ~0.50 (chance) under crop/JPEG. Two code-verified + locally-reproduced mechanisms drive the content-dependent failures: (1) the decoder reads each bit as the **highest-magnitude DCT coefficient per block**, so any content coefficient exceeding the encoded target flips the bit; (2) the default embed is in the **YUV chroma channel, which 8-bit-clamps on white/bright pixels** (a +36 chroma delta survives a white-fill round-trip as only +4, ~89% loss) — this is the mechanism behind the bright-flat / minimalist failures and the all-ones degenerate decode. No maintained fork or detector decodes this scheme reliably: the WAVES benchmark (arXiv:2401.08573) relegates DWT-DCT to supplementary appendix G.5 and targets Stable Signature / Tree-Ring / StegaStamp instead; learned encoder/decoder schemes reach ~0.98-0.99 clean but are a DIFFERENT watermark class (not what SDXL/FLUX stamp). `dwtDctSvd` does not help (SDXL embeds `dwtDct`; dwtDctSvd cannot decode it, and its clean accuracy ~0.72 is lower). **Authoritative conclusion: the open DWT-DCT mark cannot be turned from positive-only into a reliable real-world detector; keep it positive-only and rely on C2PA.** (Refuted along the way: that the library is unmaintained, and that it is robust to JPEG but only fails on geometric attacks — both did not survive verification.)
|
||||
|
||||
Consequence for the FLUX hosted-output question (BFL Playground, FLUX.2 [pro] + FLUX.1 [dev], 2026-06-19): all samples carry the signed C2PA manifest (issuer "Black Forest Labs"); the open DWT-DCT decode returned `None`, but every available FLUX carrier (textured fox AND a minimalist-flat generation) failed the positive control (28/48), so the detector is blind on them and **whether BFL hosted output embeds the open pixel watermark is UNRESOLVED** (an earlier note here wrongly asserted it absent — overstated; a later note blamed "high texture" — also wrong, flat carriers fail too). What IS established: C2PA is the reliable FLUX identifier; the `_BITS_48` pattern is correct (round-trips on chatgpt/firefly/random). Resolving the hosted question needs a hosted FLUX carrier that first passes a >=44/48 positive control, which neither a textured nor a flat prompt produced — low priority (the open mark is only a stripped-metadata fallback).
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Copilot + Designer; Bing Image Creator collected 2026-05-24 still signs as "Microsoft" and now runs **MAI-Image**, NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
|
||||
**Samsung Galaxy AI** signs supported edits with C2PA and may carry the
|
||||
proprietary `genAIType` marker. The registered visible detector covers the
|
||||
@@ -28,7 +28,7 @@ take precedence over issuer attribution. Supported mappings include Higgsfield
|
||||
AI, Topaz Labs Image API, and TikTok Ad Creative Toolbox; an unknown claim
|
||||
generator still falls back to the certificate issuer.
|
||||
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
|
||||
- **EXIF/XMP/PNG-text generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal). Additional verified generator stamps include **NovelAI** (`Software`, `Source`, and `Title` PNG text chunks), **Reve** (`Software` or XMP `CreatorTool` = `reve.com`), and **Aphrodite AI** (`Make` or `Software` = `Aphrodite AI`).
|
||||
- **App-export provenance and AIGC JSON:** supported ByteDance-family exports can place a JSON object in EXIF `ImageDescription` or `UserComment`, independently of C2PA or TC260. Exact `product` values for Doubao, Xinghui, and Dreamina are removable product provenance, but do not alone prove that the pixels were generated. Dreamina additionally requires `exportType=generation` for that verdict. A nested Aweme `aigc_type=1` or private ByteDance `aigc_label_type=1` / `2` is an AIGC disclosure; `0` is inconclusive and can occur on a Dreamina generation export. Plain Aweme, retouch, and `lv` exports are preserved. The lower-case private field is deliberately not interpreted as the normative TC260 `Label`, whose values `1` / `2` / `3` mean generated / possibly generated / suspected generated under [GB 45438-2025](https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf).
|
||||
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
|
||||
|
||||
+9
-1
@@ -4,7 +4,7 @@
|
||||
# on it, including the ComfyUI node package. The console script below carries
|
||||
# the same weight, since users have it on PATH.
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.27.0"
|
||||
version = "0.29.0"
|
||||
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11,<3.15"
|
||||
@@ -125,6 +125,14 @@ qwen-zimage = [
|
||||
text-restoration = [
|
||||
"remove-ai-watermarks[qwen-zimage,lama]",
|
||||
]
|
||||
# Draft-only OCR proposals for verified-text manifests. CPU, no torch:
|
||||
# PaddleOCR detection plus three script-chosen recognition engines. The models
|
||||
# download on first use and are cached by huggingface_hub; never bundled.
|
||||
text-draft = [
|
||||
"paddleocr>=3.3.3",
|
||||
"paddlepaddle>=3.0.0",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Adobe TrustMark decoder -- the open, keyless watermark behind Adobe Durable
|
||||
# Content Credentials (soft-binding alg ``com.adobe.trustmark.P``). Optional
|
||||
# because it pulls torch and downloads model weights on first use. identify()
|
||||
|
||||
+11
-40
@@ -21,14 +21,11 @@ changes under crop jitter or whose minimum confidence is below the threshold.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import unicodedata
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import click
|
||||
import numpy as np
|
||||
@@ -37,38 +34,15 @@ from PIL import Image
|
||||
log = logging.getLogger(__name__)
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
RESTORATION_SCRIPT = ROOT / "scripts/selective_text_restoration.py"
|
||||
|
||||
from scripts._text_eval import normalize_text # noqa: E402
|
||||
|
||||
|
||||
def _load_restoration_module() -> Any:
|
||||
spec = importlib.util.spec_from_file_location("selective_text_restoration_for_inference", RESTORATION_SCRIPT)
|
||||
if spec is None or spec.loader is None:
|
||||
raise RuntimeError(f"unable to load {RESTORATION_SCRIPT}")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def _has_script(text: str, script: str) -> bool:
|
||||
return any(script in unicodedata.name(character, "") for character in text)
|
||||
|
||||
|
||||
def choose_language(probes: dict[str, tuple[str, float]]) -> str:
|
||||
if _has_script(probes["ch"][0], "CJK"):
|
||||
return "ch"
|
||||
if _has_script(probes["ru"][0], "CYRILLIC"):
|
||||
return "ru"
|
||||
return "en"
|
||||
|
||||
|
||||
def stable_recognition(reads: list[tuple[str, float]], min_score: float = 0.85) -> str | None:
|
||||
normalized = {normalize_text(text) for text, _score in reads}
|
||||
if len(normalized) != 1 or min(score for _text, score in reads) < min_score:
|
||||
return None
|
||||
return reads[0][0]
|
||||
# Dogfoods the packaged draft API (remove_ai_watermarks.text_draft); this script
|
||||
# keeps only the CLI wrapper so the package stays the one home for the logic.
|
||||
from remove_ai_watermarks.text_draft import ( # noqa: E402
|
||||
_detect_line_boxes,
|
||||
_recognize,
|
||||
choose_language,
|
||||
stable_recognition,
|
||||
)
|
||||
|
||||
|
||||
@click.command()
|
||||
@@ -81,7 +55,6 @@ def main(source: Path, out: Path, min_score: float) -> None:
|
||||
os.environ["PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK"] = "True"
|
||||
from paddleocr import PaddleOCR, TextRecognition
|
||||
|
||||
restoration = _load_restoration_module()
|
||||
source_rgb = np.asarray(Image.open(source).convert("RGB"))
|
||||
detector = PaddleOCR(
|
||||
lang="ch",
|
||||
@@ -94,19 +67,17 @@ def main(source: Path, out: Path, min_score: float) -> None:
|
||||
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec"),
|
||||
"ch": TextRecognition(model_name="PP-OCRv5_server_rec"),
|
||||
}
|
||||
boxes = restoration.detect_line_boxes(detector, source_rgb)
|
||||
boxes = _detect_line_boxes(detector, source_rgb)
|
||||
accepted = []
|
||||
rejected = []
|
||||
for box in boxes:
|
||||
probes = {}
|
||||
for language, engine in engines.items():
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
line = restoration.TextLine(box, "", script)
|
||||
probes[language] = restoration._recognize(engine, source_rgb, line, 0.1)
|
||||
probes[language] = _recognize(engine, source_rgb, box, script, 0.1)
|
||||
language = choose_language(probes)
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
line = restoration.TextLine(box, "", script)
|
||||
reads = [restoration._recognize(engines[language], source_rgb, line, ratio) for ratio in (0.08, 0.12, 0.2)]
|
||||
reads = [_recognize(engines[language], source_rgb, box, script, ratio) for ratio in (0.08, 0.12, 0.2)]
|
||||
text = stable_recognition(reads, min_score)
|
||||
result = {
|
||||
"box": box,
|
||||
|
||||
@@ -32,7 +32,7 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
|
||||
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
|
||||
|
||||
|
||||
__version__ = "0.27.0"
|
||||
__version__ = "0.29.0"
|
||||
|
||||
__all__ = [
|
||||
"BatchSummary",
|
||||
|
||||
@@ -65,7 +65,7 @@ def _vendor(
|
||||
|
||||
# Order is product priority when a manifest mentions more than one organization.
|
||||
C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
|
||||
_vendor(b"Microsoft", "Microsoft", "Microsoft (Bing Image Creator / Designer)", "Microsoft"),
|
||||
_vendor(b"Microsoft", "Microsoft", "Microsoft (Copilot / Designer)", "Microsoft"),
|
||||
_vendor(b"Adobe", "Adobe", "Adobe Firefly", "Adobe"),
|
||||
_vendor(
|
||||
b"OpenAI",
|
||||
@@ -78,18 +78,25 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
|
||||
_vendor(b"Google", "Google LLC", "Google (Gemini / Imagen)", "Google", synthid=True),
|
||||
_vendor(b"Stability AI", "Stability AI", "Stability AI (Stable Image / DreamStudio)", "Stability AI"),
|
||||
_vendor(b"Black Forest Labs", "Black Forest Labs", "Black Forest Labs (FLUX)", "Black Forest Labs"),
|
||||
_vendor(b"volcengine", "ByteDance (Volcano Engine)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
|
||||
_vendor(
|
||||
b"volcengine",
|
||||
"ByteDance (Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
|
||||
"ByteDance",
|
||||
),
|
||||
_vendor(
|
||||
"北京火山引擎科技有限公司",
|
||||
"ByteDance (Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
|
||||
"ByteDance",
|
||||
),
|
||||
_vendor(b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
|
||||
_vendor(
|
||||
b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)", "ByteDance"
|
||||
),
|
||||
_vendor(
|
||||
b"Dreamina",
|
||||
"ByteDance (Dreamina)",
|
||||
"ByteDance (Doubao / Jimeng / Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)",
|
||||
"ByteDance",
|
||||
asserts_ai=True,
|
||||
),
|
||||
@@ -109,7 +116,7 @@ C2PA_IDENTITY_AI_ORGS = frozenset(vendor.org for vendor in C2PA_AI_VENDORS if ve
|
||||
C2PA_CLAIM_GENERATOR_PLATFORMS: tuple[tuple[str, str], ...] = (
|
||||
("adobe_firefly", "Adobe Firefly"),
|
||||
("firefly", "Adobe Firefly"),
|
||||
("dreamina", "ByteDance (Doubao / Jimeng / Volcano Engine)"),
|
||||
("dreamina", "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"),
|
||||
("higgsfield ai", "Higgsfield AI"),
|
||||
("topaz labs image api", "Topaz Labs"),
|
||||
("tiktok ad creative toolbox", "TikTok Ad Creative Toolbox"),
|
||||
|
||||
@@ -1071,6 +1071,7 @@ class QwenZImagePipeline:
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
text_manifest: VerifiedTextManifest | None = None,
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Image.Image:
|
||||
"""Execute global regeneration and masked face repair."""
|
||||
self._require_cuda()
|
||||
@@ -1078,7 +1079,24 @@ class QwenZImagePipeline:
|
||||
donor = None
|
||||
if text_manifest is not None:
|
||||
self._progress("Reconstructing the verified text donor with the Qwen VAE...")
|
||||
donor = self._qwen_vae_roundtrip(image)
|
||||
if tile and max(image.size) > tile_size:
|
||||
from remove_ai_watermarks._internal.tiling import run_tiled
|
||||
|
||||
donor = run_tiled(
|
||||
self._qwen_vae_roundtrip,
|
||||
image,
|
||||
tile_size,
|
||||
tile_overlap,
|
||||
lambda message: self._progress(
|
||||
message.replace(
|
||||
"Tiled diffusion",
|
||||
"Reconstructing the verified text donor",
|
||||
1,
|
||||
)
|
||||
),
|
||||
)
|
||||
else:
|
||||
donor = self._qwen_vae_roundtrip(image)
|
||||
global_strength = (
|
||||
resolution_adaptive_denoise(image.width, image.height) if strength is None else float(strength)
|
||||
)
|
||||
@@ -1120,7 +1138,16 @@ class QwenZImagePipeline:
|
||||
restore_verified_text,
|
||||
)
|
||||
|
||||
self._progress("Blending the Qwen-VAE fidelity anchor...")
|
||||
anchor = blend_fidelity_anchor(result, donor)
|
||||
# The anchor is OFF by default since 0.27.1: blending 15% of the Qwen-VAE
|
||||
# donor ACROSS THE WHOLE FRAME returned detector-visible OpenAI SynthID on
|
||||
# poster-scale manifests (official Content Provenance API, 2026-08-19:
|
||||
# detected x6 with the anchor, clean x6 without it, base clean throughout;
|
||||
# see docs/text-protection-research.md). ``fidelity_anchor=True`` keeps the
|
||||
# 0.27.0 research behavior for reproduction.
|
||||
if fidelity_anchor:
|
||||
self._progress("Blending the Qwen-VAE fidelity anchor...")
|
||||
anchor = blend_fidelity_anchor(result, donor)
|
||||
else:
|
||||
anchor = result
|
||||
self._progress(f"Restoring {len(text_manifest.lines)} verified text lines...")
|
||||
return restore_verified_text(image, anchor, donor, text_manifest.lines)
|
||||
@@ -13,30 +13,35 @@ import cv2
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks._internal.schema import require_schema_version
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
TEXT_MANIFEST_SCHEMA = 1
|
||||
TEXT_MANIFEST_SCHEMA = 2
|
||||
_SUPPORTED_TEXT_MANIFEST_SCHEMAS = frozenset({1, TEXT_MANIFEST_SCHEMA})
|
||||
FIDELITY_BLEND_ALPHA = 0.15
|
||||
GLYPH_FEATHER = 0.5
|
||||
GLYPH_SIDE_PAD_RATIO = 0.12
|
||||
GLYPH_SIDE_PAD_LIMIT = 1.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VerifiedTextLine:
|
||||
"""One operator-verified source line in source-pixel coordinates."""
|
||||
"""One operator-verified source region in source-pixel coordinates."""
|
||||
|
||||
box: tuple[int, int, int, int]
|
||||
text: str
|
||||
script: str
|
||||
text: str | None = None
|
||||
script: str | None = None
|
||||
angle: float = 0.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VerifiedTextManifest:
|
||||
"""Text annotations cryptographically bound to one decoded RGB source."""
|
||||
"""Verified text regions cryptographically bound to one decoded RGB source."""
|
||||
|
||||
source_pixel_sha256: str
|
||||
width: int
|
||||
@@ -55,17 +60,20 @@ def source_pixel_sha256(image: Image.Image) -> str:
|
||||
|
||||
|
||||
def load_verified_text_manifest(path: Path, source: Image.Image) -> VerifiedTextManifest:
|
||||
"""Load and validate a manually verified manifest for exactly ``source``."""
|
||||
"""Load and validate an operator-verified manifest for exactly ``source``."""
|
||||
try:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError) as exc:
|
||||
raise ValueError(f"Cannot read text manifest {path}: {exc}") from exc
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("Text manifest must be a JSON object")
|
||||
if payload.get("schema_version") != TEXT_MANIFEST_SCHEMA:
|
||||
raise ValueError(f"Text manifest schema_version must be {TEXT_MANIFEST_SCHEMA}")
|
||||
schema_version = require_schema_version(
|
||||
payload.get("schema_version"),
|
||||
contract="text manifest",
|
||||
supported=_SUPPORTED_TEXT_MANIFEST_SCHEMAS,
|
||||
)
|
||||
if payload.get("verified") is not True:
|
||||
raise ValueError("Text manifest must contain verified=true after manual review")
|
||||
raise ValueError("Text manifest must contain verified=true after operator verification")
|
||||
|
||||
rgb = source.convert("RGB")
|
||||
width = _manifest_integer(payload, "width")
|
||||
@@ -82,7 +90,7 @@ def load_verified_text_manifest(path: Path, source: Image.Image) -> VerifiedText
|
||||
raw_lines = payload.get("lines")
|
||||
if not isinstance(raw_lines, list) or not raw_lines:
|
||||
raise ValueError("Text manifest lines must be a non-empty list")
|
||||
lines = tuple(_load_line(item, width, height, index) for index, item in enumerate(raw_lines))
|
||||
lines = tuple(_load_line(item, width, height, index, schema_version) for index, item in enumerate(raw_lines))
|
||||
if list(lines) != sorted(lines, key=lambda line: (line.box[1], line.box[0])):
|
||||
raise ValueError("Text manifest lines must be in top-to-bottom, left-to-right reading order")
|
||||
return VerifiedTextManifest(actual_hash, width, height, lines)
|
||||
@@ -95,7 +103,7 @@ def _manifest_integer(payload: dict[str, Any], key: str) -> int:
|
||||
return value
|
||||
|
||||
|
||||
def _load_line(item: Any, width: int, height: int, index: int) -> VerifiedTextLine:
|
||||
def _load_line(item: Any, width: int, height: int, index: int, schema_version: int) -> VerifiedTextLine:
|
||||
if not isinstance(item, dict):
|
||||
raise ValueError(f"Text manifest line {index} must be an object")
|
||||
raw_box = item.get("box")
|
||||
@@ -109,12 +117,15 @@ def _load_line(item: Any, width: int, height: int, index: int) -> VerifiedTextLi
|
||||
x1, y1, x2, y2 = box
|
||||
if not (0 <= x1 < x2 <= width and 0 <= y1 < y2 <= height):
|
||||
raise ValueError(f"Text manifest line {index} box is outside the source dimensions")
|
||||
text = item.get("text")
|
||||
script = item.get("script")
|
||||
if not isinstance(text, str) or not text.strip():
|
||||
raise ValueError(f"Text manifest line {index} text must be non-empty")
|
||||
if not isinstance(script, str) or not script.strip():
|
||||
raise ValueError(f"Text manifest line {index} script must be non-empty")
|
||||
text: str | None = None
|
||||
script: str | None = None
|
||||
if schema_version == 1:
|
||||
text = item.get("text")
|
||||
script = item.get("script")
|
||||
if not isinstance(text, str) or not text.strip():
|
||||
raise ValueError(f"Text manifest line {index} text must be non-empty")
|
||||
if not isinstance(script, str) or not script.strip():
|
||||
raise ValueError(f"Text manifest line {index} script must be non-empty")
|
||||
angle_value = item.get("angle", 0.0)
|
||||
if isinstance(angle_value, bool) or not isinstance(angle_value, int | float):
|
||||
raise ValueError(f"Text manifest line {index} angle must be numeric")
|
||||
@@ -185,14 +196,41 @@ def restore_verified_text(
|
||||
return Image.fromarray(restored)
|
||||
|
||||
|
||||
def source_silhouette_mask(
|
||||
source_rgb: NDArray[Any],
|
||||
def _glyph_crop_box(
|
||||
box: tuple[int, int, int, int],
|
||||
angle: float = 0.0,
|
||||
width: int,
|
||||
height: int,
|
||||
left_pad: int,
|
||||
right_pad: int,
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Widen the detector box so glyph edges stay inside the silhouette crop.
|
||||
|
||||
Detector boxes can stop inside a leading flourish or trailing punctuation,
|
||||
and Paddle line boxes sit 2-5 px above the true ink bottom on the poster
|
||||
fixtures. The crop is otherwise the box itself, so those pixels never reach
|
||||
the donor composite. Start 12% sideways, expand each side independently
|
||||
while foreground reaches its boundary, and pad 8% up and 25% down.
|
||||
"""
|
||||
x1, y1, x2, y2 = box
|
||||
line_h = max(1, y2 - y1)
|
||||
pad_top = max(1, round(line_h * 0.08))
|
||||
pad_bot = max(2, round(line_h * 0.25))
|
||||
return (
|
||||
max(0, x1 - left_pad),
|
||||
max(0, y1 - pad_top),
|
||||
min(width, x2 + right_pad),
|
||||
min(height, y2 + pad_bot),
|
||||
)
|
||||
|
||||
|
||||
def _silhouette_crop_mask(
|
||||
source_rgb: NDArray[Any],
|
||||
crop_box: tuple[int, int, int, int],
|
||||
angle: float,
|
||||
) -> NDArray[Any]:
|
||||
"""Recover a thresholded glyph shape without retaining source amplitudes."""
|
||||
"""Threshold one candidate crop so its horizontal boundaries can be tested."""
|
||||
height, width = source_rgb.shape[:2]
|
||||
x1, y1, x2, y2 = _clip_box(box, width, height)
|
||||
x1, y1, x2, y2 = crop_box
|
||||
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
|
||||
support = np.ones(gray.shape, dtype=np.uint8)
|
||||
if angle:
|
||||
@@ -211,7 +249,7 @@ def source_silhouette_mask(
|
||||
background_luma = float(np.median(values))
|
||||
else:
|
||||
ring_pad = max(6, min(20, (y2 - y1) // 4))
|
||||
rx1, ry1, rx2, ry2 = _clip_box((x1, y1, x2, y2), width, height, pad=ring_pad)
|
||||
rx1, ry1, rx2, ry2 = _clip_box(crop_box, width, height, pad=ring_pad)
|
||||
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
|
||||
ring = np.ones(context.shape, dtype=bool)
|
||||
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
|
||||
@@ -225,6 +263,61 @@ def source_silhouette_mask(
|
||||
else:
|
||||
crop_mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
|
||||
crop_mask[support == 0] = 0
|
||||
return crop_mask
|
||||
|
||||
|
||||
def _anchored_components_reach_sides(
|
||||
crop_mask: NDArray[Any],
|
||||
anchor_box: tuple[int, int, int, int],
|
||||
) -> tuple[bool, bool]:
|
||||
"""Whether anchored foreground reaches the left and right crop sides."""
|
||||
count, labels, stats, _centroids = cv2.connectedComponentsWithStats(crop_mask, 8)
|
||||
x1, y1, x2, y2 = anchor_box
|
||||
anchor_counts = np.bincount(labels[y1:y2, x1:x2].reshape(-1), minlength=count)
|
||||
anchor_counts[0] = 0
|
||||
|
||||
def reaches(edge_labels: NDArray[Any]) -> bool:
|
||||
return any(
|
||||
label != 0 and anchor_counts[label] >= max(3, round(stats[label, cv2.CC_STAT_AREA] * 0.1))
|
||||
for label in np.unique(edge_labels)
|
||||
)
|
||||
|
||||
return reaches(labels[y1:y2, :2]), reaches(labels[y1:y2, -2:])
|
||||
|
||||
|
||||
def source_silhouette_mask(
|
||||
source_rgb: NDArray[Any],
|
||||
box: tuple[int, int, int, int],
|
||||
angle: float = 0.0,
|
||||
) -> NDArray[Any]:
|
||||
"""Recover a thresholded glyph shape without retaining source amplitudes."""
|
||||
height, width = source_rgb.shape[:2]
|
||||
box_x1, box_y1, box_x2, box_y2 = box
|
||||
line_h = max(1, box_y2 - box_y1)
|
||||
step = max(2, round(line_h * GLYPH_SIDE_PAD_RATIO))
|
||||
limit = max(step, round(line_h * GLYPH_SIDE_PAD_LIMIT))
|
||||
left_pad = right_pad = step
|
||||
while True:
|
||||
x1, y1, x2, y2 = _glyph_crop_box(box, width, height, left_pad, right_pad)
|
||||
crop_mask = _silhouette_crop_mask(source_rgb, (x1, y1, x2, y2), angle)
|
||||
core_y1 = max(0, box_y1 - y1)
|
||||
core_y2 = min(y2 - y1, box_y2 - y1)
|
||||
anchor_box = (max(0, box_x1 - x1), core_y1, min(x2 - x1, box_x2 - x1), core_y2)
|
||||
can_expand_left = x1 > 0 and left_pad < limit
|
||||
can_expand_right = x2 < width and right_pad < limit
|
||||
left_anchored, right_anchored = (
|
||||
_anchored_components_reach_sides(crop_mask, anchor_box)
|
||||
if can_expand_left or can_expand_right
|
||||
else (False, False)
|
||||
)
|
||||
left_touches = can_expand_left and left_anchored
|
||||
right_touches = can_expand_right and right_anchored
|
||||
if not left_touches and not right_touches:
|
||||
break
|
||||
if left_touches:
|
||||
left_pad = min(limit, left_pad + step)
|
||||
if right_touches:
|
||||
right_pad = min(limit, right_pad + step)
|
||||
result = np.zeros((height, width), dtype=np.uint8)
|
||||
result[y1:y2, x1:x2] = crop_mask
|
||||
return result
|
||||
|
||||
@@ -69,6 +69,14 @@ SDXL_ZIMAGE_OPENAI_STRENGTH = 0.15
|
||||
SDXL_ZIMAGE_GEMINI_STRENGTH = 0.25
|
||||
SDXL_ZIMAGE_UNKNOWN_STRENGTH = SDXL_ZIMAGE_GEMINI_STRENGTH
|
||||
|
||||
# qwen-zimage keeps its resolution curve for openai/unknown content, but Google
|
||||
# content takes this flat oracle-measured floor instead of the curve (whose top,
|
||||
# 0.154, left a 4.33 MP CJK-sign fixture detected x3 on the full production path
|
||||
# with a valid pixel control, 2026-08-18). 0.30 anchors measured clean in Gemini
|
||||
# across two fixtures and two work accounts (CJK sign + 18-face, 3/3 checks each,
|
||||
# 2026-08 research) and stayed clean under the vae-glyphs donor layer on top.
|
||||
QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.30
|
||||
|
||||
|
||||
# sdxl-zimage picks its strength from the VENDOR (unlike qwen-zimage, which derives it
|
||||
# from image area). An unlisted or unknown vendor falls back to the Gemini value.
|
||||
@@ -103,7 +111,8 @@ def resolve_adaptive_polish(adaptive_polish: bool | None, pipeline: str) -> bool
|
||||
def strength_default_help() -> str:
|
||||
"""Describe the live default policy without duplicating its values."""
|
||||
return (
|
||||
"profile-adaptive (qwen-zimage uses resolution-adaptive denoise; sdxl-zimage "
|
||||
"profile-adaptive (qwen-zimage uses resolution-adaptive denoise, with a "
|
||||
f"flat oracle-measured Google floor of {QWEN_ZIMAGE_GOOGLE_STRENGTH}; sdxl-zimage "
|
||||
f"uses OpenAI {SDXL_ZIMAGE_OPENAI_STRENGTH} / Google {SDXL_ZIMAGE_GEMINI_STRENGTH} / "
|
||||
f"unknown {SDXL_ZIMAGE_UNKNOWN_STRENGTH}, from the C2PA issuer)"
|
||||
)
|
||||
@@ -118,10 +127,14 @@ def resolve_strength(
|
||||
) -> float:
|
||||
"""Resolve a user override or the calibrated policy for a profile and vendor.
|
||||
|
||||
Total by design. qwen-zimage picks its strength from image area rather than from
|
||||
the vendor, so it needs ``size``; returning ``None`` for it instead would push that
|
||||
Total by design. qwen-zimage picks its strength from image area rather than
|
||||
from the vendor, so it needs ``size``; returning ``None`` for it instead would push that
|
||||
branch onto every caller and move one of the two strength policies outside this
|
||||
module. ``size`` is required for qwen-zimage without an explicit strength.
|
||||
One measured exception since 0.27.2: Google-provenance content on qwen-zimage
|
||||
takes the flat ``QWEN_ZIMAGE_GOOGLE_STRENGTH`` floor instead of the area curve
|
||||
(the curve's 0.154 top left a 4.33 MP fixture oracle-detected on the full
|
||||
production path; see the constant's comment).
|
||||
"""
|
||||
if strength is not None:
|
||||
return strength
|
||||
@@ -129,6 +142,8 @@ def resolve_strength(
|
||||
return _SDXL_ZIMAGE_STRENGTH_BY_VENDOR.get((vendor or "").casefold(), SDXL_ZIMAGE_UNKNOWN_STRENGTH)
|
||||
if size is None:
|
||||
raise ValueError("qwen-zimage resolves strength from image area, so size is required")
|
||||
if (vendor or "").casefold() == "google":
|
||||
return QWEN_ZIMAGE_GOOGLE_STRENGTH
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import resolution_adaptive_denoise
|
||||
|
||||
return resolution_adaptive_denoise(*size)
|
||||
|
||||
@@ -190,6 +190,7 @@ class WatermarkRemover:
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
text_manifest: VerifiedTextManifest | None = None,
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Path:
|
||||
"""Regenerate image pixels and write the result without AI metadata.
|
||||
|
||||
@@ -219,6 +220,7 @@ class WatermarkRemover:
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
text_manifest=text_manifest,
|
||||
fidelity_anchor=fidelity_anchor,
|
||||
)
|
||||
self._write_output(result, destination)
|
||||
return destination
|
||||
@@ -244,6 +244,7 @@ class InvisibleOptions:
|
||||
tile_size: int = 1024
|
||||
tile_overlap: int = 128
|
||||
text_manifest: Path | None = None
|
||||
fidelity_anchor: bool = False
|
||||
|
||||
|
||||
# What the invisible stage did. "unavailable" is the one outcome the caller must
|
||||
@@ -525,6 +526,7 @@ def _run_invisible(
|
||||
tile_size=opts.tile_size,
|
||||
tile_overlap=opts.tile_overlap,
|
||||
text_manifest=opts.text_manifest,
|
||||
fidelity_anchor=opts.fidelity_anchor,
|
||||
)
|
||||
say("invisible", "removed")
|
||||
return "removed"
|
||||
|
||||
@@ -321,6 +321,16 @@ _text_manifest_option = click.option(
|
||||
),
|
||||
)
|
||||
|
||||
_fidelity_anchor_option = click.option(
|
||||
"--fidelity-anchor/--no-fidelity-anchor",
|
||||
default=False,
|
||||
help=(
|
||||
"With --text-manifest: blend 15% of the Qwen-VAE donor across the whole "
|
||||
"frame. Off by default - the global blend was measured to return "
|
||||
"detector-visible OpenAI SynthID on poster-scale manifests."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
_visible_backend_option = click.option(
|
||||
"--backend",
|
||||
@@ -798,6 +808,7 @@ def cmd_erase(
|
||||
@_force_option
|
||||
@_cpu_offload_option
|
||||
@_text_manifest_option
|
||||
@_fidelity_anchor_option
|
||||
@click.pass_context
|
||||
def cmd_invisible(
|
||||
ctx: click.Context,
|
||||
@@ -818,6 +829,7 @@ def cmd_invisible(
|
||||
force: bool,
|
||||
cpu_offload: bool,
|
||||
text_manifest: Path | None,
|
||||
fidelity_anchor: bool,
|
||||
) -> None:
|
||||
"""Remove invisible AI watermarks (SynthID, StableSignature, TreeRing).
|
||||
|
||||
@@ -880,6 +892,7 @@ def cmd_invisible(
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
text_manifest=text_manifest,
|
||||
fidelity_anchor=fidelity_anchor,
|
||||
)
|
||||
except (OSError, RuntimeError, ValueError) as exc:
|
||||
console.print(f" Error: {exc}")
|
||||
@@ -891,10 +904,27 @@ def cmd_invisible(
|
||||
|
||||
|
||||
# ── Metadata operations ──
|
||||
def _print_metadata_not_a_clean_verdict() -> None:
|
||||
"""Repeat the identify empty-scan limit on metadata check and strip success.
|
||||
|
||||
``metadata --check`` and ``metadata --remove`` answer a narrower question than
|
||||
``identify``: they report embedded AI metadata only. A quiet result used to
|
||||
stop at "No AI metadata found" / "AI metadata stripped", which readers treat
|
||||
as a clean-image verdict. The pixel channel is unchanged, and this project has
|
||||
no local SynthID decoder, so the command must say so in the same words
|
||||
``identify`` already uses.
|
||||
"""
|
||||
console.print(
|
||||
" This is not the same as 'clean': a pixel watermark such as SynthID cannot be\n"
|
||||
" detected here once its metadata proxy is absent."
|
||||
)
|
||||
|
||||
|
||||
def _print_metadata_report(source: Path, has_ai: bool, metadata: dict[str, str]) -> None:
|
||||
"""Render one metadata inspection result for the generic and video commands."""
|
||||
if not has_ai:
|
||||
console.print(f" No AI metadata found in {source.name}")
|
||||
_print_metadata_not_a_clean_verdict()
|
||||
return
|
||||
|
||||
console.print(f" Warning: AI metadata detected in {source.name}:")
|
||||
@@ -961,6 +991,7 @@ def cmd_metadata(
|
||||
console.print(" the file could not be decoded, so it was copied through unchanged")
|
||||
raise SystemExit(1)
|
||||
console.print(f" AI metadata stripped -> {out}")
|
||||
_print_metadata_not_a_clean_verdict()
|
||||
|
||||
|
||||
# ── Video pipeline ──
|
||||
@@ -1102,6 +1133,7 @@ def cmd_video_metadata(
|
||||
console.print(f" still present: {', '.join(sorted(result.remaining))}")
|
||||
raise SystemExit(1)
|
||||
console.print(f" AI metadata stripped -> {result.output}")
|
||||
_print_metadata_not_a_clean_verdict()
|
||||
|
||||
|
||||
@cmd_video.command("invisible")
|
||||
@@ -1428,6 +1460,7 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
|
||||
@_force_option
|
||||
@_cpu_offload_option
|
||||
@_text_manifest_option
|
||||
@_fidelity_anchor_option
|
||||
@click.pass_context
|
||||
def cmd_all(
|
||||
ctx: click.Context,
|
||||
@@ -1450,6 +1483,7 @@ def cmd_all(
|
||||
force: bool,
|
||||
cpu_offload: bool,
|
||||
text_manifest: Path | None,
|
||||
fidelity_anchor: bool,
|
||||
) -> None:
|
||||
"""Remove ALL watermarks: visible + invisible + metadata.
|
||||
|
||||
@@ -1528,6 +1562,7 @@ def cmd_all(
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
text_manifest=text_manifest,
|
||||
fidelity_anchor=fidelity_anchor,
|
||||
),
|
||||
force=force,
|
||||
progress=progress,
|
||||
|
||||
@@ -770,6 +770,7 @@ _AI_VENDOR_TOKENS: tuple[tuple[str, str], ...] = (
|
||||
("google", "Google"),
|
||||
("firefly", "Adobe"),
|
||||
("adobe", "Adobe"),
|
||||
("copilot", "Microsoft"),
|
||||
("bing", "Microsoft"),
|
||||
("designer", "Microsoft"),
|
||||
("microsoft", "Microsoft"),
|
||||
|
||||
@@ -150,6 +150,7 @@ class InvisibleEngine:
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
text_manifest: Path | None = None,
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Path:
|
||||
"""Remove invisible watermark from an image.
|
||||
|
||||
@@ -185,8 +186,17 @@ class InvisibleEngine:
|
||||
text_manifest: Operator-verified text lines bound to the decoded source
|
||||
pixels. Enables the experimental Qwen-VAE ``vae-glyphs`` post-pass.
|
||||
Requires the ``text-restoration`` extra and the ``qwen-zimage``
|
||||
profile. Incompatible with tiling, downscaling, humanize, unsharp,
|
||||
and adaptive polish because those combinations are not calibrated.
|
||||
profile. Incompatible with downscaling, humanize, unsharp, and
|
||||
adaptive polish. Tiling is supported: the VAE donor uses the same
|
||||
overlapping tiles as the global pass, then glyph restore runs on
|
||||
the blended full frame.
|
||||
fidelity_anchor: Blend 15% of the Qwen-VAE donor across the whole frame
|
||||
before glyph restoration. OFF by default since 0.27.1: that global
|
||||
blend was measured to return detector-visible OpenAI SynthID on
|
||||
poster-scale manifests (detected x6 with the anchor vs clean x6
|
||||
without it, base clean; official Content Provenance API,
|
||||
2026-08-19 - docs/text-protection-research.md). ``True`` reproduces
|
||||
the 0.27.0 research behavior. Requires ``text_manifest``.
|
||||
|
||||
Returns:
|
||||
Path to the cleaned image.
|
||||
@@ -196,13 +206,13 @@ class InvisibleEngine:
|
||||
seed = resolve_seed(seed)
|
||||
adaptive_polish = resolve_adaptive_polish(adaptive_polish, self._remover.model_profile)
|
||||
|
||||
if fidelity_anchor and text_manifest is None:
|
||||
raise ValueError("fidelity_anchor requires a text manifest")
|
||||
if text_manifest is not None:
|
||||
if self._remover.model_profile != QWEN_ZIMAGE_PROFILE:
|
||||
raise ValueError("--text-manifest is supported only by the qwen-zimage profile")
|
||||
if max_resolution != 0:
|
||||
raise ValueError("--text-manifest requires --max-resolution 0")
|
||||
if tile:
|
||||
raise ValueError("--text-manifest is not calibrated with --tile")
|
||||
if humanize > 0.0 or unsharp > 0.0 or adaptive_polish:
|
||||
raise ValueError("--text-manifest requires humanize=0, unsharp=0, and adaptive polish disabled")
|
||||
from remove_ai_watermarks import region_eraser
|
||||
@@ -270,6 +280,7 @@ class InvisibleEngine:
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
text_manifest=verified_text,
|
||||
fidelity_anchor=fidelity_anchor,
|
||||
)
|
||||
|
||||
# Post-processing chain: decode the diffusion output ONCE, apply the
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Qwen (Tongyi Qianwen, Alibaba) visible watermark detector/localizer.
|
||||
"""Qwen (Alibaba) visible watermark detector/localizer.
|
||||
|
||||
Qwen stamps its generations with a visible "千问AI生成" text strip in the
|
||||
bottom-right corner -- the explicit AIGC label mandated by China's GB 45438-2025
|
||||
|
||||
@@ -0,0 +1,299 @@
|
||||
"""Draft operator-verifiable text lines for verified-text manifests.
|
||||
|
||||
Proposal-only OCR: PaddleOCR detection plus three script-chosen recognition
|
||||
engines (Latin / Cyrillic / CJK), accepted only when three crop paddings
|
||||
normalize identically and every confidence clears the floor. ``accepted``
|
||||
means crop-stable, NEVER ground-truth-correct: on the reference posters the
|
||||
draft's exact-text precision was 90.0% and 94.4% because high-confidence OCR
|
||||
still lost punctuation (one English comma dropped, one ideographic comma
|
||||
replaced with ASCII). Every accepted line needs a human yes/no before it may
|
||||
enter a manifest with ``verified: true``.
|
||||
|
||||
Heavy imports (paddle, and the numpy/cv2/PIL pixel stack) stay inside the
|
||||
call so importing this module costs nothing without the ``text-draft`` extra;
|
||||
``draft_available()`` reports whether the extra is installed.
|
||||
"""
|
||||
|
||||
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unicodedata
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks._internal.text_restoration import source_pixel_sha256
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
__all__ = [
|
||||
"DraftLine",
|
||||
"RejectedLine",
|
||||
"TextDraft",
|
||||
"choose_language",
|
||||
"draft_available",
|
||||
"draft_text_lines",
|
||||
"group_word_boxes",
|
||||
"source_pixel_sha256",
|
||||
"stable_recognition",
|
||||
]
|
||||
|
||||
# Crop paddings probed per line: recognition must be invariant across them.
|
||||
JITTER_RATIOS: tuple[float, ...] = (0.08, 0.12, 0.2)
|
||||
DETECT_SCORE_FLOOR = 0.5
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DraftLine:
|
||||
"""One crop-stable proposal. ``accepted`` means stable, not correct."""
|
||||
|
||||
box: tuple[int, int, int, int]
|
||||
text: str
|
||||
script: str
|
||||
language: str
|
||||
min_score: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RejectedLine:
|
||||
"""A detected line whose recognition was unstable or low-confidence."""
|
||||
|
||||
box: tuple[int, int, int, int]
|
||||
script: str
|
||||
language: str
|
||||
reads: tuple[tuple[str, float], ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TextDraft:
|
||||
"""The full proposal set for one image."""
|
||||
|
||||
accepted: tuple[DraftLine, ...] = ()
|
||||
rejected: tuple[RejectedLine, ...] = ()
|
||||
|
||||
|
||||
def normalize_text(text: str) -> str:
|
||||
"""Normalize text for layout-independent comparison (casefold + no spaces)."""
|
||||
return "".join(unicodedata.normalize("NFC", text).casefold().split())
|
||||
|
||||
|
||||
def _has_script(text: str, script: str) -> bool:
|
||||
return any(script in unicodedata.name(character, "") for character in text)
|
||||
|
||||
|
||||
def choose_language(probes: dict[str, tuple[str, float]]) -> str:
|
||||
"""Pick ``ch``/``ru``/``en`` from what each probe engine actually read."""
|
||||
if _has_script(probes["ch"][0], "CJK"):
|
||||
return "ch"
|
||||
if _has_script(probes["ru"][0], "CYRILLIC"):
|
||||
return "ru"
|
||||
return "en"
|
||||
|
||||
|
||||
def stable_recognition(reads: list[tuple[str, float]], min_score: float = 0.85) -> str | None:
|
||||
"""The proposal text when every read normalizes identically and clears the floor."""
|
||||
normalized = {normalize_text(text) for text, _score in reads}
|
||||
if len(normalized) != 1 or min(score for _text, score in reads) < min_score:
|
||||
return None
|
||||
return reads[0][0]
|
||||
|
||||
|
||||
def _vertical_overlap_ratio(left: tuple[int, int, int, int], right: tuple[int, int, int, int]) -> float:
|
||||
overlap = max(0, min(left[3], right[3]) - max(left[1], right[1]))
|
||||
return overlap / max(1, min(left[3] - left[1], right[3] - right[1]))
|
||||
|
||||
|
||||
def _row_center(box: tuple[int, int, int, int]) -> float:
|
||||
return (box[1] + box[3]) / 2
|
||||
|
||||
|
||||
def group_word_boxes(boxes: list[tuple[int, int, int, int]]) -> list[tuple[int, int, int, int]]:
|
||||
"""Merge word detections into line boxes by vertical overlap and gap."""
|
||||
groups: list[tuple[int, int, int, int]] = []
|
||||
for box in sorted(boxes, key=lambda item: (_row_center(item), item[0])):
|
||||
matches: list[int] = []
|
||||
for index, group in enumerate(groups):
|
||||
if _vertical_overlap_ratio(box, group) < 0.45:
|
||||
continue
|
||||
horizontal_gap = max(0, max(box[0], group[0]) - min(box[2], group[2]))
|
||||
line_height = min(box[3] - box[1], group[3] - group[1])
|
||||
if horizontal_gap <= max(24, line_height * 3):
|
||||
matches.append(index)
|
||||
if not matches:
|
||||
groups.append(box)
|
||||
continue
|
||||
index = max(matches, key=lambda item: _vertical_overlap_ratio(box, groups[item]))
|
||||
x1, y1, x2, y2 = groups[index]
|
||||
groups[index] = min(x1, box[0]), min(y1, box[1]), max(x2, box[2]), max(y2, box[3])
|
||||
return sorted(groups, key=_row_center)
|
||||
|
||||
|
||||
def _recognition_box(
|
||||
box: tuple[int, int, int, int],
|
||||
script: str,
|
||||
width: int,
|
||||
height: int,
|
||||
vertical_pad_ratio: float | None = None,
|
||||
) -> tuple[int, int, int, int]:
|
||||
x1, y1, x2, y2 = box
|
||||
line_height = y2 - y1
|
||||
if script == "cjk":
|
||||
left_pad = max(16, round(line_height * 0.2))
|
||||
right_pad = max(16, round(line_height * 0.6))
|
||||
return max(0, x1 - left_pad), y1, min(width, x2 + right_pad), y2
|
||||
pad_x = max(16, line_height)
|
||||
pad_y = max(8, line_height // 3) if vertical_pad_ratio is None else max(8, round(line_height * vertical_pad_ratio))
|
||||
return max(0, x1 - pad_x), max(0, y1 - pad_y), min(width, x2 + pad_x), min(height, y2 + pad_y)
|
||||
|
||||
|
||||
def _recognize(
|
||||
engine: Any,
|
||||
image: Any,
|
||||
box: tuple[int, int, int, int],
|
||||
script: str,
|
||||
vertical_pad_ratio: float | None = None,
|
||||
) -> tuple[str, float]:
|
||||
import cv2
|
||||
|
||||
height, width = image.shape[:2]
|
||||
x1, y1, x2, y2 = _recognition_box(box, script, width, height, vertical_pad_ratio)
|
||||
crop = image[y1:y2, x1:x2]
|
||||
if crop.shape[0] < 64:
|
||||
scale = 64 / crop.shape[0]
|
||||
crop = cv2.resize(crop, None, fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)
|
||||
result = next(iter(engine.predict(crop)))
|
||||
return str(result.get("rec_text", "")), float(result.get("rec_score", 0.0))
|
||||
|
||||
|
||||
def _detect_line_boxes(engine: Any, source_rgb: Any) -> list[tuple[int, int, int, int]]:
|
||||
import numpy as np
|
||||
|
||||
boxes: list[tuple[int, int, int, int]] = []
|
||||
for page in engine.predict(source_rgb):
|
||||
detected = page.get("rec_boxes", None)
|
||||
if detected is None or len(detected) == 0:
|
||||
detected = page.get("rec_polys", [])
|
||||
for score, raw_box in zip(page.get("rec_scores", []), detected, strict=False):
|
||||
if float(score) < DETECT_SCORE_FLOOR:
|
||||
continue
|
||||
points = np.asarray(raw_box, dtype=np.float32).reshape(-1)
|
||||
if points.size == 4:
|
||||
x1, y1, x2, y2 = points
|
||||
else:
|
||||
points = points.reshape(-1, 2)
|
||||
x1, y1 = points.min(axis=0)
|
||||
x2, y2 = points.max(axis=0)
|
||||
boxes.append((round(float(x1)), round(float(y1)), round(float(x2)), round(float(y2))))
|
||||
return group_word_boxes(boxes)
|
||||
|
||||
|
||||
def draft_available() -> bool:
|
||||
"""True when the ``text-draft`` extra (paddleocr + paddle) can run."""
|
||||
from remove_ai_watermarks.optional_deps import module_available
|
||||
|
||||
return module_available("paddleocr") and module_available("paddle")
|
||||
|
||||
|
||||
def _build_engines() -> tuple[Any, dict[str, Any]]:
|
||||
from paddleocr import PaddleOCR, TextRecognition
|
||||
|
||||
# enable_mkldnn=False is load-bearing on linux: paddle's oneDNN PIR executor
|
||||
# raises NotImplementedError on the text-detection graph there (seen on the
|
||||
# Modal draft container, 2026-08-19), while the reference CPU path runs. The
|
||||
# eval scripts never hit it because macOS pads take a different path.
|
||||
detector = PaddleOCR(
|
||||
lang="ch",
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
enable_mkldnn=False,
|
||||
)
|
||||
engines = {
|
||||
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec", enable_mkldnn=False),
|
||||
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec", enable_mkldnn=False),
|
||||
"ch": TextRecognition(model_name="PP-OCRv5_server_rec", enable_mkldnn=False),
|
||||
}
|
||||
return detector, engines
|
||||
|
||||
|
||||
def draft_text_lines(
|
||||
image: Path,
|
||||
*,
|
||||
min_score: float = 0.85,
|
||||
detector: Any | None = None,
|
||||
engines: dict[str, Any] | None = None,
|
||||
stable: bool = True,
|
||||
) -> TextDraft:
|
||||
"""Propose verified-text manifest lines for ``image``; never verified ones.
|
||||
|
||||
Args:
|
||||
image: path of the source image (draft boxes are in ITS pixel space).
|
||||
min_score: recognition confidence floor. With ``stable=True`` every
|
||||
jittered read must clear it; with ``stable=False`` the single
|
||||
probe for the chosen language must.
|
||||
detector/engines: injectable Paddle objects (tests use fakes); when
|
||||
None they are built from the ``text-draft`` extra, which must be
|
||||
installed (``draft_available()`` reports it).
|
||||
stable: when True (default), accept a line only if three crop paddings
|
||||
normalize identically. When False, take one recognition trio and
|
||||
accept on score alone. Automatic restoration uses box/script only,
|
||||
so the jitter gate is optional there.
|
||||
|
||||
Returns:
|
||||
``TextDraft`` with crop-stable ``accepted`` proposals and unstable
|
||||
``rejected`` lines. Both lists need human review before any manifest
|
||||
may claim ``verified: true``; accepted means crop-stable, NOT
|
||||
ground-truth-correct.
|
||||
"""
|
||||
import os
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
if detector is None or engines is None:
|
||||
if not draft_available():
|
||||
raise RuntimeError(
|
||||
"Text drafting requires PaddleOCR. Install: pip install 'remove-ai-watermarks[text-draft]'"
|
||||
)
|
||||
os.environ.setdefault("PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK", "True")
|
||||
detector, engines = _build_engines()
|
||||
|
||||
source_rgb = np.asarray(Image.open(image).convert("RGB"))
|
||||
accepted: list[DraftLine] = []
|
||||
rejected: list[RejectedLine] = []
|
||||
for box in _detect_line_boxes(detector, source_rgb):
|
||||
probes: dict[str, tuple[str, float]] = {}
|
||||
for language, engine in engines.items():
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
probes[language] = _recognize(engine, source_rgb, box, script, 0.1)
|
||||
language = choose_language(probes)
|
||||
script = "cjk" if language == "ch" else "alphabetic"
|
||||
if stable:
|
||||
reads = [_recognize(engines[language], source_rgb, box, script, ratio) for ratio in JITTER_RATIOS]
|
||||
text = stable_recognition(reads, min_score)
|
||||
else:
|
||||
text, score = probes[language]
|
||||
reads = [(text, score)]
|
||||
if not text.strip() or score < min_score:
|
||||
text = None
|
||||
if text is None:
|
||||
rejected.append(
|
||||
RejectedLine(
|
||||
box=box,
|
||||
script=script,
|
||||
language=language,
|
||||
reads=tuple((value, score) for value, score in reads),
|
||||
)
|
||||
)
|
||||
else:
|
||||
accepted.append(
|
||||
DraftLine(
|
||||
box=box,
|
||||
text=text,
|
||||
script=script,
|
||||
language=language,
|
||||
min_score=min(score for _value, score in reads),
|
||||
)
|
||||
)
|
||||
return TextDraft(accepted=tuple(accepted), rejected=tuple(rejected))
|
||||
@@ -20,7 +20,7 @@ Entries:
|
||||
- ``gemini`` -- Google Gemini / Nano Banana sparkle, bottom-right.
|
||||
- ``doubao`` -- ByteDance Doubao "豆包AI生成" text strip, bottom-right.
|
||||
- ``jimeng`` -- ByteDance Jimeng / Dreamina "★ 即梦AI" wordmark, bottom-right.
|
||||
- ``qwen`` -- Alibaba Tongyi Qianwen "千问AI生成" text strip, bottom-right.
|
||||
- ``qwen`` -- Alibaba Qwen "千问AI生成" text strip, bottom-right.
|
||||
- ``kling`` -- Kuaishou Kling "可灵AI 3.0" text strip, bottom-right.
|
||||
- ``yuanbao`` -- Tencent Yuanbao "元宝 / AI生成" two-line mark, bottom-right.
|
||||
- ``samsung`` -- Samsung Galaxy AI "Contenuti generati dall'AI" strip, bottom-left.
|
||||
@@ -651,7 +651,7 @@ _REGISTRY: tuple[KnownMark, ...] = (
|
||||
"qwen",
|
||||
"Qwen 千问AI生成 text",
|
||||
"bottom-right",
|
||||
platform="Alibaba Tongyi Qianwen (visible 千问AI生成 mark detected)",
|
||||
platform="Alibaba Qwen (visible 千问AI生成 mark detected)",
|
||||
tc260_producer_codes=("91440101MA9Y9T4H7A",),
|
||||
),
|
||||
_text_mark(
|
||||
|
||||
@@ -629,11 +629,13 @@ class TestMetadataCommand:
|
||||
result = runner.invoke(main, ["metadata", str(tmp_clean_png), "--check"])
|
||||
assert result.exit_code == 0
|
||||
assert "No AI metadata" in result.output
|
||||
assert "not the same as 'clean'" in result.output
|
||||
|
||||
def test_metadata_check_ai(self, runner, tmp_png_with_ai_metadata):
|
||||
result = runner.invoke(main, ["metadata", str(tmp_png_with_ai_metadata), "--check"])
|
||||
assert result.exit_code == 0
|
||||
assert "AI metadata detected" in result.output
|
||||
assert "not the same as 'clean'" not in result.output
|
||||
|
||||
def test_metadata_remove(self, runner, tmp_png_with_ai_metadata, tmp_path):
|
||||
output = tmp_path / "stripped.png"
|
||||
@@ -649,6 +651,7 @@ class TestMetadataCommand:
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "stripped" in result.output
|
||||
assert "not the same as 'clean'" in result.output
|
||||
|
||||
def test_metadata_remove_reports_failure_when_the_strip_was_a_no_op(self, runner, tmp_path):
|
||||
"""A file PIL cannot decode is copied through UNCHANGED by the fail-safe.
|
||||
|
||||
+10
-1
@@ -16,6 +16,7 @@ from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from remove_ai_watermarks._internal.constants import C2PA_AI_VENDORS, C2PA_CLAIM_GENERATOR_PLATFORMS
|
||||
from remove_ai_watermarks.identify import (
|
||||
ProvenanceEvidence,
|
||||
ProvenanceReport,
|
||||
@@ -1300,12 +1301,20 @@ class TestVendorOf:
|
||||
def test_registered_vendors_normalize(self):
|
||||
# Regression: these registered C2PA vendors returned None, so their claims never
|
||||
# entered clash detection (a coverage hole). They now normalize to one origin.
|
||||
assert _vendor_of("ByteDance (Doubao / Jimeng / Volcano Engine)") == "ByteDance"
|
||||
assert _vendor_of("Microsoft (Copilot / Designer)") == "Microsoft"
|
||||
assert _vendor_of("Copilot") == "Microsoft"
|
||||
assert _vendor_of("ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)") == "ByteDance"
|
||||
assert _vendor_of("Dreamina/1.2") == "ByteDance"
|
||||
assert _vendor_of("Canva (Magic Media)") == "Canva"
|
||||
assert _vendor_of("Black Forest Labs (FLUX)") == "Black Forest Labs"
|
||||
assert _vendor_of("Eleven Labs Inc.") == "ElevenLabs"
|
||||
|
||||
def test_bytedance_issuers_share_one_platform(self):
|
||||
expected = "ByteDance (Doubao / Jimeng / Dreamina / Volcano Engine)"
|
||||
platforms = {vendor.platform for vendor in C2PA_AI_VENDORS if vendor.needle == "ByteDance"}
|
||||
assert platforms == {expected}
|
||||
assert ("dreamina", expected) in C2PA_CLAIM_GENERATOR_PLATFORMS
|
||||
|
||||
|
||||
class TestIntegrityClashesHelper:
|
||||
def test_two_ai_vendors_clash(self):
|
||||
|
||||
@@ -61,7 +61,6 @@ class TestVerifiedTextMode:
|
||||
cases = (
|
||||
("sdxl-zimage", {}, "qwen-zimage"),
|
||||
("qwen-zimage", {"max_resolution": 1024}, "max-resolution 0"),
|
||||
("qwen-zimage", {"tile": True}, "not calibrated"),
|
||||
("qwen-zimage", {"humanize": 1.0}, "humanize=0"),
|
||||
("qwen-zimage", {"adaptive_polish": True}, "polish disabled"),
|
||||
)
|
||||
@@ -73,6 +72,15 @@ class TestVerifiedTextMode:
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
def test_rejects_fidelity_anchor_without_manifest(self, tmp_path):
|
||||
import pytest
|
||||
|
||||
with pytest.raises(ValueError, match="fidelity_anchor requires a text manifest"):
|
||||
self._engine().remove_watermark(
|
||||
tmp_path / "unused.png",
|
||||
fidelity_anchor=True,
|
||||
)
|
||||
|
||||
def test_loads_and_forwards_verified_manifest(self, tmp_path, monkeypatch):
|
||||
import json
|
||||
|
||||
@@ -111,6 +119,18 @@ class TestVerifiedTextMode:
|
||||
engine.remove_watermark(source, output, text_manifest=manifest)
|
||||
|
||||
assert seen["text_manifest"].lines[0].text == "Exact"
|
||||
# Leak-safe default since 0.27.1: the global 15% donor blend is OFF unless
|
||||
# explicitly requested (measured to return detector-visible OpenAI SynthID
|
||||
# on poster-scale manifests; see docs/text-protection-research.md).
|
||||
assert seen["fidelity_anchor"] is False
|
||||
|
||||
engine.remove_watermark(source, output, text_manifest=manifest, fidelity_anchor=True)
|
||||
|
||||
assert seen["fidelity_anchor"] is True
|
||||
|
||||
engine.remove_watermark(source, output, text_manifest=manifest, tile=True)
|
||||
|
||||
assert seen["tile"] is True
|
||||
|
||||
|
||||
class TestNativeOutputSize:
|
||||
|
||||
+10
-4
@@ -14,6 +14,7 @@ import pytest
|
||||
from remove_ai_watermarks._internal.utils import get_image_format, is_supported_format
|
||||
from remove_ai_watermarks._internal.watermark_profiles import (
|
||||
PROFILE_CHOICES,
|
||||
QWEN_ZIMAGE_GOOGLE_STRENGTH,
|
||||
REMOVAL_MODULES,
|
||||
SDXL_ZIMAGE_GEMINI_STRENGTH,
|
||||
SDXL_ZIMAGE_OPENAI_STRENGTH,
|
||||
@@ -204,16 +205,21 @@ class TestResolveStrength:
|
||||
|
||||
qwen-zimage picks strength from image area, so it takes the size. Returning
|
||||
None for it would push that branch onto every caller and leave one of the two
|
||||
strength policies living outside this module. The vendor is ignored here on
|
||||
purpose - the curve, not the issuer, is what was calibrated.
|
||||
strength policies living outside this module. Google content is the one
|
||||
measured exception since 0.27.2: it takes the flat oracle-measured floor
|
||||
instead of the curve (the curve's 0.154 top left a 4.33 MP fixture
|
||||
oracle-detected on the full production path).
|
||||
"""
|
||||
assert resolve_strength(None, "google", "qwen-zimage", size=(2000, 1850)) == pytest.approx(0.154)
|
||||
assert resolve_strength(None, "openai", "qwen-zimage", size=(2000, 1850)) == pytest.approx(0.154)
|
||||
assert resolve_strength(None, None, "qwen-zimage", size=(600, 500)) == pytest.approx(0.084)
|
||||
assert resolve_strength(None, "google", "qwen-zimage", size=(600, 500)) == QWEN_ZIMAGE_GOOGLE_STRENGTH
|
||||
# The floor holds at every size, not only above the curve's top rung.
|
||||
assert resolve_strength(None, "google", "qwen-zimage", size=(2000, 1850)) == QWEN_ZIMAGE_GOOGLE_STRENGTH
|
||||
|
||||
def test_qwen_zimage_without_a_size_fails_loudly(self):
|
||||
"""A missing size must not silently fall back to some vendor value."""
|
||||
with pytest.raises(ValueError, match="size is required"):
|
||||
resolve_strength(None, "google", "qwen-zimage")
|
||||
resolve_strength(None, "openai", "qwen-zimage")
|
||||
|
||||
def test_sdxl_zimage_uses_its_flat_vendor_ladder(self):
|
||||
|
||||
|
||||
@@ -646,9 +646,58 @@ def test_no_face_path_still_runs_verified_text_restoration(monkeypatch):
|
||||
|
||||
result = pipeline.run(source, strength=0.1, seed=0, text_manifest=manifest)
|
||||
|
||||
assert result is restored
|
||||
# Off by default since 0.27.1 (leak finding, docs/text-protection-research.md):
|
||||
# no whole-frame donor blend; the raw global result feeds restoration.
|
||||
blend.assert_not_called()
|
||||
restore.assert_called_once_with(source, global_result, donor, manifest.lines)
|
||||
|
||||
result = pipeline.run(source, strength=0.1, seed=0, text_manifest=manifest, fidelity_anchor=True)
|
||||
|
||||
assert result is restored
|
||||
blend.assert_called_once_with(global_result, donor)
|
||||
restore.assert_called_once_with(source, anchor, donor, manifest.lines)
|
||||
restore.assert_called_with(source, anchor, donor, manifest.lines)
|
||||
|
||||
|
||||
def test_tiled_verified_text_runs_vae_donor_per_tile(monkeypatch):
|
||||
from remove_ai_watermarks._internal import qwen_zimage_pipeline, text_restoration
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import QwenZImagePipeline
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextLine, VerifiedTextManifest
|
||||
|
||||
pipeline = object.__new__(QwenZImagePipeline)
|
||||
pipeline.device = "cuda"
|
||||
pipeline.progress_callback = None
|
||||
source = Image.new("RGB", (96, 80), (10, 20, 30))
|
||||
restored = Image.new("RGB", (96, 80), (130, 140, 150))
|
||||
pipeline._qwen_vae_roundtrip = MagicMock(side_effect=lambda tile: tile)
|
||||
pipeline._run_global = MagicMock(side_effect=lambda tile, _strength, _seed: tile)
|
||||
monkeypatch.setattr(qwen_zimage_pipeline, "detect_faces", lambda _image: [])
|
||||
restore = MagicMock(return_value=restored)
|
||||
monkeypatch.setattr(text_restoration, "restore_verified_text", restore)
|
||||
manifest = VerifiedTextManifest(
|
||||
"0" * 64,
|
||||
96,
|
||||
80,
|
||||
(VerifiedTextLine((4, 4, 20, 16), "Exact", "alphabetic"),),
|
||||
)
|
||||
|
||||
result = pipeline.run(
|
||||
source,
|
||||
strength=0.1,
|
||||
seed=0,
|
||||
tile=True,
|
||||
tile_size=64,
|
||||
tile_overlap=16,
|
||||
text_manifest=manifest,
|
||||
)
|
||||
|
||||
assert result is restored
|
||||
assert pipeline._qwen_vae_roundtrip.call_count > 1
|
||||
assert pipeline._run_global.call_count > 1
|
||||
restore.assert_called_once()
|
||||
assert restore.call_args.args[0].size == (96, 80)
|
||||
assert restore.call_args.args[1].size == (96, 80)
|
||||
assert restore.call_args.args[2].size == (96, 80)
|
||||
|
||||
|
||||
def test_watermark_remover_dispatches_to_full_pipeline(tmp_path, monkeypatch):
|
||||
@@ -858,6 +907,7 @@ def test_invisible_engine_passes_the_seed_but_never_a_step_count(tmp_image_path,
|
||||
def test_sdxl_zimage_strength_is_vendor_adaptive_and_leaves_other_profiles_alone():
|
||||
"""An SDXL global pass needs more strength than Qwen, so it gets its own policy."""
|
||||
from remove_ai_watermarks._internal.watermark_profiles import (
|
||||
QWEN_ZIMAGE_GOOGLE_STRENGTH,
|
||||
SDXL_ZIMAGE_GEMINI_STRENGTH,
|
||||
SDXL_ZIMAGE_OPENAI_STRENGTH,
|
||||
resolve_strength,
|
||||
@@ -867,11 +917,14 @@ def test_sdxl_zimage_strength_is_vendor_adaptive_and_leaves_other_profiles_alone
|
||||
assert resolve_strength(None, "google", "sdxl-zimage") == pytest.approx(SDXL_ZIMAGE_GEMINI_STRENGTH)
|
||||
# Unknown provenance takes the stricter of the two.
|
||||
assert resolve_strength(None, None, "sdxl-zimage") == pytest.approx(SDXL_ZIMAGE_GEMINI_STRENGTH)
|
||||
# An explicit value still wins, and qwen-zimage is untouched by this ladder: it
|
||||
# defers to its resolution curve rather than to a vendor value.
|
||||
# An explicit value still wins, and qwen-zimage's curve is untouched by this
|
||||
# ladder - with the one measured exception: Google content takes the flat
|
||||
# oracle floor (0.27.2), not the area curve.
|
||||
assert resolve_strength(0.4, "google", "sdxl-zimage") == pytest.approx(0.4)
|
||||
assert resolve_strength(None, "openai", "qwen-zimage", size=(2000, 1850)) == pytest.approx(0.154)
|
||||
assert resolve_strength(None, "google", "qwen-zimage", size=(2000, 1850)) == pytest.approx(0.154)
|
||||
assert resolve_strength(None, "google", "qwen-zimage", size=(2000, 1850)) == pytest.approx(
|
||||
QWEN_ZIMAGE_GOOGLE_STRENGTH
|
||||
)
|
||||
|
||||
|
||||
def test_sdxl_zimage_shares_the_fixed_seed_contract():
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
"""text_draft: proposal-only OCR for verified-text manifests (no paddle needed)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks.text_draft import (
|
||||
TextDraft,
|
||||
choose_language,
|
||||
draft_available,
|
||||
draft_text_lines,
|
||||
group_word_boxes,
|
||||
stable_recognition,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class TestPureHelpers:
|
||||
def test_choose_language_follows_unicode_script(self):
|
||||
assert choose_language({"ch": ("你好", 0.9), "ru": ("xxx", 0.1), "en": ("yyy", 0.1)}) == "ch"
|
||||
assert choose_language({"ch": ("??", 0.1), "ru": ("привет", 0.9), "en": ("yyy", 0.1)}) == "ru"
|
||||
assert choose_language({"ch": ("??", 0.1), "ru": ("xxx", 0.1), "en": ("hello", 0.9)}) == "en"
|
||||
|
||||
def test_stable_recognition_requires_identical_normalizations(self):
|
||||
# Same text modulo spacing/case (comma kept): stable.
|
||||
assert (
|
||||
stable_recognition([("Hello, World", 0.97), ("hello,world", 0.96), ("HELLO, WORLD", 0.95)])
|
||||
== "Hello, World"
|
||||
)
|
||||
# Different texts: rejected (None).
|
||||
assert stable_recognition([("Hello", 0.97), ("Hallo", 0.96), ("Hello", 0.95)]) is None
|
||||
# Identical text but one read under the floor: rejected.
|
||||
assert stable_recognition([("Hello", 0.97), ("Hello", 0.60), ("Hello", 0.95)]) is None
|
||||
assert stable_recognition([("Hello", 0.97), ("Hello", 0.96), ("Hello", 0.95)], min_score=0.99) is None
|
||||
|
||||
def test_group_word_boxes_merges_a_line_and_keeps_rows_apart(self):
|
||||
# Two words on one baseline merge; a distant lower line stays separate.
|
||||
merged = group_word_boxes([(10, 100, 60, 140), (70, 100, 120, 140)])
|
||||
assert len(merged) == 1
|
||||
assert merged[0] == (10, 100, 120, 140)
|
||||
apart = group_word_boxes([(10, 100, 60, 140), (10, 300, 60, 340)])
|
||||
assert len(apart) == 2
|
||||
|
||||
|
||||
class TestDraftAvailable:
|
||||
def test_returns_a_bool_without_importing_paddle(self):
|
||||
assert isinstance(draft_available(), bool)
|
||||
|
||||
|
||||
class _FakeDetector:
|
||||
"""Emits Paddle-style pages for one synthetic poster."""
|
||||
|
||||
def __init__(self, boxes: list[tuple[int, int, int, int]]) -> None:
|
||||
self._boxes = boxes
|
||||
|
||||
def predict(self, _image: Any) -> Any:
|
||||
import numpy as np
|
||||
|
||||
yield {
|
||||
"rec_boxes": np.asarray(self._boxes, dtype=np.float32),
|
||||
"rec_scores": [0.99] * len(self._boxes),
|
||||
}
|
||||
|
||||
|
||||
class _FakeEngine:
|
||||
"""Returns a fixed text at high score regardless of the crop."""
|
||||
|
||||
def __init__(self, text: str) -> None:
|
||||
self._text = text
|
||||
|
||||
def predict(self, _crop: Any) -> Any:
|
||||
yield {"rec_text": self._text, "rec_score": 0.97}
|
||||
|
||||
|
||||
class _JitterEngine:
|
||||
"""Changes its answer with the crop height - exactly what the gate rejects."""
|
||||
|
||||
def __init__(self, texts: list[str]) -> None:
|
||||
self._texts = texts
|
||||
self._calls = 0
|
||||
|
||||
def predict(self, crop: Any) -> Any:
|
||||
text = self._texts[self._calls % len(self._texts)]
|
||||
self._calls += 1
|
||||
yield {"rec_text": text, "rec_score": 0.97}
|
||||
|
||||
|
||||
class TestDraftTextLines:
|
||||
@staticmethod
|
||||
def _poster(tmp_path: Path) -> Path:
|
||||
path = tmp_path / "poster.png"
|
||||
Image.new("RGB", (400, 300), (255, 255, 255)).save(path)
|
||||
return path
|
||||
|
||||
def test_accepts_crop_stable_and_rejects_jittered(self, tmp_path: Path):
|
||||
path = self._poster(tmp_path)
|
||||
stable = _FakeEngine("Invoice 42")
|
||||
jitter = _JitterEngine(["Hello", "Hallo", "Hullo"])
|
||||
# All three probe engines read the same string, so language = en; the
|
||||
# jitter engine then serves as the en engine and flips across crops.
|
||||
draft = draft_text_lines(
|
||||
path,
|
||||
detector=_FakeDetector([(20, 20, 200, 60), (20, 80, 200, 120)]),
|
||||
engines={"en": jitter, "ru": stable, "ch": stable},
|
||||
)
|
||||
assert isinstance(draft, TextDraft)
|
||||
assert len(draft.rejected) == 2
|
||||
assert all(line.language == "en" for line in draft.rejected)
|
||||
assert draft.accepted == ()
|
||||
|
||||
def test_accepted_line_carries_box_text_script_and_floor(self, tmp_path: Path):
|
||||
path = self._poster(tmp_path)
|
||||
engine = _FakeEngine("Total: 1,234.56")
|
||||
draft = draft_text_lines(
|
||||
path,
|
||||
detector=_FakeDetector([(20, 20, 260, 60)]),
|
||||
engines={"en": engine, "ru": engine, "ch": engine},
|
||||
)
|
||||
(line,) = draft.accepted
|
||||
assert line.box == (20, 20, 260, 60)
|
||||
assert line.text == "Total: 1,234.56"
|
||||
assert line.script == "alphabetic"
|
||||
assert line.language == "en"
|
||||
assert line.min_score >= 0.85
|
||||
|
||||
def test_cjk_probe_switches_script_and_language(self, tmp_path: Path):
|
||||
path = self._poster(tmp_path)
|
||||
cjk = _FakeEngine("每天都是一个新的机会。")
|
||||
latin = _FakeEngine("hello")
|
||||
draft = draft_text_lines(
|
||||
path,
|
||||
detector=_FakeDetector([(30, 30, 300, 90)]),
|
||||
engines={"en": latin, "ru": latin, "ch": cjk},
|
||||
)
|
||||
(line,) = draft.accepted
|
||||
assert line.language == "ch"
|
||||
assert line.script == "cjk"
|
||||
|
||||
def test_unstable_geometry_draft_accepts_a_single_high_score_read(self, tmp_path: Path):
|
||||
path = self._poster(tmp_path)
|
||||
jitter = _JitterEngine(["Hello", "Hallo", "Hullo"])
|
||||
stable = _FakeEngine("Hello")
|
||||
# Probes disagree across engines so language is en; jitter would fail
|
||||
# the stable gate. Geometry mode uses the one en probe and accepts.
|
||||
draft = draft_text_lines(
|
||||
path,
|
||||
min_score=0.75,
|
||||
stable=False,
|
||||
detector=_FakeDetector([(20, 20, 200, 60)]),
|
||||
engines={"en": jitter, "ru": stable, "ch": stable},
|
||||
)
|
||||
(line,) = draft.accepted
|
||||
assert line.text == "Hello"
|
||||
assert line.min_score >= 0.75
|
||||
@@ -15,6 +15,7 @@ from remove_ai_watermarks._internal.text_restoration import (
|
||||
load_verified_text_manifest,
|
||||
restore_verified_text,
|
||||
source_pixel_sha256,
|
||||
source_silhouette_mask,
|
||||
)
|
||||
|
||||
|
||||
@@ -36,6 +37,17 @@ def _manifest(image: Image.Image) -> dict[str, object]:
|
||||
}
|
||||
|
||||
|
||||
def _geometry_manifest(image: Image.Image) -> dict[str, object]:
|
||||
return {
|
||||
"schema_version": 2,
|
||||
"verified": True,
|
||||
"source_pixel_sha256": source_pixel_sha256(image),
|
||||
"width": image.width,
|
||||
"height": image.height,
|
||||
"lines": [{"box": [8, 8, 40, 24], "angle": 0.0}],
|
||||
}
|
||||
|
||||
|
||||
def test_pixel_hash_ignores_container_metadata(tmp_path) -> None:
|
||||
image = Image.new("RGB", (48, 32), (10, 20, 30))
|
||||
plain = tmp_path / "plain.png"
|
||||
@@ -62,6 +74,40 @@ def test_verified_manifest_is_bound_to_source_pixels(tmp_path) -> None:
|
||||
assert loaded.lines == (VerifiedTextLine((8, 8, 40, 24), "Exact text", "alphabetic", 0.0),)
|
||||
|
||||
|
||||
def test_geometry_manifest_needs_no_transcription_or_script(tmp_path) -> None:
|
||||
source = Image.new("RGB", (48, 32), (10, 20, 30))
|
||||
path = tmp_path / "regions.json"
|
||||
path.write_text(json.dumps(_geometry_manifest(source)), encoding="utf-8")
|
||||
|
||||
loaded = load_verified_text_manifest(path, source)
|
||||
|
||||
assert loaded.lines == (VerifiedTextLine((8, 8, 40, 24), angle=0.0),)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field", ["text", "script"])
|
||||
def test_schema_one_still_requires_text_metadata(tmp_path, field) -> None:
|
||||
source = Image.new("RGB", (48, 32), (10, 20, 30))
|
||||
payload = _manifest(source)
|
||||
del payload["lines"][0][field]
|
||||
path = tmp_path / "lines.json"
|
||||
path.write_text(json.dumps(payload), encoding="utf-8")
|
||||
|
||||
with pytest.raises(ValueError, match=field):
|
||||
load_verified_text_manifest(path, source)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("schema_version", [True, 0, 3])
|
||||
def test_manifest_rejects_unsupported_schema_versions(tmp_path, schema_version) -> None:
|
||||
source = Image.new("RGB", (48, 32), (10, 20, 30))
|
||||
payload = _geometry_manifest(source)
|
||||
payload["schema_version"] = schema_version
|
||||
path = tmp_path / "lines.json"
|
||||
path.write_text(json.dumps(payload), encoding="utf-8")
|
||||
|
||||
with pytest.raises(ValueError, match="Unsupported text manifest schema"):
|
||||
load_verified_text_manifest(path, source)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("mutation", "message"),
|
||||
[
|
||||
@@ -119,10 +165,35 @@ def test_restoration_uses_lama_and_qwen_vae_core(monkeypatch) -> None:
|
||||
Image.fromarray(source),
|
||||
Image.fromarray(candidate),
|
||||
Image.fromarray(donor),
|
||||
(VerifiedTextLine((8, 8, 48, 28), "Exact text", "alphabetic"),),
|
||||
(VerifiedTextLine((8, 8, 48, 28)),),
|
||||
)
|
||||
|
||||
restored = np.asarray(result)
|
||||
assert calls
|
||||
assert np.all(restored[16, 20] == donor[16, 20])
|
||||
assert np.all(restored[0, 0] == candidate[0, 0])
|
||||
|
||||
|
||||
def test_silhouette_includes_descender_below_the_detector_box() -> None:
|
||||
source = np.full((40, 50, 3), 240, dtype=np.uint8)
|
||||
source[10:22, 18:24] = 20
|
||||
source[22:27, 18:22] = 20 # tail of a y / Cyrillic u, under the box
|
||||
|
||||
mask = source_silhouette_mask(source, (10, 10, 40, 22))
|
||||
|
||||
assert mask[24, 20] == 255
|
||||
assert mask[16, 20] == 255
|
||||
|
||||
|
||||
def test_silhouette_includes_glyph_edges_beside_the_detector_box() -> None:
|
||||
source = np.full((40, 70, 3), 240, dtype=np.uint8)
|
||||
source[10:22, 20:50] = 20
|
||||
source[14:18, 10:20] = 20 # leading flourish, left of the detector box
|
||||
source[14:18, 50:58] = 20 # punctuation or icon edge, right of the box
|
||||
source[14:18, 2:5] = 20 # separate decoration must stay outside the crop
|
||||
|
||||
mask = source_silhouette_mask(source, (20, 10, 50, 22))
|
||||
|
||||
assert mask[16, 12] == 255
|
||||
assert mask[16, 56] == 255
|
||||
assert mask[16, 3] == 0
|
||||
@@ -951,6 +951,7 @@ class TestVideoMetadataCli:
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "AI metadata detected" in result.output
|
||||
assert "not the same as 'clean'" not in result.output
|
||||
|
||||
def test_remove_reports_output(self, tmp_path: Path):
|
||||
runner = CliRunner()
|
||||
@@ -961,6 +962,7 @@ class TestVideoMetadataCli:
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "AI metadata stripped" in result.output
|
||||
assert "not the same as 'clean'" in result.output
|
||||
assert C2PA_UUID not in output.read_bytes()
|
||||
|
||||
def test_rejects_image_input(self, tmp_clean_png: Path):
|
||||
|
||||
Reference in new issue
Block a user