mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-31 01:30:35 +02:00
Reject the uncalibrated text-manifest tiling and verify Content Seal transforms
Tiled diffusion was never provider-oracle calibrated with verified text restoration: the tiled VAE donor path ran anyway and produced results no oracle had certified. The combination is now rejected at both the pipeline and the engine seam (ValueError with the reason), and the CLI help no longer implies support. The invisible help is generalized and the metadata container list corrected (MKA/OGA/Opus/AAC). scripts/contentseal_transforms.py reproduces the deterministic crop, resize, and JPEG variants of the Content Seal corpus from manifest.csv, hash-verifying every output; its README gains scripts/README.md context and new data tests. The corpus README is honest about the one crop the daily oracle limit left unchecked, and the eval CSVs carry the updated verdicts. The byte-scan SynthID suppression hoists its soft-binding lookup so the guard is computed once. Staged on top of 0.33.1; no version bump in this commit.
This commit is contained in:
@@ -4,7 +4,7 @@ name: Distribute on release
|
||||
# PyPI is handled by publish.yml. This workflow event-drives the three channels
|
||||
# that would otherwise be manual:
|
||||
# - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
|
||||
# - HF Space: factory-rebuild so it reinstalls the latest sdist from PyPI.
|
||||
# - HF Space: factory-rebuild the version already pinned in the Space repo.
|
||||
# - ComfyUI: sync, test, and publish the node against the exact library release.
|
||||
# All wait for the freshly published sdist to appear on PyPI first, since the
|
||||
# Release event fires in parallel with publish.yml's upload.
|
||||
|
||||
@@ -7,13 +7,14 @@ Remove AI provenance marks from images and video you generated yourself:
|
||||
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
|
||||
|
||||
Video support covers provenance identification, complete visible-plus-metadata
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo, and
|
||||
Kling mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo AI, and
|
||||
Kling AI mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
removal.
|
||||
|
||||
> [raiw.cc](https://raiw.cc) runs this library as a hosted service, with the GPU
|
||||
> included and nothing to install. Visible mark and metadata removal are free
|
||||
> there; invisible watermark removal is paid.
|
||||
> included and nothing to install. Visible mark and metadata removal at Standard
|
||||
> output up to 12 MP are free there; original resolution above 12 MP and invisible
|
||||
> watermark removal are paid.
|
||||
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
@@ -230,6 +231,13 @@ features, and development setup.
|
||||
| --- | --- |
|
||||
|  |  |
|
||||
|
||||
The `after` raster is generated from the tracked `before` raster by the public path:
|
||||
|
||||
```bash
|
||||
uv run remove-ai-watermarks visible demo_banana_before.png \
|
||||
--backend cv2 -o demo_banana_after.png
|
||||
```
|
||||
|
||||
### High quality invisible removal
|
||||
|
||||
`qwen-zimage` is the default profile: a Qwen-Image-2512 Lightning pass under Canny
|
||||
@@ -311,8 +319,8 @@ remove-ai-watermarks batch ./images --mode all
|
||||
|
||||
Visible mark support includes:
|
||||
|
||||
- Google Gemini and Nano Banana sparkle;
|
||||
- Doubao, Jimeng, Qwen, Kling, Yuanbao, Baidu, LibLibAI, and RunningHub labels;
|
||||
- Google Gemini and Nano Banana visible sparkle watermark;
|
||||
- Doubao, Jimeng, Qwen, Kling AI, Yuanbao, Baidu, LiblibAI, and RunningHub labels;
|
||||
- one calibrated Samsung Galaxy AI label variant.
|
||||
|
||||
Metadata and provenance inspection covers C2PA, EXIF, XMP, IPTC, common
|
||||
|
||||
+5
-4
@@ -48,16 +48,17 @@ runtime assets.
|
||||
video removal profile works, so it is also the source of truth for three shipped
|
||||
defaults: `tests/test_video_invisible.py` asserts that `noise_std`, `long_side`,
|
||||
and `fps` together match a row this manifest records as certified. Changing one
|
||||
of those three without adding the row that certifies it fails the suite. `vae` is
|
||||
deliberately outside that check because neither tracked row records one; add it
|
||||
to the assertion in the same commit as the first row that does.
|
||||
of those three without adding the row that certifies it fails the suite. Both
|
||||
historical rows mark `vae` as `unrecorded`: the repository must not infer a model
|
||||
identity from the current default. Add it to the assertion in the same commit as
|
||||
the first oracle row that records and verifies one.
|
||||
|
||||
| Column | Meaning |
|
||||
| --- | --- |
|
||||
| `date`, `source_url`, `source_sha256` | Identify the carrier. |
|
||||
| `source_width`, `source_height`, `source_fps` | Carrier geometry. Without it the actual downscale factor of a row cannot be recovered later. |
|
||||
| `duration_seconds`, `source_verdict` | Clip length submitted and the verifier's reading of the untouched carrier. |
|
||||
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The full run configuration. |
|
||||
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The run configuration. `unrecorded` means the historical run did not preserve the VAE identity. |
|
||||
| `output_sha256` | Identifies the exact submitted file. |
|
||||
| `output_verdict` | One of `detected`, `not_detected`, `indeterminate`, `refused`. |
|
||||
| `output_verdict_text` | The verifier's wording, verbatim. |
|
||||
|
||||
@@ -16,9 +16,12 @@ plus hash and is not stored.
|
||||
embedded in the watermark payload. Both survived a 512 px LANCZOS resize and
|
||||
a full-size JPEG q85 re-encode (same ID returned), so the payload is more
|
||||
robust than the detection threshold.
|
||||
- Center crops lose the seal: 50% and 33% linear center crops of two different
|
||||
images all returned "No AI signatures from Meta were found", consistent with
|
||||
the Reuters 2026-07-11 analysis (55% missed after cropping).
|
||||
- Three checked center crops lost the seal: 50% and 33% linear crops of the fox
|
||||
and the 50% crop of the text poster returned "No AI signatures from Meta were
|
||||
found". The text poster's 33% crop was not checked because the daily oracle
|
||||
limit was reached, so its empty verdict is not evidence either way. The checked
|
||||
results are consistent with the Reuters 2026-07-11 analysis (55% missed after
|
||||
cropping).
|
||||
- API outputs carry XMP `iptcExt:DigitalSourceType =
|
||||
trainedAlgorithmicMedia`, so local `identify` flags them via the existing
|
||||
Made-with-AI path. Metadata-stripping transforms fall back to unknown, and
|
||||
@@ -62,8 +65,8 @@ the calibration rows below is the race-free variant.
|
||||
|
||||
The library resolves strength per vendor with measured floors (OpenAI
|
||||
0.07675 / Google 0.27 / Microsoft InvisMark 0.15 in
|
||||
`_internal/watermark_profiles.py`). Meta Content Seal has no floor yet; the
|
||||
goal of these rows is to measure one by that same methodology: independent
|
||||
`_internal/watermark_profiles.py`). Meta Content Seal had no floor before this
|
||||
calibration; these rows measure one by the same methodology: independent
|
||||
generations, each one's first-clean boundary, floor = worst boundary plus the
|
||||
observed cross-source spread.
|
||||
|
||||
@@ -83,7 +86,16 @@ Measured (2026-08-26/27, oracle `meta.ai/identification`):
|
||||
names it explicitly on stripped files (implying the scrub runs).
|
||||
## Regeneration
|
||||
|
||||
API key is not stored in this repository. Regenerate with the script pattern
|
||||
from the session (env `MUSE_API_KEY`, endpoint
|
||||
`https://api.meta.ai/v1/images/generations`, model `muse-image-1.0`); prompts
|
||||
are recorded per file in `manifest.csv`.
|
||||
The eight deterministic crop, resize, and JPEG variants can be reproduced and
|
||||
hash-checked from the tracked originals:
|
||||
|
||||
```bash
|
||||
uv run python scripts/contentseal_transforms.py /tmp/contentseal-derived
|
||||
```
|
||||
|
||||
The Meta API generations and remote GPU outputs are not reproducible from this
|
||||
repository alone. Their prompts, exact output hashes, model/profile settings,
|
||||
and oracle results are recorded in `manifest.csv`, but the generation API is
|
||||
stochastic and the private worker environment is not tracked. `MUSE_API_KEY` and
|
||||
the anonymous detector session are deliberately absent. A new calibration must
|
||||
therefore create new manifest rows rather than claiming to recreate these bytes.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
date,source_commit,fixture,variant,model,model_revision,model_sha256,ocr_route,text_cer,img_lpips,ssim,psnr_db,output_sha256,provider_oracle,notes
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,,,,"Paddle ch",0.074,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621,not_run,Exact Qwen baseline reused from text-restoration benchmark
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,,,,"Paddle ch",0.074,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_run,Exact Qwen baseline reused from text-restoration benchmark
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-default,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.185,0.338,0.642,18.2,87729fe7e0482d2cead6d3b7c7dd386ecc75ab3d8658131c54e67528b8f9feeb,not_run,Fidelity gate failed; padded crop recognition CER 0.074 still shows two substituted characters; locally composited three-line edit; 20 DDIM steps; seed 0
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-mimic,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.222,0.345,0.644,17.9,f5198d8f2365b79880aba8ea6b0c3885142b9de21c9f7f4f6712ac4a0db8dd85,not_run,Fidelity gate failed; source glyph masks supplied as font hints; 20 DDIM steps; seed 0
|
||||
|
||||
|
@@ -1,3 +1,3 @@
|
||||
date,source_url,source_sha256,source_width,source_height,source_fps,duration_seconds,source_verdict,vae,noise_std,long_side,fps,seed,output_sha256,output_verdict,output_verdict_text,output_detected_range,track,session_id,stratum,psnr_db,temporal_residual_ratio
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,unrecorded,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,unrecorded,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
|
||||
|
||||
|
@@ -160,7 +160,7 @@ sparkle; the SynthID *pixel* itself has no local detector (oracle only).
|
||||
|
||||
Key takeaways:
|
||||
- The same model differs by *surface*: Gemini app wraps C2PA, AI Studio (API/playground) emits none -- only the pixel + sparkle survive.
|
||||
- Microsoft Designer's DALL-E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL-E).
|
||||
- Microsoft Designer's DALL·E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL·E).
|
||||
- Meta uses the IPTC `digitalSourceType` marker, not C2PA or SynthID.
|
||||
- The open imwatermark fires only on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL) -- not from re-hosts (Krea, Stability hosted SDXL) or re-encoded design exports (Recraft, Canva). Ideogram's only signal is the EXIF `Make` tag.
|
||||
- Bing and Grok web UIs are uncooperative for autonomous capture (no document_idle for screenshots; blob downloads intermittently no-op; low-res in-chat previews). Use their native download button manually if a full-res sample is needed.
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 3.6 MiB After Width: | Height: | Size: 5.8 MiB |
+4
-4
@@ -434,10 +434,10 @@ user can act on rather than after a model load.
|
||||
### Restore operator-verified text
|
||||
|
||||
`--text-manifest` enables the experimental `vae-glyphs` post-pass. It reconstructs
|
||||
the source with the Qwen VAE, blends 15% of that reconstruction into the normal
|
||||
`qwen-zimage` result, erases the annotated candidate glyphs with LaMa, and composites
|
||||
only the reconstructed glyph cores through source-derived silhouettes. It does not
|
||||
run OCR or choose which strings are correct.
|
||||
the source with the Qwen VAE, erases the annotated candidate glyphs with LaMa, and
|
||||
composites only the reconstructed glyph cores through source-derived silhouettes. It
|
||||
does not run OCR or choose which strings are correct. The optional `--fidelity-anchor`
|
||||
described below additionally blends 15% of the reconstruction across the full frame.
|
||||
|
||||
Install the combined extra and run only with an operator-verified manifest:
|
||||
|
||||
|
||||
@@ -641,7 +641,7 @@ photo only).
|
||||
|
||||
**Sources.** https://arxiv.org/abs/2206.11253 (CodeFormer) · https://github.com/sczhou/CodeFormer ·
|
||||
https://arxiv.org/pdf/2401.07519 (InstantID) ·
|
||||
https://openaccess.thecvf.com/content/WACV2024/papers/Suin_Diffuse_and_Restore... (region-adaptive) ·
|
||||
https://openaccess.thecvf.com/content/WACV2024/html/Suin_Diffuse_and_Restore_A_Region-Adaptive_Diffusion_Model_for_Identity-Preserving_Blind_WACV_2024_paper.html (region-adaptive) ·
|
||||
https://arxiv.org/pdf/2504.12809 (saliency-aware watermark removal)
|
||||
|
||||
## Provenance
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# Development
|
||||
|
||||
Standalone evaluation, calibration, and release utilities are catalogued in
|
||||
[`scripts/README.md`](../scripts/README.md). They are maintainer tools rather than
|
||||
installed commands, and their local data inputs remain outside the repository.
|
||||
|
||||
Read this reference for environment setup, dependency recovery, CI behavior, and fixture policy. The always-loaded invariants remain in [`.claude/rules/development.md`](../.claude/rules/development.md).
|
||||
|
||||
## Local environment
|
||||
|
||||
@@ -114,6 +114,7 @@ application actually uses:
|
||||
| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `qwen-zimage` | Invisible image-watermark removal, both CUDA-only profiles | `diffusion`, DiffSynth | Yes |
|
||||
| `text-restoration` | Opt-in verified Qwen-VAE glyph restoration | `qwen-zimage`, `lama` | Yes |
|
||||
| `text-draft` | Draft OCR proposals for operator verification | PaddleOCR, PaddlePaddle | Model download, no Torch |
|
||||
| `all` | Every production feature available on the active Python | All compatible rows above | Yes |
|
||||
| `dev` | Tests, linting, typing, and upstream parity checks | `video`, `detect`, upstream invisible-watermark | Yes, for parity tests |
|
||||
|
||||
@@ -130,12 +131,15 @@ flowchart LR
|
||||
qwen["qwen-zimage"] --> diffusion
|
||||
text["text-restoration"] --> qwen
|
||||
text --> lama
|
||||
draft["text-draft"]
|
||||
heif
|
||||
trustmark
|
||||
```
|
||||
|
||||
`heif` and `trustmark` are independent branches. Combine them explicitly with
|
||||
another feature when required. TrustMark requires NumPy 1.x, which has no
|
||||
`heif`, `trustmark`, and `text-draft` are independent branches. Combine them
|
||||
explicitly with another feature when required. `text-draft` is excluded from
|
||||
`all` because it proposes unverified OCR annotations and is not a production
|
||||
removal path. TrustMark requires NumPy 1.x, which has no
|
||||
CPython 3.13 or 3.14 wheels, so that branch is available only on Python
|
||||
3.11-3.12. The `all` bundle contains every production branch compatible with
|
||||
the active Python and never includes `dev`.
|
||||
|
||||
@@ -416,7 +416,7 @@ remain identical because isolated decoder hits can otherwise be content noise.
|
||||
It accepts Variant P schemas 0-2. Variant Q requires a different model, and
|
||||
schema 3 is rejected at the measured precision threshold. Its NumPy 1.x runtime
|
||||
limits the extra to Python 3.11-3.12; the rest of the package remains supported
|
||||
through Python 3.14. Through 0.29.0 the extra also resolves lightning 2.6.5
|
||||
through Python 3.14. The current TrustMark dependency line also resolves lightning 2.6.5
|
||||
(PYSEC-2026-3624, no fixed release yet); the vulnerable `load_from_checkpoint`
|
||||
path is unreachable here because TrustMark loads its checksummed checkpoints
|
||||
with plain `torch.load`. Bump lightning and cut a patch release when a fix
|
||||
|
||||
@@ -746,9 +746,9 @@ capture-less pill is too weak to attribute.
|
||||
|
||||
The set of marks that veto the pill is DERIVED from the registry rows: every mark
|
||||
under the same label regime (`tc260`) belonging to a different product. It used to
|
||||
be a hand-written list of keys, and that list drifted -- LibLibAI was registered
|
||||
alongside RunningHub and Baidu, both of which were added to it, and LibLibAI was
|
||||
not, so a confident LibLibAI detection did not suppress the pill the way its two
|
||||
be a hand-written list of keys, and that list drifted -- LiblibAI was registered
|
||||
alongside RunningHub and Baidu, both of which were added to it, and LiblibAI was
|
||||
not, so a confident LiblibAI detection did not suppress the pill the way its two
|
||||
siblings did. Marks outside the TC260 regime (Gemini, Samsung) are deliberately
|
||||
not vetoers: neither can put `jimeng` into `provenance`, so neither can enable the
|
||||
arm it would be vetoing.
|
||||
@@ -818,7 +818,7 @@ were briefly skipped there for exactly that reason.
|
||||
A mark whose removable footprint differs from what the detector localizes
|
||||
overrides `_footprint_rect` (which policy) and `_extend_match_box` (how far the
|
||||
box grows), not the whole `footprint_mask`. Baidu extends right to the corner tag
|
||||
and LibLibAI extends left to the triangle logo; both inherit every guard around
|
||||
and LiblibAI extends left to the triangle logo; both inherit every guard around
|
||||
that arithmetic.
|
||||
|
||||
Yuanbao uses the polarity-independent `contrast` front end because its standard
|
||||
@@ -1020,11 +1020,13 @@ Validation happens before model loading. The library never treats OCR confidence
|
||||
verification, and geometry-only operators do not need to invent text or script fields.
|
||||
|
||||
When enabled, `QwenZImagePipeline` reconstructs the source once through its already
|
||||
loaded Qwen VAE, runs the ordinary global and face stages, blends 15% of the VAE
|
||||
reconstruction into that clean result, and calls the shared restoration compositor.
|
||||
The compositor derives binary source and candidate silhouettes, groups nearby lines,
|
||||
uses LaMa for the initial and residual-glyph erase passes, paints fresh silhouette
|
||||
edges, then copies the Qwen-VAE core with a 0.5-pixel feather. The evaluation script
|
||||
loaded Qwen VAE, runs the ordinary global and face stages, and calls the shared
|
||||
restoration compositor. The optional fidelity anchor first blends 15% of the VAE
|
||||
reconstruction into the clean result; it is off by default because the blend returned
|
||||
detector-visible OpenAI SynthID in the measured poster fixtures. The compositor derives
|
||||
binary source and candidate silhouettes, groups nearby lines, uses LaMa for the initial
|
||||
and residual-glyph erase passes, paints fresh silhouette edges, then copies the Qwen-VAE
|
||||
core with a 0.5-pixel feather. The evaluation script
|
||||
imports these same mask and compositing helpers so the two implementations cannot
|
||||
silently drift. Silhouette crops start 12% of line height beyond each horizontal
|
||||
side, then expand each side independently while a foreground component anchored
|
||||
|
||||
+4
-4
@@ -644,10 +644,10 @@ engine.remove_watermark(
|
||||
Install `remove-ai-watermarks[text-restoration]`. The manifest schema and safety
|
||||
constraints are documented in the CLI guide. The engine verifies its decoded RGB
|
||||
hash before loading the diffusion models and rejects SDXL, downscaling, and
|
||||
postprocessing combinations that were not evaluated. Tiling is allowed: the VAE
|
||||
donor uses the same overlapping tiles as the global pass. `InvisibleOptions` exposes the
|
||||
same field for `remove_all`; after a visible-stage edit, the manifest must be built
|
||||
against the staged pixels rather than the pristine source.
|
||||
postprocessing combinations that were not evaluated. Tiling is also rejected because
|
||||
the combined tiled-restoration path has no provider-oracle calibration. `InvisibleOptions`
|
||||
exposes the same field for `remove_all`; after a visible-stage edit, the manifest must
|
||||
be built against the staged pixels rather than the pristine source.
|
||||
|
||||
Use manifest schema 1 for manually reviewed text plus script metadata. Automated
|
||||
operators that verify only text-region geometry should emit schema 2 lines with a
|
||||
|
||||
@@ -19,7 +19,7 @@ The `visible` command registers these mark keys:
|
||||
| `samsung` | `✦ Contenuti generati dall'AI` | Bottom left | Calibrated for the Italian text variant. |
|
||||
| `runninghub` | `RunningHub AI生成` | Top left | Strict visual and position gates. |
|
||||
| `baidu` | `百度 AI生成` | Bottom right | Detector and extended removal footprint. |
|
||||
| `liblib` | `LibLibAI` | Bottom center | Includes a minimum image size gate. |
|
||||
| `liblib` | `LiblibAI` | Bottom center | Includes a minimum image size gate. |
|
||||
| `jimeng_pill` | `AI生成` pill | Top left | Weak detector with additional product and background gates. |
|
||||
|
||||
`--mark auto` evaluates all registered marks and removes every selected match.
|
||||
@@ -226,7 +226,7 @@ not a universal clean verdict.
|
||||
| Kling | Kling image and video marks | No registered pixel decoder | TC260 AIGC |
|
||||
| Hailuo / MiniMax video | Hailuo composite video label | No registered pixel decoder | TC260 AIGC where present |
|
||||
| Baidu | Baidu mark | No registered pixel decoder | TC260 AIGC |
|
||||
| LibLibAI | LibLibAI mark | No registered pixel decoder | TC260 AIGC |
|
||||
| LiblibAI | LiblibAI mark | No registered pixel decoder | TC260 AIGC |
|
||||
| RunningHub | RunningHub mark | No registered pixel decoder | TC260 AIGC |
|
||||
| Samsung Galaxy AI | One locale specific mark | No registered pixel decoder | C2PA and Samsung markers |
|
||||
|
||||
|
||||
@@ -58,7 +58,7 @@ defines two more native serializations. AVI stores an `AIGC` child in
|
||||
bounded RIFF and FLV readers validate the JSON field set and skip media
|
||||
payloads. Removal remuxes either container through ffmpeg with stream copy.
|
||||
|
||||
- **HuggingFace-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** HuggingFace Jobs / Spaces can stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID. It marks the *hosting job*, not a model, so it lifts an Unknown verdict to a tentative AI via `hf_only` but never overrides a hard metadata signal. `_HF_JOB_CAVEAT` states the limit. Removal drops the chunk through the PNG metadata whitelist.
|
||||
- **Hugging Face-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** Hugging Face Jobs / Spaces can stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID. It marks the *hosting job*, not a model, so it lifts an Unknown verdict to a tentative AI via `hf_only` but never overrides a hard metadata signal. `_HF_JOB_CAVEAT` states the limit. Removal drops the chunk through the PNG metadata whitelist.
|
||||
- **No detectable signal on some downloads:** Recraft exports and some hosted
|
||||
FLUX surfaces can arrive without a supported local signal. Midjourney samples
|
||||
may carry IPTC metadata but no registered C2PA or pixel watermark. The open
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
# Maintainer scripts
|
||||
|
||||
These scripts are development and evaluation tools, not installed CLI commands. Run
|
||||
them from the repository root with `uv run python scripts/<name>.py --help`. Inputs
|
||||
under `.local-eval/` and generated reports remain untracked unless a data README
|
||||
explicitly names a tracked canonical result.
|
||||
|
||||
## Audits and release checks
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `corpus_gap_scan.py` | Compare a local image corpus with the library's `identify` results. |
|
||||
| `detection_timing.py` | Record per-method metadata and verdict timings. |
|
||||
| `detection_timing_report.py` | Aggregate timing records by method and segment. |
|
||||
| `fidelity_metrics.py` | Compute objective image-fidelity metrics for paired outputs. |
|
||||
| `invisible_quality_audit.py` | Pair originals and invisible-removal outputs for quality review. |
|
||||
| `metadata_removal_audit.py` | Check metadata detection/removal parity over a corpus. |
|
||||
| `pill_gate_audit.py` | Measure the Jimeng pill detector on the product path. |
|
||||
| `real_examples_e2e.py` | Run end-to-end confidence checks over local examples. |
|
||||
| `record_parity_audit.py` | Compare record-based and file-based identification. |
|
||||
| `resource_ceilings.py` | Measure peak RSS and runtime for fill backends. |
|
||||
| `robustness_suite.py` | Exercise CLI failures on adversarial and degenerate inputs. |
|
||||
| `sidecar_regression.py` | Compare current identification with recorded sidecars. |
|
||||
| `smoke_matrix.py` | Exercise CLI parameter choices on real local data. |
|
||||
| `video_fidelity_probe.py` | Compare delivered video fidelity with its source. |
|
||||
| `visible_eval.py` | Benchmark registered visible-mark detectors. |
|
||||
| `visible_removal_audit.py` | Audit visible-removal results over a local corpus. |
|
||||
|
||||
## Calibration and corpus preparation
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `contentseal_transforms.py` | Reproduce and hash-check deterministic Content Seal variants. |
|
||||
| `detector_response.py` | Measure detector response over mark size, contrast, background, and aspect. |
|
||||
| `fill_quality.py` | Measure visible-fill quality against constructed ground truth. |
|
||||
| `ladder_headroom.py` | Measure recall cost from the coarse scale ladder. |
|
||||
| `synthid_corpus.py` | Ingest and inspect the local SynthID reference corpus. |
|
||||
| `vendor_cohort_harvest.py` | Partition TC260 carriers by producer code. |
|
||||
| `vendor_mark_calibrate.py` | Calibrate a candidate vendor text detector. |
|
||||
| `visible_alpha_solve.py` | Rebuild visible-watermark alpha assets from controlled captures. |
|
||||
| `visible_groundtruth.py` | Consolidate blinded contact-sheet labels into ground truth. |
|
||||
| `visible_positives.py` | List corpus images carrying a registered visible mark. |
|
||||
| `visible_recall_sample.py` | Build an unbiased blinded sample for recall measurement. |
|
||||
| `visible_sheets.py` | Build blinded contact sheets for relaxation candidates. |
|
||||
|
||||
## Research and diagnostic prototypes
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `cjk_tail_probe.py` | Test a generic template for otherwise uncovered CJK labels. |
|
||||
| `controlnet_sweep.py` | Sweep the historical ControlNet removal prototype. |
|
||||
| `infer_text_lines.py` | Draft stable source-text lines without modifying pixels. |
|
||||
| `qwen_scrub_prototype.py` | Probe low-strength Qwen regeneration on a GPU. |
|
||||
| `selective_text_restoration.py` | Evaluate text restoration over a scrubbed image. |
|
||||
| `synthid_pixel_probe.py` | Run the experimental local SynthID carrier probe. |
|
||||
| `video_synthid_sweep.py` | Build oracle-gated video regeneration candidates. |
|
||||
|
||||
## Generated assets
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `render_pill_silhouette.py` | Render the synthetic Jimeng pill silhouette. |
|
||||
| `render_vendor_silhouettes.py` | Render synthetic vendor text-mark silhouettes. |
|
||||
|
||||
## Shared helpers
|
||||
|
||||
`_plain_console.py` provides plain-text fallbacks for Rich output, and
|
||||
`_text_eval.py` contains normalization helpers shared by text-evaluation scripts.
|
||||
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Reproduce the deterministic Content Seal crop, resize, and JPEG variants."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CORPUS = ROOT / "data" / "contentseal"
|
||||
MANIFEST = CORPUS / "manifest.csv"
|
||||
|
||||
|
||||
def _rows() -> dict[str, dict[str, str]]:
|
||||
with MANIFEST.open(newline="", encoding="utf-8") as stream:
|
||||
return {row["name"]: row for row in csv.DictReader(stream)}
|
||||
|
||||
|
||||
def _write_and_verify(image: Image.Image, path: Path, row: dict[str, str], *, format: str, quality: int) -> None:
|
||||
image.save(path, format=format, quality=quality)
|
||||
digest = hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
if digest != row["sha256"]:
|
||||
raise RuntimeError(f"{row['name']} hash mismatch: expected {row['sha256']}, got {digest}")
|
||||
log.info("Verified %s", path)
|
||||
|
||||
|
||||
def reproduce_transforms(output_dir: Path) -> list[Path]:
|
||||
"""Write and hash-check the eight deterministic manifest variants."""
|
||||
rows = _rows()
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
outputs: list[Path] = []
|
||||
|
||||
for prefix, source_name in (("fox", "gen_fox_forest"), ("text", "gen_text_poster")):
|
||||
with Image.open(CORPUS / rows[source_name]["file"]) as opened:
|
||||
source = opened.convert("RGB")
|
||||
|
||||
for fraction in (0.5, 0.33):
|
||||
width = int(source.width * fraction)
|
||||
height = int(source.height * fraction)
|
||||
left = (source.width - width) // 2
|
||||
top = (source.height - height) // 2
|
||||
name = f"{prefix}_crop{int(fraction * 100)}"
|
||||
path = output_dir / f"{name}.webp"
|
||||
crop = source.crop((left, top, left + width, top + height))
|
||||
_write_and_verify(crop, path, rows[name], format="WEBP", quality=95)
|
||||
outputs.append(path)
|
||||
|
||||
scale = 512 / max(source.size)
|
||||
resized = source.resize(
|
||||
(round(source.width * scale), round(source.height * scale)),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
name = f"{prefix}_res512"
|
||||
path = output_dir / f"{name}.webp"
|
||||
_write_and_verify(resized, path, rows[name], format="WEBP", quality=95)
|
||||
outputs.append(path)
|
||||
|
||||
name = f"{prefix}_jpeg85"
|
||||
path = output_dir / f"{name}.jpg"
|
||||
_write_and_verify(source, path, rows[name], format="JPEG", quality=85)
|
||||
outputs.append(path)
|
||||
|
||||
return outputs
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("output_dir", type=Path, help="Directory for regenerated variants")
|
||||
args = parser.parse_args()
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
|
||||
reproduce_transforms(args.output_dir)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -21,9 +21,9 @@ honest doubt never becomes a fabricated data point.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
SEEN_TO_MARK = {
|
||||
@@ -77,7 +77,15 @@ def metadata_provenance(path: str) -> list[str]:
|
||||
|
||||
|
||||
def main() -> None:
|
||||
root = Path(sys.argv[1] if len(sys.argv) > 1 else ".local-eval/textmark-relaxation")
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"root",
|
||||
type=Path,
|
||||
nargs="?",
|
||||
default=Path(".local-eval/textmark-relaxation"),
|
||||
help="Directory containing the blinded labelling rounds",
|
||||
)
|
||||
root = parser.parse_args().root
|
||||
out = root / "groundtruth.jsonl"
|
||||
rows: dict[str, dict] = {}
|
||||
stats: dict[str, int] = {}
|
||||
|
||||
@@ -22,10 +22,10 @@ Design decisions that matter:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import random
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
@@ -35,6 +35,8 @@ import numpy as np
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
|
||||
@@ -63,10 +65,16 @@ def corner_strip(img: NDArray[Any]) -> NDArray[Any] | None:
|
||||
|
||||
|
||||
def main() -> None:
|
||||
scan = Path(sys.argv[1])
|
||||
out = Path(sys.argv[2])
|
||||
n_tc260 = int(sys.argv[3]) if len(sys.argv) > 3 else 160
|
||||
n_google = int(sys.argv[4]) if len(sys.argv) > 4 else 80
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("scan", type=Path, help="JSONL corpus scan produced by the visible evaluation harness")
|
||||
parser.add_argument("output", type=Path, help="Directory for contact sheets and the blinded manifest")
|
||||
parser.add_argument("--tc260", type=int, default=160, help="Number of TC260 carriers to sample")
|
||||
parser.add_argument("--google", type=int, default=80, help="Number of Google-provenance carriers to sample")
|
||||
args = parser.parse_args()
|
||||
scan = args.scan
|
||||
out = args.output
|
||||
n_tc260 = args.tc260
|
||||
n_google = args.google
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
recs = [json.loads(line) for line in scan.open() if '"marks"' in line]
|
||||
|
||||
@@ -12,6 +12,7 @@ Each sheet mixes three strata in shuffled order:
|
||||
The two control strata are what make a low measured precision trustworthy.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import random
|
||||
@@ -62,9 +63,13 @@ def crop(path: str, region: tuple[int, int, int, int] | None, pad_factor: float
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with open(sys.argv[1]) as fh:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("input", type=Path, help="JSON candidate list")
|
||||
parser.add_argument("output", type=Path, help="Directory for blinded contact sheets")
|
||||
args = parser.parse_args()
|
||||
with args.input.open() as fh:
|
||||
items = json.load(fh) # [{uid,path,key,stratum,conf}]
|
||||
outdir = Path(sys.argv[2])
|
||||
outdir = args.output
|
||||
outdir.mkdir(parents=True, exist_ok=True)
|
||||
random.Random(1234).shuffle(items) # noqa: S311 -- sheet ordering, not cryptography
|
||||
|
||||
|
||||
@@ -624,16 +624,16 @@ def _populate_registry_fields(buffer: bytes, info: dict[str, Any]) -> bool:
|
||||
|
||||
if b"c2pa.watermarked" in buffer:
|
||||
info["watermarked"] = True
|
||||
soft_bindings = soft_binding_vendors_in(buffer)
|
||||
synthid = (
|
||||
[]
|
||||
if soft_binding_vendors_in(buffer)
|
||||
if soft_bindings
|
||||
else synthid_evidence_vendors_in(buffer, has_watermark_action=info.get("watermarked", False))
|
||||
)
|
||||
if ai_source and synthid:
|
||||
info["synthid_vendors"] = synthid
|
||||
info["synthid_watermark"] = synthid_verdict(", ".join(synthid))
|
||||
|
||||
soft_bindings = soft_binding_vendors_in(buffer)
|
||||
if soft_bindings:
|
||||
info["soft_binding_vendors"] = soft_bindings
|
||||
info["soft_binding"] = ", ".join(soft_bindings)
|
||||
|
||||
@@ -70,7 +70,7 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
|
||||
_vendor(
|
||||
b"OpenAI",
|
||||
"OpenAI",
|
||||
"OpenAI (ChatGPT / gpt-image / DALL-E / Sora)",
|
||||
"OpenAI (ChatGPT / GPT Image / DALL·E / Sora)",
|
||||
"OpenAI",
|
||||
synthid=True,
|
||||
synthid_requires_watermark_action=True,
|
||||
@@ -128,8 +128,8 @@ C2PA_AI_TOOLS = {
|
||||
("GPT-4o", "GPT-4o"),
|
||||
("ChatGPT", "ChatGPT"),
|
||||
("Sora", "Sora"),
|
||||
("DALL-E", "DALL-E"),
|
||||
("DALL", "DALL-E"),
|
||||
("DALL-E", "DALL·E"),
|
||||
("DALL", "DALL·E"),
|
||||
("Imagen", "Imagen"),
|
||||
("Firefly", "Firefly"),
|
||||
("Dreamina", "Dreamina"),
|
||||
@@ -138,16 +138,18 @@ C2PA_AI_TOOLS = {
|
||||
|
||||
C2PA_SOFT_BINDINGS = {
|
||||
b"com.adobe.trustmark": "Adobe TrustMark",
|
||||
b"com.adobe.icn": "Adobe (content fingerprint)",
|
||||
b"com.digimarc": "Digimarc",
|
||||
b"com.adobe.icn": "Adobe Image Comparator Network",
|
||||
b"com.digimarc": "Digimarc Validate",
|
||||
b"com.imatag.lamark": "Imatag (Lamark)",
|
||||
b"ai.steg": "Steg.AI",
|
||||
b"com.microsoft.invismark": "Microsoft InvisMark",
|
||||
b"com.microsoft.wavmark": "Microsoft WavMark",
|
||||
b"com.verimatrix": "Verimatrix",
|
||||
b"com.nagra.nexguard": "NAGRA NexGuard",
|
||||
b"com.aiwatermark": "AIWatermark (Meta PixelSeal)",
|
||||
b"ai.trufo": "Trufo",
|
||||
b"com.aiwatermark.pixelseal": "AIWatermark PixelSeal",
|
||||
b"com.aiwatermark.videoseal": "AIWatermark VideoSeal",
|
||||
b"com.aiwatermark.audioseal": "AIWatermark AudioSeal",
|
||||
b"ai.trufo": "Trufo PawPrint",
|
||||
b"app.overlai": "Overlai",
|
||||
b"com.markany": "MarkAny",
|
||||
b"com.mentaport": "Mentaport",
|
||||
|
||||
@@ -1074,29 +1074,14 @@ class QwenZImagePipeline:
|
||||
fidelity_anchor: bool = False,
|
||||
) -> Image.Image:
|
||||
"""Execute global regeneration and masked face repair."""
|
||||
if text_manifest is not None and tile:
|
||||
raise ValueError("Verified text restoration is not calibrated with tiled diffusion")
|
||||
self._require_cuda()
|
||||
seed = resolve_seed(seed)
|
||||
donor = None
|
||||
if text_manifest is not None:
|
||||
self._progress("Reconstructing the verified text donor with the Qwen VAE...")
|
||||
if tile and max(image.size) > tile_size:
|
||||
from remove_ai_watermarks._internal.tiling import run_tiled
|
||||
|
||||
donor = run_tiled(
|
||||
self._qwen_vae_roundtrip,
|
||||
image,
|
||||
tile_size,
|
||||
tile_overlap,
|
||||
lambda message: self._progress(
|
||||
message.replace(
|
||||
"Tiled diffusion",
|
||||
"Reconstructing the verified text donor",
|
||||
1,
|
||||
)
|
||||
),
|
||||
)
|
||||
else:
|
||||
donor = self._qwen_vae_roundtrip(image)
|
||||
donor = self._qwen_vae_roundtrip(image)
|
||||
global_strength = (
|
||||
resolution_adaptive_denoise(image.width, image.height) if strength is None else float(strength)
|
||||
)
|
||||
|
||||
@@ -90,7 +90,7 @@ class TextMarkConfig:
|
||||
asset_name: str # bundled alpha PNG under assets/ (e.g. "doubao_alpha.png")
|
||||
corner: Literal[
|
||||
"br", "bl", "tl", "bc"
|
||||
] # bottom-right (Doubao/Jimeng), bottom-left (Samsung), top-left (RunningHub), bottom-center (LibLibAI)
|
||||
] # bottom-right (Doubao/Jimeng), bottom-left (Samsung), top-left (RunningHub), bottom-center (LiblibAI)
|
||||
margin_floor: int # min margin in px for locate (4 for br marks, 2 for Samsung)
|
||||
# locate geometry (fraction of scale_base -- see scale_base())
|
||||
width_frac: float
|
||||
@@ -695,7 +695,7 @@ class TextMarkEngine:
|
||||
|
||||
OVERRIDABLE, and the override contract is specifically the DETECTOR'S MATCH BOX:
|
||||
a mark whose removable footprint reaches beyond what the NCC localizes -- Baidu's
|
||||
flat white tag right of the text run, LibLibAI's triangle logo left of the
|
||||
flat white tag right of the text run, LiblibAI's triangle logo left of the
|
||||
wordmark -- supplies its own extension here and inherits the rest of the
|
||||
footprint path. The blob-bbox branch never routes through an override.
|
||||
"""
|
||||
@@ -771,7 +771,7 @@ class TextMarkEngine:
|
||||
"""Footprint policy for a mark whose fill must be bounded by the DETECTOR's match
|
||||
box and never by the binary glyph blob.
|
||||
|
||||
Baidu's white tag has a flat interior a top-hat cannot answer, and LibLibAI's
|
||||
Baidu's white tag has a flat interior a top-hat cannot answer, and LiblibAI's
|
||||
blob bleeds up into background structure; in both cases the blob bbox is
|
||||
measurably wrong and the NCC match box is right. ``force`` takes priority here,
|
||||
unlike the default policy: a ``--no-detect`` caller named the mark, so the whole
|
||||
|
||||
@@ -157,7 +157,7 @@ def remove_visible(
|
||||
"""Remove every detected known visible AI mark through localize then fill.
|
||||
|
||||
The registry currently covers the Gemini sparkle; Doubao, Jimeng, Qwen, Kling,
|
||||
Yuanbao, Samsung, RunningHub, Baidu, and LibLibAI text marks; and the Jimeng
|
||||
Yuanbao, Samsung, RunningHub, Baidu, and LiblibAI text marks; and the Jimeng
|
||||
pill. Returns ``(result_bgr, [labels removed])``.
|
||||
|
||||
``source`` is a file path OR a BGR ndarray. For a PATH, metadata provenance is read
|
||||
|
||||
@@ -307,7 +307,7 @@ _seed_option = click.option(
|
||||
help="Random seed for reproducibility. Default 0: both profiles are certified "
|
||||
"at a fixed seed, because SynthID removal near the strength floor is seed-dependent.",
|
||||
)
|
||||
_hf_token_option = click.option("--hf-token", type=str, default=None, help="HuggingFace API token.")
|
||||
_hf_token_option = click.option("--hf-token", type=str, default=None, help="Hugging Face API token.")
|
||||
_humanize_option = click.option(
|
||||
"--humanize", type=float, default=0.0, help="Analog Humanizer film grain intensity (0 = off, typical: 2.0-6.0)."
|
||||
)
|
||||
@@ -859,7 +859,7 @@ def cmd_invisible(
|
||||
text_manifest: Path | None,
|
||||
fidelity_anchor: bool,
|
||||
) -> None:
|
||||
"""Remove invisible AI watermarks (SynthID, StableSignature, TreeRing).
|
||||
"""Attempt to disrupt invisible AI watermarks through pixel regeneration.
|
||||
|
||||
Regenerates the pixels with the two-stage diffusion profile. CUDA-only:
|
||||
pip install 'remove-ai-watermarks[qwen-zimage]'
|
||||
@@ -1001,8 +1001,8 @@ def cmd_metadata(
|
||||
Strips EXIF AI tags, PNG text chunks, C2PA provenance manifests, and the
|
||||
China TC260 AIGC label. Beyond images (PNG/JPEG/WebP/AVIF/HEIF/JXL) it also
|
||||
strips provenance metadata from MP4/MOV/M4V/M4A containers and, via ffmpeg,
|
||||
from WebM/MKV/AVI/FLV/MP3/WAV/FLAC/OGG. The coded image, audio, and video
|
||||
data are left untouched.
|
||||
from WebM/MKV/MKA/AVI/FLV/MP3/WAV/FLAC/OGG/OGA/Opus/AAC. The coded image,
|
||||
audio, and video data are left untouched.
|
||||
"""
|
||||
from remove_ai_watermarks.metadata import get_ai_metadata, has_ai_metadata, strip_and_verify
|
||||
|
||||
|
||||
@@ -145,7 +145,7 @@ _INVISIBLE_WM_CAVEAT = (
|
||||
"or resizing, so it confirms origin only on a pristine (un-re-encoded) file."
|
||||
)
|
||||
_HF_JOB_CAVEAT = (
|
||||
"The hf-job-id tag marks a HuggingFace-hosted job (commonly diffusion "
|
||||
"The hf-job-id tag marks a Hugging Face-hosted job (commonly diffusion "
|
||||
"generation) but names neither the model nor the content type, so it is a "
|
||||
"medium-confidence signal, not proof the pixels are AI-generated."
|
||||
)
|
||||
@@ -445,7 +445,7 @@ def evidence_from_metadata_record(
|
||||
if iptc_system:
|
||||
ai_metadata.setdefault("ai_system", f"IPTC 2025.1 AI disclosure ({iptc_system})")
|
||||
if hf_job:
|
||||
ai_metadata.setdefault("huggingface_job", f"HuggingFace-hosted job ({hf_job})")
|
||||
ai_metadata.setdefault("huggingface_job", f"Hugging Face-hosted job ({hf_job})")
|
||||
if samsung is not None:
|
||||
ai_metadata.setdefault("samsung_genai", f"Samsung Galaxy AI editing marker (genAIType={samsung})")
|
||||
|
||||
@@ -953,7 +953,7 @@ def _visible_sparkle(image_path: Path, *, image: NDArray[Any] | None = None) ->
|
||||
# metadata label); the per-engine detection thresholds live in the registry.
|
||||
# Text mark -> the platform sentence this report prints when that mark is the strongest
|
||||
# evidence, DERIVED from the registry rows so registering a mark is one edit. It was a
|
||||
# hand-maintained copy, and that class of copy is how LibLibAI ended up registered but
|
||||
# hand-maintained copy, and that class of copy is how LiblibAI ended up registered but
|
||||
# missing from the pill veto. Insertion order is the registry's, which is what fixes the
|
||||
# scan order below. The Gemini sparkle and the capture-less pill carry no platform of
|
||||
# their own (`KnownMark.platform is None`) and are excluded here: the sparkle has its
|
||||
@@ -1426,17 +1426,17 @@ def _identify_from_evidence(
|
||||
platform = "xAI (Grok / Aurora)"
|
||||
ai_vendor_claims["xai"] = "xAI"
|
||||
|
||||
# ── HuggingFace-hosted job marker (hf-job-id PNG text chunk) ─────
|
||||
# ── Hugging Face-hosted job marker (hf-job-id PNG text chunk) ─────
|
||||
# Marks the hosting job, not a model -- medium confidence (commonly diffusion
|
||||
# output). Like the visible sparkle, it lifts an otherwise-Unknown verdict to
|
||||
# a tentative AI, but never overrides a high-confidence metadata signal.
|
||||
hf_job = evidence.huggingface_job
|
||||
if hf_job:
|
||||
signals.append(Signal("hf_job", f"HuggingFace job {hf_job}", "medium"))
|
||||
watermarks.append("HuggingFace-hosted job (hf-job-id)")
|
||||
signals.append(Signal("hf_job", f"Hugging Face job {hf_job}", "medium"))
|
||||
watermarks.append("Hugging Face-hosted job (hf-job-id)")
|
||||
caveats.append(_HF_JOB_CAVEAT)
|
||||
if platform is None:
|
||||
platform = "HuggingFace-hosted job (model not identified)"
|
||||
platform = "Hugging Face-hosted job (model not identified)"
|
||||
|
||||
# ── Samsung Galaxy AI editing marker (genAIType) ─────────────────
|
||||
# Galaxy AI tools stamp a proprietary genAIType in PhotoEditor_Re_Edit_Data.
|
||||
|
||||
@@ -33,7 +33,7 @@ warnings.filterwarnings("ignore", category=UserWarning, module="huggingface_hub"
|
||||
warnings.filterwarnings("ignore", category=UserWarning, module="diffusers")
|
||||
warnings.filterwarnings("ignore", module="transformers")
|
||||
|
||||
# Suppress HuggingFace internal logging
|
||||
# Suppress Hugging Face internal logging
|
||||
os.environ["TRANSFORMERS_VERBOSITY"] = "error"
|
||||
os.environ["DIFFUSERS_VERBOSITY"] = "error"
|
||||
|
||||
@@ -104,7 +104,7 @@ class InvisibleEngine:
|
||||
global pass, vendor-adaptive strength because an SDXL global stage
|
||||
needs more of it). BOTH ARE CUDA-ONLY -- there is no CPU or MPS path
|
||||
for invisible-watermark removal.
|
||||
hf_token: HuggingFace API token.
|
||||
hf_token: Hugging Face API token.
|
||||
progress_callback: Optional callback for progress messages.
|
||||
controlnet_conditioning_scale: Canny ControlNet structure-preservation
|
||||
strength on the global stage of both profiles.
|
||||
@@ -187,9 +187,8 @@ class InvisibleEngine:
|
||||
pixels. Enables the experimental Qwen-VAE ``vae-glyphs`` post-pass.
|
||||
Requires the ``text-restoration`` extra and the ``qwen-zimage``
|
||||
profile. Incompatible with downscaling, humanize, unsharp, and
|
||||
adaptive polish. Tiling is supported: the VAE donor uses the same
|
||||
overlapping tiles as the global pass, then glyph restore runs on
|
||||
the blended full frame.
|
||||
adaptive polish. Tiling is rejected because that combination has
|
||||
no provider-oracle calibration.
|
||||
fidelity_anchor: Blend 15% of the Qwen-VAE donor across the whole frame
|
||||
before glyph restoration. OFF by default since 0.27.1: that global
|
||||
blend was measured to return detector-visible OpenAI SynthID on
|
||||
@@ -211,6 +210,8 @@ class InvisibleEngine:
|
||||
if text_manifest is not None:
|
||||
if self._remover.model_profile != QWEN_ZIMAGE_PROFILE:
|
||||
raise ValueError("--text-manifest is supported only by the qwen-zimage profile")
|
||||
if tile:
|
||||
raise ValueError("--text-manifest is not calibrated with --tile")
|
||||
if max_resolution != 0:
|
||||
raise ValueError("--text-manifest requires --max-resolution 0")
|
||||
if humanize > 0.0 or unsharp > 0.0 or adaptive_polish:
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
"""LibLibAI visible watermark detector/localizer.
|
||||
"""LiblibAI visible watermark detector/localizer.
|
||||
|
||||
LibLibAI (哩布哩布AI, USCC 91110105MACJ6K1C8A) stamps its generations with a
|
||||
white triangle logo + "LibLibAI" latin wordmark at **bottom-center** (not a
|
||||
LiblibAI (哩布哩布AI, USCC 91110105MACJ6K1C8A) stamps its generations with a
|
||||
white triangle logo + "LiblibAI" Latin wordmark at **bottom-center** (not a
|
||||
corner -- the locate box is horizontally centered). Detection matches the
|
||||
bundled font-rendered "LibLibAI" silhouette (the triangle logo is NOT rendered
|
||||
bundled font-rendered "LiblibAI" silhouette (the triangle logo is NOT rendered
|
||||
-- logos vary, the wordmark discriminates); removal is the shared **localize ->
|
||||
fill** (the glyph blob covers logo + wordmark, both bright).
|
||||
|
||||
This module supplies only LibLibAI's tuned :class:`TextMarkConfig`
|
||||
This module supplies only LiblibAI's tuned :class:`TextMarkConfig`
|
||||
(``assets/liblib_alpha.png`` from ``scripts/render_vendor_silhouettes.py``,
|
||||
never cut from an upload).
|
||||
|
||||
@@ -64,7 +64,7 @@ _ALPHA_HEIGHT_FRAC = 0.026
|
||||
_LADDER = (0.9, 1.0, 1.1)
|
||||
|
||||
_CONFIG = TextMarkConfig(
|
||||
name="LibLibAI",
|
||||
name="LiblibAI",
|
||||
asset_name="liblib_alpha.png",
|
||||
corner="bc",
|
||||
margin_floor=4,
|
||||
@@ -90,12 +90,12 @@ _CONFIG = TextMarkConfig(
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled LibLibAI alpha template (float [0,1]), or None."""
|
||||
"""The bundled LiblibAI alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
class LibLibEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible LibLibAI wordmark (bottom-center; localize -> fill)."""
|
||||
"""Detect/localize the visible LiblibAI wordmark (bottom-center; localize -> fill)."""
|
||||
|
||||
# Per-mark size floor prevents small generic icons from matching the wordmark.
|
||||
_MIN_SHORT_SIDE = 480
|
||||
|
||||
@@ -199,7 +199,7 @@ def parse_tc260_aigc_json(value: bytes) -> dict[str, str] | None:
|
||||
return fields if TC260_AIGC_FIELDS & fields.keys() else None
|
||||
|
||||
|
||||
# HuggingFace-hosted GPU jobs (Jobs / Spaces) stamp generated PNGs with this
|
||||
# Hugging Face-hosted GPU jobs (Jobs / Spaces) stamp generated PNGs with this
|
||||
# ``tEXt`` chunk key holding the job UUID. It marks the hosting job, not a
|
||||
# specific model -- a medium-confidence AI signal (commonly diffusion output).
|
||||
_HF_JOB_KEY: str = "hf-job-id"
|
||||
@@ -510,7 +510,7 @@ def has_ai_metadata(image_path: Path) -> bool:
|
||||
# only the XMP form; the raw-JSON tEXt chunk needs the PIL-based parse).
|
||||
if aigc_label(image_path) is not None:
|
||||
return True
|
||||
# HuggingFace-hosted job marker (hf-job-id PNG text chunk).
|
||||
# Hugging Face-hosted job marker (hf-job-id PNG text chunk).
|
||||
if huggingface_job(image_path):
|
||||
return True
|
||||
# xAI / Grok: no C2PA/IPTC/XMP -- only the EXIF Signature + UUID-Artist pair.
|
||||
@@ -682,10 +682,10 @@ def c2pa_cloud_manifest(image_path: Path) -> str | None:
|
||||
|
||||
|
||||
def _huggingface_job_impl(image_path: Path) -> str | None:
|
||||
"""Return the HuggingFace job id if the image carries an ``hf-job-id`` PNG
|
||||
"""Return the Hugging Face job id if the image carries an ``hf-job-id`` PNG
|
||||
text chunk, else None.
|
||||
|
||||
HuggingFace-hosted GPU jobs (Jobs / Spaces) stamp generated PNGs with an
|
||||
Hugging Face-hosted GPU jobs (Jobs / Spaces) stamp generated PNGs with an
|
||||
``hf-job-id`` ``tEXt`` chunk holding the job's UUID. It identifies the
|
||||
*hosting job*, not a specific model, and is most commonly seen on diffusion-
|
||||
generation output -- a medium-confidence AI signal, not proof of AI pixels
|
||||
@@ -1216,9 +1216,9 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
if system := iptc_ai_system(image_path):
|
||||
result.setdefault("ai_system", f"IPTC 2025.1 AI disclosure ({system})")
|
||||
|
||||
# HuggingFace-hosted job marker (hf-job-id PNG text chunk).
|
||||
# Hugging Face-hosted job marker (hf-job-id PNG text chunk).
|
||||
if job := huggingface_job(image_path):
|
||||
result.setdefault("huggingface_job", f"HuggingFace-hosted job ({job})")
|
||||
result.setdefault("huggingface_job", f"Hugging Face-hosted job ({job})")
|
||||
# Samsung Galaxy AI editing marker (genAIType in PhotoEditor_Re_Edit_Data).
|
||||
if (genai := samsung_genai(image_path)) is not None:
|
||||
result.setdefault("samsung_genai", f"Samsung Galaxy AI editing marker (genAIType={genai})")
|
||||
|
||||
@@ -179,8 +179,8 @@ _VISIBLE_PLATFORM = {
|
||||
"veo": "Google Veo",
|
||||
"seedance": "ByteDance Seedance",
|
||||
"dola": "ByteDance Dola",
|
||||
"hailuo": "MiniMax Hailuo",
|
||||
"kling": "Kuaishou Kling",
|
||||
"hailuo": "MiniMax Hailuo AI",
|
||||
"kling": "Kuaishou Kling AI",
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -17,17 +17,17 @@ localizer stays cheap (cv2/numpy, CPU) so a memory-tight caller can run it on a
|
||||
small worker; the heavy fill (MI-GAN / LaMa) is opt-in and chosen by the caller.
|
||||
|
||||
Entries:
|
||||
- ``gemini`` -- Google Gemini / Nano Banana sparkle, bottom-right.
|
||||
- ``gemini`` -- Google Gemini / Nano Banana visible watermark (sparkle), bottom-right.
|
||||
- ``doubao`` -- ByteDance Doubao "豆包AI生成" text strip, bottom-right.
|
||||
- ``jimeng`` -- ByteDance Jimeng / Dreamina "★ 即梦AI" wordmark, bottom-right.
|
||||
- ``qwen`` -- Alibaba Qwen "千问AI生成" text strip, bottom-right.
|
||||
- ``kling`` -- Kuaishou Kling "可灵AI 3.0" text strip, bottom-right.
|
||||
- ``kling`` -- Kuaishou Kling AI "可灵AI 3.0" text strip, bottom-right.
|
||||
- ``yuanbao`` -- Tencent Yuanbao "元宝 / AI生成" two-line mark, bottom-right.
|
||||
- ``samsung`` -- Samsung Galaxy AI "Contenuti generati dall'AI" strip, bottom-left.
|
||||
- ``jimeng_pill`` -- Jimeng-basic "AI生成" pill, top-left (capture-less).
|
||||
- ``runninghub`` -- RunningHub "RunningHub AI生成" text, top-left (gray front-end).
|
||||
- ``baidu`` -- Baidu "百度 AI生成" text + white tag, bottom-right.
|
||||
- ``liblib`` -- LibLibAI "LibLibAI" wordmark, bottom-center.
|
||||
- ``liblib`` -- LiblibAI "LiblibAI" wordmark, bottom-center.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -620,7 +620,7 @@ _REGISTRY: tuple[KnownMark, ...] = (
|
||||
# is None so it can never act as a TC260 sibling in _keep_pill.
|
||||
KnownMark(
|
||||
"gemini",
|
||||
"Google Gemini sparkle",
|
||||
"Google Gemini visible watermark (sparkle)",
|
||||
"bottom-right",
|
||||
True,
|
||||
"gemini",
|
||||
@@ -656,9 +656,9 @@ _REGISTRY: tuple[KnownMark, ...] = (
|
||||
),
|
||||
_text_mark(
|
||||
"kling",
|
||||
"Kling 可灵AI 3.0 text",
|
||||
"Kling AI 可灵AI 3.0 text",
|
||||
"bottom-right",
|
||||
platform="Kuaishou Kling (visible 可灵AI 3.0 mark detected)",
|
||||
platform="Kuaishou Kling AI (visible 可灵AI 3.0 mark detected)",
|
||||
tc260_producer_codes=("91110108335469089C",),
|
||||
),
|
||||
_text_mark(
|
||||
@@ -693,9 +693,9 @@ _REGISTRY: tuple[KnownMark, ...] = (
|
||||
),
|
||||
_text_mark(
|
||||
"liblib",
|
||||
"LibLibAI wordmark",
|
||||
"LiblibAI wordmark",
|
||||
"bottom-center",
|
||||
platform="LibLibAI (visible LibLibAI mark detected)",
|
||||
platform="LiblibAI (visible LiblibAI mark detected)",
|
||||
tc260_producer_codes=("91110105MACJ6K1C8A",),
|
||||
),
|
||||
# Same product as the Jimeng wordmark -- the one pair that cross-relaxes.
|
||||
@@ -794,7 +794,7 @@ def tc260_producer_vendors() -> dict[str, str]:
|
||||
def _pill_suppressors() -> set[str]:
|
||||
"""Marks whose detection vetoes the capture-less pill: same label regime as the
|
||||
pill, different product. Derived so a newly registered TC260 mark cannot be
|
||||
forgotten here -- which is exactly how LibLibAI ended up missing."""
|
||||
forgotten here -- which is exactly how LiblibAI ended up missing."""
|
||||
pill = get_mark("jimeng_pill")
|
||||
return {
|
||||
m.key
|
||||
@@ -822,9 +822,9 @@ def _keep_pill(keys: set[str], *, provenance: frozenset[str], footprint_flat: bo
|
||||
No confirmation at all -> never remove (blocks false fires on non-Jimeng content).
|
||||
|
||||
The suppressor set is DERIVED from the registry (same label regime, different
|
||||
product), not hand-listed. The hand-written list had drifted: LibLibAI was
|
||||
product), not hand-listed. The hand-written list had drifted: LiblibAI was
|
||||
registered alongside RunningHub and Baidu but never added to it, so a confident
|
||||
LibLibAI detection did not veto the pill the way its two siblings did. Marks
|
||||
LiblibAI detection did not veto the pill the way its two siblings did. Marks
|
||||
outside the TC260 regime (Gemini, Samsung) are deliberately NOT suppressors --
|
||||
neither can put ``"jimeng"`` into ``provenance``, so neither can enable the arm
|
||||
they would be vetoing."""
|
||||
|
||||
@@ -58,3 +58,12 @@ def test_default_pipeline_clearance_is_recorded() -> None:
|
||||
rows = {row["name"]: row for row in _manifest_rows()}
|
||||
for name in ("fox_modal_invisible", "text_modal_invisible"):
|
||||
assert rows[name]["oracle_verdict"] == "not_detected", name
|
||||
|
||||
|
||||
def test_deterministic_transforms_reproduce_recorded_hashes(tmp_path: Path) -> None:
|
||||
from scripts.contentseal_transforms import reproduce_transforms
|
||||
|
||||
outputs = reproduce_transforms(tmp_path)
|
||||
|
||||
assert len(outputs) == 8
|
||||
assert all(path.is_file() for path in outputs)
|
||||
|
||||
@@ -34,7 +34,10 @@ DESIGNER_STORE = {
|
||||
},
|
||||
{
|
||||
"label": "c2pa.soft-binding",
|
||||
"data": {"alg": "com.microsoft.invismark.1", "blocks": [{"value": "bf7a2993-cc1f-47e1-b1f0-cd8839aabb22"}]},
|
||||
"data": {
|
||||
"alg": "com.microsoft.invismark.1",
|
||||
"blocks": [{"value": "bf7a2993-cc1f-47e1-b1f0-cd8839aabb22"}],
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
"""Cross-corpus integrity checks for tracked evaluation tables."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
EVALUATIONS = ROOT / "data" / "evaluations"
|
||||
SHA256 = re.compile(r"^[0-9a-f]{64}$")
|
||||
|
||||
|
||||
def test_every_recorded_evaluation_sha256_is_well_formed() -> None:
|
||||
checked = 0
|
||||
for path in sorted(EVALUATIONS.rglob("*.csv")):
|
||||
with path.open(newline="", encoding="utf-8") as stream:
|
||||
for line_number, row in enumerate(csv.DictReader(stream), start=2):
|
||||
for field, value in row.items():
|
||||
if field is not None and field.endswith("sha256") and value:
|
||||
assert SHA256.fullmatch(value), f"{path.relative_to(ROOT)}:{line_number} {field}={value!r}"
|
||||
checked += 1
|
||||
assert checked > 0
|
||||
@@ -703,7 +703,8 @@ class TestIdentifyRealSamples:
|
||||
)
|
||||
apple = tmp_path / "apple.jpg"
|
||||
apple.write_bytes(
|
||||
b'\xff\xd8\xff\xe1<x:xmpmeta Iptc4xmpExt:DigitalSourceType="compositeWithTrainedAlgorithmicMedia" photoshop:Credit="Apple Photos Clean Up"></x:xmpmeta>\xff\xd9'
|
||||
b'\xff\xd8\xff\xe1<x:xmpmeta Iptc4xmpExt:DigitalSourceType="compositeWithTrainedAlgorithmicMedia" '
|
||||
b'photoshop:Credit="Apple Photos Clean Up"></x:xmpmeta>\xff\xd9'
|
||||
)
|
||||
|
||||
assert identify(muse, check_visible=False, check_invisible=False).platform == (
|
||||
|
||||
@@ -60,6 +60,7 @@ class TestVerifiedTextMode:
|
||||
manifest.write_text("{}", encoding="utf-8")
|
||||
cases = (
|
||||
("sdxl-zimage", {}, "qwen-zimage"),
|
||||
("qwen-zimage", {"tile": True}, "not calibrated with --tile"),
|
||||
("qwen-zimage", {"max_resolution": 1024}, "max-resolution 0"),
|
||||
("qwen-zimage", {"humanize": 1.0}, "humanize=0"),
|
||||
("qwen-zimage", {"adaptive_polish": True}, "polish disabled"),
|
||||
@@ -128,10 +129,6 @@ class TestVerifiedTextMode:
|
||||
|
||||
assert seen["fidelity_anchor"] is True
|
||||
|
||||
engine.remove_watermark(source, output, text_manifest=manifest, tile=True)
|
||||
|
||||
assert seen["tile"] is True
|
||||
|
||||
|
||||
class TestNativeOutputSize:
|
||||
"""Model-side latent-grid rounding must not change the public output size."""
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Tests for the LibLibAI ("LibLibAI" wordmark) visible-watermark engine.
|
||||
"""Tests for the LiblibAI ("LiblibAI" wordmark) visible-watermark engine.
|
||||
|
||||
Every tuned constant in ``liblib_engine`` was measured on the 15-frame vendor
|
||||
cohort (2026-07-22); these tests pin the load-bearing ones: the bottom-CENTER
|
||||
@@ -24,7 +24,7 @@ _MARK_FRAC = 0.10 # measured wordmark width, fraction of the frame WIDTH
|
||||
|
||||
|
||||
def _compose(w: int, h: int, bg: float = 100.0):
|
||||
"""Composite a triangle logo + the LibLibAI wordmark, bottom-center."""
|
||||
"""Composite a triangle logo + the LiblibAI wordmark, bottom-center."""
|
||||
img = np.full((h, w, 3), bg, np.float32)
|
||||
at = _alpha_template()
|
||||
gw = int(_MARK_FRAC * w)
|
||||
@@ -112,8 +112,8 @@ class TestDetectAndMask:
|
||||
assert eng.footprint_mask(img) is None
|
||||
|
||||
def test_confident_liblib_detection_suppresses_the_jimeng_pill(self):
|
||||
# A LibLibAI image is TC260 too but is not Jimeng-basic: like Doubao/Qwen/
|
||||
# Kling/RunningHub/Baidu, a confident LibLibAI detection must veto the pill.
|
||||
# A LiblibAI image is TC260 too but is not Jimeng-basic: like Doubao/Qwen/
|
||||
# Kling/RunningHub/Baidu, a confident LiblibAI detection must veto the pill.
|
||||
# It was the one mark the hand-written veto list in ``_keep_pill`` missed.
|
||||
from remove_ai_watermarks.watermark_registry import _keep_pill
|
||||
|
||||
|
||||
@@ -659,8 +659,7 @@ def test_no_face_path_still_runs_verified_text_restoration(monkeypatch):
|
||||
restore.assert_called_with(source, anchor, donor, manifest.lines)
|
||||
|
||||
|
||||
def test_tiled_verified_text_runs_vae_donor_per_tile(monkeypatch):
|
||||
from remove_ai_watermarks._internal import qwen_zimage_pipeline, text_restoration
|
||||
def test_tiled_verified_text_is_rejected_before_model_work():
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import QwenZImagePipeline
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextLine, VerifiedTextManifest
|
||||
|
||||
@@ -668,12 +667,6 @@ def test_tiled_verified_text_runs_vae_donor_per_tile(monkeypatch):
|
||||
pipeline.device = "cuda"
|
||||
pipeline.progress_callback = None
|
||||
source = Image.new("RGB", (96, 80), (10, 20, 30))
|
||||
restored = Image.new("RGB", (96, 80), (130, 140, 150))
|
||||
pipeline._qwen_vae_roundtrip = MagicMock(side_effect=lambda tile: tile)
|
||||
pipeline._run_global = MagicMock(side_effect=lambda tile, _strength, _seed: tile)
|
||||
monkeypatch.setattr(qwen_zimage_pipeline, "detect_faces", lambda _image: [])
|
||||
restore = MagicMock(return_value=restored)
|
||||
monkeypatch.setattr(text_restoration, "restore_verified_text", restore)
|
||||
manifest = VerifiedTextManifest(
|
||||
"0" * 64,
|
||||
96,
|
||||
@@ -681,23 +674,16 @@ def test_tiled_verified_text_runs_vae_donor_per_tile(monkeypatch):
|
||||
(VerifiedTextLine((4, 4, 20, 16), "Exact", "alphabetic"),),
|
||||
)
|
||||
|
||||
result = pipeline.run(
|
||||
source,
|
||||
strength=0.1,
|
||||
seed=0,
|
||||
tile=True,
|
||||
tile_size=64,
|
||||
tile_overlap=16,
|
||||
text_manifest=manifest,
|
||||
)
|
||||
|
||||
assert result is restored
|
||||
assert pipeline._qwen_vae_roundtrip.call_count > 1
|
||||
assert pipeline._run_global.call_count > 1
|
||||
restore.assert_called_once()
|
||||
assert restore.call_args.args[0].size == (96, 80)
|
||||
assert restore.call_args.args[1].size == (96, 80)
|
||||
assert restore.call_args.args[2].size == (96, 80)
|
||||
with pytest.raises(ValueError, match="not calibrated with tiled diffusion"):
|
||||
pipeline.run(
|
||||
source,
|
||||
strength=0.1,
|
||||
seed=0,
|
||||
tile=True,
|
||||
tile_size=64,
|
||||
tile_overlap=16,
|
||||
text_manifest=manifest,
|
||||
)
|
||||
|
||||
|
||||
def test_watermark_remover_dispatches_to_full_pipeline(tmp_path, monkeypatch):
|
||||
@@ -757,6 +743,29 @@ def test_watermark_remover_dispatches_qwen_tiling_to_full_pipeline(tmp_path, mon
|
||||
assert output.exists()
|
||||
|
||||
|
||||
def test_watermark_remover_rejects_tiled_verified_text_before_pipeline(tmp_path, monkeypatch):
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextLine, VerifiedTextManifest
|
||||
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
|
||||
|
||||
_mock_watermark_runtime_deps(monkeypatch)
|
||||
source = tmp_path / "source.png"
|
||||
Image.new("RGB", (96, 80), (20, 30, 40)).save(source)
|
||||
manifest = VerifiedTextManifest(
|
||||
"0" * 64,
|
||||
96,
|
||||
80,
|
||||
(VerifiedTextLine((4, 4, 20, 16), "Exact", "alphabetic"),),
|
||||
)
|
||||
remover = WatermarkRemover(device="cuda", pipeline="qwen-zimage")
|
||||
runtime = MagicMock()
|
||||
monkeypatch.setattr(remover, "_load_qwen_zimage_pipeline", lambda: runtime)
|
||||
|
||||
with pytest.raises(ValueError, match="not calibrated with tiled diffusion"):
|
||||
remover.remove_watermark(source, text_manifest=manifest, tile=True)
|
||||
|
||||
runtime.run.assert_not_called()
|
||||
|
||||
|
||||
def test_qwen_tiling_runs_global_tiles_then_one_full_frame_face_stage(monkeypatch):
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import (
|
||||
QwenZImagePipeline,
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Basic command-line contracts for standalone maintainer scripts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"script",
|
||||
[
|
||||
"visible_groundtruth.py",
|
||||
"visible_recall_sample.py",
|
||||
"visible_sheets.py",
|
||||
],
|
||||
)
|
||||
def test_script_help_exits_cleanly(script: str) -> None:
|
||||
result = subprocess.run( # noqa: S603 -- fixed interpreter and repository-owned script path
|
||||
[sys.executable, str(ROOT / "scripts" / script), "--help"],
|
||||
cwd=ROOT,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert "usage:" in result.stdout
|
||||
@@ -3,12 +3,14 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import math
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from typing import TYPE_CHECKING, cast
|
||||
from typing import TYPE_CHECKING, Any, cast
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from remove_ai_watermarks import optional_deps, video_encoding, video_invisible
|
||||
@@ -92,6 +94,50 @@ def test_regeneration_rejects_noise_outside_unit_interval(tmp_path: Path) -> Non
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("kwargs", "message"),
|
||||
[
|
||||
({"fps": 0.0}, "fps must be at least 1"),
|
||||
({"batch_size": 0}, "batch_size must be at least 1"),
|
||||
({"duration": 0.0}, "duration must be positive"),
|
||||
({"device": "tpu"}, "device must be auto, cuda, mps, or cpu"),
|
||||
],
|
||||
)
|
||||
def test_regeneration_rejects_invalid_controls_before_probing(
|
||||
tmp_path: Path,
|
||||
kwargs: dict[str, Any],
|
||||
message: str,
|
||||
) -> None:
|
||||
with pytest.raises(ValueError, match=message):
|
||||
video_invisible.regenerate_video_candidate(
|
||||
tmp_path / "source.mp4",
|
||||
tmp_path / "candidate.mp4",
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
def test_fit_size_and_paired_psnr_validate_geometry() -> None:
|
||||
assert video_invisible._fit_size(1920, 1080, 512) == (512, 288)
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
video_invisible._fit_size(0, 1080, 512)
|
||||
with pytest.raises(ValueError, match="at least"):
|
||||
video_invisible._fit_size(1920, 1080, 4)
|
||||
|
||||
frame = np.zeros((2, 2, 3), dtype=np.uint8)
|
||||
assert video_invisible.paired_psnr(frame, frame) == math.inf
|
||||
with pytest.raises(ValueError, match="matching shapes"):
|
||||
video_invisible.paired_psnr(frame, frame[:1])
|
||||
|
||||
|
||||
def test_load_runtime_rejects_invalid_device_and_missing_extra(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
with pytest.raises(ValueError, match="device must be"):
|
||||
video_invisible.load_video_vae_runtime(device="tpu")
|
||||
|
||||
monkeypatch.setattr(video_invisible, "is_available", lambda: False)
|
||||
with pytest.raises(RuntimeError, match="diffusion extra"):
|
||||
video_invisible.load_video_vae_runtime()
|
||||
|
||||
|
||||
def test_encoder_and_mux_commands_separate_streaming_frames_from_source_audio(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
@@ -298,8 +344,9 @@ def test_shipped_defaults_match_a_certified_manifest_row() -> None:
|
||||
rate now fails here until a ``not_detected`` row exists for that exact triple.
|
||||
|
||||
The tuple stops at three fields because the model is a fourth thing the oracle
|
||||
was shown and neither tracked row records it. That omission is data-driven: add
|
||||
``vae`` here in the same commit as the first row that records one.
|
||||
was shown and neither historical row recorded it. The manifest says
|
||||
``unrecorded`` rather than leaving an ambiguous empty field. Add ``vae`` here in
|
||||
the same commit as the first row that records one.
|
||||
"""
|
||||
with ORACLE_MANIFEST.open(newline="", encoding="utf-8") as stream:
|
||||
certified = {
|
||||
@@ -320,6 +367,15 @@ def test_shipped_defaults_match_a_certified_manifest_row() -> None:
|
||||
)
|
||||
|
||||
|
||||
def test_oracle_manifest_marks_missing_vae_identity_explicitly() -> None:
|
||||
with ORACLE_MANIFEST.open(newline="", encoding="utf-8") as stream:
|
||||
rows = list(csv.DictReader(stream))
|
||||
|
||||
assert rows
|
||||
assert all(row["vae"] for row in rows)
|
||||
assert all(row["vae"] == "unrecorded" or "/" in row["vae"] for row in rows)
|
||||
|
||||
|
||||
def test_stream_batches_consumes_only_one_batch_ahead() -> None:
|
||||
consumed: list[int] = []
|
||||
|
||||
|
||||
@@ -529,7 +529,7 @@ class TestMarkKnowledgeIsOnTheRow:
|
||||
|
||||
Product family, label regime, the platform sentence and the metadata signals that
|
||||
confirm the vendor all used to live in separate hand-maintained tables across
|
||||
``watermark_registry``, ``identify`` and ``api``. That is how LibLibAI ended up
|
||||
``watermark_registry``, ``identify`` and ``api``. That is how LiblibAI ended up
|
||||
registered but absent from the pill veto.
|
||||
"""
|
||||
|
||||
@@ -563,7 +563,7 @@ class TestMarkKnowledgeIsOnTheRow:
|
||||
class TestPillSuppressors:
|
||||
"""The pill veto is derived from the registry, not hand-listed.
|
||||
|
||||
The hand-written list drifted: LibLibAI was registered in the same commit as
|
||||
The hand-written list drifted: LiblibAI was registered in the same commit as
|
||||
RunningHub and Baidu, both of which were added to the veto, and it was not. A
|
||||
derived set cannot be forgotten by the next registration.
|
||||
"""
|
||||
@@ -581,7 +581,7 @@ class TestPillSuppressors:
|
||||
assert not reg._keep_pill({"liblib"}, provenance=frozenset({"jimeng"}), footprint_flat=1.0)
|
||||
|
||||
def test_pill_dropped_on_liblib_even_with_the_jimeng_wordmark(self):
|
||||
"""The veto precedes the wordmark arm, so a co-firing LibLibAI wins.
|
||||
"""The veto precedes the wordmark arm, so a co-firing LiblibAI wins.
|
||||
|
||||
This is the broader half of the change: it needs neither TC260 provenance nor
|
||||
a flat footprint, so it is reachable on more inputs than the metadata arm.
|
||||
|
||||
Reference in New Issue
Block a user