mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-31 09:40:38 +02:00
Reject the uncalibrated text-manifest tiling and verify Content Seal transforms
Tiled diffusion was never provider-oracle calibrated with verified text restoration: the tiled VAE donor path ran anyway and produced results no oracle had certified. The combination is now rejected at both the pipeline and the engine seam (ValueError with the reason), and the CLI help no longer implies support. The invisible help is generalized and the metadata container list corrected (MKA/OGA/Opus/AAC). scripts/contentseal_transforms.py reproduces the deterministic crop, resize, and JPEG variants of the Content Seal corpus from manifest.csv, hash-verifying every output; its README gains scripts/README.md context and new data tests. The corpus README is honest about the one crop the daily oracle limit left unchecked, and the eval CSVs carry the updated verdicts. The byte-scan SynthID suppression hoists its soft-binding lookup so the guard is computed once. Staged on top of 0.33.1; no version bump in this commit.
This commit is contained in:
+4
-4
@@ -434,10 +434,10 @@ user can act on rather than after a model load.
|
||||
### Restore operator-verified text
|
||||
|
||||
`--text-manifest` enables the experimental `vae-glyphs` post-pass. It reconstructs
|
||||
the source with the Qwen VAE, blends 15% of that reconstruction into the normal
|
||||
`qwen-zimage` result, erases the annotated candidate glyphs with LaMa, and composites
|
||||
only the reconstructed glyph cores through source-derived silhouettes. It does not
|
||||
run OCR or choose which strings are correct.
|
||||
the source with the Qwen VAE, erases the annotated candidate glyphs with LaMa, and
|
||||
composites only the reconstructed glyph cores through source-derived silhouettes. It
|
||||
does not run OCR or choose which strings are correct. The optional `--fidelity-anchor`
|
||||
described below additionally blends 15% of the reconstruction across the full frame.
|
||||
|
||||
Install the combined extra and run only with an operator-verified manifest:
|
||||
|
||||
|
||||
@@ -641,7 +641,7 @@ photo only).
|
||||
|
||||
**Sources.** https://arxiv.org/abs/2206.11253 (CodeFormer) · https://github.com/sczhou/CodeFormer ·
|
||||
https://arxiv.org/pdf/2401.07519 (InstantID) ·
|
||||
https://openaccess.thecvf.com/content/WACV2024/papers/Suin_Diffuse_and_Restore... (region-adaptive) ·
|
||||
https://openaccess.thecvf.com/content/WACV2024/html/Suin_Diffuse_and_Restore_A_Region-Adaptive_Diffusion_Model_for_Identity-Preserving_Blind_WACV_2024_paper.html (region-adaptive) ·
|
||||
https://arxiv.org/pdf/2504.12809 (saliency-aware watermark removal)
|
||||
|
||||
## Provenance
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# Development
|
||||
|
||||
Standalone evaluation, calibration, and release utilities are catalogued in
|
||||
[`scripts/README.md`](../scripts/README.md). They are maintainer tools rather than
|
||||
installed commands, and their local data inputs remain outside the repository.
|
||||
|
||||
Read this reference for environment setup, dependency recovery, CI behavior, and fixture policy. The always-loaded invariants remain in [`.claude/rules/development.md`](../.claude/rules/development.md).
|
||||
|
||||
## Local environment
|
||||
|
||||
@@ -114,6 +114,7 @@ application actually uses:
|
||||
| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `qwen-zimage` | Invisible image-watermark removal, both CUDA-only profiles | `diffusion`, DiffSynth | Yes |
|
||||
| `text-restoration` | Opt-in verified Qwen-VAE glyph restoration | `qwen-zimage`, `lama` | Yes |
|
||||
| `text-draft` | Draft OCR proposals for operator verification | PaddleOCR, PaddlePaddle | Model download, no Torch |
|
||||
| `all` | Every production feature available on the active Python | All compatible rows above | Yes |
|
||||
| `dev` | Tests, linting, typing, and upstream parity checks | `video`, `detect`, upstream invisible-watermark | Yes, for parity tests |
|
||||
|
||||
@@ -130,12 +131,15 @@ flowchart LR
|
||||
qwen["qwen-zimage"] --> diffusion
|
||||
text["text-restoration"] --> qwen
|
||||
text --> lama
|
||||
draft["text-draft"]
|
||||
heif
|
||||
trustmark
|
||||
```
|
||||
|
||||
`heif` and `trustmark` are independent branches. Combine them explicitly with
|
||||
another feature when required. TrustMark requires NumPy 1.x, which has no
|
||||
`heif`, `trustmark`, and `text-draft` are independent branches. Combine them
|
||||
explicitly with another feature when required. `text-draft` is excluded from
|
||||
`all` because it proposes unverified OCR annotations and is not a production
|
||||
removal path. TrustMark requires NumPy 1.x, which has no
|
||||
CPython 3.13 or 3.14 wheels, so that branch is available only on Python
|
||||
3.11-3.12. The `all` bundle contains every production branch compatible with
|
||||
the active Python and never includes `dev`.
|
||||
|
||||
@@ -416,7 +416,7 @@ remain identical because isolated decoder hits can otherwise be content noise.
|
||||
It accepts Variant P schemas 0-2. Variant Q requires a different model, and
|
||||
schema 3 is rejected at the measured precision threshold. Its NumPy 1.x runtime
|
||||
limits the extra to Python 3.11-3.12; the rest of the package remains supported
|
||||
through Python 3.14. Through 0.29.0 the extra also resolves lightning 2.6.5
|
||||
through Python 3.14. The current TrustMark dependency line also resolves lightning 2.6.5
|
||||
(PYSEC-2026-3624, no fixed release yet); the vulnerable `load_from_checkpoint`
|
||||
path is unreachable here because TrustMark loads its checksummed checkpoints
|
||||
with plain `torch.load`. Bump lightning and cut a patch release when a fix
|
||||
|
||||
@@ -746,9 +746,9 @@ capture-less pill is too weak to attribute.
|
||||
|
||||
The set of marks that veto the pill is DERIVED from the registry rows: every mark
|
||||
under the same label regime (`tc260`) belonging to a different product. It used to
|
||||
be a hand-written list of keys, and that list drifted -- LibLibAI was registered
|
||||
alongside RunningHub and Baidu, both of which were added to it, and LibLibAI was
|
||||
not, so a confident LibLibAI detection did not suppress the pill the way its two
|
||||
be a hand-written list of keys, and that list drifted -- LiblibAI was registered
|
||||
alongside RunningHub and Baidu, both of which were added to it, and LiblibAI was
|
||||
not, so a confident LiblibAI detection did not suppress the pill the way its two
|
||||
siblings did. Marks outside the TC260 regime (Gemini, Samsung) are deliberately
|
||||
not vetoers: neither can put `jimeng` into `provenance`, so neither can enable the
|
||||
arm it would be vetoing.
|
||||
@@ -818,7 +818,7 @@ were briefly skipped there for exactly that reason.
|
||||
A mark whose removable footprint differs from what the detector localizes
|
||||
overrides `_footprint_rect` (which policy) and `_extend_match_box` (how far the
|
||||
box grows), not the whole `footprint_mask`. Baidu extends right to the corner tag
|
||||
and LibLibAI extends left to the triangle logo; both inherit every guard around
|
||||
and LiblibAI extends left to the triangle logo; both inherit every guard around
|
||||
that arithmetic.
|
||||
|
||||
Yuanbao uses the polarity-independent `contrast` front end because its standard
|
||||
@@ -1020,11 +1020,13 @@ Validation happens before model loading. The library never treats OCR confidence
|
||||
verification, and geometry-only operators do not need to invent text or script fields.
|
||||
|
||||
When enabled, `QwenZImagePipeline` reconstructs the source once through its already
|
||||
loaded Qwen VAE, runs the ordinary global and face stages, blends 15% of the VAE
|
||||
reconstruction into that clean result, and calls the shared restoration compositor.
|
||||
The compositor derives binary source and candidate silhouettes, groups nearby lines,
|
||||
uses LaMa for the initial and residual-glyph erase passes, paints fresh silhouette
|
||||
edges, then copies the Qwen-VAE core with a 0.5-pixel feather. The evaluation script
|
||||
loaded Qwen VAE, runs the ordinary global and face stages, and calls the shared
|
||||
restoration compositor. The optional fidelity anchor first blends 15% of the VAE
|
||||
reconstruction into the clean result; it is off by default because the blend returned
|
||||
detector-visible OpenAI SynthID in the measured poster fixtures. The compositor derives
|
||||
binary source and candidate silhouettes, groups nearby lines, uses LaMa for the initial
|
||||
and residual-glyph erase passes, paints fresh silhouette edges, then copies the Qwen-VAE
|
||||
core with a 0.5-pixel feather. The evaluation script
|
||||
imports these same mask and compositing helpers so the two implementations cannot
|
||||
silently drift. Silhouette crops start 12% of line height beyond each horizontal
|
||||
side, then expand each side independently while a foreground component anchored
|
||||
|
||||
+4
-4
@@ -644,10 +644,10 @@ engine.remove_watermark(
|
||||
Install `remove-ai-watermarks[text-restoration]`. The manifest schema and safety
|
||||
constraints are documented in the CLI guide. The engine verifies its decoded RGB
|
||||
hash before loading the diffusion models and rejects SDXL, downscaling, and
|
||||
postprocessing combinations that were not evaluated. Tiling is allowed: the VAE
|
||||
donor uses the same overlapping tiles as the global pass. `InvisibleOptions` exposes the
|
||||
same field for `remove_all`; after a visible-stage edit, the manifest must be built
|
||||
against the staged pixels rather than the pristine source.
|
||||
postprocessing combinations that were not evaluated. Tiling is also rejected because
|
||||
the combined tiled-restoration path has no provider-oracle calibration. `InvisibleOptions`
|
||||
exposes the same field for `remove_all`; after a visible-stage edit, the manifest must
|
||||
be built against the staged pixels rather than the pristine source.
|
||||
|
||||
Use manifest schema 1 for manually reviewed text plus script metadata. Automated
|
||||
operators that verify only text-region geometry should emit schema 2 lines with a
|
||||
|
||||
@@ -19,7 +19,7 @@ The `visible` command registers these mark keys:
|
||||
| `samsung` | `✦ Contenuti generati dall'AI` | Bottom left | Calibrated for the Italian text variant. |
|
||||
| `runninghub` | `RunningHub AI生成` | Top left | Strict visual and position gates. |
|
||||
| `baidu` | `百度 AI生成` | Bottom right | Detector and extended removal footprint. |
|
||||
| `liblib` | `LibLibAI` | Bottom center | Includes a minimum image size gate. |
|
||||
| `liblib` | `LiblibAI` | Bottom center | Includes a minimum image size gate. |
|
||||
| `jimeng_pill` | `AI生成` pill | Top left | Weak detector with additional product and background gates. |
|
||||
|
||||
`--mark auto` evaluates all registered marks and removes every selected match.
|
||||
@@ -226,7 +226,7 @@ not a universal clean verdict.
|
||||
| Kling | Kling image and video marks | No registered pixel decoder | TC260 AIGC |
|
||||
| Hailuo / MiniMax video | Hailuo composite video label | No registered pixel decoder | TC260 AIGC where present |
|
||||
| Baidu | Baidu mark | No registered pixel decoder | TC260 AIGC |
|
||||
| LibLibAI | LibLibAI mark | No registered pixel decoder | TC260 AIGC |
|
||||
| LiblibAI | LiblibAI mark | No registered pixel decoder | TC260 AIGC |
|
||||
| RunningHub | RunningHub mark | No registered pixel decoder | TC260 AIGC |
|
||||
| Samsung Galaxy AI | One locale specific mark | No registered pixel decoder | C2PA and Samsung markers |
|
||||
|
||||
|
||||
@@ -58,7 +58,7 @@ defines two more native serializations. AVI stores an `AIGC` child in
|
||||
bounded RIFF and FLV readers validate the JSON field set and skip media
|
||||
payloads. Removal remuxes either container through ffmpeg with stream copy.
|
||||
|
||||
- **HuggingFace-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** HuggingFace Jobs / Spaces can stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID. It marks the *hosting job*, not a model, so it lifts an Unknown verdict to a tentative AI via `hf_only` but never overrides a hard metadata signal. `_HF_JOB_CAVEAT` states the limit. Removal drops the chunk through the PNG metadata whitelist.
|
||||
- **Hugging Face-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** Hugging Face Jobs / Spaces can stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID. It marks the *hosting job*, not a model, so it lifts an Unknown verdict to a tentative AI via `hf_only` but never overrides a hard metadata signal. `_HF_JOB_CAVEAT` states the limit. Removal drops the chunk through the PNG metadata whitelist.
|
||||
- **No detectable signal on some downloads:** Recraft exports and some hosted
|
||||
FLUX surfaces can arrive without a supported local signal. Midjourney samples
|
||||
may carry IPTC metadata but no registered C2PA or pixel watermark. The open
|
||||
|
||||
Reference in New Issue
Block a user