Compare commits

...
58 Commits
Author SHA1 Message Date
Victor Kuznetsov 86ce55ced6 Release 0.34.0 2026-08-27 21:08:07 -07:00
Victor Kuznetsov ee561308ef Keep maintainer help portable on Windows 2026-08-27 20:09:54 -07:00
Victor Kuznetsov a8d4bce14a Align watermark content with detector coverage 2026-08-27 20:05:03 -07:00
Victor Kuznetsov 17408b958e Reject the uncalibrated text-manifest tiling and verify Content Seal transforms
Tiled diffusion was never provider-oracle calibrated with verified text
restoration: the tiled VAE donor path ran anyway and produced results no
oracle had certified. The combination is now rejected at both the
pipeline and the engine seam (ValueError with the reason), and the CLI
help no longer implies support. The invisible help is generalized and
the metadata container list corrected (MKA/OGA/Opus/AAC).

scripts/contentseal_transforms.py reproduces the deterministic crop,
resize, and JPEG variants of the Content Seal corpus from manifest.csv,
hash-verifying every output; its README gains scripts/README.md context
and new data tests. The corpus README is honest about the one crop the
daily oracle limit left unchecked, and the eval CSVs carry the updated
verdicts. The byte-scan SynthID suppression hoists its soft-binding
lookup so the guard is computed once.

Staged on top of 0.33.1; no version bump in this commit.
2026-08-27 16:53:34 -07:00
Victor Kuznetsov 834342db07 Follow the seal bet in the Likely-source line
A standalone-tag file showed 'platform not specified' while the panel
below priced the Muse Content Seal removal - the platform line lagged
the attribution the seal signal and the strength router already make.
The fallback now attributes 'Meta Muse Image (attributed by the
standalone AI digital-source tag)' when no more specific attribution
applies, so the source line and the priced target agree. Apple Photos
Clean Up keeps its own attribution; the hedge names the attribution
basis, not a detection.
2026-08-27 16:41:28 -07:00
Victor Kuznetsov dcc1bf5e07 Emit the Content Seal as its own signal on standalone-AI-tag files
The strength router already bets that a file whose only provenance is
the standalone AI digital-source tag is Meta Muse output (C2PA issuers
win first), and Muse stamps every output with the invisible Content
Seal. identify now surfaces that same bet as evidence: the additive
content_seal signal (medium confidence - an attribution, not a decode,
since no public decoder exists) plus the watermark string, emitted on
exactly the standalone_iptc condition that routes the cohort. Clients
select pixel removal from the signal list instead of parsing caveats,
the way InvisMark is additive over soft_binding. The API's invisible
gate already runs on ai_from_metadata, so all/invisible behavior is
unchanged; has_invisible_target needs no edit.
2026-08-27 16:25:11 -07:00
Victor Kuznetsov d8fcd0f79b Stop double-counting a named forensic mark as SynthID provenance
A manifest that names its own forensic soft-binding algorithm carries
that vendor's mark; the generic watermark-action vendor-token inference
must not add a second, differently-attributed invisible watermark from
the same bytes. Microsoft Designer manifests triggered exactly that:
signed by Microsoft, watermarked by InvisMark, with the generation
agent named "Azure OpenAI ImageGen" - the OpenAI issuer token inside
that service name plus the InvisMark watermarked action satisfied the
OpenAI SynthID-evidence rule, and identify reported one forensic mark
as two paid pixel watermarks.

Three changes, one rule at every inference site (the verdict-scan
comment's own lesson: a rule that lives in only one copy is a rule the
others silently lack):

- c2pa.py structured path: SynthID evidence now scopes to the
  signer/generator identity strings only (signature issuer, claim
  generator), never the raw chain, and is suppressed entirely when a
  soft-binding algorithm is named.
- c2pa.py byte fallback and metadata.py synthid_source: suppressed when
  the scan names a soft-binding algorithm.
- identify.py verdict scan: same suppression.

Gemini and ChatGPT originals keep their provenance-asserted SynthID
strings; the Designer regression is pinned by
test_designer_synthid_suppression.py (agent name alone is not the
vendor's provenance, and a named soft binding suppresses the
inference).
2026-08-27 15:20:45 -07:00
Victor Kuznetsov 0a3b227f2c Merge pull request #87 from wiltodelta/dependabot/uv/minor-and-patch-c54089dda1
Bump the minor-and-patch group with 5 updates
2026-08-27 11:50:53 -07:00
Victor Kuznetsov f91f9fd9fc Record the public Microsoft Content Provenance check URL
The InvisMark strength ladder was measured against Microsoft's public
Content Provenance page, but no doc or comment carried its URL - the
API how-to link was the only address recorded anywhere, and the page
is what a human can actually check without an Azure account. Record
https://ai.azure.com/nextgen/validate in supported-signals.md, the
watermarking landscape, and the strength derivation comment, with the
honest caveat that its collapsed verdict tops out at Inconclusive
rather than the API's separate watermark-negative result.
2026-08-27 11:21:08 -07:00
Victor KuznetsovandClaude Fable 4.5 c4aecfbbdf Release 0.32.0
Co-Authored-By: Claude Fable 4.5 <noreply@anthropic.com>
2026-08-26 23:03:04 -07:00
Victor KuznetsovandClaude Fable 4.5 ab528ec0e8 Ship the measured Meta Content Seal cohort with auto routing and --vendor override
Full Meta Muse Image support in the invisible-removal path:

- QWEN_ZIMAGE_META_STRENGTH = 0.1: derived by the standard
  worst-boundary-plus-cross-source-spread method over five oracle-bracketed
  generations (data/contentseal/manifest.csv)
- Auto mode: vendor_for_strength routes a file whose only provenance is the
  standalone AI IPTC trainedAlgorithmicMedia tag onto the meta cohort; C2PA
  issuers win first, so Google/OpenAI/Microsoft routing is unchanged. Muse
  WebP outputs place the XMP in a tail chunk, so the scan uses the shared
  chunk-aware metadata.scan_head rather than a plain head read
- Explicit override: --vendor on invisible/all/batch and
  InvisibleOptions.vendor name the cohort on stripped files; naming a cohort
  asserts the watermark is present, so the no-signal gate treats it like
  --force at both the CLI and API seams
- sdxl-zimage has no measured Meta rung: an explicit meta vendor falls to
  the conservative unknown 0.25 rather than inventing one
- identify emits a Content Seal caveat pointing at the removal path
- The legacy visible 'Imagined with AI' mark stays unregistered: a dedicated
  sample hunt (newsroom mockups, community posts, press screenshots, dead
  imagine.meta.com, broken Wayback captures) found no pixel-verifiable
  capture, and the registry rule forbids encoding a corner without one.
  erase --region remains its removal path; outcome recorded in the landscape

Co-Authored-By: Claude Fable 4.5 <noreply@anthropic.com>
2026-08-26 22:59:23 -07:00
Victor KuznetsovandClaude Fable 4.5 a1811b6221 Release 0.31.2
Documentation and corpus release: Meta Muse Image Content Seal support
records (oracle-verified boundaries, wire format, removal verification)
plus the contentseal manifest guard. No runtime behavior change.

Co-Authored-By: Claude Fable 4.5 <noreply@anthropic.com>
2026-08-26 21:11:40 -07:00
Victor KuznetsovandClaude Fable 4.5 e09104e39f Document Meta Muse Image Content Seal support with oracle-verified corpus
Muse Image (muse-image-1.0, Meta Superintelligence Labs, 2026-07-07) ships
every output with Content Seal, a proprietary invisible pixel watermark, and
no visible mark. Establish support documentation and a verified corpus:

- data/contentseal/: five own generations via the Meta Model API, every
  oracle verdict recorded in manifest.csv (44 rows, settled-text protocol,
  fresh-navigation variant for calibration rows)
- Oracle: meta.ai/identification web tool only; no API endpoint exists in
  the Meta Model API (verified against dev.meta.ai/docs); internal REST
  pair documented with its server-side sliding-window rate limit
- Removal: default qwen-zimage profile clears Content Seal (oracle-verified
  on the worst source); strength floor derived at 0.1 by the standard
  worst-boundary-plus-cross-source-spread method, recorded but not encoded
  as a constant since no provenance signal routes Muse output onto a vendor
  cohort
- Seal robustness measured: survives resize, JPEG q85, metadata stripping,
  CDN WebP transcode; dies to center crops and diffusion regeneration
- tests/test_contentseal_corpus.py guards manifest integrity

Co-Authored-By: Claude Fable 4.5 <noreply@anthropic.com>
2026-08-26 21:08:47 -07:00
Victor Kuznetsov 29f17105de Release 0.31.1 with calibrated provider strengths 2026-08-26 11:22:13 -07:00
dependabot[bot] 2810b53585 Bump the minor-and-patch group with 5 updates
Bumps the minor-and-patch group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [diffusers](https://github.com/huggingface/diffusers) | `0.39.0` | `0.40.0` |
| [transformers](https://github.com/huggingface/transformers) | `5.15.0` | `5.15.1` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.27.0` | `1.28.0` |
| [onnxruntime](https://github.com/microsoft/onnxruntime) | `1.28.0` | `1.29.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.3` | `0.16.4` |


Updates `diffusers` from 0.39.0 to 0.40.0
- [Release notes](https://github.com/huggingface/diffusers/releases)
- [Commits](https://github.com/huggingface/diffusers/compare/v0.39.0...v0.40.0)

Updates `transformers` from 5.15.0 to 5.15.1
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.15.0...v5.15.1)

Updates `huggingface-hub` from 1.27.0 to 1.28.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.27.0...v1.28.0)

Updates `onnxruntime` from 1.28.0 to 1.29.0
- [Release notes](https://github.com/microsoft/onnxruntime/releases)
- [Changelog](https://github.com/microsoft/onnxruntime/blob/main/docs/ReleaseNotesWorkflow.md)
- [Commits](https://github.com/microsoft/onnxruntime/compare/v1.28.0...v1.29.0)

Updates `ruff` from 0.16.3 to 0.16.4
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.3...0.16.4)

---
updated-dependencies:
- dependency-name: diffusers
  dependency-version: 0.40.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: transformers
  dependency-version: 5.15.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.28.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: onnxruntime
  dependency-version: 1.29.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-26 08:48:21 +00:00
Victor KuznetsovandClaude Opus 5 ba5710836b Release 0.31.0
Minor, not patch: the C2PA confidence mapping changed. A cryptographically bound
manifest now reports high confidence where 0.27.0 through 0.30.1 reported medium,
because the previous gate required a trust anchor no installation has. Consumers
that branch on `confidence` will see verdicts move on files whose bytes did not.

The provenance report stays at schema 1. No field was added, removed, renamed or
retyped, and the meaning of `confidence` is unchanged -- the value it carried for a
verified claim was wrong. Bumping the output schema instead would break every
pinned schema-1 consumer on a corrected value rather than a new shape.

pre-commit: 1) maintain.sh - exit 1 on uv-secure, lightning PYSEC-2026-3624 unchanged from 0.30.1, no fixed release exists, vulnerable API unreachable (no load_from_checkpoint in project or trustmark); ruff, ruff format, pyright src/, and 1394 tests passed separately; 2) /simplify - version bump only; 3) docs sync - no version refs outside pyproject, __init__.py, uv.lock; 4) CLAUDE.md - no change

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 20:11:25 -07:00
Victor KuznetsovandClaude Opus 5 c927560614 Stop gating C2PA confidence on a trust anchor that never ships
High-confidence C2PA attribution required signingCredential.trusted, a status code
the reader emits only when a trust anchor list is loaded. None ships, so from 0.27.0
through 0.30.0 the branch was unreachable in production for every vendor: an intact,
cryptographically bound manifest scored the same medium as a fallback parse that
validated nothing, which collapsed the one distinction the official reader exists to
draw. A hand-built info dict stamping that code kept the branch green in the suite.

Confidence now follows the binding. Signer trust and certificate expiry stay visible
as their own dimensions and as caveats, because a trust list that was never
configured is a missing input, not a finding against the credential. Every committed
provenance fixture with a reader result and an intact binding now reaches high
confidence, and test_no_committed_fixture_reports_a_trusted_signer guards the
reachability itself rather than a synthesized status set.

Revocation joins binding and signature failures as disqualifying. It arrives only on
signer_validity, so a check reading the other two returned a confident AI verdict off
a credential the issuer had disowned, with an empty integrity_clashes -- quieter than
a hash mismatch on the same file. Expiry stays non-disqualifying: it does not imply
the signed bytes changed, and a signature genuinely made outside validity already
arrives as claimSignature.outsideValidity.

The rule now lives in one place. _validation_fields maps status codes to the four
dimensions and names the failures that moved one; c2pa_info_has_invalid_credential
maps dimensions to disqualified. The ingredient-reachability walk and the
user-visible reason both consume that path instead of re-classifying raw codes, so
adding this one rule no longer means editing three layers in lockstep.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 20:04:25 -07:00
Victor Kuznetsov f83da5aee4 Expose Microsoft InvisMark removal signal 2026-08-25 18:52:26 -07:00
Victor Kuznetsov 251ba9ac79 Record the ComfyUI sync PyPI CDN race and its rerun recovery
pre-commit: 1) maintain.sh - docs-only change, full gate ran this session with identical code state (exit 1 on the known lightning triage, rest green); 2) /simplify - docs only; 3) docs sync - this is the sync; 4) CLAUDE.md - no change
2026-08-25 17:17:17 -07:00
Victor Kuznetsov c5764661b4 Release 0.30.0
pre-commit: 1) maintain.sh - exit 1 on uv-secure, lightning PYSEC-2026-3624 unchanged from 0.29.0, no fixed release exists, vulnerable API unreachable (no load_from_checkpoint in project or trustmark); ruff, pyright src/, and 1391 tests passed separately; 2) /simplify - version bump only; 3) docs sync - no version refs outside pyproject, __init__.py, uv.lock; 4) CLAUDE.md - no change
2026-08-25 16:52:27 -07:00
Victor Kuznetsov 97cba11efe Read the InvisMark soft-binding algorithm and its signed watermark id
Reachable c2pa.soft-binding assertions now surface their exact alg and the
bounded printable block value next to the normalized vendor label; a value
without its algorithm stays hidden because it cannot be attributed.
com.microsoft.invismark.1 uses that value as the pixel-watermark identifier
in Microsoft Paint output, so identify names it, metadata carries it, and an
InvisMark soft binding keeps the invisible-removal gate fail-safe even after
C2PA asset binding goes invalid. Content fingerprints still do not trigger
pixel regeneration. Removal is verified against Microsoft's Content
Provenance Detection API, which reports Watermark and C2PA separately; the
protocol and the pixel-identical control requirement are documented.

Implemented in a parallel session; verified, gated, and committed by pi.

pre-commit: 1) maintain.sh - exit 1 on the known uv-secure lightning PYSEC-2026-3624 triage (no fix available, unchanged from 0.29.0); ruff, pyright src/, and 1391 tests passed separately; 2) /simplify - single-pass, clean; 3) docs sync - five docs updated by the author session, no remaining references found; 4) CLAUDE.md - invariants recorded in module-internals, no change needed
2026-08-25 15:54:26 -07:00
Victor Kuznetsov 809d980fb7 Describe the real Hugging Face Space release mechanism
The Space demo is deployed from wiltodelta/raiw-hf-space: its uv lock pins the
library version, its sync workflow mirrors the files, and a factory reboot
alone re-installs the pinned version rather than upgrading. Document the manual
pin bump, the smoke-test CI in that repository, and live-call verification of
the running Space.

pre-commit: 1) maintain.sh - docs-only change, full gate ran this session with identical code state (exit 1 on the known lightning triage, rest green); 2) /simplify - docs only; 3) docs sync - this is the sync, no other references found; 4) CLAUDE.md - existing Space mention remains accurate
2026-08-25 15:00:24 -07:00
Victor Kuznetsov 10da166f3e Record the release-verification uv cache caveat and the pending lightning bump
pre-commit: 1) maintain.sh - docs-only change, last full run minutes ago: exit 1 on the known uv-secure lightning triage, ruff/pyright/1386 tests green; 2) /simplify - docs only; 3) docs sync - these are the doc updates; 4) CLAUDE.md - no change
2026-08-25 13:36:50 -07:00
Victor Kuznetsov 8871eb3f67 Release 0.29.0
pre-commit: 1) maintain.sh - exit 1 on uv-secure, lightning PYSEC-2026-3624 unchanged from 0.28.1, no fixed release exists, vulnerable API unreachable (no load_from_checkpoint in project or trustmark); ruff, pyright src/, and 1386 tests passed separately; 2) /simplify - version bump only; 3) docs sync - no version refs outside pyproject, __init__.py, uv.lock; 4) CLAUDE.md - no change
2026-08-25 13:25:33 -07:00
Victor Kuznetsov 44ec203c22 Drop the stale untiled restriction from text restoration docs 2026-08-25 13:22:49 -07:00
Victor Kuznetsov 4beb61463a Say metadata check and strip results are not a clean verdict 2026-08-25 13:22:38 -07:00
Victor Kuznetsov 0d11e4099c Fix adaptive text restoration crop bounds 2026-08-25 12:44:05 -07:00
Victor Kuznetsov fa83c1444f Add geometry-only text manifest schema 2026-08-25 11:33:21 -07:00
Victor Kuznetsov f52e40ae24 Name Copilot, Dreamina, and Qwen in C2PA source labels 2026-08-21 14:43:39 -07:00
Victor Kuznetsov 0aa03c4e89 Release 0.28.1
pre-commit: 1 maintain.sh - uv-secure lightning PYSEC-2026-3624 known, no fix, same as 0.28.0; ruff/pyright/1378 tests passed; 2) /simplify - version bump only; 3) docs sync - no refs; 4) CLAUDE.md - no change
EOF
)
2026-08-19 21:46:14 -07:00
Victor Kuznetsov 91aa49b03a Allow verified-text restoration on tiled images
Tile the Qwen VAE donor the same way as the global pass. Glyph restore already runs on the blended full frame, so the old tile ban was an artificial gate.
2026-08-19 21:33:22 -07:00
Victor Kuznetsov ee865e95e4 Add geometry draft mode and pad silhouette descenders
pre-commit: 1 maintain.sh - not rerun full, ruff + 34 targeted tests passed; 2) /simplify - pad lives in silhouette crop, stable=False reuses probes; 3) docs sync - no README/cli refs to update; 4) CLAUDE.md - no change
EOF
)
2026-08-19 19:53:38 -07:00
Victor Kuznetsov d84fd1a125 Merge pull request #86 from wiltodelta/dependabot/uv/minor-and-patch-267694dd16
Bump the minor-and-patch group across 1 directory with 7 updates
2026-08-19 12:22:54 -07:00
Victor Kuznetsov e315eacb14 Document automated-operator verified manifests
verified:true may attest machine-verified geometry (stability-gated
boxes inside caps) set by a service; the pipeline consumes box/script
only, text is advisory, so the verification burden is on the geometry.
2026-08-19 12:21:32 -07:00
Victor Kuznetsov 3ba4ace7e0 Disable mkldnn in the draft engines (linux PIR crash) 2026-08-19 10:22:58 -07:00
dependabot[bot] 13888704f8 Bump the minor-and-patch group across 1 directory with 7 updates
Bumps the minor-and-patch group with 7 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [python-dotenv](https://github.com/theskumar/python-dotenv) | `1.2.2` | `1.2.3` |
| [c2pa-python](https://github.com/contentauth/c2pa-python) | `0.37.5` | `0.37.7` |
| [av](https://github.com/PyAV-Org/PyAV) | `18.0.0` | `18.1.0` |
| [transformers](https://github.com/huggingface/transformers) | `5.14.1` | `5.15.0` |
| diffsynth | `2.1.0` | `2.1.2` |
| [paddleocr](https://github.com/PaddlePaddle/PaddleOCR) | `3.4.1` | `3.7.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.2` | `0.16.3` |



Updates `python-dotenv` from 1.2.2 to 1.2.3
- [Release notes](https://github.com/theskumar/python-dotenv/releases)
- [Changelog](https://github.com/theskumar/python-dotenv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/theskumar/python-dotenv/compare/v1.2.2...v1.2.3)

Updates `c2pa-python` from 0.37.5 to 0.37.7
- [Release notes](https://github.com/contentauth/c2pa-python/releases)
- [Changelog](https://github.com/contentauth/c2pa-python/blob/main/docs/release-notes.md)
- [Commits](https://github.com/contentauth/c2pa-python/compare/v0.37.5...v0.37.7)

Updates `av` from 18.0.0 to 18.1.0
- [Release notes](https://github.com/PyAV-Org/PyAV/releases)
- [Changelog](https://github.com/PyAV-Org/PyAV/blob/v18.1.0/CHANGELOG.rst)
- [Commits](https://github.com/PyAV-Org/PyAV/compare/v18.0.0...v18.1.0)

Updates `transformers` from 5.14.1 to 5.15.0
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.14.1...v5.15.0)

Updates `diffsynth` from 2.1.0 to 2.1.2

Updates `paddleocr` from 3.4.1 to 3.7.0
- [Release notes](https://github.com/PaddlePaddle/PaddleOCR/releases)
- [Commits](https://github.com/PaddlePaddle/PaddleOCR/compare/v3.4.1...v3.7.0)

Updates `ruff` from 0.16.2 to 0.16.3
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.2...0.16.3)

---
updated-dependencies:
- dependency-name: python-dotenv
  dependency-version: 1.2.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: c2pa-python
  dependency-version: 0.37.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: av
  dependency-version: 18.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: transformers
  dependency-version: 5.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: diffsynth
  dependency-version: 2.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: paddleocr
  dependency-version: 3.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-19 17:19:11 +00:00
Victor Kuznetsov e938b57f8c Package the verified-text draft annotator as text_draft
Move the draft-annotation logic (PaddleOCR line detection, word-box
grouping, three script-chosen recognition engines, crop-jitter
stability gate) from the evaluation-only scripts into the installable
package, with lazy paddle imports and a text-draft extra (CPU, no
torch). draft_text_lines() returns accepted (crop-stable, NEVER
ground-truth-correct - precision on the reference posters was 90.0% /
94.4%) and rejected proposals; source_pixel_sha256 is re-exported for
manifest building. scripts/infer_text_lines.py now dogfoods the
package module instead of loading the eval script by path.
2026-08-19 10:13:22 -07:00
Victor Kuznetsov 3cd1e47935 Add the oracle-measured Google strength floor to qwen-zimage
The resolution curve's 0.154 top left the 4.33 MP CJK-sign fixture
SynthID-detected x3 in Gemini on the full production path (visible
stage -> qwen-zimage seed 0 -> resize-back -> metadata strip), with a
valid pixel-identical stripped control in the same session
(2026-08-18). Google-provenance content now resolves to the flat
QWEN_ZIMAGE_GOOGLE_STRENGTH 0.30 floor instead of the area curve -
anchors at 0.30 measured clean in Gemini on two fixtures (CJK sign +
18-face) at 3/3 checks across two work accounts, and stayed clean
under the vae-glyphs donor layer. openai/unknown content keeps the
curve; an explicit strength still wins.
2026-08-18 22:34:41 -07:00
Victor Kuznetsov 2298282b8b Thread fidelity_anchor through WatermarkRemover
The engine forwards the new kwarg into WatermarkRemover.remove_watermark,
which rejected it with TypeError on the real (non-fake) path - caught by
the deployed Modal smoke test, not by the unit fakes.
2026-08-18 21:32:57 -07:00
Victor Kuznetsov 42d427731f Turn the verified-text fidelity anchor off by default
The whole-frame 15% Qwen-VAE blend returned detector-visible OpenAI
SynthID on poster-scale manifests through the engine text-manifest
path (official Content Provenance API, 2026-08-19: restored detected
x6 with the anchor, clean x6 without it; base outputs clean x6;
pixel-identical stripped controls detected, proving the pixel channel).
Add fidelity_anchor=False to remove_watermark and InvisibleOptions and
--fidelity-anchor on the CLI to reproduce the 0.27.0 research
behavior. Text-box MAE cost of the new default is under one point on
all three fixtures (11.60->11.72, 7.79->7.86, 7.57->8.13).
2026-08-18 21:23:25 -07:00
Victor Kuznetsov 50d9f44829 Guide text-heavy content away from forced diffusion scrubs
Issue #84: text, tables, and UI screenshots are the worst case for
--force regeneration. Document that the default pipeline skips the
invisible stage without a detectable signal, that visible/metadata
stages never redraw glyphs, and why paste-back is not offered.
2026-08-18 11:03:53 -07:00
Victor Kuznetsov 2e1428406e Release 0.27.0 2026-08-18 11:02:18 -07:00
Victor Kuznetsov 21ce1dfadd Cover QuickTime TC260 placements, Luma AI tags, and MiniMax video provenance 2026-08-17 19:38:37 -07:00
Victor Kuznetsov 2369d351a9 Merge pull request #83 from wiltodelta/chore/text-preservation-evaluation
Add verified text restoration
2026-08-15 12:30:27 -07:00
Victor Kuznetsov 432b63b6d7 Add verified text restoration 2026-08-15 12:25:35 -07:00
Victor Kuznetsov 8c00525946 Evaluate selective text restoration 2026-08-15 12:20:25 -07:00
Victor Kuznetsov f8003f505a Merge pull request #82 from wiltodelta/agent/c2pa-validation
Validate C2PA credentials before attribution
2026-08-15 11:38:47 -07:00
Victor Kuznetsov 2eab24a2e1 Validate C2PA credentials before attribution 2026-08-15 11:31:59 -07:00
Victor Kuznetsov 8201ada070 Merge pull request #81 from wiltodelta/docs/video-synthid-parked-state
Record the video SynthID measurements and park the investigation
2026-08-15 11:11:05 -07:00
Victor KuznetsovandClaude Opus 5 b7d44ff1a3 Warn users that the invisible video path leaves audio untouched
The finding lived only in the research archive, whose own banner says it may
describe prototypes that were later removed. It belongs where users read
limitations: the path strips every metadata marker while copying the audio
bitstream byte for byte, so a clean local report on a clip with generated audio
is unproven rather than a guarantee.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 11:08:09 -07:00
Victor KuznetsovandClaude Opus 5 e8f65022cc Record the video SynthID measurements and park the investigation
Everything here is evidence and state, not behavior: the shipped operating point
is untouched at 512 px / 12 fps / noise_std=0.15.

The carrier was downloaded and its sha256 matched the manifest, so its properties
are now measured rather than assumed: 1920x1080 at 24 fps with an AAC audio track.
Those three fields go into the two 2026-07-31 rows, which could not previously
tell a reader what the downscale factor even was.

The geometry prize is measured end to end for the first time. Against the
untouched source, 1024 scores +3.46 dB over the shipped 512, while the entire
noise_std axis is worth 1.92 dB. Two readings that the table alone hides are
recorded with it: the temporal residual IMPROVES with resolution, because the
shared noise field lives on the latent grid and is four times finer relative to
the frame at 1920; and the frame-rate arm cannot be judged by these metrics at
all, since they price its cost and not the smoothness it buys.

Two findings that outrank the quality question. The certified row does not
reproduce -- a rerun of the same configuration on a different device and dtype
gives a different hash and different metrics, and the manifest records neither.
And the pipeline copies audio byte for byte while stripping every metadata
marker, so it can emit a file our own detector calls clean with an untouched
Google-generated audio track inside it. The mechanism is proven on two carriers;
whether that audio carries a mark the verifier reads is not, and only the oracle
can say.

The six prepared oracle submissions were never run: file upload to the verifier
failed at the tool level. Their artifacts are gone with the scratch directory,
which is the intended lifecycle for generated media, and the document says what
rebuilding costs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 11:01:14 -07:00
Victor KuznetsovandClaude Opus 5 4855586834 Halve decoder memory with strip processing and a raveled Haar pass
Each Haar pass is one flat pywt.downcoef call over a raveled strip instead of
pywt.dwt(..., axis=1)[0], and the plane is walked in strips so no full-plane
float64 intermediate exists. Exact only while the last axis is even, so
_approximation raises on an odd width rather than returning wrong bits, and
TestRaveledHaarPass pins both that raise and the downcoef/dwt equivalence a
pywt upgrade could take away.

Drops the block constructor knob: the fold chains are written for 4, nothing
ever passed another value, and a knob that silently decodes wrong is worse
than no knob.

Peak RSS 111 MB to 21 MB on a 4.3 MP image; the decoder itself 0.011s to
0.007s, which is only 0.4% of identify() now that it is under 2% of the run.
Output bits and detector verdicts over 200 sampled data/ images, two
synthesized carriers and eight degenerate shapes are byte-identical to the
pre-vectorization decoder.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 13:33:34 -07:00
Victor KuznetsovandClaude Opus 5 37789e02f6 Vectorize the DWT-DCT decode path, 15x on the decoder
Output stays bit-identical: decoder bits and detector verdicts recorded over
200 sampled data/ images plus two synthesized carriers before and after, and
the record is byte-identical.

Measured on a 1536x2816 image -- decoder 0.280s to 0.016s, warm identify()
1.757s to 1.365s with both arms timed in one process.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 11:59:56 -07:00
Victor Kuznetsov 2d03a00a39 Merge pull request #80 from wiltodelta/dependabot/uv/minor-and-patch-2677c98c32
Bump the minor-and-patch group across 1 directory with 5 updates
2026-08-12 20:49:38 -07:00
dependabot[bot] acbf6afac3 Bump the minor-and-patch group across 1 directory with 5 updates
Bumps the minor-and-patch group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [c2pa-python](https://github.com/contentauth/c2pa-python) | `0.37.2` | `0.37.5` |
| diffsynth | `2.0.18` | `2.1.0` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.26.0` | `1.27.0` |
| [packaging](https://github.com/pypa/packaging) | `26.2` | `26.3` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.1` | `0.16.2` |



Updates `c2pa-python` from 0.37.2 to 0.37.5
- [Release notes](https://github.com/contentauth/c2pa-python/releases)
- [Changelog](https://github.com/contentauth/c2pa-python/blob/main/docs/release-notes.md)
- [Commits](https://github.com/contentauth/c2pa-python/compare/v0.37.2...v0.37.5)

Updates `diffsynth` from 2.0.18 to 2.1.0

Updates `huggingface-hub` from 1.26.0 to 1.27.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.26.0...v1.27.0)

Updates `packaging` from 26.2 to 26.3
- [Release notes](https://github.com/pypa/packaging/releases)
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pypa/packaging/compare/26.2...26.3)

Updates `ruff` from 0.16.1 to 0.16.2
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.1...0.16.2)

---
updated-dependencies:
- dependency-name: c2pa-python
  dependency-version: 0.37.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: diffsynth
  dependency-version: 2.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: packaging
  dependency-version: '26.3'
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-12 22:57:56 +00:00
Victor Kuznetsov 25eaa7458c Support Python 3.11 through 3.14 2026-08-12 15:52:11 -07:00
Victor KuznetsovandClaude Opus 5 58146dbc6d Say what the hosted service is instead of offering it as a fallback
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:24:56 -07:00
Victor KuznetsovandClaude Opus 5 b08899dc00 Note that the package name is the published distribution
Renaming it strands every pip install and every pinned dependency on it,
including the ComfyUI node package, and the console script has the same
property since users have it on PATH. Recorded at the name itself, where
whoever considers changing it will be looking.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 13:52:48 -07:00
128 changed files with 8770 additions and 1874 deletions

No files matched your search

+23
View File
@@ -65,6 +65,15 @@ that is what actually happened.
Use availability checks only for paths that actually load large models.
A verdict branch must be reachable with the dependency stack as shipped. High-confidence
C2PA attribution required `signingCredential.trusted`, a code the reader emits only when a
trust anchor list is loaded, and none ships -- so the branch was dead in production for
every vendor while a hand-built info dict stamping that code kept it green from 0.27.0
through 0.30.0. Assert reachability against the committed fixtures rather than a synthesized
status set; `TestIdentifyRealSamples::test_no_committed_fixture_reports_a_trusted_signer`
is that guard. Read a capability the stack does not have as a missing input, never as a
negative finding.
## One measurement, one gate seam
A detector is split into a trust-level-blind scan and a verdict that applies the
@@ -98,6 +107,20 @@ record is byte-identical, and a green test suite does not establish that on its
change that is meant to FIX detection is the exception that proves the rule: the diff
must then be exactly the files you intended to change, named in advance.
When the baseline is a published release rather than the previous commit, get it with
`uv run --isolated --no-project --with 'remove-ai-watermarks[heif]==<version>' python <script>`:
it resolves that release from PyPI without touching the working tree or the editable
install. Quote the extra -- bare `pkg[extra]` is a glob in zsh.
Two corollaries in `dwt_dct.py`, where "close enough" has an exact meaning. The bit test
is `peak % 36 > 18.0` and for uint8 input the exact Haar value is a multiple of 0.5, so it
lands ON the threshold once per 72 blocks and a 1-ulp difference flips real bits: leave the
transform to `pywt` however slow it looks, because its C convolution contracts into an FMA
that numpy has no ufunc for (`docs/module-internals.md` carries what that costs). And
`tests/test_invisible_watermark.py` is `skipif(not is_available())` -- without the `detect`
extra the upstream-parity test never runs, so a green suite is not evidence here and the
verdict record is not optional.
## A certified operating point is data, not a constant
The video SynthID default is only meaningful as a row in
+1 -1
View File
@@ -4,7 +4,7 @@ name: Distribute on release
# PyPI is handled by publish.yml. This workflow event-drives the three channels
# that would otherwise be manual:
# - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
# - HF Space: factory-rebuild so it reinstalls the latest sdist from PyPI.
# - HF Space: factory-rebuild the version already pinned in the Space repo.
# - ComfyUI: sync, test, and publish the node against the exact library release.
# All wait for the freshly published sdist to appear on PyPI first, since the
# Release event fires in parallel with publish.yml's upload.
+20 -7
View File
@@ -28,13 +28,26 @@ jobs:
strategy:
fail-fast: false
matrix:
# Cross-OS x Python-floor/recent. The default-install surface only
# needs the core deps plus dev tooling (no GPU extra): the model-running
# paths are availability-gated and skip when torch/diffusers are absent,
# so this matrix exercises metadata, identify, visible, and the cv2
# eraser on every OS without pulling the heavy ML stack.
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.10", "3.12"]
# Exercise every supported minor. The floor and latest minor run on all
# operating systems; intermediate minors run on Linux to bound the
# matrix without leaving a declared Python version untested.
include:
- os: ubuntu-latest
python-version: "3.11"
- os: macos-latest
python-version: "3.11"
- os: windows-latest
python-version: "3.11"
- os: ubuntu-latest
python-version: "3.12"
- os: ubuntu-latest
python-version: "3.13"
- os: ubuntu-latest
python-version: "3.14"
- os: macos-latest
python-version: "3.14"
- os: windows-latest
python-version: "3.14"
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
+1
View File
@@ -54,6 +54,7 @@ gfpgan/
# deeper in the tree is not caught.
/.agents/
/.codex/
/sch_*.yaml
/.opencode/
/opencode.json
/opencode.jsonc
+43 -13
View File
@@ -2,17 +2,19 @@
Remove AI provenance marks from images and video you generated yourself:
- known visible labels such as the Gemini sparkle and vendor text marks;
- known visible labels such as the Google Gemini sparkle watermark and vendor text marks;
- invisible pixel watermarks through diffusion regeneration;
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
Video support covers provenance identification, complete visible-plus-metadata
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo, and
Kling mark removal, and oracle-certified VAE regeneration for video SynthID
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo AI, and
Kling AI mark removal, and oracle-certified VAE regeneration for video SynthID
removal.
> Try it online at [raiw.cc](https://raiw.cc) if you do not want to install Python
> or run diffusion models locally.
> [raiw.cc](https://raiw.cc) runs this library as a hosted service, with the GPU
> included and nothing to install. Visible mark and metadata removal at Standard
> output up to 12 MP are free there; original resolution above 12 MP and invisible
> watermark removal are paid.
[![PyPI](https://img.shields.io/pypi/v/remove-ai-watermarks?logo=pypi&logoColor=white)](https://pypi.org/project/remove-ai-watermarks/)
[![Python](https://img.shields.io/pypi/pyversions/remove-ai-watermarks?logo=python&logoColor=white)](https://pypi.org/project/remove-ai-watermarks/)
@@ -43,6 +45,10 @@ removal.
| Run visible, invisible, and metadata removal | `all` | Recommended |
| Process a directory | `batch` | Depends on mode |
Microsoft Paint and Photos InvisMark declarations are routed automatically to
pixel regeneration. The `all` command removes both the hidden pixel watermark and
its linked C2PA manifest; metadata stripping alone removes only the manifest.
## Installation modes
| Need | Install |
@@ -53,12 +59,12 @@ removal.
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
| Torch-free DWT-DCT detection | `remove-ai-watermarks[detect]` |
| Invisible image removal (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
| Every production feature | `remove-ai-watermarks[all]` |
| Every production feature available on the active Python | `remove-ai-watermarks[all]` |
Lower-level and specialized extras include `pixels`, `heif`, `trustmark`,
`migan`, `lama`, and `diffusion`. The
[installation guide](docs/installation.md#feature-extras) documents their exact
dependency composition and model requirements.
dependency composition, Python compatibility, and model requirements.
## Quick start
@@ -106,7 +112,9 @@ The `video metadata` command does not transcode video or audio streams. Unlike
the image command above, when `-o` is omitted it writes `<source>_clean` and
preserves the original. MP4 and MOV
inspection includes the native TC260 `AIGC` tag in
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload.
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload,
plus the QuickTime-form `meta` variants Doubao's iOS export writes (a bare
`meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list).
MKV and WebM inspection reads the normative
`Segment.Tags.Tag.SimpleTag` placement. AVI uses `LIST/INFO/AIGC`, while FLV
uses `script.onMetaData.AIGC`. The non-ISOBMFF formats are remuxed with stream
@@ -161,8 +169,8 @@ the specificity order shown below. Pass an explicit mark to restrict detection
to one provider.
Sora covers the moving Sora 2 mascot and wordmark. Veo covers both the current
four-point diamond and the legacy `Veo` text. Seedance covers the fixed boxed
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo covers the composite
`MINIMAX | hailuo AI` label, and Kling covers the bottom-right `KLING AI`
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo AI covers the composite
`MINIMAX | hailuo AI` label, and Kling AI covers the bottom-right `KLING AI`
label with its version suffix. A completed encode is published atomically. No
output is written when no stable mark is found.
HDR, PQ/HLG, and greater-than-8-bit inputs are rejected before encoding rather
@@ -198,6 +206,20 @@ image came from an AI generator, add `--force`:
remove-ai-watermarks invisible image.png -o clean.png --force
```
Typography-heavy images can opt into the experimental verified-text post-pass.
It accepts manually reviewed strings and line boxes or an operator-verified
geometry-only manifest; it never treats raw OCR output as ground truth or runs
automatically:
```bash
uv tool install --force "remove-ai-watermarks[text-restoration]"
remove-ai-watermarks invisible image.png -o clean.png \
--text-manifest verified-lines.json --force
```
See the [CLI guide](docs/cli.md#restore-operator-verified-text) for the manifest
schema, compatibility restrictions, and oracle caveats.
See the [installation guide](docs/installation.md) for Homebrew, uv, optional
features, and development setup.
@@ -209,6 +231,13 @@ features, and development setup.
| --- | --- |
| ![Image with a visible Gemini watermark](demo_banana_before.png) | ![Image after visible watermark removal](demo_banana_after.png) |
The `after` raster is generated from the tracked `before` raster by the public path:
```bash
uv run remove-ai-watermarks visible demo_banana_before.png \
--backend cv2 -o demo_banana_after.png
```
### High quality invisible removal
`qwen-zimage` is the default profile: a Qwen-Image-2512 Lightning pass under Canny
@@ -290,8 +319,9 @@ remove-ai-watermarks batch ./images --mode all
Visible mark support includes:
- Google Gemini and Nano Banana sparkle;
- Doubao, Jimeng, Qwen, Kling, Yuanbao, Baidu, LibLibAI, and RunningHub labels;
- Google Gemini and Nano Banana visible sparkle watermark;
- Doubao, Jimeng, Qwen, Kling AI, Yuanbao, Baidu, LiblibAI, and RunningHub labels;
- one calibrated Microsoft top-right white AI-badge variant;
- one calibrated Samsung Galaxy AI label variant.
Metadata and provenance inspection covers C2PA, EXIF, XMP, IPTC, common
@@ -375,7 +405,7 @@ invisible removal.
detail.
- Visible video removal recognizes the moving Sora 2 wordmark, the current Veo
diamond plus legacy `Veo` text, the Seedance boxed `AI` label, and the fixed
Dola, Hailuo, and Kling labels. It does not recognize the older Sora Turbo
Dola, Hailuo AI, and Kling AI labels. It does not recognize the older Sora Turbo
corner swirl or unregistered layouts from those providers.
The classical OpenCV backend can smear structured backgrounds; use MI-GAN or
LaMa when recovery quality matters.
+5 -4
View File
@@ -48,16 +48,17 @@ runtime assets.
video removal profile works, so it is also the source of truth for three shipped
defaults: `tests/test_video_invisible.py` asserts that `noise_std`, `long_side`,
and `fps` together match a row this manifest records as certified. Changing one
of those three without adding the row that certifies it fails the suite. `vae` is
deliberately outside that check because neither tracked row records one; add it
to the assertion in the same commit as the first row that does.
of those three without adding the row that certifies it fails the suite. Both
historical rows mark `vae` as `unrecorded`: the repository must not infer a model
identity from the current default. Add it to the assertion in the same commit as
the first oracle row that records and verifies one.
| Column | Meaning |
| --- | --- |
| `date`, `source_url`, `source_sha256` | Identify the carrier. |
| `source_width`, `source_height`, `source_fps` | Carrier geometry. Without it the actual downscale factor of a row cannot be recovered later. |
| `duration_seconds`, `source_verdict` | Clip length submitted and the verifier's reading of the untouched carrier. |
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The full run configuration. |
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The run configuration. `unrecorded` means the historical run did not preserve the VAE identity. |
| `output_sha256` | Identifies the exact submitted file. |
| `output_verdict` | One of `detected`, `not_detected`, `indeterminate`, `refused`. |
| `output_verdict_text` | The verifier's wording, verbatim. |
+101
View File
@@ -0,0 +1,101 @@
# Content Seal oracle corpus
Muse Image (`muse-image-1.0`) generations with externally verified Content Seal
verdicts, produced through the Meta Model API on 2026-08-26 and checked against
the public detector at `https://meta.ai/identification` (anonymous session, no
login). Follows the `data/synthid/originals` pattern: binaries live in
`originals/`, every derived variant is recorded in `manifest.csv` as a recipe
plus hash and is not stored.
## What this corpus establishes
- The Meta Model API image endpoint (`POST /v1/images/generations`) stamps the
same Content Seal pixel watermark as the consumer Meta AI app: all five
generations verified positive with attribution "Muse Image 1 - Meta".
- The detector response carries a per-generation ID and creation timestamp
embedded in the watermark payload. Both survived a 512 px LANCZOS resize and
a full-size JPEG q85 re-encode (same ID returned), so the payload is more
robust than the detection threshold.
- Three checked center crops lost the seal: 50% and 33% linear crops of the fox
and the 50% crop of the text poster returned "No AI signatures from Meta were
found". The text poster's 33% crop was not checked because the daily oracle
limit was reached, so its empty verdict is not evidence either way. The checked
results are consistent with the Reuters 2026-07-11 analysis (55% missed after
cropping).
- API outputs carry XMP `iptcExt:DigitalSourceType =
trainedAlgorithmicMedia`, so local `identify` flags them via the existing
Made-with-AI path. Metadata-stripping transforms fall back to unknown, and
Content Seal has no local decoder in this project: the oracle is the only
reader.
- Drift finding: the 512 px resize of `gen_fox_forest` triggers a
medium-confidence false positive "Tencent Yuanbao (visible 元宝 / AI生成
mark)" in this project's `identify`. Recorded here as a reproducible case.
## Oracle limits and wire format
There is no public or documented checking API. Verified against the official
developer documentation on 2026-08-26 (`https://dev.meta.ai/docs/`): the full
Meta Model API reference lists only Responses, Chat Completions, Messages,
Files, Images (`/v1/images/generations`, `/v1/images/edits`), and Models, with
no identification, detection, or watermark endpoint, and the image-generation,
Muse Image cookbook, and pricing pages never mention watermark, Content Seal,
or provenance at all. The API applies the seal (every generation in this corpus
carries it) while documenting nothing about it. The web tool drives an internal
REST pair, captured from the browser network log on 2026-08-26:
1. `POST https://rupload.meta.ai/gen_ai_document_gen_ai_tenant/<uuid>` with the
raw file bytes, `x-entity-type`, `x-entity-length`, `ai_detector_upload: true`,
and an anonymous `authorization: OAuth ecto1:<token>` session token minted by
the page.
2. `POST https://meta.ai/api/ai-detector` with `Bearer ecto1:<token>` and body
`{"media_id": "...", "fileName": "...", "mimeType": "..."}`.
The rate limit is enforced at that endpoint, server-side, and keyed beyond the
browser session: the API itself returns `429 {"errorType": "rate_limited"}`,
and clearing cookies and storage changed nothing, so driving the internal pair
directly does not bypass it. The Meta Model API (`api.meta.ai/v1`, where the
generation key works) has no identification endpoint; plausible paths all
return 404. Rows with an empty `oracle_verdict` were transformed but not yet
checkable. Read a verdict only from the settled page text after the
result-complete state ("Upload another file"): a wait for a verdict string can
match the previous upload's text, and the fresh-navigation protocol used for
the calibration rows below is the race-free variant.
## Strength floor calibration (qwen-zimage, seed 0)
The library resolves strength per vendor with measured floors (OpenAI
0.07675 / Google 0.27 / Microsoft InvisMark 0.15 in
`_internal/watermark_profiles.py`). Meta Content Seal had no floor before this
calibration; these rows measure one by the same methodology: independent
generations, each one's first-clean boundary, floor = worst boundary plus the
observed cross-source spread.
Measured (2026-08-26/27, oracle `meta.ai/identification`):
- Default pipeline clears Content Seal: tested samples came back clean at the
default resolution-adaptive strength (~0.1305 at 2.56 MP), including the
worst source.
- Five independent generations bracketed. First-clean boundaries:
lighthouse (0.0525, 0.06], fox (0.03, 0.0375], night_city (0.03, 0.0375],
mug <= 0.03, text <= 0.015. Cross-source spread is wide (a factor of four
between easiest and hardest).
- Derived Meta floor by the existing worst-boundary-plus-cross-source-spread
method: 0.06 + (0.0525 - 0.015) = 0.0975, rounded to **0.1**.
- Shipped as `QWEN_ZIMAGE_META_STRENGTH`: auto mode routes standalone-AI-IPTC
files onto the cohort, and `--vendor meta` / `InvisibleOptions.vendor`
names it explicitly on stripped files (implying the scrub runs).
## Regeneration
The eight deterministic crop, resize, and JPEG variants can be reproduced and
hash-checked from the tracked originals:
```bash
uv run python scripts/contentseal_transforms.py /tmp/contentseal-derived
```
The Meta API generations and remote GPU outputs are not reproducible from this
repository alone. Their prompts, exact output hashes, model/profile settings,
and oracle results are recorded in `manifest.csv`, but the generation API is
stochastic and the private worker environment is not tracked. `MUSE_API_KEY` and
the anonymous detector session are deliberately absent. A new calibration must
therefore create new manifest rows rather than claiming to recreate these bytes.
+44
View File
@@ -0,0 +1,44 @@
name,file,origin,source_url,prompt,requested_size,delivered_size,sha256,xmp_iptc_trainedAlgorithmicMedia,oracle_verdict,oracle_attribution,oracle_generation_id,oracle_created_utc,checked_at_utc,notes
gen_fox_forest,originals/gen_fox_forest.webp,meta-model-api,,A red fox resting on a mossy log in a misty forest at dawn photorealistic shallow depth of field,1024x1024,1600x1600,6d7cecf322a5ac4b2547e2f32c767b3cb3e8a0e49f274cfd00aa31be6697eed4,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:11Z,checked twice; ID stable across uploads
gen_text_poster,originals/gen_text_poster.webp,meta-model-api,,A cheerful flat-design poster that says 'WATERMARK FIELD DAY 2026' in bold rounded letters over a coral and cream striped background,1024x1024,1600x1600,873927662d8198e9c38b4ae71a2919ba5d417dfcd714649db1b0ae231e8cbf9b,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbJuIKXPAYxsd,2026-08-26T19:08Z,2026-08-26T19:08Z,
gen_lighthouse_watercolor,originals/gen_lighthouse_watercolor.webp,meta-model-api,,A watercolor illustration of a lighthouse on a cliff at golden hour seagulls in the distance soft loose washes white paper texture,1024x1024,1600x1600,69bee7c376608568361085602c5c7406900885849bd94c5279c3924cf793b27c,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbB5uWbAmWmwp,2026-08-26T19:08Z,2026-08-26T19:08Z,
gen_night_city,originals/gen_night_city.webp,meta-model-api,,A night city skyline in the rain neon reflections on wet asphalt cinematic wide shot moody,1536x1024,1920x1280,345adf6b9f777b15ef7ce7bbdfb3b355a1d9e882740809e966e9f6caafd94d48,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbH5uWbAmWmwp,2026-08-26T19:08Z,2026-08-26T19:08Z,
gen_studio_mug,originals/gen_studio_mug.webp,meta-model-api,,A minimalist studio photo of a single matte ceramic mug on a pastel blue background soft shadow clean commercial look,1024x1024,1600x1600,2b7bc7fd56fa2250c66ef74f8b3e49d9666fe147d894d5baf9461b5e36077a07,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbAsSozV2XCzw,2026-08-26T19:08Z,2026-08-26T19:08Z,
fox_crop50,,derived:gen_fox_forest,,center crop 0.5 linear each side then WEBP q95,,800x800,80b91995560a7c4b08596a26b3401a29afb82691fad193502697a49785eab7be,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost; PIL re-encode strips XMP
fox_crop33,,derived:gen_fox_forest,,center crop 0.33 linear each side then WEBP q95,,528x528,3b3aded8f4fcb98d0599431abccb73b9581854ae34c6bc869724cd5c86a1e35e,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost
fox_res512,,derived:gen_fox_forest,,resize long side 512 LANCZOS then WEBP q95,,512x512,19b85f615d56f5e9f55cd97bae0ee343f4f432e6bfa0a6fc7acb07dec2cc1d81,no_stripped_by_reencode,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:13Z,generation ID and creation time identical to the original generation: watermark payload survives the resize
fox_jpeg85,,derived:gen_fox_forest,,full-size JPEG q85 re-encode,,1600x1600,3bc7dc11bf3c16c83814df1e6485cdb6e49e21141511dda56d210cc920a8c5e3,no_stripped_by_reencode,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:13Z,payload survives JPEG recompression; local identify returns unknown (metadata stripped)
text_crop50,,derived:gen_text_poster,,center crop 0.5 linear each side then WEBP q95,,800x800,3f0d4075012ee4fec820209c5c1c32bb739998813325f49b1124f379162f9837,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost
text_crop33,,derived:gen_text_poster,,center crop 0.33 linear each side then WEBP q95,,528x528,95cef87002961831b21f6f47cd9a0824a67e2ee6f8503356fd5db6a3fcfcc998,no_stripped_by_reencode,,,,,,daily oracle limit reached before this row could be checked
text_res512,,derived:gen_text_poster,,resize long side 512 LANCZOS then WEBP q95,,512x512,b319727d02e908f22e59f148050e6b4686cbd1a622bd69db51fe4072a6f66b34,no_stripped_by_reencode,,,,,,daily oracle limit reached; also the fox twin of this transform triggered the Yuanbao false positive below
text_jpeg85,,derived:gen_text_poster,,full-size JPEG q85 re-encode,,1600x1600,76f00fcde1e974549afa5d90e1879c705e3c3631ef1f35cab901fc912cd15fa4,no_stripped_by_reencode,,,,,,daily oracle limit reached before this row could be checked
blog_qr_manhwa,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1807972953501959&version=1783318590&transcode_extension=webp,,,1920x1280,8ee7700dff59f21d7de393a619664a0b9360f148d18df289364ecea939b3dea9,yes,detected,Muse Image 1 - Meta,,,2026-08-26T18:59Z,not stored; re-downloadable from source_url
blog_fractal_poster,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1499727434782014&version=1783727678&transcode_extension=webp,,,2096x992,2a96168db026444b47ac37807fd99a88c8b7fa6866e1c93a54ab427fbe581f76,no,detected,Muse Image 1 - Meta,,,2026-08-26T18:59Z,seal survived CDN transcode without any metadata
blog_flipbook_punch,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=2521328538303639&version=1783396590&transcode_extension=webp,,,1920x1280,35f28dc2cbdf44aed190bc8efaa0fe7bfa288a2196638712dd83bc941cc294d5,yes,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,
blog_teen_cat,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1325391919706734&version=1783319609&transcode_extension=webp,,,1600x1600,c2e7df000ffa6520589d8efc5f0bcbeb759175cdaa0c827189c28a64f75d3dd7,yes,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,
blog_hero_1,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=2140990336800598&version=1783727678,,,1344x1792,7655639acb9f826cdb864d869530153efc1e1467ff414b5a63e80aa7b39b2fb7,no,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,seal survived CDN transcode without any metadata
control_synth_gradient,,synthetic-local,,PIL gradient with rectangle JPEG q90; never touched any AI pipeline,,1024x768,25f29c0b993ed1a8e41060ed50de778d875a6d94847ced180aafc5a6ad429f82,no,not_detected,,,,2026-08-26T19:01Z,negative control: No AI signatures from Meta were found
fox_nometa,,derived:gen_fox_forest,,metadata --remove only (no pixel change),,1600x1600,685d3a5c5d74e653ec7615fc7a347a0584c277fc7fb59594d388ac619ffe3985,no_stripped,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T22:19Z,metadata stripping does NOT remove Content Seal; seal lives in pixels; same generation ID as original
fox_modal_invisible,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,6daa3e9b5dde5cb443d8071c5294262742f001c02b694243764a50345769d8b8,no_stripped,not_detected,,,,2026-08-26T22:21Z,current default pipeline clears Content Seal
text_modal_invisible,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,8530a82fb856b0a00651076ac1ad8a7544b5d8873cac9bd5780704fbe10f53f8,no_stripped,not_detected,,,,2026-08-26T22:22Z,current default pipeline clears Content Seal
fox_str0.06000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,7b5e5ae75ca20f2a3cdfcfa7666104752d9557b08ac236db6f1548434940a1f3,no_stripped,not_detected,,,,2026-08-26T22:28Z,strength calibration ladder
fox_str0.04500,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,26da15694b3096e2f1717154db213e814cdd27f2fc40e5f924db2324213cf960,no_stripped,not_detected,,,,2026-08-27T02:28Z,strength calibration ladder; re-confirmed clean at 0.045
fox_str0.03750,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,b7eb0616d10042933edf22cfc30aff4df915384828289b409f6cfa4cdf2807d4,no_stripped,not_detected,,,,2026-08-26T22:30Z,fresh-page protocol; fox first-clean boundary is in (0.03 0.0375]
fox_str0.03000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,cff80d18d9d00000f47e0f97c419a37c13a13831795768aee8727aa9516d1401,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:31Z,fresh-page protocol; seal survives 0.03
fox_str0.02000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,b19183ec60b63a5b73d3e1c723e7943584912857d51a4b5f85259c5c77da0e53,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:28Z,settled-read protocol; monotonic below the 0.03 boundary
text_str0.03750,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,bacfd4eca7b13616502f74497cbe4f02ef6893918870f13f9e76fdb93b43107f,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol
text_str0.03000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,d5c6cffbf471ef8bb74ee8c0d22cc17d9da2c446b83e46c4be67e6ea4cdd09de,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol; text first-clean boundary is at or below 0.03 - cross-source spread vs fox confirmed
lh_str0.03750,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,f5169debc5fd2792399c0ea3fa88cb1d05f523d95004bec240c73db0e9aaee51,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.0375 where fox is already clean
lh_str0.03000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,110d2112d91dd53df5ccd1839e0e2b41c44aa1832adb0090032660e071fb1fd4,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,fresh-page protocol; lighthouse holds its seal below the fox boundary: cross-source spread is wide
text_str0.02500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,f46a6c53b4f3d39fccb1d62cca82ced2e163e4424d13c98e6468a8e083c0fa54,no_stripped,not_detected,,,,2026-08-27T02:28Z,mid rung inside the clean zone
text_str0.02000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,a19cbe22313e0f19768c98567ea89f2d9dc30827691fc7ec2971c52f3e7c6cc6,no_stripped,not_detected,,,,2026-08-27T02:28Z,text clean at 0.02: boundary <= 0.02 - the easiest source measured
text_str0.01500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.015,,1600x1600,aa3816c75e38f3ab5afc378d349caafad56f2bb6da859a5ee68fdea57ad00d73,no_stripped,not_detected,,,,2026-08-27T02:41Z,text clean at 0.015: boundary <= 0.015 - the easiest source measured
lh_str0.02500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,989c4a7a804bc8420ec4d86b9445bc3a37382c1834782a7f8842f781d69b05e2,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
lh_str0.02000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,6a4a4ec94292a145e7d73e6900b8ec6d0eede17137f2b91fba7757ec0bdba5ec,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
nc_str0.03750,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1920x1280,46f322335f801ee54427eb39275216465f9a7cfa3d918c698e4d90a2439a0815,no_stripped,not_detected,,,,2026-08-27T02:28Z,boundary (0.03; 0.0375]
nc_str0.03000,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1920x1280,3f42632319248ad1a91ef2457c7e0b13ed17e1d2569decbbaab67c2072e9a305,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,night_city holds seal at 0.03
mug_str0.03750,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,3b6109ae7e86b8ea74ce33315e3f2a7eda1cadb724d429f0951f13c3ead34b0f,no_stripped,not_detected,,,,2026-08-27T02:28Z,upper rung for the mug bracket
mug_str0.03000,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,1c24c984677e1528da83003ac40594f6e5ef1fa9351f1368b6d31a0101308583,no_stripped,not_detected,,,,2026-08-27T02:28Z,mug clean at 0.03: boundary <= 0.03
lh_str0.05250,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0525,,1600x1600,8912ba1ff4a26790f69224c49dd649e46835e277c375276ac9604f5339775b2a,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:41Z,bisection: lighthouse first-clean boundary bracketed to (0.0525; 0.06]
lh_str0.04500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,cfdcf7bbb27c740846a1de4edffee3b25a624c2a8e1aba2362feaa25b682e3ae,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.045: lighthouse is the worst source measured
lh_str0.06000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,4e214be00a89dc9f1f32ec39483c5e9668a1ec7f38b15e5794942479c839fcca,no_stripped,not_detected,,,,2026-08-27T02:28Z,lighthouse first-clean boundary bracketed to (0.045 0.06]
1 name file origin source_url prompt requested_size delivered_size sha256 xmp_iptc_trainedAlgorithmicMedia oracle_verdict oracle_attribution oracle_generation_id oracle_created_utc checked_at_utc notes
2 gen_fox_forest originals/gen_fox_forest.webp meta-model-api A red fox resting on a mossy log in a misty forest at dawn photorealistic shallow depth of field 1024x1024 1600x1600 6d7cecf322a5ac4b2547e2f32c767b3cb3e8a0e49f274cfd00aa31be6697eed4 yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD 2026-08-26T19:07Z 2026-08-26T19:11Z checked twice; ID stable across uploads
3 gen_text_poster originals/gen_text_poster.webp meta-model-api A cheerful flat-design poster that says 'WATERMARK FIELD DAY 2026' in bold rounded letters over a coral and cream striped background 1024x1024 1600x1600 873927662d8198e9c38b4ae71a2919ba5d417dfcd714649db1b0ae231e8cbf9b yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbJuIKXPAYxsd 2026-08-26T19:08Z 2026-08-26T19:08Z
4 gen_lighthouse_watercolor originals/gen_lighthouse_watercolor.webp meta-model-api A watercolor illustration of a lighthouse on a cliff at golden hour seagulls in the distance soft loose washes white paper texture 1024x1024 1600x1600 69bee7c376608568361085602c5c7406900885849bd94c5279c3924cf793b27c yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbB5uWbAmWmwp 2026-08-26T19:08Z 2026-08-26T19:08Z
5 gen_night_city originals/gen_night_city.webp meta-model-api A night city skyline in the rain neon reflections on wet asphalt cinematic wide shot moody 1536x1024 1920x1280 345adf6b9f777b15ef7ce7bbdfb3b355a1d9e882740809e966e9f6caafd94d48 yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbH5uWbAmWmwp 2026-08-26T19:08Z 2026-08-26T19:08Z
6 gen_studio_mug originals/gen_studio_mug.webp meta-model-api A minimalist studio photo of a single matte ceramic mug on a pastel blue background soft shadow clean commercial look 1024x1024 1600x1600 2b7bc7fd56fa2250c66ef74f8b3e49d9666fe147d894d5baf9461b5e36077a07 yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbAsSozV2XCzw 2026-08-26T19:08Z 2026-08-26T19:08Z
7 fox_crop50 derived:gen_fox_forest center crop 0.5 linear each side then WEBP q95 800x800 80b91995560a7c4b08596a26b3401a29afb82691fad193502697a49785eab7be no_stripped_by_reencode not_detected 2026-08-26T19:13Z seal lost; PIL re-encode strips XMP
8 fox_crop33 derived:gen_fox_forest center crop 0.33 linear each side then WEBP q95 528x528 3b3aded8f4fcb98d0599431abccb73b9581854ae34c6bc869724cd5c86a1e35e no_stripped_by_reencode not_detected 2026-08-26T19:13Z seal lost
9 fox_res512 derived:gen_fox_forest resize long side 512 LANCZOS then WEBP q95 512x512 19b85f615d56f5e9f55cd97bae0ee343f4f432e6bfa0a6fc7acb07dec2cc1d81 no_stripped_by_reencode detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD 2026-08-26T19:07Z 2026-08-26T19:13Z generation ID and creation time identical to the original generation: watermark payload survives the resize
10 fox_jpeg85 derived:gen_fox_forest full-size JPEG q85 re-encode 1600x1600 3bc7dc11bf3c16c83814df1e6485cdb6e49e21141511dda56d210cc920a8c5e3 no_stripped_by_reencode detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD 2026-08-26T19:07Z 2026-08-26T19:13Z payload survives JPEG recompression; local identify returns unknown (metadata stripped)
11 text_crop50 derived:gen_text_poster center crop 0.5 linear each side then WEBP q95 800x800 3f0d4075012ee4fec820209c5c1c32bb739998813325f49b1124f379162f9837 no_stripped_by_reencode not_detected 2026-08-26T19:13Z seal lost
12 text_crop33 derived:gen_text_poster center crop 0.33 linear each side then WEBP q95 528x528 95cef87002961831b21f6f47cd9a0824a67e2ee6f8503356fd5db6a3fcfcc998 no_stripped_by_reencode daily oracle limit reached before this row could be checked
13 text_res512 derived:gen_text_poster resize long side 512 LANCZOS then WEBP q95 512x512 b319727d02e908f22e59f148050e6b4686cbd1a622bd69db51fe4072a6f66b34 no_stripped_by_reencode daily oracle limit reached; also the fox twin of this transform triggered the Yuanbao false positive below
14 text_jpeg85 derived:gen_text_poster full-size JPEG q85 re-encode 1600x1600 76f00fcde1e974549afa5d90e1879c705e3c3631ef1f35cab901fc912cd15fa4 no_stripped_by_reencode daily oracle limit reached before this row could be checked
15 blog_qr_manhwa meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=1807972953501959&version=1783318590&transcode_extension=webp 1920x1280 8ee7700dff59f21d7de393a619664a0b9360f148d18df289364ecea939b3dea9 yes detected Muse Image 1 - Meta 2026-08-26T18:59Z not stored; re-downloadable from source_url
16 blog_fractal_poster meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=1499727434782014&version=1783727678&transcode_extension=webp 2096x992 2a96168db026444b47ac37807fd99a88c8b7fa6866e1c93a54ab427fbe581f76 no detected Muse Image 1 - Meta 2026-08-26T18:59Z seal survived CDN transcode without any metadata
17 blog_flipbook_punch meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=2521328538303639&version=1783396590&transcode_extension=webp 1920x1280 35f28dc2cbdf44aed190bc8efaa0fe7bfa288a2196638712dd83bc941cc294d5 yes detected Muse Image 1 - Meta 2026-08-26T19:00Z
18 blog_teen_cat meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=1325391919706734&version=1783319609&transcode_extension=webp 1600x1600 c2e7df000ffa6520589d8efc5f0bcbeb759175cdaa0c827189c28a64f75d3dd7 yes detected Muse Image 1 - Meta 2026-08-26T19:00Z
19 blog_hero_1 meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=2140990336800598&version=1783727678 1344x1792 7655639acb9f826cdb864d869530153efc1e1467ff414b5a63e80aa7b39b2fb7 no detected Muse Image 1 - Meta 2026-08-26T19:00Z seal survived CDN transcode without any metadata
20 control_synth_gradient synthetic-local PIL gradient with rectangle JPEG q90; never touched any AI pipeline 1024x768 25f29c0b993ed1a8e41060ed50de778d875a6d94847ced180aafc5a6ad429f82 no not_detected 2026-08-26T19:01Z negative control: No AI signatures from Meta were found
21 fox_nometa derived:gen_fox_forest metadata --remove only (no pixel change) 1600x1600 685d3a5c5d74e653ec7615fc7a347a0584c277fc7fb59594d388ac619ffe3985 no_stripped detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD 2026-08-26T19:07Z 2026-08-26T22:19Z metadata stripping does NOT remove Content Seal; seal lives in pixels; same generation ID as original
22 fox_modal_invisible derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600) 1600x1600 6daa3e9b5dde5cb443d8071c5294262742f001c02b694243764a50345769d8b8 no_stripped not_detected 2026-08-26T22:21Z current default pipeline clears Content Seal
23 text_modal_invisible derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600) 1600x1600 8530a82fb856b0a00651076ac1ad8a7544b5d8873cac9bd5780704fbe10f53f8 no_stripped not_detected 2026-08-26T22:22Z current default pipeline clears Content Seal
24 fox_str0.06000 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06 1600x1600 7b5e5ae75ca20f2a3cdfcfa7666104752d9557b08ac236db6f1548434940a1f3 no_stripped not_detected 2026-08-26T22:28Z strength calibration ladder
25 fox_str0.04500 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045 1600x1600 26da15694b3096e2f1717154db213e814cdd27f2fc40e5f924db2324213cf960 no_stripped not_detected 2026-08-27T02:28Z strength calibration ladder; re-confirmed clean at 0.045
26 fox_str0.03750 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1600x1600 b7eb0616d10042933edf22cfc30aff4df915384828289b409f6cfa4cdf2807d4 no_stripped not_detected 2026-08-26T22:30Z fresh-page protocol; fox first-clean boundary is in (0.03 0.0375]
27 fox_str0.03000 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1600x1600 cff80d18d9d00000f47e0f97c419a37c13a13831795768aee8727aa9516d1401 no_stripped detected Muse Image 1 - Meta 2026-08-26T22:31Z fresh-page protocol; seal survives 0.03
28 fox_str0.02000 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02 1600x1600 b19183ec60b63a5b73d3e1c723e7943584912857d51a4b5f85259c5c77da0e53 no_stripped detected Muse Image 1 - Meta 2026-08-26T22:28Z settled-read protocol; monotonic below the 0.03 boundary
29 text_str0.03750 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1600x1600 bacfd4eca7b13616502f74497cbe4f02ef6893918870f13f9e76fdb93b43107f no_stripped not_detected 2026-08-26T22:33Z fresh-page protocol
30 text_str0.03000 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1600x1600 d5c6cffbf471ef8bb74ee8c0d22cc17d9da2c446b83e46c4be67e6ea4cdd09de no_stripped not_detected 2026-08-26T22:33Z fresh-page protocol; text first-clean boundary is at or below 0.03 - cross-source spread vs fox confirmed
31 lh_str0.03750 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1600x1600 f5169debc5fd2792399c0ea3fa88cb1d05f523d95004bec240c73db0e9aaee51 no_stripped detected Muse Image 1 - Meta 2026-08-27T02:28Z seal survives 0.0375 where fox is already clean
32 lh_str0.03000 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1600x1600 110d2112d91dd53df5ccd1839e0e2b41c44aa1832adb0090032660e071fb1fd4 no_stripped detected Muse Image 1 - Meta 2026-08-27T02:28Z fresh-page protocol; lighthouse holds its seal below the fox boundary: cross-source spread is wide
33 text_str0.02500 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025 1600x1600 f46a6c53b4f3d39fccb1d62cca82ced2e163e4424d13c98e6468a8e083c0fa54 no_stripped not_detected 2026-08-27T02:28Z mid rung inside the clean zone
34 text_str0.02000 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02 1600x1600 a19cbe22313e0f19768c98567ea89f2d9dc30827691fc7ec2971c52f3e7c6cc6 no_stripped not_detected 2026-08-27T02:28Z text clean at 0.02: boundary <= 0.02 - the easiest source measured
35 text_str0.01500 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.015 1600x1600 aa3816c75e38f3ab5afc378d349caafad56f2bb6da859a5ee68fdea57ad00d73 no_stripped not_detected 2026-08-27T02:41Z text clean at 0.015: boundary <= 0.015 - the easiest source measured
36 lh_str0.02500 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025 1600x1600 989c4a7a804bc8420ec4d86b9445bc3a37382c1834782a7f8842f781d69b05e2 no_stripped below the measured detected boundary (0.03) for this source; not individually checked
37 lh_str0.02000 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02 1600x1600 6a4a4ec94292a145e7d73e6900b8ec6d0eede17137f2b91fba7757ec0bdba5ec no_stripped below the measured detected boundary (0.03) for this source; not individually checked
38 nc_str0.03750 derived:gen_night_city modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1920x1280 46f322335f801ee54427eb39275216465f9a7cfa3d918c698e4d90a2439a0815 no_stripped not_detected 2026-08-27T02:28Z boundary (0.03; 0.0375]
39 nc_str0.03000 derived:gen_night_city modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1920x1280 3f42632319248ad1a91ef2457c7e0b13ed17e1d2569decbbaab67c2072e9a305 no_stripped detected Muse Image 1 - Meta 2026-08-27T02:28Z night_city holds seal at 0.03
40 mug_str0.03750 derived:gen_studio_mug modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1600x1600 3b6109ae7e86b8ea74ce33315e3f2a7eda1cadb724d429f0951f13c3ead34b0f no_stripped not_detected 2026-08-27T02:28Z upper rung for the mug bracket
41 mug_str0.03000 derived:gen_studio_mug modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1600x1600 1c24c984677e1528da83003ac40594f6e5ef1fa9351f1368b6d31a0101308583 no_stripped not_detected 2026-08-27T02:28Z mug clean at 0.03: boundary <= 0.03
42 lh_str0.05250 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0525 1600x1600 8912ba1ff4a26790f69224c49dd649e46835e277c375276ac9604f5339775b2a no_stripped detected Muse Image 1 - Meta 2026-08-27T02:41Z bisection: lighthouse first-clean boundary bracketed to (0.0525; 0.06]
43 lh_str0.04500 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045 1600x1600 cfdcf7bbb27c740846a1de4edffee3b25a624c2a8e1aba2362feaa25b682e3ae no_stripped detected Muse Image 1 - Meta 2026-08-27T02:28Z seal survives 0.045: lighthouse is the worst source measured
44 lh_str0.06000 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06 1600x1600 4e214be00a89dc9f1f32ec39483c5e9668a1ec7f38b15e5794942479c839fcca no_stripped not_detected 2026-08-27T02:28Z lighthouse first-clean boundary bracketed to (0.045 0.06]
Binary file not shown.

After

Width:  |  Height:  |  Size: 585 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 510 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 652 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 192 KiB

+366 -3
View File
@@ -7,31 +7,394 @@ ground truth and instructions.
| Original | Provider | Content | Exercises |
| --- | --- | --- | --- |
| `ChatGPT Image May 31, 2026, 02_02_23 PM.png` | OpenAI | Light multilingual typography | Text preservation |
| `ChatGPT Image May 31, 2026, 02_03_55 PM.png` | OpenAI | Multilingual typography | Text preservation |
| `Gemini_Generated_Image_633uuy633uuy633u.png` | Google | Landscape with a Chinese sign | CJK text preservation |
| `Gemini_Generated_Image_y48j3cy48j3cy48j.png` | Google | Portrait grid | Face identity and skin texture |
## Text ground truth
`ground-truth.json` contains hand-verified OCR for the two text-bearing
`ground-truth.json` contains hand-verified OCR for the three text-bearing
originals. To regenerate an OCR seed:
```bash
uv run scripts/fidelity_metrics.py ocr \
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png" \
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png" \
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
--langs en,ru,ch \
--out data/evaluations/fidelity/ground-truth.json
```
Verify and correct the generated text by hand before using it as ground truth.
`text-lines.json` contains the verified per-line strings and source-space boxes
used by the evaluation-only selective-restoration experiment. It is not an
automatic scene-text annotation set.
`scripts/infer_text_lines.py` can generate a draft from source pixels without
existing annotations. On the two posters it proposed 20 and 18 lines at the
default threshold, but exact-text precision was only 90.0% and 94.4% because
high-confidence OCR still dropped an English comma and replaced a Chinese comma
with ASCII. Its output therefore requires manual verification of every line;
`accepted` means crop-stable, not ground-truth-correct.
## Text-preservation benchmark
`text-preservation-2026-08-13.csv` records a fixed-seed comparison of the two
current profiles and a global Z-Image Turbo prototype on all three text
fixtures. Each candidate ran through the
complete `visible -> invisible -> metadata` route with its profile default
strength and adaptive-polish setting. The Z-Image prototype instead sweeps
0.08, 0.10, 0.15, 0.20, and 0.25 without polish because its provider-specific
removal floor is not known. The output hash identifies the exact bytes
measured; generated outputs remain outside the repository.
The character-weighted added CER is 0.262 for `qwen-zimage` and 0.256 for
`sdxl-zimage`. That 0.006 absolute difference is not a
stable ordering: SDXL wins the light poster, Qwen wins the dark poster, and the
Chinese sign is tied. A paired sign test on the two non-ties is 1-1 (`p=1.0`).
The measured sample therefore does not support a general text-preservation
winner. Both profiles substantially degrade the smallest multilingual poster
text and preserve the larger Chinese sign.
Z-Image Turbo is the clear fidelity lead. At strength 0.10 its
character-weighted CER is 0.093, against the unchanged sources' 0.124 OCR
floor, while whole-image LPIPS stays between 0.047 and 0.086. Visual inspection
still finds substitutions in the smallest Cyrillic and Chinese poster lines,
so a favorable OCR score does not mean pixel-exact text. Quality drops quickly
above 0.15; at 0.25 the weighted CER is 0.428.
Provider-oracle checks on 2026-08-13 bracket the OpenAI requirement at no more
than 0.10 for this sample. Both original OpenAI controls were detected by
`openai.com/verify`; the light poster was clean at 0.08, while the dark poster
was detected at 0.08 and clean at 0.10. The light 0.10 output was not separately
checked. Google fails the entire tested sweep. The original Gemini image was
detected through C2PA, and a pixel-identical copy with AI metadata stripped was
separately detected by Gemini's built-in SynthID verifier. Z-Image outputs at
0.08, 0.10, 0.15, 0.20, and 0.25 were all still detected. At 0.25 the weighted
CER has already risen to 0.428, worse than the raw weighted CER of both current
profiles (Qwen 0.387, SDXL 0.381). Increasing strength beyond the measured grid
would therefore no longer serve the text-preservation objective without a new
mechanism or hypothesis. Z-Image is not a viable global replacement on this
evidence: no clean Google operating point was found before it lost its fidelity
advantage.
Qwen-Image-2.0 was not added to the numeric comparison. Its weights are not
published, and its hosted edit API exposes an editing instruction and seed but
no low-strength denoise control. It can be evaluated as a separate hosted edit
strategy when credentials are available, but it is not a drop-in replacement
for the partial-regeneration mechanism measured here.
The OCR floor is the source image scored against the hand-verified text. Use
`added_cer = text_cer - ocr_floor` when interpreting pipeline damage, because the
unchanged poster sources already score 0.127 CER. `oracle_rechecked=true` marks
the exact Z-Image bytes checked above; the remaining rows were not rechecked.
The table does not certify other seeds, content classes, or strengths beyond
the recorded provider verdicts.
## Text-restoration prototype
`text-restoration-2026-08-13.csv` evaluates an OCR-driven post-pass on the exact
Qwen outputs above. The prototype recognizes English and Russian with macOS
Vision and CJK with PaddleOCR, derives glyph masks independently from the
source and Qwen output, removes both sets of glyphs with block-wise LaMa, and
draws the recognized strings with new system-font pixels. It never composites
source pixels back into the result.
On the two multilingual posters, character-weighted CER fell from 0.338 and
0.305 to 0.007 on both. OpenAI Verify reported no OpenAI signals for both Qwen
controls and both restored outputs in the same run. The improvement comes with
a substantial whole-image fidelity cost: LPIPS rose from 0.107 to 0.174 and
from 0.095 to 0.162, while PSNR fell by about 10.5-11.4 dB. Visual inspection
found one residual shadow in the smallest English line of the light poster;
the dark poster was clean but the substitute fonts visibly changed typography.
The Chinese sign did not improve: CER rose from 0.074 to 0.111 because the OCR
and renderer changed punctuation. The Gemini verifier returned detected for
both the restored output and its byte-identical Qwen control on a third work
account, although that Qwen hash had previously returned clean on another work
account. This run therefore does not isolate a restoration-stage regression;
its Google verdict is inconclusive until a source-positive, Qwen-negative
control can be reproduced in the same available account.
This prototype is not ready to ship. Its strongest result establishes that
fresh-glyph reconstruction can recover literal text without reintroducing an
OpenAI signal, but portable OCR, font/style reconstruction, a tighter mask, and
a reproducible Google oracle control remain prerequisites.
### Selective restoration follow-up
`selective-text-restoration-2026-08-13.csv` compares that full compositor with
a selective prototype on the same two posters. The prototype leaves a Qwen line
unchanged when padded source and output recognition agree, and applies the same
LaMa plus fresh-system-font reconstruction only to lines whose recognized text
changed. This reduced the edited area from 15.4% and 17.0% to 5.5% on both
posters.
Under one consistent Paddle `en+ru+ch` measurement route, selective restoration
reduced Qwen CER from 0.378 to 0.101 on the light poster and from 0.413 to 0.112
on the dark poster. Its image LPIPS was 0.120 and 0.103, substantially closer to
Qwen than the full compositor's 0.174 and 0.162. These CER values must not be
mixed with the preceding table's Vision/Paddle hybrid values: the comparison
file remeasures all three variants through Paddle so their relative result is
valid on one OCR route.
OpenAI Verify returned `No OpenAI signals detected` for both selective outputs,
then detected the original light poster as `Generated with OpenAI tools` in the
same Chrome sequence. Visual inspection found the dark output clean, but the
light output still retained a local shadow around one replaced fine-text line.
Selective restoration is therefore the strongest current direction, not a
production-ready default. The next implementation needs automatic line matching
and a tighter source-plus-candidate glyph mask before it can be proposed for the
pipeline.
A mask-only follow-up added two pixels of dilation around every selected glyph.
It visually removed the light poster's shadow and improved its CER from 0.101 to
0.081 with effectively unchanged LPIPS, but increased the edited area from 5.5%
to 7.0%. Applying the same rule to the dark poster worsened CER from 0.112 to
0.147. These expanded-mask rows were not resubmitted to the provider oracle.
The opposing result rules out one global dilation constant and makes per-line
residual detection the next fidelity gate.
An automatic follow-up removed the hand-selected line list. For each known
source line, it runs a recognition-only model on padded, upscaled source and
Qwen crops. It preserves a line only when both confidence scores are at least
0.75, the source recognition is within 0.25 normalized edit distance of the
verified line, and normalized source and candidate recognition match exactly.
This reproduced the manual dark-poster line selection and additionally kept one
correct large Russian line on the light poster.
The automatic prototype also replaced uniform dilation with a second measured
LaMa pass. After the first erase, it extracts contrast remaining at the original
glyph positions, dilates only that residual, and inpaints it before drawing fresh
text. Visual inspection found no remaining double glyphs on either output. The
effective edited fractions were 4.1% and 5.7%. Image LPIPS was 0.113 and 0.104,
against 0.120 and 0.103 for the manual selective version. Detector-based page CER
was 0.123 and 0.119; the light value is inflated by the detector splitting the
preserved Cyrillic line into Latin-like fragments even though the padded Russian
recognizer reads it exactly. This is another reason that line decisions use
recognition-only crops rather than the page detector's aggregate CER.
Both exact automatic output hashes returned `No OpenAI signals detected`, and
the original light poster returned `Generated with OpenAI tools` immediately
afterward in the same Chrome sequence. The tracked script reproduced the dark
hash exactly. Its light output differed from the initial prototype in only 386
pixels by at most one channel value because separate ONNX environments rounded
differently; the script's exact `900def5a...` hash was therefore submitted and
also returned clean before the positive source control.
The script and its required verified line annotations are tracked as
`scripts/selective_text_restoration.py` and `text-lines.json`. For example:
```bash
PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK=True \
uv run scripts/selective_text_restoration.py \
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
path/to/qwen-candidate.png \
--restoration vae-glyphs \
--glyph-donor path/to/qwen-vae-reconstruction.png \
--glyph-feather 0.5 \
--output path/to/selective-output.png \
--manifest path/to/manifest.json
```
The restoration mode is deliberately explicit. `vae-glyphs` requires a
separately generated VAE reconstruction in `--glyph-donor`; it takes exact
thresholded glyph cores from that donor, uses a fresh silhouette beneath them,
and defaults to a narrow 0.5-pixel donor edge. The donor and the resulting exact bytes still require
full-pipeline oracle evaluation. `rerender` reproduces the tracked
fresh-system-font experiment. `source-glyphs` is only for compositing a
separately regenerated, oracle-evaluated layer; feeding it the watermarked
original would paste provenance-bearing pixels back into the result. The
`--keep-background --composite-mask boxes` combination is an aligned-layer
experiment and is not a production text restorer.
The `source-silhouette` follow-up keeps the source glyph geometry but discards
its pixel amplitudes: it thresholds each source line to a binary shape, samples
one foreground color, and synthesizes new antialiasing over the scrubbed image.
On a 24-image matrix spanning serif, Latin-diacritic, Cyrillic, CJK, tiny-UI,
and rotated text, the original outer-feather compositor preserved all 34
source-readable lines exactly under crop OCR, with median text-box SSIM 0.859.
An inner-antialias variant raised median text-box SSIM to 0.902 and left
whole-image SSIM unchanged at 0.672, but preserved 33/34 lines: OCR read one
middle dot as a colon. The higher-fidelity antialiasing is retained in the
evaluation script, with that punctuation miss recorded as a caveat. Its exact
outputs were not submitted to the provider oracle because the
public verifier reached its request limit; a clean verdict from the earlier
outer-feather bytes does not transfer to the new hashes.
Follow-up visual review on a typography-rich control rejected both silhouette
compositors despite their OCR and SSIM scores. They preserved literal content
and approximate glyph geometry, but changed stroke weight, color variation,
edge antialiasing, and small decorative details enough to be plainly different
from the source. OCR exactness and text-box SSIM are therefore screening metrics,
not acceptance gates for source-typography preservation. A candidate must also
retain source-like edge pixels and pass direct visual comparison at native size.
A narrower Qwen-VAE donor follow-up keeps the scrubbed background, then copies
only VAE-reconstructed pixels through the source-silhouette mask with a
0.5-pixel feather. Across the 48-case typography matrix (548 annotated lines),
text-box SSIM improved in 47/48 cases and glyph-edge error improved in 48/48.
Median text-box SSIM rose from 0.854 to 0.914, while median glyph-edge MAE fell
from 37.59 to 32.42. The median nonzero alpha area was 3.62%; on dense or large
text this is still substantially wider than the target control. These are
fidelity results only; the 48 matrix outputs were not submitted to the provider
oracle. On a separate untracked dense-typography control, the exact core-only
Qwen-VAE donor with a 1.43% source-silhouette area returned `No OpenAI signals
detected`. Its 0.5-pixel feathered sibling, which raises the nonzero donor area
to 2.75%, returned the same verdict and improved mean text-box SSIM from 0.957
to 0.960. Crop OCR recovered from 7/15 exact lines on the raw pass to 14/15,
matching the source OCR floor. Whole-image LPIPS was 0.082, but only 0.108% of
pixels were exactly unchanged and the detected face retained 0.670 of source
Laplacian variance. The tracked script reproduced the feathered file byte for
byte. These two exact-byte verdicts do not certify other images or the larger
matrix masks, and the global smoothing fails a strict unchanged-image criterion.
The opt-in production port was rechecked separately on 2026-08-15. Its current
LaMa runtime did not reproduce the earlier evaluation PNG byte for byte, but all
changed pixels were confined to the erased background outside the donor glyph
core. The exact production artifact returned `No OpenAI signals detected` in
3/3 OpenAI Verify runs, while the matched source control returned `Generated
with OpenAI tools` in 2/2 runs in the same Chrome session; expanded details
identified SynthID and no C2PA manifest on the control. The private control and
artifact hashes remain outside the public repository. This certifies only that
runtime, verified manifest, and output, not arbitrary text masks or images.
The Google result is negative. On the synthetic CJK sign case, two separate
work-account runs both detected SynthID in the resaved source control and in the
exact Qwen-VAE donor output. The candidate improved mean text-box SSIM from
0.791 to 0.835 and glyph-edge MAE from 35.48 to 22.49 with a 3.67% donor area,
but Gemini still detected it. The intermediate Qwen silhouette base and the
earlier original-sign Qwen baseline were also detected in the same account.
This isolates the current blocker upstream of text restoration: the global
Google pass did not reach the SynthID removal floor, so the OpenAI-clean
`vae-glyphs` configuration cannot be used as a Google operating point.
On the large Chinese-sign control, source and candidate recognition agreed on
all three lines. The script selected no changed lines, emitted a zero mask, and
copied the candidate byte for byte instead of re-encoding it. This corrects the
earlier unnecessary CJK rerender and proves the no-edit branch. It does not add
a Google-negative oracle result: the available Google account still cannot
reproduce a source-positive, Qwen-negative control sequence.
The earlier automatic rerender was the first variant in the experiment to pass
the recorded visual, fidelity, and OpenAI-oracle gates on its two poster
fixtures. The later typography-rich control shows that result does not
generalize to source-typography preservation. It remains evaluation-only: it
depends on verified source text and source line boxes, uses macOS system fonts,
and has not been validated on natural scene text, rotated text, or automatic
line-box discovery.
An opt-in `--detect-boxes` follow-up tested automatic geometry. Grouping Paddle
word detections by vertical overlap found exactly 20/20, 20/20, and 3/3 lines;
mean IoU with verified boxes was 0.857, 0.847, and 1.000. Reusing the annotation
crop padding was unstable and preserved only 4/20 dark-poster lines. Reducing
vertical recognition padding to 10% restored the exact 8/20 and 7/20 selection
decisions, but detector CER was 0.127 and 0.154 rather than 0.123 and 0.119. The
dark regression failed the fidelity gate, so these hashes were not submitted to
the provider oracle. The flag is retained only to reproduce that negative
evaluation and still requires verified strings and an exact line-count match.
### AnyText2 glyph-conditioned follow-up
`anytext2-restoration-2026-08-13.csv` tests the official Apache-2.0
AnyText2 checkpoint as a local text-editing pass over the exact Qwen Chinese-sign
output. The checkpoint hash was verified against ModelScope. Its own edit example
successfully replaced a masked blackboard line with the requested `DADDY`, so the
runtime and checkpoint were functional before the tracked fixture was measured.
AnyText2 failed the fidelity gate. The standard full-image detector scored the
default edit at CER 0.185 and the source-font mimic at 0.222, against 0.074 for
Qwen; image LPIPS moved from 0.289 to 0.338 and 0.345. A padded crop-recognition
cross-check removed the detector's punctuation misses: Qwen scored 0.000, the
default edit still scored 0.074 with two substituted characters, and the mimic
edit stayed at 0.222. Visual inspection agreed, with additional line-shape and
punctuation errors in the mimic output. No provider-oracle submission was made
because neither candidate preserved the requested content better than its
already-clean Qwen input.
This result covers the published SD1.5 checkpoint, three Chinese lines of no more
than 20 characters, one seed, and one fixture. The repository still describes
AnyText2XL as unreleased, and the inference wrapper truncates each requested line
after 20 characters, so the published model cannot directly cover the longer
English and Russian poster lines.
The padded cross-check also changes the interpretation of this fixture's earlier
restoration result. Its Qwen image already contains all three correct Chinese
lines; the reported 0.074 came from the detector cropping the final punctuation
from two lines. A deterministic rerender with horizontally padded recognition
and a mask-matched Hiragino Sans GB W6 font reached detector CER 0.000, but image
LPIPS worsened from 0.289 to 0.327 and the font was visibly heavier. The correct
policy on this large-text fixture is therefore to leave a Qwen line unchanged
when padded recognition confirms the source and candidate strings agree.
## Face-stage isolation
`face-stage-isolation-2026-08-13.csv` separates the two stages on the 18-face
Gemini portrait-grid fixture. All variants use the same visible-cleaned,
metadata-stripped input and seed 0. The prepared input remained positive in the
Gemini SynthID verifier, so the surrounding visible and metadata stages did not
create the clean verdicts.
| Route | Gemini oracle | ID cosine | Face LPIPS | Laplacian ratio | Image LPIPS |
| --- | --- | ---: | ---: | ---: | ---: |
| Prepared control | detected | - | - | - | - |
| Z-Image face stage only | detected | 0.927 | 0.028 | 0.612 | 0.013 |
| Qwen global stage only | clean | 0.589 | 0.217 | 0.331 | 0.288 |
| Full Qwen then Z-Image | clean | 0.852 | 0.050 | 0.492 | 0.252 |
This isolates the roles: the Qwen global pass removes the watermark, while the
masked Z-Image pass restores face identity and detail without reintroducing a
detectable whole-image signal. Z-Image face-only does not remove SynthID. The
result explains why the face stage succeeded while global Z-Image failed: the
earlier clean verdict belonged to the two-stage pipeline, not to Z-Image as a
standalone remover. This is one fixture, one seed, and one oracle pass per
variant; it does not establish a general reintroduction threshold for mask size.
## Upstream Synthid-Bypass v2 reproduction
`upstream-v2-reproduction-2026-08-13.csv` records a source-level audit and a
close reproduction of the public
[`Synthid-Bypass-v2.0.json`](https://github.com/cebeuq/Synthid-Bypass/blob/3007d0351596ae0a78b7074dae7ad179710b1e48/Synthid-Bypass-v2.0.json).
The audited workflow is SHA-256
`41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6` at
upstream commit `3007d0351596ae0a78b7074dae7ad179710b1e48`.
Tracing links backward from `SaveImage` confirms that the active global stage is
Qwen-Image-2512 Q4 with the Lightning four-step LoRA at 0.8, DiffSynth Canny at
1.0, AuraFlow shift 3, `dpmpp_2m` plus `sgm_uniform`, CFG 1, and four steps. Its
resolution helper uses adaptive level 6 over 0.08..0.15, which resolves to the
0.154 ceiling for the 2816x1536 portrait fixture. Z-Image Turbo is used only by
the face detailer: eight steps, CFG 1, `res_2s` plus `bong_tangent`, a 768 px
guide, 1024 px cap, and direct adaptive denoise
`clamp(0.10 * largest_face_ratio / 0.03, 0.05, 0.28)`. The active face regions
come from YOLOv8-face plus SAM. Although the README describes MediaPipe as part
of the strict path, the MediaPipe nodes do not reach the saved output. The
1.2-megapixel scaler is also bypassed (`mode=4`).
The published upstream pair 12 changed from SynthID-positive to clean in the
Gemini verifier and scored 0.975 face identity. On the project portrait fixture,
the prepared control was positive, while both the close global reproduction and
the close full reproduction were clean. The global stage scored 0.589 identity
and 0.217 face LPIPS; the upstream-strength face stage improved those to 0.783
and 0.083. The maintained profile's weaker face pass scored 0.852 and 0.050 on
the same control, so copying upstream's roughly doubled face strength would be a
quality regression on this fixture.
The close reproduction is intentionally labeled rather than presented as an
exact ComfyUI run. It uses full-precision DiffSynth bf16 weights instead of the
Q4 GGUF files, the maintained DiffSynth Lightning scheduler approximation
instead of the ComfyUI sampler pair, YuNet plus SAM instead of YOLOv8-face plus
SAM, and fixed seed 0 instead of randomized seeds. The two independent oracle
controls and the published upstream pair establish the direction of the result;
they do not make the local output byte-equivalent to upstream.
## Compare
```bash
uv run scripts/fidelity_metrics.py compare \
--original data/synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png \
--variant controlnet=<out>.png \
--variant qwen=<out>.png \
--variant qwen-zimage=<qwen-out>.png \
--variant sdxl-zimage=<sdxl-out>.png \
--ocr-langs ""
```
@@ -0,0 +1,4 @@
date,source_commit,fixture,variant,model,model_revision,model_sha256,ocr_route,text_cer,img_lpips,ssim,psnr_db,output_sha256,provider_oracle,notes
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,,,,"Paddle ch",0.074,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_run,Exact Qwen baseline reused from text-restoration benchmark
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-default,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.185,0.338,0.642,18.2,87729fe7e0482d2cead6d3b7c7dd386ecc75ab3d8658131c54e67528b8f9feeb,not_run,Fidelity gate failed; padded crop recognition CER 0.074 still shows two substituted characters; locally composited three-line edit; 20 DDIM steps; seed 0
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-mimic,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.222,0.345,0.644,17.9,f5198d8f2365b79880aba8ea6b0c3885142b9de21c9f7f4f6712ac4a0db8dd85,not_run,Fidelity gate failed; source glyph masks supplied as font hints; 20 DDIM steps; seed 0
1 date source_commit fixture variant model model_revision model_sha256 ocr_route text_cer img_lpips ssim psnr_db output_sha256 provider_oracle notes
2 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage Paddle ch 0.074 0.289 0.696 23.3 4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6 not_run Exact Qwen baseline reused from text-restoration benchmark
3 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage-anytext2-default AnyText2-SD1.5 b06c583a583818f3679665ef67b51363f107853c 7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8 Paddle ch 0.185 0.338 0.642 18.2 87729fe7e0482d2cead6d3b7c7dd386ecc75ab3d8658131c54e67528b8f9feeb not_run Fidelity gate failed; padded crop recognition CER 0.074 still shows two substituted characters; locally composited three-line edit; 20 DDIM steps; seed 0
4 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage-anytext2-mimic AnyText2-SD1.5 b06c583a583818f3679665ef67b51363f107853c 7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8 Paddle ch 0.222 0.345 0.644 17.9 f5198d8f2365b79880aba8ea6b0c3885142b9de21c9f7f4f6712ac4a0db8dd85 not_run Fidelity gate failed; source glyph masks supplied as font hints; 20 DDIM steps; seed 0
@@ -0,0 +1,5 @@
date,source_commit,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,zimage-face-only,0,,0.06604512532552083,18,0.927,0.028,0.612,0.013,0.959,34.7,c388e9bbf737bb9b22a7cb2216ffb8026321ca632f9db29e1e456b7c14a8553f,synthid_detected
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-global-only,0,0.153999996,,18,0.589,0.217,0.331,0.288,0.646,24.2,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-zimage-full,0,0.153999996,0.06604512532552083,18,0.852,0.050,0.492,0.252,0.671,24.7,69de191bddbfbf53348c7638bdba661b6d45d257c00a8a2646b2558670c5a4c2,clean
1 date source_commit fixture route seed global_strength face_strength face_count id_cos face_lpips lapvar_ratio img_lpips ssim psnr_db output_sha256 gemini_oracle
2 2026-08-13 2d03a00 Gemini_Generated_Image_y48j3cy48j3cy48j.png visible-metadata-control 0 18 9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b synthid_detected
3 2026-08-13 2d03a00 Gemini_Generated_Image_y48j3cy48j3cy48j.png zimage-face-only 0 0.06604512532552083 18 0.927 0.028 0.612 0.013 0.959 34.7 c388e9bbf737bb9b22a7cb2216ffb8026321ca632f9db29e1e456b7c14a8553f synthid_detected
4 2026-08-13 2d03a00 Gemini_Generated_Image_y48j3cy48j3cy48j.png qwen-global-only 0 0.153999996 18 0.589 0.217 0.331 0.288 0.646 24.2 281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f clean
5 2026-08-13 2d03a00 Gemini_Generated_Image_y48j3cy48j3cy48j.png qwen-zimage-full 0 0.153999996 0.06604512532552083 18 0.852 0.050 0.492 0.252 0.671 24.7 69de191bddbfbf53348c7638bdba661b6d45d257c00a8a2646b2558670c5a4c2 clean
@@ -1,4 +1,5 @@
{
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
"Gemini_Generated_Image_633uuy633uuy633u.png": "每天都是一个新的机会。\n用微笑开始它。\n世界也会向你微笑。"
}
@@ -0,0 +1,14 @@
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Paddle en+ru+ch",0.378,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.048,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.101,0.120,0.796,19.5,0.054799,15b40b8cdf9e97558c66d4acdcb97d46769e64f04f143f8061746bc731fbbc21,no_openai_signals_detected,source detected in same browser sequence
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.081,0.119,0.796,19.5,0.070267,5a8a0bcbe5763d1b1af0d64149cec7e35b16c5751e995d3bacf17581d64177ec,not_checked,fidelity-only follow-up
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.123,0.113,0.808,21.2,0.041279,900def5a433c039aa474789eddc4dc4c63b8126ba1aff6829d2e8d600b9f9c71,no_openai_signals_detected,source detected immediately afterward in the same Chrome sequence
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Paddle en+ru+ch",0.413,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.086,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.112,0.103,0.780,20.8,0.054912,846f39db0ad4a48a734a5019f0b0ff5d110eecb3210223407e72529dfa4fcf54,no_openai_signals_detected,source detected in same browser sequence
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.147,0.104,0.779,20.8,0.072556,8ae76e1dedeec26bebb8d35e71fd92d77f142dbb0f3440b330cdf33b79558afe,not_checked,fidelity-only follow-up
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.119,0.104,0.779,20.8,0.057110,ac4201bd2e342b1a37737319438cedb2203f6bb58cd9113f825cf9b80d913755,no_openai_signals_detected,source detected in same browser sequence
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-selective-automatic-passthrough,"Paddle detector ch; horizontal-only padded recognizer selection",0.074,0.289,0.696,23.3,0.000000,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_checked,all three source and candidate recognitions agreed exactly; output is byte-identical to candidate
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.127,0.113,0.805,20.9,0.041280,70c52fed8b8a1d98f4cda8558ae08825313e510dfb372e448370d8dcf1da73b0,not_checked,fidelity-only automatic-box follow-up; same 8/20 preserve decisions
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.154,0.104,0.778,20.8,0.057146,1fa01ce706042c003a61429d5d97fc6c928ec32278071a91926d588f1e760b00,not_checked,fidelity-only automatic-box follow-up; same 7/20 preserve decisions but worse CER
1 date source_commit fixture variant ocr_route text_cer img_lpips ssim psnr_db mask_fraction output_sha256 provider_oracle oracle_control
2 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage Paddle en+ru+ch 0.378 0.107 0.832 24.4 bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd no_openai_signals_detected source-positive control established earlier
3 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-restored Paddle en+ru+ch 0.048 0.174 0.703 13.9 0.154 3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6 no_openai_signals_detected qwen-zimage clean in same run
4 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-selective Paddle en+ru+ch 0.101 0.120 0.796 19.5 0.054799 15b40b8cdf9e97558c66d4acdcb97d46769e64f04f143f8061746bc731fbbc21 no_openai_signals_detected source detected in same browser sequence
5 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-selective-expanded-mask Paddle en+ru+ch 0.081 0.119 0.796 19.5 0.070267 5a8a0bcbe5763d1b1af0d64149cec7e35b16c5751e995d3bacf17581d64177ec not_checked fidelity-only follow-up
6 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-selective-automatic Paddle detector en+ru+ch; padded recognizer selection 0.123 0.113 0.808 21.2 0.041279 900def5a433c039aa474789eddc4dc4c63b8126ba1aff6829d2e8d600b9f9c71 no_openai_signals_detected source detected immediately afterward in the same Chrome sequence
7 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage Paddle en+ru+ch 0.413 0.095 0.818 26.0 83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713 no_openai_signals_detected source-positive control established earlier
8 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-restored Paddle en+ru+ch 0.086 0.162 0.682 14.6 0.170 95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d no_openai_signals_detected qwen-zimage clean in same run
9 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-selective Paddle en+ru+ch 0.112 0.103 0.780 20.8 0.054912 846f39db0ad4a48a734a5019f0b0ff5d110eecb3210223407e72529dfa4fcf54 no_openai_signals_detected source detected in same browser sequence
10 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-selective-expanded-mask Paddle en+ru+ch 0.147 0.104 0.779 20.8 0.072556 8ae76e1dedeec26bebb8d35e71fd92d77f142dbb0f3440b330cdf33b79558afe not_checked fidelity-only follow-up
11 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-selective-automatic Paddle detector en+ru+ch; padded recognizer selection 0.119 0.104 0.779 20.8 0.057110 ac4201bd2e342b1a37737319438cedb2203f6bb58cd9113f825cf9b80d913755 no_openai_signals_detected source detected in same browser sequence
12 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage-text-selective-automatic-passthrough Paddle detector ch; horizontal-only padded recognizer selection 0.074 0.289 0.696 23.3 0.000000 4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6 not_checked all three source and candidate recognitions agreed exactly; output is byte-identical to candidate
13 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-selective-detected-boxes Paddle ch detector boxes; 0.10-height recognizer padding 0.127 0.113 0.805 20.9 0.041280 70c52fed8b8a1d98f4cda8558ae08825313e510dfb372e448370d8dcf1da73b0 not_checked fidelity-only automatic-box follow-up; same 8/20 preserve decisions
14 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-selective-detected-boxes Paddle ch detector boxes; 0.10-height recognizer padding 0.154 0.104 0.778 20.8 0.057146 1fa01ce706042c003a61429d5d97fc6c928ec32278071a91926d588f1e760b00 not_checked fidelity-only automatic-box follow-up; same 7/20 preserve decisions but worse CER
+51
View File
@@ -0,0 +1,51 @@
{
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": [
{"box": [156, 206, 709, 296], "text": "This is a longer", "script": "alphabetic"},
{"box": [157, 293, 939, 379], "text": "sample text in English.", "script": "alphabetic"},
{"box": [165, 393, 774, 434], "text": "Typography can flow smoothly from", "script": "alphabetic"},
{"box": [161, 434, 521, 477], "text": "large to medium size.", "script": "alphabetic"},
{"box": [165, 495, 566, 522], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
{"box": [165, 524, 354, 550], "text": "rhythm, and clarity.", "script": "alphabetic"},
{"box": [165, 564, 448, 585], "text": "Even the finest text should remain", "script": "alphabetic"},
{"box": [165, 593, 330, 613], "text": "clean and readable.", "script": "alphabetic"},
{"box": [158, 673, 903, 740], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
{"box": [162, 753, 697, 793], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
{"box": [164, 807, 524, 829], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
{"box": [163, 845, 483, 868], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
{"box": [162, 951, 780, 1009], "text": "Это более длинный пример", "script": "alphabetic"},
{"box": [161, 1019, 723, 1062], "text": "текста на русском языке.", "script": "alphabetic"},
{"box": [165, 1076, 639, 1104], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
{"box": [165, 1111, 505, 1133], "text": "крупного размера к среднему.", "script": "alphabetic"},
{"box": [165, 1151, 513, 1172], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
{"box": [167, 1178, 287, 1192], "text": "ритм и ясность.", "script": "alphabetic"},
{"box": [165, 1210, 417, 1227], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
{"box": [165, 1235, 381, 1247], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
],
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": [
{"box": [165, 204, 754, 299], "text": "This is a longer", "script": "alphabetic"},
{"box": [167, 295, 1006, 385], "text": "sample text in English.", "script": "alphabetic"},
{"box": [171, 401, 831, 446], "text": "Typography can flow smoothly from", "script": "alphabetic"},
{"box": [169, 446, 560, 491], "text": "large to medium size.", "script": "alphabetic"},
{"box": [173, 507, 605, 536], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
{"box": [173, 538, 379, 567], "text": "rhythm, and clarity.", "script": "alphabetic"},
{"box": [175, 583, 481, 603], "text": "Even the finest text should remain", "script": "alphabetic"},
{"box": [173, 611, 352, 632], "text": "clean and readable.", "script": "alphabetic"},
{"box": [170, 693, 956, 757], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
{"box": [174, 775, 740, 812], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
{"box": [175, 829, 561, 852], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
{"box": [174, 869, 534, 892], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
{"box": [169, 958, 824, 1022], "text": "Это более длинный пример", "script": "alphabetic"},
{"box": [171, 1029, 764, 1074], "text": "текста на русском языке.", "script": "alphabetic"},
{"box": [175, 1090, 678, 1119], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
{"box": [175, 1125, 546, 1151], "text": "крупного размера к среднему.", "script": "alphabetic"},
{"box": [175, 1169, 552, 1192], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
{"box": [175, 1196, 312, 1217], "text": "ритм и ясность.", "script": "alphabetic"},
{"box": [175, 1233, 479, 1253], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
{"box": [177, 1259, 440, 1272], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
],
"Gemini_Generated_Image_633uuy633uuy633u.png": [
{"box": [1281, 650, 2357, 818], "text": "每天都是一个新的机会。", "script": "cjk"},
{"box": [1466, 788, 2167, 957], "text": "用微笑开始它。", "script": "cjk"},
{"box": [1367, 910, 2264, 1112], "text": "世界也会向你微笑。", "script": "cjk"}
]
}
@@ -0,0 +1,25 @@
date,source_commit,fixture,profile,route,seed,strength,adaptive_polish,text_cer,ocr_floor,added_cer,img_lpips,ssim,psnr_db,output_sha256,oracle_rechecked
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.378,0.127,0.251,0.107,0.832,24.4,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.356,0.127,0.229,0.125,0.707,25.3,9bb932d6e66b52f34ff07c3fcdbdb9c33eb601ae2cd84bec6b8c7b5e30df9926,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.413,0.127,0.286,0.095,0.818,26.0,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.422,0.127,0.295,0.099,0.754,25.9,7605b1175b0c99f481ae6929acc488f34d98bc0f2571d42a1d2972b671d9b8cb,false
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,original,reference,,,,0.037,0.037,0.000,0.000,1.000,inf,,not_applicable
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,visible-invisible-metadata,0,0.153999996,false,0.074,0.037,0.037,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,false
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,sdxl-zimage,visible-invisible-metadata,0,0.25,true,0.074,0.037,0.037,0.176,0.769,26.1,d9f89465c31bd2d38ffb111775b9bbea4577f40713fcbebf421e05d344ab27ad,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.068,0.127,-0.059,0.042,0.874,31.4,534de6422a3a03cfde44377aa9bf824a8267db3b73511240f8f07fb8dcb95eff,true
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.086,0.127,-0.041,0.047,0.862,30.8,4dba8211e8d84ebbfff72de00f54d4365d0d202394acd412d1419ae93d1e9f71,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.177,0.127,0.050,0.065,0.838,29.1,fc45c5fff6d33f15d30e05dd02580fe0addd97098b735b2894015f9abdaa43a7,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.248,0.127,0.121,0.093,0.813,27.3,1e55e11f0e289a752098370e9f1d53fed6cd4fab7ceff46fe86dd20c927f9ffb,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.343,0.127,0.216,0.120,0.787,25.7,f5a2d2829e88a790cff5bedc50fca1f878a92cdb392d530afb49d0d95c23b884,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.143,0.127,0.016,0.042,0.877,31.5,5cfa466e276ab37cf532f7458d6dd86ea4f0807e2cbdb6788b7c01ca6ee35a1a,true
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.101,0.127,-0.026,0.047,0.864,30.9,ade869ec7c5b08098495d9b9d67ee900773f189f94bbb411b043f242f5c2fa37,true
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.208,0.127,0.081,0.060,0.839,29.3,a11976963e3f2ca856ce66278e22064d51345d449fa6d672029039a79517d60a,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.341,0.127,0.214,0.070,0.819,27.6,1ab1ae075565e9d39960a80cf5d61e5b0c50d7e23283493ea0277056b3e964ab,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.538,0.127,0.411,0.084,0.799,25.6,5ff5a364d84f738de4c56ead7d4950a2f9a1aaf4bec16e97bda6a65915744a57,false
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.08,false,0.074,0.037,0.037,0.068,0.865,30.8,81419eddc3e5150956859dd10c26dd841bb93d9c8b34c01f344e6d69bca6edbf,true
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.10,false,0.074,0.037,0.037,0.086,0.844,29.8,4d1f32bd0dac0e05885ff943e7c10230b06de7d8547e6215f8e2941f732a1df5,true
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.15,false,0.000,0.037,-0.037,0.135,0.793,27.6,526c7987b34896f9513f7439b63b8a87573b59d15930fa307f3bb924435f3cec,true
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.20,false,0.000,0.037,-0.037,0.179,0.746,25.9,325364cf7a3b5d39ac0c44af82181753ca4d6fde0f1b522c896414191725e8fc,true
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.25,false,0.000,0.037,-0.037,0.218,0.703,24.5,f01d64dc0cabd45d66a93c1be39b819b988f29cda1f0a7bc0d5a1954571d807e,true
1 date source_commit fixture profile route seed strength adaptive_polish text_cer ocr_floor added_cer img_lpips ssim psnr_db output_sha256 oracle_rechecked
2 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png original reference 0.127 0.127 0.000 0.000 1.000 inf not_applicable
3 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage visible-invisible-metadata 0 0.11020972541176471 false 0.378 0.127 0.251 0.107 0.832 24.4 bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd false
4 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png sdxl-zimage visible-invisible-metadata 0 0.15 true 0.356 0.127 0.229 0.125 0.707 25.3 9bb932d6e66b52f34ff07c3fcdbdb9c33eb601ae2cd84bec6b8c7b5e30df9926 false
5 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png original reference 0.127 0.127 0.000 0.000 1.000 inf not_applicable
6 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage visible-invisible-metadata 0 0.11020972541176471 false 0.413 0.127 0.286 0.095 0.818 26.0 83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713 false
7 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png sdxl-zimage visible-invisible-metadata 0 0.15 true 0.422 0.127 0.295 0.099 0.754 25.9 7605b1175b0c99f481ae6929acc488f34d98bc0f2571d42a1d2972b671d9b8cb false
8 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png original reference 0.037 0.037 0.000 0.000 1.000 inf not_applicable
9 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage visible-invisible-metadata 0 0.153999996 false 0.074 0.037 0.037 0.289 0.696 23.3 4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6 false
10 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png sdxl-zimage visible-invisible-metadata 0 0.25 true 0.074 0.037 0.037 0.176 0.769 26.1 d9f89465c31bd2d38ffb111775b9bbea4577f40713fcbebf421e05d344ab27ad false
11 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.08 false 0.068 0.127 -0.059 0.042 0.874 31.4 534de6422a3a03cfde44377aa9bf824a8267db3b73511240f8f07fb8dcb95eff true
12 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.10 false 0.086 0.127 -0.041 0.047 0.862 30.8 4dba8211e8d84ebbfff72de00f54d4365d0d202394acd412d1419ae93d1e9f71 false
13 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.15 false 0.177 0.127 0.050 0.065 0.838 29.1 fc45c5fff6d33f15d30e05dd02580fe0addd97098b735b2894015f9abdaa43a7 false
14 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.20 false 0.248 0.127 0.121 0.093 0.813 27.3 1e55e11f0e289a752098370e9f1d53fed6cd4fab7ceff46fe86dd20c927f9ffb false
15 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.25 false 0.343 0.127 0.216 0.120 0.787 25.7 f5a2d2829e88a790cff5bedc50fca1f878a92cdb392d530afb49d0d95c23b884 false
16 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.08 false 0.143 0.127 0.016 0.042 0.877 31.5 5cfa466e276ab37cf532f7458d6dd86ea4f0807e2cbdb6788b7c01ca6ee35a1a true
17 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.10 false 0.101 0.127 -0.026 0.047 0.864 30.9 ade869ec7c5b08098495d9b9d67ee900773f189f94bbb411b043f242f5c2fa37 true
18 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.15 false 0.208 0.127 0.081 0.060 0.839 29.3 a11976963e3f2ca856ce66278e22064d51345d449fa6d672029039a79517d60a false
19 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.20 false 0.341 0.127 0.214 0.070 0.819 27.6 1ab1ae075565e9d39960a80cf5d61e5b0c50d7e23283493ea0277056b3e964ab false
20 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.25 false 0.538 0.127 0.411 0.084 0.799 25.6 5ff5a364d84f738de4c56ead7d4950a2f9a1aaf4bec16e97bda6a65915744a57 false
21 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.08 false 0.074 0.037 0.037 0.068 0.865 30.8 81419eddc3e5150956859dd10c26dd841bb93d9c8b34c01f344e6d69bca6edbf true
22 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.10 false 0.074 0.037 0.037 0.086 0.844 29.8 4d1f32bd0dac0e05885ff943e7c10230b06de7d8547e6215f8e2941f732a1df5 true
23 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.15 false 0.000 0.037 -0.037 0.135 0.793 27.6 526c7987b34896f9513f7439b63b8a87573b59d15930fa307f3bb924435f3cec true
24 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.20 false 0.000 0.037 -0.037 0.179 0.746 25.9 325364cf7a3b5d39ac0c44af82181753ca4d6fde0f1b522c896414191725e8fc true
25 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.25 false 0.000 0.037 -0.037 0.218 0.703 24.5 f01d64dc0cabd45d66a93c1be39b819b988f29cda1f0a7bc0d5a1954571d807e true
@@ -0,0 +1,7 @@
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.338,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.305,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,"Paddle ch",0.074,0.289,0.696,23.3,,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,synthid_detected,inconclusive account: clean baseline not reproduced
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-restored,"Paddle ch",0.111,0.328,0.653,18.1,0.051,2ab7666a4161ff0e12c53d9412b23d013970c7613a8a55a1b1ae8a2744e3290d,synthid_detected,inconclusive account: qwen-zimage control also detected
1 date source_commit fixture variant ocr_route text_cer img_lpips ssim psnr_db mask_fraction output_sha256 provider_oracle oracle_control
2 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage Vision en+ru; Paddle ch 0.338 0.107 0.832 24.4 bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd no_openai_signals_detected source-positive control established earlier
3 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-restored Vision en+ru; Paddle ch 0.007 0.174 0.703 13.9 0.154 3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6 no_openai_signals_detected qwen-zimage clean in same run
4 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage Vision en+ru; Paddle ch 0.305 0.095 0.818 26.0 83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713 no_openai_signals_detected source-positive control established earlier
5 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-restored Vision en+ru; Paddle ch 0.007 0.162 0.682 14.6 0.170 95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d no_openai_signals_detected qwen-zimage clean in same run
6 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage Paddle ch 0.074 0.289 0.696 23.3 4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6 synthid_detected inconclusive account: clean baseline not reproduced
7 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage-text-restored Paddle ch 0.111 0.328 0.653 18.1 0.051 2ab7666a4161ff0e12c53d9412b23d013970c7613a8a55a1b1ae8a2744e3290d synthid_detected inconclusive account: qwen-zimage control also detected
@@ -0,0 +1,6 @@
date,upstream_commit,workflow_sha256,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle,implementation
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/before/12.png,official-before,,,,10,,,,,,,056276464387c883f5fd89bd2f1415faeb425abb3153cd10efb25d1b9053bdc2,synthid_detected,upstream-published-bytes
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/after/12.png,official-after,,,,10,0.975,0.093,0.354,0.172,0.820,23.8,a822d805666224adb3e360b3ee89e14f20a341f1e7c4d3a5d3d5251e9286aefe,clean,upstream-published-bytes
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected,project-runtime-control
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-global-close-reproduction,0,0.153999996,,18,0.589,0.217,0.331,0.286,0.647,24.3,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean,diffsynth-bf16-yunet-sam
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-full-close-reproduction,0,0.153999996,0.13209025065104166,18,0.783,0.083,0.470,0.260,0.659,24.7,2d7a8f76f98b905f14d69e5f9e11979610c72c6be7b6ac298dec4659f17bd819,clean,diffsynth-bf16-yunet-sam
1 date upstream_commit workflow_sha256 fixture route seed global_strength face_strength face_count id_cos face_lpips lapvar_ratio img_lpips ssim psnr_db output_sha256 gemini_oracle implementation
2 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 comparison/before/12.png official-before 10 056276464387c883f5fd89bd2f1415faeb425abb3153cd10efb25d1b9053bdc2 synthid_detected upstream-published-bytes
3 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 comparison/after/12.png official-after 10 0.975 0.093 0.354 0.172 0.820 23.8 a822d805666224adb3e360b3ee89e14f20a341f1e7c4d3a5d3d5251e9286aefe clean upstream-published-bytes
4 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 Gemini_Generated_Image_y48j3cy48j3cy48j.png visible-metadata-control 0 18 9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b synthid_detected project-runtime-control
5 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 Gemini_Generated_Image_y48j3cy48j3cy48j.png upstream-global-close-reproduction 0 0.153999996 18 0.589 0.217 0.331 0.286 0.647 24.3 281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f clean diffsynth-bf16-yunet-sam
6 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 Gemini_Generated_Image_y48j3cy48j3cy48j.png upstream-full-close-reproduction 0 0.153999996 0.13209025065104166 18 0.783 0.083 0.470 0.260 0.659 24.7 2d7a8f76f98b905f14d69e5f9e11979610c72c6be7b6ac298dec4659f17bd819 clean diffsynth-bf16-yunet-sam
+2 -2
View File
@@ -1,3 +1,3 @@
date,source_url,source_sha256,source_width,source_height,source_fps,duration_seconds,source_verdict,vae,noise_std,long_side,fps,seed,output_sha256,output_verdict,output_verdict_text,output_detected_range,track,session_id,stratum,psnr_db,temporal_residual_ratio
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,,,,8,detected_all_frames,,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,,,,8,detected_all_frames,,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,unrecorded,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,unrecorded,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
1 date source_url source_sha256 source_width source_height source_fps duration_seconds source_verdict vae noise_std long_side fps seed output_sha256 output_verdict output_verdict_text output_detected_range track session_id stratum psnr_db temporal_residual_ratio
2 2026-07-31 https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4 79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d 1920 1080 24 8 detected_all_frames unrecorded 0.10 512 12 0 079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3 detected 26.2932 1.0072
3 2026-07-31 https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4 79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d 1920 1080 24 8 detected_all_frames unrecorded 0.15 512 12 0 1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e not_detected 25.3911 1.0578
+1 -1
View File
@@ -160,7 +160,7 @@ sparkle; the SynthID *pixel* itself has no local detector (oracle only).
Key takeaways:
- The same model differs by *surface*: Gemini app wraps C2PA, AI Studio (API/playground) emits none -- only the pixel + sparkle survive.
- Microsoft Designer's DALL-E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL-E).
- Microsoft Designer's DALL·E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL·E).
- Meta uses the IPTC `digitalSourceType` marker, not C2PA or SynthID.
- The open imwatermark fires only on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL) -- not from re-hosts (Krea, Stability hosted SDXL) or re-encoded design exports (Recraft, Canva). Ideogram's only signal is the EXIF `Make` tag.
- Bing and Grok web UIs are uncooperative for autonomous capture (no document_idle for screenshots; blob downloads intermittently no-op; low-res in-chat previews). Use their native download button manually if a full-res sample is needed.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.6 MiB

After

Width:  |  Height:  |  Size: 5.8 MiB

+87 -7
View File
@@ -16,7 +16,7 @@ defaults. This page focuses on choosing the right command.
| `metadata` and metadata-only `identify` | Default package |
| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` |
| Adobe TrustMark signals in `identify` | `remove-ai-watermarks[trustmark]` |
| Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` |
| `visible` and `erase` with OpenCV | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
| `visible` or `erase` with MI-GAN | `remove-ai-watermarks[migan]` |
| `visible` or `erase` with big-LaMa | `remove-ai-watermarks[lama]` |
@@ -26,7 +26,7 @@ defaults. This page focuses on choosing the right command.
| `video visible`, `video all`, and visible/all batch modes | `remove-ai-watermarks[video]` plus ffmpeg on PATH |
| `video invisible` and `video all --invisible` | `remove-ai-watermarks[video,diffusion]` plus ffmpeg on PATH |
| HEIC/HEIF/AVIF pixel input | Add `remove-ai-watermarks[heif]` |
| Every production command and backend | `remove-ai-watermarks[all]` |
| Every production command and Python-compatible backend | `remove-ai-watermarks[all]` |
`batch` requires the same extra as its selected mode. Extras can be combined in
one installation, for example `remove-ai-watermarks[visible,detect,heif]`.
@@ -40,7 +40,15 @@ remove-ai-watermarks identify image.png
`identify` always inspects supported metadata. When pixel extras are installed,
it also evaluates supported visible and invisible pixel signals. When no signal
is found, it reports the origin as unknown. It does not claim the image is
clean.
clean. For C2PA files, the text report shows asset integrity, claim-signature,
signer-trust, and signer-validity results separately. Confidence follows the
binding: an intact asset binding and claim signature is high confidence, and an
unanchored or expired signer is reported as a caveat rather than a lower score,
because no trust anchor list ships with the reader. A failed asset binding or
signature, or a revoked signing credential, does not confirm the claimed origin. When a structured
C2PA soft binding is present, the report also names its exact algorithm and
signed value; removing the manifest does not remove the referenced pixel
watermark or content fingerprint.
Machine readable output:
@@ -154,6 +162,11 @@ remove-ai-watermarks metadata image.png --remove -o clean.png
When `-o` is omitted, removal overwrites the source. Standard metadata is kept
unless you pass `--remove-all`.
A quiet `--check` or a successful `--remove` is not a clean verdict. The command
only inspects and strips embedded AI metadata; a pixel watermark such as SynthID
has no local decoder once that metadata proxy is gone. `identify` reports the
same limit.
The command also supports the audio and video containers listed in
[supported signals](supported-signals.md). ffmpeg must be available for the
non-ISOBMFF audio and video path.
@@ -222,11 +235,17 @@ remove-ai-watermarks video metadata input.mp4 --check
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
```
As with the generic `metadata` command, a quiet check or a successful strip is
not a clean verdict: video SynthID is not decoded locally after the metadata
proxy is gone.
Supported containers are MP4, MOV, M4V, WebM, MKV, AVI, and FLV. The operation
delegates to the same verified metadata scanner and stripper as the generic
`metadata` command, so detection and removal stay in parity. Video and audio
streams are not transcoded. For MP4 and MOV, this includes the native TC260
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`. The inspector
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`, plus the
QuickTime-form `meta` variants Doubao's iOS export writes (a bare `meta` box
as a direct `moov` child, and a keyless `hdlr=mdir` metadata list). The inspector
seeks past a large `mdat` to find a tail `moov`. Removal stream-copies the
container in bounded chunks, converts supported top-level provenance boxes to
same-size `free` boxes, and blanks the TC260 key/value in place. Box sizes,
@@ -298,7 +317,7 @@ remove-ai-watermarks video visible kling.mp4 --mark kling -o kling_clean.mp4
The command supports the moving Sora mascot and wordmark, two Veo
corner variants, the Seedance boxed `AI` label, the `Dola AI` text label, the
composite `MINIMAX | hailuo AI` label, and the bottom-right Kling label. Sora
composite `MINIMAX | hailuo AI` label, and the bottom-right Kling AI label. Sora
searches the whole frame at multiple scales. The other detectors search bounded
lower-frame regions with separate synthetic silhouettes. Kling additionally
requires its bright low-saturation label near the frame edge. Every mark
@@ -309,7 +328,7 @@ provenance-aware marks; metadata alone never creates a detection.
`--mark auto` is the default. It evaluates all providers in one decode pass and
selects the first stable match in specificity order: Sora, Veo, Seedance, Dola,
Hailuo, then Kling. Their confidence scores are independently calibrated and
Hailuo AI, then Kling AI. Their confidence scores are independently calibrated and
are not compared across providers. Pass an explicit `--mark` to scan only that
provider.
@@ -368,6 +387,23 @@ detected. Use `--force` when you know the image should be processed:
remove-ai-watermarks invisible image.png -o clean.png --force
```
### Choose a strength cohort
`--vendor` selects the cohort the default strength resolves from. `auto`
(the default) derives it from provenance: the C2PA issuer for
OpenAI/Google/Microsoft, and the standalone AI IPTC tag for Meta Content Seal
(Muse output carries no C2PA; the tag is a standard code, so C2PA evidence
always wins first). An explicit value both names the cohort for stripped files
and implies the scrub runs -- naming the cohort asserts the pixel watermark is
present -- exactly like `--force` plus a measured floor:
```bash
remove-ai-watermarks invisible muse_output.webp -o clean.png --vendor meta
```
The same option exists on `all` and `batch`, and as
`InvisibleOptions(vendor="meta")` in the Python API.
### Choose a pipeline
| Pipeline | When to use it |
@@ -395,6 +431,50 @@ schedule, CFG 1.0 and CUDA, so every one of those flags existed only to be refus
several layers down. They are not parsed at all now, which fails at the point the
user can act on rather than after a model load.
### Restore operator-verified text
`--text-manifest` enables the experimental `vae-glyphs` post-pass. It reconstructs
the source with the Qwen VAE, erases the annotated candidate glyphs with LaMa, and
composites only the reconstructed glyph cores through source-derived silhouettes. It
does not run OCR or choose which strings are correct. The optional `--fidelity-anchor`
described below additionally blends 15% of the reconstruction across the full frame.
Install the combined extra and run only with an operator-verified manifest:
```bash
uv tool install --force "remove-ai-watermarks[text-restoration]"
remove-ai-watermarks invisible image.png -o clean.png \
--pipeline qwen-zimage --text-manifest verified-lines.json --force
```
``verified: true`` may also be set by an automated operator that attests
machine-verified geometry: stability-gated detector boxes inside sane caps. Such
operators should use the geometry-only schema 2, which carries no transcription
or script metadata.
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default: it
was measured to return detector-visible OpenAI SynthID on poster-scale manifests
(official Content Provenance API, 2026-08-19). `--fidelity-anchor` restores the
0.27.0 research behavior; text-box fidelity lost by the default is well under one
MAE point on the measured fixtures.
The manifest is a JSON object with `verified: true`, decoded RGB dimensions,
`source_pixel_sha256`, and a non-empty `lines` array. Schema 1 is retained for
manually reviewed annotations: each line has an integer `[x1, y1, x2, y2]` box,
exact `text`, a non-empty `script`, and an optional angle from -30 to 30 degrees.
Schema 2 is geometry-only: each line has the box and optional angle, with no
required `text` or `script`. Lines must be in top-to-bottom, left-to-right order.
The hash binds the annotations to decoded RGB geometry and pixels, so metadata-only
container changes remain valid while a resized or edited source fails closed. The
experimental helper
`remove_ai_watermarks._internal.text_restoration.source_pixel_sha256` computes it.
This mode is supported only by `qwen-zimage` at native geometry with
`humanize=0`, `unsharp=0`, and adaptive polish disabled. `all` also accepts the flag,
but its manifest must match the pixels entering the invisible stage; if visible-mark
removal changes those pixels, the hash check rejects the run. One oracle verdict does
not certify another manifest, seed, model/runtime version, or output hash.
### Work with limited memory
Lower CUDA memory pressure:
@@ -426,7 +506,7 @@ It is a memory strategy, not a guarantee of better quality.
The `all` command and the `all` installation extra are separate concepts. The
command runs every applicable stage. Installing `remove-ai-watermarks[all]`
makes every production backend available; a smaller installation such as
makes every backend compatible with the active Python available; a smaller installation such as
`remove-ai-watermarks[visible,qwen-zimage]` can also run the command with fewer
optional backends.
+5
View File
@@ -7,6 +7,11 @@ This page records notices required by source dependencies and licensed derivativ
- The DWT-DCT implementation derives from ShieldMnt's
[`invisible-watermark`](https://github.com/ShieldMnt/invisible-watermark), licensed
under MIT. Its notice ships in `src/remove_ai_watermarks/licenses/invisible-watermark-MIT.txt`.
The decode path is a vectorized reformulation, not a transcription: it produces
the same bits and is checked against upstream's own decoder, but it no longer
corresponds line by line to `imwatermark/maxDct.py`. Diffing the two files will
show structurally different code, which is expected and does not mean the
derivation notice is stale.
## Licensed test fixtures
+2 -2
View File
@@ -521,7 +521,7 @@ held by its masked ArcFace vector -- no original pixel copied.
drift, NOT a perfect face swap. Set expectations; PuLID/InstantID are the higher-fidelity (heavier)
paths if needed.
- **Value scales with strength:** at low strength (OpenAI 0.10) faces barely drift, so FaceID is
marginal; at the higher strength a hard vendor (Google 0.30) needs, FaceID earns its keep.
marginal; at the higher strength then used for a hard vendor (Google 0.30), FaceID earned its keep.
### Build plan (staged)
@@ -641,7 +641,7 @@ photo only).
**Sources.** https://arxiv.org/abs/2206.11253 (CodeFormer) · https://github.com/sczhou/CodeFormer ·
https://arxiv.org/pdf/2401.07519 (InstantID) ·
https://openaccess.thecvf.com/content/WACV2024/papers/Suin_Diffuse_and_Restore... (region-adaptive) ·
https://openaccess.thecvf.com/content/WACV2024/html/Suin_Diffuse_and_Restore_A_Region-Adaptive_Diffusion_Model_for_Identity-Preserving_Blind_WACV_2024_paper.html (region-adaptive) ·
https://arxiv.org/pdf/2504.12809 (saliency-aware watermark removal)
## Provenance
+14 -1
View File
@@ -1,5 +1,9 @@
# Development
Standalone evaluation, calibration, and release utilities are catalogued in
[`scripts/README.md`](../scripts/README.md). They are maintainer tools rather than
installed commands, and their local data inputs remain outside the repository.
Read this reference for environment setup, dependency recovery, CI behavior, and fixture policy. The always-loaded invariants remain in [`.claude/rules/development.md`](../.claude/rules/development.md).
## Local environment
@@ -12,9 +16,18 @@ Read this reference for environment setup, dependency recovery, CI behavior, and
The optional TrustMark decoder downloads weights into its installed package directory. After pruning that extra, a leftover weights directory can make availability checks see an empty namespace package. If Pyright reports an unknown `TrustMark` import and `find_spec("trustmark")` returns a loader-less spec, remove that regenerable remnant from the active virtual environment and resync.
### Known security-gate blocks
`maintain.sh` fails while PyPI ships no fixed release for a transitive CVE. Current case (triaged 2026-08-17): `lightning` 2.6.5, pulled only by the optional `trustmark` extra, carries PYSEC-2026-3624 (RCE via `load_from_checkpoint` on an attacker-crafted checkpoint). The vulnerable API is unreachable here, because the TrustMark decoder loads only its own pinned weights downloaded from the TrustMark release, never a user-supplied checkpoint. The upstream fix is merged but unreleased, and ignores are never added, so run and report the remaining core checks (Ruff, Pyright, tests) separately until a fixed `lightning` release lands; bump it with `uv lock --upgrade-package lightning` as soon as one does.
## CI
`.github/workflows/test.yml` runs Ruff, a cross-platform test matrix over the supported-Python floor and the latest supported minor with default plus development dependencies, and a separate job that installs ffmpeg on Ubuntu to run the full-clip video test. Diffusion and model-running tests skip in that matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the OpenCV eraser remain covered across operating systems.
`.github/workflows/test.yml` runs Ruff, a test matrix over every supported Python
minor with default plus development dependencies, cross-platform coverage at the
oldest and newest minors, and a separate job that installs ffmpeg on Ubuntu to
run the full-clip video test. Diffusion and model-running tests skip in that
matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the
OpenCV eraser remain covered across operating systems.
Keep `uv.lock` compatible with `uv sync --frozen`. Dependency pull-request checks use GitHub's merge result against current `main`; if `main` moves, merge it locally and rerun the full gate because a newer linter can expose stale directives in later code.
+25 -7
View File
@@ -1,6 +1,6 @@
# Installation
Python 3.10.1 or newer is required.
Python 3.11 through 3.14 are supported.
## Default metadata mode
@@ -87,6 +87,15 @@ removal, metadata stripping and every `identify` command still run anywhere.
Video SynthID regeneration is a separate VAE path and does still run on CPU or MPS;
it needs the `diffusion` extra, not this one.
The experimental verified-text post-pass additionally needs LaMa:
```bash
uv tool install --force "remove-ai-watermarks[text-restoration]"
```
That extra includes `qwen-zimage` and `lama`; it does not add OCR. Text strings and
line boxes must be reviewed before the run.
## Feature extras
Extras are composable. Install only the capabilities and file formats the
@@ -99,12 +108,14 @@ application actually uses:
| `visible` | Visible mark detection, OpenCV inpainting, and manual erasing | `pixels` | No |
| `video` | Visible video identification/removal and timestamp preservation | `visible`, PyAV | No |
| `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No |
| `trustmark` | Adobe TrustMark detection | trustmark | Yes |
| `trustmark` | Adobe TrustMark detection on Python 3.11-3.12 | trustmark | Yes |
| `diffusion` | Torch and Diffusers runtime; video SynthID regeneration | `pixels`, Torch, Diffusers | Yes |
| `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
| `qwen-zimage` | Invisible image-watermark removal, both CUDA-only profiles | `diffusion`, DiffSynth | Yes |
| `all` | Every production feature | All rows above | Yes |
| `text-restoration` | Opt-in verified Qwen-VAE glyph restoration | `qwen-zimage`, `lama` | Yes |
| `text-draft` | Draft OCR proposals for operator verification | PaddleOCR, PaddlePaddle | Model download, no Torch |
| `all` | Every production feature available on the active Python | All compatible rows above | Yes |
| `dev` | Tests, linting, typing, and upstream parity checks | `video`, `detect`, upstream invisible-watermark | Yes, for parity tests |
Dependency composition:
@@ -118,13 +129,20 @@ flowchart LR
migan --> visible
lama --> visible
qwen["qwen-zimage"] --> diffusion
text["text-restoration"] --> qwen
text --> lama
draft["text-draft"]
heif
trustmark
```
`heif` and `trustmark` are independent branches. Combine them explicitly with
another feature when required. The `all` bundle contains every production
branch but never includes `dev`.
`heif`, `trustmark`, and `text-draft` are independent branches. Combine them
explicitly with another feature when required. `text-draft` is excluded from
`all` because it proposes unverified OCR annotations and is not a production
removal path. TrustMark requires NumPy 1.x, which has no
CPython 3.13 or 3.14 wheels, so that branch is available only on Python
3.11-3.12. The `all` bundle contains every production branch compatible with
the active Python and never includes `dev`.
Examples:
@@ -141,7 +159,7 @@ uv tool install --force "remove-ai-watermarks[video]"
# DWT-DCT and TrustMark detection without diffusion removal
uv tool install --force "remove-ai-watermarks[detect,trustmark]"
# Every production capability
# Every production capability compatible with this Python
uv tool install --force "remove-ai-watermarks[all]"
# An arbitrary minimal combination
+88 -12
View File
@@ -31,11 +31,14 @@ an unsupported locale, a different position, or a crop may be missed.
Known examples:
- The Microsoft detector covers one calibrated top-right white pill. Microsoft's
[documented feature](https://support.microsoft.com/en-us/topic/include-a-watermark-when-content-from-microsoft-365-is-ai-generated-b00a656e-ae61-4692-8086-67d004421030)
can instead use a Copilot icon, `AI-Generated` text, or another position.
- Samsung detection is calibrated for the Italian
`Contenuti generati dall'AI` text variant.
- The Jimeng top-left pill has a weak visual detector and is intentionally
subject to additional product and background checks.
- Kling support covers the calibrated variants rather than every Kling label.
- Kling AI support covers the calibrated variants rather than every Kling AI label.
Use `erase --region` when you can see and select an unsupported or missed mark.
@@ -57,12 +60,42 @@ Invisible removal does not decode and delete a payload. It regenerates the
image through a diffusion pipeline. Faces, text, colors, and fine detail can
change even when the watermark is successfully disrupted.
Text, tables, and UI screenshots are the worst case for a forced scrub: the
model redraws glyphs as plausible-but-wrong shapes. Do not run the invisible
stage on such content unless a locally-detectable invisible signal exists.
The default `all`, `invisible`, and `batch` commands already skip it then, and
`visible` (which strips AI metadata by default) or `metadata` alone never
redraws a glyph outside the filled watermark box. `--force` is for content you
know carries a pixel watermark despite no local signal; on a clean screenshot
it only buys distortion. Preserving or pasting back the original text pixels
during a real scrub is deliberately not offered: an invisible watermark such
as SynthID lives in the pixels everywhere, including inside the glyphs, so
frozen text regions would keep the watermark
([text protection research](text-protection-research.md)).
`qwen-zimage` is the default profile and `sdxl-zimage` the only alternative.
Both are CUDA only and differ only in the global regeneration model: each
conditions that stage on a canny edge map, which preserves structure but not
identity or exact texture, and each then runs the same face stage.
`qwen-zimage` is the higher fidelity of the two. Both are large, slow, and may
still alter small text or difficult faces.
Existing face evaluations favor `qwen-zimage`, but there is no blanket fidelity
ordering across content types. A fixed-seed, three-scene text comparison at the
profile defaults found no stable winner: SDXL won one poster, Qwen won one, and
the Chinese sign tied. Both are large, slow, and may still alter small text or
difficult faces. The measurements and their OCR and oracle caveats are tracked
in [`data/evaluations/fidelity/`](../data/evaluations/fidelity/README.md).
A global Z-Image Turbo prototype preserved text substantially better at low
strength, but it has no useful cross-provider operating point and is not a
supported profile. Automatic text restorers also remain research-only:
fresh-font and silhouette variants visibly changed typography. The higher-fidelity
`vae-glyphs` route is available only as an experimental opt-in with verified strings
and line geometry. It builds its donor internally but still requires an independently
clean global anchor. Automatic OCR and line-box proposals are not reliable enough to
remove those requirements, and exact oracle results do not establish a general mask,
seed, runtime, or provider operating range. Qwen-Image-2.0 is hosted-only and exposes
no equivalent low-strength denoise control. Exact experiments, controls, and pass
rates are kept in
[`text-protection-research.md`](text-protection-research.md) and the
[`fidelity` evaluation record](../data/evaluations/fidelity/README.md).
### Removal cannot be verified locally for proprietary SynthID
@@ -81,6 +114,14 @@ For important outputs:
Provider systems can change, so a result verified on one file, seed, or version
is not a permanent certification.
Meta Content Seal (Muse Image) is in the same family: no local decoder, presence
recognizable only through the XMP `trainedAlgorithmicMedia` companion tag while
that metadata survives, and removal verifiable only through the anonymous
`meta.ai/identification` oracle, which rate-limits by IP per day. The oracle's
negative is weaker than its positive: Reuters measured it missing 55% of cropped
Muse images, so treat a negative on a cropped or heavily edited file as
inconclusive rather than clean.
### Video SynthID removal is lossy and content-dependent
The `video invisible` command and `remove_video_invisible` API regenerate video
@@ -112,6 +153,21 @@ measured 25.39 dB paired PSNR and a 1.058 motion-compensated temporal-residual
ratio. This is one carrier, not a universal guarantee; hashes and exact verdicts
are tracked in `data/evaluations/video-synthid-oracle.csv`.
### The invisible video path does not touch the audio track
`remove_video_invisible` regenerates the video stream and copies the source audio
verbatim: the extracted audio bitstream of input and output has an identical
sha256, measured on two carriers. It also strips every metadata marker, so the
output can report clean from `get_ai_metadata` and `identify_video` while carrying
an untouched generated audio track. Google's verifier scores audio and visual
tracks separately, and a track this path never modifies is a track it cannot have
cleaned.
Whether a given carrier's audio actually holds a mark the verifier reads has not
been established -- that needs a provider verdict, which has not been obtained.
Treat a clean local report on a clip with generated audio as unproven, not as a
guarantee, and check the audio separately when it matters.
The shipped engine streams sampled frames in bounded batches, computes its
fidelity metrics incrementally, and pipes regenerated pixels directly to
ffmpeg. Its frame and latent memory is therefore bounded by `--batch-size`
@@ -132,9 +188,14 @@ path has not yet answered.
The two profiles resolve an unset strength differently, because different things
were measured for each.
`qwen-zimage` reads it from image area, through the resolution-adaptive denoise
curve. The vendor is deliberately ignored: the curve, not the issuer, is what was
calibrated.
`qwen-zimage` reads unknown content from the resolution-adaptive denoise curve.
Measured provider cohorts instead take flat operating points: OpenAI `0.07675`,
Google `0.27`, and Microsoft InvisMark `0.15`. OpenAI and Microsoft add one full
observed cross-source boundary spread to the worst clean source; Microsoft's three
first-clean boundaries were `0.04125`, `0.055`, and `0.095`, giving `0.14875` before
rounding up. Google's candidate was separately repeated across three valid sources
and three accounts. The small corpora make these operating points, not universal
thresholds.
`sdxl-zimage` reads it from the C2PA issuer, on a flat ladder:
@@ -160,6 +221,15 @@ certified at a fixed seed. The live resolver is
| `qwen-zimage` | CUDA only, large model stack, and limited broad certification across seeds and content. |
| `sdxl-zimage` | CUDA only. Its strength ladder is flat per vendor, not a resolution curve, because flat values are what was measured. |
Only manually verified `vae-glyphs` is an optional production stage, and it is
experimental rather than a default.
OCR plus LaMa recovered literal poster text but changed fonts and worsened whole-image
fidelity. Restricting it to OCR-mismatched lines improved the tradeoff but still
left a local shadow on one poster. The published AnyText2 SD1.5 checkpoint
substituted Chinese characters and increased CER on the sign fixture. AnyText2XL
is not published, and the released wrapper truncates individual text lines after
20 characters.
The `controlnet`, `sdxl`, `qwen` and `default` profiles were removed, not aliased
onward: a retired name is rejected at parse time rather than routed into a profile
the caller never chose. There is no `--model`, `--steps`, `--guidance-scale`,
@@ -243,11 +313,11 @@ supported AI provenance metadata without transcoding streams.
`video visible` and `remove_video_visible` additionally support the moving
Sora 2 mascot and wordmark, the current Veo four-point diamond, the legacy
`Veo` text, the Seedance boxed `AI` label, the fixed `Dola AI` text, the Hailuo
MINIMAX/Hailuo composite label, and the bottom-right Kling label with its
`Veo` text, the Seedance boxed `AI` label, the fixed `Dola AI` text, the Hailuo AI
MINIMAX/Hailuo AI composite label, and the bottom-right Kling AI label with its
version suffix. Detection requires a recurring visual candidate across
adjacent frames. Fixed-mark candidates must remain anchored rather than
drifting with a scene object. Kling also requires a bright low-saturation
drifting with a scene object. Kling AI also requires a bright low-saturation
candidate near the expected frame edge. Provider provenance can recover
low-contrast runs only after visual evidence exists for the marks that define a
provenance prior, so metadata alone does not erase a clean API export.
@@ -258,7 +328,7 @@ policies and selects the first stable result in specificity order. Use an
explicit mark when the provider is already known.
Historical Sora Turbo exports use a small OpenAI swirl in the corner rather
than the moving mascot-and-wordmark design; that earlier variant is not
detected by the `sora` video mark. Hailuo and Kling coverage is specific to the
detected by the `sora` video mark. Hailuo AI and Kling AI coverage is specific to the
verified lower-edge layouts; a new provider layout needs a separate calibrated
silhouette. Other provider video labels are not supported yet. Google video
SynthID has an oracle-certified VAE removal path, while other proprietary
@@ -347,8 +417,14 @@ The `trustmark` extra adds Adobe TrustMark decoding. The implementation retains
an additional JPEG re-encode gate and requires the binary payload and schema to
remain identical because isolated decoder hits can otherwise be content noise.
It accepts Variant P schemas 0-2. Variant Q requires a different model, and
schema 3 is rejected at the measured precision threshold. The calibration
history is in [module internals](module-internals.md#metadata-and-provenance).
schema 3 is rejected at the measured precision threshold. Its NumPy 1.x runtime
limits the extra to Python 3.11-3.12; the rest of the package remains supported
through Python 3.14. The current TrustMark dependency line also resolves lightning 2.6.5
(PYSEC-2026-3624, no fixed release yet); the vulnerable `load_from_checkpoint`
path is unreachable here because TrustMark loads its checksummed checkpoints
with plain `torch.load`. Bump lightning and cut a patch release when a fix
ships. The calibration history is in
[module internals](module-internals.md#metadata-and-provenance).
External AI versus real image classifiers are out of scope. The project
identifies concrete local provenance signals instead of shipping a generic
+280 -19
View File
@@ -42,6 +42,10 @@ Important contracts:
- `invisible` writes no output when no supported local signal is found, unless
`--force` is supplied.
- The two no-signal conditions currently share exit code `2`.
- A quiet `metadata --check` and a successful `metadata --remove` (same for
`video metadata`) end by repeating the `identify` limit: the pixel channel is
untouched and a watermark such as SynthID has no local decoder once its
metadata proxy is gone, so neither outcome is a clean verdict.
- Hard processing and write failures exit with code `1`.
- `all` can still write the completed visible and metadata stages when the
diffusion dependencies are unavailable, but exits with code `1` so the
@@ -158,7 +162,14 @@ Native MP4/MOV TC260 labels follow TC260-PG-20257A:
`moov.udta.meta.keys` maps an `AIGC` key to a raw JSON value in `ilst`.
[`_internal/isobmff.py`](../src/remove_ai_watermarks/_internal/isobmff.py) walks those
nested boxes by seeking, so detection reaches a tail `moov` without reading the
preceding `mdat`. The MP4/MOV/M4V/M4A removal path first validates the top-level
preceding `mdat`. Two Doubao iOS variants sit outside that normative placement
and are covered by the same walker (2026-08-17 corpus findings, both previously
undetected): a QuickTime-form `meta` box as a *direct* `moov` child (no FullBox
header, disambiguated by probing the child-box offset), and a QuickTime
`hdlr=mdir` metadata list under `udta.meta` whose `ilst` data items carry the
validated JSON with no `keys` box at all (content-validated, so only genuine
TC260 JSON matches; the keyless entry has no key name to blank, so its value
alone is spaced out). The MP4/MOV/M4V/M4A removal path first validates the top-level
box walk, then copies the source to a sibling temporary file in bounded chunks.
Supported C2PA/JUMBF/AI-label boxes become same-size `free` boxes with blank
payloads; TC260 removal changes the four-byte key to `free` and blanks only the
@@ -285,13 +296,13 @@ from the gate. The full-clip oracle floor is
detected while `0.15` did not.
[`video_visible.py`](../src/remove_ai_watermarks/video_visible.py) implements
the first pixel stages for Sora, Veo, Seedance, Dola, Hailuo, and Kling. The
the first pixel stages for Sora, Veo, Seedance, Dola, Hailuo AI, and Kling AI. The
Sora detector searches a normalized frame with a fully synthetic
mascot-and-text silhouette at several scales. The Veo detector uses separate
synthetic silhouettes for the current four-point diamond and legacy `Veo`
text. Seedance uses a synthetic rounded boxed-`AI` silhouette, while Dola uses
an OpenCV-font `Dola AI` silhouette. Hailuo uses a synthetic waveform,
MINIMAX/Hailuo text, separator, and ring. Kling combines synthetic font
an OpenCV-font `Dola AI` silhouette. Hailuo AI uses a synthetic waveform,
MINIMAX/Hailuo AI text, separator, and ring. Kling AI combines synthetic font
variants with a ring approximation of its swirl; the logo path rescues
wordmarks whose version or font differs, while the edge and white-label gates
reject recurring scene texture. All fixed-mark searches are bounded to the
@@ -312,23 +323,23 @@ Every per-frame result is untrusted. Each provider's floors, minimum-run policy,
fill padding and mask style are one row in `VISIBLE_MARK_POLICIES`, and every mark
enters the same `stabilize_localizations` entry point; the recurrence
implementation underneath knows nothing about providers. That policy row also
carries `accepts_provenance`, which forces `provenance=False` for Hailuo and Kling
carries `accepts_provenance`, which forces `provenance=False` for Hailuo AI and Kling AI
— they have no metadata that could confirm them, and the guarantee used to be
structural (their wrappers took no `provenance` parameter at all). Provenance can relax a low-contrast run only
after recurring visual evidence exists. Sora transition frames follow the
nearest confirmed moving position only with Sora provenance. Seedance, Dola,
Hailuo, and Kling additionally require candidates to remain anchored to the
Hailuo AI, and Kling AI additionally require candidates to remain anchored to the
start of a run. This rejects slowly drifting scene details that still have
high frame-to-frame overlap. Hailuo and Kling do not infer provenance from
high frame-to-frame overlap. Hailuo AI and Kling AI do not infer provenance from
technical encoder tags; their confirmed public samples carried no provider
metadata.
Removal runs in a second decode pass. Sora, legacy Veo text, Dola text,
Seedance, Hailuo, and Kling use box masks. Seedance deliberately fills the
Seedance, Hailuo AI, and Kling AI use box masks. Seedance deliberately fills the
complete localized box: a synthetic outline mask passed repeat detection but
left part of the real translucent border visible during visual end-to-end
review. Hailuo expands beyond the matched core to cover both provider icons.
Kling expands around the wordmark or swirl to include the version and optional
review. Hailuo AI expands beyond the matched core to cover both provider icons.
Kling AI expands around the wordmark or swirl to include the version and optional
`PRO` suffix. The square Veo diamond uses a synthetic shape mask so transparent
corners do not erase unrelated pixels. Every mask goes through the shared
`watermark_registry.fill` backends. ffmpeg encodes the changed video stream and
@@ -364,6 +375,75 @@ Regression coverage:
official `c2pa-python` reader first. Its byte-level PNG parser remains a fallback
for partial and synthetic fixtures that the official reader rejects.
Structured extraction is limited to the active manifest and the ingredient
manifests reachable from it. Validation is preserved as separate dimensions:
asset binding integrity, claim signature, signer trust, and signer certificate
validity. A matching asset hash and valid claim signature do not make an
untrusted or expired signer trusted, so those stay separate fields and separate
caveats -- but they do not lower confidence. `identify` assigns high confidence
when the asset binding and the claim signature both validate, medium confidence
to a claim that validated nothing (fallback parsing, unknown dimensions), and no
origin verdict to a binding failure, a signature failure, or a revoked signing
credential. The failed claim remains in the marker inventory and keeps the removal
gate fail-safe because a post-signing container edit can invalidate C2PA without
removing a declared pixel watermark.
Revocation reaches the disqualifying branch through `signer_validity`, not through
binding or signature. A check that read only the latter two returned a confident AI
verdict off a credential the issuer had disowned, with an empty `integrity_clashes`
-- quieter than a hash mismatch on the same file. Certificate expiry is deliberately
not disqualifying: an expired certificate does not imply the signed bytes changed,
and a signature actually made outside validity already arrives as
`claimSignature.outsideValidity`.
One rule, one place. `_validation_fields` maps status codes to the four dimensions
and also emits `c2pa_failed_codes`, the subset of failures that actually drove a
dimension to invalid; `c2pa_info_has_invalid_credential` maps those dimensions to
disqualified, and both the ingredient-reachability walk and the report consume that
single path. The displayed reason comes from `c2pa_failed_codes` rather than a
substring rescan of the full code list, so what a caller is shown as the cause
cannot become a looser rule than the verdict it explains.
The structured walk also treats an exact known AI product in a reachable
`claim_generator` as an AI assertion. This covers update chains where the active
manifest names only `c2pa-tool` while a validated ingredient names Dreamina, and
Firefly chains that identify `Adobe_Firefly` without repeating a digital source
type. Unreachable manifests remain excluded.
Reachable `c2pa.soft-binding*` assertions retain their exact `alg` and bounded,
printable block `value` in addition to the normalized vendor label. A block value
without its algorithm is not surfaced because it cannot be attributed to a
decoder or registry entry. `com.microsoft.invismark.1` uses that value as the
pixel-watermark identifier in Microsoft Paint output. An InvisMark soft binding
keeps the invisible-removal gate fail-safe even when the C2PA asset binding has
since become invalid, because metadata damage does not prove the pixel carrier
disappeared. `identify` retains the generic `soft_binding` signal for schema-1
compatibility and adds `invismark` as the stable pixel-removal signal.
Content-fingerprint soft bindings do not trigger pixel regeneration.
The SDK default enables trust verification but supplies no production trust
anchors. Consequently, an installation without an explicitly maintained C2PA
trust bundle reports every otherwise valid signer chain as untrusted --
`signingCredential.trusted` appears in no default installation, from any vendor.
Confidence therefore must not depend on it. It did from 0.27.0 through 0.30.0, which made
the high-confidence branch unreachable in production while a hand-built fixture
stamping `signingCredential.trusted` kept it green in the suite; measured on the
committed provenance fixtures, every C2PA file from OpenAI, Adobe and Black Forest
Labs came back untrusted, and an intact manifest scored the same medium as a
fallback parse that validated nothing. That collapsed the one distinction the
official reader exists to draw.
`tests/test_identify.py::TestIdentifyRealSamples::test_no_committed_fixture_reports_a_trusted_signer`
is the reachability guard: it asserts the fixtures really are untrusted and still
reach high confidence.
Read `untrusted` here as a missing input, not a finding: nothing was checked,
because there was nothing to check against. If a maintained trust bundle is ever
configured, that stops being true and the confidence mapping in
`_c2pa_credential_level` must be re-read, because only then does a failed trust
check mean the signer was rejected. Shipping or fetching an official trust bundle
requires a separate update, provenance, and availability policy; do not silently
convert `signingCredential.untrusted` into trusted based on a vendor-name match.
Vendor attribution comes from the registry in
[`_internal/constants.py`](../src/remove_ai_watermarks/_internal/constants.py). Derived
issuer and platform maps should not be maintained separately.
@@ -371,7 +451,9 @@ For an AI C2PA claim, a recognized product in `claim_generator` takes precedence
over the certificate issuer: an application can sign through an upstream model
provider without becoming that provider's product. Only exact product mappings
receive this precedence; an unknown claim generator still falls back to issuer
attribution.
attribution. An unmapped issuer org reads as unknown-signer C2PA with no platform;
that is how Ideogram was surfaced (4 corpus uploads signed "Ideogram, Inc",
2026-08-08) before its vendor row was added on 2026-08-27.
### Metadata scanning and stripping
@@ -555,9 +637,11 @@ the function it replaces does — no provenance means no relaxation, and an unkn
invisible target means scrub rather than skip.
The `detect` extra composes the shared `pixels` runtime with PyWavelets. Its
in-tree [`dwt_dct.py`](../src/remove_ai_watermarks/dwt_dct.py) decoder preserves
the upstream matrix algorithm without installing Torch or non-headless OpenCV.
The upstream MIT notice ships inside the wheel under `licenses/`.
in-tree [`dwt_dct.py`](../src/remove_ai_watermarks/dwt_dct.py) decoder reproduces
the upstream algorithm's output bit for bit without installing Torch or
non-headless OpenCV; the block scan is vectorized rather than transcribed, so
the file no longer reads line by line against `maxDct.py`. The upstream MIT
notice ships inside the wheel under `licenses/`.
`is_ai_generated` is `True` or `None`; absence of evidence is not reported as a
human-made verdict. `ai_source_kind` distinguishes fully generated content from
@@ -572,11 +656,63 @@ schemas 0-2 count as positives. Schema 3 is below the precision threshold: all
same false payload after re-encoding. The official Adobe Variant P schema-1
fixture in `data/fixtures/provenance/` is the positive regression control.
#### Why the DWT-DCT decoder looks the way it does
Two things in `dwt_dct.py` are load-bearing and neither is obvious from the code.
`_approximation` calls `pywt.dwt` twice instead of `pywt.dwt2`, and transposes
before each pass. A factorial ablation over both axes separates the two:
skipping the three detail bands `dwt2` computes and this code discards is worth
**4%**, while the transposes are worth **2.8x**, because pywt walks the axis it
transforms and on axis 0 of a C-contiguous plane that is a column walk. The
arithmetic saving is the intuitive explanation and it is the small term; four
independent profiles named it as the mechanism before the ablation contradicted
them.
Bit-identity is a hard requirement, not a preference. For uint8 input the exact
Haar LL value is a multiple of 0.5 and the bit test is `peak % 36 > 18.0`, a
threshold sitting exactly on a representable value that ~1 block in 72 lands on,
so a 1-ulp difference deterministically flips real bits. That is why the ~16x
available from a hand-rolled numpy Haar is unreachable rather than merely
untaken: pywt's C convolution contracts into an FMA that numpy has no ufunc for,
and `np.longdouble` is 64-bit on arm64 macOS.
Each pass is one flat `pywt.downcoef` call over a raveled strip rather than
`pywt.dwt(..., axis=1)[0]`, and the plane is processed in strips of `_STRIP`
block-rows so no full-plane float64 intermediate is ever materialized. The strip
height is not a tuned value: 8 through 64 all scored inside each other's noise
with unstable ordering, and only "strips at all" versus whole-plane matters.
`_approximation`'s even-last-axis check is load-bearing, not defensive. Haar's
filter is length 2, so an even row length keeps every pair inside its own row;
on an odd width the pairs walk across row boundaries and the reshape still
succeeds whenever the total is even, which would be wrong bits with no
exception. Both call sites are even by construction today, and
`TestRaveledHaarPass` pins both halves -- the `downcoef`/`dwt` equivalence,
which a pywt upgrade could take away, and the raise on an odd width.
Measured on a 1536x2816 image, all arms timed in one process: the decoder went
0.112 s (the original per-block Python loop) to 0.011 s vectorized to 0.007 s
with strips, and a warm `identify()` 1.757 s to 1.365 s on the first step and a
further 0.4% on the second. That last figure is the point at which this target
is finished: the decoder is now under 2% of `identify()`, so speed here has
stopped buying anything. What the strips buy is peak RSS in the stage, 111 MB to
21 MB on a 4.3 MP image, which is what matters on the memory-limited Space.
Both steps were verified by recording decoder output and detector verdict over
200 sampled `data/` images plus two synthesized carriers before and after: the
record is byte-identical, as are seven degenerate shapes (`1x65536` through
`8x8192`, plus an odd width) that clear the caller's area check.
Regression coverage:
- [`test_identify.py`](../tests/test_identify.py)
- [`test_trustmark_detector.py`](../tests/test_trustmark_detector.py)
- [`test_invisible_watermark.py`](../tests/test_invisible_watermark.py)
- [`test_invisible_watermark.py`](../tests/test_invisible_watermark.py) --
note `test_in_tree_decoder_matches_upstream` is the parity guard against
upstream's own decoder, and the whole module is `skipif(not is_available())`.
A green run without the `detect` extra installed has not checked parity at
all, so a decoder change still owes the before/after verdict record.
## Visible mark removal
@@ -612,9 +748,9 @@ capture-less pill is too weak to attribute.
The set of marks that veto the pill is DERIVED from the registry rows: every mark
under the same label regime (`tc260`) belonging to a different product. It used to
be a hand-written list of keys, and that list drifted -- LibLibAI was registered
alongside RunningHub and Baidu, both of which were added to it, and LibLibAI was
not, so a confident LibLibAI detection did not suppress the pill the way its two
be a hand-written list of keys, and that list drifted -- LiblibAI was registered
alongside RunningHub and Baidu, both of which were added to it, and LiblibAI was
not, so a confident LiblibAI detection did not suppress the pill the way its two
siblings did. Marks outside the TC260 regime (Gemini, Samsung) are deliberately
not vetoers: neither can put `jimeng` into `provenance`, so neither can enable the
arm it would be vetoing.
@@ -666,6 +802,29 @@ be represented by the shared base:
- [`runninghub_engine.py`](../src/remove_ai_watermarks/runninghub_engine.py)
- [`baidu_engine.py`](../src/remove_ai_watermarks/baidu_engine.py)
- [`liblib_engine.py`](../src/remove_ai_watermarks/liblib_engine.py)
- [`microsoft_engine.py`](../src/remove_ai_watermarks/microsoft_engine.py)
The measured Microsoft badge variant (2026-08-27 registration) is the first
`tr`-corner mark and the first `long`-side scale basis: the pill tracks the
render dimension, so a
1024x1536 portrait carries the same pill as 1536x1024, and a width basis
undersized the template by the aspect ratio (portrait carriers fell to
0.15-0.32 NCC until the basis was measured). The silhouette is a white pill with
its synthetic internal shapes knocked out - the holes are what separate it from
any other bright rounded corner element (a plain white pill scores below the gate
in the tests). It does not claim coverage of Microsoft's other documented icon,
wording, or position variants.
The 2026-08-27 rerun used the registered engine through
`scripts/registered_mark_calibrate.py`, rather than a copied detector
configuration. The manifest kept three evidence classes separate: 17 visually
confirmed carriers, 343 Microsoft-provenance files without a visual adjudication,
and 1200 non-overlapping no-signal controls. At the strict 0.38 gate, 15/17
confirmed carriers fired (min 0.249, p50 0.519, p90 0.578, max 0.579), while
0/1200 controls fired (p99 0.200, max 0.293). The provenance cohort produced
78/343 fires, but that is not a recall measurement because provenance identifies
the provider, not the presence of this visible layout. No provenance relaxation
ships until that cohort is visually labeled.
The detector and removal mask must use compatible geometry. A detector that
fires while producing an empty or misplaced mask is a removal failure even if
@@ -684,7 +843,7 @@ were briefly skipped there for exactly that reason.
A mark whose removable footprint differs from what the detector localizes
overrides `_footprint_rect` (which policy) and `_extend_match_box` (how far the
box grows), not the whole `footprint_mask`. Baidu extends right to the corner tag
and LibLibAI extends left to the triangle logo; both inherit every guard around
and LiblibAI extends left to the triangle logo; both inherit every guard around
that arithmetic.
Yuanbao uses the polarity-independent `contrast` front end because its standard
@@ -742,6 +901,11 @@ are CUDA-only. `controlnet`, `sdxl`, `qwen` and `default` were removed rather th
kept as a CPU path, and are rejected rather than aliased onward. There is no
content-dependent automatic router.
`qwen-zimage` normally resolves global denoise from image area for unknown content.
Measured provider cohorts bypass that curve with flat operating points. The values,
measurement derivations, and corpus limits are canonical in
[`known-limitations.md`](known-limitations.md#strength-is-content-and-seed-dependent).
For serverless cold starts, `InvisibleEngine.preload(global_only=True)` loads the
mandatory global stage and YuNet while leaving the optional Z-Image and SAM face
stack lazy until a face is detected. The default `preload()` still loads every
@@ -871,6 +1035,65 @@ orchestration, YuNet integration, SAM selection, masks, sizing helpers, and pixe
compositing are implemented for this runtime. Changing a calibrated model input
requires the same provider-oracle and identity evaluation as a model change.
#### Verified text restoration
[`_internal/text_restoration.py`](../src/remove_ai_watermarks/_internal/text_restoration.py)
implements the opt-in `vae-glyphs` stage. A versioned manifest carries manually
reviewed strings and source-space line boxes in schema 1, or verified source-space
geometry alone in schema 2, plus a SHA-256 over decoded RGB width, height, and pixels.
Validation happens before model loading. The library never treats OCR confidence as
verification, and geometry-only operators do not need to invent text or script fields.
When enabled, `QwenZImagePipeline` reconstructs the source once through its already
loaded Qwen VAE, runs the ordinary global and face stages, and calls the shared
restoration compositor. The optional fidelity anchor first blends 15% of the VAE
reconstruction into the clean result; it is off by default because the blend returned
detector-visible OpenAI SynthID in the measured poster fixtures. The compositor derives
binary source and candidate silhouettes, groups nearby lines, uses LaMa for the initial
and residual-glyph erase passes, paints fresh silhouette edges, then copies the Qwen-VAE
core with a 0.5-pixel feather. The evaluation script
imports these same mask and compositing helpers so the two implementations cannot
silently drift. Silhouette crops start 12% of line height beyond each horizontal
side, then expand each side independently while a foreground component anchored
inside the detector box still reaches that boundary, up to one line height. They
extend 8% above and 25% below. The anchored-component gate covers clipped leading
flourishes, trailing punctuation, icons, and descenders without walking into
disconnected decoration or background texture.
The stage is deliberately narrower than the engine: it rejects `sdxl-zimage`, tiles,
resolution caps, humanize, unsharp, and adaptive polish. Those combinations change
geometry or final pixels after the verified layer and have no measured oracle result.
It remains opt-in because annotations are manual and provider verdicts apply only to
the exact tested output hashes, not to the mechanism in general.
A matched stage-isolation check on the 18-face Gemini portrait grid confirms the
division of responsibility. The visible-cleaned, metadata-stripped control and the
Z-Image face-only output were both SynthID-positive; Qwen global-only and the full
Qwen-then-Z-Image output were both clean. The face stage raised identity cosine from
0.589 to 0.852 and reduced face LPIPS from 0.217 to 0.050 without reintroducing a
detectable whole-image signal. Thus Z-Image is a masked fidelity repair stage here,
not the watermark-removal stage. Exact hashes, metrics, strengths, and the one-fixture,
one-seed caveat are recorded in
[`data/evaluations/fidelity/face-stage-isolation-2026-08-13.csv`](../data/evaluations/fidelity/face-stage-isolation-2026-08-13.csv).
The public Synthid-Bypass v2 graph was subsequently audited at upstream commit
`3007d035`. Its saved-output path confirms the same division: Qwen-Image-2512
Lightning plus Canny is global, and Z-Image Turbo exists only inside the masked
face detailer. The connected face path is YOLOv8-face plus SAM; the MediaPipe nodes
described by the upstream README and the 1.2-megapixel normalization node do not
reach `SaveImage`. Upstream also applies its adaptive face strength directly,
whereas this implementation multiplies it by `FACE_DENOISE_SCALE = 0.5`.
A close reproduction on the same portrait fixture kept the control positive and
made both Qwen global-only and full Qwen-then-Z-Image outputs clean. Applying the
upstream-strength face pass raised identity from 0.589 to 0.783 and reduced face
LPIPS from 0.217 to 0.083, but remained worse than this profile's 0.852 and 0.050.
The published upstream pair 12 was also independently checked positive before and
clean after, with 0.975 identity. Exact workflow provenance, hashes, metrics,
oracle outcomes, and the DiffSynth/GGUF, scheduler, detector, and seed caveats are
recorded in
[`data/evaluations/fidelity/upstream-v2-reproduction-2026-08-13.csv`](../data/evaluations/fidelity/upstream-v2-reproduction-2026-08-13.csv).
### SDXL plus Z-Image
[`_internal/sdxl_zimage_pipeline.py`](../src/remove_ai_watermarks/_internal/sdxl_zimage_pipeline.py)
@@ -953,6 +1176,44 @@ sizes the curve's own values verify clean, which is what a resolution-scaled
requirement would predict. There is no measured size at which the shipped curve fails,
so it is left alone.
### Meta Content Seal boundaries for qwen-zimage
Measured 2026-08-26 against the anonymous oracle `meta.ai/identification`
(verdicts read from the settled page text; corpus and full row-level provenance in
`data/contentseal/manifest.csv`). Muse Image carries no C2PA, so
`vendor_for_strength` returns None for it and these images ride the default
resolution-adaptive curve; there is deliberately no `meta` entry in the flat vendor
map, because no provenance signal can establish that cohort automatically and an
unreachable constant would be dead code.
| source (2.56 MP generation) | detected at | clean from |
|---|---|---|
| gen_lighthouse_watercolor | 0.0525 | 0.06 |
| gen_fox_forest | 0.03 | 0.0375 |
| gen_night_city | 0.03 | 0.0375 |
| gen_studio_mug | -- | 0.03 |
| gen_text_poster | -- | 0.015 |
Full spread: worst first-clean boundary (0.0525, 0.06] on lighthouse, easiest
source already clean at 0.015. Following the same derivation as the OpenAI and
Microsoft floors (worst clean boundary plus one full observed cross-source
spread): 0.06 + (0.0525 - 0.015) = 0.0975, rounded up to **0.1**. Shipped as
`QWEN_ZIMAGE_META_STRENGTH` with two routing paths: auto mode routes a file
whose only provenance is the standalone AI IPTC tag onto the cohort
(`vendor_for_strength` checks C2PA issuers first, so Google/OpenAI/Microsoft
evidence always wins, and the tag's other users ship no invisible watermark
this profile targets), and `--vendor meta` / `InvisibleOptions.vendor` names the
cohort explicitly on stripped files, implying the scrub runs (naming the cohort
asserts the watermark is present). sdxl-zimage has no measured Meta rung and an
explicit meta vendor there falls to the conservative unknown 0.25. The default
resolution-adaptive curve (~0.1305 at 2.56 MP) also clears every measured
source, so default behavior needed no change. Oracle verdicts carry a
generation ID and creation timestamp embedded in the watermark payload; both
survived the 512 px resize and JPEG q85 rows, so payload recovery outlives the
detection threshold. Oracle session limits are per-IP, server-side, and
sliding-window: clearing cookies and storage does not reset them, and a burst
exhausts the window minutes after it reopens.
### Static prompt embeddings
Both stages prompt with module constants, and at CFG 1.0 DiffSynth's
+99 -3
View File
@@ -112,6 +112,18 @@ raiw.remove_all(
)
```
`vendor="meta"` names a strength cohort explicitly (the measured Content Seal
floor) for a stripped file whose provenance no longer carries the AI IPTC tag,
and implies the scrub runs:
```python
raiw.remove_all(
"muse_output.webp",
"clean.png",
invisible=InvisibleOptions(vendor="meta"),
)
```
`InvisibleOptions` carries only what `InvisibleEngine` itself takes, and uses the
engine's own parameter names and defaults. `force`, which decides whether the
engine runs at all, is a parameter of `remove_all` and `remove_batch` alongside
@@ -155,8 +167,34 @@ from remove_ai_watermarks.identify import identify
report = identify(Path("input.png"))
print(report.platform)
print(report.signals)
print(report.c2pa_validation)
```
`c2pa_validation`, when present, reports `integrity`, `signature`,
`signer_trust`, and `signer_validity` independently, plus the reader status
codes. A valid hash and signature is a high-confidence signed claim; an
unanchored or expired signer appears in `caveats` and in these fields, not as a
lower confidence, because the reader ships no trust anchors to check against and
`signer_trust` is therefore a missing input rather than a finding. A hash or
signature failure, or a revoked signing credential, does not confirm the claimed
platform or AI origin.
A consumer must read `integrity_clashes`. When a credential fails validation,
`is_ai_generated` becomes `None`, because a claim that cannot be tied to these
bytes cannot establish origin -- the manifest may have been transplanted from a
real AI image onto anything -- and the failure is reported in
`integrity_clashes` instead. That is a different question from whether an AI
watermark is physically present in the pixels, which is what
`has_invisible_target` answers, and it stays fail-safe `True` on the same file.
Reading only `is_ai_generated` turns a broken vendor manifest into silence.
`c2pa_validation["state"]` is the reader's own aggregate and is carried for
diagnostics only; no verdict is derived from it, because it collapses a
transplanted manifest and a merely expired certificate into one `Invalid`, and
its `Trusted` level depends on anchors no default installation has. Fallback parsing reports unknown validation
dimensions, while a raw marker in an unsupported or malformed container can
leave `c2pa_validation` as `None`.
Use `check_visible=False` and `check_invisible=False` for metadata-only
inspection through the compatible path-based API:
@@ -224,6 +262,11 @@ the schema 1 serializer remains available for rolling upgrades. Asking a release
an unsupported schema raises `ValueError` rather than silently returning another
shape.
Microsoft InvisMark declarations emit both the compatible generic `soft_binding`
signal and the additive `invismark` signal. Consumers should use `invismark` to route
the image through pixel removal; the generic signal also covers content fingerprints
that must not trigger regeneration.
A record carries metadata regions, not the primary coded-pixel stream: marker
segments before the JPEG scan, every PNG chunk except `IDAT`, RIFF chunks except the
coded image, the ISOBMFF provenance boxes, the container's trailer, the parsed EXIF tags the
@@ -422,7 +465,8 @@ must use the same container extension as the source.
The returned `VideoMetadataResult` records the source, output, metadata detected
before removal, and any markers remaining after the verified strip. MP4/MOV
inspection recognizes the native TC260 `AIGC` entry in
`moov.udta.meta.keys/ilst`; its removal preserves container size and encoded
`moov.udta.meta.keys/ilst` and the QuickTime-form `meta` variants Doubao's iOS
export writes; its removal preserves container size and encoded
stream bytes. MP4/MOV/M4V are copied in bounded chunks, so a large `mdat` is not
loaded into memory; publication is atomic. MKV/WebM inspection recognizes the corresponding
`Segment.Tags.Tag.SimpleTag` representation; its removal requires ffmpeg for a
@@ -519,8 +563,8 @@ independently and are not compared across detectors. Pass one of those explicit
values to restrict the scan to a single provider. The Veo detector recognizes
the current four-point diamond and the
legacy `Veo` text. Seedance recognizes the boxed `AI` label, Dola recognizes
its compact text label, Hailuo recognizes the composite MINIMAX/Hailuo label,
and Kling recognizes its bottom-right logo, wordmark, and version suffix. Each
its compact text label, Hailuo AI recognizes the composite MINIMAX/Hailuo AI label,
and Kling AI recognizes its bottom-right logo, wordmark, and version suffix. Each
variant has an independent synthetic silhouette and calibrated temporal policy.
After each accepted frame is filled, `temporal_consistency=True` motion-aligns
the preceding accepted fill and blends it only when the warped prior mask
@@ -587,6 +631,58 @@ engine = InvisibleEngine(pipeline="sdxl-zimage")
The `qwen-zimage` extra is required for both profiles: each runs the same
DiffSynth Z-Image face stage.
The opt-in verified-text stage uses the same `text_manifest` argument as the CLI:
```python
engine.remove_watermark(
Path("watermarked.png"),
Path("clean.png"),
text_manifest=Path("verified-lines.json"),
)
```
Install `remove-ai-watermarks[text-restoration]`. The manifest schema and safety
constraints are documented in the CLI guide. The engine verifies its decoded RGB
hash before loading the diffusion models and rejects SDXL, downscaling, and
postprocessing combinations that were not evaluated. Tiling is also rejected because
the combined tiled-restoration path has no provider-oracle calibration. `InvisibleOptions`
exposes the same field for `remove_all`; after a visible-stage edit, the manifest must
be built against the staged pixels rather than the pristine source.
Use manifest schema 1 for manually reviewed text plus script metadata. Automated
operators that verify only text-region geometry should emit schema 2 lines with a
`box` and optional `angle`; no placeholder transcription or script is required.
Since 0.27.1 the mode's global 15% Qwen-VAE fidelity-anchor blend is **off by
default** (`fidelity_anchor=False`): that whole-frame blend was measured to
return detector-visible OpenAI SynthID on poster-scale manifests (official
Content Provenance API, 2026-08-19 - detected x6 with the anchor, clean x6
without it, controls and base outputs validated in the same sessions). Pass
`fidelity_anchor=True` to reproduce the 0.27.0 research behavior.
### Drafting manifest lines
`remove_ai_watermarks.text_draft` proposes lines for a manifest; it never
produces verified ones:
```python
from remove_ai_watermarks.text_draft import draft_text_lines
draft = draft_text_lines(Path("watermarked.png"))
for line in draft.accepted:
print(line.box, line.script, line.min_score, line.text)
```
Install `remove-ai-watermarks[text-draft]` (CPU, no torch: PaddleOCR detection
plus three script-chosen recognition engines). A line lands in `accepted` only
when three crop paddings normalize identically and every confidence clears
`min_score` (default 0.85); `accepted` means crop-stable, NOT ground-truth-
correct - on the reference posters the draft's exact-text precision was 90.0%
and 94.4% because high-confidence OCR still lost punctuation. Every accepted
line needs a human yes/no before a manifest may claim `verified: true`.
`source_pixel_sha256` is re-exported here for building the manifest's
pixel-binding hash against the exact source the engine will decode.
`remove_watermark` takes strength, seed, tiling, resolution, and postprocessing
controls. It takes no model id, step count or guidance scale, and neither does the
constructor: each profile pins its model stack, its per-stage schedule and CFG
+30 -2
View File
@@ -11,7 +11,7 @@ A release is complete only after all four published surfaces are verified:
|---|---|---|
| PyPI | The `remove-ai-watermarks` wheel and source distribution | `publish.yml` |
| Homebrew | The `remove-ai-watermarks` formula in `wiltodelta/homebrew-tap` | `distribute.yml` |
| Hugging Face Space | A factory rebuild of `wiltodelta/remove-ai-watermarks` against the new PyPI release | `distribute.yml` |
| Hugging Face Space | The Space demo deployed on the new release through a requirements-pin bump in `wiltodelta/raiw-hf-space` | `raiw-hf-space` sync workflow; the pin bump is manual, `distribute.yml` only re-installs the pinned version |
| ComfyUI Registry | A compatible release of `wiltodelta/ComfyUI-remove-ai-watermarks` with its own node version | `distribute.yml` and the node repository's workflows |
The GitHub Release is the trigger and release record for this flow. Conda is
@@ -29,6 +29,10 @@ The package version appears in:
Update the first two, then refresh the lock file with uv. Do not edit a
line-number-specific location in `uv.lock`; its package order changes.
Before choosing a version, verify it is absent from git tags, GitHub Releases,
and PyPI. An unpublished remote tag still reserves that version; never move it,
publish the next version instead.
## Publish flow
PyPI publishing is triggered by a published GitHub Release, not by a tag push
@@ -58,9 +62,17 @@ PyPI API token from the repository.
waits for the matching source distribution to appear on PyPI, then:
- updates the Homebrew tap formula URL and SHA-256;
- triggers a factory rebuild of the Hugging Face Space;
- triggers a factory rebuild of the Hugging Face Space: this re-installs the
version pinned in the Space repo, it does NOT upgrade the demo;
- synchronizes, tests, versions, and publishes the ComfyUI nodes.
Upgrading the Hugging Face Space is a separate, manual step: bump
`remove-ai-watermarks[visible,heif]` in `wiltodelta/raiw-hf-space` (`pyproject.toml`,
`uv lock`, and the re-exported `requirements.txt`), then push. That repository's
`sync-to-hf.yml` mirrors the files onto the Space, which rebuilds on the new
pin. Its callback smoke tests run in that repository's CI on every push; they
are the scenario check for the demo surface.
The workflow can also be started manually with an optional version input.
If a distribution job fails because a repository or Hugging Face credential is
@@ -116,4 +128,20 @@ After publication, verify:
- the Homebrew formula points to the new source distribution;
- the distribution workflow completed successfully;
- the ComfyUI Registry node requires the new library version;
- the Hugging Face Space serves the new release: its `requirements.txt` pin
matches (raw file via the HF API), the runtime stage is `RUNNING` after the
rebuild, and one live `identify` and one live `visible` API call succeed
against the running Space;
- a clean install can run `remove-ai-watermarks --version`.
A clean-install check run immediately after publication can fail with "no
version of remove-ai-watermarks==X.Y.Z" even though the simple index already
lists it: uv serves its cached pre-release view of the index. Re-run with a
fresh cache (`UV_CACHE_DIR=$(mktemp -d)`) before treating it as a release
failure.
The ComfyUI sync run can fail the same way on its own: it resolves the version
from the PyPI JSON API (which updates first) but installs the test dependency
with pip against the simple index, whose CDN edges lag by minutes. Rerun the
failed `distribute.yml` comfyui job once the simple index lists the release;
nothing about the release itself is wrong.
+73 -8
View File
@@ -10,7 +10,7 @@ The `visible` command registers these mark keys:
| Key | Mark | Expected area | Important limit |
| --- | --- | --- | --- |
| `gemini` | Google Gemini sparkle | Usually bottom right | Detection includes a false positive gate. |
| `gemini` | Google Gemini visible watermark (sparkle) | Usually bottom right | Detection includes a false positive gate. |
| `doubao` | `豆包AI生成` | Bottom right | Vendor specific text detector. |
| `jimeng` | `★ 即梦AI` | Bottom right | Vendor specific text detector. |
| `qwen` | `千问AI生成` | Bottom right | Strict visual gate. |
@@ -19,7 +19,8 @@ The `visible` command registers these mark keys:
| `samsung` | `✦ Contenuti generati dall'AI` | Bottom left | Calibrated for the Italian text variant. |
| `runninghub` | `RunningHub AI生成` | Top left | Strict visual and position gates. |
| `baidu` | `百度 AI生成` | Bottom right | Detector and extended removal footprint. |
| `liblib` | `LibLibAI` | Bottom center | Includes a minimum image size gate. |
| `liblib` | `LiblibAI` | Bottom center | Includes a minimum image size gate. |
| `microsoft` | One Microsoft white AI-badge variant | Top right | Strict gate; other documented icon, text, and position variants are not covered. |
| `jimeng_pill` | `AI生成` pill | Top left | Weak detector with additional product and background gates. |
`--mark auto` evaluates all registered marks and removes every selected match.
@@ -37,8 +38,8 @@ when you can select the affected area yourself.
| `veo` | Current four-point diamond and legacy `Veo` text | Fixed bottom-right corner | Uses separate silhouettes and requires a recurring match; learned fill is preferable on structured backgrounds. |
| `seedance` | Boxed `AI` label | Fixed bottom-right corner | Requires an anchored recurring match; the full localized box is filled because a thinner synthetic shape mask leaves the real translucent rim behind. |
| `dola` | `Dola AI` text | Fixed bottom-right corner | Requires an anchored recurring match; ByteDance or BytePlus provenance can relax only an existing visual run. |
| `hailuo` | `MINIMAX \| hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. |
| `kling` | Kling swirl, `KLING AI`, version, and optional `PRO` suffix | Fixed bottom-right edge | Combines a synthetic logo rescue with font variants, an edge gate, a white-label gate, and anchored temporal recurrence. |
| `hailuo` | `MINIMAX \| hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. A TC260 label naming MiniMax as producer can relax only an existing stable run. |
| `kling` | Kling AI swirl, `KLING AI`, version, and optional `PRO` suffix | Fixed bottom-right edge | Combines a synthetic logo rescue with font variants, an edge gate, a white-label gate, and anchored temporal recurrence. |
`video identify`, `video visible`, and `video all` share this registry and the
same temporal arbiter. It is separate from the image registry because selection
@@ -86,7 +87,13 @@ The inspection and stripping code handles signals in these groups:
`identify` combines detected signals into a `ProvenanceReport`. It reports
unknown when evidence is absent. It never treats missing metadata as proof that
an image is human made.
an image is human made. C2PA presence alone is not a verified identity: the
report distinguishes asset binding, claim signature, signer trust, and signer
validity. High-confidence C2PA attribution requires an intact asset binding and
claim signature; signer trust and certificate expiry are reported as their own
dimensions and as caveats, since no trust anchor list ships to evaluate them.
Fallback claims, which validate nothing, remain medium-confidence, while a failed
binding or signature or a revoked credential contributes no origin verdict.
## File and container formats
@@ -134,6 +141,60 @@ all-media watermark policy, and current OpenAI C2PA carrying an explicit
assert SynthID. After provenance metadata is removed, a local negative result
is still inconclusive.
Microsoft Paint can name `com.microsoft.invismark.1` in a C2PA soft-binding
assertion. Inspection reports both that exact algorithm and its signed `value`,
which Paint uses as the identifier carried by the pixel watermark, and emits an
additive `invismark` signal so callers can select pixel removal without parsing
the generic `soft_binding` detail. Photos uses a parallel local writer path.
Metadata stripping removes only the embedded manifest; `invisible` and `all`
guarantee the supported InvisMark removal contract by regenerating the pixel
layer as well. The project has no validated local InvisMark decoder, so local
inspection cannot independently verify the output. Microsoft's official
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
is the external oracle: it reports pixel `Watermark` and embedded `C2PA` results
separately; a control-positive, output-negative pair is the available per-file
verification path. The API needs Azure credentials; the page a human can check
without an account is <https://ai.azure.com/nextgen/validate>. Its verdict is
weaker than the API's: it collapses watermark and C2PA into one rendered result
and tops out at `Inconclusive` rather than a watermark-negative, so treat
`Inconclusive` on a processed file as "not confirmed", not as "detected still".
Meta Muse Image stamps every output with Content Seal, a proprietary invisible
pixel watermark, and ships no visible mark (the legacy `Imagined with AI`
corner mark belongs to the pre-2026 Imagine pipeline and is not registered).
This project has no local Content Seal decoder. Meta Model API outputs and
Meta CDN copies carry an XMP `iptcExt:DigitalSourceType =
trainedAlgorithmicMedia` companion tag, which `identify` reports through the
existing Made-with-AI path; that IPTC code is a standard, not a Meta-exclusive
signal, so it cannot key a strength cohort the way the C2PA issuer does. Since
0.33.0 a standalone AI digital-source tag (no C2PA manifest) additionally emits
the additive `content_seal` signal - the strength router's Meta bet as evidence,
medium confidence, with the same caveat - so clients select pixel removal from
the signal list exactly the way InvisMark is additive over `soft_binding`. It
is an attribution, never a decode. The
external oracle is `https://meta.ai/identification`: anonymous, no login,
accepts image, video, and audio, enforces an unspecified daily identification
limit, and answers with model attribution (`Muse Image 1 - Meta`) plus a
per-generation ID and creation timestamp read from the watermark payload. No
identification endpoint exists in the Meta Model API itself (the reference at
dev.meta.ai documents only generation and edits for images), and the official
documentation never mentions the seal. The
default `qwen-zimage` profile clears Content Seal at the default
resolution-adaptive strength (oracle-verified on 2.56 MP generations); measured
strength boundaries are recorded in `data/contentseal/manifest.csv` and
[module internals](module-internals.md#meta-content-seal-boundaries-for-qwen-zimage).
The derived Meta floor (0.1 by the standard spread method) ships as a measured
cohort: auto mode routes a file whose only provenance is the standalone AI IPTC
tag to it (the tag is not Meta-exclusive; other tag users ship no invisible
watermark this profile targets, and Google/OpenAI/Microsoft C2PA evidence
always wins first), and `invisible --vendor meta` (also `all` and `batch`, and
`InvisibleOptions.vendor` in the API) names the cohort explicitly on stripped
files. An explicit vendor implies the scrub runs: naming the cohort asserts the
pixel watermark is present.
The seal survives resizing, JPEG recompression, and metadata stripping; it dies
to center crops of a third to a half, matching the Reuters 2026-07-11 finding
that Meta's detector missed 55% of cropped Muse images.
For MP4, MOV, and M4V, `video invisible` or the explicit
`video all --invisible` option can regenerate the video through a VAE and strip
source metadata. The shipped profile is oracle-certified, but it is not a local
@@ -154,16 +215,20 @@ not a universal clean verdict.
| Google Gemini | Sparkle | Diffusion regeneration for SynthID | C2PA and related source signals |
| Google Veo video | Veo diamond and legacy text | Oracle-certified VAE removal for SynthID | C2PA and related source signals |
| OpenAI image generators | None registered | Diffusion regeneration for supported invisible signals | C2PA and generator provenance |
| Meta Muse Image | None on Muse output (legacy `Imagined with AI` unregistered) | Diffusion regeneration for Content Seal, oracle-verified on the default profile | XMP IPTC `trainedAlgorithmicMedia` companion tag; no local Content Seal decoder |
| Microsoft Paint and Photos | None registered | External Microsoft oracle for InvisMark; no validated local decoder | Paint C2PA soft-binding algorithm and identifier |
| Microsoft image outputs (measured variant) | One top-right white AI-badge variant | No registered pixel decoder | C2PA attribution |
| Stable Diffusion and SDXL | None registered | Diffusion regeneration; optional open decoder | Embedded parameters and text metadata |
| FLUX | None registered | Diffusion regeneration; optional open decoder | C2PA for supported sources |
| Adobe Firefly | None registered | Optional TrustMark Variant P decoder | C2PA |
| Midjourney | None registered | No registered pixel decoder | EXIF, XMP, and IPTC signals |
| Luma AI | None registered | No registered pixel decoder | PNG text generator tags (Uni-1) |
| ByteDance generators | Doubao and Jimeng marks | No registered pixel decoder | TC260 AIGC, supported C2PA, and exact app-export AIGC disclosures |
| Qwen | Qwen mark | No registered pixel decoder | TC260 AIGC |
| Kling | Kling image and video marks | No registered pixel decoder | TC260 AIGC |
| Hailuo / MiniMax video | Hailuo composite video label | No registered pixel decoder | TC260 AIGC where present |
| Kling AI | Kling AI image and video marks | No registered pixel decoder | TC260 AIGC |
| Hailuo AI / MiniMax video | Hailuo AI composite video label | No registered pixel decoder | TC260 AIGC where present |
| Baidu | Baidu mark | No registered pixel decoder | TC260 AIGC |
| LibLibAI | LibLibAI mark | No registered pixel decoder | TC260 AIGC |
| LiblibAI | LiblibAI mark | No registered pixel decoder | TC260 AIGC |
| RunningHub | RunningHub mark | No registered pixel decoder | TC260 AIGC |
| Samsung Galaxy AI | One locale specific mark | No registered pixel decoder | C2PA and Samsung markers |
+17 -5
View File
@@ -537,8 +537,8 @@ study (section 2.2) gives empirical floors:
> were removed, and `OPENAI_STRENGTH` / `GEMINI_STRENGTH` / `UNKNOWN_STRENGTH` went
> with them. Everything from here to the end of this section is a record of what was
> measured on those profiles, kept because the oracle verdicts are still the evidence
> base. For the strength policy in force now see `module-internals.md`: `qwen-zimage`
> uses `resolution_adaptive_denoise`, `sdxl-zimage` a flat vendor ladder.
> base. For the strength policy in force now see
> [`known-limitations.md`](known-limitations.md#strength-is-content-and-seed-dependent).
The default was **vendor-adaptive** (`watermark_profiles.resolve_strength` +
`vendor_for_strength`): the tool read the C2PA issuer on the original input and picked
@@ -691,9 +691,9 @@ solid at 0.10 but at 0.15 it is near-threshold noise; (2) for reliable removal p
a strength with MARGIN above the borderline (controlnet >= 0.20), not exactly on
it; (3) **historical engineering conclusion:** this dated run argued for a
higher ControlNet strength than the then-current default. That proposal was
later superseded. The current resolver intentionally shares the 0.10/0.15
ladder between SDXL and ControlNet and uses a separate Qwen ladder; see
`_internal/watermark_profiles.py`.
later superseded. The resolver of that period shared the 0.10/0.15 ladder between
SDXL and ControlNet; the current policies are recorded in
[`known-limitations.md`](known-limitations.md#strength-is-content-and-seed-dependent).
Source images are private (faces / product shots), not committed; reproduce on any
photoreal + flat-graphic gpt-image pair, varying the seed, and re-checking the
oracle.
@@ -769,3 +769,15 @@ reproducible verification requires a fixed seed.
8. Jiang et al. (2025). **VideoMarkBench: Benchmarking Robustness of Video
Watermarking.** arXiv:2505.21620.
https://arxiv.org/abs/2505.21620
**Google floor on qwen-zimage (0.27.2, remeasured 2026-08-25).** The bracket above did not
survive the full production path: on the 4.33 MP CJK-sign fixture (visible-stage
sparkle removal -> qwen-zimage seed 0 at the curve's 0.154 top -> resize-back ->
metadata strip), the Gemini verifier detected SynthID x3 with a valid
pixel-identical stripped control in the same session. Google-provenance content
initially took a flat `0.30` floor instead of the area curve. The later provider
remeasurement established a common pass at `0.25` across three valid sources and
separately repeated `QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.27` across those sources and
three accounts. OpenAI now has its own measured flat operating point; unknown
content keeps the resolution curve. An explicit strength still wins. Certification artifacts: raiw-app
`data/certification/text-restoration-2026-08-18/` (the failing re-baseline).
+228 -2
View File
@@ -20,8 +20,8 @@ filter applied to every candidate below.
## Problem recap
The `invisible` pipeline is SDXL base 1.0 img2img to defeat SynthID. The default
strength has risen over time as Google hardens SynthID (0.05 -> 0.10 -> **~0.30**, the
current threshold for fresh Gemini output); higher strength deforms text more, which is
strength rose over time as Google hardened SynthID (0.05 -> 0.10 -> **~0.30** in
this experiment; the later measured operating point is 0.27); higher strength deforms text more, which is
exactly why text protection matters. Text is protected via Differential Diffusion with a
per-pixel change map (`preserve` ~0.9) driven by the PP-OCRv3 DB detector
(`text_protector.py`). Large text survives; **small text (sub ~8 px strokes) softens or
@@ -143,3 +143,229 @@ Deep-research workflow run `wf_118b9a03-3eb` (2026-05-29). Findings adversariall
(2/3 refutes required to kill a claim). This note records research only; no code change is
implied until a prototype validates fidelity and the SynthID-scrub guarantee on the
restored output.
## 2026-08-13 OCR plus LaMa prototype
A lightweight version of the recommended two-stage architecture was tested on
the three tracked text fixtures. It used the already-clean Qwen result as the
base, recognized the source text, removed source and Qwen glyph contours with
LaMa, and rendered the recognized strings as fresh pixels. No source pixels
were pasted back.
The result validates only part of the hypothesis. Character-weighted CER on the
two multilingual posters fell from 0.338/0.305 to 0.007/0.007, and OpenAI Verify
remained clean for both matched Qwen and restored pairs. However, replacement
fonts changed the design, whole-image LPIPS worsened by 0.067 on each poster,
and the light poster retained a shadow in one small English line. On the Chinese
sign, punctuation errors moved CER from 0.074 to 0.111. The Google verifier run
was inconclusive because the available account detected both the restored image
and the byte-identical Qwen control that a previous account had called clean.
The experiment supports a proper glyph-conditioned restorer, not shipping the
plain OCR/system-font compositor. Exact hashes, metrics, masks, and provider
verdicts are tracked in
[`data/evaluations/fidelity/text-restoration-2026-08-13.csv`](../data/evaluations/fidelity/text-restoration-2026-08-13.csv).
## 2026-08-13 AnyText2 follow-up
The official AnyText2 SD1.5 checkpoint was then tested as the glyph-conditioned
restorer. Its ModelScope entry and code are Apache 2.0, and the downloaded
checkpoint matched the published SHA-256. An official blackboard edit completed
successfully before the fixture run, establishing that the runtime reproduced
the model rather than silently exercising a fallback.
On the Chinese sign, a three-line local edit over the exact Qwen output scored
CER 0.185 under the standard detector. Font mimic from the source glyph masks
scored 0.222. Whole-image LPIPS changed from 0.289 to 0.338 and 0.345 respectively.
A padded crop-recognition check showed why the comparison matters: Qwen contained
the full correct text and scored 0.000, while default AnyText2 still scored 0.074
with two substituted characters and mimic remained at 0.222. The provider oracle
was deliberately not called because these variants had already failed the
content-fidelity gate.
AnyText2 is therefore not the missing production restorer in its published form.
The negative result is narrower than the model family: only the SD1.5 checkpoint
is public, while AnyText2XL remains listed as future work. The current wrapper
also truncates each quoted text line to 20 characters, which prevents a direct
test of several tracked English and Russian poster lines. Exact model provenance,
output hashes, and metrics are recorded in
[`data/evaluations/fidelity/anytext2-restoration-2026-08-13.csv`](../data/evaluations/fidelity/anytext2-restoration-2026-08-13.csv).
The same cross-check exposes a measurement bug in the earlier lightweight
restoration result. Paddle detection made tight boxes that omitted the final
Chinese full stop from two lines, producing Qwen CER 0.074 although the pixels
and padded recognition were correct. Adding 16-32 pixels of horizontal context
restored every punctuation mark. A deterministic rerender using the full OCR
strings and the closest of nine mask-scored CJK fonts also reached detector CER
0.000, but its heavier Hiragino Sans GB W6 glyphs raised LPIPS from 0.289 to 0.327.
The actionable design is selective restoration: compare padded source and output
recognition first, and preserve the Qwen output when they already match.
That policy was tested on the two multilingual posters. A manual prototype kept
the lines whose padded source and Qwen recognition agreed and rerendered only the
changed lines. The edited mask fell from 15.4%/17.0% in the full compositor to
5.5%/5.5%. On a single Paddle `en+ru+ch` route, CER changed from 0.378/0.413 for
Qwen to 0.101/0.112, while LPIPS was only 0.120/0.103 rather than the full
compositor's 0.174/0.162. The different OCR route is intentional and all three
variants were remeasured through it; these CER values are not directly
comparable with the earlier Vision/Paddle hybrid results.
Both selective outputs returned `No OpenAI signals detected` in OpenAI Verify,
and the original light poster returned `Generated with OpenAI tools` immediately
afterward as the positive control. The dark poster was visually clean. The light
poster still showed one local old-text shadow, so mask construction remains the
blocking defect. Exact hashes, metrics, mask fractions, and oracle controls are
in
[`data/evaluations/fidelity/selective-text-restoration-2026-08-13.csv`](../data/evaluations/fidelity/selective-text-restoration-2026-08-13.csv).
Uniformly expanding every selected glyph mask by two pixels removed that light
poster shadow and moved CER from 0.101 to 0.081 without a meaningful LPIPS cost.
It also expanded the edited area from 5.5% to 7.0%, and the identical rule made
the dark poster worse at CER 0.147 rather than 0.112. Those follow-up bytes have
not been oracle-checked. The next prototype should detect residual source glyphs
per line and expand only the failing component, rather than introducing another
poster-wide tuning constant.
The manual selection was then replaced with per-line padded recognition. A line
is left untouched only when the source recognizer is sufficiently consistent
with the verified line and normalized source/candidate recognition agree
exactly. The automatic rule reproduced the manual selection on the dark poster
and correctly kept one additional large Russian line on the light poster. It
reduced that poster's effective edited area from 5.5% to 4.1%.
Mask cleanup became a measured second pass rather than a global dilation. After
the first LaMa erase, the prototype finds contrast that remains specifically at
the original glyph positions, dilates only that residual, and erases it before
fresh text is drawn. This removed the visible double glyphs from both posters.
The automatic outputs scored LPIPS 0.113/0.104 and detector CER 0.123/0.119, with
effective edited fractions of 4.1%/5.7%. The light aggregate CER is pessimistic:
the page detector fragments its untouched Cyrillic line into Latin-like pieces,
while the padded Russian recognizer returns the exact expected text.
OpenAI Verify returned `No OpenAI signals detected` for both new hashes and then
`Generated with OpenAI tools` for the original light poster in the same browser
sequence. Reproducing the prototype from the tracked script exposed harmless
ONNX rounding on the light image: 386 pixels differed by at most one channel
value from the initially checked hash. The exact tracked output hash
`900def5a...` was therefore submitted separately, returned clean, and was
followed immediately by another positive source result. The dark tracked output
was byte-identical to the initially checked hash.
The Chinese sign provides a no-edit control for the selector. Detector boxes for
its three large lines overlap vertically, so Latin-style vertical padding
incorrectly mixed neighboring lines. Horizontal-only CJK padding captured the
terminal punctuation without mixing neighbors and made source and candidate
recognition agree exactly on all three lines.
The script then produced a zero mask and copied the Qwen candidate byte for byte.
This prevents the heavier-font regression seen in the earlier CJK compositor,
but does not resolve the unavailable Google-negative oracle control.
The result clears the measured OpenAI gate without a manual line selection, but
is not a production design yet. It still assumes verified source strings and
line boxes, relies on macOS system fonts, and needs evaluation on uncurated
layouts, rotated text, false OCR matches, automatic line-box discovery, and a
reproducible Google-negative control.
The later source-silhouette experiment removed the font lookup but did not meet
the actual visual requirement. Native-size review found changed stroke weight,
color variation, edge antialiasing, and decorative details even where crop OCR
was exact. A narrower `vae-glyphs` experiment uses the silhouette only as a
support mask: exact pixels come from a Qwen-VAE reconstruction, while a fresh
flat-color silhouette supplies only the outer edge beneath that core. Across 48
typography cases (548 annotated lines), this improved glyph-edge MAE in 48/48
and text-box SSIM in 47/48. The median values moved from 37.59 to 32.42 and from
0.854 to 0.914 respectively. One separate untracked core-only output with a
1.43% donor area returned `No OpenAI signals detected`; the tracked script then
reproduced those exact bytes. A 0.5-pixel feathered sibling with 2.75% nonzero
donor area improved mean text-box SSIM from 0.957 to 0.960, with a 0.918 minimum
across 15 verified lines, while full-image RGB SSIM reached 0.864. Crop OCR
recovered from 7/15 exact lines on the raw pass to 14/15, equal to the source's
own recognizer floor. Whole-image LPIPS was 0.082, but only 0.108% of pixels
remained exactly equal to the source and the detected face's Laplacian-variance
ratio was 0.670, confirming residual global smoothing. A same-session
OpenAI Verify sequence detected the exact source control in 1/1 check, returned
clean for the raw global pass and clean-fidelity anchor in 1/1 each, and returned
clean for the feathered output in 3/3 repeats. The tracked script reproduced
that feathered file byte for byte. This proves two materially better exact-output
Pareto points, not a general operating range. The 48 matrix outputs and other
mask sizes remain oracle-unverified.
The first Google oracle sample did not transfer. For the synthetic CJK sign,
two separate work accounts detected SynthID in both the resaved positive source
control and the exact Qwen-VAE donor candidate. The candidate's text-box SSIM
improved from 0.791 to 0.835 and its glyph-edge MAE from 35.48 to 22.49, but its
3.67% donor layer did not create a clean operating point. The Qwen silhouette
base was already detected, as was the earlier original-sign Qwen baseline in the
same account. The measured cause is therefore insufficient global Google
regeneration, not a demonstrated watermark regression from the text donor.
Google needs a stronger oracle-clean global anchor before the narrow donor can
be evaluated there.
A targeted follow-up supplied that missing anchor for one CJK case. An explicit
Qwen global pass at strength 0.30 returned no reliable SynthID signal in Gemini,
while the matched source control was detected. Applying the 0.5-pixel
`vae-glyphs` layer to that clean anchor changed 1.17% of the image and raised
mean text-box SSIM from 0.765 to 0.964 (minimum 0.963), with mean text-box MAE
falling from 20.74 to 5.38. The post-donor output then returned clean in 3/3
Gemini checks across two work accounts. A separate 18-face fixture with four
tiny UI-text lines also had a clean strength-0.30 anchor; its donor layer changed
0.70% of the image, raised mean text-box SSIM from 0.617 to 0.976 (minimum
0.972), and returned clean in 3/3 checks across the same two accounts. One
quota-exceeded response is excluded from both pass rates. These are two exact
oracle-certified outputs, not a general operating range: other layouts, masks,
strengths, seeds, and donor areas remain unverified.
An automatic-box follow-up merged Paddle word detections by vertical overlap.
It found exactly 20/20 poster lines and 3/3 sign lines, with mean IoU 0.857,
0.847, and 1.000 against the verified boxes. That structural match was not
sufficient: reusing the annotation crop padding changed recognition decisions,
reduced preserved dark-poster lines from 7 to 4, and expanded its edited
fraction from 5.7% to 11.2%.
A recognition-only sweep isolated the crop sensitivity. Limiting vertical
padding around detector boxes to 8-12% of line height reproduced the verified-
box decision vectors on both posters; 10% was used for a full follow-up. It kept
8/20 and 7/20 lines with edited fractions of 4.1% and 5.7%, and visual inspection
found no double glyphs. Whole-image LPIPS remained 0.113/0.104, but detector CER
was 0.127/0.154 instead of 0.123/0.119. The dark regression failed the fidelity
gate, so these hashes were not submitted to the provider oracle. The opt-in flag
remains only to reproduce the negative result. Count, IoU, and matching decision
vectors are therefore insufficient gates for automatic boxes; the next design
needs box rectification or recognition stability under crop jitter before it
can replace verified geometry.
Removing verified strings was tested separately with an annotation-seed dry
run. It detects boxes, chooses `en`, `ru`, or `ch` from Unicode script, and
accepts a draft only when three crop paddings normalize identically and every
confidence is at least 0.85. One execution proposed 20 and 18 poster lines, but
exact-text precision against the held-out annotations was only 90.0% and 94.4%.
The stable errors were punctuation: one lost English comma and an ideographic
comma consistently replaced by ASCII. Thus crop agreement and confidence do
not verify literal text. The helper is retained only to seed annotations; every
proposal needs manual review before it can enter the restoration compositor.
## Production-path anchor leak and fix (2026-08-19)
The engine `text_manifest=` path (unlike the research compositor above) blends a
15% Qwen-VAE fidelity anchor across the WHOLE frame before glyph restoration.
Certified through the official OpenAI Content Provenance API on the exact
production path (qwen-zimage, seed 0, no strength override, untiled, LANCZOS
resize-back, metadata strip; library 0.27.0 @ 50d9f448):
- controls (pixel-identical metadata-stripped sources): `synthid: detected` -
the pixel channel was proven live before every session;
- base outputs: `clean` x6 (two 20-line posters, 3 attempts each);
- text-restored WITH the anchor: `synthid: detected` x6, deterministic;
- text-restored with `FIDELITY_BLEND_ALPHA = 0` (runtime ablation, base bytes
identical between runs): `clean` x6.
The anchor was therefore the leak channel: a 15% global donor admixture over a
scrubbed frame returns enough OpenAI SynthID for the provider detector. Fidelity
cost of dropping it (source-referenced text-box MAE): 11.60->11.72, 7.79->7.86,
and 7.57->8.13 across the two posters and the CJK sign - well under one point
against base MAEs of 24.7/16.8/10.2. Since 0.27.1 the anchor is off by default
(`fidelity_anchor` parameter on `remove_watermark` / `InvisibleOptions` /
`--fidelity-anchor`); Google rows were not re-queried because the default
strength leaves that fixture's base detected (the vendor-strength lane tracks
that separately). Certification artifacts: raiw-app
`data/certification/text-restoration-2026-08-19-A1/` (sha256-stamped rows).
+22 -9
View File
@@ -24,7 +24,7 @@ Performance depends on format, enabled extras, and hardware. Measure it locally
## Tier A -- self-evident oracles (full corpus, unattended)
Properties that are true or false without anyone labelling anything. These are the
Properties that are true or false without anyone labeling anything. These are the
backbone: they scale to large local datasets and catch regressions with zero human cost.
### A1. Recorded-verdict regression
@@ -39,6 +39,15 @@ confidence), reviewed once, then re-baselined. Lost detections are the alarm.
Implemented as `scripts/sidecar_regression.py` (resumable, ~1.5 h at 8 workers).
A 2026-08-15 metadata-only C2PA regression audit over the local historical
corpus caught two structured-parser gaps before release: an exact Firefly claim
generator without a repeated source type, and a Dreamina generator carried by a
reachable ingredient under a generic update manifest. Both now have focused
tests. The same audit confirmed that invalid hash/signature claims lose origin
attribution without losing the C2PA inventory, and exposed the absence of
production trust anchors in the SDK defaults. Dataset-derived counts and
identifiers remain in the gitignored audit output.
#### Local run protocol
Re-run `identify` against locally recorded sidecars, classify losses separately from intended new detections, and keep generated reports under `.local-eval/`. Do not commit dataset-derived counts or identifiers.
@@ -119,7 +128,7 @@ which read like twelve broken features. It was one bad harness parameter (`--ste
sitting on top of one real bug. An error that is IDENTICAL across unrelated rows is
evidence of a common cause, not of many faults -- check the shared input first.
## Tier B -- constructed ground truth (automatable, no labelling)
## Tier B -- constructed ground truth (automatable, no labeling)
Where reality gives no answer key, build one. This is the tier that closes the two biggest
holes: fill quality and detector response at the edge of the operating range.
@@ -160,7 +169,7 @@ Peak RSS and wall time per backend x input size, up to 25 MP. The memory-constra
tier is a real constraint (MI-GAN must stay ~0.6-0.9 GB by cropping around the mask); a
regression here is invisible today and would only surface under load.
## Tier C -- human-labelled accuracy (bounded by labelling effort)
## Tier C -- human-labeled accuracy (bounded by labeling effort)
The machinery exists: `visible_recall_sample.py` -> `visible_sheets.py` ->
`visible_groundtruth.py` -> `visible_eval.py`.
@@ -177,10 +186,14 @@ report recall from the detector-sampled set.
## Tier D -- external oracles (manual, not automatable here)
SynthID removal cannot be verified locally by design -- no public decoder exists. Each
vendor has its own oracle and it covers only that vendor's content: `openai.com/verify` for
OpenAI (more accessible, the automation candidate), the Gemini app for Google (manual,
rate-limited). A quiet metadata proxy is **not** proof the pixel watermark is gone.
Proprietary watermark removal cannot be verified locally by design -- no public decoder
exists. Each vendor has its own oracle and it covers only that vendor's content:
`openai.com/verify` for OpenAI, the Gemini app for Google, and Microsoft's
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
for InvisMark. The Microsoft API reports `Watermark` and `C2PA` separately. It therefore
needs a pixel-identical metadata-stripped control: the control must lose `C2PA` while
remaining `Watermark`-positive before a candidate's negative result can be attributed to
pixel regeneration. A quiet metadata proxy is **not** proof the pixel watermark is gone.
Scope honestly: this tier certifies strength floors on a handful of images per vendor, and
that is all it can do. See `docs/synthid.md`.
@@ -278,7 +291,7 @@ on one file. The bar is never "handles it" but **never raises and never silently
4. **B2 detector curves** -- cheap, and directly guards the geometry class of bug.
5. **A5 contract sweep over a representative local set**.
6. **B4 resource ceilings**, **E robustness**.
7. **C recall expansion** -- gated by labelling appetite.
7. **C recall expansion** -- gated by labeling appetite.
8. **D oracles** -- manual, per release.
Every tier writes a versioned snapshot so runs are comparable over time; a run that cannot
@@ -313,7 +326,7 @@ Per mark, what actually goes away when metadata is stripped:
### Measure before improving
Use Tier B2 detector-response curves before tuning a detector with sparse labelled
Use Tier B2 detector-response curves before tuning a detector with sparse labeled
examples. Sweep size, contrast, aspect, and background texture so geometry regressions
are visible without exposing private evaluation statistics.
+114
View File
@@ -171,6 +171,120 @@ Three readings, all MEASURED, none of them about SynthID:
so the `max(temporal_baseline, 1e-6)` denominator collapses and the ratio
inflates. This is the predicted defect, now observed rather than argued.
## Measurements on the real carrier (2026-08-06)
The public Veo 3 off-road sample was downloaded and its sha256 matched the manifest
exactly, so everything below is measured on the same file the 2026-07-31
calibration used. Its properties, none of which the manifest recorded at the time:
H.264 1920x1080 at 24 fps, 8.000 s, 5,155,063 bps video, plus an **AAC 48 kHz
stereo 192,241 bps audio track of 7.914 s**.
The shipped profile therefore delivers 512x288 at 12 fps: **7.1% of the frame area
and 3.6% of the pixels per second.**
### The geometry prize, measured end to end
Scored with `scripts/video_fidelity_probe.py` against the untouched 1920x1080
source, all at `noise_std=0.15`:
| Geometry | Frame area | PSNR | SSIM | Temporal | Bitrate |
| --- | --- | --- | --- | --- | --- |
| 512x288 @12 (shipped) | 7.1% | 25.3971 | 0.8215 | 1.0407 | 2187 kbps |
| 768x432 @12 | 16.0% | 27.3796 | 0.8556 | 1.0323 | 3782 kbps |
| 1024x576 @12 | 28.4% | 28.8542 | 0.8819 | 1.0262 | 5604 kbps |
| 512x288 @24 | 7.1% | 25.6142 | 0.8270 | 1.0995 | 3181 kbps |
**512 to 1024 is worth +3.46 dB, against 1.92 dB for the entire `noise_std` axis.**
The geometry lever is roughly twice the noise lever, which is the first hard number
this investigation has for the size of the prize. What it still costs in removal is
unknown: none of these was ever submitted.
Two readings that are not obvious from the table:
- **Temporal residual improves as resolution rises** (1.0407 to 1.0262). The shared
noise field lives on the latent grid, so at 1920 it is four times finer relative
to the frame than at 512, where it stretches into low-frequency blotches. Higher
resolution plausibly makes the perturbation *less* visible, not more.
- **The frame-rate arm looks bad here and the metric is the wrong judge.** 24 fps
buys +0.22 dB and worsens the temporal ratio to 1.0995, but what 24 fps actually
buys is smoothness, which neither PSNR nor SSIM measures at all. The metric sees
only the cost. Do not decide the fps question from this table.
Encoder cost on an M-series MPS device, 8 seconds of source: 51 s at 512/12, 180 s
at 768/12, 380 s at 1024/12, 190 s at 512/24. Native 1920x1080 at 24 fps
extrapolates to roughly 46 minutes for the same 8 seconds, around 350x real time. A
native run was started and killed before producing anything, so that figure remains
an extrapolation.
### The certified row does not reproduce
Re-running the certified configuration (512/12/0.15, seed 0) on this machine gave
25.5002 dB and 1.0675 against the manifest's 25.3911 and 1.0578, and a different
output sha256. The manifest records neither device nor dtype, and the original was
almost certainly CUDA fp16 while this was MPS fp32. **The oracle certified a
specific file that cannot be rebuilt**, so any re-run produces an artifact the
verifier never saw. Device and dtype belong in the schema.
### Audio passes through untouched, and the output still reads clean
Established on two carriers, by comparing the extracted audio bitstream sha256 of
input and output:
- Veo carrier: `674063df...` in, `674063df...` out.
- A Lyria 3 track generated for this purpose: `6ee4c041...` in, `6ee4c041...` out.
The Lyria case is the sharper one because its provenance is known. Google's own
generator stamps it: `get_ai_metadata` reads a C2PA manifest naming
`Google C2PA Core Generator Library`, issuer `Google LLC`, `trainedAlgorithmicMedia`.
After the shipped path runs, that output reports `{}` -- every metadata marker
stripped, `remaining_metadata` empty, visuals regenerated -- while the audio the
generator produced is bit-for-bit intact.
So the pipeline can produce a file that **our own detector calls clean while an
untouched Google-generated audio track sits inside it**. The mechanism is proven;
whether that audio carries an audible-to-the-verifier SynthID mark is not, and only
the oracle can say. If it does, this is a shipped defect rather than a quality
question, and it outranks everything else in this document.
Worth recording alongside it: the Veo carrier itself returns `{}` from
`get_ai_metadata`. The public sample carries no C2PA at all, which is exactly why it
needed an oracle verdict in the first place.
## State when this work was parked (2026-08-06)
Landed in `main` through PR #78: the manifest-backed default pin, the
scaling-factor gate and its single seam, the sweep loading through the engine's
loader, `scripts/video_fidelity_probe.py` with its tests, the extended manifest
schema, and this document. **The shipped operating point did not move**: it is
still 512 px / 12 fps / `noise_std=0.15`.
A product requirement arrived late and reframes the target: output resolution,
frame rate and bitrate must not be reduced. Resolution, frame rate and crf are
parameters and can be preserved, but zero quality loss is not achievable by this
mechanism at all -- removal works by rebuilding pixels, so a regeneration can never
return its input. The honest goal is to preserve container properties and make the
perceptual loss imperceptible, judged by DISTS and VMAF rather than PSNR. Two
consequences follow: crf 18 on this path against crf 14 on the visible path is an
unexplained asymmetry and a free quality win, and the oracle's 100 MB limit means a
higher-quality profile is *harder* to certify than the one it replaces.
Nothing is blocked on analysis. Six oracle submissions were prepared and never run,
because file upload to the verifier failed at the tool level:
1. A Lyria carrier and the pipeline's output from it, answering the audio question.
2. A multiplexed control-plus-candidate file at the shipped geometry, validating
whether the verifier reads a concatenated pair by time range. Verified locally
frame by frame: half one is bit-identical to the control, half two to the
candidate.
3. Three more multiplexed pairs for 768, 1024 and 24 fps, worth three queries
instead of six if the instrument validates.
Every generated artifact for those submissions has since been lost with its
scratch directory, which is the intended lifecycle for generated media. Rebuilding
costs one download and roughly 25 minutes of encoding; the Lyria track is the
exception, since regenerating it would produce a different track, and the copies
that survive are the only ones.
## The single most important unknown
**How much the 512 px downscale contributes to removal.** Every quality gain routes
+21 -18
View File
@@ -10,7 +10,7 @@ Who embeds what, and whether it is locally detectable (so we know which gaps are
**Root cause and external confirmation (deep-research 2026-06-19, adversarially verified).** This is the SCHEME's ceiling, not our usage — there is no better decoder to adopt. The imwatermark maintainers state verbatim (both the ShieldMnt and Stability-AI READMEs) that the algorithm "cannot guarantee to decode the original watermarks 100% accurately even though we don't apply any attack." Independent measurement (WMAdapter, arXiv:2406.08337 Table 2) puts dwtDct at only **~0.79 bit accuracy on CLEAN images (~38/48 bits — already below our 44 gate)**, collapsing to ~0.50 (chance) under crop/JPEG. Two code-verified + locally-reproduced mechanisms drive the content-dependent failures: (1) the decoder reads each bit as the **highest-magnitude DCT coefficient per block**, so any content coefficient exceeding the encoded target flips the bit; (2) the default embed is in the **YUV chroma channel, which 8-bit-clamps on white/bright pixels** (a +36 chroma delta survives a white-fill round-trip as only +4, ~89% loss) — this is the mechanism behind the bright-flat / minimalist failures and the all-ones degenerate decode. No maintained fork or detector decodes this scheme reliably: the WAVES benchmark (arXiv:2401.08573) relegates DWT-DCT to supplementary appendix G.5 and targets Stable Signature / Tree-Ring / StegaStamp instead; learned encoder/decoder schemes reach ~0.98-0.99 clean but are a DIFFERENT watermark class (not what SDXL/FLUX stamp). `dwtDctSvd` does not help (SDXL embeds `dwtDct`; dwtDctSvd cannot decode it, and its clean accuracy ~0.72 is lower). **Authoritative conclusion: the open DWT-DCT mark cannot be turned from positive-only into a reliable real-world detector; keep it positive-only and rely on C2PA.** (Refuted along the way: that the library is unmaintained, and that it is robust to JPEG but only fails on geometric attacks — both did not survive verification.)
Consequence for the FLUX hosted-output question (BFL Playground, FLUX.2 [pro] + FLUX.1 [dev], 2026-06-19): all samples carry the signed C2PA manifest (issuer "Black Forest Labs"); the open DWT-DCT decode returned `None`, but every available FLUX carrier (textured fox AND a minimalist-flat generation) failed the positive control (28/48), so the detector is blind on them and **whether BFL hosted output embeds the open pixel watermark is UNRESOLVED** (an earlier note here wrongly asserted it absent — overstated; a later note blamed "high texture" — also wrong, flat carriers fail too). What IS established: C2PA is the reliable FLUX identifier; the `_BITS_48` pattern is correct (round-trips on chatgpt/firefly/random). Resolving the hosted question needs a hosted FLUX carrier that first passes a >=44/48 positive control, which neither a textured nor a flat prompt produced — low priority (the open mark is only a stripped-metadata fallback).
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Copilot + Designer; Bing Image Creator collected 2026-05-24 still signs as "Microsoft" and now runs **MAI-Image**, NOT OpenAI/DALL·E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; verified samples disproved the earlier assumption that Canva downloads always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
**Samsung Galaxy AI** signs supported edits with C2PA and may carry the
proprietary `genAIType` marker. The registered visible detector covers the
@@ -28,7 +28,7 @@ take precedence over issuer attribution. Supported mappings include Higgsfield
AI, Topaz Labs Image API, and TikTok Ad Creative Toolbox; an unknown claim
generator still falls back to the certificate issuer.
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` now maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance (Dreamina)" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance Volcano Engine"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance Dreamina" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
- **EXIF/XMP/PNG-text generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal). Additional verified generator stamps include **NovelAI** (`Software`, `Source`, and `Title` PNG text chunks), **Reve** (`Software` or XMP `CreatorTool` = `reve.com`), and **Aphrodite AI** (`Make` or `Software` = `Aphrodite AI`).
- **App-export provenance and AIGC JSON:** supported ByteDance-family exports can place a JSON object in EXIF `ImageDescription` or `UserComment`, independently of C2PA or TC260. Exact `product` values for Doubao, Xinghui, and Dreamina are removable product provenance, but do not alone prove that the pixels were generated. Dreamina additionally requires `exportType=generation` for that verdict. A nested Aweme `aigc_type=1` or private ByteDance `aigc_label_type=1` / `2` is an AIGC disclosure; `0` is inconclusive and can occur on a Dreamina generation export. Plain Aweme, retouch, and `lv` exports are preserved. The lower-case private field is deliberately not interpreted as the normative TC260 `Label`, whose values `1` / `2` / `3` mean generated / possibly generated / suspected generated under [GB 45438-2025](https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf).
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
@@ -45,7 +45,7 @@ pair. On the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). The label says only "this is AI", but its `ContentProducer` names the signing entity — `001` + `1` + an 18-char Unified Social Credit Code + a 5-digit product suffix, normalized by `metadata.uscc_of`, or for a few generators a bare product name. `KnownMark.tc260_producer_codes` maps the codes settled per vendor by `scripts/vendor_cohort_harvest.py` to registry mark keys, so an AIGC image relaxes the detector of the vendor it actually carries rather than a guessed pair; an unmapped or absent producer falls back to ByteDance's two products. A code identifies a legal entity, not necessarily one brand, so a hosting or aggregating platform that signs for several apps is a recall bet rather than a proof.
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item. The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item; Doubao's iOS export additionally writes two QuickTime-form variants (a bare `meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list whose `ilst` data items carry the JSON with no key name) — both verified on retained carriers (2026-08-17). The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
Native MKV/WebM is a sixth serialization. TC260-PG-20257A stores
`TagName=AIGC` and the raw JSON `TagString` in
@@ -58,18 +58,19 @@ defines two more native serializations. AVI stores an `AIGC` child in
bounded RIFF and FLV readers validate the JSON field set and skip media
payloads. Removal remuxes either container through ffmpeg with stream copy.
- **HuggingFace-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** HuggingFace Jobs / Spaces can stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID. It marks the *hosting job*, not a model, so it lifts an Unknown verdict to a tentative AI via `hf_only` but never overrides a hard metadata signal. `_HF_JOB_CAVEAT` states the limit. Removal drops the chunk through the PNG metadata whitelist.
- **Hugging Face-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** Hugging Face Jobs / Spaces can stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID. It marks the *hosting job*, not a model, so it lifts an Unknown verdict to a tentative AI via `hf_only` but never overrides a hard metadata signal. `_HF_JOB_CAVEAT` states the limit. Removal drops the chunk through the PNG metadata whitelist.
- **No detectable signal on some downloads:** Recraft exports and some hosted
FLUX surfaces can arrive without a supported local signal. Midjourney samples
may carry IPTC metadata but no registered C2PA or pixel watermark. The open
DWT-DCT decoder only applies when the producing pipeline actually ran its
encoder and the carrier remains decodable.
- **Invisible but NOT locally detectable (proprietary, API/oracle only — same wall as SynthID):** Amazon Titan Image Generator + Nova Canvas (Bedrock `DetectGeneratedContent` API), Kakao (new SynthID image adopter, May 2026), NVIDIA Cosmos (SynthID video). No local detector possible; treat like SynthID.
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest (Digimarc/Imatag/Steg.AI/...) stay name-only (proprietary decoders).
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, preserve the structured assertion's signed `value`, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest stay name-only unless a compatible decoder is independently verified.
- **Microsoft Paint and Photos InvisMark (reverse-engineered 2026-08-20):** Paint receives a per-generation GUID from remote prompt moderation, embeds it into locally generated pixels, and records the same value in `c2pa.soft-binding` under `com.microsoft.invismark.1`. The C2PA soft-binding registry independently identifies that algorithm as Microsoft Responsible AI InvisMark for image and video. Paint's 144-bit writer framing does not match the public repository's 100-bit pretrained checkpoint interface, so compatibility is not assumed. The parser reports the signed identifier; there is no validated local pixel decoder. Microsoft's external Content Provenance Detection API is the removal oracle because it reports `Watermark` separately from `C2PA`; a metadata-stripped, pixel-identical control must remain watermark-positive before an output-negative result is attributed to pixel removal. The no-account web page for the same check is <https://ai.azure.com/nextgen/validate> (its `Inconclusive` verdict is weaker than the API's separate watermark result). Sources: https://xusheng.dev/posts/reversing/mspaint_invisible_watermark/main/, https://github.com/c2pa-org/softbinding-algorithm-list/blob/main/softbinding-algorithm-list.json, and https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection
- **Built in the dated batch:** soft-binding vendor detection, IPTC Photo
Metadata AI-disclosure fields, C2PA detection and stripping for supported
ISOBMFF video, the optional Adobe TrustMark decoder, and temporally stabilized
visible Sora, Veo, Seedance, Dola, Hailuo, and Kling removal. Other visible
visible Sora, Veo, Seedance, Dola, Hailuo AI, and Kling AI removal. Other visible
video logos and proprietary audio-watermark detection remain outside the
package.
Metadata stripping for supported audio containers is a separate implemented
@@ -115,12 +116,12 @@ every frame, then requires an anchored temporal run. This extra anchor check
was necessary because a moving clean scene detail could retain enough adjacent
overlap to pass a recurrence-only gate.
**Hailuo and Kling use larger fixed composite labels.** Verified Hailuo exports
carry a lower-edge waveform, `MINIMAX`, separator, Hailuo ring, and
`hailuo AI` text. Verified Kling exports carry a bottom-right swirl,
**Hailuo AI and Kling AI use larger fixed composite labels.** Verified Hailuo AI exports
carry a lower-edge waveform, `MINIMAX`, separator, Hailuo AI ring, and
`hailuo AI` text. Verified Kling AI exports carry a bottom-right swirl,
`KLING AI`, a changing version suffix, and sometimes `PRO`. The detectors use
only synthetic primitives and fonts. Hailuo expands the matched core to cover
the complete composite. Kling combines a version-independent text core with a
only synthetic primitives and fonts. Hailuo AI expands the matched core to cover
the complete composite. Kling AI combines a version-independent text core with a
synthetic ring rescue, then requires the recurring candidate to reach the
expected frame edge and contain enough bright low-saturation pixels. Those
extra gates were added after clean Luma and PixVerse scene details passed shape
@@ -130,17 +131,17 @@ instead uses Florence-2 to identify arbitrary watermarks before LaMa
inpainting. That is broader, but it carries a much heavier model and a less
auditable detection boundary than the provider-specific synthetic path here.
**The faint-visible-mark precision/recall wall is fundamental, not a heuristic artifact.** The visible-watermark-detection literature has moved to LEARNED segmentation / object-detection (WDNet WACV'21 arXiv:2012.07616; SLBR ACM MM'21, open code+weights; the PRCV'18 large-scale detector; Su et al. survey 2025), but three verified findings bound what a learned detector actually buys: (1) a claim that a confidence threshold "cleanly separates" true from false matches even with a learned CNN front-end was **REFUTED** in verification (arXiv:1705.08593) -- the precision/recall wall persists even with learned features. (2) Learned detectors need a LARGE, pattern-diverse labeled dataset trained on synthetic composites (PRCV'18: 60k images / 80 watermark classes; CLWD: 60k / 160 marks), and off-distribution degradation is a documented real axis (models trained on limited-pattern LVW transfer worse; diversity of training patterns drives generalization). (3) Inference is cheap (WDNet ~8 ms at 256x256) -- the cost is the data pipeline, not runtime. Net: a learned detector shifts the frontier but does NOT remove the wall; for a SINGLE mark the cheapest next step is a small patch classifier (real-sparkle vs false-positive) on top of the existing NCC localizer, not a full segmentation model. SLBR is a ready baseline. The current NCC + false-positive gate (core-ring brightness margin + gradient-NCC crispness + white-core saturation) is a sound operating point, and the residual miss is the information-theoretic wall the literature confirms.
**Learned detectors change the visible-mark precision/recall tradeoff; they do not establish a universal separator.** The visible-watermark-detection literature has moved to learned segmentation and object detection (WDNet WACV'21 arXiv:2012.07616; SLBR ACM MM'21, open code and weights; the PRCV'18 large-scale detector; Su et al. survey 2025). The cited arXiv:1705.08593 paper reports a method that significantly reduces false matches and eliminates them after rejecting a small fraction of matches on its electron-microscopy task; it does not prove an information-theoretic limit for watermark detection. Learned watermark detectors also rely on large, pattern-diverse labeled datasets built with synthetic composites (PRCV'18: 60k images / 80 watermark classes; CLWD: 60k / 160 marks), and transfer depends on the diversity represented in training. Inference can be cheap (WDNet reports roughly 8 ms at 256x256), while the labeled-data and evaluation pipeline remain the larger project cost. A learned detector is therefore a possible future operating point, not evidence that the current NCC gate is theoretically optimal.
**Visible-mark landscape beyond the registry.** Meta stamps a visible "Imagined with AI" mark (bottom-LEFT, a small symbol) on its OWN Meta AI / "Imagine" output; for third-party images it relies on C2PA / IPTC, not a visible mark. Samsung Galaxy AI additionally uses a **four-star icon** variant in a corner alongside the localized text wordmark `samsung_engine` calibrates (only the Italian text variant is covered) -- the icon is a distinct, uncovered variant. Every source agrees visible + metadata marks are trivially removable (crop / screenshot, ~2 s), which is the tool's premise.
**Visible-mark landscape beyond the registry.** Since Muse Image (2026-07) Meta's own generation output carries no visible mark, only the invisible proprietary Content Seal; the legacy visible "Imagined with AI" mark belongs to the pre-2026 Meta AI / "Imagine" pipeline and remains unregistered (string verified, position not). For third-party images Meta relies on C2PA / IPTC, not a visible mark. The Samsung detector covers only the measured Italian text wordmark; no separate icon-only variant has been verified. Visible overlays and embedded metadata can usually be removed without changing the underlying generation model, which is the tool's premise.
**Regulatory driver -- China GB 45438-2025 is the strongest VISIBLE-mark mandate.** The CAC / TC260 "Measures for Labeling AI-Generated Synthesized Content" (issued March 2025, **effective 2025-09-01**, technical standard **GB 45438-2025**, building on the TC260 Aug-2023 practice guide) MANDATE a **visible** label for AI images -- a visible textual mark whose height must be **>= 5% of the image's shortest side** -- plus the metadata (implicit) label. Several such CJK text marks are now registered; see [supported signals](supported-signals.md) for the current list. By contrast EU AI Act Article 50 mandates only the MACHINE-READABLE mark (enforceable 2026-08-02, grace to 2026-12-02); a visible label is proposed and modality-specific (visible for images) but is NOT a hard "fixed icon" mandate -- a claim that Art 50 requires a clearly-visible fixed icon for images was refuted in verification. Primary-source dates verified against the article/standard text, not search summaries.
## Uncovered visible marks: implementation specs (deep-research 2026-07-18)
Compatibility testing showed that TC260-labelled images can still produce no visible-mark
Compatibility testing showed that TC260-labeled images can still produce no visible-mark
detection. The main causes were a fixed Doubao localization defect and genuinely
uncovered vendors. Verification status is labelled per claim; treat (b)/(c) as leads,
uncovered vendors. Verification status is labeled per claim; treat (b)/(c) as leads,
not ground truth.
**GB 45438-2025 clause 5.2, the binding constraint for every Chinese mark (VERIFIED (a) -- full standard text extracted from the TC260-hosted PDF).** Verbatim requirements for an image's explicit label:
@@ -152,7 +153,7 @@ not ground truth.
Two consequences we can exploit: (1) the 5% floor is a **scale prior** -- a compliant CN mark's glyphs are large (>= 51 px on a 1024² image), so a CN silhouette ladder can be anchored at ~5-10% of the short side instead of swept broadly, which should cut false fires; (2) every compliant string shares the tail `AI生成` / `AI合成`, so a shared suffix silhouette plus a per-vendor prefix may beat five independent templates. NOT established: whether 文字高度 means cap height, em box, or rendered bounding box (a ~1.3x spread in template scale). Sources: `https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf`, parent CAC measure `https://www.cac.gov.cn/2025-03/14/c_1743654684782215.htm` (the CAC text itself specifies no size or corner).
**Alibaba Qwen -- two surfaces that differ (API tier VERIFIED (a)).** Model Studio docs state verbatim that the API adds a `Qwen-Image` watermark 在图像右下角 and 默认值为 false -- so API output is **unwatermarked by default**, and when enabled the mark is a LATIN wordmark, not CJK, and not GB-compliant in wording. The consumer app's `千问AI生成` (bottom-right) is (b) secondary only -- no Alibaba primary page states it. So Qwen needs TWO templates, and its absence is never evidence of a clean image. Source: `https://help.aliyun.com/zh/model-studio/qwen-image-api`.
**Alibaba Cloud Qwen -- two surfaces that differ (API tier VERIFIED (a)).** Model Studio docs state verbatim that the API adds a `Qwen-Image` watermark 在图像右下角 and 默认值为 false -- so API output is **unwatermarked by default**, and when enabled the mark is a LATIN wordmark, not CJK, and not GB-compliant in wording. The consumer app's `千问AI生成` (bottom-right) is (b) secondary only -- no Alibaba primary page states it. So Qwen needs TWO templates, and its absence is never evidence of a clean image. Source: `https://help.aliyun.com/zh/model-studio/qwen-image-api`.
**星绘 is ByteDance (VERIFIED (a): Baidu Baike + App Store listing, now branded 豆包旗下, team folded into Doubao April 2025).** So `星绘AI生成` is very likely the Doubao house style -- same typeface, same corner, possibly the same top-left `AI生成` pill. Starting from the Doubao `TextMarkConfig` and swapping the two lead glyphs is the cheap path. String/position themselves are (c) inferred.
@@ -160,9 +161,11 @@ Two consequences we can exploit: (1) the 5% floor is a **scale prior** -- a comp
**Tencent Yuanbao: RESOLVED 2026-07-25, registered (`yuanbao_engine.py`).** The standard mark is a compact two-line italic `元宝` over `AI生成` block at bottom-right. It switches between light and dark strokes with the scene, so detection uses polarity-independent local contrast rather than a white top-hat. The separate one-line overlay variant remains evidence-limited to one example.
**Meta `Imagined with AI` (string VERIFIED (a) from Meta's own newsroom; POSITION NOT VERIFIED).** Sources conflict on placement. Do not encode a corner without a verified sample. `identify` reads the supported IPTC disclosure; it does not decode Meta's proprietary invisible watermark. Source: `https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/`.
**Meta Muse Image / Content Seal: current state VERIFIED 2026-08-26 against live artifacts.** Muse Image (launched 2026-07-07, first image model from Meta Superintelligence Labs) puts no visible mark on output. Every output carries Content Seal, a proprietary invisible pixel watermark; research lineage is open (`github.com/facebookresearch/content-seal`, Pixel Seal / VideoSeal / Watermark Anything), but the deployed implementation is proprietary and unpublished. API outputs and Meta CDN copies also carry XMP `iptcExt:DigitalSourceType = trainedAlgorithmicMedia` (a standard IPTC code, not Meta-exclusive). The only reader is the anonymous web oracle `meta.ai/identification`, which returns model attribution plus a per-generation ID and timestamp embedded in the payload. Verified robustness (our corpus, `data/contentseal/`): seal survives CDN WebP transcode, 512 px resize, JPEG q85, and full metadata stripping; it is lost to center crops of 33-50% linear size (Reuters measured 55% detector misses on cropped images, 2026-07-11). Removal is fully supported and oracle-verified on the default profile, with a measured Meta strength floor (0.1) shipped behind auto IPTC routing and an explicit `--vendor meta` override; see [supported signals](supported-signals.md) and [module internals](module-internals.md). Legacy paragraph retained below.
**Samsung English/other locales: still not established.** Samsung's own support page says only that "A Galaxy AI watermark will appear on AI-generated images" -- no string, no corner. Every community thread carrying the exact English string returned HTTP 403 to WebFetch, so the search paraphrase (bottom-left) is deliberately NOT recorded as fact. Feature-tier detail (b): the mark is applied by Generative Edit / sketch-to-image but reportedly NOT by Object Eraser, so Samsung absence is feature-dependent. The four-star icon variant: nothing found.
**Meta `Imagined with AI` (string VERIFIED (a) from Meta's own newsroom; POSITION NOT VERIFIED).** Sources conflict on placement. Do not encode a corner without a verified sample. A dedicated 2026-08-27 sample hunt failed to obtain one: the Feb-2024 newsroom images are UI mockups whose photos carry no in-pixel mark; community posts (Threads `C8_rS_MuEId` titled "lower left corner", a Facebook share) corroborate bottom-LEFT verbally but their files defeat pixel verification; press screenshots predate the mark (Dec 2023); `imagine.meta.com` is dead (redirects to meta.ai) and its Wayback captures are broken SPA error pages. The mark's generator no longer exists (Muse output has no visible mark), so no fresh sample can be made. Removal stays on the generic `erase --region` path until a legacy capture surfaces. Source: `https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/`.
**Samsung English/other locales: still not established.** Samsung's own support page says only that "A Galaxy AI watermark will appear on AI-generated images" -- no string, no corner. Every community thread carrying the exact English string returned HTTP 403 to WebFetch, so the search paraphrase (bottom-left) is deliberately NOT recorded as fact. Feature-tier detail (b): the mark is applied by Generative Edit / sketch-to-image but reportedly NOT by Object Eraser, so Samsung absence is feature-dependent. No separate icon-only variant was established.
**The one document that would settle ByteDance placement is BLOCKED.** Douyin's 《抖音关于人工智能生成内容标识的水印与元数据规范》 aims to give AI tools a unified watermark style and position, which would cover Doubao / Jimeng / 星绘 at once. Both mirrors return HTTP 403 to WebFetch; a secondary report (b, unconfirmed) says the watermark is `AI生成` + tool name + company name placed **top-left** -- which would explain the Jimeng pill's top-left position but contradicts the GB annex's bottom-right example. Worth one retry through Chrome MCP with a real browser session.
+33 -22
View File
@@ -1,9 +1,13 @@
[project]
# This name is the published PyPI distribution, not a local label: changing it
# strands every `pip install remove-ai-watermarks` and every pinned dependency
# on it, including the ComfyUI node package. The console script below carries
# the same weight, since users have it on PATH.
name = "remove-ai-watermarks"
version = "0.26.3"
version = "0.34.0"
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
readme = "README.md"
requires-python = ">=3.10.1"
requires-python = ">=3.11,<3.15"
license = {text = "Apache-2.0"}
keywords = [
"ai-watermark",
@@ -33,7 +37,6 @@ classifiers = [
"License :: OSI Approved :: Apache Software License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
@@ -61,7 +64,10 @@ dependencies = [
[project.optional-dependencies]
pixels = [
"numpy>=1.24.0",
"numpy>=1.24.0; python_version < '3.13'",
# NumPy 1.x has no CPython 3.13+ wheels. Requiring 2.x prevents installers
# from attempting an unsupported source build on the newer Python lines.
"numpy>=2; python_version >= '3.13'",
"opencv-python-headless>=4.8.0",
]
# Optional HEIC/AVIF pixel decode. Metadata scanning handles these containers
@@ -71,12 +77,10 @@ heif = [
]
visible = ["remove-ai-watermarks[pixels]"]
# Video visible removal uses the shared pixel runtime. PyAV packetizes processed
# VFR frames with explicit PTS before system ffmpeg encodes them. PyAV 18 requires
# Python 3.11; the 16.x wheel line still covers Python 3.10.
# VFR frames with explicit PTS before system ffmpeg encodes them.
video = [
"remove-ai-watermarks[visible]",
"av>=16,<17; python_version < '3.11'",
"av>=18,<19; python_version >= '3.11'",
"av>=18,<19",
]
# Open DWT-DCT watermarks used by Stable Diffusion / SDXL / FLUX. The in-tree
# decoder avoids the upstream invisible-watermark package's mandatory torch and
@@ -116,12 +120,27 @@ qwen-zimage = [
"diffsynth>=2.0.17,<3",
"torchvision>=0.20.0",
]
# Opt-in verified-text reconstruction over qwen-zimage. LaMa removes the changed
# candidate glyphs before exact Qwen-VAE cores are composited back.
text-restoration = [
"remove-ai-watermarks[qwen-zimage,lama]",
]
# Draft-only OCR proposals for verified-text manifests. CPU, no torch:
# PaddleOCR detection plus three script-chosen recognition engines. The models
# download on first use and are cached by huggingface_hub; never bundled.
text-draft = [
"paddleocr>=3.3.3",
"paddlepaddle>=3.0.0",
"huggingface-hub>=0.20.0",
]
# Adobe TrustMark decoder -- the open, keyless watermark behind Adobe Durable
# Content Credentials (soft-binding alg ``com.adobe.trustmark.P``). Optional
# because it pulls torch and downloads model weights on first use. identify()
# guards the import and skips the TrustMark signal when absent.
trustmark = [
"trustmark>=0.8.0",
# TrustMark requires NumPy <2, whose last release has no CPython 3.13+
# wheels. Keep this feature installable only on its real runtime range.
"trustmark>=0.8.0; python_version < '3.13'",
]
# Universal region eraser backend -- big-LaMa via onnxruntime (Carve/LaMa-ONNX,
# Apache-2.0). CPU, no torch. Model (~200 MB) is downloaded on first use and
@@ -129,10 +148,7 @@ trustmark = [
# eraser backend needs none of this.
lama = [
"remove-ai-watermarks[visible]",
# ONNX Runtime 1.24 dropped CPython 3.10 wheels; keep the project's
# supported 3.10 line on the last compatible release series.
"onnxruntime>=1.16.0,<1.24; python_version < '3.11'",
"onnxruntime>=1.16.0; python_version >= '3.11'",
"onnxruntime>=1.24.0",
"huggingface-hub>=0.20.0",
]
# Lightweight inpaint backend -- MI-GAN via onnxruntime (andraniksargsyan/migan,
@@ -142,8 +158,7 @@ lama = [
# LaMa, the quality-first `auto` choice, is too large. Same runtime as `lama`.
migan = [
"remove-ai-watermarks[visible]",
"onnxruntime>=1.16.0,<1.24; python_version < '3.11'",
"onnxruntime>=1.16.0; python_version >= '3.11'",
"onnxruntime>=1.24.0",
"huggingface-hub>=0.20.0",
]
dev = [
@@ -156,14 +171,10 @@ dev = [
"ruff>=0.4.0",
"pyright>=1.1.0",
"invisible-watermark>=0.2.0",
# maintain.sh helpers; they only support newer Pythons, so gate them by
# marker to keep the py3.10 resolution (and CI matrix) solvable.
"uv-outdated>=0.1.0; python_version >= '3.12'",
"uv-secure>=0.12.0; python_version >= '3.12'",
]
# ``qwen-zimage`` already pulls ``diffusion``; naming both would suggest diffusion is
# independently sufficient for a removal, which it is not.
all = ["remove-ai-watermarks[video,heif,detect,trustmark,qwen-zimage,lama,migan]"]
all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan]"]
[project.scripts]
remove-ai-watermarks = "remove_ai_watermarks.cli:main"
@@ -200,7 +211,7 @@ pythonpath = ["src"]
addopts = "-v --tb=short"
[tool.ruff]
target-version = "py310"
target-version = "py311"
line-length = 120
exclude = ["_refs"]
extend-exclude = ["*.md"]
@@ -223,7 +234,7 @@ quote-style = "double"
indent-style = "space"
[tool.pyright]
pythonVersion = "3.10"
pythonVersion = "3.11"
typeCheckingMode = "strict"
exclude = ["_refs"]
+69
View File
@@ -0,0 +1,69 @@
# Maintainer scripts
These scripts are development and evaluation tools, not installed CLI commands. Run
them from the repository root with `uv run python scripts/<name>.py --help`. Inputs
under `.local-eval/` and generated reports remain untracked unless a data README
explicitly names a tracked canonical result.
## Audits and release checks
| Script | Purpose |
| --- | --- |
| `corpus_gap_scan.py` | Compare a local image corpus with the library's `identify` results. |
| `detection_timing.py` | Record per-method metadata and verdict timings. |
| `detection_timing_report.py` | Aggregate timing records by method and segment. |
| `fidelity_metrics.py` | Compute objective image-fidelity metrics for paired outputs. |
| `invisible_quality_audit.py` | Pair originals and invisible-removal outputs for quality review. |
| `metadata_removal_audit.py` | Check metadata detection/removal parity over a corpus. |
| `pill_gate_audit.py` | Measure the Jimeng pill detector on the product path. |
| `real_examples_e2e.py` | Run end-to-end confidence checks over local examples. |
| `record_parity_audit.py` | Compare record-based and file-based identification. |
| `resource_ceilings.py` | Measure peak RSS and runtime for fill backends. |
| `robustness_suite.py` | Exercise CLI failures on adversarial and degenerate inputs. |
| `sidecar_regression.py` | Compare current identification with recorded sidecars. |
| `smoke_matrix.py` | Exercise CLI parameter choices on real local data. |
| `video_fidelity_probe.py` | Compare delivered video fidelity with its source. |
| `visible_eval.py` | Benchmark registered visible-mark detectors. |
| `visible_removal_audit.py` | Audit visible-removal results over a local corpus. |
## Calibration and corpus preparation
| Script | Purpose |
| --- | --- |
| `contentseal_transforms.py` | Reproduce and hash-check deterministic Content Seal variants. |
| `detector_response.py` | Measure detector response over mark size, contrast, background, and aspect. |
| `fill_quality.py` | Measure visible-fill quality against constructed ground truth. |
| `ladder_headroom.py` | Measure recall cost from the coarse scale ladder. |
| `registered_mark_calibrate.py` | Measure a registered detector without conflating visual positives, metadata cohorts, adjudicated negatives, and unlabeled controls. |
| `synthid_corpus.py` | Ingest and inspect the local SynthID reference corpus. |
| `vendor_cohort_harvest.py` | Partition TC260 carriers by producer code. |
| `vendor_mark_calibrate.py` | Calibrate a candidate vendor text detector. |
| `visible_alpha_solve.py` | Rebuild visible-watermark alpha assets from controlled captures. |
| `visible_groundtruth.py` | Consolidate blinded contact-sheet labels into ground truth. |
| `visible_positives.py` | List corpus images carrying a registered visible mark. |
| `visible_recall_sample.py` | Build an unbiased blinded sample for recall measurement. |
| `visible_sheets.py` | Build blinded contact sheets for relaxation candidates. |
## Research and diagnostic prototypes
| Script | Purpose |
| --- | --- |
| `cjk_tail_probe.py` | Test a generic template for otherwise uncovered CJK labels. |
| `controlnet_sweep.py` | Sweep the historical ControlNet removal prototype. |
| `infer_text_lines.py` | Draft stable source-text lines without modifying pixels. |
| `qwen_scrub_prototype.py` | Probe low-strength Qwen regeneration on a GPU. |
| `selective_text_restoration.py` | Evaluate text restoration over a scrubbed image. |
| `synthid_pixel_probe.py` | Run the experimental local SynthID carrier probe. |
| `video_synthid_sweep.py` | Build oracle-gated video regeneration candidates. |
## Generated assets
| Script | Purpose |
| --- | --- |
| `render_pill_silhouette.py` | Render the synthetic Jimeng pill silhouette. |
| `render_vendor_silhouettes.py` | Render synthetic vendor text-mark silhouettes. |
## Shared helpers
`_plain_console.py` provides plain-text fallbacks for Rich output, and
`_text_eval.py` contains normalization helpers shared by text-evaluation scripts.
+34
View File
@@ -0,0 +1,34 @@
"""Pure text-normalization helpers shared by evaluation scripts."""
from __future__ import annotations
import unicodedata
def normalize_text(text: str) -> str:
"""Normalize text for layout-independent evaluation comparisons."""
return "".join(unicodedata.normalize("NFC", text).casefold().split())
def levenshtein_normalized(left: str, right: str) -> float:
"""Return normalized Levenshtein distance without changing either input."""
if not left and not right:
return 0.0
previous = list(range(len(right) + 1))
for left_index, left_character in enumerate(left, start=1):
current = [left_index]
for right_index, right_character in enumerate(right, start=1):
current.append(
min(
current[-1] + 1,
previous[right_index] + 1,
previous[right_index - 1] + (left_character != right_character),
)
)
previous = current
return previous[-1] / max(len(left), len(right))
def normalized_edit_distance(left: str, right: str) -> float:
"""Normalize text, then return Levenshtein distance over the result."""
return levenshtein_normalized(normalize_text(left), normalize_text(right))
@@ -0,0 +1,26 @@
# Unregistered visible-mark candidates
Synthetic detection silhouettes for marks that are measured but NOT registered:
none of these separates its positives from clean negatives yet, so shipping them
would attribute and fill corners on content that carries no mark (a false fill
is the worse error). Assets are regenerated by
`scripts/render_vendor_silhouettes.py`; a candidate is deleted from here on the
day its mark registers (the asset moves to the package `assets/`).
Measurements below are local calibration snapshots. Candidate pools are
provenance cohorts, not automatically visible-mark positives; comparison controls
are independently selected no-signal images and are not adjudicated negatives.
Each needs capture-solved alphas or vendor-accurate font work before it can
ship; `scripts/vendor_mark_calibrate.py` is the candidate-detector harness.
| Asset | Mark | Evidence | Result |
| --- | --- | --- | --- |
| `samsung_en_alpha.png` | Samsung Galaxy AI label, English locale ("AI-generated content", bottom-left) | 5 corpus files | POS 0.11-0.30 vs NEG max 0.40 (binary); tophat/gray tried, no separation. The registered Italian engine scores 0.18-0.31 on the same files -- same layout class, wrong glyph template. |
| `gemini_text_alpha.png` | "Generated with Gemini" text label (bottom-right; the registered gemini mark is the sparkle icon) | 3 corpus files | POS 0.07-0.22 vs NEG max 0.32; coverage gate finds the blob (0.29-0.35) but the Arial silhouette misses Google's letterforms. |
| `notebooklm_alpha.png` | NotebookLM wordmark (bottom-right) | 12 corpus files | locate geometry not yet fitted; POS max 0.12. |
| `dola_alpha.png` | DolaAI on images (the video mark is registered) | 12 corpus files | POS 0.11-0.21 vs NEG max 0.30. |
| `mindvideo_alpha.png` | MindVideo.AI (top-right) | 11 corpus files | POS 0.29-0.32 vs NEG max 0.30 -- borderline overlap, not shippable. |
| `xinghui_alpha.png` | 星绘AI生成 (parked before this set) | -- | prior parking, unchanged. |
| `qingyan_alpha.png` | 清言·AI生成 (parked before this set) | -- | prior parking, unchanged. |
| `hailuo_alpha.png` | Hailuo AI image wordmark (parked before this set; the VIDEO label is registered) | -- | prior parking, unchanged. |
| `catlogo_alpha.png` | outline cat-head + AI生成 (parked before this set) | -- | prior parking, unchanged. |
Binary file not shown.

After

Width:  |  Height:  |  Size: 3.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.7 KiB

+1 -1
View File
@@ -1,7 +1,7 @@
"""Can one GENERIC template cover the CJK AI labels no per-vendor detector fires on?
THE OPPORTUNITY
Corpus inspection of doubao-provenance misses turned up `千问AI生成` (Alibaba Qwen) and
Corpus inspection of doubao-provenance misses turned up `千问AI生成` (Alibaba Cloud Qwen) and
`百度 AI生成` (Baidu) sitting in the same bottom-right corner as the marks we do cover,
bold and plainly legible, with no detector able to fire on either. `docs/...landscape`
puts uncovered vendors at ~6% of sampled images -- larger than any tuning gain left in
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/env python3
"""Reproduce the deterministic Content Seal crop, resize, and JPEG variants."""
from __future__ import annotations
import argparse
import csv
import hashlib
import logging
from pathlib import Path
from PIL import Image
log = logging.getLogger(__name__)
ROOT = Path(__file__).resolve().parents[1]
CORPUS = ROOT / "data" / "contentseal"
MANIFEST = CORPUS / "manifest.csv"
def _rows() -> dict[str, dict[str, str]]:
with MANIFEST.open(newline="", encoding="utf-8") as stream:
return {row["name"]: row for row in csv.DictReader(stream)}
def _write_and_verify(image: Image.Image, path: Path, row: dict[str, str], *, format: str, quality: int) -> None:
image.save(path, format=format, quality=quality)
digest = hashlib.sha256(path.read_bytes()).hexdigest()
if digest != row["sha256"]:
raise RuntimeError(f"{row['name']} hash mismatch: expected {row['sha256']}, got {digest}")
log.info("Verified %s", path)
def reproduce_transforms(output_dir: Path) -> list[Path]:
"""Write and hash-check the eight deterministic manifest variants."""
rows = _rows()
output_dir.mkdir(parents=True, exist_ok=True)
outputs: list[Path] = []
for prefix, source_name in (("fox", "gen_fox_forest"), ("text", "gen_text_poster")):
with Image.open(CORPUS / rows[source_name]["file"]) as opened:
source = opened.convert("RGB")
for fraction in (0.5, 0.33):
width = int(source.width * fraction)
height = int(source.height * fraction)
left = (source.width - width) // 2
top = (source.height - height) // 2
name = f"{prefix}_crop{int(fraction * 100)}"
path = output_dir / f"{name}.webp"
crop = source.crop((left, top, left + width, top + height))
_write_and_verify(crop, path, rows[name], format="WEBP", quality=95)
outputs.append(path)
scale = 512 / max(source.size)
resized = source.resize(
(round(source.width * scale), round(source.height * scale)),
Image.Resampling.LANCZOS,
)
name = f"{prefix}_res512"
path = output_dir / f"{name}.webp"
_write_and_verify(resized, path, rows[name], format="WEBP", quality=95)
outputs.append(path)
name = f"{prefix}_jpeg85"
path = output_dir / f"{name}.jpg"
_write_and_verify(source, path, rows[name], format="JPEG", quality=85)
outputs.append(path)
return outputs
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("output_dir", type=Path, help="Directory for regenerated variants")
args = parser.parse_args()
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
reproduce_transforms(args.output_dir)
if __name__ == "__main__":
main()
+1 -1
View File
@@ -39,7 +39,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parent))
from detection_timing import COMPONENTS as _TIMED
# Taken from the script that WROTE the rows, in its order, so a probe added, removed
# or reordered there cannot silently leave a column missing or mislabelled here.
# or reordered there cannot silently leave a column missing or mislabeled here.
COMPONENTS = tuple(name for name, _ in _TIMED)
METHODS = (
("cold_extract_evidence_ms", "extract_provenance_evidence (cold)"),
+9 -8
View File
@@ -1,12 +1,11 @@
# /// script
# requires-python = ">=3.10"
# requires-python = ">=3.11"
# dependencies = [
# "click",
# "numpy",
# "opencv-python-headless",
# "opencv-python-headless<5",
# "pillow",
# "scikit-image",
# "rapidfuzz",
# "torch",
# "lpips",
# "paddleocr",
@@ -29,7 +28,7 @@ metrics run only where faces are detected, text metrics only where text is.
Two subcommands:
ocr -- OCR images (PaddleOCR PP-OCRv6) into a JSON {basename: text} file.
ocr -- OCR images (PaddleOCR defaults) into a JSON {basename: text} file.
Run this on the ORIGINALS, hand-verify/correct the file, and it
becomes the ground truth for ``compare --ground-truth`` -- the clean
way to score text, since OCR-vs-OCR is doubly noisy (errors on both
@@ -54,6 +53,7 @@ Usage:
from __future__ import annotations
import json
import sys
import unicodedata
from dataclasses import dataclass, field
from pathlib import Path
@@ -65,7 +65,10 @@ import numpy as np
from _plain_console import Console, Table
console = Console()
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
from scripts._text_eval import levenshtein_normalized # noqa: E402
# ── helpers ──────────────────────────────────────────────────────────
@@ -89,7 +92,7 @@ def _norm(text: str) -> str:
return "".join(unicodedata.normalize("NFC", text).split())
# ── text: PaddleOCR (PP-OCRv6) ───────────────────────────────────────
# ── text: PaddleOCR defaults ─────────────────────────────────────────
# Our lang codes -> PaddleOCR lang. The 'ch' model also reads Latin; 'ru' reads
# Cyrillic + Latin. Multiple langs in one image -> run each model, union detections.
@@ -165,9 +168,7 @@ def _ocr_lines(bgr: np.ndarray, langs: list[str], min_score: float = 0.5) -> lis
def _cer(ref: str, hyp: str) -> float:
from rapidfuzz.distance import Levenshtein
return Levenshtein.normalized_distance(_norm(ref), _norm(hyp))
return levenshtein_normalized(_norm(ref), _norm(hyp))
# ── face: detection + ArcFace + texture ──────────────────────────────
+99
View File
@@ -0,0 +1,99 @@
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "click",
# "huggingface-hub>=0.20.0",
# "numpy",
# "onnxruntime>=1.24.0",
# "opencv-python-headless<5",
# "paddleocr>=3.3.3",
# "paddlepaddle",
# "pillow",
# ]
# ///
"""Infer stable source-text lines without modifying an image.
This evaluation-only dry run proposes line annotations for selective text
restoration. Every proposal still needs human verification: stable OCR can lose
punctuation with high confidence. It separately flags lines whose recognition
changes under crop jitter or whose minimum confidence is below the threshold.
"""
from __future__ import annotations
import json
import logging
import os
import sys
from pathlib import Path
import click
import numpy as np
from PIL import Image
log = logging.getLogger(__name__)
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
# Dogfoods the packaged draft API (remove_ai_watermarks.text_draft); this script
# keeps only the CLI wrapper so the package stays the one home for the logic.
from remove_ai_watermarks.text_draft import ( # noqa: E402
_detect_line_boxes,
_recognize,
choose_language,
stable_recognition,
)
@click.command()
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
@click.option("--out", required=True, type=click.Path(dir_okay=False, path_type=Path))
@click.option("--min-score", default=0.85, show_default=True, type=click.FloatRange(0.0, 1.0))
def main(source: Path, out: Path, min_score: float) -> None:
"""Write draft line text for SOURCE; manually verify every proposal."""
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
os.environ["PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK"] = "True"
from paddleocr import PaddleOCR, TextRecognition
source_rgb = np.asarray(Image.open(source).convert("RGB"))
detector = PaddleOCR(
lang="ch",
use_doc_orientation_classify=False,
use_doc_unwarping=False,
use_textline_orientation=False,
)
engines = {
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec"),
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec"),
"ch": TextRecognition(model_name="PP-OCRv5_server_rec"),
}
boxes = _detect_line_boxes(detector, source_rgb)
accepted = []
rejected = []
for box in boxes:
probes = {}
for language, engine in engines.items():
script = "cjk" if language == "ch" else "alphabetic"
probes[language] = _recognize(engine, source_rgb, box, script, 0.1)
language = choose_language(probes)
script = "cjk" if language == "ch" else "alphabetic"
reads = [_recognize(engines[language], source_rgb, box, script, ratio) for ratio in (0.08, 0.12, 0.2)]
text = stable_recognition(reads, min_score)
result = {
"box": box,
"script": script,
"language": language,
"reads": [{"text": value, "score": score} for value, score in reads],
}
if text is None:
rejected.append(result)
else:
accepted.append({"box": box, "text": text, "script": script, "min_score": min(score for _, score in reads)})
payload = {"source": source.name, "accepted": accepted, "rejected": rejected}
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
log.info("Accepted %s lines and rejected %s uncertain lines", len(accepted), len(rejected))
if __name__ == "__main__":
main()
+1 -1
View File
@@ -1,5 +1,5 @@
# /// script
# requires-python = ">=3.10"
# requires-python = ">=3.11"
# dependencies = [
# "diffusers>=0.35.0",
# "transformers>=4.51.0",
+2 -2
View File
@@ -81,7 +81,7 @@ def find_visible_positive(mark: str) -> Path | None:
class Results:
"""Collects one row per checked behaviour.
"""Collects one row per checked behavior.
A FAIL keeps the command's OUTPUT. That is not cosmetic: the first run of this harness
discarded it, an `all` invocation failed once with exit 1, and because the output was
@@ -166,7 +166,7 @@ def check_visible(res: Results, tmp: Path) -> None:
"""Real marked images -> the PRODUCT'S DECISION is honoured, and a removed mark clears.
The success criterion is not "the mark is always gone" -- it is "the product did what it
decided, and the decision is right". Two designed behaviours make a blind re-detect
decided, and the decision is right". Two designed behaviors make a blind re-detect
misleading:
* The pill is GATED (`_keep_pill`): a low-confidence pill with no corroboration is
deliberately NOT removed, so `visible` correctly writes nothing and exits 2. That is
+175
View File
@@ -0,0 +1,175 @@
"""Measure a registered visible-mark detector on independently labeled arms.
The input is a JSONL manifest with exactly two fields per row:
``path``
Absolute path, or a path relative to the manifest.
``arm``
``positive`` for a visually confirmed carrier, ``metadata`` for a
provenance-only cohort, ``negative`` for an independently adjudicated
no-mark image, or ``control`` for an unlabeled comparison image with no
known local signal.
The arms stay separate because metadata names a provider, not the presence of a
visible mark, and missing local signals do not make an image a true negative. The
script imports the registered engine and reads its shipped gate, so calibration
cannot silently use a copied configuration.
Input images and manifests are read-only. Keep private inputs and generated
manifests outside the repository or in a gitignored evaluation directory.
uv run python scripts/registered_mark_calibrate.py microsoft manifest.jsonl
"""
from __future__ import annotations
import argparse
import json
import math
import sys
from collections import Counter
from pathlib import Path
from typing import Any, Literal, TypedDict, cast
REPO = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(REPO / "src"))
from remove_ai_watermarks import watermark_registry # noqa: E402
from remove_ai_watermarks.image_io import imread # noqa: E402
Arm = Literal["positive", "metadata", "negative", "control"]
_ARMS: tuple[Arm, ...] = ("positive", "metadata", "negative", "control")
class ManifestRow(TypedDict):
path: str
arm: Arm
class ArmSummary(TypedDict):
n: int
unreadable: int
min: float | None
p50: float | None
p90: float | None
p99: float | None
max: float | None
fires: int
def _percentile(ordered: list[float], fraction: float) -> float | None:
"""Return a nearest-rank percentile from values sorted in ascending order."""
if not ordered:
return None
return ordered[max(0, math.ceil(fraction * len(ordered)) - 1)]
def summarize(values: list[float], *, unreadable: int, fires: int) -> ArmSummary:
"""Summarize one independently defined arm without inferring its label."""
ordered = sorted(values)
return {
"n": len(values),
"unreadable": unreadable,
"min": _percentile(ordered, 0.0),
"p50": _percentile(ordered, 0.5),
"p90": _percentile(ordered, 0.9),
"p99": _percentile(ordered, 0.99),
"max": _percentile(ordered, 1.0),
"fires": fires,
}
def load_manifest(path: Path) -> list[ManifestRow]:
"""Load and validate a manifest, resolving relative paths beside it."""
rows: list[ManifestRow] = []
seen: set[Path] = set()
with path.open(encoding="utf-8") as manifest:
for line_number, line in enumerate(manifest, 1):
if not line.strip():
continue
raw = json.loads(line)
if not isinstance(raw, dict) or set(raw) != {"path", "arm"}:
raise ValueError(f"{path}:{line_number}: expected exactly path and arm")
raw_path = raw["path"]
raw_arm = raw["arm"]
if not isinstance(raw_path, str) or not raw_path:
raise ValueError(f"{path}:{line_number}: path must be a non-empty string")
if raw_arm not in _ARMS:
raise ValueError(f"{path}:{line_number}: arm must be one of {', '.join(_ARMS)}")
image_path = Path(raw_path)
if not image_path.is_absolute():
image_path = path.parent / image_path
image_path = image_path.resolve()
if image_path in seen:
raise ValueError(f"{path}:{line_number}: duplicate image path {image_path}")
seen.add(image_path)
rows.append({"path": str(image_path), "arm": cast("Arm", raw_arm)})
if not rows:
raise ValueError(f"{path}: manifest is empty")
return rows
def measure(mark: str, rows: list[ManifestRow]) -> tuple[float, dict[Arm, ArmSummary]]:
"""Score every row with the registered engine and its current shipped gate."""
registered_mark = watermark_registry.get_mark(mark)
# Calibration intentionally resolves the registry's concrete engine so it
# cannot drift onto a copied configuration.
engine = watermark_registry._engine(mark)
config = getattr(engine, "config", None)
gate = getattr(config, "detect_ncc_threshold", None)
if not isinstance(gate, int | float):
raise ValueError(f"registered mark {mark!r} does not expose a text-detector NCC gate")
scores: dict[Arm, list[float]] = {arm: [] for arm in _ARMS}
unreadable: Counter[Arm] = Counter()
fires: Counter[Arm] = Counter()
for row in rows:
image = imread(row["path"])
if image is None:
unreadable[row["arm"]] += 1
continue
detection: Any = registered_mark.detect(image)
scores[row["arm"]].append(float(detection.confidence))
fires[row["arm"]] += bool(detection.detected)
threshold = float(gate)
return threshold, {arm: summarize(scores[arm], unreadable=unreadable[arm], fires=fires[arm]) for arm in _ARMS}
def _format_stat(value: float | None) -> str:
return "-" if value is None else f"{value:.3f}"
def print_report(mark: str, gate: float, summaries: dict[Arm, ArmSummary]) -> None:
"""Print a compact human-readable calibration report."""
print(f"registered mark: {mark} gate: {gate:.3f}")
print(
f"{'arm':10s} {'n':>5s} {'bad':>5s} {'min':>7s} {'p50':>7s} {'p90':>7s} {'p99':>7s} {'max':>7s} {'fires':>7s}"
)
for arm in _ARMS:
row = summaries[arm]
print(
f"{arm:10s} {row['n']:5d} {row['unreadable']:5d} "
f"{_format_stat(row['min']):>7s} {_format_stat(row['p50']):>7s} "
f"{_format_stat(row['p90']):>7s} {_format_stat(row['p99']):>7s} "
f"{_format_stat(row['max']):>7s} {row['fires']:7d}"
)
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("mark", choices=watermark_registry.mark_keys())
parser.add_argument("manifest", type=Path)
parser.add_argument("--json", action="store_true", help="emit machine-readable JSON")
args = parser.parse_args()
rows = load_manifest(args.manifest)
gate, summaries = measure(args.mark, rows)
if args.json:
print(json.dumps({"mark": args.mark, "gate": gate, "arms": summaries}, sort_keys=True))
else:
print_report(args.mark, gate, summaries)
if __name__ == "__main__":
main()
+123 -17
View File
@@ -12,11 +12,14 @@ from __future__ import annotations
import sys
from pathlib import Path
from typing import Any
from typing import TYPE_CHECKING, Any
import numpy as np
from PIL import Image, ImageDraw, ImageFont
if TYPE_CHECKING:
from collections.abc import Callable
_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(_ROOT / "src"))
@@ -42,7 +45,7 @@ MARKS = {
"kling_alpha.png": "可灵AI 3.0",
# The "cat-logo" candidate stamps an outline cat-head plus bold "AI生成",
# bottom-right. It remains unregistered pending sufficient calibration coverage.
"catlogo_alpha.png": "CATLOGO", # sentinel: drawn by draw_catlogo(), not font-rendered
"catlogo_alpha.png": "cat logo + AI生成",
# RunningHub top-left text mark.
"runninghub_alpha.png": "RunningHub AI生成",
# LibLibAI bottom-center wordmark.
@@ -53,6 +56,19 @@ MARKS = {
"hailuo_alpha.png": "Hailuo AI",
# Baidu bottom-right text run.
"baidu_alpha.png": "百度",
# Measured Microsoft top-right white AI-badge variant. Sentinel: drawn by
# draw_msbadge(), not font-rendered.
"microsoft_alpha.png": "Made with AI",
# Samsung Galaxy AI label, English locale (the registered samsung_alpha.png is
# the Italian "Contenuti generati dall'AI" silhouette; EN is the literal
# translation with the same leading sparkle). Sentinel: draw_samsung_en().
"samsung_en_alpha.png": "AI-generated content",
# Gemini text-form label (the registered gemini mark is the sparkle icon).
"gemini_text_alpha.png": "Generated with Gemini",
# Candidate wordmarks measured on a local evaluation corpus (unregistered).
"notebooklm_alpha.png": "NotebookLM",
"dola_alpha.png": "DolaAI",
"mindvideo_alpha.png": "MindVideo.AI",
}
_REGISTERED = {f"{key}_alpha.png" for key in mark_keys()} & MARKS.keys()
@@ -74,6 +90,24 @@ MARK_OPTS: dict[str, dict[str, Any]] = {
}
def _fit_font(
font_path: str,
reaches_target: Callable[[ImageFont.FreeTypeFont], bool],
*,
index: int = 0,
) -> ImageFont.FreeTypeFont:
"""Return the smallest 8-200 px font that reaches a render target."""
low, high = 8, 200
while low < high:
size = (low + high) // 2
font = ImageFont.truetype(font_path, size, index=index)
if reaches_target(font):
high = size
else:
low = size + 1
return ImageFont.truetype(font_path, low, index=index)
def render(text: str, width: int = 335, opts: dict[str, Any] | None = None) -> np.ndarray:
"""Binary glyph silhouette (255 = glyph), sized to the doubao asset's convention.
@@ -91,13 +125,11 @@ def render(text: str, width: int = 335, opts: dict[str, Any] | None = None) -> n
probe = Image.new("L", (10, 10))
d0 = ImageDraw.Draw(probe)
lines = text.split("\n")
size = 8
while size < 200: # grow until the LONGEST line fills the target width
f = ImageFont.truetype(font_path, size, index=font_index)
if max(d0.textbbox((0, 0), ln, font=f)[2] for ln in lines) >= width * 0.98:
break
size += 1
font = ImageFont.truetype(font_path, size, index=font_index)
font = _fit_font(
font_path,
lambda f: max(d0.textbbox((0, 0), ln, font=f)[2] for ln in lines) >= width * 0.98,
index=font_index,
)
boxes = [d0.textbbox((0, 0), ln, font=font) for ln in lines]
line_h = max(bb[3] - bb[1] for bb in boxes)
gap = max(1, int(line_h * gap_frac))
@@ -138,13 +170,7 @@ def draw_catlogo(width: int = 335) -> np.ndarray:
probe = Image.new("L", (10, 10))
d0 = ImageDraw.Draw(probe)
text = "AI生成"
size = 8
while size < 200:
f = ImageFont.truetype(_FONT, size)
if d0.textbbox((0, 0), text, font=f)[2] >= width * 0.60:
break
size += 1
font = ImageFont.truetype(_FONT, size)
font = _fit_font(_FONT, lambda f: d0.textbbox((0, 0), text, font=f)[2] >= width * 0.60)
bb = d0.textbbox((0, 0), text, font=font)
tw, th = bb[2] - bb[0], bb[3] - bb[1]
cs = int(th * 1.08)
@@ -182,10 +208,90 @@ def draw_catlogo(width: int = 335) -> np.ndarray:
return np.array(im)
def _star_pts(cx: float, cy: float, r: float, waist: float) -> list[tuple[float, float]]:
return [
(cx, cy - r),
(cx + r * waist, cy - r * waist),
(cx + r, cy),
(cx + r * waist, cy + r * waist),
(cx, cy + r),
(cx - r * waist, cy + r * waist),
(cx - r, cy),
(cx - r * waist, cy - r * waist),
]
def _sparkle(draw: ImageDraw.ImageDraw, cx: float, cy: float, r: float) -> None:
"""Draw the four-point cutout used by synthetic candidate silhouettes."""
draw.polygon(_star_pts(cx, cy, r, 0.22), fill=0)
def draw_msbadge(width: int = 335) -> np.ndarray:
"""Synthetic silhouette for the measured Microsoft top-right white pill.
The top-hat front-end sees the bright pill with dark-text holes, so the template
carries the same holes -- that is what discriminates this pill from any other
white rounded element in the top-right corner. The text and four-point cutout
approximate the measured internal shape; they do not assert one universal
Microsoft icon or wording. Geometry measured on 17 visually confirmed carriers
on 2026-08-27: pill 0.152W x 0.040W, margins ~0.010W right / ~0.007W top,
glyph height ~0.39 of pill height.
"""
h = round(width / 3.78)
im = Image.new("L", (width, h), 0)
d = ImageDraw.Draw(im)
d.rounded_rectangle([0, 0, width - 1, h - 1], radius=h // 2, fill=255)
font_path = "/System/Library/Fonts/Supplemental/Arial Bold.ttf"
text = "Made with AI"
probe = Image.new("L", (10, 10))
d0 = ImageDraw.Draw(probe)
font = _fit_font(font_path, lambda f: d0.textbbox((0, 0), text, font=f)[3] >= h * 0.39)
bb = d0.textbbox((0, 0), text, font=font)
th = bb[3] - bb[1]
r = h * 0.20 # sparkle radius, ~half the text height
pad_l = h * 0.22
cx = pad_l + r
cy = h / 2 - 1
tx = int(pad_l + 2 * r + h * 0.22)
d.text((tx - bb[0], (h - th) // 2 - bb[1]), text, font=font, fill=0)
_sparkle(d, cx, cy, r)
return np.array(im)
def draw_samsung_en(width: int = 335) -> np.ndarray:
"""Samsung Galaxy AI English label: "AI-generated content" with the leading
4-point sparkle, light-gray glyphs (same class as the registered Italian asset)."""
text = "AI-generated content"
font_path = "/System/Library/Fonts/Supplemental/Arial.ttf"
probe = Image.new("L", (10, 10))
d0 = ImageDraw.Draw(probe)
font = _fit_font(font_path, lambda f: d0.textbbox((0, 0), text, font=f)[2] >= width * 0.80)
bb = d0.textbbox((0, 0), text, font=font)
tw, th = bb[2] - bb[0], bb[3] - bb[1]
r = th * 0.55
gap = th * 0.45
im = Image.new("L", (int(tw + gap + 2 * r + 8), th + 8), 0)
d = ImageDraw.Draw(im)
# sparkle as bright glyph (this silhouette is light-glyph class, not a pill)
d.polygon(_star_pts(4 + r, 4 + th / 2, r, 0.22), fill=255)
d.text((4 + 2 * r + gap - bb[0], 4 - bb[1]), text, font=font, fill=255)
arr = np.array(im)
ys, xs = np.where(arr > 0)
return arr[ys.min() : ys.max() + 1, xs.min() : xs.max() + 1]
_CUSTOM_RENDERERS = {
"catlogo_alpha.png": draw_catlogo,
"microsoft_alpha.png": draw_msbadge,
"samsung_en_alpha.png": draw_samsung_en,
}
def main() -> None:
try:
for name, text in MARKS.items():
sil = draw_catlogo() if text == "CATLOGO" else render(text, opts=MARK_OPTS.get(name))
renderer = _CUSTOM_RENDERERS.get(name)
sil = renderer() if renderer is not None else render(text, opts=MARK_OPTS.get(name))
output_dir = _PACKAGE_ASSETS if name in _REGISTERED else _CANDIDATE_ASSETS
output_dir.mkdir(parents=True, exist_ok=True)
output = output_dir / name
+528
View File
@@ -0,0 +1,528 @@
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "click",
# "huggingface-hub>=0.20.0",
# "numpy",
# "onnxruntime>=1.24.0",
# "opencv-python-headless<5",
# "paddleocr>=3.3.3",
# "paddlepaddle",
# "pillow",
# ]
# ///
"""Evaluation-only text restoration over a scrubbed image.
``vae-glyphs`` composites only thresholded glyph-core pixels from a separately
generated VAE reconstruction over a fresh silhouette edge. ``source-glyphs``
can preserve typeface, layout, color, and antialiasing when
SOURCE is itself a regenerated layer. It must not be treated as safe when SOURCE
is the watermarked original: a provider oracle detected SynthID after that exact
paste-back experiment. ``source-silhouette`` instead transfers only a
thresholded glyph shape, then synthesizes fresh flat-color pixels and
antialiasing. ``rerender`` retains the system-font negative control. This is
not a production stage and does not add PaddleOCR to the package graph.
"""
from __future__ import annotations
import json
import logging
import math
import os
import shutil
import sys
import unicodedata
from dataclasses import asdict
from pathlib import Path
from typing import Any
import click
import cv2
import numpy as np
from PIL import Image, ImageDraw, ImageFont
log = logging.getLogger(__name__)
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
sys.path.insert(0, str(ROOT / "src"))
from remove_ai_watermarks import region_eraser # noqa: E402
from remove_ai_watermarks._internal.text_restoration import VerifiedTextLine as TextLine # noqa: E402
from remove_ai_watermarks._internal.text_restoration import ( # noqa: E402
composite_fresh_text_edges,
composite_reconstructed_glyphs,
group_text_lines,
residual_glyph_mask,
source_silhouette_mask,
)
from scripts._text_eval import normalize_text, normalized_edit_distance # noqa: E402
if ROOT not in Path(region_eraser.__file__).resolve().parents:
raise RuntimeError("selective_text_restoration imported outside the current worktree")
REGULAR_FONT = Path("/System/Library/Fonts/Supplemental/Arial.ttf")
BOLD_FONT = Path("/System/Library/Fonts/Supplemental/Arial Bold.ttf")
CJK_FONT = Path("/System/Library/Fonts/STHeiti Medium.ttc")
def should_preserve_line(
expected: str,
source_text: str,
source_score: float,
candidate_text: str,
candidate_score: float,
) -> bool:
if min(source_score, candidate_score) < 0.75:
return False
if normalized_edit_distance(expected, source_text) > 0.25:
return False
return normalize_text(source_text) == normalize_text(candidate_text)
def composite_source_glyphs(
source_rgb: np.ndarray,
background_rgb: np.ndarray,
glyph_mask: np.ndarray,
*,
feather: float = 0.7,
) -> np.ndarray:
"""Composite exact source pixels inside a glyph mask with an outer feather."""
if source_rgb.shape != background_rgb.shape or source_rgb.shape[:2] != glyph_mask.shape:
raise ValueError("source, background, and glyph mask dimensions must match")
blurred = cv2.GaussianBlur(glyph_mask, (0, 0), feather) if feather > 0 else glyph_mask
alpha = np.maximum(glyph_mask, blurred).astype(np.float32) / 255.0
alpha = alpha[..., None]
combined = source_rgb.astype(np.float32) * alpha + background_rgb.astype(np.float32) * (1.0 - alpha)
return np.clip(combined, 0, 255).astype(np.uint8)
def source_box_mask(
shape: tuple[int, int],
boxes: list[tuple[int, int, int, int]],
) -> np.ndarray:
"""Build a padded text-line mask for aligned regenerated layer compositing."""
height, width = shape
mask = np.zeros((height, width), dtype=np.uint8)
for x1, y1, x2, y2 in boxes:
pad = max(8, (y2 - y1) // 4)
x1, y1, x2, y2 = _clip_box((x1, y1, x2, y2), width, height, pad=pad)
mask[y1:y2, x1:x2] = 255
return mask
def _clip_box(box: tuple[int, int, int, int], width: int, height: int, pad: int = 0) -> tuple[int, int, int, int]:
x1, y1, x2, y2 = box
return max(0, x1 - pad), max(0, y1 - pad), min(width, x2 + pad), min(height, y2 + pad)
def foreground_mask(source_rgb: np.ndarray, box: tuple[int, int, int, int]) -> np.ndarray:
height, width = source_rgb.shape[:2]
line_height = box[3] - box[1]
x1, y1, x2, y2 = _clip_box(box, width, height, pad=max(6, int(line_height * 0.12)))
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
ring_pad = max(8, min(24, (y2 - y1) // 5))
rx1, ry1, rx2, ry2 = _clip_box((x1, y1, x2, y2), width, height, pad=ring_pad)
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
ring = np.ones(context.shape, dtype=bool)
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
background_luma = float(np.median(context[ring])) if ring.any() else float(np.median(gray))
low, high = float(np.percentile(gray, 4)), float(np.percentile(gray, 96))
dark_contrast, light_contrast = background_luma - low, high - background_luma
contrast = max(light_contrast, dark_contrast)
threshold = max(24.0, min(72.0, contrast * 0.32))
if light_contrast > dark_contrast:
mask = (gray.astype(np.float32) >= background_luma + threshold).astype(np.uint8) * 255
else:
mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((2, 2), np.uint8))
dilation = 5 if line_height >= 48 else 3
mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * dilation + 1,) * 2))
result = np.zeros((height, width), dtype=np.uint8)
result[y1:y2, x1:x2] = mask
return result
def _line_language(line: TextLine) -> str:
if line.script == "cjk":
return "ch"
return "ru" if any("CYRILLIC" in unicodedata.name(character, "") for character in line.text) else "en"
def _recognition_box(
line: TextLine,
width: int,
height: int,
vertical_pad_ratio: float | None = None,
) -> tuple[int, int, int, int]:
x1, y1, x2, y2 = line.box
line_height = y2 - y1
if line.script == "cjk":
left_pad = max(16, round(line_height * 0.2))
right_pad = max(16, round(line_height * 0.6))
return max(0, x1 - left_pad), y1, min(width, x2 + right_pad), y2
pad_x = max(16, line_height)
pad_y = max(8, line_height // 3) if vertical_pad_ratio is None else max(8, round(line_height * vertical_pad_ratio))
return max(0, x1 - pad_x), max(0, y1 - pad_y), min(width, x2 + pad_x), min(height, y2 + pad_y)
def _recognize(
engine: Any,
image: np.ndarray,
line: TextLine,
vertical_pad_ratio: float | None = None,
) -> tuple[str, float]:
height, width = image.shape[:2]
x1, y1, x2, y2 = _recognition_box(line, width, height, vertical_pad_ratio)
crop = image[y1:y2, x1:x2]
if crop.shape[0] < 64:
scale = 64 / crop.shape[0]
crop = cv2.resize(crop, None, fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)
result = next(iter(engine.predict(crop)))
return str(result.get("rec_text", "")), float(result.get("rec_score", 0.0))
def _sample_text_color(
source_rgb: np.ndarray,
mask: np.ndarray,
box: tuple[int, int, int, int],
) -> tuple[int, int, int]:
height, width = source_rgb.shape[:2]
x1, y1, x2, y2 = _clip_box(box, width, height, pad=2)
crop = source_rgb[y1:y2, x1:x2]
active = mask[y1:y2, x1:x2] > 0
pixels = crop[active]
luma = pixels.mean(axis=1)
background_luma = float(crop[[0, -1], :, :].reshape(-1, 3).mean(axis=1).mean())
if background_luma >= 128:
selected = pixels[luma <= np.percentile(luma, 20)]
else:
selected = pixels[luma >= np.percentile(luma, 80)]
return tuple(int(value) for value in np.median(selected, axis=0))
def _render_line(image: Image.Image, line: TextLine, color: tuple[int, int, int]) -> None:
font_path = CJK_FONT if line.script == "cjk" else (BOLD_FONT if line.box[3] - line.box[1] >= 55 else REGULAR_FONT)
target_width, target_height = line.box[2] - line.box[0], line.box[3] - line.box[1]
draw = ImageDraw.Draw(image)
low, high = 4, max(8, target_height * 2)
font = ImageFont.truetype(str(font_path), low)
while low <= high:
size = (low + high) // 2
candidate = ImageFont.truetype(str(font_path), size)
bounds = draw.textbbox((0, 0), line.text, font=candidate)
if bounds[2] - bounds[0] <= target_width * 1.03 and bounds[3] - bounds[1] <= target_height * 1.08:
font, low = candidate, size + 1
else:
high = size - 1
bounds = draw.textbbox((0, 0), line.text, font=font)
y = line.box[1] + math.floor((target_height - (bounds[3] - bounds[1])) / 2) - bounds[1]
draw.text((line.box[0], y), line.text, fill=color, font=font)
def _write_manifest(path: Path | None, payload: dict[str, Any]) -> None:
if path is None:
return
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
def _vertical_overlap_ratio(left: tuple[int, int, int, int], right: tuple[int, int, int, int]) -> float:
overlap = max(0, min(left[3], right[3]) - max(left[1], right[1]))
return overlap / max(1, min(left[3] - left[1], right[3] - right[1]))
def group_word_boxes(boxes: list[tuple[int, int, int, int]]) -> list[tuple[int, int, int, int]]:
groups: list[tuple[int, int, int, int]] = []
for box in sorted(boxes, key=lambda item: ((item[1] + item[3]) / 2, item[0])):
matches = []
for index, group in enumerate(groups):
if _vertical_overlap_ratio(box, group) < 0.45:
continue
horizontal_gap = max(0, max(box[0], group[0]) - min(box[2], group[2]))
line_height = min(box[3] - box[1], group[3] - group[1])
if horizontal_gap <= max(24, line_height * 3):
matches.append(index)
if not matches:
groups.append(box)
continue
index = max(matches, key=lambda item: _vertical_overlap_ratio(box, groups[item]))
x1, y1, x2, y2 = groups[index]
groups[index] = min(x1, box[0]), min(y1, box[1]), max(x2, box[2]), max(y2, box[3])
return sorted(groups, key=lambda item: ((item[1] + item[3]) / 2, item[0]))
def detect_line_boxes(
engine: Any,
source_rgb: np.ndarray,
expected_count: int | None = None,
) -> list[tuple[int, int, int, int]]:
boxes: list[tuple[int, int, int, int]] = []
for page in engine.predict(source_rgb):
detected = page.get("rec_boxes", None)
if detected is None or len(detected) == 0:
detected = page.get("rec_polys", [])
for score, raw_box in zip(page.get("rec_scores", []), detected, strict=False):
if float(score) < 0.5:
continue
points = np.asarray(raw_box, dtype=np.float32).reshape(-1)
if points.size == 4:
x1, y1, x2, y2 = points
else:
points = points.reshape(-1, 2)
x1, y1 = points.min(axis=0)
x2, y2 = points.max(axis=0)
boxes.append((round(float(x1)), round(float(y1)), round(float(x2)), round(float(y2))))
lines = group_word_boxes(boxes)
if expected_count is not None and len(lines) != expected_count:
raise click.ClickException(f"detected {len(lines)} source lines; expected exactly {expected_count}")
return lines
def _load_lines(path: Path, key: str) -> list[TextLine]:
payload = json.loads(path.read_text(encoding="utf-8"))
return [
TextLine(tuple(item["box"]), item["text"], item["script"], float(item.get("angle", 0.0)))
for item in payload[key]
]
@click.command()
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
@click.argument("candidate", type=click.Path(exists=True, dir_okay=False, path_type=Path))
@click.option("--output", required=True, type=click.Path(dir_okay=False, path_type=Path))
@click.option(
"--lines-json",
default=ROOT / "data/evaluations/fidelity/text-lines.json",
type=click.Path(exists=True, dir_okay=False, path_type=Path),
)
@click.option("--source-key", help="Key in lines JSON; defaults to source basename.")
@click.option(
"--detect-boxes",
is_flag=True,
help="Detect source boxes; fail unless their count matches verified lines.",
)
@click.option(
"--restoration",
type=click.Choice(("vae-glyphs", "source-glyphs", "source-silhouette", "rerender")),
required=True,
help="Choose VAE glyph cores, regenerated pixels, fresh source shapes, or the system-font control.",
)
@click.option(
"--glyph-donor",
type=click.Path(exists=True, dir_okay=False, path_type=Path),
help="VAE reconstruction used only by --restoration vae-glyphs.",
)
@click.option(
"--glyph-feather",
type=click.FloatRange(min=0.0),
default=0.5,
show_default=True,
help="Outer donor-edge feather used only by --restoration vae-glyphs.",
)
@click.option(
"--selection",
type=click.Choice(("all", "changed")),
default="all",
show_default=True,
help="Restore every verified line or only OCR-confirmed changes.",
)
@click.option(
"--erase-background/--keep-background",
default=True,
show_default=True,
help="Erase candidate glyphs before compositing, or directly blend an aligned regenerated glyph layer.",
)
@click.option(
"--composite-mask",
type=click.Choice(("glyphs", "boxes")),
default="glyphs",
show_default=True,
help="Composite isolated glyphs or complete aligned text-line boxes.",
)
@click.option("--manifest", type=click.Path(dir_okay=False, path_type=Path))
def main(
source: Path,
candidate: Path,
output: Path,
lines_json: Path,
source_key: str | None,
detect_boxes: bool,
restoration: str,
glyph_donor: Path | None,
glyph_feather: float,
selection: str,
erase_background: bool,
composite_mask: str,
manifest: Path | None,
) -> None:
"""Restore SOURCE text over the scrubbed CANDIDATE."""
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
if restoration == "rerender":
for font in (REGULAR_FONT, BOLD_FONT, CJK_FONT):
if not font.exists():
raise click.ClickException(f"required evaluation font is unavailable: {font}")
if restoration == "vae-glyphs" and glyph_donor is None:
raise click.ClickException("--glyph-donor is required for --restoration vae-glyphs")
if restoration != "vae-glyphs" and glyph_donor is not None:
raise click.ClickException("--glyph-donor is only valid with --restoration vae-glyphs")
os.environ["PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK"] = "True"
source_rgb = np.asarray(Image.open(source).convert("RGB"))
candidate_rgb = np.asarray(Image.open(candidate).convert("RGB"))
if candidate_rgb.shape != source_rgb.shape:
raise click.ClickException("source and candidate dimensions must match")
donor_rgb = np.asarray(Image.open(glyph_donor).convert("RGB")) if glyph_donor else None
if donor_rgb is not None and donor_rgb.shape != source_rgb.shape:
raise click.ClickException("source and glyph donor dimensions must match")
lines = _load_lines(lines_json, source_key or source.name)
annotation_boxes = [line.box for line in lines]
if detect_boxes:
from paddleocr import PaddleOCR
page_engine = PaddleOCR(
lang="ch",
use_doc_orientation_classify=False,
use_doc_unwarping=False,
use_textline_orientation=False,
)
boxes = detect_line_boxes(page_engine, source_rgb, len(lines))
lines = [TextLine(box, line.text, line.script) for line, box in zip(lines, boxes, strict=True)]
vertical_pad_ratio = 0.1 if detect_boxes else None
decisions = []
selected = list(lines)
if selection == "all":
decisions = [{"line": asdict(line), "selected": True, "reason": "all-lines"} for line in lines]
else:
from paddleocr import TextRecognition
engines = {
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec"),
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec"),
"ch": TextRecognition(model_name="PP-OCRv5_server_rec"),
}
selected = []
for line in lines:
language = _line_language(line)
source_text, source_score = _recognize(engines[language], source_rgb, line, vertical_pad_ratio)
candidate_text, candidate_score = _recognize(engines[language], candidate_rgb, line, vertical_pad_ratio)
preserve = should_preserve_line(line.text, source_text, source_score, candidate_text, candidate_score)
decisions.append(
{
"line": asdict(line),
"source_text": source_text,
"source_score": source_score,
"candidate_text": candidate_text,
"candidate_score": candidate_score,
"preserve": preserve,
"selected": not preserve,
}
)
if not preserve:
selected.append(line)
output.parent.mkdir(parents=True, exist_ok=True)
mask_path = output.with_name(output.stem + "_mask.png")
manifest_common = {
"source": source.name,
"candidate": candidate.name,
"output": output.name,
"mask": mask_path.name,
"glyph_donor": glyph_donor.name if glyph_donor else None,
"glyph_feather": glyph_feather if restoration == "vae-glyphs" else None,
"restoration": restoration,
"selection": selection,
"erase_background": erase_background,
"composite_mask": composite_mask,
"box_source": "detector" if detect_boxes else "verified_annotations",
"annotation_boxes": annotation_boxes,
"decisions": decisions,
}
if not selected:
combined = np.zeros(source_rgb.shape[:2], dtype=np.uint8)
shutil.copyfile(candidate, output)
Image.fromarray(combined).save(mask_path)
payload = {
**manifest_common,
"mask_fraction": 0.0,
"source_glyph_fraction": 0.0,
"source_layer_fraction": 0.0,
}
_write_manifest(manifest, payload)
log.info("Copied %s unchanged because every line passed", output)
return
if restoration in {"source-silhouette", "vae-glyphs"}:
source_masks = [source_silhouette_mask(source_rgb, line.box, line.angle) for line in selected]
candidate_masks = [source_silhouette_mask(candidate_rgb, line.box, line.angle) for line in selected]
line_masks = []
for line, source_mask, candidate_mask in zip(selected, source_masks, candidate_masks, strict=True):
radius = 5 if line.box[3] - line.box[1] >= 48 else 3
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * radius + 1,) * 2)
line_masks.append(cv2.dilate(np.maximum(source_mask, candidate_mask), kernel))
masks = line_masks
else:
source_masks = [foreground_mask(source_rgb, line.box) for line in selected]
candidate_masks = [foreground_mask(candidate_rgb, line.box) for line in selected]
masks = [np.maximum(left, right) for left, right in zip(source_masks, candidate_masks, strict=True)]
del candidate_masks
groups = group_text_lines(selected)
if erase_background:
background = cv2.cvtColor(candidate_rgb, cv2.COLOR_RGB2BGR)
for group in groups:
background = region_eraser.erase_lama(
background,
np.maximum.reduce([masks[index] for index in group]),
)
background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
residual_masks = [
residual_glyph_mask(background_rgb, mask, line.box) for line, mask in zip(selected, masks, strict=True)
]
for group in groups:
residual = np.maximum.reduce([residual_masks[index] for index in group])
if np.any(residual):
background = region_eraser.erase_lama(background, residual)
background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
else:
background_rgb = candidate_rgb
residual_masks = []
source_glyph_mask = np.maximum.reduce(source_masks)
source_layer_mask = source_glyph_mask
if restoration == "source-glyphs":
source_layer_mask = (
source_box_mask(source_rgb.shape[:2], [line.box for line in selected])
if composite_mask == "boxes"
else source_glyph_mask
)
restored = composite_source_glyphs(source_rgb, background_rgb, source_layer_mask, feather=3.0)
Image.fromarray(restored).save(output)
elif restoration in {"source-silhouette", "vae-glyphs"}:
restored = composite_fresh_text_edges(source_rgb, background_rgb, selected, source_masks)
if restoration == "vae-glyphs":
if donor_rgb is None:
raise RuntimeError("VAE glyph restoration requires a loaded donor")
restored = composite_reconstructed_glyphs(
donor_rgb,
restored,
source_layer_mask,
feather=glyph_feather,
)
Image.fromarray(restored).save(output)
else:
rendered = Image.fromarray(background_rgb)
for line, source_mask in zip(selected, source_masks, strict=True):
_render_line(rendered, line, _sample_text_color(source_rgb, source_mask, line.box))
rendered.save(output)
combined = np.maximum.reduce([*masks, *residual_masks])
if restoration == "source-glyphs" and not erase_background:
combined = source_layer_mask
Image.fromarray(combined).save(mask_path)
payload = {
**manifest_common,
"mask_fraction": float((combined > 0).mean()),
"source_glyph_fraction": float((source_glyph_mask > 0).mean()),
"source_layer_fraction": float((source_layer_mask > 0).mean()),
}
_write_manifest(manifest, payload)
log.info("Wrote %s with %.4f edited fraction", output, payload["mask_fraction"])
if __name__ == "__main__":
main()
+2 -2
View File
@@ -22,7 +22,7 @@ import shutil
import subprocess
import sys
from collections import Counter
from datetime import datetime, timezone
from datetime import UTC, datetime
from pathlib import Path
import click
@@ -180,7 +180,7 @@ def ingest(
"c2pa_issuer": issuer,
"synthid_metadata": "yes" if synthid else "",
"verified_via": verified_via,
"added": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
"added": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
"notes": notes,
}
)
+1 -1
View File
@@ -15,7 +15,7 @@ THE KEY
``001191110102MACQD9K64010000`` -> USCC ``91110102MACQD9K640``, which names a
specific legal entity. So the metadata partitions carriers into per-ENTITY
cohorts without looking at a single pixel. A cohort is a LABEL: once one frame
in it is eyeballed, every frame in it is a labelled example of that vendor's
in it is eyeballed, every frame in it is a labeled example of that vendor's
mark. That is what turns "one confirmed positive" into "30+ per vendor".
CLAUDE.md's "the generic TC260 label names no specific vendor" is about the
+4 -4
View File
@@ -14,7 +14,7 @@ WHERE THE POSITIVES COME FROM
absence) visible directly.
NEGATIVES
The 432 frames hand-labelled `present: []` in the 2026-07-18 round -- already-adjudicated
The 432 frames hand-labeled `present: []` in the 2026-07-18 round -- already-adjudicated
no-visible-mark images, so the false-fire arm rests on human labels rather than on the
absence of a detection.
@@ -115,7 +115,7 @@ def load_sets(cohort: str) -> tuple[list[str], list[str]]:
# The 2026-07-18 labels are in the vocabulary of the REGISTERED marks only
# (gemini/doubao/jimeng/jimeng_pill): `present: []` means "no registered mark", NOT
# "no mark at all" -- 146 of the 432 sit in a TC260 cohort, and qwen-cohort frames
# visibly carrying 千问AI生成 are labelled `present: []` there (measured 2026-07-21:
# visibly carrying 千问AI生成 are labeled `present: []` there (measured 2026-07-21:
# they made up the clean arm's whole top tail, clean p99 0.37 -> 0.69). A gate read
# off that arm is meaningless, so the clean arm excludes every frame in ANY TC260
# cohort -- cohort membership is the cheap proxy for "may carry a CJK AI label".
@@ -168,7 +168,7 @@ def report(pos: list[dict[str, Any]], neg: list[dict[str, Any]], name: str) -> N
p = np.array([r["score"] for r in pos])
n = np.array([r["score"] for r in neg])
print(f"\n{'=' * 78}\n{name}: candidate-cohort vs hand-labelled clean\n{'=' * 78}")
print(f"\n{'=' * 78}\n{name}: candidate-cohort vs hand-labeled clean\n{'=' * 78}")
print(f"\n{'arm':10s} {'n':>5s} {'p10':>7s} {'p50':>7s} {'p90':>7s} {'p95':>7s} {'p99':>7s} {'max':>7s}")
for label, arr in (("cohort", p), ("clean", n)):
if not len(arr):
@@ -602,7 +602,7 @@ def main() -> None:
crossfire(pools, cand, a.workers, a.gate)
return
print(f"cohort {a.cohort}: {len(pos_paths)} candidates | clean: {len(neg_paths)} hand-labelled")
print(f"cohort {a.cohort}: {len(pos_paths)} candidates | clean: {len(neg_paths)} hand-labeled")
print(f"scale_basis={a.scale_basis} overrides={overrides}")
pos = run(pos_paths, a.asset, name, a.workers, a.scale_basis, overrides)
neg = run(neg_paths, a.asset, name, a.workers, a.scale_basis, overrides)
+4 -4
View File
@@ -1,7 +1,7 @@
"""Benchmark harness for the visible-mark detectors.
Run this before AND after any detector change. It re-runs perception over the
hand-labelled ground truth and reports, per mark, how often a fire is correct --
hand-labeled ground truth and reports, per mark, how often a fire is correct --
with Wilson intervals, so a change inside the noise is visible as such.
uv run python scripts/visible_eval.py # score current code
@@ -10,10 +10,10 @@ with Wilson intervals, so a change inside the noise is visible as such.
WHAT THIS SET CAN AND CANNOT MEASURE -- read before quoting a number:
* PRECISION: sound. Every labelled crop is centred on the region a detector
* PRECISION: sound. Every labeled crop is centered on the region a detector
pointed at, so "the detector fired mark K here, was K actually there" is
exactly the question the labels answer.
* RECALL: NOT measurable here, and the harness refuses to print it. The labelled
* RECALL: NOT measurable here, and the harness refuses to print it. The labeled
images were SAMPLED WHERE DETECTORS FIRED (relaxation additions plus controls),
so images carrying a mark that every detector missed are absent by construction.
Computing recall on this set would divide by a denominator that excludes exactly
@@ -133,7 +133,7 @@ def report(res: dict, prev: dict | None = None) -> None:
conf = {k: v for k, v in res["confusion"][m].items() if k != m}
if conf:
print(f" {m:12s} {dict(sorted(conf.items(), key=lambda kv: -kv[1]))}")
print("\n'scope' = images whose crop could rule on that mark; 'missed' = labelled marks it did not fire on.")
print("\n'scope' = images whose crop could rule on that mark; 'missed' = labeled marks it did not fire on.")
print("NOTE: 'missed' is NOT recall -- this set was sampled where detectors fired, so images")
print(" every detector missed are absent by construction. Use it only to catch a change")
print(" LOSING marks it used to find; an unbiased random sample is needed for true recall.")
+18 -9
View File
@@ -1,6 +1,6 @@
"""Consolidate the hand-labelled contact-sheet rounds into ONE ground-truth file.
"""Consolidate the hand-labeled contact-sheet rounds into ONE ground-truth file.
Ground truth is `uid -> the set of visible marks actually present`, hand-labelled
Ground truth is `uid -> the set of visible marks actually present`, hand-labeled
blind against contact sheets with a two-sided control in every round. Rounds so far:
2026-07-18 text-mark/pill round : 423 cells (doubao / jimeng / jimeng_pill arms)
@@ -10,20 +10,20 @@ DATA SAFETY: treat the input dataset as sensitive. This script reads a gitignore
dataset and writes a gitignored ground-truth file. Neither the images nor this
output may be committed; only the harness is. See the repo CLAUDE.md.
The labels record what the LABELLER SAW in the crop, one of:
The labels record what the labeler saw in the crop, one of:
doubao | jimeng | pill | sparkle | other_ai_label | none | uncertain
`other_ai_label` is a real visible AI label from a vendor we do NOT have a mark for
(千问 / 百度 / 星绘 / 抖音); it is NOT a positive for any registered mark, but it is
also not "clean" -- it is exactly what the relaxed jimeng detector confuses.
`uncertain` rows are EXCLUDED from scoring rather than coerced, so a labeller's
`uncertain` rows are EXCLUDED from scoring rather than coerced, so a labeler's
honest doubt never becomes a fabricated data point.
"""
from __future__ import annotations
import argparse
import csv
import json
import sys
from pathlib import Path
SEEN_TO_MARK = {
@@ -32,7 +32,7 @@ SEEN_TO_MARK = {
"pill": "jimeng_pill",
"sparkle": "gemini",
}
# Which marks a crop centred on `key` lets the labeller rule on (same corner = visible
# Which marks a crop centered on `key` lets the labeler rule on (same corner = visible
# in the same crop). Doubao and Jimeng share the bottom-right corner.
_ADJUDICATES = {
"doubao": ("doubao", "jimeng"),
@@ -46,6 +46,7 @@ ROUNDS = [
("textmark", "labels.csv", "manifest.csv"),
("gemini", "gemini_labels.csv", "gemini_manifest.csv"),
]
_HELP_DESCRIPTION = "Consolidate hand-labeled contact-sheet rounds into one ground-truth file."
def metadata_provenance(path: str) -> list[str]:
@@ -77,7 +78,15 @@ def metadata_provenance(path: str) -> list[str]:
def main() -> None:
root = Path(sys.argv[1] if len(sys.argv) > 1 else ".local-eval/textmark-relaxation")
parser = argparse.ArgumentParser(description=_HELP_DESCRIPTION)
parser.add_argument(
"root",
type=Path,
nargs="?",
default=Path(".local-eval/textmark-relaxation"),
help="Directory containing the blinded labeling rounds",
)
root = parser.parse_args().root
out = root / "groundtruth.jsonl"
rows: dict[str, dict] = {}
stats: dict[str, int] = {}
@@ -97,8 +106,8 @@ def main() -> None:
m["uid"],
{"uid": m["uid"], "path": m["path"], "present": [], "seen": [], "rounds": [], "adjudicated": []},
)
# ADJUDICATION SCOPE -- load-bearing. A crop centred on one mark only lets
# the labeller rule on marks visible IN THAT CROP. A pill crop (top-left)
# ADJUDICATION SCOPE -- load-bearing. A crop centered on one mark only lets
# the labeler rule on marks visible IN THAT CROP. A pill crop (top-left)
# says nothing about a bottom-right wordmark, so scoring jimeng against a
# pill-round image would book real detections as false fires (~61% of pills
# carry a wordmark). Bottom-right marks co-adjudicate each other: one crop
+17 -9
View File
@@ -1,8 +1,8 @@
"""Build an UNBIASED random sample for measuring visible-mark RECALL.
Every earlier labelling round sampled where detectors FIRED, so images that every
Every earlier labeling round sampled where detectors FIRED, so images that every
detector missed were absent by construction and recall was unmeasurable. This round
samples at random within a provenance class and shows the labeller the corners where
samples at random within a provenance class and shows the labeler the corners where
a mark can physically be, so a MISSED mark is visible as such.
Design decisions that matter:
@@ -13,19 +13,19 @@ Design decisions that matter:
recall over all uploads would mostly measure how often each vendor appears.
* NATIVE RESOLUTION crops, never a downscaled whole image: a 220px preview destroys a
faint mark (measured in an earlier round), which would inflate the miss count with
the labeller's own blindness rather than the detector's.
the labeler's own blindness rather than the detector's.
* BOTH corners per image (top-left pill, bottom-right wordmark/strip/sparkle), so one
pass adjudicates every registered mark instead of one mark per crop.
* The detector's verdict is NOT shown and is not in the sheet order -- the manifest
holds it and must not be opened until labelling ends.
holds it and must not be opened until labeling ends.
"""
from __future__ import annotations
import argparse
import csv
import json
import random
import sys
from pathlib import Path
from typing import TYPE_CHECKING, Any
@@ -35,6 +35,8 @@ import numpy as np
if TYPE_CHECKING:
from numpy.typing import NDArray
import sys
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from remove_ai_watermarks.image_io import imread
@@ -63,10 +65,16 @@ def corner_strip(img: NDArray[Any]) -> NDArray[Any] | None:
def main() -> None:
scan = Path(sys.argv[1])
out = Path(sys.argv[2])
n_tc260 = int(sys.argv[3]) if len(sys.argv) > 3 else 160
n_google = int(sys.argv[4]) if len(sys.argv) > 4 else 80
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("scan", type=Path, help="JSONL corpus scan produced by the visible evaluation harness")
parser.add_argument("output", type=Path, help="Directory for contact sheets and the blinded manifest")
parser.add_argument("--tc260", type=int, default=160, help="Number of TC260 carriers to sample")
parser.add_argument("--google", type=int, default=80, help="Number of Google-provenance carriers to sample")
args = parser.parse_args()
scan = args.scan
out = args.output
n_tc260 = args.tc260
n_google = args.google
out.mkdir(parents=True, exist_ok=True)
recs = [json.loads(line) for line in scan.open() if '"marks"' in line]
+11 -6
View File
@@ -1,17 +1,18 @@
"""Build BLIND contact sheets for hand-labelling relaxation additions.
"""Build BLIND contact sheets for hand-labeling relaxation additions.
Crops are centered on the DETECTED REGION (not the corner), padded by ~0.9x the mark
size, and resized to 240px with INTER_NEAREST -- a downscaled preview destroys a faint
mark, so nothing here may smooth. The manifest is written to a separate file that must
NOT be read until labelling is finished.
NOT be read until labeling is finished.
Each sheet mixes three strata in shuffled order:
add - the relaxation additions whose precision we are measuring
pos - strict-consistent detections (a mark is really there): labeller sensitivity
clean - verified-clean negatives (no mark can be there): labeller specificity
pos - strict-consistent detections (a mark is really there): labeler sensitivity
clean - verified-clean negatives (no mark can be there): labeler specificity
The two control strata are what make a low measured precision trustworthy.
"""
import argparse
import csv
import json
import random
@@ -62,9 +63,13 @@ def crop(path: str, region: tuple[int, int, int, int] | None, pad_factor: float
def main() -> None:
with open(sys.argv[1]) as fh:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("input", type=Path, help="JSON candidate list")
parser.add_argument("output", type=Path, help="Directory for blinded contact sheets")
args = parser.parse_args()
with args.input.open() as fh:
items = json.load(fh) # [{uid,path,key,stratum,conf}]
outdir = Path(sys.argv[2])
outdir = args.output
outdir.mkdir(parents=True, exist_ok=True)
random.Random(1234).shuffle(items) # noqa: S311 -- sheet ordering, not cryptography
+1 -1
View File
@@ -32,7 +32,7 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
__version__ = "0.26.3"
__version__ = "0.34.0"
__all__ = [
"BatchSummary",
+383 -4
View File
@@ -8,6 +8,7 @@ import json
import logging
import re
import struct
from collections import deque
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING, Any, cast
@@ -37,6 +38,41 @@ with contextlib.suppress(Exception):
_C2PA_READER_AVAILABLE = _C2paReader is not None
_PNG_HEADER = struct.Struct(">I4s")
_CONTENT_BINDING_MATCHES = (
"assertion.dataHash.match",
"assertion.bmffHash.match",
"assertion.boxesHash.match",
"assertion.collectionHash.match",
"assertion.multiAssetHash.match",
)
_CONTENT_BINDING_FAILURE_MARKERS = (
"Hash.incorrectFileCount",
"Hash.malformed",
"Hash.mismatch",
"Hash.missingPart",
"assertion.hashedURI.mismatch",
"claim.hardBindings.missing",
"hashedUri.mismatch",
"ingredient.manifest.mismatch",
)
_SIGNATURE_FAILURES = frozenset(
{
"claimSignature.mismatch",
"claimSignature.missing",
"claimSignature.outsideValidity",
}
)
# A credential the issuer itself has disowned, disqualifying like a hash mismatch.
# ``signingCredential.expired`` is deliberately absent: an expired certificate does not
# imply the signed bytes changed, and a signature actually made outside validity already
# arrives as ``claimSignature.outsideValidity`` above.
_CREDENTIAL_FAILURES = frozenset(
{
"signingCredential.invalid",
"signingCredential.ocsp.revoked",
}
)
@dataclass(frozen=True)
class _PngChunk:
@@ -185,6 +221,184 @@ def _active_manifest(store: dict[str, Any]) -> dict[str, Any]:
return cast("dict[str, Any]", active) if isinstance(active, dict) else {}
def _manifest_chain(store: dict[str, Any]) -> list[dict[str, Any]]:
"""Return the active manifest followed by credential-usable reachable ingredients."""
manifests = store.get("manifests")
active_label = store.get("active_manifest")
if not isinstance(manifests, dict) or not isinstance(active_label, str):
return []
typed_manifests = cast("dict[object, object]", manifests)
pending = deque([active_label])
seen: set[str] = set()
chain: list[dict[str, Any]] = []
while pending:
label = pending.popleft()
if label in seen:
continue
seen.add(label)
manifest_value = typed_manifests.get(label)
if not isinstance(manifest_value, dict):
continue
manifest = cast("dict[str, Any]", manifest_value)
chain.append(manifest)
ingredients = manifest.get("ingredients")
if not isinstance(ingredients, list):
continue
for ingredient_value in cast("list[object]", ingredients):
if not isinstance(ingredient_value, dict):
continue
ingredient = cast("dict[object, object]", ingredient_value)
child = ingredient.get("active_manifest")
if (
isinstance(child, str)
and child not in seen
and not _store_has_invalid_credential(cast("dict[str, Any]", ingredient))
):
pending.append(child)
return chain
def _status_codes(store: dict[str, Any]) -> tuple[list[str], list[str], list[str]]:
"""Return ordered status codes for this store's active credential."""
successes: list[str] = []
informationals: list[str] = []
failures: list[str] = []
def extend(status_set: object) -> None:
if not isinstance(status_set, dict):
return
mapping = cast("dict[object, object]", status_set)
for key, target in (
("success", successes),
("informational", informationals),
("failure", failures),
):
values = mapping.get(key)
if not isinstance(values, list):
continue
for value in cast("list[object]", values):
if not isinstance(value, dict):
continue
code = cast("dict[object, object]", value).get("code")
if isinstance(code, str) and code:
target.append(code)
validation_results = store.get("validation_results")
if isinstance(validation_results, dict):
results = cast("dict[object, object]", validation_results)
extend(results.get("activeManifest"))
if not (successes or informationals or failures):
# Older readers expose only the flat non-success list. It still carries the
# exact codes needed to reject a mismatched binding or invalid credential.
values = store.get("validation_status")
if isinstance(values, list):
for value in cast("list[object]", values):
if not isinstance(value, dict):
continue
code = cast("dict[object, object]", value).get("code")
if isinstance(code, str) and code:
failures.append(code)
return successes, informationals, failures
def _store_has_invalid_credential(store: dict[str, Any]) -> bool:
"""Return whether this store's own active credential failed validation.
Deliberately routed through the same codes -> dimensions -> disqualified path the
report uses, rather than re-testing the raw codes here. When this walk classified
failures on its own, adding one rule meant editing it and
:func:`c2pa_info_has_invalid_credential` in lockstep, and the ingredient
reachability walk was free to drift away from the verdict it feeds.
"""
return c2pa_info_has_invalid_credential(_validation_fields(store))
def c2pa_info_has_invalid_credential(info: dict[str, Any]) -> bool:
"""Return whether parsed C2PA info reports a failed binding, signature, or credential."""
return (
info.get("c2pa_integrity") == "invalid"
or info.get("c2pa_signature") == "invalid"
or info.get("c2pa_signer_validity") == "invalid"
)
def c2pa_info_has_invismark(info: dict[str, Any]) -> bool:
"""Return whether parsed C2PA info declares Microsoft InvisMark."""
soft_bindings = info.get("soft_binding_vendors")
return isinstance(soft_bindings, list) and "Microsoft InvisMark" in soft_bindings
def c2pa_info_has_removal_hint(info: dict[str, Any]) -> bool:
"""Return whether a C2PA AI or watermark claim should keep removal fail-safe."""
return bool(info.get("ai_source_kind") or info.get("synthid_watermark") or c2pa_info_has_invismark(info))
def _has_suffix(codes: list[str], suffixes: tuple[str, ...]) -> bool:
return any(code.endswith(suffix) for code in codes for suffix in suffixes)
def _validation_fields(store: dict[str, Any]) -> dict[str, Any]:
successes, informationals, failures = _status_codes(store)
all_codes = list(dict.fromkeys([*successes, *informationals, *failures]))
disqualifying = [
code
for code in failures
if any(marker in code for marker in _CONTENT_BINDING_FAILURE_MARKERS)
or code in _SIGNATURE_FAILURES
or code in _CREDENTIAL_FAILURES
]
binding_failed = any(marker in code for code in disqualifying for marker in _CONTENT_BINDING_FAILURE_MARKERS)
binding_matched = _has_suffix(successes, _CONTENT_BINDING_MATCHES)
signature_failed = any(code in _SIGNATURE_FAILURES for code in disqualifying)
signature_validated = "claimSignature.validated" in successes
if binding_failed:
integrity = "invalid"
elif binding_matched:
integrity = "valid"
else:
integrity = "unknown"
if signature_failed:
signature = "invalid"
elif signature_validated:
signature = "valid"
else:
signature = "unknown"
if "signingCredential.trusted" in successes:
signer_trust = "trusted"
elif "signingCredential.untrusted" in failures:
signer_trust = "untrusted"
else:
signer_trust = "unknown"
if any(code in _CREDENTIAL_FAILURES for code in disqualifying):
signer_validity = "invalid"
elif "signingCredential.expired" in failures:
signer_validity = "expired"
elif "claimSignature.insideValidity" in successes:
signer_validity = "valid"
else:
signer_validity = "unknown"
state = store.get("validation_state")
return {
"c2pa_validation_source": "reader",
"c2pa_validation_state": state if isinstance(state, str) and state else "unknown",
"c2pa_integrity": integrity,
"c2pa_signature": signature,
"c2pa_signer_trust": signer_trust,
"c2pa_signer_validity": signer_validity,
"c2pa_validation_codes": all_codes,
# The subset of failures that actually drove a dimension to invalid. Callers
# rendering "why" must use this rather than re-classifying the full code list,
# or the reason shown and the verdict reached stop being the same rule.
"c2pa_failed_codes": disqualifying,
}
def _claim_generator_from_store(store: dict[str, Any]) -> str | None:
active = _active_manifest(store)
direct = active.get("claim_generator")
@@ -199,6 +413,154 @@ def _claim_generator_from_store(store: dict[str, Any]) -> str | None:
return None
def _structured_manifest_fields(store: dict[str, Any]) -> dict[str, Any]:
"""Extract provenance only from the active manifest and its ingredient graph."""
chain = _manifest_chain(store)
if not chain:
return {}
info: dict[str, Any] = {}
issuers: list[str] = []
tools: list[str] = []
actions: list[str] = []
source_types: list[str] = []
soft_binding_algorithms: list[str] = []
soft_binding_values: list[str] = []
# Raw signer/generator identity strings, used to scope SynthID evidence to
# the vendor that actually asserted the manifest. A vendor token appearing
# anywhere else in the chain (e.g. Microsoft Designer's "Azure OpenAI
# ImageGen" softwareAgent) is a service name, not that vendor's provenance.
identity_strings: list[str] = []
claim_generator_asserts_ai = False
def add_tool_matches(value: str, *, asserts_ai: bool = False) -> None:
nonlocal claim_generator_asserts_ai
matches = _ordered_matches(value.encode(), C2PA_AI_TOOLS)
tools.extend(matches)
if asserts_ai and matches:
claim_generator_asserts_ai = True
for manifest in chain:
signature_value = manifest.get("signature_info")
if isinstance(signature_value, dict):
signature = cast("dict[object, object]", signature_value)
for key in ("issuer", "common_name", "certificate_issuer"):
value = signature.get(key)
if isinstance(value, str):
issuers.extend(_ordered_matches(value.encode(), C2PA_ISSUERS))
identity_strings.append(value)
direct_generator = manifest.get("claim_generator")
if isinstance(direct_generator, str):
identity_strings.append(direct_generator)
add_tool_matches(direct_generator, asserts_ai=True)
candidates = manifest.get("claim_generator_info")
if isinstance(candidates, list):
for candidate_value in cast("list[object]", candidates):
if not isinstance(candidate_value, dict):
continue
name = cast("dict[object, object]", candidate_value).get("name")
if isinstance(name, str):
add_tool_matches(name, asserts_ai=True)
identity_strings.append(name)
assertions = manifest.get("assertions")
if not isinstance(assertions, list):
continue
for assertion_value in cast("list[object]", assertions):
if not isinstance(assertion_value, dict):
continue
assertion = cast("dict[object, object]", assertion_value)
label = assertion.get("label")
data = assertion.get("data")
if isinstance(label, str) and label.startswith("c2pa.soft-binding") and isinstance(data, dict):
soft_binding = cast("dict[object, object]", data)
algorithm = soft_binding.get("alg")
if isinstance(algorithm, str):
soft_binding_algorithms.append(algorithm)
blocks = soft_binding.get("blocks")
if isinstance(blocks, list):
for block_value in cast("list[object]", blocks):
if not isinstance(block_value, dict):
continue
value = cast("dict[object, object]", block_value).get("value")
if isinstance(value, str) and value.isprintable() and len(value) <= 256:
soft_binding_values.append(value)
if not (isinstance(label, str) and label.startswith("c2pa.actions") and isinstance(data, dict)):
continue
action_values = cast("dict[object, object]", data).get("actions")
if not isinstance(action_values, list):
continue
for action_value in cast("list[object]", action_values):
if not isinstance(action_value, dict):
continue
action = cast("dict[object, object]", action_value)
action_name = action.get("action")
if isinstance(action_name, str):
actions.append(C2PA_ACTIONS.get(action_name.encode(), action_name.removeprefix("c2pa.")))
source_type = action.get("digitalSourceType")
if isinstance(source_type, str):
source_types.append(source_type)
software_agent = action.get("softwareAgent")
if isinstance(software_agent, dict):
name = cast("dict[object, object]", software_agent).get("name")
if isinstance(name, str):
add_tool_matches(name)
issuers = list(dict.fromkeys(issuers))
tools = list(dict.fromkeys(tools))
actions = list(dict.fromkeys(actions))
if issuers:
info["issuer"] = ", ".join(issuers)
if tools:
info["ai_tool"] = ", ".join(tools)
if actions:
info["actions"] = ", ".join(actions)
if claim_generator_asserts_ai:
info["c2pa_identity_ai"] = True
generated = any(
"trainedAlgorithmicMedia" in value and "compositeWithTrainedAlgorithmicMedia" not in value
for value in source_types
)
enhanced = any("compositeWithTrainedAlgorithmicMedia" in value for value in source_types)
procedural = any("algorithmicMedia" in value for value in source_types)
if generated:
info.update(source_type="trainedAlgorithmicMedia (AI-generated)", ai_source_kind="generated")
elif enhanced:
info.update(source_type="compositeWithTrainedAlgorithmicMedia (AI-enhanced)", ai_source_kind="enhanced")
elif procedural:
info["source_type"] = "algorithmicMedia"
has_watermark_action = any(action.startswith("watermarked") for action in actions)
if has_watermark_action:
info["watermarked"] = True
if info.get("ai_source_kind") and not soft_binding_algorithms:
# Evidence scope: only the signer/generator identity strings above, never
# the whole chain - a vendor named inside another vendor's manifest (the
# Designer case) must not turn into that vendor's SynthID provenance. A
# manifest that names its own forensic soft-binding algorithm carries
# that vendor's mark and is excluded from the generic inference entirely.
identity_bytes = json.dumps(identity_strings, ensure_ascii=False).encode()
synthid = synthid_evidence_vendors_in(identity_bytes, has_watermark_action=has_watermark_action)
if synthid:
info["synthid_vendors"] = synthid
info["synthid_watermark"] = synthid_verdict(", ".join(synthid))
if soft_binding_algorithms:
soft_binding_algorithms = list(dict.fromkeys(soft_binding_algorithms))
algorithms = "\n".join(soft_binding_algorithms).encode()
soft_bindings = soft_binding_vendors_in(algorithms)
if soft_bindings:
info["soft_binding_vendors"] = soft_bindings
info["soft_binding"] = ", ".join(soft_bindings)
info["soft_binding_algorithm"] = ", ".join(soft_binding_algorithms)
if soft_binding_values:
info["soft_binding_value"] = ", ".join(dict.fromkeys(soft_binding_values))
return info
def synthid_verdict(vendors: str) -> str:
"""Describe why supported provenance establishes a SynthID watermark."""
return f"present according to {vendors} provenance"
@@ -262,12 +624,16 @@ def _populate_registry_fields(buffer: bytes, info: dict[str, Any]) -> bool:
if b"c2pa.watermarked" in buffer:
info["watermarked"] = True
synthid = synthid_evidence_vendors_in(buffer, has_watermark_action=info.get("watermarked", False))
soft_bindings = soft_binding_vendors_in(buffer)
synthid = (
[]
if soft_bindings
else synthid_evidence_vendors_in(buffer, has_watermark_action=info.get("watermarked", False))
)
if ai_source and synthid:
info["synthid_vendors"] = synthid
info["synthid_watermark"] = synthid_verdict(", ".join(synthid))
soft_bindings = soft_binding_vendors_in(buffer)
if soft_bindings:
info["soft_binding_vendors"] = soft_bindings
info["soft_binding"] = ", ".join(soft_bindings)
@@ -276,16 +642,29 @@ def _populate_registry_fields(buffer: bytes, info: dict[str, Any]) -> bool:
def _base_info(byte_count: int, *, fallback: bool = False) -> dict[str, Any]:
container = "C2PA manifest" if fallback else "C2PA manifest store"
return {
info: dict[str, Any] = {
"has_c2pa": True,
"type": "C2PA (Coalition for Content Provenance and Authenticity)",
"c2pa_manifest": f"{container} ({byte_count} bytes)",
}
if fallback:
info.update(
c2pa_validation_source="fallback",
c2pa_validation_state="unknown",
c2pa_integrity="unknown",
c2pa_signature="unknown",
c2pa_signer_trust="unknown",
c2pa_signer_validity="unknown",
c2pa_validation_codes=[],
c2pa_failed_codes=[],
)
return info
def _info_from_store(store: dict[str, Any], encoded: bytes) -> dict[str, Any]:
info = _base_info(len(encoded))
_populate_registry_fields(encoded, info)
info.update(_structured_manifest_fields(store))
info.update(_validation_fields(store))
generator = _claim_generator_from_store(store)
if generator is not None:
info["claim_generator"] = generator
+36 -15
View File
@@ -65,12 +65,12 @@ def _vendor(
# Order is product priority when a manifest mentions more than one organization.
C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
_vendor(b"Microsoft", "Microsoft", "Microsoft (Bing Image Creator / Designer)", "Microsoft"),
_vendor(b"Microsoft", "Microsoft", "Microsoft (Copilot / Designer)", "Microsoft"),
_vendor(b"Adobe", "Adobe", "Adobe Firefly", "Adobe"),
_vendor(
b"OpenAI",
"OpenAI",
"OpenAI (ChatGPT / gpt-image / DALL-E / Sora)",
"OpenAI (ChatGPT / GPT Image / DALL·E / Sora)",
"OpenAI",
synthid=True,
synthid_requires_watermark_action=True,
@@ -78,25 +78,35 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
_vendor(b"Google", "Google LLC", "Google (Gemini / Imagen)", "Google", synthid=True),
_vendor(b"Stability AI", "Stability AI", "Stability AI (Stable Image / DreamStudio)", "Stability AI"),
_vendor(b"Black Forest Labs", "Black Forest Labs", "Black Forest Labs (FLUX)", "Black Forest Labs"),
_vendor(b"volcengine", "ByteDance (Volcano Engine)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
_vendor(
b"volcengine",
"ByteDance (Volcano Engine)",
"ByteDance Volcano Engine",
"Volcano Engine",
),
_vendor(
"北京火山引擎科技有限公司",
"ByteDance (Volcano Engine)",
"ByteDance (Doubao / Jimeng / Volcano Engine)",
"ByteDance",
"ByteDance Volcano Engine",
"Volcano Engine",
),
_vendor(b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
_vendor(b"Byteplus", "BytePlus (ByteDance)", "BytePlus (ByteDance)", "BytePlus"),
_vendor(
b"Dreamina",
"ByteDance (Dreamina)",
"ByteDance (Doubao / Jimeng / Volcano Engine)",
"ByteDance",
"ByteDance Dreamina",
"Dreamina",
asserts_ai=True,
),
_vendor(b"Canva", "Canva", "Canva (Magic Media)", "Canva"),
_vendor(b"Eleven Labs", "ElevenLabs", "ElevenLabs", "ElevenLabs"),
_vendor(b"fal-ai", "fal.ai", "fal.ai", "fal.ai", asserts_ai=True),
_vendor(b"Bria", "Bria Artificial Intelligence", "Bria AI", "Bria", asserts_ai=True),
# Ideogram signs its downloads' Content Credentials with "Ideogram, Inc"; the
# issuer token is the org prefix (same substring-match class as "Bria" in
# "Bria Artificial Intelligence"). Found as an unmapped signer on 4 corpus
# uploads 2026-08-08 that identify reported as unknown-signer C2PA.
_vendor(b"Ideogram", "Ideogram", "Ideogram", "Ideogram", asserts_ai=True),
_vendor(b"Truepic", "Truepic", None, None),
)
@@ -107,6 +117,9 @@ C2PA_IDENTITY_AI_ORGS = frozenset(vendor.org for vendor in C2PA_AI_VENDORS if ve
# Keep this attribution beside the issuer registry so every C2PA consumer has one
# canonical source rather than maintaining a derived product map in identify.py.
C2PA_CLAIM_GENERATOR_PLATFORMS: tuple[tuple[str, str], ...] = (
("adobe_firefly", "Adobe Firefly"),
("firefly", "Adobe Firefly"),
("dreamina", "ByteDance Dreamina"),
("higgsfield ai", "Higgsfield AI"),
("topaz labs image api", "Topaz Labs"),
("tiktok ad creative toolbox", "TikTok Ad Creative Toolbox"),
@@ -118,25 +131,28 @@ C2PA_AI_TOOLS = {
("GPT-4o", "GPT-4o"),
("ChatGPT", "ChatGPT"),
("Sora", "Sora"),
("DALL-E", "DALL-E"),
("DALL", "DALL-E"),
("DALL-E", "DALL·E"),
("DALL", "DALL·E"),
("Imagen", "Imagen"),
("Firefly", "Firefly"),
("Dreamina", "Dreamina"),
)
}
C2PA_SOFT_BINDINGS = {
b"com.adobe.trustmark": "Adobe TrustMark",
b"com.adobe.icn": "Adobe (content fingerprint)",
b"com.digimarc": "Digimarc",
b"com.adobe.icn": "Adobe Image Comparator Network",
b"com.digimarc": "Digimarc Validate",
b"com.imatag.lamark": "Imatag (Lamark)",
b"ai.steg": "Steg.AI",
b"com.microsoft.invismark": "Microsoft InvisMark",
b"com.microsoft.wavmark": "Microsoft WavMark",
b"com.verimatrix": "Verimatrix",
b"com.nagra.nexguard": "NAGRA NexGuard",
b"com.aiwatermark": "AIWatermark (Meta PixelSeal)",
b"ai.trufo": "Trufo",
b"com.aiwatermark.pixelseal": "AIWatermark PixelSeal",
b"com.aiwatermark.videoseal": "AIWatermark VideoSeal",
b"com.aiwatermark.audioseal": "AIWatermark AudioSeal",
b"ai.trufo": "Trufo PawPrint",
b"app.overlai": "Overlai",
b"com.markany": "MarkAny",
b"com.mentaport": "Mentaport",
@@ -167,6 +183,11 @@ AI_GENERATOR_TOKENS = frozenset(
"dreamstudio",
"novelai",
"reve.com",
# Luma AI stamps PNG tEXt Source="Luma AI" / Comment="Generated by
# Luma AI's Uni-1 model (https://lumalabs.ai)"; the space-bearing token
# avoids matching incidental "luma" runs (luma/chroma key names etc.).
"luma ai",
"lumalabs",
"aphrodite ai",
"apple photos clean up",
"fal-ai",
@@ -183,7 +204,7 @@ C2PA_ACTIONS = {f"c2pa.{action}".encode(): action for action in _C2PA_ACTION_NAM
# what stops a newly registered TC260 vendor from silently falling back to ByteDance.
#
# What a TC260 label confirms when its producer is absent or unmapped. Historical
# behaviour, kept as the fallback so an unrecognized producer never regresses to no
# behavior, kept as the fallback so an unrecognized producer never regresses to no
# relaxation at all: ByteDance's two products are the ones the relaxed band was
# calibrated on (see _text_mark_engine._DEFAULT_PROVENANCE_NCC_FACTOR).
TC260_FALLBACK_VENDORS: frozenset[str] = frozenset({"doubao", "jimeng"})
+110 -55
View File
@@ -12,7 +12,10 @@ so pixel, video, and audio data is preserved bit-for-bit.
TC260-PG-20257A video metadata is nested instead:
``moov.udta.meta.keys/ilst``. Its detector seeks through those boxes without
reading media payloads, and its stripper blanks the validated key/value in
place so fast-start media offsets remain valid.
place so fast-start media offsets remain valid. Two serialization variants are
covered: the ISO form (``meta`` as a FullBox under ``udta``) and the QuickTime
form Doubao's iOS export writes (a bare ``meta`` box as a direct ``moov`` child,
no FullBox header).
This file intentionally avoids dependencies on format-specific libraries
(pillow-heif, pillow-jxl, pymp4) so it works on systems where they aren't
@@ -184,68 +187,116 @@ def _tc260_key_indices(
return found
# The box types a TC260-bearing ``meta`` box opens with or contains: ISO files
# have ``hdlr`` then ``keys``/``ilst``; QuickTime metadata lists have ``hdlr``
# and ``ilst`` only. Which payload offset (0 vs 4) yields such children is what
# disambiguates the QuickTime form (no FullBox header) from the ISO one.
_META_CHILD_TYPES = frozenset({b"hdlr", b"keys", b"ilst"})
def _meta_child_boxes(
stream: BinaryIO,
meta_payload: int,
meta_end: int,
) -> Iterator[tuple[int, int, bytes, int]]:
"""Yield the child boxes of one ``meta`` box in either serialized form.
ISO serializes ``meta`` as a FullBox, so its children start 4 bytes into
the payload; QuickTime writes a bare box, so they start at 0. Both real
forms open with a recognized child (``hdlr``), so the form is picked by
which offset's first box type is one of ``_META_CHILD_TYPES``; a wrong
probe reads garbage header bytes that match no known type.
"""
for offset in (0, 4):
boxes = iter_file_boxes(stream, meta_payload + offset, meta_end)
first = next(boxes, None)
if first is not None and first[2] in _META_CHILD_TYPES:
yield first
yield from boxes
return
def _iter_tc260_meta_boxes(
stream: BinaryIO,
moov_payload: int,
moov_end: int,
) -> Iterator[tuple[int, int]]:
"""Yield ``(payload, end)`` of every ``meta`` box that may hold a TC260 label.
The normative ISO placement is ``moov.udta.meta``; Doubao's iOS MOV export
instead stores the label in a QuickTime-form ``meta`` box that hangs
directly off ``moov``. Both are yielded so one consumer covers them.
"""
for _start, end, box_type, payload in iter_file_boxes(stream, moov_payload, moov_end):
if box_type == b"udta":
for _udta_start, udta_end, udta_type, udta_payload in iter_file_boxes(stream, payload, end):
if udta_type == b"meta":
yield udta_payload, udta_end
elif box_type == b"meta":
yield payload, end
def _tc260_aigc_regions(
stream: BinaryIO,
file_size: int,
) -> list[tuple[int, int, int, int, bytes]]:
) -> list[tuple[tuple[int, int] | None, int, int, bytes]]:
"""Locate validated native TC260 entries without reading media payloads.
Each tuple is ``(key_start, key_end, value_start, value_end, value)``.
Each tuple is ``(key_span, value_start, value_end, value)``; ``key_span`` is
the byte span of the ``AIGC`` key when the normative ``keys`` box maps the
item, or None for the QuickTime metadata-list form (``hdlr=mdir``) Doubao's
iOS export writes, where the JSON sits in a bare ``ilst`` data item with no
key name to blank.
"""
regions: list[tuple[int, int, int, int, bytes]] = []
regions: list[tuple[tuple[int, int] | None, int, int, bytes]] = []
for _moov_start, moov_end, moov_type, moov_payload in iter_file_boxes(stream, 0, file_size):
if moov_type != b"moov":
continue
for _udta_start, udta_end, udta_type, udta_payload in iter_file_boxes(
stream,
moov_payload,
moov_end,
):
if udta_type != b"udta":
continue
for _meta_start, meta_end, meta_type, meta_payload in iter_file_boxes(
for meta_payload, meta_end in _iter_tc260_meta_boxes(stream, moov_payload, moov_end):
keys: dict[int, tuple[int, int]] = {}
ilst_boxes: list[tuple[int, int]] = []
keyed = False
for _child_start, child_end, child_type, child_payload in _meta_child_boxes(
stream,
udta_payload,
udta_end,
meta_payload,
meta_end,
):
if meta_type != b"meta" or meta_payload + 4 > meta_end:
continue
keys: dict[int, tuple[int, int]] = {}
ilst_boxes: list[tuple[int, int]] = []
for _child_start, child_end, child_type, child_payload in iter_file_boxes(
if child_type == b"keys":
keyed = True
keys.update(_tc260_key_indices(stream, child_payload, child_end))
elif child_type == b"ilst":
ilst_boxes.append((child_payload, child_end))
if not ilst_boxes:
continue
for ilst_payload, ilst_end in ilst_boxes:
for _item_start, item_end, item_type, item_payload in iter_file_boxes(
stream,
meta_payload + 4,
meta_end,
ilst_payload,
ilst_end,
):
if child_type == b"keys":
keys.update(_tc260_key_indices(stream, child_payload, child_end))
elif child_type == b"ilst":
ilst_boxes.append((child_payload, child_end))
if not keys:
continue
for ilst_payload, ilst_end in ilst_boxes:
for _item_start, item_end, item_type, item_payload in iter_file_boxes(
index = int.from_bytes(item_type, "big")
key_span = keys.get(index)
if keyed and key_span is None:
# A keyed (ISO) meta box maps items through ``keys``;
# an unmapped index is not an AIGC entry, and reading
# its value would pull arbitrary metadata (e.g. cover
# art) through the JSON parser on every scan. Only the
# keyless QuickTime list falls through to content
# validation below.
continue
for _data_start, data_end, data_type, data_payload in iter_file_boxes(
stream,
ilst_payload,
ilst_end,
item_payload,
item_end,
):
index = int.from_bytes(item_type, "big")
key_span = keys.get(index)
if key_span is None:
value_start = data_payload + 8
value_size = data_end - value_start
if data_type != b"data" or value_size < 0 or value_size > MAX_TC260_VALUE_BYTES:
continue
for _data_start, data_end, data_type, data_payload in iter_file_boxes(
stream,
item_payload,
item_end,
):
value_start = data_payload + 8
value_size = data_end - value_start
if data_type != b"data" or value_size < 0 or value_size > MAX_TC260_VALUE_BYTES:
continue
stream.seek(value_start)
value = stream.read(value_size)
if len(value) == value_size and parse_tc260_aigc_json(value) is not None:
regions.append((*key_span, value_start, data_end, value))
stream.seek(value_start)
value = stream.read(value_size)
if len(value) == value_size and parse_tc260_aigc_json(value) is not None:
regions.append((key_span, value_start, data_end, value))
return regions
@@ -257,7 +308,7 @@ def tc260_aigc_payloads(path: str | Path) -> tuple[bytes, ...]:
return ()
stream.seek(0, 2)
file_size = stream.tell()
return tuple(region[4] for region in _tc260_aigc_regions(stream, file_size))
return tuple(region[3] for region in _tc260_aigc_regions(stream, file_size))
except OSError:
return ()
@@ -268,6 +319,8 @@ def blank_tc260_aigc_tags(data: bytes) -> tuple[bytes, int]:
Removing a nested ``ilst`` item would shift ``mdat`` in a fast-start MP4 and
invalidate its chunk offsets. Replacing the four-byte key with ``free`` and
the JSON value with spaces keeps every box size and media offset unchanged.
A keyless QuickTime metadata-list entry has no key name, so only its value
is blanked.
"""
if not is_isobmff(data):
return data, 0
@@ -276,9 +329,10 @@ def blank_tc260_aigc_tags(data: bytes) -> tuple[bytes, int]:
return data, 0
out = bytearray(data)
key_spans: set[tuple[int, int]] = set()
for key_start, key_end, value_start, value_end, _value in regions:
key_spans.add((key_start, key_end))
out[key_start:key_end] = b"free"
for key_span, value_start, value_end, _value in regions:
if key_span is not None:
key_spans.add(key_span)
out[key_span[0] : key_span[1]] = b"free"
out[value_start:value_end] = b" " * (value_end - value_start)
return bytes(out), len(key_spans)
@@ -456,7 +510,7 @@ def strip_isobmff_media_file(
max_scan=max_box_scan,
)
tc260_regions = _tc260_aigc_regions(stream, file_size) if targets is not None else []
tc260_key_spans = {(region[0], region[1]) for region in tc260_regions}
tc260_key_spans = {region[0] for region in tc260_regions if region[0] is not None}
with atomic_video_output(output_path) as temporary_path:
with source_path.open("rb") as source_stream, temporary_path.open("r+b") as temporary:
@@ -466,9 +520,10 @@ def strip_isobmff_media_file(
temporary.seek(box_start + 4)
temporary.write(b"free")
_overwrite_range(temporary, payload_start, box_end, byte=b"\x00")
for key_start, _key_end, value_start, value_end, _value in tc260_regions:
temporary.seek(key_start)
temporary.write(b"free")
for key_span, value_start, value_end, _value in tc260_regions:
if key_span is not None:
temporary.seek(key_span[0])
temporary.write(b"free")
_overwrite_range(temporary, value_start, value_end, byte=b" ")
temporary.flush()
os.fsync(temporary.fileno())
@@ -30,6 +30,8 @@ from remove_ai_watermarks._internal.watermark_profiles import resolve_seed
if TYPE_CHECKING:
from collections.abc import Callable
from remove_ai_watermarks._internal.text_restoration import VerifiedTextManifest
log = logging.getLogger(__name__)
QWEN_IMAGE_2512_MODEL_ID = "Qwen/Qwen-Image-2512"
@@ -973,6 +975,30 @@ class QwenZImagePipeline:
result = result.resize(image.size, Image.Resampling.LANCZOS)
return result.convert("RGB")
def _qwen_vae_roundtrip(self, image: Image.Image) -> Image.Image:
"""Reconstruct source pixels through the already loaded Qwen VAE."""
import torch
pipe, _controlnet_input_cls = self._load_qwen()
source_width, source_height = image.size
pad_width = (-source_width) % 8
pad_height = (-source_height) % 8
padded = image.convert("RGB")
if pad_width or pad_height:
padded = Image.fromarray(
np.pad(
np.asarray(padded),
((0, pad_height), (0, pad_width), (0, 0)),
mode="edge",
)
)
pipe.load_models_to_device(["vae"])
tensor = pipe.preprocess_image(padded).to(device=self.device, dtype=self.torch_dtype)
with torch.inference_mode():
latents = pipe.vae.encode(tensor)
decoded = pipe.vae.decode(latents)
return pipe.vae_output_to_image(decoded).crop((0, 0, source_width, source_height)).convert("RGB")
@staticmethod
def _detail_size(
crop_size: tuple[int, int],
@@ -1044,10 +1070,18 @@ class QwenZImagePipeline:
tile: bool = False,
tile_size: int = 1024,
tile_overlap: int = 128,
text_manifest: VerifiedTextManifest | None = None,
fidelity_anchor: bool = False,
) -> Image.Image:
"""Execute global regeneration and masked face repair."""
if text_manifest is not None and tile:
raise ValueError("Verified text restoration is not calibrated with tiled diffusion")
self._require_cuda()
seed = resolve_seed(seed)
donor = None
if text_manifest is not None:
self._progress("Reconstructing the verified text donor with the Qwen VAE...")
donor = self._qwen_vae_roundtrip(image)
global_strength = (
resolution_adaptive_denoise(image.width, image.height) if strength is None else float(strength)
)
@@ -1068,14 +1102,37 @@ class QwenZImagePipeline:
boxes = detect_faces(image)
if not boxes:
self._progress("No faces detected; keeping the Qwen global result.")
return global_result
masks = self._sam_masks(image, boxes)
face_strength = largest_face_denoise(boxes, image.size) * FACE_DENOISE_SCALE
return self._run_faces(
image,
global_result,
boxes,
masks,
strength=face_strength,
seed=seed,
result = global_result
else:
masks = self._sam_masks(image, boxes)
face_strength = largest_face_denoise(boxes, image.size) * FACE_DENOISE_SCALE
result = self._run_faces(
image,
global_result,
boxes,
masks,
strength=face_strength,
seed=seed,
)
if text_manifest is None:
return result
if donor is None:
raise RuntimeError("Verified text restoration requires a Qwen-VAE donor")
from remove_ai_watermarks._internal.text_restoration import (
blend_fidelity_anchor,
restore_verified_text,
)
# The anchor is OFF by default since 0.27.1: blending 15% of the Qwen-VAE
# donor ACROSS THE WHOLE FRAME returned detector-visible OpenAI SynthID on
# poster-scale manifests (official Content Provenance API, 2026-08-19:
# detected x6 with the anchor, clean x6 without it, base clean throughout;
# see docs/text-protection-research.md). ``fidelity_anchor=True`` keeps the
# 0.27.0 research behavior for reproduction.
if fidelity_anchor:
self._progress("Blending the Qwen-VAE fidelity anchor...")
anchor = blend_fidelity_anchor(result, donor)
else:
anchor = result
self._progress(f"Restoring {len(text_manifest.lines)} verified text lines...")
return restore_verified_text(image, anchor, donor, text_manifest.lines)
@@ -0,0 +1,446 @@
"""Opt-in restoration of verified text from a Qwen VAE reconstruction."""
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false
from __future__ import annotations
import hashlib
import json
import math
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
import cv2
import numpy as np
from PIL import Image
from remove_ai_watermarks._internal.schema import require_schema_version
if TYPE_CHECKING:
from collections.abc import Sequence
from pathlib import Path
from numpy.typing import NDArray
TEXT_MANIFEST_SCHEMA = 2
_SUPPORTED_TEXT_MANIFEST_SCHEMAS = frozenset({1, TEXT_MANIFEST_SCHEMA})
FIDELITY_BLEND_ALPHA = 0.15
GLYPH_FEATHER = 0.5
GLYPH_SIDE_PAD_RATIO = 0.12
GLYPH_SIDE_PAD_LIMIT = 1.0
@dataclass(frozen=True)
class VerifiedTextLine:
"""One operator-verified source region in source-pixel coordinates."""
box: tuple[int, int, int, int]
text: str | None = None
script: str | None = None
angle: float = 0.0
@dataclass(frozen=True)
class VerifiedTextManifest:
"""Verified text regions cryptographically bound to one decoded RGB source."""
source_pixel_sha256: str
width: int
height: int
lines: tuple[VerifiedTextLine, ...]
def source_pixel_sha256(image: Image.Image) -> str:
"""Hash decoded RGB geometry and bytes, independent of container metadata."""
rgb = image.convert("RGB")
digest = hashlib.sha256()
digest.update(rgb.width.to_bytes(8, "big"))
digest.update(rgb.height.to_bytes(8, "big"))
digest.update(rgb.tobytes())
return digest.hexdigest()
def load_verified_text_manifest(path: Path, source: Image.Image) -> VerifiedTextManifest:
"""Load and validate an operator-verified manifest for exactly ``source``."""
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise ValueError(f"Cannot read text manifest {path}: {exc}") from exc
if not isinstance(payload, dict):
raise ValueError("Text manifest must be a JSON object")
schema_version = require_schema_version(
payload.get("schema_version"),
contract="text manifest",
supported=_SUPPORTED_TEXT_MANIFEST_SCHEMAS,
)
if payload.get("verified") is not True:
raise ValueError("Text manifest must contain verified=true after operator verification")
rgb = source.convert("RGB")
width = _manifest_integer(payload, "width")
height = _manifest_integer(payload, "height")
if (width, height) != rgb.size:
raise ValueError(f"Text manifest dimensions {width}x{height} do not match source {rgb.width}x{rgb.height}")
expected_hash = payload.get("source_pixel_sha256")
if not isinstance(expected_hash, str) or len(expected_hash) != 64:
raise ValueError("Text manifest source_pixel_sha256 must be a 64-character SHA-256")
actual_hash = source_pixel_sha256(rgb)
if expected_hash.casefold() != actual_hash:
raise ValueError("Text manifest source_pixel_sha256 does not match the decoded source pixels")
raw_lines = payload.get("lines")
if not isinstance(raw_lines, list) or not raw_lines:
raise ValueError("Text manifest lines must be a non-empty list")
lines = tuple(_load_line(item, width, height, index, schema_version) for index, item in enumerate(raw_lines))
if list(lines) != sorted(lines, key=lambda line: (line.box[1], line.box[0])):
raise ValueError("Text manifest lines must be in top-to-bottom, left-to-right reading order")
return VerifiedTextManifest(actual_hash, width, height, lines)
def _manifest_integer(payload: dict[str, Any], key: str) -> int:
value = payload.get(key)
if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
raise ValueError(f"Text manifest {key} must be a positive integer")
return value
def _load_line(item: Any, width: int, height: int, index: int, schema_version: int) -> VerifiedTextLine:
if not isinstance(item, dict):
raise ValueError(f"Text manifest line {index} must be an object")
raw_box = item.get("box")
if (
not isinstance(raw_box, list)
or len(raw_box) != 4
or any(isinstance(value, bool) or not isinstance(value, int) for value in raw_box)
):
raise ValueError(f"Text manifest line {index} box must contain four integers")
box = tuple(raw_box)
x1, y1, x2, y2 = box
if not (0 <= x1 < x2 <= width and 0 <= y1 < y2 <= height):
raise ValueError(f"Text manifest line {index} box is outside the source dimensions")
text: str | None = None
script: str | None = None
if schema_version == 1:
text = item.get("text")
script = item.get("script")
if not isinstance(text, str) or not text.strip():
raise ValueError(f"Text manifest line {index} text must be non-empty")
if not isinstance(script, str) or not script.strip():
raise ValueError(f"Text manifest line {index} script must be non-empty")
angle_value = item.get("angle", 0.0)
if isinstance(angle_value, bool) or not isinstance(angle_value, int | float):
raise ValueError(f"Text manifest line {index} angle must be numeric")
angle = float(angle_value)
if not math.isfinite(angle) or abs(angle) > 30.0:
raise ValueError(f"Text manifest line {index} angle must be between -30 and 30 degrees")
return VerifiedTextLine(box, text, script, angle)
def blend_fidelity_anchor(clean: Image.Image, donor: Image.Image) -> Image.Image:
"""Blend 15% Qwen-VAE reconstruction into the oracle-clean pipeline output."""
clean_rgb = np.asarray(clean.convert("RGB"), dtype=np.float32)
donor_rgb = np.asarray(donor.convert("RGB"), dtype=np.float32)
if clean_rgb.shape != donor_rgb.shape:
raise ValueError("Clean result and Qwen-VAE donor dimensions must match")
blended = np.rint(clean_rgb * (1.0 - FIDELITY_BLEND_ALPHA) + donor_rgb * FIDELITY_BLEND_ALPHA)
return Image.fromarray(np.clip(blended, 0, 255).astype(np.uint8))
def restore_verified_text(
source: Image.Image,
candidate: Image.Image,
donor: Image.Image,
lines: tuple[VerifiedTextLine, ...],
) -> Image.Image:
"""Erase candidate glyphs, then composite verified Qwen-VAE glyph cores."""
from remove_ai_watermarks import region_eraser
if not region_eraser.lama_available():
raise RuntimeError(
"Verified text restoration requires LaMa. Install: pip install 'remove-ai-watermarks[text-restoration]'"
)
source_rgb = np.asarray(source.convert("RGB"))
candidate_rgb = np.asarray(candidate.convert("RGB"))
donor_rgb = np.asarray(donor.convert("RGB"))
if source_rgb.shape != candidate_rgb.shape or source_rgb.shape != donor_rgb.shape:
raise ValueError("Source, candidate, and Qwen-VAE donor dimensions must match")
source_masks = [source_silhouette_mask(source_rgb, line.box, line.angle) for line in lines]
for index, mask in enumerate(source_masks):
if not np.any(mask):
raise ValueError(f"Verified text line {index} produced no source glyph pixels")
candidate_masks = [source_silhouette_mask(candidate_rgb, line.box, line.angle) for line in lines]
erase_masks = []
for line, source_mask, candidate_mask in zip(lines, source_masks, candidate_masks, strict=True):
radius = 5 if line.box[3] - line.box[1] >= 48 else 3
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * radius + 1,) * 2)
erase_masks.append(cv2.dilate(np.maximum(source_mask, candidate_mask), kernel))
del candidate_masks
groups = group_text_lines(lines)
background = cv2.cvtColor(candidate_rgb, cv2.COLOR_RGB2BGR)
for group in groups:
background = region_eraser.erase_lama(background, np.maximum.reduce([erase_masks[index] for index in group]))
background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
residual_masks = [
residual_glyph_mask(background_rgb, mask, line.box) for line, mask in zip(lines, erase_masks, strict=True)
]
for group in groups:
residual = np.maximum.reduce([residual_masks[index] for index in group])
if np.any(residual):
background = region_eraser.erase_lama(background, residual)
del erase_masks, residual_masks
restored = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
restored = composite_fresh_text_edges(source_rgb, restored, lines, source_masks)
source_glyph_mask = np.maximum.reduce(source_masks)
restored = composite_reconstructed_glyphs(donor_rgb, restored, source_glyph_mask)
return Image.fromarray(restored)
def _glyph_crop_box(
box: tuple[int, int, int, int],
width: int,
height: int,
left_pad: int,
right_pad: int,
) -> tuple[int, int, int, int]:
"""Widen the detector box so glyph edges stay inside the silhouette crop.
Detector boxes can stop inside a leading flourish or trailing punctuation,
and Paddle line boxes sit 2-5 px above the true ink bottom on the poster
fixtures. The crop is otherwise the box itself, so those pixels never reach
the donor composite. Start 12% sideways, expand each side independently
while foreground reaches its boundary, and pad 8% up and 25% down.
"""
x1, y1, x2, y2 = box
line_h = max(1, y2 - y1)
pad_top = max(1, round(line_h * 0.08))
pad_bot = max(2, round(line_h * 0.25))
return (
max(0, x1 - left_pad),
max(0, y1 - pad_top),
min(width, x2 + right_pad),
min(height, y2 + pad_bot),
)
def _silhouette_crop_mask(
source_rgb: NDArray[Any],
crop_box: tuple[int, int, int, int],
angle: float,
) -> NDArray[Any]:
"""Threshold one candidate crop so its horizontal boundaries can be tested."""
height, width = source_rgb.shape[:2]
x1, y1, x2, y2 = crop_box
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
support = np.ones(gray.shape, dtype=np.uint8)
if angle:
box_width, box_height = x2 - x1, y2 - y1
theta = math.radians(abs(angle))
cosine, sine = math.cos(theta), math.sin(theta)
denominator = cosine * cosine - sine * sine
rect_width = (box_width * cosine - box_height * sine) / denominator
rect_height = (box_height * cosine - box_width * sine) / denominator
rotated = cv2.boxPoints(
((box_width / 2, box_height / 2), (max(1.0, rect_width * 0.92), max(1.0, rect_height * 0.62)), -angle)
)
support.fill(0)
cv2.fillConvexPoly(support, np.rint(rotated).astype(np.int32), 1)
values = gray[support > 0]
background_luma = float(np.median(values))
else:
ring_pad = max(6, min(20, (y2 - y1) // 4))
rx1, ry1, rx2, ry2 = _clip_box(crop_box, width, height, pad=ring_pad)
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
ring = np.ones(context.shape, dtype=bool)
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
background_luma = float(np.median(context[ring])) if ring.any() else float(np.median(gray))
values = gray.reshape(-1)
low, high = float(np.percentile(values, 2)), float(np.percentile(values, 98))
dark_contrast, light_contrast = background_luma - low, high - background_luma
threshold = max(16.0, min(56.0, max(light_contrast, dark_contrast) * 0.22))
if light_contrast > dark_contrast:
crop_mask = (gray.astype(np.float32) >= background_luma + threshold).astype(np.uint8) * 255
else:
crop_mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
crop_mask[support == 0] = 0
return crop_mask
def _anchored_components_reach_sides(
crop_mask: NDArray[Any],
anchor_box: tuple[int, int, int, int],
) -> tuple[bool, bool]:
"""Whether anchored foreground reaches the left and right crop sides."""
count, labels, stats, _centroids = cv2.connectedComponentsWithStats(crop_mask, 8)
x1, y1, x2, y2 = anchor_box
anchor_counts = np.bincount(labels[y1:y2, x1:x2].reshape(-1), minlength=count)
anchor_counts[0] = 0
def reaches(edge_labels: NDArray[Any]) -> bool:
return any(
label != 0 and anchor_counts[label] >= max(3, round(stats[label, cv2.CC_STAT_AREA] * 0.1))
for label in np.unique(edge_labels)
)
return reaches(labels[y1:y2, :2]), reaches(labels[y1:y2, -2:])
def source_silhouette_mask(
source_rgb: NDArray[Any],
box: tuple[int, int, int, int],
angle: float = 0.0,
) -> NDArray[Any]:
"""Recover a thresholded glyph shape without retaining source amplitudes."""
height, width = source_rgb.shape[:2]
box_x1, box_y1, box_x2, box_y2 = box
line_h = max(1, box_y2 - box_y1)
step = max(2, round(line_h * GLYPH_SIDE_PAD_RATIO))
limit = max(step, round(line_h * GLYPH_SIDE_PAD_LIMIT))
left_pad = right_pad = step
while True:
x1, y1, x2, y2 = _glyph_crop_box(box, width, height, left_pad, right_pad)
crop_mask = _silhouette_crop_mask(source_rgb, (x1, y1, x2, y2), angle)
core_y1 = max(0, box_y1 - y1)
core_y2 = min(y2 - y1, box_y2 - y1)
anchor_box = (max(0, box_x1 - x1), core_y1, min(x2 - x1, box_x2 - x1), core_y2)
can_expand_left = x1 > 0 and left_pad < limit
can_expand_right = x2 < width and right_pad < limit
left_anchored, right_anchored = (
_anchored_components_reach_sides(crop_mask, anchor_box)
if can_expand_left or can_expand_right
else (False, False)
)
left_touches = can_expand_left and left_anchored
right_touches = can_expand_right and right_anchored
if not left_touches and not right_touches:
break
if left_touches:
left_pad = min(limit, left_pad + step)
if right_touches:
right_pad = min(limit, right_pad + step)
result = np.zeros((height, width), dtype=np.uint8)
result[y1:y2, x1:x2] = crop_mask
return result
def residual_glyph_mask(
background_rgb: NDArray[Any],
original_mask: NDArray[Any],
box: tuple[int, int, int, int],
) -> NDArray[Any]:
"""Find glyph-like contrast left after the first inpaint pass."""
residual = _foreground_mask(background_rgb, box)
residual = cv2.bitwise_and(residual, original_mask)
return cv2.dilate(residual, np.ones((5, 5), np.uint8), iterations=1)
def composite_fresh_text_edges(
source_rgb: NDArray[Any],
background_rgb: NDArray[Any],
lines: tuple[VerifiedTextLine, ...],
masks: list[NDArray[Any]],
) -> NDArray[Any]:
"""Render fresh antialiased edges for source-derived glyph masks."""
restored = background_rgb
for line, mask in zip(lines, masks, strict=True):
color = _sample_text_color(source_rgb, mask, line.box)
restored = composite_fresh_silhouette(restored, mask, color)
return restored
def composite_reconstructed_glyphs(
donor_rgb: NDArray[Any],
background_rgb: NDArray[Any],
glyph_mask: NDArray[Any],
*,
feather: float = GLYPH_FEATHER,
) -> NDArray[Any]:
"""Composite an exact reconstructed core with a narrow donor edge."""
if donor_rgb.shape != background_rgb.shape or donor_rgb.shape[:2] != glyph_mask.shape:
raise ValueError("donor, background, and glyph mask dimensions must match")
blurred = cv2.GaussianBlur(glyph_mask, (0, 0), feather) if feather > 0 else glyph_mask
alpha = np.maximum(glyph_mask, blurred).astype(np.float32) / 255.0
combined = donor_rgb.astype(np.float32) * alpha[..., None] + background_rgb.astype(np.float32) * (
1.0 - alpha[..., None]
)
return np.clip(np.rint(combined), 0, 255).astype(np.uint8)
def composite_fresh_silhouette(
background_rgb: NDArray[Any],
glyph_mask: NDArray[Any],
color: tuple[int, int, int],
*,
feather: float = 0.35,
) -> NDArray[Any]:
"""Render a binary source shape with fresh color and antialiasing."""
if background_rgb.shape[:2] != glyph_mask.shape:
raise ValueError("background and glyph mask dimensions must match")
antialiased = cv2.GaussianBlur(glyph_mask, (0, 0), feather) if feather > 0 else glyph_mask
alpha = antialiased.astype(np.float32)[..., None] / 255.0
foreground = np.empty_like(background_rgb)
foreground[:, :] = color
combined = foreground.astype(np.float32) * alpha + background_rgb.astype(np.float32) * (1.0 - alpha)
return np.clip(combined, 0, 255).astype(np.uint8)
def _clip_box(box: tuple[int, int, int, int], width: int, height: int, pad: int = 0) -> tuple[int, int, int, int]:
x1, y1, x2, y2 = box
return max(0, x1 - pad), max(0, y1 - pad), min(width, x2 + pad), min(height, y2 + pad)
def _foreground_mask(source_rgb: NDArray[Any], box: tuple[int, int, int, int]) -> NDArray[Any]:
height, width = source_rgb.shape[:2]
line_height = box[3] - box[1]
x1, y1, x2, y2 = _clip_box(box, width, height, pad=max(6, int(line_height * 0.12)))
gray = cv2.cvtColor(source_rgb[y1:y2, x1:x2], cv2.COLOR_RGB2GRAY)
ring_pad = max(8, min(24, (y2 - y1) // 5))
rx1, ry1, rx2, ry2 = _clip_box((x1, y1, x2, y2), width, height, pad=ring_pad)
context = cv2.cvtColor(source_rgb[ry1:ry2, rx1:rx2], cv2.COLOR_RGB2GRAY)
ring = np.ones(context.shape, dtype=bool)
ring[y1 - ry1 : y2 - ry1, x1 - rx1 : x2 - rx1] = False
background_luma = float(np.median(context[ring])) if ring.any() else float(np.median(gray))
low, high = float(np.percentile(gray, 4)), float(np.percentile(gray, 96))
dark_contrast, light_contrast = background_luma - low, high - background_luma
threshold = max(24.0, min(72.0, max(light_contrast, dark_contrast) * 0.32))
if light_contrast > dark_contrast:
mask = (gray.astype(np.float32) >= background_luma + threshold).astype(np.uint8) * 255
else:
mask = (gray.astype(np.float32) <= background_luma - threshold).astype(np.uint8) * 255
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((2, 2), np.uint8))
dilation = 5 if line_height >= 48 else 3
mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * dilation + 1,) * 2))
result = np.zeros((height, width), dtype=np.uint8)
result[y1:y2, x1:x2] = mask
return result
def _sample_text_color(
source_rgb: NDArray[Any], mask: NDArray[Any], box: tuple[int, int, int, int]
) -> tuple[int, int, int]:
height, width = source_rgb.shape[:2]
x1, y1, x2, y2 = _clip_box(box, width, height, pad=2)
crop = source_rgb[y1:y2, x1:x2]
pixels = crop[mask[y1:y2, x1:x2] > 0]
luma = pixels.mean(axis=1)
background_luma = float(crop[[0, -1], :, :].reshape(-1, 3).mean(axis=1).mean())
selected = (
pixels[luma <= np.percentile(luma, 20)] if background_luma >= 128 else pixels[luma >= np.percentile(luma, 80)]
)
return tuple(int(value) for value in np.median(selected, axis=0))
def group_text_lines(lines: Sequence[VerifiedTextLine]) -> list[list[int]]:
"""Group nearby same-script lines for a shared LaMa erase pass."""
groups: list[list[int]] = []
for index, line in enumerate(lines):
if not groups:
groups.append([index])
continue
previous = lines[groups[-1][-1]]
gap = line.box[1] - previous.box[3]
if line.script != previous.script or gap > max(60, int((previous.box[3] - previous.box[1]) * 1.1)):
groups.append([index])
else:
groups[-1].append(index)
return groups
@@ -69,9 +69,50 @@ SDXL_ZIMAGE_OPENAI_STRENGTH = 0.15
SDXL_ZIMAGE_GEMINI_STRENGTH = 0.25
SDXL_ZIMAGE_UNKNOWN_STRENGTH = SDXL_ZIMAGE_GEMINI_STRENGTH
# qwen-zimage keeps its resolution curve for unknown content, but measured vendor
# cohorts bypass it. Google remained detectable through 0.24375; 0.25 cleared all
# three valid sources and 0.27 was separately repeated clean across them and three
# accounts, so the independently checked 0.27 candidate is the operating floor.
QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.27
# sdxl-zimage picks its strength from the VENDOR (unlike qwen-zimage, which derives it
# from image area). An unlisted or unknown vendor falls back to the Gemini value.
# The two OpenAI sources first cleared at 0.06225 and 0.0695. Add one full observed
# cross-source spread (0.00725) to the worst clean boundary: 0.0695 + 0.00725.
QWEN_ZIMAGE_OPENAI_STRENGTH = 0.07675
# Microsoft's public detector (https://ai.azure.com/nextgen/validate) returned
# Inconclusive rather than an API-level watermark-negative verdict. Three valid
# Paint sources first cleared at 0.04125,
# 0.055, and 0.095. Add one full observed cross-source spread to the worst clean
# boundary: 0.095 + (0.095 - 0.04125) = 0.14875, rounded up to 0.15. This is a
# measured corpus margin, not a universal InvisMark threshold.
QWEN_ZIMAGE_MICROSOFT_STRENGTH = 0.15
# Meta Muse Image stamps every output with Content Seal, but no provenance signal
# survives to route it: the outputs carry no C2PA, and their IPTC
# trainedAlgorithmicMedia companion tag is a standard code many platforms use, so
# it cannot key this cohort the way an issuer keys the others. The floor is
# therefore selected by an explicit --vendor meta override, never by detection.
# Derivation (oracle meta.ai/identification, 2026-08-26/27, corpus in
# data/contentseal/): five independent 2.56 MP generations bracketed at
# lighthouse (0.0525, 0.06], fox (0.03, 0.0375], night_city (0.03, 0.0375],
# mug <= 0.03, text <= 0.015. Worst clean boundary plus one full observed
# cross-source spread: 0.06 + (0.0525 - 0.015) = 0.0975, rounded up to 0.1.
# sdxl-zimage has no measured Meta floor; its vendor map stays without a meta
# entry so an explicit --vendor meta there falls to the unknown 0.25, which is
# above this floor and therefore conservative.
QWEN_ZIMAGE_META_STRENGTH = 0.1
_QWEN_ZIMAGE_FLAT_STRENGTH_BY_VENDOR: dict[str, float] = {
"google": QWEN_ZIMAGE_GOOGLE_STRENGTH,
"openai": QWEN_ZIMAGE_OPENAI_STRENGTH,
"microsoft": QWEN_ZIMAGE_MICROSOFT_STRENGTH,
"meta": QWEN_ZIMAGE_META_STRENGTH,
}
# sdxl-zimage always picks its strength from the vendor. qwen-zimage instead uses
# the vendor only for measured cohorts and image area for unknown content. An
# unlisted or unknown SDXL vendor falls back to the Gemini value.
_SDXL_ZIMAGE_STRENGTH_BY_VENDOR: dict[str, float] = {
"openai": SDXL_ZIMAGE_OPENAI_STRENGTH,
"google": SDXL_ZIMAGE_GEMINI_STRENGTH,
@@ -103,7 +144,10 @@ def resolve_adaptive_polish(adaptive_polish: bool | None, pipeline: str) -> bool
def strength_default_help() -> str:
"""Describe the live default policy without duplicating its values."""
return (
"profile-adaptive (qwen-zimage uses resolution-adaptive denoise; sdxl-zimage "
"profile-adaptive (qwen-zimage uses resolution-adaptive denoise, with a "
f"flat OpenAI {QWEN_ZIMAGE_OPENAI_STRENGTH} / Google {QWEN_ZIMAGE_GOOGLE_STRENGTH} / "
f"Microsoft InvisMark {QWEN_ZIMAGE_MICROSOFT_STRENGTH} / Meta Content Seal "
f"{QWEN_ZIMAGE_META_STRENGTH} floors; sdxl-zimage "
f"uses OpenAI {SDXL_ZIMAGE_OPENAI_STRENGTH} / Google {SDXL_ZIMAGE_GEMINI_STRENGTH} / "
f"unknown {SDXL_ZIMAGE_UNKNOWN_STRENGTH}, from the C2PA issuer)"
)
@@ -118,10 +162,14 @@ def resolve_strength(
) -> float:
"""Resolve a user override or the calibrated policy for a profile and vendor.
Total by design. qwen-zimage picks its strength from image area rather than from
the vendor, so it needs ``size``; returning ``None`` for it instead would push that
Total by design. qwen-zimage picks its strength from image area rather than
from the vendor, so it needs ``size``; returning ``None`` for it instead would push that
branch onto every caller and move one of the two strength policies outside this
module. ``size`` is required for qwen-zimage without an explicit strength.
Measured vendor exceptions bypass the area curve: OpenAI and Microsoft use one
additional observed cross-source spread over their worst clean boundaries, while
Google uses a separately repeated cross-source candidate. See the constants'
comments for the exact derivations.
"""
if strength is not None:
return strength
@@ -129,21 +177,63 @@ def resolve_strength(
return _SDXL_ZIMAGE_STRENGTH_BY_VENDOR.get((vendor or "").casefold(), SDXL_ZIMAGE_UNKNOWN_STRENGTH)
if size is None:
raise ValueError("qwen-zimage resolves strength from image area, so size is required")
vendor_strength = _QWEN_ZIMAGE_FLAT_STRENGTH_BY_VENDOR.get((vendor or "").casefold())
if vendor_strength is not None:
return vendor_strength
from remove_ai_watermarks._internal.qwen_zimage_pipeline import resolution_adaptive_denoise
return resolution_adaptive_denoise(*size)
def vendor_for_strength(image_path: Path) -> Literal["openai", "google"] | None:
"""Select the strength cohort using the input's SynthID provenance evidence."""
def vendor_for_strength(image_path: Path) -> Literal["openai", "google", "microsoft", "meta"] | None:
"""Select the strength cohort from non-invalid pixel-watermark provenance.
OpenAI / Google / Microsoft come from their C2PA issuers. Meta is the
fallback cohort: Muse Image carries no C2PA at all, and its only readable
companion is the IPTC ``trainedAlgorithmicMedia`` XMP tag -- a standard code
other platforms also use. Attributing that tag to Meta is a measured bet,
not an identification: the other tag users in this project's model (ByteDance
products, X) ship no invisible pixel watermark this profile targets, so the
worst misroute spends the Meta floor (0.1) where the resolution curve would
have spent a similar amount, and Google/OpenAI files never reach this arm
because their C2PA matched first."""
try:
from remove_ai_watermarks._internal.c2pa import (
c2pa_info_has_invalid_credential,
c2pa_info_has_invismark,
extract_c2pa_info,
)
from remove_ai_watermarks.metadata import synthid_source
evidence = (synthid_source(image_path) or "").casefold()
info = extract_c2pa_info(image_path)
evidence = (synthid_source(image_path, c2pa_info=info) or "").casefold()
except Exception:
return None
if "google" in evidence:
return "google"
if "openai" in evidence:
return "openai"
if not c2pa_info_has_invalid_credential(info) and c2pa_info_has_invismark(info):
return "microsoft"
if _standalone_iptc_ai_tag(image_path):
return "meta"
return None
def _standalone_iptc_ai_tag(image_path: Path) -> bool:
"""True when the file carries an AI IPTC marker with no C2PA around it.
Mirrors identify's ``standalone_iptc`` condition (the tag is only
trustworthy as platform evidence when no manifest supersedes it) without
importing the heavy identify module: the shared chunk-aware
:func:`metadata.scan_head` window -- Muse WebP outputs place their XMP
packet in a tail chunk up to hundreds of KB past a plain head read, which
is exactly what scan_head's extensions exist to catch.
"""
try:
from remove_ai_watermarks.metadata import IPTC_AI_MARKERS, c2pa_marker_in, scan_head
scan = scan_head(image_path)
except Exception:
return False
return any(marker in scan for marker in IPTC_AI_MARKERS) and not c2pa_marker_in(scan)
@@ -26,6 +26,8 @@ if TYPE_CHECKING:
from collections.abc import Callable
from pathlib import Path
from remove_ai_watermarks._internal.text_restoration import VerifiedTextManifest
logger = logging.getLogger(__name__)
try:
@@ -187,6 +189,8 @@ class WatermarkRemover:
tile: bool = False,
tile_size: int = 1024,
tile_overlap: int = 128,
text_manifest: VerifiedTextManifest | None = None,
fidelity_anchor: bool = False,
) -> Path:
"""Regenerate image pixels and write the result without AI metadata.
@@ -203,6 +207,10 @@ class WatermarkRemover:
resolved_strength = resolve_strength(strength, vendor, self.model_profile, size=source.size)
if not 0.0 <= resolved_strength <= 1.0:
raise ValueError(f"Strength must be between 0.0 and 1.0, got {resolved_strength}")
if text_manifest is not None and self.model_profile == SDXL_ZIMAGE_PROFILE:
raise ValueError("Verified text restoration is supported only by the qwen-zimage profile")
if text_manifest is not None and tile:
raise ValueError("Verified text restoration is not calibrated with tiled diffusion")
result = self._load_qwen_zimage_pipeline().run(
source,
@@ -211,6 +219,8 @@ class WatermarkRemover:
tile=tile,
tile_size=tile_size,
tile_overlap=tile_overlap,
text_manifest=text_manifest,
fidelity_anchor=fidelity_anchor,
)
self._write_output(result, destination)
return destination
+22 -13
View File
@@ -52,8 +52,8 @@ _MIN_DETECT_SHORT_SIDE = 200
# This used to be ONE shared 0.7 for every text mark. Measured 2026-07-18 on the
# `auto` path (the default -- no flag, driven by TC260 metadata), it turned out to
# mean two completely different things per mark. Blind hand-label of the ADDITIONS
# (accepted with provenance, rejected without) over a labelled TC260 evaluation set,
# two-sided control (labeller sensitivity 100%/96%, specificity 100%/100%):
# (accepted with provenance, rejected without) over a labeled TC260 evaluation set,
# two-sided control (labeler sensitivity 100%/96%, specificity 100%/100%):
#
# mark band precision 95% CI n
# doubao whole arm 76% 61-87% 42
@@ -89,8 +89,8 @@ class TextMarkConfig:
name: str # short label for log lines (e.g. "Doubao")
asset_name: str # bundled alpha PNG under assets/ (e.g. "doubao_alpha.png")
corner: Literal[
"br", "bl", "tl", "bc"
] # bottom-right (Doubao/Jimeng), bottom-left (Samsung), top-left (RunningHub), bottom-center (LibLibAI)
"br", "bl", "tl", "tr", "bc"
] # br (Doubao/Jimeng), bl (Samsung), tl (RunningHub), tr (Microsoft), bc (LiblibAI)
margin_floor: int # min margin in px for locate (4 for br marks, 2 for Samsung)
# locate geometry (fraction of scale_base -- see scale_base())
width_frac: float
@@ -131,7 +131,7 @@ class TextMarkConfig:
template_blur: float = 0.0
# Which image dimension the mark's size and margins scale with. VENDOR-SPECIFIC,
# measured, not assumed -- see TextMarkEngine.scale_base. "short" = min(h, w), "width" = w.
scale_basis: Literal["short", "width"] = "width"
scale_basis: Literal["short", "width", "long"] = "width"
# Scale rungs ``_ladder_best`` sweeps (the detection comb). PER-MARK: a vendor
# whose stamp sizes do not land on the shared 3-rung comb carries its own ladder
# (measured for 千问, whose marks sit in two size modes ~1.6x apart -- one fraction
@@ -309,7 +309,7 @@ class TextMarkEngine:
provenance relaxation it stopped trying because many Jimeng false additions
were actually Doubao marks.
Measured separability on hand-labelled examples, scoring BOTH templates
Measured separability on hand-labeled examples, scoring BOTH templates
against the same glyph blob:
feature separability (0.5 = useless, 1.0 = perfect)
@@ -480,10 +480,19 @@ class TextMarkEngine:
China's GB 45438-2025 clause 5.2(e) mandates glyph height >= 5% of "the
shortest side" for CN marks, which is why a short-side basis is the natural
prior -- but Jimeng's measured behaviour overrides the prior, and measurement
prior -- but Jimeng's measured behavior overrides the prior, and measurement
wins over the standard's wording.
"long" (max of the two sides) is the Microsoft badge's measured basis: the
pill tracks the RENDER dimension, so on a 1024x1536 portrait it scales with
the 1536 (a width basis undersized the template by the aspect ratio and the
portrait carriers fell to 0.15-0.32 NCC; measured 2026-08-27).
"""
return min(image.shape[:2]) if self.config.scale_basis == "short" else image.shape[1]
if self.config.scale_basis == "short":
return min(image.shape[:2])
if self.config.scale_basis == "long":
return max(image.shape[:2])
return image.shape[1]
def locate(self, image: NDArray[Any]) -> TextMarkLocation:
"""Anchor the watermark box in the configured corner, scaled by ``scale_basis``.
@@ -499,14 +508,14 @@ class TextMarkEngine:
wm_h = max(16, int(base * c.height_frac))
margin_x = max(c.margin_floor, int(base * c.margin_x_frac))
margin_b = max(c.margin_floor, int(base * c.margin_bottom_frac))
if c.corner == "br":
if c.corner == "br" or c.corner == "tr":
x = max(0, w - margin_x - wm_w)
elif c.corner == "bc": # bottom-center: horizontally centered, margin_x unused
x = max(0, (w - wm_w) // 2)
else:
x = min(margin_x, max(0, w - wm_w))
# "tl" anchors at the top instead: margin_bottom_frac is then the TOP margin.
y = min(margin_b, max(0, h - wm_h)) if c.corner == "tl" else max(0, h - margin_b - wm_h)
# "tl"/"tr" anchor at the top instead: margin_bottom_frac is then the TOP margin.
y = min(margin_b, max(0, h - wm_h)) if c.corner in ("tl", "tr") else max(0, h - margin_b - wm_h)
wm_w = min(wm_w, w - x)
wm_h = min(wm_h, h - y)
return TextMarkLocation(x=x, y=y, w=wm_w, h=wm_h)
@@ -695,7 +704,7 @@ class TextMarkEngine:
OVERRIDABLE, and the override contract is specifically the DETECTOR'S MATCH BOX:
a mark whose removable footprint reaches beyond what the NCC localizes -- Baidu's
flat white tag right of the text run, LibLibAI's triangle logo left of the
flat white tag right of the text run, LiblibAI's triangle logo left of the
wordmark -- supplies its own extension here and inherits the rest of the
footprint path. The blob-bbox branch never routes through an override.
"""
@@ -771,7 +780,7 @@ class TextMarkEngine:
"""Footprint policy for a mark whose fill must be bounded by the DETECTOR's match
box and never by the binary glyph blob.
Baidu's white tag has a flat interior a top-hat cannot answer, and LibLibAI's
Baidu's white tag has a flat interior a top-hat cannot answer, and LiblibAI's
blob bleeds up into background structure; in both cases the blob bbox is
measurably wrong and the NCC match box is right. ``force`` takes priority here,
unlike the default policy: a ``--no-detect`` caller named the mark, so the whole
+19 -8
View File
@@ -73,7 +73,7 @@ def _tc260_vendors(path: Path) -> frozenset[str]:
An absent, unreadable or unmapped producer falls back to the historical pair rather
than to nothing: the caller has already established that the AIGC signal fired, so
the image IS China-AIGC labelled, and dropping to no relaxation would lose the
the image IS China-AIGC labeled, and dropping to no relaxation would lose the
detections the fallback recovers today. The re-read is deliberately isolated -- a
failure here must narrow the answer, never discard the rest of the provenance.
"""
@@ -156,9 +156,10 @@ def remove_visible(
) -> tuple[NDArray[Any], list[str]]:
"""Remove every detected known visible AI mark through localize then fill.
The registry currently covers the Gemini sparkle; Doubao, Jimeng, Qwen, Kling,
Yuanbao, Samsung, RunningHub, Baidu, and LibLibAI text marks; and the Jimeng
pill. Returns ``(result_bgr, [labels removed])``.
The registry currently covers the Gemini visible watermark; Doubao, Jimeng,
Qwen, Kling AI, Yuanbao, Samsung, RunningHub, Baidu, and LiblibAI text marks;
one Microsoft top-right AI-badge variant; and the Jimeng pill. Returns
``(result_bgr, [labels removed])``.
``source`` is a file path OR a BGR ndarray. For a PATH, metadata provenance is read
automatically (so ``sensitivity="auto"`` recovers a moved/faint mark whenever the
@@ -184,7 +185,7 @@ def remove_visible(
from remove_ai_watermarks import watermark_registry
# Reject a removed sensitivity loudly; `Sensitivity` is a Literal and not enforced
# at runtime, so a 0.15 caller would otherwise get `auto` behaviour in silence.
# at runtime, so a 0.15 caller would otherwise get `auto` behavior in silence.
watermark_registry.validate_sensitivity(sensitivity)
loaded = _load_visible_input(source)
result, removed = watermark_registry.remove_auto_marks(
@@ -232,6 +233,7 @@ class InvisibleOptions:
strength: float | None = None
pipeline: str = "qwen-zimage"
vendor: str | None = None
seed: int | None = None
hf_token: str | None = None
humanize: float = 0.0
@@ -243,6 +245,8 @@ class InvisibleOptions:
tile: bool = False
tile_size: int = 1024
tile_overlap: int = 128
text_manifest: Path | None = None
fidelity_anchor: bool = False
# What the invisible stage did. "unavailable" is the one outcome the caller must
@@ -329,12 +333,13 @@ class _SourceEvidence:
if evidence is None:
return True
try:
from remove_ai_watermarks._internal.c2pa import c2pa_info_has_removal_hint
from remove_ai_watermarks.identify import identify_from_evidence
report = identify_from_evidence(evidence, image_path=self._path, check_invisible=True)
except Exception:
return True
return bool(report.ai_from_metadata)
return report.ai_from_metadata or c2pa_info_has_removal_hint(evidence.c2pa_info)
def _provenance_from_report(report: Any, path: Path) -> frozenset[str]:
@@ -484,13 +489,17 @@ def _run_invisible(
if not is_available():
say("invisible", "unavailable")
return "unavailable"
if not (force or evidence.has_invisible_target()):
if not (force or opts.vendor is not None or evidence.has_invisible_target()):
say("invisible", "no-signal")
return "no-signal"
from remove_ai_watermarks._internal.watermark_profiles import resolve_strength, vendor_for_strength
vendor = vendor_for_strength(vendor_source)
# An explicit vendor override wins over detection and implies the scrub runs:
# naming the cohort (e.g. "meta" for Muse Image Content Seal, which carries no
# provenance to detect) asserts the pixel watermark is present, so the no-signal
# gate must not skip it.
vendor = opts.vendor or vendor_for_strength(vendor_source)
# Report the strength the engine will actually execute, resolved the same way it
# resolves it, so the reported value cannot drift from the executed one.
with suppress(Exception):
@@ -522,6 +531,8 @@ def _run_invisible(
tile=opts.tile,
tile_size=opts.tile_size,
tile_overlap=opts.tile_overlap,
text_manifest=opts.text_manifest,
fidelity_anchor=opts.fidelity_anchor,
)
say("invisible", "removed")
return "removed"
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.4 KiB

+133 -24
View File
@@ -274,6 +274,32 @@ _strength_option = click.option(
default=None,
help=f"Denoising strength (0.0-1.0). Default: {strength_default_help()}.",
)
# Explicit strength-cohort override. Auto-detection reads the C2PA issuer, so it
# covers OpenAI / Google / Microsoft; Meta Content Seal has no provenance signal
# (no C2PA; the IPTC tag is a standard code), and an unknown or stripped manifest
# also leaves the resolution-adaptive curve in charge -- this flag is the way to
# name the cohort when the user knows what the file does not say.
_vendor_option = click.option(
"--vendor",
type=click.Choice(["auto", "openai", "google", "microsoft", "meta"]),
default="auto",
help=(
"Strength cohort for the invisible-removal default, and it implies the scrub "
"runs even without a local signal: naming the cohort asserts the pixel "
"watermark is present. auto: derive from C2PA provenance, else "
"resolution-adaptive. Set explicitly when the source is known but unreadable "
"(e.g. meta for Muse Image Content Seal, which never carries C2PA)."
),
)
def _explicit_vendor(vendor: str | None) -> str | None:
"""Normalize --vendor's ``auto`` default to None for the engine/API seam.
One helper so the three diffusion commands cannot drift on the spelling."""
return None if vendor in (None, "auto") else vendor
_seed_option = click.option(
"--seed",
type=int,
@@ -281,7 +307,7 @@ _seed_option = click.option(
help="Random seed for reproducibility. Default 0: both profiles are certified "
"at a fixed seed, because SynthID removal near the strength floor is seed-dependent.",
)
_hf_token_option = click.option("--hf-token", type=str, default=None, help="HuggingFace API token.")
_hf_token_option = click.option("--hf-token", type=str, default=None, help="Hugging Face API token.")
_humanize_option = click.option(
"--humanize", type=float, default=0.0, help="Analog Humanizer film grain intensity (0 = off, typical: 2.0-6.0)."
)
@@ -311,6 +337,26 @@ _cpu_offload_option = click.option(
),
)
_text_manifest_option = click.option(
"--text-manifest",
type=click.Path(exists=True, dir_okay=False, path_type=Path),
default=None,
help=(
"Experimental verified-text restoration manifest. Requires qwen-zimage, "
"the text-restoration extra, native untiled geometry, and no postprocessing."
),
)
_fidelity_anchor_option = click.option(
"--fidelity-anchor/--no-fidelity-anchor",
default=False,
help=(
"With --text-manifest: blend 15% of the Qwen-VAE donor across the whole "
"frame. Off by default - the global blend was measured to return "
"detector-visible OpenAI SynthID on poster-scale manifests."
),
)
_visible_backend_option = click.option(
"--backend",
@@ -495,7 +541,7 @@ def _should_skip_invisible_scrub(force: bool, image_path: Path) -> bool:
@click.option("-v", "--verbose", is_flag=True, help="Enable verbose logging.")
@click.pass_context
def main(ctx: click.Context, verbose: bool) -> None:
"""Remove visible and invisible AI watermarks from images, plus provenance metadata from video."""
"""Remove visible and invisible AI watermarks, plus metadata provenance marks, from images and video."""
from dotenv import load_dotenv
load_dotenv() # Load .env (e.g. HF_TOKEN)
@@ -776,6 +822,7 @@ def cmd_erase(
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
@_output_option
@_strength_option
@_vendor_option
@_pipeline_option
@_seed_option
@_hf_token_option
@@ -787,12 +834,15 @@ def cmd_erase(
@_tile_options
@_force_option
@_cpu_offload_option
@_text_manifest_option
@_fidelity_anchor_option
@click.pass_context
def cmd_invisible(
ctx: click.Context,
source: Path,
output: Path | None,
strength: float | None,
vendor: str | None,
pipeline: str,
seed: int | None,
hf_token: str | None,
@@ -806,8 +856,10 @@ def cmd_invisible(
tile_overlap: int,
force: bool,
cpu_offload: bool,
text_manifest: Path | None,
fidelity_anchor: bool,
) -> None:
"""Remove invisible AI watermarks (SynthID, StableSignature, TreeRing).
"""Attempt to disrupt invisible AI watermarks through pixel regeneration.
Regenerates the pixels with the two-stage diffusion profile. CUDA-only:
pip install 'remove-ai-watermarks[qwen-zimage]'
@@ -827,11 +879,17 @@ def cmd_invisible(
if output is None:
output = source.with_stem(source.stem + "_clean")
# An explicit --vendor wins over detection (see the option help) and implies the
# scrub runs: naming the cohort asserts the pixel watermark is present, so the
# no-signal gate must not skip it. Resolved BEFORE the gate for the same reason.
resolved_vendor = _explicit_vendor(vendor)
# Gate BEFORE building the engine: skip the destructive regeneration when no
# invisible AI watermark is locally detectable (it would only degrade a clean
# image -- dominant paid score-0 cause), so the common skip path pays nothing for
# engine construction. A skip never claims the image is clean; --force overrides.
if _should_skip_invisible_scrub(force, source):
# engine construction. A skip never claims the image is clean; --force and an
# explicit --vendor override.
if _should_skip_invisible_scrub(force or resolved_vendor is not None, source):
_no_invisible_signal_exit(source)
def progress_cb(msg: str) -> None:
@@ -846,27 +904,40 @@ def cmd_invisible(
)
# Detect the SynthID vendor from the ORIGINAL (before processing strips C2PA) so the
# displayed and executed strength agree on the vendor-adaptive default.
vendor = vendor_for_strength(source)
# displayed and executed strength agree on the vendor-adaptive default. An explicit
# --vendor override wins over detection: it names a cohort the file cannot prove
# (Meta Content Seal never carries C2PA; a stripped manifest proves nothing).
detected_vendor = vendor_for_strength(source) if resolved_vendor is None else None
vendor_label = resolved_vendor or detected_vendor
vendor_note = " (override)" if resolved_vendor else ""
console.print(f" Input: {source.name}")
console.print(f" Pipeline: {pipeline}")
console.print(f" Strength: {_resolved_strength_for_display(source, strength, vendor, pipeline)}")
console.print(
f" Strength: {_resolved_strength_for_display(source, strength, vendor_label, pipeline)}"
+ (f" [vendor: {vendor_label}{vendor_note}]" if vendor_label else "")
)
t0 = time.monotonic()
result_path = engine.remove_watermark(
image_path=source,
output_path=output,
strength=strength,
seed=seed,
humanize=humanize,
unsharp=unsharp,
adaptive_polish=adaptive_polish,
max_resolution=max_resolution,
vendor=vendor,
tile=tile,
tile_size=tile_size,
tile_overlap=tile_overlap,
)
try:
result_path = engine.remove_watermark(
image_path=source,
output_path=output,
strength=strength,
seed=seed,
humanize=humanize,
unsharp=unsharp,
adaptive_polish=adaptive_polish,
max_resolution=max_resolution,
vendor=vendor_label,
tile=tile,
tile_size=tile_size,
tile_overlap=tile_overlap,
text_manifest=text_manifest,
fidelity_anchor=fidelity_anchor,
)
except (OSError, RuntimeError, ValueError) as exc:
console.print(f" Error: {exc}")
raise SystemExit(1) from exc
elapsed = time.monotonic() - t0
size_kb = result_path.stat().st_size / 1024
@@ -874,10 +945,27 @@ def cmd_invisible(
# ── Metadata operations ──
def _print_metadata_not_a_clean_verdict() -> None:
"""Repeat the identify empty-scan limit on metadata check and strip success.
``metadata --check`` and ``metadata --remove`` answer a narrower question than
``identify``: they report embedded AI metadata only. A quiet result used to
stop at "No AI metadata found" / "AI metadata stripped", which readers treat
as a clean-image verdict. The pixel channel is unchanged, and this project has
no local SynthID decoder, so the command must say so in the same words
``identify`` already uses.
"""
console.print(
" This is not the same as 'clean': a pixel watermark such as SynthID cannot be\n"
" detected here once its metadata proxy is absent."
)
def _print_metadata_report(source: Path, has_ai: bool, metadata: dict[str, str]) -> None:
"""Render one metadata inspection result for the generic and video commands."""
if not has_ai:
console.print(f" No AI metadata found in {source.name}")
_print_metadata_not_a_clean_verdict()
return
console.print(f" Warning: AI metadata detected in {source.name}:")
@@ -913,8 +1001,8 @@ def cmd_metadata(
Strips EXIF AI tags, PNG text chunks, C2PA provenance manifests, and the
China TC260 AIGC label. Beyond images (PNG/JPEG/WebP/AVIF/HEIF/JXL) it also
strips provenance metadata from MP4/MOV/M4V/M4A containers and, via ffmpeg,
from WebM/MKV/AVI/FLV/MP3/WAV/FLAC/OGG. The coded image, audio, and video
data are left untouched.
from WebM/MKV/MKA/AVI/FLV/MP3/WAV/FLAC/OGG/OGA/Opus/AAC. The coded image,
audio, and video data are left untouched.
"""
from remove_ai_watermarks.metadata import get_ai_metadata, has_ai_metadata, strip_and_verify
@@ -944,6 +1032,7 @@ def cmd_metadata(
console.print(" the file could not be decoded, so it was copied through unchanged")
raise SystemExit(1)
console.print(f" AI metadata stripped -> {out}")
_print_metadata_not_a_clean_verdict()
# ── Video pipeline ──
@@ -1085,6 +1174,7 @@ def cmd_video_metadata(
console.print(f" still present: {', '.join(sorted(result.remaining))}")
raise SystemExit(1)
console.print(f" AI metadata stripped -> {result.output}")
_print_metadata_not_a_clean_verdict()
@cmd_video.command("invisible")
@@ -1356,6 +1446,13 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
verdict = "AI-generated (fully synthetic)"
console.print(f"\n Verdict: {verdict} (confidence: {report.confidence})")
console.print(f" Platform: {report.platform or 'undetermined'}")
if report.c2pa_validation is not None:
validation = report.c2pa_validation
console.print(
" C2PA validation: "
f"integrity={validation['integrity']}, signature={validation['signature']}, "
f"signer trust={validation['signer_trust']}, signer validity={validation['signer_validity']}"
)
if report.is_ai_generated is None:
console.print(
@@ -1392,6 +1489,7 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
@_visible_backend_option
@_visible_sensitivity_option
@_strength_option
@_vendor_option
@_pipeline_option
@_seed_option
@_hf_token_option
@@ -1403,6 +1501,8 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
@_tile_options
@_force_option
@_cpu_offload_option
@_text_manifest_option
@_fidelity_anchor_option
@click.pass_context
def cmd_all(
ctx: click.Context,
@@ -1411,6 +1511,7 @@ def cmd_all(
backend: str,
sensitivity: str,
strength: float | None,
vendor: str | None,
pipeline: str,
seed: int | None,
hf_token: str | None,
@@ -1424,6 +1525,8 @@ def cmd_all(
tile_overlap: int,
force: bool,
cpu_offload: bool,
text_manifest: Path | None,
fidelity_anchor: bool,
) -> None:
"""Remove ALL watermarks: visible + invisible + metadata.
@@ -1489,6 +1592,7 @@ def cmd_all(
sensitivity=_parse_sensitivity(sensitivity),
invisible=InvisibleOptions(
strength=strength,
vendor=_explicit_vendor(vendor),
pipeline=pipeline,
seed=seed,
hf_token=hf_token,
@@ -1501,6 +1605,8 @@ def cmd_all(
tile=tile,
tile_size=tile_size,
tile_overlap=tile_overlap,
text_manifest=text_manifest,
fidelity_anchor=fidelity_anchor,
),
force=force,
progress=progress,
@@ -1581,6 +1687,7 @@ def _batch_engine(mode: str, options: InvisibleOptions) -> object | None:
@_visible_backend_option
@_visible_sensitivity_option
@_humanize_option
@_vendor_option
@_pipeline_option
@_seed_option
@_hf_token_option
@@ -1598,6 +1705,7 @@ def cmd_batch(
mode: str,
output_dir: Path | None,
strength: float | None,
vendor: str | None,
pipeline: str,
seed: int | None,
hf_token: str | None,
@@ -1635,6 +1743,7 @@ def cmd_batch(
invisible_options = InvisibleOptions(
strength=strength,
vendor=_explicit_vendor(vendor),
pipeline=pipeline,
seed=seed,
hf_token=hf_token,
+106 -39
View File
@@ -1,7 +1,11 @@
"""DWT-DCT decoder compatible with invisible-watermark's ``dwtDct`` path.
Derived from ShieldMnt/invisible-watermark ``imwatermark/maxDct.py`` (MIT),
trimmed to the matrix path used by Stable Diffusion, SDXL, and FLUX.
trimmed to the matrix path used by Stable Diffusion, SDXL, and FLUX. The block
scan is vectorized rather than transcribed, so the file no longer reads line by
line against upstream; what it preserves is the output, bit for bit. See
[`docs/module-internals.md`](../../docs/module-internals.md) for the
measurements and for why a faster hand-rolled transform is not available.
Copyright (c) 2021 ShieldMnt
@@ -23,7 +27,39 @@ if TYPE_CHECKING:
from numpy.typing import NDArray
_DEFAULT_SCALES = (0, 36, 36)
_DEFAULT_BLOCK = 4
# Fixed by the format being decoded, and the 4-way fold chains in `_frame_bits`
# are written for it. It was a constructor parameter while the block scan was a
# generic Python loop; nothing ever passed another value, and a knob that now
# silently returns wrong bits is worse than no knob.
_BLOCK = 4
# Block-rows of the approximation band handled per strip. Not a tuned value:
# every height measured landed inside the others' noise. What matters is strips
# at all rather than a full-plane intermediate, not this number.
_STRIP = 16
def _approximation(rows: NDArray[Any]) -> NDArray[Any]:
"""One Haar pass along the last axis, approximation band only.
``pywt.dwt(x, "haar", axis=1)[0]`` computes and allocates the detail band as
well, and dispatches per row. Flattening lets one ``downcoef`` call do the
whole plane, and it is the same numbers in the same order **only while the
last axis is even**: Haar's filter is length 2, so an even row length makes
every pair fall inside its own row with no boundary extension. On an odd
width the pairs walk across row boundaries and the reshape below still
succeeds whenever the total is even -- wrong bits, no exception, and the
upstream-parity test is `skipif`-gated. Hence the explicit check.
The transform itself stays inside pywt however slow that looks. Its C
convolution contracts into an FMA that no numpy expression reproduces, and
the caller's threshold is ``peak % 36 > 18.0`` against values that are exact
multiples of 0.5, so a 1-ulp difference flips real bits.
"""
height, width = rows.shape
if width % 2:
raise RuntimeError(f"row length {width} is odd; the raveled Haar pass requires an even last axis")
return pywt.downcoef("a", rows.ravel(), "haar").reshape(height, width // 2)
class _DecodeMaxDct:
@@ -33,54 +69,85 @@ class _DecodeMaxDct:
self,
wm_lengths: tuple[int, ...],
scales: tuple[int, int, int] = _DEFAULT_SCALES,
block: int = _DEFAULT_BLOCK,
) -> None:
self._wm_lengths = wm_lengths
self._scales = scales
self._block = block
def decode(self, bgr: NDArray[Any]) -> dict[int, NDArray[Any]]:
row, col, _channels = bgr.shape
yuv = cv2.cvtColor(bgr, cv2.COLOR_BGR2YUV)
trimmed = yuv[: row // 4 * 4, : col // 4 * 4]
scores_by_length = {wm_len: ([0] * wm_len, [0] * wm_len) for wm_len in self._wm_lengths}
for channel in range(2):
if self._scales[channel] <= 0:
continue
ca1, _detail = pywt.dwt2(yuv[: row // 4 * 4, : col // 4 * 4, channel], "haar")
self._decode_frame(ca1, self._scales[channel], scores_by_length)
per_channel = [
self._plane_bits(trimmed, channel, self._scales[channel])
for channel in range(2)
if self._scales[channel] > 0
]
# Each channel restarts the bit index at 0, so the buckets come from a
# per-channel arange rather than one running counter.
index = np.concatenate([np.arange(bits.size) for bits in per_channel] or [np.zeros(0, dtype=np.int64)])
weights = np.concatenate(per_channel or [np.zeros(0)])
return {
wm_len: np.asarray(sums) * 255 > np.asarray(counts) * 127
for wm_len, (sums, counts) in scores_by_length.items()
}
decoded: dict[int, NDArray[Any]] = {}
for wm_len in self._wm_lengths:
bucket = index % wm_len
sums = np.bincount(bucket, weights=weights, minlength=wm_len)
counts = np.bincount(bucket, minlength=wm_len)
decoded[wm_len] = sums * 255 > counts * 127
return decoded
def _decode_frame(
self,
frame: NDArray[Any],
scale: int,
scores_by_length: dict[int, tuple[list[int], list[int]]],
) -> None:
row, col = frame.shape
bit_index = 0
for i in range(row // self._block):
for j in range(col // self._block):
block = frame[
i * self._block : i * self._block + self._block,
j * self._block : j * self._block + self._block,
]
inferred = self._infer_bit(block, scale)
for wm_len, (sums, counts) in scores_by_length.items():
bucket = bit_index % wm_len
sums[bucket] += inferred
counts[bucket] += 1
bit_index += 1
def _plane_bits(self, trimmed: NDArray[Any], channel: int, scale: int) -> NDArray[Any]:
"""Block bits for one color plane, a strip of block-rows at a time.
def _infer_bit(self, block: NDArray[Any], scale: int) -> int:
position = int(np.argmax(np.abs(block.flatten()[1:]))) + 1
i, j = position // self._block, position % self._block
value = abs(float(block[i][j]))
return int((value % scale) > 0.5 * scale)
``dwt2`` is ``dwtn``: it transforms along axis 0, then along axis 1 over
both halves, and three of the four bands it returns are discarded here.
Only the approximation is ever asked for, and transposing between the
two passes lets pywt walk a contiguous axis instead of a column -- that
access pattern, not the arithmetic saved, is where the time goes.
Strips mean no full-plane float64 intermediate is ever materialized, and
they are seam-free for the reason ``_approximation`` documents: output
row ``k`` reads input rows ``2k`` and ``2k + 1`` only. Strips start on
multiples of ``2 * _BLOCK``, so neither a pair nor a 4x4 block straddles
one.
"""
if trimmed.shape[0] == 0 or trimmed.shape[1] == 0:
# Reachable: a 1x65536 image clears the caller's area check and
# trims to an empty plane. Left to dwt2 so the exception stays the
# one this module has always raised -- returning empty bits here
# instead would silently turn a raise into an all-false verdict.
return pywt.dwt2(trimmed[:, :, channel], "haar")[0]
rows = trimmed.shape[0] // (2 * _BLOCK)
cols = trimmed.shape[1] // (2 * _BLOCK)
if rows == 0 or cols == 0:
return np.zeros(0, dtype=np.float64)
width = cols * 2 * _BLOCK
pieces: list[NDArray[Any]] = []
for start in range(0, rows, _STRIP):
stop = min(start + _STRIP, rows)
strip = trimmed[start * 2 * _BLOCK : stop * 2 * _BLOCK, :width]
columns = cv2.transpose(cv2.extractChannel(strip, channel))
band = _approximation(cv2.transpose(_approximation(columns)))
pieces.append(self._frame_bits(band, scale))
return np.concatenate(pieces)
def _frame_bits(self, frame: NDArray[Any], scale: int) -> NDArray[Any]:
"""One bit per 4x4 block, in row-major block order.
Upstream's per-block loop, said to numpy once instead of to the
interpreter ~135k times per image. Zeroing the DC term in the absolute
band says "ignore index 0" without materializing a ``(nblocks, 16)``
copy, and the 4-way ``np.maximum`` chains reduce over contiguous rows.
"""
rows = frame.shape[0] // _BLOCK
cols = frame.shape[1] // _BLOCK
band = np.abs(frame[: rows * _BLOCK, : cols * _BLOCK])
band[::_BLOCK, ::_BLOCK] = 0.0
folded = np.maximum(np.maximum(band[0::4], band[1::4]), np.maximum(band[2::4], band[3::4]))
folded = folded.reshape(rows * cols, _BLOCK)
peak = np.maximum(np.maximum(folded[:, 0], folded[:, 1]), np.maximum(folded[:, 2], folded[:, 3]))
return ((peak % scale) > 0.5 * scale).astype(np.float64)
def decode_dwt_dct(bgr: NDArray[Any], wm_len: int) -> NDArray[Any]:
+245 -47
View File
@@ -28,6 +28,9 @@ from typing import TYPE_CHECKING, Any, cast
from remove_ai_watermarks._internal.c2pa import (
c2pa_info_from_manifest_store,
c2pa_info_has_invalid_credential,
c2pa_info_has_invismark,
c2pa_info_has_removal_hint,
cbor_text_after,
extract_c2pa_info,
soft_binding_vendors_in,
@@ -114,13 +117,35 @@ _SYNTHID_CAVEAT = (
"SynthID presence comes from supported provenance here; the pixel watermark is not locally "
"decoded (proprietary decoder). Confirm via the Gemini app or openai.com/verify."
)
_C2PA_UNTRUSTED_CAVEAT = (
"The C2PA claim signature and asset binding validate, but no trust anchor list is configured here, "
"so the signer identity was never checked against one; treat the named platform as a signed claim, "
"not a verified identity."
)
_C2PA_EXPIRED_CAVEAT = (
"The C2PA signing certificate has expired and no trusted timestamp establishes when the claim was "
"signed. The binding to these bytes still holds; only the signing time is unproven."
)
_C2PA_UNVALIDATED_CAVEAT = (
"The C2PA marker was parsed without cryptographic validation; treat its origin and watermark "
"assertions as unverified claims."
)
_C2PA_INVALID_CAVEAT = (
"The embedded C2PA claim no longer validates against this asset. Its origin and watermark assertions "
"are retained only as removal hints, not as verified provenance."
)
_IPTC_ONLY_CAVEAT = "The IPTC 'Made with AI' tag flags AI provenance but does not identify the specific platform."
_CONTENT_SEAL_CAVEAT = (
"Meta Muse Image outputs carry the invisible Content Seal pixel watermark, which has no "
"local decoder; `invisible` removes it (auto when this tag is present, or `--vendor meta` "
"on stripped files) and meta.ai/identification verifies it."
)
_INVISIBLE_WM_CAVEAT = (
"The open invisible watermark is fragile: it does not survive JPEG re-encoding "
"or resizing, so it confirms origin only on a pristine (un-re-encoded) file."
)
_HF_JOB_CAVEAT = (
"The hf-job-id tag marks a HuggingFace-hosted job (commonly diffusion "
"The hf-job-id tag marks a Hugging Face-hosted job (commonly diffusion "
"generation) but names neither the model nor the content type, so it is a "
"medium-confidence signal, not proof the pixels are AI-generated."
)
@@ -131,6 +156,10 @@ _C2PA_CLOUD_CAVEAT = (
"It marks Content Credentials, not AI origin: the cloud manifest may describe a "
"human edit, and reading it needs a network fetch this tool does not make."
)
_SOFT_BINDING_CAVEAT = (
"Removing the embedded C2PA manifest does not remove its soft binding: the named "
"watermark or fingerprint may remain in the pixels and re-link the asset to provenance."
)
_SAMSUNG_GENAI_CAVEAT = (
"Samsung's genAIType marker shows a Galaxy AI editing tool (Generative Edit, "
"Sketch to Image, ...) touched the image; it is an undocumented proprietary "
@@ -374,6 +403,8 @@ def evidence_from_metadata_record(
"actions",
"synthid_watermark",
"soft_binding",
"soft_binding_algorithm",
"soft_binding_value",
):
if key in c2pa_info:
ai_metadata.setdefault(key, str(c2pa_info[key]))
@@ -414,7 +445,7 @@ def evidence_from_metadata_record(
if iptc_system:
ai_metadata.setdefault("ai_system", f"IPTC 2025.1 AI disclosure ({iptc_system})")
if hf_job:
ai_metadata.setdefault("huggingface_job", f"HuggingFace-hosted job ({hf_job})")
ai_metadata.setdefault("huggingface_job", f"Hugging Face-hosted job ({hf_job})")
if samsung is not None:
ai_metadata.setdefault("samsung_genai", f"Samsung Galaxy AI editing marker (genAIType={samsung})")
@@ -452,19 +483,27 @@ class ProvenanceReport:
# (C2PA AI issuer / SynthID provenance, IPTC, AIGC, local gen params, EXIF/xAI, or
# an open DWT-DCT decode) -- as opposed to a visible mark, provenance-only
# TrustMark, or a weak medium-confidence hint (hf-job, Samsung genAIType). This
# is exactly the set of signals an invisible/diffusion scrub targets: a
# visible-only or no-signal image has it False. Equivalent to
# ``confidence == "high"``;
# surfaced as a field so callers gate on intent, not on the string.
# is the set of signals an invisible/diffusion scrub targets: a visible-only
# or no-signal image has it False. Callers should gate on intent, not on the
# confidence string.
ai_from_metadata: bool = False
watermarks: list[str] = field(default_factory=list[str])
signals: list[Signal] = field(default_factory=list["Signal"])
caveats: list[str] = field(default_factory=list[str])
# Contradictions between independent provenance signals (e.g. two different
# AI vendors both claiming the image, or camera-capture credentials next to
# AI-generation markers). Non-empty means the provenance is internally
# inconsistent -- a strong tell of spoofed, transplanted, or laundered metadata.
# AI-generation markers), and credentials that failed validation. Non-empty means
# the provenance is internally inconsistent -- a strong tell of spoofed,
# transplanted, or laundered metadata. A failed credential sends
# ``is_ai_generated`` to None and lands here instead, so a consumer that reads
# only the verdict field turns a broken AI manifest into silence; see the
# ``integrity_clashes`` note in docs/python-api.md.
integrity_clashes: list[str] = field(default_factory=list[str])
# Orthogonal C2PA checks. A valid content binding does not make an untrusted
# signer identity trusted, and an expired credential does not by itself mean the
# signed bytes changed. Fallback parsing reports each dimension as unknown;
# None means no C2PA result exists.
c2pa_validation: dict[str, Any] | None = None
def to_dict(
self,
@@ -500,9 +539,41 @@ class ProvenanceReport:
],
"caveats": list(self.caveats),
"integrity_clashes": list(self.integrity_clashes),
"c2pa_validation": self.c2pa_validation,
}
def _c2pa_validation(info: dict[str, Any]) -> dict[str, Any] | None:
source = info.get("c2pa_validation_source")
if not isinstance(source, str):
return None
codes = info.get("c2pa_validation_codes")
return {
"source": source,
"state": str(info.get("c2pa_validation_state", "unknown")),
"integrity": str(info.get("c2pa_integrity", "unknown")),
"signature": str(info.get("c2pa_signature", "unknown")),
"signer_trust": str(info.get("c2pa_signer_trust", "unknown")),
"signer_validity": str(info.get("c2pa_signer_validity", "unknown")),
"codes": list(cast("list[object]", codes)) if isinstance(codes, list) else [],
}
def _c2pa_credential_level(info: dict[str, Any]) -> str:
"""Return invalid, verified, or unverified for provenance attribution.
``verified`` means the reader tied this manifest to these bytes: the hard binding
matched and the claim signature validated. Signer trust is deliberately NOT a
condition -- no trust anchors ship, so gating on it made this branch unreachable.
Read the trust-anchor paragraph in docs/module-internals.md before changing this.
"""
if c2pa_info_has_invalid_credential(info):
return "invalid"
if info.get("c2pa_integrity") == "valid" and info.get("c2pa_signature") == "valid":
return "verified"
return "unverified"
def extract_provenance_evidence(image_path: Path) -> ProvenanceEvidence:
"""Read all file-backed metadata needed by provenance verdict logic once."""
return ProvenanceEvidence(
@@ -720,6 +791,7 @@ _AI_VENDOR_TOKENS: tuple[tuple[str, str], ...] = (
("google", "Google"),
("firefly", "Adobe"),
("adobe", "Adobe"),
("copilot", "Microsoft"),
("bing", "Microsoft"),
("designer", "Microsoft"),
("microsoft", "Microsoft"),
@@ -746,6 +818,8 @@ _AI_VENDOR_TOKENS: tuple[tuple[str, str], ...] = (
("fal-ai", "fal.ai"),
("bria", "Bria"),
("apple photos clean up", "Apple"),
("luma ai", "Luma AI"),
("lumalabs", "Luma AI"),
)
@@ -879,7 +953,7 @@ def _visible_sparkle(image_path: Path, *, image: NDArray[Any] | None = None) ->
# metadata label); the per-engine detection thresholds live in the registry.
# Text mark -> the platform sentence this report prints when that mark is the strongest
# evidence, DERIVED from the registry rows so registering a mark is one edit. It was a
# hand-maintained copy, and that class of copy is how LibLibAI ended up registered but
# hand-maintained copy, and that class of copy is how LiblibAI ended up registered but
# missing from the pill veto. Insertion order is the registry's, which is what fixes the
# scan order below. The Gemini sparkle and the capture-less pill carry no platform of
# their own (`KnownMark.platform is None`) and are excluded here: the sparkle has its
@@ -1031,7 +1105,7 @@ def _collect_visible_signals(
sparkle_conf = _visible_sparkle(image_path, image=image)
if sparkle_conf is not None and sparkle_conf >= _SPARKLE_THRESHOLD:
signals.append(Signal("visible_sparkle", f"NCC confidence {sparkle_conf:.2f}", "medium"))
watermarks.append(f"Visible Gemini sparkle (confidence {sparkle_conf:.2f})")
watermarks.append(f"Google Gemini visible watermark (sparkle; confidence {sparkle_conf:.2f})")
if platform is None:
platform = "Google Gemini family (visible sparkle detected)"
@@ -1085,10 +1159,16 @@ def _identify_from_evidence(
# ── C2PA Content Credentials ────────────────────────────────────
has_c2pa = bool(info) or c2pa_marker_in(head)
c2pa_validation = _c2pa_validation(info)
c2pa_level = _c2pa_credential_level(info)
c2pa_usable = c2pa_level != "invalid"
# The reader already named which failures moved a dimension; re-deriving that here
# by substring made the displayed reason a second, looser rule than the verdict.
failed_c2pa_codes = [str(code) for code in cast("list[object]", info.get("c2pa_failed_codes", []))]
issuers = [info["issuer"]] if info.get("issuer") else _issuers_in(region)
# Full AI generation (trainedAlgorithmicMedia) vs an AI-enhanced real photo
# (compositeWithTrainedAlgorithmicMedia). The structured kind is parsed once in
# _internal.c2pa._populate_registry_fields (covers PNG + any container the c2pa-python
# _internal.c2pa._structured_manifest_fields (covers PNG + any container the c2pa-python
# reader handles); fall back to a raw head scan for the non-PNG raw-blob path
# where extract_c2pa_info returns {}. Full generation wins when both appear.
source_kind = _metadata_source_kind(info, head)
@@ -1098,8 +1178,11 @@ def _identify_from_evidence(
# Restricted to the ``asserts_ai`` vendors (distinctive brand strings), so it
# does not reopen the incidental-mention problem the common-word issuers have.
issuer_blob = " ".join(issuers)
c2pa_identity_ai = has_c2pa and any(org in issuer_blob for org in C2PA_IDENTITY_AI_ORGS)
c2pa_is_ai = source_kind is not None or c2pa_identity_ai
c2pa_identity_ai = has_c2pa and (
bool(info.get("c2pa_identity_ai")) or any(org in issuer_blob for org in C2PA_IDENTITY_AI_ORGS)
)
c2pa_claims_ai = source_kind is not None or c2pa_identity_ai
c2pa_is_ai = c2pa_usable and c2pa_claims_ai
# Generator string (for the signal detail): structured for PNG, CBOR-scanned
# for other containers. Best-effort -- some manifests key it as
# `claim_generator_info` (Pixel), so this can be None even when a device is
@@ -1119,20 +1202,43 @@ def _identify_from_evidence(
camera_label
or signer_label
or (_claim_generator_platform(generator) if c2pa_is_ai else None)
or (_claim_generator_platform(str(info.get("ai_tool"))) if c2pa_is_ai and info.get("ai_tool") else None)
or _attribute_platform(issuers, is_ai=c2pa_is_ai)
)
if has_c2pa
if has_c2pa and c2pa_usable
else None
)
if has_c2pa:
detail = ", ".join(filter(None, [", ".join(issuers), generator, info.get("source_type")]))
signals.append(Signal("c2pa", detail or "C2PA manifest present", "high"))
watermarks.append(f"C2PA Content Credentials ({', '.join(issuers) or 'unknown signer'})")
if c2pa_level == "invalid":
suffix = f"; {', '.join(failed_c2pa_codes)}" if failed_c2pa_codes else ""
signals.append(Signal("c2pa", f"C2PA manifest present, credential integrity invalid{suffix}", "medium"))
watermarks.append("C2PA Content Credentials (invalid asset binding or signature)")
caveats.append(_C2PA_INVALID_CAVEAT)
else:
signals.append(
Signal("c2pa", detail or "C2PA manifest present", "high" if c2pa_level == "verified" else "medium")
)
watermarks.append(f"C2PA Content Credentials ({', '.join(issuers) or 'unknown signer'})")
if c2pa_level == "verified":
# A missing trust bundle is not a signer that failed against one.
if info.get("c2pa_signer_trust") != "trusted":
caveats.append(_C2PA_UNTRUSTED_CAVEAT)
if info.get("c2pa_signer_validity") == "expired":
caveats.append(_C2PA_EXPIRED_CAVEAT)
else:
caveats.append(_C2PA_UNVALIDATED_CAVEAT)
# Record the AI-origin vendor for clash detection only when the source is
# actually AI -- classify the issuer attribution / generator, NOT the
# resolved `platform` (which may be a camera device token whose label,
# e.g. "Google Pixel", would mis-normalize to an AI vendor).
if c2pa_is_ai and (v := (_vendor_of(_attribute_platform(issuers, is_ai=True)) or _vendor_of(generator))):
if c2pa_is_ai and (
v := (
_vendor_of(_attribute_platform(issuers, is_ai=True))
or _vendor_of(generator)
or _vendor_of(str(info.get("ai_tool", "")))
)
):
ai_vendor_claims["c2pa"] = v
# ── C2PA cloud-manifest reference (Durable Content Credentials) ─
@@ -1167,38 +1273,99 @@ def _identify_from_evidence(
# reusing the derived `has_c2pa` / `source_kind` above, which are broader:
# the file path's answer must not move.
trained_source = b"trainedAlgorithmicMedia" in head or b"TrainedAlgorithmicMedia" in head
if not synthid and trained_source and c2pa_marker_in(head) and (vendors := synthid_evidence_vendors_in(region)):
# Same suppression as every other inference site: bytes that name their own
# forensic soft-binding algorithm carry that vendor's mark, and the generic
# vendor-token inference must not add a second, differently-attributed
# invisible watermark (Microsoft Designer: "Azure OpenAI ImageGen" agent +
# the InvisMark watermarked action read as "SynthID per OpenAI").
soft_binding_vendors = soft_binding_vendors_in(region)
if (
not synthid
and trained_source
and c2pa_marker_in(head)
and not soft_binding_vendors
and (vendors := synthid_evidence_vendors_in(region))
):
synthid = synthid_verdict(", ".join(vendors))
if synthid:
watermarks.append(f"SynthID watermark ({synthid})")
watermarks.append(
f"SynthID watermark ({synthid})"
if c2pa_usable
else f"SynthID watermark claimed by invalid C2PA credentials ({synthid})"
)
caveats.append(_SYNTHID_CAVEAT)
if v := _vendor_of(synthid):
if c2pa_usable and (v := _vendor_of(synthid)):
ai_vendor_claims["synthid"] = v
# ── C2PA soft-binding: a named forensic/third-party watermark vendor ─
# (Adobe TrustMark, Digimarc, Imatag, ...). Present in the manifest even when
# the watermark itself can't be decoded; names whose watermark stamped the pixels.
soft_binding = meta.get("soft_binding") or (", ".join(v) if (v := soft_binding_vendors_in(region)) else None)
soft_binding = meta.get("soft_binding") or (", ".join(soft_binding_vendors) if soft_binding_vendors else None)
if soft_binding:
signals.append(Signal("soft_binding", f"C2PA soft binding: {soft_binding}", "high"))
watermarks.append(f"Forensic watermark soft binding ({soft_binding})")
soft_binding_algorithm = meta.get("soft_binding_algorithm") or info.get("soft_binding_algorithm")
soft_binding_value = meta.get("soft_binding_value") or info.get("soft_binding_value")
soft_binding_details = "; ".join(
str(value) for value in (soft_binding, soft_binding_algorithm, soft_binding_value) if value
)
signals.append(
Signal(
"soft_binding",
f"C2PA soft binding: {soft_binding_details}",
"high" if c2pa_level == "verified" else "medium",
)
)
if c2pa_info_has_invismark(info):
# Keep the generic soft-binding row for schema-1 compatibility, and add
# the pixel-removal target as its own stable signal for clients that need
# to select diffusion without parsing human-readable detail.
signals.append(
Signal(
"invismark",
f"Microsoft InvisMark pixel watermark: {soft_binding_details}",
"high" if c2pa_level == "verified" else "medium",
)
)
watermarks.append(f"Forensic watermark soft binding ({soft_binding_details})")
caveats.append(_SOFT_BINDING_CAVEAT)
# ── IPTC "Made with AI" (Meta etc.), only meaningful without C2PA ─
iptc = any(m in head for m in IPTC_AI_MARKERS)
if iptc and not has_c2pa:
standalone_iptc = iptc and not has_c2pa
if standalone_iptc:
signals.append(Signal("iptc", "digitalSourceType (Made with AI)", "high"))
watermarks.append("IPTC digitalSourceType (Made with AI)")
# Muse Image stamps every output with the invisible Content Seal, and this
# tag is the only provenance such a file carries - the same measured bet
# the strength router makes (vendor_for_strength -> "meta"). Emit the seal
# as its own stable signal, the way InvisMark is additive over
# soft_binding, so clients select pixel removal from the signal list
# instead of parsing caveats. It is an attribution, not a decode: no
# public Content Seal decoder exists, hence "medium".
signals.append(
Signal(
"content_seal",
"Meta Muse Content Seal pixel watermark (attributed by the standalone AI digital-source tag)",
"medium",
)
)
watermarks.append("Invisible Content Seal watermark (Meta Muse attribution)")
caveats.append(_IPTC_ONLY_CAVEAT)
caveats.append(_CONTENT_SEAL_CAVEAT)
if platform is None:
# Apple Photos Clean Up (Apple Intelligence object removal) marks
# the edit with photoshop:Credit / IPTC "Apple Photos Clean Up"
# next to compositeWithTrainedAlgorithmicMedia. It was detected but
# previously never attributed.
platform = (
"Apple Photos (Clean Up AI edit)"
if b"Apple Photos Clean Up" in head
else "Made-with-AI tag (e.g. Meta AI); platform not specified"
)
if b"Apple Photos Clean Up" in head:
platform = "Apple Photos (Clean Up AI edit)"
else:
# The platform line follows the same measured bet the seal signal
# and the strength router make: Muse Image is the tag writer whose
# outputs this profile targets, so a hedged Muse attribution is
# more useful than "platform not specified" while the panel below
# already prices the Content Seal removal. The hedge stays in the
# wording - it names the attribution basis, not a detection.
platform = "Meta Muse Image (attributed by the standalone AI digital-source tag)"
# ── IPTC 2025.1 AI-disclosure fields (Iptc4xmpExt:AISystemUsed etc.) ─
iptc_ai = any(m in head for m in IPTC_AI_FIELD_MARKERS)
@@ -1259,17 +1426,17 @@ def _identify_from_evidence(
platform = "xAI (Grok / Aurora)"
ai_vendor_claims["xai"] = "xAI"
# ── HuggingFace-hosted job marker (hf-job-id PNG text chunk) ─────
# ── Hugging Face-hosted job marker (hf-job-id PNG text chunk) ─────
# Marks the hosting job, not a model -- medium confidence (commonly diffusion
# output). Like the visible sparkle, it lifts an otherwise-Unknown verdict to
# a tentative AI, but never overrides a high-confidence metadata signal.
hf_job = evidence.huggingface_job
if hf_job:
signals.append(Signal("hf_job", f"HuggingFace job {hf_job}", "medium"))
watermarks.append("HuggingFace-hosted job (hf-job-id)")
signals.append(Signal("hf_job", f"Hugging Face job {hf_job}", "medium"))
watermarks.append("Hugging Face-hosted job (hf-job-id)")
caveats.append(_HF_JOB_CAVEAT)
if platform is None:
platform = "HuggingFace-hosted job (model not identified)"
platform = "Hugging Face-hosted job (model not identified)"
# ── Samsung Galaxy AI editing marker (genAIType) ─────────────────
# Galaxy AI tools stamp a proprietary genAIType in PhotoEditor_Re_Edit_Data.
@@ -1311,8 +1478,18 @@ def _identify_from_evidence(
exif_gen = any(s.name == "exif_generator" for s in signals)
xai_sig = any(s.name == "xai_signature" for s in signals)
ai_from_metadata = bool(
(has_c2pa and (c2pa_is_ai or synthid))
or iptc
(has_c2pa and c2pa_usable and (c2pa_is_ai or synthid))
or standalone_iptc
or iptc_ai
or aigc
or local_keys
or invisible_wm
or exif_gen
or xai_sig
)
high_ai_from_metadata = bool(
(has_c2pa and c2pa_level == "verified" and (c2pa_is_ai or synthid))
or standalone_iptc
or iptc_ai
or aigc
or local_keys
@@ -1330,7 +1507,7 @@ def _identify_from_evidence(
if ai_from_metadata:
is_ai: bool | None = True
confidence = "high"
confidence = "high" if high_ai_from_metadata else "medium"
elif visible_only or hf_only or samsung_only:
is_ai = True
confidence = "medium"
@@ -1339,7 +1516,17 @@ def _identify_from_evidence(
confidence = "none"
# ── Integrity clashes: contradictions between independent signals ─
clashes = _integrity_clashes(ai_vendor_claims, camera_label, camera_has_ai_marker=bool(ai_vendor_claims))
clashes = _integrity_clashes(
ai_vendor_claims,
camera_label if c2pa_usable else None,
camera_has_ai_marker=bool(ai_vendor_claims),
)
if c2pa_level == "invalid":
clashes.insert(
0,
"C2PA credentials failed integrity validation"
+ (f": {', '.join(failed_c2pa_codes)}" if failed_c2pa_codes else "."),
)
caveats.append(_STRIP_CAVEAT)
# De-duplicate while preserving order.
@@ -1352,12 +1539,13 @@ def _identify_from_evidence(
confidence=confidence,
# Meaningful for the same digitalSourceType whether carried by C2PA or a
# standalone IPTC/XMP label. Other AI signals leave it None.
ai_source_kind=source_kind if (is_ai and (has_c2pa or iptc)) else None,
ai_source_kind=(source_kind if (is_ai and ((has_c2pa and c2pa_usable) or standalone_iptc)) else None),
ai_from_metadata=ai_from_metadata,
watermarks=watermarks,
signals=signals,
caveats=caveats,
integrity_clashes=clashes,
c2pa_validation=c2pa_validation,
)
@@ -1434,13 +1622,14 @@ def has_invisible_target(image_path: Path) -> bool:
The decision gate for the diffusion scrub (``invisible`` / ``all`` / ``batch``):
regenerating pixels removes an AI-specific invisible watermark (SynthID,
open DWT-DCT) but degrades a real photo, so it must not run when there is
nothing to remove. Runs :func:`identify` with ``check_visible=False`` -- a
visible mark is handled by the separate visible pass and is NOT a diffusion
target -- and ``check_invisible=True`` so an open watermark counts. Returns
``report.ai_from_metadata`` (C2PA AI issuer / SynthID provenance, IPTC, AIGC, local
gen params, EXIF/xAI, or open DWT-DCT). TrustMark alone does not trigger the
scrub because it also protects human-authored work and therefore is not an AI
signal by itself.
nothing to remove. It runs the same evidence pipeline as :func:`identify`
with visible checks disabled and invisible checks enabled, so a visible mark
is handled by the separate visible pass and is NOT a diffusion target. Returns
True for ``report.ai_from_metadata`` (C2PA AI issuer / SynthID provenance,
IPTC, AIGC, local gen params, EXIF/xAI, or open DWT-DCT), and also when an
invalid C2PA claim retains an AI or watermark removal hint. TrustMark alone
does not trigger the scrub because it also protects human-authored work and
therefore is not an AI signal by itself.
IMPORTANT -- this cannot prove a pixel SynthID is absent: SynthID is detectable
only through its C2PA proxy, so a metadata-stripped AI image reads as no signal
@@ -1451,8 +1640,17 @@ def has_invisible_target(image_path: Path) -> bool:
watermark on a paid removal is worse than over-regenerating a clean image.
"""
try:
report = identify(image_path, check_visible=False, check_invisible=True)
evidence = extract_provenance_evidence(image_path)
report = _identify_from_evidence(
evidence,
image_path=image_path,
check_visible=False,
check_invisible=True,
)
except Exception: # unreadable / detector error -> do not skip the removal
logger.debug("has_invisible_target: identify failed, defaulting to run", exc_info=True)
return True
return report.ai_from_metadata
# An asset edit can invalidate the C2PA binding while leaving the declared pixel
# watermark intact. Keep the removal gate fail-safe without promoting that broken
# claim back into the provenance verdict.
return report.ai_from_metadata or c2pa_info_has_removal_hint(evidence.c2pa_info)
+44 -2
View File
@@ -18,6 +18,7 @@ from typing import TYPE_CHECKING
from ._internal.watermark_profiles import (
DEFAULT_PROFILE,
QWEN_ZIMAGE_PROFILE,
REMOVAL_MODULES,
resolve_adaptive_polish,
resolve_seed,
@@ -32,7 +33,7 @@ warnings.filterwarnings("ignore", category=UserWarning, module="huggingface_hub"
warnings.filterwarnings("ignore", category=UserWarning, module="diffusers")
warnings.filterwarnings("ignore", module="transformers")
# Suppress HuggingFace internal logging
# Suppress Hugging Face internal logging
os.environ["TRANSFORMERS_VERBOSITY"] = "error"
os.environ["DIFFUSERS_VERBOSITY"] = "error"
@@ -103,7 +104,7 @@ class InvisibleEngine:
global pass, vendor-adaptive strength because an SDXL global stage
needs more of it). BOTH ARE CUDA-ONLY -- there is no CPU or MPS path
for invisible-watermark removal.
hf_token: HuggingFace API token.
hf_token: Hugging Face API token.
progress_callback: Optional callback for progress messages.
controlnet_conditioning_scale: Canny ControlNet structure-preservation
strength on the global stage of both profiles.
@@ -148,6 +149,8 @@ class InvisibleEngine:
tile: bool = False,
tile_size: int = 1024,
tile_overlap: int = 128,
text_manifest: Path | None = None,
fidelity_anchor: bool = False,
) -> Path:
"""Remove invisible watermark from an image.
@@ -180,6 +183,19 @@ class InvisibleEngine:
Engages only when the long side exceeds ``tile_size``.
tile_size: Tile dimension in px (default 1024).
tile_overlap: Overlap between adjacent tiles in px (default 128).
text_manifest: Operator-verified text lines bound to the decoded source
pixels. Enables the experimental Qwen-VAE ``vae-glyphs`` post-pass.
Requires the ``text-restoration`` extra and the ``qwen-zimage``
profile. Incompatible with downscaling, humanize, unsharp, and
adaptive polish. Tiling is rejected because that combination has
no provider-oracle calibration.
fidelity_anchor: Blend 15% of the Qwen-VAE donor across the whole frame
before glyph restoration. OFF by default since 0.27.1: that global
blend was measured to return detector-visible OpenAI SynthID on
poster-scale manifests (detected x6 with the anchor vs clean x6
without it, base clean; official Content Provenance API,
2026-08-19 - docs/text-protection-research.md). ``True`` reproduces
the 0.27.0 research behavior. Requires ``text_manifest``.
Returns:
Path to the cleaned image.
@@ -189,6 +205,25 @@ class InvisibleEngine:
seed = resolve_seed(seed)
adaptive_polish = resolve_adaptive_polish(adaptive_polish, self._remover.model_profile)
if fidelity_anchor and text_manifest is None:
raise ValueError("fidelity_anchor requires a text manifest")
if text_manifest is not None:
if self._remover.model_profile != QWEN_ZIMAGE_PROFILE:
raise ValueError("--text-manifest is supported only by the qwen-zimage profile")
if tile:
raise ValueError("--text-manifest is not calibrated with --tile")
if max_resolution != 0:
raise ValueError("--text-manifest requires --max-resolution 0")
if humanize > 0.0 or unsharp > 0.0 or adaptive_polish:
raise ValueError("--text-manifest requires humanize=0, unsharp=0, and adaptive polish disabled")
from remove_ai_watermarks import region_eraser
if not region_eraser.lama_available():
raise RuntimeError(
"Verified text restoration requires LaMa. Install: "
"pip install 'remove-ai-watermarks[text-restoration]'"
)
from PIL import Image, ImageOps
# Resolution policy: a max_resolution cap (0 = none) bounds memory on huge
@@ -205,6 +240,11 @@ class InvisibleEngine:
# Full-res original, kept for the adaptive-polish detail target (image is
# reassigned to the resized copy below; PIL resize returns a new object).
reference_pil = image
verified_text = None
if text_manifest is not None:
from remove_ai_watermarks._internal.text_restoration import load_verified_text_manifest
verified_text = load_verified_text_manifest(text_manifest, reference_pil)
# Both profiles run at the input's native geometry, so only the explicit max
# cap can move it, and it can only ever scale down.
@@ -240,6 +280,8 @@ class InvisibleEngine:
tile=tile,
tile_size=tile_size,
tile_overlap=tile_overlap,
text_manifest=verified_text,
fidelity_anchor=fidelity_anchor,
)
# Post-processing chain: decode the diffusion output ONCE, apply the
@@ -99,7 +99,7 @@ def detect_invisible_watermark(image_path: Path, *, image: NDArray[Any] | None =
# ``image`` lets a caller that has already decoded these pixels hand them in
# (mirrors gemini_engine.detect_sparkle_confidence). The decoder only reads the
# array -- it converts colour spaces into fresh buffers -- so no copy is needed.
# array -- it converts color spaces into fresh buffers -- so no copy is needed.
img = image if image is not None else image_io.imread(image_path)
if img is None:
return None
+9 -9
View File
@@ -1,6 +1,6 @@
"""Kling (可灵, Kuaishou) visible watermark detector/localizer.
"""Kling AI (可灵AI, Kuaishou) visible watermark detector/localizer.
Kling stamps its generations with a thin, light-gray "可灵AI 3.0" text strip in the
Kling AI stamps its generations with a thin, light-gray "可灵AI 3.0" text strip in the
bottom-right corner, preceded by the vendor's spiral logo (not part of the detection
silhouette -- logos vary between releases, the text run is what discriminates).
Known variants: an "Omni" suffix release, a latin "KlingAI 3.0" release, and a
@@ -9,7 +9,7 @@ suffix variants are only caught when the core run is bold enough (measured below
Detection matches the bundled glyph silhouette against the corner; removal is the
shared **localize -> fill** (the glyph-bbox :meth:`footprint_mask` feeds
``region_eraser``), NOT reverse-alpha. This module supplies only Kling's tuned
``region_eraser``), NOT reverse-alpha. This module supplies only Kling AI's tuned
:class:`TextMarkConfig` (``assets/kling_alpha.png`` -- a font-rendered synthetic
silhouette from ``scripts/render_vendor_silhouettes.py``, never cut from an
upload). It also feeds ``identify`` as the medium-confidence ``visible_kling``
@@ -26,7 +26,7 @@ producer USCC 91110108335469089C names the entity, 2026-07-21; harness
* ``alpha_height_frac`` comes from the silhouette aspect (0.239) at the fitted
width, matching the aspect the fit converged on (0.25).
* Gate 0.35, one step above the clean arm's max: on the cohort-vs-clean run
(cohort-contamination-guarded, 286 hand-labelled clean frames) the clean arm
(cohort-contamination-guarded, 286 hand-labeled clean frames) the clean arm
scored p99 0.304 / max 0.320, and every cohort frame >= 0.35 carries a visible
可灵AI 3.0 mark (9 of ~19 eyeballed visible marks fire = ~47% recall of visible
marks; the misses are the faint "Omni"-suffix release, the latin "KlingAI"
@@ -38,7 +38,7 @@ producer USCC 91110108335469089C names the entity, 2026-07-21; harness
provenance relaxation exists for this mark.
* No rival margin: at the shipped gate the template fires on 1 of 400
Doubao-marked frames (0.2%, a 豆包 frame sitting INSIDE the Kling cohort, still
below the gate), 0 of 298 Jimeng-marked frames and 0 of 286 hand-labelled clean
below the gate), 0 of 298 Jimeng-marked frames and 0 of 286 hand-labeled clean
frames, and a 0.10 rival margin costs zero genuine Kling detections -- so it is
simply unnecessary (same conclusion shape as Qwen).
"""
@@ -74,7 +74,7 @@ LOGO_MIN_LUMA = 150
TOPHAT_DELTA = 12
DETECT_MIN_COVERAGE = 0.04 # unused by the tophat front-end (kept for config parity)
# Calibrated 2026-07-21 on the vendor cohort vs 286 hand-labelled clean frames
# Calibrated 2026-07-21 on the vendor cohort vs 286 hand-labeled clean frames
# (cohort-contamination-guarded): clean p99 0.304 / max 0.320, and every cohort
# frame scoring >= 0.35 carries a visible 可灵AI 3.0 mark. 0.35 was picked over
# 0.33 (also zero clean fires) for margin against unseen clean content at a cost
@@ -87,7 +87,7 @@ _ALPHA_WIDTH_FRAC = 0.12
_ALPHA_HEIGHT_FRAC = 0.0287
_CONFIG = TextMarkConfig(
name="Kling",
name="Kling AI",
asset_name="kling_alpha.png",
corner="br",
margin_floor=4,
@@ -114,7 +114,7 @@ _CONFIG = TextMarkConfig(
def _alpha_template() -> NDArray[Any] | None:
"""The bundled Kling alpha template (float [0,1]), or None."""
"""The bundled Kling AI alpha template (float [0,1]), or None."""
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
@@ -124,7 +124,7 @@ def _glyph_silhouette() -> NDArray[Any] | None:
class KlingEngine(TextMarkEngine):
"""Detect/localize the visible Kling "可灵AI 3.0" watermark (locate -> mask; mask feeds the fill)."""
"""Detect/localize the visible Kling AI "可灵AI 3.0" watermark (locate -> mask; mask feeds the fill)."""
def __init__(self) -> None:
super().__init__(_CONFIG)
+8 -8
View File
@@ -1,13 +1,13 @@
"""LibLibAI visible watermark detector/localizer.
"""LiblibAI visible watermark detector/localizer.
LibLibAI (哩布哩布AI, USCC 91110105MACJ6K1C8A) stamps its generations with a
white triangle logo + "LibLibAI" latin wordmark at **bottom-center** (not a
LiblibAI (哩布哩布AI, USCC 91110105MACJ6K1C8A) stamps its generations with a
white triangle logo + "LiblibAI" Latin wordmark at **bottom-center** (not a
corner -- the locate box is horizontally centered). Detection matches the
bundled font-rendered "LibLibAI" silhouette (the triangle logo is NOT rendered
bundled font-rendered "LiblibAI" silhouette (the triangle logo is NOT rendered
-- logos vary, the wordmark discriminates); removal is the shared **localize ->
fill** (the glyph blob covers logo + wordmark, both bright).
This module supplies only LibLibAI's tuned :class:`TextMarkConfig`
This module supplies only LiblibAI's tuned :class:`TextMarkConfig`
(``assets/liblib_alpha.png`` from ``scripts/render_vendor_silhouettes.py``,
never cut from an upload).
@@ -64,7 +64,7 @@ _ALPHA_HEIGHT_FRAC = 0.026
_LADDER = (0.9, 1.0, 1.1)
_CONFIG = TextMarkConfig(
name="LibLibAI",
name="LiblibAI",
asset_name="liblib_alpha.png",
corner="bc",
margin_floor=4,
@@ -90,12 +90,12 @@ _CONFIG = TextMarkConfig(
def _alpha_template() -> NDArray[Any] | None:
"""The bundled LibLibAI alpha template (float [0,1]), or None."""
"""The bundled LiblibAI alpha template (float [0,1]), or None."""
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
class LibLibEngine(TextMarkEngine):
"""Detect/localize the visible LibLibAI wordmark (bottom-center; localize -> fill)."""
"""Detect/localize the visible LiblibAI wordmark (bottom-center; localize -> fill)."""
# Per-mark size floor prevents small generic icons from matching the wordmark.
_MIN_SHORT_SIDE = 480
+42 -11
View File
@@ -199,7 +199,7 @@ def parse_tc260_aigc_json(value: bytes) -> dict[str, str] | None:
return fields if TC260_AIGC_FIELDS & fields.keys() else None
# HuggingFace-hosted GPU jobs (Jobs / Spaces) stamp generated PNGs with this
# Hugging Face-hosted GPU jobs (Jobs / Spaces) stamp generated PNGs with this
# ``tEXt`` chunk key holding the job UUID. It marks the hosting job, not a
# specific model -- a medium-confidence AI signal (commonly diffusion output).
_HF_JOB_KEY: str = "hf-job-id"
@@ -413,7 +413,7 @@ def _scan_head_impl(image_path: Path, size: int) -> bytes:
# packet larger than this is not a provenance label.
_DECODED_TEXT_LIMIT = 512 * 1024
# Decoder values that are binary payloads with their own readers, not metadata text.
# An ICC profile is colour data and can run to hundreds of kilobytes; appending it
# An ICC profile is color data and can run to hundreds of kilobytes; appending it
# would bloat the buffer every later detector re-scans, for no signal.
_DECODER_BINARY_KEYS = frozenset({"icc_profile"})
@@ -510,7 +510,7 @@ def has_ai_metadata(image_path: Path) -> bool:
# only the XMP form; the raw-JSON tEXt chunk needs the PIL-based parse).
if aigc_label(image_path) is not None:
return True
# HuggingFace-hosted job marker (hf-job-id PNG text chunk).
# Hugging Face-hosted job marker (hf-job-id PNG text chunk).
if huggingface_job(image_path):
return True
# xAI / Grok: no C2PA/IPTC/XMP -- only the EXIF Signature + UUID-Artist pair.
@@ -682,10 +682,10 @@ def c2pa_cloud_manifest(image_path: Path) -> str | None:
def _huggingface_job_impl(image_path: Path) -> str | None:
"""Return the HuggingFace job id if the image carries an ``hf-job-id`` PNG
"""Return the Hugging Face job id if the image carries an ``hf-job-id`` PNG
text chunk, else None.
HuggingFace-hosted GPU jobs (Jobs / Spaces) stamp generated PNGs with an
Hugging Face-hosted GPU jobs (Jobs / Spaces) stamp generated PNGs with an
``hf-job-id`` ``tEXt`` chunk holding the job's UUID. It identifies the
*hosting job*, not a specific model, and is most commonly seen on diffusion-
generation output -- a medium-confidence AI signal, not proof of AI pixels
@@ -792,7 +792,7 @@ def _iptc_ai_system_impl(image_path: Path) -> str | None:
return iptc_ai_system_in(scan_head(image_path))
def synthid_source(image_path: Path) -> str | None:
def synthid_source(image_path: Path, *, c2pa_info: dict[str, Any] | None = None) -> str | None:
"""Return the vendor name(s) when provenance establishes SynthID.
This is provenance-based, not a local pixel decode. Google states that all
@@ -808,14 +808,25 @@ def synthid_source(image_path: Path) -> str | None:
Args:
image_path: Path to the image (PNG, JPEG, WebP, or ISOBMFF container).
c2pa_info: Already extracted normalized C2PA info, for callers that also need
another claim from the same manifest.
Returns:
Comma-joined vendor name(s) (e.g. ``"OpenAI"``) or None.
"""
from remove_ai_watermarks._internal.c2pa import extract_c2pa_info, synthid_evidence_vendors_in
from remove_ai_watermarks._internal.c2pa import (
c2pa_info_has_invalid_credential,
extract_c2pa_info,
synthid_evidence_vendors_in,
)
# PNG: the caBX chunk parser gives a clean, structured issuer.
vendors = extract_c2pa_info(image_path).get("synthid_vendors")
# Prefer the official reader's structured result. A failed asset binding or
# signature cannot establish the claim, and the raw fallback below must not
# resurrect it from the same damaged manifest bytes.
c2pa = extract_c2pa_info(image_path) if c2pa_info is None else c2pa_info
if c2pa_info_has_invalid_credential(c2pa):
return None
vendors = c2pa.get("synthid_vendors")
if vendors:
return ", ".join(vendors)
@@ -828,6 +839,13 @@ def synthid_source(image_path: Path) -> str | None:
ai_source = b"trainedAlgorithmicMedia" in data or b"TrainedAlgorithmicMedia" in data
if not (has_c2pa and ai_source):
return None
from remove_ai_watermarks._internal.c2pa import soft_binding_vendors_in
# A scan that names its own forensic soft-binding algorithm carries that
# vendor's mark; the generic vendor-token inference must not add a second,
# differently-attributed invisible watermark from the same bytes.
if soft_binding_vendors_in(data):
return None
matched = synthid_evidence_vendors_in(data)
return ", ".join(matched) if matched else None
@@ -1151,6 +1169,14 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
"actions",
"synthid_watermark",
"soft_binding",
"soft_binding_algorithm",
"soft_binding_value",
"c2pa_validation_source",
"c2pa_validation_state",
"c2pa_integrity",
"c2pa_signature",
"c2pa_signer_trust",
"c2pa_signer_validity",
):
if key in c2pa:
result.setdefault(key, str(c2pa[key]))
@@ -1169,6 +1195,11 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
if (aigc := aigc_label(image_path)) is not None:
producer = aigc.get("ContentProducer", "")
result["aigc_label"] = f"China AIGC label (TC260){f'; producer {producer}' if producer else ''}"
# The structural producer beside its display rendering: a machine
# consumer (video-visible provenance) must not parse the formatted
# sentence above, whose wording can change without notice.
if producer:
result["aigc_producer"] = producer
app_scan = scan_head(image_path)
app_provenance, app_generator = _app_metadata_evidence(app_scan)
@@ -1185,9 +1216,9 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
if system := iptc_ai_system(image_path):
result.setdefault("ai_system", f"IPTC 2025.1 AI disclosure ({system})")
# HuggingFace-hosted job marker (hf-job-id PNG text chunk).
# Hugging Face-hosted job marker (hf-job-id PNG text chunk).
if job := huggingface_job(image_path):
result.setdefault("huggingface_job", f"HuggingFace-hosted job ({job})")
result.setdefault("huggingface_job", f"Hugging Face-hosted job ({job})")
# Samsung Galaxy AI editing marker (genAIType in PhotoEditor_Re_Edit_Data).
if (genai := samsung_genai(image_path)) is not None:
result.setdefault("samsung_genai", f"Samsung Galaxy AI editing marker (genAIType={genai})")
@@ -0,0 +1,104 @@
"""Microsoft top-right AI-badge detector/localizer.
This engine covers one measured Microsoft output variant: a white pill with dark
internal shapes in the top-right corner. The evaluated files used both "Made with
AI" and "AI-Generated" wording. This is narrower than Microsoft's documented
watermark feature, which can use a Copilot icon or text and can place the mark in
other positions. A Microsoft provenance signal therefore does not establish that
this exact visible variant is present.
Detection matches a synthetic pill silhouette (white pill with the sparkle and
text KNOCKED OUT) against the top-hat blob of the located box: the holes are what
discriminate this pill from any other bright rounded element in the corner.
Removal is the shared **localize -> fill**; the glyph-bbox :meth:`footprint_mask`
covers the whole pill including its text.
The tuned numbers below were remeasured on 2026-08-27 with the registered engine
and ``scripts/registered_mark_calibrate.py``. The arms were kept distinct: 17
visually confirmed carriers, 343 Microsoft-provenance files whose visible-mark
status was not adjudicated, and 1200 non-overlapping no-signal controls:
* Geometry is single-mode and tight: pill 0.152 x 0.040 of the LONG side
(aspect 3.73-3.89 over 720..1536 px), margins ~0.010/0.007 of the same basis. One size
mode, so the shared 3-rung ladder is untouched and the locate box simply
wraps the pill with NCC slack.
* STRICT ONLY (``provenance_ncc_factor`` 1.0): controls max 0.293 / p99
0.200; confirmed carriers p50 0.519 / p90 0.578 / max 0.579, with 15/17
above the 0.38 gate. Provenance alone does not relax the gate: only 78/343
files in that cohort crossed it, and the rest were not visually labeled.
* Front-end "binary": the pill is a bold opaque overlay; the tophat blob is
solid with dark-text holes, exactly the template's shape.
"""
# pyright: reportUnusedFunction=false
from __future__ import annotations
from typing import TYPE_CHECKING, Any
from remove_ai_watermarks import _text_mark_engine
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
if TYPE_CHECKING:
from numpy.typing import NDArray
# Locate geometry as a fraction of the image LONG side (measured; scale_basis="long":
# on 1024x1536 portraits the pill tracks the 1536, and a width basis undersized
# the template until the portrait carriers fell to 0.15-0.32 NCC).
# The box wraps the measured pill rect (0.152W x 0.040W) with NCC slack; margins
# sit inside the pill's own ~0.010W-right / ~0.007W-top insets.
WM_WIDTH_FRAC = 0.170
WM_HEIGHT_FRAC = 0.055
MARGIN_RIGHT_FRAC = 0.004
MARGIN_TOP_FRAC = 0.003
# Glyph appearance: a bright near-white pill (luma ~245), gray-scale (sat < 60).
MAX_SATURATION = 60
LOGO_MIN_LUMA = 170
TOPHAT_DELTA = 10
# Calibrated 2026-08-27: non-overlapping no-signal controls (n=1200) max 0.293 /
# p99 0.200; visually confirmed carriers (n=17) p50 0.519 / p90 0.578 / max
# 0.579, with 15/17 above the 0.38 gate. The two misses score 0.249 and 0.315.
DETECT_MIN_COVERAGE = 0.30 # the pill fills most of its box; content corners do not
DETECT_NCC_THRESHOLD = 0.38
# Pill silhouette geometry (fraction of width): 0.152W x 0.040W, aspect ~3.78.
_ALPHA_NATIVE_WIDTH = 335
_ALPHA_WIDTH_FRAC = 0.152
_ALPHA_HEIGHT_FRAC = 0.040
_CONFIG = TextMarkConfig(
name="Microsoft top-right AI badge",
asset_name="microsoft_alpha.png",
corner="tr",
margin_floor=2,
width_frac=WM_WIDTH_FRAC,
height_frac=WM_HEIGHT_FRAC,
margin_x_frac=MARGIN_RIGHT_FRAC,
margin_bottom_frac=MARGIN_TOP_FRAC,
max_saturation=MAX_SATURATION,
logo_min_luma=LOGO_MIN_LUMA,
tophat_delta=TOPHAT_DELTA,
morph_open_size=5,
detect_min_coverage=DETECT_MIN_COVERAGE,
detect_ncc_threshold=DETECT_NCC_THRESHOLD,
alpha_width_frac=_ALPHA_WIDTH_FRAC,
alpha_height_frac=_ALPHA_HEIGHT_FRAC,
min_gw=24,
detect_frontend="binary",
scale_basis="long",
provenance_ncc_factor=1.0,
)
def _alpha_template() -> NDArray[Any] | None:
"""The bundled Microsoft pill template (float [0,1]), or None."""
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
class MicrosoftEngine(TextMarkEngine):
"""Detect/localize the measured Microsoft top-right AI badge."""
def __init__(self) -> None:
super().__init__(_CONFIG)
+1 -1
View File
@@ -64,7 +64,7 @@ _MASK_W, _MASK_H = 0.205, 0.115 # width of W, height of W
#
# Measured through the PRODUCT path (the `_keep_pill` gate), not the raw detector, by
# ``scripts/pill_gate_audit.py`` -- the raw path bypasses the gate and reads as a
# disaster that the shipped behaviour does not have. Re-run it when the gate changes.
# disaster that the shipped behavior does not have. Re-run it when the gate changes.
_FLAT_TEXTURE_MAX = 6.0
_silhouette: NDArray[Any] | None = None
+4 -4
View File
@@ -1,4 +1,4 @@
"""Qwen (Tongyi Qianwen, Alibaba) visible watermark detector/localizer.
"""Qwen (Alibaba) visible watermark detector/localizer.
Qwen stamps its generations with a visible "千问AI生成" text strip in the
bottom-right corner -- the explicit AIGC label mandated by China's GB 45438-2025
@@ -23,7 +23,7 @@ whose producer USCC 91440101MA9Y9T4H7A names the entity, 2026-07-21; harness
ratio 1.64 -- wider than the shared 3-rung ladder's 1.5625 span), so a single
fraction on the shared ladder covers ~75% of marks and the rest land in the
comb's collapse zone. Qwen therefore carries its OWN 2-rung ladder
(``TextMarkConfig.ladder``), one rung centred on each mode; the shared
(``TextMarkConfig.ladder``), one rung centered on each mode; the shared
default is untouched for every other mark.
* The mark also sits FARTHER off the corner than Doubao's box assumes (right
margin ~0.025 vs 0.004 of the short side), so Doubao's locate box clipped the
@@ -38,7 +38,7 @@ whose producer USCC 91440101MA9Y9T4H7A names the entity, 2026-07-21; harness
arm would be mostly false fills. No provenance relaxation exists for this
mark.
* No rival margin: at the shipped gate the template fires on 0 of 400
Doubao-marked frames, 0 of 298 Jimeng-marked frames and 0 of 286 hand-labelled
Doubao-marked frames, 0 of 298 Jimeng-marked frames and 0 of 286 hand-labeled
clean frames (the shared tail correlates at ~0.22, far below the gate), while
a 0.10 rival margin would have suppressed ~10% of genuine Qwen detections.
"""
@@ -74,7 +74,7 @@ LOGO_MIN_LUMA = 150
TOPHAT_DELTA = 12
DETECT_MIN_COVERAGE = 0.04 # unused by the tophat front-end (kept for config parity)
# Calibrated 2026-07-21 on the vendor cohort vs 286 hand-labelled clean frames
# Calibrated 2026-07-21 on the vendor cohort vs 286 hand-labeled clean frames
# (cohort-contamination-guarded): clean p99 0.301 / max 0.316, and every cohort
# frame scoring >= 0.45 carries a visible 千问AI生成 mark (86% of the eyeballed
# visible marks fire, the misses being white-on-near-white contrast losses).
@@ -32,7 +32,7 @@ whose producer USCC names the entity, harvested 2026-07-22 by
* STRICT ONLY (``provenance_ncc_factor`` 1.0): raw gray NCC is
contrast-DEPENDENT and the sub-gate band of a corner-anchored gray match is
unmeasured beyond the clean arm, so no provenance relaxation exists.
* Gate 0.34: on 283 hand-labelled clean frames (cohort-contamination-guarded)
* Gate 0.34: on 283 hand-labeled clean frames (cohort-contamination-guarded)
corner-anchored gray NCC p99 is 0.264 / max 0.304, while the 4 positives
score 0.38-0.54. 0.34 sits above the clean max with a small margin; the
positives are few, so the margin is deliberately thin on the recall side.
@@ -75,14 +75,14 @@ LOGO_MIN_LUMA = 150
TOPHAT_DELTA = 12
DETECT_MIN_COVERAGE = 0.04 # unused by the gray front-end (kept for config parity)
# Calibrated 2026-07-22 on the vendor cohort vs 283 hand-labelled clean frames:
# Calibrated 2026-07-22 on the vendor cohort vs 283 hand-labeled clean frames:
# corner-anchored gray NCC, clean p99 0.264 / max 0.304; positives 0.38-0.54.
DETECT_NCC_THRESHOLD = 0.34
# Detection-silhouette geometry (fraction of the image width), measured on the
# positives: mark width is ~0.320 of width on all three frame sizes (266px at 832,
# 345px at 1080, 491px at 1536), and the NCC is razor-sharp in size (0.537 on-size,
# 0.223 at +5.6% -- the same comb behaviour Qwen measured), so the nominal sits
# 0.223 at +5.6% -- the same comb behavior Qwen measured), so the nominal sits
# exactly on the measured size with a TIGHT ladder around it, not the shared 3 rungs
# (whose nearest rung landed 5.6% off and collapsed the match to 0.22).
_ALPHA_WIDTH_FRAC = 0.32
+299
View File
@@ -0,0 +1,299 @@
"""Draft operator-verifiable text lines for verified-text manifests.
Proposal-only OCR: PaddleOCR detection plus three script-chosen recognition
engines (Latin / Cyrillic / CJK), accepted only when three crop paddings
normalize identically and every confidence clears the floor. ``accepted``
means crop-stable, NEVER ground-truth-correct: on the reference posters the
draft's exact-text precision was 90.0% and 94.4% because high-confidence OCR
still lost punctuation (one English comma dropped, one ideographic comma
replaced with ASCII). Every accepted line needs a human yes/no before it may
enter a manifest with ``verified: true``.
Heavy imports (paddle, and the numpy/cv2/PIL pixel stack) stay inside the
call so importing this module costs nothing without the ``text-draft`` extra;
``draft_available()`` reports whether the extra is installed.
"""
# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportUnknownParameterType=false, reportMissingTypeArgument=false, reportMissingTypeStubs=false, reportMissingImports=false, reportArgumentType=false, reportAssignmentType=false, reportReturnType=false, reportCallIssue=false, reportIndexIssue=false, reportOperatorIssue=false
from __future__ import annotations
import unicodedata
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
from remove_ai_watermarks._internal.text_restoration import source_pixel_sha256
if TYPE_CHECKING:
from pathlib import Path
__all__ = [
"DraftLine",
"RejectedLine",
"TextDraft",
"choose_language",
"draft_available",
"draft_text_lines",
"group_word_boxes",
"source_pixel_sha256",
"stable_recognition",
]
# Crop paddings probed per line: recognition must be invariant across them.
JITTER_RATIOS: tuple[float, ...] = (0.08, 0.12, 0.2)
DETECT_SCORE_FLOOR = 0.5
@dataclass(frozen=True)
class DraftLine:
"""One crop-stable proposal. ``accepted`` means stable, not correct."""
box: tuple[int, int, int, int]
text: str
script: str
language: str
min_score: float
@dataclass(frozen=True)
class RejectedLine:
"""A detected line whose recognition was unstable or low-confidence."""
box: tuple[int, int, int, int]
script: str
language: str
reads: tuple[tuple[str, float], ...]
@dataclass(frozen=True)
class TextDraft:
"""The full proposal set for one image."""
accepted: tuple[DraftLine, ...] = ()
rejected: tuple[RejectedLine, ...] = ()
def normalize_text(text: str) -> str:
"""Normalize text for layout-independent comparison (casefold + no spaces)."""
return "".join(unicodedata.normalize("NFC", text).casefold().split())
def _has_script(text: str, script: str) -> bool:
return any(script in unicodedata.name(character, "") for character in text)
def choose_language(probes: dict[str, tuple[str, float]]) -> str:
"""Pick ``ch``/``ru``/``en`` from what each probe engine actually read."""
if _has_script(probes["ch"][0], "CJK"):
return "ch"
if _has_script(probes["ru"][0], "CYRILLIC"):
return "ru"
return "en"
def stable_recognition(reads: list[tuple[str, float]], min_score: float = 0.85) -> str | None:
"""The proposal text when every read normalizes identically and clears the floor."""
normalized = {normalize_text(text) for text, _score in reads}
if len(normalized) != 1 or min(score for _text, score in reads) < min_score:
return None
return reads[0][0]
def _vertical_overlap_ratio(left: tuple[int, int, int, int], right: tuple[int, int, int, int]) -> float:
overlap = max(0, min(left[3], right[3]) - max(left[1], right[1]))
return overlap / max(1, min(left[3] - left[1], right[3] - right[1]))
def _row_center(box: tuple[int, int, int, int]) -> float:
return (box[1] + box[3]) / 2
def group_word_boxes(boxes: list[tuple[int, int, int, int]]) -> list[tuple[int, int, int, int]]:
"""Merge word detections into line boxes by vertical overlap and gap."""
groups: list[tuple[int, int, int, int]] = []
for box in sorted(boxes, key=lambda item: (_row_center(item), item[0])):
matches: list[int] = []
for index, group in enumerate(groups):
if _vertical_overlap_ratio(box, group) < 0.45:
continue
horizontal_gap = max(0, max(box[0], group[0]) - min(box[2], group[2]))
line_height = min(box[3] - box[1], group[3] - group[1])
if horizontal_gap <= max(24, line_height * 3):
matches.append(index)
if not matches:
groups.append(box)
continue
index = max(matches, key=lambda item: _vertical_overlap_ratio(box, groups[item]))
x1, y1, x2, y2 = groups[index]
groups[index] = min(x1, box[0]), min(y1, box[1]), max(x2, box[2]), max(y2, box[3])
return sorted(groups, key=_row_center)
def _recognition_box(
box: tuple[int, int, int, int],
script: str,
width: int,
height: int,
vertical_pad_ratio: float | None = None,
) -> tuple[int, int, int, int]:
x1, y1, x2, y2 = box
line_height = y2 - y1
if script == "cjk":
left_pad = max(16, round(line_height * 0.2))
right_pad = max(16, round(line_height * 0.6))
return max(0, x1 - left_pad), y1, min(width, x2 + right_pad), y2
pad_x = max(16, line_height)
pad_y = max(8, line_height // 3) if vertical_pad_ratio is None else max(8, round(line_height * vertical_pad_ratio))
return max(0, x1 - pad_x), max(0, y1 - pad_y), min(width, x2 + pad_x), min(height, y2 + pad_y)
def _recognize(
engine: Any,
image: Any,
box: tuple[int, int, int, int],
script: str,
vertical_pad_ratio: float | None = None,
) -> tuple[str, float]:
import cv2
height, width = image.shape[:2]
x1, y1, x2, y2 = _recognition_box(box, script, width, height, vertical_pad_ratio)
crop = image[y1:y2, x1:x2]
if crop.shape[0] < 64:
scale = 64 / crop.shape[0]
crop = cv2.resize(crop, None, fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)
result = next(iter(engine.predict(crop)))
return str(result.get("rec_text", "")), float(result.get("rec_score", 0.0))
def _detect_line_boxes(engine: Any, source_rgb: Any) -> list[tuple[int, int, int, int]]:
import numpy as np
boxes: list[tuple[int, int, int, int]] = []
for page in engine.predict(source_rgb):
detected = page.get("rec_boxes", None)
if detected is None or len(detected) == 0:
detected = page.get("rec_polys", [])
for score, raw_box in zip(page.get("rec_scores", []), detected, strict=False):
if float(score) < DETECT_SCORE_FLOOR:
continue
points = np.asarray(raw_box, dtype=np.float32).reshape(-1)
if points.size == 4:
x1, y1, x2, y2 = points
else:
points = points.reshape(-1, 2)
x1, y1 = points.min(axis=0)
x2, y2 = points.max(axis=0)
boxes.append((round(float(x1)), round(float(y1)), round(float(x2)), round(float(y2))))
return group_word_boxes(boxes)
def draft_available() -> bool:
"""True when the ``text-draft`` extra (paddleocr + paddle) can run."""
from remove_ai_watermarks.optional_deps import module_available
return module_available("paddleocr") and module_available("paddle")
def _build_engines() -> tuple[Any, dict[str, Any]]:
from paddleocr import PaddleOCR, TextRecognition
# enable_mkldnn=False is load-bearing on linux: paddle's oneDNN PIR executor
# raises NotImplementedError on the text-detection graph there (seen on the
# Modal draft container, 2026-08-19), while the reference CPU path runs. The
# eval scripts never hit it because macOS pads take a different path.
detector = PaddleOCR(
lang="ch",
use_doc_orientation_classify=False,
use_doc_unwarping=False,
use_textline_orientation=False,
enable_mkldnn=False,
)
engines = {
"en": TextRecognition(model_name="en_PP-OCRv5_mobile_rec", enable_mkldnn=False),
"ru": TextRecognition(model_name="eslav_PP-OCRv5_mobile_rec", enable_mkldnn=False),
"ch": TextRecognition(model_name="PP-OCRv5_server_rec", enable_mkldnn=False),
}
return detector, engines
def draft_text_lines(
image: Path,
*,
min_score: float = 0.85,
detector: Any | None = None,
engines: dict[str, Any] | None = None,
stable: bool = True,
) -> TextDraft:
"""Propose verified-text manifest lines for ``image``; never verified ones.
Args:
image: path of the source image (draft boxes are in ITS pixel space).
min_score: recognition confidence floor. With ``stable=True`` every
jittered read must clear it; with ``stable=False`` the single
probe for the chosen language must.
detector/engines: injectable Paddle objects (tests use fakes); when
None they are built from the ``text-draft`` extra, which must be
installed (``draft_available()`` reports it).
stable: when True (default), accept a line only if three crop paddings
normalize identically. When False, take one recognition trio and
accept on score alone. Automatic restoration uses box/script only,
so the jitter gate is optional there.
Returns:
``TextDraft`` with crop-stable ``accepted`` proposals and unstable
``rejected`` lines. Both lists need human review before any manifest
may claim ``verified: true``; accepted means crop-stable, NOT
ground-truth-correct.
"""
import os
import numpy as np
from PIL import Image
if detector is None or engines is None:
if not draft_available():
raise RuntimeError(
"Text drafting requires PaddleOCR. Install: pip install 'remove-ai-watermarks[text-draft]'"
)
os.environ.setdefault("PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK", "True")
detector, engines = _build_engines()
source_rgb = np.asarray(Image.open(image).convert("RGB"))
accepted: list[DraftLine] = []
rejected: list[RejectedLine] = []
for box in _detect_line_boxes(detector, source_rgb):
probes: dict[str, tuple[str, float]] = {}
for language, engine in engines.items():
script = "cjk" if language == "ch" else "alphabetic"
probes[language] = _recognize(engine, source_rgb, box, script, 0.1)
language = choose_language(probes)
script = "cjk" if language == "ch" else "alphabetic"
if stable:
reads = [_recognize(engines[language], source_rgb, box, script, ratio) for ratio in JITTER_RATIOS]
text = stable_recognition(reads, min_score)
else:
text, score = probes[language]
reads = [(text, score)]
if not text.strip() or score < min_score:
text = None
if text is None:
rejected.append(
RejectedLine(
box=box,
script=script,
language=language,
reads=tuple((value, score) for value, score in reads),
)
)
else:
accepted.append(
DraftLine(
box=box,
text=text,
script=script,
language=language,
min_score=min(score for _value, score in reads),
)
)
return TextDraft(accepted=tuple(accepted), rejected=tuple(rejected))
+6 -4
View File
@@ -1,8 +1,8 @@
"""High-level video processing API.
The product path covers provenance identification, container-level AI metadata
removal, temporally stabilized visible Sora, Veo, Seedance, Dola, Hailuo, and
Kling removal, and an oracle-certified opt-in VAE profile for video SynthID.
removal, temporally stabilized visible Sora, Veo, Seedance, Dola, Hailuo AI, and
Kling AI removal, and an oracle-certified opt-in VAE profile for video SynthID.
The visible pixel path reuses the image package's shared fill backends.
"""
@@ -179,8 +179,8 @@ _VISIBLE_PLATFORM = {
"veo": "Google Veo",
"seedance": "ByteDance Seedance",
"dola": "ByteDance Dola",
"hailuo": "MiniMax Hailuo",
"kling": "Kuaishou Kling",
"hailuo": "MiniMax Hailuo AI",
"kling": "Kuaishou Kling AI",
}
@@ -239,6 +239,7 @@ def _visible_removal_plan(
from remove_ai_watermarks.video_visible import (
VISIBLE_MARK_POLICIES,
has_bytedance_video_provenance,
has_hailuo_video_provenance,
has_sora_provenance,
has_veo_provenance,
stabilize_localizations,
@@ -249,6 +250,7 @@ def _visible_removal_plan(
"veo": has_veo_provenance,
"seedance": has_bytedance_video_provenance,
"dola": has_bytedance_video_provenance,
"hailuo": has_hailuo_video_provenance,
}.get(selected_mark)
policy = VISIBLE_MARK_POLICIES[selected_mark]
regions = stabilize_localizations(
+25 -8
View File
@@ -4,8 +4,8 @@ Supported marks use fully synthetic silhouettes made from geometric primitives,
OpenCV's built-in font, and Pillow's bundled font. Sora detection searches the
full frame because the wordmark moves. Veo detection covers both the current
four-point diamond and legacy ``Veo`` text. Seedance detects the boxed ``AI``
label, Dola detects its compact text label, Hailuo detects the composite
MINIMAX/Hailuo label, and Kling detects its version-independent wordmark core.
label, Dola detects its compact text label, Hailuo AI detects the composite
MINIMAX/Hailuo AI label, and Kling AI detects its version-independent wordmark core.
A single frame is never enough to authorize removal: the temporal arbiter
requires the candidate to recur at the same location across adjacent frames.
This keeps isolated lookalikes in clean videos from becoming removal masks.
@@ -890,11 +890,14 @@ class VisibleMarkPolicy:
Every value here is MEASURED per provider; the arbiter itself
(:func:`_stabilize_localizations`) is shared and knows nothing about providers.
``accepts_provenance`` is load-bearing rather than cosmetic. Hailuo and Kling have
no metadata that could confirm them, so their rows force ``provenance=False``; that
used to be guaranteed structurally by wrappers that took no ``provenance``
parameter at all, and this flag is what preserves the guarantee now that one entry
point serves every mark.
``accepts_provenance`` is load-bearing rather than cosmetic. A vendor with no
metadata that could confirm it forces ``provenance=False``; that used to be
guaranteed structurally by wrappers that took no ``provenance`` parameter at
all, and this flag is what preserves the guarantee now that one entry point
serves every mark. Kling keeps the flag off (its TC260 producer code is image
evidence; no kling video corpus row ties a producer to the moving mark), while
Hailuo accepts it through the MiniMax TC260 label
(:func:`has_hailuo_video_provenance`).
``padding_fraction`` and ``mask_style`` belong to the removal plan rather than the
arbiter, but they are per-provider constants like the rest, so they live on the same
@@ -969,7 +972,9 @@ VISIBLE_MARK_POLICIES: dict[str, VisibleMarkPolicy] = {
anchor_iou=0.80,
padding_fraction=0.12,
mask_style="box",
accepts_provenance=False,
# No provenance_weak_floor: the measured weak floor stays the entry bar;
# a MiniMax TC260 label only drops the strong-frame requirement for an
# already-stable run (see has_hailuo_video_provenance).
),
"kling": VisibleMarkPolicy(
weak_floor=_KLING_WEAK_CONFIDENCE,
@@ -1458,3 +1463,15 @@ def has_bytedance_video_provenance(markers: dict[str, str]) -> bool:
).lower()
source_type = markers.get("source_type", "").lower()
return ("bytedance" in identity or "byteplus" in identity) and "trainedalgorithmicmedia" in source_type
def has_hailuo_video_provenance(markers: dict[str, str]) -> bool:
"""Whether a TC260 label names MiniMax, Hailuo's maker, as the producer.
Hailuo video exports carry no C2PA; their TC260 ``ContentProducer`` is the
bare name ``MiniMax`` (verified on retained MiniMax-hailuo clips). The
producer travels as its own structural marker (``aigc_producer``, set in
``metadata.get_ai_metadata``), so this matches the field exactly rather
than parsing the human-readable ``aigc_label`` sentence.
"""
return markers.get("aigc_producer", "").strip().lower() == "minimax"
+43 -19
View File
@@ -17,17 +17,18 @@ localizer stays cheap (cv2/numpy, CPU) so a memory-tight caller can run it on a
small worker; the heavy fill (MI-GAN / LaMa) is opt-in and chosen by the caller.
Entries:
- ``gemini`` -- Google Gemini / Nano Banana sparkle, bottom-right.
- ``gemini`` -- Google Gemini / Nano Banana visible watermark (sparkle), bottom-right.
- ``doubao`` -- ByteDance Doubao "豆包AI生成" text strip, bottom-right.
- ``jimeng`` -- ByteDance Jimeng / Dreamina "★ 即梦AI" wordmark, bottom-right.
- ``qwen`` -- Alibaba Tongyi Qianwen "千问AI生成" text strip, bottom-right.
- ``kling`` -- Kuaishou Kling "可灵AI 3.0" text strip, bottom-right.
- ``qwen`` -- Alibaba Cloud Qwen "千问AI生成" text strip, bottom-right.
- ``kling`` -- Kuaishou Kling AI "可灵AI 3.0" text strip, bottom-right.
- ``yuanbao`` -- Tencent Yuanbao "元宝 / AI生成" two-line mark, bottom-right.
- ``samsung`` -- Samsung Galaxy AI "Contenuti generati dall'AI" strip, bottom-left.
- ``jimeng_pill`` -- Jimeng-basic "AI生成" pill, top-left (capture-less).
- ``runninghub`` -- RunningHub "RunningHub AI生成" text, top-left (gray front-end).
- ``baidu`` -- Baidu "百度 AI生成" text + white tag, bottom-right.
- ``liblib`` -- LibLibAI "LibLibAI" wordmark, bottom-center.
- ``liblib`` -- LiblibAI "LiblibAI" wordmark, bottom-center.
- ``microsoft`` -- one measured Microsoft white AI-badge variant, top-right.
"""
from __future__ import annotations
@@ -147,7 +148,7 @@ _REMOVED_SENSITIVITIES = {
"vendor made it or where the mark is. If you can see a mark the detector missed, "
"act on what you see: erase(image, region=(x, y, w, h)), or the CLI "
"`--mark <name> --no-detect` for a known text mark. Use sensitivity='auto' for "
"the default evidence-driven behaviour."
"the default evidence-driven behavior."
)
}
@@ -156,7 +157,7 @@ def validate_sensitivity(value: str) -> Sensitivity:
"""Reject a removed sensitivity LOUDLY instead of silently falling back to ``auto``.
``Sensitivity`` is a ``Literal``, which is not enforced at runtime, so a caller
upgrading from 0.15 would pass ``"assume_ai"`` and quietly get ``auto`` behaviour --
upgrading from 0.15 would pass ``"assume_ai"`` and quietly get ``auto`` behavior --
a silent semantic change on the one release where they most need to be told.
"""
if value in _REMOVED_SENSITIVITIES:
@@ -368,7 +369,7 @@ _GEMINI_AUTO_MIN_CONF = GEMINI_SPARKLE_TRUST_CONF
# that never had a mark rather than on moved ones.
#
# Measured blind on 954 unique Google-metadata uploads (detector never saw the
# metadata), hand-labelled against a two-sided control (labeller sensitivity ~88%,
# metadata), hand-labeled against a two-sided control (labeler sensitivity ~88%,
# specificity 100%). "Additions" = accepted with provenance but not without:
#
# band precision 95% CI population
@@ -409,6 +410,7 @@ _ENGINE_CLASS: dict[str, tuple[str, str]] = {
"runninghub": ("runninghub_engine", "RunningHubEngine"),
"baidu": ("baidu_engine", "BaiduEngine"),
"liblib": ("liblib_engine", "LibLibEngine"),
"microsoft": ("microsoft_engine", "MicrosoftEngine"),
}
@@ -485,7 +487,14 @@ def fill(image: NDArray[Any], mask: NDArray[Any], *, backend: Backend = "auto")
def _gemini_wrap(d: Any, *, provenance: bool) -> MarkDetection:
gate = _GEMINI_PROVENANCE_MIN_CONF if provenance else _GEMINI_AUTO_MIN_CONF
detected = bool(d.detected) and d.confidence >= gate
return MarkDetection("gemini", "Google Gemini sparkle", "bottom-right", detected, d.confidence, d.region)
return MarkDetection(
"gemini",
"Google Gemini visible watermark (sparkle)",
"bottom-right",
detected,
d.confidence,
d.region,
)
def _gemini_detect(image: NDArray[Any], *, provenance: bool = False) -> MarkDetection:
@@ -561,12 +570,14 @@ def _text_mark(
label_regime: str | None = "tc260",
provenance_signals: tuple[str, ...] = ("aigc",),
tc260_producer_codes: tuple[str, ...] = (),
provenance_platform_tokens: tuple[str, ...] = (),
) -> KnownMark:
"""Build a text-mark registry row from its shared detector and mask adapters.
``product`` defaults to the key (one mark, one product); pass it only when two
marks share a product. ``label_regime`` and ``provenance_signals`` default to the
China-AIGC label because every text mark registered so far except Samsung uses it.
China-AIGC label because every text mark registered so far except Samsung and
Microsoft uses it.
"""
return KnownMark(
key,
@@ -580,6 +591,7 @@ def _text_mark(
_text_mark_mask(key),
provenance_signals=provenance_signals,
tc260_producer_codes=tc260_producer_codes,
provenance_platform_tokens=provenance_platform_tokens,
_detect_both=_text_mark_detect_both(key, label, location),
)
@@ -616,11 +628,11 @@ def _pill_features(image: NDArray[Any]) -> dict[str, float]:
_REGISTRY: tuple[KnownMark, ...] = (
# Gemini is a Google C2PA/SynthID product, not a China-AIGC labeller: label_regime
# Gemini is a Google C2PA/SynthID product, not a China-AIGC labeler: label_regime
# is None so it can never act as a TC260 sibling in _keep_pill.
KnownMark(
"gemini",
"Google Gemini sparkle",
"Google Gemini visible watermark (sparkle)",
"bottom-right",
True,
"gemini",
@@ -651,14 +663,14 @@ _REGISTRY: tuple[KnownMark, ...] = (
"qwen",
"Qwen 千问AI生成 text",
"bottom-right",
platform="Alibaba Tongyi Qianwen (visible 千问AI生成 mark detected)",
platform="Alibaba Cloud Qwen (visible 千问AI生成 mark detected)",
tc260_producer_codes=("91440101MA9Y9T4H7A",),
),
_text_mark(
"kling",
"Kling 可灵AI 3.0 text",
"Kling AI 可灵AI 3.0 text",
"bottom-right",
platform="Kuaishou Kling (visible 可灵AI 3.0 mark detected)",
platform="Kuaishou Kling AI (visible 可灵AI 3.0 mark detected)",
tc260_producer_codes=("91110108335469089C",),
),
_text_mark(
@@ -693,11 +705,23 @@ _REGISTRY: tuple[KnownMark, ...] = (
),
_text_mark(
"liblib",
"LibLibAI wordmark",
"LiblibAI wordmark",
"bottom-center",
platform="LibLibAI (visible LibLibAI mark detected)",
platform="LiblibAI (visible LiblibAI mark detected)",
tc260_producer_codes=("91110105MACJ6K1C8A",),
),
# One measured Microsoft visible-mark variant: a white top-right pill with
# dark internal shapes. Microsoft's documented feature also permits other
# icon, text, and placement variants, which this detector does not cover.
_text_mark(
"microsoft",
"Microsoft top-right AI badge",
"top-right",
label_regime=None,
provenance_signals=(),
platform="Microsoft (visible top-right AI badge detected)",
provenance_platform_tokens=("microsoft",),
),
# Same product as the Jimeng wordmark -- the one pair that cross-relaxes.
KnownMark(
"jimeng_pill",
@@ -794,7 +818,7 @@ def tc260_producer_vendors() -> dict[str, str]:
def _pill_suppressors() -> set[str]:
"""Marks whose detection vetoes the capture-less pill: same label regime as the
pill, different product. Derived so a newly registered TC260 mark cannot be
forgotten here -- which is exactly how LibLibAI ended up missing."""
forgotten here -- which is exactly how LiblibAI ended up missing."""
pill = get_mark("jimeng_pill")
return {
m.key
@@ -822,9 +846,9 @@ def _keep_pill(keys: set[str], *, provenance: frozenset[str], footprint_flat: bo
No confirmation at all -> never remove (blocks false fires on non-Jimeng content).
The suppressor set is DERIVED from the registry (same label regime, different
product), not hand-listed. The hand-written list had drifted: LibLibAI was
product), not hand-listed. The hand-written list had drifted: LiblibAI was
registered alongside RunningHub and Baidu but never added to it, so a confident
LibLibAI detection did not veto the pill the way its two siblings did. Marks
LiblibAI detection did not veto the pill the way its two siblings did. Marks
outside the TC260 regime (Gemini, Samsung) are deliberately NOT suppressors --
neither can put ``"jimeng"`` into ``provenance``, so neither can enable the arm
they would be vetoing."""
+22 -4
View File
@@ -2,7 +2,9 @@
from __future__ import annotations
from typing import TYPE_CHECKING
import struct
import zlib
from pathlib import Path
import cv2
import numpy as np
@@ -10,9 +12,6 @@ import pytest
from PIL import Image
from PIL.PngImagePlugin import PngInfo
if TYPE_CHECKING:
from pathlib import Path
@pytest.fixture
def clean_photo(tmp_path: Path) -> Path:
@@ -76,3 +75,22 @@ def tmp_clean_png(tmp_path: Path) -> Path:
path = tmp_path / "clean.png"
img.save(path, pnginfo=pnginfo)
return path
@pytest.fixture
def tampered_chatgpt_png(tmp_path: Path) -> Path:
"""Add valid PNG metadata after signing so the C2PA asset hash no longer matches."""
source = Path(__file__).resolve().parents[1] / "data" / "fixtures" / "provenance" / "chatgpt-1.png"
data = source.read_bytes()
iend = data.rfind(b"\x00\x00\x00\x00IEND")
assert iend >= 0
kind = b"tEXt"
payload = b"c2pa-test\x00benign post-signing metadata mutation"
chunk = (
struct.pack(">I", len(payload)) + kind + payload + struct.pack(">I", zlib.crc32(kind + payload) & 0xFFFFFFFF)
)
target = tmp_path / "tampered-chatgpt.png"
target.write_bytes(data[:iend] + chunk + data[iend:])
assert target.read_bytes() != data
return target
+10
View File
@@ -245,6 +245,7 @@ class TestInvisibleOptionsMirrorTheEngine:
tile=True,
tile_size=768,
tile_overlap=64,
text_manifest=tmp_path / "verified-lines.json",
)
seen: dict[str, object] = {}
@@ -511,6 +512,15 @@ class TestSourceEvidenceHolder:
assert holder.visible_provenance() == api.visible_provenance(DOUBAO)
assert holder.has_invisible_target() == identify.has_invisible_target(DOUBAO)
def test_holder_preserves_invalid_c2pa_removal_hint(self, tampered_chatgpt_png):
from remove_ai_watermarks import api, identify
holder = api._SourceEvidence(tampered_chatgpt_png)
assert identify.identify(tampered_chatgpt_png, check_visible=False).ai_from_metadata is False
assert holder.has_invisible_target() is True
assert holder.has_invisible_target() == identify.has_invisible_target(tampered_chatgpt_png)
def test_extraction_failure_fails_safe_in_both_directions(self, monkeypatch, tmp_path):
"""No provenance means no relaxation; an unknown invisible target means SCRUB.
Leaving a watermark on a paid removal is worse than over-regenerating."""
+40 -2
View File
@@ -228,8 +228,8 @@ class TestVisibleCommand:
# The transparent corners must remain transparent.
assert out[0, 0, 3] == 0
assert out[199, 199, 3] == 0
# The opaque centre remains opaque (the watermark region default is bottom-right,
# which doesn't overlap the centre square at 200x200).
# The opaque center remains opaque (the watermark region default is bottom-right,
# which doesn't overlap the center square at 200x200).
assert out[100, 100, 3] == 255
def test_visible_keeps_alpha_opaque_in_watermark_region(self, runner, tmp_path):
@@ -426,6 +426,39 @@ class TestInvisibleCommand:
assert result.exit_code == 0, result.output
mock_engine.remove_watermark.assert_called_once()
def test_invisible_explicit_vendor_implies_force_and_sets_cohort(self, runner, sample_png, tmp_path):
"""--vendor meta names a cohort the file cannot prove (Content Seal carries
no C2PA), so it must both bypass the no-signal skip and arrive at the engine
as the vendor, where it resolves to the measured Meta floor (not the
area-curve value the same size would otherwise get)."""
mock_cls, mock_engine = _mock_invisible_engine()
output = tmp_path / "clean.png"
with (
patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True),
patch("remove_ai_watermarks.cli.InvisibleEngine", mock_cls, create=True),
patch("remove_ai_watermarks.invisible_engine.InvisibleEngine", mock_cls),
):
result = runner.invoke(main, ["invisible", str(sample_png), "-o", str(output), "--vendor", "meta"])
assert result.exit_code == 0, result.output
kwargs = mock_engine.remove_watermark.call_args.kwargs
assert kwargs["vendor"] == "meta"
assert "0.1" in result.output # the resolved Meta floor, printed
assert "0.094" not in result.output # not the sample's area-curve value (200x200 -> ~0.0944)
def test_invisible_vendor_auto_keeps_detection_semantics(self, runner, sample_png, tmp_path):
"""--vendor auto is the default spelled out: detection still runs and a
no-signal file still skips."""
mock_cls, mock_engine = _mock_invisible_engine()
output = tmp_path / "clean.png"
with (
patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True),
patch("remove_ai_watermarks.cli.InvisibleEngine", mock_cls, create=True),
patch("remove_ai_watermarks.invisible_engine.InvisibleEngine", mock_cls),
):
result = runner.invoke(main, ["invisible", str(sample_png), "-o", str(output), "--vendor", "auto"])
assert result.exit_code == 2, result.output
mock_engine.remove_watermark.assert_not_called()
def test_invisible_runs_without_force_when_signal_present(self, runner, tmp_path):
"""An image carrying an AI metadata signal IS a scrub target, so the run
proceeds with no --force needed."""
@@ -629,11 +662,13 @@ class TestMetadataCommand:
result = runner.invoke(main, ["metadata", str(tmp_clean_png), "--check"])
assert result.exit_code == 0
assert "No AI metadata" in result.output
assert "not the same as 'clean'" in result.output
def test_metadata_check_ai(self, runner, tmp_png_with_ai_metadata):
result = runner.invoke(main, ["metadata", str(tmp_png_with_ai_metadata), "--check"])
assert result.exit_code == 0
assert "AI metadata detected" in result.output
assert "not the same as 'clean'" not in result.output
def test_metadata_remove(self, runner, tmp_png_with_ai_metadata, tmp_path):
output = tmp_path / "stripped.png"
@@ -649,6 +684,7 @@ class TestMetadataCommand:
)
assert result.exit_code == 0
assert "stripped" in result.output
assert "not the same as 'clean'" in result.output
def test_metadata_remove_reports_failure_when_the_strip_was_a_no_op(self, runner, tmp_path):
"""A file PIL cannot decode is copied through UNCHANGED by the fail-safe.
@@ -723,6 +759,8 @@ class TestIdentifyCommand:
result = runner.invoke(main, ["identify", str(sample), "--no-visible"])
assert result.exit_code == 0
assert "AI-generated (fully synthetic)" in result.output
assert "C2PA validation: integrity=valid, signature=valid" in result.output
assert "signer trust=untrusted, signer validity=expired" in result.output
def test_identify_json_is_valid(self, runner, tmp_png_with_ai_metadata):
result = runner.invoke(main, ["identify", str(tmp_png_with_ai_metadata), "--no-visible", "--json"])
+69
View File
@@ -0,0 +1,69 @@
"""Tests for the Content Seal oracle corpus layout.
Mirrors the synthid corpus guard: the manifest is the source of truth for
which binaries exist, and every recorded hash must match the file it names.
Derived rows are recipes, not stored files, so only originals are checked
against disk.
"""
from __future__ import annotations
import csv
import hashlib
import re
from pathlib import Path
CORPUS_DIR = Path(__file__).resolve().parent.parent / "data" / "contentseal"
MANIFEST = CORPUS_DIR / "manifest.csv"
ORIGINALS = CORPUS_DIR / "originals"
_SHA256 = re.compile(r"^[0-9a-f]{64}$")
_VALID_VERDICTS = {"detected", "not_detected", ""}
_VALID_ORIGINS = {"meta-model-api", "derived", "meta-blog-cdn", "synthetic-local"}
def _manifest_rows() -> list[dict[str, str]]:
with open(MANIFEST, newline="") as f:
return list(csv.DictReader(f))
def test_manifest_original_rows_match_binaries_and_hashes() -> None:
rows = [r for r in _manifest_rows() if r["file"]]
stored = {path.name for path in ORIGINALS.iterdir() if path.is_file()}
assert {r["file"].removeprefix("originals/") for r in rows} == stored
for row in rows:
digest = hashlib.sha256((CORPUS_DIR / row["file"]).read_bytes()).hexdigest()
assert digest == row["sha256"], row["file"]
def test_manifest_rows_are_well_formed() -> None:
rows = _manifest_rows()
assert len({row["sha256"] for row in rows}) == len(rows), "duplicate sha256"
for row in rows:
assert _SHA256.match(row["sha256"]), row["name"]
assert row["origin"].split(":")[0] in _VALID_ORIGINS, row["name"]
assert row["oracle_verdict"] in _VALID_VERDICTS, row["name"]
# Every oracle verdict must carry its check timestamp.
if row["oracle_verdict"]:
assert row["checked_at_utc"], row["name"]
# Detection rows must name the oracle attribution.
if row["oracle_verdict"] == "detected":
assert "Muse Image 1" in row["oracle_attribution"], row["name"]
def test_default_pipeline_clearance_is_recorded() -> None:
"""The verified claim that the default profile clears Content Seal must stay."""
rows = {row["name"]: row for row in _manifest_rows()}
for name in ("fox_modal_invisible", "text_modal_invisible"):
assert rows[name]["oracle_verdict"] == "not_detected", name
def test_deterministic_transforms_reproduce_recorded_hashes(tmp_path: Path) -> None:
from scripts.contentseal_transforms import reproduce_transforms
outputs = reproduce_transforms(tmp_path)
assert len(outputs) == 8
assert all(path.is_file() for path in outputs)
@@ -0,0 +1,88 @@
"""Regression: a manifest that names its own forensic soft binding must not
also report a SynthID watermark from the generic vendor-token inference.
Microsoft Designer manifests sign as Microsoft, carry the InvisMark
``c2pa.watermarked`` action, and name their generation agent
"Azure OpenAI ImageGen". The OpenAI issuer token inside that agent name plus
the watermarked action used to satisfy the OpenAI SynthID-evidence rule,
double-counting one forensic mark as two pixel watermarks.
"""
from __future__ import annotations
from remove_ai_watermarks._internal.c2pa import c2pa_info_from_manifest_store
DESIGNER_STORE = {
"active_manifest": "designer",
"manifests": {
"designer": {
"signature_info": {"issuer": "Microsoft Corporation", "common_name": "Microsoft Corporation"},
"claim_generator_info": [{"name": "Microsoft Responsible AI Provenance", "version": "1.0"}],
"assertions": [
{
"label": "c2pa.actions",
"data": {
"actions": [
{
"action": "c2pa.created",
"softwareAgent": {"name": "Azure OpenAI ImageGen"},
"digitalSourceType": "http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia",
},
{"action": "c2pa.watermarked"},
]
},
},
{
"label": "c2pa.soft-binding",
"data": {
"alg": "com.microsoft.invismark.1",
"blocks": [{"value": "bf7a2993-cc1f-47e1-b1f0-cd8839aabb22"}],
},
},
],
}
},
}
def test_named_soft_binding_suppresses_generic_synthid_evidence() -> None:
info = c2pa_info_from_manifest_store(DESIGNER_STORE)
assert info["ai_source_kind"] == "generated"
assert info["soft_binding_algorithm"] == "com.microsoft.invismark.1"
assert info.get("synthid_watermark") is None
assert info.get("synthid_vendors") is None
def test_vendor_agent_name_alone_is_not_the_vendors_provenance() -> None:
"""The identity-scoped inference must not fire on a service name either.
Same manifest without the soft binding: the "Azure OpenAI ImageGen" agent
is not an OpenAI signature or claim generator, so no OpenAI SynthID
evidence may be derived from it.
"""
store = {
"active_manifest": "designer",
"manifests": {
"designer": {
"signature_info": {"issuer": "Microsoft Corporation", "common_name": "Microsoft Corporation"},
"claim_generator_info": [{"name": "Microsoft Responsible AI Provenance", "version": "1.0"}],
"assertions": [
{
"label": "c2pa.actions",
"data": {
"actions": [
{
"action": "c2pa.created",
"softwareAgent": {"name": "Azure OpenAI ImageGen"},
"digitalSourceType": "http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia",
},
{"action": "c2pa.watermarked"},
]
},
}
],
}
},
}
info = c2pa_info_from_manifest_store(store)
assert info.get("synthid_watermark") is None
+23
View File
@@ -0,0 +1,23 @@
"""Cross-corpus integrity checks for tracked evaluation tables."""
from __future__ import annotations
import csv
import re
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
EVALUATIONS = ROOT / "data" / "evaluations"
SHA256 = re.compile(r"^[0-9a-f]{64}$")
def test_every_recorded_evaluation_sha256_is_well_formed() -> None:
checked = 0
for path in sorted(EVALUATIONS.rglob("*.csv")):
with path.open(newline="", encoding="utf-8") as stream:
for line_number, row in enumerate(csv.DictReader(stream), start=2):
for field, value in row.items():
if field is not None and field.endswith("sha256") and value:
assert SHA256.fullmatch(value), f"{path.relative_to(ROOT)}:{line_number} {field}={value!r}"
checked += 1
assert checked > 0
+10 -2
View File
@@ -20,7 +20,7 @@ import pytest
_SCRIPTS = Path(__file__).resolve().parent.parent / "scripts"
def _load_assign():
def _load_module():
# fidelity_metrics is a standalone PEP723 script, not an installed module; load it by
# path with scripts/ on sys.path so its `_plain_console` shim import resolves.
sys.path.insert(0, str(_SCRIPTS))
@@ -35,7 +35,15 @@ def _load_assign():
pytest.skip(f"fidelity_metrics import deps missing: {exc}")
finally:
sys.path.remove(str(_SCRIPTS))
return mod.assign_faces_one_to_one
return mod
def _load_assign():
return _load_module().assign_faces_one_to_one
def test_cer_remains_case_sensitive() -> None:
assert _load_module()._cer("A", "a") == 1.0
def test_distinct_faces_match_nearest() -> None:
Loaded 100 of 128 files, more files were not shown because too many files have changed in this diff. Show more