Compare commits

...
282 Commits
Author SHA1 Message Date
Victor Kuznetsov d8fcd0f79b Stop double-counting a named forensic mark as SynthID provenance
A manifest that names its own forensic soft-binding algorithm carries
that vendor's mark; the generic watermark-action vendor-token inference
must not add a second, differently-attributed invisible watermark from
the same bytes. Microsoft Designer manifests triggered exactly that:
signed by Microsoft, watermarked by InvisMark, with the generation
agent named "Azure OpenAI ImageGen" - the OpenAI issuer token inside
that service name plus the InvisMark watermarked action satisfied the
OpenAI SynthID-evidence rule, and identify reported one forensic mark
as two paid pixel watermarks.

Three changes, one rule at every inference site (the verdict-scan
comment's own lesson: a rule that lives in only one copy is a rule the
others silently lack):

- c2pa.py structured path: SynthID evidence now scopes to the
  signer/generator identity strings only (signature issuer, claim
  generator), never the raw chain, and is suppressed entirely when a
  soft-binding algorithm is named.
- c2pa.py byte fallback and metadata.py synthid_source: suppressed when
  the scan names a soft-binding algorithm.
- identify.py verdict scan: same suppression.

Gemini and ChatGPT originals keep their provenance-asserted SynthID
strings; the Designer regression is pinned by
test_designer_synthid_suppression.py (agent name alone is not the
vendor's provenance, and a named soft binding suppresses the
inference).
2026-08-27 15:20:45 -07:00
Victor Kuznetsov 0a3b227f2c Merge pull request #87 from wiltodelta/dependabot/uv/minor-and-patch-c54089dda1
Bump the minor-and-patch group with 5 updates
2026-08-27 11:50:53 -07:00
Victor Kuznetsov f91f9fd9fc Record the public Microsoft Content Provenance check URL
The InvisMark strength ladder was measured against Microsoft's public
Content Provenance page, but no doc or comment carried its URL - the
API how-to link was the only address recorded anywhere, and the page
is what a human can actually check without an Azure account. Record
https://ai.azure.com/nextgen/validate in supported-signals.md, the
watermarking landscape, and the strength derivation comment, with the
honest caveat that its collapsed verdict tops out at Inconclusive
rather than the API's separate watermark-negative result.
2026-08-27 11:21:08 -07:00
Victor KuznetsovandClaude Fable 4.5 c4aecfbbdf Release 0.32.0
Co-Authored-By: Claude Fable 4.5 <noreply@anthropic.com>
2026-08-26 23:03:04 -07:00
Victor KuznetsovandClaude Fable 4.5 ab528ec0e8 Ship the measured Meta Content Seal cohort with auto routing and --vendor override
Full Meta Muse Image support in the invisible-removal path:

- QWEN_ZIMAGE_META_STRENGTH = 0.1: derived by the standard
  worst-boundary-plus-cross-source-spread method over five oracle-bracketed
  generations (data/contentseal/manifest.csv)
- Auto mode: vendor_for_strength routes a file whose only provenance is the
  standalone AI IPTC trainedAlgorithmicMedia tag onto the meta cohort; C2PA
  issuers win first, so Google/OpenAI/Microsoft routing is unchanged. Muse
  WebP outputs place the XMP in a tail chunk, so the scan uses the shared
  chunk-aware metadata.scan_head rather than a plain head read
- Explicit override: --vendor on invisible/all/batch and
  InvisibleOptions.vendor name the cohort on stripped files; naming a cohort
  asserts the watermark is present, so the no-signal gate treats it like
  --force at both the CLI and API seams
- sdxl-zimage has no measured Meta rung: an explicit meta vendor falls to
  the conservative unknown 0.25 rather than inventing one
- identify emits a Content Seal caveat pointing at the removal path
- The legacy visible 'Imagined with AI' mark stays unregistered: a dedicated
  sample hunt (newsroom mockups, community posts, press screenshots, dead
  imagine.meta.com, broken Wayback captures) found no pixel-verifiable
  capture, and the registry rule forbids encoding a corner without one.
  erase --region remains its removal path; outcome recorded in the landscape

Co-Authored-By: Claude Fable 4.5 <noreply@anthropic.com>
2026-08-26 22:59:23 -07:00
Victor KuznetsovandClaude Fable 4.5 a1811b6221 Release 0.31.2
Documentation and corpus release: Meta Muse Image Content Seal support
records (oracle-verified boundaries, wire format, removal verification)
plus the contentseal manifest guard. No runtime behavior change.

Co-Authored-By: Claude Fable 4.5 <noreply@anthropic.com>
2026-08-26 21:11:40 -07:00
Victor KuznetsovandClaude Fable 4.5 e09104e39f Document Meta Muse Image Content Seal support with oracle-verified corpus
Muse Image (muse-image-1.0, Meta Superintelligence Labs, 2026-07-07) ships
every output with Content Seal, a proprietary invisible pixel watermark, and
no visible mark. Establish support documentation and a verified corpus:

- data/contentseal/: five own generations via the Meta Model API, every
  oracle verdict recorded in manifest.csv (44 rows, settled-text protocol,
  fresh-navigation variant for calibration rows)
- Oracle: meta.ai/identification web tool only; no API endpoint exists in
  the Meta Model API (verified against dev.meta.ai/docs); internal REST
  pair documented with its server-side sliding-window rate limit
- Removal: default qwen-zimage profile clears Content Seal (oracle-verified
  on the worst source); strength floor derived at 0.1 by the standard
  worst-boundary-plus-cross-source-spread method, recorded but not encoded
  as a constant since no provenance signal routes Muse output onto a vendor
  cohort
- Seal robustness measured: survives resize, JPEG q85, metadata stripping,
  CDN WebP transcode; dies to center crops and diffusion regeneration
- tests/test_contentseal_corpus.py guards manifest integrity

Co-Authored-By: Claude Fable 4.5 <noreply@anthropic.com>
2026-08-26 21:08:47 -07:00
Victor Kuznetsov 29f17105de Release 0.31.1 with calibrated provider strengths 2026-08-26 11:22:13 -07:00
dependabot[bot] 2810b53585 Bump the minor-and-patch group with 5 updates
Bumps the minor-and-patch group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [diffusers](https://github.com/huggingface/diffusers) | `0.39.0` | `0.40.0` |
| [transformers](https://github.com/huggingface/transformers) | `5.15.0` | `5.15.1` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.27.0` | `1.28.0` |
| [onnxruntime](https://github.com/microsoft/onnxruntime) | `1.28.0` | `1.29.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.3` | `0.16.4` |


Updates `diffusers` from 0.39.0 to 0.40.0
- [Release notes](https://github.com/huggingface/diffusers/releases)
- [Commits](https://github.com/huggingface/diffusers/compare/v0.39.0...v0.40.0)

Updates `transformers` from 5.15.0 to 5.15.1
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.15.0...v5.15.1)

Updates `huggingface-hub` from 1.27.0 to 1.28.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.27.0...v1.28.0)

Updates `onnxruntime` from 1.28.0 to 1.29.0
- [Release notes](https://github.com/microsoft/onnxruntime/releases)
- [Changelog](https://github.com/microsoft/onnxruntime/blob/main/docs/ReleaseNotesWorkflow.md)
- [Commits](https://github.com/microsoft/onnxruntime/compare/v1.28.0...v1.29.0)

Updates `ruff` from 0.16.3 to 0.16.4
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.3...0.16.4)

---
updated-dependencies:
- dependency-name: diffusers
  dependency-version: 0.40.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: transformers
  dependency-version: 5.15.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.28.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: onnxruntime
  dependency-version: 1.29.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-26 08:48:21 +00:00
Victor KuznetsovandClaude Opus 5 ba5710836b Release 0.31.0
Minor, not patch: the C2PA confidence mapping changed. A cryptographically bound
manifest now reports high confidence where 0.27.0 through 0.30.1 reported medium,
because the previous gate required a trust anchor no installation has. Consumers
that branch on `confidence` will see verdicts move on files whose bytes did not.

The provenance report stays at schema 1. No field was added, removed, renamed or
retyped, and the meaning of `confidence` is unchanged -- the value it carried for a
verified claim was wrong. Bumping the output schema instead would break every
pinned schema-1 consumer on a corrected value rather than a new shape.

pre-commit: 1) maintain.sh - exit 1 on uv-secure, lightning PYSEC-2026-3624 unchanged from 0.30.1, no fixed release exists, vulnerable API unreachable (no load_from_checkpoint in project or trustmark); ruff, ruff format, pyright src/, and 1394 tests passed separately; 2) /simplify - version bump only; 3) docs sync - no version refs outside pyproject, __init__.py, uv.lock; 4) CLAUDE.md - no change

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 20:11:25 -07:00
Victor KuznetsovandClaude Opus 5 c927560614 Stop gating C2PA confidence on a trust anchor that never ships
High-confidence C2PA attribution required signingCredential.trusted, a status code
the reader emits only when a trust anchor list is loaded. None ships, so from 0.27.0
through 0.30.0 the branch was unreachable in production for every vendor: an intact,
cryptographically bound manifest scored the same medium as a fallback parse that
validated nothing, which collapsed the one distinction the official reader exists to
draw. A hand-built info dict stamping that code kept the branch green in the suite.

Confidence now follows the binding. Signer trust and certificate expiry stay visible
as their own dimensions and as caveats, because a trust list that was never
configured is a missing input, not a finding against the credential. Every committed
provenance fixture with a reader result and an intact binding now reaches high
confidence, and test_no_committed_fixture_reports_a_trusted_signer guards the
reachability itself rather than a synthesized status set.

Revocation joins binding and signature failures as disqualifying. It arrives only on
signer_validity, so a check reading the other two returned a confident AI verdict off
a credential the issuer had disowned, with an empty integrity_clashes -- quieter than
a hash mismatch on the same file. Expiry stays non-disqualifying: it does not imply
the signed bytes changed, and a signature genuinely made outside validity already
arrives as claimSignature.outsideValidity.

The rule now lives in one place. _validation_fields maps status codes to the four
dimensions and names the failures that moved one; c2pa_info_has_invalid_credential
maps dimensions to disqualified. The ingredient-reachability walk and the
user-visible reason both consume that path instead of re-classifying raw codes, so
adding this one rule no longer means editing three layers in lockstep.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 20:04:25 -07:00
Victor Kuznetsov f83da5aee4 Expose Microsoft InvisMark removal signal 2026-08-25 18:52:26 -07:00
Victor Kuznetsov 251ba9ac79 Record the ComfyUI sync PyPI CDN race and its rerun recovery
pre-commit: 1) maintain.sh - docs-only change, full gate ran this session with identical code state (exit 1 on the known lightning triage, rest green); 2) /simplify - docs only; 3) docs sync - this is the sync; 4) CLAUDE.md - no change
2026-08-25 17:17:17 -07:00
Victor Kuznetsov c5764661b4 Release 0.30.0
pre-commit: 1) maintain.sh - exit 1 on uv-secure, lightning PYSEC-2026-3624 unchanged from 0.29.0, no fixed release exists, vulnerable API unreachable (no load_from_checkpoint in project or trustmark); ruff, pyright src/, and 1391 tests passed separately; 2) /simplify - version bump only; 3) docs sync - no version refs outside pyproject, __init__.py, uv.lock; 4) CLAUDE.md - no change
2026-08-25 16:52:27 -07:00
Victor Kuznetsov 97cba11efe Read the InvisMark soft-binding algorithm and its signed watermark id
Reachable c2pa.soft-binding assertions now surface their exact alg and the
bounded printable block value next to the normalized vendor label; a value
without its algorithm stays hidden because it cannot be attributed.
com.microsoft.invismark.1 uses that value as the pixel-watermark identifier
in Microsoft Paint output, so identify names it, metadata carries it, and an
InvisMark soft binding keeps the invisible-removal gate fail-safe even after
C2PA asset binding goes invalid. Content fingerprints still do not trigger
pixel regeneration. Removal is verified against Microsoft's Content
Provenance Detection API, which reports Watermark and C2PA separately; the
protocol and the pixel-identical control requirement are documented.

Implemented in a parallel session; verified, gated, and committed by pi.

pre-commit: 1) maintain.sh - exit 1 on the known uv-secure lightning PYSEC-2026-3624 triage (no fix available, unchanged from 0.29.0); ruff, pyright src/, and 1391 tests passed separately; 2) /simplify - single-pass, clean; 3) docs sync - five docs updated by the author session, no remaining references found; 4) CLAUDE.md - invariants recorded in module-internals, no change needed
2026-08-25 15:54:26 -07:00
Victor Kuznetsov 809d980fb7 Describe the real Hugging Face Space release mechanism
The Space demo is deployed from wiltodelta/raiw-hf-space: its uv lock pins the
library version, its sync workflow mirrors the files, and a factory reboot
alone re-installs the pinned version rather than upgrading. Document the manual
pin bump, the smoke-test CI in that repository, and live-call verification of
the running Space.

pre-commit: 1) maintain.sh - docs-only change, full gate ran this session with identical code state (exit 1 on the known lightning triage, rest green); 2) /simplify - docs only; 3) docs sync - this is the sync, no other references found; 4) CLAUDE.md - existing Space mention remains accurate
2026-08-25 15:00:24 -07:00
Victor Kuznetsov 10da166f3e Record the release-verification uv cache caveat and the pending lightning bump
pre-commit: 1) maintain.sh - docs-only change, last full run minutes ago: exit 1 on the known uv-secure lightning triage, ruff/pyright/1386 tests green; 2) /simplify - docs only; 3) docs sync - these are the doc updates; 4) CLAUDE.md - no change
2026-08-25 13:36:50 -07:00
Victor Kuznetsov 8871eb3f67 Release 0.29.0
pre-commit: 1) maintain.sh - exit 1 on uv-secure, lightning PYSEC-2026-3624 unchanged from 0.28.1, no fixed release exists, vulnerable API unreachable (no load_from_checkpoint in project or trustmark); ruff, pyright src/, and 1386 tests passed separately; 2) /simplify - version bump only; 3) docs sync - no version refs outside pyproject, __init__.py, uv.lock; 4) CLAUDE.md - no change
2026-08-25 13:25:33 -07:00
Victor Kuznetsov 44ec203c22 Drop the stale untiled restriction from text restoration docs 2026-08-25 13:22:49 -07:00
Victor Kuznetsov 4beb61463a Say metadata check and strip results are not a clean verdict 2026-08-25 13:22:38 -07:00
Victor Kuznetsov 0d11e4099c Fix adaptive text restoration crop bounds 2026-08-25 12:44:05 -07:00
Victor Kuznetsov fa83c1444f Add geometry-only text manifest schema 2026-08-25 11:33:21 -07:00
Victor Kuznetsov f52e40ae24 Name Copilot, Dreamina, and Qwen in C2PA source labels 2026-08-21 14:43:39 -07:00
Victor Kuznetsov 0aa03c4e89 Release 0.28.1
pre-commit: 1 maintain.sh - uv-secure lightning PYSEC-2026-3624 known, no fix, same as 0.28.0; ruff/pyright/1378 tests passed; 2) /simplify - version bump only; 3) docs sync - no refs; 4) CLAUDE.md - no change
EOF
)
2026-08-19 21:46:14 -07:00
Victor Kuznetsov 91aa49b03a Allow verified-text restoration on tiled images
Tile the Qwen VAE donor the same way as the global pass. Glyph restore already runs on the blended full frame, so the old tile ban was an artificial gate.
2026-08-19 21:33:22 -07:00
Victor Kuznetsov ee865e95e4 Add geometry draft mode and pad silhouette descenders
pre-commit: 1 maintain.sh - not rerun full, ruff + 34 targeted tests passed; 2) /simplify - pad lives in silhouette crop, stable=False reuses probes; 3) docs sync - no README/cli refs to update; 4) CLAUDE.md - no change
EOF
)
2026-08-19 19:53:38 -07:00
Victor Kuznetsov d84fd1a125 Merge pull request #86 from wiltodelta/dependabot/uv/minor-and-patch-267694dd16
Bump the minor-and-patch group across 1 directory with 7 updates
2026-08-19 12:22:54 -07:00
Victor Kuznetsov e315eacb14 Document automated-operator verified manifests
verified:true may attest machine-verified geometry (stability-gated
boxes inside caps) set by a service; the pipeline consumes box/script
only, text is advisory, so the verification burden is on the geometry.
2026-08-19 12:21:32 -07:00
Victor Kuznetsov 3ba4ace7e0 Disable mkldnn in the draft engines (linux PIR crash) 2026-08-19 10:22:58 -07:00
dependabot[bot] 13888704f8 Bump the minor-and-patch group across 1 directory with 7 updates
Bumps the minor-and-patch group with 7 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [python-dotenv](https://github.com/theskumar/python-dotenv) | `1.2.2` | `1.2.3` |
| [c2pa-python](https://github.com/contentauth/c2pa-python) | `0.37.5` | `0.37.7` |
| [av](https://github.com/PyAV-Org/PyAV) | `18.0.0` | `18.1.0` |
| [transformers](https://github.com/huggingface/transformers) | `5.14.1` | `5.15.0` |
| diffsynth | `2.1.0` | `2.1.2` |
| [paddleocr](https://github.com/PaddlePaddle/PaddleOCR) | `3.4.1` | `3.7.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.2` | `0.16.3` |



Updates `python-dotenv` from 1.2.2 to 1.2.3
- [Release notes](https://github.com/theskumar/python-dotenv/releases)
- [Changelog](https://github.com/theskumar/python-dotenv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/theskumar/python-dotenv/compare/v1.2.2...v1.2.3)

Updates `c2pa-python` from 0.37.5 to 0.37.7
- [Release notes](https://github.com/contentauth/c2pa-python/releases)
- [Changelog](https://github.com/contentauth/c2pa-python/blob/main/docs/release-notes.md)
- [Commits](https://github.com/contentauth/c2pa-python/compare/v0.37.5...v0.37.7)

Updates `av` from 18.0.0 to 18.1.0
- [Release notes](https://github.com/PyAV-Org/PyAV/releases)
- [Changelog](https://github.com/PyAV-Org/PyAV/blob/v18.1.0/CHANGELOG.rst)
- [Commits](https://github.com/PyAV-Org/PyAV/compare/v18.0.0...v18.1.0)

Updates `transformers` from 5.14.1 to 5.15.0
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.14.1...v5.15.0)

Updates `diffsynth` from 2.1.0 to 2.1.2

Updates `paddleocr` from 3.4.1 to 3.7.0
- [Release notes](https://github.com/PaddlePaddle/PaddleOCR/releases)
- [Commits](https://github.com/PaddlePaddle/PaddleOCR/compare/v3.4.1...v3.7.0)

Updates `ruff` from 0.16.2 to 0.16.3
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.2...0.16.3)

---
updated-dependencies:
- dependency-name: python-dotenv
  dependency-version: 1.2.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: c2pa-python
  dependency-version: 0.37.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: av
  dependency-version: 18.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: transformers
  dependency-version: 5.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: diffsynth
  dependency-version: 2.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: paddleocr
  dependency-version: 3.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-19 17:19:11 +00:00
Victor Kuznetsov e938b57f8c Package the verified-text draft annotator as text_draft
Move the draft-annotation logic (PaddleOCR line detection, word-box
grouping, three script-chosen recognition engines, crop-jitter
stability gate) from the evaluation-only scripts into the installable
package, with lazy paddle imports and a text-draft extra (CPU, no
torch). draft_text_lines() returns accepted (crop-stable, NEVER
ground-truth-correct - precision on the reference posters was 90.0% /
94.4%) and rejected proposals; source_pixel_sha256 is re-exported for
manifest building. scripts/infer_text_lines.py now dogfoods the
package module instead of loading the eval script by path.
2026-08-19 10:13:22 -07:00
Victor Kuznetsov 3cd1e47935 Add the oracle-measured Google strength floor to qwen-zimage
The resolution curve's 0.154 top left the 4.33 MP CJK-sign fixture
SynthID-detected x3 in Gemini on the full production path (visible
stage -> qwen-zimage seed 0 -> resize-back -> metadata strip), with a
valid pixel-identical stripped control in the same session
(2026-08-18). Google-provenance content now resolves to the flat
QWEN_ZIMAGE_GOOGLE_STRENGTH 0.30 floor instead of the area curve -
anchors at 0.30 measured clean in Gemini on two fixtures (CJK sign +
18-face) at 3/3 checks across two work accounts, and stayed clean
under the vae-glyphs donor layer. openai/unknown content keeps the
curve; an explicit strength still wins.
2026-08-18 22:34:41 -07:00
Victor Kuznetsov 2298282b8b Thread fidelity_anchor through WatermarkRemover
The engine forwards the new kwarg into WatermarkRemover.remove_watermark,
which rejected it with TypeError on the real (non-fake) path - caught by
the deployed Modal smoke test, not by the unit fakes.
2026-08-18 21:32:57 -07:00
Victor Kuznetsov 42d427731f Turn the verified-text fidelity anchor off by default
The whole-frame 15% Qwen-VAE blend returned detector-visible OpenAI
SynthID on poster-scale manifests through the engine text-manifest
path (official Content Provenance API, 2026-08-19: restored detected
x6 with the anchor, clean x6 without it; base outputs clean x6;
pixel-identical stripped controls detected, proving the pixel channel).
Add fidelity_anchor=False to remove_watermark and InvisibleOptions and
--fidelity-anchor on the CLI to reproduce the 0.27.0 research
behavior. Text-box MAE cost of the new default is under one point on
all three fixtures (11.60->11.72, 7.79->7.86, 7.57->8.13).
2026-08-18 21:23:25 -07:00
Victor Kuznetsov 50d9f44829 Guide text-heavy content away from forced diffusion scrubs
Issue #84: text, tables, and UI screenshots are the worst case for
--force regeneration. Document that the default pipeline skips the
invisible stage without a detectable signal, that visible/metadata
stages never redraw glyphs, and why paste-back is not offered.
2026-08-18 11:03:53 -07:00
Victor Kuznetsov 2e1428406e Release 0.27.0 2026-08-18 11:02:18 -07:00
Victor Kuznetsov 21ce1dfadd Cover QuickTime TC260 placements, Luma AI tags, and MiniMax video provenance 2026-08-17 19:38:37 -07:00
Victor Kuznetsov 2369d351a9 Merge pull request #83 from wiltodelta/chore/text-preservation-evaluation
Add verified text restoration
2026-08-15 12:30:27 -07:00
Victor Kuznetsov 432b63b6d7 Add verified text restoration 2026-08-15 12:25:35 -07:00
Victor Kuznetsov 8c00525946 Evaluate selective text restoration 2026-08-15 12:20:25 -07:00
Victor Kuznetsov f8003f505a Merge pull request #82 from wiltodelta/agent/c2pa-validation
Validate C2PA credentials before attribution
2026-08-15 11:38:47 -07:00
Victor Kuznetsov 2eab24a2e1 Validate C2PA credentials before attribution 2026-08-15 11:31:59 -07:00
Victor Kuznetsov 8201ada070 Merge pull request #81 from wiltodelta/docs/video-synthid-parked-state
Record the video SynthID measurements and park the investigation
2026-08-15 11:11:05 -07:00
Victor KuznetsovandClaude Opus 5 b7d44ff1a3 Warn users that the invisible video path leaves audio untouched
The finding lived only in the research archive, whose own banner says it may
describe prototypes that were later removed. It belongs where users read
limitations: the path strips every metadata marker while copying the audio
bitstream byte for byte, so a clean local report on a clip with generated audio
is unproven rather than a guarantee.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 11:08:09 -07:00
Victor KuznetsovandClaude Opus 5 e8f65022cc Record the video SynthID measurements and park the investigation
Everything here is evidence and state, not behavior: the shipped operating point
is untouched at 512 px / 12 fps / noise_std=0.15.

The carrier was downloaded and its sha256 matched the manifest, so its properties
are now measured rather than assumed: 1920x1080 at 24 fps with an AAC audio track.
Those three fields go into the two 2026-07-31 rows, which could not previously
tell a reader what the downscale factor even was.

The geometry prize is measured end to end for the first time. Against the
untouched source, 1024 scores +3.46 dB over the shipped 512, while the entire
noise_std axis is worth 1.92 dB. Two readings that the table alone hides are
recorded with it: the temporal residual IMPROVES with resolution, because the
shared noise field lives on the latent grid and is four times finer relative to
the frame at 1920; and the frame-rate arm cannot be judged by these metrics at
all, since they price its cost and not the smoothness it buys.

Two findings that outrank the quality question. The certified row does not
reproduce -- a rerun of the same configuration on a different device and dtype
gives a different hash and different metrics, and the manifest records neither.
And the pipeline copies audio byte for byte while stripping every metadata
marker, so it can emit a file our own detector calls clean with an untouched
Google-generated audio track inside it. The mechanism is proven on two carriers;
whether that audio carries a mark the verifier reads is not, and only the oracle
can say.

The six prepared oracle submissions were never run: file upload to the verifier
failed at the tool level. Their artifacts are gone with the scratch directory,
which is the intended lifecycle for generated media, and the document says what
rebuilding costs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 11:01:14 -07:00
Victor KuznetsovandClaude Opus 5 4855586834 Halve decoder memory with strip processing and a raveled Haar pass
Each Haar pass is one flat pywt.downcoef call over a raveled strip instead of
pywt.dwt(..., axis=1)[0], and the plane is walked in strips so no full-plane
float64 intermediate exists. Exact only while the last axis is even, so
_approximation raises on an odd width rather than returning wrong bits, and
TestRaveledHaarPass pins both that raise and the downcoef/dwt equivalence a
pywt upgrade could take away.

Drops the block constructor knob: the fold chains are written for 4, nothing
ever passed another value, and a knob that silently decodes wrong is worse
than no knob.

Peak RSS 111 MB to 21 MB on a 4.3 MP image; the decoder itself 0.011s to
0.007s, which is only 0.4% of identify() now that it is under 2% of the run.
Output bits and detector verdicts over 200 sampled data/ images, two
synthesized carriers and eight degenerate shapes are byte-identical to the
pre-vectorization decoder.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 13:33:34 -07:00
Victor KuznetsovandClaude Opus 5 37789e02f6 Vectorize the DWT-DCT decode path, 15x on the decoder
Output stays bit-identical: decoder bits and detector verdicts recorded over
200 sampled data/ images plus two synthesized carriers before and after, and
the record is byte-identical.

Measured on a 1536x2816 image -- decoder 0.280s to 0.016s, warm identify()
1.757s to 1.365s with both arms timed in one process.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 11:59:56 -07:00
Victor Kuznetsov 2d03a00a39 Merge pull request #80 from wiltodelta/dependabot/uv/minor-and-patch-2677c98c32
Bump the minor-and-patch group across 1 directory with 5 updates
2026-08-12 20:49:38 -07:00
dependabot[bot] acbf6afac3 Bump the minor-and-patch group across 1 directory with 5 updates
Bumps the minor-and-patch group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [c2pa-python](https://github.com/contentauth/c2pa-python) | `0.37.2` | `0.37.5` |
| diffsynth | `2.0.18` | `2.1.0` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.26.0` | `1.27.0` |
| [packaging](https://github.com/pypa/packaging) | `26.2` | `26.3` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.1` | `0.16.2` |



Updates `c2pa-python` from 0.37.2 to 0.37.5
- [Release notes](https://github.com/contentauth/c2pa-python/releases)
- [Changelog](https://github.com/contentauth/c2pa-python/blob/main/docs/release-notes.md)
- [Commits](https://github.com/contentauth/c2pa-python/compare/v0.37.2...v0.37.5)

Updates `diffsynth` from 2.0.18 to 2.1.0

Updates `huggingface-hub` from 1.26.0 to 1.27.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.26.0...v1.27.0)

Updates `packaging` from 26.2 to 26.3
- [Release notes](https://github.com/pypa/packaging/releases)
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pypa/packaging/compare/26.2...26.3)

Updates `ruff` from 0.16.1 to 0.16.2
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.1...0.16.2)

---
updated-dependencies:
- dependency-name: c2pa-python
  dependency-version: 0.37.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: diffsynth
  dependency-version: 2.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: packaging
  dependency-version: '26.3'
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-12 22:57:56 +00:00
Victor Kuznetsov 25eaa7458c Support Python 3.11 through 3.14 2026-08-12 15:52:11 -07:00
Victor KuznetsovandClaude Opus 5 58146dbc6d Say what the hosted service is instead of offering it as a fallback
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:24:56 -07:00
Victor KuznetsovandClaude Opus 5 b08899dc00 Note that the package name is the published distribution
Renaming it strands every pip install and every pinned dependency on it,
including the ComfyUI node package, and the console script has the same
property since users have it on PATH. Recorded at the name itself, where
whoever considers changing it will be looking.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 13:52:48 -07:00
Victor Kuznetsov b1319b1d5a Release 0.26.3 2026-08-09 21:15:15 -07:00
Victor Kuznetsov 353bc5f12c Harden TrustMark detection with an official fixture 2026-08-09 21:08:09 -07:00
Victor KuznetsovandClaude Opus 5 370556ffe6 Ignore other harnesses' generated config before it appears
These directories are empty today, so git shows nothing and the gap is
invisible. The moment the sync writes into one, a routine running
git add -A commits another harness's generated config, and the harness
audit then fails after the fact rather than before it. Paths are the
canonical untrackedConsumerPaths list, anchored to the repo root so a
same-named source file deeper in the tree is not caught.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 17:04:05 -07:00
Victor KuznetsovandClaude Opus 5 14a27d224d Drop the settings.json that only repeats global defaults
All four plugins it enabled are enabled globally, and its allow list says
nothing under the global defaultMode: bypassPermissions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 16:51:37 -07:00
Victor Kuznetsov 015df40f49 Release 0.26.2 and document distribution surfaces 2026-08-09 00:27:10 -07:00
Victor Kuznetsov f9beef365f Remove conda distribution support 2026-08-08 21:18:10 -07:00
Victor Kuznetsov 9b656513f1 Improve C2PA attribution and ByteDance metadata handling 2026-08-08 19:34:40 -07:00
github-actions[bot] ab03ecc507 Sync conda recipe with v0.26.1 2026-08-07 01:21:28 +00:00
Victor Kuznetsov 14b7247e0b Fix SynthID provenance evidence and release 0.26.1 2026-08-06 18:17:40 -07:00
Victor Kuznetsov d3eeba3a33 Merge pull request #78 from wiltodelta/research/video-synthid-quality-groundwork
Make the video SynthID operating point measurable and hard to move silently
2026-08-05 22:35:17 -07:00
Victor KuznetsovandClaude Opus 5 613b349358 Cover the fidelity probe, and run it where ffmpeg actually exists
The probe shipped with no tests while every sibling research script has them,
and it needs neither a model nor a GPU, so nothing justified the gap. Its whole
output is a ranking and a mispaired comparison still prints a plausible number,
so the pairing tests carry more weight than the metric ones.

The centerpiece builds a correctly phased and a wrong-phased delivery of the
same clip. Both have six frames, so the frame-count check the probe used to rely
on passes for both; the misaligned one scores near 9 dB against 40+. A separate
assertion pins that the probe binds the engine's sampler rather than a copy,
which is the contract a same-phase private copy would otherwise satisfy
silently. This file now holds the only constraint on that sampler's phase.

CI ran none of this. The test matrix installs no ffmpeg, and the job that does
install it ran one class from another file, so five of these tests would have
skipped on every runner. The video-e2e job now collects this file too.

Recorded in .claude/rules/development.md: mutating the sampler's >= to > proves
nothing, because the 1e-9 epsilon makes both branches agree on exact equality.
The phase or the period is what has to move. That inert mutation passed a green
suite here before the real one failed five tests.

Dropped the downscale test's PSNR bound: solid colors survive both the downscale
and the upscale exactly, so any score there pins the local ffmpeg's chroma
rounding rather than the probe. Geometry is the assertion that means something.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 22:24:49 -07:00
Victor Kuznetsov acd7092477 Merge remote-tracking branch 'origin/main' into research/video-synthid-quality-groundwork 2026-08-05 21:46:19 -07:00
github-actions[bot] e08644dcf2 Sync conda recipe with v0.26.0 2026-08-06 04:23:38 +00:00
Victor Kuznetsov 1a38c556f8 Merge pull request #77 from wiltodelta/claude/image-metadata-detection-benchmark-0bb6ec
Add versioned metadata and pixel evidence transports
2026-08-05 21:19:32 -07:00
Victor Kuznetsov a83952e375 Add versioned forensic metadata transports 2026-08-05 21:10:55 -07:00
Victor KuznetsovandClaude Opus 5 1124c591be Match the vendor registries against metadata, not coded pixels
The registries are raw substrings and the shortest tokens are four and five bytes
(`Bria`, `Adobe`, `Canva`). Over a megabyte of compressed pixel data such a sequence
turns up by chance: `Bria` matched inside the entropy-coded scan of 4 of 14,707
corpus JPEGs, in none of which the manifest names Bria. The rate is what a four-byte
pattern predicts on that corpus, and the Bria entry asserts AI, so a chance match can
declare an image AI-generated rather than merely mislabel its signer.

`_metadata_region` gives the registry scans the container's metadata: JPEG marker
segments before the coded scan, PNG chunks other than IDAT, both trailers, and
whatever `scan_head` appended past the window. Every other check keeps the full
buffer -- their markers are long and distinctive. A container that does not parse is
returned whole, since dropping real evidence to avoid a chance match is the wrong
trade. `c2pa_marker_in` already refuses a bare `c2pa` substring for this reason;
this is the same defence for the registries.

Verified the way the rules require for a change that MOVES a verdict: over all 48,905
corpus images, exactly one file changed, the one named in advance, from
"C2PA Content Credentials (Bria Artificial Intelligence)" to "(unknown signer)".
Record-path parity is 0 disagreements, down from 75 when this work started.

The audit's own baseline comparison is fixed here too. It compared confidence and
signals only, and so reported "0 changed" for the run whose single intended
correction was a watermark line -- the change it exists to show.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:10:55 -07:00
Victor KuznetsovandClaude Opus 5 9a29dcac8a Keep the pixel forensics in the library, drop the ai-score tooling
`scripts/ai_score.py` and the dataset scanner that fed it are gone: the detector
they trained is not something this project runs, and the corpus lived outside the
repository anyway. Nothing else referenced them.

The scanner's pixel layer was worth keeping, so it moves into the package as
`pixel_evidence.py` -- six families of scale-robust statistics (block-DCT histograms
and Benford deviation, FFT band energies and CFA peaks, high-pass residual, error
level, gradient, colour) measured in a single shared decode. The arithmetic was
verified against the scanner over 60 corpus images, families and artifacts alike,
before the scanner was removed; that comparison is no longer possible, which is why
the tests now pin behavior instead: determinism, empty-not-wrong on images too small
for a family, and one failing family not taking the others with it.

It has no consumer. Nothing in the package reads it, and the module says so.

`artifacts=True` returns the spatial layer -- perceptual hash, 128px thumbnail,
coarse ELA/residual/phase maps. Those identify the source image rather than describe
it, so they are opt-in and separate: everything else is a scalar or a fixed-length
histogram nothing can be reconstructed from.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:10:55 -07:00
Victor KuznetsovandClaude Opus 5 b5da0510c9 Release 0.26.0
Updates the three version sources the release doc names -- `pyproject.toml`,
`__init__.py`, and the root package entry in `uv.lock` -- and carries the marker
simplifications uv produced when it re-resolved the lock.

The release itself is not started here: the tag, push, and GitHub Release are the
remaining steps, and PyPI publishing triggers on the published Release rather than
on a tag push.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:10:38 -07:00
Victor KuznetsovandClaude Opus 5 bebff368fc Decide the SynthID proxy in the verdict, where both extractors meet
A full-corpus audit of the record path against the file path found 75 of 48,905
images disagreeing, and 74 were one gap: the SynthID byte scan for containers whose
manifest no parser reaches lived in `get_ai_metadata`, an extractor the record path
does not run. The record silently reported no SynthID for images `identify` flagged.

Moving the scan into `identify_from_evidence` fixes it by construction rather than by
copying the rule into a second extractor -- the same shape `soft_binding` already
uses. Its byte checks mirror `metadata.synthid_source` literally instead of reusing
the broader `has_c2pa` / `c2pa_source_kind` derived above, so the file path's answers
do not move: verdicts over a 4,000-image sample are byte-identical.

`scripts/record_parity_audit.py` is the audit itself, now repeatable. It walks a
dataset, judges every image through both seams with the record round-tripped through
JSON, and reports disagreements by field and by signal. The rule in
`.claude/rules/development.md` says to re-run both sides of this seam after changing
either; this is what to run.

Both timing and audit scripts now put the package's OWN `src` on the path. From a
worktree an editable install resolves to the main checkout, so the audit imported a
different tree than the one under test -- the failure the same rules file warns about,
reproduced within an hour of writing it down.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:10:38 -07:00
Victor KuznetsovandClaude Opus 5 2668f1302d Read WebP metadata past the scan window and surface C2PA reader failures
Three gaps found while measuring the record path against the file path, each one
a signal the library could not see:

WebP stores `XMP ` after the pixels, so on any WebP above the scan window a fixed
read stops short of the label. `_riff_late_metadata` steps over the coded image to
reach it, the RIFF analogue of the existing PNG and ISOBMFF readers. Three corpus
files hid an IPTC "Made with AI" tag and a C2PA `trainedAlgorithmicMedia` there.
The decoder-backed fallback now covers only what it is actually for -- metadata the
raw bytes do not spell, such as a compressed PNG `zTXt` packet.

A C2PA reader failure returned the same `None` as a file with no manifest, so a
verdict could fall back to the raw byte scan with no trace anywhere. Failures now
log at warning and only genuine ones do: a file without credentials never reaches
that branch, and an unsupported container is demoted to debug through the reader's
own `C2paError.NotSupported`. The first corpus run with it found a truncated PNG.

`scan_dataset.py` never registered the pillow-heif opener it declares as a
dependency, so every HEIC was scanned as unreadable -- no EXIF, and a pixel layer
that was 397 of 406 features NaN instead of 136.

`_riff_late_metadata` caps its total like `isobmff.scan_c2pa_region` does. Clamping
each chunk to the bytes remaining is not enough on its own: one chunk can declare a
length spanning most of the file, and this runs on the memoized verdict path over
images from arbitrary sources.

Also lands `identify_metadata_record` and `ProvenanceReport.to_dict()`, the
one-call entry point and the versioned JSON contract for the record path.

Record-vs-file equality holds over 3,478 corpus images, and the eight files these
fixes recovered still report AI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:10:38 -07:00
Victor KuznetsovandClaude Opus 5 0c5961a0ed Add a portable metadata record so collection and verdict can run apart
`collect_metadata_record` returns a JSON-safe record carrying an image's
provenance metadata regions -- never its pixels -- and the existing
`evidence_from_metadata_record` + `identify_from_evidence` build the verdict
from it without opening the file. The contract is equality with
`identify(path, metadata only)`, verified over the tracked fixtures and over a
local corpus of 3,478 images (every file carrying a rare signal, plus a random
slice): zero differences.

Three placements defeated earlier drafts and each is now a rule with a test:
the `scan_head` buffer is the head CONCATENATED with late metadata, so a
structural walk must read the raw head instead; Samsung splits its evidence
between a post-EOI trailer and the coded scan; and PIL's info keys must be
emitted in the file path's candidate order, since the first token match wins.

Also fix a real detection gap found while establishing that equality: a label
the decoder can read but a raw byte scan cannot -- a compressed PNG `zTXt`
packet, or a WebP XMP chunk past the scan window -- was invisible to
`identify`. Eight corpus files carrying a China TC260 AIGC label or an IPTC
"Made with AI" tag were reported as no signal at all.

`scripts/detection_timing.py` and its report script measure the metadata path
per method; they write outside the repository and are read-only over a dataset.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:10:38 -07:00
Victor KuznetsovandClaude Opus 5 8fe0b0110f Make the video SynthID operating point measurable and hard to move silently
The shipped profile was certified by one oracle row, but only noise_std was
pinned: long_side and fps -- two thirds of what the verifier was actually shown
-- could move with a green suite. The test now derives the pin from
data/evaluations/video-synthid-oracle.csv, so a default without a certifying row
fails.

The certified profile is a perturbation-to-signal ratio, not a bare noise_std.
sd-vae-ft-mse publishes no scaling_factor key, so 0.18215 comes from the
AutoencoderKL class default under an upper-unbounded diffusers pin. The loader
now gates that value, carries it on VideoVaeRuntime, and passes it into encode
and decode so the validated value is the applied value. video_synthid_sweep.py
loads through the same function: the harness producing the certified rows was
the one path exempt from the gate it exists to feed.

psnr_db is measured against the already-resized frame and before the encoder, so
it cannot see the downscale, the decimation, or the codec, and no in-loop metric
can. scripts/video_fidelity_probe.py scores the delivered file end to end,
streaming the way the engine does and sharing its frame-selection rule rather
than copying it -- a frame-count check cannot catch a rule that reorders frames
without changing how many.

The manifest gains source geometry, vae, track, verbatim verdict and session
fields. The two 2026-07-31 rows keep them empty: they were never recorded and
are not recoverable. Verdicts now have four states, because the verifier's
unclear reading logged as not_detected is the silent regression the manifest
exists to prevent.

docs/video-synthid-quality-research.md records the research behind this: the
noise axis is worth about 2 dB and is nearly exhausted, resolution is the real
prize but is an uncertified destruction axis rather than a free win, and every
proposed autoencoder swap was refuted. First local measurements included.

Verified: engine output is byte-identical before and after the refactor on a
locally built clip, at noise_std 0.00 and 0.15.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 11:17:38 -07:00
Victor Kuznetsov f481e6f944 Merge pull request #76 from wiltodelta/refactor/invisible-options-are-engine-knobs
Make InvisibleOptions engine knobs only and pin the forwarding
2026-08-04 19:38:07 -07:00
Victor KuznetsovandClaude Opus 5 4a896cd4b5 Make InvisibleOptions engine knobs only and pin the forwarding
`InvisibleOptions` promises in its docstring that every default mirrors
`InvisibleEngine`. Two fields made that promise cost something to keep: `force` is
not an engine parameter at all, and `controlnet_scale` was a third spelling of the
engine's `controlnet_conditioning_scale`. The mirror test carried an exception
table for each. This removes both, so the comparison needs no exceptions -- a field
that needs one is a field that belongs somewhere else.

`force` decides WHETHER the engine runs, which is settled before it is built, so it
joins `backend` and `sensitivity` as a parameter of `remove_all` and `remove_batch`
and is threaded to `_run_invisible` as its own argument. `controlnet_scale` takes
the engine's own name; the click option stays `--controlnet-scale` and is now
translated exactly once instead of at three forwarding sites.

Safe to do today: both symbols landed after 0.25.0 and have never been published.

The forwarding turned out to be the weaker half. A defaults comparison cannot see a
hardcoded literal at the seam, and `_run_invisible` passed the entire suite with
`controlnet_conditioning_scale` pinned to a constant. Each of the two knobs also
reaches the engine through TWO paths -- `remove_all` versus `remove_batch(mode="all")`
for `force`, `_run_invisible` versus `_batch_engine` for the scale -- and guarding one
left the other free to hardcode with a green suite. So:

  * `test_every_field_arrives_at_the_engine_with_the_caller_s_value` drives the real
    seam with all 13 fields set off their defaults; mutating any one of them to its
    default fails it.
  * `test_force_reaches_the_scrub_gate_in_every_scrubbing_mode` and
    `test_batch_controlnet_scale_flows_to_the_cached_engine` are parametrized over
    both modes, so neither path can be pinned alone.

Also fixes an order-dependent test surfaced by the added tests reshuffling the xdist
shards. `test_visible_path_decodes_file_once` counted every `image_io.imread` in the
process, but the Gemini engine loads its own bundled capture assets on first
construction, so the count was 3 on a cold engine and 1 on a warm one and the test
passed only when an earlier test happened to build the engine first. It now counts
decodes of the SOURCE, which is the invariant it exists for, and still fails when the
shared decode is broken. The production path was never wrong: the source bitmap is
decoded exactly once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 19:33:54 -07:00
Victor KuznetsovandClaude Opus 5 13095fb45c Verify every doc claim against the source and fix what drifted
Every code-referencing claim in the docs, the README and the rules files was
checked against src/, and each finding was re-derived independently before it
was applied. 35 held, 5 were false positives.

Two of them were code, not text. `InvisibleOptions` promises in its docstring to
mirror `InvisibleEngine`, and two defaults had silently stopped:
`max_resolution=None` reached `_target_size`'s `max_resolution > 0` and raised
`TypeError` on every library call that left the options alone, and
`cpu_offload=True` made a library run slower than the identical CLI run. Both are
fixed, and `TestInvisibleOptionsMirrorTheEngine` compares the two signatures
field by field rather than pinning the two values that happen to be known. A
companion assertion in `TestTargetSize` reads the engine's own declared default,
so a drift on the engine side -- which the mirror check alone would accept,
because both sides would still agree -- fails too.

The user-facing docs: README called `invisible` GPU-optional where it raises
without CUDA, and gave the image `metadata` command `video metadata`'s output
rule, promising the source survives a command that overwrites it. Yuanbao was
missing from the supported-mark list. `veo` was listed among the video policies
that require a run anchor, though its row sets no `anchor_iou`.
`known-limitations` called ControlNet the default profile and contradicted
itself ninety lines below. An unescaped pipe truncated the `hailuo` table row.
The `dev` extra, the CI shape, ffmpeg's role, the sdist boundary and the
strength-curve range were corrected, and `remove_all`/`remove_batch`, the pill
gate, `erase --keep-metadata` and `all`'s CUDA failure mode were documented.

Research notes that described removed modules, extras and flags in the present
tense now say so once in the page banner instead of sentence by sentence, which
covers the whole page rather than the lines that happened to be noticed, and one
fixture is referred to by role rather than by name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 11:38:04 -07:00
Victor Kuznetsov 0261ec1bd7 Merge pull request #75 from wiltodelta/refactor/detection-path-and-library-pipeline
Collapse the duplicated detection path and lift the image pipeline into the library
2026-08-04 10:21:43 -07:00
Victor KuznetsovandClaude Opus 5 78d9e81d0f Collapse the duplicated detection path and lift the image pipeline into the library
The visible-mark path had grown three copies of one ladder sweep, four
near-identical `detect` arms, and four hand-rolled `footprint_mask` overrides;
mark knowledge sat in five hand-maintained tables across three modules; and the
flagship `all`/`batch` pipeline existed only in cli.py, written twice with
divergent behavior.

Detection is now one measurement. `_ladder_best` replaces the three sweeps,
`_scan`/`_verdict` replace the four arms, and the winning box travels to the
mask on `TextMarkDetection.match_box` instead of being swept a second time.
`detect_both` returns the strict and relaxed verdicts from one scan, which
halves the arbiter's perception cost (260 -> 130 matchTemplate calls on a 2048²
image, verdicts identical field for field). A per-mark demotion goes in the new
`_post_gate` hook, never in a `detect` override -- an override is invisible to
the single-pass path, which is how the RunningHub and Yuanbao anchor gates
briefly stopped applying.

Everything about a mark is now one registry row: product, label regime, the
platform sentence `identify` reports, the metadata signals that confirm it, and
its TC260 producer codes. `identify._VISIBLE_MARK_PLATFORM`, the signal mapping
in `api.visible_provenance`, `_PRODUCT_OF` and the pill veto are derived from
those rows.

`api.remove_all` / `api.remove_batch` are the library form of the `all` and
`batch` commands; the CLI is a wrapper that owns console text and exit codes.
Progress is a `(stage, detail)` pair of stable tokens, so the CLI keys its
wording off structure rather than parsing the library's prose back.

Two intentional behavior changes, both verified against a recorded 811-image
sample of detector verdicts, removal-mask hashes, arbiter decisions and
`identify` reports:

  * A TC260 label now relaxes the vendor its `ContentProducer` names rather than
    ByteDance's pair on every China-AIGC image. 333 of 811 samples move; on 185
    of them the previously relaxed pair was simply the wrong vendor, and the
    mark actually present never reached the relaxed gate its own
    `provenance_ncc_factor` was calibrated for.
  * A confident LibLibAI detection suppresses the Jimeng pill, like every other
    TC260 product's mark. It was registered alongside RunningHub and Baidu, both
    of which were added to the hand-written veto list, and it was not. 1 sample
    moves, and it is exactly the co-firing case.

Nothing else in that record changes: detector verdicts, mask hashes and
`identify` verdicts are byte-identical, and all 200 calibration constants are
untouched.

Also: `aigc_label` and friends plus `extract_c2pa_info` are memoized on
(path, mtime_ns, size) -- size because this package rewrites in place; the
native TC260 container readers route on magic bytes instead of the file
extension, so a mislabeled AVI or FLV is no longer invisible; `identify` shares
one pixel decode between the DWT-DCT and visible stages (TrustMark keeps its own
Pillow decode, which is not substitutable); and the six `stabilize_*` video
wrappers collapse into one policy table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 22:49:45 -07:00
Victor KuznetsovandClaude Opus 5 480f478484 Refresh the dependency lockfile
Patch and minor only: cffi, coverage, hf-xet, modelscope-hub, typer. The majors
the resolver held back (numpy 2, opencv 5, tokenizers 0.23) are the ones
.github/dependabot.yml already documents as blocked upstream.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 22:49:18 -07:00
github-actions[bot] cb4590c61f Sync conda recipe with v0.25.0 2026-08-04 00:03:04 +00:00
Victor KuznetsovandClaude Opus 5 38399e4eb0 Release 0.25.0
Breaking, on top of the released 0.24.0.

Removed from the CLI: --model, --steps, --guidance-scale, --device, and the
deprecated --auto. Removed from InvisibleEngine and WatermarkRemover: the
model_id, num_inference_steps and guidance_scale parameters, remove_watermark_batch,
and the remover's region/region_feather path. Each pinned a value the two
remaining profiles fix -- the model stack, the per-stage distilled schedule,
CFG 1.0, CUDA -- so their only outcome was an error raised several frames below
the caller.

Also removed: the public remove_ai_watermarks.upscaler module, the
--min-resolution and --upscaler options and the published esrgan extra (0.24.0
shipped them unreachable); the pytorch-xpu index; and "diffusion" from the "all"
extra, which qwen-zimage already pulls.

Behaviour changes a caller can see:
- invisible-watermark removal now requires the qwen-zimage extra, not diffusion.
  Both profiles run the DiffSynth Z-Image face stage, so a torch+diffusers-only
  environment used to pass the availability gate and then die there. Every
  install hint names qwen-zimage now.
- get_device() answers cuda or cpu only, and the CUDA-only refusal names the
  resolved device rather than the raw argument.
- adaptive_polish is tri-state. Unset follows the profile (off for qwen-zimage,
  on for sdxl-zimage) and is resolved inside the engine, so a library caller and
  a CLI caller on one profile now produce the same pixels; they did not before.

ComfyUI node 0.1.15 is already published and tracks this surface.

pre-commit: 1) maintain.sh - exit 0 (1093 tests, Pyright 0 errors, no
vulnerabilities); 2) /simplify - n/a, version bump only; 3) docs sync - version
appears in pyproject.toml, __init__.py and uv.lock, all three updated; 4)
CLAUDE.md - no rule change

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 16:59:10 -07:00
Victor KuznetsovandClaude Opus 5 2f72257996 Fix what the verification pass found in the knob removal
An adversarial review of 52b2c11 (five independent audits, each finding put to
two skeptics, plus a completeness critic) found four defects that commit
introduced and several stale claims it should have caught.

The install hint no longer installs -- again. Folding five hints into one
INVISIBLE_EXTRA constant dropped the shell quoting the originals had, so the
printed remediation was `pip install remove-ai-watermarks[qwen-zimage]`. Bare
brackets are a glob in zsh, the macOS default shell: it dies with "no matches
found" before pip runs. That is the exact failure 52b2c11 existed to stop
producing, reintroduced in a different form by a bulk replace. The constant is
quoted now, and a test asserts the quotes rather than the bare substring -- the
old assertions passed either way, which is why nothing caught it.

Three tests were not guarding what they claimed:

- The commit's headline behaviour change, per-profile polish resolution inside
  the engine, had no test at all. Rebinding resolve_adaptive_polish to the
  pre-commit `bool(value)` left the full suite green. Now covered by a test that
  drives the real engine and observes whether humanizer.adaptive_polish ran;
  that mutation now fails it.
- TestAvailability still asserted the pre-commit (torch, diffusers) contract, so
  in a diffusion-only environment it was simply wrong, and comparing each gate to
  a tuple copied from itself could never catch the two gates disagreeing -- the
  drift the shared REMOVAL_MODULES was introduced to prevent. Replaced with a
  test that simulates each module's absence and requires BOTH gates to close.
- Both CUDA-refusal guards skipped in every environment, including CI: they were
  gated on the diffusion stack, which no CI job installs. The refusal fires
  before any torch attribute is read, so they now run everywhere; only the dtype
  assertion keeps its skip.

Also: the retired-knob test covered `invisible` but not `all` or `batch`, though
all three declared those options separately; and smoke_matrix.py still called
remove_watermark(region=...), a parameter 52b2c11 deleted, with the resulting
TypeError swallowed into a skip by a broad except.

Stale documentation the previous sweep missed: known-limitations still described
an MPS out-of-memory fallback and a lighter-pipeline escape that no code can
produce; module-internals declared Canny thresholds of 100/200 as a compatibility
contract while the code uses 13/64, attributed enable_model_cpu_offload to
deleted profiles, and still warned that the engine and CLI defaults differ (this
commit's predecessor made them identical); cli.md gated `all` on the `diffusion`
extra; python-api claimed "cuda" was the only accepted explicit device when
"auto" is too. The claim that `device` is not a parameter was wrong in both
module-internals and .claude/rules/development.md -- it is one, deliberately, and
now says so. `--cpu-offload` help and the pipeline's CUDA guard both still
pointed at MPS.

Not fixed here, reported instead -- both are outside this repo:
- ComfyUI-remove-ai-watermarks nodes.py:332 passes num_inference_steps and
  guidance_scale (plus min_resolution/upscaler from bf4bfc1). distribute.yml's
  comfyui job runs on every release and fails the release if the node sync fails,
  so 0.25.0 needs that node updated first.
- raiw-app modal_app.py:422-425 forwards the same two kwargs into
  remove_watermark. Latent: it is pinned to 1a77e24 and nothing supplies a value
  today, so it fires on the next pin bump.

pre-commit: 1) maintain.sh - exit 0 (1093 tests, Pyright 0 errors, no
vulnerabilities); 2) /simplify - not re-run, this commit is the applied output of
a five-dimension adversarial review; 3) docs sync - grepped MPS/mps, the extras
names and every symbol touched across README, docs/, scripts/, .claude/; updated
6 docs; 4) CLAUDE.md - corrected the device claim in .claude/rules/development.md
and added the shell-quoting rule

Verified by execution, not assertion: smoke_matrix --quick 51 pass / 0 fail,
_knob_rows driven directly 10 pass / 0 fail / 7 skip (no CUDA), the install hint
rendered and round-tripped through zsh, and each new test confirmed to fail
under the mutation it is meant to catch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 16:31:39 -07:00
Victor KuznetsovandClaude Opus 5 52b2c115e8 Delete every knob the fixed profiles cannot honor
The CLI still advertised --model, --steps, --guidance-scale, --device and a
deprecated --auto. Each pinned a value the two surviving profiles fix -- the
model stack, the per-stage distilled schedule, CFG 1.0, CUDA -- so the only
outcome any of them had was an error raised several frames below the caller,
under a message naming an internal profile. A flag whose sole result is a
refusal is worse than no flag: it advertises a capability that does not exist,
and it lets a wrapper thread a value that will silently do nothing. They are
gone from the parser, from InvisibleEngine, and from WatermarkRemover, so the
failure is now a TypeError or a Click "No such option" at the point the caller
can act on.

The install hint was wrong in the same way. is_available() checked torch and
diffusers, then told the user to install [diffusion] -- which contains neither
DiffSynth nor the Z-Image face stage both profiles run. Following the advice
produced a second, different failure. The module list and the extra name now
live once in watermark_profiles (REMOVAL_MODULES, INVISIBLE_EXTRA) and are read
by both the CLI gate and the remover's precondition, which cannot drift apart
because they are the same tuple.

The adaptive-polish default moved out of the argument parser. It was resolved by
reading Click's parameter source, which put per-profile data in the CLI layer,
left the engine declaring the opposite default (False vs True) so a library
caller and a CLI caller on one profile got different output, and lost the polish
entirely for anything that supplies the flag non-interactively. The flag is now
tri-state (default=None) and resolve_adaptive_polish owns the per-profile
answer. The seed follows the same rule: the CLI stopped pre-resolving it.

Dead code removed with it: six scan_*_video wrappers and the _scan_video helper
none of them had a caller for, PNG_METADATA_KEYS, feather_region_composite and
the remover region path that was only reachable from a no-caller convenience
wrapper, remove_watermark_batch on both layers, try_empty_device_cache, the
_generate/_run_qwen_zimage pass-through pair, self.model_id, and the _internal
PEP 562 shim that no caller ever went through. get_device now answers cuda or
cpu only: mps and xpu travelled one frame to the same CUDA-only refusal while
costing a device probe each, and that refusal now names the resolved device, so
device=None on a CUDA-less host says 'cpu' rather than 'None'. The XPU wheel
index went with them.

Docs: README, cli, installation, python-api, supported-signals,
known-limitations and module-internals all still described the removed profiles,
the CPU/MPS/XPU ladder, a `default`->`sdxl` alias, and the wrong extra.
known-limitations still listed the retired SDXL strength ladder as current.
scripts/smoke_matrix.py and real_examples_e2e.py drove --device mps.

Next release is 0.25.0, not a patch: this removes public parameters and
narrows a published extra on top of the released 0.24.0.

pre-commit: 1) maintain.sh - exit 0 (1091 tests, Pyright 0 errors, no
vulnerabilities); 2) /simplify - 4 agents, 11 findings applied, 2 skipped
(dropping the `device` parameter entirely, which raiw-app pins; folding
diffsynth into the `diffusion` extra, which video-only callers do not need);
3) docs sync - grepped every removed identifier across README, docs/, scripts/,
.claude/; updated 9 docs; 4) CLAUDE.md - added the no-error-only-knobs rule to
.claude/rules/development.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 15:38:40 -07:00
Victor KuznetsovandClaude Opus 5 bf4bfc1ab7 Remove the unreachable ESRGAN upscale chain
The min-resolution floor lifted small inputs toward SDXL's ~1024 training size,
and Real-ESRGAN was an optional way to do that lifting. Both surviving profiles
run at native geometry, so the engine forced the floor to 0 on every path; the
floor never fired, `upscaling` was never true, and nothing downstream of it could
execute. Gone: upscaler.py, _esrgan_upscale, the min_resolution and upscaler
parameters, _target_size's floor branch, --min-resolution, --upscaler,
_warn_if_esrgan_unavailable and the `esrgan` extra. max_resolution stays and is
now the only lever on geometry; it can only scale down.

scripts/smoke_matrix.py was the one live consumer and neither gate saw it -
Pyright is scoped to src/ and Ruff cannot resolve its function-local import - so
`--diffusion` would have died at import. Its knob rows were written for the
removed profiles besides (--pipeline sdxl, --steps 20, --guidance-scale 5.0,
--device mps), so they are rewritten rather than patched: most now assert a knob
is REJECTED, which is the coverage worth having when the CLI accepts a value the
library refuses several layers down. Accepted-knob rows skip without CUDA, so the
row count is host-dependent and verification-plan.md no longer claims a fixed 68.

This removes a public module, a CLI option and a published extra, so the next
release is 0.25.0, not a patch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 14:54:26 -07:00
github-actions[bot] 95a6964e04 Sync conda recipe with v0.24.0 2026-08-03 21:27:17 +00:00
Victor KuznetsovandClaude Opus 5 1a77e24b99 Stop the engine from supplying a model id the remover must reject
InvisibleEngine substituted DEFAULT_MODEL_ID whenever model_id was None. When
b0ca205 tightened the remover's fixed-stack check from `not in {None,
DEFAULT_MODEL_ID}` to `is not None`, that substitution turned every single
InvisibleEngine construction into a ValueError - including the deployed Modal
worker's setup(), which is how it was found.

The library suite missed it because these two are tested from opposite sides:
every remover test builds WatermarkRemover directly with model_id unset, and
every engine test mocks the remover away. Nothing exercised the seam between
them. TestEngineDoesNotFabricateAModelId now does, without a GPU.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 13:24:53 -07:00
Victor KuznetsovandClaude Opus 5 691f5d88c2 Skip the face-stage dtype guard without the qwen-zimage extra
CI's base test job installs the library without that extra, so the test died on
`import transformers` before reaching a single assertion. It monkeypatches the
Z-Image and SAM loaders rather than calling them, but the modules still have to
be importable to be patched, so pytest.importorskip is the right gate: the guard
still runs everywhere the extra is present and skips where it cannot.

Local runs could not have caught this - the dev environment has the extra.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 12:23:21 -07:00
Victor KuznetsovandClaude Opus 5 b0ca2054f6 Keep only the two-stage profiles and make CUDA a precondition
qwen-zimage becomes the default and sdxl-zimage the only alternative. The
controlnet, sdxl, qwen and default profiles are gone, and with them the CPU and
MPS paths for invisible-watermark removal: neither matched the two-stage
recipe's face preservation, so keeping them advertised a quality this library no
longer delivers. Visible-mark removal and every identify command still run
anywhere.

Retired names are rejected rather than remapped. Silently routing --pipeline
sdxl onward would run an old script at a different strength, on a different
model, at a different quality, and report success.

CUDA is now checked when the remover is constructed instead of when the model
loads. Auto-detection cheerfully returned mps on a Mac, so the failure arrived
several layers down, after the dependency check and the pipeline import, in a
message naming whichever internal pipeline happened to raise. _DEVICES collapses
to {"cuda"} and the cpu/mps float32 branch goes with it.

resolve_strength stays total. It briefly returned None for qwen-zimage, meaning
"ask the resolution curve", which pushed a branch onto both callers and left one
of the two strength policies outside the strength module; the CLI copy had
already grown an `or 0.0` guarding a path its own comment called unreachable. It
now takes the image size and answers for both profiles, so the displayed value
cannot drift from the executed one.

Deletion fallout removed with it: img2img_runner and progress.py (the MPS
recovery path and its progress monitor had no callers left), viable_steps, the
fp16 degenerate-output retry, the fp16 VAE fix, and the Qwen img2img call
builders. try_empty_device_cache moved into watermark_remover rather than
leaving a module whose docstring outlived its code. _HAS_DIFFUSERS routes
through optional_deps.module_available, which is what the rest of the library
uses and what correctly rejects a pruned namespace remnant.

--steps, --guidance-scale and --model now have exactly one legal value each and
are still accepted at parse time, then rejected in remove(). Their help text
says so, but validating them beside the option would be better.

Not addressed, and worth its own decision: invisible_engine forces
min_resolution to 0 for both profiles, so the --min-resolution floor, --upscaler,
_esrgan_upscale, upscaler.py and the esrgan extra are all unreachable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 12:16:14 -07:00
Victor KuznetsovandClaude Opus 5 3d43bac6a5 Load the face stage in its own dtype, not the global stage's
The sdxl-zimage profile crashed on every image containing a face. The remover
gives it torch.float16, because SDXL ships fp16 weights and an fp16-safe VAE,
and that dtype reached the inherited _load_zimage while _zimage_vram_config
hardcodes bfloat16 for its offload, onload and computation dtypes. Z-Image was
therefore built bf16 and handed fp16 latents, dying in the VAE with "Input type
(c10::Half) and bias type (c10::BFloat16) should be the same". Zero-face inputs
never enter _run_faces, so the profile passed every timing run it was given, and
its tests avoid model downloads, so nothing exercised the loader.

Every face-stage loader now reads _face_stage_dtype(), the computation dtype of
the VRAM config it is paired with. SAM is routed through it too: it never
crashed, since it casts its own inputs and leaves through .float(), but it read
the same field and would have re-landed the bug for the next profile with a
different global dtype. That field was never the global dtype on this profile
anyway - _load_sdxl hardcodes fp16 for its own ControlNet, VAE and pipeline - so
its only readers were face-stage code.

This also fixes a second instance transitively: the persisted prompt-embedding
cache restores payloads at the DiffSynth pipe's dtype, which was fp16 into a
bf16 stack before this change.

For qwen-zimage the whole change is a strict no-op. The remover already hands it
bfloat16, the same value _face_stage_dtype() returns, so production is
untouched; verified on an H100 against the deployed pin.

The guard asserts the dtype the Z-Image and SAM loaders actually receive rather
than comparing the accessor to the config it derives from, which would restate
the implementation and pass for any consistently wrong value. Both assertions
were mutation-tested against the pre-fix line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 09:41:52 -07:00
github-actions[bot] 40853ae102 Sync conda recipe with v0.23.0 2026-08-03 04:31:36 +00:00
Victor KuznetsovandClaude Opus 5 7234cf7f4d Release 0.23.0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 21:29:34 -07:00
Victor Kuznetsov db33823d8e Merge pull request #74 from wiltodelta/feat/sdxl-zimage-profile
Add the sdxl-zimage profile: the same recipe on an SDXL global stage
2026-08-02 21:27:54 -07:00
Victor KuznetsovandClaude Opus 5 a9e64b9628 Add the sdxl-zimage profile: the same recipe on an SDXL global stage
SdxlZImagePipeline subclasses QwenZImagePipeline and overrides only _run_global
and preload, so the face stage is inherited rather than copied and cannot drift
between the two profiles. A test asserts the shared methods are the same objects.

Four things are architecture-bound and swap with the model: the ControlNet, the
four-step distillation LoRA (SDXL-Lightning at its documented 1.0, not the
reference graph's 0.8, which belongs to a different LoRA), the sampler (Euler
trailing, no AuraFlow shift), and the latent grid at 8 px against Qwen's 16.

Strength is architecture-bound too, which is the easy mistake and cost two wrong
conclusions before it was caught. An SDXL global pass leaves SynthID at the
strength Qwen needs: through the Gemini app on a native 2816x1536 original, 0.154
is FOUND while 0.20, 0.25 and 0.30 are clean. The profile therefore takes a flat
vendor policy - OpenAI 0.15, Gemini 0.25, unknown following Gemini - rather than
resolution_adaptive_denoise, because flat values are what was measured and no
size dependence has been established for this stage.

requested_steps exists because the runtimes truncate differently: DiffSynth sets
sigma_start = denoising_strength and runs every requested step, while Diffusers
img2img truncates the step count, so four steps at 0.15 executes zero and returns
a bare VAE round-trip.

Also records both measured provider boundaries for the shipped qwen-zimage curve
- OpenAI detected at 0.06 and clean from 0.08, Gemini detected at 0.08 and clean
from 0.10 - together with the two low-resolution Gemini verdicts that explain why
the curve's sub-1 MP rungs are not under-driven despite looking short against a
boundary measured at 4.33 MP. The curve is left unchanged; nothing measured fails.

The profile is not deployed and not production-ready: every verdict so far comes
from one fixture and one seed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 21:23:43 -07:00
Victor Kuznetsov a4d8de4367 Merge pull request #73 from wiltodelta/docs/face-stage-compositing
Document how the face stage actually composites
2026-08-02 20:47:57 -07:00
Victor KuznetsovandClaude Opus 5 af51afb586 Document how the face stage actually composites
The parameter names invite a misreading. _run_faces regenerates the ENTIRE
expanded crop with Z-Image and only then cross-fades on a blurred SAM mask, so
the generation is conditioned on a fully noised neighbourhood and the pixels the
mask later discards were regenerated too.

Records the alternative that has never been tried here - passing the mask into
the sampler as a latent noise mask, so only masked pixels are denoised and the
edge transition happens inside the generation - because the face stage is the
largest measured quality contributor, worth 3.5 dB and 6.1 dB inside the face
boxes on the two fixtures that have one.

Also records that FACE_DENOISE_SCALE = 0.5 is coupled to this compositing choice
rather than independently calibrated: regenerating a whole crop and blending is
stronger than masked denoising, so changing the compositing without revisiting
the scale would move output strength by about a factor of two.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 20:44:20 -07:00
Victor Kuznetsov f51c5b09dd Merge pull request #72 from wiltodelta/docs/qwen-zimage-denoise-boundary
Record the qwen-zimage global denoise boundary against the Gemini oracle
2026-08-02 20:16:58 -07:00
Victor KuznetsovandClaude Opus 5 9b07b3f0ec Validate the bottom of the qwen-zimage denoise curve against Gemini
Downscaled a Gemini original to 0.57 and 1.40 MP and ran the deployed worker on
both, so the profile applied its own low-end strengths of 0.0896 and 0.1066. Both
come back clean in the Gemini app.

That rules out the failure mode the ladder raised: 0.08 failed at 4.33 MP, and the
curve sends sub-1 MP images to 0.084-0.094, which looked like it might mean small
Gemini uploads were under-processed in production. They are not, at these sizes.

Written as validation of the shipped curve rather than as evidence that the
boundary moves with resolution. 0.0896 sits inside the untested gap at 4.33 MP,
where only 0.08 and 0.10 were probed, so it may clear at both sizes; the direction
of any resolution dependence stays unproven. These are also downscales rather than
natively small Gemini outputs, which remain untested.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 19:53:51 -07:00
Victor KuznetsovandClaude Opus 5 84583dd479 Drop the unproven claim that Gemini removal is resolution-sensitive
The document asserted it in two places and recommended capping Gemini at 1536
with 0.30, or native-calibrating to ~0.35+. Nothing measured that, and the one
relevant measurement points the other way: the 2026-06-14 deployed-worker re-test
cleared Gemini at 0.15 on two NATIVE 2816x1536 images, the same rung as capped
1536. The document already recorded that as contradicting the "native >= 0.30"
guess, then kept the guess anyway in the historical-certification paragraph and
restated it as fact in the strength floors.

Replaced with what was measured, plus an explicit statement that the direction is
unproven and the low-resolution end has never been through the Gemini oracle on
any pipeline.

Also removes the same appeal from the qwen-zimage denoise-boundary note added
earlier in this branch, which had used the unproven trend as reassurance that the
bottom of the adaptive curve is safe. It is not reassurance; it is an open
question, and it is now written as one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 19:49:44 -07:00
Victor KuznetsovandClaude Opus 5 56d7aff3a1 Record the qwen-zimage global denoise boundary against the Gemini oracle
A ladder on one native 2816x1536 Gemini original at seed 0, verified through the
Gemini app, puts the boundary between 0.08 and 0.10: 0.154, 0.12 and 0.10 read
clean, 0.08 reads SynthID FOUND. Fidelity rises monotonically all the way down,
so 0.10 buys +1.54 dB whole-image and +0.98 dB inside the face boxes over the
0.154 the profile ships for that size.

Recorded with the two constraints that stop it being acted on directly. It
brackets rather than calibrates - one image, one seed, and shipping the lowest
clean rung means shipping at the measured cliff edge. And the untested end is the
bottom, not the top: every Gemini oracle fixture is 2816x1536, so the Google-side
certification only ever covered 0.154, while the curve sends sub-1 MP images to
0.084-0.094. The resolution trend already recorded in this document says lower
processing resolution needs less strength, which is the shape the curve has, but
that is an inference and no small Gemini original has ever been through the
oracle. Downscaling is valid test material since SynthID survives it by design.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 19:46:12 -07:00
Victor Kuznetsov 40cd122683 Merge pull request #71 from wiltodelta/feat/static-prompt-embedding-cache
Serve the static prompt embeddings from a persistent cache
2026-08-02 15:39:37 -07:00
Victor KuznetsovandClaude Opus 5 39523aba09 Serve the static prompt embeddings from a persistent cache
Both qwen-zimage stages prompt with module constants, and at CFG 1.0 DiffSynth's
PipelineUnitRunner reuses the positive embedding for the negative side rather
than encoding it, so exactly one embedding per stage is ever computed. Persist it
and neither text encoder has to be loaded at all.

Measured on an H100 volume: this drops 15.45 GiB (Qwen2.5-VL) and 7.49 GiB
(Z-Image) of an 87.6 GiB per-request read, worth a median 11.76 s and 4.10 s of
load time paired within five containers. A nine-face fixture returned
sha256 c8567e11077de32a both with and without the cache, so the output is
byte-identical and the provider-oracle clearance is untouched.

The cache key carries the cache version, model id, pipeline output params and the
exact prompt, so a model bump or a prompt edit recomputes instead of reading a
stale embedding. The write is atomic because a torn file must never read back as
a hit, and a miss after the text encoder was already dropped raises rather than
calling a model that is not loaded.

_model_cache_dir now prefers HF_HOME: on a scale-to-zero runner that is the only
persistently mounted path, so anything below it is re-derived every request. The
YuNet download follows the same root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 15:36:55 -07:00
Victor KuznetsovandClaude Opus 5 94730f77d3 Merge the v0.20.1 residency fix into main
Joins fix/qwen-vram-residency, which carried the global-stack residency
change on the v0.20.1 line that raiw-app pins. The change itself is already
on main via port/qwen-vram-residency-main, reapplied there because the
package layout moved under _internal/ in between, so this merge is history
only and its tree is identical to the commit before it.

Conflicts resolved in favour of main throughout, including dropping the
noai/watermark_remover.py the branch resurrected; that module now lives at
_internal/watermark_remover.py.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 21:53:40 -07:00
Victor KuznetsovandClaude Opus 5 36e3743289 Keep the qwen-zimage global stack resident on a card that can hold it
Port of the same change made on the v0.20.1 line, reapplied here because the
package layout moved under _internal/ in the meantime.

The mandatory Qwen stack was configured to offload to disk unconditionally.
DiffSynth implements that by dropping the weights to the meta device and
re-reading every parameter through its DiskMap on the next onload, and the
pipeline moves between text encoder, transformer and VAE on every pass, so
each generation paid a full model reload. That is the right trade on a
consumer card, where it is what makes a 20B model runnable at all, and pure
waste on a card that can simply hold the stack.

Residency is now resolved from total VRAM, mirroring how the optional
Z-Image face stack is already gated. Above the floor the config passes no
"disk" value anywhere, which is what actually disables the behavior:
DiffSynth latches disk_offload once from offload_dtype, so pointing every
device at CUDA while leaving the sentinel would keep both the meta-drop and
the re-read.

Measured on an H100: a warm global pass went from 37.3s at 0.8 GiB resident
to 2.2s at 28.7 GiB, with both stacks resident peaking at 48.0 GiB of 79.2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 21:22:52 -07:00
Victor KuznetsovandClaude Opus 5 a64d81f446 Keep the qwen-zimage global stack resident on a card that can hold it
The mandatory Qwen stack was configured to offload to disk unconditionally.
DiffSynth implements that by dropping the weights to the meta device and
re-reading every parameter through its DiskMap on the next onload, and the
pipeline moves between text encoder, transformer and VAE on every pass, so
each generation paid a full model reload. That is the right trade on a
consumer card, where it is what makes a 20B model runnable at all, and pure
waste on a card that can simply hold the stack.

Residency is now resolved from total VRAM, mirroring how the optional
Z-Image face stack is already gated. Above the floor the config passes no
"disk" value anywhere, which is what actually disables the behavior:
DiffSynth latches disk_offload once from offload_dtype, so pointing every
device at CUDA while leaving the sentinel would keep both the meta-drop and
the re-read.

The floor is set equal to the face floor rather than lower because that is
the configuration measured with both stacks resident; a tighter gate is
plausible but unvalidated. cpu_offload now forces both stacks to stream, so
a caller asking for low VRAM no longer gets the larger stack pinned anyway.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 21:06:13 -07:00
Victor Kuznetsov 9c71dc9c79 Refresh dependencies and compatibility guards 2026-07-31 17:24:05 -07:00
Victor Kuznetsov be7553b55f Merge pull request #70 from wiltodelta/feat/video-watermarks
Complete product video watermark pipeline
2026-07-31 17:00:56 -07:00
Victor Kuznetsov a6c0c1c6f0 Rewrite internal watermark pipeline and preserve behavior 2026-07-31 16:53:41 -07:00
Victor Kuznetsov 7c922e5133 Preserve one encoded frame per input frame 2026-07-31 15:31:00 -07:00
Victor Kuznetsov 7e63856a90 Split video encoding from audio muxing 2026-07-31 15:22:41 -07:00
Victor Kuznetsov 9b9f9c6ced Open source input before frame pipe 2026-07-31 15:07:15 -07:00
Victor Kuznetsov 7818e60824 Prevent ffmpeg stderr backpressure 2026-07-31 14:59:57 -07:00
Victor Kuznetsov 0fac8cd000 Bound ffmpeg filter threads 2026-07-31 14:43:01 -07:00
Victor Kuznetsov 166dab9d03 Bound ffmpeg video encoder threads 2026-07-31 14:33:50 -07:00
Victor Kuznetsov 7fb0e65455 Trace stalled Linux video encoder 2026-07-31 14:24:08 -07:00
Victor Kuznetsov 1fd859b58d Prevent ffmpeg pipe probe deadlock 2026-07-31 12:05:42 -07:00
Victor Kuznetsov f9a4e2ba10 Diagnose stalled video end-to-end test 2026-07-31 11:54:45 -07:00
Victor Kuznetsov 0e7a52820b Bound full-clip video CI runtime 2026-07-31 11:30:57 -07:00
Victor Kuznetsov de3ed75de5 Merge main into video watermark pipeline 2026-07-31 11:13:59 -07:00
Victor Kuznetsov 5767c1d7b3 Finalize video package boundary and product wording 2026-07-31 10:52:44 -07:00
github-actions[bot] 63d7cfa0a5 Sync conda recipe with v0.22.0 2026-07-31 17:43:40 +00:00
Victor Kuznetsov 6315b58628 feat: complete product video watermark pipeline 2026-07-31 10:42:39 -07:00
Victor Kuznetsov 08dc078d91 Release 0.22.0 with composable feature extras 2026-07-31 10:39:13 -07:00
Victor Kuznetsov 8975916b11 Harden automatic video watermark removal for product use 2026-07-30 18:36:04 -07:00
Victor Kuznetsov 9c9e81c756 Compact CLAUDE.md and route development guidance 2026-07-30 18:32:25 -07:00
Victor Kuznetsov fc177bacde Correct OpenCV Dependabot compatibility cutoff 2026-07-30 18:18:49 -07:00
Victor Kuznetsov 0c4b26bb77 Fix OpenCV Dependabot compatibility bound 2026-07-30 16:29:29 -07:00
Victor Kuznetsov 3d941069e5 Keep Dependabot updates within compatible dependency lines 2026-07-30 16:21:50 -07:00
Victor Kuznetsov 4e97076b22 Merge external metadata evidence into main 2026-07-30 12:49:06 -07:00
Victor Kuznetsov 07b678fa4c Add Hailuo, Kling, AVI, and FLV video coverage 2026-07-29 23:30:33 -07:00
Victor Kuznetsov 63cde7f32a Add streaming video SynthID regeneration 2026-07-29 22:03:23 -07:00
Victor Kuznetsov c9b8585105 Add video SynthID experiment harness 2026-07-29 21:03:46 -07:00
Victor Kuznetsov 41b6fd0682 Add Seedance and Dola video removal 2026-07-29 17:56:52 -07:00
Victor Kuznetsov f330dd9c94 Add video metadata and visible watermark removal 2026-07-29 17:28:11 -07:00
github-actions[bot] a67b209d66 Sync conda recipe with v0.21.2 2026-07-29 23:48:36 +00:00
Victor Kuznetsov 7c9bec0eba Ignore collector diagnostics in metadata evidence 2026-07-29 16:46:42 -07:00
github-actions[bot] 37f3b3b1c9 Sync conda recipe with v0.21.1 2026-07-29 21:34:11 +00:00
Victor Kuznetsov 4ef180541f Add external metadata evidence ingestion 2026-07-29 14:32:25 -07:00
github-actions[bot] c013704ded Sync conda recipe with v0.21.0 2026-07-29 20:17:17 +00:00
Victor Kuznetsov 408c4c9d5d Separate provenance extraction and detection, release 0.21.0 2026-07-29 13:13:39 -07:00
github-actions[bot] 28d23a3023 Sync conda recipe with v0.20.2 2026-07-28 19:14:22 +00:00
Victor Kuznetsov 59823b2eaf Recover malformed metadata containers, release 0.20.2 2026-07-28 12:12:26 -07:00
Victor Kuznetsov 1d333fe855 Use a dedicated ComfyUI release token 2026-07-26 12:11:35 -07:00
Victor Kuznetsov 5ea160f4ce Include ComfyUI in library releases 2026-07-26 11:56:09 -07:00
github-actions[bot] 643557b7bf Sync conda recipe with v0.20.1 2026-07-26 18:41:23 +00:00
Victor Kuznetsov e562b440db Improve Qwen face fidelity, release 0.20.1 2026-07-26 11:39:29 -07:00
Victor Kuznetsov ab525c1b90 Add global-only preload for qwen-zimage 2026-07-26 10:52:26 -07:00
Victor Kuznetsov 685f0679bd Automate conda recipe synchronization 2026-07-25 21:57:22 -07:00
Victor Kuznetsov d21dc62bbb Sync conda recipe with v0.20.0 2026-07-25 21:32:52 -07:00
Victor Kuznetsov 03cd00f132 Restructure documentation, validate metadata removal, consolidate assets 2026-07-25 21:08:04 -07:00
Victor Kuznetsov 214c9bb3e7 Add Tencent Yuanbao visible watermark removal 2026-07-25 20:52:15 -07:00
Victor Kuznetsov 952ce822fd Merge pull request #64 from salomonMuriel/feature/cuda-cpu-offload
Add --cpu-offload flag for low-VRAM CUDA cards
2026-07-25 19:08:24 -07:00
Victor Kuznetsov fbca7f7ed9 Sync CPU offload with dependency updates 2026-07-25 19:05:41 -07:00
Victor Kuznetsov 51d222f4ea Merge pull request #63 from wiltodelta/dependabot/uv/minor-and-patch-e56a0901c7
chore(deps): bump the minor-and-patch group across 1 directory with 5 updates
2026-07-25 19:04:06 -07:00
Victor Kuznetsov d6cf5f4e84 Sync dependency updates with current main 2026-07-25 19:01:40 -07:00
Victor Kuznetsov b8b69d93be Finish CPU offload support on current main 2026-07-25 18:44:44 -07:00
Victor Kuznetsov 88949c3c30 Isolate Qwen Z-Image tests from GPU extras 2026-07-25 17:13:04 -07:00
Victor Kuznetsov 43312b04eb Exclude Markdown from Ruff formatting 2026-07-25 17:10:47 -07:00
Victor Kuznetsov b0419a9d8a Merge Qwen Z-Image pipeline into main 2026-07-25 17:00:05 -07:00
Victor Kuznetsov b6204a24ce Add high-fidelity Qwen Z-Image removal pipeline 2026-07-25 16:53:22 -07:00
Victor KuznetsovandClaude cdec348349 Add v2 forensic features to the scorer, split without hash leakage
Train now dedups by sha256 (earliest date wins), splits by hash group,
and drops post-cutoff rows whose hash was seen in training, so neither
holdout reports memorized duplicates. The feature schema is explicit
(v1 = the original 97 structural features, v2 adds CFA peaks, DCT AC
histograms and JPEG quant/Huffman/scan stats), stored in the bundle and
read back at scoring time; legacy bundles default to v1. Vectors are
fixed-width with NaN padding, so a sparse record no longer shifts every
column. Scoring runs in batches instead of one predict_proba per record.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 19:24:39 -07:00
dependabot[bot] bb9f9c959d chore(deps): bump the minor-and-patch group across 1 directory with 5 updates
Bumps the minor-and-patch group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [pillow-heif](https://github.com/bigcat88/pillow_heif) | `1.4.0` | `1.5.0` |
| [c2pa-python](https://github.com/contentauth/c2pa-python) | `0.36.0` | `0.37.1` |
| [transformers](https://github.com/huggingface/transformers) | `5.13.1` | `5.14.1` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.23.0` | `1.24.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.15.21` | `0.16.0` |



Updates `pillow-heif` from 1.4.0 to 1.5.0
- [Release notes](https://github.com/bigcat88/pillow_heif/releases)
- [Changelog](https://github.com/bigcat88/pillow_heif/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bigcat88/pillow_heif/compare/v1.4.0...v1.5.0)

Updates `c2pa-python` from 0.36.0 to 0.37.1
- [Release notes](https://github.com/contentauth/c2pa-python/releases)
- [Changelog](https://github.com/contentauth/c2pa-python/blob/main/docs/release-notes.md)
- [Commits](https://github.com/contentauth/c2pa-python/compare/v0.36.0...v0.37.1)

Updates `transformers` from 5.13.1 to 5.14.1
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.13.1...v5.14.1)

Updates `huggingface-hub` from 1.23.0 to 1.24.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.23.0...v1.24.0)

Updates `ruff` from 0.15.21 to 0.16.0
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.21...0.16.0)

---
updated-dependencies:
- dependency-name: c2pa-python
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.24.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: pillow-heif
  dependency-version: 1.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.15.22
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: transformers
  dependency-version: 5.14.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-23 23:25:34 +00:00
Victor Kuznetsov 11afe68411 chore(release): v0.19.0 2026-07-23 16:20:27 -07:00
Victor Kuznetsov c11181b9b5 Detect and attribute Bria AI, fal.ai, and Apple Photos Clean Up
Corpus-mined vendor gaps (42k-file metadata scan, 2026-07-23):
- Bria AI signs C2PA as "Bria Artificial Intelligence" with source type
  empty (no trainedAlgorithmicMedia), so identify was completely blind;
  registered with asserts_ai like Dreamina.
- fal.ai ("fal - Features & Labels Inc.", fal-ai/<model> generators)
  was detected via the source type but never attributed; registered,
  also asserts_ai as a pure generative platform.
- Apple Photos Clean Up (Apple Intelligence object removal) was detected
  as a generic made-with-AI tag; now attributed as an AI edit via the
  photoshop:Credit marker, and the credit value joins
  AI_GENERATOR_TOKENS so removal strips it in parity.
Each new test was verified red without its fix.
2026-07-23 16:15:48 -07:00
Victor Kuznetsov eb544806f6 Add AI-generation scorer trained on metadata-labeled corpus
scripts/ai_score.py trains a gradient-boosted classifier on labels
derived from the scan_dataset metadata (C2PA AI generators, TC260,
local pipelines vs camera/screenshot/editor output) and scores every
file from pixel and container statistics alone, so metadata-stripped
files still get a score. Temporal holdout on the production corpus:
AUC 0.966 / AP 0.988. CLIP ViT-L/14 was evaluated as an alternative
and lost on both accuracy (0.82) and cost, so the model uses the
structural features the scanner already collects; scoring is CPU-only
and I/O-bound.
2026-07-23 16:01:35 -07:00
Salomón MurielandClaude Opus 4.8 39fdd59f6c Add --cpu-offload flag for low-VRAM CUDA cards
The invisible/SynthID diffusion pass loads the whole SDXL fp16 pipeline
into VRAM via `pipeline.to("cuda")`. On an 8 GB card the weights alone
(~7 GB) leave no room for activations, so the run OOMs and there is no
in-tool way to recover short of falling back to CPU (~9 min/image).

Add an opt-in `--cpu-offload` flag (default off) that calls diffusers'
`enable_model_cpu_offload()` instead: submodules are streamed to the GPU
on demand, dropping peak VRAM to roughly the largest single submodule at
the cost of per-step transfers. CUDA-only; a no-op on cpu/mps. Threaded
through `invisible`, `all`, and `batch` to keep the knob set identical
across the three, mirroring the existing `--device`/`--pipeline` options.

Measured on a GTX 1070 Ti (8 GB): `invisible --pipeline sdxl --cpu-offload`
runs the SynthID scrub on-GPU in ~2.5 min vs ~9 min on CPU, where the
default full-VRAM path OOMs.

Test drives the placement decision with a mock pipeline (no model/GPU),
gated on torch so it runs under the `gpu` extra and skips the core CI
matrix, consistent with the model-running test policy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 16:06:43 -05:00
Victor Kuznetsov dfe69772cd Polish the scanner after review: shared b64 helper, tri-state pixel mode
Route the pixel layer's base64 through the capped _b64 helper, replace
the pixels/pixels_full booleans with one pixel_mode parameter, record a
pixel skipped marker on oversized files, read EXIF from the already-read
bytes instead of re-reading the file, chunk the noise residual
convolution and replace the mgrid with 1D broadcast in the FFT features
(bit-identical values, ~300 MB less peak memory at 2048px), and fix the
raw-only docstring to account for the derived pixel layer.
2026-07-23 11:54:08 -07:00
Victor Kuznetsov 68fb3dded6 Fix numpy-free metadata mode after the pixel-layer merge
Guard module-level numpy touchpoints so the metadata-only mode imports
and runs without numpy installed (deferred annotations, lazy DCT basis);
pixel modes now exit early with a clear numpy requirement message.
2026-07-23 11:13:13 -07:00
Victor Kuznetsov 47ef60c2a1 Add optional pixel forensics layer to the dataset scanner
--pixels adds aggregate, non-reconstructable pixel statistics (block-DCT
histograms with Benford deviation, high-pass noise stats, FFT band
energies and CFA peaks, ELA stats, gradient and color histograms) plus
per-section timing for pipeline latency planning. --pixels-full adds the
privacy-lifting artifacts behind an explicit flag (perceptual hash,
128px thumbnail, coarse ELA/noise-residual/FFT-phase maps). Expensive
maps are computed once and shared between the scalar and artifact paths
(2.2x speedup vs the first split implementation). numpy is required only
for the pixel modes; the metadata-only mode stays numpy-free.
2026-07-23 11:06:45 -07:00
Victor Kuznetsov a981eb84b1 Make the dataset scanner raw-only and harden format sniffing
Strip all derived output: the library verdict, the XMP edit-trail
parse, and the IJG quality estimate. The script now collects raw bytes
plus mechanical container decodes only, is fully standalone (no
remove_ai_watermarks import), and resumes interrupted runs. Add GIF,
BMP, and TIFF magic bytes to sniff_format (GIF files in the wild were
reported as unknown). Validated on the full 42k-file corpus: 100%
coverage, zero error records.
2026-07-22 20:32:36 -07:00
Victor Kuznetsov df62800bf1 Add standalone dataset metadata/forensics scanner script
Single-file read-only scanner producing one JSONL record per image:
AI verdict via identify (metadata-only, no visible marks), full EXIF/
IPTC/XMP/PNG/JPEG/WebP/ISOBMFF inventories, C2PA store, JPEG encoder
forensics (quant tables, IJG quality, scan script, Huffman, subsampling),
XMP edit-trail fields, hashes/timestamps, macOS download provenance,
Live Photo pairing id. Runs standalone (pillow/piexif/c2pa-python) with
the library optional; supports gzip output, resume after interruption,
and head-only scanning of oversized files.
2026-07-22 17:53:02 -07:00
Victor Kuznetsov 80cdaff2d5 chore(release): v0.18.0 2026-07-22 13:17:00 -07:00
Victor Kuznetsov f1a5eecf98 Register RunningHub, Baidu, and LibLibAI visible marks; park Qingyan and MiniMax (measured)
New engines, each calibrated on its TC260 USCC cohort and validated by a
full-corpus sweep (42009 files):
- runninghub: top-left corner (new corner="tl"), faint mid-gray text via
  the new raw-grayscale "gray" detection front-end, anchor-position gate
- baidu: text-run-only template (pill is a bright-blob magnet), load-bearing
  Doubao+Qwen rival margins, corner-extended footprint for the white tag
- liblib: bottom-center (new corner="bc"), Arial silhouette (font is the
  discriminative lever against latin UI text), logo-extended footprint

Qingyan parked (no clean-arm separation at any render/box), MiniMax/Hailuo
parked (1 visible frame, the xinghui rule); silhouettes kept as starting
points.
2026-07-22 13:03:03 -07:00
Victor Kuznetsov ba29eccc45 chore(release): v0.17.0 2026-07-22 09:15:07 -07:00
Victor Kuznetsov 5d63b9161f Register the Kling 可灵AI 3.0 visible text mark; park Yuanbao and cat-logo (measured)
Kling (USCC cohort 91110108335469089C, n=30): kling_engine.py, gate 0.35
(clean p99 0.304 / max 0.320), strict-only, unimodal 0.12/short on the
shared ladder, fitted locate box, no rival margin (crossfire 1/400 doubao
below gate, 0 jimeng, 0 clean), parity 9/9 detect->fill->re-detect.
Suppresses the jimeng pill like doubao/qwen. identify gains visible_kling.

Yuanbao: measured negative -- the two-line italic block does not separate
from clean corners on either front-end at any render/box/font setting;
the fitted recipe stays in render_vendor_silhouettes.py MARK_OPTS.

cat-logo: cohort has only 2 unique carriers, parked on evidence; the
draw_catlogo silhouette already separates (0.50 vs clean max 0.333), so
registration is a gate pick once more uniques arrive.

vendor_mark_calibrate: --fit-geometry takes locate-box overrides (two-line
marks were clipped by the inherited box) and the aspect sweep reaches 0.62.
2026-07-22 08:49:06 -07:00
Victor Kuznetsov 3a659ededc chore(release): v0.16.0 2026-07-21 18:35:04 -07:00
Victor Kuznetsov 008319c6a5 Register the Qwen 千问AI生成 visible text mark
Calibrated on the 117-frame TC260-producer cohort (vendor_cohort_harvest +
vendor_mark_calibrate, both committed here): per-mark 2-rung ladder
(0.78, 1.27) for the two measured size modes, fitted locate box (the mark
sits ~0.025 of the short side off the edge; doubao's box clipped the first
glyph), measured template aspect 0.26, gate 0.45 (clean p99 0.301).
Strict-only (the sub-gate band is non-Qwen banners), no rival margin
(0 cross-fires on 400 doubao / 298 jimeng / 286 clean frames).
83/83 real marks detector-clean after cv2 fill.

TextMarkConfig gains a per-mark ladder field; the shipped 3-rung default
is unchanged for every other mark.
2026-07-21 16:41:16 -07:00
Victor KuznetsovandClaude Opus 4.8 1d93704a4d Restructure the open items into a continuation list for the next session
The section had accumulated incremental edits: defects numbered out of order
(1, 2, 3a, 3, 5, 4), closed items mixed into the open table, and "what to do
next" spread across four subsections that partly repeated each other.

Now it opens with START HERE -- the prioritized next actions and the reason each
sits where it does -- followed by open defects only, renumbered 1-5. Closed items
move to their own subsection, keeping the faint-mask post-mortem because how it
escaped both parity and its own regression test is the instructive part.

Adds a table of the completed full runs with their artifacts and row counts, each
verified against the file on disk. That exists because the artifacts are the
answer to "did we actually cover X" -- relaunching a sweep costs hours and returns
nothing new, and this session nearly did exactly that before checking. It also
names the two fast confirmations (real_examples_e2e ~2 min, robustness_suite
~3 min) that a later change should run instead.

Adds invisible_engine.py:346 as an open defect: it still discards imwrite's
success flag, the same shape as the crash fixed in the previous commit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 08:31:06 -07:00
Victor KuznetsovandClaude Opus 4.8 633fc3aa52 Report failed writes instead of crashing, and reject directories at parse time
The Tier E adversarial sweep (new, scripts/robustness_suite.py) drove the real CLI
over truncated, corrupt, zero-byte, absurdly-shaped and bomb inputs, unicode and
RTL paths, hostile output directories and concurrent runs. It found two crashes;
the /simplify review then reproduced a third and worse one.

1. A FAILED WRITE CRASHED ON THE SIZE REPORT. image_io.imwrite is contractually
   non-raising and returns False, but write_bgr_with_alpha discarded that bool and
   returned None, so no caller could tell a failed write from a successful one.
   Every write site then ran output.stat() to print the size, so a read-only
   destination died with a bare FileNotFoundError pointing at the stat rather than
   the write. The fix is deliberately NOT uniform: single-image commands exit via
   the new cli._write_output_or_exit; api._write_visible_result RAISES so a library
   caller gets an accurate error instead of a confusing FileNotFoundError from the
   downstream metadata strip; and the batch sites raise but never SystemExit,
   because the batch loop counts per-image exceptions and aborting would kill the
   whole run.

2. BATCH LOST DATA SILENTLY. Into a read-only output directory it wrote ZERO files
   for 2 inputs and exited 0 -- no traceback, no error, an empty output directory a
   wrapping service would read as a completed run. The robustness harness could not
   see this class at all, since it scored exit codes and traceback markers and this
   failure has neither; it now asserts on the artifacts written.

3. A DIRECTORY PASSED AS THE IMAGE crashed the metadata scanner with
   IsADirectoryError, because click.Path(exists=True) accepts directories. Fixed
   with dir_okay=False on all six source arguments, so argument parsing refuses it.

Also adds Tier B4 (scripts/resource_ceilings.py): peak RSS per fill backend from
1 MP to 25 MP, one fresh process per cell. migan 603->775 MB and lama 4679->4779 MB
are flat in input size, confirming the crop-around-the-mask design and both
documented figures; cv2 is the only backend that grows (74->440 MB, 5.9x). The
harness's own no-op check originally allocated a full-frame temp before reading
peak RSS and inflated the numbers with input size -- it now compares only the mask
box, and the conclusion survived re-measurement.

And scripts/real_examples_e2e.py, which drives every command over real corpus
examples and checks the outcome rather than the exit code: 6/6 provenance classes
identified, 10/10 metadata strips re-scan clean, all three fill backends write,
diffusion on MPS writes genuinely changed images. It records samsung as a real
partial (the faintest mark, 0.431 -> 0.404 against a 0.40 gate on the weakest of
its 3 corpus positives) and treats the gated pill's refusal to act as correct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 22:45:02 -07:00
Victor Kuznetsov c2afb92832 Merge pull request #61 from wiltodelta/dependabot/uv/minor-and-patch-d57359b79f
chore(deps): bump the minor-and-patch group with 4 updates
2026-07-20 15:58:12 -07:00
Victor KuznetsovandClaude Opus 4.8 52eb40c2ca Mask faint text marks with the detector's match box, not a response threshold
The faint-mask fallback added for the tophat front-end thresholded the
max-normalized uint8 response at 0.5 -- which selects every non-zero pixel,
not "half the peak" as its comment claimed -- and filled ~120% of the corner
box on textured frames. Measured on 14 real faint-path frames (cv2 fill,
detector re-run after): the detector's own best-match box fills a 58.7%-median
corner box vs 120.9% for the threshold, both 100% detector-clean. Detection and
the mask now read one method, _tophat_best, whose score gates detection and
whose argmax box bounds the fill, so the two cannot drift by construction --
which is how the mismatch arose. The 0.5 constant is deleted.

Parity could not catch this (a mask that fills everything is trivially
detector-clean) and the regression test could not either: its flat fixture
gives every threshold the same box, so mutating the constant to 99.0 stayed
green. The fixture now carries texture and asserts the mask area is bounded,
not merely non-empty; it reproduces the corpus number (127% pre-fix).

Also lands the Tier B2 verification harnesses that found and bounded this:
- detector_response.py: response curves (detected AND maskable per cell); found
  the size response is a comb, contrast is near-irrelevant, no unmaskable cells.
- ladder_headroom.py: measured that a denser scale ladder recovers 7.6% of
  misses for a 2.52%->3.05% false-fire rise, and the one landscape rung that
  helps is a geometry shift that helps and hurts equally (1.7:1) -- do not add.
- cjk_tail_probe.py: a generic shared-tail (AI生成) template does not separate
  uncovered vendors from clean corners (0.407 vs clean p99 0.298).

Records the visible-parity re-run confirming the earlier front-end fix (doubao
91.8% -> 99.3%), and dedups the thrice-written stamp forward model into one
fill_quality.composite.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 15:41:18 -07:00
Victor KuznetsovandClaude Opus 4.8 7d00debdca Record the full visible-parity run and the per-mark residual split
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 11:11:27 -07:00
Victor KuznetsovandClaude Opus 4.8 ac20cf3a3d Record open defects, unrun verification tiers and the stale torch triage
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 11:10:12 -07:00
Victor KuznetsovandClaude Opus 4.8 21d7e05014 Drop AGENTS.md, CLAUDE.md is the single source
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 11:02:09 -07:00
Victor KuznetsovandClaude Opus 4.8 cd16e8129d Resync AGENTS.md with CLAUDE.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 10:57:52 -07:00
Victor KuznetsovandClaude Opus 4.8 836d87ed68 Mask faint text marks the tophat front-end detects but binarization loses
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 10:57:08 -07:00
Victor KuznetsovandClaude Opus 4.8 c150180acf Add full-surface verification harnesses and corpus sweep plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 08:14:50 -07:00
Victor KuznetsovandClaude Opus 4.8 cfefd9d819 Remove assume_ai, add tophat front-end and rival margin, fix two CLI defects
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 08:14:50 -07:00
Victor Kuznetsov a8f3536d3e Refactor watermark detection and provenance handling 2026-07-16 17:40:46 -07:00
Victor KuznetsovandClaude Opus 4.8 9618ac93c8 docs: record where the HF Space source lives and how it deploys
The Space's demo code (app.py etc.) lives in a SEPARATE private repo,
wiltodelta/raiw-hf-space, which nothing documented -- so finding it cost a long
detour through the Space's commit authorship and a hunt for a write token that
never existed locally. Record it, plus the deploy flow that replaced the old
web-UI editing: push to that repo's main -> sync-to-hf.yml mirrors the files via
HfApi.upload_folder (adds a commit on top of the Space history, never a force
push). Also call out the two automations that both touch the Space so they are
not confused: sync-to-hf.yml ships demo-code changes, while this repo's
distribute.yml factory-rebuilds the Space on a library release so its
`remove-ai-watermarks>=` pin re-resolves.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 10:24:46 -07:00
dependabot[bot] 976935f5fe chore(deps): bump the minor-and-patch group with 4 updates
Bumps the minor-and-patch group with 4 updates: [torch](https://github.com/pytorch/pytorch), [transformers](https://github.com/huggingface/transformers), [huggingface-hub](https://github.com/huggingface/huggingface_hub) and [ruff](https://github.com/astral-sh/ruff).


Updates `torch` from 2.12.1 to 2.13.0
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/compare/v2.12.1...v2.13.0)

Updates `transformers` from 5.13.0 to 5.13.1
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.13.0...v5.13.1)

Updates `huggingface-hub` from 1.22.0 to 1.23.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.22.0...v1.23.0)

Updates `ruff` from 0.15.20 to 0.15.21
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.20...0.15.21)

---
updated-dependencies:
- dependency-name: torch
  dependency-version: 2.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: transformers
  dependency-version: 5.13.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.15.21
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-15 08:46:35 +00:00
Victor KuznetsovandClaude Opus 4.8 71dae8fe25 chore(release): bump version to 0.15.3
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 17:17:01 +03:00
Victor KuznetsovandClaude Opus 4.8 9fe12c3997 docs: note content-based strip routing + SDXL watermarker fixes
Document both fixes in CLAUDE.md (the metadata.py and watermark_remover.py
bullets). Also add a --backend flag to the visible-removal audit script so a
realistic quality pass can run the production MI-GAN fill instead of cv2
(removal SUCCESS is backend-independent, but only migan/lama reflect the
recovered-region quality a user actually gets).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 17:16:56 +03:00
Victor KuznetsovandClaude Opus 4.8 183793fbf2 fix(invisible): disable diffusers SDXL watermarker on removal output
The SDXL removal pipelines (sdxl + controlnet) were built without
add_watermarker=False, so diffusers embedded its default open "Stable
Diffusion XL" DWT-DCT invisible watermark on every output whenever
invisible-watermark is installed (the detect extra). A watermark REMOVER was
therefore replacing one detectable AI watermark (SynthID) with another: the
cleaned output re-read as AI (identify -> "Open invisible watermark: Stable
Diffusion XL"), observed on the SynthID validation sample.

Both SDXL loaders now call a shared _disable_sdxl_watermarker helper (mirrors
_maybe_add_fp16_vae; the ControlNetModel sub-model and the Qwen loader never
call it, since only the pipeline accepts the kwarg). Verified end to end: the
affected outputs re-run clean (is_ai=None, no open watermark).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 17:16:49 +03:00
Victor KuznetsovandClaude Opus 4.8 7f22852593 fix(metadata): route strip output format by content, not extension
remove_ai_metadata chose its save format (and the lossless-JPEG fast path)
from the OUTPUT file extension. On the ~2% of real uploads whose extension
lies about their content (a PNG served as .jpg is the common case, ~0.9% of
the corpus), the default flow -- which inherits the source's own extension --
re-encoded a lossless PNG/WebP into a real JPEG, silently degrading the pixels
and breaking the "work with originals" invariant.

Sniff the actual container from magic bytes (_sniff_image_format, reusing the
12-byte head already read for the ISOBMFF check) and route on content: a
misnamed lossless source (source-extension format != content) is preserved in
its true format, while a correctly-named source still honors a deliberate
output-extension conversion (source.png -> output.jpg). The JPEG-lossless gate
is likewise content-gated.

Found by a new metadata-removal parity audit over the local corpus
(scripts/metadata_removal_audit.py): 18170/18173 carriers strip cleanly, and
this fix takes the 208 pixel-integrity failures (all misnamed PNGs) to 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 17:16:36 +03:00
Victor Kuznetsov d9204e2e1f chore(release): bump version to 0.15.2 2026-07-13 12:27:06 +03:00
Victor KuznetsovandClaude Opus 4.8 2bdaa09e5f fix(metadata): strip bare AIGC in APP11 and AIGC in a standard PNG text chunk
Full-corpus strip audit surfaced 24 china_aigc survivors on real uploads:
- 19 JPEG carried the bare AIGC{...} blob in APP11 (0xEB). That marker's branch
  in _jpeg_app_carries_ai only tested for a C2PA/JUMBF manifest and RETURNED, so
  a bare AIGC there slipped past the generic AIGC check. The specific
  C2PA(APP11)/XMP(APP1)/IPTC(APP13) checks now fall through to the generic
  _is_aigc_exif_value drop, which runs for every APP marker they did not claim.
- PNG carried the {"AIGC":{...}} block in a STANDARD text chunk (Description).
  _is_ai_key keeps that key, so removal now also drops a text value carrying an
  AIGC block (_is_aigc_exif_value broadened to accept str; wired into the PNG
  re-save value filter), in parity with aigc_label's detection.

Verified on the corpus: decodable china_aigc survivors 27 -> 0; the 3 remaining
are truncated files the strip fail-safe (v0.15.1) copies through by design.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:27:05 +03:00
Victor Kuznetsov 72f6734b45 chore(release): bump version to 0.15.1 2026-07-13 11:54:53 +03:00
Victor KuznetsovandClaude Opus 4.8 220803c4d0 fix(metadata): remove_ai_metadata is fail-safe on a truncated/corrupt image
PIL raises OSError decoding a truncated file, which crashed remove_ai_metadata
(the PNG/WebP PIL re-save path) -- a direct library caller like a web worker
500s on a partial upload. ~0.2% of the real upload corpus is truncated. The
strip now probes decodability first and, on failure, copies the input through
unchanged and returns rather than raising (we cannot strip what we cannot parse),
mirroring strip_c2pa_boxes' fail-safe. identify already handled these.

The CLI `metadata --remove` on an unreadable file therefore now exits 0 with the
input passed through, not a clean error (exit 1) -- `visible`, which must decode
to remove a mark, still exits 1. Test updated to the per-command contract.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 11:54:33 +03:00
Victor Kuznetsov 1dc6fee91a chore(release): bump version to 0.15.0 2026-07-13 10:50:20 +03:00
Victor KuznetsovandClaude Opus 4.8 a4c901ff39 fix: metadata-strip parity, input robustness, and detection/clash coverage
Bug fixes (each with a regression test):
- metadata strip parity across every marker placement: IPTC digitalSourceType
  in XMP, the Samsung post-EOI trailer, the China TC260 AIGC block in EXIF
  UserComment, a bare AIGC block in a non-standard APP segment, and the ISOBMFF
  EXIF path (AIGC + xAI) are all now stripped -- anything a scanner flags, the
  strip reaches
- Samsung genAIType detected when its trailer sits past the 512 KB scan window
  (file-tail read on large photos)
- crashes on edge inputs: Gemini detector on images with a short side < 16px,
  footprint_mask on a zero-size ndarray, the humanizer on chromatic_shift >=
  width, and the CLI on unreadable/corrupt/empty input (clean error, not a
  traceback)
- WebP written losslessly (cv2 quality 101), not lossy at 100
- the IPTC digitalSourceType algorithmicMedia (procedural, not trained on
  sampled data) is no longer flagged as AI-generated, so clean procedural
  content is not scrubbed
- c2pa source-type: compositeWithTrainedAlgorithmicMedia is checked before the
  bare algorithmicMedia token, so an AI-enhanced composite is not misclassified

Detection:
- integrity-clash coverage now normalizes ByteDance / Canva / ElevenLabs /
  Black Forest Labs, so a transplanted manifest next to an independent
  conflicting stamp is caught; the generic China TC260 AIGC label is attributed
  to a co-present TC260 vendor, so a legit Doubao image (its own C2PA + TC260
  label) does not clash (corpus-validated: 0 new clashes on 5000 carriers)

CLI:
- batch exits non-zero (with a warning) when any image errors or a GPU-missing
  SynthID scrub is skipped, and copies the input through so the output dir stays
  complete -- it used to always exit 0 and could silently drop files

Perf:
- GeminiEngine reused as a process-wide singleton with a precomputed template
  ladder: -24% on the identify sparkle path, detection byte-identical

Internal: one shared _ai_exif_targets rule set feeds both EXIF scrubbers so
their coverage cannot drift; docs synced; maintain.sh hardened so the uv-secure
internal teardown crash no longer aborts the gate (still fails on a real finding).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 10:49:24 +03:00
Victor KuznetsovandClaude Opus 4.8 190dc89d23 perf(visible): crop MI-GAN around the mask so peak RAM is bounded by mark size
erase_migan fed the whole frame to the ONNX model, so peak RSS scaled with the
upload (~0.6 GB at 4 MP up to ~2.4 GB at 25 MP). Mirror erase_lama: crop a padded
region around the mask (pad = max(256, 2*bbox)), feed only that crop (at native
resolution -- MI-GAN accepts arbitrary dims, unlike LaMa's fixed 512 square), and
paste only masked pixels back. Peak RSS is now bounded by the mark size
(~0.6-0.9 GB), so a memory-tight host (a 1-2 GB web worker) can run MI-GAN on a
25 MP upload.

Fill quality is unchanged: verified by eye on real Gemini/Doubao marks plus a
ground-truth reconstruction sweep -- a tighter view if anything reduces the GAN's
hallucination of large background structure.

Extract the shared padded-crop-box math into _padded_crop_box (used by both
erase_lama and erase_migan).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 08:22:59 +03:00
Victor KuznetsovandClaude Opus 4.8 f69fedcf2d chore(release): bump version to 0.14.1
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 11:13:02 +03:00
Victor KuznetsovandClaude Opus 4.8 731af94639 docs(release): note HOMEBREW_TAP_TOKEN rotation on distribute.yml Bad credentials
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 11:13:02 +03:00
Victor KuznetsovandClaude Opus 4.8 49869ab02b feat(identify): detect Dreamina C2PA + Tencent Cloud AIGC schema
Two AI-provenance metadata types mined from the retained corpus that
identify previously read as no-signal:

- Dreamina (ByteDance's international Jimeng brand) signs C2PA as
  "Bytedance Pte. Ltd." with a "Dreamina/x.y" claim generator and NO
  digitalSourceType, so the generator name is the only AI signal. Add a
  C2paAiVendor row with a new asserts_ai flag (identity-AI: presence
  asserts AI without trainedAlgorithmicMedia) plus the derived
  C2PA_IDENTITY_AI_ORGS view, folded into identify's c2pa_is_ai. Keyed on
  the Dreamina generator token, not the "Bytedance Pte" issuer, so non-AI
  CapCut edits signed by the same entity stay unattributed. 7/7 corpus
  files now attribute to ByteDance.

- Tencent Cloud's TC260 AIGC variant uses a ServiceProvider/ServiceUser
  schema (vs the producer-side ContentProducer schema), embedded in EXIF
  ImageDescription; add those field names to _TC260_FIELDS so the generic
  {"AIGC":{...}} gate accepts it. 11/11 corpus files now flagged.

Test-first: reproducing tests in test_identify.py / test_metadata.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 10:54:56 +03:00
Victor Kuznetsov 41e20a0606 Merge pull request #60 from wiltodelta/claude/mystifying-cori-a4c1e7
v0.14.0: visible-mark localize→fill rewrite, backend priority, fixes
2026-07-11 09:53:07 +03:00
Victor KuznetsovandClaude Opus 4.8 e3e332c4b1 Merge origin/main into claude/mystifying-cori-a4c1e7
main landed #58 (pill-gate fix, superseded by this branch's localize->fill
rewrite) and #57 (deps bump). Resolved the 6 code/test/doc conflicts by keeping
this branch's post-rewrite versions; the deps bump auto-merged into
uv.lock/pyproject. Full gate green after resolution: ruff, pyright 0, 730 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 09:50:21 +03:00
Victor KuznetsovandClaude Opus 4.8 b579385c6f feat(visible): white-core rescue for the Gemini false-positive gate
The FP gate demotes a low-gradient match, but a real FAINT sparkle also has soft
edges, so metadata-stripped faint sparkles were dropped. Keep a low-grad match
that is a strong (conf >= 0.52), bright, near-WHITE-core sparkle: a real sparkle
core is white, a clean bright corner that shape-matches (sky/sun) is colored
(_core_saturation). Recovers ~14/20 stripped faint sparkles under the DEFAULT
strict/auto (no metadata, no flag) at ~1.25% clean false-fire (baseline 0.55%);
the ~0.51-scoring bright-background FPs stay demoted (below 0.52).

A learned classifier on the same features measured WORSE than the tuned gate
(tier-1: MLP 86.7% recall vs the gate's 90.8% at equal false-fire), so the
heuristic stays; a patch-CNN with richer features is roadmapped P2 with low
expected value -- the precision/recall wall is fundamental (deep-research +
tier-1 both confirm it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 09:41:58 +03:00
Victor KuznetsovandClaude Opus 4.8 bd60be10f9 docs: fold visible-mark + detection-method research into watermarking-landscape
Deep-research 2026-07-10 (adversarially verified): the Gemini sparkle is
tier-gated (visible on Free/Pro, absent on Ultra/AI-Studio/API; no official
visible-mark detector or published glyph spec); the faint-visible-mark
precision/recall wall is fundamental (learned CNN front-end does not cleanly
separate true/false, arXiv:1705.08593 refuted); learned detectors need large
synthetic-composite datasets + carry off-distribution risk; landscape adds
Meta bottom-left + Samsung star-icon variants; China GB 45438-2025 is the
strongest visible-mark mandate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 10:00:12 +03:00
Victor KuznetsovandClaude Opus 4.8 27a921b54f docs(visible): record the 0.12.1-vs-0.14 head-to-head + backend quality
Full-dataset validation of reverse-alpha (v0.12.1) vs the current localize->fill:
doubao/jimeng identical (100% coverage + clearance across all backends); gemini
strict coverage a few points below reverse-alpha (the FP tightening), every
missed mark recovered under assume_ai, clearance ~98% both, no outside-box
damage. Clearance is fill-independent (cv2/MI-GAN/LaMa all strip the mark shape);
the difference is visual fill quality on textured/structured backgrounds -- LaMa
best, MI-GAN can ghost/hallucinate, cv2 smears -- which motivates auto = LaMa >
MI-GAN > cv2. Added to module-internals, known-limitations, and the CLAUDE.md
compact list.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 19:18:25 +03:00
Victor KuznetsovandClaude Opus 4.8 c858006e93 feat(visible): auto fill prefers LaMa > MI-GAN > cv2, warn on cv2 fallback
The auto backend now resolves best-first: LaMa (highest quality, recovers the
textured/structured backgrounds the classical fill smears) > MI-GAN > cv2. Both
learned backends share the same onnxruntime availability check, so auto cannot
tell them apart and always prefers the better one; a memory-tight deployment
that cannot afford LaMa's ~4.7 GB peak pins MI-GAN explicitly via
`--backend migan` / `backend="migan"` (the deployment's call, not the library's).
cv2 stays the no-deps floor and now emits a one-time quality warning when auto
falls back to it, since it smears texture/structure.

Motivated by a v0.12.1 reverse-alpha vs 0.14 localize->fill head-to-head:
reverse-alpha recovered structured backgrounds more cleanly than any inpaint;
LaMa closes most of that gap, MI-GAN can ghost/hallucinate, cv2 is weakest.
doubao/jimeng removal is identical between versions; gemini strict coverage is
4pp lower (all recovered via assume_ai) with cleaner clearance and no
outside-box damage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 18:24:20 +03:00
Victor KuznetsovandClaude Opus 4.8 9756189eaf style: drop dead Candidate fields, simplify resolve_backend, US spelling
- Candidate carries only the fields the arbiter reads (key, label,
  detected_strict, detected_relaxed, features); location/region/confidence were
  vestigial from the removed best_auto_mark max-by-confidence path.
- resolve_backend returns preferred_inpaint_backend() directly (typed Literal)
  instead of an identity ternary.
- colour/normalise/behaviour -> US spelling across code comments and docs.

No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 16:18:13 +03:00
Victor KuznetsovandClaude Opus 4.8 178fed69a7 fix(visible): thread detection into mask + guard removal/IO edge cases
Resolve 10 code-review findings on the v0.14.0 localize->fill path, several
release-blocking:

- gemini: build the removal mask from the decision's provenance-aware region
  instead of a strict internal re-detect. A relaxed/assume_ai sparkle was
  re-demoted by the FP gate into a None mask and reported removed while left in
  the image; this also drops the redundant double-detect.
- registry: report a mark removed only when a fill actually happened (remove()
  returns a None region for an empty mask), so a no-op is never claimed.
- api/cli: add write_noop so the CLI `visible` no-mark path writes nothing and
  cannot clobber a pre-existing -o file (was write-then-unlink -> data loss);
  create output.parent; skip the same-file copy (SameFileError on in-place).
- cli: catch the missing migan/lama backend RuntimeError on the visible/all
  paths (matches `erase`); route the single-mark relaxation through the shared
  resolve_relax instead of an inline copy.
- metadata: keep_standard=False no longer takes the AI-only lossless JPEG
  short-circuit (it left standard metadata); defer a malformed-marker JPEG to
  the PIL fallback instead of reporting a partial strip as complete.
- invisible: register the HEIF opener before Image.open (HEIC --force) and
  RGB-convert before the PNG temp (CMYK JPEG).
- pill: normalize via to_bgr so a 4-channel BGRA array cannot crash cvtColor.

Regression tests for each; docs synced (resolve_relax, write_noop,
best_auto_mark -> detect_marks).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 16:14:46 +03:00
Victor KuznetsovandClaude Opus 4.8 1f5d64e6c5 style: unify section dividers and logger variable naming
- cli.py section dividers -> the codebase-majority `# ── Title ──` form
- logger variable unified to `logger` across identify, isobmff, c2pa,
  trustmark_detector, invisible_watermark (matching the 9-file majority)
- cosmetic only, no behavior change (719 tests / pyright / ruff green)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 14:31:40 +03:00
Victor KuznetsovandClaude Opus 4.8 1a955b096a feat(visible): localize->fill rewrite, sensitivity/backend + api, HEIC + lossless IO
- Replace reverse-alpha removal with localize -> fill (template-free mask + one
  shared cv2/MI-GAN/big-LaMa fill) for every mark; drops the colour-shift / dark-pit
  failure modes, version-robust to a moved or re-rendered mark
- Separate perception/decision/action: engines report Candidates, a pure
  decide(candidates, Context) arbiter owns all policy (sensitivity + provenance +
  pill gate), remove_auto_marks orchestrates -- behavior-preserving (corpus 46/46/92)
- Three orthogonal knobs replace --method: --backend cv2|migan|lama,
  --sensitivity auto|strict|assume-ai, provenance (auto from metadata)
- Add high-level api.remove_visible / visible_provenance (lazy top-level re-export);
  visible --mark auto delegates to it so CLI and library share ONE path
- Read+write HEIC/AVIF on the pixel path via pillow-heif; imwrite preserves the input
  format at max quality (JPEG q100/4:4:4); a no-op copies the original bytes verbatim
- Lossless byte-level JPEG metadata strip (no DCT re-encode); consolidate the two
  remove_ai_metadata into one, delete legacy noai/cleaner + best_auto_mark
- Bump 0.13.0 -> 0.14.0

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 14:20:52 +03:00
dependabot[bot] 3c6cfd3a9b chore(deps): bump the minor-and-patch group across 1 directory with 6 updates (#57)
Bumps the minor-and-patch group with 6 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [pillow](https://github.com/python-pillow/Pillow) | `12.2.0` | `12.3.0` |
| [diffusers](https://github.com/huggingface/diffusers) | `0.38.0` | `0.39.0` |
| [transformers](https://github.com/huggingface/transformers) | `5.12.1` | `5.13.0` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.20.1` | `1.22.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.15.19` | `0.15.20` |
| [pyright](https://github.com/RobertCraigie/pyright-python) | `1.1.410` | `1.1.411` |



Updates `pillow` from 12.2.0 to 12.3.0
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/12.2.0...12.3.0)

Updates `diffusers` from 0.38.0 to 0.39.0
- [Release notes](https://github.com/huggingface/diffusers/releases)
- [Commits](https://github.com/huggingface/diffusers/compare/v0.38.0...v0.39.0)

Updates `transformers` from 5.12.1 to 5.13.0
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.12.1...v5.13.0)

Updates `huggingface-hub` from 1.20.1 to 1.22.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.20.1...v1.22.0)

Updates `ruff` from 0.15.19 to 0.15.20
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.19...0.15.20)

Updates `pyright` from 1.1.410 to 1.1.411
- [Release notes](https://github.com/RobertCraigie/pyright-python/releases)
- [Commits](https://github.com/RobertCraigie/pyright-python/compare/v1.1.410...v1.1.411)

---
updated-dependencies:
- dependency-name: pillow
  dependency-version: 12.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: diffusers
  dependency-version: 0.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: transformers
  dependency-version: 5.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.15.20
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: pyright
  dependency-version: 1.1.411
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-07 12:37:13 +03:00
Victor KuznetsovandClaude Opus 4.8 3524e8442e fix(visible): safe-inpaint pill gate, cut metadata-only false fires (#58)
Verified 0.13.0 pill removal on a 32k real-upload corpus. The metadata-OR-wordmark
gate was only ~1/3 precise: TC260 metadata confirms Jimeng-class provenance, not pill
presence, so the weak edge-NCC detector's false fires (textured ceilings/walls, where
inpaint visibly smears) were admitted whenever metadata was present.

Split into two arms (_keep_pill): the reliable bottom-right wordmark (~94% precise,
survives metadata stripping) removes the pill unrestricted; the metadata-only arm
removes it ONLY when the top-left footprint is flat enough for an invisible inpaint
(PillEngine.footprint_is_flat, median-Sobel <= _FLAT_TEXTURE_MAX). Keeps real
flat-scene pills and harmless flat false fires; leaves the damaging textured false
fires untouched. Corpus: 270 -> 118 removals, ~90 true preserved, damaging FP -> ~0.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 11:29:17 +03:00
Victor KuznetsovandClaude Opus 4.8 a8fd02a8f7 fix(visible): safe-inpaint pill gate, cut metadata-only false fires
Verified 0.13.0 pill removal on a 32k real-upload corpus. The metadata-OR-wordmark
gate was only ~1/3 precise: TC260 metadata confirms Jimeng-class provenance, not pill
presence, so the weak edge-NCC detector's false fires (textured ceilings/walls, where
inpaint visibly smears) were admitted whenever metadata was present.

Split into two arms (_keep_pill): the reliable bottom-right wordmark (~94% precise,
survives metadata stripping) removes the pill unrestricted; the metadata-only arm
removes it ONLY when the top-left footprint is flat enough for an invisible inpaint
(PillEngine.footprint_is_flat, median-Sobel <= _FLAT_TEXTURE_MAX). Keeps real
flat-scene pills and harmless flat false fires; leaves the damaging textured false
fires untouched. Corpus: 270 -> 118 removals, ~90 true preserved, damaging FP -> ~0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 11:26:28 +03:00
Victor KuznetsovandClaude Opus 4.8 0e5a4cbc54 feat(visible): capture-less AI生成 pill (#54), inpaint fallback, MI-GAN backend (#56)
- Add the Jimeng-basic top-left "AI生成" pill as a CAPTURE-LESS mark
  (pill_engine.py): synthetic-silhouette edge-NCC detect + inpaint-only removal.
  Gated in remove_auto_marks: kept only when Jimeng is confirmed (TC260 metadata
  OR the bottom-right "★ 即梦AI" wordmark fired -- the wordmark keeps recall on
  metadata-STRIPPED uploads) AND Doubao did not fire.
- Add an inpaint-fallback removal path + MI-GAN ONNX backend (migan extra, MIT,
  ~28 MB / ~1 GB peak -- droplet-friendly) alongside big-LaMa. New
  --method auto|reverse-alpha|inpaint (shared across visible/all/batch) and
  erase --backend migan; footprint_mask on each engine.
- auto is deterministic: reverse-alpha for capture marks (recovers exact pixels,
  lighter -- measured cleaner than MI-GAN on structured backgrounds) and inpaint
  only for the capture-less pill.
- --mark auto now removes EVERY detected mark in one pass (remove_auto_marks),
  so a Jimeng-basic image's top-left pill AND bottom-right wordmark both clear.
- Bump 0.12.1 -> 0.13.0.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 20:38:23 +03:00
Victor KuznetsovandClaude Opus 4.8 0f54c6b54d fix(identify): kill three visible-detector false positives
Bright-background photos/renders and a tiny app icon were flagged as
AI-generated by the visible detectors. Two failure modes:

- Gemini sparkle on a bright background (snow+sky photo, white product
  render) scored ~0.51. The FP gate only demoted on a low core-ring
  brightness margin, which a bright background makes high. Add a gradient
  floor (_SPARKLE_FP_GRAD 0.55): a real sparkle is a crisp star (grad
  ~0.97-1.0), a smooth luminance blob that NCC-matches the diamond is not
  (the two FPs measured grad 0.105 / 0.463). The OR is a strict superset
  of the old margin-only demotion, so it cannot regress dark/mid (kept by
  margin) or white-bg (kept by confidence) real sparkles.

- A 48x48 geometric icon matched the Doubao/Jimeng CJK silhouette at
  0.41/0.47 NCC. Purely a small-size artifact (the same icon at >=256px
  collapses to ~0.06-0.10). Guard text-mark detection below a 200px short
  side (_MIN_DETECT_SHORT_SIDE); real marks ship on full-resolution
  renders (smallest captured sample 1086px).

Corpus re-sweep flips only OpenAI content and already-cleaned outputs,
all sub-0.5, so no provenance verdict changes. Add synthetic regression
fixtures for both modes; docs/module-internals.md updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 09:44:24 -07:00
Victor KuznetsovandClaude Opus 4.6 f269b75ded chore: add ruff scripts per-file-ignores, Claude Code settings, bump deps
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-24 12:32:44 -07:00
Victor KuznetsovandClaude Opus 4.8 0d9d7dcf6a docs: compact CLAUDE.md, relocate incident/CVE detail to docs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 10:28:40 -07:00
dependabot[bot] 3bbcc0ee94 chore(deps): bump actions/checkout from 6 to 7 in the actions group (#52)
Bumps the actions group with 1 update: [actions/checkout](https://github.com/actions/checkout).


Updates `actions/checkout` from 6 to 7
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-24 09:51:00 -07:00
dependabot[bot] d96e3f6334 chore(deps): bump the minor-and-patch group with 5 updates (#53)
Bumps the minor-and-patch group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [c2pa-python](https://github.com/contentauth/c2pa-python) | `0.35.1` | `0.36.0` |
| [torch](https://github.com/pytorch/pytorch) | `2.12.0` | `2.12.1` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.19.0` | `1.20.1` |
| [pytest](https://github.com/pytest-dev/pytest) | `9.1.0` | `9.1.1` |
| [ruff](https://github.com/astral-sh/ruff) | `0.15.17` | `0.15.19` |


Updates `c2pa-python` from 0.35.1 to 0.36.0
- [Release notes](https://github.com/contentauth/c2pa-python/releases)
- [Changelog](https://github.com/contentauth/c2pa-python/blob/main/docs/release-notes.md)
- [Commits](https://github.com/contentauth/c2pa-python/compare/v0.35.1...v0.36.0)

Updates `torch` from 2.12.0 to 2.12.1
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/compare/v2.12.0...v2.12.1)

Updates `huggingface-hub` from 1.19.0 to 1.20.1
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.19.0...v1.20.1)

Updates `pytest` from 9.1.0 to 9.1.1
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.1.0...9.1.1)

Updates `ruff` from 0.15.17 to 0.15.19
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.17...0.15.19)

---
updated-dependencies:
- dependency-name: c2pa-python
  dependency-version: 0.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: torch
  dependency-version: 2.12.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.20.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: pytest
  dependency-version: 9.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.15.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-24 09:50:56 -07:00
Victor Kuznetsov ee4af800ce chore(release): v0.12.1 2026-06-23 15:15:04 -07:00
Victor KuznetsovandClaude Opus 4.8 675590e8b2 perf(invisible): read the fp16 weight variant to halve the cold-start weight load
InvisibleEngine loads SDXL/ControlNet in fp16 on CUDA/XPU but called from_pretrained
without variant="fp16", so it read the full fp32 weight files (~7 GB) and downcast in
memory. _load_from_pretrained now passes variant="fp16" when torch_dtype is float16,
reading the half-precision files (~3.5 GB) instead - roughly halving the cold-start
weight read + host->device transfer (a phase-timed Modal run measured weight load as
~half of the ~25s cold start). Falls back to the default weights when a checkpoint ships
no fp16 variant (a custom --model), so the worst case is the prior behavior. fp32
(cpu/mps) and bf16 (qwen) never request the variant.

Tests: TestFp16WeightVariant (variant requested on fp16, fallback on missing, never on
fp32).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 14:42:47 -07:00
Victor KuznetsovandClaude Opus 4.8 c8dbd0c3f9 docs(claude-md): record that new visible marks are blocked on real flat captures
Per user decision 2026-06-22: synthetic font-rendered alpha reconstruction is
rejected as below the quality bar; the reverse-alpha alpha map must be solved
from real controlled flat captures (visible_alpha_solve.py). Meta AI, more
Samsung locales, and any Grok visible mark are parked until captures exist.
Future sessions must not propose synthetic or derive assets from the corpus.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 16:28:28 -07:00
Victor KuznetsovandClaude Opus 4.8 abb7be7e9b feat(identify): detect + strip NovelAI / Reve / Aphrodite generator stamps
Mined from the retained corpus 2026-06-22 (open-world EXIF/PNG-text/XMP scan,
minus the registry): three AI image generators that stamp a plain generator
name and no C2PA, so identify read them as no-signal -- and under the P0#5
no-signal skip would have skipped the scrub.

- NovelAI (anime SD): PNG tEXt Software/Source/Title. exif_generator now reads
  PNG text chunks (via img.info), not only EXIF/XMP.
- Reve (reve.com): EXIF Software / XMP CreatorTool. Token is the full
  "reve.com", not bare "reve" (would false-fire on "forever"/"reverie").
- Aphrodite AI: EXIF Make / Software.

Detection/removal parity: NovelAI stamps an AI-shaped VALUE under a non-AI KEY
(Title/Source), which _is_ai_key alone keeps. New _is_ai_value drops a text
chunk by value-token match on removal, mirroring exif_generator -- else the
cleaned file still read as NovelAI (verified on a real corpus file).

Tests: TestExifGenerator gains NovelAI PNG-text, Reve, Reve-not-overmatched,
Aphrodite, and a NovelAI detect/remove parity regression. Docs synced
(module-internals, watermarking-landscape, CLAUDE.md).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 16:23:33 -07:00
Victor KuznetsovandClaude Opus 4.8 19f9ab0947 feat(invisible): skip the diffusion scrub when no invisible watermark is detectable (P0#5)
Regenerating pixels removes SynthID / open watermarks but degrades a real
photo, so running it on a clean image is the dominant paid score-0 cause on
no-watermark uploads. Gate invisible/all/batch on identify.has_invisible_target:
when no invisible AI signal is locally detectable and --force is unset, skip the
regeneration. Per-command semantics:
  - invisible: write no output, exit EXIT_NO_INVISIBLE_SIGNAL (2)
  - all: skip step 2 but keep visible-removed pixels + strip metadata, exit 0
  - batch: skip the scrub; copy the input through in invisible mode
A skip never claims the image is clean (a pixel SynthID is undetectable once its
metadata proxy is gone); the message says so and routes to --force. The gate
fails safe (a detector error runs the removal).

has_invisible_target wraps identify(check_visible=False, check_invisible=True)
and returns the new ProvenanceReport.ai_from_metadata field (the confidence==high
union), so the raiw.cc worker can reuse the same gate. Gate placed before engine
construction so the skip path is cheap; shared via cli._should_skip_invisible_scrub.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 11:37:01 -07:00
Victor KuznetsovandClaude Opus 4.8 5a612adfef chore(release): v0.12.0
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 11:14:55 -07:00
Victor KuznetsovandClaude Opus 4.8 9695a7cbd2 docs(qwen): drop false 'pending seed-repeat cert' from pipeline docstring
The qwen oracle floors are certified, not pending. Near-threshold scrub is
seed-non-deterministic, but the prod path pins one fixed seed, so a certified
floor reproduces run-to-run -- pinning the seed is the release gate, not a seed
sweep. Reword the docstring so it stops implying an open seed-repeat gate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 10:30:42 -07:00
Victor KuznetsovandClaude Opus 4.8 78e2ae65ad docs: neutralize local-pull path reference in doubao research note
Replace the `data/spaces/originals/` path with a generic "local corpus of
pristine originals" so the committed public doc carries no reference to the
local working-data pull (the data itself is gitignored). The analysis scripts'
default paths are left untouched (operational tooling, no content/provenance).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 09:41:23 -07:00
Victor KuznetsovandClaude Opus 4.8 719c7e9313 chore(gitignore): ignore local-only data/spaces working dir
21k user-pull images under data/spaces/ were untracked but not ignored, so a
stray `git add -A` could have committed them. Add the ignore entry alongside the
other local-data paths; the dir stays local analysis only, never a committed corpus.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 09:32:22 -07:00
Victor KuznetsovandClaude Opus 4.8 d5dd24140c fix(qwen): native-geometry img2img + pipeline-aware strength; record dropped auto/mixed/Z-Image leads
- watermark_remover: _build_qwen_kwargs now passes explicit height/width (via
  _qwen_target_size, floored to /16). Without it QwenImageImg2ImgPipeline defaults to
  1024x1024 and silently squishes non-square inputs, distorting the scene and garbling text.
- watermark_profiles: resolve_strength gains a `pipeline` arg + a Qwen strength ladder
  (_QWEN_VENDOR_STRENGTH, Gemini 0.25), so `--pipeline qwen` gets its certified floor
  automatically; retires the manual "pass --strength 0.25 for Gemini on qwen" workaround.
- fidelity_metrics: replace per-face nearest matching (collided on multi-face images when a
  variant dropped a face, corrupting the identity metric) with a collision-free one-to-one
  assignment (assign_faces_one_to_one). lapvar/LPIPS were always bbox-anchored and immune.
  Regression-guarded by tests/test_fidelity_matching.py.
- docs: record the measured outcomes of the qwen-improvement arc. The Qwen ControlNet
  face-fix is CLOSED (no permissive Qwen detail/tile ControlNet exists; canny carries edges,
  not skin grain). The `--pipeline auto` router + faces+text mixed dual-pass were prototyped
  and DROPPED (controlnet wins faces AND display text: abba CER 0.114 vs qwen 0.379).
  Z-Image-Turbo was tried and dropped (same regeneration limits). qwen stays a manual opt-in;
  controlnet is the default for everything.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 21:52:56 -07:00
Victor KuznetsovandClaude Opus 4.8 8f64869bfc docs: capture the Qwen-improvement research (ship vs improve)
Cited deep-research report (22 sources, 3-vote adversarial verification, 5 refuted)
behind the "ship qwen as-is or improve first?" decision. Verdict: shippable now as
an opt-in text lane; strongest improvement lead is adding a Qwen-Image ControlNet
(InstantX / DiffSynth, Apache-2.0, diffusers QwenImageControlNetPipeline) for face/
skin structure; Z-Image-Turbo (6B, Apache-2.0) is the best cheaper text-preserving
substitute. No improvement has measured face-fidelity at our scrub floors yet --
validate with scripts/fidelity_metrics.py first. Linked from known-limitations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 15:58:46 -07:00
Victor Kuznetsov 0d9033d63a Merge branch 'claude/modest-carson-d72243': corpus-mining provenance + removal fixes
Retained-corpus mining (2026-06-20) fixes, all gate-green:
- C2PA vendor coverage (Volcano Engine CJK legal name, ElevenLabs; TikTok/PixelBin vetted out)
- identify AI-generated vs AI-enhanced (ai_source_kind) + shared GEMINI_SPARKLE_TRUST_CONF (detect/remove threshold unify)
- text-mark over-subtraction guard (Doubao/Jimeng/Samsung)
- region-targeted regeneration for AI-enhanced composites (feather_region_composite + remove_watermark(region=))

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

# Conflicts:
#	CLAUDE.md
2026-06-20 15:39:29 -07:00
Victor KuznetsovandClaude Opus 4.8 737305858d docs: sync module map for the corpus-mining provenance + removal fixes
Update CLAUDE.md and docs/module-internals.md for: ProvenanceReport.ai_source_kind
(generated vs enhanced) and the shared GEMINI_SPARKLE_TRUST_CONF; the text-mark
over-subtraction guard; noai/tiling.feather_region_composite + the region-targeted
WatermarkRemover.remove_watermark(region=) path; the new C2PA vendor rows (Volcano
Engine Chinese legal name, ElevenLabs) and the documented TikTok/PixelBin
exclusion. Record the rejected gemini-gate-lowering experiment.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 15:34:39 -07:00
Victor KuznetsovandClaude Opus 4.8 c1971a3e8d feat(invisible): region-targeted regeneration for AI-enhanced composites
For AI-enhanced composites (digitalSourceType compositeWithTrainedAlgorithmicMedia,
identify ai_source_kind == "enhanced"; roadmap P1#8): regenerate ONLY the AI
region and preserve the real photo elsewhere, instead of regenerating the whole
frame.

- noai.tiling.feather_region_composite(base, regenerated, box, *, feather): pure,
  model-free compositor that blends the regenerated AI box back over the original
  with a feathered seam, leaving pixels OUTSIDE the box exactly equal to base.
  Fully unit-tested (outside-box exactness, interior == regenerated, hard paste at
  feather 0, monotonic seam ramp, dtype/grayscale/clamp/empty-box/shape-mismatch).
- WatermarkRemover.remove_watermark(region=, region_feather=) and the module-level
  convenience function thread it through: the remover regenerates (or tiles) the
  frame, then composites only the AI box back over the original input. The box is
  caller-supplied -- a C2PA composite manifest carries no reliable machine-readable
  region, so none is fabricated. The no-model lossless region path stays
  region_eraser.erase.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 15:34:39 -07:00
Victor KuznetsovandClaude Opus 4.8 33fddbc6fa fix(visible): over-subtraction guard for Doubao/Jimeng/Samsung text marks
Port the Gemini sparkle dark-pit guard (commit 41f6797) to the shared
TextMarkEngine reverse-alpha base (roadmap P0#8): on a dark or mid-tone
background the captured alpha can over-estimate this image's mark opacity, and
reverse-alpha leaves a darker-than-background glyph ghost instead of recovering
the true pixels. The sparkle-only fix left the text marks unhandled.

_reverse_alpha_oversubtracts predicts the reverse-alpha output PER PIXEL over the
glyph body from the INPUT ((obs - a*logo)/(1-a), the remover's own math); when
the predicted body lands more than _OVERSUB_DARK_MARGIN (25) gray levels below
the local background ring it abandons the reverse-alpha output for the footprint
and inpaints it from the original surroundings (_inpaint_footprint, wider dilate/
radius than the thin residual pass). Predicting per-pixel from the input (not the
produced output, which depends on which placement the remover picked) keeps a
cleanly captured full-strength mark byte-identical -- it predicts back to the
background everywhere, so the guard never trips on it (verified across all three
engines on white/mid/dark/midgray backgrounds).

Regression-guarded by tests/test_text_mark_oversubtraction.py: predicate True on
faint / False on clean, end-to-end no-dark-pit acceptance, clean-mark byte
identity, and textured-background footprint recovery.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 15:34:39 -07:00
Victor KuznetsovandClaude Opus 4.8 0c215b5b2f feat(identify): C2PA vendor coverage, AI-enhanced split, detect/remove threshold unify
Retained-corpus mining (2026-06-20) surfaced three provenance gaps; all are
oracle-free and regression-guarded.

- C2PA vendor coverage (roadmap): register Volcano Engine under its Chinese
  legal entity 北京火山引擎科技有限公司 (the latin "volcengine" needle misses
  those certs) -> normalizes to the same ByteDance platform; register ElevenLabs
  ("Eleven Labs Inc.", pure generative-AI) as a generator. Document the
  deliberate exclusion of TikTok Inc. and PixelBin.io/"Fynd" (provenance/transform
  signers, not generators) so they are not re-added.

- AI-generated vs AI-enhanced (roadmap): ProvenanceReport.ai_source_kind splits
  the C2PA digital-source-type into "generated" (trainedAlgorithmicMedia) vs
  "enhanced" (compositeWithTrainedAlgorithmicMedia) so a caller branches a
  full-frame scrub from a region-targeted clean. Parsed once in
  noai.c2pa._populate_registry_fields (PNG + any c2pa-python-readable container),
  with a raw head-scan fallback in identify for the non-PNG raw-blob path. CLI
  verdict reads "AI-generated (fully synthetic)" vs "AI-enhanced (real content
  with an AI-composited region)"; surfaced in --json.

- Detect-vs-remove threshold desync (P0#7): identify's sparkle threshold and the
  removal arbitration gate were two independent 0.5 constants. Unify them into the
  single GEMINI_SPARKLE_TRUST_CONF (identify imports it) so they can never drift.
  Lowering the gate to recover faint sub-0.5 sparkles was evaluated and REJECTED:
  a real Doubao text mark scores ~0.40-0.42 as a gemini match with a higher
  core-ring brightness margin than a genuine faint sparkle, so neither confidence
  nor the brightness gate separates them in [0.35, 0.5) -- lowering would trade a
  rare miss for false-positive removals on clean images. Regression-guarded by
  TestSparkleDetectRemoveAlignment (real demo sparkle at borderline opacities;
  identify and best_auto_mark must agree on either side of the line).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 15:34:20 -07:00
Victor KuznetsovandClaude Opus 4.8 e29d6624b9 docs(known-limitations): seed is a weak text-only quality lever; mark qwen floors certified
Measured (openai_1, 0.10, seeds 0-4): seed barely moves whole-image fidelity
(img LPIPS 0.062-0.065, SSIM/PSNR flat) but shifts text legibility (OCR CER
0.241-0.290, ~17% spread) -- it changes which details regenerate, not the level.
So per-image best-of-N-seed is a weak text-only lever (pin a seed in prod; reserve
best-of-N for text-heavy premium). Also retitle the qwen section "certified floors"
and drop the now-stale "uncertified / run seed-repeat / floor 0.30" tails.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 15:26:27 -07:00
Victor KuznetsovandClaude Opus 4.8 7dddfef14e docs: certify qwen scrub floors (OpenAI 0.10 seed-robust, Gemini 0.25)
Oracle seed-repeat + floor refinement (2026-06-20, data/qwen_in):
- OpenAI floor 0.10 is SEED-ROBUST: 0.05 and 0.075 still detected; 0.10 clean on
  seeds 0-4 (5/5) -> a random seed is safe.
- Gemini floor lowered 0.30 -> 0.25 (0.20 still detected, 0.25 clean on both
  images). Single-seed (seed 0): the Gemini oracle rate-limits volume seed-repeat,
  so pin a seed in prod rather than relying on seed-robustness there.

Re-measured fidelity at the certified floors (controlnet 0.15 vs Qwen 0.25 for
Gemini): faces still favor controlnet (ArcFace 0.546 vs 0.382, lapvar 0.62 vs
0.40); the short-CJK text case is now a TIE (gemini_1 0.037 vs 0.037 -- the earlier
Qwen 0.000 was at 0.30, not the floor). Qwen's text win holds on substantial
Latin/mixed text (OpenAI 0.385 vs 0.241 / 0.341 vs 0.290). Update watermark_profiles
comment, CLAUDE.md, module-internals, known-limitations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 15:16:51 -07:00
Victor KuznetsovandClaude Opus 4.8 373b910a60 docs: fix the qwen-vs-controlnet face comparison to oracle-confirmed scrub floors
The face fidelity numbers cited an equal-strength compare (both 0.15), but Qwen at
0.15 does NOT clear Gemini SynthID -- so that output is un-scrubbed and the compare
is invalid. Per the methodology rule (compare fidelity only between outputs where
SynthID is removed in BOTH), restate faces at each pipeline's scrub floor
(controlnet 0.15 / Qwen 0.30): ArcFace identity 0.546 vs 0.331, lapvar 0.62 vs 0.40,
face LPIPS 0.09 vs 0.19 -- controlnet still wins faces, conclusion unchanged. Drop
the "equal strength" framing in CLAUDE.md / module-internals / known-limitations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 14:33:11 -07:00
Victor KuznetsovandClaude Opus 4.8 2d5b26ed18 test(eval): vision-transcribed ground truth for qwen_in + clean text-CER numbers
data/qwen_in/ground_truth.json is transcribed by vision (PaddleOCR mangled the
stylized Cyrillic), so the text metric scores variants against an accurate
reference instead of noisy OCR-vs-OCR. Re-measured text CER (controlnet vs qwen)
with this ground truth confirms qwen wins text across EN/RU/ZH: openai_1 0.385 vs
0.241, openai_2 0.341 vs 0.290, gemini_1 (ZH) 0.037 vs 0.000 (perfect Chinese even
at the higher 0.30 strength). Faces still favor controlnet. Refresh the numbers in
docs/known-limitations.md to this cleaner methodology.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 14:26:23 -07:00
Victor KuznetsovandClaude Opus 4.8 e29c156279 test(eval): fix the qwen_in pipeline-fidelity eval set + PaddleOCR ground-truth flow
- data/qwen_in/: a stable, committed set of 4 AI-generated images (OpenAI +
  Google, carrying SynthID/C2PA -- same class as data/samples fixtures) used to
  compare the controlnet/sdxl/qwen pipelines for fidelity. Two text-multi-script
  (incl. RU/CJK), one EN poster, one face grid. README documents the set + the
  ground-truth workflow. data/ is sdist-excluded so the wheel is unaffected.
- scripts/fidelity_metrics.py: switch text OCR from EasyOCR to PaddleOCR
  (PP-OCRv6, higher accuracy esp. CJK, single multilingual stack); split into
  `ocr` (seed a {basename: text} ground truth) and `compare` (--ground-truth for
  a clean CER vs the hand-verified reference instead of noisy OCR-vs-OCR). Spatial
  IoU-NMS keeps the best-scoring read per line so wrong-script models don't inject
  garbage over Cyrillic/CJK.
- Oracle methodology: validate the OpenAI arm FIRST (openai.com/verify is more
  accessible and the strongest Playwright/Chrome-MCP automation candidate; the
  Gemini app is more manual). Recorded in CLAUDE.md + docs/synthid.md.

Ground-truth JSON (data/qwen_in/ground_truth.json) lands in a follow-up once
hand-verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 14:17:04 -07:00
Victor KuznetsovandClaude Opus 4.8 a2c33af284 feat(scripts): fidelity_metrics.py + correct the qwen-vs-controlnet claim
Add scripts/fidelity_metrics.py: an objective eval harness comparing
watermark-removal outputs against the original (reference) across four groups
-- OCR character error rate (EasyOCR), ArcFace identity cosine (insightface),
face texture (LPIPS + Laplacian-variance ratio), and whole-image LPIPS/SSIM/
PSNR. PEP 723 inline deps so it stays out of the package / uv.lock; metrics
self-gate (faces only where faces, text only where text).

The metrics overturned an eyeball conclusion: at EQUAL strength Qwen beats
controlnet on TEXT (OpenAI typography 0.10: OCR CER 0.25 vs 0.37) but controlnet
beats Qwen on FACES (gemini_3, 18 faces, 0.15 each: Laplacian-variance retention
0.62 vs 0.41, face LPIPS 0.09 vs 0.13 -- Qwen smooths faces MORE; ArcFace
identity ~tied). So Qwen is the better TEXT-preserving remover, not a universal
fidelity win. Correct the earlier "qwen keeps faces faithful where controlnet
plasticizes" claim in CLAUDE.md, module-internals.md, known-limitations.md, README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 09:58:22 -07:00
Victor KuznetsovandClaude Opus 4.8 76e3d4154c feat(invisible): add Qwen-Image img2img pipeline (--pipeline qwen)
A third diffusion pipeline alongside sdxl/controlnet: Qwen-Image (20B MMDiT,
Apache-2.0 code AND weights) img2img. The scrub still comes from the img2img
strength; Qwen preserves text (incl. CJK) and structure markedly better than
SDXL at the scrub floor, so it over-regenerates real photos far less (directly
targets the controlnet over-regeneration that degrades real uploads).

- watermark_profiles: QWEN_MODEL_ID, normalize_profile accepts "qwen".
- WatermarkRemover: _load_qwen_pipeline (bf16, loads Qwen base unless --model
  overridden, clear ImportError if diffusers lacks the class), _run_qwen (no
  MPS fallback -- 20B is CUDA/cloud-class), dispatch in _generate_one/preload,
  pure _build_qwen_kwargs (true_cfg_scale, not guidance_scale).
- Shared _base_load_kwargs() across all three loaders (dtype + token).
- CLI --pipeline gains "qwen"; invisible_engine threads it through.
- scripts/qwen_scrub_prototype.py: standalone PEP 723 GPU experiment.

Prototype oracle floors (Modal A100-80GB, single seed, controls SynthID-positive,
PENDING seed-repeat cert): OpenAI clears at strength ~0.10, Gemini at ~0.30 (0.20
still detected), with CJK text + faces faithful where controlnet plasticizes. The
Gemini floor is higher than the shared default ladder, so pass an explicit
--strength for Gemini on this pipeline until a Qwen-specific ladder is certified.

The model-running path is CUDA-only (untestable locally); unit tests cover the
pure call-shape (_build_qwen_kwargs) and profile normalization without torch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 20:44:36 -07:00
Victor KuznetsovandClaude Opus 4.8 0c0c6c6b03 feat(invisible): sliding-window tiled diffusion for large inputs (--tile)
Add a lossless alternative to the --max-resolution downscale for large
images that OOM on MPS/GPU: regenerate in overlapping, feather-blended
tiles at native resolution.

- noai/tiling.py: pure plan_tiles (uniform tiles, last flush to edge) +
  feather_weights (strictly-positive separable taper -> partition-of-unity
  blend) + run_tiled (per-tile generate callable, decoupled from the
  pipeline). Unit-tested without the model.
- WatermarkRemover.remove_watermark: refactor _generate into _generate_one
  + a tiled branch that engages only when --tile is set and the long side
  exceeds tile_size (ControlNet canny is rebuilt per tile).
- Thread tile/tile_size/tile_overlap through InvisibleEngine and the
  invisible/all/batch CLI commands via a shared _tile_options decorator.

Verified end-to-end on the real SDXL pipeline (forced 2x2 tiling on a
1024px sample, MPS): non-degenerate output, no gross seam at tile borders.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 11:54:58 -07:00
Victor KuznetsovandClaude Opus 4.8 d5845a72f3 feat(metadata): blank AI-generator tokens in AVIF/HEIF Exif meta-box items
Closes a documented coverage gap (P2#9): an AI Software/Make/Artist/ImageDescription
token in an EXIF item (its TIFF bytes live in mdat/idat) survived remove_ai_metadata
because the top-level box stripper and (absent pillow-heif) the PIL EXIF reader can't
reach it. New isobmff.blank_ai_exif_tokens finds EXIF TIFF blocks by their II/MM
byte-order header, validates each with piexif (a coincidental II/MM run in pixels
won't parse as a TIFF IFD, so it's ignored), and overwrites any AI_GENERATOR_TOKENS-
bearing value with same-length spaces -- so box sizes and iloc offsets stay valid and
the coded image is untouched (mirrors blank_ai_xmp_packets; no iinf/iloc surgery, no
exiftool dep). Camera/editor EXIF without an AI token is preserved. Wired into
remove_ai_metadata's ISOBMFF path. Covers the realistic AI-generator-token case; xAI-
signature-in-meta-box-EXIF (Grok is JPEG-only) stays out.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 10:43:35 -07:00
Victor KuznetsovandClaude Opus 4.8 3f5d6a0af1 docs(landscape): back the DWT-DCT positive-only limitation with researched root cause + citations
Deep-research (2026-06-19, adversarially verified) confirms the open imwatermark
dwtDct mark is fragile by scheme, not by our usage: maintainers admit no 100%
clean-decode guarantee; measured ~0.79 bit accuracy clean (~38/48, below our 44
gate). Root causes (code-verified + locally reproduced): per-block max-coefficient
bit read (content flips bits) and YUV chroma 8-bit clamping on bright pixels (the
bright-flat / all-ones failure). No maintained fork or detector does this scheme
reliably (WAVES relegates it to an appendix; learned schemes are a different class;
dwtDctSvd cannot decode SDXL's dwtDct). Conclusion: keep it positive-only, rely on
C2PA. Sources: imwatermark READMEs, arXiv:2406.08337 (WMAdapter), arXiv:2401.08573
(WAVES), diffusers SDXL watermark.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 10:27:08 -07:00
Victor Kuznetsov f97fdc5b92 chore(release): v0.11.4 2026-06-19 10:06:04 -07:00
Victor KuznetsovandClaude Opus 4.8 4c8a57ec7b docs: dwtDct detector is carrier-fragile (all-ones = artifact), FLUX open-mark unresolvable
Final characterization after a positive-control sweep. The imwatermark dwtDct
round-trip fails (28-39/48, below the 44 gate) not on "high texture" as a prior
note claimed, but on a broad carrier class: the FLUX fox, doubao, a minimalist-FLAT
FLUX generation, AND a clean synthetic bright-flat fill with NO watermark all fail
identically. The degenerate all-ones decode is therefore a CARRIER ARTIFACT, not a
watermark (the no-watermark synthetic image reproduces it; a double-embed test shows
no interference). detect_invisible_watermark is positive-only: trust a hit, treat a
None as inconclusive unless a same-carrier positive control first recovers >=44.

Consequence: whether BFL hosted FLUX embeds the open DWT-DCT is unresolvable with
this detector on the available carriers (textured AND flat FLUX both fail the
control). C2PA stays the reliable FLUX signal. Low priority to chase further.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 10:03:34 -07:00
Victor KuznetsovandClaude Opus 4.8 a0a349cc66 docs: correct overstated FLUX open-watermark claim; record detector content-fragility
Earlier notes asserted BFL hosted output has no open DWT-DCT watermark. That was
overstated: the test carriers were high-texture fox images where a clean
encode->decode round-trip of a KNOWN-embedded watermark recovers only 28-35/48
bits (below the safe 44 gate), so the detector would miss a present mark there --
the None is inconclusive, not proof of absence.

Verified positive-control (2026-06-19): imwatermark dwtDct round-trips 48/48 on
synthetic carriers and on chatgpt-1.png (48/48) / firefly-1.png (45/48), but
FAILS on flux-1.png (28/48) and doubao-1.png (39/48). So invisible_watermark
detection is a positive-only signal: trust a hit, treat a miss on busy content as
inconclusive. Affects all open SD/SDXL/FLUX DWT-DCT detection. C2PA stays the
reliable FLUX identifier; whether BFL hosted embeds the open mark is unresolved
(needs a low-texture hosted sample).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 10:03:34 -07:00
Victor KuznetsovandClaude Opus 4.8 99e57c872f perf(text-mark): footprint-sized arrays in reverse-alpha CPU path
The reverse-alpha text-mark engine (Doubao/Jimeng/Samsung) allocated
full-frame arrays where only the glyph footprint is ever read:

  - _fixed_alpha_map / _aligned_alpha_map each built a full (h, w) float32
    alpha map non-zero only inside the glyph box, and two were held at once
    during removal (~96 MB of mostly-zeros on a 12 MP frame);
  - extract_mask built a full (h, w) uint8 mask that every caller cropped to
    the located box (~12 MB, rebuilt per text-mark detector on the
    memory-tight identify path).

Both now return footprint-sized arrays: the alpha helpers return the
glyph-sized block plus its placement (ax, ay, gw, gh), and extract_mask
returns the box-sized mask. _apply_reverse_alpha consumes the block
directly; the residual inpaint embeds it into one full-frame uint8 mask only
at cv2.inpaint time (which needs a full-frame mask). remove_watermark_
reverse_alpha tracks the winning region alongside best_amap to place it.

Peak allocation drops from O(image*4)x2 + O(image) to O(footprint)x2 +
one gated O(image*1) uint8 mask -- a win every consumer gets, motivated by
the 512 MB raiw.cc worker that OOMs on large decodes. GPU path untouched.

Byte-identical to the old full-frame path (verified: 17 output hashes
across the three engines, inpaint/no-inpaint, detect, and the real
doubao-1.png fixture, unchanged before/after). tests/test_text_mark_memory.py
guards it by reconstructing the old full-frame path inline and asserting
equality, so the proof survives a cv2/asset bump, and pins the O(footprint)
shape so a regression to full-frame fails loudly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 10:01:07 -07:00
Victor KuznetsovandClaude Opus 4.8 9614615001 docs(landscape): confirm BFL hosted = C2PA-only on FLUX.1 [dev] too
Lossless-PNG check across both BFL Playground model lines (FLUX.2 [pro] and
FLUX.1 [dev]) confirms the open DWT-DCT pixel watermark is absent on hosted
output regardless of model or container; only the signed C2PA manifest is present.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 09:42:03 -07:00
Victor KuznetsovandClaude Opus 4.8 9e307d020e test(c2pa): add real FLUX.2 BFL C2PA fixtures (PNG + JPEG)
flux-1.png / flux-1.jpg are real Black Forest Labs FLUX.2 [pro] Playground
outputs (signed C2PA, issuer "Black Forest Labs" + trainedAlgorithmicMedia,
manifests verified to contain no personal data). flux-1.jpg is the first
committed JPEG-with-C2PA fixture, exercising the c2pa-python non-PNG reader path
end to end. Regression tests assert both attribute to "Black Forest Labs (FLUX)".

Also documents the verified finding (n=2, 2026-06-19): BFL's hosted output carries
the signed C2PA manifest but NOT the open invisible-watermark DWT-DCT (decodes to
degenerate all-ones, chance-level vs the FLUX reference) -- the open pixel mark is
dev-inference-code-optional only. So a hosted FLUX.2 image is identified by C2PA
alone, with no open-pixel fallback once C2PA is stripped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 09:37:40 -07:00
Victor KuznetsovandClaude Opus 4.8 d4d9429328 feat(identify): attribute Canva and BytePlus C2PA; fix BytePlus->Adobe mislabel
Mining the local production corpus (25,725 imgs) surfaced two AI vendors signing
C2PA that the registry missed:
- Canva (Magic Media) signed "Canva" + trainedAlgorithmicMedia -> detected AI but
  no platform attributed (disproves the old "Canva exports strip C2PA" assumption).
- BytePlus (ByteDance international: Seedream/Seededit) signs "Byteplus Pte. Ltd.";
  the bare volcengine needle missed it, so its output was mis-attributed to "Adobe
  Firefly" via an incidental "Adobe XMP" string the fallback byte-scan picked up.

Adding both to C2PA_AI_VENDORS lets the clean manifest issuer attribute them
directly. Corpus re-run: 16 platform changes, all improvements (3 Adobe->ByteDance
fixes, 4 None/TC260->ByteDance, 9 None->Canva), 0 regressions. An attempted
signer-based attribution fallback was measured and dropped: it regressed 18 images
(friendly ByteDance label -> raw Chinese cert org; IPTC tool name pre-empted).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 21:57:21 -07:00
Victor Kuznetsov be566e8868 chore(release): v0.11.3 2026-06-18 17:28:21 -07:00
Victor KuznetsovandClaude Opus 4.8 9f6c26a439 refactor(c2pa): read manifests via official c2pa-python, keep byte-scan fallback
extract_c2pa_info now uses the c2pa-python Reader first (any container, whole
manifest store incl. ingredient manifests), falling back to the hand-rolled caBX
parser for blobs the validator rejects (synthetic/partial, broken wheel). The
issuer/source-type/SynthID/soft-binding registry scan is shared by both paths
(_populate_registry_fields), so the return-dict contract is unchanged. Also
replaces the dead `from c2pa import has_c2pa_metadata` import in metadata.py with
a real Reader presence check. c2pa-python added as a core dep (MIT/Apache, ~+5MB
RSS, no torch; wheels cover the CI matrix).

Validated on the full local spaces corpus (25,725 imgs): 0 regressions; 384
manifests newly parsed (379 non-PNG JPEG/WebP + 2 PNGs the byte-scanner missed);
3 false Adobe/Microsoft->Google attributions fixed via real-manifest parsing.

The docs/module-internals.md section for this change already landed in 41f6797.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 17:24:58 -07:00
Victor KuznetsovandClaude Opus 4.8 41f67973ce fix(visible): inpaint mid-tone Gemini sparkle instead of a dark diamond
The free `visible` path over-subtracted a faint Gemini sparkle on a
mid-tone background into a darker-than-background brown diamond instead
of removing it (2026-06-18 prod NPS report, "the watermark was not
removed, just its color changed"). The existing over-subtraction guard
only tripped when reverse-alpha drove a footprint pixel fully negative
(the issue #30 dark-background black-pit case); on a mid-tone background
the over-subtraction darkens the core well below the background without
any pixel crossing zero, so the gate missed it and shipped the dark mark.

Add a second over-subtraction signal to `_reverse_alpha_oversubtracts`:
predict the reverse-alpha output at the bright core, (core - a*logo)/(1-a),
and route to the footprint inpaint when it lands more than
`_OVERSUB_DARK_MARGIN` (25) gray levels below the local background ring.
Calibrated wide: clean removals predict within ~12 of background
(demo_banana ~-1), the prod regression ~-40, the issue #30 dark case ~-82.
Corpus-validated on the 479 detected Gemini images: 10 switch reverse-alpha
to inpaint, all of them dark-diamond cases that improve or match; the
other 469 stay byte-identical. demo_banana stays on the reverse-alpha
path (byte-identical).

Also crop both reverse-alpha helpers to the region they actually touch,
a pure O(image) -> O(mark) win that is byte-identical to the full-frame
math (a uint8<->float32 round-trip is exact):
- `GeminiEngine._core_and_bg` converts only the footprint+ring crop to
  gray, not the whole frame (~70 ms -> 0.1 ms on a 12 MP image; it runs
  for both the alpha-gain estimate and the new gate). Verified identical
  across 479 images; detector confidence unchanged.
- `TextMarkEngine._apply_reverse_alpha` computes the blend on the glyph
  crop only (`amap` is zero outside it, so the math is a no-op there):
  ~275 ms -> ~2 ms per placement on a 12 MP frame, up to 2 placements per
  removal. Verified identical across 142 Doubao/Jimeng placements.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 17:19:41 -07:00
Victor Kuznetsov 09fdb4544a fix(invisible): preserve native output dimensions 2026-06-18 16:44:21 -07:00
Victor KuznetsovandClaude Sonnet 4.6 61aa76a591 perf(identify): decode the image once for all visible-mark detectors
identify(check_visible=True) ran the Gemini-sparkle detector and the
Doubao/Jimeng text-mark detector each with its own image_io.imread, so the
same bitmap was fully decoded twice. On a memory-constrained host (the raiw.cc
512 MB web worker, which runs identify on every upload) that doubled the peak
decode allocation and contributed to OOM restarts.

Decode once in identify() and pass the BGR array to both detectors. The detect
methods already accept an NDArray, so this only threads the pre-decoded array
through: detect_sparkle_confidence and the two _visible_* helpers gain an
optional image= param that, when None, preserves the old self-read behavior
(so direct callers and the cv2-missing/unreadable paths are unchanged).

Only the visible path is deduplicated; the optional check_invisible decoders
are unaffected (and off on the web hot path). Adds a test asserting
identify(check_visible=True, check_invisible=False) decodes exactly once.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 11:13:17 -07:00
dependabot[bot] cd0a79df38 chore(deps): bump the minor-and-patch group with 5 updates (#50)
Bumps the minor-and-patch group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [transformers](https://github.com/huggingface/transformers) | `5.10.2` | `5.12.1` |
| [accelerate](https://github.com/huggingface/accelerate) | `1.13.0` | `1.14.0` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.18.0` | `1.19.0` |
| [pytest](https://github.com/pytest-dev/pytest) | `9.0.3` | `9.1.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.15.16` | `0.15.17` |


Updates `transformers` from 5.10.2 to 5.12.1
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.10.2...v5.12.1)

Updates `accelerate` from 1.13.0 to 1.14.0
- [Release notes](https://github.com/huggingface/accelerate/releases)
- [Commits](https://github.com/huggingface/accelerate/compare/v1.13.0...v1.14.0)

Updates `huggingface-hub` from 1.18.0 to 1.19.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.18.0...v1.19.0)

Updates `pytest` from 9.0.3 to 9.1.0
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.3...9.1.0)

Updates `ruff` from 0.15.16 to 0.15.17
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.16...0.15.17)

---
updated-dependencies:
- dependency-name: transformers
  dependency-version: 5.12.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: accelerate
  dependency-version: 1.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: pytest
  dependency-version: 9.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.15.17
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-18 10:07:41 -07:00
Victor KuznetsovandClaude Opus 4.8 4c6b56f888 lower(strength): drop vendor-adaptive floor to OpenAI 0.10 / Google 0.15
A 2026-06-14 oracle re-test on the deployed Modal controlnet worker (v0.10.0)
cleared SynthID at OpenAI 0.10 (2 photoreal) and Google 0.15 (2 native
2816x1536, retiring the "native >= 0.30" guess), while a pixel sweep showed the
2026-06-04 cert floors (0.20/0.30) over-regenerated for no efficacy gain
(Google MAE -20% at 0.15). Lowers OPENAI_STRENGTH 0.20->0.10, GEMINI_STRENGTH
and UNKNOWN_STRENGTH 0.30->0.15.

Caveats documented in watermark_profiles.py + docs: removal near this floor is
seed-non-deterministic (a service must pin a verified seed), and the n=2 re-test
did not cover flat-graphic hard cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 13:17:11 -07:00
Victor KuznetsovandClaude Opus 4.8 41a2af2ecb fix(cli): preserve SynthID uncertainty in no-visible-mark message
The 'no signal' branch of the visible no-mark path claimed 'No AI provenance
signal found either', which reads as 'the image is clean'. A missing metadata
proxy is not proof an invisible pixel watermark (SynthID) is absent: it cannot
be detected once metadata is gone and may have been stripped upstream. The
message now preserves that uncertainty and routes to both 'all' (regenerate
pixels) and 'erase'. Regression-guarded by the SynthID/all asserts in
test_cli.py. CLAUDE.md visible-command note updated to match.

Also adds a 'Scope and non-goals' section (CLAUDE.md + README): removing
AI-provenance marks on the user's own content is in scope; stripping
stock/paid-content watermarks (Shutterstock/Getty/iStock, classifieds) is out
of scope by principle, not by difficulty.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 19:30:49 -07:00
Victor KuznetsovandClaude Opus 4.8 d8cdc9f478 docs: correct stale strength-ladder values in remove_watermark docstring
The convenience wrapper's docstring still quoted the pre-2026-06 ladder
(0.10 OpenAI / 0.15 Google / 0.15 unknown). The live constants in
watermark_profiles.py are 0.20 / 0.30 / 0.30, applied to both the controlnet
and sdxl pipelines. Docstring only; behaviour was already correct via
vendor_for_strength + resolve_strength.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 09:51:09 -07:00
Victor Kuznetsov 6237429610 chore(release): v0.11.2 2026-06-12 21:37:04 -07:00
Victor KuznetsovandClaude Opus 4.8 30b56f0ea3 fix(cli): stop silent passthrough when visible finds no known mark
When `visible --mark auto` (or an explicit `--mark` with detection on) found
no registered mark, it exited 0 without writing output -- which a wrapping
service reads as success and re-serves the unchanged input. ~74% of real
uploads carry no registered visible mark, so this was the dominant "it didn't
work" / NPS score-0 failure mode.

Now it runs a cheap metadata-only identify, prints actionable guidance (route
to `all` for an invisible/metadata mark, or `erase` for an arbitrary logo),
writes no output file, and exits EXIT_NO_VISIBLE_MARK (2) -- distinct from
success (0) and a hard error (1) so the caller can surface the message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 21:36:56 -07:00
Victor KuznetsovandClaude Opus 4.8 b08405bece chore(release): v0.11.1
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 12:15:20 -07:00
Victor KuznetsovandClaude Opus 4.8 28569bd05d fix(gemini): recover sub-0.85 corner sparkles via top-K fusion selection
The 256->512 detection-search widening (v0.8) let a large, low-gradient
shape match outrank a genuine mid-size corner sparkle whose raw NCC sits
below the 0.85 corner-promote gate, so `identify` read `unknown` on Gemini
images that v0.7.2 caught (reporter osachub: scale-48 sparkle on light
bedding -- true sparkle spatial 0.775 / grad 0.960 / fusion 0.676, but the
size-weighted argmax locked onto a decoy at spatial 0.628 / grad 0.036).

detect_watermark now keeps the top-K (_SELECT_TOPK=3) size-weighted
candidates (NMS-deduped) plus the corner-promote candidate, scores each by
full fusion (spatial+gradient+variance) via the extracted _grad_var_scores
helper, and selects the highest -- the gradient term lifts the true sparkle
over the decoy. Ranking by the SIZE-WEIGHTED score (not a raw-NCC argmax)
preserves tiny-patch suppression: a raw-NCC argmax re-admitted 16-18px
content false positives (14/65 doubao + 4/11 jimeng visible images). Top-K
adds zero flips on the doubao/jimeng corpora and leaves the 495-image Gemini
set unchanged (479 detected) while recovering the reporter's image at 0.676.

- _grad_var_scores: gradient/variance scoring factored out of detect_watermark
- confidence = best_fused (drop the duplicated fusion recompute)
- tests: rename test_promotion_is_what_rescues_it ->
  test_size_weighted_search_alone_traps_on_the_decoy (corner-promote is no
  longer the sole rescue path); add a deterministic regression test mirroring
  the real spatial/grad signature
- docs: module-internals.md detector section + CLAUDE.md mechanism map

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 12:04:20 -07:00
Victor KuznetsovandClaude Fable 5 9feea4ac1e Slim CLAUDE.md: move module internals, limitations, landscape research to docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 15:50:03 -07:00
Victor KuznetsovandClaude Fable 5 3055aa6c4a test: patch is_available in full-pipeline all tests (fix no-gpu CI)
test_all_basic / test_all_visible_step_uses_registry asserted exit 0 but did
not patch is_available, so on CI (core+dev only, no gpu) they took the skip
branch and hit the new non-zero exit. Passed locally where gpu is present.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:07:05 -07:00
Victor KuznetsovandClaude Fable 5 c8bc4b7c68 chore(release): v0.11.0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:03:51 -07:00
Victor KuznetsovandClaude Fable 5 a8e218acf6 Make all fail loudly when the gpu extra is missing
Step 2 (invisible/SynthID) was skipped with a quiet inline warning and the
run still exited 0, so a missing [gpu] extra was mistaken for a clean result
(recurring #14/#47). Add a prominent end-of-run banner and a non-zero exit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 09:58:49 -07:00
Victor KuznetsovandClaude Fable 5 ad7e4ee08b feat(identify): close 3 detector gaps found on the spaces corpus (06-05..06-11)
- AIGC: parse the bare ``AIGC{...}`` blob form (label glued to its JSON in a
  JPEG APP segment near the JFIF header), and scan both raw-JSON forms in one
  fall-through loop so a quoted ``"AIGC"`` later in an XMP packet no longer
  shadows a real bare label earlier in the file (3 files read unknown before).
- Integrity clash rule 2: a camera device + an AI marker from the SAME C2PA
  manifest (Google Pixel Magic Editor / Pixel Studio edit chain) is a legitimate
  edit chain, not a contradiction. Fire only when the AI marker's source is
  independent of the camera's manifest; pure cameras (Leica/Sony/Nikon) are
  unaffected (2 Pixel files mis-flagged before).
- New c2pa_cloud_manifest detector: surface a C2PA 2.4 Durable Content
  Credentials cloud-manifest reference (Adobe cai-manifests.adobe.com) as a
  medium provenance signal when the embedded manifest is stripped. Provenance
  only, never asserts is_ai (2 files read fully unknown before).

identify reuses its already-loaded scan head for the cloud check (no second
read). +7 tests; CLAUDE.md + README synced.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 09:28:15 -07:00
Victor KuznetsovandClaude Opus 4.8 22bc171806 ci: bump checkout to v6 (Node 24), note dismissed torch alert
actions/checkout@v4 ran on the deprecated Node 20; bump to v6 to match
test.yml/publish.yml. Document the dismissed Dependabot torch alert
(GHSA-rrmf-rvhw-rf47, not_used: no torch.jit usage, gpu-extra-only, no patch).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 16:00:35 -07:00
Victor KuznetsovandClaude Opus 4.8 d763581ed3 chore(release): v0.10.3
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 15:53:50 -07:00
Victor KuznetsovandClaude Opus 4.8 0d99f403fb ci: auto-distribute releases to Homebrew tap + HF Space
distribute.yml fans a published GitHub Release out to the channels that
would otherwise be manual: it waits for the sdist on PyPI, bumps the
Homebrew formula (HOMEBREW_TAP_TOKEN) and factory-rebuilds the HF Space
(HF_TOKEN). PyPI stays on publish.yml; conda-forge on its autotick bot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 15:47:03 -07:00
Victor KuznetsovandClaude Opus 4.8 e78e5f1154 docs: address HN feedback in README (scope, limitations, honest use case)
From the HN front-page discussion (news.ycombinator.com/item?id=48200569):
- Threat model: drop the 'third-party classifiers' overclaim. State scope
  honestly: it removes SynthID / visible marks / provenance metadata, does NOT
  defeat trained AI-vs-real classifiers (Hive), and watermarks are a weak trust
  signal to begin with.
- Replace the 'preserving art / historical record' use case (criticized as not
  holding) with the defensible one: clearing an overstated AI label from your
  own lightly-AI-edited photo.
- Add a Limitations section: lossless visible/metadata vs lossy content-dependent
  SynthID path, no local self-verify, large images not tiled yet, out-of-scope.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 11:41:39 -07:00
Victor KuznetsovandClaude Opus 4.8 0a77d3198e chore(release): v0.10.2
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 10:38:50 -07:00
Victor KuznetsovandClaude Opus 4.8 9aea5f240f chore: improve discoverability (PyPI keywords/classifiers, README badges)
Research-informed metadata for organic dev discovery:
- pyproject: add a keywords field (was absent; biggest PyPI search gap) and
  expand classifiers (audience, console, security, AI, utilities); rewrite the
  summary noun-first, naming Nano Banana / SynthID / C2PA verbatim.
- README: add PyPI version, Python versions, downloads, and license badges.

GitHub topics (comfyui, watermark-remover) and the repo description were
updated out of band. PyPI metadata ships on the next release.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 10:34:43 -07:00
Victor KuznetsovandClaude Opus 4.8 c3ddf8a801 docs: document Homebrew, conda-forge, and ComfyUI distribution channels
- README: add Homebrew install, conda (conda-forge, in review), and a
  ComfyUI custom-nodes section.
- CLAUDE.md: per-channel release/bump cadence (Homebrew formula, conda-forge
  autotick bot, ComfyUI Registry); note pip_check: false on the conda recipe.
- Add packaging/conda/recipe.yaml (v1, noarch core-only), verified green on
  conda-forge/staged-recipes PR #33674.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 19:29:40 -07:00
298 changed files with 49793 additions and 9043 deletions

No files matched your search

+153
View File
@@ -0,0 +1,153 @@
---
globs: ["src/**/*.py", "tests/**/*.py", "scripts/**/*.py", "pyproject.toml", "uv.lock", "maintain.sh", ".github/workflows/*.yml"]
description: Command contracts, project gate, typing boundaries, model-adjacent test invariants, and the detection-path measurement rule.
---
# Development invariants
## Command contracts
Every single-image command declares `source` with `dir_okay=False`; `batch` declares its directory with `file_okay=False`. Keep `tests/test_cli_robustness.py::TestDirectoryInputIsRejected` as the regression guard.
Exit-code and no-signal behavior is a public contract. Read the command-line section of [`../../docs/module-internals.md`](../../docs/module-internals.md) before changing it.
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
`device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. It is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds.
The same rule applies to install hints: name the extra that actually makes the command work (`qwen-zimage`, not `diffusion`), and keep the printed command shell-quoted -- bare `pkg[extra]` is a glob in zsh.
## Local gate
Run `bash maintain.sh` from the repository root. The authoritative type gate is scoped to `src/`; full-project Pyright can exhaust Node memory on the ML dependency graph.
Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.
From a worktree, `uv run` imports the package from the MAIN checkout -- that is where the editable install points. A script measuring a worktree's edit must insert that worktree's `src` at `sys.path[0]` and assert `module.__file__` resolves inside it, or it silently compares unmodified code against itself.
## Model-adjacent tests
Do not classify an entire module as untestable because its main path downloads a model. Keep pure behavior covered without downloads, including:
- target-size selection in `test_invisible_engine.py`;
- unsharp and adaptive-polish helpers in `test_humanizer.py`;
- tiling geometry and blending in `test_tiling.py`;
- prompt-embedding cache keying, storage round-trip, and the cross-pipeline reuse
that lets a stack load without its text encoder, in `test_qwen_zimage_pipeline.py`;
- the face stack's dtype, in `test_qwen_zimage_pipeline.py`. A subclass that changes
the pipeline dtype for its own global model must not change the inherited face
stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a
face. When one profile inherits another's stage, guard the invariants that stage
relies on, not just the code path.
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
mirror another, compare them field by field rather than pinning the values you happen
to know about, so the next field added on one side and not the other fails at the
seam. Two of these defaults drifted in practice and neither needed a GPU to catch;
the incident is recorded in `docs/module-internals.md`. Keep the comparison free of
an exception table: a field that needs one is a field that belongs elsewhere, which
is what `force` turned out to be.
A defaults comparison is not a forwarding test, and the two fail differently. Pin the
VALUE at the seam, not just the name -- `_run_invisible` passed the whole suite with
`controlnet_conditioning_scale` hardcoded, because nothing asserted the caller's value
arrived. `test_every_field_arrives_at_the_engine_with_the_caller_s_value` drives the
real seam with every field set off its default, so one test covers the whole bag
instead of one assertion per knob.
Count the seams before believing a knob is covered. Each of `force` and
`controlnet_conditioning_scale` reaches the engine through TWO paths -- `remove_all`
versus `remove_batch(mode="all")` for the first, `_run_invisible` versus `_batch_engine`
for the second -- and in both cases guarding one path left the other free to hardcode a
constant with a green suite. The mode-parametrized guards in
`TestRemoveBatchLibrary::test_force_reaches_the_scrub_gate_in_every_scrubbing_mode` and
`TestBatchCommand::test_batch_controlnet_scale_flows_to_the_cached_engine` exist because
that is what actually happened.
Use availability checks only for paths that actually load large models.
A verdict branch must be reachable with the dependency stack as shipped. High-confidence
C2PA attribution required `signingCredential.trusted`, a code the reader emits only when a
trust anchor list is loaded, and none ships -- so the branch was dead in production for
every vendor while a hand-built info dict stamping that code kept it green from 0.27.0
through 0.30.0. Assert reachability against the committed fixtures rather than a synthesized
status set; `TestIdentifyRealSamples::test_no_committed_fixture_reports_a_trusted_signer`
is that guard. Read a capability the stack does not have as a missing input, never as a
negative finding.
## One measurement, one gate seam
A detector is split into a trust-level-blind scan and a verdict that applies the
threshold, so `detect` and `detect_both` reach the same numbers by construction. Two
rules follow, and both were broken in practice before they were written down:
- A per-mark demotion goes in the `_post_gate` hook (or, for a whole-scan precondition
like LibLibAI's size floor, in `_scan`) -- never in a `detect` override. An override
is invisible to `detect_both`, so the RunningHub and Yuanbao anchor gates silently
stopped applying on the arbiter's perception path. `TestSinglePassPerception` is the
guard: it asserts `detect_both` equals two `detect` calls field for field.
- Detection and the removal mask must read ONE sweep. The winning box travels on
`TextMarkDetection.match_box` and the registry threads the detection into the mask
builder; a mask path that re-runs its own sweep is how the two drift apart.
The C2PA manifest-store JSON is NOT stable across reads: the reader regenerates manifest
URNs and instance ids. Compare the derived `c2pa_info`, never the raw store.
A third seam reaches the same verdict: `collect_metadata_record` ->
`evidence_from_metadata_record` -> `identify_from_evidence`, the path a caller uses when
collection and verdict run on different machines. Its contract is equality with
`identify(path, check_visible=False, check_invisible=False)` on the same image, and it
can break from EITHER side -- a region the collector stops walking, or a placement the
file path learns to read and the record does not. `tests/test_metadata_record.py` pins
it over the tracked fixtures; a separate local evaluation corpus catches placements the
fixtures do not cover. Change either side and re-run both.
Before changing anything in the detection path, record the detectors' exact verdicts
over a local sample first and diff them after. A refactor here is only correct if that
record is byte-identical, and a green test suite does not establish that on its own. A
change that is meant to FIX detection is the exception that proves the rule: the diff
must then be exactly the files you intended to change, named in advance.
When the baseline is a published release rather than the previous commit, get it with
`uv run --isolated --no-project --with 'remove-ai-watermarks[heif]==<version>' python <script>`:
it resolves that release from PyPI without touching the working tree or the editable
install. Quote the extra -- bare `pkg[extra]` is a glob in zsh.
Two corollaries in `dwt_dct.py`, where "close enough" has an exact meaning. The bit test
is `peak % 36 > 18.0` and for uint8 input the exact Haar value is a multiple of 0.5, so it
lands ON the threshold once per 72 blocks and a 1-ulp difference flips real bits: leave the
transform to `pywt` however slow it looks, because its C convolution contracts into an FMA
that numpy has no ufunc for (`docs/module-internals.md` carries what that costs). And
`tests/test_invisible_watermark.py` is `skipif(not is_available())` -- without the `detect`
extra the upstream-parity test never runs, so a green suite is not evidence here and the
verdict record is not optional.
## A certified operating point is data, not a constant
The video SynthID default is only meaningful as a row in
`data/evaluations/video-synthid-oracle.csv`, so
`test_shipped_defaults_match_a_certified_manifest_row` derives the pin from that
manifest instead of restating literals. Pinning `noise_std` alone had let `long_side`
and `fps` -- two thirds of what the oracle was actually shown -- move with a green
suite.
The certified profile is a perturbation-to-signal ratio, not a bare `noise_std`, so
the latent scaling factor is gated in `load_video_vae_runtime`, carried on
`VideoVaeRuntime`, and passed into encode and decode: the validated value and the
applied value are one measurement. Anything that produces oracle evidence loads
through that function. `video_synthid_sweep.py` hand-rolled the load and was the one
path exempt from the gate it exists to feed, which is exactly backwards.
Prove a video-path refactor the same way the detection path is proven, and without
needing an oracle carrier: build a clip from a tracked fixture with ffmpeg, run the
engine before and after, and require an identical output sha256. Keep the generated
media outside the repository.
Frame sampling is compared as `timestamp + 1e-9 >= next_sample_time`, so mutating
that `>=` to `>` changes nothing and a green suite proves nothing. Mutate the phase
or the period instead -- starting the accumulator half a period late shifts the
selection by one frame while keeping the count identical, which is the drift a
frame-count check cannot see and what
`test_pairing_follows_the_engine_sampling_rule_not_just_the_frame_count` exists to
catch.
Environment setup, dependency recovery, CI behavior, and fixture policy: [`../../docs/development.md`](../../docs/development.md).
-14
View File
@@ -1,14 +0,0 @@
{
"permissions": {
"allow": [
"WebSearch",
"WebFetch"
]
},
"enabledPlugins": {
"pyright-lsp@claude-plugins-official": true,
"context7@claude-plugins-official": true,
"code-simplifier@claude-plugins-official": true,
"claude-md-management@claude-plugins-official": true
}
}
+1 -1
View File
@@ -1,3 +1,3 @@
# HuggingFace token (optional; only needed for gated/private models)
# Get yours at: https://huggingface.co/settings/tokens
HF_TOKEN=
# HF_TOKEN=
+15
View File
@@ -14,6 +14,21 @@ updates:
update-types:
- "minor"
- "patch"
# The `all` extra must remain jointly resolvable. These candidates violate
# upstream constraints: TrustMark still requires NumPy 1.x, OpenCV 4.12+
# requires NumPy 2 on newer Python versions, and this project keeps tokenizers
# below 0.23 for the stable Transformers path. Keep Dependabot on the
# compatible lines until those constraints move.
ignore:
- dependency-name: "numpy"
versions:
- ">=2"
- dependency-name: "opencv-python-headless"
versions:
- ">=4.12"
- dependency-name: "tokenizers"
versions:
- ">=0.23"
- package-ecosystem: "github-actions"
directory: "/"
+179
View File
@@ -0,0 +1,179 @@
name: Distribute on release
# Fans a published GitHub Release out to the channels that need a nudge.
# PyPI is handled by publish.yml. This workflow event-drives the three channels
# that would otherwise be manual:
# - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
# - HF Space: factory-rebuild so it reinstalls the latest sdist from PyPI.
# - ComfyUI: sync, test, and publish the node against the exact library release.
# All wait for the freshly published sdist to appear on PyPI first, since the
# Release event fires in parallel with publish.yml's upload.
on:
release:
types: [published]
workflow_dispatch:
inputs:
version:
description: "Version to distribute (no leading v), e.g. 0.10.3. Blank = latest on PyPI."
required: false
jobs:
resolve:
runs-on: ubuntu-latest
outputs:
version: ${{ steps.v.outputs.version }}
url: ${{ steps.sdist.outputs.url }}
sha: ${{ steps.sdist.outputs.sha }}
steps:
- name: Resolve target version
id: v
run: |
set -euo pipefail
if [ "${{ github.event_name }}" = "release" ]; then
v="${GITHUB_REF_NAME#v}"
elif [ -n "${{ github.event.inputs.version }}" ]; then
v="${{ github.event.inputs.version }}"
else
v=$(curl -fsSL https://pypi.org/pypi/remove-ai-watermarks/json \
| python3 -c "import sys,json;print(json.load(sys.stdin)['info']['version'])")
fi
echo "version=$v" >> "$GITHUB_OUTPUT"
echo "Target version: $v"
- name: Wait for the sdist on PyPI, capture url + sha256
id: sdist
run: |
set -euo pipefail
v="${{ steps.v.outputs.version }}"
for i in $(seq 1 30); do
if rel=$(curl -fsSL "https://pypi.org/pypi/remove-ai-watermarks/$v/json" 2>/dev/null); then
url=$(echo "$rel" | python3 -c "import sys,json;d=json.load(sys.stdin);print(next((u['url'] for u in d['urls'] if u['packagetype']=='sdist'),''))")
sha=$(echo "$rel" | python3 -c "import sys,json;d=json.load(sys.stdin);print(next((u['digests']['sha256'] for u in d['urls'] if u['packagetype']=='sdist'),''))")
if [ -n "$url" ] && [ -n "$sha" ]; then
echo "url=$url" >> "$GITHUB_OUTPUT"
echo "sha=$sha" >> "$GITHUB_OUTPUT"
echo "Found sdist for $v"
exit 0
fi
fi
echo "sdist for $v not on PyPI yet (attempt $i), waiting..."
sleep 20
done
echo "::error::sdist for $v did not appear on PyPI in time"
exit 1
homebrew:
needs: resolve
runs-on: ubuntu-latest
steps:
- name: Checkout the tap
uses: actions/checkout@v7
with:
repository: wiltodelta/homebrew-tap
token: ${{ secrets.HOMEBREW_TAP_TOKEN }}
- name: Bump the formula
env:
URL: ${{ needs.resolve.outputs.url }}
SHA: ${{ needs.resolve.outputs.sha }}
VERSION: ${{ needs.resolve.outputs.version }}
run: |
set -euo pipefail
F=Formula/remove-ai-watermarks.rb
sed -i -E "s|url \"[^\"]*\"|url \"$URL\"|" "$F"
sed -i -E "s|sha256 \"[^\"]*\"|sha256 \"$SHA\"|" "$F"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add "$F"
git commit -m "remove-ai-watermarks $VERSION" || { echo "Formula already current"; exit 0; }
git push
hf-space:
needs: resolve
runs-on: ubuntu-latest
steps:
- name: Factory-rebuild the HuggingFace Space
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python3 -m pip install --quiet huggingface_hub
python3 - <<'PY'
import os
from huggingface_hub import HfApi
HfApi(token=os.environ["HF_TOKEN"]).restart_space(
"wiltodelta/remove-ai-watermarks", factory_reboot=True
)
print("HF Space factory rebuild triggered")
PY
comfyui:
needs: resolve
runs-on: ubuntu-latest
steps:
- name: Sync and publish the ComfyUI nodes
env:
GH_TOKEN: ${{ secrets.COMFYUI_RELEASE_TOKEN }}
run: |
set -euo pipefail
repo=wiltodelta/ComfyUI-remove-ai-watermarks
workflow=sync-library.yml
dispatched_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)
gh workflow run "$workflow" \
--repo "$repo" \
--ref main \
--raw-field library_version="${{ needs.resolve.outputs.version }}"
run_id=""
for attempt in $(seq 1 30); do
run_id=$(
gh run list \
--repo "$repo" \
--workflow "$workflow" \
--event workflow_dispatch \
--limit 10 \
--json databaseId,createdAt \
| python3 -c '
import json
import sys
dispatched_at = sys.argv[1]
runs = json.load(sys.stdin)
print(next(
(run["databaseId"] for run in runs if run["createdAt"] >= dispatched_at),
"",
))
' "$dispatched_at"
)
if [ -n "$run_id" ]; then
break
fi
echo "ComfyUI sync run not visible yet (attempt $attempt), waiting..."
sleep 2
done
if [ -z "$run_id" ]; then
echo "::error::The dispatched ComfyUI sync run did not appear"
exit 1
fi
for attempt in $(seq 1 180); do
read -r status conclusion <<< "$(
gh run view "$run_id" \
--repo "$repo" \
--json status,conclusion \
--jq '[.status, (.conclusion // "")] | @tsv'
)"
if [ "$status" = "completed" ]; then
if [ "$conclusion" = "success" ]; then
echo "ComfyUI sync run $run_id succeeded"
exit 0
fi
echo "::error::ComfyUI sync run $run_id completed with $conclusion"
exit 1
fi
echo "ComfyUI sync run $run_id is $status (attempt $attempt), waiting..."
sleep 5
done
echo "::error::ComfyUI sync run $run_id did not finish in time"
exit 1
+1 -1
View File
@@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-latest
environment: pypi
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
+62 -9
View File
@@ -13,7 +13,7 @@ jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
- name: Sync dev environment
run: uv sync --frozen --extra dev
@@ -28,15 +28,28 @@ jobs:
strategy:
fail-fast: false
matrix:
# Cross-OS x Python-floor/recent. The default-install surface only
# needs the core deps plus dev tooling (no GPU extra): the model-running
# paths are availability-gated and skip when torch/diffusers are absent,
# so this matrix exercises metadata, identify, visible, and the cv2
# eraser on every OS without pulling the heavy ML stack.
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.10", "3.12"]
# Exercise every supported minor. The floor and latest minor run on all
# operating systems; intermediate minors run on Linux to bound the
# matrix without leaving a declared Python version untested.
include:
- os: ubuntu-latest
python-version: "3.11"
- os: macos-latest
python-version: "3.11"
- os: windows-latest
python-version: "3.11"
- os: ubuntu-latest
python-version: "3.12"
- os: ubuntu-latest
python-version: "3.13"
- os: ubuntu-latest
python-version: "3.14"
- os: macos-latest
python-version: "3.14"
- os: windows-latest
python-version: "3.14"
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
with:
python-version: ${{ matrix.python-version }}
@@ -44,3 +57,43 @@ jobs:
run: uv sync --frozen --extra dev
- name: Run tests
run: uv run pytest -q
transport-contract-python-314:
name: versioned transport contracts py3.14
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
with:
python-version: "3.14"
- name: Create clean runtime contract environment
run: uv venv --python 3.14 .venv-contract
- name: Install runtime contract dependencies
run: >-
uv pip install --python .venv-contract/bin/python
".[pixels,heif]" "numpy>=2,<3" "pytest>=8"
- name: Run versioned transport contract tests
run: >-
.venv-contract/bin/python -m pytest -q
tests/test_forensic_metadata.py
tests/test_metadata_record.py
tests/test_pixel_evidence.py
video-e2e:
name: video full-clip end-to-end
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
with:
python-version: "3.12"
- name: Install ffmpeg
run: sudo apt-get update && sudo apt-get install --yes ffmpeg
- name: Sync dev environment
run: uv sync --frozen --extra dev
- name: Run full-clip video test
run: >-
uv run pytest -vv -o faulthandler_timeout=60
tests/test_video.py::TestVideoVisibleFullClip
tests/test_video_fidelity_probe.py
+18 -10
View File
@@ -4,6 +4,8 @@ __pycache__/
*.egg-info/
dist/
build/
tmp/
.sc/
# Environment secrets
.env
@@ -18,8 +20,7 @@ Thumbs.db
*.swp
*.swo
# SynthID corpus reference fills (synthetic black/white calibration tiles,
# regenerable; the labeled pos/neg/cleaned images ARE tracked, see README)
# Generated calibration inputs
data/synthid_corpus/refs/
# Reference materials
@@ -33,11 +34,7 @@ yolov8n.pt
.claude/settings.local.json
.claude/scheduled_tasks.lock
# Visible-watermark alpha calibration. The solid black/gray/white CAPTURES are
# committed (content-free: a solid colour + the watermark; the source for
# scripts/visible_alpha_solve.py so the alpha assets are reproducible). The
# synthetic seeds (regenerable) and any real-content validation download (a real
# generated scene, kept local for privacy) are NOT committed.
# Generated calibration inputs and local evaluation artifacts
data/doubao_capture/seeds/
data/jimeng_capture/seeds/
data/jimeng_capture/captures/jimeng_content_*.png
@@ -46,7 +43,18 @@ data/gemini_capture/captures/gemini_content_*.png
data/samsung_capture/seeds/
data/samsung_capture/captures/samsung_content_*
# Leftover GFPGAN weights dir from the retired face-restore experiments
# (GFPGAN wrote RetinaFace/parsing weights to a CWD ./gfpgan/weights/ working
# dir on first use). Runtime artifact, never committed.
# Runtime model artifacts
gfpgan/
# Local evaluation data
.local-eval/
# Other harnesses' generated config. Claude's sync owns these; a committed copy
# is a second source nobody reconciles. Anchored so a same-named source file
# deeper in the tree is not caught.
/.agents/
/.codex/
/sch_*.yaml
/.opencode/
/opencode.json
/opencode.jsonc
+33 -75
View File
@@ -1,96 +1,54 @@
# Remove-AI-Watermarks
# Remove AI Watermarks
You are a **principal Python engineer** maintaining a CLI tool and library for removing visible and invisible AI watermarks from images.
You are a **principal Python engineer** maintaining a CLI tool and library for removing visible and invisible AI provenance watermarks.
## Scope and non-goals
The project gives users control over provenance marks on content they generated or edited themselves. It does not automatically remove stock-agency, marketplace, classifieds, tiled-preview, or other marks that protect a third party's paid or copyrighted asset.
- Add visible templates only for AI-generation labels.
- Do not add stock, agency, or classifieds marks to `watermark_registry.py`.
- Keep `erase --region` generic and user-directed; do not build an automatic stock-watermark remover on it.
Full boundary and legal context: [`docs/legal-and-safety.md`](docs/legal-and-safety.md).
## How to run
- `uv run remove-ai-watermarks all <image.png> -o <output.png>` — full pipeline (visible + invisible + metadata). Same diffusion knobs as `invisible` below, plus the visible-pass `--inpaint/--no-inpaint`/`--inpaint-method`.
- `uv run remove-ai-watermarks invisible <image.png> -o <out.png>` — diffusion SynthID removal. **Full knob set** (kept identical across `invisible`/`all`/`batch`): `--strength` (vendor-adaptive default), `--steps`, `--guidance-scale` (CFG, default 7.5), `--pipeline sdxl|controlnet` (default `controlnet`), `--controlnet-scale`, `--model` (HF model id, default SDXL base), `--device`, `--seed`, `--hf-token`, `--max-resolution`/`--min-resolution`, `--upscaler lanczos|esrgan`, `--humanize` (Analog Humanizer grain), `--unsharp` (final sharpen), and `--adaptive-polish/--no-adaptive-polish` (**ON by default**; detail-targeted polish that self-gates to a no-op where there is no deficit). `--auto` is deprecated and now a no-op that only warns (the polish it used to enable is ON by default).
- `uv run remove-ai-watermarks visible <image.png> -o <out.png>` — known-visible-mark removal, CPU, no GPU. Reverse-alpha based: each mark is removed by inverting its captured alpha map. `--mark auto` (default) picks the strongest detected of the Gemini sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, and the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-LEFT, locale-specific — Italian variant calibrated); `--mark gemini` / `--mark doubao` / `--mark jimeng` / `--mark samsung` force one (choices come from the registry). Gemini/Doubao recover pixels exactly with no inpaint at native; **Jimeng and Samsung add an always-on thin residual inpaint over the glyph footprint** (their marks re-rasterize per image, so reverse-alpha alone leaves a faint outline). For arbitrary logos/objects use `erase`.
- `uv run remove-ai-watermarks erase <image.png> --region x,y,w,h -o <out.png>` — universal region eraser (any logo/object, any position). `--backend cv2` (default, no deps) or `--backend lama` (big-LaMa via onnxruntime, extra `lama`); `--region` is repeatable.
- `uv run remove-ai-watermarks identify <image>` — provenance verdict (platform + watermark inventory + confidence); `--json` for machine output, `--no-visible` to skip the cv2 sparkle detector
- `uv run remove-ai-watermarks metadata <image.png> --check` — inspect AI metadata (C2PA, EXIF, PNG chunks)
- `uv run remove-ai-watermarks metadata <image.png> --remove -o <out.png>` — strip all AI metadata
- `uv run remove-ai-watermarks batch <directory>` — process every supported image in a directory (output defaults to `<directory>_clean/`, set with `-o`). `--mode visible|invisible|metadata|all` (default `visible`); the invisible/all path reuses the **full `invisible` knob set above** (`--strength`/`--steps`/`--guidance-scale`/`--pipeline`/`--controlnet-scale`/`--model`/`--device`/`--max-resolution`/`--min-resolution`/`--upscaler`/`--seed`/`--hf-token`/`--humanize`/`--unsharp`/`--adaptive-polish`), plus `--inpaint/--no-inpaint` for the visible pass. `--adaptive-polish` is ON by default; `--auto` is deprecated and a no-op that only warns. One engine cached per pipeline; the polish is resolved once before the loop.
```bash
uv run remove-ai-watermarks --help
bash maintain.sh
```
## Test and lint
- **CI** (`.github/workflows/test.yml`): runs on push to `main` + every PR. A `lint` job (ubuntu: `ruff check` + `ruff format --check`) plus a `test` matrix (ubuntu/macos/windows x py3.10/3.12) that does `uv sync --frozen --extra dev` then `pytest`. The matrix installs only core + dev (no `gpu` extra), so the GPU/model-running tests skip there and it exercises the metadata/identify/visible/cv2-eraser surface on all three OSes. Keep `uv.lock` valid (don't break `--frozen`) when editing `pyproject.toml`. `publish.yml` stays release-only and now verifies the release tag matches the `pyproject.toml` version (fails the build on a mismatch) before building, then uploads via `uv publish` (PyPI trusted publishing over OIDC, no token — replaced the `pypa/gh-action-pypi-publish` action so the upload no longer depends on that action's bundled twine accepting the Metadata-Version; the `id-token: write` permission + `pypi` environment + workflow filename are unchanged, so PyPI's trusted-publisher entry still matches). **Release flow:** bump the version in `pyproject.toml` + `src/remove_ai_watermarks/__init__.py` + `uv.lock` (the project's own `[[package]]` entry — find it with `grep -n 'name = "remove-ai-watermarks"' uv.lock`, the `version =` line right below it, ~line 2246), commit `chore(release): vX.Y.Z`, `git tag -a vX.Y.Z -m vX.Y.Z` (annotated — `git tag` without `-m` errors here), push `main` + the tag, then `gh release create vX.Y.Z` — **PyPI publish triggers on the GitHub Release `published` event, NOT on the tag push**, so the tag alone does not publish. **Sdist must exclude `data/`** (`[tool.hatch.build.targets.sdist] exclude = ["/data"]`): hatchling's default sdist bundles all VCS-tracked files, so the committed `data/` test corpora (the multi-hundred-MB synthid_corpus images + the visible-mark captures) pushed the **0.8.0** sdist past PyPI's per-project file-size limit (400 "File too large") — the wheel uploaded but the sdist was rejected, so 0.8.0 shipped wheel-only and 0.8.1 carried the fix. The wheel only ships `src/` (via `[tool.hatch.build.targets.wheel] packages`), so it was never affected. **A failed PyPI upload of one artifact still leaves the other live and you cannot re-upload the same version** — fix the build and cut the next patch. **Build backend is unpinned `hatchling`** (`[build-system] requires`) since 2026-06-09. History: it was pinned `<1.31` because hatchling 1.30.0 made Metadata-Version 2.5 (PEP 794) the default and the twine bundled in `pypa/gh-action-pypi-publish@release/v1` rejected it (`"'2.5' is not a valid Metadata-Version"`), which **failed the v0.8.3 PyPI upload on 2026-06-01**; hatchling 1.30.1 reverted the default to 2.4. After the workflow moved to `uv publish` (whose uploader accepts 2.5) the pin was belt-and-suspenders only, and once v0.9.0 + v0.10.0 both published wheel+sdist through that path (verified on PyPI) it was dropped. If a future hatchling flips the default to 2.5 again and some consumer chokes, re-pin with a dated comment.
- `bash maintain.sh` — uv-outdated, uv-secure, ruff check/fix, ruff format, pyright (scoped `src/`, see the OOM note below), pytest -n auto. The helper tools live in the `dev` extra (`pytest-xdist`, plus `uv-outdated`/`uv-secure` marker-gated to py3.12+ so the py3.10 resolution stays solvable) — a bare env without `--extra dev` does not have them.
- **Strict pyright is clean across `src/` (0 errors).** The cv2/torch/diffusers boundary files (`gemini_engine`, `region_eraser`, `doubao_engine`, `humanizer`, `invisible_engine`, `noai/watermark_remover`) carry a documented per-file `# pyright:` relax pragma that turns off only the unknown-type / untyped-third-party rules — those libs ship no usable types, so strict typing there fights the ecosystem. Pure-logic files stay fully strict; `typings/piexif/__init__.pyi` is a local stub so `metadata.py`/`extractor.py` resolve piexif. Public ndarray-returning signatures on the relaxed engines are still annotated `NDArray[Any]` so strict consumers (`cli.py`) stay clean. When touching a relaxed file, prefer fixing real issues over widening the pragma; keep the pragma scoped to genuinely-untyped boundaries. (`uv-secure` is clean since idna was bumped 3.11 -> 3.16, fixing GHSA-65pc-fj4g-8rjx, and aiohttp 3.13.5 -> 3.14.0 via `uv lock --upgrade-package aiohttp`, fixing GHSA-hg6j-4rv6-33pg + GHSA-jg22-mg44-37j8. (The old basicsr Dependabot alert (GHSA-86w8-vhw6-q9qq) is resolved by removal: the experimental `restore` extra was retired and basicsr is no longer anywhere in the dependency tree.)
- **Full-project `uv run pyright` (no path) OOMs/crashes node on this ML-heavy repo** (emits a `libnode` stack frame, no summary) — a known environment limit, not a code error. Gate with `uv run --extra dev --extra gpu pyright src/` (completes, authoritative) or scope to changed files; also run `uv run ruff check` and `uv run pytest` directly.
- Run `uv run` from the repo root — from another cwd it falls back to a bare env without numpy/cv2/torch.
- **Stale `trustmark` remnant in site-packages after an extras change:** the `trustmark` package downloads model weights INTO its own package dir, so when a narrower `uv sync` prunes the package, a `trustmark/models/` directory survives as an empty namespace package. Symptom: pyright `"TrustMark" is unknown import symbol` on `trustmark_detector.py` and `find_spec("trustmark")` returning a loader-less spec (so `is_available()` lies True). Fix: `rm -rf .venv/lib/python3.12/site-packages/trustmark` (regenerable weights cache).
- To add a dev tool (pytest/ruff/pyright) into the env, use `uv sync --frozen --extra dev --extra gpu`, **never `uv pip install`** — `uv pip install` re-resolves and rewrites `uv.lock`, which silently bumped `transformers` to a build incompatible with the pinned `diffusers` (`cannot import name 'Qwen3VLForConditionalGeneration'`) and broke every `identify`/metadata import. Recovery: `git checkout uv.lock && uv sync --frozen --extra gpu --extra dev`. The `gpu` extra holds `diffusers`/`transformers`/`torch`, so a bare `uv sync` (no extras) removes them; `noai/__init__` is now **lazy** (PEP 562 `__getattr__`, so importing `identify`/`metadata` no longer pulls `watermark_remover`/torch), so a bare env breaks only when the removal pipeline is actually invoked, not on import. `maintain.sh`'s `uv sync --all-extras` also pulls the heavy `trustmark`/`lama` wheels (pytorch-lightning, onnxruntime) — fine on a good connection, but on flaky DNS sync only `--extra gpu --extra dev` and run the lint/test steps by hand.
- Metadata/C2PA tests assert against real committed fixtures in `data/samples/` (`chatgpt-*.png` = OpenAI C2PA, `firefly-1.png` = Adobe, `mj-*` = Midjourney IPTC, `doubao-1.png` = ByteDance Doubao with the China TC260 `<TC260:AIGC>` XMP label **and** a visible "豆包AI生成" text mark bottom-right; `grok-1.jpg` = xAI Grok with its EXIF-only `Signature:` blob + UUID `Artist` and no C2PA/SynthID/IPTC); synthetic byte blobs cover the JPEG/ISOBMFF format paths. The "non-AI / clean photo" control is no longer in `data/samples/` -- the `clean_photo` conftest fixture serves a verified-negative image from the corpus `neg/` set (skips if the corpus is absent).
- SynthID reference corpus: `scripts/synthid_corpus.py` ingests labeled images into `data/synthid_corpus/`. The labeled `images/` (`pos/` `neg/` `cleaned/`) are **committed** (public repo -- review every image for private content before adding; `manifest.csv` is kept in sync with the files on disk, one row per tracked image); only the synthetic `refs/` calibration fills are gitignored. See its README for the collection protocol and verification oracles. **`cleaned/` examples must be produced by a CURRENT shipped removal method** -- the default SDXL img2img pass (optionally `--max-resolution`). Do NOT archive cleaned outputs from methods that are no longer in the pipeline (ctrlregen, the old text/face-protection, IP-Adapter FaceID, CodeFormer) or from the experimental opt-in paths (controlnet, face restore) as corpus examples; a cleaned reference should represent the canonical removal, and a removed method's output is not a reproducible example. Keep those experiment outputs in a local working dir, never in the committed corpus.
Run `uv` from the repository root. Command selection, options, defaults, and examples live in [`docs/cli.md`](docs/cli.md). Before changing command routing, no-signal behavior, or exit codes, read the command-line section of [`docs/module-internals.md`](docs/module-internals.md).
## Configuration
- GPU/ML modules (invisible_engine, watermark_remover) are optional — guard imports with `is_available()` checks
- Optional detection extras: `detect` (imwatermark — open SD/SDXL/FLUX watermark) and `trustmark` (Adobe TrustMark decoder; pulls torch + downloads weights). Both are guarded by `is_available()` and skipped by `identify` when absent.
- Optional `esrgan` extra (spandrel only): Real-ESRGAN pre-diffusion super-resolution for small inputs (`upscaler.py`, CLI `--upscaler esrgan` on `invisible`/`all`/`batch`). Guarded by `upscaler.is_available()`; the default upscaler stays Lanczos (cv2, no deps) and the engine falls back to Lanczos when the extra is absent or the model errors. spandrel is MIT and pulls NO basicsr (only torch/torchvision/safetensors/numpy/einops); Real-ESRGAN weights are BSD-3-Clause and download on first use via `torch.hub` (never bundled). Kept OUT of `all` (heavy + model download).
- Tests for the *model-running* paths are limited to availability checks (multi-GB downloads). But the **pure helpers inside ML-adjacent modules are unit-tested without any download** and must stay that way: `_target_size` (native-vs-downscale-cap-vs-upscale-floor, `test_invisible_engine.py`), `humanizer.unsharp_mask`/`adaptive_polish` (`test_humanizer.py`), and the MPS->CPU fallback control flow via mocked pipelines (`test_img2img_runner.py`, 100% cover). Don't skip these as "ML, needs a model" — only `remove_watermark`/the diffusion bodies do.
GPU and ML modules are optional. Guard their imports with `is_available()`.
## Key modules
Optional features and installation groups are documented in [`docs/installation.md`](docs/installation.md). Model-running paths may use availability tests, while pure helpers in ML-adjacent modules must remain unit-tested without downloads.
- `noai/c2pa.py` — PNG chunk parser; use `extract_c2pa_chunk(path)` to get raw caBX payload, `has_c2pa_metadata(path)` to detect. Do not reimplement chunk parsing. `extract_c2pa_info(path)` sets `synthid_watermark`/`synthid_vendors` when the manifest is signed by a SynthID-using vendor, and `soft_binding`/`soft_binding_vendors` when a `c2pa.soft-binding` `alg` names a forensic-watermark vendor (`soft_binding_vendors_in(buffer)` is the shared byte-scan, used by both the PNG parser and the non-PNG binary path). PNG/caBX chunk reads are clamped to the remaining file size (`safe_length = min(length, remaining)`; skipped chunks use seek) so a malformed huge `length` cannot drive a multi-GB allocation (shared safety discipline matching `isobmff.scan_c2pa_region`).
- `noai/constants.py` — PNG_SIGNATURE, C2PA_CHUNK_TYPE, C2PA_SIGNATURES, and `C2PA_AI_VENDORS` — the single `C2paAiVendor` registry of C2PA-signing vendors (issuer byte, resolved org name, the `identify` platform label, and a `synthid` flag), from which `C2PA_ISSUERS`, `SYNTHID_C2PA_ISSUERS` (issuers that pair SynthID with C2PA: Google, OpenAI), and `identify._ISSUER_PLATFORM` are all **derived** — plus `C2PA_SOFT_BINDINGS` (soft-binding `alg` prefix → forensic-watermark vendor: Adobe TrustMark, Digimarc, Imatag, Steg.AI, Microsoft, ...). Add a new C2PA vendor as one `C2PA_AI_VENDORS` entry (never edit the derived dicts), a new soft-binding to `C2PA_SOFT_BINDINGS`; not inline.
- `metadata.py` — `scan_head(path, size=1MB)` is the shared input for every C2PA/AIGC/IPTC byte scan: first `size` bytes plus the payloads of any provenance metadata found beyond that window — for ISOBMFF, the late provenance boxes from `isobmff.scan_c2pa_region` (catches a manifest after a large `mdat`); for **PNG**, the late `tEXt`/`iTXt`/`zTXt`/`eXIf`/`iCCP` chunks from `_png_late_metadata` (catches an XMP/EXIF packet appended after a large `IDAT`, e.g. a TC260 AIGC label at ~2.7 MB). Behavior-neutral (`f.read(size)`) for non-ISOBMFF inputs and for any file that fits within `size`. Use it instead of `open().read(1MB)` for any new marker scan. **Memoized per (path, size, mtime)** (added 2026-06-09, `_scan_head_cached` lru_cache, `maxsize=8`): one `identify`/`get_ai_metadata` call fans out to ~8 byte-scan detectors that each re-read the same file head, so the cache turns those into a single read; the mtime key invalidates on change, a stat failure falls back to an uncached read. `synthid_source(path)` returns the vendor name(s) if the C2PA manifest implies a SynthID pixel watermark, else None. Format-agnostic: PNG via the caBX parser, JPEG/WebP/AVIF/HEIF/JXL via a binary scan (C2PA marker + SynthID issuer + AI-source marker). `get_ai_metadata` surfaces the verdict, and `metadata --check` prints it as a callout. Both `get_ai_metadata` and `has_ai_metadata` guard the PIL open with `except Exception` (HEIC/unknown formats raise non-OSError) and fall through to the binary scan. `xai_signature(path)` detects xAI/Grok's EXIF-only scheme (`ImageDescription` = `Signature: <base64>` + UUID `Artist`); it feeds `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify`. `iptc_ai_system(path)` detects the IPTC Photo Metadata 2025.1 AI-disclosure XMP properties (`IPTC_AI_FIELD_MARKERS` = `AISystemUsed`/`AISystemVersionUsed`/`AIPromptInformation`/`AIPromptWriterName`) and returns the `AISystemUsed` generator name (or `"fields present"`). `remove_ai_metadata` routes **ISOBMFF video** (`.mp4`/`.mov`/`.m4v`) through the same `isobmff.strip_c2pa_boxes` as AVIF/HEIF (MP4 is ISOBMFF), and `_scrub_ai_exif` removes the xAI signature + AI-generator EXIF tags on JPEG output. `strip_c2pa_boxes` is **fail-safe** on a malformed box: it returns the original bytes unchanged with a logged warning instead of truncating the tail to EOF (detection-only `scan_c2pa_region` still stops at a malformed box). `_png_late_metadata` clamps each late-chunk read to the remaining file size (`safe_length = min(length, remaining)`) so a malformed `length` cannot drive a multi-GB allocation, AND advances the cursor by `safe_length` (not the raw `length`) so an inflated length cannot jump past EOF and abort the scan, silently skipping a genuine AI-label chunk after it.
- `identify.py` — the OpenAI rollout caveat is keyed on `_vendor_of(synthid) == "OpenAI"` (not a raw substring over the issuer + verdict blob). `identify(path)` aggregates every locally-readable signal (C2PA issuer→platform, C2PA soft-binding forensic-watermark vendor, IPTC "Made with AI" + IPTC 2025.1 `AISystemUsed`, embedded SD/ComfyUI params, SynthID proxy, xAI/Grok EXIF signature via `metadata.xai_signature`, the China TC260 AIGC label via `metadata.aigc_label`, the HuggingFace `hf-job-id` job marker via `metadata.huggingface_job`, the Samsung Galaxy AI editing marker via `metadata.samsung_genai`, the visible marks — Gemini sparkle plus the ByteDance Doubao 豆包AI生成 / Jimeng 即梦AI / Samsung Galaxy AI "Contenuti generati dall'AI" text marks via the `watermark_registry` — open invisible watermark, Adobe TrustMark via `trustmark_detector`) into one `ProvenanceReport`. `is_ai_generated` is True or None (never asserted False — stripped metadata is not proof of clean origin). The `hf_job`, visible-mark, and Samsung `samsung_genai` signals are **medium** confidence: each lifts an otherwise-Unknown verdict to a tentative AI (`hf_only` / `visible_only` / `samsung_only`, parallel branches; `visible_only` fires on any `visible_*` signal) but is excluded from the high-confidence `ai_from_metadata` set, so none overrides a hard metadata signal. **Visible-mark detection** (`check_visible`, signals `visible_sparkle` / `visible_doubao` / `visible_jimeng` / `visible_samsung`): the Gemini sparkle keeps its own file-level path (`_visible_sparkle` → `gemini_engine.detect_sparkle_confidence`, promoted only at confidence ≥ `_SPARKLE_THRESHOLD` 0.5; corpus-tuned to separate Gemini sparkles ≥0.56 from non-sparkle ≤0.49), while Doubao/Jimeng/Samsung reuse the registry detectors (`_visible_text_marks` → `watermark_registry`, iterating `_VISIBLE_MARK_PLATFORM`), each gated by its own engine NCC threshold via `MarkDetection.detected` (Doubao 0.4, Jimeng 0.45, Samsung 0.4). Doubao/Jimeng are normally also caught by the TC260 AIGC metadata label and Samsung by its C2PA + `genAIType` marker, so the visible path is their stripped-metadata fallback. Visible marks set `platform` only when no harder signal already did, and (like the sparkle) are excluded from integrity-clash vendor claims. The cv2 dependency lives in the engines, not here. **`import identify` is deliberately light** (~21 MB; ~36 MB with cv2 loaded by a visible-mark run, ~106 MB for a full `check_visible` run): it imports only the pure `noai.c2pa`/`noai.constants` submodules, and `noai/__init__` is lazy (see "Test and lint"), so torch/diffusers are NOT pulled at import even in a full `gpu`/`detect` install — fits a 512 MB host. The heavy paths are opt-in: `check_invisible=True` needs the `detect`/`trustmark` extras (each pulls **torch**; TrustMark also **downloads weights**), so on a core-only deploy leave `check_invisible` off (it is a no-op there anyway). Before the lazy `__init__`, the mere presence of torch in the env inflated `import identify` to ~420 MB. **C2PA platform attribution is device-token-first, issuer-scan fallback** (`_device_platform` scans manifest bytes for `_DEVICE_C2PA_PLATFORM` tokens, then `_attribute_platform`/`_ISSUER_PLATFORM`). **Why, verified on real signed files 2026-05-26:** the old issuer-only byte-scan matched ANY issuer substring anywhere, so multi-entity manifests mis-attributed -- Leica→"Truepic" (a signing authority in the trust chain), Nikon→"Adobe Firefly" (XMP-toolkit "Adobe" + the sample's "Adobe_MAX" name), Pixel→"Google (Gemini)" ("Google LLC" cert org), Truepic→"Google". A distinctive device token wins instead. **Token distinctiveness is load-bearing:** bare `b"Truepic"` mis-fires (it appears in unrelated trust chains -- it mis-attributed the OpenAI `chatgpt-1.png` fixture), so the token is the specific `b"Truepic_Lens"` from the Lens SDK claim generator; likewise `b"Pixel Camera"` (cert CN) not bare `b"Pixel"`. `_DEVICE_C2PA_PLATFORM` lists ONLY tokens **verified against a real C2PA file**: Leica (`lc_c2pa`/`Leica Camera`), Nikon (`NIKON`), Pixel (`Pixel Camera` -- from a real Pixel 10 Pro file attached to c2pa-rs issue #1609/#1554), Sony (`sony.sig`/`sony.cert` -- Sony's own C2PA assertion namespace, verified on a real Sony PXW-Z300 file; NOT bare "Sony" which is a common EXIF Make), Truepic (`Truepic_Lens`). Canon/Bria have **no public direct-download C2PA sample** (checked exhaustively: GitHub issue/PR attachments, contentcredentials gallery, HF datasets -- all upload-to-verify or token-gated; Canon's only public file was a self-signed hobbyist CR3, not factory), so they stay unmapped until a real file is captured (same fixture discipline as Grok/Doubao). The Sony sample is video (MP4) -- our ISOBMFF C2PA path detects it; Sony Alpha stills likely share the `sony.*` namespace but are not separately verified. **Samsung Galaxy + ASUS Gallery live in a separate `_SIGNER_C2PA_PLATFORM` (scaLine truncated
- `watermark_registry.py` — **single catalog of known visible watermarks**, the unified "find known marks in their usual places, recognize, remove" entry. **Reverse-alpha based by policy**: a mark is listed only once a real alpha map has been captured for it, and removal inverts that map (`original = (wm - a*logo)/(1-a)`) — Gemini recovers cleanly with no inpaint (its sparkle alpha comes from a pure-black capture, so it is near-exact), while **Doubao, Jimeng, and Samsung all add an always-on THIN residual inpaint** over the glyph footprint (their text marks re-rasterize + jitter a few px per image, so a single capture cannot pixel-cancel them; the inpaint blends into the reverse-alpha-recovered pixels). Arbitrary-region inpainting still lives in `region_eraser`/`erase`. Each `KnownMark` ties a key to {usual `location`, `in_auto` flag, `recovery` (="reverse-alpha"), a `detect` adapter → uniform `MarkDetection`, a `remove` adapter}. Entries today: `gemini` (bottom-right sparkle), `doubao` (bottom-right "豆包AI生成"), `jimeng` (bottom-right "★ 即梦AI"), and `samsung` (bottom-**LEFT** "✦ Contenuti generati dall'AI", Samsung Galaxy AI, Italian locale). `detect_marks` scans all; `best_auto_mark` picks the highest-confidence detection. **Cross-engine confidences aren't directly comparable**, so the gemini adapter applies the corpus-validated 0.5 sparkle threshold (`_GEMINI_AUTO_MIN_CONF`) for its `detected` flag — otherwise the gemini engine's loose internal threshold weakly fires (~0.36) on the Doubao text and hijacks `auto`. The shape-keyed Doubao/Jimeng/Samsung NCC detectors don't cross-fire (jimeng scores ~0.22 on the Doubao strip, well under its 0.45 threshold; Samsung is bottom-left so it shares no corner with the others, and scored 0.0 on Doubao/Jimeng captures and they 0.0 on a real Samsung photo), so `auto` picks the right one. `cli.cmd_visible` is registry-driven: `--mark auto` → `best_auto_mark`, `--mark <key>` → that mark; `--mark` choices come from `mark_keys()`. **`cli._remove_visible_auto` is the shared visible-removal helper used by `cmd_all`/`cmd_batch` too** (they no longer hardcode `GeminiEngine`), so `all`/`batch` remove Doubao/Jimeng/Samsung text marks, not just the Gemini sparkle (regression-guarded by `test_all_visible_step_uses_registry`). The three text-mark adapters were consolidated 2026-06-09: a single `_text_mark(key, label, location)` builds the registry row from one parameterized `_text_mark_detect`/`_text_mark_remove` pair (reverse-alpha only when detected/forced AND `reverse_alpha_available`, else skipped — no inpaint); the gemini adapters stay bespoke. Add a new visible mark = one `_text_mark(...)` row + its `TextMarkConfig` (with a captured alpha map); do not re-add per-mark `if` branches or copy-paste adapters. **Alpha-on-save policy (issue #30):** `cli._write_bgr_with_alpha` rejoins the input's alpha plane **unchanged** — it must NOT zero alpha in the watermark bbox. Reverse-alpha (and `erase` inpaint) recover real pixels there, so zeroing alpha punched a transparent hole that renders as a solid **white box** on any non-transparent viewer (Gemini app exports are opaque RGBA, so every user hit it; regression-guarded by `test_visible_keeps_alpha_opaque_in_watermark_region`). The registry `remove()` still returns its region (used for `inpaint_residual` positioning), but the CLI no longer uses it to clear alpha.
- `gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU). `detect_sparkle_confidence(path)` is the file-level entry point used by `identify.py`. The public entry points normalize a grayscale (2D) or RGBA (4-channel) input to BGR up front so a non-BGR image does not crash the cv2 pipeline. **Detection localization (issue #36):** `detect_watermark`'s global multi-scale NCC search applies a size weight (`(scale/96)**0.5`) that suppresses tiny-patch false positives but can let a larger, mediocre match (e.g. a bright collar in a portrait) outrank a small, near-perfect sparkle in the corner — so a faint sparkle on a busy background scored below threshold and read as clean (the regression osachub reported from widening the search window 256px->512px between v0.7.2 and v0.8.8). `_corner_promote` adds a bottom-right-corner raw-NCC pass on top of the global search: a match with raw NCC >= `_CORNER_PROMOTE_NCC` 0.85 that beats the global pick overrides it (it only ever replaces a lower-fidelity pick, so it cannot weaken an existing detection), rescuing the buried sparkle without reverting the wider window. The corner side is **relative-clamped** (`_CORNER_PROMOTE_FRAC` 0.20 of the short side, clamped to `[_CORNER_PROMOTE_MIN` 96, `_CORNER_PROMOTE_MAX` 384`]`): a fixed 256px is a true corner on a large image but covers ~70% of a small portrait, where a real photo raw-matches the star at ~0.81 (relative tightening drops that worst case to ~0.69, while the upper clamp stops the corner ballooning on huge images where a real photo reached ~0.83 at 512px). The 0.85 gate sits midway between the worst real-photo corner match (~0.78 across native + downscaled negatives) and a genuine faint sparkle (~0.93), so promotion adds true detections with zero corpus false positives (Gemini's sparkle sits ~60-160px from the corner at fixed margins, covered by the [96, 384] band at every measured size). Regression-guarded by `test_gemini_engine.py::TestCornerPromotion`. **Removal is reverse-alpha with an over-subtraction guard** (`remove_watermark` → `_reverse_alpha_blend`, else `_inpaint_footprint`): the sparkle alpha is computed (`alpha = max(R,G,B)/255`) from the bundled sparkle-on-black captures `assets/gemini_bg_{96,48}.png` (the capture max is ~130, NOT 255 — the sparkle is a ~51%-opaque white overlay, so `alpha` maxes at ~0.51, which is CORRECT for the capture, not under-exposed). The alpha is near-exact only when the real mark's effective opacity matches the capture, which holds on bright/flat backgrounds — re-verified clean on `demo_banana_before.png` 2026-05-31. **Issue #30 (dark-background black pit):** on a dark/textured background (e.g. grass, ~73) the real sparkle's effective opacity is LOWER than the captured 0.51, so the fixed-alpha reverse blend OVER-subtracts (`watermarked - a*logo` goes negative) and drives the footprint to black — the white sparkle becomes a black diamond. `remove_watermark` now detects this via `_reverse_alpha_oversubtracts` (fraction of footprint pixels with `alpha >= _FOOTPRINT_ALPHA` 0.1 whose numerator < 0 exceeds `_OVERSUB_FOOTPRINT_FRAC` 0.05) and **inpaints the footprint** (`_inpaint_footprint`, cv2 NS over the dilated alpha mask) from the surrounding pixels instead. **Behavior-neutral on the working case:** a bright background over-subtracts at ~0% so reverse-alpha is used and the output is byte-identical to before (verified: demo_banana 0.0 frac vs issue-#30 grass 0.61 frac; regression-guarded by `test_gemini_engine.py::TestOverSubtractionGuard`, which composites the sparkle at a reduced effective alpha to reproduce the mismatch). **Under-subtraction (the symmetric case, fixed 2026-06-03):** some real Gemini sparkles are rendered MORE opaque than the captured ~0.51, so the fixed-alpha reverse blend UNDER-subtracts and leaves a bright sparkle residual the detector still fires on (measured on the spaces corpus: a visible-removal audit through the registry path left a detectable sparkle on a meaningful fraction of marks, all under-removals, NOT a background-brightness class — failures and successes had the same input confidence and the same background-luma distribution; the discriminator was the removal delta itself). `remove_watermark` now estimates a per-image alpha gain (`_estimate_alpha_gain`: effective sparkle opacity at the bright core vs the local background ring, `a_eff/a_cap`, clamped `[1.0, _ALPHA_GAIN_MAX` 1.94`]`) and scales the alpha to match before the over-sub/blend branch. The gain cleanly separates on the corpus (under-removed marks ~1.47, cleanly-removed ~1.00), and a deadband (`_ALPHA_GAIN_DEADBAND` 1.05) keeps a matching sparkle **byte-identical** to the pre-fix output, so the fix is purely additive (0 regressions on the audit set; the over-sub guard still runs on the scaled alpha as the safety net for an over-shooting estimate). Regression-guarded by `test_gemini_engine.py::TestUnderSubtractionGain` (composites a more-opaque-than-captLine truncated
- `_text_mark_engine.py` — **shared base for the three reverse-alpha text-mark engines (Doubao/Jimeng/Samsung), extracted 2026-06-09** (they were ~90% byte-identical clones). `TextMarkEngine(config: TextMarkConfig)` owns the whole `locate → extract_mask → detect → _fixed/_aligned_alpha_map → _apply_reverse_alpha → remove_watermark_reverse_alpha` pipeline (+ the asset-keyed `load_alpha_template`/`glyph_silhouette`/`template_match_score` caches). Each engine module is now a thin subclass: it supplies only its `TextMarkConfig` (the tuned constants, the bundled asset, and the bounded structural deltas — `corner` br/bl, `margin_floor` 4/2, `morph_open_size` 5/3, `min_gw` 8/16) plus the test-facing module shims (`_alpha_template`/`_glyph_silhouette`/`_template_match_score` + the constants). Behavior is byte-exact vs the old per-engine code (the three engine test suites pass unchanged). Gemini stays a SEPARATE engine (its multi-size fixed-slot sparkle model is genuinely different). Add a new text mark = a new `TextMarkConfig` + a thin subclass + one registry `_text_mark(...)` row. The engine bullets below describe each mark's calibration history; the LOGIC lives here.
- `doubao_engine.py` — **a thin `_text_mark_engine.TextMarkEngine` subclass (config only) since 2026-06-09.** visible Doubao "豆包AI生成" remover/detector (cv2/numpy, no GPU). `DoubaoEngine.locate` anchors a bottom-right box by **geometry** (mark scales with image WIDTH), `extract_mask` pulls the light, low-chroma glyphs (the detection candidate) using a per-pixel channel-spread proxy `sat = roi.max(axis=2) - roi.min(axis=2)` (no HSV conversion). `detect` is **shape-consistent**: it matches the bundled alpha glyph silhouette (`assets/doubao_alpha.png`) against the candidate via zero-mean normalized correlation (`_template_match_score`, cv2 `TM_CCOEFF_NORMED`), gated at `DETECT_NCC_THRESHOLD` 0.4 over a small `DETECT_MIN_COVERAGE` floor. Keying on glyph SHAPE (not coverage heuristics) fixed #23 (corpus FP 7/1243). **Removal = reverse-alpha + thin residual inpaint** (`remove_watermark_reverse_alpha`): `original = (wm - a*logo)/(1-a)` from the bundled alpha map + `_ALPHA_LOGO_BGR` (pure white) + `_ALPHA_*_FRAC` geometry, then a deliberately THIN inpaint (`_RESIDUAL_*`, `INPAINT_NS`) over the glyph footprint clears leftover edges without smearing. **Alpha is rebuilt by `scripts/visible_alpha_solve.py` (the careful gray-self solve: cubic background fit, mean over channels, full halo, unblurred), same recipe as Jimeng** — the captures are committed in `data/doubao_capture/captures/`. **Removal aligns ALWAYS** (no `_ALPHA_NATIVE_BAND` fast-path): it tries fixed geometry AND `_aligned_alpha_map`'s `TM_CCOEFF_NORMED` scale+position search and keeps the lower-residual one — the mark is re-rasterized and a few px off per image, so fixed geometry alone leaves a visible outline even at 2048. **The locate box (`WM_*`) is generous (0.22 wide, margins 0.004) and reaches close to the corner** — a tight box (the old 0.185 / margin 0.012) let a corner-ward shift fall OUTSIDE the alignment search, so the align missed and a readable outline survived; regression-guarded by `test_recovers_shifted_mark_on_texture` (composes the alpha shifted on a known texture; old box ~29 vs new ~1 mean residual). **Issue #13 follow-up defect (found 2026-05-31): the SHIPPED Doubao removal left a clearly READABLE "豆包AI生成" outline on the real `doubao-1.png` sample, while `detect` returned conf 0.0 (it is fooled by a thin outline) so `test_reverse_alpha_removes_mark` passed and the old "56/56 clean" claim was detector-measured, not visual.** Root cause: bad alpha (under-estimated, max ~0.65) + fixed-no-inpaint + tight box; the careful rebuild + always-align + thin inpaint + wide box takes it from a readable outline to faint texture-level traces (parity with Jimeng — a single capture cannot pixel-cancel a per-image re-rasterized mark). **Lesson: a detector-only removal test is insufficient; assert visual residual (the textured-shift test).** **`extract_mask` guards a degenerate ROI (`bh < 16 or bw < 16` -> empty mask, skips cv2):** the always-align removal scores each placement with a residual `detect(out)`, and on an extremely wide/short image (e.g. 2048x1, `test_wide_short_does_not_raise`) that fed cv2's GaussianBlur a ~1-px-tall ROI and **faulted natively on Windows py3.12 (access violation, non-deterministic — one CI cell went red while a re-run passed)**; the old at-native path never ran `detect` on degenerate sizes. Real images always clear the guard (the `WM_*` box floors are `max(16, …)` height / `max(40, …)` width), so it only short-circuits slivers. `reverse_alpha_available` is just "asset present"; the registry gates removal on `detect`. The shipped third-party `_refs/zhengsuanfa_doubao_alpha_120x20.png` is NOT a usable alpha (verified 2026-05-29). Arbitrary-region inpainting is `region_eraser`/`erase`.
- `jimeng_engine.py` — **a thin `TextMarkEngine` subclass (config only) since 2026-06-09.** visible Jimeng / Dreamina "★ 即梦AI" remover/detector (cv2/numpy, no GPU), built 2026-05-30 from issue #13's solid captures (@powersee). Shares the base with `doubao_engine`: `locate` anchors a bottom-right box by **geometry** (scales with WIDTH), `extract_mask` pulls the light low-chroma glyphs (white top-hat + grayish + min-luma), `detect` matches the bundled "即梦AI" glyph silhouette (`assets/jimeng_alpha.png`) via `TM_CCOEFF_NORMED` over a coverage floor. Threshold `DETECT_NCC_THRESHOLD` **0.45** cleanly separates real Jimeng marks (>=0.81) from the Doubao strip (0.21) and other AI output (0.0), so the two ByteDance marks don't cross-fire in `--mark auto`. **Logo is pure white (255,255,255)** (`_ALPHA_LOGO_BGR`; the white capture + an L-pair-solve confirm ~254.6); compositing is **sRGB, not linear** (a linear-light solve tripled the cross-residual). **Alpha rebuilt by `scripts/visible_alpha_solve.py` from the GRAY capture** (`data/jimeng_capture/captures/`, the solid captures now committed): `a = (I - B)/(255 - B)`, B a per-capture **cubic** background fit over the non-glyph pixels, **averaged over channels, full halo extent (down to a~0.02), unblurred**. Gray (bg ~132) is the deliberate choice over black: it is the best proxy for real content (the mark sits on bright photo areas, not on black), and the careful build drops the gray self-residual to ~1.3. **The mask quality, not the method, was the earlier limit** — a max-channel / quadratic-bg / blurred / halo-truncated build (and a black-dominated LS) left a visible outline (lesson from issue #13: when reverse-alpha leaves a ghost, suspect the captured alpha map before adding heuristics or switching method). Geometry emitted by the solver at `_ALPHA_NATIVE_WIDTH` 2048: `_ALPHA_WIDTH_FRAC` 0.202, `_ALPHA_HEIGHT_FRAC` 0.058, margins ~0.029. **Removal = reverse-alpha + a deliberately THIN residual inpaint** (`remove_watermark_reverse_alpha`, `_RESIDUAL_DILATE` 5 over the `_RESIDUAL_ALPHA_FLOOR` 0.05 footprint, `_RESIDUAL_INPAINT_RADIUS` 2, `INPAINT_NS`): a single 2048 alpha cannot pixel-cancel the mark re-rasterized at another resolution (alpha maps from independent captures correlate 0.998, not 1.0; off-native reverse-alpha alone only halves the mark), so a tight inpaint clears the residual edges WITHOUT the texture/edge smear a wide full-footprint pass caused. **Placement ALWAYS tries fixed geometry AND `_aligned_alpha_map`'s NCC scale+position search, keeping the lower-residual** — the mark re-rasterizes + jitters a few px per image even at the captured width, so fixed geometry alone misses (there is no `_ALPHA_NATIVE_BAND` fast-path; the scale search `_ALPHA_ALIGN_SEARCH` is fine-stepped, and the `WM_*` locate box is generous so a corner-ward shift stays inside the search — the same widen that fixed Doubao). Verified clean on the solid captures (native 2048; faint self-residual ~1.3 visible only on a dead-flat field, hidden by real texture) and a real 1440-wide Jimeng download (off-native, table edge preserved). `reverse_alpha_available` is just "asset present"; the registry gates on `detect`. **No committed real sample** (the real content download stays gitignored; only the solid calibration captures are committed) — `tests/test_jimeng_engine.py` synthesizes a mark from the bundled alpha asset, and `test_recovers_shifted_mark_on_texture` guards the align-on-shift path that the Doubao defect exposed. Jimeng images are independently caught by the China TC260 AIGC label in `metadata`/`identify`, so this engine is the visible-mark *removal* path, not a new `identify` signal.
- `samsung_engine.py` — **a thin `TextMarkEngine` subclass (config only) since 2026-06-09.** visible Samsung Galaxy AI "✦ Contenuti generati dall'AI" remover/detector (cv2/numpy, no GPU), built 2026-06-05 from issue #37's flat captures (@f-liva). Shares the base but anchored **bottom-LEFT** (Doubao/Jimeng are bottom-right): `locate` anchors a bottom-left box by **geometry** (scales with WIDTH), `extract_mask` pulls the light low-chroma glyphs (white top-hat + grayish + min-luma — `LOGO_MIN_LUMA` is lowered to **110** because the mark is faint, peak alpha ~0.38, so on a mid/dark background its glyph luma is lower than Jimeng's), `detect` matches the bundled glyph silhouette (`assets/samsung_alpha.png`) via `TM_CCOEFF_NORMED` over a coverage floor. Threshold `DETECT_NCC_THRESHOLD` **0.40** (real marks ~0.79 on a real photo, ~0.57/0.71 on the black/gray captures; 0.0 on Doubao/Jimeng captures, and Doubao/Jimeng score 0.0 on a real Samsung photo — no cross-fire, also because the corner differs). **Logo is pure white (255,255,255)** (`_ALPHA_LOGO_BGR`; white capture confirms). **Alpha solved by `scripts/visible_alpha_solve.py samsung` from the GRAY capture** (`data/samsung_capture/captures/`, the flat black/gray/white captures committed; the solver gained a `corner="bl"` mode + left-margin logging for this), same careful recipe as Jimeng (cubic background, mean-channel, full halo, unblurred). Geometry emitted at `_ALPHA_NATIVE_WIDTH` **1086** (the flat-edit capture width): `_ALPHA_WIDTH_FRAC` 0.3195, `_ALPHA_HEIGHT_FRAC` 0.0378, `_ALPHA_MARGIN_LEFT_FRAC` 0.0110, `_ALPHA_MARGIN_BOTTOM_FRAC` 0.0064. **Removal = reverse-alpha + a deliberately THIN residual inpaint** (`remove_watermark_reverse_alpha`, same `_RESIDUAL_*` recipe as Jimeng) with **always-try fixed AND `_aligned_alpha_map` NCC scale+position search, keep the lower-residual** (`_ALPHA_ALIGN_SEARCH` widened to (0.85, 1.18, 23) because the flat captures are far off the real-photo width). **Resolution caveat:** the flat captures arrived at 1086 wide while real photos are ~2958 wide (the mark scales with width, so the captured glyph ~334px is ~2.7x smaller than the ~903px real-photo glyph); width-scale + NCC-align still removes it cleanly (verified on a real 2958-wide @f-liva photo: re-detect 0.79→0.00, no readable text or outline on the recovered wooden table — checked **visually**, not just by the detector, per the Gemini self-verify lesson), but a flat capture at the real photo resolution would make the alpha pixel-sharp instead of upscaled (open quality upgrade, noted in `data/samsung_capture/README.md`). **The mark is locale-specific** (text differs per language); this build is the Italian "Contenuti generati dall'AI" variant — other locales need their own captured template. `reverse_alpha_available` is just "asset present"; the registry gates on `detect`. **No committed real sample** (the real photo stays gitignored; only the flat calibration captures are committed) — `tests/test_samsung_engine.py` synthesizes a mark from the bundled alpha asset (bottom-left geometry), with `test_recovers_shifted_mark_on_texture` guarding the align-on-shift path. Samsung Galaxy AI edits are independently caught by C2PA + the `genAIType` marker in `metadata`/`identify`, so this engine is the visible-mark *removal* path; it also feeds `identify` as the medium-confidence `visible_samsung` signal via the registry (the stripped-metadata fallback).
- `region_eraser.py` — universal region eraser (`erase` CLI). `erase(image, boxes=|mask=, backend=)` accepts grayscale (2D) and RGBA (4-channel) inputs on **both** backends (`erase_cv2` and `erase_lama` each split off any alpha plane and re-attach it unchanged, and promote grayscale to BGR for processing — LaMa would otherwise crash on grayscale and drop alpha on BGRA): `boxes_to_mask` → `cv2.inpaint` (`cv2` backend, default, no deps) or big-LaMa via onnxruntime (`lama` backend, extra `lama`, `Carve/LaMa-ONNX` Apache-2.0 model downloaded on first use, never bundled). `erase_lama` crops a padded region around the mask, runs LaMa at its fixed 512² input, pastes only masked pixels back (untouched areas stay pixel-exact). Lazy `_get_lama_session` singleton; `lama_available()` guards the optional import. **LaMa-ONNX costs ~3.5-4 GB peak RAM and ~5-6 s/call on CPU** (FFC working set, not arena — `enable_cpu_mem_arena=False` does not help), so it does NOT fit a minimal droplet; the cv2 backend (tens of MB, ~30 ms) does. LaMa quality at low RAM = serverless/GPU, mirroring how raiw.cc offloads SDXL to fal.
- `invisible_watermark.py` — `detect_invisible_watermark(path)` decodes the OPEN DWT-DCT watermarks (public decoder, no key) embedded by Stable Diffusion / SDXL / FLUX via the `imwatermark` library. Known fixed patterns (verified against upstream source) live in `_BITS_48` (SDXL 48-bit, FLUX.2 48-bit) and `_SD1_STRING` ("StableDiffusionV1", SD 1.x/2.x). Optional dep (extra `detect`); returns None when absent. The `detect` extra pulls **torch** transitively (invisible-watermark declares torch a hard dep, and `WatermarkDecoder` eagerly imports `rivaGan` -> `torch` at import time), so detection needs torch present even though dwtDct runs CPU-only on cv2/numpy/pywavelets — no GPU and no separate `gpu` extra required. **Unlike SynthID this is locally detectable**, but the watermark is fragile (does not survive JPEG re-encode/resize — verified gone after JPEG q90), so it confirms origin only on pristine files. Add new known patterns here. The file carries a top-of-module pyright pragma because imwatermark/cv2 ship no type stubs.
- `trustmark_detector.py` — `detect_trustmark(path)` decodes the OPEN, keyless **Adobe TrustMark** watermark (the soft binding behind Adobe Durable Content Credentials, `alg` `com.adobe.trustmark.P`) via the optional `trustmark` package (extra `trustmark`; pulls torch, downloads model weights on first use). Mirrors `invisible_watermark.py` (lazy singleton guarded by a double-checked `threading.Lock` so concurrent callers do not double-download the weights, top-of-module pyright pragma, returns None when absent). It detects *provenance*, not AI origin as such (TrustMark also marks human-authored content), so `identify` lists it as a watermark without setting `is_ai_generated`. Other soft-binding vendors (Digimarc/Imatag/Steg.AI/...) have no public decoder — they are only *named* via the `C2PA_SOFT_BINDINGS` scan, not decoded. **False-positive gate (added 2026-05-29):** TrustMark's `wm_present` is a BCH error-correction validity flag that spuriously validates on a content-correlated fraction of un-watermarked images — AI-generated textures trip it far more than camera photos (verified 2026-05-29 on real files: it fires on Gemini/OpenAI/Doubao output that *cannot* carry Adobe's watermark, with a random-bytes decoded secret, while signal-free camera photos did not trip it). A genuine TrustMark is a *durable* soft binding engineered to survive re-encoding, so `detect_trustmark` re-decodes after a mild JPEG round-trip (`_survives_reencode`, `_REENCODE_QUALITY` 95) and requires the same schema both times; every observed false positive collapsed (none survived even q95), so the gate is the durability property the watermark guarantees. The second decode runs only on the rare initial hit, so the cost is negligible. Do NOT remove the gate to "catch more" — a lone TrustMark hit without it is almost always content noise.
- `noai/watermark_remover.py` — the `WatermarkRemover` class has two diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id` -- both use the same SDXL base, `DEFAULT_MODEL_ID`). **`sdxl`** (renamed from `default` 2026-06-09; `default` kept as a back-compat alias via `normalize_profile`) runs plain SDXL img2img (`_run_img2img`); it is the lighter opt-down alternative (no ControlNet weights). **`controlnet`** (**the DEFAULT pipeline since 2026-06-09** for `invisible`/`all`/`batch` and both engine ctors; `_run_controlnet`, `_load_controlnet_pipeline`) runs `StableDiffusionXLControlNetImg2ImgPipeline` with the SDXL-native canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` (`watermark_profiles.CONTROLNET_CANNY_MODEL`): the control image is `cv2.Canny(gray, 100, 200)` stacked to 3 channels (`_CANNY_LOW`/`_CANNY_HIGH`, prompt `_CONTROLNET_PROMPT` / `_CONTROLNET_NEGATIVE`). **Removal comes from the img2img regeneration (`strength`); the ControlNet only PRESERVES text and face STRUCTURE via the edge map.** No original pixels are copied or frozen, BUT **validation 2026-06-04 disproved the old "so SynthID does not survive" claim: SynthID CAN survive controlnet on photoreal/high-detail content.** At the shared low removal strength the canny edge-conditioning keeps the regeneration so close to the original that the pixel perturbation that destroys SynthID does not happen (oracle-confirmed: an OpenAI bracelet photo + a 9-face grid read **SynthID-detected** after controlnet at strength 0.10/0.15, but **SynthID-not-detected** after the `default` pipeline at the SAME strength + resolution -- only the pipeline differed). **But the reverse also holds: a flat-graphic logo/poster SURVIVED `default` while clearing controlnet** -- removal at the low strength is content×pipeline dependent and neither pipeline is universally safe; the real lever is a higher strength. See the controlnet Known-limitations bullet for the full table + root cause. Canny holds face STRUCTURE but NOT identity (the regenerated face drifts in likeness -- canny carries edges, not identity). The drifted cleaned face is the LEAST-AI state we can reach without re-introducing SynthID; the library does NOT ship a face-restore extra. Every restore approach we evaluated (GFPGAN-on-cleaned, PhotoMaker-V2 txt2img, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps, 2026-06-04 - 2026-06-08 Modal cert sweeps) regenerated the face from an ArcFace embedding via SDXL diffusion -- which makes the output face look MORE AI-generated, not less. Empirical conclusion in `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up". For production face preservation, ship the cleaned image as-is. `controlnet_conditioning_scale` (ctor arg, default 1.0) is the structure-preservation knob. Same dtype rule as `default` (fp32 on cpu/mps, fp16 only on cuda/xpu; the fp16-fixed SDXL VAE `_SDXL_FP16_VAE_ID` is swapped in on fp16 GPUs -- issue #29) and the same MPS->CPU fallback (reload on cpu/fp32, drop a non-cpu generator, retry once).
- **`auto_config.py` + the content-detection layer were REMOVED 2026-06-09.** History: `auto_config.plan()` was a content-adaptive planner that detected faces/text/edges (bundled OpenCV YuNet + PP-OCRv3 DBNet models) to route the pipeline and toggle the adaptive polish. Once `controlnet` became the default-and-only auto pipeline (it no longer downgrades a structure-less image to `sdxl`) and the adaptive polish was confirmed to **self-gate by detail level** (`humanizer.adaptive_polish` no-ops when the cleaned image already meets the input's Laplacian variance, so it does real work only on over-smoothed photo/face texture and ~nothing on text/flat), the detection no longer changed any behavior — it only annotated a `reason` string. So the whole layer was deleted: `auto_config.py`, `tests/test_auto_config.py`, and the two detection assets (`assets/face_detection_yunet_2023mar.onnx`, `assets/text_detection_ppocrv3_2023may.onnx`, ~2.6 MB). **`--auto` is now a DEPRECATED no-op** (`cli._resolve_auto_polish`): controlnet is already the default pipeline AND the adaptive polish is ON by default, so `--auto` has nothing left to do — it only prints a deprecation warning and passes `adaptive_polish` through unchanged (an explicit `--no-adaptive-polish` still wins). (Originally it re-enabled the polish; once the polish default flipped to ON the same day, the parameter-source branch became dead and was dropped.) The **adaptive polish itself lives on** in `humanizer.adaptive_polish` (CLI `--adaptive-polish/--no-adaptive-polish`, **ON by default since 2026-06-09** — it self-gates to a no-op where there is no detail deficit, so default-on is safe; uses the full-res original as the detail reference) — see the `humanizer` test note. `batch` resolves the polish once before the loop (one warning) and caches the invisible engine per pipeline (`ctx.obj["_inv_engines"]`).
- `upscaler.py` — optional Real-ESRGAN pre-diffusion super-resolution for small inputs (spandrel boundary, top-of-file pyright pragma). `is_available()` gates on spandrel+torch (via `importlib.util.find_spec`); `upscale(bgr, device=None)` loads a lazily-built spandrel `ImageModelDescriptor` singleton (double-checked lock) and upscales by the model's native factor (x2), with a non-CPU→CPU device fallback mirroring the diffusion engine's MPS→CPU retry. Weights (`RealESRGAN_x2plus.pth`, BSD-3-Clause) download on first use to the `torch.hub` checkpoints cache; never bundled. Used only when UPscaling to the `min_resolution` floor (a `max_resolution` downscale always uses Lanczos). The wiring is `InvisibleEngine._esrgan_upscale(pil, target)` — Real-ESRGAN at native factor, then a Lanczos resize to the exact target, falling back to a plain Lanczos resize if the extra is absent or the model errors (so an optional upscaler can never break removal). The default `--upscaler` is `lanczos` (cv2, no deps). **ESRGAN is a generic photo/texture GAN with no face/glyph prior**, so it best fits photo/texture content and can degrade faces (glassy/asymmetric eyes -- the diffusion pass regenerates faces so the full-pipeline final recovers) and thin/small text (the GAN invents wrong strokes, and low-strength diffusion will not fix it). Verified 2026-06-04: isolated upscale lap-var ~5x Lanczos on faces+textures but glassy eyes; end-to-end `invisible` final lap-var 1634 vs Lanczos 663 with natural faces (diffusion cleaned the artifact). Kept a **manual opt-in knob** (the auto plan never selects it) with `lanczos` the default; not content-gated by design (use Lanczos for text-heavy inputs). spandrel is MIT and pulls no basicsr. Unit-tested without the model: `tests/test_upscaler.py` (availability guard + the not-installed RuntimeError) and `tests/test_invisible_engine.py::TestEsrganUpscale` (the three `_esrgan_upscale` branches via a monkeypatched `upscaler`).
- `image_io.py` — Unicode-safe cv2 IO (issue #17). `imread(path, flags=None)` / `imwrite(path, img)` wrap `np.fromfile`+`cv2.imdecode` / `cv2.imencode`+`tofile` so non-ASCII paths work on Windows -- bare `cv2.imread`/`cv2.imwrite` use the platform ANSI code-page API there and fail (empty decode + `can't open/read file`) on Chinese/Cyrillic/accented filenames. `imread` keeps `cv2.imread` semantics (defaults to `IMREAD_COLOR`, returns `None` on missing/empty/undecodable). **Every cv2 file read/write in the package routes through here; do not call `cv2.imread`/`cv2.imwrite` directly.** `imwrite` returns `False` on an unwritable path (`OSError` caught) instead of raising, matching `cv2.imwrite` semantics. macOS/Linux already accept UTF-8 paths, so it is behavior-neutral there (the bug only reproduces on Windows). **`to_bgr(image)` (added 2026-06-09)** is the shared channel normalizer: promotes 2D grayscale / (h,w,1) / 4-channel BGRA to 3-channel BGR (a 3-channel input is returned unchanged, no copy). Use it instead of inlining the `cvtColor(GRAY2BGR/BGRA2BGR)` branch — the gemini engine and the `TextMarkEngine` base both route through it so a grayscale/BGRA input (a real Gemini-app export is opaque RGBA) does not crash the `axis=2` channel reductions. cv2/numpy are imported lazily inside the functions, so the module is cheap to import in a bare env.
## Test and lint
### Doubao clean-reverse-alpha distillation (re-investigated 2026-05-29)
`maintain.sh` runs dependency freshness and security checks, Ruff, Pyright scoped to `src/`, and the parallel test suite. Full-project Pyright is not the project gate because the ML dependency graph can exhaust Node memory.
**RESOLVED 2026-05-29: black+gray Doubao captures were obtained and a reverse-alpha is built** (`doubao_engine.remove_watermark_reverse_alpha`, `assets/doubao_alpha.png`; see the `doubao_engine.py` bullet above). The captures (`data/doubao_capture/captures/`, now committed) confirmed the alpha-composite model: on black `captured = a*logo`, logo pure white. **UPDATE 2026-05-31 (issue #13 follow-up): the first build was NOT "exact"** — it left a readable "豆包AI生成" outline on the real sample (the detector was fooled, conf 0.0). The alpha is now rebuilt by `scripts/visible_alpha_solve.py` (the careful gray-self solve shared with Jimeng), removal always-aligns + thin-inpaints, and the locate box was widened; see the `doubao_engine.py` bullet. The notes below (the failed content-image distillation) are retained as the record of why controlled captures were necessary.
Command, gate, typing, and model-test invariants auto-load from [`.claude/rules/development.md`](.claude/rules/development.md). Environment recovery, CI behavior, and fixture policy live in [`docs/development.md`](docs/development.md).
**Conclusion (historical): pure reverse-alpha distilled from content images does NOT work, and the blocker is the WRONG kind of data, not too little of it.** The earlier framing ("need ~5-8 PRISTINE same-resolution originals") is obsolete -- `data/spaces/originals/` holds plenty. Curate them with `DoubaoEngine.detect` + an NCC filter against a clean glyph template, keeping only marks at offset ≈ (0,0): that yields e.g. **15 pixel-aligned 2048² marks** (sub-pixel drift, not the ±50 px the old lossy/mixed-res scrapes had), plus 1086x1448 / 1792x2400 clusters. With those, LaMa-clean `O` + weighted-LS (and per-pixel I-on-O regression) for `α` (+ logo colour) was tried end-to-end and **still leaves a persistent ghost outline.**
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Treat the release as complete only after PyPI, Homebrew, the Hugging Face Space, and the ComfyUI Registry are verified; conda is not published. Keep the source-distribution public allowlist.
Diagnosed why, empirically (cached stacks, `/tmp/doubao_distill`): (1) the mark is a clean white overlay with **no dark halo** -- over glyph pixels ~54% are brighter than the clean bg, only ~4% darker -- so the white-logo model `I=(1-α)O+α·255` is correct; (2) but content backgrounds are almost never dark *under* the mark (median darkest available bg over glyph pixels = **58/255**; only ~13% of mark pixels are ever observed on a bg < 40), so on bright backgrounds the equation is ill-conditioned and `α` is unidentifiable; (3) LaMa's `O` is a plausible **hallucination**, not the true pre-mark background, which compounds the error, and per-pixel regression on ~15 obs overfits into colour noise.
## Module architecture
**Why Gemini's engine is clean (verified in GeminiWatermarkTool `src/core/watermark_engine.cpp`): its alpha map is the watermark stamped on a PURE-BLACK background**, where `watermarked = α·255 + (1-α)·0 = α·255`, so `alpha = capture/255` exactly -- no estimation. (`gemini_bg_*.png` is literally the sparkle in grey on black.) So the real Doubao unlock is the same controlled capture, **not more content images**. Black/white/gray seeds exist (`data/doubao_capture/seeds/seed_*_1x1_2048x2048.png`); a capture run (feed a black seed through doubao.com edit mode, download the *original*) was requested from the #13 reporter 2026-05-29. With ~2-3 black captures we get `α = capture/255` for free, Gemini-quality.
[`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem.
**Until black captures arrive, the shipped direction is precise canonical glyph mask + inpaint (cv2 default, lama optional), NOT reverse-alpha.** The consensus glyph silhouette across the aligned marks distills cleanly (proto: a tight "豆包AI生成" strip, width ≈ 0.156 × image-width) and is good both as an exact inpaint mask and as an NCC localiser -- the latter also fixes the #23 detector false-positives (match the real glyph shape, not any bright low-saturation corner). Do **not** retry content-image reverse-alpha: it is data-limited by physics (no dark-background observations), not by effort.
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md).
## Watermarking landscape (research 2026-05-24)
## Data safety
Who embeds what, and whether it is locally detectable (so we know which gaps are fillable). See `identify.py` for what we read.
- **Locally detectable (open decoder, no key/API):** Stable Diffusion / SDXL / FLUX via `imwatermark` DWT-DCT (now covered by `invisible_watermark.py`). FLUX uses the same library (`black-forest-labs/flux2` `src/flux2/watermark.py`, 48-bit `0b001010101111111010000111100111001111010100101110`); SDXL is the diffusers `WATERMARK_MESSAGE` (`0b101100111110110010010000011110111011000110011110`). Caveat: fragile to re-encoding.
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), and **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`). Still unsampled: Canva (its downloads are re-encoded design *exports* that strip C2PA, so a Canva "positive" is inconclusive — skipped), Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag). **Samsung Galaxy AI** (Generative Edit / Sketch to Image / Portrait Studio on Galaxy S23 FE / S24 / S25, One UI 7+) signs C2PA as "Samsung Galaxy" with the standard `trainedAlgorithmicMedia` source type AND a proprietary `genAIType` marker; verified on real signed files 2026-05-29 (the standard scan catches the source type; `genAIType` additionally catches a Galaxy S24 file that omits it). It ALSO burns a **visible** localized wordmark into the pixels — a sparkle + "generated with AI" string in the bottom-LEFT corner (issue #37; the Italian "✦ Contenuti generati dall'AI" variant is calibrated) — removed by `samsung_engine.py` / `visible --mark samsung` (reverse-alpha, see the engine bullet); detection feeds `identify` as the medium `visible_samsung` signal. The string is locale-specific, so each locale needs its own captured alpha template. **ASUS Gallery** also signs edited photos as C2PA (`com.asus.gallery`) but with no AI source type — a signer, not an AI marker. **Black Forest Labs (FLUX)** API output signs C2PA: `claim_generator_info "Black Forest Labs API"` + a `c2pa.ai_generated_content` assertion + `trainedAlgorithmicMedia` (issuer `b"Black Forest Labs"` added to `C2PA_ISSUERS`, platform "Black Forest Labs (FLUX)"). **ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. All three verified on real signed files 2026-05-29.
- **EXIF/XMP generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal).
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).** Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC** — only EXIF `Artist` = a UUID and EXIF `ImageDescription` = `Signature: <base64>` (a crypto signature, unverifiable locally without xAI's public key). This empirically kills the earlier unverified "xAI signs C2PA as xAI" lead — xAI is not even a C2PA member. `exif_generator` misses it (neither field holds an `AI_GENERATOR_TOKENS` token), so a dedicated detector `xai_signature(path)` matches the pair (`ImageDescription ~ ^Signature: [A-Za-z0-9+/=]{64,}` AND UUID `Artist`); wired into `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify` (signal `xai_signature`, platform "xAI (Grok / Aurora)"). **Format confirmed stable across n=3 genuine generations:** exactly three EXIF tags (`Artist`, `ExifOffset`, `ImageDescription`), `Signature:` prefix constant, base64 payload 300-1004 chars. Two capture facts: (a) the `Artist` UUID **equals the public image id** in the asset URL (`https://imagine-public.x.ai/imagine-public/images/<uuid>.jpg`), so it is NOT a private per-user secret — only the `Signature` blob is; (b) the Grok web-UI image is a re-encoded **WebP with no signature** — the EXIF survives only in the *original* JPEG (download button or that public tokenless URL), which is why screenshots / re-encodes are metadata-stripped. A real fixture `data/samples/grok-1.jpg` plus **synthetic** JPEG fixtures (fake UUID + fake `Signature:` blob) cover the detector; never add a real Grok image carrying private content (the repo is public). **Stripped on removal too:** `remove_ai_metadata` now calls `_scrub_ai_exif` on the JPEG EXIF, which deletes the xAI Signature+UUID-Artist pair **and** any `Software`/`Make`/`Artist`/`ImageDescription` tag holding an `AI_GENERATOR_TOKENS` token (so Ideogram's `Make="Ideogram AI"` is scrubbed too), while keeping genuine camera/editor EXIF. The shared `_is_xai_signature_pair` helper (module-level compiled regexes) is the single source of truth for the pattern, used by both `xai_signature` and `_scrub_ai_exif`. (AVIF/HEIF/JXL still strip only C2PA boxes via `isobmff`, not EXIF — unchanged.)
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). **Doubao** (ByteDance) uses it (verified on the real #13 sample 2026-05-25; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **three serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on the corpus 2026-05-30) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on the corpus 2026-05-28, `ContentProducer="doubao"`), **and** a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on the corpus 2026-05-30, `ContentProducer="001191440300708461136T1308L"`). Both generic forms (the PNG chunk and the bare `{"AIGC":...}` object) are gated on at least one TC260 field (`_TC260_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
- **HuggingFace-hosted job (caught by `metadata.huggingface_job`, surfaced by `identify` as the `hf_job` signal, MEDIUM confidence):** HuggingFace Jobs / Spaces stamp generated PNGs with an `hf-job-id` tEXt chunk holding the job UUID (3 on the corpus 2026-05-28, no other signal). It marks the *hosting job*, not a model — most commonly diffusion output — so it lifts an Unknown verdict to a tentative AI via `hf_only` (parallel to the visible sparkle) but never overrides a hard metadata signal; `_HF_JOB_CAVEAT` states the limit (job, not model; not proof of AI pixels). Stripped on removal (the PNG save whitelist keeps only `STANDARD_METADATA_KEYS`, so `hf-job-id` and the `AIGC` chunk are both dropped). The exact writer is not authoritatively documented (HF Jobs are generic GPU jobs), hence medium not high.
- **No detectable signal on download (correctly reported `unknown`):** **Recraft** (PNG export is a re-encoded design export — strips everything), **Krea hosting FLUX 2** (no imwatermark despite FLUX — the host omits the encoder, same as Stability's hosted SDXL), and Midjourney (embeds nothing). Lesson: the imwatermark detector only fires on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL), not from re-hosts (Krea/Stability) or re-encoded exports (Recraft/Canva).
- **Invisible but NOT locally detectable (proprietary, API/oracle only — same wall as SynthID):** Amazon Titan Image Generator + Nova Canvas (Bedrock `DetectGeneratedContent` API), Kakao (new SynthID image adopter, May 2026), NVIDIA Cosmos (SynthID video). No local detector possible; treat like SynthID.
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html We now READ the soft-binding `alg` (`C2PA_SOFT_BINDINGS` / `soft_binding_vendors_in`) to name the forensic-watermark vendor, and locally DECODE the one open scheme, Adobe TrustMark (`trustmark_detector`); the rest (Digimarc/Imatag/Steg.AI/...) stay name-only (proprietary decoders).
- **Built 2026-05-26 (this batch):** soft-binding `alg` vendor detection; IPTC Photo Metadata 2025.1 AI-disclosure fields (`AISystemUsed` etc.); **video C2PA metadata** detect + strip for MP4/MOV/M4V (free — `isobmff.py` is format-agnostic, MP4 is ISOBMFF); Adobe TrustMark open decoder. NOT done (out of cheap reach, per the feasibility review): visible video-logo removal (needs a video frame pipeline) and audio (SynthID/ElevenLabs/Resemble/Suno all oracle-only or unmarked). **Box detection window — now handled (v0.6.8):** detection no longer relies on a fixed first-MB read. `metadata.scan_head(path, size)` reads the first `size` bytes and, for ISOBMFF, appends the payloads of late provenance boxes found by `isobmff.scan_c2pa_region` (a file-seeking top-level box walker that skips past `mdat` by size without reading it), so a C2PA/AIGC/IPTC manifest placed AFTER a large `mdat` in a streaming/non-faststart MP4 is now caught. Every C2PA/marker byte scan (`has_ai_metadata`, `aigc_label`, `iptc_ai_system`, `synthid_source`, `exif_generator` XMP, `get_ai_metadata` soft-binding, and `identify`) goes through `scan_head`; it is behavior-neutral for non-ISOBMFF inputs (exactly `f.read(size)`). **Meta-box XMP removal — now handled (v0.6.9):** an AI-label XMP packet stored as a meta-box `mime` item (HEIF/AVIF; out of reach of the top-level box stripper) is blanked in place by `isobmff.blank_ai_xmp_packets` — it locates the packet by its `<?xpacket begin … end?>` delimiters and, if it carries an AI marker (`_AI_LABEL_MARKERS`), overwrites it with spaces of the SAME length, so box sizes / `iloc` offsets stay valid and the coded image is untouched (selective: plain non-AI XMP is left alone, mirroring the top-level uuid logic). Wired into `remove_ai_metadata`'s ISOBMFF branch after `strip_c2pa_boxes`. The remaining gap is an `Exif` meta-box *item* (rare; the AI labels are XMP) — still needs `iinf`/`iloc` surgery or exiftool.
- **Regulatory driver (context, not a code change):** AI-content labeling mandates are expanding, which pushes more generators toward exactly the C2PA + watermark signals we read. The full per-jurisdiction table lives in README "## Legal" -- keep it there, not duplicated here. Newly added + primary-source verified 2026-05-26: **EU AI Act Article 50** machine-readable marking applicable **2026-08-02** (verified against the article text); **South Korea AI Framework Act Art. 31(3)** in force since **22 January 2026** (verified via Kim & Chang + FPF/Korea Times; Enforcement Decree accepts an invisible-watermark label); **California AB 853** (amends the CA AI Transparency Act) latent-disclosure duty operative **2026-08-02**, requiring a disclosure "permanent or extraordinarily difficult to remove" (verified against the leginfo bill text -- this is the exact disclosure our tool strips); **India IT Amendment Rules 2026** in force **2026-02-20** (verified via Chambers), which prominently-label + permanent-provenance-id all synthetic media AND **expressly prohibit removing/suppressing the label or metadata** -- the first major all-content removal ban outside China. **Removal liability (README "## Legal" disclaimer):** the tool is lawful general-purpose software; liability sits with the remover and is intent-gated -- downstream acts (fraud/deception/IP), plus US DMCA 17 USC 1202 (removing copyright-management info to conceal infringement), plus the removal-as-such bans in China + India. When extending the README table, verify each date/article against the statute/bill text before committing, not against search summaries.
Follow [`data/README.md`](data/README.md) for public fixture, calibration, oracle, and evaluation layout. Store each tracked binary once and keep generated evaluation outputs outside the repository.
## Known limitations
## Rules and conventions
- `invisible` pipeline processes at **native resolution for inputs whose long side is >= 1024px**, and **auto-upscales smaller inputs UP to a 1024px floor** (`min_resolution=1024`, the default; `--min-resolution 0` disables) before diffusion -- SDXL img2img distorts badly on a tiny latent (a 381x512 portrait wrecks at native, the #36 follow-up), and the output is restored to the original input size so the floor is a transparent quality boost (it adds time/memory on small inputs). The floor upscale uses Lanczos by default; **`--upscaler esrgan`** (opt-in, the `esrgan` extra) runs Real-ESRGAN first for better detail before the Lanczos resize to the exact target (`upscaler.py` / `InvisibleEngine._esrgan_upscale`, falls back to Lanczos if the extra is absent). `max_resolution=0` (default) means no downscale cap, matching the hosted raiw.cc backend (fal fast-sdxl, no pre-downscale). The old forced downscale-to-1024 -> upscale-back round-trip for LARGE images was the main quality loss (issue #10) and is gone; at strength ~0.05 SDXL img2img does not need a downscale. **Final `--unsharp` post-filter (`humanizer.unsharp_mask`, opt-in, default 0):** applied LAST (after the face-restore pass, else it would be smoothed over) to counter the soft/over-smoothed look diffusion + restoration leave (an AI tell); ~0.5-0.8 safe, higher risks halos. Pairs with `--humanize` (grain adds sensor-noise texture, unsharp adds crispness). `--max-resolution N` re-introduces an opt-in long-side cap purely to bound GPU/MPS memory on very large inputs (it reintroduces the lossy round-trip). For huge images that OOM at native, tile-based diffusion is still the proper long-term fix. **Concrete MPS data points (the OOM is memory-tier-dependent, NOT a hard MPS limit):** on a ~24 GB unified-memory machine (verified 2026-05-25, 1254x1254 gpt-image SDXL, fp32) native res OOMs at the *UNet* step (peak ~17 GiB), not only the VAE decode, and the auto-fallback in `img2img_runner` reloads on CPU and finishes (slow, ~13 min) -- the output is still weight-identical and defeats SynthID, so "looks hung/crashed" on Mac is usually this CPU fallback, not a pipeline error. On a **32 GB** unified-memory machine the same default SDXL pass runs entirely on MPS with **no CPU fallback** (verified 2026-05-31, 1122x1402 gpt-image, `all`/default, ~155 s end-to-end), so 32 GB clears the native-res UNet peak that 24 GB could not. Adding `enable_vae_tiling()` alone does NOT prevent the 24 GB OOM (the peak is the UNet, not the VAE). The fast Mac workarounds for memory-constrained machines are fp16 on MPS (roughly halves memory) or `--max-resolution` to cap the long side; neither is wired as the default. The `controlnet` pipeline adds the canny ControlNet weights on top of SDXL, so its peak is a bit higher than the plain `default` pass; the same MPS->CPU fallback covers an OOM. The native-vs-cap-vs-floor decision lives in the pure helper `invisible_engine._target_size(w, h, max_resolution, min_resolution)` (returns `None` for native, a target tuple for a downscale cap OR an upscale floor; cap takes precedence, the floor is skipped on a min>max misconfig) so it is unit-tested (`tests/test_invisible_engine.py::TestTargetSize`, the #10/#15/#36 regression guard) without loading the model -- keep that logic in the helper, don't re-inline it.
- **fp16 VAE black-output fix (issue #29, 2026-05-30):** on a **CUDA/XPU fp16** backend the stock SDXL VAE overflows to NaN and the *plain* img2img path decodes to an **all-black** image (reproduced on the raiw.cc result: a 1086x1448 input -> a uniformly black 4.6 KB PNG, mean 0). `watermark_remover._load_pipeline` / `_load_controlnet_pipeline` swap in the fp16-fixed SDXL VAE (`madebyollin/sdxl-vae-fp16-fix` = `_SDXL_FP16_VAE_ID`) when `_needs_fp16_vae_fix(model_id, DEFAULT_MODEL_ID, is_fp16)` is true -- only the default SDXL checkpoint on fp16. **cpu/mps run fp32** (the stock VAE is fine there, which is why the bug never reproduces on Mac). A custom non-SDXL `model_id` keeps its own VAE (the fp16-fix VAE is SDXL-architecture-specific). The decision is a pure helper, unit-tested without a download (`tests/test_platform.py::TestFp16VaeFix`); the actual black->clean recovery needs a CUDA GPU. **Confirmed on real CUDA hardware 2026-06-03:** running `all` on a 1086x1448 OpenAI gpt-image (the #29 repro size) at fp16 produced a normal (non-black) output, so the fp16-fix VAE swap resolves the all-black decode. (It was not reproducible on this MPS machine, which runs fp32, so the verification had to happen on an NVIDIA box.) **Follow-up safety net (issue #41, 2026-06-04):** the swap is gated to `model_id == DEFAULT_MODEL_ID`, so a custom model, a stale pre-fix install, or a fal/custom loader can still hit the black decode -- a new reporter did (gpt-image 1448x1086, the #29 size, with the exact `image_processor.py:142 invalid value encountered in cast` warning the NaN->0 cast emits). `remove_watermark` now adds a model-agnostic backstop: after generation, if the run was fp16 AND the output is degenerate (`_is_degenerate_image`: mean and std both below `_DEGENERATE_THRESHOLD` 1.0 -- a uniform all-black/NaN frame; the variance guard spares a legitimately dark-but-textured photo), it rebuilds the pipeline in fp32 on the SAME device and re-runs once. fp32 is the verified-clean path, so the user never gets a black image regardless of model_id/version. Mirrors the existing MPS->CPU fallback's self-mutation pattern (reset `torch_dtype` + clear `_pipeline`/`_controlnet_pipeline`); `batch` inherits it through `remove_watermark`, and once one image trips it the rest of the batch stays on the safe fp32. The detector is a pure helper, unit-tested without a model (`tests/test_platform.py::TestDegenerateOutputGuard`); the full fp16->detect->fp32-retry chain was verified e2e on this MPS machine by forcing fp16 with the swap disabled (first pass black, guard fired, retry produced a normal image). CAVEAT: the fp32 retry uses ~2x memory, so on a VRAM-constrained GPU it can OOM (a visible error, still better than a silent black frame; the MPS->CPU fallback covers that path). The reporter's "CPU also black" symptom is NOT reproducible here -- fp32 (cpu/mps) decodes clean -- so it points at an old version or a non-fp32 run, pending their version + command.
- Pyright first run is slow (2-3 min) due to ML deps (torch/diffusers/transformers stubs); full-project `uv run pyright` can stall for many minutes — scope it to changed files.
- A third-party PIL plugin autoload (e.g. an HEIF/AVIF plugin) can raise a non-OSError (`ModuleNotFoundError`), not `UnidentifiedImageError`, when opening a file. Code that opens user-supplied or unknown-format files should `except Exception`, not just `OSError`/`UnidentifiedImageError`.
- **rich was dropped (CLI + scripts print plain text via `click.echo`).** `cli.py` renders through small `_Console`/`_Table`/`_Progress` shims; the analysis scripts (`scripts/synthid_corpus.py`, `synthid_pixel_probe.py`, `text_detection_benchmark.py`, `corpus_gap_scan.py`) import `Console`/`Table` from the shared `scripts/_plain_console.py` shim (markup like `[bold]`/`[/]` is stripped, tables render aligned). Consequences: (1) `rich` is NOT a dependency, so anything that imports it breaks a clean `uv sync --frozen` (CI installs core+dev only) — this exact gap red-failed CI after the refactor when those 4 scripts still imported rich; if you add a script, use the `_plain_console` shim, not rich. (2) The old `[gpu]`-bracket-eaten bug (#19) is gone — plain `click.echo` prints `pip install 'remove-ai-watermarks[gpu]'` verbatim, no escaping needed (regression-guarded by `tests/test_cli.py::TestGpuHintMarkup`). (3) No Unicode glyphs / colors / progress bars in CLI output by design.
- Metadata detection for AVIF/HEIF/JPEG-XL relies on a binary scan for `C2PA_UUID` + `IPTC_AI_MARKERS`, plus EXIF `Software` / XMP `CreatorTool` generator tags via `metadata.exif_generator` (validated with synthesized AVIF/JPEG fixtures + an XMP raw-scan fixture). C2PA removal in those containers is implemented via `noai/isobmff.py` (top-level ``uuid`` / ``jumb`` box stripper, no re-encoding), which now also drops a top-level XMP ``uuid`` box that carries an AI label (matched by AI-marker content, not by the XMP UUID, so byte-order-robust) and covers MP4/MOV/M4V/M4A by content sniff. **Non-ISOBMFF audio/video removal is via ffmpeg** (`_FFMPEG_STRIP_EXTS` -> `_strip_with_ffmpeg`): WebM/Matroska (EBML), MP3 (ID3), WAV/FLAC/OGG (RIFF/Vorbis) are stripped losslessly with `ffmpeg -map_metadata -1 -map_chapters -1 -c copy` (codec data untouched). Requires ffmpeg on PATH; raises `RuntimeError` if absent or if ffmpeg can't parse the file. Verified end-to-end (a real ffmpeg-made WAV/MP3 with a `title=Suno AI` tag -> tag gone, audio bytes preserved). **Meta-box XMP now handled (`isobmff.blank_ai_xmp_packets`, v0.6.9):** an AI-label XMP packet stored as a meta-box `mime` item (AVIF/HEIF) is blanked in place (overwritten with spaces of the same length, so `iloc` offsets and the coded image stay valid). **Still NOT built:** an `Exif` *item* inside the `meta` box (rare -- AI labels are XMP) needs full `iinf`/`iloc` surgery (offset rewrite) with corruption risk -- exiftool (R/W/C for HEIC/AVIF EXIF+XMP, verified on exiftool.org 2026-05-27) would do it but is a non-installed binary dep, so it stays a documented gap. **Audio watermark DETECTION (Resemble PerTh) was evaluated and NOT built (2026-05-26):** `resemble-perth`'s `PerthImplicitWatermarker.get_watermark()` returns a raw bit-array with **no presence/confidence flag** (clean audio decodes to arbitrary bits too), so reliably distinguishing watermarked-from-clean needs either Resemble's fixed payload or a confidence API -- neither is public, and there's no real Resemble sample to calibrate against. Same wall-class as the SynthID pixel detector: the decode exists, reliable presence-detection does not. (perth's top-level `PerthImplicitWatermarker` is also gated to None unless `librosa` is importable.)
- **SynthID technical reference: `docs/synthid.md`** — primary-source-cited doc covering mechanism (post-hoc encoder/decoder pair, 136-bit payload at 512x512, pixel-space, model weights NOT modified), robustness numbers (arXiv:2510.09263: ~99.98% TPR@0.1%FPR across 30 transforms including JPEG/crop/resize/color/noise), removal attacks and forensic detectability (arXiv:2605.09203: all 6 attacks detectable at >98% TPR@1%FPR), detectability limits (no public decoder, metadata-proxy only), oracle scope, and adoption landscape. Read that doc first before adding notes here.
Topic-specific rules live in `.claude/rules/*.md` and are auto-loaded when matching files are touched.
- **SynthID detection is metadata-only.** There is no reliable *local* detector of the SynthID *pixel* watermark — Google's decoder is proprietary, no public spec or API (only a waitlisted portal). Authoritative confirmation: Google DeepMind's own paper "SynthID-Image: Image watermarking at internet scale" (Gowal et al., arXiv:2510.09263) states the verification service is restricted to "trusted testers" and does not release detector weights or a reproducible algorithm — so a local pixel detector is infeasible by design, not just unbuilt. https://arxiv.org/abs/2510.09263 We detect SynthID by its C2PA companion (`synthid_source` / `SYNTHID_C2PA_ISSUERS`), which is reliable while the manifest is intact but says nothing once C2PA is stripped. **Surface-dependent blind spot (verified 2026-05-24):** the same Google model emits different metadata per surface -- the Gemini *app* wraps outputs in Google C2PA, but the *API/playground* (AI Studio, Nano Banana / gemini-2.5-flash-image) emits the SynthID *pixel* watermark (confirmed via the Gemini-app oracle) + the visible sparkle but **no C2PA/IPTC at all**, so `synthid_source` returns None despite SynthID being present. Only the pixel oracle or the visible-sparkle detector catches those. (Meta AI is another surface mismatch: it writes the IPTC `digitalSourceType=trainedAlgorithmicMedia` marker, not C2PA and not SynthID.) Google→SynthID is long-standing; OpenAI→SynthID is confirmed by OpenAI's Help Center (ChatGPT/Codex/API "include both C2PA metadata and SynthID watermarks", updated 2026-05-21) but time-gated (pre-rollout OpenAI images carry C2PA without SynthID), so the OpenAI verdict is hedged "likely". Oracles: Gemini app "Verify with SynthID" (Google), openai.com/verify (OpenAI). **Each vendor's oracle detects only its OWN content (verified on the page 2026-05-31):** `openai.com/research/verify` states verbatim "OpenAI generation signals will only be detected if the image was generated with our tools" and "Content could also still be AI-generated by another company's model, which the tool currently does not detect" -- SynthID is shared tech but the verifier is keyed to its own vendor's payload, so a Google-SynthID image reads clean on OpenAI's verifier and vice-versa. **This explains the recurring "oracle says clean but `identify` still flags SynthID" report (#14):** the oracle reads the *pixel* watermark (gone after our SDXL pass), while `identify` reads the *C2PA-metadata proxy* (still present if the manifest survived). Different signals, not a contradiction -- strip the metadata too (`metadata --remove` / `all`) and the proxy goes quiet, but a quiet proxy is not proof the pixel watermark is gone. **SynthID is durable to JPEG re-encode by design, so a GitHub-recompressed issue attachment is still a valid SynthID test subject** (verified 2026-06-01 on issue #14's pic3: the GitHub-served JPEG survived re-encoding and openai.com/verify still detected SynthID). Do NOT dismiss issue-attachment JPEGs as "not faithful originals" when reproducing a SynthID-survival report: the recompression strips the **C2PA metadata** (so `identify` reads Unknown on the attachment) but NOT the **pixel watermark** that openai.com/verify reads. A true byte-original only matters for the metadata/C2PA path, not for the pixel-SynthID-removal test. (Contrast the open imwatermark above, which IS fragile to JPEG.) The spectral phase-coherence approach from `github.com/aloshdenny/reverse-SynthID` was evaluated (May 2026) and **does not work for real-content detection**: on its own shipped codebook + validation set, watermarked and cleaned images were indistinguishable (conf within noise, cleaned often higher); it only fires on pure-black 1024x1024 reference images at exact resolution (the controlled case it was calibrated on). The README's "90% / conf=0.91" reproduces only in that lab condition. Do not build a production detector on it; if revisited, it is experimental/diagnostic only and needs a per-resolution, per-model reference corpus. A from-scratch gpt-image pilot (2026-05-24) confirmed this independently: 5 independent solid-black gpt-image outputs share a near-identical fixed signature (pairwise residual correlation **0.92**, avg-template retains 97% energy), so the watermark/carrier IS strongly present and consistent on flat content — but the carrier frequencies extracted from it do NOT discriminate real content (carrier-to-random ratio: cleaned 1.86 > watermarked 1.53; a non-gpt-image image scored highest at 3.67). The signature drowns in content texture. Net: a perfectly consistent solid-color signature still yields no real-content pixel detector with magnitude/carrier methods. A corpus discrimination test (2026-05-24, `scripts/synthid_pixel_probe.py`, raw zero-mean residual NCC) independently re-confirms this: at matched resolution, SynthID positives do NOT cluster apart from negatives (within-Gemini 0.07; at 1024 px pos-vs-neg >= pos-vs-pos). The only high correlations Line truncated
- **External AI-vs-real classifier models are out of scope (decided 2026-05-24).** Generic HuggingFace detectors (`Organika/sdxl-detector` Swin Transformer, `umm-maybe/AI-image-detector`, and fine-tunes) exist and report ~0.98 on their *own* SDXL-vs-real validation sets, but they are per-generator and the model cards themselves note degraded accuracy off-distribution; they are untested on gpt-image / Gemini Nano Banana (the metadata-stripped surfaces we care about), and our own light SDXL pass would likely defeat them the same way it defeats SynthID. Detection here stays local + signal-based (metadata + visible sparkle); do not add a bundled classifier dependency.
- **DEFAULT STRENGTH IS VENDOR-ADAPTIVE, ONE LADDER FOR BOTH PIPELINES (raised + unified 2026-06-09; vendor-adaptive since 2026-06-01, SUPERSEDES every fixed-default claim in this bullet and the next).** `resolve_strength(strength, vendor)` + `vendor_for_strength(path)` (`watermark_profiles.py`) read the C2PA issuer (`metadata.synthid_source`) on the ORIGINAL input and pick `OPENAI_STRENGTH` **0.20** / `GEMINI_STRENGTH` **0.30** / `UNKNOWN_STRENGTH` **0.30** when `--strength` is unset; explicit `--strength` always wins. **The SAME ladder applies to BOTH pipelines** (`sdxl` and `controlnet`) -- these are the 2026-06-04 Modal-cert controlnet floors. **Why one ladder (NOT a per-pipeline split):** the cert was run on controlnet and does NOT transfer to `sdxl` by symmetry (opposite hard cases -- controlnet leaves SynthID on photoreal, `sdxl` on flat graphics), BUT on its OWN hard case (flat fills) `sdxl` is the WEAKER remover (plain img2img barely perturbs a flat region at low strength), so it needs AT LEAST controlnet's strength -- hence the certified floor is the right floor for `sdxl` too. It is a MARGIN argument for `sdxl`, not a fresh certification (no local SynthID detector to self-verify); raise `--strength` if an oracle still reads a flat `sdxl` output. The higher strength costs little quality because `controlnet` is now the default pipeline AND the only `--auto` pick, so `sdxl` is reached only via an explicit `--pipeline sdxl` (a deliberate opt-down for inputs without faces/text), where over-regeneration has nothing to damage. (A short-lived per-pipeline split ladder -- `sdxl` 0.15/0.20 vs controlnet 0.20/0.30 -- existed on 2026-06-09 before being unified the same day; the `resolve_strength` `pipeline` param and the `CONTROLNET_*_STRENGTH` constants were removed.) The CLI detects the vendor from the pristine source (before the visible pass / metadata-strip removes C2PA from the temp file) and passes it to display calls so display and execution agree; `cmd_invisible`/`cmd_all`/`batch` thread `vendor`. **This replaces the single 0.30 default AND the prior "do NOT build a vendor-adaptive default" policy** -- both came from the now-debunked region-rescrub-contaminated study (the per-region re-scrub that contaminated those numbers was removed in the controlnet refactor). Basis: the oracle-verified June 2026 controlled study (clean v0.8.6, protect OFF): OpenAI clears at 0.05 across 1024-1600 (n=4, resolution-independent); Google needs 0.15 on the capped-1536 path (n=4). `docs/synthid.md` §2.2 (data) + §5.2 (the adaptive default) are authoritative. **CAVEAT (oracle pass 2026-06-04): the OpenAI 0.10 default is content-dependent, NOT universal -- a flat-graphic OpenAI logo/poster still read SynthID-detected after `default` at 0.10, and photoreal images after controlnet at 0.10/0.15 (low-change regions under-perturbed). Removal at 0.10/0.15 is content×pipeline dependent (see the controlnet Known-limitations bullet); the lever is a higher strength, oracle-revalidated per content type. Do NOT assume the vendor-adaptive default clears every image.** CAVEAT: Google's 0.15 was validated only on `--max-resolution 1536`; native large Gemini (2816) was not locally measurable (OOM on M-series) and is pending GPU validation on raiw.cc -- if it survives 0.15 native, raise `--strength`. **Everything below in this bullet about a fixed 0.10/0.30 default is HISTORICAL; trust the vendor-adaptive constants + docs/synthid.md.**
- **SynthID removal: strength + oracle scope.** Default strength is vendor-adaptive (see the bullet above); `docs/synthid.md` §2.2 is authoritative for the numbers. **Oracle scope (load-bearing):** the Gemini app "Verify with SynthID" is the ONLY valid SynthID oracle (detects Google's mark on any image); `openai.com/verify` is scoped to OpenAI provenance (its own C2PA), NOT a SynthID oracle -- a negative there is meaningless for SynthID. There is no local SynthID detector, so the tool cannot self-check; if the oracle still reads SynthID, raise `--strength` to the lowest value that verifies clean. Only the `sdxl` (plain SDXL img2img; `default` is a back-compat alias) and `controlnet` (SDXL + canny ControlNet) profiles exist; the local `invisible` default is weight-for-weight identical to raiw.cc prod (`fal-ai/fast-sdxl` = `stabilityai/stable-diffusion-xl-base-1.0`, runtime-downloaded, not bundled). **Forensic-stealth caveat** (arXiv:2605.09203): defeating the SynthID verifier is NOT forensic invisibility -- independent detectors flag *removal-processed* images vs genuinely-clean ones at >98% TPR@1%FPR, so do not over-claim "indistinguishable from a real photo".
- **`controlnet` pipeline (text/face STRUCTURE preservation, THE DEFAULT since 2026-06-09; `--pipeline default` opts down to plain SDXL).** SDXL + the canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` via `StableDiffusionXLControlNetImg2ImgPipeline` (`watermark_remover._run_controlnet` / `_load_controlnet_pipeline`). **Removal still comes from the img2img regeneration (`strength`); the ControlNet only PRESERVES text and face STRUCTURE by conditioning on the canny edge map** (`cv2.Canny(gray, 100, 200)`, 3-channel). Canny preserves edges, NOT face identity (a regenerated face drifts in likeness). The drifted cleaned face is the LEAST-AI state we can reach without re-introducing SynthID; **the library does NOT ship a face-restore extra** (every approach evaluated 2026-06-04 - 2026-06-08 -- GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps -- regenerated the face via SDXL and made it look MORE AI-generated). Full empirical conclusion in `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up". For production face preservation, ship the cleaned image as-is. No original pixels are copied or frozen, **BUT removal at the low vendor-adaptive strength is CONTENT × PIPELINE dependent and NEITHER pipeline clears all content -- oracle-validated against the OpenAI verifier 2026-06-04 (8 images, strength 0.10/0.15, `--max-resolution 1536`).** The survivors FLIP by content type: **photoreal** (a 9-face grid, a bracelet product photo) SURVIVES controlnet but CLEARS `default` (controlnet's dense edge map keeps the regen too close to the original, so the SynthID-destroying perturbation never happens; plain img2img perturbs photoreal texture enough); **flat graphic** (a logo/poster with large flat color fills) SURVIVES `default` but CLEARS controlnet (at low strength img2img barely changes flat fills so SynthID persists there, while controlnet repaints them more freely); a flat **text** card cleared under both. **Root cause is insufficient STRENGTH, not the pipeline: at 0.10 the low-change regions -- dense-edge photoreal under controlnet, large flat fills under `default` -- are not perturbed enough to destroy SynthID. The vendor-adaptive 0.10 from the June study is NOT universally sufficient (that study's content happened to clear at 0.10).** The robust fix is a HIGHER strength, oracle-revalidated per content type (controlnet can be cranked harder without losing structure; a lower `controlnet_conditioning_scale` also frees the regen on photoreal). So at today's default strength **both pipelines AND `--auto` can LEAVE SynthID on some content** -- a removal-priority caller (raiw.cc) MUST oracle-validate strength across content types before adopting, not pick a pipeline and assume removal. **Follow-up same day: re-running the two photoreal survivors through controlnet at an explicit `--strength 0.15` cleared BOTH on the oracle -- BUT one of them (the bracelet) had SURVIVED the SAME 0.15 controlnet config in the first pass (only the random, unset seed differed). So removal near the threshold is SEED-NON-DETERMINISTIC: the same image+pipeline+strength+resolution can pass or fail run-to-run (img2img uses `seed=None`/random unless `--seed` is passed, and there is no local SynthID detector to self-verify). 0.15 is the borderline, NOT a robust floor -- pick a strength with MARGIN (controlnet ~>= 0.20) rather than exactly on it; the content×pipeline table's 0.15 data point is near-threshold noise. A confirming run at `--strength 0.20` controlnet cleared BOTH photoreal survivors on the oracle (ladder: 0.10 grid detected → 0.15 borderline/non-deterministic → 0.20 both clean), so **0.20 is the recommended robust controlnet floor for OpenAI photoreal** (one margin run, not an N-run repeatability proof -- a service should add margin or verify repeatability since there is no local SynthID detector to self-check). **Engineering follow-up DONE 2026-06-09 (three coupled changes):** (1) **strength raised + unified** -- `resolve_strength(strength, vendor)` now applies ONE vendor-adaptive ladder (the certified controlnet floors 0.20/0.30/0.30) to BOTH pipelines; see the DEFAULT STRENGTH bullet above for why one ladder covers `sdxl`. (2) **`controlnet` is now the DEFAULT pipeline** (CLI `--pipeline` default = `controlnet` + both engine ctors). Rationale: with the certified higher ladder it clears BOTH content classes that flipped in the content-x-pipeline table (photoreal AND flat graphic), whereas plain SDXL left SynthID on flat graphics -- so controlnet is the more removal-robust default. Cost: every non-`--auto` run now downloads the canny ControlNet weights + a higher memory peak (MPS->CPU fallback covers OOM). (3) **the plain-SDXL profile was renamed `default` -> `sdxl`** (`watermark_profiles.SDXL_PROFILE`/`normalize_profile`); `default` stays as a back-compat CLI/ctor alias (the `--pipeline` Choice accepts `sdxl`/`controlnet`/`default`, a click callback `_noLine truncated
| File | Covers |
|---|---|
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, and the detection-path measurement rule |
+415 -407
View File
@@ -1,464 +1,472 @@
# Remove-AI-Watermarks
# Remove AI Watermarks
Remove **visible** and **invisible** AI watermarks from images generated by Google Gemini (Nano Banana), ChatGPT / DALL-E, Stable Diffusion, Adobe Firefly, Midjourney, and other AI models.
Remove AI provenance marks from images and video you generated yourself:
Strips SynthID, C2PA Content Credentials, EXIF/XMP "Made with AI" labels, and visible sparkle overlays — all in one command.
- known visible labels such as the Gemini sparkle and vendor text marks;
- invisible pixel watermarks through diffusion regeneration;
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
> ## Try it online: [raiw.cc](https://raiw.cc)
>
> No Python, no GPU, no setup. Visible-watermark and metadata removal are **free**. Invisible-watermark removal (SynthID / SDXL regeneration) normally needs a local GPU and ~2 GB of models. On **[raiw.cc](https://raiw.cc)** it runs on cloud GPUs in one click for a small per-image fee.
Video support covers provenance identification, complete visible-plus-metadata
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo, and
Kling mark removal, and oracle-certified VAE regeneration for video SynthID
removal.
> [raiw.cc](https://raiw.cc) runs this library as a hosted service, with the GPU
> included and nothing to install. Visible mark and metadata removal are free
> there; invisible watermark removal is paid.
[![PyPI](https://img.shields.io/pypi/v/remove-ai-watermarks?logo=pypi&logoColor=white)](https://pypi.org/project/remove-ai-watermarks/)
[![Python](https://img.shields.io/pypi/pyversions/remove-ai-watermarks?logo=python&logoColor=white)](https://pypi.org/project/remove-ai-watermarks/)
[![Downloads](https://static.pepy.tech/badge/remove-ai-watermarks/month)](https://pepy.tech/project/remove-ai-watermarks)
[![License](https://img.shields.io/pypi/l/remove-ai-watermarks?color=blue)](LICENSE)
[![Tests](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml/badge.svg)](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml)
[![Sponsor](https://img.shields.io/badge/Sponsor-GitHub-db61a2?logo=githubsponsors&logoColor=white)](https://github.com/sponsors/wiltodelta)
If this tool saves you time, consider [sponsoring its development](https://github.com/sponsors/wiltodelta).
> This project is for lawful use on content you own. It does not target stock
> agency previews or other watermarks that protect third party paid content.
> See [scope, safety, and legal notes](docs/legal-and-safety.md).
> **Intended for lawful use only.** Publishing and running this software is lawful; responsibility for any downstream use, and for compliance with local law, rests entirely with the user. Some jurisdictions restrict removing an AI label as such (see [Legal](#legal)). The authors do not condone use for deception, fraud, or any unlawful activity.
## Choose what you want to do
## Features
| Goal | Command | GPU |
| --- | --- | --- |
| Find provenance signals and watermarks | `identify` | No |
| Remove known visible AI marks | `visible` | No |
| Erase a region you select | `erase` | No |
| Strip AI metadata | `metadata` | No |
| Identify supported video provenance | `video identify` | No |
| Remove visible marks and AI metadata from video | `video all` | No |
| Strip AI metadata from video | `video metadata` | No |
| Remove a registered visible AI mark from video | `video visible` | No |
| Process a directory of videos | `video batch` | Depends on mode |
| Remove video SynthID with the certified VAE profile | `video invisible` | Recommended |
| Regenerate an image to disrupt invisible watermarks | `invisible` | Required (CUDA) |
| Run visible, invisible, and metadata removal | `all` | Recommended |
| Process a directory | `batch` | Depends on mode |
- **Visible watermark removal** — a registry of known marks in their usual places: the Gemini / Nano Banana sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, and the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-left, locale-specific). Each is removed by **reverse-alpha blending** against a captured alpha map (`original = (wm − α·logo)/(1−α)`), recovering the true pixels rather than inpainting a guess. The Gemini sparkle recovers cleanly on its own on bright backgrounds; it adapts the alpha to each image's sparkle opacity, so a more-opaque-than-captured sparkle is still fully removed (and on a dark background, where the fixed alpha would over-subtract and leave a dark spot, it automatically inpaints the small sparkle footprint instead); the Doubao, Jimeng, and Samsung text marks re-rasterize slightly per image, so a thin residual inpaint over the glyph footprint clears the leftover edges (the alpha maps are reproducibly rebuilt from controlled captures by `scripts/visible_alpha_solve.py`). Fast, offline, no GPU. `visible --mark auto` finds and removes the strongest detected mark. (For arbitrary logos/objects, see `erase`.)
- **Universal region eraser (`erase`)** — remove any logo / watermark / object inside boxes you specify, regardless of position or colour. Default cv2 inpainting (CPU, instant); optional big-LaMa via onnxruntime (`lama` extra) for higher quality
- **Invisible watermark removal** — SynthID, StableSignature, TreeRing via diffusion-based regeneration (needs a local GPU, or run it with no setup on [raiw.cc](https://raiw.cc))
- **AI metadata stripping** — EXIF, PNG text chunks, C2PA provenance manifests (PNG / JPEG / AVIF / HEIF / JPEG-XL, **MP4 / MOV / M4V / M4A** at the container level, and **WebM / MP3 / WAV / FLAC / OGG** losslessly via ffmpeg), XMP DigitalSourceType
- **"Made with AI" label removal** — removes the AI-disclosure metadata that platforms read to apply automatic labels (useful for clearing a false-positive label from a human-edited photograph)
- **Analog Humanizer** — optional film grain and chromatic aberration post-processing
- **Text and face preservation (default)** — the default pipeline is a canny ControlNet that keeps text and face structure sharp through the removal pass (without copying original pixels, so SynthID is still removed). Use `--pipeline sdxl` for plain SDXL img2img (lighter, no extra model download) on inputs without text or faces. Canny preserves face *structure*, not *identity* (the regenerated face drifts in likeness). The library does not ship a face-restore extra: every approach evaluated (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned) regenerated the face via SDXL and made the output look more AI-generated than the cleaned image. The cleaned controlnet output is the least-AI face state achievable without re-introducing SynthID.
- **Batch processing** — process entire directories
- **Detection** — three-stage NCC watermark detection with confidence scoring
- **Provenance detection (`identify`)** — aggregate C2PA issuer, the C2PA soft-binding forensic-watermark vendor (Adobe TrustMark, Digimarc, Imatag, ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, embedded SD/ComfyUI params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the China TC260 AIGC label (XMP, PNG chunk, or EXIF), the HuggingFace `hf-job-id` job marker, the SynthID metadata proxy, the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), the open SD/SDXL/FLUX invisible watermark, and (with the `trustmark` extra) the open Adobe TrustMark watermark into one origin-platform + watermark-inventory verdict (`--json` for machine output)
Microsoft Paint and Photos InvisMark declarations are routed automatically to
pixel regeneration. The `all` command removes both the hidden pixel watermark and
its linked C2PA manifest; metadata stripping alone removes only the manifest.
## Installation modes
| Need | Install |
| --- | --- |
| Metadata inspection and stripping | `remove-ai-watermarks` |
| Visible detection and removal | `remove-ai-watermarks[visible]` |
| Visible video processing | `remove-ai-watermarks[video]` |
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
| Torch-free DWT-DCT detection | `remove-ai-watermarks[detect]` |
| Invisible image removal (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
| Every production feature available on the active Python | `remove-ai-watermarks[all]` |
Lower-level and specialized extras include `pixels`, `heif`, `trustmark`,
`migan`, `lama`, and `diffusion`. The
[installation guide](docs/installation.md#feature-extras) documents their exact
dependency composition, Python compatibility, and model requirements.
## Quick start
Install the metadata-focused default CLI:
```bash
uv tool install remove-ai-watermarks
```
Inspect an image:
```bash
remove-ai-watermarks identify image.png
```
For visible watermark removal, install the pixel dependencies:
```bash
uv tool install --force "remove-ai-watermarks[visible]"
```
Then remove a known visible mark and AI metadata:
```bash
remove-ai-watermarks visible image.png -o clean.png
```
Strip metadata without running visible inpainting or diffusion:
```bash
remove-ai-watermarks metadata image.png --remove -o clean.png
```
Without `-o` this command overwrites the source in place.
Inspect or remove AI metadata from an MP4, MOV, M4V, WebM, MKV, AVI, or FLV
file:
```bash
remove-ai-watermarks video metadata input.mp4 --check
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
```
The `video metadata` command does not transcode video or audio streams. Unlike
the image command above, when `-o` is omitted it writes `<source>_clean` and
preserves the original. MP4 and MOV
inspection includes the native TC260 `AIGC` tag in
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload,
plus the QuickTime-form `meta` variants Doubao's iOS export writes (a bare
`meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list).
MKV and WebM inspection reads the normative
`Segment.Tags.Tag.SimpleTag` placement. AVI uses `LIST/INFO/AIGC`, while FLV
uses `script.onMetaData.AIGC`. The non-ISOBMFF formats are remuxed with stream
copy for removal.
Use the product-oriented video path to identify or clean a file:
```bash
uv tool install --force "remove-ai-watermarks[video]"
remove-ai-watermarks video identify input.mp4
remove-ai-watermarks video all input.mp4 -o clean.mp4
```
`video all` removes a stable registered visible mark when present and always
strips verified AI metadata. If neither signal is found, it still writes a
same-container passthrough, so application callers get one predictable output
contract. Proprietary invisible-video removal is excluded by default.
`--invisible` opts into the lossy, oracle-certified video SynthID profile.
Process a directory with the same contract:
```bash
remove-ai-watermarks video batch ./videos --mode all
```
Remove a supported visible video mark:
```bash
remove-ai-watermarks video visible input.mp4 -o clean.mp4
remove-ai-watermarks video visible veo.mp4 --mark veo -o veo_clean.mp4
remove-ai-watermarks video visible seedance.mp4 --mark seedance -o seedance_clean.mp4
remove-ai-watermarks video visible dola.mp4 --mark dola -o dola_clean.mp4
remove-ai-watermarks video visible hailuo.mp4 --mark hailuo -o hailuo_clean.mp4
remove-ai-watermarks video visible kling.mp4 --mark kling -o kling_clean.mp4
```
This path scans the complete sequence before changing pixels. It accepts only a
mark that repeats at a stable position across adjacent frames, then reuses the
same OpenCV, MI-GAN, or LaMa fill backends as image removal. Audio is copied
without re-encoding and is allowed to reach its natural end; the video stream
is transcoded because its pixels change. By default, a guarded optical-flow
pass motion-aligns the preceding accepted fill and blends it only when the
nearby source context agrees; use `--no-temporal-consistency` to disable it.
The encoder preserves supported
8-bit source chroma sampling, color tags, and MP4/MOV track timescale instead
of relying on ffmpeg's implicit raw-BGR defaults. Variable frame intervals are
preserved through a timestamped in-memory NUT bridge instead of being flattened
to the average frame rate. Non-zero source start timestamps are retained
together with the copied audio offset. The default `--mark auto`
scans all providers in one decode pass and selects the first stable match in
the specificity order shown below. Pass an explicit mark to restrict detection
to one provider.
Sora covers the moving Sora 2 mascot and wordmark. Veo covers both the current
four-point diamond and the legacy `Veo` text. Seedance covers the fixed boxed
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo covers the composite
`MINIMAX | hailuo AI` label, and Kling covers the bottom-right `KLING AI`
label with its version suffix. A completed encode is published atomically. No
output is written when no stable mark is found.
HDR, PQ/HLG, and greater-than-8-bit inputs are rejected before encoding rather
than silently reduced through OpenCV's 8-bit BGR boundary.
Remove video SynthID:
```bash
uv tool install --force "remove-ai-watermarks[video,diffusion]"
remove-ai-watermarks video invisible input.mp4 -o clean.mp4
```
This path regenerates the complete sequence with one latent-noise field shared
across time, copies complete audio, strips source metadata, and publishes the
completed encode atomically. The default `noise_std=0.15` profile passed both
the two-carrier calibration and a complete public eight-second Veo oracle
check. Google does not publish a local decoder, so a fresh provider check
remains useful for unusually important files or after provider changes, but it
is not a product result state.
For invisible watermark removal, install the `qwen-zimage` extra. **An NVIDIA GPU
is required**: both profiles are CUDA-only, and there is no CPU or MPS fallback.
```bash
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
remove-ai-watermarks invisible image.png -o clean.png
```
If the local detectors cannot confirm an invisible watermark but you know the
image came from an AI generator, add `--force`:
```bash
remove-ai-watermarks invisible image.png -o clean.png --force
```
Typography-heavy images can opt into the experimental verified-text post-pass.
It accepts manually reviewed strings and line boxes or an operator-verified
geometry-only manifest; it never treats raw OCR output as ground truth or runs
automatically:
```bash
uv tool install --force "remove-ai-watermarks[text-restoration]"
remove-ai-watermarks invisible image.png -o clean.png \
--text-manifest verified-lines.json --force
```
See the [CLI guide](docs/cli.md#restore-operator-verified-text) for the manifest
schema, compatibility restrictions, and oracle caveats.
See the [installation guide](docs/installation.md) for Homebrew, uv, optional
features, and development setup.
## Examples
| Before (Watermarked) | After (Cleaned) |
### Visible Gemini mark
| Before | After |
| --- | --- |
| ![Before](demo_banana_before.png) | ![After](demo_banana_after.png) |
| ![Image with a visible Gemini watermark](demo_banana_before.png) | ![Image after visible watermark removal](demo_banana_after.png) |
## Supported models
### High quality invisible removal
| AI model | Visible watermark | Invisible watermark | Metadata | Our approach |
| --- | --- | --- | --- | --- |
| **Google Gemini / Nano Banana / Gemini 3 Pro** | ✅ Sparkle logo | ✅ SynthID v1 + v2 (default SDXL pipeline, native resolution) | ✅ C2PA + EXIF | Alpha reversal + diffusion + metadata strip |
| **OpenAI DALL-E 3 / ChatGPT** | — | — | ✅ C2PA manifest | Metadata strip |
| **OpenAI ChatGPT Images 2.0** (gpt-image-2) | — | ✅ SynthID + content-specific pixel watermark (since May 2026; no local decoder, openai.com/verify oracle) | ✅ C2PA manifest (verified) | Diffusion regeneration + metadata strip |
| **Stable Diffusion / SDXL (AUTOMATIC1111, ComfyUI)** | — | ✅ DWT-DCT (imwatermark — locally detectable) | ✅ PNG text chunks | Diffusion regeneration + metadata strip |
| **Black Forest Labs FLUX** | — | ✅ DWT-DCT (imwatermark — locally detectable) | ✅ C2PA (FLUX.2 Pro) | Diffusion regeneration + metadata strip |
| **Adobe Firefly** | — | — | ✅ Content Credentials (C2PA) | Metadata strip |
| **Stability AI** (DreamStudio / Stable Image) | — | — | ✅ C2PA ("Stability AI Ltd") | Metadata strip |
| **Microsoft Designer / Bing Image Creator** | — | ✅ SynthID via DALL-E backend (Designer) | ✅ C2PA (Bing runs MAI-Image, signed "Microsoft") | Metadata strip |
| **xAI Grok (Aurora)** | — | — | ✅ EXIF signature scheme (no C2PA): `Signature:` blob + UUID `Artist` | Detected (`identify`); metadata strip |
| **Midjourney** | — | — | ✅ EXIF + XMP (prompt, model, seed) | Metadata strip |
| **Meta AI** | — | — | ✅ IPTC "Made with AI" (digitalSourceType) | Metadata strip (removes the label) |
| **Doubao** (ByteDance) / China AIGC generators | ✅ "豆包AI生成" text strip (bottom-right) | — | ✅ TC260 AIGC label (`<TC260:AIGC>` XMP, `AIGC` PNG chunk, or EXIF JSON) **+ C2PA** signed by ByteDance Volcano Engine (`volcengine`) | Reverse-alpha (captured α map) + thin residual inpaint, NCC-aligned across resolutions, + metadata strip |
| **Jimeng / Dreamina** (即梦AI, ByteDance) | ✅ "★ 即梦AI" wordmark (bottom-right) | — | ✅ TC260 AIGC label + C2PA (Volcano Engine) | Reverse-alpha (captured α map) + residual inpaint over the glyph footprint, NCC-aligned across resolutions, + metadata strip |
| **Samsung Galaxy AI** (Generative Edit, Sketch to Image, ...) | ✅ "✦ Contenuti generati dall'AI" strip (bottom-left, locale-specific) | — | ✅ C2PA (signer "Samsung Galaxy") + `trainedAlgorithmicMedia` / proprietary `genAIType` marker | Reverse-alpha (captured α map) + thin residual inpaint, NCC-aligned across resolutions, + metadata strip |
| **Black Forest Labs** (FLUX API) | — | — | ✅ C2PA (`Black Forest Labs API` + `c2pa.ai_generated_content` + `trainedAlgorithmicMedia`) | Metadata strip |
| **StableSignature** (Meta) | — | ✅ In-model watermark | — | Diffusion regeneration |
| **TreeRing** | — | ✅ Latent space watermark | — | Diffusion regeneration |
`qwen-zimage` is the default profile: a Qwen-Image-2512 Lightning pass under Canny
ControlNet, followed by SAM-masked Z-Image repair of any detected face. The
alternative, `sdxl-zimage`, swaps the global stage for SDXL and keeps the same face
stage. Both are CUDA only.
> Visible overlays are used by Google Gemini / Nano Banana (sparkle logo), by ByteDance's Doubao ("豆包AI生成" corner text) and Jimeng / Dreamina ("★ 即梦AI" wordmark), and by Samsung Galaxy AI ("✦ Contenuti generati dall'AI" strip, bottom-left, locale-specific). All are removed on CPU by reverse-alpha against a captured alpha map (Jimeng and Samsung add a thin residual inpaint over the glyph footprint, since their marks re-rasterize per image). Other services rely on invisible watermarks and/or metadata; our diffusion-based regeneration works against any invisible watermark in pixel or frequency domain. For a visible mark from any other source (any position, any colour), use the universal `erase --region` command.
```bash
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
remove-ai-watermarks invisible image.png -o clean.png --force
```
> **Detection:** `remove-ai-watermarks identify <image>` reports the origin platform and watermark inventory for all the signals above — C2PA issuer, the C2PA soft-binding forensic-watermark vendor (TrustMark / Digimarc / Imatag / ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, the China TC260 AIGC label (XMP, PNG chunk, or EXIF), the HuggingFace `hf-job-id` job marker, embedded generation params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the SynthID metadata proxy, the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), and (with the `[detect]` / `[trustmark]` extras) the open SD/SDXL/FLUX and Adobe TrustMark invisible watermarks. SynthID and the proprietary soft-binding watermarks (Digimarc etc.) have no local decoder, so they are reported by metadata proxy / vendor name only.
| OpenAI example before | OpenAI example after |
| --- | --- |
| [![OpenAI portrait grid before qwen-zimage](data/synthid/originals/ChatGPT%20Image%20May%2030,%202026,%2010_31_08%20AM.png)](data/synthid/originals/ChatGPT%20Image%20May%2030,%202026,%2010_31_08%20AM.png) | [![OpenAI portrait grid after qwen-zimage](docs/images/qwen-zimage/ChatGPT/ChatGPT%20Image%20May%2030,%202026,%2010_31_08%20AM_full_clean.png)](docs/images/qwen-zimage/ChatGPT/ChatGPT%20Image%20May%2030,%202026,%2010_31_08%20AM_full_clean.png) |
| Gemini example before | Gemini example after |
| --- | --- |
| [![Gemini sign before qwen-zimage](data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png)](data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png) | [![Gemini sign after qwen-zimage](docs/images/qwen-zimage/Gemini/Gemini_Generated_Image_633uuy633uuy633u_full_clean.png)](docs/images/qwen-zimage/Gemini/Gemini_Generated_Image_633uuy633uuy633u_full_clean.png) |
These exact output files were checked with the matching provider verifiers. That
result applies to these files, not to every seed, image, or future watermark
version.
## Common recipes
### Remove every detected visible mark
```bash
remove-ai-watermarks visible image.png -o clean.png
```
The default `--mark auto` checks all registered visible marks and removes every
match. If the mark is visible to you but the detector misses it, select its
region explicitly:
```bash
remove-ai-watermarks erase image.png \
--region 1640,1930,400,100 \
-o clean.png
```
`--region` uses `x,y,width,height` and may be repeated.
### Use a learned fill backend
The `visible` extra uses OpenCV inpainting when no learned backend is installed.
For more difficult backgrounds, the learned-backend extras include the same
pixel dependencies automatically:
```bash
uv tool install --force "remove-ai-watermarks[migan]"
remove-ai-watermarks visible image.png -o clean.png --backend migan
```
```bash
uv tool install --force "remove-ai-watermarks[lama]"
remove-ai-watermarks visible image.png -o clean.png --backend lama
```
### Reduce CUDA memory use
```bash
remove-ai-watermarks invisible image.png -o clean.png \
--cpu-offload --force
```
CPU offload lowers CUDA memory pressure by moving model components between CPU
and GPU, at the cost of speed.
### Process a directory
```bash
remove-ai-watermarks batch ./images --mode visible
remove-ai-watermarks batch ./images --mode all
```
## What the tool can recognize
Visible mark support includes:
- Google Gemini and Nano Banana sparkle;
- Doubao, Jimeng, Qwen, Kling, Yuanbao, Baidu, LibLibAI, and RunningHub labels;
- one calibrated Samsung Galaxy AI label variant.
Metadata and provenance inspection covers C2PA, EXIF, XMP, IPTC, common
generator parameters, China TC260 AIGC labels, and several vendor specific
signals. Optional decoders add support for open DWT-DCT watermarks and Adobe
TrustMark.
The exact support matrix, including important locale and detector limits, lives
in [supported signals](docs/supported-signals.md).
## How it works
### Removing the Gemini / Nano Banana sparkle watermark
Visible removal follows three steps:
Google Gemini (internally codenamed **Nano Banana**) adds a visible sparkle logo to generated images using alpha blending:
1. Detect a registered mark in its expected area.
2. Build a mask around the mark.
3. Fill only the masked region with OpenCV, MI-GAN, or LaMa.
```text
watermarked = α × logo + (1 − α) × original
```
Metadata removal uses format aware stripping. JPEG metadata removal preserves
the encoded image scan instead of recompressing it. Native MP4/MOV TC260 values
are blanked without changing box sizes or media offsets. Other supported
containers use their corresponding metadata path.
We reverse this with a known alpha map (extracted from Gemini / Nano Banana output on a pure-black background):
Invisible removal is different. It regenerates the image through a diffusion
pipeline to disrupt pixel and frequency domain watermarks. This changes the
image and cannot guarantee that a proprietary verifier will reject every
output.
```text
original = (watermarked − α × logo) / (1 − α)
```
See [supported signals](docs/supported-signals.md) and
[known limitations](docs/known-limitations.md) for the full technical boundary.
A three-stage NCC (Normalized Cross-Correlation) detector finds the watermark position and scale dynamically, so it works even if the image was resized or cropped. After removal, residual sparkle-edge artifacts are cleaned via gradient-masked inpainting.
## Python API
**Speed**: ~0.05s per image. No GPU needed.
### Removing the Doubao "豆包AI生成" text watermark
Doubao (ByteDance) stamps every output with a light, semi-transparent "豆包AI生成" text strip in the bottom-right corner — the visible AIGC label mandated by China's TC260 standard. It is a fixed semi-transparent white overlay, so it is removed by **reverse-alpha blending**: `original = (watermarked - α·logo) / (1 - α)`, recovering the true pixels instead of hallucinating them. The α map is solved from controlled black/gray captures (rebuildable with `scripts/visible_alpha_solve.py`). Like the Jimeng mark, Doubao re-rasterizes its text slightly per image, so reverse-alpha is followed by a thin residual inpaint over the glyph footprint to clear the leftover edges, and the α template is NCC-aligned to the actual mark (handling per-image scale/position jitter). Detection matches the same glyph silhouette against the corner (normalized correlation), so it keys on the "豆包AI生成" shape, not on textured corners.
**Speed**: ~0.05s, no GPU needed.
### Removing the Jimeng "★ 即梦AI" wordmark
Jimeng / Dreamina (即梦AI, also ByteDance, distinct from Doubao) stamps a "★ 即梦AI" wordmark — a four-point sparkle followed by the 即梦AI characters — in the bottom-right corner. It is a fixed semi-transparent **pure-white** overlay, solved from controlled black / gray / white captures the same way as Doubao. `visible --mark auto` detects and removes it (or force it with `--mark jimeng`). One difference from Doubao: Jimeng re-rasterizes its mark slightly differently per image, so a single alpha map does not cancel it pixel-for-pixel — reverse-alpha knocks the mark down and a residual inpaint over the glyph footprint clears the remaining outline. The two ByteDance marks do not confuse `auto`: detection keys on each mark's own glyph shape (the Jimeng detector scores far below its threshold on a Doubao strip, and vice versa).
```bash
remove-ai-watermarks visible jimeng.png -o clean.png # --mark auto picks Jimeng
remove-ai-watermarks visible jimeng.png --mark jimeng -o clean.png
```
### Removing the Samsung Galaxy AI "✦ Contenuti generati dall'AI" mark
Samsung's on-device Generative AI edits (Generative Edit, Sketch to Image, Portrait Studio) burn a visible sparkle + "generated with AI" string into the **bottom-left** corner — a faint, low-opacity semi-transparent white overlay. It is solved from controlled black / gray / white captures the same way as Jimeng and removed by reverse-alpha plus a thin residual inpaint over the glyph footprint (the mark re-rasterizes per image, and the flat captures are smaller than real photos, so the alpha template is NCC-aligned and width-scaled to the actual mark). `visible --mark auto` detects and removes it (or force it with `--mark samsung`); being bottom-left it never confuses the bottom-right Gemini/Doubao/Jimeng marks. The string is **locale-specific** — this build is calibrated for the Italian "Contenuti generati dall'AI" variant; other locales need their own captured template (open a sample on issue #37).
```bash
remove-ai-watermarks visible samsung.jpg -o clean.jpg # --mark auto picks Samsung
remove-ai-watermarks visible samsung.jpg --mark samsung -o clean.jpg
```
### Universal region eraser
For any visible mark the dedicated engines do not cover — a logo anywhere, any colour — `erase --region x,y,w,h` inpaints the box you specify. The default `cv2` backend is instant and dependency-free; the optional `lama` backend (big-LaMa via onnxruntime, `lama` extra, ~200 MB model downloaded on first use) gives much cleaner fills on textured regions at the cost of ~3-4 GB RAM per call.
### Removing SynthID and other invisible watermarks
Google embeds **SynthID** into every image generated by Gemini / Nano Banana. Other AI services use StableSignature, TreeRing, and similar schemes. These imperceptible frequency-domain patterns survive cropping, resizing, and JPEG compression.
The removal pipeline (default profile, SDXL):
```text
image → encode to latent space (VAE) at native resolution
→ add controlled noise (forward diffusion)
→ denoise (reverse diffusion, ~50 steps; strength is vendor-adaptive:
0.20 OpenAI / 0.30 Google / 0.30 unknown, same for both pipelines;
override with --strength)
→ decode back to pixels (VAE)
```
- Large inputs run at native resolution (no down-then-up round-trip, which was the main quality loss in issue #10); use `--max-resolution N` only to cap GPU/MPS memory on very large inputs. Small inputs (long side under 1024 px) are auto-upscaled to a 1024 px floor before diffusion, because SDXL distorts on a tiny latent, and the result is restored to the original size (a transparent quality boost). Disable the floor with `--min-resolution 0`. The floor upscale uses Lanczos by default; `--upscaler esrgan` (the `esrgan` extra) runs Real-ESRGAN first for sharper detail and falls back to Lanczos if the extra is absent. ESRGAN is a generic photo/texture GAN with no face/glyph prior, so it is best for photo/texture content -- it can degrade faces (the diffusion pass regenerates them, so the final recovers) and thin text; keep Lanczos for text-heavy inputs.
> **Default strength is vendor-adaptive (no flag needed).** The tool reads the C2PA issuer to detect which vendor's SynthID is present and picks the strength accordingly: **OpenAI gpt-image → `0.20`**, **Google Gemini → `0.30`**, **unknown source → `0.30`**. The **same ladder applies to both pipelines** — these are the oracle-certified `controlnet` floors (June 2026 Modal cert, multi-seed). They also cover plain `sdxl`: the two pipelines have opposite hard cases (controlnet leaves SynthID on photoreal, sdxl on flat graphics), but on its own hard case sdxl is the weaker remover, so it needs at least controlnet's strength — using one certified ladder is the safe choice (margin-based for sdxl, not separately certified). The dominant factor is the vendor (Google's SynthID is ~3x more robust). There is no local SynthID detector, so if the oracle still reads SynthID, raise `--strength`; if you care more about preserving fine detail, lower it. (Caveat: Google's `0.30` was validated only at `--max-resolution 1536`; a very large native Gemini image may need ~`0.35`+.)
>
> **The default pipeline is `controlnet` — it preserves text and face structure.** It runs the same SDXL img2img scrub but adds a canny ControlNet that conditions the regeneration on the image's edge map, so text and structure stay sharp at the strengths that remove SynthID. The watermark removal still comes from the img2img regeneration (`--strength`); the ControlNet only preserves structure — no original pixels are copied or frozen. The default strength ladder (OpenAI `0.20` / Google `0.30`) is the oracle-certified controlnet floor. `--controlnet-scale` tunes the preservation strength (higher = closer to the original structure). Runs fp32 on mps/cpu (fp16 only on cuda/xpu, where the fp16-fixed SDXL VAE is loaded automatically). Pass `--pipeline sdxl` for plain SDXL img2img (lighter, no extra model download) on inputs without text or faces.
>
> **No face-restore extra in the library.** Every ArcFace-based regeneration approach we evaluated (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps, 2026-06-04 - 2026-06-08 Modal cert sweeps) regenerated the face via SDXL diffusion — the output face pixels were diffusion-fresh (SynthID not re-introduced), but the face inherently looked more AI-generated than the cleaned image (SDXL "clean skin" gloss, lost original identity precision). The cleaned image from the main controlnet 0.20 pass is the least-AI face state we can reach without re-introducing SynthID. Empirical conclusion in `docs/synthid-robust-identity-research-2026-06-08.md`.
SDXL is the default since May 2026: empirically defeats SynthID v2 on Gemini 3 Pro outputs, where the older SD-1.5 pipeline at 768 px did not. The SD-1.5 path was removed once it was verified not to handle v2. Note the scope: this defeats the SynthID *verifier*, which is not the same as being forensically indistinguishable from a real photo. Recent work ([arXiv:2605.09203](https://arxiv.org/abs/2605.09203)) shows watermark-removal pipelines leave detectable traces, so a separate "this image was processed" classifier can still flag the output.
> **Oracle vs `identify` can disagree, and that is expected.** An online verifier reads the actual SynthID *pixel* watermark and detects only its own vendor's content — [openai.com/research/verify](https://openai.com/research/verify/) states "OpenAI generation signals will only be detected if the image was generated with our tools". Our `identify` cannot decode the pixel watermark (no vendor ships a local decoder), so it infers SynthID from the **C2PA metadata** instead. So after the SDXL pass the oracle can read "no SynthID" (pixel watermark gone) while `identify` still reports SynthID from a surviving C2PA manifest. They measure different signals. Run `metadata --remove` (or `all`) to also strip the manifest; note that a quiet metadata proxy is not proof the pixel watermark itself is gone.
> **Technical deep-dive:** see [`docs/synthid.md`](docs/synthid.md) for a primary-source-cited breakdown of how SynthID works mechanically (post-hoc encoder/decoder, 136-bit payload, pixel-space embedding), what it empirically survives (JPEG, crop, resize: ~99.98% TPR at 0.1% FPR from arXiv:2510.09263), what removes it, and the forensic-stealth tradeoff (all known removal attacks are detectable at >98% TPR@1%FPR per arXiv:2605.09203).
**Text and face preservation** (the default pipeline; `--pipeline sdxl` opts down to plain SDXL): a canny ControlNet keeps text and face *structure* sharp through the removal pass, without copying or freezing any original pixels (so SynthID is still removed). Tune the preservation strength with `--controlnet-scale`. Canny preserves structure but not face *identity*: the regenerated face drifts in likeness. The library does not ship a face-restore extra (see the callout above).
**Analog Humanizer**: optional film grain and chromatic aberration injection that mimics a photo of a screen, raising the bar for AI-generated image classifiers. (It frustrates generic classifiers but does not guarantee forensic invisibility — see the [arXiv:2605.09203](https://arxiv.org/abs/2605.09203) note above.)
### Stripping C2PA, EXIF, and "Made with AI" metadata
AI tools embed generation metadata that social platforms use to show "Made with AI" labels:
- **EXIF tags** — prompt, seed, model hash, sampler settings (Stable Diffusion, Midjourney)
- **XMP DigitalSourceType** — `trainedAlgorithmicMedia` tag used by Instagram, Facebook, and X (Twitter) to show "Made with AI"
- **PNG text chunks** — ComfyUI workflows, AUTOMATIC1111 parameters
- **C2PA Content Credentials** — cryptographic provenance manifests from Google Imagen, OpenAI DALL-E, Adobe Firefly
The cleaner parses each layer, removes AI-related fields, and preserves standard metadata (Author, Copyright, Title).
## Installation
### Recommended
Install as an isolated CLI tool — no need to manage virtual environments:
```bash
# Using pipx (https://pipx.pypa.io)
pipx install git+https://github.com/wiltodelta/remove-ai-watermarks.git
# Or using uv (https://docs.astral.sh/uv)
uv tool install git+https://github.com/wiltodelta/remove-ai-watermarks.git
```
To update to the latest version:
```bash
pipx upgrade remove-ai-watermarks
# or
uv tool upgrade remove-ai-watermarks
```
### Install from repository
**Prerequisites:** Python 3.10+ and `pip` (or [`uv`](https://docs.astral.sh/uv/)).
```bash
# 1. Clone the repository
git clone https://github.com/wiltodelta/remove-ai-watermarks.git
cd remove-ai-watermarks
# 2. Install the package in editable mode
pip install -e .
# Or, if you use uv:
uv pip install -e .
```
After installation the `remove-ai-watermarks` command is available system-wide.
> **Note**: The base install covers visible watermark removal and metadata stripping.
> For invisible watermark removal (SynthID etc.), install GPU dependencies:
>
> ```bash
> pip install -e ".[gpu]" # or: uv pip install -e ".[gpu]"
> ```
>
> To let `identify` decode the open Stable Diffusion / SDXL / FLUX invisible
> watermarks, install the `detect` extra (adds the `invisible-watermark` decoder):
>
> ```bash
> pip install -e ".[detect]" # or: uv pip install -e ".[detect]"
> ```
>
> To also decode the open **Adobe TrustMark** watermark (behind Adobe Durable
> Content Credentials), install the `trustmark` extra (pulls torch and downloads
> model weights on first use):
>
> ```bash
> pip install -e ".[trustmark]" # or: uv pip install -e ".[trustmark]"
> ```
>
> For sharper upscaling of small inputs before diffusion (`--upscaler esrgan`,
> Real-ESRGAN), install the `esrgan` extra. It loads via spandrel (MIT, no basicsr);
> the Real-ESRGAN weights (BSD-3-Clause) download on first use:
>
> ```bash
> pip install -e ".[esrgan]" # or: uv pip install -e ".[esrgan]"
> ```
#### Invisible watermark removal
Invisible removal uses diffusion models and a GPU for reasonable speed.
```bash
# On first run, the model (~2 GB) will be downloaded automatically.
# Device is auto-detected: CUDA (Linux/Windows) > MPS (macOS) > CPU.
# To force a device: --device cuda / --device mps / --device cpu
# Optional: set a HuggingFace token for gated/private models
cp .env.example .env
# Edit .env and set HF_TOKEN=hf_your_token_here
```
#### Developer setup
```bash
# Install with dev dependencies (pytest, ruff, pyright)
pip install -e ".[dev]"
# Or with uv:
uv pip install -e ".[dev]"
# Run tests
pytest
# Run linters
./maintain.sh
```
## Usage
### CLI
```bash
# Remove all watermarks from a single image (visible + invisible + metadata)
remove-ai-watermarks all image.png -o clean.png
# Process an entire directory
remove-ai-watermarks batch ./images/ --mode all
```
#### Individual commands
```bash
# Identify provenance: where an image was made + its watermark inventory.
# Aggregates C2PA, IPTC "Made with AI", embedded SD/ComfyUI params, EXIF/XMP
# generator tags (incl. inside AVIF/HEIF), the SynthID proxy, the visible Gemini
# sparkle, and (with the [detect] extra) the open SD/SDXL/FLUX invisible
# watermark into one verdict. Reports "unknown"
# (never "clean") when no signal is found, since stripped metadata is not proof
# of a clean origin. Add --json for machine-readable output.
remove-ai-watermarks identify image.png
# Visible watermark only — fast, offline, CPU. --mark auto (default) finds the
# strongest known mark (Gemini sparkle / Doubao "豆包AI生成" / Jimeng "即梦AI" /
# Samsung Galaxy AI "Contenuti generati dall'AI"); force one with
# --mark gemini / doubao / jimeng / samsung. Removed by reverse-alpha (true-pixel recovery).
remove-ai-watermarks visible image.png -o clean.png
# Erase arbitrary region(s) — universal, any logo/watermark/object, any position.
# Default cv2 inpainting (CPU). --backend lama uses big-LaMa (extra 'lama').
remove-ai-watermarks erase image.png --region 1640,1930,400,100 -o clean.png
# Invisible watermark only (SynthID etc.) — requires GPU
remove-ai-watermarks invisible image.png -o clean.png --humanize 4.0 --unsharp 0.5
# --humanize adds film grain, --unsharp counters the soft "AI" look (both opt-in).
# Large images run at native resolution; small ones are upscaled to a 1024 floor
# first (disable with --min-resolution 0); --upscaler esrgan uses Real-ESRGAN for
# that floor upscale (needs the 'esrgan' extra). On a very large image that OOMs the
# GPU/MPS, cap the long side: --max-resolution 2048
# Strength is vendor-adaptive by default (OpenAI 0.20 / Google 0.30, same
# for both pipelines); override with --strength. controlnet (text/face
# structure preservation) is the default pipeline; --pipeline sdxl opts down
# to plain SDXL for non-structure inputs. Tune structure preservation with
# --controlnet-scale, the CFG with --guidance-scale (default 7.5), and the
# diffusion model with --model (default: SDXL base).
# --adaptive-polish (ON by default) restores the input's detail level (sparing
# text) to counter the over-smoothed look; it self-limits to a no-op where
# there is no detail deficit. Disable with --no-adaptive-polish.
# Check / strip AI metadata (C2PA, EXIF, "Made with AI" labels)
# --check also flags SynthID-bearing sources: a C2PA manifest signed by
# Google or OpenAI implies an invisible SynthID watermark in the pixels
# (both vendors pair the two). Adobe Firefly / Microsoft sign C2PA without
# SynthID, so they are reported as C2PA only.
remove-ai-watermarks metadata image.png --check
remove-ai-watermarks metadata image.png --remove
# Batch with a specific mode
remove-ai-watermarks batch ./images/ --mode visible
# Batch accepts the full invisible knob set (--strength/--guidance-scale/--model/
# --pipeline/...); --adaptive-polish is on by default (--no-adaptive-polish to disable)
remove-ai-watermarks batch ./images/ --mode all
```
### Python API
The visible-removal API requires `remove-ai-watermarks[visible]`.
```python
from remove_ai_watermarks.gemini_engine import GeminiEngine
import cv2
import remove_ai_watermarks as raiw
engine = GeminiEngine()
image = cv2.imread("watermarked.png")
result, removed = raiw.remove_visible("watermarked.png", "clean.png")
print(removed)
# Detect
result = engine.detect_watermark(image)
print(f"Detected: {result.detected} (confidence: {result.confidence:.1%})")
# Remove
clean = engine.remove_watermark(image)
cv2.imwrite("clean.png", clean)
```
#### Invisible removal (diffusion)
```python
from pathlib import Path
from remove_ai_watermarks.invisible_engine import InvisibleEngine
# pipeline: "controlnet" (default, preserves text/face structure) or "sdxl" (plain).
# model_id=None uses the SDXL base; controlnet_conditioning_scale tunes preservation.
engine = InvisibleEngine(pipeline="controlnet")
engine.remove_watermark(
Path("watermarked.png"),
Path("clean.png"),
strength=None, # None = vendor-adaptive default (OpenAI 0.20 / Google 0.30)
num_inference_steps=50,
guidance_scale=None, # None = the library default (7.5)
seed=None, # set for reproducible output
adaptive_polish=True, # detail-targeted polish, self-gating (default on in the CLI)
min_resolution=1024, # upscale tiny inputs to this floor before diffusion
max_resolution=0, # 0 = native; set only to cap GPU/MPS memory
upscaler="lanczos", # or "esrgan" for the floor upscale (needs the 'esrgan' extra)
provenance = raiw.identify_video("input.mp4")
report = raiw.inspect_video_metadata("input.mp4")
complete = raiw.remove_video_all("input.mp4", "clean.mp4")
batch = raiw.remove_video_batch("videos", "videos_clean")
cleaned = raiw.remove_video_metadata("input.mp4")
synthid_cleaned = raiw.remove_video_invisible("input.mp4", "synthid_clean.mp4")
visible = raiw.remove_video_visible("input.mp4", "clean.mp4")
print(visible.mark)
veo = raiw.remove_video_visible("veo.mp4", "veo_clean.mp4", mark="veo")
seedance = raiw.remove_video_visible(
"seedance.mp4",
"seedance_clean.mp4",
mark="seedance",
)
dola = raiw.remove_video_visible("dola.mp4", "dola_clean.mp4", mark="dola")
```
### Metadata stripping
The high level API accepts a file path or a BGR NumPy array. For path inputs it
also reads provenance metadata, preserves alpha, and can strip AI metadata from
the written result.
```python
from remove_ai_watermarks.metadata import has_ai_metadata, remove_ai_metadata
from pathlib import Path
See the [Python API guide](docs/python-api.md) for visible removal, the full
`remove_all` and `remove_batch` pipeline, provenance inspection, metadata
stripping, and diffusion usage.
if has_ai_metadata(Path("image.png")):
remove_ai_metadata(Path("image.png"), Path("clean.png"))
```
## ComfyUI
## Requirements
The separate
[ComfyUI Remove AI Watermarks](https://github.com/wiltodelta/ComfyUI-remove-ai-watermarks)
package provides nodes for visible removal, detection, region erasing, and
invisible removal.
- Python ≥ 3.10
- **Visible removal / metadata**: CPU only, no GPU required
- **Invisible removal**: GPU recommended (CUDA or MPS), works on CPU (slow)
## Important limitations
## Troubleshooting
- A missing local signal means unknown, not clean. Proprietary pixel
watermarks may remain after metadata has been stripped.
- Visible removal reconstructs a small region. Results depend on the background
and selected fill backend.
- Invisible removal changes the whole image and may alter faces, text, or fine
detail.
- Visible video removal recognizes the moving Sora 2 wordmark, the current Veo
diamond plus legacy `Veo` text, the Seedance boxed `AI` label, and the fixed
Dola, Hailuo, and Kling labels. It does not recognize the older Sora Turbo
corner swirl or unregistered layouts from those providers.
The classical OpenCV backend can smear structured backgrounds; use MI-GAN or
LaMa when recovery quality matters.
- Video SynthID regeneration changes resolution, frame rate, and image detail.
The shipped profile is oracle-certified, but no public local decoder can
certify an arbitrary output at runtime. Recheck unusually important outputs
after provider changes.
- Invisible-watermark removal requires CUDA. Both profiles refuse any other
device at construction rather than falling back to one that cannot run them.
Visible removal, metadata stripping and `identify` still run anywhere.
- Provider watermark systems can change. Validate important outputs with the
provider's own verifier when one is available.
**SSL certificate error** (`CERTIFICATE_VERIFY_FAILED`):
The shipped `video invisible` command uses the certified `noise_std=0.15`
profile. The companion `scripts/video_synthid_sweep.py` research harness builds
a matched re-encode control plus VAE-regenerated candidates and leaves the
verifier verdict blank:
```bash
# Install certifi (the tool auto-detects it)
pip install certifi
# macOS only: run the Python certificate installer
/Applications/Python\ 3.*/Install\ Certificates.command
uv run --extra video --extra diffusion python scripts/video_synthid_sweep.py input.mp4 -o sweep/
```
**First run is slow** — this is expected. The tool downloads model weights (~2 GB) on first launch. Subsequent runs use cached models.
To score what an output actually cost, use `scripts/video_fidelity_probe.py`:
the engine's own PSNR is measured before the resize and the encode, so only the
probe sees the delivered picture.
## Credits
```bash
uv run --extra video python scripts/video_fidelity_probe.py input.mp4 input_clean.mp4
```
- [noai-watermark](https://github.com/mertizci/noai-watermark) by mertizci — invisible watermark removal engine
- [GeminiWatermarkTool](https://github.com/allenk/GeminiWatermarkTool) by Allen Kuo (MIT) — visible watermark removal algorithm
- [controlnet-canny-sdxl-1.0](https://huggingface.co/xinsir/controlnet-canny-sdxl-1.0) by xinsir — SDXL canny ControlNet used by the `controlnet` pipeline to preserve text/face structure
- NeuralBleach (MIT) — analog humanizer technique
The control must still be SynthID-positive before a negative candidate can
count as removal evidence. In the 2026-07-29 two-clip calibration, both matched
controls were positive in Gemini's built-in SynthID verifier; the stronger
candidate was negative on both carriers, while a weaker candidate was
negative on one. A later adversarial follow-up that asked ordinary Gemini to
reinterpret the pixel result returned `UNAVAILABLE`; that follow-up was not a
verifier rerun and does not invalidate the built-in verdicts. A 2026-07-31
full-clip check on a public eight-second Veo sample found `0.10` still detected
and `0.15` not detected, so `0.15` is now the certified default. The
reproducible hashes and verdicts live in
`data/evaluations/video-synthid-oracle.csv`.
## Roadmap
## Documentation
Tracked but not yet implemented:
Start with the [documentation index](docs/index.md).
- **SynthID-Image v2 automated regression test**. The default SDXL profile defeats v2 per manual checks against the [Gemini app](https://support.google.com/gemini/answer/16722517)'s "Verify with SynthID" feature on a Gemini 3 Pro output (May 2026). An automated end-to-end test would need either programmatic access to the [SynthID Detector portal](https://blog.google/innovation-and-ai/products/google-synthid-ai-content-detector/) (waitlist for media professionals and researchers) or an offline surrogate detector. The spectral phase-coherence surrogate from [reverse-SynthID](https://github.com/aloshdenny/reverse-SynthID) was evaluated and does not separate watermarked from cleaned real-content images (it only fires on controlled solid-color references at exact resolution), so it is not a usable oracle. Open.
- **Local SynthID *pixel* detector**. Not feasible today: Google's decoder is proprietary, and magnitude/carrier spectral methods do not separate real content (confirmed by three independent evaluations, including a from-scratch gpt-image pilot; see CLAUDE.md). Blocked on either (a) a programmatic generation path (OpenAI / Gemini API) to build a per-(model, resolution) labeled corpus at scale, or (b) a raw watermarked-output dataset. If data arrives, the next approach to try is a learned classifier on diverse content rather than a fixed carrier codebook.
- **Grow the SynthID reference corpus** (`data/synthid_corpus/`) with oracle-labeled samples per model and resolution (Gemini app for Google, openai.com/verify for OpenAI). Prerequisite for any pixel-detector attempt and for an automated removal-regression set.
- **Real non-PNG C2PA fixtures**. SynthID-source detection for JPEG / WebP / AVIF is currently covered only by synthetic byte blobs; replace with real vendor-emitted files to ground the binary-scan path.
- **Maintenance debt**. Strict pyright is now clean across `src/` (0 errors): pure-logic files are fully typed, the cv2 / torch / diffusers boundary files carry a documented per-file relax pragma, and a local `typings/piexif` stub covers piexif. Remaining: full-project `pyright` (no path) still OOMs node on this ML-heavy repo, so it must be scoped to `src/`; narrowing the boundary pragmas back toward full strict (as upstream stubs improve) is the long tail. (`uv-secure` is already clean since `idna` was bumped to 3.16.)
- **AVIF / HEIF `Exif` item inside the `meta` box**. An AI-label *XMP* packet in a `meta`-box item is now blanked in place (v0.6.9), but EXIF stored as a `meta`-box `Exif` *item* is still not removed — it needs full `iinf`/`iloc` surgery (offset rewrite, corruption risk) or `exiftool` (a non-bundled binary dependency). Low priority: the AI labels we target are XMP, not EXIF, so an EXIF-only meta-box case is rare.
- **More C2PA device signers**. Leica, Nikon, Google Pixel, Sony, and Truepic capture cameras are mapped (each verified against a real signed file); **Samsung Galaxy AI**, **Black Forest Labs (FLUX)**, and **ByteDance Volcano Engine** (Doubao / Jimeng) are now attributed too (verified on real signed files). Canon is still deferred until a real signed sample surfaces — no public direct-download C2PA file exists for it today (upload-to-verify / news-agency-licensed only).
- **Resemble PerTh audio detection** — evaluated, not feasible with the public API: `get_watermark()` returns a raw bit array with no presence/confidence flag, so watermarked vs. clean audio can't be reliably separated without Resemble's fixed payload or a confidence service. Same wall as the SynthID pixel detector.
- **Video pipeline (`noai-video`)**: per-frame inpainting and tracking for Sora 2 dynamic logo, Veo 3.1 badge, Kling, Runway. Separate package, not folded into this repo.
- [Installation](docs/installation.md)
- [CLI guide](docs/cli.md)
- [Python API](docs/python-api.md)
- [Supported signals](docs/supported-signals.md)
- [Known limitations](docs/known-limitations.md)
- [Scope, safety, and legal notes](docs/legal-and-safety.md)
- [Module internals](docs/module-internals.md)
- [Release and distribution](docs/release-and-distribution.md)
Won't fix:
Research notes and historical experiments are listed separately in the
[documentation index](docs/index.md). They explain past decisions but do not
define the current public API.
- **Nightshade / Glaze / PhotoGuard removal**. These are defensive perturbations used by artists to protect their work from being scraped into AI training sets. Removing them attacks artists, not AI provenance. Out of scope.
## Contributing
## Legal
Install the development environment and run the project gate:
Watermarking and provenance for AI-generated content is now regulated in several jurisdictions. The table below summarises the May 2026 status. None of this is legal advice.
```bash
uv sync --frozen --extra dev
bash maintain.sh
```
| Jurisdiction | Instrument | Status (May 2026) | Relevance |
| --- | --- | --- | --- |
| EU | AI Act, Article 50 | Transparency duties apply from **2 August 2026**. Legacy generative systems (placed on the market before that date) get a grandfathering period to **2 December 2026** for the Article 50(2) marking duty, under the Digital Omnibus (Commission proposal Nov 2025; co-legislator political agreement 7 May 2026). Article 50 guidelines and a marking Code of Practice are being finalised through 2026. | Removing mandated provenance markers with intent to deceive may be sanctioned under national implementations. |
| US (federal) | COPIED Act (S. 1396, 119th Cong.) | **Reintroduced April 2025; not enacted** (referred to Senate Commerce Committee). | If passed, would set NIST provenance standards and prohibit tampering with / removing provenance information. The tool itself is lawful; usage may not be. |
| US (state) | CA AB 2655, TX SB 751 (2019), similar | TX SB 751 (2019) in force; **CA AB 2655 struck down** by a federal court (E.D. Cal., Aug 2025, *Kohls v. Bonta*) as preempted by **Section 230**; the court did not reach the First Amendment (the companion law AB 2839 was separately enjoined on First Amendment grounds). | Content-specific (election deepfakes, sexual deepfakes). Not tool-specific. |
| US (state) | CA AB 853 (amends the California AI Transparency Act) | Core provider duties operative **2 August 2026** (delayed from 1 January 2026); large platforms 1 January 2027; capture devices 1 January 2028. | Covered providers (1M+ monthly users) must embed a latent disclosure that is "permanent or extraordinarily difficult to remove" and offer a free detection tool. Removing that disclosure is what this tool does. |
| South Korea | AI Framework Act (Basic Act on AI), Article 31 | In force since **22 January 2026** (one-year transition after promulgation). | Art. 31(3): AI output "difficult to distinguish from reality" must be labeled so users "clearly recognize" it; the draft Enforcement Decree accepts a machine-readable (invisible-watermark) label. Artistic/creative works get a presentation exception. |
| China | Measures for Labeling AI-Generated Content (+ GB 45438-2025) | In force since **1 September 2025**. | Mandatory explicit (visible) + implicit (metadata) labels across image / audio / video; tampering with, forging, or removing labels is prohibited. |
| India | IT (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules, 2026 | In force since **20 February 2026** (notified 10 February 2026). | All "synthetically generated information" must be **prominently labelled** and carry **permanent metadata / a provenance identifier**; the rules expressly **prohibit modifying, suppressing, or removing** that label or metadata. Covers image, audio, and audio-visual content. |
| UK | Online Safety Act 2023 / Ofcom guidance | In force, but **no statutory AI-provenance or watermarking obligation**. | Ofcom encourages watermarking / provenance metadata as voluntary "attribution measures"; platform duties, not user obligations. |
## Threat model
This tool defends already-distributed AI imagery against automatic detection systems (social-platform "Made with AI" labels, third-party classifiers, content-policy filters). It does **not** retroactively anonymise generation.
In particular, **SynthID** (Google DeepMind) is embedded across Google's generative media stack — Imagen (images), Veo (video), Lyria (audio) — and Gemini app image outputs (Nano Banana / Gemini 3 Pro, which we verified positive via the Gemini app's SynthID oracle); Google reported over 10 billion items watermarked by December 2025. It carries a **multi-bit payload** — the research paper's SynthID-O variant encodes 136-bit payloads in 512x512 images ([arxiv 2510.09263](https://arxiv.org/abs/2510.09263)). The payload is believed to encode a user / session identifier. If the original watermarked file ever passed through a system controlled by the prompt originator (a saved Gemini account history, a screenshot uploaded to a Google product, a backup), Google retains the ability to link that original to the generating account. Stripping the watermark from a copy you possess does not erase Google's server-side record.
Use cases where the threat model fits:
- You generated the image yourself, want to publish it as your own work, and accept the consequences if Google ever publishes their detector logs.
- You are running a security / robustness evaluation.
- You are preserving art or historical record against false-positive "AI-generated" labels.
Use cases where the threat model **does not** fit:
- Generating an image, expecting that removing the watermark anonymises you to Google. It doesn't.
- Distributing AI-generated content while claiming human authorship. The watermark is one of several traceability layers.
This tool is intended for legitimate purposes such as:
- Privacy protection (removing metadata that leaks user account identifiers).
- Art preservation and fair-use research.
- Removing false-positive "Made with AI" labels from human-edited photographs.
- Security research and watermark robustness study.
**Who bears the liability.** This is general-purpose software and is itself lawful to publish and run; legal responsibility attaches to the person who removes a marker and to how the result is then used, and the hinge is intent. Removing AI provenance to pass AI-generated content off as human-made, to commit fraud, to produce non-consensual deepfakes, or to conceal copyright infringement can expose the remover to liability. Two kinds of exposure are worth knowing:
- **The downstream act.** Deception, fraud, defamation, IP infringement, or breaking a platform's terms — judged by intent and harm, not by the act of editing metadata itself. In the US, the **DMCA (17 U.S.C. § 1202)** specifically bars removing "copyright management information" *with intent to conceal or enable infringement*.
- **The removal itself.** Some jurisdictions penalise tampering with the label/metadata as such, regardless of downstream use — notably **China** (Labeling Measures) and **India** (IT Amendment Rules 2026), which expressly prohibit removing or suppressing the AI label and provenance metadata. The US **COPIED Act** would do the same if enacted.
Legitimate uses — publishing your own work, privacy (stripping metadata that leaks an account identifier), security / robustness research, or removing a false-positive "Made with AI" label from a human-edited photograph — are generally lawful. Users are solely responsible for ensuring their use complies with all applicable laws. The authors do not condone use of this tool for deception, fraud, or any activity that violates applicable laws or regulations. None of this is legal advice.
See [module internals](docs/module-internals.md) before changing a subsystem
with documented invariants.
## License
+80
View File
@@ -0,0 +1,80 @@
# Repository data
Tracked data is organized by purpose:
```text
data/
fixtures/
provenance/ Real format and provenance fixtures used by tests
(source records live in fixtures/README.md)
calibration/
<vendor>/ Minimal controlled inputs needed to rebuild detector assets
synthid/
originals/ Canonical provider-oracle fixtures, stored once
manifest.csv Provenance and verification record for each original
full-pipeline-quality.csv
Reusable full-pipeline evaluation selection
evaluations/
fidelity/ Evaluation instructions and hand-verified ground truth
video-synthid-oracle.csv
Reproducible full-clip Gemini SynthID verdicts
```
## Storage rules
1. Store each binary image once. Evaluation manifests and documentation point
to its canonical location.
2. Put executable test fixtures in `fixtures/`.
3. Put only the minimal reproducible detector inputs in `calibration/`.
4. Put externally verified SynthID originals in `synthid/originals/` and keep
both CSV files synchronized.
5. Keep evaluation outputs outside the repository. Record reproducible
commands, hashes, and oracle verdicts instead of committing another corpus
copy. A small curated before-and-after example may live in `docs/images/`
when it is part of the public documentation.
6. Keep third-party fixture license notices in `licenses/`, outside directories
that tests enumerate as media inputs.
7. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
Unregistered research candidates belong in
`scripts/assets/visible-mark-candidates/` so they are not shipped in the
wheel.
The source distribution excludes `data/`; the wheel contains only package
runtime assets.
## Video SynthID oracle manifest
`evaluations/video-synthid-oracle.csv` is the only evidence that the shipped
video removal profile works, so it is also the source of truth for three shipped
defaults: `tests/test_video_invisible.py` asserts that `noise_std`, `long_side`,
and `fps` together match a row this manifest records as certified. Changing one
of those three without adding the row that certifies it fails the suite. `vae` is
deliberately outside that check because neither tracked row records one; add it
to the assertion in the same commit as the first row that does.
| Column | Meaning |
| --- | --- |
| `date`, `source_url`, `source_sha256` | Identify the carrier. |
| `source_width`, `source_height`, `source_fps` | Carrier geometry. Without it the actual downscale factor of a row cannot be recovered later. |
| `duration_seconds`, `source_verdict` | Clip length submitted and the verifier's reading of the untouched carrier. |
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The full run configuration. |
| `output_sha256` | Identifies the exact submitted file. |
| `output_verdict` | One of `detected`, `not_detected`, `indeterminate`, `refused`. |
| `output_verdict_text` | The verifier's wording, verbatim. |
| `output_detected_range` | Time range the verifier reported for the output. |
| `track` | `visual`, `audio`, `both`, or empty. The verifier scores tracks separately and this path copies source audio unchanged. |
| `session_id` | Groups rows submitted in one oracle session, so per-session drift stays visible. |
| `stratum` | Content class of the carrier, for stratified certification. |
| `psnr_db`, `temporal_residual_ratio` | Fidelity measurements. Neither is a watermark verdict. |
Record `indeterminate` when the verifier answers with its unclear state rather
than a negative: an unclear reading logged as `not_detected` is exactly the
silent regression this manifest exists to prevent. Leave a field empty when it
was not recorded, and never backfill it with a plausible value.
`psnr_db` is measured against the already-resized frame and before the encode,
so it excludes the downscale, the decimation, and the codec. Rows stay
comparable to each other only while that definition holds.
The two 2026-07-31 rows predate this schema; their empty fields were never
recorded and are not recoverable from the row.
@@ -1,17 +1,17 @@
# Doubao visible watermark capture
> **Status (captured 2026-05-29; alpha rebuilt 2026-05-31):** the black/gray/white captures were
> taken and are now **committed** in `captures/` (solid colour + watermark, content-free). The alpha
> map is rebuilt by `scripts/visible_alpha_solve.py doubao` (the careful gray-self solve shared with
> Jimeng). The first build claimed "pixel-exact" but left a readable outline on the real sample (issue
> #13 follow-up); removal now reverse-alphas, NCC-aligns, and applies a thin residual inpaint. See the
> `doubao_engine.py` notes in the root `CLAUDE.md`. The text below is kept as the historical capture plan.
> **Status (captured 2026-05-29; asset rebuilt 2026-05-31):** the
> black and gray captures are committed here. The generated alpha
> asset is now a detection silhouette and mask-shaping input only. Current
> removal localizes the mark and sends its footprint to the shared fill backend.
> Reverse-alpha recovery was retired. The details below preserve the historical
> capture rationale.
Goal: capture the Doubao "豆包AI生成" visible watermark over known flat backgrounds so we can
build a per-pixel alpha map and a reverse-alpha-blend remover, the same way the Gemini sparkle
engine works (`src/remove_ai_watermarks/gemini_engine.py`).
rebuild and validate the detection silhouette used by
`src/remove_ai_watermarks/doubao_engine.py`.
## What we already know (verified from prior art, 2026-05-26)
## Historical findings from the capture work
- Blend model: **alpha compositing with a white logo** `watermarked = a*logo + (1-a)*original`,
`logo = (255,255,255)`. Inversion: `original = (watermarked - a*logo) / (1-a)`.
@@ -19,14 +19,13 @@ engine works (`src/remove_ai_watermarks/gemini_engine.py`).
both say "alpha map"). One commercial blog (pixelcleanai) claims "screen blend" instead; the gray
capture below settles it empirically.
- Position: **bottom-right corner**, small margins (right ~8-20px, bottom ~5px), scales with image size.
Confirmed by our sample `data/samples/doubao-1.png` (2048x2048) plus three sources.
Confirmed by our sample `data/fixtures/provenance/doubao-1.png` (2048x2048) plus three sources.
- Size **scales with resolution**. Third-party numbers (~90x18 at <=1024, ~180x40 at >1024) are
approximate and calibrated for ~1024-1280 outputs; at 2048 the strip is much larger. A shipped
third-party alpha map is only 120x20, too small for our 2K/4K target -> capture fresh.
- The planning assumption was that clean inversion leaves residue on textured backgrounds, so the
remover would pair the alpha map with inpainting. After the capture this turned out unnecessary at
the native width (recovery is pixel-exact there and inpaint is off); the shipped remover is
reverse-alpha only, with a residual inpaint applied off-native only.
- The original implementation used reverse-alpha recovery. That path was later
removed in favor of localize then fill because the latter also handles moved
and re-rendered marks.
## Use doubao.com specifically
@@ -35,7 +34,7 @@ the captured template matches our target.
## How to capture (image-edit path, most reliable)
For each seed in `seeds/`:
For each locally generated solid-color seed:
1. Open Doubao image generation, use the image-edit / reference mode, upload the seed.
2. Prompt (Chinese preferred):
@@ -70,16 +69,15 @@ extracted by per-pixel min/median.
- Original download, never a screenshot. PNG preferred; if Doubao only gives JPEG, note it.
- No crop / edit / re-save. Default settings, watermark left ON.
## Naming, drop into `captures/`
## Committed inputs
```
doubao_black_1x1_1.png
doubao_white_1x1_1.png
doubao_gray128_1x1_1.png
doubao_black_16x9_1.png
doubao_content_1x1_1.png
doubao_gray_1x1_1.png
```
The black and gray captures feed `scripts/visible_alpha_solve.py doubao`.
## Also report back
1. Which resolutions and aspect ratios the Doubao UI actually offers.
@@ -1,8 +1,10 @@
# Gemini (Nano Banana) visible sparkle capture
> **Status (captured 2026-05-31):** black/gray/white captures taken and **committed**
> in `captures/` (solid colour + the sparkle, content-free). The sparkle-on-black assets
> **Status (captured 2026-05-31):** the required black capture is committed
> here. The sparkle-on-black assets
> `gemini_bg_{96,48}.png` are rebuilt by `scripts/visible_alpha_solve.py gemini`.
> They support detection and footprint geometry. Current pixel recovery uses the
> shared fill backend, not reverse-alpha inversion.
Google Gemini (Nano Banana) stamps a four-point sparkle icon in the bottom-right corner
via alpha compositing: `watermarked = a*logo + (1-a)*original`. Unlike the Doubao/Jimeng
@@ -20,7 +22,7 @@ text marks, the sparkle is captured over a **pure-black** background, where
## How to capture (image-edit path)
For each solid-colour seed (`seeds/seed_{black,gray,white}_2048.png`, gitignored):
For each locally generated black, gray, or white 2048 px seed:
1. Open Gemini image generation, image-edit / reference mode, upload the seed.
2. Prompt: `Recreate this image exactly as it is, keep it identical, do not add or change anything`
@@ -28,16 +30,15 @@ For each solid-colour seed (`seeds/seed_{black,gray,white}_2048.png`, gitignored
Black is the key one (sparkle on black -> exact alpha). Gray/white cross-check.
## Naming, drop into `captures/`
## Committed input
```
gemini_black_2048.png # the key capture (sparkle on black)
gemini_gray_2048.png
gemini_white_2048.png
```
The solid captures are **committed** (content-free). The synthetic `seeds/` and any
real-content `gemini_content_*.png` validation download are gitignored (local-only).
The gray and white captures were calibration cross-checks and are not needed to
rebuild the assets. Synthetic seeds and real-content validation downloads stay
local.
Rebuild the assets with:
```
@@ -1,15 +1,13 @@
# Jimeng (即梦AI) visible watermark capture
> **Status (completed 2026-05-30):** solid black/gray/white Jimeng captures were
> obtained (issue #13, from @powersee) and the alpha map was solved. Removal is
> reverse-alpha plus a residual inpaint over the glyph footprint; see the
> `jimeng_engine.py` notes in the root `CLAUDE.md`. The text below is kept as the
> capture plan.
> **Status (completed 2026-05-30):** solid black and gray Jimeng captures were
> obtained and the detection asset was solved. The asset now supports detection
> and mask geometry only. Current removal localizes the wordmark and uses the
> shared fill backend. The text below records the capture and calibration work.
Goal: capture the Jimeng / Dreamina "★ 即梦AI" visible wordmark over known flat
backgrounds so we can build a per-pixel alpha map and a reverse-alpha remover, the
same way the Gemini sparkle and Doubao strip engines work
(`src/remove_ai_watermarks/gemini_engine.py`, `doubao_engine.py`).
backgrounds so we can rebuild and validate the silhouette used by
`src/remove_ai_watermarks/jimeng_engine.py`.
## What we learned (verified from the captures, 2026-05-30)
@@ -32,11 +30,10 @@ same way the Gemini sparkle and Doubao strip engines work
width; a real 1440-wide download matched width_frac ~0.21.
- **Per-image render variation:** the alpha maps solved independently from the
black and the gray capture correlate 0.998 but not 1.0 (mean |Δa| ~0.02). Jimeng
re-rasterizes the mark per generation AND jitters its position a few px, so a
single alpha map does NOT pixel-cancel the mark the way Doubao's deterministic
overlay does. Removal therefore: NCC-aligns the alpha to the actual mark (always,
not only off-native), reverse-alphas, then clears the residual with a THIN inpaint
over the glyph footprint (a wide full-footprint pass smeared the texture/edges).
re-rasterizes the mark per generation and jitters its position a few pixels.
This was one reason the old reverse-alpha path needed alignment and residual
inpainting. The current path instead detects the silhouette, localizes the
glyph footprint, and fills that footprint.
## How to capture (image-edit path, most reliable)
@@ -56,19 +53,15 @@ pure white (the mark is nearly invisible on white, as expected).
- Original download, never a screenshot. PNG preferred; if Jimeng only gives JPEG, note it.
- No crop / edit / re-save. Default settings, watermark left ON.
## Naming, drop into `captures/`
## Committed inputs
```
jimeng_cap_A.png # black seed run through Jimeng
jimeng_cap_B.png # white seed
jimeng_cap_C.png # gray seed
jimeng_content_1.png # a normal-content download, for end-to-end validation
```
The solid `jimeng_cap_{A,B,C}.png` captures are **committed** (content-free: a solid
colour + the watermark; the source for `scripts/visible_alpha_solve.py jimeng`). The
synthetic `seeds/` and the real-content `jimeng_content_*.png` validation download are
gitignored (local-only). Rebuild the alpha asset with:
The A and C captures feed
`scripts/visible_alpha_solve.py jimeng`. Rebuild the detection asset with:
```
uv run python scripts/visible_alpha_solve.py jimeng # or: all
@@ -1,15 +1,14 @@
# Samsung Galaxy AI visible watermark capture
> **Status (built 2026-06-05):** flat black/gray/white Samsung Galaxy AI captures
> were obtained (issue #37, from @f-liva) and the alpha map was solved. Removal is
> reverse-alpha plus a thin residual inpaint over the glyph footprint; see the
> `samsung_engine.py` notes in the root `CLAUDE.md`. The text below is the capture
> plan and the open quality follow-up.
> **Status (built 2026-06-05):** flat black and gray Samsung Galaxy AI captures
> were obtained and the detection asset was solved. The asset now supports
> detection and mask geometry only. Current removal localizes the wordmark and
> uses the shared fill backend. The text below records the capture plan and its
> locale and resolution limits.
Goal: capture the Samsung Galaxy AI "✦ Contenuti generati dall'AI" visible wordmark
over known flat backgrounds so we can build a per-pixel alpha map and a reverse-alpha
remover, the same way the Gemini sparkle and the Doubao / Jimeng strips work
(`src/remove_ai_watermarks/gemini_engine.py`, `doubao_engine.py`, `jimeng_engine.py`).
over known flat backgrounds so we can rebuild and validate the silhouette used
by `src/remove_ai_watermarks/samsung_engine.py`.
## What we learned (verified from the captures, 2026-06-05)
@@ -51,15 +50,11 @@ On a Samsung Galaxy AI device (set the UI language to the target locale):
## Files
- `captures/samsung_black_1.png`, `samsung_gray_1.png`, `samsung_white_1.png` --
portrait flat edits (1086 wide), the primary calibration set.
- `captures/samsung_black_2.png`, `samsung_gray_2.png`, `samsung_white_2.png` --
a second (landscape 1920) set.
- `captures/samsung_content_*` -- real-photo validation downloads, **gitignored**
(user content, repo is public).
- `seeds/` -- synthetic solid-color inputs, gitignored (regenerable).
- `samsung_black_1.png` and `samsung_gray_1.png` are the retained solver inputs.
- White, secondary-resolution, and real-content captures were validation
material and are not required to rebuild the asset.
Rebuild the alpha asset with:
Rebuild the detection asset with:
```
uv run python scripts/visible_alpha_solve.py samsung
+89
View File
@@ -0,0 +1,89 @@
# Content Seal oracle corpus
Muse Image (`muse-image-1.0`) generations with externally verified Content Seal
verdicts, produced through the Meta Model API on 2026-08-26 and checked against
the public detector at `https://meta.ai/identification` (anonymous session, no
login). Follows the `data/synthid/originals` pattern: binaries live in
`originals/`, every derived variant is recorded in `manifest.csv` as a recipe
plus hash and is not stored.
## What this corpus establishes
- The Meta Model API image endpoint (`POST /v1/images/generations`) stamps the
same Content Seal pixel watermark as the consumer Meta AI app: all five
generations verified positive with attribution "Muse Image 1 - Meta".
- The detector response carries a per-generation ID and creation timestamp
embedded in the watermark payload. Both survived a 512 px LANCZOS resize and
a full-size JPEG q85 re-encode (same ID returned), so the payload is more
robust than the detection threshold.
- Center crops lose the seal: 50% and 33% linear center crops of two different
images all returned "No AI signatures from Meta were found", consistent with
the Reuters 2026-07-11 analysis (55% missed after cropping).
- API outputs carry XMP `iptcExt:DigitalSourceType =
trainedAlgorithmicMedia`, so local `identify` flags them via the existing
Made-with-AI path. Metadata-stripping transforms fall back to unknown, and
Content Seal has no local decoder in this project: the oracle is the only
reader.
- Drift finding: the 512 px resize of `gen_fox_forest` triggers a
medium-confidence false positive "Tencent Yuanbao (visible 元宝 / AI生成
mark)" in this project's `identify`. Recorded here as a reproducible case.
## Oracle limits and wire format
There is no public or documented checking API. Verified against the official
developer documentation on 2026-08-26 (`https://dev.meta.ai/docs/`): the full
Meta Model API reference lists only Responses, Chat Completions, Messages,
Files, Images (`/v1/images/generations`, `/v1/images/edits`), and Models, with
no identification, detection, or watermark endpoint, and the image-generation,
Muse Image cookbook, and pricing pages never mention watermark, Content Seal,
or provenance at all. The API applies the seal (every generation in this corpus
carries it) while documenting nothing about it. The web tool drives an internal
REST pair, captured from the browser network log on 2026-08-26:
1. `POST https://rupload.meta.ai/gen_ai_document_gen_ai_tenant/<uuid>` with the
raw file bytes, `x-entity-type`, `x-entity-length`, `ai_detector_upload: true`,
and an anonymous `authorization: OAuth ecto1:<token>` session token minted by
the page.
2. `POST https://meta.ai/api/ai-detector` with `Bearer ecto1:<token>` and body
`{"media_id": "...", "fileName": "...", "mimeType": "..."}`.
The rate limit is enforced at that endpoint, server-side, and keyed beyond the
browser session: the API itself returns `429 {"errorType": "rate_limited"}`,
and clearing cookies and storage changed nothing, so driving the internal pair
directly does not bypass it. The Meta Model API (`api.meta.ai/v1`, where the
generation key works) has no identification endpoint; plausible paths all
return 404. Rows with an empty `oracle_verdict` were transformed but not yet
checkable. Read a verdict only from the settled page text after the
result-complete state ("Upload another file"): a wait for a verdict string can
match the previous upload's text, and the fresh-navigation protocol used for
the calibration rows below is the race-free variant.
## Strength floor calibration (qwen-zimage, seed 0)
The library resolves strength per vendor with measured floors (OpenAI
0.07675 / Google 0.27 / Microsoft InvisMark 0.15 in
`_internal/watermark_profiles.py`). Meta Content Seal has no floor yet; the
goal of these rows is to measure one by that same methodology: independent
generations, each one's first-clean boundary, floor = worst boundary plus the
observed cross-source spread.
Measured (2026-08-26/27, oracle `meta.ai/identification`):
- Default pipeline clears Content Seal: tested samples came back clean at the
default resolution-adaptive strength (~0.1305 at 2.56 MP), including the
worst source.
- Five independent generations bracketed. First-clean boundaries:
lighthouse (0.0525, 0.06], fox (0.03, 0.0375], night_city (0.03, 0.0375],
mug <= 0.03, text <= 0.015. Cross-source spread is wide (a factor of four
between easiest and hardest).
- Derived Meta floor by the existing worst-boundary-plus-cross-source-spread
method: 0.06 + (0.0525 - 0.015) = 0.0975, rounded to **0.1**.
- Shipped as `QWEN_ZIMAGE_META_STRENGTH`: auto mode routes standalone-AI-IPTC
files onto the cohort, and `--vendor meta` / `InvisibleOptions.vendor`
names it explicitly on stripped files (implying the scrub runs).
## Regeneration
API key is not stored in this repository. Regenerate with the script pattern
from the session (env `MUSE_API_KEY`, endpoint
`https://api.meta.ai/v1/images/generations`, model `muse-image-1.0`); prompts
are recorded per file in `manifest.csv`.
+44
View File
@@ -0,0 +1,44 @@
name,file,origin,source_url,prompt,requested_size,delivered_size,sha256,xmp_iptc_trainedAlgorithmicMedia,oracle_verdict,oracle_attribution,oracle_generation_id,oracle_created_utc,checked_at_utc,notes
gen_fox_forest,originals/gen_fox_forest.webp,meta-model-api,,A red fox resting on a mossy log in a misty forest at dawn photorealistic shallow depth of field,1024x1024,1600x1600,6d7cecf322a5ac4b2547e2f32c767b3cb3e8a0e49f274cfd00aa31be6697eed4,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:11Z,checked twice; ID stable across uploads
gen_text_poster,originals/gen_text_poster.webp,meta-model-api,,A cheerful flat-design poster that says 'WATERMARK FIELD DAY 2026' in bold rounded letters over a coral and cream striped background,1024x1024,1600x1600,873927662d8198e9c38b4ae71a2919ba5d417dfcd714649db1b0ae231e8cbf9b,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbJuIKXPAYxsd,2026-08-26T19:08Z,2026-08-26T19:08Z,
gen_lighthouse_watercolor,originals/gen_lighthouse_watercolor.webp,meta-model-api,,A watercolor illustration of a lighthouse on a cliff at golden hour seagulls in the distance soft loose washes white paper texture,1024x1024,1600x1600,69bee7c376608568361085602c5c7406900885849bd94c5279c3924cf793b27c,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbB5uWbAmWmwp,2026-08-26T19:08Z,2026-08-26T19:08Z,
gen_night_city,originals/gen_night_city.webp,meta-model-api,,A night city skyline in the rain neon reflections on wet asphalt cinematic wide shot moody,1536x1024,1920x1280,345adf6b9f777b15ef7ce7bbdfb3b355a1d9e882740809e966e9f6caafd94d48,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbH5uWbAmWmwp,2026-08-26T19:08Z,2026-08-26T19:08Z,
gen_studio_mug,originals/gen_studio_mug.webp,meta-model-api,,A minimalist studio photo of a single matte ceramic mug on a pastel blue background soft shadow clean commercial look,1024x1024,1600x1600,2b7bc7fd56fa2250c66ef74f8b3e49d9666fe147d894d5baf9461b5e36077a07,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbAsSozV2XCzw,2026-08-26T19:08Z,2026-08-26T19:08Z,
fox_crop50,,derived:gen_fox_forest,,center crop 0.5 linear each side then WEBP q95,,800x800,80b91995560a7c4b08596a26b3401a29afb82691fad193502697a49785eab7be,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost; PIL re-encode strips XMP
fox_crop33,,derived:gen_fox_forest,,center crop 0.33 linear each side then WEBP q95,,528x528,3b3aded8f4fcb98d0599431abccb73b9581854ae34c6bc869724cd5c86a1e35e,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost
fox_res512,,derived:gen_fox_forest,,resize long side 512 LANCZOS then WEBP q95,,512x512,19b85f615d56f5e9f55cd97bae0ee343f4f432e6bfa0a6fc7acb07dec2cc1d81,no_stripped_by_reencode,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:13Z,generation ID and creation time identical to the original generation: watermark payload survives the resize
fox_jpeg85,,derived:gen_fox_forest,,full-size JPEG q85 re-encode,,1600x1600,3bc7dc11bf3c16c83814df1e6485cdb6e49e21141511dda56d210cc920a8c5e3,no_stripped_by_reencode,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:13Z,payload survives JPEG recompression; local identify returns unknown (metadata stripped)
text_crop50,,derived:gen_text_poster,,center crop 0.5 linear each side then WEBP q95,,800x800,3f0d4075012ee4fec820209c5c1c32bb739998813325f49b1124f379162f9837,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost
text_crop33,,derived:gen_text_poster,,center crop 0.33 linear each side then WEBP q95,,528x528,95cef87002961831b21f6f47cd9a0824a67e2ee6f8503356fd5db6a3fcfcc998,no_stripped_by_reencode,,,,,,daily oracle limit reached before this row could be checked
text_res512,,derived:gen_text_poster,,resize long side 512 LANCZOS then WEBP q95,,512x512,b319727d02e908f22e59f148050e6b4686cbd1a622bd69db51fe4072a6f66b34,no_stripped_by_reencode,,,,,,daily oracle limit reached; also the fox twin of this transform triggered the Yuanbao false positive below
text_jpeg85,,derived:gen_text_poster,,full-size JPEG q85 re-encode,,1600x1600,76f00fcde1e974549afa5d90e1879c705e3c3631ef1f35cab901fc912cd15fa4,no_stripped_by_reencode,,,,,,daily oracle limit reached before this row could be checked
blog_qr_manhwa,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1807972953501959&version=1783318590&transcode_extension=webp,,,1920x1280,8ee7700dff59f21d7de393a619664a0b9360f148d18df289364ecea939b3dea9,yes,detected,Muse Image 1 - Meta,,,2026-08-26T18:59Z,not stored; re-downloadable from source_url
blog_fractal_poster,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1499727434782014&version=1783727678&transcode_extension=webp,,,2096x992,2a96168db026444b47ac37807fd99a88c8b7fa6866e1c93a54ab427fbe581f76,no,detected,Muse Image 1 - Meta,,,2026-08-26T18:59Z,seal survived CDN transcode without any metadata
blog_flipbook_punch,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=2521328538303639&version=1783396590&transcode_extension=webp,,,1920x1280,35f28dc2cbdf44aed190bc8efaa0fe7bfa288a2196638712dd83bc941cc294d5,yes,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,
blog_teen_cat,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1325391919706734&version=1783319609&transcode_extension=webp,,,1600x1600,c2e7df000ffa6520589d8efc5f0bcbeb759175cdaa0c827189c28a64f75d3dd7,yes,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,
blog_hero_1,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=2140990336800598&version=1783727678,,,1344x1792,7655639acb9f826cdb864d869530153efc1e1467ff414b5a63e80aa7b39b2fb7,no,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,seal survived CDN transcode without any metadata
control_synth_gradient,,synthetic-local,,PIL gradient with rectangle JPEG q90; never touched any AI pipeline,,1024x768,25f29c0b993ed1a8e41060ed50de778d875a6d94847ced180aafc5a6ad429f82,no,not_detected,,,,2026-08-26T19:01Z,negative control: No AI signatures from Meta were found
fox_nometa,,derived:gen_fox_forest,,metadata --remove only (no pixel change),,1600x1600,685d3a5c5d74e653ec7615fc7a347a0584c277fc7fb59594d388ac619ffe3985,no_stripped,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T22:19Z,metadata stripping does NOT remove Content Seal; seal lives in pixels; same generation ID as original
fox_modal_invisible,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,6daa3e9b5dde5cb443d8071c5294262742f001c02b694243764a50345769d8b8,no_stripped,not_detected,,,,2026-08-26T22:21Z,current default pipeline clears Content Seal
text_modal_invisible,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,8530a82fb856b0a00651076ac1ad8a7544b5d8873cac9bd5780704fbe10f53f8,no_stripped,not_detected,,,,2026-08-26T22:22Z,current default pipeline clears Content Seal
fox_str0.06000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,7b5e5ae75ca20f2a3cdfcfa7666104752d9557b08ac236db6f1548434940a1f3,no_stripped,not_detected,,,,2026-08-26T22:28Z,strength calibration ladder
fox_str0.04500,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,26da15694b3096e2f1717154db213e814cdd27f2fc40e5f924db2324213cf960,no_stripped,not_detected,,,,2026-08-27T02:28Z,strength calibration ladder; re-confirmed clean at 0.045
fox_str0.03750,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,b7eb0616d10042933edf22cfc30aff4df915384828289b409f6cfa4cdf2807d4,no_stripped,not_detected,,,,2026-08-26T22:30Z,fresh-page protocol; fox first-clean boundary is in (0.03 0.0375]
fox_str0.03000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,cff80d18d9d00000f47e0f97c419a37c13a13831795768aee8727aa9516d1401,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:31Z,fresh-page protocol; seal survives 0.03
fox_str0.02000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,b19183ec60b63a5b73d3e1c723e7943584912857d51a4b5f85259c5c77da0e53,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:28Z,settled-read protocol; monotonic below the 0.03 boundary
text_str0.03750,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,bacfd4eca7b13616502f74497cbe4f02ef6893918870f13f9e76fdb93b43107f,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol
text_str0.03000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,d5c6cffbf471ef8bb74ee8c0d22cc17d9da2c446b83e46c4be67e6ea4cdd09de,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol; text first-clean boundary is at or below 0.03 - cross-source spread vs fox confirmed
lh_str0.03750,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,f5169debc5fd2792399c0ea3fa88cb1d05f523d95004bec240c73db0e9aaee51,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.0375 where fox is already clean
lh_str0.03000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,110d2112d91dd53df5ccd1839e0e2b41c44aa1832adb0090032660e071fb1fd4,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,fresh-page protocol; lighthouse holds its seal below the fox boundary: cross-source spread is wide
text_str0.02500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,f46a6c53b4f3d39fccb1d62cca82ced2e163e4424d13c98e6468a8e083c0fa54,no_stripped,not_detected,,,,2026-08-27T02:28Z,mid rung inside the clean zone
text_str0.02000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,a19cbe22313e0f19768c98567ea89f2d9dc30827691fc7ec2971c52f3e7c6cc6,no_stripped,not_detected,,,,2026-08-27T02:28Z,text clean at 0.02: boundary <= 0.02 - the easiest source measured
text_str0.01500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.015,,1600x1600,aa3816c75e38f3ab5afc378d349caafad56f2bb6da859a5ee68fdea57ad00d73,no_stripped,not_detected,,,,2026-08-27T02:41Z,text clean at 0.015: boundary <= 0.015 - the easiest source measured
lh_str0.02500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,989c4a7a804bc8420ec4d86b9445bc3a37382c1834782a7f8842f781d69b05e2,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
lh_str0.02000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,6a4a4ec94292a145e7d73e6900b8ec6d0eede17137f2b91fba7757ec0bdba5ec,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
nc_str0.03750,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1920x1280,46f322335f801ee54427eb39275216465f9a7cfa3d918c698e4d90a2439a0815,no_stripped,not_detected,,,,2026-08-27T02:28Z,boundary (0.03; 0.0375]
nc_str0.03000,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1920x1280,3f42632319248ad1a91ef2457c7e0b13ed17e1d2569decbbaab67c2072e9a305,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,night_city holds seal at 0.03
mug_str0.03750,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,3b6109ae7e86b8ea74ce33315e3f2a7eda1cadb724d429f0951f13c3ead34b0f,no_stripped,not_detected,,,,2026-08-27T02:28Z,upper rung for the mug bracket
mug_str0.03000,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,1c24c984677e1528da83003ac40594f6e5ef1fa9351f1368b6d31a0101308583,no_stripped,not_detected,,,,2026-08-27T02:28Z,mug clean at 0.03: boundary <= 0.03
lh_str0.05250,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0525,,1600x1600,8912ba1ff4a26790f69224c49dd649e46835e277c375276ac9604f5339775b2a,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:41Z,bisection: lighthouse first-clean boundary bracketed to (0.0525; 0.06]
lh_str0.04500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,cfdcf7bbb27c740846a1de4edffee3b25a624c2a8e1aba2362feaa25b682e3ae,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.045: lighthouse is the worst source measured
lh_str0.06000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,4e214be00a89dc9f1f32ec39483c5e9668a1ec7f38b15e5794942479c839fcca,no_stripped,not_detected,,,,2026-08-27T02:28Z,lighthouse first-clean boundary bracketed to (0.045 0.06]
1 name file origin source_url prompt requested_size delivered_size sha256 xmp_iptc_trainedAlgorithmicMedia oracle_verdict oracle_attribution oracle_generation_id oracle_created_utc checked_at_utc notes
2 gen_fox_forest originals/gen_fox_forest.webp meta-model-api A red fox resting on a mossy log in a misty forest at dawn photorealistic shallow depth of field 1024x1024 1600x1600 6d7cecf322a5ac4b2547e2f32c767b3cb3e8a0e49f274cfd00aa31be6697eed4 yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD 2026-08-26T19:07Z 2026-08-26T19:11Z checked twice; ID stable across uploads
3 gen_text_poster originals/gen_text_poster.webp meta-model-api A cheerful flat-design poster that says 'WATERMARK FIELD DAY 2026' in bold rounded letters over a coral and cream striped background 1024x1024 1600x1600 873927662d8198e9c38b4ae71a2919ba5d417dfcd714649db1b0ae231e8cbf9b yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbJuIKXPAYxsd 2026-08-26T19:08Z 2026-08-26T19:08Z
4 gen_lighthouse_watercolor originals/gen_lighthouse_watercolor.webp meta-model-api A watercolor illustration of a lighthouse on a cliff at golden hour seagulls in the distance soft loose washes white paper texture 1024x1024 1600x1600 69bee7c376608568361085602c5c7406900885849bd94c5279c3924cf793b27c yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbB5uWbAmWmwp 2026-08-26T19:08Z 2026-08-26T19:08Z
5 gen_night_city originals/gen_night_city.webp meta-model-api A night city skyline in the rain neon reflections on wet asphalt cinematic wide shot moody 1536x1024 1920x1280 345adf6b9f777b15ef7ce7bbdfb3b355a1d9e882740809e966e9f6caafd94d48 yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbH5uWbAmWmwp 2026-08-26T19:08Z 2026-08-26T19:08Z
6 gen_studio_mug originals/gen_studio_mug.webp meta-model-api A minimalist studio photo of a single matte ceramic mug on a pastel blue background soft shadow clean commercial look 1024x1024 1600x1600 2b7bc7fd56fa2250c66ef74f8b3e49d9666fe147d894d5baf9461b5e36077a07 yes detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbAsSozV2XCzw 2026-08-26T19:08Z 2026-08-26T19:08Z
7 fox_crop50 derived:gen_fox_forest center crop 0.5 linear each side then WEBP q95 800x800 80b91995560a7c4b08596a26b3401a29afb82691fad193502697a49785eab7be no_stripped_by_reencode not_detected 2026-08-26T19:13Z seal lost; PIL re-encode strips XMP
8 fox_crop33 derived:gen_fox_forest center crop 0.33 linear each side then WEBP q95 528x528 3b3aded8f4fcb98d0599431abccb73b9581854ae34c6bc869724cd5c86a1e35e no_stripped_by_reencode not_detected 2026-08-26T19:13Z seal lost
9 fox_res512 derived:gen_fox_forest resize long side 512 LANCZOS then WEBP q95 512x512 19b85f615d56f5e9f55cd97bae0ee343f4f432e6bfa0a6fc7acb07dec2cc1d81 no_stripped_by_reencode detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD 2026-08-26T19:07Z 2026-08-26T19:13Z generation ID and creation time identical to the original generation: watermark payload survives the resize
10 fox_jpeg85 derived:gen_fox_forest full-size JPEG q85 re-encode 1600x1600 3bc7dc11bf3c16c83814df1e6485cdb6e49e21141511dda56d210cc920a8c5e3 no_stripped_by_reencode detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD 2026-08-26T19:07Z 2026-08-26T19:13Z payload survives JPEG recompression; local identify returns unknown (metadata stripped)
11 text_crop50 derived:gen_text_poster center crop 0.5 linear each side then WEBP q95 800x800 3f0d4075012ee4fec820209c5c1c32bb739998813325f49b1124f379162f9837 no_stripped_by_reencode not_detected 2026-08-26T19:13Z seal lost
12 text_crop33 derived:gen_text_poster center crop 0.33 linear each side then WEBP q95 528x528 95cef87002961831b21f6f47cd9a0824a67e2ee6f8503356fd5db6a3fcfcc998 no_stripped_by_reencode daily oracle limit reached before this row could be checked
13 text_res512 derived:gen_text_poster resize long side 512 LANCZOS then WEBP q95 512x512 b319727d02e908f22e59f148050e6b4686cbd1a622bd69db51fe4072a6f66b34 no_stripped_by_reencode daily oracle limit reached; also the fox twin of this transform triggered the Yuanbao false positive below
14 text_jpeg85 derived:gen_text_poster full-size JPEG q85 re-encode 1600x1600 76f00fcde1e974549afa5d90e1879c705e3c3631ef1f35cab901fc912cd15fa4 no_stripped_by_reencode daily oracle limit reached before this row could be checked
15 blog_qr_manhwa meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=1807972953501959&version=1783318590&transcode_extension=webp 1920x1280 8ee7700dff59f21d7de393a619664a0b9360f148d18df289364ecea939b3dea9 yes detected Muse Image 1 - Meta 2026-08-26T18:59Z not stored; re-downloadable from source_url
16 blog_fractal_poster meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=1499727434782014&version=1783727678&transcode_extension=webp 2096x992 2a96168db026444b47ac37807fd99a88c8b7fa6866e1c93a54ab427fbe581f76 no detected Muse Image 1 - Meta 2026-08-26T18:59Z seal survived CDN transcode without any metadata
17 blog_flipbook_punch meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=2521328538303639&version=1783396590&transcode_extension=webp 1920x1280 35f28dc2cbdf44aed190bc8efaa0fe7bfa288a2196638712dd83bc941cc294d5 yes detected Muse Image 1 - Meta 2026-08-26T19:00Z
18 blog_teen_cat meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=1325391919706734&version=1783319609&transcode_extension=webp 1600x1600 c2e7df000ffa6520589d8efc5f0bcbeb759175cdaa0c827189c28a64f75d3dd7 yes detected Muse Image 1 - Meta 2026-08-26T19:00Z
19 blog_hero_1 meta-blog-cdn https://lookaside.fbsbx.com/elementpath/media/?media_id=2140990336800598&version=1783727678 1344x1792 7655639acb9f826cdb864d869530153efc1e1467ff414b5a63e80aa7b39b2fb7 no detected Muse Image 1 - Meta 2026-08-26T19:00Z seal survived CDN transcode without any metadata
20 control_synth_gradient synthetic-local PIL gradient with rectangle JPEG q90; never touched any AI pipeline 1024x768 25f29c0b993ed1a8e41060ed50de778d875a6d94847ced180aafc5a6ad429f82 no not_detected 2026-08-26T19:01Z negative control: No AI signatures from Meta were found
21 fox_nometa derived:gen_fox_forest metadata --remove only (no pixel change) 1600x1600 685d3a5c5d74e653ec7615fc7a347a0584c277fc7fb59594d388ac619ffe3985 no_stripped detected Muse Image 1 - Meta AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD 2026-08-26T19:07Z 2026-08-26T22:19Z metadata stripping does NOT remove Content Seal; seal lives in pixels; same generation ID as original
22 fox_modal_invisible derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600) 1600x1600 6daa3e9b5dde5cb443d8071c5294262742f001c02b694243764a50345769d8b8 no_stripped not_detected 2026-08-26T22:21Z current default pipeline clears Content Seal
23 text_modal_invisible derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600) 1600x1600 8530a82fb856b0a00651076ac1ad8a7544b5d8873cac9bd5780704fbe10f53f8 no_stripped not_detected 2026-08-26T22:22Z current default pipeline clears Content Seal
24 fox_str0.06000 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06 1600x1600 7b5e5ae75ca20f2a3cdfcfa7666104752d9557b08ac236db6f1548434940a1f3 no_stripped not_detected 2026-08-26T22:28Z strength calibration ladder
25 fox_str0.04500 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045 1600x1600 26da15694b3096e2f1717154db213e814cdd27f2fc40e5f924db2324213cf960 no_stripped not_detected 2026-08-27T02:28Z strength calibration ladder; re-confirmed clean at 0.045
26 fox_str0.03750 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1600x1600 b7eb0616d10042933edf22cfc30aff4df915384828289b409f6cfa4cdf2807d4 no_stripped not_detected 2026-08-26T22:30Z fresh-page protocol; fox first-clean boundary is in (0.03 0.0375]
27 fox_str0.03000 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1600x1600 cff80d18d9d00000f47e0f97c419a37c13a13831795768aee8727aa9516d1401 no_stripped detected Muse Image 1 - Meta 2026-08-26T22:31Z fresh-page protocol; seal survives 0.03
28 fox_str0.02000 derived:gen_fox_forest modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02 1600x1600 b19183ec60b63a5b73d3e1c723e7943584912857d51a4b5f85259c5c77da0e53 no_stripped detected Muse Image 1 - Meta 2026-08-26T22:28Z settled-read protocol; monotonic below the 0.03 boundary
29 text_str0.03750 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1600x1600 bacfd4eca7b13616502f74497cbe4f02ef6893918870f13f9e76fdb93b43107f no_stripped not_detected 2026-08-26T22:33Z fresh-page protocol
30 text_str0.03000 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1600x1600 d5c6cffbf471ef8bb74ee8c0d22cc17d9da2c446b83e46c4be67e6ea4cdd09de no_stripped not_detected 2026-08-26T22:33Z fresh-page protocol; text first-clean boundary is at or below 0.03 - cross-source spread vs fox confirmed
31 lh_str0.03750 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1600x1600 f5169debc5fd2792399c0ea3fa88cb1d05f523d95004bec240c73db0e9aaee51 no_stripped detected Muse Image 1 - Meta 2026-08-27T02:28Z seal survives 0.0375 where fox is already clean
32 lh_str0.03000 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1600x1600 110d2112d91dd53df5ccd1839e0e2b41c44aa1832adb0090032660e071fb1fd4 no_stripped detected Muse Image 1 - Meta 2026-08-27T02:28Z fresh-page protocol; lighthouse holds its seal below the fox boundary: cross-source spread is wide
33 text_str0.02500 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025 1600x1600 f46a6c53b4f3d39fccb1d62cca82ced2e163e4424d13c98e6468a8e083c0fa54 no_stripped not_detected 2026-08-27T02:28Z mid rung inside the clean zone
34 text_str0.02000 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02 1600x1600 a19cbe22313e0f19768c98567ea89f2d9dc30827691fc7ec2971c52f3e7c6cc6 no_stripped not_detected 2026-08-27T02:28Z text clean at 0.02: boundary <= 0.02 - the easiest source measured
35 text_str0.01500 derived:gen_text_poster modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.015 1600x1600 aa3816c75e38f3ab5afc378d349caafad56f2bb6da859a5ee68fdea57ad00d73 no_stripped not_detected 2026-08-27T02:41Z text clean at 0.015: boundary <= 0.015 - the easiest source measured
36 lh_str0.02500 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025 1600x1600 989c4a7a804bc8420ec4d86b9445bc3a37382c1834782a7f8842f781d69b05e2 no_stripped below the measured detected boundary (0.03) for this source; not individually checked
37 lh_str0.02000 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02 1600x1600 6a4a4ec94292a145e7d73e6900b8ec6d0eede17137f2b91fba7757ec0bdba5ec no_stripped below the measured detected boundary (0.03) for this source; not individually checked
38 nc_str0.03750 derived:gen_night_city modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1920x1280 46f322335f801ee54427eb39275216465f9a7cfa3d918c698e4d90a2439a0815 no_stripped not_detected 2026-08-27T02:28Z boundary (0.03; 0.0375]
39 nc_str0.03000 derived:gen_night_city modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1920x1280 3f42632319248ad1a91ef2457c7e0b13ed17e1d2569decbbaab67c2072e9a305 no_stripped detected Muse Image 1 - Meta 2026-08-27T02:28Z night_city holds seal at 0.03
40 mug_str0.03750 derived:gen_studio_mug modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375 1600x1600 3b6109ae7e86b8ea74ce33315e3f2a7eda1cadb724d429f0951f13c3ead34b0f no_stripped not_detected 2026-08-27T02:28Z upper rung for the mug bracket
41 mug_str0.03000 derived:gen_studio_mug modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03 1600x1600 1c24c984677e1528da83003ac40594f6e5ef1fa9351f1368b6d31a0101308583 no_stripped not_detected 2026-08-27T02:28Z mug clean at 0.03: boundary <= 0.03
42 lh_str0.05250 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0525 1600x1600 8912ba1ff4a26790f69224c49dd649e46835e277c375276ac9604f5339775b2a no_stripped detected Muse Image 1 - Meta 2026-08-27T02:41Z bisection: lighthouse first-clean boundary bracketed to (0.0525; 0.06]
43 lh_str0.04500 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045 1600x1600 cfdcf7bbb27c740846a1de4edffee3b25a624c2a8e1aba2362feaa25b682e3ae no_stripped detected Muse Image 1 - Meta 2026-08-27T02:28Z seal survives 0.045: lighthouse is the worst source measured
44 lh_str0.06000 derived:gen_lighthouse_watercolor modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06 1600x1600 4e214be00a89dc9f1f32ec39483c5e9668a1ec7f38b15e5794942479c839fcca no_stripped not_detected 2026-08-27T02:28Z lighthouse first-clean boundary bracketed to (0.045 0.06]
Binary file not shown.

After

Width:  |  Height:  |  Size: 585 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 510 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 652 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 192 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.2 MiB

+400
View File
@@ -0,0 +1,400 @@
# Pipeline fidelity evaluation
This evaluation compares diffusion pipelines with
`scripts/fidelity_metrics.py`. The images themselves have one canonical home
in `data/synthid/originals/`; this directory stores only evaluation-specific
ground truth and instructions.
| Original | Provider | Content | Exercises |
| --- | --- | --- | --- |
| `ChatGPT Image May 31, 2026, 02_02_23 PM.png` | OpenAI | Light multilingual typography | Text preservation |
| `ChatGPT Image May 31, 2026, 02_03_55 PM.png` | OpenAI | Multilingual typography | Text preservation |
| `Gemini_Generated_Image_633uuy633uuy633u.png` | Google | Landscape with a Chinese sign | CJK text preservation |
| `Gemini_Generated_Image_y48j3cy48j3cy48j.png` | Google | Portrait grid | Face identity and skin texture |
## Text ground truth
`ground-truth.json` contains hand-verified OCR for the three text-bearing
originals. To regenerate an OCR seed:
```bash
uv run scripts/fidelity_metrics.py ocr \
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png" \
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png" \
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
--langs en,ru,ch \
--out data/evaluations/fidelity/ground-truth.json
```
Verify and correct the generated text by hand before using it as ground truth.
`text-lines.json` contains the verified per-line strings and source-space boxes
used by the evaluation-only selective-restoration experiment. It is not an
automatic scene-text annotation set.
`scripts/infer_text_lines.py` can generate a draft from source pixels without
existing annotations. On the two posters it proposed 20 and 18 lines at the
default threshold, but exact-text precision was only 90.0% and 94.4% because
high-confidence OCR still dropped an English comma and replaced a Chinese comma
with ASCII. Its output therefore requires manual verification of every line;
`accepted` means crop-stable, not ground-truth-correct.
## Text-preservation benchmark
`text-preservation-2026-08-13.csv` records a fixed-seed comparison of the two
current profiles and a global Z-Image Turbo prototype on all three text
fixtures. Each candidate ran through the
complete `visible -> invisible -> metadata` route with its profile default
strength and adaptive-polish setting. The Z-Image prototype instead sweeps
0.08, 0.10, 0.15, 0.20, and 0.25 without polish because its provider-specific
removal floor is not known. The output hash identifies the exact bytes
measured; generated outputs remain outside the repository.
The character-weighted added CER is 0.262 for `qwen-zimage` and 0.256 for
`sdxl-zimage`. That 0.006 absolute difference is not a
stable ordering: SDXL wins the light poster, Qwen wins the dark poster, and the
Chinese sign is tied. A paired sign test on the two non-ties is 1-1 (`p=1.0`).
The measured sample therefore does not support a general text-preservation
winner. Both profiles substantially degrade the smallest multilingual poster
text and preserve the larger Chinese sign.
Z-Image Turbo is the clear fidelity lead. At strength 0.10 its
character-weighted CER is 0.093, against the unchanged sources' 0.124 OCR
floor, while whole-image LPIPS stays between 0.047 and 0.086. Visual inspection
still finds substitutions in the smallest Cyrillic and Chinese poster lines,
so a favorable OCR score does not mean pixel-exact text. Quality drops quickly
above 0.15; at 0.25 the weighted CER is 0.428.
Provider-oracle checks on 2026-08-13 bracket the OpenAI requirement at no more
than 0.10 for this sample. Both original OpenAI controls were detected by
`openai.com/verify`; the light poster was clean at 0.08, while the dark poster
was detected at 0.08 and clean at 0.10. The light 0.10 output was not separately
checked. Google fails the entire tested sweep. The original Gemini image was
detected through C2PA, and a pixel-identical copy with AI metadata stripped was
separately detected by Gemini's built-in SynthID verifier. Z-Image outputs at
0.08, 0.10, 0.15, 0.20, and 0.25 were all still detected. At 0.25 the weighted
CER has already risen to 0.428, worse than the raw weighted CER of both current
profiles (Qwen 0.387, SDXL 0.381). Increasing strength beyond the measured grid
would therefore no longer serve the text-preservation objective without a new
mechanism or hypothesis. Z-Image is not a viable global replacement on this
evidence: no clean Google operating point was found before it lost its fidelity
advantage.
Qwen-Image-2.0 was not added to the numeric comparison. Its weights are not
published, and its hosted edit API exposes an editing instruction and seed but
no low-strength denoise control. It can be evaluated as a separate hosted edit
strategy when credentials are available, but it is not a drop-in replacement
for the partial-regeneration mechanism measured here.
The OCR floor is the source image scored against the hand-verified text. Use
`added_cer = text_cer - ocr_floor` when interpreting pipeline damage, because the
unchanged poster sources already score 0.127 CER. `oracle_rechecked=true` marks
the exact Z-Image bytes checked above; the remaining rows were not rechecked.
The table does not certify other seeds, content classes, or strengths beyond
the recorded provider verdicts.
## Text-restoration prototype
`text-restoration-2026-08-13.csv` evaluates an OCR-driven post-pass on the exact
Qwen outputs above. The prototype recognizes English and Russian with macOS
Vision and CJK with PaddleOCR, derives glyph masks independently from the
source and Qwen output, removes both sets of glyphs with block-wise LaMa, and
draws the recognized strings with new system-font pixels. It never composites
source pixels back into the result.
On the two multilingual posters, character-weighted CER fell from 0.338 and
0.305 to 0.007 on both. OpenAI Verify reported no OpenAI signals for both Qwen
controls and both restored outputs in the same run. The improvement comes with
a substantial whole-image fidelity cost: LPIPS rose from 0.107 to 0.174 and
from 0.095 to 0.162, while PSNR fell by about 10.5-11.4 dB. Visual inspection
found one residual shadow in the smallest English line of the light poster;
the dark poster was clean but the substitute fonts visibly changed typography.
The Chinese sign did not improve: CER rose from 0.074 to 0.111 because the OCR
and renderer changed punctuation. The Gemini verifier returned detected for
both the restored output and its byte-identical Qwen control on a third work
account, although that Qwen hash had previously returned clean on another work
account. This run therefore does not isolate a restoration-stage regression;
its Google verdict is inconclusive until a source-positive, Qwen-negative
control can be reproduced in the same available account.
This prototype is not ready to ship. Its strongest result establishes that
fresh-glyph reconstruction can recover literal text without reintroducing an
OpenAI signal, but portable OCR, font/style reconstruction, a tighter mask, and
a reproducible Google oracle control remain prerequisites.
### Selective restoration follow-up
`selective-text-restoration-2026-08-13.csv` compares that full compositor with
a selective prototype on the same two posters. The prototype leaves a Qwen line
unchanged when padded source and output recognition agree, and applies the same
LaMa plus fresh-system-font reconstruction only to lines whose recognized text
changed. This reduced the edited area from 15.4% and 17.0% to 5.5% on both
posters.
Under one consistent Paddle `en+ru+ch` measurement route, selective restoration
reduced Qwen CER from 0.378 to 0.101 on the light poster and from 0.413 to 0.112
on the dark poster. Its image LPIPS was 0.120 and 0.103, substantially closer to
Qwen than the full compositor's 0.174 and 0.162. These CER values must not be
mixed with the preceding table's Vision/Paddle hybrid values: the comparison
file remeasures all three variants through Paddle so their relative result is
valid on one OCR route.
OpenAI Verify returned `No OpenAI signals detected` for both selective outputs,
then detected the original light poster as `Generated with OpenAI tools` in the
same Chrome sequence. Visual inspection found the dark output clean, but the
light output still retained a local shadow around one replaced fine-text line.
Selective restoration is therefore the strongest current direction, not a
production-ready default. The next implementation needs automatic line matching
and a tighter source-plus-candidate glyph mask before it can be proposed for the
pipeline.
A mask-only follow-up added two pixels of dilation around every selected glyph.
It visually removed the light poster's shadow and improved its CER from 0.101 to
0.081 with effectively unchanged LPIPS, but increased the edited area from 5.5%
to 7.0%. Applying the same rule to the dark poster worsened CER from 0.112 to
0.147. These expanded-mask rows were not resubmitted to the provider oracle.
The opposing result rules out one global dilation constant and makes per-line
residual detection the next fidelity gate.
An automatic follow-up removed the hand-selected line list. For each known
source line, it runs a recognition-only model on padded, upscaled source and
Qwen crops. It preserves a line only when both confidence scores are at least
0.75, the source recognition is within 0.25 normalized edit distance of the
verified line, and normalized source and candidate recognition match exactly.
This reproduced the manual dark-poster line selection and additionally kept one
correct large Russian line on the light poster.
The automatic prototype also replaced uniform dilation with a second measured
LaMa pass. After the first erase, it extracts contrast remaining at the original
glyph positions, dilates only that residual, and inpaints it before drawing fresh
text. Visual inspection found no remaining double glyphs on either output. The
effective edited fractions were 4.1% and 5.7%. Image LPIPS was 0.113 and 0.104,
against 0.120 and 0.103 for the manual selective version. Detector-based page CER
was 0.123 and 0.119; the light value is inflated by the detector splitting the
preserved Cyrillic line into Latin-like fragments even though the padded Russian
recognizer reads it exactly. This is another reason that line decisions use
recognition-only crops rather than the page detector's aggregate CER.
Both exact automatic output hashes returned `No OpenAI signals detected`, and
the original light poster returned `Generated with OpenAI tools` immediately
afterward in the same Chrome sequence. The tracked script reproduced the dark
hash exactly. Its light output differed from the initial prototype in only 386
pixels by at most one channel value because separate ONNX environments rounded
differently; the script's exact `900def5a...` hash was therefore submitted and
also returned clean before the positive source control.
The script and its required verified line annotations are tracked as
`scripts/selective_text_restoration.py` and `text-lines.json`. For example:
```bash
PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK=True \
uv run scripts/selective_text_restoration.py \
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
path/to/qwen-candidate.png \
--restoration vae-glyphs \
--glyph-donor path/to/qwen-vae-reconstruction.png \
--glyph-feather 0.5 \
--output path/to/selective-output.png \
--manifest path/to/manifest.json
```
The restoration mode is deliberately explicit. `vae-glyphs` requires a
separately generated VAE reconstruction in `--glyph-donor`; it takes exact
thresholded glyph cores from that donor, uses a fresh silhouette beneath them,
and defaults to a narrow 0.5-pixel donor edge. The donor and the resulting exact bytes still require
full-pipeline oracle evaluation. `rerender` reproduces the tracked
fresh-system-font experiment. `source-glyphs` is only for compositing a
separately regenerated, oracle-evaluated layer; feeding it the watermarked
original would paste provenance-bearing pixels back into the result. The
`--keep-background --composite-mask boxes` combination is an aligned-layer
experiment and is not a production text restorer.
The `source-silhouette` follow-up keeps the source glyph geometry but discards
its pixel amplitudes: it thresholds each source line to a binary shape, samples
one foreground color, and synthesizes new antialiasing over the scrubbed image.
On a 24-image matrix spanning serif, Latin-diacritic, Cyrillic, CJK, tiny-UI,
and rotated text, the original outer-feather compositor preserved all 34
source-readable lines exactly under crop OCR, with median text-box SSIM 0.859.
An inner-antialias variant raised median text-box SSIM to 0.902 and left
whole-image SSIM unchanged at 0.672, but preserved 33/34 lines: OCR read one
middle dot as a colon. The higher-fidelity antialiasing is retained in the
evaluation script, with that punctuation miss recorded as a caveat. Its exact
outputs were not submitted to the provider oracle because the
public verifier reached its request limit; a clean verdict from the earlier
outer-feather bytes does not transfer to the new hashes.
Follow-up visual review on a typography-rich control rejected both silhouette
compositors despite their OCR and SSIM scores. They preserved literal content
and approximate glyph geometry, but changed stroke weight, color variation,
edge antialiasing, and small decorative details enough to be plainly different
from the source. OCR exactness and text-box SSIM are therefore screening metrics,
not acceptance gates for source-typography preservation. A candidate must also
retain source-like edge pixels and pass direct visual comparison at native size.
A narrower Qwen-VAE donor follow-up keeps the scrubbed background, then copies
only VAE-reconstructed pixels through the source-silhouette mask with a
0.5-pixel feather. Across the 48-case typography matrix (548 annotated lines),
text-box SSIM improved in 47/48 cases and glyph-edge error improved in 48/48.
Median text-box SSIM rose from 0.854 to 0.914, while median glyph-edge MAE fell
from 37.59 to 32.42. The median nonzero alpha area was 3.62%; on dense or large
text this is still substantially wider than the target control. These are
fidelity results only; the 48 matrix outputs were not submitted to the provider
oracle. On a separate untracked dense-typography control, the exact core-only
Qwen-VAE donor with a 1.43% source-silhouette area returned `No OpenAI signals
detected`. Its 0.5-pixel feathered sibling, which raises the nonzero donor area
to 2.75%, returned the same verdict and improved mean text-box SSIM from 0.957
to 0.960. Crop OCR recovered from 7/15 exact lines on the raw pass to 14/15,
matching the source OCR floor. Whole-image LPIPS was 0.082, but only 0.108% of
pixels were exactly unchanged and the detected face retained 0.670 of source
Laplacian variance. The tracked script reproduced the feathered file byte for
byte. These two exact-byte verdicts do not certify other images or the larger
matrix masks, and the global smoothing fails a strict unchanged-image criterion.
The opt-in production port was rechecked separately on 2026-08-15. Its current
LaMa runtime did not reproduce the earlier evaluation PNG byte for byte, but all
changed pixels were confined to the erased background outside the donor glyph
core. The exact production artifact returned `No OpenAI signals detected` in
3/3 OpenAI Verify runs, while the matched source control returned `Generated
with OpenAI tools` in 2/2 runs in the same Chrome session; expanded details
identified SynthID and no C2PA manifest on the control. The private control and
artifact hashes remain outside the public repository. This certifies only that
runtime, verified manifest, and output, not arbitrary text masks or images.
The Google result is negative. On the synthetic CJK sign case, two separate
work-account runs both detected SynthID in the resaved source control and in the
exact Qwen-VAE donor output. The candidate improved mean text-box SSIM from
0.791 to 0.835 and glyph-edge MAE from 35.48 to 22.49 with a 3.67% donor area,
but Gemini still detected it. The intermediate Qwen silhouette base and the
earlier original-sign Qwen baseline were also detected in the same account.
This isolates the current blocker upstream of text restoration: the global
Google pass did not reach the SynthID removal floor, so the OpenAI-clean
`vae-glyphs` configuration cannot be used as a Google operating point.
On the large Chinese-sign control, source and candidate recognition agreed on
all three lines. The script selected no changed lines, emitted a zero mask, and
copied the candidate byte for byte instead of re-encoding it. This corrects the
earlier unnecessary CJK rerender and proves the no-edit branch. It does not add
a Google-negative oracle result: the available Google account still cannot
reproduce a source-positive, Qwen-negative control sequence.
The earlier automatic rerender was the first variant in the experiment to pass
the recorded visual, fidelity, and OpenAI-oracle gates on its two poster
fixtures. The later typography-rich control shows that result does not
generalize to source-typography preservation. It remains evaluation-only: it
depends on verified source text and source line boxes, uses macOS system fonts,
and has not been validated on natural scene text, rotated text, or automatic
line-box discovery.
An opt-in `--detect-boxes` follow-up tested automatic geometry. Grouping Paddle
word detections by vertical overlap found exactly 20/20, 20/20, and 3/3 lines;
mean IoU with verified boxes was 0.857, 0.847, and 1.000. Reusing the annotation
crop padding was unstable and preserved only 4/20 dark-poster lines. Reducing
vertical recognition padding to 10% restored the exact 8/20 and 7/20 selection
decisions, but detector CER was 0.127 and 0.154 rather than 0.123 and 0.119. The
dark regression failed the fidelity gate, so these hashes were not submitted to
the provider oracle. The flag is retained only to reproduce that negative
evaluation and still requires verified strings and an exact line-count match.
### AnyText2 glyph-conditioned follow-up
`anytext2-restoration-2026-08-13.csv` tests the official Apache-2.0
AnyText2 checkpoint as a local text-editing pass over the exact Qwen Chinese-sign
output. The checkpoint hash was verified against ModelScope. Its own edit example
successfully replaced a masked blackboard line with the requested `DADDY`, so the
runtime and checkpoint were functional before the tracked fixture was measured.
AnyText2 failed the fidelity gate. The standard full-image detector scored the
default edit at CER 0.185 and the source-font mimic at 0.222, against 0.074 for
Qwen; image LPIPS moved from 0.289 to 0.338 and 0.345. A padded crop-recognition
cross-check removed the detector's punctuation misses: Qwen scored 0.000, the
default edit still scored 0.074 with two substituted characters, and the mimic
edit stayed at 0.222. Visual inspection agreed, with additional line-shape and
punctuation errors in the mimic output. No provider-oracle submission was made
because neither candidate preserved the requested content better than its
already-clean Qwen input.
This result covers the published SD1.5 checkpoint, three Chinese lines of no more
than 20 characters, one seed, and one fixture. The repository still describes
AnyText2XL as unreleased, and the inference wrapper truncates each requested line
after 20 characters, so the published model cannot directly cover the longer
English and Russian poster lines.
The padded cross-check also changes the interpretation of this fixture's earlier
restoration result. Its Qwen image already contains all three correct Chinese
lines; the reported 0.074 came from the detector cropping the final punctuation
from two lines. A deterministic rerender with horizontally padded recognition
and a mask-matched Hiragino Sans GB W6 font reached detector CER 0.000, but image
LPIPS worsened from 0.289 to 0.327 and the font was visibly heavier. The correct
policy on this large-text fixture is therefore to leave a Qwen line unchanged
when padded recognition confirms the source and candidate strings agree.
## Face-stage isolation
`face-stage-isolation-2026-08-13.csv` separates the two stages on the 18-face
Gemini portrait-grid fixture. All variants use the same visible-cleaned,
metadata-stripped input and seed 0. The prepared input remained positive in the
Gemini SynthID verifier, so the surrounding visible and metadata stages did not
create the clean verdicts.
| Route | Gemini oracle | ID cosine | Face LPIPS | Laplacian ratio | Image LPIPS |
| --- | --- | ---: | ---: | ---: | ---: |
| Prepared control | detected | - | - | - | - |
| Z-Image face stage only | detected | 0.927 | 0.028 | 0.612 | 0.013 |
| Qwen global stage only | clean | 0.589 | 0.217 | 0.331 | 0.288 |
| Full Qwen then Z-Image | clean | 0.852 | 0.050 | 0.492 | 0.252 |
This isolates the roles: the Qwen global pass removes the watermark, while the
masked Z-Image pass restores face identity and detail without reintroducing a
detectable whole-image signal. Z-Image face-only does not remove SynthID. The
result explains why the face stage succeeded while global Z-Image failed: the
earlier clean verdict belonged to the two-stage pipeline, not to Z-Image as a
standalone remover. This is one fixture, one seed, and one oracle pass per
variant; it does not establish a general reintroduction threshold for mask size.
## Upstream Synthid-Bypass v2 reproduction
`upstream-v2-reproduction-2026-08-13.csv` records a source-level audit and a
close reproduction of the public
[`Synthid-Bypass-v2.0.json`](https://github.com/cebeuq/Synthid-Bypass/blob/3007d0351596ae0a78b7074dae7ad179710b1e48/Synthid-Bypass-v2.0.json).
The audited workflow is SHA-256
`41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6` at
upstream commit `3007d0351596ae0a78b7074dae7ad179710b1e48`.
Tracing links backward from `SaveImage` confirms that the active global stage is
Qwen-Image-2512 Q4 with the Lightning four-step LoRA at 0.8, DiffSynth Canny at
1.0, AuraFlow shift 3, `dpmpp_2m` plus `sgm_uniform`, CFG 1, and four steps. Its
resolution helper uses adaptive level 6 over 0.08..0.15, which resolves to the
0.154 ceiling for the 2816x1536 portrait fixture. Z-Image Turbo is used only by
the face detailer: eight steps, CFG 1, `res_2s` plus `bong_tangent`, a 768 px
guide, 1024 px cap, and direct adaptive denoise
`clamp(0.10 * largest_face_ratio / 0.03, 0.05, 0.28)`. The active face regions
come from YOLOv8-face plus SAM. Although the README describes MediaPipe as part
of the strict path, the MediaPipe nodes do not reach the saved output. The
1.2-megapixel scaler is also bypassed (`mode=4`).
The published upstream pair 12 changed from SynthID-positive to clean in the
Gemini verifier and scored 0.975 face identity. On the project portrait fixture,
the prepared control was positive, while both the close global reproduction and
the close full reproduction were clean. The global stage scored 0.589 identity
and 0.217 face LPIPS; the upstream-strength face stage improved those to 0.783
and 0.083. The maintained profile's weaker face pass scored 0.852 and 0.050 on
the same control, so copying upstream's roughly doubled face strength would be a
quality regression on this fixture.
The close reproduction is intentionally labeled rather than presented as an
exact ComfyUI run. It uses full-precision DiffSynth bf16 weights instead of the
Q4 GGUF files, the maintained DiffSynth Lightning scheduler approximation
instead of the ComfyUI sampler pair, YuNet plus SAM instead of YOLOv8-face plus
SAM, and fixed seed 0 instead of randomized seeds. The two independent oracle
controls and the published upstream pair establish the direction of the result;
they do not make the local output byte-equivalent to upstream.
## Compare
```bash
uv run scripts/fidelity_metrics.py compare \
--original data/synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png \
--variant qwen-zimage=<qwen-out>.png \
--variant sdxl-zimage=<sdxl-out>.png \
--ocr-langs ""
```
@@ -0,0 +1,4 @@
date,source_commit,fixture,variant,model,model_revision,model_sha256,ocr_route,text_cer,img_lpips,ssim,psnr_db,output_sha256,provider_oracle,notes
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,,,,"Paddle ch",0.074,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621,not_run,Exact Qwen baseline reused from text-restoration benchmark
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-default,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.185,0.338,0.642,18.2,87729fe7e0482d2cead6d3b7c7dd386ecc75ab3d8658131c54e67528b8f9feeb,not_run,Fidelity gate failed; padded crop recognition CER 0.074 still shows two substituted characters; locally composited three-line edit; 20 DDIM steps; seed 0
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-mimic,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.222,0.345,0.644,17.9,f5198d8f2365b79880aba8ea6b0c3885142b9de21c9f7f4f6712ac4a0db8dd85,not_run,Fidelity gate failed; source glyph masks supplied as font hints; 20 DDIM steps; seed 0
1 date source_commit fixture variant model model_revision model_sha256 ocr_route text_cer img_lpips ssim psnr_db output_sha256 provider_oracle notes
2 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage Paddle ch 0.074 0.289 0.696 23.3 4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621 not_run Exact Qwen baseline reused from text-restoration benchmark
3 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage-anytext2-default AnyText2-SD1.5 b06c583a583818f3679665ef67b51363f107853c 7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8 Paddle ch 0.185 0.338 0.642 18.2 87729fe7e0482d2cead6d3b7c7dd386ecc75ab3d8658131c54e67528b8f9feeb not_run Fidelity gate failed; padded crop recognition CER 0.074 still shows two substituted characters; locally composited three-line edit; 20 DDIM steps; seed 0
4 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage-anytext2-mimic AnyText2-SD1.5 b06c583a583818f3679665ef67b51363f107853c 7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8 Paddle ch 0.222 0.345 0.644 17.9 f5198d8f2365b79880aba8ea6b0c3885142b9de21c9f7f4f6712ac4a0db8dd85 not_run Fidelity gate failed; source glyph masks supplied as font hints; 20 DDIM steps; seed 0
@@ -0,0 +1,5 @@
date,source_commit,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,zimage-face-only,0,,0.06604512532552083,18,0.927,0.028,0.612,0.013,0.959,34.7,c388e9bbf737bb9b22a7cb2216ffb8026321ca632f9db29e1e456b7c14a8553f,synthid_detected
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-global-only,0,0.153999996,,18,0.589,0.217,0.331,0.288,0.646,24.2,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-zimage-full,0,0.153999996,0.06604512532552083,18,0.852,0.050,0.492,0.252,0.671,24.7,69de191bddbfbf53348c7638bdba661b6d45d257c00a8a2646b2558670c5a4c2,clean
1 date source_commit fixture route seed global_strength face_strength face_count id_cos face_lpips lapvar_ratio img_lpips ssim psnr_db output_sha256 gemini_oracle
2 2026-08-13 2d03a00 Gemini_Generated_Image_y48j3cy48j3cy48j.png visible-metadata-control 0 18 9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b synthid_detected
3 2026-08-13 2d03a00 Gemini_Generated_Image_y48j3cy48j3cy48j.png zimage-face-only 0 0.06604512532552083 18 0.927 0.028 0.612 0.013 0.959 34.7 c388e9bbf737bb9b22a7cb2216ffb8026321ca632f9db29e1e456b7c14a8553f synthid_detected
4 2026-08-13 2d03a00 Gemini_Generated_Image_y48j3cy48j3cy48j.png qwen-global-only 0 0.153999996 18 0.589 0.217 0.331 0.288 0.646 24.2 281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f clean
5 2026-08-13 2d03a00 Gemini_Generated_Image_y48j3cy48j3cy48j.png qwen-zimage-full 0 0.153999996 0.06604512532552083 18 0.852 0.050 0.492 0.252 0.671 24.7 69de191bddbfbf53348c7638bdba661b6d45d257c00a8a2646b2558670c5a4c2 clean
@@ -0,0 +1,5 @@
{
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
"Gemini_Generated_Image_633uuy633uuy633u.png": "每天都是一个新的机会。\n用微笑开始它。\n世界也会向你微笑。"
}
@@ -0,0 +1,14 @@
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Paddle en+ru+ch",0.378,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.048,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.101,0.120,0.796,19.5,0.054799,15b40b8cdf9e97558c66d4acdcb97d46769e64f04f143f8061746bc731fbbc21,no_openai_signals_detected,source detected in same browser sequence
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.081,0.119,0.796,19.5,0.070267,5a8a0bcbe5763d1b1af0d64149cec7e35b16c5751e995d3bacf17581d64177ec,not_checked,fidelity-only follow-up
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.123,0.113,0.808,21.2,0.041279,900def5a433c039aa474789eddc4dc4c63b8126ba1aff6829d2e8d600b9f9c71,no_openai_signals_detected,source detected immediately afterward in the same Chrome sequence
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Paddle en+ru+ch",0.413,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.086,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.112,0.103,0.780,20.8,0.054912,846f39db0ad4a48a734a5019f0b0ff5d110eecb3210223407e72529dfa4fcf54,no_openai_signals_detected,source detected in same browser sequence
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.147,0.104,0.779,20.8,0.072556,8ae76e1dedeec26bebb8d35e71fd92d77f142dbb0f3440b330cdf33b79558afe,not_checked,fidelity-only follow-up
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.119,0.104,0.779,20.8,0.057110,ac4201bd2e342b1a37737319438cedb2203f6bb58cd9113f825cf9b80d913755,no_openai_signals_detected,source detected in same browser sequence
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-selective-automatic-passthrough,"Paddle detector ch; horizontal-only padded recognizer selection",0.074,0.289,0.696,23.3,0.000000,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_checked,all three source and candidate recognitions agreed exactly; output is byte-identical to candidate
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.127,0.113,0.805,20.9,0.041280,70c52fed8b8a1d98f4cda8558ae08825313e510dfb372e448370d8dcf1da73b0,not_checked,fidelity-only automatic-box follow-up; same 8/20 preserve decisions
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.154,0.104,0.778,20.8,0.057146,1fa01ce706042c003a61429d5d97fc6c928ec32278071a91926d588f1e760b00,not_checked,fidelity-only automatic-box follow-up; same 7/20 preserve decisions but worse CER
1 date source_commit fixture variant ocr_route text_cer img_lpips ssim psnr_db mask_fraction output_sha256 provider_oracle oracle_control
2 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage Paddle en+ru+ch 0.378 0.107 0.832 24.4 bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd no_openai_signals_detected source-positive control established earlier
3 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-restored Paddle en+ru+ch 0.048 0.174 0.703 13.9 0.154 3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6 no_openai_signals_detected qwen-zimage clean in same run
4 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-selective Paddle en+ru+ch 0.101 0.120 0.796 19.5 0.054799 15b40b8cdf9e97558c66d4acdcb97d46769e64f04f143f8061746bc731fbbc21 no_openai_signals_detected source detected in same browser sequence
5 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-selective-expanded-mask Paddle en+ru+ch 0.081 0.119 0.796 19.5 0.070267 5a8a0bcbe5763d1b1af0d64149cec7e35b16c5751e995d3bacf17581d64177ec not_checked fidelity-only follow-up
6 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-selective-automatic Paddle detector en+ru+ch; padded recognizer selection 0.123 0.113 0.808 21.2 0.041279 900def5a433c039aa474789eddc4dc4c63b8126ba1aff6829d2e8d600b9f9c71 no_openai_signals_detected source detected immediately afterward in the same Chrome sequence
7 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage Paddle en+ru+ch 0.413 0.095 0.818 26.0 83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713 no_openai_signals_detected source-positive control established earlier
8 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-restored Paddle en+ru+ch 0.086 0.162 0.682 14.6 0.170 95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d no_openai_signals_detected qwen-zimage clean in same run
9 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-selective Paddle en+ru+ch 0.112 0.103 0.780 20.8 0.054912 846f39db0ad4a48a734a5019f0b0ff5d110eecb3210223407e72529dfa4fcf54 no_openai_signals_detected source detected in same browser sequence
10 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-selective-expanded-mask Paddle en+ru+ch 0.147 0.104 0.779 20.8 0.072556 8ae76e1dedeec26bebb8d35e71fd92d77f142dbb0f3440b330cdf33b79558afe not_checked fidelity-only follow-up
11 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-selective-automatic Paddle detector en+ru+ch; padded recognizer selection 0.119 0.104 0.779 20.8 0.057110 ac4201bd2e342b1a37737319438cedb2203f6bb58cd9113f825cf9b80d913755 no_openai_signals_detected source detected in same browser sequence
12 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage-text-selective-automatic-passthrough Paddle detector ch; horizontal-only padded recognizer selection 0.074 0.289 0.696 23.3 0.000000 4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6 not_checked all three source and candidate recognitions agreed exactly; output is byte-identical to candidate
13 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-selective-detected-boxes Paddle ch detector boxes; 0.10-height recognizer padding 0.127 0.113 0.805 20.9 0.041280 70c52fed8b8a1d98f4cda8558ae08825313e510dfb372e448370d8dcf1da73b0 not_checked fidelity-only automatic-box follow-up; same 8/20 preserve decisions
14 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-selective-detected-boxes Paddle ch detector boxes; 0.10-height recognizer padding 0.154 0.104 0.778 20.8 0.057146 1fa01ce706042c003a61429d5d97fc6c928ec32278071a91926d588f1e760b00 not_checked fidelity-only automatic-box follow-up; same 7/20 preserve decisions but worse CER
+51
View File
@@ -0,0 +1,51 @@
{
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": [
{"box": [156, 206, 709, 296], "text": "This is a longer", "script": "alphabetic"},
{"box": [157, 293, 939, 379], "text": "sample text in English.", "script": "alphabetic"},
{"box": [165, 393, 774, 434], "text": "Typography can flow smoothly from", "script": "alphabetic"},
{"box": [161, 434, 521, 477], "text": "large to medium size.", "script": "alphabetic"},
{"box": [165, 495, 566, 522], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
{"box": [165, 524, 354, 550], "text": "rhythm, and clarity.", "script": "alphabetic"},
{"box": [165, 564, 448, 585], "text": "Even the finest text should remain", "script": "alphabetic"},
{"box": [165, 593, 330, 613], "text": "clean and readable.", "script": "alphabetic"},
{"box": [158, 673, 903, 740], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
{"box": [162, 753, 697, 793], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
{"box": [164, 807, 524, 829], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
{"box": [163, 845, 483, 868], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
{"box": [162, 951, 780, 1009], "text": "Это более длинный пример", "script": "alphabetic"},
{"box": [161, 1019, 723, 1062], "text": "текста на русском языке.", "script": "alphabetic"},
{"box": [165, 1076, 639, 1104], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
{"box": [165, 1111, 505, 1133], "text": "крупного размера к среднему.", "script": "alphabetic"},
{"box": [165, 1151, 513, 1172], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
{"box": [167, 1178, 287, 1192], "text": "ритм и ясность.", "script": "alphabetic"},
{"box": [165, 1210, 417, 1227], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
{"box": [165, 1235, 381, 1247], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
],
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": [
{"box": [165, 204, 754, 299], "text": "This is a longer", "script": "alphabetic"},
{"box": [167, 295, 1006, 385], "text": "sample text in English.", "script": "alphabetic"},
{"box": [171, 401, 831, 446], "text": "Typography can flow smoothly from", "script": "alphabetic"},
{"box": [169, 446, 560, 491], "text": "large to medium size.", "script": "alphabetic"},
{"box": [173, 507, 605, 536], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
{"box": [173, 538, 379, 567], "text": "rhythm, and clarity.", "script": "alphabetic"},
{"box": [175, 583, 481, 603], "text": "Even the finest text should remain", "script": "alphabetic"},
{"box": [173, 611, 352, 632], "text": "clean and readable.", "script": "alphabetic"},
{"box": [170, 693, 956, 757], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
{"box": [174, 775, 740, 812], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
{"box": [175, 829, 561, 852], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
{"box": [174, 869, 534, 892], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
{"box": [169, 958, 824, 1022], "text": "Это более длинный пример", "script": "alphabetic"},
{"box": [171, 1029, 764, 1074], "text": "текста на русском языке.", "script": "alphabetic"},
{"box": [175, 1090, 678, 1119], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
{"box": [175, 1125, 546, 1151], "text": "крупного размера к среднему.", "script": "alphabetic"},
{"box": [175, 1169, 552, 1192], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
{"box": [175, 1196, 312, 1217], "text": "ритм и ясность.", "script": "alphabetic"},
{"box": [175, 1233, 479, 1253], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
{"box": [177, 1259, 440, 1272], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
],
"Gemini_Generated_Image_633uuy633uuy633u.png": [
{"box": [1281, 650, 2357, 818], "text": "每天都是一个新的机会。", "script": "cjk"},
{"box": [1466, 788, 2167, 957], "text": "用微笑开始它。", "script": "cjk"},
{"box": [1367, 910, 2264, 1112], "text": "世界也会向你微笑。", "script": "cjk"}
]
}
@@ -0,0 +1,25 @@
date,source_commit,fixture,profile,route,seed,strength,adaptive_polish,text_cer,ocr_floor,added_cer,img_lpips,ssim,psnr_db,output_sha256,oracle_rechecked
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.378,0.127,0.251,0.107,0.832,24.4,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.356,0.127,0.229,0.125,0.707,25.3,9bb932d6e66b52f34ff07c3fcdbdb9c33eb601ae2cd84bec6b8c7b5e30df9926,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.413,0.127,0.286,0.095,0.818,26.0,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.422,0.127,0.295,0.099,0.754,25.9,7605b1175b0c99f481ae6929acc488f34d98bc0f2571d42a1d2972b671d9b8cb,false
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,original,reference,,,,0.037,0.037,0.000,0.000,1.000,inf,,not_applicable
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,visible-invisible-metadata,0,0.153999996,false,0.074,0.037,0.037,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,false
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,sdxl-zimage,visible-invisible-metadata,0,0.25,true,0.074,0.037,0.037,0.176,0.769,26.1,d9f89465c31bd2d38ffb111775b9bbea4577f40713fcbebf421e05d344ab27ad,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.068,0.127,-0.059,0.042,0.874,31.4,534de6422a3a03cfde44377aa9bf824a8267db3b73511240f8f07fb8dcb95eff,true
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.086,0.127,-0.041,0.047,0.862,30.8,4dba8211e8d84ebbfff72de00f54d4365d0d202394acd412d1419ae93d1e9f71,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.177,0.127,0.050,0.065,0.838,29.1,fc45c5fff6d33f15d30e05dd02580fe0addd97098b735b2894015f9abdaa43a7,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.248,0.127,0.121,0.093,0.813,27.3,1e55e11f0e289a752098370e9f1d53fed6cd4fab7ceff46fe86dd20c927f9ffb,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.343,0.127,0.216,0.120,0.787,25.7,f5a2d2829e88a790cff5bedc50fca1f878a92cdb392d530afb49d0d95c23b884,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.143,0.127,0.016,0.042,0.877,31.5,5cfa466e276ab37cf532f7458d6dd86ea4f0807e2cbdb6788b7c01ca6ee35a1a,true
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.101,0.127,-0.026,0.047,0.864,30.9,ade869ec7c5b08098495d9b9d67ee900773f189f94bbb411b043f242f5c2fa37,true
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.208,0.127,0.081,0.060,0.839,29.3,a11976963e3f2ca856ce66278e22064d51345d449fa6d672029039a79517d60a,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.341,0.127,0.214,0.070,0.819,27.6,1ab1ae075565e9d39960a80cf5d61e5b0c50d7e23283493ea0277056b3e964ab,false
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.538,0.127,0.411,0.084,0.799,25.6,5ff5a364d84f738de4c56ead7d4950a2f9a1aaf4bec16e97bda6a65915744a57,false
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.08,false,0.074,0.037,0.037,0.068,0.865,30.8,81419eddc3e5150956859dd10c26dd841bb93d9c8b34c01f344e6d69bca6edbf,true
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.10,false,0.074,0.037,0.037,0.086,0.844,29.8,4d1f32bd0dac0e05885ff943e7c10230b06de7d8547e6215f8e2941f732a1df5,true
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.15,false,0.000,0.037,-0.037,0.135,0.793,27.6,526c7987b34896f9513f7439b63b8a87573b59d15930fa307f3bb924435f3cec,true
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.20,false,0.000,0.037,-0.037,0.179,0.746,25.9,325364cf7a3b5d39ac0c44af82181753ca4d6fde0f1b522c896414191725e8fc,true
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.25,false,0.000,0.037,-0.037,0.218,0.703,24.5,f01d64dc0cabd45d66a93c1be39b819b988f29cda1f0a7bc0d5a1954571d807e,true
1 date source_commit fixture profile route seed strength adaptive_polish text_cer ocr_floor added_cer img_lpips ssim psnr_db output_sha256 oracle_rechecked
2 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png original reference 0.127 0.127 0.000 0.000 1.000 inf not_applicable
3 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage visible-invisible-metadata 0 0.11020972541176471 false 0.378 0.127 0.251 0.107 0.832 24.4 bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd false
4 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png sdxl-zimage visible-invisible-metadata 0 0.15 true 0.356 0.127 0.229 0.125 0.707 25.3 9bb932d6e66b52f34ff07c3fcdbdb9c33eb601ae2cd84bec6b8c7b5e30df9926 false
5 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png original reference 0.127 0.127 0.000 0.000 1.000 inf not_applicable
6 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage visible-invisible-metadata 0 0.11020972541176471 false 0.413 0.127 0.286 0.095 0.818 26.0 83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713 false
7 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png sdxl-zimage visible-invisible-metadata 0 0.15 true 0.422 0.127 0.295 0.099 0.754 25.9 7605b1175b0c99f481ae6929acc488f34d98bc0f2571d42a1d2972b671d9b8cb false
8 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png original reference 0.037 0.037 0.000 0.000 1.000 inf not_applicable
9 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage visible-invisible-metadata 0 0.153999996 false 0.074 0.037 0.037 0.289 0.696 23.3 4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6 false
10 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png sdxl-zimage visible-invisible-metadata 0 0.25 true 0.074 0.037 0.037 0.176 0.769 26.1 d9f89465c31bd2d38ffb111775b9bbea4577f40713fcbebf421e05d344ab27ad false
11 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.08 false 0.068 0.127 -0.059 0.042 0.874 31.4 534de6422a3a03cfde44377aa9bf824a8267db3b73511240f8f07fb8dcb95eff true
12 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.10 false 0.086 0.127 -0.041 0.047 0.862 30.8 4dba8211e8d84ebbfff72de00f54d4365d0d202394acd412d1419ae93d1e9f71 false
13 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.15 false 0.177 0.127 0.050 0.065 0.838 29.1 fc45c5fff6d33f15d30e05dd02580fe0addd97098b735b2894015f9abdaa43a7 false
14 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.20 false 0.248 0.127 0.121 0.093 0.813 27.3 1e55e11f0e289a752098370e9f1d53fed6cd4fab7ceff46fe86dd20c927f9ffb false
15 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png zimage-global visible-invisible-metadata 0 0.25 false 0.343 0.127 0.216 0.120 0.787 25.7 f5a2d2829e88a790cff5bedc50fca1f878a92cdb392d530afb49d0d95c23b884 false
16 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.08 false 0.143 0.127 0.016 0.042 0.877 31.5 5cfa466e276ab37cf532f7458d6dd86ea4f0807e2cbdb6788b7c01ca6ee35a1a true
17 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.10 false 0.101 0.127 -0.026 0.047 0.864 30.9 ade869ec7c5b08098495d9b9d67ee900773f189f94bbb411b043f242f5c2fa37 true
18 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.15 false 0.208 0.127 0.081 0.060 0.839 29.3 a11976963e3f2ca856ce66278e22064d51345d449fa6d672029039a79517d60a false
19 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.20 false 0.341 0.127 0.214 0.070 0.819 27.6 1ab1ae075565e9d39960a80cf5d61e5b0c50d7e23283493ea0277056b3e964ab false
20 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png zimage-global visible-invisible-metadata 0 0.25 false 0.538 0.127 0.411 0.084 0.799 25.6 5ff5a364d84f738de4c56ead7d4950a2f9a1aaf4bec16e97bda6a65915744a57 false
21 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.08 false 0.074 0.037 0.037 0.068 0.865 30.8 81419eddc3e5150956859dd10c26dd841bb93d9c8b34c01f344e6d69bca6edbf true
22 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.10 false 0.074 0.037 0.037 0.086 0.844 29.8 4d1f32bd0dac0e05885ff943e7c10230b06de7d8547e6215f8e2941f732a1df5 true
23 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.15 false 0.000 0.037 -0.037 0.135 0.793 27.6 526c7987b34896f9513f7439b63b8a87573b59d15930fa307f3bb924435f3cec true
24 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.20 false 0.000 0.037 -0.037 0.179 0.746 25.9 325364cf7a3b5d39ac0c44af82181753ca4d6fde0f1b522c896414191725e8fc true
25 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png zimage-global visible-invisible-metadata 0 0.25 false 0.000 0.037 -0.037 0.218 0.703 24.5 f01d64dc0cabd45d66a93c1be39b819b988f29cda1f0a7bc0d5a1954571d807e true
@@ -0,0 +1,7 @@
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.338,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.305,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,"Paddle ch",0.074,0.289,0.696,23.3,,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,synthid_detected,inconclusive account: clean baseline not reproduced
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-restored,"Paddle ch",0.111,0.328,0.653,18.1,0.051,2ab7666a4161ff0e12c53d9412b23d013970c7613a8a55a1b1ae8a2744e3290d,synthid_detected,inconclusive account: qwen-zimage control also detected
1 date source_commit fixture variant ocr_route text_cer img_lpips ssim psnr_db mask_fraction output_sha256 provider_oracle oracle_control
2 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage Vision en+ru; Paddle ch 0.338 0.107 0.832 24.4 bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd no_openai_signals_detected source-positive control established earlier
3 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_02_23 PM.png qwen-zimage-text-restored Vision en+ru; Paddle ch 0.007 0.174 0.703 13.9 0.154 3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6 no_openai_signals_detected qwen-zimage clean in same run
4 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage Vision en+ru; Paddle ch 0.305 0.095 0.818 26.0 83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713 no_openai_signals_detected source-positive control established earlier
5 2026-08-13 2d03a00 ChatGPT Image May 31, 2026, 02_03_55 PM.png qwen-zimage-text-restored Vision en+ru; Paddle ch 0.007 0.162 0.682 14.6 0.170 95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d no_openai_signals_detected qwen-zimage clean in same run
6 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage Paddle ch 0.074 0.289 0.696 23.3 4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6 synthid_detected inconclusive account: clean baseline not reproduced
7 2026-08-13 2d03a00 Gemini_Generated_Image_633uuy633uuy633u.png qwen-zimage-text-restored Paddle ch 0.111 0.328 0.653 18.1 0.051 2ab7666a4161ff0e12c53d9412b23d013970c7613a8a55a1b1ae8a2744e3290d synthid_detected inconclusive account: qwen-zimage control also detected
@@ -0,0 +1,6 @@
date,upstream_commit,workflow_sha256,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle,implementation
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/before/12.png,official-before,,,,10,,,,,,,056276464387c883f5fd89bd2f1415faeb425abb3153cd10efb25d1b9053bdc2,synthid_detected,upstream-published-bytes
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/after/12.png,official-after,,,,10,0.975,0.093,0.354,0.172,0.820,23.8,a822d805666224adb3e360b3ee89e14f20a341f1e7c4d3a5d3d5251e9286aefe,clean,upstream-published-bytes
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected,project-runtime-control
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-global-close-reproduction,0,0.153999996,,18,0.589,0.217,0.331,0.286,0.647,24.3,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean,diffsynth-bf16-yunet-sam
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-full-close-reproduction,0,0.153999996,0.13209025065104166,18,0.783,0.083,0.470,0.260,0.659,24.7,2d7a8f76f98b905f14d69e5f9e11979610c72c6be7b6ac298dec4659f17bd819,clean,diffsynth-bf16-yunet-sam
1 date upstream_commit workflow_sha256 fixture route seed global_strength face_strength face_count id_cos face_lpips lapvar_ratio img_lpips ssim psnr_db output_sha256 gemini_oracle implementation
2 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 comparison/before/12.png official-before 10 056276464387c883f5fd89bd2f1415faeb425abb3153cd10efb25d1b9053bdc2 synthid_detected upstream-published-bytes
3 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 comparison/after/12.png official-after 10 0.975 0.093 0.354 0.172 0.820 23.8 a822d805666224adb3e360b3ee89e14f20a341f1e7c4d3a5d3d5251e9286aefe clean upstream-published-bytes
4 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 Gemini_Generated_Image_y48j3cy48j3cy48j.png visible-metadata-control 0 18 9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b synthid_detected project-runtime-control
5 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 Gemini_Generated_Image_y48j3cy48j3cy48j.png upstream-global-close-reproduction 0 0.153999996 18 0.589 0.217 0.331 0.286 0.647 24.3 281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f clean diffsynth-bf16-yunet-sam
6 2026-08-13 3007d0351596ae0a78b7074dae7ad179710b1e48 41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6 Gemini_Generated_Image_y48j3cy48j3cy48j.png upstream-full-close-reproduction 0 0.153999996 0.13209025065104166 18 0.783 0.083 0.470 0.260 0.659 24.7 2d7a8f76f98b905f14d69e5f9e11979610c72c6be7b6ac298dec4659f17bd819 clean diffsynth-bf16-yunet-sam
@@ -0,0 +1,3 @@
date,source_url,source_sha256,source_width,source_height,source_fps,duration_seconds,source_verdict,vae,noise_std,long_side,fps,seed,output_sha256,output_verdict,output_verdict_text,output_detected_range,track,session_id,stratum,psnr_db,temporal_residual_ratio
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
1 date source_url source_sha256 source_width source_height source_fps duration_seconds source_verdict vae noise_std long_side fps seed output_sha256 output_verdict output_verdict_text output_detected_range track session_id stratum psnr_db temporal_residual_ratio
2 2026-07-31 https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4 79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d 1920 1080 24 8 detected_all_frames 0.10 512 12 0 079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3 detected 26.2932 1.0072
3 2026-07-31 https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4 79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d 1920 1080 24 8 detected_all_frames 0.15 512 12 0 1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e not_detected 25.3911 1.0578
+15
View File
@@ -0,0 +1,15 @@
# Provenance fixtures
These files exercise real container, metadata, and pixel-provenance formats.
They are test inputs, not evaluation corpora.
## Adobe TrustMark
`adobe-trustmark-p.png` is Adobe's official `images/ghost_P.png` Variant P
example from [`adobe/trustmark`](https://github.com/adobe/trustmark) at commit
`0ed40cbe8188f664fd9cbbeacd969807de27440a`.
- Source: `https://raw.githubusercontent.com/adobe/trustmark/0ed40cbe8188f664fd9cbbeacd969807de27440a/images/ghost_P.png`
- SHA-256: `e58c5825ed7e5d9fb04710ea541b61bd55879cad65554c0d46260aa24b3d0755`
- Expected signal: TrustMark Variant P, schema 1
- License: MIT, reproduced in `../licenses/adobe-trustmark-MIT.txt`
Binary file not shown.

After

Width:  |  Height:  |  Size: 360 KiB

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Binary file not shown.

After

Width:  |  Height:  |  Size: 495 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 MiB

File renamed without changes.
File renamed without changes.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 715 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 373 KiB

+25
View File
@@ -0,0 +1,25 @@
Copyright 2023 Adobe
All Rights Reserved.
NOTICE: Adobe permits you to use, modify, and distribute this file in
accordance with the terms of the license agreement accompanying it.
MIT License
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.7 MiB

-41
View File
@@ -1,41 +0,0 @@
# OpenAI ChatGPT Images 2.0 sample
Reference image generated by OpenAI's ChatGPT Images 2.0 (`gpt-image-2`, launched 2026-04-21). Kept in the repo so the C2PA manifest parser and invisible-watermark pipeline can be re-verified against a real production output.
## `amur-leopard.png`
- Resolution: 1055 x 1491, PNG, 3.0 MB
- Downloaded: 2026-04-22
- Content: AI-generated infographic about the Amur leopard (*Panthera pardus orientalis*), chosen to exercise the model's new accurate-text-rendering feature
### Embedded C2PA manifest (caBX chunk, 23607 bytes)
Parsed by `remove-ai-watermarks metadata --check`:
| Field | Value |
| --- | --- |
| `claim_generator` | GPT-4o |
| `c2pa_spec` | 2.2.0 |
| `digital_source_type` | http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia |
| `c2pa_actions` | created, converted |
| Signer | OpenAI OpCo, LLC / OpenAI Media Service API |
| Claim signing CA | Trufo C2PA Claim Signing CA (2025) -> Trufo C2PA Root CA (ECC P384) |
| Timestamp authority | OpenAI TSA Issuing CA -> OpenAI TSA Root CA |
The `trainedAlgorithmicMedia` tag is what triggers "Made with AI" labels on Instagram, Facebook, and X.
### Invisible pixel-level watermark
OpenAI's system card for Images 2.0 states the model embeds an "imperceptible, robust, and content-specific" pixel-level watermark alongside C2PA. This sample was downloaded 2026-04-22, before OpenAI's 19 May 2026 rollout of Google's SynthID watermark across ChatGPT / Codex / the API, so it likely predates SynthID and carries only the original content-specific watermark. Since that rollout, the openai.com/verify tool (in preview) is the public oracle for both signals; there is still no local decoder, so bypass cannot be verified empirically here without the oracle.
## Reproducing the removal
```bash
# C2PA strip only
remove-ai-watermarks metadata amur-leopard.png --remove -o amur-leopard.clean.png
# Full pipeline (visible + diffusion regeneration + metadata)
remove-ai-watermarks all amur-leopard.png -o amur-leopard.all.png --device mps
```
Note: the diffusion step runs at native resolution by default (no pre-downscale), so fine text on a text-heavy infographic like this one is preserved. Pass `--max-resolution N` only if a very large image OOMs the GPU/MPS (that reintroduces a lossy downscale then upscale round-trip).
Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.0 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 317 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 425 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 815 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 314 KiB

@@ -1,11 +1,10 @@
# SynthID reference corpus
# SynthID oracle fixtures
A locally-collected, labeled image corpus for SynthID work. Two downstream uses:
A compact set of externally verified originals for two uses:
1. **Per-resolution spectral codebook** for an experimental SynthID detector
(carrier frequencies are resolution-dependent, so labels must record the
exact native resolution).
2. **Removal regression set** — verify that our pipeline turns a SynthID-positive
1. **Signal and detector-limit research** with exact native resolutions and
recorded verification sources.
2. **Removal regression set**: verify that a pipeline turns a SynthID-positive
image into a negative one.
There is no reliable local detector of the SynthID pixel watermark (Google's
@@ -15,23 +14,40 @@ external oracle, recorded per image in `verified_via` (see below).
## Layout
```
data/synthid_corpus/
README.md # this protocol (committed)
manifest.csv # labels + provenance (committed; one row per tracked image)
images/ # the labeled corpus (committed)
pos/ # SynthID present
neg/ # SynthID absent (incl. reviewed real photos)
cleaned/ # our pipeline output from a pos image
refs/ # synthetic black/white calibration fills (gitignored, regenerable)
data/synthid/
README.md # collection and verification protocol
manifest.csv # one row per committed original
full-pipeline-quality.csv # reusable evaluation selection
originals/ # one canonical copy of each verified source
```
The labeled images are committed so the corpus is reproducible and the removal
regression set runs in CI. `manifest.csv` is kept in sync with the files on
disk (one row per tracked image; dangling rows are pruned when files are
removed). Before adding any image, confirm it carries no private or
identifiable content you would not publish -- this is a public repo and git
history is permanent. The synthetic `refs/` fills stay gitignored (regenerable,
not part of the labeled set).
The originals are committed so the evaluation set is reproducible and its
hashes can be checked in CI. `manifest.csv` is kept in sync with
`originals/`, one row per file. Generated outputs do not belong here. Store
them outside the repository and record only a reproducible verdict in the
evaluation manifest or documentation.
## Reusable removal-quality set
`full-pipeline-quality.csv` is the canonical reusable input set for
full-pipeline visual-quality and watermark-removal tests. Its fixtures are
cleared for permanent public test reuse. Tests, benchmarks, and manual
evaluation bundles may read these images repeatedly without requesting new
permission.
The CSV preserves each platform's original filename and maps it to the one
canonical corpus copy. Use `corpus_path` to read the input, but preserve
`source_filename` in generated result names and keep OpenAI and Gemini outputs
in separate provider groups. This avoids duplicating large PNGs while retaining
the names needed for the corresponding provider oracle.
Rows with a dated negative `final_clean_oracle` were checked after the complete
`visible -> qwen-zimage -> metadata` route with the matching provider oracle.
`not-checked-in-final-candidate` means the input is available as an additional
quality stress fixture, but a newly generated output still needs the matching
oracle before claiming removal. The checked result applies to those exact
output bytes; re-run the oracle after changing the pipeline, seed, model
versions, or runtime settings.
## Verification levels (`verified_via`)
@@ -40,37 +56,31 @@ Ground-truth quality, strongest first:
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
- `synthid-portal` — checked via Google's SynthID Detector portal.
- `c2pa-metadata` — issuer-only proxy (Google/OpenAI C2PA manifest present). Weaker: the C2PA can be stripped while the pixel watermark remains.
- `c2pa-metadata` — supported provenance evidence (Google AI C2PA, or OpenAI
C2PA with an explicit `c2pa.watermarked.*` action). Weaker than a provider
oracle: C2PA can be stripped while the pixel watermark remains.
- `third-party` — label asserted by an external dataset, not independently verified.
- `none` — unverified.
Prefer `gemini-app` for any image that will train the codebook or gate a test.
Prefer `gemini-app` for any Google image that will gate a test.
## What to collect
For the **codebook** (per target resolution, e.g. 1024x1024, 1024x1536, 1536x2816):
- 30-50+ SynthID-positive outputs per resolution (more is better; ~150-200 per
resolution materially improves carrier discovery).
- At each target resolution, also a batch of **pure-black (#000000)** and
**pure-white (#FFFFFF)** fills generated by the SynthID model — these isolate
the content-independent carrier (the watermark is most of the signal there).
For the **regression set**:
- A handful of `pos` images, their `cleaned` counterparts (run through our
pipeline), and the cleaned re-verified via `gemini-app` (should read negative).
- `neg` controls: non-AI photos and outputs from non-SynthID models (SD,
Midjourney, Firefly) verified negative.
- A small number of positive originals with a recorded provider oracle.
- A quality-set row for every original used by repeatable evaluation.
- Re-run the provider oracle on each newly generated output. Do not commit the
generated output as another corpus copy.
The corpus is committed to a public repo: review every image before adding it
and keep out anything private or identifiable you would not publish.
## Ingesting
Use `scripts/synthid_corpus.py` — it copies a file in, records its sha256,
resolution, format, and C2PA issuer (via our own detector), and appends a row
to `manifest.csv`:
Use `scripts/synthid_corpus.py` to copy a file into `originals/`, record its
SHA-256, resolution, format, and C2PA issuer through our detector, and append a
row to `manifest.csv`:
```bash
uv run python scripts/synthid_corpus.py ingest path/to/*.png \
@@ -82,6 +92,10 @@ uv run python scripts/synthid_corpus.py status # counts by label / resolution
## Autonomous collection via Chrome MCP
> Historical collection notes. Browser interfaces and download behavior can
> change independently of this repository. These steps do not define package
> behavior.
Generation can be driven through the browser (the account must be logged in):
- **Gemini** (`gemini.google.com`): type `Create an image: <prompt>`, wait, hover the
@@ -122,8 +136,8 @@ Download; Leonardo serves the original JPEG in-chat (download button matches).
## Per-platform watermark map (observed, May 2026)
What each platform actually embeds, verified by byte-scan (and Gemini-app oracle
where noted). The detector's coverage is complementary: metadata catches C2PA /
This is a dated observation table, not a live provider compatibility promise.
The detector's coverage is complementary: metadata catches C2PA /
IPTC; `exif_generator` catches EXIF `Make`/`Software` + XMP `CreatorTool`;
`invisible_watermark.py` (imwatermark) catches the open SD/SDXL/FLUX DWT-DCT
watermark on pristine files; the visible detector catches the Gemini-family
+8
View File
@@ -0,0 +1,8 @@
provider,source_filename,corpus_path,sha256,original_signal_verified_via,final_clean_oracle
openai,"ChatGPT Image May 30, 2026, 10_31_08 AM.png","originals/ChatGPT Image May 30, 2026, 10_31_08 AM.png",28ff8732b037f98a4ef5bc277bbcdaa32e5eb9ccbd00b6c8c616e46ef68ae8a0,openai-verify,openai-verify-negative-2026-07-25
openai,"ChatGPT Image May 31, 2026, 02_02_23 PM.png","originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png",228e54413a3ef62b313176dcdba774a56a58c36b54227d1440bc26cbe0aefd82,c2pa-metadata,not-checked-in-final-candidate
openai,"ChatGPT Image May 31, 2026, 02_03_55 PM.png","originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png",05b836ecfe40fd689177fda74384ae4fdcc446505bbc4281cd3cbb6523eb669e,openai-verify,openai-verify-negative-2026-07-25
gemini,"Gemini_Generated_Image_3mc4t93mc4t93mc4.png","originals/Gemini_Generated_Image_3mc4t93mc4t93mc4.png",2c33e75a2db614ce74c83cc0a6ac6c3ac735aca83ab88c9c9345843b124f7856,gemini-app,gemini-app-negative-2026-07-25
gemini,"Gemini_Generated_Image_633uuy633uuy633u.png","originals/Gemini_Generated_Image_633uuy633uuy633u.png",4affd7f27767a445db6abf741355743ba8d95108ad922c9fff045feed8492236,gemini-app,gemini-app-negative-2026-07-25
gemini,"Gemini_Generated_Image_akdbeiakdbeiakdb.png","originals/Gemini_Generated_Image_akdbeiakdbeiakdb.png",8c1a6fb03ef3d45a1f958fb3401e4264e409ff88c2a793061db7f29023454d0e,gemini-app,gemini-app-negative-2026-07-25
gemini,"Gemini_Generated_Image_y48j3cy48j3cy48j.png","originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png",45d79a683134fcba1b147b2aedb669783d474e1fb8a4df329729a0904fd1b46b,gemini-app,gemini-app-negative-2026-07-25
1 provider source_filename corpus_path sha256 original_signal_verified_via final_clean_oracle
2 openai ChatGPT Image May 30, 2026, 10_31_08 AM.png originals/ChatGPT Image May 30, 2026, 10_31_08 AM.png 28ff8732b037f98a4ef5bc277bbcdaa32e5eb9ccbd00b6c8c616e46ef68ae8a0 openai-verify openai-verify-negative-2026-07-25
3 openai ChatGPT Image May 31, 2026, 02_02_23 PM.png originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png 228e54413a3ef62b313176dcdba774a56a58c36b54227d1440bc26cbe0aefd82 c2pa-metadata not-checked-in-final-candidate
4 openai ChatGPT Image May 31, 2026, 02_03_55 PM.png originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png 05b836ecfe40fd689177fda74384ae4fdcc446505bbc4281cd3cbb6523eb669e openai-verify openai-verify-negative-2026-07-25
5 gemini Gemini_Generated_Image_3mc4t93mc4t93mc4.png originals/Gemini_Generated_Image_3mc4t93mc4t93mc4.png 2c33e75a2db614ce74c83cc0a6ac6c3ac735aca83ab88c9c9345843b124f7856 gemini-app gemini-app-negative-2026-07-25
6 gemini Gemini_Generated_Image_633uuy633uuy633u.png originals/Gemini_Generated_Image_633uuy633uuy633u.png 4affd7f27767a445db6abf741355743ba8d95108ad922c9fff045feed8492236 gemini-app gemini-app-negative-2026-07-25
7 gemini Gemini_Generated_Image_akdbeiakdbeiakdb.png originals/Gemini_Generated_Image_akdbeiakdbeiakdb.png 8c1a6fb03ef3d45a1f958fb3401e4264e409ff88c2a793061db7f29023454d0e gemini-app gemini-app-negative-2026-07-25
8 gemini Gemini_Generated_Image_y48j3cy48j3cy48j.png originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png 45d79a683134fcba1b147b2aedb669783d474e1fb8a4df329729a0904fd1b46b gemini-app gemini-app-negative-2026-07-25
+8
View File
@@ -0,0 +1,8 @@
sha256,filename,label,source,model,width,height,format,c2pa_issuer,synthid_metadata,verified_via,added,notes
05b836ecfe40fd689177fda74384ae4fdcc446505bbc4281cd3cbb6523eb669e,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",pos,ChatGPT,gpt-image,1122,1402,png,OpenAI,yes,openai-verify,2026-06-04T00:07:53Z,June 2026 strength-study subject; openai.com/verify: SynthID detected (docs/synthid.md 2.2)
28ff8732b037f98a4ef5bc277bbcdaa32e5eb9ccbd00b6c8c616e46ef68ae8a0,"ChatGPT Image May 30, 2026, 10_31_08 AM.png",pos,ChatGPT,gpt-image,1448,1086,png,OpenAI,yes,openai-verify,2026-06-04T00:07:53Z,June 2026 strength-study subject; openai.com/verify: SynthID detected (docs/synthid.md 2.2)
4affd7f27767a445db6abf741355743ba8d95108ad922c9fff045feed8492236,Gemini_Generated_Image_633uuy633uuy633u.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,gemini-app,2026-06-04T00:08:05Z,June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
8c1a6fb03ef3d45a1f958fb3401e4264e409ff88c2a793061db7f29023454d0e,Gemini_Generated_Image_akdbeiakdbeiakdb.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,gemini-app,2026-06-04T00:08:05Z,June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
45d79a683134fcba1b147b2aedb669783d474e1fb8a4df329729a0904fd1b46b,Gemini_Generated_Image_y48j3cy48j3cy48j.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,gemini-app,2026-06-04T00:08:05Z,June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
2c33e75a2db614ce74c83cc0a6ac6c3ac735aca83ab88c9c9345843b124f7856,Gemini_Generated_Image_3mc4t93mc4t93mc4.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,gemini-app,2026-06-04T00:08:05Z,June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
228e54413a3ef62b313176dcdba774a56a58c36b54227d1440bc26cbe0aefd82,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",pos,ChatGPT,gpt-image,1122,1402,png,OpenAI,yes,c2pa-metadata,2026-07-25T20:21:33Z,Reusable full-pipeline quality fixture; original filename preserved in quality-set manifest; cleared for public test reuse
1 sha256 filename label source model width height format c2pa_issuer synthid_metadata verified_via added notes
2 05b836ecfe40fd689177fda74384ae4fdcc446505bbc4281cd3cbb6523eb669e ChatGPT Image May 31, 2026, 02_03_55 PM.png pos ChatGPT gpt-image 1122 1402 png OpenAI yes openai-verify 2026-06-04T00:07:53Z June 2026 strength-study subject; openai.com/verify: SynthID detected (docs/synthid.md 2.2)
3 28ff8732b037f98a4ef5bc277bbcdaa32e5eb9ccbd00b6c8c616e46ef68ae8a0 ChatGPT Image May 30, 2026, 10_31_08 AM.png pos ChatGPT gpt-image 1448 1086 png OpenAI yes openai-verify 2026-06-04T00:07:53Z June 2026 strength-study subject; openai.com/verify: SynthID detected (docs/synthid.md 2.2)
4 4affd7f27767a445db6abf741355743ba8d95108ad922c9fff045feed8492236 Gemini_Generated_Image_633uuy633uuy633u.png pos Gemini app gemini 2816 1536 png Google LLC yes gemini-app 2026-06-04T00:08:05Z June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
5 8c1a6fb03ef3d45a1f958fb3401e4264e409ff88c2a793061db7f29023454d0e Gemini_Generated_Image_akdbeiakdbeiakdb.png pos Gemini app gemini 2816 1536 png Google LLC yes gemini-app 2026-06-04T00:08:05Z June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
6 45d79a683134fcba1b147b2aedb669783d474e1fb8a4df329729a0904fd1b46b Gemini_Generated_Image_y48j3cy48j3cy48j.png pos Gemini app gemini 2816 1536 png Google LLC yes gemini-app 2026-06-04T00:08:05Z June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
7 2c33e75a2db614ce74c83cc0a6ac6c3ac735aca83ab88c9c9345843b124f7856 Gemini_Generated_Image_3mc4t93mc4t93mc4.png pos Gemini app gemini 2816 1536 png Google LLC yes gemini-app 2026-06-04T00:08:05Z June 2026 strength-study subject; Gemini-app Verify with SynthID: detected (docs/synthid.md 2.2)
8 228e54413a3ef62b313176dcdba774a56a58c36b54227d1440bc26cbe0aefd82 ChatGPT Image May 31, 2026, 02_02_23 PM.png pos ChatGPT gpt-image 1122 1402 png OpenAI yes c2pa-metadata 2026-07-25T20:21:33Z Reusable full-pipeline quality fixture; original filename preserved in quality-set manifest; cleared for public test reuse
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.4 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.6 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.7 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.5 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.7 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.8 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.7 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.0 MiB

Loaded 100 of 298 files, more files were not shown because too many files have changed in this diff. Show more