Commit Graph
309 Commits
Author SHA1 Message Date
Victor KuznetsovandClaude Opus 4.8 836d87ed68 Mask faint text marks the tophat front-end detects but binarization loses
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 10:57:08 -07:00
Victor KuznetsovandClaude Opus 4.8 c150180acf Add full-surface verification harnesses and corpus sweep plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 08:14:50 -07:00
Victor KuznetsovandClaude Opus 4.8 cfefd9d819 Remove assume_ai, add tophat front-end and rival margin, fix two CLI defects
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 08:14:50 -07:00
Victor Kuznetsov a8f3536d3e Refactor watermark detection and provenance handling 2026-07-16 17:40:46 -07:00
Victor KuznetsovandClaude Opus 4.8 9618ac93c8 docs: record where the HF Space source lives and how it deploys
The Space's demo code (app.py etc.) lives in a SEPARATE private repo,
wiltodelta/raiw-hf-space, which nothing documented -- so finding it cost a long
detour through the Space's commit authorship and a hunt for a write token that
never existed locally. Record it, plus the deploy flow that replaced the old
web-UI editing: push to that repo's main -> sync-to-hf.yml mirrors the files via
HfApi.upload_folder (adds a commit on top of the Space history, never a force
push). Also call out the two automations that both touch the Space so they are
not confused: sync-to-hf.yml ships demo-code changes, while this repo's
distribute.yml factory-rebuilds the Space on a library release so its
`remove-ai-watermarks>=` pin re-resolves.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 10:24:46 -07:00
Victor KuznetsovandClaude Opus 4.8 71dae8fe25 chore(release): bump version to 0.15.3
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
v0.15.3
2026-07-14 17:17:01 +03:00
Victor KuznetsovandClaude Opus 4.8 9fe12c3997 docs: note content-based strip routing + SDXL watermarker fixes
Document both fixes in CLAUDE.md (the metadata.py and watermark_remover.py
bullets). Also add a --backend flag to the visible-removal audit script so a
realistic quality pass can run the production MI-GAN fill instead of cv2
(removal SUCCESS is backend-independent, but only migan/lama reflect the
recovered-region quality a user actually gets).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 17:16:56 +03:00
Victor KuznetsovandClaude Opus 4.8 183793fbf2 fix(invisible): disable diffusers SDXL watermarker on removal output
The SDXL removal pipelines (sdxl + controlnet) were built without
add_watermarker=False, so diffusers embedded its default open "Stable
Diffusion XL" DWT-DCT invisible watermark on every output whenever
invisible-watermark is installed (the detect extra). A watermark REMOVER was
therefore replacing one detectable AI watermark (SynthID) with another: the
cleaned output re-read as AI (identify -> "Open invisible watermark: Stable
Diffusion XL"), observed on the SynthID validation sample.

Both SDXL loaders now call a shared _disable_sdxl_watermarker helper (mirrors
_maybe_add_fp16_vae; the ControlNetModel sub-model and the Qwen loader never
call it, since only the pipeline accepts the kwarg). Verified end to end: the
affected outputs re-run clean (is_ai=None, no open watermark).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 17:16:49 +03:00
Victor KuznetsovandClaude Opus 4.8 7f22852593 fix(metadata): route strip output format by content, not extension
remove_ai_metadata chose its save format (and the lossless-JPEG fast path)
from the OUTPUT file extension. On the ~2% of real uploads whose extension
lies about their content (a PNG served as .jpg is the common case, ~0.9% of
the corpus), the default flow -- which inherits the source's own extension --
re-encoded a lossless PNG/WebP into a real JPEG, silently degrading the pixels
and breaking the "work with originals" invariant.

Sniff the actual container from magic bytes (_sniff_image_format, reusing the
12-byte head already read for the ISOBMFF check) and route on content: a
misnamed lossless source (source-extension format != content) is preserved in
its true format, while a correctly-named source still honors a deliberate
output-extension conversion (source.png -> output.jpg). The JPEG-lossless gate
is likewise content-gated.

Found by a new metadata-removal parity audit over the local corpus
(scripts/metadata_removal_audit.py): 18170/18173 carriers strip cleanly, and
this fix takes the 208 pixel-integrity failures (all misnamed PNGs) to 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 17:16:36 +03:00
Victor Kuznetsov d9204e2e1f chore(release): bump version to 0.15.2 v0.15.2 2026-07-13 12:27:06 +03:00
Victor KuznetsovandClaude Opus 4.8 2bdaa09e5f fix(metadata): strip bare AIGC in APP11 and AIGC in a standard PNG text chunk
Full-corpus strip audit surfaced 24 china_aigc survivors on real uploads:
- 19 JPEG carried the bare AIGC{...} blob in APP11 (0xEB). That marker's branch
  in _jpeg_app_carries_ai only tested for a C2PA/JUMBF manifest and RETURNED, so
  a bare AIGC there slipped past the generic AIGC check. The specific
  C2PA(APP11)/XMP(APP1)/IPTC(APP13) checks now fall through to the generic
  _is_aigc_exif_value drop, which runs for every APP marker they did not claim.
- PNG carried the {"AIGC":{...}} block in a STANDARD text chunk (Description).
  _is_ai_key keeps that key, so removal now also drops a text value carrying an
  AIGC block (_is_aigc_exif_value broadened to accept str; wired into the PNG
  re-save value filter), in parity with aigc_label's detection.

Verified on the corpus: decodable china_aigc survivors 27 -> 0; the 3 remaining
are truncated files the strip fail-safe (v0.15.1) copies through by design.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:27:05 +03:00
Victor Kuznetsov 72f6734b45 chore(release): bump version to 0.15.1 v0.15.1 2026-07-13 11:54:53 +03:00
Victor KuznetsovandClaude Opus 4.8 220803c4d0 fix(metadata): remove_ai_metadata is fail-safe on a truncated/corrupt image
PIL raises OSError decoding a truncated file, which crashed remove_ai_metadata
(the PNG/WebP PIL re-save path) -- a direct library caller like a web worker
500s on a partial upload. ~0.2% of the real upload corpus is truncated. The
strip now probes decodability first and, on failure, copies the input through
unchanged and returns rather than raising (we cannot strip what we cannot parse),
mirroring strip_c2pa_boxes' fail-safe. identify already handled these.

The CLI `metadata --remove` on an unreadable file therefore now exits 0 with the
input passed through, not a clean error (exit 1) -- `visible`, which must decode
to remove a mark, still exits 1. Test updated to the per-command contract.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 11:54:33 +03:00
Victor Kuznetsov 1dc6fee91a chore(release): bump version to 0.15.0 v0.15.0 2026-07-13 10:50:20 +03:00
Victor KuznetsovandClaude Opus 4.8 a4c901ff39 fix: metadata-strip parity, input robustness, and detection/clash coverage
Bug fixes (each with a regression test):
- metadata strip parity across every marker placement: IPTC digitalSourceType
  in XMP, the Samsung post-EOI trailer, the China TC260 AIGC block in EXIF
  UserComment, a bare AIGC block in a non-standard APP segment, and the ISOBMFF
  EXIF path (AIGC + xAI) are all now stripped -- anything a scanner flags, the
  strip reaches
- Samsung genAIType detected when its trailer sits past the 512 KB scan window
  (file-tail read on large photos)
- crashes on edge inputs: Gemini detector on images with a short side < 16px,
  footprint_mask on a zero-size ndarray, the humanizer on chromatic_shift >=
  width, and the CLI on unreadable/corrupt/empty input (clean error, not a
  traceback)
- WebP written losslessly (cv2 quality 101), not lossy at 100
- the IPTC digitalSourceType algorithmicMedia (procedural, not trained on
  sampled data) is no longer flagged as AI-generated, so clean procedural
  content is not scrubbed
- c2pa source-type: compositeWithTrainedAlgorithmicMedia is checked before the
  bare algorithmicMedia token, so an AI-enhanced composite is not misclassified

Detection:
- integrity-clash coverage now normalizes ByteDance / Canva / ElevenLabs /
  Black Forest Labs, so a transplanted manifest next to an independent
  conflicting stamp is caught; the generic China TC260 AIGC label is attributed
  to a co-present TC260 vendor, so a legit Doubao image (its own C2PA + TC260
  label) does not clash (corpus-validated: 0 new clashes on 5000 carriers)

CLI:
- batch exits non-zero (with a warning) when any image errors or a GPU-missing
  SynthID scrub is skipped, and copies the input through so the output dir stays
  complete -- it used to always exit 0 and could silently drop files

Perf:
- GeminiEngine reused as a process-wide singleton with a precomputed template
  ladder: -24% on the identify sparkle path, detection byte-identical

Internal: one shared _ai_exif_targets rule set feeds both EXIF scrubbers so
their coverage cannot drift; docs synced; maintain.sh hardened so the uv-secure
internal teardown crash no longer aborts the gate (still fails on a real finding).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 10:49:24 +03:00
Victor KuznetsovandClaude Opus 4.8 190dc89d23 perf(visible): crop MI-GAN around the mask so peak RAM is bounded by mark size
erase_migan fed the whole frame to the ONNX model, so peak RSS scaled with the
upload (~0.6 GB at 4 MP up to ~2.4 GB at 25 MP). Mirror erase_lama: crop a padded
region around the mask (pad = max(256, 2*bbox)), feed only that crop (at native
resolution -- MI-GAN accepts arbitrary dims, unlike LaMa's fixed 512 square), and
paste only masked pixels back. Peak RSS is now bounded by the mark size
(~0.6-0.9 GB), so a memory-tight host (a 1-2 GB web worker) can run MI-GAN on a
25 MP upload.

Fill quality is unchanged: verified by eye on real Gemini/Doubao marks plus a
ground-truth reconstruction sweep -- a tighter view if anything reduces the GAN's
hallucination of large background structure.

Extract the shared padded-crop-box math into _padded_crop_box (used by both
erase_lama and erase_migan).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 08:22:59 +03:00
Victor KuznetsovandClaude Opus 4.8 f69fedcf2d chore(release): bump version to 0.14.1
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
v0.14.1
2026-07-11 11:13:02 +03:00
Victor KuznetsovandClaude Opus 4.8 731af94639 docs(release): note HOMEBREW_TAP_TOKEN rotation on distribute.yml Bad credentials
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 11:13:02 +03:00
Victor KuznetsovandClaude Opus 4.8 49869ab02b feat(identify): detect Dreamina C2PA + Tencent Cloud AIGC schema
Two AI-provenance metadata types mined from the retained corpus that
identify previously read as no-signal:

- Dreamina (ByteDance's international Jimeng brand) signs C2PA as
  "Bytedance Pte. Ltd." with a "Dreamina/x.y" claim generator and NO
  digitalSourceType, so the generator name is the only AI signal. Add a
  C2paAiVendor row with a new asserts_ai flag (identity-AI: presence
  asserts AI without trainedAlgorithmicMedia) plus the derived
  C2PA_IDENTITY_AI_ORGS view, folded into identify's c2pa_is_ai. Keyed on
  the Dreamina generator token, not the "Bytedance Pte" issuer, so non-AI
  CapCut edits signed by the same entity stay unattributed. 7/7 corpus
  files now attribute to ByteDance.

- Tencent Cloud's TC260 AIGC variant uses a ServiceProvider/ServiceUser
  schema (vs the producer-side ContentProducer schema), embedded in EXIF
  ImageDescription; add those field names to _TC260_FIELDS so the generic
  {"AIGC":{...}} gate accepts it. 11/11 corpus files now flagged.

Test-first: reproducing tests in test_identify.py / test_metadata.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 10:54:56 +03:00
Victor KuznetsovandGitHub 41e20a0606 Merge pull request #60 from wiltodelta/claude/mystifying-cori-a4c1e7
v0.14.0: visible-mark localize→fill rewrite, backend priority, fixes
v0.14.0
2026-07-11 09:53:07 +03:00
Victor KuznetsovandClaude Opus 4.8 e3e332c4b1 Merge origin/main into claude/mystifying-cori-a4c1e7
main landed #58 (pill-gate fix, superseded by this branch's localize->fill
rewrite) and #57 (deps bump). Resolved the 6 code/test/doc conflicts by keeping
this branch's post-rewrite versions; the deps bump auto-merged into
uv.lock/pyproject. Full gate green after resolution: ruff, pyright 0, 730 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 09:50:21 +03:00
Victor KuznetsovandClaude Opus 4.8 b579385c6f feat(visible): white-core rescue for the Gemini false-positive gate
The FP gate demotes a low-gradient match, but a real FAINT sparkle also has soft
edges, so metadata-stripped faint sparkles were dropped. Keep a low-grad match
that is a strong (conf >= 0.52), bright, near-WHITE-core sparkle: a real sparkle
core is white, a clean bright corner that shape-matches (sky/sun) is colored
(_core_saturation). Recovers ~14/20 stripped faint sparkles under the DEFAULT
strict/auto (no metadata, no flag) at ~1.25% clean false-fire (baseline 0.55%);
the ~0.51-scoring bright-background FPs stay demoted (below 0.52).

A learned classifier on the same features measured WORSE than the tuned gate
(tier-1: MLP 86.7% recall vs the gate's 90.8% at equal false-fire), so the
heuristic stays; a patch-CNN with richer features is roadmapped P2 with low
expected value -- the precision/recall wall is fundamental (deep-research +
tier-1 both confirm it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 09:41:58 +03:00
Victor KuznetsovandClaude Opus 4.8 bd60be10f9 docs: fold visible-mark + detection-method research into watermarking-landscape
Deep-research 2026-07-10 (adversarially verified): the Gemini sparkle is
tier-gated (visible on Free/Pro, absent on Ultra/AI-Studio/API; no official
visible-mark detector or published glyph spec); the faint-visible-mark
precision/recall wall is fundamental (learned CNN front-end does not cleanly
separate true/false, arXiv:1705.08593 refuted); learned detectors need large
synthetic-composite datasets + carry off-distribution risk; landscape adds
Meta bottom-left + Samsung star-icon variants; China GB 45438-2025 is the
strongest visible-mark mandate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 10:00:12 +03:00
Victor KuznetsovandClaude Opus 4.8 27a921b54f docs(visible): record the 0.12.1-vs-0.14 head-to-head + backend quality
Full-dataset validation of reverse-alpha (v0.12.1) vs the current localize->fill:
doubao/jimeng identical (100% coverage + clearance across all backends); gemini
strict coverage a few points below reverse-alpha (the FP tightening), every
missed mark recovered under assume_ai, clearance ~98% both, no outside-box
damage. Clearance is fill-independent (cv2/MI-GAN/LaMa all strip the mark shape);
the difference is visual fill quality on textured/structured backgrounds -- LaMa
best, MI-GAN can ghost/hallucinate, cv2 smears -- which motivates auto = LaMa >
MI-GAN > cv2. Added to module-internals, known-limitations, and the CLAUDE.md
compact list.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 19:18:25 +03:00
Victor KuznetsovandClaude Opus 4.8 c858006e93 feat(visible): auto fill prefers LaMa > MI-GAN > cv2, warn on cv2 fallback
The auto backend now resolves best-first: LaMa (highest quality, recovers the
textured/structured backgrounds the classical fill smears) > MI-GAN > cv2. Both
learned backends share the same onnxruntime availability check, so auto cannot
tell them apart and always prefers the better one; a memory-tight deployment
that cannot afford LaMa's ~4.7 GB peak pins MI-GAN explicitly via
`--backend migan` / `backend="migan"` (the deployment's call, not the library's).
cv2 stays the no-deps floor and now emits a one-time quality warning when auto
falls back to it, since it smears texture/structure.

Motivated by a v0.12.1 reverse-alpha vs 0.14 localize->fill head-to-head:
reverse-alpha recovered structured backgrounds more cleanly than any inpaint;
LaMa closes most of that gap, MI-GAN can ghost/hallucinate, cv2 is weakest.
doubao/jimeng removal is identical between versions; gemini strict coverage is
4pp lower (all recovered via assume_ai) with cleaner clearance and no
outside-box damage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 18:24:20 +03:00
Victor KuznetsovandClaude Opus 4.8 9756189eaf style: drop dead Candidate fields, simplify resolve_backend, US spelling
- Candidate carries only the fields the arbiter reads (key, label,
  detected_strict, detected_relaxed, features); location/region/confidence were
  vestigial from the removed best_auto_mark max-by-confidence path.
- resolve_backend returns preferred_inpaint_backend() directly (typed Literal)
  instead of an identity ternary.
- colour/normalise/behaviour -> US spelling across code comments and docs.

No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 16:18:13 +03:00
Victor KuznetsovandClaude Opus 4.8 178fed69a7 fix(visible): thread detection into mask + guard removal/IO edge cases
Resolve 10 code-review findings on the v0.14.0 localize->fill path, several
release-blocking:

- gemini: build the removal mask from the decision's provenance-aware region
  instead of a strict internal re-detect. A relaxed/assume_ai sparkle was
  re-demoted by the FP gate into a None mask and reported removed while left in
  the image; this also drops the redundant double-detect.
- registry: report a mark removed only when a fill actually happened (remove()
  returns a None region for an empty mask), so a no-op is never claimed.
- api/cli: add write_noop so the CLI `visible` no-mark path writes nothing and
  cannot clobber a pre-existing -o file (was write-then-unlink -> data loss);
  create output.parent; skip the same-file copy (SameFileError on in-place).
- cli: catch the missing migan/lama backend RuntimeError on the visible/all
  paths (matches `erase`); route the single-mark relaxation through the shared
  resolve_relax instead of an inline copy.
- metadata: keep_standard=False no longer takes the AI-only lossless JPEG
  short-circuit (it left standard metadata); defer a malformed-marker JPEG to
  the PIL fallback instead of reporting a partial strip as complete.
- invisible: register the HEIF opener before Image.open (HEIC --force) and
  RGB-convert before the PNG temp (CMYK JPEG).
- pill: normalize via to_bgr so a 4-channel BGRA array cannot crash cvtColor.

Regression tests for each; docs synced (resolve_relax, write_noop,
best_auto_mark -> detect_marks).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 16:14:46 +03:00
Victor KuznetsovandClaude Opus 4.8 1f5d64e6c5 style: unify section dividers and logger variable naming
- cli.py section dividers -> the codebase-majority `# ── Title ──` form
- logger variable unified to `logger` across identify, isobmff, c2pa,
  trustmark_detector, invisible_watermark (matching the 9-file majority)
- cosmetic only, no behavior change (719 tests / pyright / ruff green)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 14:31:40 +03:00
Victor KuznetsovandClaude Opus 4.8 1a955b096a feat(visible): localize->fill rewrite, sensitivity/backend + api, HEIC + lossless IO
- Replace reverse-alpha removal with localize -> fill (template-free mask + one
  shared cv2/MI-GAN/big-LaMa fill) for every mark; drops the colour-shift / dark-pit
  failure modes, version-robust to a moved or re-rendered mark
- Separate perception/decision/action: engines report Candidates, a pure
  decide(candidates, Context) arbiter owns all policy (sensitivity + provenance +
  pill gate), remove_auto_marks orchestrates -- behavior-preserving (corpus 46/46/92)
- Three orthogonal knobs replace --method: --backend cv2|migan|lama,
  --sensitivity auto|strict|assume-ai, provenance (auto from metadata)
- Add high-level api.remove_visible / visible_provenance (lazy top-level re-export);
  visible --mark auto delegates to it so CLI and library share ONE path
- Read+write HEIC/AVIF on the pixel path via pillow-heif; imwrite preserves the input
  format at max quality (JPEG q100/4:4:4); a no-op copies the original bytes verbatim
- Lossless byte-level JPEG metadata strip (no DCT re-encode); consolidate the two
  remove_ai_metadata into one, delete legacy noai/cleaner + best_auto_mark
- Bump 0.13.0 -> 0.14.0

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 14:20:52 +03:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3c6cfd3a9b chore(deps): bump the minor-and-patch group across 1 directory with 6 updates (#57)
Bumps the minor-and-patch group with 6 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [pillow](https://github.com/python-pillow/Pillow) | `12.2.0` | `12.3.0` |
| [diffusers](https://github.com/huggingface/diffusers) | `0.38.0` | `0.39.0` |
| [transformers](https://github.com/huggingface/transformers) | `5.12.1` | `5.13.0` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.20.1` | `1.22.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.15.19` | `0.15.20` |
| [pyright](https://github.com/RobertCraigie/pyright-python) | `1.1.410` | `1.1.411` |



Updates `pillow` from 12.2.0 to 12.3.0
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/12.2.0...12.3.0)

Updates `diffusers` from 0.38.0 to 0.39.0
- [Release notes](https://github.com/huggingface/diffusers/releases)
- [Commits](https://github.com/huggingface/diffusers/compare/v0.38.0...v0.39.0)

Updates `transformers` from 5.12.1 to 5.13.0
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v5.12.1...v5.13.0)

Updates `huggingface-hub` from 1.20.1 to 1.22.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.20.1...v1.22.0)

Updates `ruff` from 0.15.19 to 0.15.20
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.19...0.15.20)

Updates `pyright` from 1.1.410 to 1.1.411
- [Release notes](https://github.com/RobertCraigie/pyright-python/releases)
- [Commits](https://github.com/RobertCraigie/pyright-python/compare/v1.1.410...v1.1.411)

---
updated-dependencies:
- dependency-name: pillow
  dependency-version: 12.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: diffusers
  dependency-version: 0.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: transformers
  dependency-version: 5.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.15.20
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: pyright
  dependency-version: 1.1.411
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-07 12:37:13 +03:00
3524e8442e fix(visible): safe-inpaint pill gate, cut metadata-only false fires (#58)
Verified 0.13.0 pill removal on a 32k real-upload corpus. The metadata-OR-wordmark
gate was only ~1/3 precise: TC260 metadata confirms Jimeng-class provenance, not pill
presence, so the weak edge-NCC detector's false fires (textured ceilings/walls, where
inpaint visibly smears) were admitted whenever metadata was present.

Split into two arms (_keep_pill): the reliable bottom-right wordmark (~94% precise,
survives metadata stripping) removes the pill unrestricted; the metadata-only arm
removes it ONLY when the top-left footprint is flat enough for an invisible inpaint
(PillEngine.footprint_is_flat, median-Sobel <= _FLAT_TEXTURE_MAX). Keeps real
flat-scene pills and harmless flat false fires; leaves the damaging textured false
fires untouched. Corpus: 270 -> 118 removals, ~90 true preserved, damaging FP -> ~0.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
v0.13.0
2026-07-07 11:29:17 +03:00
Victor KuznetsovandClaude Opus 4.8 a8fd02a8f7 fix(visible): safe-inpaint pill gate, cut metadata-only false fires
Verified 0.13.0 pill removal on a 32k real-upload corpus. The metadata-OR-wordmark
gate was only ~1/3 precise: TC260 metadata confirms Jimeng-class provenance, not pill
presence, so the weak edge-NCC detector's false fires (textured ceilings/walls, where
inpaint visibly smears) were admitted whenever metadata was present.

Split into two arms (_keep_pill): the reliable bottom-right wordmark (~94% precise,
survives metadata stripping) removes the pill unrestricted; the metadata-only arm
removes it ONLY when the top-left footprint is flat enough for an invisible inpaint
(PillEngine.footprint_is_flat, median-Sobel <= _FLAT_TEXTURE_MAX). Keeps real
flat-scene pills and harmless flat false fires; leaves the damaging textured false
fires untouched. Corpus: 270 -> 118 removals, ~90 true preserved, damaging FP -> ~0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 11:26:28 +03:00
0e5a4cbc54 feat(visible): capture-less AI生成 pill (#54), inpaint fallback, MI-GAN backend (#56)
- Add the Jimeng-basic top-left "AI生成" pill as a CAPTURE-LESS mark
  (pill_engine.py): synthetic-silhouette edge-NCC detect + inpaint-only removal.
  Gated in remove_auto_marks: kept only when Jimeng is confirmed (TC260 metadata
  OR the bottom-right "★ 即梦AI" wordmark fired -- the wordmark keeps recall on
  metadata-STRIPPED uploads) AND Doubao did not fire.
- Add an inpaint-fallback removal path + MI-GAN ONNX backend (migan extra, MIT,
  ~28 MB / ~1 GB peak -- droplet-friendly) alongside big-LaMa. New
  --method auto|reverse-alpha|inpaint (shared across visible/all/batch) and
  erase --backend migan; footprint_mask on each engine.
- auto is deterministic: reverse-alpha for capture marks (recovers exact pixels,
  lighter -- measured cleaner than MI-GAN on structured backgrounds) and inpaint
  only for the capture-less pill.
- --mark auto now removes EVERY detected mark in one pass (remove_auto_marks),
  so a Jimeng-basic image's top-left pill AND bottom-right wordmark both clear.
- Bump 0.12.1 -> 0.13.0.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 20:38:23 +03:00
Victor KuznetsovandClaude Opus 4.8 0f54c6b54d fix(identify): kill three visible-detector false positives
Bright-background photos/renders and a tiny app icon were flagged as
AI-generated by the visible detectors. Two failure modes:

- Gemini sparkle on a bright background (snow+sky photo, white product
  render) scored ~0.51. The FP gate only demoted on a low core-ring
  brightness margin, which a bright background makes high. Add a gradient
  floor (_SPARKLE_FP_GRAD 0.55): a real sparkle is a crisp star (grad
  ~0.97-1.0), a smooth luminance blob that NCC-matches the diamond is not
  (the two FPs measured grad 0.105 / 0.463). The OR is a strict superset
  of the old margin-only demotion, so it cannot regress dark/mid (kept by
  margin) or white-bg (kept by confidence) real sparkles.

- A 48x48 geometric icon matched the Doubao/Jimeng CJK silhouette at
  0.41/0.47 NCC. Purely a small-size artifact (the same icon at >=256px
  collapses to ~0.06-0.10). Guard text-mark detection below a 200px short
  side (_MIN_DETECT_SHORT_SIDE); real marks ship on full-resolution
  renders (smallest captured sample 1086px).

Corpus re-sweep flips only OpenAI content and already-cleaned outputs,
all sub-0.5, so no provenance verdict changes. Add synthetic regression
fixtures for both modes; docs/module-internals.md updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 09:44:24 -07:00
Victor KuznetsovandClaude Opus 4.6 f269b75ded chore: add ruff scripts per-file-ignores, Claude Code settings, bump deps
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-24 12:32:44 -07:00
Victor KuznetsovandClaude Opus 4.8 0d9d7dcf6a docs: compact CLAUDE.md, relocate incident/CVE detail to docs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 10:28:40 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3bbcc0ee94 chore(deps): bump actions/checkout from 6 to 7 in the actions group (#52)
Bumps the actions group with 1 update: [actions/checkout](https://github.com/actions/checkout).


Updates `actions/checkout` from 6 to 7
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-24 09:51:00 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
d96e3f6334 chore(deps): bump the minor-and-patch group with 5 updates (#53)
Bumps the minor-and-patch group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [c2pa-python](https://github.com/contentauth/c2pa-python) | `0.35.1` | `0.36.0` |
| [torch](https://github.com/pytorch/pytorch) | `2.12.0` | `2.12.1` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.19.0` | `1.20.1` |
| [pytest](https://github.com/pytest-dev/pytest) | `9.1.0` | `9.1.1` |
| [ruff](https://github.com/astral-sh/ruff) | `0.15.17` | `0.15.19` |


Updates `c2pa-python` from 0.35.1 to 0.36.0
- [Release notes](https://github.com/contentauth/c2pa-python/releases)
- [Changelog](https://github.com/contentauth/c2pa-python/blob/main/docs/release-notes.md)
- [Commits](https://github.com/contentauth/c2pa-python/compare/v0.35.1...v0.36.0)

Updates `torch` from 2.12.0 to 2.12.1
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/compare/v2.12.0...v2.12.1)

Updates `huggingface-hub` from 1.19.0 to 1.20.1
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.19.0...v1.20.1)

Updates `pytest` from 9.1.0 to 9.1.1
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.1.0...9.1.1)

Updates `ruff` from 0.15.17 to 0.15.19
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.17...0.15.19)

---
updated-dependencies:
- dependency-name: c2pa-python
  dependency-version: 0.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: torch
  dependency-version: 2.12.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.20.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: pytest
  dependency-version: 9.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.15.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-24 09:50:56 -07:00
Victor Kuznetsov ee4af800ce chore(release): v0.12.1 v0.12.1 2026-06-23 15:15:04 -07:00
Victor KuznetsovandClaude Opus 4.8 675590e8b2 perf(invisible): read the fp16 weight variant to halve the cold-start weight load
InvisibleEngine loads SDXL/ControlNet in fp16 on CUDA/XPU but called from_pretrained
without variant="fp16", so it read the full fp32 weight files (~7 GB) and downcast in
memory. _load_from_pretrained now passes variant="fp16" when torch_dtype is float16,
reading the half-precision files (~3.5 GB) instead - roughly halving the cold-start
weight read + host->device transfer (a phase-timed Modal run measured weight load as
~half of the ~25s cold start). Falls back to the default weights when a checkpoint ships
no fp16 variant (a custom --model), so the worst case is the prior behavior. fp32
(cpu/mps) and bf16 (qwen) never request the variant.

Tests: TestFp16WeightVariant (variant requested on fp16, fallback on missing, never on
fp32).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 14:42:47 -07:00
Victor KuznetsovandClaude Opus 4.8 c8dbd0c3f9 docs(claude-md): record that new visible marks are blocked on real flat captures
Per user decision 2026-06-22: synthetic font-rendered alpha reconstruction is
rejected as below the quality bar; the reverse-alpha alpha map must be solved
from real controlled flat captures (visible_alpha_solve.py). Meta AI, more
Samsung locales, and any Grok visible mark are parked until captures exist.
Future sessions must not propose synthetic or derive assets from the corpus.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 16:28:28 -07:00
Victor KuznetsovandClaude Opus 4.8 abb7be7e9b feat(identify): detect + strip NovelAI / Reve / Aphrodite generator stamps
Mined from the retained corpus 2026-06-22 (open-world EXIF/PNG-text/XMP scan,
minus the registry): three AI image generators that stamp a plain generator
name and no C2PA, so identify read them as no-signal -- and under the P0#5
no-signal skip would have skipped the scrub.

- NovelAI (anime SD): PNG tEXt Software/Source/Title. exif_generator now reads
  PNG text chunks (via img.info), not only EXIF/XMP.
- Reve (reve.com): EXIF Software / XMP CreatorTool. Token is the full
  "reve.com", not bare "reve" (would false-fire on "forever"/"reverie").
- Aphrodite AI: EXIF Make / Software.

Detection/removal parity: NovelAI stamps an AI-shaped VALUE under a non-AI KEY
(Title/Source), which _is_ai_key alone keeps. New _is_ai_value drops a text
chunk by value-token match on removal, mirroring exif_generator -- else the
cleaned file still read as NovelAI (verified on a real corpus file).

Tests: TestExifGenerator gains NovelAI PNG-text, Reve, Reve-not-overmatched,
Aphrodite, and a NovelAI detect/remove parity regression. Docs synced
(module-internals, watermarking-landscape, CLAUDE.md).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 16:23:33 -07:00
Victor KuznetsovandClaude Opus 4.8 19f9ab0947 feat(invisible): skip the diffusion scrub when no invisible watermark is detectable (P0#5)
Regenerating pixels removes SynthID / open watermarks but degrades a real
photo, so running it on a clean image is the dominant paid score-0 cause on
no-watermark uploads. Gate invisible/all/batch on identify.has_invisible_target:
when no invisible AI signal is locally detectable and --force is unset, skip the
regeneration. Per-command semantics:
  - invisible: write no output, exit EXIT_NO_INVISIBLE_SIGNAL (2)
  - all: skip step 2 but keep visible-removed pixels + strip metadata, exit 0
  - batch: skip the scrub; copy the input through in invisible mode
A skip never claims the image is clean (a pixel SynthID is undetectable once its
metadata proxy is gone); the message says so and routes to --force. The gate
fails safe (a detector error runs the removal).

has_invisible_target wraps identify(check_visible=False, check_invisible=True)
and returns the new ProvenanceReport.ai_from_metadata field (the confidence==high
union), so the raiw.cc worker can reuse the same gate. Gate placed before engine
construction so the skip path is cheap; shared via cli._should_skip_invisible_scrub.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 11:37:01 -07:00
Victor KuznetsovandClaude Opus 4.8 5a612adfef chore(release): v0.12.0
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
v0.12.0
2026-06-22 11:14:55 -07:00
Victor KuznetsovandClaude Opus 4.8 9695a7cbd2 docs(qwen): drop false 'pending seed-repeat cert' from pipeline docstring
The qwen oracle floors are certified, not pending. Near-threshold scrub is
seed-non-deterministic, but the prod path pins one fixed seed, so a certified
floor reproduces run-to-run -- pinning the seed is the release gate, not a seed
sweep. Reword the docstring so it stops implying an open seed-repeat gate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 10:30:42 -07:00
Victor KuznetsovandClaude Opus 4.8 78e2ae65ad docs: neutralize local-pull path reference in doubao research note
Replace the `data/spaces/originals/` path with a generic "local corpus of
pristine originals" so the committed public doc carries no reference to the
local working-data pull (the data itself is gitignored). The analysis scripts'
default paths are left untouched (operational tooling, no content/provenance).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 09:41:23 -07:00
Victor KuznetsovandClaude Opus 4.8 719c7e9313 chore(gitignore): ignore local-only data/spaces working dir
21k user-pull images under data/spaces/ were untracked but not ignored, so a
stray `git add -A` could have committed them. Add the ignore entry alongside the
other local-data paths; the dir stays local analysis only, never a committed corpus.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 09:32:22 -07:00
Victor KuznetsovandClaude Opus 4.8 d5dd24140c fix(qwen): native-geometry img2img + pipeline-aware strength; record dropped auto/mixed/Z-Image leads
- watermark_remover: _build_qwen_kwargs now passes explicit height/width (via
  _qwen_target_size, floored to /16). Without it QwenImageImg2ImgPipeline defaults to
  1024x1024 and silently squishes non-square inputs, distorting the scene and garbling text.
- watermark_profiles: resolve_strength gains a `pipeline` arg + a Qwen strength ladder
  (_QWEN_VENDOR_STRENGTH, Gemini 0.25), so `--pipeline qwen` gets its certified floor
  automatically; retires the manual "pass --strength 0.25 for Gemini on qwen" workaround.
- fidelity_metrics: replace per-face nearest matching (collided on multi-face images when a
  variant dropped a face, corrupting the identity metric) with a collision-free one-to-one
  assignment (assign_faces_one_to_one). lapvar/LPIPS were always bbox-anchored and immune.
  Regression-guarded by tests/test_fidelity_matching.py.
- docs: record the measured outcomes of the qwen-improvement arc. The Qwen ControlNet
  face-fix is CLOSED (no permissive Qwen detail/tile ControlNet exists; canny carries edges,
  not skin grain). The `--pipeline auto` router + faces+text mixed dual-pass were prototyped
  and DROPPED (controlnet wins faces AND display text: abba CER 0.114 vs qwen 0.379).
  Z-Image-Turbo was tried and dropped (same regeneration limits). qwen stays a manual opt-in;
  controlnet is the default for everything.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 21:52:56 -07:00
Victor KuznetsovandClaude Opus 4.8 8f64869bfc docs: capture the Qwen-improvement research (ship vs improve)
Cited deep-research report (22 sources, 3-vote adversarial verification, 5 refuted)
behind the "ship qwen as-is or improve first?" decision. Verdict: shippable now as
an opt-in text lane; strongest improvement lead is adding a Qwen-Image ControlNet
(InstantX / DiffSynth, Apache-2.0, diffusers QwenImageControlNetPipeline) for face/
skin structure; Z-Image-Turbo (6B, Apache-2.0) is the best cheaper text-preserving
substitute. No improvement has measured face-fidelity at our scrub floors yet --
validate with scripts/fidelity_metrics.py first. Linked from known-limitations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 15:58:46 -07:00
Victor Kuznetsov 0d9033d63a Merge branch 'claude/modest-carson-d72243': corpus-mining provenance + removal fixes
Retained-corpus mining (2026-06-20) fixes, all gate-green:
- C2PA vendor coverage (Volcano Engine CJK legal name, ElevenLabs; TikTok/PixelBin vetted out)
- identify AI-generated vs AI-enhanced (ai_source_kind) + shared GEMINI_SPARKLE_TRUST_CONF (detect/remove threshold unify)
- text-mark over-subtraction guard (Doubao/Jimeng/Samsung)
- region-targeted regeneration for AI-enhanced composites (feather_region_composite + remove_watermark(region=))

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

# Conflicts:
#	CLAUDE.md
2026-06-20 15:39:29 -07:00