Commit Graph
24 Commits
Author SHA1 Message Date
Victor Kuznetsov 03cd00f132 Restructure documentation, validate metadata removal, consolidate assets 2026-07-25 21:08:04 -07:00
Victor KuznetsovandClaude Opus 4.8 a4c901ff39 fix: metadata-strip parity, input robustness, and detection/clash coverage
Bug fixes (each with a regression test):
- metadata strip parity across every marker placement: IPTC digitalSourceType
  in XMP, the Samsung post-EOI trailer, the China TC260 AIGC block in EXIF
  UserComment, a bare AIGC block in a non-standard APP segment, and the ISOBMFF
  EXIF path (AIGC + xAI) are all now stripped -- anything a scanner flags, the
  strip reaches
- Samsung genAIType detected when its trailer sits past the 512 KB scan window
  (file-tail read on large photos)
- crashes on edge inputs: Gemini detector on images with a short side < 16px,
  footprint_mask on a zero-size ndarray, the humanizer on chromatic_shift >=
  width, and the CLI on unreadable/corrupt/empty input (clean error, not a
  traceback)
- WebP written losslessly (cv2 quality 101), not lossy at 100
- the IPTC digitalSourceType algorithmicMedia (procedural, not trained on
  sampled data) is no longer flagged as AI-generated, so clean procedural
  content is not scrubbed
- c2pa source-type: compositeWithTrainedAlgorithmicMedia is checked before the
  bare algorithmicMedia token, so an AI-enhanced composite is not misclassified

Detection:
- integrity-clash coverage now normalizes ByteDance / Canva / ElevenLabs /
  Black Forest Labs, so a transplanted manifest next to an independent
  conflicting stamp is caught; the generic China TC260 AIGC label is attributed
  to a co-present TC260 vendor, so a legit Doubao image (its own C2PA + TC260
  label) does not clash (corpus-validated: 0 new clashes on 5000 carriers)

CLI:
- batch exits non-zero (with a warning) when any image errors or a GPU-missing
  SynthID scrub is skipped, and copies the input through so the output dir stays
  complete -- it used to always exit 0 and could silently drop files

Perf:
- GeminiEngine reused as a process-wide singleton with a precomputed template
  ladder: -24% on the identify sparkle path, detection byte-identical

Internal: one shared _ai_exif_targets rule set feeds both EXIF scrubbers so
their coverage cannot drift; docs synced; maintain.sh hardened so the uv-secure
internal teardown crash no longer aborts the gate (still fails on a real finding).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 10:49:24 +03:00
Victor KuznetsovandClaude Opus 4.8 b579385c6f feat(visible): white-core rescue for the Gemini false-positive gate
The FP gate demotes a low-gradient match, but a real FAINT sparkle also has soft
edges, so metadata-stripped faint sparkles were dropped. Keep a low-grad match
that is a strong (conf >= 0.52), bright, near-WHITE-core sparkle: a real sparkle
core is white, a clean bright corner that shape-matches (sky/sun) is colored
(_core_saturation). Recovers ~14/20 stripped faint sparkles under the DEFAULT
strict/auto (no metadata, no flag) at ~1.25% clean false-fire (baseline 0.55%);
the ~0.51-scoring bright-background FPs stay demoted (below 0.52).

A learned classifier on the same features measured WORSE than the tuned gate
(tier-1: MLP 86.7% recall vs the gate's 90.8% at equal false-fire), so the
heuristic stays; a patch-CNN with richer features is roadmapped P2 with low
expected value -- the precision/recall wall is fundamental (deep-research +
tier-1 both confirm it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 09:41:58 +03:00
Victor KuznetsovandClaude Opus 4.8 178fed69a7 fix(visible): thread detection into mask + guard removal/IO edge cases
Resolve 10 code-review findings on the v0.14.0 localize->fill path, several
release-blocking:

- gemini: build the removal mask from the decision's provenance-aware region
  instead of a strict internal re-detect. A relaxed/assume_ai sparkle was
  re-demoted by the FP gate into a None mask and reported removed while left in
  the image; this also drops the redundant double-detect.
- registry: report a mark removed only when a fill actually happened (remove()
  returns a None region for an empty mask), so a no-op is never claimed.
- api/cli: add write_noop so the CLI `visible` no-mark path writes nothing and
  cannot clobber a pre-existing -o file (was write-then-unlink -> data loss);
  create output.parent; skip the same-file copy (SameFileError on in-place).
- cli: catch the missing migan/lama backend RuntimeError on the visible/all
  paths (matches `erase`); route the single-mark relaxation through the shared
  resolve_relax instead of an inline copy.
- metadata: keep_standard=False no longer takes the AI-only lossless JPEG
  short-circuit (it left standard metadata); defer a malformed-marker JPEG to
  the PIL fallback instead of reporting a partial strip as complete.
- invisible: register the HEIF opener before Image.open (HEIC --force) and
  RGB-convert before the PNG temp (CMYK JPEG).
- pill: normalize via to_bgr so a 4-channel BGRA array cannot crash cvtColor.

Regression tests for each; docs synced (resolve_relax, write_noop,
best_auto_mark -> detect_marks).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 16:14:46 +03:00
Victor KuznetsovandClaude Opus 4.8 1a955b096a feat(visible): localize->fill rewrite, sensitivity/backend + api, HEIC + lossless IO
- Replace reverse-alpha removal with localize -> fill (template-free mask + one
  shared cv2/MI-GAN/big-LaMa fill) for every mark; drops the colour-shift / dark-pit
  failure modes, version-robust to a moved or re-rendered mark
- Separate perception/decision/action: engines report Candidates, a pure
  decide(candidates, Context) arbiter owns all policy (sensitivity + provenance +
  pill gate), remove_auto_marks orchestrates -- behavior-preserving (corpus 46/46/92)
- Three orthogonal knobs replace --method: --backend cv2|migan|lama,
  --sensitivity auto|strict|assume-ai, provenance (auto from metadata)
- Add high-level api.remove_visible / visible_provenance (lazy top-level re-export);
  visible --mark auto delegates to it so CLI and library share ONE path
- Read+write HEIC/AVIF on the pixel path via pillow-heif; imwrite preserves the input
  format at max quality (JPEG q100/4:4:4); a no-op copies the original bytes verbatim
- Lossless byte-level JPEG metadata strip (no DCT re-encode); consolidate the two
  remove_ai_metadata into one, delete legacy noai/cleaner + best_auto_mark
- Bump 0.13.0 -> 0.14.0

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 14:20:52 +03:00
0e5a4cbc54 feat(visible): capture-less AI生成 pill (#54), inpaint fallback, MI-GAN backend (#56)
- Add the Jimeng-basic top-left "AI生成" pill as a CAPTURE-LESS mark
  (pill_engine.py): synthetic-silhouette edge-NCC detect + inpaint-only removal.
  Gated in remove_auto_marks: kept only when Jimeng is confirmed (TC260 metadata
  OR the bottom-right "★ 即梦AI" wordmark fired -- the wordmark keeps recall on
  metadata-STRIPPED uploads) AND Doubao did not fire.
- Add an inpaint-fallback removal path + MI-GAN ONNX backend (migan extra, MIT,
  ~28 MB / ~1 GB peak -- droplet-friendly) alongside big-LaMa. New
  --method auto|reverse-alpha|inpaint (shared across visible/all/batch) and
  erase --backend migan; footprint_mask on each engine.
- auto is deterministic: reverse-alpha for capture marks (recovers exact pixels,
  lighter -- measured cleaner than MI-GAN on structured backgrounds) and inpaint
  only for the capture-less pill.
- --mark auto now removes EVERY detected mark in one pass (remove_auto_marks),
  so a Jimeng-basic image's top-left pill AND bottom-right wordmark both clear.
- Bump 0.12.1 -> 0.13.0.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 20:38:23 +03:00
Victor KuznetsovandClaude Opus 4.8 0f54c6b54d fix(identify): kill three visible-detector false positives
Bright-background photos/renders and a tiny app icon were flagged as
AI-generated by the visible detectors. Two failure modes:

- Gemini sparkle on a bright background (snow+sky photo, white product
  render) scored ~0.51. The FP gate only demoted on a low core-ring
  brightness margin, which a bright background makes high. Add a gradient
  floor (_SPARKLE_FP_GRAD 0.55): a real sparkle is a crisp star (grad
  ~0.97-1.0), a smooth luminance blob that NCC-matches the diamond is not
  (the two FPs measured grad 0.105 / 0.463). The OR is a strict superset
  of the old margin-only demotion, so it cannot regress dark/mid (kept by
  margin) or white-bg (kept by confidence) real sparkles.

- A 48x48 geometric icon matched the Doubao/Jimeng CJK silhouette at
  0.41/0.47 NCC. Purely a small-size artifact (the same icon at >=256px
  collapses to ~0.06-0.10). Guard text-mark detection below a 200px short
  side (_MIN_DETECT_SHORT_SIDE); real marks ship on full-resolution
  renders (smallest captured sample 1086px).

Corpus re-sweep flips only OpenAI content and already-cleaned outputs,
all sub-0.5, so no provenance verdict changes. Add synthetic regression
fixtures for both modes; docs/module-internals.md updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 09:44:24 -07:00
Victor KuznetsovandClaude Opus 4.8 41f67973ce fix(visible): inpaint mid-tone Gemini sparkle instead of a dark diamond
The free `visible` path over-subtracted a faint Gemini sparkle on a
mid-tone background into a darker-than-background brown diamond instead
of removing it (2026-06-18 prod NPS report, "the watermark was not
removed, just its color changed"). The existing over-subtraction guard
only tripped when reverse-alpha drove a footprint pixel fully negative
(the issue #30 dark-background black-pit case); on a mid-tone background
the over-subtraction darkens the core well below the background without
any pixel crossing zero, so the gate missed it and shipped the dark mark.

Add a second over-subtraction signal to `_reverse_alpha_oversubtracts`:
predict the reverse-alpha output at the bright core, (core - a*logo)/(1-a),
and route to the footprint inpaint when it lands more than
`_OVERSUB_DARK_MARGIN` (25) gray levels below the local background ring.
Calibrated wide: clean removals predict within ~12 of background
(demo_banana ~-1), the prod regression ~-40, the issue #30 dark case ~-82.
Corpus-validated on the 479 detected Gemini images: 10 switch reverse-alpha
to inpaint, all of them dark-diamond cases that improve or match; the
other 469 stay byte-identical. demo_banana stays on the reverse-alpha
path (byte-identical).

Also crop both reverse-alpha helpers to the region they actually touch,
a pure O(image) -> O(mark) win that is byte-identical to the full-frame
math (a uint8<->float32 round-trip is exact):
- `GeminiEngine._core_and_bg` converts only the footprint+ring crop to
  gray, not the whole frame (~70 ms -> 0.1 ms on a 12 MP image; it runs
  for both the alpha-gain estimate and the new gate). Verified identical
  across 479 images; detector confidence unchanged.
- `TextMarkEngine._apply_reverse_alpha` computes the blend on the glyph
  crop only (`amap` is zero outside it, so the math is a no-op there):
  ~275 ms -> ~2 ms per placement on a 12 MP frame, up to 2 placements per
  removal. Verified identical across 142 Doubao/Jimeng placements.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 17:19:41 -07:00
Victor KuznetsovandClaude Sonnet 4.6 61aa76a591 perf(identify): decode the image once for all visible-mark detectors
identify(check_visible=True) ran the Gemini-sparkle detector and the
Doubao/Jimeng text-mark detector each with its own image_io.imread, so the
same bitmap was fully decoded twice. On a memory-constrained host (the raiw.cc
512 MB web worker, which runs identify on every upload) that doubled the peak
decode allocation and contributed to OOM restarts.

Decode once in identify() and pass the BGR array to both detectors. The detect
methods already accept an NDArray, so this only threads the pre-decoded array
through: detect_sparkle_confidence and the two _visible_* helpers gain an
optional image= param that, when None, preserves the old self-read behavior
(so direct callers and the cv2-missing/unreadable paths are unchanged).

Only the visible path is deduplicated; the optional check_invisible decoders
are unaffected (and off on the web hot path). Adds a test asserting
identify(check_visible=True, check_invisible=False) decodes exactly once.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 11:13:17 -07:00
Victor KuznetsovandClaude Opus 4.8 28569bd05d fix(gemini): recover sub-0.85 corner sparkles via top-K fusion selection
The 256->512 detection-search widening (v0.8) let a large, low-gradient
shape match outrank a genuine mid-size corner sparkle whose raw NCC sits
below the 0.85 corner-promote gate, so `identify` read `unknown` on Gemini
images that v0.7.2 caught (reporter osachub: scale-48 sparkle on light
bedding -- true sparkle spatial 0.775 / grad 0.960 / fusion 0.676, but the
size-weighted argmax locked onto a decoy at spatial 0.628 / grad 0.036).

detect_watermark now keeps the top-K (_SELECT_TOPK=3) size-weighted
candidates (NMS-deduped) plus the corner-promote candidate, scores each by
full fusion (spatial+gradient+variance) via the extracted _grad_var_scores
helper, and selects the highest -- the gradient term lifts the true sparkle
over the decoy. Ranking by the SIZE-WEIGHTED score (not a raw-NCC argmax)
preserves tiny-patch suppression: a raw-NCC argmax re-admitted 16-18px
content false positives (14/65 doubao + 4/11 jimeng visible images). Top-K
adds zero flips on the doubao/jimeng corpora and leaves the 495-image Gemini
set unchanged (479 detected) while recovering the reporter's image at 0.676.

- _grad_var_scores: gradient/variance scoring factored out of detect_watermark
- confidence = best_fused (drop the duplicated fusion recompute)
- tests: rename test_promotion_is_what_rescues_it ->
  test_size_weighted_search_alone_traps_on_the_decoy (corner-promote is no
  longer the sole rescue path); add a deterministic regression test mirroring
  the real spatial/grad signature
- docs: module-internals.md detector section + CLAUDE.md mechanism map

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 12:04:20 -07:00
Victor KuznetsovandClaude Opus 4.8 2fcd00ced0 fix: address whole-project code review (visible all/batch, engine consolidation, I/O)
Nine findings from a high-effort project-wide review, fixed and verified
(571 passed, ruff/pyright clean):

Correctness:
- all/batch now remove Doubao/Jimeng/Samsung visible text marks: the visible
  step routes through the registry (new cli._remove_visible_auto) instead of a
  hardcoded GeminiEngine, so they no longer leave the wordmark intact.
- batch always reads the original source (dropped the out_path-reuse that
  re-processed already-cleaned outputs on a re-run).
- img2img_runner only retries the diffusion call on the deprecated-callback
  TypeError; any other TypeError now propagates instead of double-running.
- gemini detect/remove and the reverse-alpha engines normalize channels via a
  new image_io.to_bgr, fixing a grayscale/BGRA crash in the FP-gate path.
- _png_late_metadata advances its cursor by the clamped length, so a malformed
  chunk length no longer aborts the late AI-label scan.

Cleanup / efficiency:
- Consolidate the ~90%-identical Doubao/Jimeng/Samsung engines into a shared
  config-driven _text_mark_engine.TextMarkEngine base; each engine is now a thin
  subclass (TextMarkConfig + test shims). Behavior is byte-exact (the three
  engine test suites pass unchanged). Registry adapters collapse to one
  _text_mark(...) row each. Gemini stays a separate engine.
- scan_head is memoized per (path, size, mtime), so identify() reads the file
  head once instead of ~8 times.
- invisible_engine post-processing decodes/encodes the output once (chained in
  memory) instead of 2-4 times across stages.
- Remove the orphaned get_model_id_for_profile (+ CONTROLNET_PROFILE); derive
  the --strength help from the strength constants (strength_default_help) so it
  cannot drift; share the --pipeline/--strength click options; simplify the
  retired --auto resolver.

Net -835 lines. Tests added for the registry-routed visible pass, to_bgr,
the polish/model/guidance wiring, and strength_default_help. CLAUDE.md updated
for the new base module, the engine/registry changes, image_io.to_bgr, and the
scan_head cache.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 13:21:13 -07:00
Victor KuznetsovandClaude Opus 4.8 2c0b174dfa fix(gemini): self-verify repair for under-removed sparkles
After reverse-alpha, re-detect the sparkle; when one survives at or above the
registry fail line (conf >= 0.5) -- an alpha mismatch the per-image gain estimate
could not fully correct -- inpaint the footprint and keep that only when it lowers
the re-detect confidence. The footprint inpaint reconstructs the slot from its
darker surroundings, so it physically removes the bright sparkle; purely additive,
the common clean removal re-detects below 0.5 and is returned untouched.

Measured on the spaces visible-removal audit: gemini removal-audit failures drop
15 -> 11 (4 genuine rescues), doubao 65/65 and jimeng 11/11 unchanged, zero
regressions on the 468 already-clean removals.

An offset+scale alignment search was prototyped on the remaining 11 fails and
rejected: an audit "ceiling" suggested +4 more, but those were NCC-gaming -- the
lower-scoring placement left the sparkle as bright or brighter, just reshaping the
residual so the contrast-invariant shape-NCC scored lower (a5a9: first-pass slot
~76 at background level vs the "aligned win" ~164). A brightness sanity check
rejected every one, so it contributed nothing and was removed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 16:45:18 -07:00
Victor KuznetsovandClaude Opus 4.8 32a0779e1d fix(gemini): demote sparkle false positives with a core-brightness gate
detect_watermark's shape-only NCC (spatial/gradient/var fusion) fires on ornate
or flat content (text strips, banners, hatching) that coincidentally matches the
diamond shape. The NCC is contrast-invariant, so it cannot see the defining
property of a real Gemini sparkle: a bright WHITE overlay whose core sits above
the local background.

The fusion now demotes (caps confidence to 0.30) a match that is BOTH
low-confidence (< _SPARKLE_FP_CONF 0.65) AND has a low core-ring brightness
margin (_core_ring_margin < _SPARKLE_FP_MARGIN 5). Real sparkles escape via
EITHER high confidence (white-bg sparkles score >=0.79 despite a low margin) OR
high margin (dark/mid backgrounds, incl. the #36 faint-corner case), so both
must fail to demote. The gate is monotonic -- it only removes detections, never
adds -- so it cannot regress the verified-negative corpus (already 0 FPs).

On the spaces corpus it demoted 16/495 flagged sparkles (13 no AI metadata =
content FPs; the 3 AI-meta ones were visually FPs / a near-invisible
white-on-white sparkle whose AI verdict is held by metadata), and dropped the
removal-audit failures 20 -> 15.

- _core_and_bg shared helper (core 75th-pct brightness vs background-ring median);
  _estimate_alpha_gain refactored onto it, new _core_ring_margin wrapper.
- TestSparkleFalsePositiveGate: margin high/low, strong-sparkle kept (incl. on
  white via high conf), blurred no-core blob demoted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 22:02:28 -07:00
Victor KuznetsovandClaude Opus 4.8 e7fb64dca1 fix(gemini): remove more-opaque sparkles via per-image alpha gain
The captured sparkle alpha peaks ~0.51, but some real Gemini sparkles are
rendered more opaque. The fixed-alpha reverse blend then UNDER-subtracts and
leaves a bright residual the detector still fires on. A visible-removal audit
through the registry path on the spaces corpus showed this as a meaningful
fraction of marks -- all under-removals, not a background-brightness class
(failures and successes had the same input confidence and background luma; the
discriminator was the removal delta itself).

remove_watermark now estimates a per-image alpha gain (_estimate_alpha_gain:
effective sparkle opacity at the bright core vs the local background ring,
a_eff/a_cap, clamped [1.0, 1.94]) and scales the alpha to match before the
over-sub/blend branch. A 1.05 deadband keeps a sparkle that already matches the
capture byte-identical to the pre-fix output, so the fix is purely additive
(0 regressions on the audit set; failures dropped substantially). The over-sub
guard still runs on the scaled alpha as the safety net for an over-shoot.

- _estimate_alpha_gain + _ALPHA_GAIN_MAX/_DEADBAND/_CORE_FRAC in gemini_engine.
- TestUnderSubtractionGain asserts on footprint pixels, NOT the detector (its
  NCC is degenerate on a flat synthetic bg; the real corpus removal drops the
  detector ~0.80 -> ~0.27).
- scripts/visible_removal_audit.py: the detect -> remove -> re-detect audit tool
  that found and validated this (operates on gitignored data/spaces only).
- CLAUDE.md + README: document the under-subtraction gain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 19:48:40 -07:00
Victor KuznetsovandClaude Opus 4.8 175609b60a fix(gemini): rescue small corner sparkle buried by the size weight (#36)
detect_watermark's size-weighted global NCC search lets a larger, mediocre
match (e.g. a bright collar in a portrait) outrank a small, near-perfect
sparkle in the bottom-right corner, so a faint sparkle on a busy background
scored below threshold and the image read as clean -- the regression from
widening the search window 256px->512px between v0.7.2 and v0.8.8.

Add _corner_promote: a bottom-right-corner raw-NCC pass that overrides the
global pick when the corner holds a match with raw NCC >= 0.85 that beats it.
It only ever replaces a lower-fidelity pick (cannot weaken an existing
detection) and keeps the wider window for variant margins. The corner side is
relative-clamped (0.20 of the short side, [96, 384]) so it stays a true corner
at every scale: a fixed 256px covers ~70% of a small portrait, where a real
photo raw-matches the star at ~0.81; relative tightening drops that to ~0.69.
The 0.85 gate sits between the worst real-photo corner match (~0.78) and a
genuine faint sparkle (~0.93): zero false positives across native + downscaled
negatives, headshot rescued from below-threshold to 0.71.

Factor the shared multi-scale matchTemplate loop into _scan_scales.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 16:51:03 -07:00
Victor KuznetsovandClaude Opus 4.8 9ca2811938 fix(gemini): inpaint sparkle footprint when reverse-alpha over-subtracts (#30)
On a dark/textured background (e.g. grass) the captured alpha map over-estimates
the real Gemini sparkle's effective opacity (~0.51 captured vs ~0.31 effective),
so the fixed-alpha reverse blend over-subtracts (watermarked - alpha*logo goes
negative) and drives the footprint to black -- the white sparkle turns into a
black diamond (issue #30, reported by @CoolZimo1).

remove_watermark now detects this via _reverse_alpha_oversubtracts (fraction of
footprint pixels with a negative numerator > 5%) and inpaints the small sparkle
footprint from the surrounding pixels (cv2 NS, cropped to a padded box) instead.
Behavior-neutral on the working case: a bright background over-subtracts at ~0%,
so reverse-alpha is used and the output is byte-identical to before (verified:
demo_banana 0.0 frac vs the issue-#30 grass image 0.61 frac; issue-#30 footprint
recovers to background grass with no pit, residual sparkle conf 0.25 < 0.35).

Guard is scoped to GeminiEngine: doubao/jimeng already NCC-align their alpha to
the actual mark per image, which sidesteps the fixed-alpha mismatch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 09:17:32 -07:00
Victor KuznetsovandClaude Opus 4.8 1708857772 fix(gemini): expand sparkle search area 256 -> 512px from corner
The 256px limit caused misses when Gemini places the sparkle further from the
corner than the standard 160px (margin 64 + logo 96). Observed variant at ~300px
reported in issue #30. 512px covers all known Gemini margin variations with room
to spare; matchTemplate on a 512x512 region is still fast on CPU.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 10:42:04 -07:00
Victor KuznetsovandClaude Opus 4.8 5d0e6c3a65 fix: harden metadata parsers and engines; sync docs (full-repo review)
Apply fixes from a full-repo review (code, tests, docs).

Security / correctness:
- Clamp attacker-controlled PNG/caBX chunk lengths to the remaining file
  size in metadata.py and noai/c2pa.py (a malformed length no longer drives
  a multi-GB read); skipped chunks seek instead of read.
- noai/isobmff.strip_c2pa_boxes is now fail-safe on a malformed box: return
  the original bytes with a warning instead of silently truncating the tail,
  so metadata --remove can no longer emit a corrupt file.
- doubao_engine._fixed_alpha_map clamps the glyph box to the image (no crash
  on degenerate width-vs-height).
- watermark_remover._run_region_hires gates the phaseCorrelate offset on
  response and magnitude (a spurious shift no longer garbles text) and drops
  the generator after a CPU fallback (no MPS/CPU device mismatch).

Robustness:
- gemini_engine, doubao_engine, region_eraser normalize grayscale and RGBA
  inputs to BGR at the engine entry points.
- image_io.imwrite returns False on an unwritable path (matches cv2).
- invisible_engine guards a None imread result before use.
- trustmark_detector._decoder uses a double-checked threading lock.
- ctrlregen.tiling.tile_positions raises on overlap >= tile.
- humanizer chromatic shift no longer wraps opposite-edge pixels.
- identify OpenAI caveat keyed on the normalized vendor, not a substring.
- Remove the dead "visible --detect-threshold" CLI option.
- publish.yml verifies the release tag matches the package version.

Docs:
- README strength 0.05 to 0.10; .env.example HF_TOKEN marked optional;
  doubao_capture README updated to reverse-alpha-only; CLAUDE.md synced with
  the new behaviors and the batch command.

Tests: new test_security_clamp.py for the read clamp and isobmff fail-safe;
erase CLI coverage; integrity-clash rule 2 end-to-end; multi-tag EXIF
survival and cross-format strip guards; channel/size, tiling, humanizer, and
imwrite regressions. Full suite 493 passed, 2 skipped; ruff and pyright src/
clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 18:00:39 -07:00
Victor KuznetsovandClaude Opus 4.7 888c8c2556 chore(types): clear strict-pyright debt across src (0 errors)
Make `pyright src/` strict-clean via a hybrid: pure-logic files are fully typed
(piexif gets a local typings/ stub; PIL info-dict loops guard isinstance(key, str);
progress returns Callable[..., None]; availability checks use importlib.util.find_spec
instead of unused imports), while the irreducibly-untyped cv2/torch/diffusers boundary
files carry a documented per-file `# pyright:` relax pragma (or a ctrlregen
executionEnvironment) that disables only the unknown-type rules. Public ndarray-returning
signatures on the relaxed engines are annotated NDArray[Any] so strict consumers (cli.py)
stay clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 14:00:15 -07:00
test-userandClaude Opus 4.7 7b47fa9f6a fix(io): Unicode-safe cv2 image IO + un-eat the [gpu] install hint (v0.6.6)
Two CLI/IO robustness bugs surfaced by issues #17 and #19.

#17 -- non-ASCII image paths (Chinese/Cyrillic/accented) failed on Windows:
cv2.imread/imwrite use the platform ANSI code-page API, so the decode came back
empty with a "can't open/read file" warning. New image_io.imread/imwrite route
through np.fromfile+cv2.imdecode / cv2.imencode+tofile (Unicode-safe, byte-
identical output, cv2.imread None-semantics preserved); all 8 cv2 read/write
call sites now go through it. Behavior-neutral on macOS/Linux (already accept
UTF-8 paths), so the fix is correct-by-construction for the Windows-only bug.

#19 (incidental) -- rich parsed the "[gpu]" in the GPU-extra install hint as a
style tag and dropped it, so the printed command was the un-installable
"pip install 'remove-ai-watermarks'". Escaped as \[gpu] at both call sites.

Tests: test_image_io.py (non-ASCII round-trip, alpha, missing/empty/garbage
semantics); test_cli.py::TestGpuHintMarkup (install hint keeps the extra).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 11:52:48 -07:00
test-userandClaude Opus 4.7 fa104bcade feat(identify): provenance command (platform + watermark inventory)
New 'identify' command and identify.py module: upload an image, get one
ProvenanceReport answering where it was made and what watermarks it
carries. Aggregates every locally-readable signal:

- C2PA Content Credentials -> generating platform (issuer + generator).
- IPTC digitalSourceType 'Made with AI' (Meta and others).
- Embedded SD/ComfyUI generation parameters (local pipelines).
- SynthID metadata proxy (Google / OpenAI C2PA companion).
- Visible Gemini sparkle (cv2 fallback for the stripped-metadata case),
  promoted only at confidence >= 0.5 (corpus-tuned: Gemini sparkles
  score >= 0.56, non-sparkle <= 0.49).

is_ai_generated is True or None, never asserted False -- stripped
metadata leaves no local proof of a clean origin, so absence of signals
is reported as 'unknown' with an explicit caveat. The SynthID *pixel*
watermark remains locally undecodable; the report says so.

Non-PNG containers (JPEG/WebP/AVIF/HEIF/JXL) get the same issuer +
generator attribution via a binary scan (the caBX parser is PNG-only).
The cv2 dependency is isolated in gemini_engine.detect_sparkle_confidence
so identify.py stays type-clean. CLI supports --json and --no-visible.

Validated against the 109-image corpus: 14/14 positives flagged AI,
93/94 negatives clean (the one 'neg' flagged is a Meta image that
genuinely carries the IPTC tag -- correct), zero true errors.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:19:26 -07:00
test-userandClaude Opus 4.7 eb1f65ae45 fix(gemini): no-op remove_watermark when nothing detected
Reverse alpha blending applied at the assumed default position painted
a visible inverse-sparkle artifact onto clean or edited images. The
function now returns an unmodified copy when detection fails, instead
of falling back to the hardcoded Gemini corner. Bump to 0.3.5.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 12:17:25 -07:00
test-userandClaude Opus 4.6 7eb32fedee refactor: enforce strict linting and type checking across codebase
- Expand ruff rules (B, S, SIM, RET, COM, C4, G, PT, PIE, T20, DTZ, ICN, TCH, RUF, ANN)
- Switch pyright to strict mode with relaxed test environment
- Replace try-except-pass with contextlib.suppress throughout
- Move type-only imports into TYPE_CHECKING blocks
- Replace ambiguous Unicode chars (en dash, multiplication sign, Greek alpha) with ASCII
- Move color-matcher from base deps to [gpu], remove unused requests dep
- Add pyright to dev deps, update dependabot to uv ecosystem
- Fix hardcoded version in test_version, unused unpacked vars in tests
- Update maintain.sh, CLAUDE.md, .gitignore, .claude/settings.json
- Remove obsolete .agents/rules/project.md
- Upgrade all dependencies (Pygments vulnerability fix)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 11:42:42 -07:00
test-user e5d8970add Add project files, tests, and documentation for GitHub release
- CLI with visible, invisible, all, metadata, and batch commands
- Gemini watermark removal via reverse alpha blending
- Invisible watermark removal via diffusion regeneration (SynthID, TreeRing)
- AI metadata stripping (EXIF, PNG text, C2PA)
- Face protection (YOLO/Haar) and analog humanizer
- 137 tests covering all CLI modes and core engines
- Ruff and Pyright clean
2026-03-25 11:15:05 -07:00