mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-10 08:00:32 +02:00
Read WebP metadata past the scan window and surface C2PA reader failures
Three gaps found while measuring the record path against the file path, each one a signal the library could not see: WebP stores `XMP ` after the pixels, so on any WebP above the scan window a fixed read stops short of the label. `_riff_late_metadata` steps over the coded image to reach it, the RIFF analogue of the existing PNG and ISOBMFF readers. Three corpus files hid an IPTC "Made with AI" tag and a C2PA `trainedAlgorithmicMedia` there. The decoder-backed fallback now covers only what it is actually for -- metadata the raw bytes do not spell, such as a compressed PNG `zTXt` packet. A C2PA reader failure returned the same `None` as a file with no manifest, so a verdict could fall back to the raw byte scan with no trace anywhere. Failures now log at warning and only genuine ones do: a file without credentials never reaches that branch, and an unsupported container is demoted to debug through the reader's own `C2paError.NotSupported`. The first corpus run with it found a truncated PNG. `scan_dataset.py` never registered the pillow-heif opener it declares as a dependency, so every HEIC was scanned as unreadable -- no EXIF, and a pixel layer that was 397 of 406 features NaN instead of 136. `_riff_late_metadata` caps its total like `isobmff.scan_c2pa_region` does. Clamping each chunk to the bytes remaining is not enough on its own: one chunk can declare a length spanning most of the file, and this runs on the memoized verdict path over images from arbitrary sources. Also lands `identify_metadata_record` and `ProvenanceReport.to_dict()`, the one-call entry point and the versioned JSON contract for the record path. Record-vs-file equality holds over 3,478 corpus images, and the eight files these fixes recovered still report AI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
0c5961a0ed
commit
2668f1302d
@@ -39,7 +39,6 @@ DATA SAFETY
|
||||
Read-only over a local dataset. Writes only to the given output prefix, which
|
||||
belongs outside the repository.
|
||||
|
||||
uv run python scripts/detection_timing.py data/spaces/originals .local-eval/timing/run
|
||||
uv run python scripts/detection_timing.py <dataset> <prefix> --limit 200
|
||||
"""
|
||||
|
||||
|
||||
@@ -72,6 +72,18 @@ from typing import Any
|
||||
from PIL import Image
|
||||
from PIL.IptcImagePlugin import getiptcinfo
|
||||
|
||||
# Pillow cannot open HEIC/HEIF without this opener, and it does not auto-register.
|
||||
# Skipping it does not fail loudly: HEIC records lose EXIF and most pixel features
|
||||
# while the scan continues as if the container were merely unreadable. Deliberately a
|
||||
# copy of `image_io._register_heif` rather than an import: this script keeps the
|
||||
# minimal dependency set its docstring advertises and never imports the package. The
|
||||
# suppression is as wide as the original's, so a broken libheif degrades the scan
|
||||
# instead of killing it at import.
|
||||
with contextlib.suppress(Exception):
|
||||
import pillow_heif
|
||||
|
||||
pillow_heif.register_heif_opener()
|
||||
|
||||
SUPPORTED = {
|
||||
".png",
|
||||
".jpg",
|
||||
|
||||
Reference in New Issue
Block a user