mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-09 23:50:40 +02:00
Read WebP metadata past the scan window and surface C2PA reader failures
Three gaps found while measuring the record path against the file path, each one a signal the library could not see: WebP stores `XMP ` after the pixels, so on any WebP above the scan window a fixed read stops short of the label. `_riff_late_metadata` steps over the coded image to reach it, the RIFF analogue of the existing PNG and ISOBMFF readers. Three corpus files hid an IPTC "Made with AI" tag and a C2PA `trainedAlgorithmicMedia` there. The decoder-backed fallback now covers only what it is actually for -- metadata the raw bytes do not spell, such as a compressed PNG `zTXt` packet. A C2PA reader failure returned the same `None` as a file with no manifest, so a verdict could fall back to the raw byte scan with no trace anywhere. Failures now log at warning and only genuine ones do: a file without credentials never reaches that branch, and an unsupported container is demoted to debug through the reader's own `C2paError.NotSupported`. The first corpus run with it found a truncated PNG. `scan_dataset.py` never registered the pillow-heif opener it declares as a dependency, so every HEIC was scanned as unreadable -- no EXIF, and a pixel layer that was 397 of 406 features NaN instead of 136. `_riff_late_metadata` caps its total like `isobmff.scan_c2pa_region` does. Clamping each chunk to the bytes remaining is not enough on its own: one chunk can declare a length spanning most of the file, and this runs on the memoized verdict path over images from arbitrary sources. Also lands `identify_metadata_record` and `ProvenanceReport.to_dict()`, the one-call entry point and the versioned JSON contract for the record path. Record-vs-file equality holds over 3,478 corpus images, and the eight files these fixes recovered still report AI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
0c5961a0ed
commit
2668f1302d
+10
-10
@@ -191,29 +191,29 @@ built from on another machine, in another process, or later.
|
||||
```python
|
||||
import json
|
||||
|
||||
from remove_ai_watermarks.identify import evidence_from_metadata_record, identify_from_evidence
|
||||
from remove_ai_watermarks.identify import identify_metadata_record
|
||||
from remove_ai_watermarks.metadata_record import collect_metadata_record
|
||||
|
||||
record = collect_metadata_record(Path("input.png")) # reads the file
|
||||
blob = json.dumps(record) # ship it anywhere
|
||||
|
||||
evidence = evidence_from_metadata_record(json.loads(blob), path=Path("input.png"))
|
||||
report = identify_from_evidence(evidence) # reads nothing
|
||||
report = identify_metadata_record(json.loads(blob), path=Path("input.png")) # reads nothing
|
||||
payload = report.to_dict() # versioned JSON contract
|
||||
```
|
||||
|
||||
The verdict is the same one `identify(path, check_visible=False,
|
||||
check_invisible=False)` returns for that file. That equality is the record's whole
|
||||
contract, and it is verified two ways: over the tracked provenance fixtures in
|
||||
`tests/test_metadata_record.py`, and over a local corpus, where 3,478 images
|
||||
(every file carrying a rare signal, plus a random slice) produced identical
|
||||
reports through both paths.
|
||||
contract and is verified over the tracked provenance fixtures and a separate local
|
||||
evaluation corpus. `ProvenanceReport.to_dict()` is the stable service boundary: it
|
||||
adds a `schema_version`, contains only JSON-safe values, and deliberately omits the
|
||||
local source path.
|
||||
|
||||
A record carries metadata regions, never pixels: marker segments before the JPEG
|
||||
scan, every PNG chunk except `IDAT`, RIFF chunks except the coded image, the
|
||||
ISOBMFF provenance boxes, the container's trailer, the parsed EXIF tags the
|
||||
verdict reads by name, PIL's info mapping, and the C2PA manifest store. Typical
|
||||
size is 13 kB (p90 93 kB); the tail belongs to images carrying a large embedded
|
||||
manifest, where the store itself dominates.
|
||||
verdict reads by name, PIL's info mapping, and the C2PA manifest store. Record size
|
||||
is bounded by those metadata regions and trailers; images with large embedded
|
||||
manifests naturally produce larger records.
|
||||
|
||||
The `path` argument is metadata: it labels the report and is never opened by
|
||||
either function, so a record collected elsewhere can be judged against a path that
|
||||
|
||||
Reference in New Issue
Block a user