CalvinBackup/remove-ai-watermarks
1
0
Fork 0
mirror of https://github.com/wiltodelta/remove-ai-watermarks.git synced 2026-08-09 15:36:01 +02:00
Code Issues Packages Projects Releases Wiki Activity

Add a portable metadata record so collection and verdict can run apart

Browse Source 
`collect_metadata_record` returns a JSON-safe record carrying an image's
provenance metadata regions -- never its pixels -- and the existing
`evidence_from_metadata_record` + `identify_from_evidence` build the verdict
from it without opening the file. The contract is equality with
`identify(path, metadata only)`, verified over the tracked fixtures and over a
local corpus of 3,478 images (every file carrying a rare signal, plus a random
slice): zero differences.

Three placements defeated earlier drafts and each is now a rule with a test:
the `scan_head` buffer is the head CONCATENATED with late metadata, so a
structural walk must read the raw head instead; Samsung splits its evidence
between a post-EOI trailer and the coded scan; and PIL's info keys must be
emitted in the file path's candidate order, since the first token match wins.

Also fix a real detection gap found while establishing that equality: a label
the decoder can read but a raw byte scan cannot -- a compressed PNG `zTXt`
packet, or a WebP XMP chunk past the scan window -- was invisible to
`identify`. Eight corpus files carrying a China TC260 AIGC label or an IPTC
"Made with AI" tag were reported as no signal at all.

`scripts/detection_timing.py` and its report script measure the metadata path
per method; they write outside the repository and are read-only over a dataset.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Victor Kuznetsov
2026-08-05 21:10:38 -07:00
co-authored by Claude Opus 5
parent f481e6f944
commit 0c5961a0ed
9 changed files with 1159 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/module-internals.md
+25
View File
@@ -386,6 +386,31 @@ metadata extraction from verdict logic:
- `identify` preserves the path-based API and adds the optional registered
visible-mark and open invisible-watermark decoders after extraction.
### Portable metadata record
[`metadata_record.py`](../src/remove_ai_watermarks/metadata_record.py) produces the
record `evidence_from_metadata_record` consumes, so collection and verdict can run
on different machines. Its contract is equality with the file path, and the three
defects found while establishing that equality are the reason each rule exists:
- It walks the file's RAW head, never the `scan_head` buffer. That buffer is the
head concatenated with late metadata payloads, so a structural walk runs off the
end of the real head and parses appended bytes as chunks — which produced 11 MB
records and a phantom AIGC signal.
- Samsung Galaxy AI splits its evidence: the `PhotoEditor_Re_Edit_Data` marker sits
in the post-EOI trailer while the `genAIType` value it is gated on can sit inside
the entropy-coded scan (measured on one file: marker at 580 619, value at
382 953). A marked file therefore keeps the whole tail window, not just the
trailer.
- PIL's info keys are emitted in the file path's own candidate order
(`Software`, `Source`, `Title`, `Description`, then EXIF). `generator_from_metadata`
returns the FIRST candidate carrying a known token, so dict order alone changed
the reported platform on nine NovelAI files.
Pixel forensics are deliberately absent: nothing in the provenance path reads them.
Verified over 3,609 research-scan records — dropping every pixel section changed no
verdict.
The DWT-DCT detector and the visible-mark stage share a single decode of the
source, held by
a per-call `_SharedDecode`. It exposes two accessors because the two arms need