Commit Graph
2 Commits
Author SHA1 Message Date
Victor Kuznetsov a981eb84b1 Make the dataset scanner raw-only and harden format sniffing
Strip all derived output: the library verdict, the XMP edit-trail
parse, and the IJG quality estimate. The script now collects raw bytes
plus mechanical container decodes only, is fully standalone (no
remove_ai_watermarks import), and resumes interrupted runs. Add GIF,
BMP, and TIFF magic bytes to sniff_format (GIF files in the wild were
reported as unknown). Validated on the full 42k-file corpus: 100%
coverage, zero error records.
2026-07-22 20:32:36 -07:00
Victor Kuznetsov df62800bf1 Add standalone dataset metadata/forensics scanner script
Single-file read-only scanner producing one JSONL record per image:
AI verdict via identify (metadata-only, no visible marks), full EXIF/
IPTC/XMP/PNG/JPEG/WebP/ISOBMFF inventories, C2PA store, JPEG encoder
forensics (quant tables, IJG quality, scan script, Huffman, subsampling),
XMP edit-trail fields, hashes/timestamps, macOS download provenance,
Live Photo pairing id. Runs standalone (pillow/piexif/c2pa-python) with
the library optional; supports gzip output, resume after interruption,
and head-only scanning of oversized files.
2026-07-22 17:53:02 -07:00