Route the pixel layer's base64 through the capped _b64 helper, replace
the pixels/pixels_full booleans with one pixel_mode parameter, record a
pixel skipped marker on oversized files, read EXIF from the already-read
bytes instead of re-reading the file, chunk the noise residual
convolution and replace the mgrid with 1D broadcast in the FFT features
(bit-identical values, ~300 MB less peak memory at 2048px), and fix the
raw-only docstring to account for the derived pixel layer.
Guard module-level numpy touchpoints so the metadata-only mode imports
and runs without numpy installed (deferred annotations, lazy DCT basis);
pixel modes now exit early with a clear numpy requirement message.
--pixels adds aggregate, non-reconstructable pixel statistics (block-DCT
histograms with Benford deviation, high-pass noise stats, FFT band
energies and CFA peaks, ELA stats, gradient and color histograms) plus
per-section timing for pipeline latency planning. --pixels-full adds the
privacy-lifting artifacts behind an explicit flag (perceptual hash,
128px thumbnail, coarse ELA/noise-residual/FFT-phase maps). Expensive
maps are computed once and shared between the scalar and artifact paths
(2.2x speedup vs the first split implementation). numpy is required only
for the pixel modes; the metadata-only mode stays numpy-free.
Strip all derived output: the library verdict, the XMP edit-trail
parse, and the IJG quality estimate. The script now collects raw bytes
plus mechanical container decodes only, is fully standalone (no
remove_ai_watermarks import), and resumes interrupted runs. Add GIF,
BMP, and TIFF magic bytes to sniff_format (GIF files in the wild were
reported as unknown). Validated on the full 42k-file corpus: 100%
coverage, zero error records.
Single-file read-only scanner producing one JSONL record per image:
AI verdict via identify (metadata-only, no visible marks), full EXIF/
IPTC/XMP/PNG/JPEG/WebP/ISOBMFF inventories, C2PA store, JPEG encoder
forensics (quant tables, IJG quality, scan script, Huffman, subsampling),
XMP edit-trail fields, hashes/timestamps, macOS download provenance,
Live Photo pairing id. Runs standalone (pillow/piexif/c2pa-python) with
the library optional; supports gzip output, resume after interruption,
and head-only scanning of oversized files.
Victor Kuznetsov