Expand provenance metadata discovery

This commit is contained in:
Victor Kuznetsov committed 2026-09-12 13:36:46 -07:00
1 parent 927429d349
commit c4edc2820d
17 files changed
+446 -79

No files matched your search

+8 -5
View File
@@ -133,6 +133,8 @@ inspection includes the native TC260 `AIGC` tag in
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload,
plus the QuickTime-form `meta` variants Doubao's iOS export writes (a bare
`meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list).
Keyed `workflow` and `prompt` entries in the same metadata-list structure are
also detected and removed, including ComfyUI exports.
MKV and WebM inspection reads the normative
`Segment.Tags.Tag.SimpleTag` placement. AVI uses `LIST/INFO/AIGC`, while FLV
uses `script.onMetaData.AIGC`. The non-ISOBMFF formats are remuxed with stream
@@ -350,9 +352,9 @@ Visible mark support includes:
- one calibrated Samsung Galaxy AI label variant.
Metadata and provenance inspection covers C2PA, EXIF, XMP, IPTC, common
generator parameters, China TC260 AIGC labels, and several vendor specific
signals. Optional decoders add support for open DWT-DCT watermarks and Adobe
TrustMark.
generator parameters in image and video containers, China TC260 AIGC labels,
and several vendor specific signals. Optional decoders add support for open
DWT-DCT watermarks and Adobe TrustMark.
The exact support matrix, including important locale and detector limits, lives
in [supported signals](docs/supported-signals.md). The
@@ -369,8 +371,9 @@ Visible removal follows three steps:
Metadata removal uses format aware stripping. JPEG metadata removal preserves
the encoded image scan instead of recompressing it. Native MP4/MOV TC260 values
are blanked without changing box sizes or media offsets. Other supported
containers use their corresponding metadata path.
and keyed generation parameters are blanked without changing box sizes or
media offsets. Other supported containers use their corresponding metadata
path.
Invisible removal is different. It regenerates the image through a diffusion
pipeline to disrupt pixel and frequency domain watermarks. This changes the
+4 -3
View File
@@ -316,10 +316,11 @@ delegates to the same verified metadata scanner and stripper as the generic
streams are not transcoded. For MP4 and MOV, this includes the native TC260
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`, plus the
QuickTime-form `meta` variants Doubao's iOS export writes (a bare `meta` box
as a direct `moov` child, and a keyless `hdlr=mdir` metadata list). The inspector
seeks past a large `mdat` to find a tail `moov`. Removal stream-copies the
as a direct `moov` child, and a keyless `hdlr=mdir` metadata list). Keyed
`workflow` and `prompt` entries in the same metadata list are also inspected.
The inspector seeks past a large `mdat` to find a tail `moov`. Removal stream-copies the
container in bounded chunks, converts supported top-level provenance boxes to
same-size `free` boxes, and blanks the TC260 key/value in place. Box sizes,
same-size `free` boxes, and blanks the TC260 or generation key/value in place. Box sizes,
media offsets, and encoded stream bytes do not move; the result is atomically
published only after the complete copy succeeds.
+11 -1
View File
@@ -222,7 +222,9 @@ alone is spaced out). The MP4/MOV/M4V/M4A removal path first validates the top-l
box walk, then copies the source to a sibling temporary file in bounded chunks.
Supported C2PA/JUMBF/AI-label boxes become same-size `free` boxes with blank
payloads; TC260 removal changes the four-byte key to `free` and blanks only the
validated JSON value with same-length spaces. This preserves every box size,
validated JSON value with same-length spaces. Keyed generation metadata such
as `workflow` and `prompt` uses the same structural walk; removal blanks the
complete key and value in place. This preserves every box size,
`stco`/`co64` offset, encoded stream byte, and source-sized memory bound.
Publication is atomic, and a malformed top-level walk is copied unchanged. A
generic `AIGC` key whose value has no TC260 field is ignored.
@@ -541,10 +543,18 @@ Key contracts:
asserting that the pixels were generated; explicit `aigc_info` discriminator
values and Dreamina `exportType=generation` do assert AI origin. Ordinary
Aweme, retouch, and `lv` editor exports are preserved.
- EXIF reads fall back to Pillow's decoded `getexif()` mapping when a container
exposes no ordinary `info["exif"]` blob. This covers ImageMagick PNG raw EXIF
profiles and keeps portable-record identification aligned with the file path.
- Structured `generationParams.modelName` JSON is reduced to its recognized
model name before attribution, so prompts and the surrounding JSON never
become a platform label.
- ISOBMFF containers use
[`_internal/isobmff.py`](../src/remove_ai_watermarks/_internal/isobmff.py).
- Native MP4/MOV TC260 `AIGC` entries are read from
`moov.udta.meta.keys/ilst` and blanked without changing box sizes.
- Native MP4/MOV `workflow` and `prompt` entries use that keyed metadata-list
reader and the same offset-preserving removal path.
- Native MKV/WebM TC260 `AIGC` entries are read from
`Segment.Tags.Tag.SimpleTag` and removed through the ffmpeg stream-copy path.
- Native AVI and FLV TC260 entries are read from `LIST/INFO/AIGC` and
+2 -1
View File
@@ -568,7 +568,8 @@ field records that source audio is copied if present and not watermark-verified.
MP4/MOV
inspection recognizes the native TC260 `AIGC` entry in
`moov.udta.meta.keys/ilst` and the QuickTime-form `meta` variants Doubao's iOS
export writes; its removal preserves container size and encoded
export writes. It also recognizes keyed `workflow` and `prompt` generation
metadata in the same metadata-list structure; removal preserves container size and encoded
stream bytes. MP4/MOV/M4V are copied in bounded chunks, so a large `mdat` is not
loaded into memory; publication is atomic. MKV/WebM inspection recognizes the corresponding
`Segment.Tags.Tag.SimpleTag` representation; its removal requires ffmpeg for a
+6 -3
View File
@@ -80,17 +80,20 @@ The learned backends download model files on first use.
The inspection and stripping code handles signals in these groups:
- C2PA Content Credentials and supported cloud manifest references;
- EXIF and XMP generator fields;
- EXIF and XMP generator fields, including structured
`generationParams.modelName` values from supported generators;
- exact app-export provenance and AIGC disclosures from supported
ByteDance-family products, with product-only provenance excluded from the
generated-image verdict;
- IPTC AI disclosure fields;
- PNG text chunks and embedded generation parameters;
- PNG text chunks and embedded generation parameters, including keyed
`workflow` and `prompt` tags in MP4/MOV metadata lists;
- China TC260 AIGC labels in supported image placements and the normative
MP4/MOV `moov.udta.meta.keys/ilst`, MKV/WebM
`Segment.Tags.Tag.SimpleTag`, AVI `LIST/INFO/AIGC`, and FLV
`script.onMetaData.AIGC` placements;
- xAI and Grok EXIF signature fields;
- xAI and Grok EXIF signature fields in ordinary EXIF and ImageMagick PNG raw
EXIF profiles;
- Samsung AI editing markers;
- Hugging Face job metadata;
- open Stable Diffusion style DWT-DCT watermarks with the `detect` extra;
+14 -5
View File
@@ -27,6 +27,9 @@ provider. For AI claims, exact product mappings in `claim_generator` therefore
take precedence over issuer attribution. Supported mappings include Higgsfield
AI, Recraft, Topaz Labs Image API, and TikTok Ad Creative Toolbox; an unknown
claim generator still falls back to the certificate issuer.
Exact claim-generator mappings also cover `Grok Imagine` as xAI Grok Imagine
and `Suno` as Suno. Both can carry a valid AI source claim without a separately
readable issuer, so issuer-only attribution leaves the platform blank.
**OpenRouter raster metadata survey (2026-09-02):** a paid Recraft V3 API
output carried a valid C2PA claim with `claim_generator` `recraft.ai`, a
@@ -37,11 +40,11 @@ container metadata. Their absence is evidence about those exact files and API
routes, not a claim that every Krea or Riverflow export is metadata-free.
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance Volcano Engine"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the same content as **"Byteplus Pte. Ltd."**. The bare `volcengine` needle missed it, so BytePlus output was mis-attributed to "Adobe Firefly" through an incidental "Adobe XMP" toolkit string. Issuer `b"Byteplus"` maps directly to "BytePlus (ByteDance)". ByteDance's consumer app **Dreamina** (the international Jimeng brand) signs as **"Bytedance Pte. Ltd."** with a `Dreamina/x.y` claim generator but, unlike the Volcano Engine surface, ships **no `trainedAlgorithmicMedia`**. Issuer `b"Dreamina"` maps to "ByteDance Dreamina" with **`asserts_ai=True`**. Registering the broader **issuer** `b"Bytedance Pte"` was deliberately avoided because that same entity also signs non-AI CapCut edits; keying on the `Dreamina` generator token is precise.
- **EXIF/XMP/PNG-text generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal). Additional verified generator stamps include **NovelAI** (`Software`, `Source`, and `Title` PNG text chunks), **Reve** (`Software` or XMP `CreatorTool` = `reve.com`), and **Aphrodite AI** (`Make` or `Software` = `Aphrodite AI`).
- **EXIF/XMP/PNG-text generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal). Additional verified generator stamps include **NovelAI** (`Software`, `Source`, and `Title` PNG text chunks), **Reve** (`Software` or XMP `CreatorTool` = `reve.com`), and **Aphrodite AI** (`Make` or `Software` = `Aphrodite AI`). A JSON `generationParams.modelName` value is parsed structurally and reduced to a recognized model name, rather than exposing its surrounding prompt JSON as a platform label.
- **App-export provenance and AIGC JSON:** supported ByteDance-family exports can place a JSON object in EXIF `ImageDescription` or `UserComment`, independently of C2PA or TC260. Exact `product` values for Doubao, Xinghui, and Dreamina are removable product provenance, but do not alone prove that the pixels were generated. Dreamina additionally requires `exportType=generation` for that verdict. A nested Aweme `aigc_type=1` or private ByteDance `aigc_label_type=1` / `2` is an AIGC disclosure; `0` is inconclusive and can occur on a Dreamina generation export. Plain Aweme, retouch, and `lv` exports are preserved. The lower-case private field is deliberately not interpreted as the normative TC260 `Label`, whose values `1` / `2` / `3` mean generated / possibly generated / suspected generated under [GB 45438-2025](https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf).
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
- **xAI / Grok legacy JPEG EXIF signature scheme (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC** — only EXIF `Artist` = a UUID and EXIF `ImageDescription` = `Signature: <base64>` (a crypto signature, unverifiable locally without xAI's public key). This empirically kills the earlier unverified "xAI signs C2PA as xAI" lead — xAI is not even a C2PA member. `exif_generator` misses it (neither field holds an `AI_GENERATOR_TOKENS` token), so a dedicated detector `xai_signature(path)` matches the pair (`ImageDescription ~ ^Signature: [A-Za-z0-9+/=]{64,}` AND UUID `Artist`); wired into `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify` (signal `xai_signature`, platform "xAI (Grok / Aurora)").
The sampled Grok JPEG downloads from the Aurora route carry **no C2PA, no XMP, no SynthID, no IPTC** — only EXIF `Artist` = a UUID and EXIF `ImageDescription` = `Signature: <base64>` (a crypto signature, unverifiable locally without xAI's public key). This is route-specific, not a product-wide no-C2PA claim: newer Grok Imagine output can carry a valid AI-source C2PA claim whose `claim_generator` is `Grok Imagine`. `exif_generator` misses the legacy pair (neither field holds an `AI_GENERATOR_TOKENS` token), so a dedicated detector `xai_signature(path)` matches it (`ImageDescription ~ ^Signature: [A-Za-z0-9+/=]{64,}` AND UUID `Artist`); wired into `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify` (signal `xai_signature`, platform "xAI (Grok / Aurora)").
**Format confirmed stable across n=3 genuine generations:** exactly three EXIF tags (`Artist`, `ExifOffset`, `ImageDescription`), `Signature:` prefix constant, base64 payload 300-1004 chars. Two capture facts: (a) the `Artist` UUID **equals the public image id** in the asset URL (`https://imagine-public.x.ai/imagine-public/images/<uuid>.jpg`), so it is NOT a private per-user secret — only the `Signature` blob is; (b) the Grok web-UI image is a re-encoded **WebP with no signature** — the EXIF survives only in the *original* JPEG (download button or that public tokenless URL), which is why screenshots / re-encodes are metadata-stripped. A real fixture `data/fixtures/provenance/grok-1.jpg` plus **synthetic** JPEG fixtures (fake UUID + fake `Signature:` blob) cover the detector; never add a real Grok image carrying private content (the repo is public).
@@ -49,7 +52,10 @@ Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC*
JPEG EXIF, which deletes the xAI Signature and UUID Artist pair plus supported
AI generator values while retaining unrelated camera and editor EXIF. The
shared `xai_signature_pair` helper is the single source of truth for the
pair. On the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
pair. Converted PNGs can retain the same pair in an ImageMagick raw EXIF
profile; Pillow exposes that form through `getexif()` even when `info["exif"]`
is absent, so detection, removal, and portable records share that fallback. On
the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). The label says only "this is AI", but its `ContentProducer` names the signing entity — `001` + `1` + an 18-char Unified Social Credit Code + a 5-digit product suffix, normalized by `metadata.uscc_of`, or for a few generators a bare product name. `KnownMark.tc260_producer_codes` maps the codes settled per vendor by `scripts/vendor_cohort_harvest.py` to registry mark keys, so an AIGC image relaxes only the detector of the product it actually carries; an unmapped or absent producer leaves every product strict. The same mapping supplies the manufacturer for provenance-conflict checks. A code identifies a legal entity, not necessarily one brand, so a hosting or aggregating platform that signs for several apps is a recall bet rather than a proof.
@@ -88,7 +94,10 @@ payloads. Removal remuxes either container through ffmpeg with stream copy.
Native TC260 MP4/MOV tags do not live in those top-level provenance boxes.
`tc260_aigc_payloads` separately seeks through `moov.udta.meta.keys/ilst`, so
the normative tag is also found when a large `mdat` precedes `moov`.
the normative tag is also found when a large `mdat` precedes `moov`. The parser
canonicalizes known fields across PascalCase and lowerCamelCase serializations.
The same keyed metadata-list walk exposes generation `workflow` and `prompt`
entries, including ComfyUI exports, and blanks them in place during removal.
**Meta-box XMP and EXIF removal are handled in place:** an AI-label XMP packet
stored as a meta-box `mime` item is blanked by
+25 -19
View File
@@ -29,6 +29,7 @@ from __future__ import annotations
import csv
import logging
import re
from collections import Counter
from datetime import date
from importlib.metadata import version
@@ -40,7 +41,14 @@ from _plain_console import Console, Table
from remove_ai_watermarks.identify import _metadata_region as identify_metadata_region
from remove_ai_watermarks.identify import identify
from remove_ai_watermarks.metadata import scan_head
from remove_ai_watermarks.metadata import (
IPTC_AI_FIELD_MARKERS,
IPTC_AI_MARKERS,
aigc_label_from_metadata,
c2pa_marker_in,
samsung_genai_in,
scan_head,
)
log = logging.getLogger(__name__)
console = Console()
@@ -51,22 +59,10 @@ console = Console()
# below is the primary guard, this list is the second. Group: C2PA/JUMBF infra,
# AI source-type / labeling schemes, and distinctive generator name strings.
MARKERS: tuple[bytes, ...] = (
# C2PA / JUMBF infrastructure and AI source-type / labeling schemes.
b"c2pa",
b"jumbf",
b"contentauth",
# AI source-type / labeling schemes that are meaningful by presence.
b"trainedAlgorithmicMedia",
b"digitalSourceType",
b'"AIGC"',
b"<TC260:AIGC>",
b"TC260:AIGC",
b"tc260.org.cn",
b"AISystemUsed",
b"SynthID",
b"hf-job-id",
b"genAIType",
b"PhotoEditor_Re_Edit",
b"Signature:",
# Distinctive multi-word generator strings only. Bare single words (Luma,
# Gemini, Sora, ...) are omitted: they collide with unrelated metadata prose
# (e.g. "Luma" in Lightroom's EnhanceDenoiseLumaAmount), defeating precision.
@@ -80,12 +76,11 @@ MARKERS: tuple[bytes, ...] = (
b"Adobe Firefly",
b"Black Forest",
b"volcengine",
b"Doubao",
b"\xe8\xb1\x86\xe5\x8c\x85",
b"Nano Banana",
b"Stability AI",
b"Samsung Galaxy",
)
_XAI_SIGNATURE_RE = re.compile(rb"Signature:\s*[A-Za-z0-9+/=]{64,}")
_UUID_RE = re.compile(rb"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}", re.IGNORECASE)
REPORT_FIELDS: tuple[str, ...] = (
"path",
@@ -122,9 +117,20 @@ def _row(rep, *, path: str, suffix: str, lib_version: str) -> dict[str, str]: #
def _marker_hits(region: bytes) -> list[str]:
"""Return marker labels found case-insensitively in one metadata region."""
"""Return high-precision marker labels found in one metadata region."""
folded = region.lower()
return sorted({marker.decode("latin-1", "replace") for marker in MARKERS if marker.lower() in folded})
hits = {marker.decode("latin-1", "replace") for marker in MARKERS if marker.lower() in folded}
if c2pa_marker_in(region):
hits.add("C2PA/JUMBF")
if aigc_label_from_metadata(region) is not None:
hits.add("TC260 AIGC")
if any(marker in region for marker in IPTC_AI_MARKERS + IPTC_AI_FIELD_MARKERS):
hits.add("IPTC AI disclosure")
if samsung_genai_in(region) is not None:
hits.add("Samsung genAIType")
if _XAI_SIGNATURE_RE.search(region) and _UUID_RE.search(region):
hits.add("xAI signature pair")
return sorted(hits)
def _marker_hits_for_path(path: Path) -> list[str]:
@@ -119,6 +119,8 @@ C2PA_IDENTITY_AI_ORGS = frozenset(vendor.org for vendor in C2PA_AI_VENDORS if ve
# Keep this attribution beside the issuer registry so every C2PA consumer has one
# canonical source rather than maintaining a derived product map in identify.py.
C2PA_CLAIM_GENERATOR_PLATFORMS: tuple[tuple[str, str], ...] = (
("grok imagine", "xAI Grok Imagine"),
("suno", "Suno"),
("adobe_firefly", "Adobe Firefly"),
("firefly", "Adobe Firefly"),
("dreamina", "ByteDance Dreamina"),
@@ -211,6 +213,7 @@ AI_GENERATOR_TOKENS = frozenset(
"invokeai",
"imagen",
"gpt-image",
"nano banana",
"nightcafe",
"ideogram",
"leonardo",
+72 -16
View File
@@ -43,6 +43,7 @@ from remove_ai_watermarks.metadata import (
C2PA_UUID,
IPTC_AI_FIELD_MARKERS,
IPTC_AI_MARKERS,
LOCAL_GENERATION_METADATA_KEYS,
MAX_TC260_VALUE_BYTES,
parse_tc260_aigc_json,
)
@@ -152,12 +153,12 @@ def iter_file_boxes(
pos = box_end
def _tc260_key_indices(
def _metadata_key_indices(
stream: BinaryIO,
payload_off: int,
box_end: int,
) -> dict[int, tuple[int, int]]:
"""Map every exact ``AIGC`` key index to its byte span."""
) -> dict[int, tuple[bytes, tuple[int, int]]]:
"""Map bounded metadata key indices to their names and byte spans."""
if payload_off + 8 > box_end:
return {}
stream.seek(payload_off)
@@ -166,7 +167,7 @@ def _tc260_key_indices(
return {}
entry_count = struct.unpack(">I", prefix[4:8])[0]
pos = payload_off + 8
found: dict[int, tuple[int, int]] = {}
found: dict[int, tuple[bytes, tuple[int, int]]] = {}
for index in range(1, entry_count + 1):
if pos + 8 > box_end:
return {}
@@ -179,10 +180,12 @@ def _tc260_key_indices(
if entry_size < 8 or entry_end > box_end:
return {}
name_start = pos + 8
if entry_end - name_start == 4:
name_size = entry_end - name_start
if 0 < name_size <= 128:
stream.seek(name_start)
if stream.read(4) == b"AIGC":
found[index] = (name_start, entry_end)
name = stream.read(name_size)
if len(name) == name_size:
found[index] = (name, (name_start, entry_end))
pos = entry_end
return found
@@ -248,12 +251,23 @@ def _tc260_aigc_regions(
iOS export writes, where the JSON sits in a bare ``ilst`` data item with no
key name to blank.
"""
regions: list[tuple[tuple[int, int] | None, int, int, bytes]] = []
return [
(key_span, value_start, value_end, value)
for key_name, key_span, value_start, value_end, value in _metadata_items(stream, file_size)
if (key_name is None or key_name == b"AIGC") and parse_tc260_aigc_json(value) is not None
]
def _metadata_items(
stream: BinaryIO,
file_size: int,
) -> Iterator[tuple[bytes | None, tuple[int, int] | None, int, int, bytes]]:
"""Yield bounded keyed or keyless metadata-list values from a media file."""
for _moov_start, moov_end, moov_type, moov_payload in iter_file_boxes(stream, 0, file_size):
if moov_type != b"moov":
continue
for meta_payload, meta_end in _iter_tc260_meta_boxes(stream, moov_payload, moov_end):
keys: dict[int, tuple[int, int]] = {}
keys: dict[int, tuple[bytes, tuple[int, int]]] = {}
ilst_boxes: list[tuple[int, int]] = []
keyed = False
for _child_start, child_end, child_type, child_payload in _meta_child_boxes(
@@ -263,7 +277,7 @@ def _tc260_aigc_regions(
):
if child_type == b"keys":
keyed = True
keys.update(_tc260_key_indices(stream, child_payload, child_end))
keys.update(_metadata_key_indices(stream, child_payload, child_end))
elif child_type == b"ilst":
ilst_boxes.append((child_payload, child_end))
if not ilst_boxes:
@@ -275,8 +289,8 @@ def _tc260_aigc_regions(
ilst_end,
):
index = int.from_bytes(item_type, "big")
key_span = keys.get(index)
if keyed and key_span is None:
key = keys.get(index)
if keyed and key is None:
# A keyed (ISO) meta box maps items through ``keys``;
# an unmapped index is not an AIGC entry, and reading
# its value would pull arbitrary metadata (e.g. cover
@@ -284,6 +298,7 @@ def _tc260_aigc_regions(
# keyless QuickTime list falls through to content
# validation below.
continue
key_name, key_span = key if key is not None else (None, None)
for _data_start, data_end, data_type, data_payload in iter_file_boxes(
stream,
item_payload,
@@ -295,9 +310,8 @@ def _tc260_aigc_regions(
continue
stream.seek(value_start)
value = stream.read(value_size)
if len(value) == value_size and parse_tc260_aigc_json(value) is not None:
regions.append((key_span, value_start, data_end, value))
return regions
if len(value) == value_size:
yield key_name, key_span, value_start, data_end, value
def tc260_aigc_payloads(path: str | Path, *, strict: bool = False) -> tuple[bytes, ...]:
@@ -315,6 +329,42 @@ def tc260_aigc_payloads(path: str | Path, *, strict: bool = False) -> tuple[byte
return ()
def _ai_metadata_regions(
stream: BinaryIO,
file_size: int,
) -> list[tuple[tuple[int, int], int, int, str, bytes]]:
"""Locate keyed generation metadata without reading media payloads."""
regions: list[tuple[tuple[int, int], int, int, str, bytes]] = []
for name_bytes, key_span, value_start, value_end, value in _metadata_items(stream, file_size):
if name_bytes is None or key_span is None:
continue
try:
name = name_bytes.decode("utf-8")
except UnicodeDecodeError:
continue
if name.lower() in LOCAL_GENERATION_METADATA_KEYS:
regions.append((key_span, value_start, value_end, name, value))
return regions
def ai_metadata_tags(path: str | Path, *, strict: bool = False) -> dict[str, str]:
"""Read keyed AI-generation metadata from an ISOBMFF container."""
try:
with open(path, "rb") as stream:
if not is_isobmff(stream.read(8)):
return {}
stream.seek(0, 2)
file_size = stream.tell()
return {
name: value.decode("utf-8", "replace")
for _key_span, _value_start, _value_end, name, value in _ai_metadata_regions(stream, file_size)
}
except OSError:
if strict:
raise
return {}
def blank_tc260_aigc_tags(data: bytes) -> tuple[bytes, int]:
"""Blank native TC260 values in place while preserving every box offset.
@@ -576,7 +626,9 @@ def strip_isobmff_media_file(
max_scan=max_box_scan,
)
tc260_regions = _tc260_aigc_regions(stream, file_size) if targets is not None else []
ai_metadata_regions = _ai_metadata_regions(stream, file_size) if targets is not None else []
tc260_key_spans = {region[0] for region in tc260_regions if region[0] is not None}
ai_metadata_key_spans = {region[0] for region in ai_metadata_regions}
with atomic_video_output(output_path) as temporary_path:
with source_path.open("rb") as source_stream, temporary_path.open("r+b") as temporary:
@@ -591,6 +643,10 @@ def strip_isobmff_media_file(
temporary.seek(key_span[0])
temporary.write(b"free")
_overwrite_range(temporary, value_start, value_end, byte=b" ")
for key_span, value_start, value_end, _name, _value in ai_metadata_regions:
temporary.seek(key_span[0])
temporary.write(b"free" + b" " * (key_span[1] - key_span[0] - 4))
_overwrite_range(temporary, value_start, value_end, byte=b" ")
temporary.flush()
os.fsync(temporary.fileno())
shutil.copymode(source_path, temporary_path)
@@ -601,7 +657,7 @@ def strip_isobmff_media_file(
source_path,
)
return 0, 0
return len(targets), len(tc260_key_spans)
return len(targets), len(tc260_key_spans | ai_metadata_key_spans)
def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
+2 -3
View File
@@ -51,6 +51,7 @@ from remove_ai_watermarks.metadata import (
AIGC_MARKERS,
IPTC_AI_FIELD_MARKERS,
IPTC_AI_MARKERS,
LOCAL_GENERATION_METADATA_KEYS,
aigc_label,
aigc_label_from_metadata,
c2pa_cloud_manifest_in,
@@ -106,9 +107,7 @@ _ISSUER_PLATFORM: tuple[tuple[str, str], ...] = tuple(
# PNG-text / EXIF keys that indicate a local diffusion pipeline (vs. a hosted
# platform's C2PA). Subset of AI_METADATA_KEYS; excludes the C2PA/Software keys.
_LOCAL_GEN_KEYS = frozenset(
AI_METADATA_KEYS & {"parameters", "prompt", "negative_prompt", "workflow", "comfyui", "invokeai_metadata", "dream"}
)
_LOCAL_GEN_KEYS = LOCAL_GENERATION_METADATA_KEYS
_STRIP_CAVEAT = (
"Absence of metadata is not proof the image is clean: C2PA, EXIF, and PNG "
+81 -21
View File
@@ -59,6 +59,9 @@ AI_METADATA_KEYS: frozenset[str] = frozenset(
"Software",
]
)
LOCAL_GENERATION_METADATA_KEYS: frozenset[str] = frozenset(
AI_METADATA_KEYS & {"parameters", "prompt", "negative_prompt", "workflow", "comfyui", "invokeai_metadata", "dream"}
)
AI_KEYWORDS: tuple[str, ...] = (
"stable_diffusion",
@@ -169,6 +172,7 @@ TC260_AIGC_FIELDS: frozenset[str] = frozenset(
"ServiceUser",
}
)
_TC260_CANONICAL_FIELDS: dict[str, str] = {field.casefold(): field for field in TC260_AIGC_FIELDS}
MAX_TC260_VALUE_BYTES = 1024 * 1024
@@ -197,7 +201,10 @@ def parse_tc260_aigc_json(value: bytes) -> dict[str, str] | None:
return None
if not isinstance(parsed, dict):
return None
fields = {str(key): str(item) for key, item in cast("dict[object, object]", parsed).items()}
fields = {
_TC260_CANONICAL_FIELDS.get(str(key).casefold(), str(key)): str(item)
for key, item in cast("dict[object, object]", parsed).items()
}
return fields if TC260_AIGC_FIELDS & fields.keys() else None
@@ -877,11 +884,47 @@ def generator_from_metadata(candidates: Iterable[str], scan: bytes = b"") -> str
for value in itertools.chain(candidates, creator_tools):
if app_generator := app_generator_from_metadata(value):
return app_generator
if structured_generator := _structured_generator_from_metadata(value):
return structured_generator
if any(token in value.lower() for token in AI_GENERATOR_TOKENS):
return value.strip()
return None
def _structured_generator_from_metadata(value: str) -> str | None:
"""Extract a bounded generator name without returning its surrounding JSON."""
from remove_ai_watermarks._internal.constants import AI_GENERATOR_TOKENS
if len(value) > MAX_TC260_VALUE_BYTES:
return None
try:
parsed = json.loads(value)
except ValueError:
return None
if not isinstance(parsed, dict):
return None
generation_params = cast("dict[object, object]", parsed).get("generationParams")
if not isinstance(generation_params, dict):
return None
model_name = cast("dict[object, object]", generation_params).get("modelName")
if not isinstance(model_name, str) or not model_name.strip() or len(model_name) > 200:
return None
normalized = model_name.strip()
return normalized if any(token in normalized.lower() for token in AI_GENERATOR_TOKENS) else None
def exif_bytes_from_image(image: Any) -> bytes | None:
"""Return EXIF bytes from either a normal EXIF field or a decoded raw profile."""
value = image.info.get("exif")
if isinstance(value, bytes) and value:
return value
with contextlib.suppress(Exception):
exif = image.getexif()
if exif:
return exif.tobytes()
return None
_APP_PROVENANCE_PRODUCTS: dict[str, str] = {
"doubao": "ByteDance Doubao",
"xinghui": "ByteDance Xinghui",
@@ -894,12 +937,17 @@ _APP_AIGC_LABEL_RE = re.compile(r'"aigc_label_type"\s*:\s*[12](?=\s*[,}])', re.I
_APP_AIGC_TYPE_RE = re.compile(r'"aigc_type"\s*:\s*1(?=\s*[,}])', re.IGNORECASE)
def _normalized_app_metadata(value: str | bytes) -> str:
text = value.decode("latin-1", "ignore") if isinstance(value, bytes) else value
def _normalized_app_metadata(value: object) -> str:
if isinstance(value, bytes):
text = value.decode("latin-1", "ignore")
elif isinstance(value, str):
text = value
else:
return ""
return text.replace('\\"', '"')
def _app_metadata_evidence(value: str | bytes) -> tuple[str | None, str | None]:
def _app_metadata_evidence(value: object) -> tuple[str | None, str | None]:
"""Return removable product provenance and stronger AI-origin evidence."""
normalized = _normalized_app_metadata(value)
products = tuple(match.group(1).lower() for match in _APP_PRODUCT_RE.finditer(normalized))
@@ -962,7 +1010,7 @@ def exif_generator(image_path: Path) -> str | None:
with Image.open(image_path) as img:
info = img.info
exif_bytes = info.get("exif")
exif_bytes = exif_bytes_from_image(img)
# PNG tEXt/iTXt chunks land in img.info too (same idiom as the other
# PNG-text readers in this module); NovelAI stamps Software/Source/Title.
for key in ("Software", "Source", "Title", "Description"):
@@ -1034,7 +1082,7 @@ def _xai_signature_impl(image_path: Path) -> bool:
from PIL import Image
with Image.open(image_path) as img:
exif_bytes = img.info.get("exif")
exif_bytes = exif_bytes_from_image(img)
if not exif_bytes:
return False
tags = piexif.load(exif_bytes).get("0th", {})
@@ -1063,8 +1111,7 @@ def _is_aigc_exif_value(raw: object) -> bool:
return False
if b"AIGC" not in raw:
return False
text = bytes(raw).decode("latin-1", "ignore")
return any(field in text for field in TC260_AIGC_FIELDS)
return aigc_label_from_metadata(bytes(raw)) is not None
def _ai_exif_targets(loaded: dict[str, Any]) -> list[tuple[str, int, bytes, str]]:
@@ -1100,15 +1147,18 @@ def _ai_exif_targets(loaded: dict[str, Any]) -> list[tuple[str, int, bytes, str]
if xai_signature_pair(_exif_text(ifd0, piexif.ImageIFD.ImageDescription), _exif_text(ifd0, piexif.ImageIFD.Artist)):
add("0th", ifd0, piexif.ImageIFD.ImageDescription, "ImageDescription")
add("0th", ifd0, piexif.ImageIFD.Artist, "Artist")
# (b) known AI generator token in a 0th text tag.
for tag, name in (
(piexif.ImageIFD.Software, "Software"),
(piexif.ImageIFD.Make, "Make"),
(piexif.ImageIFD.Artist, "Artist"),
(piexif.ImageIFD.ImageDescription, "ImageDescription"),
# (b) known AI generator token or structured generator name in a text tag.
for ifd_key, ifd, tag, name in (
("0th", ifd0, piexif.ImageIFD.Software, "Software"),
("0th", ifd0, piexif.ImageIFD.Make, "Make"),
("0th", ifd0, piexif.ImageIFD.Artist, "Artist"),
("0th", ifd0, piexif.ImageIFD.ImageDescription, "ImageDescription"),
("Exif", ifde, piexif.ExifIFD.UserComment, "UserComment"),
):
if any(token in _exif_text(ifd0, tag).lower() for token in AI_GENERATOR_TOKENS):
add("0th", ifd0, tag, name)
value = ifd.get(tag)
text = value.decode("latin-1", "replace") if isinstance(value, bytes) else ""
if any(token in text.lower() for token in AI_GENERATOR_TOKENS) or _structured_generator_from_metadata(text):
add(ifd_key, ifd, tag, name)
# (c) TC260 AIGC block in ImageDescription (0th) or UserComment (Exif sub-IFD).
if _is_aigc_exif_value(ifd0.get(piexif.ImageIFD.ImageDescription)):
add("0th", ifd0, piexif.ImageIFD.ImageDescription, "ImageDescription")
@@ -1171,6 +1221,11 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
except Exception as exc:
logger.debug("PIL could not open %s for AI-metadata scan: %s", image_path, exc)
from remove_ai_watermarks._internal.isobmff import ai_metadata_tags
for key, value in ai_metadata_tags(image_path).items():
result.setdefault(key, value[:200] + ("…" if len(value) > 200 else ""))
# C2PA manifest fields from the single canonical parser (_internal/c2pa.py).
c2pa = extract_c2pa_info(image_path)
for key in (
@@ -1632,11 +1687,11 @@ def remove_ai_metadata(
with open(source_path, "rb") as f:
head = f.read(26)
if source_path.suffix.lower() in _STREAMING_ISOBMFF_EXTS and is_isobmff(head):
stripped, tc260_blanked = strip_isobmff_media_file(source_path, output_path)
stripped, native_ai_blanked = strip_isobmff_media_file(source_path, output_path)
logger.info(
"Stream-blanked %d AI-provenance box(es) and %d native TC260 tag(s) → %s",
"Stream-blanked %d AI-provenance box(es) and %d native AI metadata tag(s) → %s",
stripped,
tc260_blanked,
native_ai_blanked,
output_path,
)
return output_path
@@ -1714,8 +1769,9 @@ def remove_ai_metadata(
return output_path
# Read image and filter metadata
with Image.open(source_path) as img:
img = img.copy()
with Image.open(source_path) as source_image:
source_exif_bytes = exif_bytes_from_image(source_image)
img = source_image.copy()
# Pick the save format. Honor the caller's output extension (so a deliberate
# source.png -> output.jpg conversion still works) UNLESS the SOURCE is misnamed
# -- a lossless PNG/WebP whose extension lies (served as .jpg). There the output
@@ -1762,6 +1818,10 @@ def remove_ai_metadata(
if _keep_standard_text_metadata(key, value, keep_standard):
kept_meta[key] = str(value) if not isinstance(value, str) else value
if source_exif_bytes:
with contextlib.suppress(Exception):
exif_data = piexif.load(source_exif_bytes)
# Apply cleaned metadata
if save_kwargs["format"] == "PNG" and kept_meta:
pnginfo = PngInfo()
+9 -1
View File
@@ -50,6 +50,7 @@ from remove_ai_watermarks._internal.schema import require_schema_version
from remove_ai_watermarks.metadata import (
QUICK_SCAN_BYTES,
SAMSUNG_EDITOR_MARKER,
exif_bytes_from_image,
exif_text,
read_file_tail,
)
@@ -267,7 +268,10 @@ def _decoder_info(image_path: Path) -> dict[str, Any]:
with Image.open(image_path) as img:
# PIL types this mapping with a non-string key union (a DPI tuple key
# exists), so the keys are normalized here rather than assumed.
return {str(key): value for key, value in img.info.items()}
info = {str(key): value for key, value in img.info.items()}
if "exif" not in info and (exif_bytes := exif_bytes_from_image(img)):
info["exif"] = exif_bytes
return info
except (UnidentifiedImageError, ImportError) as exc: # unsupported optional decoder
logger.debug("PIL info unavailable for %s: %s", image_path, exc)
return {}
@@ -392,6 +396,10 @@ def collect_metadata_record(
issues.append({"stage": "trailer", "code": "collection-failed"})
try:
info = _decoder_info(image_path)
if container == "isobmff":
from remove_ai_watermarks._internal.isobmff import ai_metadata_tags
info.update(ai_metadata_tags(image_path, strict=True))
exif = _exif_pairs(info)
except Exception as exc:
logger.debug("metadata decoder collection failed for %s: %s", image_path, exc)
+2
View File
@@ -464,6 +464,8 @@ class TestIdentifyNonPng:
@pytest.mark.parametrize(
("claim_generator", "platform"),
[
("Grok Imagine", "xAI Grok Imagine"),
("Suno", "Suno"),
("Higgsfield AI", "Higgsfield AI"),
("recraft.ai", "Recraft"),
("Topaz Labs Image API", "Topaz Labs"),
+106
View File
@@ -945,6 +945,30 @@ class TestExifGenerator:
def test_clean_image_is_none(self, tmp_clean_png: Path):
assert exif_generator(tmp_clean_png) is None
def test_nested_generation_model_in_webp_user_comment(self, tmp_path: Path):
import json
payload = json.dumps(
{
"generationParams": {
"modelName": "Nano Banana synthetic fixture",
"prompt": "private prompt must not become the platform label",
}
}
)
exif = piexif.dump({"Exif": {piexif.ExifIFD.UserComment: payload.encode("utf-8")}})
source = tmp_path / "generated.webp"
Image.new("RGB", (32, 32)).save(source, "WEBP", lossless=True, exif=exif)
assert exif_generator(source) == "Nano Banana synthetic fixture"
assert (
identify(source, check_visible=False).platform == "Nano Banana synthetic fixture (embedded generator tag)"
)
output = tmp_path / "clean.webp"
remove_ai_metadata(source, output)
assert exif_generator(output) is None
_FAKE_SIG = "A" * 120 # 64+ base64 chars; real Grok payloads are 300-1004
_FAKE_UUID = "12345678-1234-1234-1234-123456789abc"
@@ -970,12 +994,48 @@ def _grok_jpeg(tmp_path: Path, *, signature: str = _FAKE_SIG, artist: str = _FAK
return path
def _grok_imagemagick_png(tmp_path: Path) -> Path:
"""Write the ImageMagick raw-profile EXIF form seen on converted PNGs."""
exif = piexif.dump(
{
"0th": {
piexif.ImageIFD.ImageDescription: f"Signature: {_FAKE_SIG}".encode("latin1"),
piexif.ImageIFD.Artist: _FAKE_UUID.encode("latin1"),
piexif.ImageIFD.Make: b"Canon",
},
"Exif": {},
"GPS": {},
"1st": {},
}
)
profile = (
"\nexif\n"
+ f"{len(exif):8d}\n"
+ "\n".join(exif.hex()[start : start + 72] for start in range(0, len(exif) * 2, 72))
)
info = PngInfo()
info.add_text("Raw profile type exif", profile)
path = tmp_path / "grok-imagemagick.png"
Image.new("RGB", (64, 64), (70, 80, 90)).save(path, pnginfo=info)
return path
class TestXaiSignature:
"""xAI / Grok's EXIF Signature + UUID-Artist provenance scheme."""
def test_signature_plus_uuid_detected(self, tmp_path: Path):
assert xai_signature(_grok_jpeg(tmp_path)) is True
def test_imagemagick_png_raw_exif_profile_detected(self, tmp_path: Path):
path = _grok_imagemagick_png(tmp_path)
with Image.open(path) as image:
assert "exif" not in image.info
assert image.getexif()
assert xai_signature(path) is True
assert "xai_signature" in get_ai_metadata(path)
def test_real_grok_sample_detected(self):
# Real committed Grok download (data/fixtures/provenance/grok-1.jpg); the EXIF
# Signature + UUID-Artist pair is the only AI signal it carries.
@@ -1169,6 +1229,21 @@ class TestRemoveAiExif:
assert xai_signature(out) is False
assert has_ai_metadata(out) is False
def test_grok_raw_profile_stripped_on_png_output(self, tmp_path: Path):
src = _grok_imagemagick_png(tmp_path)
out = tmp_path / "clean.png"
remove_ai_metadata(src, out)
assert xai_signature(out) is False
with Image.open(out) as image:
assert image.getexif().get(piexif.ImageIFD.Make) == "Canon"
def test_non_text_exif_value_does_not_break_app_metadata_scrub(self):
from remove_ai_watermarks.metadata import _app_metadata_evidence
assert _app_metadata_evidence((1, 2)) == (None, None)
def test_generator_make_token_stripped(self, tmp_path: Path):
# Ideogram's EXIF Make="Ideogram AI" must be scrubbed on removal.
exif = piexif.dump({"0th": {piexif.ImageIFD.Make: b"Ideogram AI"}, "Exif": {}, "GPS": {}, "1st": {}})
@@ -1391,6 +1466,37 @@ class TestAIGCLabel:
assert info["Label"] == "1"
assert info["ContentProducer"] == "001191440300708461136T1308L"
def test_parses_lower_camel_case_tc260_fields(self, tmp_path: Path):
import json
from remove_ai_watermarks.metadata import aigc_label
p = tmp_path / "aigc-lower-camel.jpg"
payload = json.dumps(
{
"data": {
"AIGC": {
"label": "1",
"contentProducer": "synthetic-producer",
"produceId": "synthetic-id",
}
}
}
)
exif = {"Exif": {piexif.ExifIFD.UserComment: payload.encode("ascii")}}
Image.new("RGB", (32, 32)).save(p, exif=piexif.dump(exif))
info = aigc_label(p)
assert info == {
"Label": "1",
"ContentProducer": "synthetic-producer",
"ProduceID": "synthetic-id",
}
out = tmp_path / "clean-lower-camel.jpg"
remove_ai_metadata(p, out)
assert aigc_label(out) is None
def test_has_ai_metadata_detects_raw_json_exif_form(self, tmp_path: Path):
assert has_ai_metadata(self._aigc_exif_jpeg(tmp_path))
+49
View File
@@ -15,6 +15,7 @@ from pathlib import Path
import numpy as np
import pytest
from PIL import Image
from PIL.PngImagePlugin import PngInfo
from remove_ai_watermarks.identify import (
PROVENANCE_REPORT_SCHEMA_VERSION,
@@ -157,6 +158,54 @@ class TestPlacementsTheRecordCouldDrop:
assert "xai_signature" in {s.name for s in identify(path, check_visible=False, check_invisible=False).signals}
_assert_same_verdict(path)
def test_an_imagemagick_raw_exif_profile_survives(self, tmp_path: Path):
"""Pillow decodes this PNG EXIF profile through getexif, not img.info."""
import piexif
exif = piexif.dump(
{
"0th": {
piexif.ImageIFD.ImageDescription: b"Signature: " + b"A" * 80,
piexif.ImageIFD.Artist: b"3f2504e0-4f89-11d3-9a0c-0305e82c3301",
}
}
)
profile = (
"\nexif\n"
+ f"{len(exif):8d}\n"
+ "\n".join(exif.hex()[start : start + 72] for start in range(0, len(exif) * 2, 72))
)
info = PngInfo()
info.add_text("Raw profile type exif", profile)
path = tmp_path / "grok-converted.png"
Image.fromarray(np.zeros((64, 64, 3), dtype=np.uint8)).save(path, "PNG", pnginfo=info)
assert "xai_signature" in {s.name for s in identify(path, check_visible=False, check_invisible=False).signals}
_assert_same_verdict(path)
def test_isobmff_generation_tags_survive(self, tmp_path: Path):
def box(kind: bytes, payload: bytes) -> bytes:
return (8 + len(payload)).to_bytes(4, "big") + kind + payload
def key(name: bytes) -> bytes:
return box(b"mdta", name)
def value(index: int, payload: bytes) -> bytes:
return box(index.to_bytes(4, "big"), box(b"data", b"\x00" * 8 + payload))
keys = box(b"keys", b"\x00" * 4 + (2).to_bytes(4, "big") + key(b"workflow") + key(b"title"))
ilst = box(b"ilst", value(1, b'{"app":"ComfyUI"}') + value(2, b"standard title"))
meta = box(b"meta", b"\x00" * 4 + keys + ilst)
path = tmp_path / "workflow.mp4"
path.write_bytes(
b"\x00\x00\x00\x18ftypmp42\x00\x00\x00\x00mp42isom"
+ box(b"mdat", b"synthetic-video")
+ box(b"moov", box(b"udta", meta))
)
assert "gen_params" in {s.name for s in identify(path, check_visible=False, check_invisible=False).signals}
_assert_same_verdict(path)
class TestRecordShape:
def test_the_record_survives_json(self, tmp_path: Path):
+10 -1
View File
@@ -25,7 +25,16 @@ def _report(**overrides):
def test_gap_markers_are_case_insensitive():
assert "c2pa" in gap._marker_hits(b"UPPERCASE C2PA MANIFEST")
assert "ComfyUI" in gap._marker_hits(b"COMFYUI WORKFLOW")
def test_gap_markers_require_structural_provenance_evidence():
noise = b"c2pa digitalSourceType Samsung Galaxy Doubao PhotoEditor_Re_Edit Signature: " + b"A" * 80
assert gap._marker_hits(noise) == []
tc260 = b'{"AIGC":{"label":"1","contentProducer":"synthetic"}}'
assert "TC260 AIGC" in gap._marker_hits(tc260)
assert "Samsung genAIType" in gap._marker_hits(b'PhotoEditor_Re_Edit_Data{"genAIType":1}')
def test_gap_candidates_cover_more_than_blind_unknowns():
+42
View File
@@ -65,6 +65,30 @@ def _video_with_tc260(path: Path, *, media_payload: bytes = _VIDEO_PAYLOAD) -> P
return path
def _video_with_comfyui_metadata(path: Path, *, media_payload: bytes = _VIDEO_PAYLOAD) -> Path:
workflow = b'{"nodes":[{"type":"KSampler"}],"app":"ComfyUI"}'
prompt = b'{"1":{"class_type":"KSampler"}}'
keys = _box(
b"keys",
b"\x00\x00\x00\x00"
+ (4).to_bytes(4, "big")
+ _metadata_key(b"workflow")
+ _metadata_key(b"prompt")
+ _metadata_key(b"title")
+ _metadata_key(b"software"),
)
ilst = _box(
b"ilst",
_metadata_value(1, workflow)
+ _metadata_value(2, prompt)
+ _metadata_value(3, b"standard title")
+ _metadata_value(4, b"ordinary editor"),
)
meta = _box(b"meta", b"\x00\x00\x00\x00" + keys + ilst)
path.write_bytes(_MP4_FTYP + _box(b"mdat", media_payload) + _box(b"moov", _box(b"udta", meta)))
return path
def _hdlr(handler: bytes) -> bytes:
# version/flags + pre_defined + handler_type + reserved[3]; the TC260 walker
# never parses it, but a real-shaped hdlr keeps the fixture honest.
@@ -564,6 +588,24 @@ class TestVideoDependencies:
class TestVideoMetadataApi:
def test_inspects_and_removes_comfyui_mp4_metadata(self, tmp_path: Path):
from remove_ai_watermarks.video import inspect_video_metadata, remove_video_metadata
source = _video_with_comfyui_metadata(tmp_path / "source.mp4")
output = tmp_path / "clean.mp4"
report = inspect_video_metadata(source)
assert set(report.markers) >= {"workflow", "prompt"}
result = remove_video_metadata(source, output)
cleaned = output.read_bytes()
assert result.remaining == {}
assert len(cleaned) == source.stat().st_size
assert _VIDEO_PAYLOAD in cleaned
assert b"standard title" in cleaned
assert b"ordinary editor" in cleaned
assert b"ComfyUI" not in cleaned
def test_top_level_api_is_lazy_exported(self):
import remove_ai_watermarks as raiw