mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-19 20:17:12 +02:00
Cover QuickTime TC260 placements, Luma AI tags, and MiniMax video provenance
This commit is contained in:
+3
-1
@@ -229,7 +229,9 @@ Supported containers are MP4, MOV, M4V, WebM, MKV, AVI, and FLV. The operation
|
||||
delegates to the same verified metadata scanner and stripper as the generic
|
||||
`metadata` command, so detection and removal stay in parity. Video and audio
|
||||
streams are not transcoded. For MP4 and MOV, this includes the native TC260
|
||||
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`. The inspector
|
||||
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`, plus the
|
||||
QuickTime-form `meta` variants Doubao's iOS export writes (a bare `meta` box
|
||||
as a direct `moov` child, and a keyless `hdlr=mdir` metadata list). The inspector
|
||||
seeks past a large `mdat` to find a tail `moov`. Removal stream-copies the
|
||||
container in bounded chunks, converts supported top-level provenance boxes to
|
||||
same-size `free` boxes, and blanks the TC260 key/value in place. Box sizes,
|
||||
|
||||
@@ -12,6 +12,10 @@ Read this reference for environment setup, dependency recovery, CI behavior, and
|
||||
|
||||
The optional TrustMark decoder downloads weights into its installed package directory. After pruning that extra, a leftover weights directory can make availability checks see an empty namespace package. If Pyright reports an unknown `TrustMark` import and `find_spec("trustmark")` returns a loader-less spec, remove that regenerable remnant from the active virtual environment and resync.
|
||||
|
||||
### Known security-gate blocks
|
||||
|
||||
`maintain.sh` fails while PyPI ships no fixed release for a transitive CVE. Current case (triaged 2026-08-17): `lightning` 2.6.5, pulled only by the optional `trustmark` extra, carries PYSEC-2026-3624 (RCE via `load_from_checkpoint` on an attacker-crafted checkpoint). The vulnerable API is unreachable here, because the TrustMark decoder loads only its own pinned weights downloaded from the TrustMark release, never a user-supplied checkpoint. The upstream fix is merged but unreleased, and ignores are never added, so run and report the remaining core checks (Ruff, Pyright, tests) separately until a fixed `lightning` release lands; bump it with `uv lock --upgrade-package lightning` as soon as one does.
|
||||
|
||||
## CI
|
||||
|
||||
`.github/workflows/test.yml` runs Ruff, a test matrix over every supported Python
|
||||
|
||||
@@ -158,7 +158,14 @@ Native MP4/MOV TC260 labels follow TC260-PG-20257A:
|
||||
`moov.udta.meta.keys` maps an `AIGC` key to a raw JSON value in `ilst`.
|
||||
[`_internal/isobmff.py`](../src/remove_ai_watermarks/_internal/isobmff.py) walks those
|
||||
nested boxes by seeking, so detection reaches a tail `moov` without reading the
|
||||
preceding `mdat`. The MP4/MOV/M4V/M4A removal path first validates the top-level
|
||||
preceding `mdat`. Two Doubao iOS variants sit outside that normative placement
|
||||
and are covered by the same walker (2026-08-17 corpus findings, both previously
|
||||
undetected): a QuickTime-form `meta` box as a *direct* `moov` child (no FullBox
|
||||
header, disambiguated by probing the child-box offset), and a QuickTime
|
||||
`hdlr=mdir` metadata list under `udta.meta` whose `ilst` data items carry the
|
||||
validated JSON with no `keys` box at all (content-validated, so only genuine
|
||||
TC260 JSON matches; the keyless entry has no key name to blank, so its value
|
||||
alone is spaced out). The MP4/MOV/M4V/M4A removal path first validates the top-level
|
||||
box walk, then copies the source to a sibling temporary file in bounded chunks.
|
||||
Supported C2PA/JUMBF/AI-label boxes become same-size `free` boxes with blank
|
||||
payloads; TC260 removal changes the four-byte key to `free` and blanks only the
|
||||
|
||||
+2
-1
@@ -431,7 +431,8 @@ must use the same container extension as the source.
|
||||
The returned `VideoMetadataResult` records the source, output, metadata detected
|
||||
before removal, and any markers remaining after the verified strip. MP4/MOV
|
||||
inspection recognizes the native TC260 `AIGC` entry in
|
||||
`moov.udta.meta.keys/ilst`; its removal preserves container size and encoded
|
||||
`moov.udta.meta.keys/ilst` and the QuickTime-form `meta` variants Doubao's iOS
|
||||
export writes; its removal preserves container size and encoded
|
||||
stream bytes. MP4/MOV/M4V are copied in bounded chunks, so a large `mdat` is not
|
||||
loaded into memory; publication is atomic. MKV/WebM inspection recognizes the corresponding
|
||||
`Segment.Tags.Tag.SimpleTag` representation; its removal requires ffmpeg for a
|
||||
|
||||
@@ -37,7 +37,7 @@ when you can select the affected area yourself.
|
||||
| `veo` | Current four-point diamond and legacy `Veo` text | Fixed bottom-right corner | Uses separate silhouettes and requires a recurring match; learned fill is preferable on structured backgrounds. |
|
||||
| `seedance` | Boxed `AI` label | Fixed bottom-right corner | Requires an anchored recurring match; the full localized box is filled because a thinner synthetic shape mask leaves the real translucent rim behind. |
|
||||
| `dola` | `Dola AI` text | Fixed bottom-right corner | Requires an anchored recurring match; ByteDance or BytePlus provenance can relax only an existing visual run. |
|
||||
| `hailuo` | `MINIMAX \| hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. |
|
||||
| `hailuo` | `MINIMAX \| hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. A TC260 label naming MiniMax as producer can relax only an existing stable run. |
|
||||
| `kling` | Kling swirl, `KLING AI`, version, and optional `PRO` suffix | Fixed bottom-right edge | Combines a synthetic logo rescue with font variants, an edge gate, a white-label gate, and anchored temporal recurrence. |
|
||||
|
||||
`video identify`, `video visible`, and `video all` share this registry and the
|
||||
@@ -162,6 +162,7 @@ not a universal clean verdict.
|
||||
| FLUX | None registered | Diffusion regeneration; optional open decoder | C2PA for supported sources |
|
||||
| Adobe Firefly | None registered | Optional TrustMark Variant P decoder | C2PA |
|
||||
| Midjourney | None registered | No registered pixel decoder | EXIF, XMP, and IPTC signals |
|
||||
| Luma AI | None registered | No registered pixel decoder | PNG text generator tags (Uni-1) |
|
||||
| ByteDance generators | Doubao and Jimeng marks | No registered pixel decoder | TC260 AIGC, supported C2PA, and exact app-export AIGC disclosures |
|
||||
| Qwen | Qwen mark | No registered pixel decoder | TC260 AIGC |
|
||||
| Kling | Kling image and video marks | No registered pixel decoder | TC260 AIGC |
|
||||
|
||||
@@ -45,7 +45,7 @@ pair. On the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
|
||||
in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). The label says only "this is AI", but its `ContentProducer` names the signing entity — `001` + `1` + an 18-char Unified Social Credit Code + a 5-digit product suffix, normalized by `metadata.uscc_of`, or for a few generators a bare product name. `KnownMark.tc260_producer_codes` maps the codes settled per vendor by `scripts/vendor_cohort_harvest.py` to registry mark keys, so an AIGC image relaxes the detector of the vendor it actually carries rather than a guessed pair; an unmapped or absent producer falls back to ByteDance's two products. A code identifies a legal entity, not necessarily one brand, so a hosting or aggregating platform that signs for several apps is a recall bet rather than a proof.
|
||||
|
||||
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item. The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
|
||||
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item; Doubao's iOS export additionally writes two QuickTime-form variants (a bare `meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list whose `ilst` data items carry the JSON with no key name) — both verified on retained carriers (2026-08-17). The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
|
||||
|
||||
Native MKV/WebM is a sixth serialization. TC260-PG-20257A stores
|
||||
`TagName=AIGC` and the raw JSON `TagString` in
|
||||
|
||||
Reference in New Issue
Block a user