From 01a2665181111846b37cb76d1d2ba2fd90e3ea3e Mon Sep 17 00:00:00 2001 From: Victor Kuznetsov Date: Thu, 1 Oct 2026 22:10:00 -0700 Subject: [PATCH] Reconcile research follow-up status --- data/image-watermark-examples.csv | 4 +- data/video-watermark-examples.csv | 6 +- docs/ai-generated-image-classifiers.md | 32 ++++++++++ docs/module-internals.md | 7 ++- docs/research-sweep-2026-09.md | 86 ++++++++++++++++++-------- 5 files changed, 103 insertions(+), 32 deletions(-) diff --git a/data/image-watermark-examples.csv b/data/image-watermark-examples.csv index 0b2a897..0ac0a9b 100644 --- a/data/image-watermark-examples.csv +++ b/data/image-watermark-examples.csv @@ -6,9 +6,9 @@ qwen,Alibaba Qwen consumer app,visible text,data/fixtures/visible/qwen/provider- kling,Kuaishou Kling AI,visible text,data/fixtures/visible/kling/provider-original-direct.png,available,authenticated Kling IMAGE 3.0 original; Latin KlingAI 3.0 mark detected at confidence 0.43; TC260 producer maps to Kuaishou,subject to Kling AI terms; noncommercial fixture yuanbao,Tencent Yuanbao,visible two-line text,data/fixtures/visible/yuanbao/provider-original.png,available,authenticated Yuanbao original; detected by Yuanbao engine at confidence 0.49; TC260 producer metadata maps to Tencent,repository samsung,Samsung Galaxy AI,visible Italian text,data/calibration/samsung/samsung_gray_1.png,available,original provider capture and detector calibration,repository -runninghub,RunningHub,visible text,,missing,no redistributable provider original found, +runninghub,RunningHub,visible text,data/fixtures/visible/runninghub/example.png,partial,synthetic detector fixture available; no redistributable provider original found,repository baidu,Baidu,visible text and tag,data/fixtures/visible/baidu/provider-original.jpeg,available,detected by Baidu engine; distributed under the OpenNoMark repository's CC BY-NC-SA 4.0 license,CC-BY-NC-SA-4.0; excluded from Apache-2.0 -liblib,LiblibAI,visible wordmark,data/fixtures/visible/liblib/current-provider-output.png,partial,"authenticated current LiblibAI original carries a top-left AI生成 pill rather than the registered historical bottom-center LiblibAI wordmark; TC260 producer metadata maps to LiblibAI; historical positive remains missing",repository +liblib,LiblibAI,visible wordmark,data/fixtures/visible/liblib/current-provider-output.png,partial,"authenticated current LiblibAI original carries a top-left AI生成 pill rather than the registered historical bottom-center LiblibAI wordmark; TC260 producer metadata maps to LiblibAI; a synthetic historical-wordmark fixture is available at data/fixtures/visible/liblib/example.png, but a historical provider positive remains missing",repository microsoft,Microsoft image output,visible top-right AI badge,data/fixtures/visible/microsoft/provider-original.png,available,authenticated Copilot original detected by Microsoft engine at confidence 0.54; also has valid C2PA and InvisMark,repository jimeng_pill,ByteDance Jimeng,visible AI pill,data/fixtures/visible/jimeng_pill/provider-published-example.jpg,available,"real Jimeng output embedded in page 29 of Zhonglun W&D's 2026 Bank Finance Legal Newsletter; direct jimeng_pill detection confidence 0.28; the same image also carries the Jimeng wordmark",fair-use research fixture; excluded from Apache-2.0 google_synthid,Google,image SynthID,data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png,available,provider oracle recorded in data/synthid/manifest.csv,repository diff --git a/data/video-watermark-examples.csv b/data/video-watermark-examples.csv index 55864b8..d4534d0 100644 --- a/data/video-watermark-examples.csv +++ b/data/video-watermark-examples.csv @@ -1,8 +1,8 @@ key,provider,watermark_type,path,status,verification,license sora,OpenAI Sora,visible moving mascot and wordmark,data/fixtures/visible/sora/provider-original.mp4,available,"Apache-2.0 upstream original; detected by Sora engine on 300/300 frames; C2PA claim names Sora and OpenAI/Truepic, with valid integrity and signature but an expired untrusted signer certificate",Apache-2.0 veo,Google Veo,visible fixed text,data/fixtures/visible/veo/provider-original.mp4,available,"authenticated Google Flow Veo 3.1 Fast original; detected by Veo engine on 192/192 frames; valid C2PA and provider-confirmed SynthID",repository -seedance,ByteDance Seedance,visible boxed AI label,,missing,"OpenRouter Seedance 2.0 Mini with watermark=true produced BytePlus C2PA but no visible boxed AI label; direct consumer original still needed", -doubao,ByteDance Doubao,visible fixed text,,missing,no redistributable provider original found, -dola,ByteDance Dola,visible fixed text,,missing,no redistributable provider original found, +seedance,ByteDance Seedance,visible boxed AI label,data/fixtures/visible/seedance/example.mp4,partial,"synthetic detector fixture available; OpenRouter Seedance 2.0 Mini with watermark=true produced BytePlus C2PA but no visible boxed AI label; direct consumer original still needed",repository +doubao,ByteDance Doubao,visible fixed text,data/fixtures/visible/doubao/example.mp4,partial,"synthetic detector fixture available; a real public carrier is retained locally but is not publication-cleared",repository +dola,ByteDance Dola,visible fixed text,data/fixtures/visible/dola/example.mp4,partial,synthetic detector fixture available; no redistributable provider original found,repository hailuo,MiniMax Hailuo AI,visible composite label,data/fixtures/visible/hailuo/provider-original.mp4,available,"authenticated Hailuo AI MiniMax H3 original downloaded with With Watermark selected; detected by Hailuo engine on 124/124 frames; TC260 AIGC label names MiniMax",repository kling,Kuaishou Kling AI,visible fixed logo and wordmark,data/fixtures/visible/kling/provider-original.mp4,available,"authenticated Kling AI VIDEO 3.0 original; detected by Kling engine on 121/121 frames; no locally readable provenance metadata",Kling AI terms; noncommercial fixture diff --git a/docs/ai-generated-image-classifiers.md b/docs/ai-generated-image-classifiers.md index 2225fff..6473625 100644 --- a/docs/ai-generated-image-classifiers.md +++ b/docs/ai-generated-image-classifiers.md @@ -1597,6 +1597,38 @@ Local reproducibility artifacts are under `contamination-scan-2026-08-26`, and `taxonomy-continuation-clean-2026-08-26` run directories. +## Modern frozen-backbone campaign closed, 2026-09-27 + +A second pre-registered representation campaign tested exact-revision, +Apache-2.0 frozen vision backbones with universal ridge heads against the same +product contract. The candidates were `google/siglip2-base-patch16-224` at +revision `75de2d55ec2d0b4efc50b3e9ad70dba96a7b2fa2` and +`facebook/PE-Core-B16-224` at revision +`a16450b46fef32363459920c2685a1b4ef13dcd9`. The protocol was frozen before +inference and used 28,743 unique source images. Its comparison cells contained +1,873 AI-test images, 3,000 fresh Open Images photographs, 24 Kodak images, and +300 FLUX images. + +| Model at its frozen operating point | AI test | Fresh photo false positives | Kodak false positives | FLUX | +| --- | ---: | ---: | ---: | ---: | +| Production Model 1 baseline | 1,676/1,873 (89.5%) | 35/3,000 (1.17%) | 0/24 | 256/300 (85.3%) | +| SigLIP2 universal ridge | 1,602/1,873 (85.5%) | 74/3,000 (2.47%) | 0/24 | 125/300 (41.7%) | +| PE-Core B universal ridge | 1,715/1,873 (91.6%) | 86/3,000 (2.87%) | 0/24 | 170/300 (56.7%) | + +SigLIP2 failed the AI-test, fresh-photo, and FLUX gates. PE-Core improved the +AI-test cell but failed the fresh-photo and FLUX gates. Matching the baseline's +exact 35-false-positive photo budget after the fact did not recover the lost +coverage: SigLIP2 reached 1,505/1,873 AI and 98/300 FLUX, PE-Core universal +reached 1,625/1,873 and 133/300, and PE-Core photo-only reached 1,709/1,873 and +97/300. No threshold matched the baseline photo budget, AI coverage, and FLUX +coverage simultaneously. + +The bounded newer-backbone hypothesis is therefore closed as a Model 1 +replacement. Keep `clip-l-ft.pt`, its ONNX graph, hashes, and Hub revision +unchanged. Reproducibility artifacts are local under +`.local-eval/synthid/ai-photo-2026-08-22/modern-representation-campaign-2026-09-27/`; +the campaign did not enter the publication stage. + ### Wild extras, not SynthID | Hypothesis | 2026-08-23 | Use | diff --git a/docs/module-internals.md b/docs/module-internals.md index ea11432..ca32ebc 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -743,8 +743,11 @@ Key contracts: A.1 string (`"Label":"1","ContentProducer":"...","ProduceID":"..."`) under the default SM2 user ID, and MiniMax's `LabelMataData` spelling is accepted beside the guide's `Md`. Only the label signature is checked: MiniMax's binding - signature also verifies, but no verdict reads it, and its content hash is not - recomputed because MiniMax names no `CntSel`. The key travels in the file, so + signature also verifies, but no verdict reads it. The guide's appendix A.2 + defines `DFT` for non-JPEG/PNG files as all bytes except the metadata label; + MiniMax names no `CntSel`, and measured whole-file, direct DFT byte-range, + metadata-box, and media-payload interpretations do not reproduce its content + hash. The key travels in the file, so only a key pinned in `TC260_PRODUCERS` (the source of `TC260_SIGNING_KEYS`) names a signer, and a pinned signer is checked with the pinned point whatever parity the file claims; an unpinned diff --git a/docs/research-sweep-2026-09.md b/docs/research-sweep-2026-09.md index 91b6ee6..93c5c24 100644 --- a/docs/research-sweep-2026-09.md +++ b/docs/research-sweep-2026-09.md @@ -33,22 +33,28 @@ This page is an index. Each finding lives in its canonical page. | Dependencies | TrustMark extra on every supported Python (`trustmark>=0.9.2`) | [installation.md](installation.md) | | Live captures | Gemini Omni, Grok, Vidu, Wan, Apple, Google Photos, Nano Banana 2 Lite and Gemini Omni Flash through the API; Runway, Higgsfield and Dreamina | [watermarking-landscape.md](watermarking-landscape.md#live-captures-2026-09-23-and-2026-09-24) | -## Open items +## Follow-up status -- **Google Photos Reimagine manifests.** Magic Editor, and so Reimagine, is not - offered in Google Photos on iOS, so no Reimagine file is captured. The +The original follow-up list now mixes completed experiments, deliberately +excluded claims, and acquisition gaps. Each row below names its current status +as of 2026-10-01 so this section can serve as the research backlog without +discarding the measured results. + +- **Closed - Google Photos Reimagine manifests.** Magic Editor, and so + Reimagine, is not offered in Google Photos on iOS, so no Reimagine file is + captured. The question it was meant to answer is settled without one: Google's checker found SynthID in all four Photos AI edits tested on 2026-09-25, none of which records SynthID in its manifest. -- **Vidu and Wan marks on more samples.** Both are registered from one real - export each (`vidu` video, `wan` image); their gates are provisional until a +- **Open - Vidu and Wan marks on more samples.** Both are registered from one + real export each (`vidu` video, `wan` image); their gates are provisional until a wider cohort is measured, and a Wan video label is not yet captured. Public Vidu feed videos (2026-09-26) carry the TC260 label naming ShengShu, and one names Tongyi Yunqi as producer with Vidu as propagator, but none shows the visible `Vidu` mark; Wan's public demo videos show no mark either. Both marks appear only on account downloads. -- **TC260 practice guides.** The audio placements from TC260-PG-202510A are - now read (the guide PDFs load in a browser from +- **Partially complete - TC260 practice guides.** The audio placements from + TC260-PG-202510A are now read (the guide PDFs load in a browser from `tc260.org.cn/tc260/sjzn/list.shtml`, not to scripted requests). The text-file guide (TC260-PG-20258A) covers documents this image and video tool does not process. The security-protection guide (TC260-PG-202511A) records a @@ -66,15 +72,25 @@ This page is an index. Each finding lives in its canonical page. point; changing one `ProduceID` digit fails it. MiniMax's own spellings depart from the guide: `TBSData.Type` is `LabelMataData` (the guide's `Md`) and `Binding` with `BType` `0` (its `Bnd`); the second signature verifies over the - compact JSON of the `Bindings` entry, also on all four. That entry's SM3 - content hash matched neither the whole file nor the file without its label, - and it names no `CntSel`, so the content binding is unverified. Other producers fill the same + compact JSON of the `Bindings` entry, also on all four. TC260-PG-202511A + appendix A.2 was re-read from the + [official PDF](https://www.tc260.org.cn/tc260/sjzn/202508/63ab55a3265f45c7865e9b8be15d4a29/files/dc379ec776f94abc8f1806affa1cda35.pdf) + on 2026-10-01. MiniMax omits its `CntSel`, and none of the guide-derived + byte selections reproduced its SM3 content hash on four measured videos, so + the content binding remains unverified + ([module internals](module-internals.md)). Other producers fill the same fields with opaque vendor strings (Qwen a base64 digest, Vidu 32 hex digits, Wan and HappyHorse `L-`/`K-` prefixed values), not `SecurityData`. The key sits in the file it signs, so a check proves authenticity only against a pinned MiniMax key. `identify` now verifies it with a built-in SM3/SM2 and a pinned - MiniMax key ([module internals](module-internals.md)). -- **Experiments.** Chroma restoration after regeneration (arXiv:2508.21072) was + MiniMax key. The remaining work is + MiniMax content-binding verification if its byte-selection rule becomes known, + and Honor's unsupported RSA/content form if an original export and signing + contract become available. +- **Partially complete - experiments.** Five of the six follow-ups are closed: + Z-Image-Turbo, Perth and PixelSeal, re-watermarking, VACE/ROSE, and newer + frozen-backbone probes. Chroma + restoration after regeneration (arXiv:2508.21072) was measured on 2026-09-27 and is not shipped: transplanted source chroma drew one explicit "inconclusive" and two unclear Gemini answers where the untouched outputs were all clean, and ghosts redrawn objects @@ -126,12 +142,20 @@ This page is an index. Each finding lives in its canonical page. profile; retain its harness and local evidence for future comparison only. The existing source-conditioned Z-Image face-repair stage remains part of `qwen-zimage`, `sdxl-zimage`, and `chroma-zimage`. - Still open: linear probes on newer frozen backbones for `classify` and RAVEN - view-synthesis removal - (arXiv:2601.08832, no code released as of 2026-09-24). + The pre-registered modern-backbone campaign found that neither frozen + SigLIP2 Base nor PE-Core B could match the production model's combined photo, + AI, and FLUX constraints, so the replacement hypothesis is closed + ([classifier research](ai-generated-image-classifiers.md#modern-frozen-backbone-campaign-closed-2026-09-27)). + Only RAVEN view-synthesis removal remains blocked. RAVEN's + [official repository](https://github.com/fahadshamshad/raven-watermark-removal) + now exists, but its 2026-08-08 README is still a release placeholder: the + authors say they will publish code after their SynthID testing and Google + Security Bounty submission. There is no implementation to evaluate as of + 2026-10-01 (arXiv:2601.08832). MiniMax-Remover (CC-BY-NC-4.0 weights) and ProPainter (S-Lab License, - non-commercial) are excluded on license. -- **New C2PA signers without a vendor row.** The + non-commercial) are deliberately excluded on license and are not backlog + items. +- **Open - new C2PA signers without a vendor row.** The [C2PA conformance list](https://github.com/c2pa-org/conformance-public/blob/main/conforming-products/conforming-products-list.json) names generator certificates `Amazon Bedrock`, `Getty Images AI-Generated Image` and `AI-Modified Image` (and the iStock pair), `Jasper`, @@ -148,7 +172,7 @@ This page is an index. Each finding lives in its canonical page. carrying C2PA: their marketing and model-page images are unsigned. A vivo X300 camera capture carries a vivo device manifest (`digitalCapture`) that the bundled reader cannot open (`unknown algorithm`). -- **Chinese phone galleries.** Huawei's help pages say the Xiaoyi photo-edit AI +- **Open - Chinese phone galleries.** Huawei's help pages say the Xiaoyi photo-edit AI watermark can be switched off; vivo's Album setting is already covered. The public-web follow-up below found Honor YOYO metadata and additional primary-source labeling evidence, but not a controlled gallery export set. A live @@ -159,11 +183,12 @@ This page is an index. Each finding lives in its canonical page. of an unedited control, an ordinary crop, and a generative edit are still needed per app before selecting an AI attribution rule. Record the device, OS/app version, region, operation, and original download path with each sample. -- **Claims not supported by a primary source.** Kling embedding SynthID +- **Closed - claims not supported by a primary source.** Kling embedding SynthID (repeated by remover blogs; Kuaishou is not in Google's adopter list), Hailuo pixel watermarks, and platforms stripping C2PA on upload. None is used by the library. -- **Google Photos small edits (further work deferred, 2026-10-01).** The +- **Partially complete - Google Photos small edits (further work deferred, + 2026-10-01).** The maintainer deferred further photographic tiny Magic Eraser captures and verification retries for the inconclusive small edits. Resume only on an explicit request; retain the existing evidence and uncertainty. On @@ -193,8 +218,13 @@ This page is an index. Each finding lives in its canonical page. negative and one inconclusive reply. The quota refusal, synthetic capture, pixel comparison, mode caveat, and hash-bound outcomes are recorded in [synthid.md](synthid.md). -- **Captures blocked by region or device.** Doubao, Jimeng, Samsung. Higgsfield, - Runway and Dreamina were captured on paid plans on 2026-09-24. +- **Partially complete - direct captures blocked by region or device.** Direct, + controlled captures remain blocked for Doubao, Jimeng, and Samsung. The + public-web follow-up below later found one real marked Doubao video and + validated its visible mark and TC260 metadata, but the file remains local + because publication has not been cleared; it is not a controlled account + capture or a redistributable fixture. Higgsfield, Runway, and Dreamina were + captured on paid plans on 2026-09-24. ### Public-web sample search, 2026-10-01 UTC @@ -251,10 +281,16 @@ provider exports. No third-party photograph was added to tracked fixtures. [Getty's generator page](https://www.gettyimages.com/ai/generation), [iStock's generator page](https://www.istockphoto.com/ai/generation), and [Reface's product site](https://reface.ai/) did not carry locally recognized - C2PA. Jasper explicitly distinguishes input/mask/output examples, but an - unsigned web derivative cannot establish its signed export's manifest shape. + C2PA. A 2026-10-01 recheck of Jasper's current official Image landing page + reached its pre-rendered endpoint outputs directly and again found no C2PA. + Jasper explicitly distinguishes input/mask/output examples, but an unsigned + web derivative cannot establish its signed export's manifest shape. Conformance certificates alone still do not justify fabricating that shape. -- **Visible-image/video gaps remain.** The downloaded RunningHub workflow +- **Visible-image/video provider-original gaps remain.** Every registered mark + already has a publication-safe synthetic detector fixture in the + [visible-mark gallery](../data/fixtures/visible/README.md); the sample ledgers + now point to those fixtures and distinguish them from provider evidence. The + downloaded RunningHub workflow illustrations are explanatory/branding assets, not verified output examples; Liblib API documentation's cat illustration has no visible historical wordmark. A public [Inkpainting model page](https://www.liblib.art/modelinfo/2acd393c23924ff9b5363c9c1ec9129f)