feat(metadata): SynthID-source detection, C2PA parser consolidation, corpus + tests

Detect SynthID-bearing images via their C2PA companion: a manifest signed by a
SynthID-using vendor (Google/OpenAI) on AI-generated content implies an
invisible SynthID pixel watermark. Verified end-to-end against the vendor
oracles (openai.com/verify, Gemini "Verify with SynthID").

- metadata: synthid_source() + synthid_watermark verdict in get_ai_metadata,
  surfaced as a `metadata --check` callout. Format-agnostic (PNG caBX parser +
  JPEG/WebP/AVIF/HEIF/JXL binary scan).
- constants: SYNTHID_C2PA_ISSUERS {Google, OpenAI}; +opened/placed actions.
- c2pa: single CBOR-aware parser (_cbor_text_after) replaces glitchy regex
  (fixes fGPT-4o claim_generator); removed duplicate _scan_png_c2pa_chunk from
  metadata; shared synthid_verdict / synthid_vendors_in helpers.
- corpus: scripts/synthid_corpus.py ingest tool + data/synthid_corpus/
  (manifest tracked, images gitignored) for a labeled reference set.
- tests: +38 across C2PA parser internals, extract/inject round-trip, ISOBMFF
  container stripping, all IPTC AI markers, and invisible watermark strength
  tiers (SynthID/StableSignature/TreeRing/StegaStamp/RingID/RivaGAN/...).

Pixel-level SynthID detection remains out of reach locally (Google's decoder is
proprietary); a from-scratch spectral pilot confirmed it does not separate real
content. See CLAUDE.md for the full evaluation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
test-user
2026-05-24 11:32:46 -07:00
co-authored by Claude Opus 4.7
parent c1ff4e1cd9
commit f07ce10c72
14 changed files with 785 additions and 107 deletions
+75
View File
@@ -0,0 +1,75 @@
# SynthID reference corpus
A locally-collected, labeled image corpus for SynthID work. Two downstream uses:
1. **Per-resolution spectral codebook** for an experimental SynthID detector
(carrier frequencies are resolution-dependent, so labels must record the
exact native resolution).
2. **Removal regression set** — verify that our pipeline turns a SynthID-positive
image into a negative one.
There is no reliable local detector of the SynthID pixel watermark (Google's
decoder is proprietary). The ground-truth label therefore comes from an
external oracle, recorded per image in `verified_via` (see below).
## Layout
```
data/synthid_corpus/
README.md # this protocol (committed)
manifest.csv # labels + provenance (committed, reviewable)
images/ # the actual files (gitignored, local-only)
pos/ # SynthID present
neg/ # SynthID absent
cleaned/ # our pipeline output from a pos image
```
Images are gitignored on purpose: the corpus is large, may contain personal or
licensed content, and SynthID-positive outputs are best kept local. The
`manifest.csv` (sha256 + labels + extracted metadata) is the durable artifact.
## Verification levels (`verified_via`)
Ground-truth quality, strongest first:
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
- `synthid-portal` — checked via Google's SynthID Detector portal.
- `c2pa-metadata` — issuer-only proxy (Google/OpenAI C2PA manifest present). Weaker: the C2PA can be stripped while the pixel watermark remains.
- `third-party` — label asserted by an external dataset, not independently verified.
- `none` — unverified.
Prefer `gemini-app` for any image that will train the codebook or gate a test.
## What to collect
For the **codebook** (per target resolution, e.g. 1024x1024, 1024x1536, 1536x2816):
- 30-50+ SynthID-positive outputs per resolution (more is better; ~150-200 per
resolution materially improves carrier discovery).
- At each target resolution, also a batch of **pure-black (#000000)** and
**pure-white (#FFFFFF)** fills generated by the SynthID model — these isolate
the content-independent carrier (the watermark is most of the signal there).
For the **regression set**:
- A handful of `pos` images, their `cleaned` counterparts (run through our
pipeline), and the cleaned re-verified via `gemini-app` (should read negative).
- `neg` controls: non-AI photos and outputs from non-SynthID models (SD,
Midjourney, Firefly) verified negative.
Avoid personal or identifiable content; the corpus stays local.
## Ingesting
Use `scripts/synthid_corpus.py` — it copies a file in, records its sha256,
resolution, format, and C2PA issuer (via our own detector), and appends a row
to `manifest.csv`:
```bash
uv run python scripts/synthid_corpus.py ingest path/to/*.png \
--label pos --source "Gemini app" --model gemini-3-pro \
--verified-via gemini-app --notes "1024x1024 batch"
uv run python scripts/synthid_corpus.py status # counts by label / resolution / verification
```
+4
View File
@@ -0,0 +1,4 @@
sha256,filename,label,source,model,width,height,format,c2pa_issuer,synthid_metadata,verified_via,added,notes
4ef377bde1a1d4eff141972841938643b173f5052992a018b9a21b31ac31731e,"4ef377bd-ChatGPT Image May 23, 2026, 02_43_02 PM.png",pos,ChatGPT,gpt-image,1254,1254,png,OpenAI,yes,openai-verify,2026-05-23T21:48:12Z,fresh post-rollout 2026-05-23; openai.com/verify: SynthID+C2PA detected
d09f84c0e4c6d8b336bf4a9a7277314e940dcb5052ae7051e785cbb3bb42d656,d09f84c0-Gemini_Generated_Image_vq7wkwvq7wkwvq7w.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,c2pa-metadata,2026-05-23T21:52:40Z,"user: latest Gemini, SynthID v2"
47188e88f956291bd38ab6906e5f21eb273d4a697ddc8b4479deac9f48915e1a,47188e88-disco_synthid_removed.png,cleaned,our pipeline (invisible/SDXL),stabilityai/stable-diffusion-xl-base-1.0,1254,1254,png,,,openai-verify,2026-05-23T22:06:54Z,cleaned from 4ef377bd disco; openai.com/verify: SynthID NOT detected (defeated)
1 sha256 filename label source model width height format c2pa_issuer synthid_metadata verified_via added notes
2 4ef377bde1a1d4eff141972841938643b173f5052992a018b9a21b31ac31731e 4ef377bd-ChatGPT Image May 23, 2026, 02_43_02 PM.png pos ChatGPT gpt-image 1254 1254 png OpenAI yes openai-verify 2026-05-23T21:48:12Z fresh post-rollout 2026-05-23; openai.com/verify: SynthID+C2PA detected
3 d09f84c0e4c6d8b336bf4a9a7277314e940dcb5052ae7051e785cbb3bb42d656 d09f84c0-Gemini_Generated_Image_vq7wkwvq7wkwvq7w.png pos Gemini app gemini 2816 1536 png Google LLC yes c2pa-metadata 2026-05-23T21:52:40Z user: latest Gemini, SynthID v2
4 47188e88f956291bd38ab6906e5f21eb273d4a697ddc8b4479deac9f48915e1a 47188e88-disco_synthid_removed.png cleaned our pipeline (invisible/SDXL) stabilityai/stable-diffusion-xl-base-1.0 1254 1254 png openai-verify 2026-05-23T22:06:54Z cleaned from 4ef377bd disco; openai.com/verify: SynthID NOT detected (defeated)