mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-25 06:52:28 +02:00
feat(metadata): SynthID-source detection, C2PA parser consolidation, corpus + tests
Detect SynthID-bearing images via their C2PA companion: a manifest signed by a
SynthID-using vendor (Google/OpenAI) on AI-generated content implies an
invisible SynthID pixel watermark. Verified end-to-end against the vendor
oracles (openai.com/verify, Gemini "Verify with SynthID").
- metadata: synthid_source() + synthid_watermark verdict in get_ai_metadata,
surfaced as a `metadata --check` callout. Format-agnostic (PNG caBX parser +
JPEG/WebP/AVIF/HEIF/JXL binary scan).
- constants: SYNTHID_C2PA_ISSUERS {Google, OpenAI}; +opened/placed actions.
- c2pa: single CBOR-aware parser (_cbor_text_after) replaces glitchy regex
(fixes fGPT-4o claim_generator); removed duplicate _scan_png_c2pa_chunk from
metadata; shared synthid_verdict / synthid_vendors_in helpers.
- corpus: scripts/synthid_corpus.py ingest tool + data/synthid_corpus/
(manifest tracked, images gitignored) for a labeled reference set.
- tests: +38 across C2PA parser internals, extract/inject round-trip, ISOBMFF
container stripping, all IPTC AI markers, and invisible watermark strength
tiers (SynthID/StableSignature/TreeRing/StegaStamp/RingID/RivaGAN/...).
Pixel-level SynthID detection remains out of reach locally (Google's decoder is
proprietary); a from-scratch spectral pilot confirmed it does not separate real
content. See CLAUDE.md for the full evaluation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
c1ff4e1cd9
commit
f07ce10c72
@@ -0,0 +1,75 @@
|
||||
# SynthID reference corpus
|
||||
|
||||
A locally-collected, labeled image corpus for SynthID work. Two downstream uses:
|
||||
|
||||
1. **Per-resolution spectral codebook** for an experimental SynthID detector
|
||||
(carrier frequencies are resolution-dependent, so labels must record the
|
||||
exact native resolution).
|
||||
2. **Removal regression set** — verify that our pipeline turns a SynthID-positive
|
||||
image into a negative one.
|
||||
|
||||
There is no reliable local detector of the SynthID pixel watermark (Google's
|
||||
decoder is proprietary). The ground-truth label therefore comes from an
|
||||
external oracle, recorded per image in `verified_via` (see below).
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
data/synthid_corpus/
|
||||
README.md # this protocol (committed)
|
||||
manifest.csv # labels + provenance (committed, reviewable)
|
||||
images/ # the actual files (gitignored, local-only)
|
||||
pos/ # SynthID present
|
||||
neg/ # SynthID absent
|
||||
cleaned/ # our pipeline output from a pos image
|
||||
```
|
||||
|
||||
Images are gitignored on purpose: the corpus is large, may contain personal or
|
||||
licensed content, and SynthID-positive outputs are best kept local. The
|
||||
`manifest.csv` (sha256 + labels + extracted metadata) is the durable artifact.
|
||||
|
||||
## Verification levels (`verified_via`)
|
||||
|
||||
Ground-truth quality, strongest first:
|
||||
|
||||
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
|
||||
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
|
||||
- `synthid-portal` — checked via Google's SynthID Detector portal.
|
||||
- `c2pa-metadata` — issuer-only proxy (Google/OpenAI C2PA manifest present). Weaker: the C2PA can be stripped while the pixel watermark remains.
|
||||
- `third-party` — label asserted by an external dataset, not independently verified.
|
||||
- `none` — unverified.
|
||||
|
||||
Prefer `gemini-app` for any image that will train the codebook or gate a test.
|
||||
|
||||
## What to collect
|
||||
|
||||
For the **codebook** (per target resolution, e.g. 1024x1024, 1024x1536, 1536x2816):
|
||||
|
||||
- 30-50+ SynthID-positive outputs per resolution (more is better; ~150-200 per
|
||||
resolution materially improves carrier discovery).
|
||||
- At each target resolution, also a batch of **pure-black (#000000)** and
|
||||
**pure-white (#FFFFFF)** fills generated by the SynthID model — these isolate
|
||||
the content-independent carrier (the watermark is most of the signal there).
|
||||
|
||||
For the **regression set**:
|
||||
|
||||
- A handful of `pos` images, their `cleaned` counterparts (run through our
|
||||
pipeline), and the cleaned re-verified via `gemini-app` (should read negative).
|
||||
- `neg` controls: non-AI photos and outputs from non-SynthID models (SD,
|
||||
Midjourney, Firefly) verified negative.
|
||||
|
||||
Avoid personal or identifiable content; the corpus stays local.
|
||||
|
||||
## Ingesting
|
||||
|
||||
Use `scripts/synthid_corpus.py` — it copies a file in, records its sha256,
|
||||
resolution, format, and C2PA issuer (via our own detector), and appends a row
|
||||
to `manifest.csv`:
|
||||
|
||||
```bash
|
||||
uv run python scripts/synthid_corpus.py ingest path/to/*.png \
|
||||
--label pos --source "Gemini app" --model gemini-3-pro \
|
||||
--verified-via gemini-app --notes "1024x1024 batch"
|
||||
|
||||
uv run python scripts/synthid_corpus.py status # counts by label / resolution / verification
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
sha256,filename,label,source,model,width,height,format,c2pa_issuer,synthid_metadata,verified_via,added,notes
|
||||
4ef377bde1a1d4eff141972841938643b173f5052992a018b9a21b31ac31731e,"4ef377bd-ChatGPT Image May 23, 2026, 02_43_02 PM.png",pos,ChatGPT,gpt-image,1254,1254,png,OpenAI,yes,openai-verify,2026-05-23T21:48:12Z,fresh post-rollout 2026-05-23; openai.com/verify: SynthID+C2PA detected
|
||||
d09f84c0e4c6d8b336bf4a9a7277314e940dcb5052ae7051e785cbb3bb42d656,d09f84c0-Gemini_Generated_Image_vq7wkwvq7wkwvq7w.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,c2pa-metadata,2026-05-23T21:52:40Z,"user: latest Gemini, SynthID v2"
|
||||
47188e88f956291bd38ab6906e5f21eb273d4a697ddc8b4479deac9f48915e1a,47188e88-disco_synthid_removed.png,cleaned,our pipeline (invisible/SDXL),stabilityai/stable-diffusion-xl-base-1.0,1254,1254,png,,,openai-verify,2026-05-23T22:06:54Z,cleaned from 4ef377bd disco; openai.com/verify: SynthID NOT detected (defeated)
|
||||
|
Reference in New Issue
Block a user