Signal and detector-limit research with exact native resolutions and recorded verification sources.
Removal regression set: verify that a pipeline turns a SynthID-positive image into a negative one.

There is no reliable local detector of the SynthID pixel watermark (Google's decoder is proprietary). The ground-truth label therefore comes from an external oracle, recorded per image in verified_via (see below).

Layout

data/synthid/
  README.md                  # collection and verification protocol
  manifest.csv               # one row per committed original
  full-pipeline-quality.csv  # reusable evaluation selection
  originals/                 # one canonical copy of each verified source

The originals are committed so the evaluation set is reproducible and its hashes can be checked in CI. manifest.csv is kept in sync with originals/, one row per file. Generated outputs do not belong here. Store them outside the repository and record only a reproducible verdict in the evaluation manifest or documentation.

Reusable removal-quality set

full-pipeline-quality.csv is the canonical reusable input set for full-pipeline visual-quality and watermark-removal tests. Its fixtures are cleared for permanent public test reuse. Tests, benchmarks, and manual evaluation bundles may read these images repeatedly without requesting new permission.

The CSV preserves each platform's original filename and maps it to the one canonical corpus copy. Use corpus_path to read the input, but preserve source_filename in generated result names and keep OpenAI and Gemini outputs in separate provider groups. This avoids duplicating large PNGs while retaining the names needed for the corresponding provider oracle.

Rows with a dated negative final_clean_oracle were checked after the complete visible -> qwen-zimage -> metadata route with the matching provider oracle. not-checked-in-final-candidate means the input is available as an additional quality stress fixture, but a newly generated output still needs the matching oracle before claiming removal. The checked result applies to those exact output bytes; re-run the oracle after changing the pipeline, seed, model versions, or runtime settings.

Verification levels (`verified_via`)

Ground-truth quality, strongest first:

gemini-app — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
openai-verify — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
synthid-portal — checked via Google's SynthID Detector portal.
c2pa-metadata — issuer-only proxy (Google/OpenAI C2PA manifest present). Weaker: the C2PA can be stripped while the pixel watermark remains.
third-party — label asserted by an external dataset, not independently verified.
none — unverified.

Prefer gemini-app for any Google image that will gate a test.

What to collect

For the regression set:

A small number of positive originals with a recorded provider oracle.
A quality-set row for every original used by repeatable evaluation.
Re-run the provider oracle on each newly generated output. Do not commit the generated output as another corpus copy.

The corpus is committed to a public repo: review every image before adding it and keep out anything private or identifiable you would not publish.

Ingesting

Use scripts/synthid_corpus.py to copy a file into originals/, record its SHA-256, resolution, format, and C2PA issuer through our detector, and append a row to manifest.csv:

uv run python scripts/synthid_corpus.py ingest path/to/*.png \
    --label pos --source "Gemini app" --model gemini-3-pro \
    --verified-via gemini-app --notes "1024x1024 batch"

uv run python scripts/synthid_corpus.py status   # counts by label / resolution / verification

Autonomous collection via Chrome MCP

Historical collection notes. Browser interfaces and download behavior can change independently of this repository. These steps do not define package behavior.

Generation can be driven through the browser (the account must be logged in):

Gemini (gemini.google.com): type Create an image: <prompt>, wait, hover the result, click the download icon (top-right). Single, reliable click. Outputs carry Google C2PA + SynthID. Occasionally the composer stalls in a "generating" state -> start a New chat to reset.

ChatGPT (chatgpt.com): the UI download is flaky (the fullscreen viewer races and can grab the previous image; the share-modal path works but is multi-step). Reliable path is an in-page fetch of the rendered image, which preserves the original bytes (C2PA intact, unlike a canvas re-encode):

// run in the ChatGPT tab via the browser MCP javascript tool
(async () => {
  const imgs = [...document.querySelectorAll('img')].filter(i => i.naturalWidth >= 400);
  const img = imgs[imgs.length - 1];                 // newest large image
  const b = await (await fetch(img.currentSrc || img.src)).blob();
  const a = document.createElement('a');
  a.href = URL.createObjectURL(b); a.download = 'dl.png';
  document.body.appendChild(a); a.click(); a.remove();
  return 'size=' + b.size;                            // do NOT return the src (privacy guard blocks query strings)
})()

Gotcha: confirm the returned size differs from the previous image before ingesting -- if the new image has not finished rendering, the script grabs the prior one (the corpus dedups by sha256, but the notes would mislabel it). ChatGPT also shows an A/B "which is better?" picker; click Skip first.

Originals, not previews. Some platforms render a low-res preview in the chat (Grok serves a ~20KB 1024px JPEG/PNG; the in-page <img> fetch grabs that, not the original). Previews are re-encoded and strip metadata, so a "clean" preview is not proof the original is clean. Always pull the original via the platform's native Download / lightbox button and sanity-check the file size (a 20KB "1024x1024" is a preview). ChatGPT's in-chat <img> is the full-res oaiusercontent original, so fetch+blob is fine there; Grok needs the lightbox Download; Leonardo serves the original JPEG in-chat (download button matches).

Per-platform watermark map (observed, May 2026)

This is a dated observation table, not a live provider compatibility promise. The detector's coverage is complementary: metadata catches C2PA / IPTC; exif_generator catches EXIF Make/Software + XMP CreatorTool; invisible_watermark.py (imwatermark) catches the open SD/SDXL/FLUX DWT-DCT watermark on pristine files; the visible detector catches the Gemini-family sparkle; the SynthID pixel itself has no local detector (oracle only).

Platform	C2PA issuer	SynthID pixel	IPTC "Made with AI"	Visible sparkle	imwatermark	Corpus label
Gemini app	Google	yes	-	yes	-	pos
ChatGPT / gpt-image	OpenAI	yes	-	-	-	pos
Microsoft Designer	OpenAI + Microsoft	yes (via OpenAI)	-	-	-	pos
Bing Image Creator	Microsoft (MAI-Image)	no	-	-	-	pos (C2PA "Microsoft", not OpenAI)
Google AI Studio (Nano Banana)	none	yes (oracle-confirmed)	-	yes	-	pos (metadata blind spot)
Stability AI (Brand Studio)	Stability AI Ltd	no	-	-	no	pos (C2PA only)
Ideogram	none	no	-	-	no	pos (EXIF `Make="Ideogram AI"` only)
Meta AI	none	no	yes	-	-	neg (for SynthID)
Leonardo.ai	none	no	no	-	no	neg
Recraft	none (export strips)	no	no	-	no	neg (re-encoded export, no signal)
Krea (FLUX 2 host)	none	no	no	-	no	neg (host omits the imwatermark encoder)
Grok (xAI)	none (non-adopter)	no	no	-	no	neg (captured: clean low-res preview)

Key takeaways:

The same model differs by surface: Gemini app wraps C2PA, AI Studio (API/playground) emits none -- only the pixel + sparkle survive.
Microsoft Designer's DALL-E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own MAI-Image and signs C2PA as "Microsoft" (not OpenAI/DALL-E).
Meta uses the IPTC digitalSourceType marker, not C2PA or SynthID.
The open imwatermark fires only on pristine output from a pipeline that runs the encoder (diffusers default, official BFL) -- not from re-hosts (Krea, Stability hosted SDXL) or re-encoded design exports (Recraft, Canva). Ideogram's only signal is the EXIF Make tag.
Bing and Grok web UIs are uncooperative for autonomous capture (no document_idle for screenshots; blob downloads intermittently no-op; low-res in-chat previews). Use their native download button manually if a full-res sample is needed.

README.md

SynthID oracle fixtures