remove-ai-watermarks/data/synthid_corpus

SynthID reference corpus

A locally-collected, labeled image corpus for SynthID work. Two downstream uses:

1. Per-resolution spectral codebook for an experimental SynthID detector (carrier frequencies are resolution-dependent, so labels must record the exact native resolution).
2. Removal regression set — verify that our pipeline turns a SynthID-positive image into a negative one.

There is no reliable local detector of the SynthID pixel watermark (Google's decoder is proprietary). The ground-truth label therefore comes from an external oracle, recorded per image in verified_via (see below).

Layout

data/synthid_corpus/
  README.md        # this protocol (committed)
  manifest.csv     # labels + provenance (committed, reviewable)
  images/          # the actual files (gitignored, local-only)
    pos/           # SynthID present
    neg/           # SynthID absent
    cleaned/       # our pipeline output from a pos image

Images are gitignored on purpose: the corpus is large, may contain personal or licensed content, and SynthID-positive outputs are best kept local. The manifest.csv (sha256 + labels + extracted metadata) is the durable artifact.

Verification levels (verified_via)

Ground-truth quality, strongest first:

• gemini-app — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
• openai-verify — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
• synthid-portal — checked via Google's SynthID Detector portal.
• c2pa-metadata — issuer-only proxy (Google/OpenAI C2PA manifest present). Weaker: the C2PA can be stripped while the pixel watermark remains.
• third-party — label asserted by an external dataset, not independently verified.
• none — unverified.

Prefer gemini-app for any image that will train the codebook or gate a test.

What to collect

For the codebook (per target resolution, e.g. 1024x1024, 1024x1536, 1536x2816):

• 30-50+ SynthID-positive outputs per resolution (more is better; ~150-200 per resolution materially improves carrier discovery).
• At each target resolution, also a batch of pure-black (#000000) and pure-white (#FFFFFF) fills generated by the SynthID model — these isolate the content-independent carrier (the watermark is most of the signal there).

For the regression set:

• A handful of pos images, their cleaned counterparts (run through our pipeline), and the cleaned re-verified via gemini-app (should read negative).
• neg controls: non-AI photos and outputs from non-SynthID models (SD, Midjourney, Firefly) verified negative.

Avoid personal or identifiable content; the corpus stays local.

Ingesting

Use scripts/synthid_corpus.py — it copies a file in, records its sha256, resolution, format, and C2PA issuer (via our own detector), and appends a row to manifest.csv:

uv run python scripts/synthid_corpus.py ingest path/to/*.png \
    --label pos --source "Gemini app" --model gemini-3-pro \
    --verified-via gemini-app --notes "1024x1024 batch"

uv run python scripts/synthid_corpus.py status   # counts by label / resolution / verification

Autonomous collection via Chrome MCP

Generation can be driven through the browser (the account must be logged in):

• Gemini (gemini.google.com): type Create an image: <prompt>, wait, hover the result, click the download icon (top-right). Single, reliable click. Outputs carry Google C2PA + SynthID. Occasionally the composer stalls in a "generating" state -> start a New chat to reset.

• ChatGPT (chatgpt.com): the UI download is flaky (the fullscreen viewer races and can grab the previous image; the share-modal path works but is multi-step). Reliable path is an in-page fetch of the rendered image, which preserves the original bytes (C2PA intact, unlike a canvas re-encode):

  // run in the ChatGPT tab via the browser MCP javascript tool
  (async () => {
    const imgs = [...document.querySelectorAll('img')].filter(i => i.naturalWidth >= 400);
    const img = imgs[imgs.length - 1];                 // newest large image
    const b = await (await fetch(img.currentSrc || img.src)).blob();
    const a = document.createElement('a');
    a.href = URL.createObjectURL(b); a.download = 'dl.png';
    document.body.appendChild(a); a.click(); a.remove();
    return 'size=' + b.size;                            // do NOT return the src (privacy guard blocks query strings)
  })()
  

  Gotcha: confirm the returned size differs from the previous image before ingesting -- if the new image has not finished rendering, the script grabs the prior one (the corpus dedups by sha256, but the notes would mislabel it). ChatGPT also shows an A/B "which is better?" picker; click Skip first.