Track the labeled SynthID corpus; complete metadata-source test coverage

Corpus images were gitignored (local-only). The negatives were reviewed and
cleared for publishing, so the labeled set is now committed (regular git, 65 MB
across 25 files) -- making the removal regression set reproducible and CI-able.

Corpus:
- Track data/synthid_corpus/images/ (pos 9, neg 15, cleaned 1); keep only the
  synthetic refs/ calibration fills gitignored.
- Reconcile manifest.csv to the on-disk files: 117 -> 25 rows (92 dangling rows
  for removed images pruned; dedup left one cleaned output, f6dd47a5).
- Rewrite the corpus README layout/policy (images committed; review every image
  for private content before adding -- public repo, permanent history).

Test fixtures:
- Remove data/samples/not-ai-1/2/3 (personal iPhone photos, incl. GPS EXIF).
- Add the clean_photo conftest fixture serving a verified-negative image from
  the corpus neg/ set; repoint the three "non-AI / clean photo" tests onto it
  (skips if the corpus is absent).

Metadata-source coverage (close the last sub-variant gaps):
- c2pa digitalSourceType: algorithmicMedia (procedural, not flagged AI) and
  compositeWithTrainedAlgorithmicMedia (AI + SynthID proxy).
- exif_generator: EXIF Artist and ImageDescription fields (Software/Make/XMP
  CreatorTool were already covered).

All 8 metadata-source kinds are now tested at both the unit and identify()
level. 313 tests pass. CLAUDE.md updated (corpus tracked, clean_photo fixture).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
test-user
2026-05-25 14:46:47 -07:00
co-authored by Claude Opus 4.7
parent 3ebdee57b8
commit 03fb460f77
36 changed files with 82 additions and 115 deletions
+17 -4
View File
@@ -2,10 +2,7 @@
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from pathlib import Path
from pathlib import Path
import cv2
import numpy as np
@@ -13,6 +10,22 @@ import pytest
from PIL import Image
from PIL.PngImagePlugin import PngInfo
CORPUS_NEG_DIR = Path(__file__).resolve().parent.parent / "data" / "synthid_corpus" / "images" / "neg"
@pytest.fixture
def clean_photo() -> Path:
"""A verified-negative real photo from the corpus neg/ set.
Used by the "non-AI image" assertions (no SynthID, verdict unknown). These
are real photos with no AI provenance, the ground truth for "must not false-
positive". Skips if the corpus is not checked out.
"""
files = sorted(CORPUS_NEG_DIR.glob("*")) if CORPUS_NEG_DIR.exists() else []
if not files:
pytest.skip("no corpus neg/ images present")
return files[0]
@pytest.fixture
def tmp_image_path(tmp_path: Path) -> Path: