Track the labeled SynthID corpus; complete metadata-source test coverage

Corpus images were gitignored (local-only). The negatives were reviewed and
cleared for publishing, so the labeled set is now committed (regular git, 65 MB
across 25 files) -- making the removal regression set reproducible and CI-able.

Corpus:
- Track data/synthid_corpus/images/ (pos 9, neg 15, cleaned 1); keep only the
  synthetic refs/ calibration fills gitignored.
- Reconcile manifest.csv to the on-disk files: 117 -> 25 rows (92 dangling rows
  for removed images pruned; dedup left one cleaned output, f6dd47a5).
- Rewrite the corpus README layout/policy (images committed; review every image
  for private content before adding -- public repo, permanent history).

Test fixtures:
- Remove data/samples/not-ai-1/2/3 (personal iPhone photos, incl. GPS EXIF).
- Add the clean_photo conftest fixture serving a verified-negative image from
  the corpus neg/ set; repoint the three "non-AI / clean photo" tests onto it
  (skips if the corpus is absent).

Metadata-source coverage (close the last sub-variant gaps):
- c2pa digitalSourceType: algorithmicMedia (procedural, not flagged AI) and
  compositeWithTrainedAlgorithmicMedia (AI + SynthID proxy).
- exif_generator: EXIF Artist and ImageDescription fields (Software/Make/XMP
  CreatorTool were already covered).

All 8 metadata-source kinds are now tested at both the unit and identify()
level. 313 tests pass. CLAUDE.md updated (corpus tracked, clean_photo fixture).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
test-user
2026-05-25 14:46:47 -07:00
co-authored by Claude Opus 4.7
parent 3ebdee57b8
commit 03fb460f77
36 changed files with 82 additions and 115 deletions
+20 -2
View File
@@ -224,8 +224,8 @@ class TestSynthIDSource:
assert synthid_source(SAMPLES_DIR / "firefly-1.png") is None
assert "synthid_watermark" not in get_ai_metadata(SAMPLES_DIR / "firefly-1.png")
def test_non_ai_image_is_not_synthid_source(self):
assert synthid_source(SAMPLES_DIR / "not-ai-1.jpeg") is None
def test_non_ai_image_is_not_synthid_source(self, clean_photo: Path):
assert synthid_source(clean_photo) is None
class TestSynthIDSourceNonPng:
@@ -377,6 +377,24 @@ class TestExifGenerator:
Image.new("RGB", (64, 64)).save(path, exif=exif)
assert exif_generator(path) is None
def test_artist_tag_ai_tool_detected(self, tmp_path: Path):
# exif_generator also reads the EXIF Artist field for an AI token.
exif = piexif.dump({"0th": {piexif.ImageIFD.Artist: b"Midjourney"}, "Exif": {}, "GPS": {}, "1st": {}})
path = tmp_path / "artist.jpg"
Image.new("RGB", (64, 64)).save(path, exif=exif)
assert exif_generator(path) == "Midjourney"
def test_imagedescription_tag_ai_tool_detected(self, tmp_path: Path):
# ...and the EXIF ImageDescription field.
exif = piexif.dump(
{"0th": {piexif.ImageIFD.ImageDescription: b"Made with Stable Diffusion"}, "Exif": {}, "GPS": {}, "1st": {}}
)
path = tmp_path / "desc.jpg"
Image.new("RGB", (64, 64)).save(path, exif=exif)
result = exif_generator(path)
assert result is not None
assert "Stable Diffusion" in result
def test_xmp_creatortool_scan_covers_unopenable(self, tmp_path: Path):
# PIL can't open this fake HEIF; the raw XMP CreatorTool scan still works.
path = tmp_path / "fake.heic"