Decide the SynthID proxy in the verdict, where both extractors meet

A full-corpus audit of the record path against the file path found 75 of 48,905
images disagreeing, and 74 were one gap: the SynthID byte scan for containers whose
manifest no parser reaches lived in `get_ai_metadata`, an extractor the record path
does not run. The record silently reported no SynthID for images `identify` flagged.

Moving the scan into `identify_from_evidence` fixes it by construction rather than by
copying the rule into a second extractor -- the same shape `soft_binding` already
uses. Its byte checks mirror `metadata.synthid_source` literally instead of reusing
the broader `has_c2pa` / `c2pa_source_kind` derived above, so the file path's answers
do not move: verdicts over a 4,000-image sample are byte-identical.

`scripts/record_parity_audit.py` is the audit itself, now repeatable. It walks a
dataset, judges every image through both seams with the record round-tripped through
JSON, and reports disagreements by field and by signal. The rule in
`.claude/rules/development.md` says to re-run both sides of this seam after changing
either; this is what to run.

Both timing and audit scripts now put the package's OWN `src` on the path. From a
worktree an editable install resolves to the main checkout, so the audit imported a
different tree than the one under test -- the failure the same rules file warns about,
reproduced within an hour of writing it down.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Victor Kuznetsov
2026-08-05 21:10:38 -07:00
co-authored by Claude Opus 5
parent 2668f1302d
commit bebff368fc
5 changed files with 299 additions and 4 deletions
+19 -2
View File
@@ -30,6 +30,8 @@ from remove_ai_watermarks._internal.c2pa import (
cbor_text_after,
extract_c2pa_info,
soft_binding_vendors_in,
synthid_vendors_in,
synthid_verdict,
)
from remove_ai_watermarks._internal.constants import (
C2PA_AI_TOOLS,
@@ -980,9 +982,24 @@ def _identify_from_evidence(
platform = f"C2PA signer: {cloud_vendor} (cloud manifest)"
# ── SynthID metadata proxy ──────────────────────────────────────
# get_ai_metadata already sets synthid_watermark for both PNG (caBX parser)
# and non-PNG (its own synthid_source fallback), so no extra scan is needed.
# Structured first (the PNG caBX parser and the manifest store both fill
# `synthid_watermark`), then the byte scan for the containers that keep the
# manifest where no parser reaches it.
#
# The scan lives HERE, in the verdict, and not in extraction, for the same reason
# `soft_binding` below does: extraction has two implementations -- one reading a
# file, one reading a portable record -- and a rule that lives in only one of them
# is a rule the other silently lacks. It did: 74 corpus images reported SynthID
# through `identify` and not through the record, because `get_ai_metadata`'s own
# fallback has no counterpart on the record side. `get_ai_metadata` keeps its copy
# for its own callers; the verdict no longer depends on which extractor ran.
synthid = meta.get("synthid_watermark")
# The literal byte checks mirror `metadata.synthid_source` exactly rather than
# reusing the derived `has_c2pa` / `c2pa_source_kind` above, which are broader:
# the file path's answer must not move.
trained_source = b"trainedAlgorithmicMedia" in head or b"TrainedAlgorithmicMedia" in head
if not synthid and trained_source and c2pa_marker_in(head) and (vendors := synthid_vendors_in(head)):
synthid = synthid_verdict(", ".join(vendors))
if synthid:
watermarks.append(f"SynthID watermark, inferred from C2PA metadata ({synthid})")
caveats.append(_SYNTHID_CAVEAT)