Match the vendor registries against metadata, not coded pixels

The registries are raw substrings and the shortest tokens are four and five bytes
(`Bria`, `Adobe`, `Canva`). Over a megabyte of compressed pixel data such a sequence
turns up by chance: `Bria` matched inside the entropy-coded scan of 4 of 14,707
corpus JPEGs, in none of which the manifest names Bria. The rate is what a four-byte
pattern predicts on that corpus, and the Bria entry asserts AI, so a chance match can
declare an image AI-generated rather than merely mislabel its signer.

`_metadata_region` gives the registry scans the container's metadata: JPEG marker
segments before the coded scan, PNG chunks other than IDAT, both trailers, and
whatever `scan_head` appended past the window. Every other check keeps the full
buffer -- their markers are long and distinctive. A container that does not parse is
returned whole, since dropping real evidence to avoid a chance match is the wrong
trade. `c2pa_marker_in` already refuses a bare `c2pa` substring for this reason;
this is the same defence for the registries.

Verified the way the rules require for a change that MOVES a verdict: over all 48,905
corpus images, exactly one file changed, the one named in advance, from
"C2PA Content Credentials (Bria Artificial Intelligence)" to "(unknown signer)".
Record-path parity is 0 disagreements, down from 75 when this work started.

The audit's own baseline comparison is fixed here too. It compared confidence and
signals only, and so reported "0 changed" for the run whose single intended
correction was a watermark line -- the change it exists to show.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Victor Kuznetsov
2026-08-05 21:10:55 -07:00
co-authored by Claude Opus 5
parent 9a29dcac8a
commit 1124c591be
4 changed files with 147 additions and 12 deletions
+14 -6
View File
@@ -162,18 +162,26 @@ def _summarize(rows: list[dict[str, Any]], baseline: Path | None) -> None:
was = previous.get(Path(row["path"]).name)
if was is None:
continue
before = was.get("file") or {"confidence": was.get("confidence"), "signals": was.get("signals")}
if before.get("confidence") != row["file"]["confidence"] or before.get("signals") != row["file"]["signals"]:
# The WHOLE verdict, not a chosen subset. A first version compared confidence
# and signals only and reported "0 changed" for a run whose single intended
# correction was a watermark line -- the change it existed to show.
before = was.get("file") or {}
if before and before != row["file"]:
changed.append((row["path"], before, row["file"]))
print(f"\nverdicts changed against the baseline: {len(changed)}")
gained: collections.Counter[str] = collections.Counter()
moved: collections.Counter[str] = collections.Counter()
for _, before, after in changed:
for name in set(after["signals"]) - set(before.get("signals") or []):
gained[f"gained {name}"] += 1
moved[f"gained signal {name}"] += 1
for name in set(before.get("signals") or []) - set(after["signals"]):
gained[f"LOST {name}"] += 1
for label, count in gained.most_common():
moved[f"LOST signal {name}"] += 1
for field in (*COMPARED, "watermarks"):
if before.get(field) != after.get(field):
moved[f"{field} changed"] += 1
for label, count in moved.most_common():
print(f" {label}: {count}")
for path, before, after in changed[:10]:
print(f" {Path(path).name}\n before: {before}\n after: {after}")
def main() -> int: