mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-26 07:22:29 +02:00
Stop gating C2PA confidence on a trust anchor that never ships
High-confidence C2PA attribution required signingCredential.trusted, a status code the reader emits only when a trust anchor list is loaded. None ships, so from 0.27.0 through 0.30.0 the branch was unreachable in production for every vendor: an intact, cryptographically bound manifest scored the same medium as a fallback parse that validated nothing, which collapsed the one distinction the official reader exists to draw. A hand-built info dict stamping that code kept the branch green in the suite. Confidence now follows the binding. Signer trust and certificate expiry stay visible as their own dimensions and as caveats, because a trust list that was never configured is a missing input, not a finding against the credential. Every committed provenance fixture with a reader result and an intact binding now reaches high confidence, and test_no_committed_fixture_reports_a_trusted_signer guards the reachability itself rather than a synthesized status set. Revocation joins binding and signature failures as disqualifying. It arrives only on signer_validity, so a check reading the other two returned a confident AI verdict off a credential the issuer had disowned, with an empty integrity_clashes -- quieter than a hash mismatch on the same file. Expiry stays non-disqualifying: it does not imply the signed bytes changed, and a signature genuinely made outside validity already arrives as claimSignature.outsideValidity. The rule now lives in one place. _validation_fields maps status codes to the four dimensions and names the failures that moved one; c2pa_info_has_invalid_credential maps dimensions to disqualified. The ingredient-reachability walk and the user-visible reason both consume that path instead of re-classifying raw codes, so adding this one rule no longer means editing three layers in lockstep. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
f83da5aee4
commit
c927560614
@@ -65,6 +65,15 @@ that is what actually happened.
|
||||
|
||||
Use availability checks only for paths that actually load large models.
|
||||
|
||||
A verdict branch must be reachable with the dependency stack as shipped. High-confidence
|
||||
C2PA attribution required `signingCredential.trusted`, a code the reader emits only when a
|
||||
trust anchor list is loaded, and none ships -- so the branch was dead in production for
|
||||
every vendor while a hand-built info dict stamping that code kept it green from 0.27.0
|
||||
through 0.30.0. Assert reachability against the committed fixtures rather than a synthesized
|
||||
status set; `TestIdentifyRealSamples::test_no_committed_fixture_reports_a_trusted_signer`
|
||||
is that guard. Read a capability the stack does not have as a missing input, never as a
|
||||
negative finding.
|
||||
|
||||
## One measurement, one gate seam
|
||||
|
||||
A detector is split into a trust-level-blind scan and a verdict that applies the
|
||||
@@ -98,6 +107,11 @@ record is byte-identical, and a green test suite does not establish that on its
|
||||
change that is meant to FIX detection is the exception that proves the rule: the diff
|
||||
must then be exactly the files you intended to change, named in advance.
|
||||
|
||||
When the baseline is a published release rather than the previous commit, get it with
|
||||
`uv run --isolated --no-project --with 'remove-ai-watermarks[heif]==<version>' python <script>`:
|
||||
it resolves that release from PyPI without touching the working tree or the editable
|
||||
install. Quote the extra -- bare `pkg[extra]` is a glob in zsh.
|
||||
|
||||
Two corollaries in `dwt_dct.py`, where "close enough" has an exact meaning. The bit test
|
||||
is `peak % 36 > 18.0` and for uint8 input the exact Haar value is a multiple of 0.5, so it
|
||||
lands ON the threshold once per 72 blocks and a 1-ulp difference flips real bits: leave the
|
||||
|
||||
+5
-3
@@ -41,9 +41,11 @@ remove-ai-watermarks identify image.png
|
||||
it also evaluates supported visible and invisible pixel signals. When no signal
|
||||
is found, it reports the origin as unknown. It does not claim the image is
|
||||
clean. For C2PA files, the text report shows asset integrity, claim-signature,
|
||||
signer-trust, and signer-validity results separately. An intact claim from an
|
||||
untrusted or expired signer is reported at medium confidence; a failed asset
|
||||
binding or signature does not confirm the claimed origin. When a structured
|
||||
signer-trust, and signer-validity results separately. Confidence follows the
|
||||
binding: an intact asset binding and claim signature is high confidence, and an
|
||||
unanchored or expired signer is reported as a caveat rather than a lower score,
|
||||
because no trust anchor list ships with the reader. A failed asset binding or
|
||||
signature, or a revoked signing credential, does not confirm the claimed origin. When a structured
|
||||
C2PA soft binding is present, the report also names its exact algorithm and
|
||||
signed value; removing the manifest does not remove the referenced pixel
|
||||
watermark or content fingerprint.
|
||||
|
||||
@@ -379,13 +379,31 @@ Structured extraction is limited to the active manifest and the ingredient
|
||||
manifests reachable from it. Validation is preserved as separate dimensions:
|
||||
asset binding integrity, claim signature, signer trust, and signer certificate
|
||||
validity. A matching asset hash and valid claim signature do not make an
|
||||
untrusted or expired signer trusted. `identify` therefore assigns high confidence
|
||||
only when all four dimensions validate, medium confidence to an intact but
|
||||
untrusted or unvalidated claim, and no origin verdict to a hash/signature failure.
|
||||
The failed claim remains in the marker inventory and keeps the removal gate
|
||||
fail-safe because a post-signing container edit can invalidate C2PA without
|
||||
untrusted or expired signer trusted, so those stay separate fields and separate
|
||||
caveats -- but they do not lower confidence. `identify` assigns high confidence
|
||||
when the asset binding and the claim signature both validate, medium confidence
|
||||
to a claim that validated nothing (fallback parsing, unknown dimensions), and no
|
||||
origin verdict to a binding failure, a signature failure, or a revoked signing
|
||||
credential. The failed claim remains in the marker inventory and keeps the removal
|
||||
gate fail-safe because a post-signing container edit can invalidate C2PA without
|
||||
removing a declared pixel watermark.
|
||||
|
||||
Revocation reaches the disqualifying branch through `signer_validity`, not through
|
||||
binding or signature. A check that read only the latter two returned a confident AI
|
||||
verdict off a credential the issuer had disowned, with an empty `integrity_clashes`
|
||||
-- quieter than a hash mismatch on the same file. Certificate expiry is deliberately
|
||||
not disqualifying: an expired certificate does not imply the signed bytes changed,
|
||||
and a signature actually made outside validity already arrives as
|
||||
`claimSignature.outsideValidity`.
|
||||
|
||||
One rule, one place. `_validation_fields` maps status codes to the four dimensions
|
||||
and also emits `c2pa_failed_codes`, the subset of failures that actually drove a
|
||||
dimension to invalid; `c2pa_info_has_invalid_credential` maps those dimensions to
|
||||
disqualified, and both the ingredient-reachability walk and the report consume that
|
||||
single path. The displayed reason comes from `c2pa_failed_codes` rather than a
|
||||
substring rescan of the full code list, so what a caller is shown as the cause
|
||||
cannot become a looser rule than the verdict it explains.
|
||||
|
||||
The structured walk also treats an exact known AI product in a reachable
|
||||
`claim_generator` as an AI assertion. This covers update chains where the active
|
||||
manifest names only `c2pa-tool` while a validated ingredient names Dreamina, and
|
||||
@@ -405,8 +423,24 @@ Content-fingerprint soft bindings do not trigger pixel regeneration.
|
||||
|
||||
The SDK default enables trust verification but supplies no production trust
|
||||
anchors. Consequently, an installation without an explicitly maintained C2PA
|
||||
trust bundle reports otherwise valid signer chains as untrusted and keeps their
|
||||
attribution at medium confidence. Shipping or fetching an official trust bundle
|
||||
trust bundle reports every otherwise valid signer chain as untrusted --
|
||||
`signingCredential.trusted` appears in no default installation, from any vendor.
|
||||
Confidence therefore must not depend on it. It did from 0.27.0 through 0.30.0, which made
|
||||
the high-confidence branch unreachable in production while a hand-built fixture
|
||||
stamping `signingCredential.trusted` kept it green in the suite; measured on the
|
||||
committed provenance fixtures, every C2PA file from OpenAI, Adobe and Black Forest
|
||||
Labs came back untrusted, and an intact manifest scored the same medium as a
|
||||
fallback parse that validated nothing. That collapsed the one distinction the
|
||||
official reader exists to draw.
|
||||
`tests/test_identify.py::TestIdentifyRealSamples::test_no_committed_fixture_reports_a_trusted_signer`
|
||||
is the reachability guard: it asserts the fixtures really are untrusted and still
|
||||
reach high confidence.
|
||||
|
||||
Read `untrusted` here as a missing input, not a finding: nothing was checked,
|
||||
because there was nothing to check against. If a maintained trust bundle is ever
|
||||
configured, that stops being true and the confidence mapping in
|
||||
`_c2pa_credential_level` must be re-read, because only then does a failed trust
|
||||
check mean the signer was rejected. Shipping or fetching an official trust bundle
|
||||
requires a separate update, provenance, and availability policy; do not silently
|
||||
convert `signingCredential.untrusted` into trusted based on a vendor-name match.
|
||||
|
||||
|
||||
+20
-3
@@ -160,9 +160,26 @@ print(report.c2pa_validation)
|
||||
|
||||
`c2pa_validation`, when present, reports `integrity`, `signature`,
|
||||
`signer_trust`, and `signer_validity` independently, plus the reader status
|
||||
codes. A valid hash and signature with an untrusted or expired signer is a
|
||||
medium-confidence signed claim. A hash or signature failure does not confirm the
|
||||
claimed platform or AI origin. Fallback parsing reports unknown validation
|
||||
codes. A valid hash and signature is a high-confidence signed claim; an
|
||||
unanchored or expired signer appears in `caveats` and in these fields, not as a
|
||||
lower confidence, because the reader ships no trust anchors to check against and
|
||||
`signer_trust` is therefore a missing input rather than a finding. A hash or
|
||||
signature failure, or a revoked signing credential, does not confirm the claimed
|
||||
platform or AI origin.
|
||||
|
||||
A consumer must read `integrity_clashes`. When a credential fails validation,
|
||||
`is_ai_generated` becomes `None`, because a claim that cannot be tied to these
|
||||
bytes cannot establish origin -- the manifest may have been transplanted from a
|
||||
real AI image onto anything -- and the failure is reported in
|
||||
`integrity_clashes` instead. That is a different question from whether an AI
|
||||
watermark is physically present in the pixels, which is what
|
||||
`has_invisible_target` answers, and it stays fail-safe `True` on the same file.
|
||||
Reading only `is_ai_generated` turns a broken vendor manifest into silence.
|
||||
|
||||
`c2pa_validation["state"]` is the reader's own aggregate and is carried for
|
||||
diagnostics only; no verdict is derived from it, because it collapses a
|
||||
transplanted manifest and a merely expired certificate into one `Invalid`, and
|
||||
its `Trusted` level depends on anchors no default installation has. Fallback parsing reports unknown validation
|
||||
dimensions, while a raw marker in an unsupported or malformed container can
|
||||
leave `c2pa_validation` as `None`.
|
||||
|
||||
|
||||
@@ -88,9 +88,11 @@ The inspection and stripping code handles signals in these groups:
|
||||
unknown when evidence is absent. It never treats missing metadata as proof that
|
||||
an image is human made. C2PA presence alone is not a verified identity: the
|
||||
report distinguishes asset binding, claim signature, signer trust, and signer
|
||||
validity. High-confidence C2PA attribution requires all four; intact but
|
||||
untrusted or fallback claims are medium-confidence, while a failed binding or
|
||||
signature contributes no origin verdict.
|
||||
validity. High-confidence C2PA attribution requires an intact asset binding and
|
||||
claim signature; signer trust and certificate expiry are reported as their own
|
||||
dimensions and as caveats, since no trust anchor list ships to evaluate them.
|
||||
Fallback claims, which validate nothing, remain medium-confidence, while a failed
|
||||
binding or signature or a revoked credential contributes no origin verdict.
|
||||
|
||||
## File and container formats
|
||||
|
||||
|
||||
@@ -62,6 +62,16 @@ _SIGNATURE_FAILURES = frozenset(
|
||||
"claimSignature.outsideValidity",
|
||||
}
|
||||
)
|
||||
# A credential the issuer itself has disowned, disqualifying like a hash mismatch.
|
||||
# ``signingCredential.expired`` is deliberately absent: an expired certificate does not
|
||||
# imply the signed bytes changed, and a signature actually made outside validity already
|
||||
# arrives as ``claimSignature.outsideValidity`` above.
|
||||
_CREDENTIAL_FAILURES = frozenset(
|
||||
{
|
||||
"signingCredential.invalid",
|
||||
"signingCredential.ocsp.revoked",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -293,16 +303,24 @@ def _status_codes(store: dict[str, Any]) -> tuple[list[str], list[str], list[str
|
||||
|
||||
|
||||
def _store_has_invalid_credential(store: dict[str, Any]) -> bool:
|
||||
"""Return whether this store's own active credential failed validation."""
|
||||
_, _, failures = _status_codes(store)
|
||||
return any(marker in code for code in failures for marker in _CONTENT_BINDING_FAILURE_MARKERS) or any(
|
||||
code in _SIGNATURE_FAILURES for code in failures
|
||||
)
|
||||
"""Return whether this store's own active credential failed validation.
|
||||
|
||||
Deliberately routed through the same codes -> dimensions -> disqualified path the
|
||||
report uses, rather than re-testing the raw codes here. When this walk classified
|
||||
failures on its own, adding one rule meant editing it and
|
||||
:func:`c2pa_info_has_invalid_credential` in lockstep, and the ingredient
|
||||
reachability walk was free to drift away from the verdict it feeds.
|
||||
"""
|
||||
return c2pa_info_has_invalid_credential(_validation_fields(store))
|
||||
|
||||
|
||||
def c2pa_info_has_invalid_credential(info: dict[str, Any]) -> bool:
|
||||
"""Return whether parsed C2PA info reports a failed binding or signature."""
|
||||
return info.get("c2pa_integrity") == "invalid" or info.get("c2pa_signature") == "invalid"
|
||||
"""Return whether parsed C2PA info reports a failed binding, signature, or credential."""
|
||||
return (
|
||||
info.get("c2pa_integrity") == "invalid"
|
||||
or info.get("c2pa_signature") == "invalid"
|
||||
or info.get("c2pa_signer_validity") == "invalid"
|
||||
)
|
||||
|
||||
|
||||
def c2pa_info_has_invismark(info: dict[str, Any]) -> bool:
|
||||
@@ -324,9 +342,16 @@ def _validation_fields(store: dict[str, Any]) -> dict[str, Any]:
|
||||
successes, informationals, failures = _status_codes(store)
|
||||
all_codes = list(dict.fromkeys([*successes, *informationals, *failures]))
|
||||
|
||||
binding_failed = any(marker in code for code in failures for marker in _CONTENT_BINDING_FAILURE_MARKERS)
|
||||
disqualifying = [
|
||||
code
|
||||
for code in failures
|
||||
if any(marker in code for marker in _CONTENT_BINDING_FAILURE_MARKERS)
|
||||
or code in _SIGNATURE_FAILURES
|
||||
or code in _CREDENTIAL_FAILURES
|
||||
]
|
||||
binding_failed = any(marker in code for code in disqualifying for marker in _CONTENT_BINDING_FAILURE_MARKERS)
|
||||
binding_matched = _has_suffix(successes, _CONTENT_BINDING_MATCHES)
|
||||
signature_failed = any(code in _SIGNATURE_FAILURES for code in failures)
|
||||
signature_failed = any(code in _SIGNATURE_FAILURES for code in disqualifying)
|
||||
signature_validated = "claimSignature.validated" in successes
|
||||
|
||||
if binding_failed:
|
||||
@@ -349,7 +374,7 @@ def _validation_fields(store: dict[str, Any]) -> dict[str, Any]:
|
||||
else:
|
||||
signer_trust = "unknown"
|
||||
|
||||
if any(code in failures for code in ("signingCredential.invalid", "signingCredential.ocsp.revoked")):
|
||||
if any(code in _CREDENTIAL_FAILURES for code in disqualifying):
|
||||
signer_validity = "invalid"
|
||||
elif "signingCredential.expired" in failures:
|
||||
signer_validity = "expired"
|
||||
@@ -367,6 +392,10 @@ def _validation_fields(store: dict[str, Any]) -> dict[str, Any]:
|
||||
"c2pa_signer_trust": signer_trust,
|
||||
"c2pa_signer_validity": signer_validity,
|
||||
"c2pa_validation_codes": all_codes,
|
||||
# The subset of failures that actually drove a dimension to invalid. Callers
|
||||
# rendering "why" must use this rather than re-classifying the full code list,
|
||||
# or the reason shown and the verdict reached stop being the same rule.
|
||||
"c2pa_failed_codes": disqualifying,
|
||||
}
|
||||
|
||||
|
||||
@@ -610,6 +639,7 @@ def _base_info(byte_count: int, *, fallback: bool = False) -> dict[str, Any]:
|
||||
c2pa_signer_trust="unknown",
|
||||
c2pa_signer_validity="unknown",
|
||||
c2pa_validation_codes=[],
|
||||
c2pa_failed_codes=[],
|
||||
)
|
||||
return info
|
||||
|
||||
|
||||
@@ -118,8 +118,13 @@ _SYNTHID_CAVEAT = (
|
||||
"decoded (proprietary decoder). Confirm via the Gemini app or openai.com/verify."
|
||||
)
|
||||
_C2PA_UNTRUSTED_CAVEAT = (
|
||||
"The C2PA claim signature and asset binding validate, but the signer identity is not anchored "
|
||||
"to a trusted credential here; treat the named platform as a signed claim, not a verified identity."
|
||||
"The C2PA claim signature and asset binding validate, but no trust anchor list is configured here, "
|
||||
"so the signer identity was never checked against one; treat the named platform as a signed claim, "
|
||||
"not a verified identity."
|
||||
)
|
||||
_C2PA_EXPIRED_CAVEAT = (
|
||||
"The C2PA signing certificate has expired and no trusted timestamp establishes when the claim was "
|
||||
"signed. The binding to these bytes still holds; only the signing time is unproven."
|
||||
)
|
||||
_C2PA_UNVALIDATED_CAVEAT = (
|
||||
"The C2PA marker was parsed without cryptographic validation; treat its origin and watermark "
|
||||
@@ -474,17 +479,20 @@ class ProvenanceReport:
|
||||
# an open DWT-DCT decode) -- as opposed to a visible mark, provenance-only
|
||||
# TrustMark, or a weak medium-confidence hint (hf-job, Samsung genAIType). This
|
||||
# is the set of signals an invisible/diffusion scrub targets: a visible-only
|
||||
# or no-signal image has it False. An intact C2PA AI claim from an untrusted
|
||||
# signer is deliberately medium-confidence while this remains True; callers
|
||||
# should gate on intent, not on the confidence string.
|
||||
# or no-signal image has it False. Callers should gate on intent, not on the
|
||||
# confidence string.
|
||||
ai_from_metadata: bool = False
|
||||
watermarks: list[str] = field(default_factory=list[str])
|
||||
signals: list[Signal] = field(default_factory=list["Signal"])
|
||||
caveats: list[str] = field(default_factory=list[str])
|
||||
# Contradictions between independent provenance signals (e.g. two different
|
||||
# AI vendors both claiming the image, or camera-capture credentials next to
|
||||
# AI-generation markers). Non-empty means the provenance is internally
|
||||
# inconsistent -- a strong tell of spoofed, transplanted, or laundered metadata.
|
||||
# AI-generation markers), and credentials that failed validation. Non-empty means
|
||||
# the provenance is internally inconsistent -- a strong tell of spoofed,
|
||||
# transplanted, or laundered metadata. A failed credential sends
|
||||
# ``is_ai_generated`` to None and lands here instead, so a consumer that reads
|
||||
# only the verdict field turns a broken AI manifest into silence; see the
|
||||
# ``integrity_clashes`` note in docs/python-api.md.
|
||||
integrity_clashes: list[str] = field(default_factory=list[str])
|
||||
# Orthogonal C2PA checks. A valid content binding does not make an untrusted
|
||||
# signer identity trusted, and an expired credential does not by itself mean the
|
||||
@@ -547,15 +555,16 @@ def _c2pa_validation(info: dict[str, Any]) -> dict[str, Any] | None:
|
||||
|
||||
|
||||
def _c2pa_credential_level(info: dict[str, Any]) -> str:
|
||||
"""Return invalid, verified, or unverified for provenance attribution."""
|
||||
"""Return invalid, verified, or unverified for provenance attribution.
|
||||
|
||||
``verified`` means the reader tied this manifest to these bytes: the hard binding
|
||||
matched and the claim signature validated. Signer trust is deliberately NOT a
|
||||
condition -- no trust anchors ship, so gating on it made this branch unreachable.
|
||||
Read the trust-anchor paragraph in docs/module-internals.md before changing this.
|
||||
"""
|
||||
if c2pa_info_has_invalid_credential(info):
|
||||
return "invalid"
|
||||
if (
|
||||
info.get("c2pa_integrity") == "valid"
|
||||
and info.get("c2pa_signature") == "valid"
|
||||
and info.get("c2pa_signer_trust") == "trusted"
|
||||
and info.get("c2pa_signer_validity") == "valid"
|
||||
):
|
||||
if info.get("c2pa_integrity") == "valid" and info.get("c2pa_signature") == "valid":
|
||||
return "verified"
|
||||
return "unverified"
|
||||
|
||||
@@ -1148,11 +1157,9 @@ def _identify_from_evidence(
|
||||
c2pa_validation = _c2pa_validation(info)
|
||||
c2pa_level = _c2pa_credential_level(info)
|
||||
c2pa_usable = c2pa_level != "invalid"
|
||||
failed_c2pa_codes = [
|
||||
str(code)
|
||||
for code in cast("list[object]", info.get("c2pa_validation_codes", []))
|
||||
if "mismatch" in str(code) or "invalid" in str(code)
|
||||
]
|
||||
# The reader already named which failures moved a dimension; re-deriving that here
|
||||
# by substring made the displayed reason a second, looser rule than the verdict.
|
||||
failed_c2pa_codes = [str(code) for code in cast("list[object]", info.get("c2pa_failed_codes", []))]
|
||||
issuers = [info["issuer"]] if info.get("issuer") else _issuers_in(region)
|
||||
# Full AI generation (trainedAlgorithmicMedia) vs an AI-enhanced real photo
|
||||
# (compositeWithTrainedAlgorithmicMedia). The structured kind is parsed once in
|
||||
@@ -1208,12 +1215,14 @@ def _identify_from_evidence(
|
||||
Signal("c2pa", detail or "C2PA manifest present", "high" if c2pa_level == "verified" else "medium")
|
||||
)
|
||||
watermarks.append(f"C2PA Content Credentials ({', '.join(issuers) or 'unknown signer'})")
|
||||
if c2pa_level == "unverified":
|
||||
caveats.append(
|
||||
_C2PA_UNTRUSTED_CAVEAT
|
||||
if info.get("c2pa_integrity") == "valid" and info.get("c2pa_signature") == "valid"
|
||||
else _C2PA_UNVALIDATED_CAVEAT
|
||||
)
|
||||
if c2pa_level == "verified":
|
||||
# A missing trust bundle is not a signer that failed against one.
|
||||
if info.get("c2pa_signer_trust") != "trusted":
|
||||
caveats.append(_C2PA_UNTRUSTED_CAVEAT)
|
||||
if info.get("c2pa_signer_validity") == "expired":
|
||||
caveats.append(_C2PA_EXPIRED_CAVEAT)
|
||||
else:
|
||||
caveats.append(_C2PA_UNVALIDATED_CAVEAT)
|
||||
# Record the AI-origin vendor for clash detection only when the source is
|
||||
# actually AI -- classify the issuer attribution / generator, NOT the
|
||||
# resolved `platform` (which may be a camera device token whose label,
|
||||
|
||||
+100
-9
@@ -16,6 +16,7 @@ from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from remove_ai_watermarks._internal.c2pa import c2pa_info_from_manifest_store
|
||||
from remove_ai_watermarks._internal.constants import C2PA_AI_VENDORS, C2PA_CLAIM_GENERATOR_PLATFORMS
|
||||
from remove_ai_watermarks.identify import (
|
||||
ProvenanceEvidence,
|
||||
@@ -72,10 +73,77 @@ class TestProvenanceEvidence:
|
||||
report = identify_from_evidence(evidence)
|
||||
|
||||
assert report.is_ai_generated is True
|
||||
assert report.confidence == "medium"
|
||||
# Intact binding and signature. The signer is not anchored, which is a missing
|
||||
# input here (no trust bundle ships), not a finding against the credential.
|
||||
assert report.confidence == "high"
|
||||
assert report.platform == "Adobe Firefly"
|
||||
|
||||
def test_fully_validated_c2pa_claim_is_high_confidence(self, tmp_path: Path):
|
||||
def test_revoked_signing_credential_is_disqualifying(self, tmp_path: Path):
|
||||
"""A credential the issuer disowned cannot establish origin.
|
||||
|
||||
Revocation arrives on its own dimension, not as a binding or signature failure,
|
||||
so a check that reads only those two returned an AI verdict off a dead cert with
|
||||
an empty ``integrity_clashes`` -- quieter than a hash mismatch on the same file.
|
||||
|
||||
The evidence comes from :func:`c2pa_info_from_manifest_store`, not a hand-written
|
||||
dict of what it is believed to emit, so the assertion follows the producer when
|
||||
its contract changes.
|
||||
"""
|
||||
path = tmp_path / "revoked.png"
|
||||
info = c2pa_info_from_manifest_store(
|
||||
{
|
||||
"active_manifest": "created",
|
||||
"validation_results": {
|
||||
"activeManifest": {
|
||||
"success": [
|
||||
{"code": "assertion.dataHash.match"},
|
||||
{"code": "claimSignature.validated"},
|
||||
],
|
||||
"failure": [{"code": "signingCredential.ocsp.revoked"}],
|
||||
}
|
||||
},
|
||||
"manifests": {
|
||||
"created": {
|
||||
"signature_info": {"issuer": "OpenAI"},
|
||||
"assertions": [
|
||||
{
|
||||
"label": "c2pa.actions.v2",
|
||||
"data": {
|
||||
"actions": [
|
||||
{
|
||||
"action": "c2pa.created",
|
||||
"digitalSourceType": "trainedAlgorithmicMedia",
|
||||
}
|
||||
]
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
},
|
||||
}
|
||||
)
|
||||
assert info["c2pa_signer_validity"] == "invalid"
|
||||
evidence = ProvenanceEvidence(
|
||||
path=path,
|
||||
c2pa_info=info,
|
||||
ai_metadata={},
|
||||
scan=b"jumb c2pa OpenAI trainedAlgorithmicMedia",
|
||||
iptc_ai_system=None,
|
||||
aigc_label=None,
|
||||
exif_generator=None,
|
||||
xai_signature=False,
|
||||
huggingface_job=None,
|
||||
samsung_genai=None,
|
||||
)
|
||||
|
||||
report = identify_from_evidence(evidence)
|
||||
|
||||
assert report.is_ai_generated is None
|
||||
assert report.platform is None
|
||||
assert report.confidence == "none"
|
||||
assert any("revoked" in clash for clash in report.integrity_clashes)
|
||||
|
||||
def test_anchored_signer_is_also_high_confidence(self, tmp_path: Path):
|
||||
path = tmp_path / "validated.png"
|
||||
info = {
|
||||
"has_c2pa": True,
|
||||
@@ -83,7 +151,7 @@ class TestProvenanceEvidence:
|
||||
"source_type": "trainedAlgorithmicMedia (AI-generated)",
|
||||
"ai_source_kind": "generated",
|
||||
"c2pa_validation_source": "reader",
|
||||
"c2pa_validation_state": "Valid",
|
||||
"c2pa_validation_state": "Trusted",
|
||||
"c2pa_integrity": "valid",
|
||||
"c2pa_signature": "valid",
|
||||
"c2pa_signer_trust": "trusted",
|
||||
@@ -112,6 +180,7 @@ class TestProvenanceEvidence:
|
||||
assert report.is_ai_generated is True
|
||||
assert report.confidence == "high"
|
||||
assert report.platform == "OpenAI (ChatGPT / gpt-image / DALL-E / Sora)"
|
||||
assert not any("not anchored" in caveat for caveat in report.caveats)
|
||||
|
||||
def test_external_metadata_record_builds_equivalent_evidence(self, tmp_path: Path):
|
||||
path = tmp_path / "external.jpg"
|
||||
@@ -553,7 +622,7 @@ class TestIdentifyRealSamples:
|
||||
def test_openai_chatgpt(self):
|
||||
r = identify(SAMPLES_DIR / "chatgpt-1.png", check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.confidence == "medium"
|
||||
assert r.confidence == "high"
|
||||
assert r.platform
|
||||
assert "OpenAI" in r.platform
|
||||
assert any("C2PA" in w for w in r.watermarks)
|
||||
@@ -622,15 +691,14 @@ class TestIdentifyRealSamples:
|
||||
# both invisible/metadata targets, so the diffusion scrub should run.
|
||||
assert has_invisible_target(SAMPLES_DIR / "chatgpt-1.png") is True
|
||||
assert has_invisible_target(SAMPLES_DIR / "mj-1.png") is True
|
||||
# ai_from_metadata records scrub intent even when an untrusted signer
|
||||
# makes the provenance verdict medium-confidence.
|
||||
# ai_from_metadata records scrub intent independently of the confidence string.
|
||||
assert identify(SAMPLES_DIR / "chatgpt-1.png", check_visible=False).ai_from_metadata is True
|
||||
|
||||
def test_untrusted_but_intact_c2pa_is_medium_confidence(self):
|
||||
def test_untrusted_but_intact_c2pa_is_high_confidence_with_a_caveat(self):
|
||||
report = identify(SAMPLES_DIR / "chatgpt-1.png", check_visible=False, check_invisible=False)
|
||||
|
||||
assert report.is_ai_generated is True
|
||||
assert report.confidence == "medium"
|
||||
assert report.confidence == "high"
|
||||
assert report.ai_from_metadata is True
|
||||
assert report.c2pa_validation is not None
|
||||
assert report.c2pa_validation["source"] == "reader"
|
||||
@@ -640,7 +708,30 @@ class TestIdentifyRealSamples:
|
||||
assert report.c2pa_validation["signer_trust"] == "untrusted"
|
||||
assert report.c2pa_validation["signer_validity"] == "expired"
|
||||
assert "assertion.dataHash.match" in report.c2pa_validation["codes"]
|
||||
assert any("not anchored" in caveat for caveat in report.caveats)
|
||||
# What was not established is said, not folded into the confidence string.
|
||||
assert any("never checked against one" in caveat for caveat in report.caveats)
|
||||
assert any("only the signing time is unproven" in caveat for caveat in report.caveats)
|
||||
|
||||
def test_no_committed_fixture_reports_a_trusted_signer(self):
|
||||
"""The reachability guard for :func:`_c2pa_credential_level`.
|
||||
|
||||
The SDK ships no production trust anchors, so ``signingCredential.trusted``
|
||||
appears in no default installation. Gating high confidence on it made that branch
|
||||
dead in production for every vendor while a hand-built dict kept it green in the
|
||||
suite. These fixtures are the producer; if one ever comes back trusted, a bundle
|
||||
got configured and the confidence mapping needs re-reading, not this assertion
|
||||
deleted.
|
||||
"""
|
||||
checked = 0
|
||||
for path in sorted(SAMPLES_DIR.iterdir()):
|
||||
report = identify(path, check_visible=False, check_invisible=False)
|
||||
if report.c2pa_validation is None:
|
||||
continue
|
||||
checked += 1
|
||||
assert report.c2pa_validation["signer_trust"] != "trusted"
|
||||
if report.c2pa_validation["integrity"] == "valid" and report.c2pa_validation["signature"] == "valid":
|
||||
assert report.confidence == "high", path.name
|
||||
assert checked >= 3
|
||||
|
||||
def test_hash_mismatch_does_not_confirm_origin_but_keeps_scrub_fail_safe(self, tampered_chatgpt_png: Path):
|
||||
report = identify(tampered_chatgpt_png, check_visible=False, check_invisible=False)
|
||||
|
||||
@@ -272,7 +272,18 @@ class TestC2PA:
|
||||
|
||||
assert c2pa_info_has_removal_hint(info) is False
|
||||
|
||||
def test_invalid_ingredient_does_not_taint_active_validation_or_supply_claims(self):
|
||||
@pytest.mark.parametrize(
|
||||
"ingredient_failure",
|
||||
[
|
||||
# One exclusion rule, reached through two different dimensions: a broken
|
||||
# binding and a credential the issuer disowned. The walk classified only the
|
||||
# first for a while, so a revoked child manifest stayed reachable and kept
|
||||
# donating its claim generator to the parent's attribution.
|
||||
"assertion.dataHash.mismatch",
|
||||
"signingCredential.ocsp.revoked",
|
||||
],
|
||||
)
|
||||
def test_invalid_ingredient_does_not_taint_active_validation_or_supply_claims(self, ingredient_failure: str):
|
||||
store = {
|
||||
"active_manifest": "update",
|
||||
"validation_results": {
|
||||
@@ -283,13 +294,7 @@ class TestC2PA:
|
||||
],
|
||||
"failure": [{"code": "signingCredential.untrusted"}],
|
||||
},
|
||||
"ingredientDeltas": [
|
||||
{
|
||||
"validationDeltas": {
|
||||
"failure": [{"code": "assertion.dataHash.mismatch"}],
|
||||
}
|
||||
}
|
||||
],
|
||||
"ingredientDeltas": [{"validationDeltas": {"failure": [{"code": ingredient_failure}]}}],
|
||||
},
|
||||
"manifests": {
|
||||
"update": {
|
||||
@@ -297,11 +302,7 @@ class TestC2PA:
|
||||
"ingredients": [
|
||||
{
|
||||
"active_manifest": "created",
|
||||
"validation_results": {
|
||||
"activeManifest": {
|
||||
"failure": [{"code": "assertion.dataHash.mismatch"}],
|
||||
}
|
||||
},
|
||||
"validation_results": {"activeManifest": {"failure": [{"code": ingredient_failure}]}},
|
||||
}
|
||||
],
|
||||
"assertions": [],
|
||||
|
||||
Reference in New Issue
Block a user