mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-09-01 02:00:36 +02:00
1761 lines
101 KiB
Markdown
1761 lines
101 KiB
Markdown
# SynthID detector and pixel-only removal research plan
|
|
|
|
> Research plan, not a statement of current product capability. The shipped
|
|
> behavior remains documented in [supported signals](supported-signals.md),
|
|
> [known limitations](known-limitations.md), and
|
|
> [module internals](module-internals.md).
|
|
|
|
## Objective
|
|
|
|
Build two independent provider tracks with four capabilities:
|
|
|
|
1. a local, metadata-independent detector for the OpenAI SynthID image signal;
|
|
2. a local, metadata-independent detector for the Google SynthID image signal;
|
|
3. a pixel-only remover for the OpenAI signal that does not use diffusion,
|
|
VAE reconstruction, semantic regeneration, or generative inpainting;
|
|
4. an independently calibrated pixel-only remover for the Google signal under
|
|
the same constraints.
|
|
|
|
OpenAI is the first research track because it has a documented remote verifier.
|
|
Google follows with the same experimental protocol but its own corpus, labels,
|
|
model, thresholds, and oracle. No carrier, feature, score, or operating point
|
|
transfers between providers until a held-out experiment demonstrates that it
|
|
does.
|
|
|
|
The detector target is signal presence, not payload recovery, provider
|
|
classification, or general AI-image classification. The removal target is a
|
|
minimal pixel residual that makes a source-positive image negative in the
|
|
matching provider oracle while preserving the image's geometry and semantics.
|
|
|
|
## Non-negotiable evidence rules
|
|
|
|
1. **Detector before remover.** A remover may be prototyped against synthetic
|
|
carriers, but no real-image removal claim is made until the local detector
|
|
passes its own held-out gate.
|
|
2. **Provider-specific ground truth.** OpenAI and Google labels come from their
|
|
matching verifier. C2PA is recorded separately and is never the pixel label.
|
|
3. **No metadata or export leakage.** Training and evaluation operate on decoded
|
|
pixels after controlled metadata stripping and matched re-encoding. Geometry,
|
|
filename, file size, chunks, encoder settings, and source directories cannot
|
|
be model inputs.
|
|
4. **Signal identifiability is a gate.** A classifier trained only on provider
|
|
positives and unrelated negatives can learn the provider's generator or
|
|
export fingerprint. It is called a provider classifier, not a SynthID
|
|
detector, until at least one causal control succeeds.
|
|
5. **The oracle is held out from optimization.** Candidate algorithms and
|
|
hyperparameters are selected locally. Oracle batches are immutable and
|
|
registered before submission. A remote binary verdict is never used as an
|
|
online loss or hill-climbing signal.
|
|
6. **A local score decrease is not removal evidence.** Removal requires a
|
|
source-positive, output-negative result from the matching provider oracle.
|
|
7. **One hypothesis does not become several signals.** Correlated spatial,
|
|
spectral, and color statistics derived from one residual are reported as one
|
|
line of evidence unless independent controls separate them.
|
|
8. **Every result is reproducible.** Input and output hashes, code revision,
|
|
model artifact hash, preprocessing, transform lineage, seed, score, threshold,
|
|
quality metrics, oracle result, session, and timestamp are retained.
|
|
|
|
## Oracle boundary
|
|
|
|
OpenAI documents a synchronous Content Provenance API at
|
|
`POST /v1/content_provenance_checks`. For images it returns separate `c2pa` and
|
|
`synthid` entries and supports PNG, JPEG, and WebP. It remains a remote,
|
|
OpenAI-scoped verifier, not a released local decoder. The same documentation
|
|
explicitly says not to use repeated queries to reverse-engineer, remove, or
|
|
evade a watermark. Adaptive detector or remover research against that endpoint
|
|
therefore requires explicit OpenAI authorization or a separate research oracle
|
|
whose terms permit the work. Without that authorization, the OpenAI track may
|
|
develop local hypotheses but stops before oracle-driven calibration and removal
|
|
certification.
|
|
|
|
Google's Gemini verification flow is a Google-scoped oracle. Its documented
|
|
result can be detected, not detected, or unclear, and the consumer flow has a
|
|
small rolling quota. An unclear result is `indeterminate`, never a negative.
|
|
|
|
For every permitted oracle batch:
|
|
|
|
- verify the untouched source first;
|
|
- submit one file per request;
|
|
- record C2PA and SynthID outcomes independently when both are returned;
|
|
- preserve the exact submitted bytes and SHA-256 outside the public repository;
|
|
- retry only a transient failure on the same bytes;
|
|
- record `detected`, `not_detected`, `indeterminate`, or `refused` verbatim;
|
|
- submit a matched transform-only control before attributing a negative result
|
|
to an experimental edit;
|
|
- reserve a final temporal holdout that no feature, threshold, or remover has
|
|
seen.
|
|
|
|
## Data design
|
|
|
|
### Corpus layers
|
|
|
|
Each provider gets a separate corpus with five layers:
|
|
|
|
| Layer | Purpose | Required controls |
|
|
| --- | --- | --- |
|
|
| Verified positives | Learn and evaluate the real signal | Matching provider oracle, original bytes |
|
|
| Same-provider hard negatives | Separate watermark from generator identity | Same surface or model, oracle-negative |
|
|
| External hard negatives | Measure false positives | Cameras, scans, edited photos, other generators, synthetic graphics |
|
|
| Low-texture probes | Expose weak shared structure | Solid colors, gradients, ramps, checkerboards, sparse edges |
|
|
| Causal pairs | Attribute a residual to the watermark | Same underlying pixels, positive and confirmed negative |
|
|
|
|
The causal-pair layer is the most valuable and the hardest to obtain. An
|
|
authorized encoder-off pair is ideal. A provider output and a pixel-only
|
|
processed version become a usable pair only after the original is positive and
|
|
the processed bytes are negative in the provider oracle. Public third-party
|
|
pairs are discovery material until their provenance and both labels are
|
|
independently verified.
|
|
|
|
A negative created by a remover trained against detector A cannot train or
|
|
validate detector A in the same experiment. Reserve it for detector B or a later
|
|
model epoch after closing the originating experiment. Otherwise detector and
|
|
remover can certify each other's shared blind spot.
|
|
|
|
If no same-provider hard negatives or causal pairs can be obtained, learned
|
|
real-image models stay explicitly labeled as provider classifiers. Spectral
|
|
repeatability on solid fills alone does not clear this gate.
|
|
|
|
### Strata
|
|
|
|
Record and split by:
|
|
|
|
- provider, product surface, model family, and generation date window;
|
|
- native width, height, aspect ratio, file format, and color mode;
|
|
- photoreal, face, text-heavy, flat graphic, illustration, low texture, and
|
|
high texture content;
|
|
- untouched, metadata-stripped, lossless-normalized, JPEG/WebP, resized,
|
|
cropped, and color-adjusted lineage;
|
|
- generation session and prompt family;
|
|
- parent hash for every derivative.
|
|
|
|
Store research media under `.local-eval/synthid/`, not in the public repository.
|
|
Only cleared fixtures may enter `data/synthid/`. Tracked files contain schemas,
|
|
scripts, synthetic fixtures, aggregate verdicts, and non-sensitive hashes.
|
|
|
|
### Split discipline
|
|
|
|
Deduplicate by decoded-pixel hash and perceptual similarity before splitting.
|
|
Keep every derivative, prompt sibling, and semantic near-duplicate in one hash
|
|
group. Split by group into train, validation, locked test, and a later temporal
|
|
test collected after the detector is frozen. A random image-level split is
|
|
invalid because it leaks transform and generation-family fingerprints.
|
|
|
|
The negative set must be large enough to support the claimed operating point.
|
|
At zero observed false positives, roughly 3,000 independent negatives are
|
|
needed merely to put the one-sided 95% upper bound near 0.1% by the rule of
|
|
three. The final evaluation should prefer at least 10,000 hard and ordinary
|
|
negatives per provider, report confidence intervals, and report every negative
|
|
stratum separately rather than hiding a weak stratum in an aggregate.
|
|
|
|
## Detector program
|
|
|
|
### Experiment D0: oracle and corpus integrity
|
|
|
|
Goal: prove that labels and bytes mean what the manifest says.
|
|
|
|
- Separate SynthID from C2PA in every response.
|
|
- Confirm that metadata stripping changes C2PA but does not silently define the
|
|
SynthID label.
|
|
- Confirm every positive used for evaluation on its matching oracle.
|
|
- Mutation-test manifest ingestion with swapped provider, duplicate bytes,
|
|
derivative leakage, an `indeterminate` result mislabeled as negative, and a
|
|
changed file after hashing.
|
|
- Measure permitted same-byte verifier reproducibility on a small preregistered
|
|
sample. Do not turn retries into adaptive querying.
|
|
|
|
Gate: no ambiguous label path, no cross-provider oracle substitution, and no
|
|
train/test family leakage.
|
|
|
|
### Experiment D1: export and generator confounds
|
|
|
|
Goal: determine how accurately SynthID can appear to be detected when no
|
|
watermark-specific evidence is available.
|
|
|
|
Train deliberately confounded baselines on file/container fields, dimensions,
|
|
RGB thumbnails, and generator-vs-camera content. Then repeat after canonical
|
|
decode, metadata removal, resolution matching, and hard-negative balancing.
|
|
|
|
Gate: any proposed signal feature must beat the canonicalized confound baseline
|
|
on same-provider hard negatives and on a temporal holdout. Otherwise the result
|
|
is generator attribution.
|
|
|
|
### Experiment D2: low-texture carrier discovery
|
|
|
|
Goal: test whether a repeatable carrier component is observable when scene
|
|
texture is suppressed.
|
|
|
|
Extend the existing `scripts/synthid_pixel_probe.py` measurements from grayscale
|
|
NCC to:
|
|
|
|
- per-channel and opponent-color residuals;
|
|
- two-dimensional FFT magnitude and circular phase coherence;
|
|
- wavelet bands and multi-scale autocorrelation;
|
|
- resolution and aspect-ratio registration;
|
|
- cross-color, cross-session, cross-model, and cross-date agreement;
|
|
- matched clean synthetic fills and same-provider oracle-negative probes.
|
|
|
|
Use leave-one-color, leave-one-session, and leave-one-resolution-out tests.
|
|
Fixed bins discovered and evaluated on the same images are descriptive only.
|
|
|
|
Gate: a template learned on one subset must detect held-out positive probes above
|
|
matched negatives and survive a later collection window. Failure kills the
|
|
fixed-carrier branch but not the content-dependent branch.
|
|
|
|
### Experiment D3: classical real-image detector
|
|
|
|
Goal: establish the strongest interpretable baseline before a neural model.
|
|
|
|
Candidate features include locally normalized high-pass residuals, FFT and
|
|
wavelet energy ratios, circular phase coherence, color-channel agreement,
|
|
block periodicity, and correlations against provider/resolution templates.
|
|
Fit regularized logistic regression and a shallow tree model. Calibration uses
|
|
only validation data.
|
|
|
|
Report TPR at 0.1% FPR as the primary metric, with bootstrap confidence
|
|
intervals. AUROC, average precision, and an arbitrary accuracy percentage are
|
|
secondary. Also report worst-stratum FPR, temporal-holdout TPR, and score drift.
|
|
|
|
Gate: advance the classical detector only if the locked test shows a stable
|
|
watermark-specific advantage over confound baselines. Do not choose a threshold
|
|
from the locked test.
|
|
|
|
### Mechanism hypotheses for D3-D5
|
|
|
|
The next detector epoch tests these hypotheses as one preregistered program:
|
|
|
|
1. **Counterfactual labels.** Train and calibrate on source-matched
|
|
clean/watermarked examples, ideally the same underlying image before and
|
|
after encoding. A result that vanishes when source, date, codec, and
|
|
dimensions are balanced is a provider fingerprint, not watermark evidence.
|
|
2. **Canonical full-field evidence.** Preserve a 512x512 image-level field,
|
|
absolute residual amplitude, and chroma alongside normalized residuals.
|
|
Compare it against the frozen patch baseline on source-disjoint pairs.
|
|
Failure to improve paired low-FPR detection rejects the added global context.
|
|
3. **Registration.** Score several bicubic canonical views spanning small scale,
|
|
crop, and offset changes, then aggregate before calibration. This tests
|
|
whether synchronization is distributed across the image rather than fixed to
|
|
one global phase or local grid.
|
|
4. **Independent detection head.** Learn one presence logit directly. Treat
|
|
payload-like or phase-consistency heads as auxiliary evidence, not as the
|
|
presence decision. Their inclusion must improve a held-out paired test, not
|
|
only attack-surface scores.
|
|
5. **Symmetric transformation channel.** Apply the identical sampled codec,
|
|
resize, crop, color, noise, or overlay operation to both members of a pair. A
|
|
transform is admitted only when the reference decoder, where available,
|
|
confirms that the transformed positive remains valid.
|
|
6. **Encoder versions.** Compare one universal model with version- or
|
|
epoch-specific experts on a cross-version transfer matrix. A version split is
|
|
useful only if it improves held-out likelihood without source metadata at
|
|
inference.
|
|
7. **Content-dependent watermarkability.** Measure flat, low-contrast,
|
|
monochrome, logo, and pixel-art strata separately. Allow a
|
|
content-conditioned expert or abstention instead of forcing one global
|
|
operating point.
|
|
8. **Two-sided calibration.** Calibrate empirical evidence for both `not
|
|
watermarked` and `watermarked`, returning positive, negative, or abstain.
|
|
External generator corpora enter afterward as an untouched false-positive
|
|
challenge, not as a substitute for counterfactual negatives.
|
|
|
|
### Experiment D4: learned residual detector
|
|
|
|
Goal: learn watermark presence from counterfactual image-level evidence that a
|
|
fixed template misses.
|
|
|
|
The primary model uses a full canonical field with raw RGB or luminance/chroma,
|
|
absolute-amplitude fine and coarse residuals, and optional stationary-wavelet
|
|
and complex-frequency branches. Every optional representation is encoded
|
|
separately and fused late; early channel concatenation is not a valid ablation.
|
|
Locally normalized patch evidence remains a frozen baseline, not the primary
|
|
input. Multi-view registration is aggregated into one image-level presence
|
|
logit. Payload-like, phase, localization, and content-watermarkability heads are
|
|
auxiliary and must prove an incremental held-out benefit.
|
|
|
|
Use group-aware pair sampling and apply every sampled transformation
|
|
symmetrically to the clean and watermarked members. Never expose metadata,
|
|
paths, native dimensions without normalization, or encoder-specific byte
|
|
patterns to the network. If causal pairs remain unavailable, do not advance a
|
|
learned model from provider classification to this experiment.
|
|
|
|
Train provider-specific models first. A shared backbone with provider-specific
|
|
heads is a later ablation, not the default architecture. Keep a second detector
|
|
family completely outside remover training so it can reveal surrogate overfit.
|
|
Compare a universal head with version-specific experts, and calibrate the
|
|
selected model with two-sided empirical evidence so ambiguous inputs abstain.
|
|
|
|
Gate for a detector release candidate:
|
|
|
|
- empirical FPR at or below 0.1% on the locked negative set;
|
|
- one-sided 95% TPR lower bound at or above 90% on untouched positives;
|
|
- no declared hard-negative stratum above 0.5% FPR;
|
|
- useful TPR on the temporal holdout and transformation suite;
|
|
- meaningful discrimination on causal pairs or same-provider hard negatives;
|
|
- calibrated `abstain` behavior outside supported providers and strata.
|
|
|
|
These are research gates, not promises that the proprietary decoder's operating
|
|
point has been reproduced.
|
|
|
|
### Experiment D5: robustness and drift
|
|
|
|
Evaluate identity, JPEG and WebP, resize, crop, padding, rotation, color changes,
|
|
blur, noise, overlays, screenshots, and combinations. Preserve a matched
|
|
transform-only positive control for every attack family. Report both average
|
|
and worst-transform TPR at the frozen threshold.
|
|
|
|
Repeat a small fixed collection after provider model or surface changes. A
|
|
shift in score distribution opens a new model epoch; it does not silently
|
|
recalibrate the old threshold.
|
|
|
|
## Localization program
|
|
|
|
A whole-image detector is not automatically a useful removal loss. Test whether
|
|
its evidence is spatially causal with three independent methods:
|
|
|
|
- tile occlusion and replacement with distortion-matched controls;
|
|
- detector-gradient attribution for the learned model;
|
|
- phase or template energy mapped back to spatial blocks.
|
|
|
|
For each source, create top-ranked, random, and bottom-ranked edits with equal
|
|
pixel norm and the same codec path. The oracle batch is registered before any
|
|
results return.
|
|
|
|
Gate: top-ranked edits must reduce matching-oracle detection more often than
|
|
random edits under paired analysis. Aggregate per-image differences and confirm
|
|
the direction with a sign test. If localization does not transfer, do not build
|
|
a region remover around it.
|
|
|
|
## Pixel-only removal program
|
|
|
|
All candidates preserve image dimensions and avoid a generative decoder. Run
|
|
the following ladder in order.
|
|
|
|
### Experiment R1: analytical carrier subtraction
|
|
|
|
Estimate provider, model-epoch, and geometry-specific residual components from
|
|
verified probes or causal pairs. Test:
|
|
|
|
- complex FFT projection with conjugate symmetry;
|
|
- wavelet-band projection;
|
|
- per-channel and opponent-color residual subtraction;
|
|
- spatially varying strength based on local texture and detector attribution.
|
|
|
|
Sweep signed amplitude, not only attenuation. Include a sham edit with identical
|
|
pixel norm outside candidate bins. This is the highest-value path because a real
|
|
shared post-hoc residual could be removed at very high fidelity.
|
|
|
|
### Experiment R2: constrained per-image optimization
|
|
|
|
Optimize pixels against an ensemble of frozen local detectors. The objective
|
|
combines detector margin with L-infinity and L2 bounds, LPIPS or DISTS, MS-SSIM,
|
|
edge consistency, OCR preservation, and face-embedding preservation where
|
|
applicable. Expectation over transformations covers lossless export, JPEG/WebP,
|
|
small resize, and color conversion so the result is not a fragile local
|
|
adversarial example.
|
|
|
|
The optimization may read only local detectors. The provider oracle evaluates
|
|
a frozen candidate batch afterward and never supplies gradients, search
|
|
direction, or per-step feedback.
|
|
|
|
### Experiment R3: feed-forward residual remover
|
|
|
|
If R2 transfers to the provider oracle, distill successful minimal residuals
|
|
into a compact image-to-residual network. Constrain the output amplitude and
|
|
frequency distribution explicitly. Train on one detector ensemble and select
|
|
on the held-out detector family. Preserve the per-image optimizer as the
|
|
reference implementation.
|
|
|
|
### Experiment R4: detector-remover co-evaluation
|
|
|
|
Evaluate four combinations separately:
|
|
|
|
1. local detector positive, provider oracle positive;
|
|
2. local detector negative, provider oracle positive;
|
|
3. local detector positive, provider oracle negative;
|
|
4. both negative.
|
|
|
|
Case 2 is the critical surrogate-overfit failure. Add every permitted example
|
|
to a future hard set only after the current experiment is closed; never tune and
|
|
report on the same oracle failure.
|
|
|
|
## Removal success gate
|
|
|
|
A provider-specific remover is a release candidate only when a locked,
|
|
source-positive evaluation shows all of the following:
|
|
|
|
- at least 90% matching-oracle `not_detected` results, with `indeterminate`
|
|
counted as failure;
|
|
- a positive matched transform-only control for every evaluated source;
|
|
- exact-size output with no semantic regeneration stage;
|
|
- median PSNR at least 40 dB and fifth-percentile PSNR at least 35 dB;
|
|
- median SSIM at least 0.99, plus LPIPS or DISTS reported rather than optimized
|
|
silently;
|
|
- no OCR regression on text strata and no material face-identity regression on
|
|
face strata under preregistered thresholds;
|
|
- no worse oracle-negative rate after a standard downstream JPEG/WebP/resize
|
|
suite;
|
|
- byte-identical pass-through for detector-negative inputs by default;
|
|
- a significant paired advantage over random and norm-matched sham edits;
|
|
- no claim of forensic cleanliness without a separately trained removal-artifact
|
|
detector and a held-out evaluation.
|
|
|
|
If pixel-only methods fail this gate, retain diffusion regeneration as the
|
|
explicit fallback. Do not combine its success with pixel-only results.
|
|
|
|
## Provider sequence
|
|
|
|
### OpenAI first
|
|
|
|
1. Resolve the oracle authorization gate.
|
|
2. Build the corpus schema and confound challenge.
|
|
3. Verify or reject the shared-carrier hypothesis on low-texture probes.
|
|
4. Train classical and learned detectors.
|
|
5. Freeze the detector and temporal test.
|
|
6. Run R1, then R2, then R3 only after each preceding gate passes.
|
|
7. Submit one preregistered final oracle batch.
|
|
|
|
OpenAI work establishes the experimental machinery, not parameters for Google.
|
|
|
|
### Google second
|
|
|
|
Repeat the full sequence with Google-native positives, Google hard negatives,
|
|
and the Gemini oracle. Spend the small manual oracle budget on controls and
|
|
decisive boundary points, not uniform sweeps. Use local detector uncertainty to
|
|
choose a batch before submission, then freeze it. Test Gemini app and AI Studio
|
|
surfaces as separate strata because their export and metadata paths differ.
|
|
|
|
## Implementation order
|
|
|
|
The research harness remains outside the public API until the gates pass.
|
|
|
|
1. **Implemented:** use the private corpus schema and auditor documented in
|
|
[`data/synthid/research-manifest.md`](../data/synthid/research-manifest.md) to
|
|
record provider, surface, model epoch, session, content stratum, parent hash,
|
|
transform lineage, separate C2PA and SynthID outcomes, oracle session, and
|
|
artifact hashes.
|
|
2. **Implemented:** build a label-free local inventory before promotion so byte-identical files,
|
|
decoded-pixel duplicates, and unsupported formats are visible without
|
|
inferring evidence from directory names.
|
|
3. **Implemented:** add a corpus auditor that rejects hash-group leakage, missing parent links,
|
|
ambiguous labels, and unsupported oracle-provider pairs.
|
|
4. **Harness implemented; evidence run pending:** run the manifest-driven D1
|
|
challenge over container, thumbnail, and canonical decoded-content
|
|
baselines. Freeze its validation threshold and report same-provider negative
|
|
cohorts separately.
|
|
5. Generalize `scripts/synthid_pixel_probe.py` into reusable feature extraction
|
|
while preserving its current synthetic tests.
|
|
6. Add reproducible train/evaluate commands whose output is a versioned model
|
|
card and metrics snapshot, never an unversioned console claim.
|
|
7. Add analytical and optimization removal harnesses with norm-matched controls.
|
|
8. Reuse the existing fidelity scripts for PSNR, SSIM, OCR, face, and edge
|
|
measurements, adding only missing metrics.
|
|
9. Package provider-specific detector weights behind an optional dependency only
|
|
after the detector gate passes.
|
|
10. Add a runtime remover and CLI surface only after the removal gate passes.
|
|
|
|
Pure feature, manifest, split, threshold, and residual-constraint logic must be
|
|
unit-tested without model downloads. Real model and oracle runs stay explicit
|
|
research jobs.
|
|
|
|
The inventory, manifest auditor, and D1 confound harness now exist as local
|
|
research tools. D1 has not produced a real-corpus metric yet because existing
|
|
artifacts have not been promoted into evidence-bearing provider manifests. This
|
|
is an evidence gap, not permission to infer labels from their paths. The next
|
|
real D1 run begins only after ordinary rows cover train, validation, locked test,
|
|
same-provider hard negatives, and a two-class temporal holdout.
|
|
|
|
## Empirical log
|
|
|
|
### 2026-08-09: D1 confound pilot and codec challenge
|
|
|
|
The manifest-driven D1 harness was run on the frozen Google pilot containing
|
|
five oracle-positive images and 330 deduplicated exact-geometry external
|
|
photographs. The manifest passed byte, decoded-pixel, lineage, and split
|
|
verification. It is not evidence-ready because the locked test contains no
|
|
same-provider hard negative.
|
|
|
|
The container-only baseline separated the labels perfectly because the pilot
|
|
still exposes format and export-geometry differences. That result is a measured
|
|
confound, not watermark evidence. The canonical 8x8 decoded-content baseline
|
|
was much weaker: locked-test AUC was 0.620 and temporal AUC was 0.533, and its
|
|
validation-frozen threshold detected neither held-out positive. The existing
|
|
positive-only RGB plus HSV S/V ensemble still detected all five positives and
|
|
emitted no positive verdict on the 330 frozen external images.
|
|
|
|
A codec challenge re-encoded every positive without changing geometry. The
|
|
ensemble remained positive on five of five JPEG-95 outputs and five of five
|
|
WebP-95 outputs; JPEG-90 retained three of five. This rejects a bare PNG versus
|
|
JPEG container explanation, but does not exclude a generator or source-pipeline
|
|
correlate. With only five positives, the one-sided 95% lower bound on TPR is
|
|
54.9%. With zero positives among 330 external images, the one-sided 95% upper
|
|
bound on FPR is 0.904%, still nine times the 0.1% detector target. Four positives
|
|
also participated in model fitting, leaving only one independent temporal
|
|
positive. The next valid detector claim still requires new oracle-positive
|
|
images and ordinary same-provider oracle-negative controls.
|
|
|
|
A post-freeze source-provenance challenge then added three exact-geometry
|
|
Google originals that had not influenced fitting or threshold selection. Each
|
|
carried the same signed Google LLC C2PA issuer, trained-algorithmic-media source
|
|
type, and explicit SynthID-present assertion as the detected temporal control.
|
|
The ensemble abstained on all three: two lacked active carrier support in both
|
|
branches, and the third passed RGB evidence but missed the HSV S/V evidence and
|
|
support gates. These are provider-signed embedding assertions rather than
|
|
matching-oracle pixel labels, but zero positives in three new same-geometry
|
|
images falsifies the current ensemble as a general Google SynthID detector. The
|
|
measured phase family is retained only as an epoch- or surface-specific
|
|
correlate pending a broader oracle-labeled corpus.
|
|
|
|
A follow-up cross-epoch check found that the correlate is not entirely confined
|
|
to the original five images. RGB and HSV models fitted only on the three later
|
|
images ranked each of the five earlier images above all 329 external negatives
|
|
in both color spaces (AUC 1.0), but thresholds derived from the later fitting
|
|
scores transferred poorly: the strict RGB-plus-HSV decision retained only one
|
|
of five earlier positives at zero false positives on the 279-image holdout. An
|
|
eight-fold leave-one-positive-out refit across both source groups then detected
|
|
six of eight excluded positives with the same RGB-plus-HSV conjunction and each
|
|
branch frozen just above its 50-image calibration maximum. RGB alone put all
|
|
eight excluded positives above that maximum, while HSV missed the geometric and
|
|
low-texture images; the per-fold RGB operating points also produced between one
|
|
and eight false positives on 279 held-out negatives.
|
|
This is evidence for a transferable but content- and epoch-sensitive Google
|
|
pixel correlate, not a shippable detector. The experiment still lacks ordinary
|
|
same-provider oracle negatives, and its external-negative set is too small for
|
|
the 0.1% false-positive target.
|
|
|
|
### 2026-08-09: fixed spectral-template baseline rejected
|
|
|
|
An exploratory Google template was reconstructed from four public, purported
|
|
clean/marked pairs. Their provenance and oracle status could not be established,
|
|
so they were used for discovery only. A scalar phase-consensus score and its
|
|
threshold were selected on those pairs, four older Google-oracle positives, and
|
|
30 external negatives.
|
|
|
|
The frozen threshold then produced six false positives on a new 100-image
|
|
external holdout. It also detected only one of four newly generated Gemini
|
|
images collected after threshold selection. Duplicate-image removal cannot
|
|
reduce the false-positive rate enough to approach the 0.1% detector gate, and
|
|
the temporal result is far below the required sensitivity. The baseline is
|
|
therefore rejected, not recalibrated on the holdout.
|
|
|
|
This result rules out the fixed phase template as a detector or removal loss.
|
|
It does not rule out a content-adaptive or model-epoch-specific signal. The
|
|
next detector must learn from independently labeled provider data, must retain
|
|
the failed holdout unchanged, and must demonstrate discrimination from export
|
|
format and generator identity.
|
|
|
|
### 2026-08-09: cross-color low-texture consensus rejected
|
|
|
|
A polarity-invariant consensus template was trained on black, white, and red
|
|
Gemini low-texture probes and frozen before evaluation. Its median score fell
|
|
from 0.574 on the training groups to 0.354 on held-out probes of the same
|
|
colors and 0.011 on unseen blue, green, and gray probes. External negatives had
|
|
a median score of 0.0047. The separation therefore depended on the training
|
|
colors and did not generalize across the intended low-texture stratum.
|
|
|
|
This branch is rejected as a current Google detector. It remains useful as a
|
|
negative control demonstrating why high training coherence is not evidence of
|
|
a shared carrier.
|
|
|
|
### 2026-08-09: external V3 codebook detector pilot
|
|
|
|
The numeric format-v2 V3 artifact from `reverse-SynthID` was loaded with pickle
|
|
disabled and evaluated independently; no third-party code was imported or
|
|
executed. A 256-bin phase score with a discovery-frozen threshold of 0.5
|
|
detected five of five Google-oracle-positive images and produced zero false
|
|
positives on 194 external images collected before and after threshold freeze.
|
|
The pilot includes four older positives and one newly generated temporal
|
|
positive. It is promising discovery evidence, not a released detector:
|
|
|
|
- zero errors on 194 negatives cannot support a 0.1% FPR claim;
|
|
- the set lacks enough same-provider, oracle-negative hard controls;
|
|
- the phase profile is a third-party artifact whose positive provenance is not
|
|
independently reproducible from our current corpus;
|
|
- a secondary `active_weight_fraction` shortcut failed on an exact-resolution
|
|
holdout, firing on 17 of 50 new negatives, and is rejected as a
|
|
native-resolution/content confound.
|
|
|
|
The next detector milestone is to reproduce the phase carrier from our own
|
|
deduplicated positives, expand the negative set to at least the rule-of-three
|
|
minimum, and add same-provider hard negatives before fitting or publishing an
|
|
operating point.
|
|
|
|
### 2026-08-09: independently fitted phase-carrier pilot
|
|
|
|
An exact-geometry phase model was then fitted from four of our own
|
|
Google-oracle-positive images. The first unconstrained search was a clear
|
|
multiple-comparisons failure: training scores were 0.983-0.991, while the
|
|
newly confirmed fern positive scored 0.061 and an exact-resolution negative
|
|
reached 0.105. Leave-one-out discovery detected only two of four older
|
|
positives. That unrestricted model is rejected.
|
|
|
|
A second model used only the 256 frequency coordinates proposed by the numeric
|
|
V3 artifact, while relearning phase, weight, and expected magnitude entirely
|
|
from our four positives. This restricted model scored the fern validation
|
|
positive at 0.740 and the four fitting positives at 0.936-0.963. A threshold of
|
|
0.43 was frozen after 50 validation negatives whose maximum was 0.305.
|
|
|
|
The phase-only epoch failed its first new holdout with three false positives in
|
|
121 exact-size images. All three had active carrier weight between 0.07 and
|
|
0.14, so the failed epoch was closed and a new rule was frozen: phase score at
|
|
least 0.43 and active weight at least 0.50. That second epoch produced zero
|
|
false positives on 279 newly downloaded exact-size images.
|
|
|
|
A separate 3,000-image COCO check also produced zero joint-rule positives, but
|
|
it is not counted toward the supported-stratum FPR bound. Every COCO image had
|
|
been upscaled to the model geometry and every active weight was at most 0.201.
|
|
The rule therefore abstained because the inputs were outside its measured
|
|
carrier-support distribution. Treat active weight as an `abstain` gate, not as
|
|
negative SynthID evidence.
|
|
|
|
This is the first positive local-detector pilot derived from our labels, but it
|
|
does not clear the detector gate. It still depends on third-party candidate
|
|
coordinates, has only five confirmed positives including the fitting images,
|
|
has only 279 fresh negatives inside the current support stratum, and has no
|
|
same-provider hard negatives. The next epoch requires a larger independently
|
|
labeled positive set, at least 3,000 native-support negatives, same-provider
|
|
oracle negatives, and a new temporal positive that has not influenced feature
|
|
or threshold selection.
|
|
|
|
### 2026-08-09: color-space spectral comparison
|
|
|
|
The restricted phase-carrier experiment was repeated in RGB, full-range
|
|
YCbCr, YCoCg, an orthogonal opponent basis, CIE Lab, and HSV. Every branch
|
|
started from the same 102 spatial frequencies: the top 256 external codebook
|
|
coordinates contained 102 unique `(row, column)` pairs, which were expanded
|
|
over all three components. Each branch then independently selected 256 of the
|
|
306 component-frequency candidates and relearned phase, expected magnitude,
|
|
and weight from the same four oracle-positive fitting images. The score was
|
|
the phase score multiplied by active carrier weight, so weak spectral support
|
|
reduced rather than merely qualified the evidence.
|
|
|
|
Thresholds were frozen from the fern validation positive and 50 exact-size
|
|
validation negatives before scoring the 279-image comparison set. The
|
|
comparison set was locked for this color-space branch, although it had already
|
|
served as the second epoch's RGB negative set and is therefore not a globally
|
|
virgin corpus.
|
|
|
|
| Space | Validation gap | Comparison negative max | Fern minus negative max | False positives |
|
|
|---|---:|---:|---:|---:|
|
|
| RGB | 0.509 | 0.097 | 0.459 | 0/279 |
|
|
| YCbCr | 0.330 | 0.193 | 0.312 | 0/279 |
|
|
| YCoCg | 0.437 | 0.163 | 0.390 | 0/279 |
|
|
| Opponent | 0.351 | 0.167 | 0.339 | 0/279 |
|
|
| Lab | 0.408 | 0.126 | 0.382 | 0/279 |
|
|
| HSV | 0.530 | 0.084 | 0.495 | 0/279 |
|
|
|
|
HSV had the best observed worst-negative margin, narrowly ahead of RGB, but
|
|
did not generally shift paired negatives below RGB. After normalizing each
|
|
space by its fern score, the median paired HSV-minus-RGB difference was 0.0015
|
|
and a two-sided sign test gave `p=0.632` (144 higher, 135 lower). The result is
|
|
therefore a tail observation, not evidence that HSV dominates RGB.
|
|
|
|
Channel decomposition localized the useful effect. HSV hue contributed only
|
|
0.012 of fern evidence and its fern-minus-holdout-maximum channel gap was
|
|
-0.024; saturation and value contributed 0.333 and 0.233, with positive gaps
|
|
of 0.273 and 0.194. In YCbCr, YCoCg, opponent, and Lab, the luminance-like
|
|
channel supplied the largest and most stable separation, while chroma-channel
|
|
gaps were much smaller. This supports a carrier expressed across intensity and
|
|
contrast structure, with useful nonlinear saturation/value projections, not a
|
|
hue-specific watermark.
|
|
|
|
This experiment does not raise the detector's evidence grade. It still has
|
|
only one held-out confirmed positive, reuses the same third-party spatial
|
|
frequency prior, lacks same-provider hard negatives, and evaluates only one
|
|
native geometry. A future HSV S+V ablation must freeze its own operating point
|
|
and use a new negative epoch and new oracle-positive images; dropping hue now
|
|
would be a post-holdout model choice.
|
|
|
|
### 2026-08-09: pixel-only R1 pilot did not transfer
|
|
|
|
Three non-generative candidate families were frozen locally before Gemini
|
|
verification:
|
|
|
|
1. local visible-region inpainting changed 0.087% of pixels at 47.64 dB PSNR
|
|
and 0.99963 SSIM;
|
|
2. sparse complex-spectrum subtraction used 256, 4,096, or 16,384 numeric V3
|
|
bins while preserving native 2816x1536 geometry;
|
|
3. deterministic spatial fragmentation combined smooth local warps, one global
|
|
resampling operation, mild color changes, and codec round-trips.
|
|
|
|
Local inpainting left the 256-bin phase score effectively unchanged. Healthy
|
|
Gemini sessions continued to detect both the target and its spatially shifted
|
|
sham, refuting an earlier one-off `not detected` response.
|
|
|
|
The widest spectral candidate reduced the frozen 256-bin phase score from
|
|
0.639 to 0.124 while retaining 51.94 dB PSNR and 0.99854 SSIM. A
|
|
pixel-identical, metadata-normalized source control was detected in two healthy
|
|
oracle sessions. The 16,384-bin candidate remained detected in one of them and
|
|
the second hit quota before returning a verdict. The 4,096-bin candidates also
|
|
remained detected. Therefore the external phase carrier is correlated with the
|
|
Google signal but is not a sufficient removal loss.
|
|
|
|
One additional profile returned `not detected` for the 16,384-bin candidate,
|
|
then also returned `not detected` for the untouched source. That entire session
|
|
is invalidated rather than counted as removal success. This failure adds a new
|
|
oracle-health rule: every account/session batch starts with the same
|
|
pixel-identical positive control; candidate verdicts are discarded when that
|
|
control is not positive. Quota and tool-unavailable responses remain
|
|
`indeterminate`.
|
|
|
|
The frozen fragmentation batch is not yet oracle-resolved because healthy
|
|
profiles exhausted their rolling quota. Its balanced and strong variants also
|
|
fall below the release fidelity gate locally, so even a future negative oracle
|
|
result would be a mechanism-discovery result, not a releasable remover.
|
|
|
|
A follow-up bounded-warp batch replaced RMS-normalized displacement with strict
|
|
per-axis maxima of 1.0, 1.8, and 2.8 pixels. The 1.0-pixel variant retained
|
|
37.79 dB PSNR and 0.9784 SSIM but left the local phase score essentially
|
|
unchanged at 0.643. The 2.8-pixel variant reduced the score only to 0.503 while
|
|
falling to 28.21 dB and 0.8570 SSIM. A multi-transform bounded variant reached
|
|
0.159 locally only at 21.97 dB and 0.6749 SSIM. Spatial fragmentation therefore
|
|
does not approach the release fidelity gate in this implementation and remains
|
|
a preregistered mechanism probe for the next healthy oracle window, not a
|
|
product candidate.
|
|
|
|
### 2026-08-09: positive-only RGB plus S/V detector epoch
|
|
|
|
The color-space result was converted into a deliberately one-sided research
|
|
detector. It requires exact 2816x1536 geometry, RGB evidence of at least 0.302,
|
|
HSV saturation-plus-value evidence of at least 0.302, and active carrier
|
|
support of at least 0.50 in both branches. It emits `positive` only when every
|
|
condition passes and emits `abstain` otherwise; it never claims that SynthID is
|
|
absent.
|
|
|
|
Leave-one-positive-out fitting passed all four older Google positives. Their
|
|
minimum held-out RGB evidence was 0.527, minimum held-out S+V evidence was
|
|
0.452, and minimum active support was 0.600. The final frozen models detected
|
|
all five available oracle-positive controls. A new Picsum epoch downloaded
|
|
IDs 600-999; after exact-geometry validation and content-hash deduplication
|
|
against all earlier epochs, 330 images remained. None produced a positive
|
|
verdict. However, 328 of 330 abstained for insufficient carrier support and
|
|
only two reached both support gates before falling below the evidence
|
|
thresholds. This validates a positive-only signal catcher inside one geometry,
|
|
not a two-sided detector or a general FPR claim.
|
|
|
|
### 2026-08-09: ensemble phase projection is not a sufficient removal loss
|
|
|
|
An analytical pixel-only attack removed the positive complex projection onto
|
|
the learned RGB phases and HSV S/V phases. One iteration at strength 0.5
|
|
cleared the local ensemble at 56.03 dB PSNR, 0.99909 SSIM, and a maximum pixel
|
|
change of two levels. A norm-matched random-noise control remained locally
|
|
positive.
|
|
|
|
The provider result rejected this loss as sufficient. In a Gemini session
|
|
where the untouched fern control returned `SynthID Status: Detected`, all
|
|
three preregistered projection candidates at strengths 0.5, 0.75, and 1.0 also
|
|
returned detected. The strongest candidate had negative local RGB and S+V
|
|
evidence at 51.97 dB PSNR. The ensemble therefore tracks a real correlate but
|
|
does not cover all evidence used by Google's decoder.
|
|
|
|
### 2026-08-09: 16x32 periodic-tile carrier hypothesis
|
|
|
|
The failed global projection exposed a more specific spatial structure. Of
|
|
the top 256 external V3 bins at 1536x2816, 208 (81.25%) lie exactly on an FFT
|
|
lattice spaced by 96 rows and 88 columns. This corresponds to a spatial cell
|
|
of `1536 / 96 = 16` rows by `2816 / 88 = 32` columns. Instead of subtracting a
|
|
small list of harmonics, a new pixel-only probe high-pass filters the image,
|
|
folds the residual modulo 16x32 over 8,448 repeats, and subtracts the complete
|
|
zero-mean folded template.
|
|
|
|
On the fern positive with its visible sparkle already removed, the mildest
|
|
locally clearing candidate used Gaussian sigma 0.6 and strength 1.5. The
|
|
phase-aligned template cleared the ensemble at 57.03 dB PSNR and 0.99911 SSIM;
|
|
the same template shifted by one row and one column remained positive at
|
|
57.02 dB and 0.99912 SSIM. Two additional aligned/shifted pairs reproduced the
|
|
direction at 55.96 and 54.19 dB.
|
|
|
|
The per-image estimator was then run on the four older positives with a frozen
|
|
strength ladder. The minimum locally clearing strength was 2.0 for three
|
|
images and 1.5 for one. All four phase-aligned outputs abstained, while all
|
|
four same-strength one-pixel-shifted controls remained positive. Selected
|
|
outputs retained at least 55.67 dB PSNR and 0.99874 SSIM. This is the strongest
|
|
local causal evidence so far because effect direction follows exact tile
|
|
phase at matched distortion across five contents.
|
|
|
|
It is not a removal result. In a healthy Google verifier session, the fern
|
|
source control was detected and the mild aligned sigma-0.6, strength-1.5
|
|
candidate was also detected. The stronger local variants therefore remain
|
|
unresolved rather than inheriting a negative label from the local ensemble.
|
|
Cross-resolution tile discovery and same-provider hard negatives also remain
|
|
open.
|
|
|
|
### 2026-08-09: provider-specific phase losses failed both remote verifiers
|
|
|
|
The Google result above and a separate OpenAI experiment closed the first
|
|
analytical-loss epoch. The OpenAI branch fitted an 8-pixel phase pattern from
|
|
OpenAI low-texture outputs and produced a 46.70 dB candidate whose local phase
|
|
score changed sign. A single frozen submission to OpenAI Verify still returned
|
|
`SynthID detected`; C2PA was absent because the candidate had been re-encoded.
|
|
This confirms that metadata removal and a local phase-score reversal do not
|
|
establish pixel-watermark removal.
|
|
|
|
The experiments are not cross-provider replications. Google images were sent
|
|
only to Google Verify AI and the OpenAI candidate only to OpenAI Verify. No
|
|
carrier, threshold, response, or verdict was transferred between tracks.
|
|
OpenAI's published verifier guidance prohibits repeated queries for watermark
|
|
reverse engineering or evasion, so that one failed query did not start an
|
|
adaptive loop. Further OpenAI candidates remain local hypotheses unless an
|
|
authorized research oracle becomes available.
|
|
|
|
### 2026-08-09: JPEG-robust periodic EOT surrogate pilot
|
|
|
|
Small residual CNNs were trained separately for each provider as differentiable
|
|
attack surfaces. They are provider classifiers, not SynthID detectors: their
|
|
corpora lack current same-provider oracle-negative controls, and several
|
|
external negatives approach the positive range. Two same-architecture models
|
|
were used for optimization and a third model with a different first kernel and
|
|
channel schedule was kept outside the attack.
|
|
|
|
For Google, all three models scored the temporal fern positive at a 75th-patch
|
|
percentile between 0.938 and 0.962. A patch-local L-infinity-8 attack initially
|
|
cleared the models and survived JPEG, but visual inspection exposed rectangular
|
|
grid boundaries; that candidate was rejected despite favorable scalar metrics.
|
|
A replacement optimized one 16x32 residual cell repeated over the full image,
|
|
with differentiable JPEG-95 and JPEG-90 branches. At half strength it retained
|
|
36.80 dB PSNR and 0.9241 SSIM against the icon-clean source. The three PNG
|
|
scores fell to 0.024, 0.047, and 0.016; after an actual JPEG-90 round trip they
|
|
were 0.182, 0.243, and 0.016. The independent RGB and HSV S+V phase ensemble
|
|
also abstained, and remained below threshold after JPEG-90, JPEG-95, WebP-95,
|
|
and a 0.98 resize round trip.
|
|
|
|
For OpenAI, the held-out complex market image scored 0.936, 0.929, and 0.813
|
|
on its three provider classifiers. An independently optimized 8x8 periodic EOT
|
|
residual was then attenuated in low-texture regions. The selected local
|
|
candidate retained 34.97 dB PSNR and 0.9356 SSIM. Its PNG scores were 0.030,
|
|
0.053, and 0.099; after actual JPEG-90 they were 0.042, 0.063, and 0.178.
|
|
WebP-95 behaved similarly, while a 0.98 resize raised the independent model to
|
|
0.692 and remains a measured weakness. A second optimization that included a
|
|
resize branch improved the two optimized models but transferred worse to the
|
|
independent model, so it was rejected.
|
|
|
|
The periodic residuals were then tested for cross-image transfer without
|
|
further optimization. The OpenAI 8x8 cell learned from the market image was
|
|
applied with the same low-texture attenuation to 11 other OpenAI positives.
|
|
Every image moved downward on all three models, both as PNG and after an actual
|
|
JPEG-90 round trip. Median PNG scores changed from 0.971, 0.970, and 0.931 to
|
|
0.016, 0.033, and 0.036. Each model had 11 of 11 paired decreases, giving a
|
|
two-sided exact sign-test `p=0.00098`; per-image PSNR ranged from 33.50 to
|
|
35.99 dB. Two complex scenes retained materially higher scores on the
|
|
independent model, so the transfer is strong but not uniform.
|
|
|
|
The Google 16x32 EOT cell was likewise applied to the four older images after
|
|
their own locally clearing phase-tile subtraction. All four remained phase
|
|
ensemble abstentions, and every CNN score decreased in the lossless outputs.
|
|
JPEG-90 preserved the direction but restored the two optimized models to
|
|
roughly 0.58-0.78 on the hardest images. Fidelity was approximately 37.25 dB
|
|
PSNR. With only four paired images this is descriptive transfer evidence, not a
|
|
statistical or oracle gate.
|
|
|
|
These are positive local robustness results, not watermark-removal results.
|
|
Neither candidate meets the release fidelity gate, neither CNN ensemble has
|
|
cleared the detector-identifiability gate, and neither selected EOT candidate
|
|
has a matching-provider negative oracle verdict. A three-file Google batch is
|
|
hash-frozen outside the repository for the next healthy quota window: the
|
|
positive source, the stronger phase-only ablation, and the combined periodic
|
|
EOT candidate. Results will be accepted only in that fixed order without
|
|
between-query adaptation. Two later attempts detected the positive source, then
|
|
returned a usage-limit response for the phase-only ablation. In the latest
|
|
attempt the verifier requested a retry after 17 hours. Both responses are
|
|
`indeterminate`, and the combined candidate was not submitted out of order. The
|
|
OpenAI candidate is not scheduled for the public verifier under the current
|
|
usage restriction.
|
|
|
|
### 2026-08-09: OpenAI fidelity and architecture-transfer frontier
|
|
|
|
Further local experiments tested whether the OpenAI periodic result could move
|
|
above the release fidelity gate without losing transport robustness. Chroma-only
|
|
projection reached 47.43 dB PSNR, but an actual JPEG round trip removed the
|
|
perturbation and restored the two classifier scores to approximately 0.9. Mixed
|
|
luminance/chroma projections and stronger texture masks did not improve that
|
|
frontier. A regularized Adam attack reached 41.21 dB on the source image, but
|
|
JPEG-90 restored the optimized classifiers to 0.90 and 0.91, and the independent
|
|
architecture scored 0.89. This rejected post-hoc color projection and simple
|
|
regularization as fidelity remedies.
|
|
|
|
A second experiment put actual OpenCV JPEG and WebP results in the forward pass
|
|
and used a straight-through gradient. At 40.08 dB, actual JPEG-90 still scored
|
|
0.89 and 0.91 on the optimized models. Removing the luminance penalty improved
|
|
the first model to 0.76 but left the second at 0.88 and reduced fidelity to
|
|
38.78 dB. The exact-codec forward pass therefore corrected an invalid transport
|
|
approximation, but did not produce a new Pareto candidate.
|
|
|
|
An architecture-diverse fourth classifier used fine RGB residuals, broader RGB
|
|
residuals, and luminance/opponent-color residuals. The market image remained
|
|
outside its training split. Its 75th-patch percentile ranged from 0.983 to 0.988
|
|
on four held-out OpenAI positives; the median across 120 held-out COCO negatives
|
|
was 0.041 and the maximum was 0.521. This is useful attack-surface diversity, not
|
|
a certified detector, because no current same-provider oracle-negative corpus is
|
|
available.
|
|
|
|
Adding that model to periodic optimization did not improve architecture transfer.
|
|
The preregistered local winner retained 34.86 dB and scored 0.057, 0.197, and
|
|
0.021 after JPEG-90 on the three optimized models, but the independent model
|
|
scored 0.573. A per-image half-resolution perturbation produced a higher-fidelity
|
|
41.86 dB candidate; JPEG-90 scores on its optimized models were 0.587, 0.572,
|
|
and 0.471, while the independent model remained at 0.875. Optimizing only the
|
|
two original models produced the same failure on both independent architectures.
|
|
|
|
The measured local frontier is therefore unchanged. The 34.97 dB periodic
|
|
candidate remains the strongest transport-robust and cross-architecture result;
|
|
the candidates above 40 dB are architecture-specific and cannot support a
|
|
removal claim. No additional public OpenAI verifier requests were made during
|
|
these experiments.
|
|
|
|
### 2026-08-09: mechanism reset and paired open-method control
|
|
|
|
The first detector epochs optimized the wrong statistical problem. The
|
|
[technical reference](synthid.md#11-post-hoc-model-independent-design) now
|
|
records the paper's paired training target, symmetric transformation channel,
|
|
independent detection logit, and two-sided calibration. Applied to the initial
|
|
pilots, those details reject three implicit assumptions: that a carrier phase
|
|
can stand in for the detection score, that mean patch classification is an
|
|
adequate image-level decoder, and that a sigmoid threshold calibrated on
|
|
unrelated negatives is a deployable decision rule.
|
|
|
|
A bounded open-method control tested the architecture concern without claiming
|
|
that another watermark reproduces SynthID. TrustMark P was used only because
|
|
its public encoder creates causal clean/watermarked pairs and its public decoder
|
|
can validate transformed labels. Ninety public COCO images were split by source
|
|
image into 60 training, 15 calibration, and 15 test pairs. Each pair received
|
|
the same identity, JPEG-90, 0.8 resize round trip, or 5% crop transformation.
|
|
The official decoder detected 14/15, 15/15, 14/15, and 15/15 transformed marked
|
|
test images respectively, with zero detections on the corresponding clean
|
|
images.
|
|
|
|
The existing normalized residual patch recipe reached only 0.707 identity AUC
|
|
and 0.668 aggregate AUC on the held-out pairs. A small full-field model that
|
|
retained RGB amplitude plus fine and coarse residuals reached 0.947 identity AUC
|
|
and 0.841 aggregate AUC. At a threshold above every calibration negative, their
|
|
aggregate TPRs were 5% and 20% respectively. Both models failed to reproduce
|
|
the official decoder's JPEG robustness. This small proxy is not a TrustMark
|
|
benchmark and says nothing directly about SynthID accuracy. It does falsify the
|
|
claim that the current patch-normalized architecture and training recipe are an
|
|
adequate generic neural-watermark detector.
|
|
|
|
The resulting eight falsifiable hypotheses now define the canonical
|
|
[D3-D5 mechanism program](#mechanism-hypotheses-for-d3-d5). In particular,
|
|
external generator corpora remain a final false-positive challenge and do not
|
|
substitute for counterfactual negatives.
|
|
|
|
Until the counterfactual-label and full-field gates pass, the current residual
|
|
CNN remains a useful vendor-triage stage for a future cascade, but it is not a
|
|
SynthID detector and should not be optimized as the final decision surface.
|
|
|
|
### 2026-08-09: paired wavelet and spectral ablation
|
|
|
|
The paired spectral harness was extended with three-level undecimated `db2`
|
|
wavelets, complex Fourier phase coherence and power, cepstral peaks, and a
|
|
cyclic clean/marked permutation control. The implementation streams wavelet and
|
|
spectrum field accumulators rather than stacking transformed fields across
|
|
pairs.
|
|
|
|
On 60 public TrustMark P training pairs at canonical size 256, the true
|
|
residuals had mean inter-pair RGB NCC of 0.097-0.121. Cyclically mismatching each
|
|
clean image with the next marked image reduced all three channels to about
|
|
-0.017. The strongest coherent Fourier cluster occurred around vertical offsets
|
|
9-11 and was strongest in the blue channel. Wavelet repeatability was greatest
|
|
in horizontal detail bands at levels 1 and 2. These are causal residual
|
|
measurements for the open proxy, not evidence about SynthID's carrier.
|
|
|
|
The representations then faced a source-disjoint single-image test with 60
|
|
training, 15 calibration, and 15 test pairs under identity, JPEG-90, 0.8 resize,
|
|
and 5% crop transformations. A fixed spectral-template score reached only 0.618
|
|
identity AUC and collapsed to 0.498 under crop. A 318-feature wavelet/spectral
|
|
summary reached 0.653 aggregate AUC and zero TPR at the threshold above every
|
|
calibration negative.
|
|
|
|
Full-field neural ablations at size 128 gave the following aggregate results:
|
|
|
|
| Input | AUC | Paired wins | Test FPR | Test TPR |
|
|
| --- | ---: | ---: | ---: | ---: |
|
|
| Spatial RGB plus residuals | 0.723 | 57/60 | 0% | 11.7% |
|
|
| SWT maps only | 0.671 | 59/60 | 0% | 0% |
|
|
| Complex FFT maps only | 0.516 | 41/60 | 0% | 0% |
|
|
| Spatial plus SWT, late fusion | 0.665 | 59/60 | 5.0% | 8.3% |
|
|
| Spatial plus SWT plus FFT, early fusion | 0.639 | 57/60 | 1.7% | 1.7% |
|
|
| Spatial plus SWT plus FFT, late fusion | 0.698 | 54/60 | 0% | 0% |
|
|
|
|
The table uses one threshold above every calibration negative; test FPR is
|
|
reported separately because a clean calibration result does not guarantee a
|
|
clean locked test.
|
|
|
|
The SWT branch contains weak, unusually consistent paired evidence, but it did
|
|
not improve the spatial model's source-disjoint discrimination in this small
|
|
proxy. The FFT map branch was indistinguishable from chance. Therefore neither
|
|
representation advances into the primary detector by default. SWT remains a
|
|
late-fusion ablation for a larger paired corpus; complex spectral analysis
|
|
remains a residual-discovery diagnostic unless a future held-out test reverses
|
|
this result.
|
|
|
|
### 2026-08-09: closed-corpus architecture and score ablations
|
|
|
|
A temporally split closed three-class corpus then tested whether the open-proxy
|
|
architecture result transferred to real external-source images. These labels
|
|
establish source classes, not watermark presence; they can measure
|
|
generalization and false positives but cannot substitute for counterfactual
|
|
clean/marked labels.
|
|
|
|
The frozen normalized-patch baseline was repeated over three training seeds.
|
|
Individual original-image AUCs ranged from 0.864 to 0.879. Averaging the three
|
|
models increased AUC to 0.884, but a threshold above every validation negative
|
|
still produced one false positive among 259 test negatives. At that operating
|
|
point the two positive-source TPRs were only 2.45% and 8.97%. Calibrating near
|
|
1% validation FPR yielded 1.54% test FPR and TPRs of 6.68% and 18.16%.
|
|
Accepted positives had 98.8-100% vendor accuracy, so vendor attribution is
|
|
useful only after an independently reliable presence decision.
|
|
|
|
A naive full-field spatial model reached only 0.599 AUC. Adding a separately
|
|
encoded stationary-wavelet branch and late fusion reduced AUC to 0.597. Both
|
|
had effectively zero strict TPR. The result does not contradict the causal
|
|
open proxy: it shows that global average pooling over source-labeled images
|
|
does not isolate the transferable signal and that SWT does not repair the
|
|
source shift.
|
|
|
|
Post-hoc image aggregation also failed to create an operating point. Averaging
|
|
presence logits raised AUC from 0.884 to 0.890, but increased strict test FPR to
|
|
0.77% while reducing TPR. Median, lower-quartile, upper-quartile, and top-tail
|
|
aggregation either preserved the overlap or suppressed almost every positive.
|
|
A separately trained binary presence head plus vendor head reached 0.880
|
|
ensemble AUC, 0.77% strict test FPR, and TPRs of 1.11% and 7.26%. Decoupling the
|
|
heads is architecturally faithful but insufficient without faithful labels.
|
|
|
|
These ablations close spectral feature engineering, naive full-field pooling,
|
|
and score aggregation as the next detector step. The next valid training epoch
|
|
requires same-content counterfactual pairs or independent watermark oracle
|
|
labels, matched transformations, and an image-level detection loss. External
|
|
generator corpora, including difficult non-target providers, remain hard
|
|
negative and FPR-challenge sets only.
|
|
|
|
### 2026-08-10: low-content controls and registered phase carrier
|
|
|
|
A same-resolution low-content matrix compared independently generated solid
|
|
outputs from two target model families against three per-image controls: exact
|
|
mean fill, amplitude-matched Gaussian noise, and a phase-randomized residual
|
|
with preserved Fourier magnitude. Raw stationary-wavelet summaries transferred
|
|
between the two target families with AUCs of 0.982 and 1.000, and reached 0.973
|
|
when blue and green were held out by color. This was not watermark evidence.
|
|
The frozen classifier accepted every one of 1,869 external negatives because
|
|
it had learned the distinction between real generator texture and artificial
|
|
controls. Removing absolute wavelet energy reduced external-negative
|
|
acceptance only to 61.6%, with similar 58.1-67.3% acceptance across all three
|
|
source classes. Both low-content wavelet branches are rejected as presence
|
|
detectors until real non-target solid outputs provide matched negatives.
|
|
|
|
The numeric V3 audit loader was then extended to support both dense and sparse
|
|
format-v2 profiles without pickle. Exact-profile evaluation exposed a sharp
|
|
encoder-version boundary. The 1024x1024 profile accepted none of 231 target
|
|
provider images and none of 26 exact-geometry negatives. The 1536x2816 profile
|
|
accepted 30 of 55 target-provider images, including all four temporal-test
|
|
images, while rejecting the one exact-geometry negative available in the
|
|
closed corpus. The independently fitted phase model accepted 24 of those 55
|
|
and also accepted all four temporal-test images. This is positive evidence for
|
|
a geometry- and epoch-specific carrier, not a universal SynthID decoder.
|
|
|
|
On the four temporal-test positives, the fixed V3 score survived JPEG-95 and a
|
|
75% downscale on all four images, survived JPEG-85 on two, and failed after a
|
|
5% center crop or a one-pixel translation on all four. Bounded analytical
|
|
translation registration recovered all four shifted images and selected the
|
|
known `(-1, -1)` offset. Searching up to 16 pixels produced no positives among
|
|
50 exact-resolution and 144 canonicalized frozen negatives. The shared
|
|
registration implementation now serves both the numeric V3 probe and the
|
|
independently fitted phase model.
|
|
|
|
A discovery-only scale-and-translation view search recovered all four 5%
|
|
cropped temporal images with the independently fitted model after lowering the
|
|
active-support gate from 0.50 to 0.40. It produced zero positives on the 194
|
|
frozen negatives and on the same preregistered 3,000-image COCO challenge used
|
|
by the identity scorer. The latter result has a zero-error one-sided 95% bound
|
|
of 0.0998% only for that abstention challenge: every COCO image remained
|
|
outside carrier support, with a maximum active fraction of 0.201. The scale
|
|
rule is not frozen because its support threshold was selected after inspecting
|
|
the crop examples. It requires a new temporal positive holdout before it can
|
|
join the detector rule.
|
|
|
|
The current actionable research candidate remains a positive-only,
|
|
provider-specific expert for the supported 1536x2816 carrier epoch. Identity
|
|
and bounded translation views use the frozen phase and support thresholds;
|
|
unsupported geometry, insufficient carrier magnitude, and ambiguous phase
|
|
return `abstain`. Vendor attribution may select the expert that supplied
|
|
accepted evidence, but it must not turn an abstention into a provider label.
|
|
The next calibration gate still requires at least 3,000 native-support
|
|
negatives, same-provider oracle negatives, matched non-target solid outputs,
|
|
and a new temporal positive that influenced neither profile nor threshold.
|
|
|
|
### 2026-08-10: 2048 periodic-tile detector
|
|
|
|
The phase hypothesis transferred to a second native geometry when the model
|
|
and operating point were kept separate. A 256-peak 2048x2048 model learned from
|
|
111 train positives used the already frozen phase threshold of 0.43, active
|
|
support threshold of 0.40, and a translation search of plus or minus four
|
|
pixels. It accepted 31 of 35 validation positives and 27 of 30 locked-test
|
|
positives while accepting none of 49 validation or 38 test negatives.
|
|
Translation registration recovered the shifted positives without changing
|
|
those identity counts. A threshold recalibrated from only 49 validation
|
|
negatives reached 30 of 30 test positives but accepted one test negative, so it
|
|
was rejected in favor of the transferred rule.
|
|
|
|
The wider native-geometry challenge exposed the remaining uncertainty. The
|
|
frozen 2048 rule accepted two of 182 earlier external-provider images, for two
|
|
accepted source negatives among all 269 native negatives. Both cases passed at
|
|
zero translation with high phase and support, and both also passed an
|
|
independently learned HSV phase branch. They are operational false positives
|
|
under source labels, but source provenance does not establish watermark
|
|
absence. They may instead expose a shared encoder or upstream backend. Without
|
|
an independent watermark oracle they cannot be relabeled either way. The same
|
|
experiment rejected the 1024x1024 and 768x1376 experts: they accepted 9 of 26
|
|
and 4 of 9 native source negatives, respectively.
|
|
|
|
The 2048 carrier has a concrete periodic mechanism. Its 256 peaks reduced to
|
|
108 unique spatial frequencies. Translating the frequency coordinates by 128
|
|
rows preserved 56 coordinates, while the maximum overlap in each of 1,000
|
|
uniform random controls was two. The permutation estimate was 0.001, and the
|
|
128-bin spacing implies a 16x16 spatial tile. A separate detector therefore
|
|
folded a high-pass residual modulo 16x16, averaged 16,384 repetitions, and
|
|
correlated the normalized tile against a train-positive template. After
|
|
float64 serialization and validation-only threshold calibration, the fixed
|
|
tile accepted 34 of 35 validation and 29 of 30 test positives, none of the 49
|
|
calibration or 38 held-out test negatives, and the same two of 182 earlier
|
|
source negatives. This second representation supports a real periodic carrier
|
|
rather than an arbitrary set of FFT peaks, but it does not resolve the two
|
|
labels.
|
|
|
|
The fixed tile also accepted none of a preregistered 3,000-image general-image
|
|
challenge after deterministic 2048x2048 canonicalization. The maximum
|
|
normalized correlation was 0.106 against a threshold of 0.174. The zero-error
|
|
one-sided 95% upper bound is 0.0998% for that challenge. Unlike the earlier
|
|
sparse-phase result, the tile score evaluates every image without a weak
|
|
carrier-support abstention. The challenge is still not native provider data
|
|
and cannot replace the required oracle-negative calibration.
|
|
|
|
Symmetric attack evaluation established the robustness boundary. The fixed
|
|
tile accepted 29 of 30 original test positives, all 30 after a 75% downscale
|
|
round trip, 21 after JPEG-95, three after JPEG-85, and none after a 5% crop,
|
|
with no accepted held-out negatives under the identity threshold. A
|
|
validation-calibrated JPEG-95 tile threshold recovered 27 of 30 but accepted
|
|
one of 38 test negatives. Requiring both codec-conditioned tile and phase
|
|
scores reduced JPEG-95 to 16 of 30 with no held-out-negative acceptance, but
|
|
still accepted one of the 182 earlier source negatives. The corresponding
|
|
JPEG-85 consensus accepted 5 of 30 positives and none of all 269 native source
|
|
negatives. Scale-and-translation phase search recovered 15 of 35 validation
|
|
and 14 of 30 test crops with no held-out-negative acceptance, but remains
|
|
discovery-only because the test transformation had already influenced the
|
|
branch. Low-frequency peak subsets and transform-augmented phase training
|
|
improved JPEG sensitivity only by raising validation false positives to
|
|
2-10%, so both were rejected.
|
|
|
|
The reproducible implementation is `scripts/synthid_periodic_tile_probe.py`.
|
|
It stores the normalized template in float64 and calibrates only after loading
|
|
the serialized artifact; an earlier float32 experiment moved a boundary score
|
|
by approximately 2.5e-10 and demonstrated why calibration-before-serialization
|
|
is invalid. The resulting research detector is positive-only and limited to a
|
|
confirmed 2048x2048 carrier epoch. An accepted expert may suggest the encoder
|
|
family, but the two cross-source carrier matches prohibit a stronger vendor
|
|
claim until an oracle distinguishes direct provider output from shared-backend
|
|
output.
|
|
|
|
A pixel-space ablation then tested whether the frozen tile merely predicted the
|
|
local scores or controlled them. At twice the train-median tile norm, aligned
|
|
subtraction changed the fixed-tile decision from 29 of 30 accepted originals to
|
|
zero and the independently fitted sparse-phase decision from 27 of 30 to zero.
|
|
The median fidelity was 53.74 dB PSNR and 0.99681 SSIM. Cyclically shifting the
|
|
same tile by one row and column left one phase acceptance, while a seeded
|
|
zero-mean random tile orthogonal to the learned template left 13; their median
|
|
PSNRs were within 0.13 dB of the aligned edit. For both local representations,
|
|
the aligned edit reduced the score more than either control on all 30 paired
|
|
images, with a two-sided sign-test p-value of 1.86e-9 for each comparison.
|
|
The same aligned edit reversed both local decisions on each of the two disputed
|
|
cross-source matches, while the orthogonal control left both phase decisions
|
|
accepted. That two-item result is descriptive but makes an accidental threshold
|
|
crossing less likely; it still cannot distinguish a shared carrier from direct
|
|
provider output.
|
|
|
|
This is the strongest local causal evidence for the 16x16 mechanism, but the
|
|
strength was selected after inspecting this locked test and is therefore
|
|
discovery-only. The shifted control also suppressed the phase representation
|
|
substantially, so local score reversal cannot certify signal removal. A matching
|
|
provider oracle must still compare untouched sources, aligned candidates, and
|
|
norm-matched controls before any removal claim. The reproducible local harness
|
|
is `scripts/synthid_periodic_tile_ablation.py`; its report records input and
|
|
model hashes without writing derivative images.
|
|
|
|
### 2026-08-10: confirmatory oracle-batch readiness
|
|
|
|
The corpus audit found no new 2048x2048 positive collected after the tile rule
|
|
and subtraction strength were frozen. Existing test images cannot be renamed as
|
|
a temporal holdout, so the confirmatory oracle batch remains blocked on new
|
|
provider output at the supported native geometry.
|
|
|
|
The batch mechanics are ready in `scripts/synthid_oracle_batch.py`. For each new
|
|
source, it preregisters five requests in a fixed order: untouched source,
|
|
lossless re-encode, aligned subtraction, cyclic-shifted subtraction, and an
|
|
orthogonal norm-matched tile. It refuses to write media inside the repository,
|
|
records artifact and decoded-pixel hashes, pins both model artifacts and the
|
|
matching provider, and detects any post-registration byte change. Oracle results
|
|
live in a separate manifest-bound file so the preregistration remains immutable;
|
|
the evaluator requires complete ordered results, verbatim responses,
|
|
timezone-aware timestamps, and separate SynthID and C2PA verdicts.
|
|
|
|
A causal success requires the source, re-encode, shifted, and orthogonal views
|
|
all to remain `detected` while only the aligned view becomes `not_detected`.
|
|
`Indeterminate` and refusal remain indeterminate. A one-source dry run on an old
|
|
test positive verified all five artifacts and the local scoring path, but is
|
|
pipeline validation only and contributes no confirmatory evidence.
|
|
|
|
### 2026-08-10: first confirmatory oracle attempt
|
|
|
|
Four new 2048x2048 Gemini outputs were generated after the local rule and
|
|
strength were frozen, then registered as a 20-request batch. The manifest and
|
|
all 20 artifacts passed hash verification before submission. The first source
|
|
group produced these ordered results:
|
|
|
|
| Role | Local tile/phase decision | Provider result |
|
|
| --- | --- | --- |
|
|
| source | detected / detected | SynthID indeterminate; Google C2PA present |
|
|
| re-encode control | detected / detected | Google AI signal detected |
|
|
| aligned subtraction | detected / not detected | Google AI signal detected |
|
|
| shifted control | detected / detected | Google AI signal detected |
|
|
| orthogonal control | detected / detected | refused: verification quota exceeded |
|
|
|
|
The quota response instructed the account to wait 24 hours. No adaptive retry
|
|
was made, and the remaining 15 preregistered requests were not submitted. The
|
|
batch cannot receive its preregistered aggregate verdict until complete ordered
|
|
results exist.
|
|
|
|
Even incomplete, this attempt rejects the frozen removal recipe for the first
|
|
temporal source: aligned subtraction crossed the phase threshold but remained
|
|
above the tile threshold and did not clear the provider oracle. The remaining
|
|
sources can still measure transfer and disagreement between the two local
|
|
experts, but they cannot turn this first aligned result into a universal
|
|
pixel-only removal success.
|
|
|
|
### 2026-08-10: positive-only runtime detector
|
|
|
|
Removal is deferred while the transferred tile signal is exposed as a bounded
|
|
detector. `src/remove_ai_watermarks/synthid_detector.py` loads the frozen model
|
|
as a bundled pickle-free runtime asset and returns `detected`, `not_detected`,
|
|
or `unsupported` without resizing input. The direct API and
|
|
`detect-synthid` CLI initially covered only native 2048x2048 images; `identify`
|
|
consumes a positive match as high-confidence evidence but never treats a
|
|
negative or unsupported result as proof of absence.
|
|
|
|
The operating point and model are unchanged from the locked experiment. The
|
|
detector accepted all four post-freeze Gemini outputs, including the source
|
|
whose frozen phase score missed, while retaining the prior 29-of-30 locked-test
|
|
sensitivity and zero accepted validation/test negatives. The unresolved two
|
|
external-source matches remain the reason a runtime positive names the carrier
|
|
but does not attribute a provider.
|
|
|
|
### 2026-08-10: calibrated native-geometry extension
|
|
|
|
The frozen 16x16 template and threshold transfer across the common native
|
|
geometries in the provider-positive evidence. Non-divisible image dimensions
|
|
use direct modulo folding with per-cell sample counts; divisible dimensions,
|
|
including 2048x2048, retain the original numerical path. All prior 2048x2048
|
|
records were replayed and matched exactly, including floating-point scores.
|
|
|
|
The runtime registry was expanded only after the unchanged threshold accepted
|
|
none of 60,000 public COCO challenge views, 3,000 at each of 20 target
|
|
geometries. This is a geometry extension of the same positive-only carrier
|
|
expert, not an OpenAI pixel detector or a proprietary payload decoder.
|
|
|
|
### 2026-08-11: calibrated image-size range
|
|
|
|
The fixed template was then evaluated on every provider-positive image in the
|
|
local evidence set rather than only its common geometries. It accepted 3,928 of
|
|
4,698 images across 757 exact geometries. Sensitivity separated by pixel count:
|
|
1,987 of 2,021 images at or above three megapixels crossed the threshold, while
|
|
1,940 of 2,672 images from one through two megapixels did. This establishes a
|
|
carrier-family boundary, not universal SynthID recall: explicit C2PA watermark
|
|
actions also occur below threshold, and three strong carriers use a different
|
|
cyclic phase.
|
|
|
|
Two public-image geometry challenges tested whether geometry itself creates
|
|
false matches. The first balanced 5,000 COCO images across all 757 observed
|
|
geometries, with every geometry present in both development and final partitions;
|
|
the maximum fixed score was 0.12549 and none crossed the unchanged 0.17357
|
|
threshold. The second transformed the same 5,000 source images at 256 generated
|
|
geometries from one through 18 megapixels, covering every width/height remainder
|
|
pair modulo 16. Its maximum was 0.16387 and again none crossed the threshold.
|
|
Runtime support therefore uses that challenged pixel-count interval instead of
|
|
an enumerated geometry registry, still without resizing.
|
|
|
|
A cyclic-registration branch was rejected after the public-image challenge.
|
|
At a low threshold it recovered 130 fixed-phase misses but false-fired on
|
|
tracked OpenAI and Adobe controls. Raising the threshold above all development
|
|
controls recovered only 23 in-range provider positives while adding 84
|
|
cross-source matches beyond the fixed detector. Three strong shifted Google
|
|
carriers remain a documented miss rather than weakening the runtime rule.
|
|
|
|
Top-level source labels were not treated as watermark-negative truth. Of 237
|
|
non-Google-labeled corpus images crossing the fixed threshold, 81 retained
|
|
SynthID provenance. High-scoring Adobe manifests explicitly named Gemini Flash
|
|
as the remote provider, and perceptual matching found direct Google derivatives
|
|
under editing or relabeling categories. The remaining source labels cannot prove
|
|
absence of the carrier, so they constrain provider attribution but do not justify
|
|
raising the pixel threshold.
|
|
|
|
### 2026-08-11: spatial-resampling boundary
|
|
|
|
Arbitrary decoded geometry did not imply arbitrary resize robustness. A
|
|
stratified sample of 80 fixed-positive provider images was rescaled in memory at
|
|
0.5, 0.625, 0.75, 0.875, 1.125, 1.25, and 1.5. The unchanged 16x16 detector
|
|
accepted none of the 80 images at every nonidentity scale. Restoring each view
|
|
to its source dimensions recovered 58-80, which isolates the failure to the
|
|
scaled sampling lattice rather than simple information destruction.
|
|
|
|
Scaling the template and modulo fold to the corresponding integer periods 8,
|
|
10, 12, 14, 18, 20, and 24 recovered 73-80 of the discovery images at the
|
|
native threshold. Searching all eight periods raised control scores, so a
|
|
single threshold was frozen above the maximum of 3,000 deterministically
|
|
resized COCO development views. It accepted none of 2,000 final controls and
|
|
672 of 800 source-disjoint provider positives, with 77-88 accepted at each
|
|
scale. Six tracked OpenAI and Adobe controls remained below the frozen threshold.
|
|
|
|
This did not generalize to arbitrary scaling. Factors 0.8, 0.9, 1.1, 1.2, and
|
|
1.333 create noninteger carrier periods; rounding the period accepted zero of
|
|
80 images in the fixed-phase branch at every factor, and cyclic registration
|
|
accepted only 1-16. Period-specific thresholds improved sensitivity but accepted
|
|
five of the 2,000 final controls. The integer-period bank is therefore retained
|
|
as a research result, not added to the runtime. A universal resize branch needs
|
|
fractional-period registration or reliable reconstruction of the pre-resize
|
|
geometry, followed by a new multiple-search calibration.
|
|
|
|
A discovery follow-up tested that fractional-period direction with the 30
|
|
strongest template harmonics. A continuous 7.5-24.5 spectral search placed the
|
|
true period within 0.05 pixels among its top three candidates for 58 of 60
|
|
transformed positives. Searching nine neighboring integer reconstruction
|
|
geometries recovered 44 of 60 at the native threshold; even the true source
|
|
geometry recovered only 48, so resampling damage sets a separate upper bound.
|
|
The full search then failed a frozen 250-development/150-final COCO challenge:
|
|
the calibration threshold accepted two final controls. A preselected product
|
|
of canonical correlation and spectral confidence increased positive
|
|
acceptance from 49 to 52 of 60 at its development threshold but accepted the
|
|
same two final controls. This fractional FFT baseline is rejected for runtime
|
|
use; better period estimation alone does not solve multiple-search specificity.
|
|
|
|
### 2026-08-11: multiscale calibration and opt-in registered detector
|
|
|
|
The fractional search was revisited as a multiple-search calibration problem,
|
|
not as period estimation alone. A source-disjoint challenge used 200 confirmed
|
|
fixed-carrier positives, 40 independent sources at each scale 0.8, 0.9, 1.1,
|
|
1.2, and 1.333. Controls were 4,900 unique COCO images resized into the positive
|
|
pixel-count domain, split into 2,900 development and 2,000 final items. The ten
|
|
frozen candidates combined exact fractional canonical correlation, quadrant
|
|
agreement, and a three-level scale-space score through mean or minimum rules.
|
|
Pure box-counting dimensions, multifractal width, lacunarity, roughness,
|
|
wavelet-energy slope, and a generic log-Mellin response had already failed the
|
|
smaller pilot and were not promoted to the large challenge.
|
|
|
|
At the development-only threshold, the three-way mean accepted 177 of 200
|
|
positives versus 163 for canonical correlation alone, with 16 positives rescued
|
|
and two lost in the paired decision. Each produced one final false positive, on
|
|
different source images, so the development threshold was rejected. Recalibrating
|
|
above all 4,900 COCO controls left 176 of 200 three-way positives and 160 of 200
|
|
canonical-only positives. This recalibration is exploratory rather than a new
|
|
validation because the final controls had then been inspected.
|
|
|
|
A second source-disjoint corpus froze those thresholds before scoring 105 new
|
|
positives across scales 0.5, 0.65, 0.8, 1.0, 1.2, 1.333, and 1.5, plus all 879
|
|
content-deduplicated Kodak and Picsum controls available locally. Natural
|
|
2816x1536 controls had a heavier null tail than upscaled COCO: the three-way
|
|
mean accepted 82 of 105 positives and six controls at the COCO threshold,
|
|
whereas canonical-only accepted 79 positives and ten controls. A threshold
|
|
above all 5,779 controls retained 253 of the first 305 positives for the
|
|
three-way mean. Four coarse pixel-count bins recovered only six more positives,
|
|
so size normalization did not explain the main 0.5x miss.
|
|
|
|
At 0.5x, a fixed period-8 branch raised canonical correlation for seven of the
|
|
15 discovery positives but also won the unconstrained search on 629 of 879
|
|
controls. Searching it unconditionally merely raised the null threshold. The
|
|
frozen remedy requires period-8 native correlation of at least 0.35 and takes
|
|
the maximum of the old and period-8 three-way scores. It added four positives
|
|
without an accepted control in the discovery corpus. On the earlier 4,900
|
|
controls, the frozen gate never opened and none crossed the final three-way
|
|
threshold `0.2460603834083705`.
|
|
|
|
The improvement itself was then tested on 50 additional source-disjoint 0.5x
|
|
positives that influenced neither gate nor threshold. The ungated three-way
|
|
score accepted 6; the frozen gate accepted 17, rescuing 11 and losing none. The
|
|
exact paired sign test was `p = 0.0009765625`. These results support an opt-in
|
|
scale-registered detector over 250,000 through 10,000,000 decoded pixels. The
|
|
runtime also requires both sides to be at least 64 pixels so each canonical
|
|
quadrant can retain a full 16-by-16 tile. These results do not establish a
|
|
universal SynthID decoder: the control maximum helped set the operating point,
|
|
33 of 50 new 0.5x positives still missed, and crop, strong codec changes, other
|
|
carrier epochs, and provider attribution remain outside the claim. The fast
|
|
native fold remains the default and the `identify` path.
|
|
|
|
That period-8 conclusion was overturned by a later symmetric negative test.
|
|
Every one of the 11,506 known non-Google Spaces controls was downscaled by 0.5,
|
|
matching the positive transformation; 11,273 outputs remained in the registered
|
|
geometry range. The direct analytic 8-by-8 template was dominated by shared
|
|
resize structure: 80.8% of its energy projected onto a 2-by-2 lattice and 94.0%
|
|
onto a 4-by-4 lattice. The production period-8 gate opened on 73 controls, and
|
|
all 73 crossed the registered threshold. They spanned hosted jobs, Firefly,
|
|
China AIGC-labeled generators, Meta-tagged images, OpenAI, and local tools. No
|
|
tested secondary condition separated the 11 rescued positives from those 73
|
|
controls. The explicit period-8 rescue is therefore rejected and not part of
|
|
the runtime detector. The 0.5x rescue result is retained here as an incident
|
|
record against asymmetric transformed-positive evaluation.
|
|
|
|
The same symmetric corpus also invalidated the main single-threshold registered
|
|
rule: 68 of 11,273 supported controls crossed `0.2460603834083705`. Fixed
|
|
thresholds for nine recovered-period bins eliminated those errors while
|
|
retaining 238 of 355 transformed positives, but one of 4,900 COCO controls then
|
|
crossed through the third spectral candidate. Requiring the canonical winner to
|
|
match the strongest spectral candidate removed that case and all 68 Spaces
|
|
errors, with 239 of 355 positives retained. This version was not shipped: a
|
|
fresh, content-deduplicated 1,000-image Open Images validation corpus produced
|
|
six false positives.
|
|
|
|
All six Open Images errors concentrated their apparent match in one
|
|
middle-frequency ring. Their correlations disappeared or changed sign in the
|
|
two higher-frequency rings of the 16-by-16 carrier, unlike the accepted Gemini
|
|
positives. Adding a frozen minimum correlation of `0.075` across those two rings
|
|
removed the six development errors and retained 229 of 355 positives. A
|
|
previously untouched reserve of 499 supported Open Images controls then produced
|
|
five amplitude-gate candidates and zero final positives; their maximum
|
|
high-band score was `0.004682175209264222`.
|
|
|
|
The retained positives by scale were 0/65 at 0.5, 8/15 at 0.65, 53/55 at 0.8,
|
|
33/40 at 0.9, 13/15 at 1.0, 30/40 at 1.1, 38/55 at 1.2, 40/55 at 1.333, and
|
|
14/15 at 1.5. A physically transformed template recovered strong correlations
|
|
for some period-8 positives, but 975 top-candidate Spaces controls fully
|
|
overlapped every tested amplitude, spectral-band, and joint feature. No
|
|
zero-control operating point retained a period-8 positive. The calibrated
|
|
runtime therefore uses the top-candidate, period-bin, and high-band gates and
|
|
makes no 0.5x detection claim.
|
|
|
|
### 2026-08-12: Registered color and phase-lock challenge
|
|
|
|
A crop-specific research branch tested whether the recovered 16x16 carrier is
|
|
better represented in a perceptual color space or a shift-tolerant directional
|
|
transform. Every branch reused the direct RGB period-and-phase registration,
|
|
fitted on the first 30 positives, selected on the next 20 positives and 200
|
|
controls, and reported the remaining 50 positives and 299 controls separately.
|
|
Only records whose recovered period was 15.5 through 16.5 were eligible.
|
|
|
|
The frozen RGB template's channel norms were `0.887:1.000:0.930` for R:G:B,
|
|
including `0.903:1.000:0.916` above FFT radius 4.5. An SVD assigned 79.80% of
|
|
template energy to a nearly equal-channel component, 18.39% to a
|
|
green-vs-magenta component, and 1.81% to a red-vs-blue component. The common
|
|
component had strong diagonal energy, while 98.6% of the coarse
|
|
green-opponent orientation energy was axis-aligned. This measured carrier does
|
|
not support a fixed `0.85:1.00:0.70` channel rule or an exclusively diagonal
|
|
decoder.
|
|
|
|
Nonlinear cube-root LMS coordinates from the OKLab transform exposed the
|
|
signal most strongly in the direct color-fold comparison, but the
|
|
development-selected full-vector candidate retained 32 of 50 final positives
|
|
and accepted six of 299 controls. This is evidence that nonlinear color
|
|
projections can improve carrier SNR, not evidence that the deployed embedder
|
|
uses Lab or OKLab internally.
|
|
|
|
A three-level DTCWT probe retained all six orientations rather than only the
|
|
`+/-45` degree pair. Its development-selected near-vertical green energy
|
|
candidate retained 37 of 50 final positives and accepted eight controls. The
|
|
green-opponent near-vertical candidate retained 39 positives and accepted one
|
|
control. Separating energy, normalized complex correlation, and phase-only
|
|
coherence changed the development choice to all-orientation green-opponent
|
|
phase coherence, which retained 37 positives and accepted four controls. The
|
|
directional representation therefore exposes real carrier structure but fails
|
|
the low-false-positive gate.
|
|
|
|
An explicit FFT probe then froze 8, 16, 32, or 64 carrier bins from the fit
|
|
positives and compared signed cosine phase coherence with the proposed
|
|
`abs(cos(delta_phase)) > 0.90` rule. The absolute rule never produced a usable
|
|
operating point; treating phase and antiphase as equivalent increased the null
|
|
overlap. With 64 bins, the development-selected green-channel signed cosine
|
|
retained 26 of 50 final positives and initially accepted none of 299 controls.
|
|
That result was underpowered because only 39 of 499 controls selected the
|
|
eligible period-16 branch.
|
|
|
|
The green phase candidate and its threshold were frozen before a full
|
|
period-and-phase search over 1,000 COCO controls. It accepted five controls.
|
|
Among the 129 eligible controls, the maximum score was `0.6910271977636098`,
|
|
above the frozen `0.5443545641095117` threshold. Raising the threshold above
|
|
that maximum retained only eight of 100 positives. Explicit registered phase
|
|
lock is therefore a useful diagnostic of this carrier family but is rejected
|
|
as a standalone detector. Because it shares the RGB registration stage, it is
|
|
also not independent confirmation of the carrier.
|
|
|
|
### 2026-08-12: Green phase perturbation and blind SWT removal
|
|
|
|
A second removal probe tested the specific claim that changing only phase at
|
|
the recovered Green-channel carrier bins can preserve spectral magnitude and
|
|
image fidelity while defeating detection. It used the 100-image public Gemini
|
|
set at native period 16 and modified conjugate FFT pairs together so the
|
|
inverse remained real. Before pixel quantization, the maximum absolute change
|
|
in FFT magnitude was `5.24e-10` for the 32-pair inversion. After clipping and
|
|
rounding to 8-bit pixels, the maximum magnitude change relative to the largest
|
|
source coefficient was 0.175%.
|
|
|
|
The native periodic detector accepted 59 of the 100 sources. Inverting the 32
|
|
strongest unique Green-channel carrier pairs left one of those 59 positive.
|
|
Across all 100 images, PSNR was at least 43.36 dB and SSIM at least 0.99048;
|
|
the medians were 52.79 dB and 0.99739. Median signed phase coherence moved from
|
|
0.485 to -0.371 and the median full RGB tile score moved from 0.230 to 0.041.
|
|
This reproduces the proposed fidelity and local-score effect against the
|
|
recovered periodic detector.
|
|
|
|
It does not establish SynthID removal. The current test targets the same
|
|
carrier used by the local score, and earlier wider complex-spectrum and
|
|
phase-projection candidates already cleared local phase models at 51.94-56.03
|
|
dB while remaining detected in healthy Google verifier sessions. The 32-pair
|
|
Green candidate is therefore a preregistered oracle candidate, not a remover.
|
|
The official SynthID-Image description also identifies a post-hoc neural,
|
|
content-dependent encoder-decoder rather than a fixed Fourier codebook; it
|
|
does not document phase bins as its decision surface.
|
|
|
|
The preregistered Google-oracle challenge then used three of the strongest
|
|
local positives. Each fixed group contained the original, its 32-pair Green
|
|
phase inversion, and an RMS-matched random-noise sham. All three phase
|
|
candidates fell below the local `0.17357069773071196` threshold, with PSNR
|
|
from 55.41 to 57.86 dB and SSIM from 0.99709 to 0.99807. The three shams
|
|
remained locally positive. Nine files were submitted to Gemini's `Verify with
|
|
SynthID` tool in separate chats with the unchanged prompt. Eight verdicts
|
|
persisted and all eight reported Google AI signals. This includes all three
|
|
phase candidates, both recoverable source controls, and all three shams. The
|
|
second source request was submitted, but its chat did not persist and no
|
|
verdict could be recovered; it was not retried outside the fixed upload budget.
|
|
|
|
This falsifies the operative removal claim: Green-carrier phase inversion
|
|
clears the local periodic correlate on three of three images but clears the
|
|
Google oracle on zero of three. It is evidence that the local template observes
|
|
one stable component or correlate of SynthID, not that these FFT coefficients
|
|
form the proprietary decoder's sufficient decision surface. Do not productize
|
|
the Green phase inversion or report its local score change as SynthID removal.
|
|
|
|
The three oracle-confirmed phase candidates were then treated as adversarial
|
|
hard positives for detector analysis, without using them to relax the existing
|
|
runtime thresholds. Both runtime modes missed all three. Their native scores
|
|
were 0.126-0.160 against the 0.174 threshold; their registered decision scores
|
|
were 0, 0, and 0.553 against the threshold of 1. A phase-invariant magnitude
|
|
profile was therefore tested as a possible rescue. The development-selected
|
|
all-bin RGB profile retained 39 of 50 final positives but accepted one of 299
|
|
final Open Images controls and still missed all three hard positives.
|
|
|
|
A frozen-template, top-32 Blue-magnitude profile did accept all three hard
|
|
positives and initially accepted none of 499 Open Images controls, but this was
|
|
a post-hoc hard-positive-selected branch with low ordinary sensitivity: 17 of
|
|
70 development and final positives. Its frozen threshold accepted 39 of 1,000
|
|
fresh COCO controls. Combining it with a native-score floor of 0.10 happened to
|
|
accept none of all 3,000 COCO controls while retaining the three hard positives,
|
|
but failed the next independent challenge: 90 of 11,506 metadata-labeled
|
|
non-Google controls passed both gates. Accepted controls included Adobe
|
|
Firefly, local diffusion, Samsung, Microsoft, and several other generator
|
|
families. The phase-invariant magnitude rescue is therefore rejected as a
|
|
provider-specific detector; it measures a shared periodic rendering family.
|
|
|
|
Channel separation exposed a narrower attack-specific rescue. The Green-only
|
|
phase inversion leaves Red and Blue pixels unchanged, but the runtime RGB score
|
|
normalizes all three channels jointly. On the three oracle-confirmed hard
|
|
positives, separate signed correlations remained 0.781-0.804 in Red and
|
|
0.798-0.825 in Blue while Green moved to -0.400 through -0.645. A Red threshold
|
|
of `0.3770237083693908`, frozen above 200 Open Images calibration controls,
|
|
retained 16 of 70 development and final ordinary positives and accepted none
|
|
of the remaining 299 Open Images controls. It also accepted all three oracle
|
|
hard positives and none of 3,000 fresh COCO controls.
|
|
|
|
Red correlation alone remained a generic periodic-renderer correlate: it
|
|
accepted 96 of 11,506 metadata-labeled non-Google controls. Every one of those
|
|
96 had positive Green correlation. The attack-specific conjunction of Red at
|
|
or above the frozen threshold and negative Green accepted none of 15,005 total
|
|
controls across three independent corpora, while retaining all three
|
|
oracle-confirmed phase candidates. Across the 100-image synthetic inversion
|
|
probe, it recovered 19 of the 58 locally detected sources that the attack had
|
|
moved below the RGB threshold. One additional attacked image still passed the
|
|
existing detector.
|
|
|
|
This channel-discordance rule is the first zero-control rescue for the
|
|
oracle-confirmed hard positives, but it is not added to runtime. Its form was
|
|
motivated after inspecting the attack, only three examples carry external
|
|
oracle labels, and its synthetic rescue sensitivity is 19 of 58. A valid next
|
|
gate must preregister this unchanged rule, obtain new oracle-confirmed
|
|
Green-perturbed positives, and challenge a newly collected control set plus
|
|
resize, crop, and codec views. Until then it is an attack diagnostic, not a
|
|
universal SynthID detector.
|
|
|
|
A blind three-level `db2` SWT probe then soft-thresholded only the Green
|
|
horizontal, vertical, or diagonal detail bands at 0.25, 0.5, 1, and 2 times the
|
|
band's robust noise estimate. Diagonal HH thresholding left all 59 original
|
|
positives detected at every strength. Vertical thresholding cleared four at
|
|
strength 1 and five at strength 2, but the respective minimum fidelity fell to
|
|
32.57 dB and 0.96971 SSIM, then 28.52 dB and 0.93646. At strength 1, vertical
|
|
score reduction exceeded diagonal reduction on 97 of 100 paired images
|
|
(`p = 2.63e-25`, exact two-sided sign test). Blind HH soft-thresholding is
|
|
therefore rejected; the stronger vertical response is consistent with the
|
|
separately measured axis-aligned green-opponent carrier component.
|
|
|
|
### 2026-08-10: OpenAI periodic-carrier challenge
|
|
|
|
The OpenAI track repeated the Google carrier method without runtime provenance
|
|
features. C2PA selected the research labels, but every candidate score consumed
|
|
only decoded RGB pixels. The temporally split corpus contained 4,615 OpenAI
|
|
training positives, 1,280 validation positives, 1,364 development-test
|
|
positives, 8,668 difficult external-source controls, and 3,000 COCO controls.
|
|
The development test is not called a locked holdout because earlier
|
|
representations had already exposed it.
|
|
|
|
High-pass residuals were folded modulo 8x8, 16x16, and 32x32 and normalized as
|
|
in the Google fixed-tile expert. A single 16x16 OpenAI template accepted only
|
|
one of 1,280 validation positives at the threshold above all 2,287 validation
|
|
controls. A 16-template geometry bank improved that to 12 positives but
|
|
accepted one of 3,372 development-test controls. The corresponding
|
|
development-test sensitivity was 9 of 1,364. The frozen Google template did
|
|
not transfer: at its unchanged threshold it accepted 14 validation positives
|
|
and 572 validation controls.
|
|
|
|
The apparent native-size signal was a confound. At 1024x1536, 91.6% of the
|
|
learned template energy projected onto a 2x2 periodic lattice, compared with
|
|
5.6% for the frozen Google template. Firefly, China AIGC-labeled, local
|
|
diffusion, and unidentified hosted-generator controls reached correlations of
|
|
0.92-0.98 against the uncorrected OpenAI template. This is operational evidence
|
|
for a shared decoder or upscaler lattice, not an OpenAI watermark carrier.
|
|
|
|
Projecting the 2x2 nuisance lattice out of every 16x16 fold isolated a weaker
|
|
epoch-specific residual. The 1024x1536 expert accepted 5 of 55 validation and
|
|
10 of 51 development-test positives while accepting none of the full
|
|
validation or development-test controls. Nine of the ten development-test
|
|
matches occurred from July 12 through July 14; later dates contributed only
|
|
one of 27. The 1536x1024 expert accepted 6 of 37 validation and 2 of 51
|
|
development-test positives with no development-test controls. Amplitude,
|
|
active-support, channel-consensus, payload-invariant power, and spherical
|
|
multi-template branches did not improve this frontier without false positives.
|
|
|
|
The OpenAI branch therefore remains research-only and is not added to the
|
|
runtime detector. The next valid training epoch requires same-content
|
|
OpenAI-oracle `detected` and `not_detected` pairs, or provider-authorized decoder
|
|
labels, followed by a fresh temporal holdout. Source provenance alone cannot
|
|
distinguish a true watermark carrier from the measured generator lattice.
|
|
|
|
### 2026-08-10: OpenAI content-dependent decoder challenge
|
|
|
|
The periodic result did not exhaust the neural-watermark hypothesis. OpenAI's
|
|
ChatGPT Images 2.0 system card describes its watermark as content-specific, and
|
|
the SynthID-Image paper says the encoder is content-dependent by design. The
|
|
paper also separates watermark detection from payload recovery and evaluates
|
|
the external SynthID-O model at its preferred 512x512 input size. These facts
|
|
support a learned full-field decoder, but they do not establish that OpenAI and
|
|
Google deploy identical model weights, payloads, or pixel spectra.
|
|
|
|
A second corpus audit found 261 same-provider candidate controls. Their signed
|
|
provenance contained `created` and `converted` actions without the
|
|
`c2pa.watermarked.unbound` action, but a subsequent software-agent audit showed
|
|
that they do not all come from the positive generation epoch. Only 118 name
|
|
`gpt-image 2.0`: 97 training, 16 validation, and five development-test images.
|
|
Another 108 name an earlier version and 35 do not expose a usable version. The
|
|
full 261-image split remains 217/26/18. These are useful confound controls, but
|
|
even the exact-version subset is action-derived rather than a provider-oracle
|
|
negative label.
|
|
|
|
Two pixel-only native-resolution experts were trained with the mixed-version
|
|
control set. A
|
|
local residual CNN accepted 132 of 1,280 validation positives and 149 of 1,364
|
|
development-test positives, with zero same-provider or COCO controls in both
|
|
splits and two of 1,373 difficult development-test controls. A separate
|
|
spectral, autocorrelation, bit-plane, and residual-error expert accepted 175
|
|
validation and 167 development-test positives with no accepted controls. A
|
|
validation-selected linear combination accepted 243 positives in each split,
|
|
with no accepted validation or development-test control. The zero-error
|
|
development-test bound across 3,391 controls is 0.0883% at one-sided 95%
|
|
confidence, but the same-provider subset contains only 18 images and therefore
|
|
does not independently establish provider specificity.
|
|
|
|
The exact-version audit did not invalidate the native score counts, but it
|
|
weakened their attribution. None of the 16 validation or five development-test
|
|
`gpt-image 2.0` controls crossed the residual, forensic, ensemble, canonical,
|
|
or JPEG-equalized thresholds. That is directionally useful, but five temporal
|
|
test controls cannot establish deployment specificity. Perceptual-hash and
|
|
local CLIP searches compared all 118 exact-version controls with 7,259
|
|
positives. They recovered a few near-content matches but no clean before/after
|
|
watermark pair. Sub-pixel affine alignment left content, edge, and photometric
|
|
changes in the residual, while 8x8, 16x16, and 32x32 folded residual
|
|
correlations varied in sign and magnitude. Those candidates cannot be used as
|
|
causal watermark differences.
|
|
|
|
Transformation challenges rejected the combined native expert as a SynthID
|
|
decoder. Of its 243 accepted development-test positives, 121 remained accepted
|
|
after a 5% crop round trip, six after a 75% resize round trip, and none after a
|
|
JPEG-95 round trip. No transformed same-provider control was accepted. The
|
|
codec fragility is inconsistent with treating the score as the documented
|
|
robust watermark without further evidence.
|
|
|
|
The 512x512 branch also failed the held-out specificity gate. Its selected
|
|
model accepted 238 validation and 194 development-test positives, but also
|
|
accepted two of 18 same-provider, four of 1,373 difficult, and nine of 2,000
|
|
COCO development-test controls. Finally, a CNN trained and evaluated only
|
|
after applying the same JPEG-95 round trip to every positive and control
|
|
accepted eight validation positives and six development-test positives while
|
|
accepting five of 2,000 development-test COCO controls. Equalizing the codec
|
|
therefore removed nearly all of the apparent signal instead of revealing a
|
|
robust hidden decoder feature.
|
|
|
|
An independent three-CNN OpenAI surrogate was also evaluated from its
|
|
published weights, without adopting its code or claims. At the published 0.5
|
|
threshold its ensemble marked 82.0% of COCO validation controls and 83.4% of
|
|
COCO development-test controls as watermarked. It also marked 56.3% of
|
|
difficult development-test controls and 15 of 16 exact-version OpenAI
|
|
validation controls, compared with 87.3% of OpenAI validation positives. The
|
|
repository does not publish its training corpus or logs, and its implementation
|
|
applies supposedly shared pair augmentation independently. This result rejects
|
|
the model as a SynthID decoder and demonstrates why same-content-looking
|
|
regenerations are not sufficient negative labels.
|
|
|
|
A final transform-augmented full-field branch tested the closest practical
|
|
approximation to the documented neural decoder. A 512x512 CNN consumed both
|
|
RGB and a fixed high-pass residual and trained with random crop/resize,
|
|
JPEG 70-100, horizontal flip, and mild color transforms. Training sampling
|
|
balanced watermarked positives, difficult controls, and the exact-version
|
|
OpenAI candidate controls. Selection required a score above every control for
|
|
both native and JPEG-95 validation views. All four fixed training epochs
|
|
accepted zero of 500 selection positives. On the full validation set the
|
|
frozen model accepted zero of 1,280 OpenAI positives in native, JPEG-95,
|
|
75%-resize, and 5%-crop views, while accepting up to two unrelated controls.
|
|
On development test it accepted zero of 1,364 OpenAI positives in native,
|
|
JPEG-95, and resize views. The crop view accepted one OpenAI positive and one
|
|
of 1,373 difficult controls. Transform augmentation therefore did not recover
|
|
a usable weaker OpenAI signal.
|
|
|
|
The measured OpenAI-native separation is best classified as an export or
|
|
rendering noiseprint, not a validated SynthID detector. None of these models is
|
|
added to runtime. Transform-augmented training has now also failed. A
|
|
defensible next epoch needs provider-authorized labels or clean same-content
|
|
before/after watermark pairs, plus a fresh temporal holdout that has not
|
|
influenced representation or threshold selection.
|
|
|
|
## Decision record
|
|
|
|
The program has four possible honest outcomes per provider:
|
|
|
|
| Outcome | Product consequence |
|
|
| --- | --- |
|
|
| Causal signal and detector both generalize | Continue to pixel-only removal |
|
|
| Detector works but causal attribution fails | Ship no SynthID detector claim; retain as provenance research |
|
|
| Detector generalizes but pixel-only removal does not transfer | Keep local detection, retain regeneration fallback |
|
|
| Pixel-only removal clears the oracle with quality gates | Productize provider-specific detector and remover |
|
|
|
|
Stopping at a failed gate is a result. It prevents a local surrogate, export
|
|
fingerprint, or quality metric from being mistaken for control over SynthID.
|
|
|
|
## Immediate first milestone
|
|
|
|
The first milestone produces no shipping code. Harness code already exists for
|
|
items 1 and 5; milestone delivery means a completed evidence-bearing artifact,
|
|
not merely an available script. It delivers:
|
|
|
|
1. the private-corpus schema and auditor;
|
|
2. an OpenAI authorization decision for use of the remote provenance verifier;
|
|
3. an independently verified status for candidate causal pairs;
|
|
4. a canonicalized OpenAI pilot set with hard negatives;
|
|
5. the D1 confound report;
|
|
6. the D2 low-texture carrier report with leave-one-group-out results;
|
|
7. a go or no-go decision for real-image detector training.
|
|
|
|
Only after this milestone should implementation of D3 and D4 begin.
|
|
|
|
## Primary sources
|
|
|
|
- OpenAI, [Content provenance](https://developers.openai.com/api/docs/guides/content-provenance).
|
|
- OpenAI, [ChatGPT Images 2.0 system card](https://deploymentsafety.openai.com/chatgpt-images-2-0/automated-evaluations-and-adversarial-testing).
|
|
- Google, [Verify AI-generated images, videos, and audio](https://support.google.com/gemini/answer/16722517?hl=en).
|
|
- Gowal et al., [SynthID-Image: Image watermarking at internet scale](https://arxiv.org/abs/2510.09263).
|