mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-14 21:40:47 +02:00
Add frozen photo-classifier research archive to docs
Record the AI-versus-camera freeze, sister SynthID research pages, and that identify stays provenance-only.
This commit is contained in:
6 files changed
+2725
-1
No files matched your search
@@ -39,7 +39,7 @@ Before a release, read [`docs/release-and-distribution.md`](docs/release-and-dis
|
||||
|
||||
[`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem.
|
||||
|
||||
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md).
|
||||
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md).
|
||||
|
||||
## Data safety
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -35,6 +35,10 @@ The current behavior is defined by the code, tests, README, and user guides.
|
||||
- [ControlNet removal research](controlnet-removal-pipeline-research.md)
|
||||
- [Qwen improvement research](qwen-improvement-research.md)
|
||||
- [Doubao reverse-alpha research](research-doubao-distillation.md)
|
||||
- [AI-generated image classifiers](ai-generated-image-classifiers.md) (photo AI-versus-camera freeze, not a product verdict)
|
||||
- [SynthID source classifiers](synthid-classifiers.md)
|
||||
- [SynthID local detector research](synthid-detector-research.md)
|
||||
- [SynthID mark removal research](synthid-removal-research.md)
|
||||
- [SynthID identity research](synthid-robust-identity-research.md)
|
||||
- [SynthID identity follow-up](synthid-robust-identity-research-2026-06-08.md)
|
||||
- [Video SynthID quality research](video-synthid-quality-research.md)
|
||||
|
||||
@@ -0,0 +1,518 @@
|
||||
# SynthID source classifiers (research)
|
||||
|
||||
> Research archive for metadata-free OpenAI/Gemini source finding and
|
||||
> provider-lineage classifiers. These are not SynthID payload decoders and are
|
||||
> not shipped product verdicts. Current behavior:
|
||||
> [supported signals](supported-signals.md) and
|
||||
> [known limitations](known-limitations.md).
|
||||
>
|
||||
> Sister pages: [general AI-generated image classifiers](ai-generated-image-classifiers.md),
|
||||
> [SynthID local detector](synthid-detector-research.md),
|
||||
> [SynthID mark removal](synthid-removal-research.md), and
|
||||
> [mechanism reference](synthid.md).
|
||||
|
||||
A source classifier is reliable only when photographs and non-target generators
|
||||
are explicit negatives, a strict rule can abstain, and any watermark claim uses
|
||||
an independent oracle. CLIP content embeddings and the 124-d origin-locked
|
||||
residual bank are different features for different jobs.
|
||||
|
||||
Krawetz's Gemini-chat TPR critique is a verifier-quality claim, not a
|
||||
feature we can ship. [Lead Stories, 2026-07](https://leadstories.com/analysis/2026/07/google-gemini-synthid-detector-confuses-results-within-same-chat.html)
|
||||
documented Gemini repeating the first file's SynthID verdict inside a
|
||||
chat; Google said that was fixed 2026-07-16. The OpenAI provenance API is
|
||||
a different endpoint.
|
||||
|
||||
## Provider names from pixels
|
||||
|
||||
The task is: given a file with no metadata, is this OpenAI, Gemini, or
|
||||
unknown, with almost no errors on camera photographs or other generators. That
|
||||
is this section. `unknown` does not mean `not AI`; it includes AI images from
|
||||
other providers and target-provider images the strict rule misses. This is not
|
||||
a general AI-generation detector or a SynthID detector. Firefly, PixelBin, and
|
||||
other generators have to sit in the test, because a head that only sees OpenAI
|
||||
versus Gemini versus COCO will call Firefly a provider.
|
||||
|
||||
Three-way `openai` / `google` / `other` on Model 1 embeddings fails the
|
||||
Firefly gate. CLIP-L-ft test accuracy 0.53; Firefly 35/31/18. CLIP-H 0.57;
|
||||
Firefly 36/33/15. OpenAI versus Gemini AUC on CLIP-L-ft is 0.845; on the
|
||||
124-d lattice bank it is 0.989. They are two pipelines, not one class.
|
||||
|
||||
### Renderer, not front-end (cross-carrier validation, 2026-08-26)
|
||||
|
||||
A local C2PA-labeled collection's issuer field allows a provider-label audit the
|
||||
family labels cannot express: Bing Image Creator rows signed `Microsoft,
|
||||
OpenAI` (renderer DALL-E, n=152), Microsoft-native rows (MAI-Image / Designer,
|
||||
n=85), Designer rows signed `Microsoft, Google LLC` (renderer Imagen, n=36),
|
||||
and the unattributed Instagram `made_with_ai` label set (n=275). Scored with
|
||||
the frozen 124-feature openai/google/no_ai cascade, no training, metadata
|
||||
used only to build cells: argmax called 124/152 (81.6%) of the Bing+DALL-E
|
||||
cell `openai` and 34/36 (94.4%) of the Microsoft+Imagen cell `google`. Pixel
|
||||
attribution therefore tracks the renderer behind the front-end, which is the
|
||||
correct semantic for a stripped file: a Bing export is an OpenAI-rendered
|
||||
image. The Microsoft-native cell split 48 openai / 35 google, consistent with
|
||||
Designer historically routing between DALL-E and Imagen rather than one
|
||||
pipeline. The Instagram `made_with_ai` set is not a Meta pixel class: argmax
|
||||
leaned `google` at 196/275 with 68 `openai`, and the photo-first 0.50 margin
|
||||
sent 182/275 to `no_ai`, so the label marks detected AI content of mixed
|
||||
origin, not Meta-rendered pixels. A real Meta class needs fresh
|
||||
`Imagined with AI` generations with known provenance; the catalog label is
|
||||
not one. Artifacts:
|
||||
`provider-renderer-cells-2026-08-26/report.json`.
|
||||
|
||||
### A Meta class from muse-image-1.0, 2026-08-26
|
||||
|
||||
The Meta Model API (`api.meta.ai/v1`, OpenAI-compatible images endpoint) made
|
||||
a known-provenance Meta corpus possible without any account browser session:
|
||||
61 images generated from a 61-prompt grid spanning portraits, product shots,
|
||||
scenes, food, animals, architecture, illustration styles, text posters, and
|
||||
abstract work across all three aspect ratios, plus the five oracle-verified
|
||||
oracle-verified content-seal samples already on disk. All 66 are content-hash
|
||||
unique, delivered at 1600x1600 / 1920x1280 / 1280x1920, and every API row
|
||||
carries IPTC `trainedAlgorithmicMedia` plus a Content Seal generation id
|
||||
recorded in the manifest. A four-head control with a `microsoft_native` class
|
||||
(83 issuer-verified rows) failed first: repeated-split argmax recall 0.38 mean
|
||||
with 0.16-0.56 spread, because Designer routes between renderers and the class
|
||||
is mixed. The same recipe with `meta_muse_image` in its place holds: argmax
|
||||
recall 0.86 mean (0.75-1.00) on just 66 images, openai 0.952 and google 0.956
|
||||
unharmed, and the frozen cross-carrier cells keep their renderer semantics
|
||||
(Bing+DALL-E 119/146 openai, Microsoft+Imagen 32/36 google) with only small
|
||||
meta leakage (14 and 3 argmax rows). The class label names what the corpus is:
|
||||
muse-image-1.0 API output, not the unverified assumption that the consumer
|
||||
Imagine feature renders identically. The full-data model calls 59/66 meta rows
|
||||
correctly, with seven leaking to google and none to openai. The photo-first
|
||||
0.50 margin does not transfer to a 66-image class (0.398 mean recall); argmax
|
||||
is the honest operating rule until the corpus grows. Muse Image is therefore
|
||||
a separable fourth provider class on pixels, unlike the Microsoft front-end.
|
||||
Artifacts: `meta-muse-corpus-2026-08-26/`, `four-head-provider-2026-08-26/`,
|
||||
`meta-provider-cells-2026-08-26/`.
|
||||
|
||||
A paired chat-vs-API check then closed the label question. Two prompts from
|
||||
the API grid (weathered fisherman portrait, white-sneaker product shot) were
|
||||
submitted to the consumer Meta AI chat imagine flow in the user's own logged-in
|
||||
browser session; the chat delivered 1280x1920 and 1920x1280 WebP files from the
|
||||
`t39.105495-1` CDN family, each carrying IPTC `trainedAlgorithmicMedia` in the
|
||||
downloaded bytes. Scored by the frozen API-trained four-head model, which had
|
||||
never seen a chat image: both chat rows landed on `meta_muse_image` under
|
||||
argmax (0.251 and 0.233) with score profiles matching their API twins almost
|
||||
exactly, and the margin rule agreed pairwise (fisherman meta at margin, sneaker
|
||||
no_ai at margin 0.50, same as its API counterpart). The chat imagine pipeline
|
||||
and `muse-image-1.0` are therefore pixel-indistinguishable to the provider
|
||||
classifier on this pair, so `meta_muse_image` honestly names both
|
||||
distributions. Caveat: n=2 paired prompts from one session; this is a
|
||||
consistency result, not a deployment-scale equivalence claim. Artifact:
|
||||
`meta-chat-check-2026-08-26/`.
|
||||
|
||||
Collapsing OpenAI and Gemini into one pixel class versus other generators
|
||||
does not fix that. Binary ridge AUC 0.686, TPR 75% at FPR 45%. Canva 98%,
|
||||
Microsoft 75%, Firefly 68% leak into the union; FLUX HF hold stays out at
|
||||
3%. Training the same union only against photographs recreates Model 1 with
|
||||
a narrower train set (fresh FPR 2.1%, Firefly still 95%).
|
||||
|
||||
`provider-report-clip-l-ft.json`, `provider-union-report.json`.
|
||||
|
||||
### Strict abstention does not recover provider identity
|
||||
|
||||
An `unknown` class makes the claim narrower, but neither existing
|
||||
representation has a useful high-precision operating point. A
|
||||
leave-one-foreign-family-out ridge on the 124-d residual bank set each cut
|
||||
above every calibration negative. Mean OpenAI recall fell to 1.3% and mean
|
||||
Google recall to 4.4%, while a held-out foreign family was still attributed to
|
||||
a provider up to 5.0% of the time and photographs were called OpenAI up to
|
||||
0.52% of the time.
|
||||
|
||||
The CLIP-L-ft head has the same failure at a 0.1% calibration-negative cut.
|
||||
On 400 disjoint queries per provider, OpenAI recall was 1.0% and Google recall
|
||||
was 0.5%. It still called 1/11 held-out FLUX images OpenAI and 1/750 COCO
|
||||
photographs OpenAI. Zero calibration false positives reduced both provider
|
||||
recalls to zero. These are abstaining content classifiers, not precise source
|
||||
finders.
|
||||
|
||||
A nonlinear head and local-neighbor decision do not expose a hidden provider
|
||||
partition in the same embeddings. Across five balanced 768-128-3 MLP seeds,
|
||||
strict test recall ranged from 0.5-7.0% for OpenAI and 0.75-1.0% for Google,
|
||||
with 6-19 false attributions among 6,550 locked negatives. The best
|
||||
calibration-selected `k=3` local-neighbor rule reached 0.5% OpenAI and 3.25%
|
||||
Google recall and still called one TC260 image Google.
|
||||
|
||||
### Pixel finetuning learns the train split, not an exact provider boundary
|
||||
|
||||
The last two CLIP-L vision blocks were then finetuned directly for
|
||||
`openai` / `google` / `unknown`: 9,063 fit images, 3,537 disjoint calibration
|
||||
images, 400 balanced steps, and random JPEG 40-95, 85-100% crop, and mild blur.
|
||||
Each provider cut was placed above every calibration negative. Calibration
|
||||
recall was 4.3% OpenAI and 5.2% Google.
|
||||
|
||||
The time-disjoint locked result was 6/400 OpenAI and 10/400 Google. One Google
|
||||
image and one TC260 image were called OpenAI. All 500 unseen-AI controls and
|
||||
all 4,945 locked photographs stayed `unknown`, including 3,000 fresh Open
|
||||
Images, but that photo specificity does not repair an AI-source error. An
|
||||
oracle cut above both locked OpenAI errors leaves only 1/400 OpenAI; it is an
|
||||
upper bound, not a valid post-test threshold. The model is not shippable.
|
||||
|
||||
The independent high-frequency route is already closed at the tested
|
||||
capacity. A four-layer opponent-residual patch CNN reached AUC 0.44-0.56
|
||||
against foreign generators, reversed to 0.15 on a fresh era, and accepted
|
||||
95-100% of several held-out Firefly, Microsoft, fal.ai, and PixelBin families
|
||||
at its photo-median threshold. It learned AI rendering versus photography,
|
||||
not vendor identity.
|
||||
|
||||
### External surrogate and forensic-descriptor audit
|
||||
|
||||
The public
|
||||
[`newideas99/gpt-image-synthid-detector`](https://github.com/newideas99/gpt-image-synthid-detector/tree/5495e09)
|
||||
does not supply a causal SynthID contrast. Its negatives are lightly
|
||||
regenerated positive images, so the trained ResNet/EfficientNet ensemble can
|
||||
read the regeneration pipeline. On a blind 517-file local pilot, OpenAI versus
|
||||
all AUC was 0.630. At the repository's 0.5 cut it retained 92/100 OpenAI and
|
||||
accepted 307/417 negatives, including 104/120 Open Images, 26/30 COCO, 34/50
|
||||
Google, and 8/10 Firefly. A later exact repeat on the hash-disjoint v7 challenge
|
||||
retained 172/200 OpenAI but accepted 110/200 Google. It therefore fails source
|
||||
specificity before any photograph gate is considered. It is a visual-domain
|
||||
classifier, not an independent confirmation signal.
|
||||
|
||||
The current [`aloshdenny/reverse-SynthID`](https://github.com/aloshdenny/reverse-SynthID/tree/b110836)
|
||||
V4 codebook also adds no useful hybrid evidence. A pickle-free exact inference
|
||||
repeat on v7 accepted 77/200 Google and 76/200 OpenAI at its published 0.52
|
||||
threshold. Applied only to v11 `unknown` rows, that threshold would rescue 26
|
||||
Google files while misrouting two OpenAI files. The older V3 published cut
|
||||
would add two v11 Google misses, but it previously accepted 5/499 controls and
|
||||
6/1,000 fresh Open Images. A 1%-recall OR rule with that measured false-positive
|
||||
history is also rejected.
|
||||
|
||||
The public [`Ristellise/REGRET`](https://github.com/Ristellise/REGRET/tree/7d449034bf323987e7e608e7886e029ed20fd847)
|
||||
SPAM model is another forensic descriptor, not a decoder. The audited pickle
|
||||
contained only an sklearn pipeline, scaler, logistic regression, and numeric
|
||||
numpy globals; inference used an exact restricted allowlist. At the published
|
||||
0.5 cut it accepted 139/200 Google and 141/200 OpenAI. On the disjoint public
|
||||
extension, the same published cut also accepted 214/600 ImageNet photographs,
|
||||
14/75 BigGAN, 17/75 Midjourney, 52/75 SDXL, and 10/75 VQDM. It adds no safe v11
|
||||
rescue.
|
||||
|
||||
[`vordme2010/synthid-dataset`](https://github.com/vordme2010/synthid-dataset/tree/133a27088f6f4d695c79db9a1a70fa8e7fa3adad)
|
||||
publishes a useful flat-field corpus but an invalid open-world classifier
|
||||
contrast. Its Tier-1 matrix has 500 Gemini-flat positives and 1,500 synthetic,
|
||||
spectrum-matched, or phase-scrambled negatives, with no real negative. The
|
||||
33 features include noise scale and radial power as well as six hand-selected
|
||||
carrier bins. Rebuilding the repository's seed-42 RBF SVM from the safe numeric
|
||||
matrix, without loading joblib, accepted 1/200 current Google and 0/200 OpenAI
|
||||
on v7. The reported AUC above 0.999 measures the synthetic negative recipe and
|
||||
flat renderer epoch; it cannot confirm the current source finder or a SynthID
|
||||
payload.
|
||||
|
||||
[Forensic Self-Descriptions](https://github.com/ductai199x/Forensic-Self-Descriptions-CVPR25/tree/50f2eae)
|
||||
(CVPR 2025) is a genuinely different representation: constrained prediction
|
||||
residuals are summarized as a 960-d forensic descriptor. Its ready attribution
|
||||
head is not usable here. Three current OpenAI files were all called `Real`,
|
||||
while one Microsoft file was called `GPT-Image 1/1.5` at confidence 0.863.
|
||||
|
||||
A custom source head on a native 256-pixel center crop was more informative.
|
||||
PCA-64 plus logistic regression, with each provider cut above every calibration
|
||||
negative, gave OpenAI AUC 0.872 and 3/50 strict OpenAI recall with 0/388 test
|
||||
false attributions. Its Google head failed on Bytedance, Canva, FLUX,
|
||||
Microsoft, and an Open Images photograph. On a later unused-hash challenge the
|
||||
OpenAI FSD head alone accepted 14/300 OpenAI, 3/300 TC260, and 3/90 PixelBin.
|
||||
FSD is therefore an independent source cue, not a sufficient classifier and
|
||||
not a watermark statistic.
|
||||
|
||||
### Rejected narrow two-signal cascades, 2026-08-23
|
||||
|
||||
The visible Gemini sparkle is absent from the target generation and is not a
|
||||
feature or a gate. Two metadata-free intersections did survive a new challenge:
|
||||
|
||||
| Cascade | Development | Unused-hash challenge |
|
||||
| --- | --- | --- |
|
||||
| Google provider probability >= 0.8876249 **and** registered-v3 lattice >= 1.0 | 3/45 Google, 0/30 lattice-positive foreign files | 3/300 Google, 0/690 OpenAI/TC260/PixelBin |
|
||||
| OpenAI provider probability >= 0.8822824 **and** FSD probability >= 0.9912320 | The two cuts were frozen independently | 3/300 OpenAI, 0/690 Google/TC260/PixelBin, 0/500 photographs |
|
||||
|
||||
The Google result is a low-recall intersection of a content head and an
|
||||
origin-locked renderer lattice. A two-pixel crop destroys the lattice, so an
|
||||
indeterminate result says nothing about SynthID. It is evidence for a narrow
|
||||
Gemini/Imagen pipeline, not a public decoder.
|
||||
|
||||
All three OpenAI challenge positives were 1086x1448 RGB PNGs, the measured
|
||||
ChatGPT export geometry. Thirty known non-OpenAI files at exactly 1086x1448
|
||||
(TC260, Apple, Microsoft, Canva, Firefly, Google, and others) produced zero
|
||||
first-gate positives. A scan of 1,194 unique files at that geometry with no
|
||||
provider or platform label produced 29 pixel-gate candidates and four final
|
||||
hybrid candidates. Three had only PNG image-data chunks; the fourth carried
|
||||
generic DPI/orientation fields but no provider or provenance label. Visual
|
||||
inspection found four plausible AI generations. They remain candidates, not
|
||||
ground truth, until a held-out oracle or original source confirms them.
|
||||
|
||||
The full luma 16-32-pixel phase-ring head is not a third confirmation. It
|
||||
rejected all three known and all four unlabeled OpenAI hybrid candidates while
|
||||
accepting 3/30 known non-OpenAI same-geometry controls. Its global test AUC was
|
||||
0.600 for OpenAI and it stays closed.
|
||||
|
||||
These cascades established that independent intersections could suppress the
|
||||
observed false positives, but about 1% recall is not useful for the source-hunt
|
||||
task. They are rejected as operating detectors. The FSD implementation and
|
||||
weights are also research-only under CC BY-NC-SA 4.0.
|
||||
|
||||
Local artifacts: `lattice-pixel-hybrid-challenge.json`,
|
||||
`fsd-openai-challenge-report.json`,
|
||||
`unlabeled-1086x1448-openai-hybrid-candidates.json`. The research entry point
|
||||
is `.local-eval/synthid/ai-photo-2026-08-22/hybrid_source_finder.py`.
|
||||
|
||||
### Canonical forensic source finder, 2026-08-23
|
||||
|
||||
An overlooked pixel representation materially improves the strict result. It
|
||||
normalizes each file to 512x512, then extracts 1,794 metadata-free features
|
||||
from opponent-color residuals: patch power and variation, phase coherence,
|
||||
autocorrelation, low-bit correlations, and residual-error histograms. The
|
||||
OpenAI branch intersects its canonical forensic score with a weak provider
|
||||
pixel vote. The Gemini branch intersects a forensic/provider fusion with the
|
||||
registered-v3 lattice. The visible Gemini sparkle remains excluded.
|
||||
|
||||
The single-scale cuts were frozen before a third hash-disjoint challenge. The
|
||||
result was:
|
||||
|
||||
| Challenge cell | OpenAI candidate | Gemini candidate | Any candidate |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| OpenAI | 51/300 | 1/300 | 52/300 (17.3%) |
|
||||
| Google | 0/300 | 34/300 | 34/300 (11.3%) |
|
||||
| Open Images photographs | 1/1,000 | 0/1,000 | 1/1,000 (0.1%) |
|
||||
| TC260 | 0/7 | 0/7 | 0/7 |
|
||||
|
||||
For the requested union, 86/600 OpenAI-or-Google targets were found, 14.3%
|
||||
recall instead of the rejected cascades' 1%. One OpenAI file crossed through
|
||||
the Gemini branch, and one known photograph crossed through the OpenAI branch.
|
||||
The seven remaining unused TC260 files are too few to establish specificity,
|
||||
and no fresh PixelBin, Firefly, Microsoft, or other generator hashes remained
|
||||
outside the earlier model and challenge manifests.
|
||||
|
||||
This v3 result established a strict source-candidate finder, not an exact
|
||||
identifier or a SynthID decoder. It does not recover a payload, and an
|
||||
`unknown` result says nothing about whether SynthID is present.
|
||||
|
||||
Local artifacts: `source-finder-v3-selection.json`,
|
||||
`source-finder-v3-challenge.json`, `google-canonical-forensic.json`, and
|
||||
`google-forensic-pixel-fusion.json`. The same local research entry point now
|
||||
implements this rule without the FSD dependency.
|
||||
|
||||
### Multiscale strict finder, 2026-08-24
|
||||
|
||||
The 512x512 normalization was discarding usable scale-specific evidence. A
|
||||
frozen union now takes the strongest OpenAI forensic response over 512, 640,
|
||||
768, and native views, gated by the provider pixel head. The Google branch
|
||||
takes the strongest forensic/provider fusion over 384, 512, 640, 768, and
|
||||
native views, then requires the same registered-v3 lattice. Development on
|
||||
the second and third challenges found 539/1,200 targets and 0/1,657
|
||||
photograph/TC260 controls.
|
||||
|
||||
The views and cuts were written before opening a fourth challenge whose hashes
|
||||
were disjoint from every model manifest and the first three challenges:
|
||||
|
||||
| Challenge cell | OpenAI candidate | Gemini candidate | Any candidate |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| OpenAI | 92/300 | 11/300 | 103/300 (34.3%) |
|
||||
| Google | 0/300 | 173/300 | 173/300 (57.7%) |
|
||||
| Open Images photographs | 0/1,000 | 0/1,000 | 0/1,000 |
|
||||
| TC260 | 1/25 | 0/25 | 1/25 |
|
||||
|
||||
For the requested union, the blind result is 276/600, 46.0% recall, with
|
||||
1/1,025 non-target candidates. This is 3.2 times the single-scale v3 recall
|
||||
and 46 times the rejected 1% cascades. The one false candidate is TC260, not a
|
||||
camera photograph. Eleven OpenAI files crossed through the Gemini branch;
|
||||
that is a provider-attribution error but still a correct hit for the declared
|
||||
OpenAI-or-Google union.
|
||||
|
||||
This remains a source-candidate finder, not an exact identifier or a SynthID
|
||||
decoder. Fresh unused paths from the other generator families were not
|
||||
available for v4, so the 0.1% observed non-target rate is not an open-world
|
||||
precision claim. Robustness to crop, resize, re-encoding, and screenshot
|
||||
capture is also not established. Keep the models and paths in `.local-eval`;
|
||||
do not add a runtime or public CLI until a new temporal challenge with fresh
|
||||
foreign-generator families establishes positive precision.
|
||||
|
||||
Local artifacts: `source-finder-v4-selection.json`,
|
||||
`source-finder-v4-rule.json`, `source-finder-v4-challenge.json`, and
|
||||
`multiscale-forensic-development.json`. The local research entry point
|
||||
implements the frozen multiscale rule and still uses no metadata or visible
|
||||
sparkle.
|
||||
|
||||
A post-hoc OR over every per-view zero-development-error OpenAI cut is
|
||||
rejected. It raised v4 OpenAI recall to 153/300 but also accepted 6/1,000
|
||||
photographs and 3/25 TC260 controls. The apparent union of many individually
|
||||
strict cuts was multiple-testing overfit, not additional independent evidence.
|
||||
|
||||
### Original-export hybrids, 2026-08-24
|
||||
|
||||
Three more hash-disjoint challenges tested whether multiscale fusion could be
|
||||
made useful without metadata. The v5 ExtraTrees union improved exact provider
|
||||
recall to 363/600 (60.5%) and provider-union recall to 373/600 (62.2%), but it
|
||||
also accepted 8/1,000 photograph and foreign-generator controls. A revised
|
||||
Google confirmation removed those eight development errors. Adding an
|
||||
AI-versus-camera gate in v6 did not transfer: exact recall fell to 327/600
|
||||
(54.5%) and union recall was 341/600 (56.8%).
|
||||
|
||||
The specificity failures exposed a stronger but narrower signal. Current
|
||||
OpenAI exports in these sets are PNGs produced with adaptive scanline filters,
|
||||
while the earlier TC260 error was a PNG encoded with filter zero on every row.
|
||||
A strict PNG parser now requires a non-interlaced PNG with at least one adaptive
|
||||
filter before the OpenAI branch can emit a result. This reads the image container
|
||||
and pixels, not EXIF, C2PA, a filename, or a visible label. It also changes the
|
||||
claim: a re-encoded OpenAI JPEG must abstain.
|
||||
|
||||
The complete frozen v7 rule reached 215/400 exact provider matches (53.8%) and
|
||||
221/400 provider-union matches (55.3%) on a new challenge. Its cells were
|
||||
114/200 exact OpenAI and 101/200 exact Google. The PNG gate repaired the
|
||||
observed specificity problem, but the old OpenAI forensic head remained the
|
||||
recall bottleneck.
|
||||
|
||||
A subsequent v8 development hybrid trains an ExtraTrees OpenAI head on v4-v5
|
||||
multiscale forensic scores, pixel probabilities, and PNG encoding structure.
|
||||
Model selection used v6. The final 0.47 precision cut was chosen after v7 had
|
||||
been opened, so the following is a post-hoc development measurement, not
|
||||
another blind result:
|
||||
|
||||
| v7 cell under v8 development rule | OpenAI | Gemini | Unknown |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| OpenAI | 190/200 | 3/200 | 7/200 |
|
||||
| Google | 0/200 | 102/200 | 98/200 |
|
||||
|
||||
That is 292/400 exact provider matches (73.0%) and 295/400 provider-union
|
||||
matches (73.8%).
|
||||
|
||||
The remaining Google miss set contained two different export pipelines: PNG
|
||||
and JPEG. A second development branch parses only JPEG codestream parameters,
|
||||
including quantization tables, chroma sampling, and progressive encoding; it
|
||||
explicitly skips APP0-APP15 and COM segments. Training one Google model per
|
||||
encoding class on v4 and selecting zero-validation-error cuts on v5-v6 raised
|
||||
the v11 transfer result to:
|
||||
|
||||
| v7 cell under v11 development rule | OpenAI | Gemini | Unknown |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| OpenAI | 190/200 | 4/200 | 6/200 |
|
||||
| Google | 0/200 | 126/200 | 74/200 |
|
||||
|
||||
This is 316/400 exact provider matches (79.0%) and 320/400 provider-union
|
||||
matches (80.0%).
|
||||
|
||||
This is the best local source finder in the campaign, but it is still not a
|
||||
SynthID detector, payload decoder, or open-world precision proof. The v8 rule
|
||||
is post-hoc, and neither v8 nor v11 has an independent open-world negative
|
||||
proof. The OpenAI branch is intentionally scoped to original-style PNG exports.
|
||||
A new temporal blind challenge with new foreign generators and PNG
|
||||
camera/editor controls is required before a runtime or public CLI is justified.
|
||||
|
||||
Local artifacts: `source-finder-v7-selection.json`,
|
||||
`source-finder-v7-challenge.json`, `source-finder-v8-rule.json`,
|
||||
`source-finder-v8-openai-extra-trees.joblib`, and
|
||||
`source-finder-v11-google-per-codec.joblib`.
|
||||
|
||||
### Published few-shot attribution also fails the open-world gate
|
||||
|
||||
[OmniDFA](https://arxiv.org/abs/2509.25682) is a purpose-built few-shot source
|
||||
attributor rather than a generic content embedding. Its published `part1`
|
||||
checkpoint is the correct unseen-generator fold for DALL-E 2 and DALL-E 3:
|
||||
those generators are in `part1` validation and absent from its training list.
|
||||
The same checkpoint has seen Imagen, so its Google result is not a clean
|
||||
unseen-Imagen benchmark; the OpenAI result is sufficient to reject the shared
|
||||
runtime.
|
||||
|
||||
With 20 support images per provider and provider-specific similarity plus
|
||||
margin cuts calibrated to zero false attributions over 160 negatives, a
|
||||
content-hash-disjoint 745-image evaluation produced:
|
||||
|
||||
| Cell | Result |
|
||||
| --- | ---: |
|
||||
| OpenAI recall | 9/50 (18%) |
|
||||
| Google recall | 4/50 (8%) |
|
||||
| Microsoft called OpenAI | 3/15 (20%) |
|
||||
| Kodak called OpenAI | 3/24 (12.5%) |
|
||||
| Canva called Google | 1/15 (6.7%) |
|
||||
| fal.ai called OpenAI | 1/15 (6.7%) |
|
||||
| xAI called OpenAI | 1/15 (6.7%) |
|
||||
| unseen Higgsfield called OpenAI | 1/11 (9.1%) |
|
||||
| fresh Open Images / COCO false attributions | 0/100 / 0/50 |
|
||||
|
||||
Provider multimodality is not the missing fix. Choosing 1-10 spherical
|
||||
prototypes only by calibration recall selected five: test recall fell to 16%
|
||||
OpenAI and 4% Google, while false attributions remained on Firefly (2/15),
|
||||
Kodak (2/24), Microsoft, ByteDance, TC260, and Made-with-AI samples.
|
||||
|
||||
Native files already fail, so JPEG, resize, crop, and screenshot variants were
|
||||
not run for OmniDFA. Do not add a provider-attribution runtime or CLI from that
|
||||
model. General exact OpenAI/Gemini identification remains unsupported. The
|
||||
strict source finder above emits candidates; it does not read the SynthID
|
||||
payload.
|
||||
|
||||
### 124-d lattice as pipeline ID, not a vendor CLIP head
|
||||
|
||||
Provider-class ridge on 124 native residual features (70/30 once, not a
|
||||
watermark gate). OpenAI L1 n=285, Google corpus n=533, foreign n=218, COCO
|
||||
n=289: OpenAI vs COCO 0.965; Google vs COCO 0.999; OpenAI vs Google 0.989;
|
||||
OpenAI vs foreign 0.725; Google vs foreign 0.922. OpenAI vs Firefly-class
|
||||
is the weak cell.
|
||||
|
||||
One-vs-rest: Google head at TPR 90% has FPR 0% vs COCO, 14% vs foreign, 2%
|
||||
vs OpenAI. A Gemini-like pixel class is close to what `pipeline_lattice`
|
||||
already is. An OpenAI-like pixel class on this bank would label Firefly as
|
||||
OpenAI about half the time and is not shippable.
|
||||
|
||||
Three-class `openai` / `google` / `no_ai` on 2,000 catalog OpenAI, 2,000
|
||||
catalog Google, and 1,936 COCO photos. Photo-first margin 0.50: openai
|
||||
74.7%, google 78.9%, no_ai 99.8%; Kodak 24/24 `no_ai`. Other generators
|
||||
are leakage, not classes:
|
||||
|
||||
| Platform | n | openai | google | no_ai |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Firefly | 106 | 37 | 27 | 42 |
|
||||
| Microsoft | 117 | 39 | 22 | 56 |
|
||||
| PixelBin | 90 | 14 | 46 | 30 |
|
||||
| HuggingFace job | 82 | 3 | 62 | 17 |
|
||||
| ByteDance C2PA | 86 | 5 | 35 | 46 |
|
||||
| SD / Comfy | 120 | 30 | 11 | 79 |
|
||||
| fal.ai | 98 | 11 | 18 | 69 |
|
||||
| Made-with-AI tag | 115 | 4 | 32 | 79 |
|
||||
| TC260 | 118 | 1 | 13 | 104 |
|
||||
| xAI | 114 | 1 | 20 | 93 |
|
||||
| Canva | 76 | 11 | 3 | 62 |
|
||||
| Apple Clean Up | 114 | 1 | 23 | 90 |
|
||||
| Aweme | 37 | 2 | 7 | 28 |
|
||||
| FLUX | 11 | 0 | 0 | 11 |
|
||||
| Reve | 10 | 0 | 0 | 10 |
|
||||
| NovelAI | 9 | 0 | 0 | 9 |
|
||||
| Higgsfield | 11 | 1 | 5 | 5 |
|
||||
|
||||
PixelBin and HuggingFace jobs lean `google` (shared renderer lineage).
|
||||
FLUX, NovelAI, and Reve stay `no_ai`. Local probe:
|
||||
`uv run python .local-eval/synthid/prc-oklab-attack-2026-08-15/classify_openai_gemini.py image.png`.
|
||||
|
||||
## Research lattice expert (google-lineage renderer)
|
||||
|
||||
Not a watermark and not in `identify`. `scripts/synthid_runtime/`
|
||||
`detect_synthid` re-check on 628 frozen holdouts, seed 20260822, threshold
|
||||
1.0.
|
||||
|
||||
| Family | n | detected | rate | max score |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Google / Gemini | 80 | 45 | 0.56 | 3.03 |
|
||||
| Firefly | 84 | 15 | 0.18 | 3.00 |
|
||||
| PixelBin | 80 | 11 | 0.14 | 2.48 |
|
||||
| Microsoft | 60 | 2 | 0.03 | 2.40 |
|
||||
| OpenAI | 80 | 1 | 0.01 | 1.38 |
|
||||
| xAI | 40 | 1 | 0.03 | 1.26 |
|
||||
| FLUX HF | 40 | 0 | 0 | 0.85 |
|
||||
| TC260 | 40 | 0 | 0 | 0.97 |
|
||||
| Kodak | 24 | 0 | 0 | 0.49 |
|
||||
| Open Images fresh | 60 | 0 | 0 | 0.64 |
|
||||
| COCO hold | 40 | 0 | 0 | 0.74 |
|
||||
|
||||
Firefly 18% and PixelBin 14% match the 2026-08-16 signed-foreign rates
|
||||
(24% and 14%) in order of magnitude. Both Microsoft hits have issuer
|
||||
`Microsoft, Google LLC`. A 2 px crop killed every sampled positive,
|
||||
including Firefly and PixelBin. Google TPR 56% is mixed collection eras, not
|
||||
the oracle-positive 147/148 cell. Honest name:
|
||||
`google_lineage_renderer` = Gemini/Imagen + Firefly + PixelBin.
|
||||
|
||||
Registered-v3 photographic controls remain 0/5,993 Open Images and 0/2,366
|
||||
COCO. Against 223 C2PA-named non-Google generators on 2026-08-16: 29
|
||||
accepted (0.130), Firefly 0.241. `.local-eval/synthid/lattice-check-2026-08-22/`.
|
||||
|
||||
## Local collection note (2026-08-21)
|
||||
|
||||
Unlabeled rows in the local C2PA-labeled collection are not photographs.
|
||||
Microsoft and Firefly rows also carry `synthid_from_provenance=true`, so
|
||||
that flag is not an OpenAI-plus-Gemini class.
|
||||
@@ -0,0 +1,591 @@
|
||||
# SynthID local detector research
|
||||
|
||||
> Research archive for the hunt for a local, keyless SynthID pixel detector.
|
||||
> Not a statement of current product capability. Shipped behavior:
|
||||
> [supported signals](supported-signals.md) and
|
||||
> [known limitations](known-limitations.md).
|
||||
>
|
||||
> Sister pages: [SynthID source classifiers](synthid-classifiers.md),
|
||||
> [SynthID mark removal](synthid-removal-research.md),
|
||||
> [mechanism reference](synthid.md). Dated measurements live in the
|
||||
> [chronological plan](synthid-detector-removal-plan.md).
|
||||
|
||||
## Result
|
||||
|
||||
The hunt for a local, keyless OpenAI SynthID pixel detector is open.
|
||||
Google has no public payload decoder, and nothing in this project
|
||||
reproduces one. Closed routes below failed to read the official oracle
|
||||
contrast on photographs. The remaining work is a new waveform, a new
|
||||
pair mint, or a new statistic, not a rerun of those routes.
|
||||
|
||||
The mark behaves like a keyed spread-spectrum residual: a content-adaptive
|
||||
`x' = x + g(x)` with a paired decoder (DeepMind patent family, optional
|
||||
secret in intermediate layers, encoder/decoder ensembles that do not
|
||||
recognize each other). Without that pair, the residual sits under the
|
||||
scene. Keyless energy, TrustMark transfer, L1 distillation, a flat-field
|
||||
stamp, and a 16-32 band student all failed to read the official oracle
|
||||
contrast on photographs.
|
||||
|
||||
S4, 2026-08-15: the two providers are not doing the same thing. Cross-image
|
||||
correlation of the folded residual is high for Google (tile16 pos-pos
|
||||
`+0.326`, chance `0.036`) and at chance for OpenAI (`+0.032`). Google
|
||||
shares one fixed phase-coherent pattern, also present in its controls at
|
||||
about half the amplitude. OpenAI shares nothing, which is what a
|
||||
content-dependent post-hoc encoder produces. Comb experts that work on
|
||||
Google therefore cannot be reused as an OpenAI watermark detector.
|
||||
|
||||
M2, 2026-08-15, re-verified 2026-08-16: Google unwatermarked pairs cannot
|
||||
be minted. `imagen-*` ids 404 from the model garden. `gemini-*-image`
|
||||
rejects `addWatermark` (`Cannot find field`). Current Gemini API docs
|
||||
state that all generated images include a SynthID watermark. There is no
|
||||
encoder-off Google path.
|
||||
|
||||
What the product uses for the *watermark* is signed provenance and
|
||||
`verify-openai-synthid`. The periodic-lattice expert is research-only under
|
||||
`scripts/synthid_runtime/` and is not called from `identify` or the CLI.
|
||||
Lineage measurements of that expert are in
|
||||
[SynthID source classifiers](synthid-classifiers.md).
|
||||
|
||||
## Public GitHub sweep, 2026-08-24
|
||||
|
||||
GitHub's primary-repository search returned 78 repositories with `synthid` in
|
||||
the name and 81 with it in the description, 133 unique repositories in total.
|
||||
The fork-inclusive searches were much larger and capped or dominated by copies
|
||||
(860 name hits and 1,311 description hits), so this is not a claim that every
|
||||
fork or unindexed private repository was read. The sweep also used targeted
|
||||
code searches for detector, verifier, correlation, codebook, bypass, and
|
||||
removal implementations. SynthID-Text, visible-corner-mark removers, metadata
|
||||
cleaners, wrappers, and literal ports were classified before inspecting the
|
||||
remaining image-pixel candidates at pinned commits.
|
||||
|
||||
The audits are not null results. They separate three claims that must not be
|
||||
collapsed: a repeatable image statistic exists, that statistic is a calibrated
|
||||
single-image SynthID detector, and changing its score transfers to the
|
||||
provider's production decoder. The repositories support parts of the first
|
||||
claim. None supports the second or third on the current challenge data.
|
||||
|
||||
| Repository | Retained contribution | Rejected use | Decisive result |
|
||||
| --- | --- | --- | --- |
|
||||
| [`Rinne414/SynthID-detector`](https://github.com/Rinne414/SynthID-detector/tree/b08abff06b67c37db9dfa7c6ed63760a446d560f) | A reproducible fixed-residual correlator and a useful spoofability negative control | Current-image detector or fixed-template remover | All three templates detected 0/400 current OpenAI and Google images. Adding the expected template spoofed the public detector, while subtracting 0.25-2x from an official positive left SynthID `detected` down to 25.72 dB |
|
||||
| [`newideas99/gpt-image-synthid-detector`](https://github.com/newideas99/gpt-image-synthid-detector/tree/5495e09c62dc26112f9a0e6cfab469f9a10f37e3) | Evidence that a CNN can learn a renderer/regeneration-domain signature, plus a white-box shortcut baseline | Payload decoder, independent hybrid confirmation, or removal oracle | The published cut retained 172/200 OpenAI but accepted 110/200 Google. A gradient drove the surrogate effectively to zero, but official SynthID stayed `detected` through +/-8/255 at 34.30 dB |
|
||||
| [`aloshdenny/reverse-SynthID`](https://github.com/aloshdenny/reverse-SynthID/tree/b11083676fd3ee3ff97ce9d03c0e409e46905902) | Cross-color phase consensus and further evidence for a repeating Google renderer-lineage pattern | The published V3/V4 thresholds as presence detectors, or Round-06 as a verified quiet remover | V4 accepted 77/200 Google and 76/200 OpenAI at the same cut. Round-06 has no published per-image oracle evidence and uses regeneration/distortion with 14/11 dB PSNR floors |
|
||||
| [`Ristellise/REGRET`](https://github.com/Ristellise/REGRET/tree/7d449034bf323987e7e608e7886e029ed20fd847) | A compact 147-d second-order SPAM descriptor and a safely auditable sklearn model | Google or OpenAI detector, or a high-precision hybrid rescue | The published cut accepted 139/200 Google, 141/200 OpenAI, and 214/600 disjoint ImageNet photographs |
|
||||
| [`vordme2010/synthid-dataset`](https://github.com/vordme2010/synthid-dataset/tree/133a27088f6f4d695c79db9a1a70fa8e7fa3adad) | A public 2026 Gemini flat-field corpus, safe numeric features, and explicit candidate carrier bins | Its reported near-perfect Tier-1 AUC as open-world SynthID evidence | Training used 500 Gemini flats against 1,500 synthetic or phase-scrambled negatives and zero real photographs. Rebuilding the SVM from the safe matrix accepted 1/200 Google and 0/200 OpenAI on v7 |
|
||||
|
||||
Useful forensic signal is therefore not synonymous with a production
|
||||
watermark decoder. The Google lattice idea is retained only as a
|
||||
research-lineage component; it cannot be labelled SynthID. The fixed template
|
||||
and CNN gradient are retained as negative controls. None of these public
|
||||
implementations is shipped as a detector or remover. Detailed classifier and
|
||||
attack measurements live in [SynthID source classifiers](synthid-classifiers.md) and
|
||||
[mark removal](synthid-removal-research.md).
|
||||
|
||||
### Google verifier surfaces found in the sweep
|
||||
|
||||
Two real but non-local verifier paths surfaced. The historical Vertex SDK
|
||||
exposed `WatermarkVerificationModel.from_pretrained("imageverification@001")`.
|
||||
The class still exists in `google-cloud-aiplatform` 1.165.1, but warns that it
|
||||
was deprecated 2025-06-24 with removal scheduled for 2026-06-24. Calls in two
|
||||
projects with Vertex enabled now reject the publisher model id as invalid. A
|
||||
repository using `response.watermark_detected` is also stale: the actual SDK
|
||||
property was `watermark_verification_result`, backed by the prediction's
|
||||
`decision` field.
|
||||
|
||||
A newer unlisted upload route,
|
||||
`POST https://synthid.googleapis.com/upload/v1:verifyContent`, is live enough
|
||||
to identify itself as the SynthID API and return `SERVICE_DISABLED` for a real
|
||||
public image. It has no public discovery document, did not appear in the
|
||||
project's available-services listing, and no authorized successful response
|
||||
was obtained. Enabling a cloud service was deliberately not attempted. This
|
||||
may be an access-gated research oracle, but it is not a local decoder and does
|
||||
not satisfy the metadata-free offline product requirement. Gemini chat remains
|
||||
the only Google verifier actually evidenced by the public bypass corpora.
|
||||
|
||||
## Closed detector routes
|
||||
|
||||
| Route | Close | Why |
|
||||
| --- | --- | --- |
|
||||
| Wavelets / FFT / cepstrum as a single-image detector | 2026-08-09 | TrustMark proxy: 318-d wavelet/spectral summary AUC 0.653 and 0 TPR at a clean calibration cut. Complex FFT maps AUC 0.516. Spatial RGB still won |
|
||||
| `aloshdenny/reverse-SynthID` V3 phase codebook | 2026-08-09/24 | Pickle-free numeric audit. The original pilot was 5/5 Google positives and 0/194 then-available negatives. The broader audit was 5/355 Google, 5/499 controls, and 6/1,000 fresh Open Images. On v7 it accepted 4/200 Google and 1/200 OpenAI. As a v11 `unknown` rescue it adds 2/200 Google on this test, but the historical control errors reject a 1%-recall OR rule |
|
||||
| `aloshdenny/reverse-SynthID` V4 | 2026-08-13/24 | Commit `b110836`. Better-of-two profiles: 141/355 Google positives and 191/499 controls. Frozen 1,000 Open Images: 386 accepted. Paired AUC 0.517. An exact repeat on the hash-disjoint v7 challenge accepted 77/200 Google and 76/200 OpenAI. Strict GUI aspect routing reduced support but not false acceptance. The threshold sits 0.02 above chance phase similarity |
|
||||
| `cebeuq/Synthid-Bypass` as a local decoder | 2026-08-13 | 12 public before/after diffusion pairs. Fixed carrier 10/12 before, 0/12 after. Gemini Verify still saw SynthID on two `before` files the carrier missed. Reconstruction removal, not a payload read |
|
||||
| S4: one content-independent pattern for both providers | 2026-08-15 | Google has a shared phase-coherent tile. OpenAI is at chance. See Result |
|
||||
| M2: mint Google encoder-off pairs | 2026-08-15 | No current Google path emits an unwatermarked image. See Result |
|
||||
| U1: one feature space for both providers | 2026-08-15 | Transfer 0.535 and 0.469 |
|
||||
| Hand statistics vs a perceptually masked keyed mark (M1b) | 2026-08-15 | At equal RMS, fixed 0.670, keyed 0.669, keyed+masked 0.546. Masking, not keying, defeats hand features |
|
||||
| Decoder-substitution / foreign VAE as a verified negative | 2026-08-17 | OpenAI oracle stayed `detected` at 22.3 dB. Attacking a positive does not mint a labelled negative |
|
||||
| `aoxo/reverse-synthid` as extra Google templates | 2026-08-19 | Hugging Face dump scored against the frozen era bank. Extra family means raised foreign scores instead of isolating a mark |
|
||||
| Origin-locked lattice as SynthID | 2026-08-16 | Two-pixel crop kills it; published SynthID keeps 99.97% TPR under aggressive crop. See classifiers for `google_lineage_renderer` |
|
||||
| Keyless energy in 16-32 px | 2026-08-20 | L1 AUC 0.53; official decoder reads phase structure, not energy |
|
||||
| TrustMark / open-encoder transfer | 2026-08-20 | E3 leave-one-encoder-out at chance on TrustMark (0.505 ridge, 0.510 conv) |
|
||||
| L1 distillation of oracle bits | 2026-08-20 | Geometry-only AUC 0.78-0.83 beats pixel 124-d (0.64). ChatGPT export and `opened` C2PA predict `not_detected`. Inside 1254x1254, permutation p=0.45 |
|
||||
| Flat 16-32 matched filter | 2026-08-21 | Leave-one-out residual correlation 0.645 on gray flats; 0.007-0.025 on COCO photo residuals. L1 AUC 0.70 was a size confound (size-matched 0.59 / 0.44) |
|
||||
| `gpt-image-1` as encoder-off pair | 2026-08-21 | Does not stamp. `gpt-image-2` does |
|
||||
| CNN on raw L1 bits | 2026-08-21 | Do not train. Labels are export geometry and presentation, not the mark |
|
||||
| 128-photo causal wavelet student on the flat template | 2026-08-21 | Photo 16-32 jam flips only at a=24 / 18.6 dB (`a*_photo < 0.53` failed). A student cannot beat a matched filter if the photo-domain mark is that quiet |
|
||||
| Photo 16-32 amplitude without the flat-arm template | 2026-08-23 | 11 COCO `gpt-image-2` edits plus the red-dot pair. Residual is the camera JPEG resized onto the edit raster, top-left 32 px dropped. Median 16-32 residual RMS 12.2, same order as 8-16 (9.0) and 32-64 (16.3). On the five tightest pairs (full residual RMS < 16): 16-32 median 2.69, still ~24x the 0.11 flat stamp and not louder than 8-16 (3.81). The photo residual does not isolate a 16-32 codeword |
|
||||
| Sort-by-luma 1D chroma stego | 2026-08-23 | Flat LOO 0.914 but a luma-order shuffle is already 0.792. COCO plus tiny noise: 0.642 vs shuffle 0.640. The sort invents the correlation |
|
||||
| Retinex / local-gain then 16-32 LOO | 2026-08-23 | 0.742 vs plain 16-32 LOO 0.752 on the same eight gray flats. No new carrier |
|
||||
| Distant 16-32 patch coupling | 2026-08-23 | Near lag 0.13 of a tile: -0.013. Far lag ~500 px: +0.054. No nonlocal lock |
|
||||
| Bispectrum of the 16-32 ring | 2026-08-23 | Degenerate: 1.0 on flats and 1.0 on COCO. Not a statistic |
|
||||
| Length-255 m-sequence vs 16x16 fold | 2026-08-23 | Max abs corr 0.059. Chance for n=255 is `1/sqrt(255) ≈ 0.063` |
|
||||
| Row-as-audio pitch peak | 2026-08-23 | Peak-to-mean 51 on flats, 46 on COCO noise residuals. Shared FFT peakiness, not a pitch |
|
||||
| 16x16 origin search of the flat 16-32 template on photos | 2026-08-23 | Zero-shift NCC 0.006-0.024, best of 256 shifts 0.013-0.026. Origin is not hiding a lock |
|
||||
| Affine search of the 16-32 residual vs the flat template | 2026-08-23 | 11 `gpt-image-2` photos: zero-deg median 0.009, best rotation in ±4° 0.020, lift 0.011. Six COCO noise residuals: lift 0.008. Flats already peak at 0°. No hidden rotation or 3% scale lock |
|
||||
| Fourier-angle scramble of 16-32 (true `(r, θ)` of the annulus) | 2026-08-23 | s1 24.1 dB official `not_detected`, cartesian replicate 24.8 dB `not_detected`. Decoder reads phase in that annulus, including angular organization. Not quieter than scramble |
|
||||
| Baker-map of the 16-32 band | 2026-08-23 | s1 27.8 dB still `detected` |
|
||||
| Poisson / photon-shot embedding | 2026-08-23 | Flat Anscombe LOO 0.7518 vs linear 0.7519. s1 Poisson 30.7 dB still `detected` |
|
||||
| Nested LSB plus SynthID | 2026-08-23 | s1 55.9 dB still `detected`. Extra open stego does not hide or replace the mark |
|
||||
| ICC sRGB rewrite | 2026-08-23 | No-op, PSNR inf. Mark is not in a profile that a pixel-preserving assign changes |
|
||||
| Seam carve 8 columns of 16-32 energy | 2026-08-23 | s1 27.7 dB still `detected` |
|
||||
| Extra OpenAI image ids as a new ensemble | 2026-08-23 | `gpt-image-1.5` and `gpt-image-1-mini` are encoder-off: C2PA `created, converted`, official `not_detected`, 16-32 corr vs the `gpt-image-2` gray stamp 0.003. Dated `gpt-image-2-2026-04-21` is the same family: C2PA `watermarked.unbound`, 16-32 corr 0.189. `chatgpt-image-latest` generate stays 403 until org verification |
|
||||
| DIRE reconstruction error as a payload | 2026-08-23 | SDXL float32, 512 px. VAE RMS: gray 1.24, s1 9.15, photo edit 9.89, camera 11.84. DDIM DIRE RMS: gray 2.40, photo 30.9, s1 31.5, camera 33.0. Rank follows texture. Float16 DDIM NaN'd on MPS. Not a payload |
|
||||
| HEIF / AV1 as a hidden carrier | 2026-08-23 | s1 HEIF q80 46.3 dB, q50 39.3 dB, AV1 CRF 32 37.2 dB, all official `detected`. Codec round-trip does not mint a new statistic |
|
||||
| Print-scan simulation as the kill | 2026-08-23 | s1 24.95 dB still `detected`. Same PSNR class as 16-32 scramble, which kills. Generic 25 dB is not the mark |
|
||||
| Mellin / log-polar \|F\| of the 16-32 residual | 2026-08-23 | Flat LOO 0.996-1.000, photo 0.996, COCO 0.991-0.995. The annulus envelope, not a payload. Scale chirps `r^{iα}`: flat 0.51, photo -0.27, COCO -0.58. No photo lock |
|
||||
| Zernike n≤6 and PCET of the residual | 2026-08-23 | Flat LOO 0.77 / 0.75, same order as cartesian 16-32. Photo -0.007 / 0.044. COCO 0.23 / 0.35, above photos. Moments of the flat stamp, not a photograph decoder |
|
||||
| Fresnel quadratic-phase of 16-32 (z=8,16,32,64) | 2026-08-23 | Flat LOO 0.750-0.753, photo 0.012, COCO -0.006. Unitary copy of the cartesian residual. No new shell |
|
||||
| 8-bin Fourier orientation energy | 2026-08-23 | 0.999 flat, 0.966 photo, 0.995 COCO. Saturates |
|
||||
| Chroma 16-32 as the payload | 2026-08-23 | Gray-flat Y LOO 0.682, Cb 0.064, Cr 0.025. s1 Y scramble 24.6 dB `not_detected`; Cb 45.0 dB and Cr 43.9 dB stay `detected`. Mark is in luma |
|
||||
| One 90° Fourier sector of 16-32 | 2026-08-23 | s1 27.5 dB still `detected`. Decoder needs the whole annulus, not one wedge |
|
||||
| Radial-phase-only scramble of 16-32 | 2026-08-23 | s1 25.0 dB `not_detected`. Phase as a function of radius, over the full ring, is enough to kill. Combined with Fourier-angle scramble, either polar coordinate of the annulus phase is a kill if the whole ring is hit |
|
||||
| [`Rinne414/SynthID-detector`](https://github.com/Rinne414/SynthID-detector/tree/b08abff06b67c37db9dfa7c6ed63760a446d560f) fixed residual template | 2026-08-23/24 | The repository reports 0/33 held-out AI images and identifies its original result as self-correlation. Its pinned pre-May GPT-Image2 template first detected 0/11 current source-matched edits and 0/11 COCO sources. An exact repeat on 200 current OpenAI and 200 current Google images produced zero detections for all three published templates at the 0.08 cut; maximum GPT-Image2 score on OpenAI was 0.0167. Adding the expected template can spoof the public correlator, but that template is not present in held-out content. Independent repeat of the rejected fixed-template route, not a payload decoder |
|
||||
| [`newideas99/gpt-image-synthid-detector`](https://github.com/newideas99/gpt-image-synthid-detector/tree/5495e09) CNN ensemble | 2026-08-24 | Exact frozen inference retained 172/200 OpenAI but accepted 110/200 Google at the published 0.5 cut. The training negatives are regenerated positives, while its confound probe transforms only positives and has no foreign-image control. A full-frame white-box attack drove its probability from 0.99995 to 0.25956 at ±2/255 and effectively zero at ±4/255, but the official OpenAI decoder stayed `detected` through ±8/255. Renderer/confound classifier, not a mark decoder |
|
||||
| [`Ristellise/REGRET`](https://github.com/Ristellise/REGRET/tree/7d449034bf323987e7e608e7886e029ed20fd847) SPAM model | 2026-08-24 | At the published cut, exact model inference accepted 139/200 Google, 141/200 OpenAI, 214/600 disjoint ImageNet photographs, and substantial fractions of foreign generators. The apparent high-precision tail did not transfer |
|
||||
| [`vordme2010/synthid-dataset`](https://github.com/vordme2010/synthid-dataset/tree/133a27088f6f4d695c79db9a1a70fa8e7fa3adad) Tier-1 SVM | 2026-08-24 | Rebuilt without loading joblib, using the repository's 2,000x33 numeric matrix, reference phases, noise template, seed-42 split, scaler, and RBF SVM. It accepted 1/200 current Google and no current OpenAI rows. Its published AUC above 0.999 distinguishes Gemini flats from generated derivatives, not current photographs |
|
||||
| Generic GitHub heuristics and literal ports | 2026-08-24 | `hackerfactor/reverse-SynthID-C`, `BIRSAx2/ripmark`, and `xiaoyao9184/reverse-from-synthid` reproduce or wrap the already-rejected reverse-SynthID codebook. `AI-SCERN` uses uncalibrated annular energy, decoder-grid, and FFT-symmetry heuristics with no weights or held-out corpus. LSB flips, ±1 dither, and 100.5% resize tools publish no matching-oracle evidence. None adds an independent signal |
|
||||
|
||||
## Information budget on photographs (2026-08-21)
|
||||
|
||||
On 11 `gpt-image-2` photo edits, the flat-derived 16-32 template at full
|
||||
amplitude against per-image whitening has median `d' = 0.93` (range
|
||||
0.68-2.46). That is a best-case single-image AUC of about 0.75 even with a
|
||||
perfect, perfectly aligned template. A P5 gate (FPR 0.1%, TPR 90%) needs
|
||||
`d' >= 4.37`. The deficit is 4.7x in amplitude, 13.4 dB, *unless* the
|
||||
photo-domain mark is louder than the flat measurement. Titration said it
|
||||
is not: fish 16-32 additive jam stays `detected` through a=16 / 21.9 dB.
|
||||
|
||||
2026-08-23, without using that flat stamp as `G`: the aligned photo
|
||||
residual in 16-32 is redraw, not a codeword. Median RMS 12.2 across 12
|
||||
pairs; 2.69 on the five tightest. Neighboring octaves are as loud.
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/photo-band-amplitude-2026-08-23.json`.
|
||||
|
||||
Do not report an AUC from those 11 pairs as a detector result. With n=11
|
||||
the standard error on AUC is about 0.12. Notes:
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/agent-detector-claude.md`.
|
||||
|
||||
## External literature (surveyed 2026-08-23)
|
||||
|
||||
Primary sources, not abstracts. Each row is mapped onto a closed or open
|
||||
route in this campaign. Mechanism detail stays in
|
||||
[synthid.md](synthid.md). Removal papers are on
|
||||
[mark removal](synthid-removal-research.md). Classifier papers are on
|
||||
[SynthID source classifiers](synthid-classifiers.md).
|
||||
|
||||
### Official mark, not a public decoder
|
||||
|
||||
| Source | What it is | Map to this campaign |
|
||||
| --- | --- | --- |
|
||||
| Gowal et al., [arXiv:2510.09263](https://arxiv.org/abs/2510.09263) | Post-hoc encoder `f` / decoder `g`. Detection logit is not payload recovery. SynthID-O (partner variant) 136 bits at 512x512. TPR at 0.1% FPR 99.98% aggregated, 99.97% on the hardest spatial crop+resize. Trains against sampled semantics-preserving transforms, including weak VAE regeneration. Production decoder unpublished | Matches the architecture we treat as keyed `x' = x + g(x)`. Explains why a two-pixel crop kills `pipeline_lattice` but not the official oracle, and why a 22.3 dB foreign VAE still reads `detected` |
|
||||
| DeepMind [US12094474B1](https://patents.google.com/patent/US12094474B1/en) and continuation [US20250149048A1](https://patents.google.com/patent/US20250149048A1/en) | Residual U-Net encoder, separate decoder, optional key, encoder/decoder ensembles that need not recognize each other | Constraint, not a recipe. Ensemble non-recognition is why one recovered Google tile cannot be reused as an OpenAI detector (S4) |
|
||||
| Dathathri et al., [Nature 634:818-823 (2024)](https://www.nature.com/articles/s41586-024-08025-4) | SynthID-Text: tournament sampling of LLM tokens, open-source | Different system. Image/audio/video remain proprietary |
|
||||
| OpenAI, [advancing content provenance](https://openai.com/index/advancing-content-provenance/) (2026-05-19, audio 2026-07-31) and [content provenance API](https://developers.openai.com/api/docs/guides/content-provenance) | ChatGPT / API / Codex images carry C2PA plus SynthID. Audio from 2026-07-31. `POST /v1/content_provenance_checks`. `not_detected` does not rule out another vendor | This is the oracle. C2PA and SynthID are independent entries. Do not abuse the endpoint as an adaptive reverse-engineering loop |
|
||||
|
||||
### Keyless presence detectors in the literature
|
||||
|
||||
| Source | Claim | Caveat against our gates |
|
||||
| --- | --- | --- |
|
||||
| Ao et al., [arXiv:2603.06723](https://arxiv.org/abs/2603.06723) (AWPD / FSNet, SAFE@CVPR 2026) | Leave-one-algorithm-out presence detection. SynthID held out: FSNet Acc 0.894 / F1 0.886, ResNet-50 Acc 0.845 / F1 0.812, ConvNeXt V2 Acc 0.866. LSB and Patchwork both fail below 60%. Hypothesis: modern invisible marks share dense high-frequency spectral anomalies | UniFreq Table 3: SynthID is 2,000 images, all AIGC, zero photographs, from `imagen-4.0-fast-generate-001` only, resized to 256x256. No OpenAI. No 0.1% FPR. Closest published analog to E3/L1 distillation. Our TrustMark leave-one-encoder-out was chance (0.505 ridge). Their high-frequency commonality is the same energy that L1 failed to read as an OpenAI payload |
|
||||
| `aloshdenny/reverse-SynthID` V3/V4; Google to [The Verge, 2026-04-14](https://www.theverge.com/ai-artificial-intelligence/911579/google-synthid-ai-watermarking-system-reverse-engineered) | Spectral codebook from averaged Gemini flats. Spokesperson Myriam Khan: "It is incorrect to say this tool can systematically remove SynthID watermarks." The author also said the bypass confuses the decoder rather than deleting a payload | Our pickle-free V4 audit: paired AUC 0.517, frozen Open Images 386/1000 accepted. Not 0.1% FPR |
|
||||
| Krawetz, [Hacker Factor "Reversing SynthID"](https://www.hackerfactor.com/blog/index.php?/archives/1092-Reversing-SynthID.html) and ["Meta's Un-Stable Signature"](https://hackerfactor.com/blog/index.php?/archives/1098-Metas-Un-Stable-Signature.html) | Gemini chat TPR closer to 1/20 than the paper's 99.97%. Detector weak on flats. reverse-SynthID field accuracy ~70% vs the author's 90% | Gemini-app chat is not the OpenAI provenance API. Our gray `gpt-image-2` flats are `detected` on the official OpenAI oracle. Do not mix the two verifiers |
|
||||
| vitotitto LAB-a logistic (community, tiny holdout) | Reported 97.7% AUC on 20/20 | Not a P5 gate. Ignore as a detector claim |
|
||||
|
||||
### Different embedding loci (not post-hoc SynthID)
|
||||
|
||||
These papers are often cited as if they were SynthID. They are not.
|
||||
|
||||
| Source | Locus | Why it is not this mark |
|
||||
| --- | --- | --- |
|
||||
| Gunn, Zhao, Song, [arXiv:2410.07369](https://arxiv.org/abs/2410.07369) (PRC, ICLR 2025) | Pseudorandom error-correcting code in the *initial diffusion latent* | Cryptographic undetectability is a latent-code property. SynthID-Image is applied after the VAE has already decoded pixels. Our OKLab "PRC-style" noise was a category error |
|
||||
| Francati et al., [arXiv:2509.10577](https://arxiv.org/abs/2509.10577) (EuroS&P 2026) | Crop-and-resize flipped about half of PRC latent signs and blocked belief-propagation decode | Confirms PRC is origin-locked in latent space. Matches why a 2 px shift kills `pipeline_lattice` and does not kill OpenAI SynthID |
|
||||
| Fernandez et al. Stable Signature; Wen et al. Tree-Ring | Fine-tuned VAE decoder, or ring constraints on initial noise | In-generation. Google's paper is explicit that SynthID-Image does not modify the generator |
|
||||
| TrustMark ([arXiv:2311.18297](https://arxiv.org/abs/2311.18297)), HiDDeN, StegaStamp | Open post-hoc encoder/decoder pairs | Transfer from these is E3: chance on TrustMark. WAVES ([arXiv:2401.08573](https://arxiv.org/abs/2401.08573)) stress-tests this family, not production SynthID |
|
||||
|
||||
Do not train a student on AWPD's Imagen-only 256 crop and call it an
|
||||
OpenAI SynthID detector. That would repeat L1's contamination: a
|
||||
generator/size cue, not a payload.
|
||||
|
||||
### Adjacent literature, not SynthID
|
||||
|
||||
These papers are the published versions of the wild hypotheses and of
|
||||
Model 1. They detect *something else*: a camera pipeline, a generator
|
||||
fingerprint, an open watermark, or steganography. None of them is a
|
||||
keyless OpenAI payload decoder.
|
||||
|
||||
**Classical spread-spectrum and open neural marks**
|
||||
|
||||
| Source | What it is | Map |
|
||||
| --- | --- | --- |
|
||||
| Cox, Kilian, Leighton, Shamoon, IEEE TIP 1997 | i.i.d. Gaussian vector in perceptually significant spectral components. Informed detector. Collusion-resistant by construction | Ancestor of keyed CDMA. Our length-255 m-sequence vs 16x16 fold was at chance because we did not have the key, and because OpenAI is content-adaptive, not a fixed Gold code |
|
||||
| Zhu et al., HiDDeN, ECCV 2018 | Joint encoder / noise layer / decoder | Open pair. AWPD leave-one-out Acc 0.985. Transfer to SynthID is E3 |
|
||||
| Tancik, Mildenhall, Ng, StegaStamp, 2020 | 100 bits at 400x400, trained through print-scan | Open pair. Survives recapture. WAVES: TPR at 1% FPR collapses from 1.00 to 0.01 under regeneration |
|
||||
| Wen et al., [arXiv:2305.20030](https://arxiv.org/abs/2305.20030) (Tree-Ring, NeurIPS 2023) | Pattern in the *initial diffusion noise*, recovered by DDIM inversion | In-generation, origin-locked in latent Fourier space. Crop-invariant by design. Not post-hoc SynthID |
|
||||
| Fernandez et al., Stable Signature, ICCV 2023 | Fine-tune the LDM VAE decoder so every decode carries a bit string | In-generation. Gowal is explicit that SynthID-Image does not modify the generator |
|
||||
| Lin and Juarez, [arXiv:2506.10502](https://arxiv.org/abs/2506.10502) (USENIX 2025) | Public-knowledge attack that removes Tree-Ring | Confirms Tree-Ring is a different object with a different kill |
|
||||
|
||||
**Steganalysis as a presence detector**
|
||||
|
||||
Fridrich and Kodovsky Spatial Rich Models (TIFS 2012) and Boroumand,
|
||||
Chen, Fridrich SRNet (TIFS 2018) detect sub-bit-per-pixel spatial
|
||||
stego by high-pass residuals, with pooling disabled in the front of
|
||||
SRNet so the weak signal is not averaged away. AWPD cites both and
|
||||
says they drift on modern deep / generative marks. That matches our
|
||||
wavelet/FFT single-image detector (AUC 0.653, 0 TPR at a clean cut)
|
||||
and the 16-32 energy miss on photographs: a residual energy detector
|
||||
without the matching key is steganalysis of a mark that was trained
|
||||
not to look like LSB.
|
||||
|
||||
**Generator fingerprints in the Fourier domain**
|
||||
|
||||
Corvi, Cozzolino, Poggi, Nagano, Verdoliva,
|
||||
[arXiv:2304.06408](https://arxiv.org/abs/2304.06408) (CVPRW 2023):
|
||||
GAN, diffusion, and VQ-GAN images show spectral peaks and anomalous
|
||||
autocorrelation; real vs synthetic differ in mid-high radial and
|
||||
angular power. reverse-SynthID averaged Gemini flats and called the
|
||||
peak a watermark codebook. Corvi's result says many generators leave
|
||||
*some* peak. Our V4 Open Images 386/1000 is what a generator-fingerprint
|
||||
detector looks like when you calibrate it as if it were a payload.
|
||||
|
||||
Yao and Juarez, [arXiv:2512.11771](https://arxiv.org/abs/2512.11771)
|
||||
("Smudged Fingerprints"): 14 fingerprinting methods across RGB,
|
||||
frequency, and learned features; removal attacks >80% white-box, >50%
|
||||
black-box. A fingerprint you can see without a key is a fingerprint
|
||||
you can wipe without a key.
|
||||
|
||||
### Image investigation and data hiding (any method)
|
||||
|
||||
These are not SynthID papers. They are the rest of the toolkit: how
|
||||
people hide bits in pictures, and how people tell a picture was
|
||||
touched. Several of our wild hypotheses already had a published form
|
||||
here.
|
||||
|
||||
Hiding is not one problem. Cover modification (change an existing
|
||||
image), coverless / generative (sample an image that already carries
|
||||
the bits), and signed metadata (C2PA) fail under different attacks.
|
||||
|
||||
**Cover modification, classical**
|
||||
|
||||
| Source | Hide how | Detect / limit |
|
||||
| --- | --- | --- |
|
||||
| LSB, Patchwork (Bender et al., IBM SJ 1996) | Flip low bits, or luminance of random pixel pairs | AWPD Acc < 0.60. Sparse or ±1 amplitude. SRNet / FSNet average it away |
|
||||
| Westfeld F5 (2001), Fridrich nsF5 | JPEG DCT coefficients, matrix embedding | Histogram attacks on F5; nsF5 was the shrinkage fix. Domain is the codec, not a mid-band residual |
|
||||
| Pevny, Filler, Bas HUGO (2010); Holub and Fridrich WOW (2012); Holub, Fridrich, Denemark UNIWARD (IH&MMSec 2013 / EURASIP 2014) | Content-adaptive costs, bits placed in texture via syndrome-trellis codes (Filler, IH 2011) | The modern spatial/JPEG floor. Distortion is *designed* to look like the cover. A 16-32 energy detector is the wrong statistic |
|
||||
| Cox et al. 1997 | Spread-spectrum in significant DCT | Keyed. Already mapped above |
|
||||
|
||||
**Cover modification, neural (hide a whole image, not 32 bits)**
|
||||
|
||||
| Source | What it does | Map |
|
||||
| --- | --- | --- |
|
||||
| Baluja, NeurIPS 2017, "Hiding Images in Plain Sight" | Full-resolution secret image distributed across all bits of a same-size cover | Capacity is the point. Not robust, not keyed provenance. HiDDeN later added a noise layer so the secret survives JPEG |
|
||||
| Jing et al., HiNet, ICCV 2021 | Invertible neural net: hide and recover as one bijection | High-capacity lossless-ish hiding. Recovery needs the exact inverse, not an official oracle |
|
||||
| Yang et al., PRIS, [arXiv:2309.13620](https://arxiv.org/abs/2309.13620) | Invertible net plus robustness modules | Same family, trained through distortion |
|
||||
|
||||
A HiNet-style student on OpenAI pairs would learn the *edit*, not a
|
||||
SynthID codeword. That is the L1 contamination again.
|
||||
|
||||
**Coverless / generative hiding**
|
||||
|
||||
The cover is never modified because there is no cover. The sampler
|
||||
emits an image whose latents already encode the message.
|
||||
|
||||
| Source | Locus | Map |
|
||||
| --- | --- | --- |
|
||||
| Yang et al., Gaussian Shading, [arXiv:2404.04956](https://arxiv.org/abs/2404.04956) (CVPR 2024) | Map bits onto Gaussian latents indistinguishable from ordinary noise. Recover by DDIM inversion | Sibling of Tree-Ring / PRC. Training-free, performance-lossless *for the generator*. Not a post-hoc pixel stamp |
|
||||
| Peng et al., StegaDDPM (ACM MM 2023) and later LDStega | Bits in the diffusion sampling distribution | Spatial SRNet is the wrong detector (NS-DSer, [arXiv:2602.10219](https://arxiv.org/abs/2602.10219): move steganalysis into noise space) |
|
||||
| CRoSS, Pulsar, MDDM | Message-to-noise projections | Same locus. A pixel scramble does not invert the sampler |
|
||||
|
||||
**Passive forensics (the picture was touched, no secret assumed)**
|
||||
|
||||
| Source | Cue | Map |
|
||||
| --- | --- | --- |
|
||||
| Krawetz, "A Picture's Worth", 2007 (ELA) | Re-JPEG at lower quality, subtract | Already measured: COCO 3.13, s1 1.97, gray stamp 0.49. Codec history, not a payload. Farid publicly called ELA as likely to mislabel originals as it is to catch edits |
|
||||
| Farid, IEEE TIFS 2009, JPEG ghosts | Difference energy vs a sweep of JPEG qualities; spliced regions ghost at their original Q | Local: s1, a `gpt-image-2` photo, and a camera JPEG all minimize at Q90. Codec, not a payload |
|
||||
| Popescu and Farid, TR2004-515 | Copy-move via duplicated regions | Not generation, not a watermark |
|
||||
| Popescu and Farid, IEEE TSP 2005 | Resampling periodic correlations | Affine search cousin. A rotated SynthID residual is a different question |
|
||||
| Wang et al., DIRE, [arXiv:2303.09295](https://arxiv.org/abs/2303.09295) (ICCV 2023) | Diffusion reconstruction error: generated images reconstruct, cameras do not | Model 1 sibling. SDXL float32 at 512: DIRE RMS camera 33.0, s1 31.5, photo edit 30.9, gray stamp 2.40. Texture rank, not a payload. Inverse of the foreign-VAE remover: there the mark survived 22.3 dB |
|
||||
| Wang, Wang, Zhang, Owens, Efros, [arXiv:1912.11035](https://arxiv.org/abs/1912.11035) (CVPR 2020, CNNDetect) | One ProGAN classifier, heavy JPEG/crop aug, transfers to many CNNs | Ancestor of "train on one generator". Ojha showed the sink-class failure once diffusion arrived. We required Firefly for that reason |
|
||||
|
||||
C2PA is the non-pixel stack: a signed manifest, stripped by
|
||||
`metadata --remove`. Durable Content Credentials (spec 2.4) add a
|
||||
soft binding that can re-link a stripped file to a repository. That
|
||||
is provenance, not hiding.
|
||||
|
||||
Do not train on ELA, JPEG ghosts, DIRE, or a HiNet reconstruction and
|
||||
name the score SynthID.
|
||||
|
||||
### Waveforms that can live in a picture
|
||||
|
||||
A mark is a function on the pixel lattice. The literature does not
|
||||
use one wave. It picks a basis whose symmetries match the attack it
|
||||
fears, then hides a keyed coefficient vector in that basis. Cartesian
|
||||
16-32 is one shell. Polar, scale, and diffraction are different
|
||||
shells.
|
||||
|
||||
**Standing waves on a rectangle.** DFT / DCT / DST. A 2-D sinusoid
|
||||
`cos(2π(ux + vy)/N)`. JPEG lives here. Our octave scramble destroys
|
||||
one annular *radius* of these frequencies, not one orientation. A
|
||||
Gabor packet is the same sinusoid windowed in space.
|
||||
|
||||
**Circular and log-radial waves.** Functions of `(r, θ)`, not
|
||||
`(x, y)`.
|
||||
|
||||
| Basis | Wave | Invariance it buys |
|
||||
| --- | --- | --- |
|
||||
| Fourier-Mellin / log-polar (Ó Ruanaidh and Pun, Signal Processing 1998) | `r^{iα} exp(ikθ)` after a DFT magnitude | Rotation and scale become translations |
|
||||
| Logarithmic radial harmonics (IH 2002) | Same family, added in pixels, detected by complex correlation | RST without going through the unstable log-polar resample |
|
||||
| Polar harmonic transforms: PCET, PCT, PST | `exp(±i2π n r²)` and polar cos/sin | Rotation. Moments, not a dense codebook |
|
||||
| Zernike / pseudo-Zernike | Orthogonal polynomials on the disk, radial part related to Bessel | Rotation. Classical moment watermark |
|
||||
| Circular chirp (SPIE 6072, 2006) | Polar map of a 1-D chirp onto a ring | JPEG (tune chirp rate) plus rotation (the ring) |
|
||||
| Tree-Ring (Wen 2023) | Concentric rings in the *latent* Fourier plane | Crop/flip by construction. Not a pixel wave |
|
||||
|
||||
The 2026-08-23 file named polar-1632 is cartesian annular phase
|
||||
shuffle, a scramble replicate (`not_detected` at 25.6 dB). True
|
||||
Fourier-angle scramble of the same annulus is `not_detected` at 24.1 dB.
|
||||
Radial-phase-only scramble is `not_detected` at 25.0 dB. One 90°
|
||||
sector plus its conjugate stays `detected` at 27.5 dB. Affine/rotation
|
||||
search of the 16-32 residual against the flat template does not lock
|
||||
(lift 0.011, COCO 0.008).
|
||||
|
||||
**Scale chirps.** Hyperbolic / Mellin monomials (arXiv:1208.5842):
|
||||
real 1-D Mellin patterns tiled in 2-D. Run 2026-08-23:
|
||||
`r^{iα}` coefficients flat LOO 0.51, photo -0.27, COCO -0.58.
|
||||
Log-polar `|F|` and a 64-bin radial Mellin profile saturate on COCO
|
||||
(0.99). Not a payload, and not quieter than cartesian 16-32.
|
||||
|
||||
**Directional packets.** Dual-tree complex wavelets (approximate
|
||||
analytic wave), Gabor/Morlet, ridgelets, curvelets, shearlets,
|
||||
contourlets, bandelets. Multiplicative spread-spectrum on curvelet
|
||||
coefficients is a published detector-design paper, not a SynthID
|
||||
decoder. Our wavelet summary AUC 0.653 already said a *generic*
|
||||
packet energy is not the OpenAI payload.
|
||||
|
||||
**Optical diffraction, actual wave physics.**
|
||||
|
||||
| Transform | What the wave is | Map |
|
||||
| --- | --- | --- |
|
||||
| Fresnel | Quadratic phase `exp(iπ r² / λz)`, a radial chirp. Energy concentrates with distance | Holographic watermarks embed the diffracted field, not the picture |
|
||||
| Fractional Fourier | Rotation of the time-frequency plane. Order is an extra key | Extra key space on top of DFT |
|
||||
| Double random phase encoding (Réfrégier and Javidi, Optics Letters 1995) | Random phase at input *and* Fourier plane. Ciphertext is stationary white noise | Closest optical analog of a keyed `x' = x + g(x)` that looks like noise without both screens |
|
||||
| Cascaded Fresnel / CGH | Two holograms at different depths | Hardware key. Not a pixel residual we can match-filter |
|
||||
|
||||
A Fresnel/FrFT student without the distance or the fractional order
|
||||
is another unmatched filter. Fresnel at z=8, 16, 32, 64 on the 16-32
|
||||
residual is a unitary copy of the cartesian band (flat LOO 0.75, photo
|
||||
0.012). FrFT is the same unmatched-filter claim with an extra order
|
||||
key; it does not isolate a photograph lock the four distances missed.
|
||||
|
||||
**Cyclostationary and clock-warped waves.** Periodic clock change
|
||||
(Martin, Chabert, Lacaze, EUSIPCO 2005) turns a stationary host into
|
||||
a cyclostationary one by a periodic time warp. Our H2 multiplicative
|
||||
cyclostationarity test already put official positives *below*
|
||||
controls. Do not rerun the same statistic in a new basis and call it
|
||||
new evidence.
|
||||
|
||||
**How to read a candidate wave.** Name the coordinates `(x,y)` vs
|
||||
`(r,θ)` vs `log r`, the generating function (sinusoid, chirp,
|
||||
Bessel, Zernike), additive vs multiplicative, and the symmetry it
|
||||
claims. Then pick one attack that kills that symmetry and one that
|
||||
should not. Cartesian 16-32 scramble and a 2 px shift are done
|
||||
(scramble kills, shift does not). True Fourier-angle scramble kills s1
|
||||
at 24.1 dB. Radial-phase scramble kills at 25.0 dB. A 90° sector does
|
||||
not. Y scramble kills; Cb/Cr do not. Affine, Mellin, Zernike, PCET,
|
||||
and Fresnel of the 16-32 residual do not lock on photographs.
|
||||
|
||||
## Still open or incomplete
|
||||
|
||||
- Google: no public payload decoder and no usable programmatic oracle.
|
||||
The Gemini app remains quota-limited. Vertex was 403.
|
||||
- `chatgpt-image-latest` is an optional third pair mint, not a blocker.
|
||||
`gpt-image-1`, `gpt-image-1.5`, and `gpt-image-1-mini` are encoder-off.
|
||||
`gpt-image-2` and dated `gpt-image-2-2026-04-21` are encoder-on, same
|
||||
16-32 stamp (corr 0.189 on gray). Latest would only matter if ChatGPT's
|
||||
current consumer id used a different ensemble. `images.generate` is 403
|
||||
until the organization is verified.
|
||||
- Physical print-scan: Brother DCP-L2520DW is idle and accepting. No
|
||||
`scanimage`, no ImageCapture pyobjc. Recapture needs a human at the
|
||||
flatbed. Simulation at 24.95 dB stayed `detected`. Do not print
|
||||
unattended.
|
||||
- Adding unmarked context (frame, pad, hstack, collage) to hunt a
|
||||
residual is a presentation gate, not a mark statistic. Those rungs
|
||||
belong with identity and layout below, and with
|
||||
[mark removal](synthid-removal-research.md).
|
||||
- Camera-to-ChatGPT "unchanged crop" pairs are dirty counterfactuals.
|
||||
The instructed edit redraws; aligned PSNR on the frame pair was
|
||||
16.6 dB. Compare pixels only after registration and a frozen residual
|
||||
bound, as in the `gpt-image-2` red-dot row.
|
||||
|
||||
A Gemini app caption that an image "looks like a photograph" is not a
|
||||
SynthID verdict. Use the provider pixel check or signed provenance.
|
||||
|
||||
## Wild hypotheses
|
||||
|
||||
Battery 2026-08-23, no official oracle. Local numbers:
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/wild-hypotheses-2026-08-23.json`.
|
||||
Prepared attack rasters wait in `wild-attacks-2026-08-23/` for a later
|
||||
verifier window. Jacobian / adaptive queries against
|
||||
`verify-openai-synthid` stay out: the endpoint forbids reverse-engineering.
|
||||
|
||||
### Tested locally, not a mark
|
||||
|
||||
| Hypothesis | Result |
|
||||
| --- | --- |
|
||||
| Chroma ordered by luma is a 1D payload | Correlation is an artifact of the sort |
|
||||
| Mark is `g(x)` of a Retinex envelope | LOO unchanged vs the raw 16-32 residual |
|
||||
| Nonlocal patch agreement in 16-32 | Far patches are not coupled |
|
||||
| Quadratic phase coupling (bispectrum) | Statistic saturates on camera images too |
|
||||
| CDMA Gold / m-sequence in the 16x16 fold | At chance for length 255 |
|
||||
| Rows as a pitched waveform | COCO noise has the same peak-to-mean |
|
||||
| The photo mark is the flat stamp at an unknown 16-phase | Exhaustive shift does not lift NCC |
|
||||
| The photo mark is the flat stamp at a small rotation or 3% scale | Affine search lift 0.011, COCO noise 0.008 |
|
||||
| Self-keyed predictor from a 16 px luma thumb | Already 0.48 vs L1, 0.50 vs COCO (`self-keyed.json`) |
|
||||
| Global multiplicative cyclostationarity (H2) | Official positives below controls, 2026-08-14 |
|
||||
| Mellin radial profile / log-polar \|F\| | Saturates on COCO (0.99). Annulus envelope |
|
||||
| Scale chirps `r^{iα}` | Flat 0.51, photo -0.27, COCO -0.58 |
|
||||
| Zernike n≤6 | Flat 0.77, photo -0.007, COCO 0.23 |
|
||||
| PCET polar harmonics | Flat 0.75, photo 0.044, COCO 0.35 |
|
||||
| Fresnel z=8..64 of 16-32 | Copy of cartesian LOO |
|
||||
| 8-bin orientation energy | Saturates on COCO |
|
||||
| Chroma 16-32 on gray flats | Cb 0.064, Cr 0.025 vs Y 0.682. Stamp is luma |
|
||||
|
||||
### Not a watermark, might still be an AI-vs-camera extra
|
||||
|
||||
Battery 2026-08-23, `remaining-hypotheses-2026-08-23.json`. Kodak corpus was empty here, camera arm is COCO plus the photo-pair sources.
|
||||
|
||||
| Hypothesis | Result |
|
||||
| --- | --- |
|
||||
| Absence of camera PRNU | Gray highpass RMS 0.25 vs COCO 14.6 was texture. Wiener residual on *photographs*: `gpt-image-2` edits 4.61 vs camera 8.05. Direction is camera-like leftover of the donor JPEG, not a missing sensor, and not a payload |
|
||||
| JPEG ELA | Flat 0.49, s1 1.97, COCO 3.13. Compression history |
|
||||
| CFA interpolation error (Bayer as detector) | Ratio photo-edit 0.117 vs camera 0.184 vs gray stamp 0.588. Weak camera vote, overlaps. Inverse of the Bayer remover, not SynthID |
|
||||
| JPEG ghosts (Farid) | s1, a `gpt-image-2` photo, and a camera JPEG all minimize at Q90 with similar spreads. Codec, not a payload |
|
||||
| PNG Paeth / filter-byte residual | `gpt-image-2` PNG is 99.9% Paeth. Camera re-saved PNG 73% Paeth, s1 74%. Export encoder fingerprint |
|
||||
| Perfect-circle / text-edge rate | Hough circles per MP 385 (edits) vs 536 (camera). Edge fraction 0.052 vs 0.072. Too noisy for a payload or a provider |
|
||||
| Face-gated 16-32 | s1: 3 Haar faces, inside RMS 12.6 vs outside 10.9. One photo with faces: outside/in 1.49. No systematic "spend the mark off faces" |
|
||||
| CLIP-space watermark | That is Model 1. Not a payload decoder |
|
||||
|
||||
### Non-local, 2026-08-23
|
||||
|
||||
Battery `.local-eval/synthid/prc-oklab-attack-2026-08-15/nonlocal-2026-08-23/`.
|
||||
Oracle after AI-metadata strip, 25 s gap, then one dated-id retry after a 429.
|
||||
|
||||
| Hypothesis | Result |
|
||||
| --- | --- |
|
||||
| HEIF q80 / q50 | 46.3 / 39.3 dB, official `detected` |
|
||||
| AV1 CRF 32 still | 37.2 dB, `detected` |
|
||||
| Print-scan simulation (0.35 deg, blur σ 0.9, 300/360 dpi, JPEG q42) | 24.95 dB, still `detected`. Same PSNR class as 16-32 scramble, which kills |
|
||||
| Physical print-scan | Brother DCP-L2520DW idle and accepting. No `scanimage`, no ImageCapture pyobjc. Not printed unattended |
|
||||
| `gpt-image-1.5` gray reproduce | C2PA `created, converted`. Oracle `not_detected`. 16-32 corr vs `gpt-image-2` stamp 0.003. Encoder-off. Dirty redraw 24.0 dB |
|
||||
| `gpt-image-1-mini` gray reproduce | Same C2PA. Oracle `not_detected`. Corr 0.003. PSNR 31.1 dB |
|
||||
| `gpt-image-2-2026-04-21` gray reproduce | C2PA `created, converted, watermarked.unbound`. Official `detected`. 16-32 corr vs unversioned `gpt-image-2` stamp 0.189. Same encoder family. PSNR 47.5 dB |
|
||||
| `chatgpt-image-latest` generate | 403, org verification. Listed among image models, not callable |
|
||||
| DIRE SDXL VAE + DDIM float32 | VAE RMS 512: gray 1.24, s1 9.15, photo edit 9.89, camera 11.84. DIRE RMS: gray 2.40, photo 30.9, s1 31.5, camera 33.0. Texture rank, not a payload |
|
||||
|
||||
### Still blocked after that battery
|
||||
|
||||
| Hypothesis | Why it stays blocked |
|
||||
| --- | --- |
|
||||
| Physical print-scan recapture | Human at the flatbed |
|
||||
| Encoder ensemble that does not recognize itself | Needs another actual SynthID encoder/decoder pair. Extra OpenAI ids did not mint one. Open-proxy transfer is already closed: source-matched TrustMark, VideoSeal, DWT-DCT, and WAM pairs produced held-out-family AUC 0.437-0.562, so another generic public pair does not answer this hypothesis |
|
||||
| `chatgpt-image-latest` as a third pair mint | `images.generate` 403 until org verification. Optional: only if ChatGPT latest used a different ensemble than `gpt-image-2` |
|
||||
|
||||
Do not train on official decoder bits beyond the frozen L1 study. Do not
|
||||
mint a 128-photo student on the flat stamp. The information budget and
|
||||
the 2026-08-23 residual measurement both say the photo-domain mark is
|
||||
under the scene.
|
||||
|
||||
## Oracle and seeds
|
||||
|
||||
Official `POST /v1/content_provenance_checks`, metadata stripped, pixels
|
||||
preserved. Raw files stay gitignored under
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/`.
|
||||
|
||||
| Seed | Size |
|
||||
| --- | --- |
|
||||
| s1 | 1024x1536 |
|
||||
| s2 | 1024x1536 |
|
||||
| s3 | 1086x1448 |
|
||||
|
||||
All three `detected` before attacks.
|
||||
|
||||
## Source-matched pairs (2026-08-21)
|
||||
|
||||
`gpt-image-2` (ChatGPT Images 2.0) can mint encoder-on/off pairs.
|
||||
`chatgpt-image-latest` is optional and 403 until org verification.
|
||||
|
||||
| Input | C2PA | Oracle | Aligned PSNR |
|
||||
| --- | --- | --- | ---: |
|
||||
| Camera JPEG 960x1280 | none | not_detected | - |
|
||||
| ChatGPT UI + frame 1086x1448 | watermarked.unbound | detected | 16.6 dB (redraw) |
|
||||
| `gpt-image-1` edit, high fidelity, red dot | created, converted only | not_detected | 30.9 dB |
|
||||
| `gpt-image-1.5` gray reproduce 1024 | created, converted only | not_detected | 24.0 dB (redraw) |
|
||||
| `gpt-image-1-mini` gray reproduce 1024 | created, converted only | not_detected | 31.1 dB |
|
||||
| `gpt-image-2` edit, red dot 1024x1536 | watermarked.unbound | detected | 28.2 dB |
|
||||
| `gpt-image-2-2026-04-21` gray reproduce 1024 | watermarked.unbound | detected | 47.5 dB |
|
||||
|
||||
The `gpt-image-2` photo residual is broadband (std 9.1), not a 16-32 stamp.
|
||||
COCO `gpt-image-2` edits: 11/12 `watermarked.unbound`, one safety block.
|
||||
|
||||
Solid gray/white through `gpt-image-2` "reproduce exactly" is
|
||||
`watermarked.unbound`. Gray 128 at 1024x1536: PSNR 41.9 dB vs the flat,
|
||||
source `not_detected`, output `detected`. Other gray levels 32-220: PSNR
|
||||
38-49 dB. Eight gray 1024x1536 residuals, leave-one-out correlation after
|
||||
highpass: 0.645 (shuffle 0). That shared stamp does not transfer to COCO
|
||||
photo residuals (0.007-0.025).
|
||||
|
||||
## Presentation gate versus residual
|
||||
|
||||
An official `not_detected` is not one event. Collage, photo-pad, two-panel
|
||||
self-concat, and 8x8 scramble can be a *presentation gate* (this upload is
|
||||
not a single photograph). Blur sigma 7 and in-band 16-32 destruction sit on
|
||||
the single-photo manifold and are residual damage; those belong in
|
||||
[mark removal](synthid-removal-research.md).
|
||||
|
||||
Do not train a surrogate on mixed collage misses if the goal is a mark
|
||||
student.
|
||||
|
||||
Identity (s1, 2026-08-19): JPEG q80/q20, crop 2 px, crop 8 px, keep 0.75
|
||||
and 0.50, rot90, rotate 5, flip, grayscale all stay `detected`.
|
||||
|
||||
Token / layout (s1): each quadrant at 512x768 `detected`; `hstack` and
|
||||
`vstack` of marked\|marked `not_detected`; 4x4 tile scramble `detected`;
|
||||
8x8 `not_detected`. Two copies of a detecting image still miss. Tomography
|
||||
3x3: all nine cells `not_detected` (a 1/9 window is below support).
|
||||
|
||||
Preprocess E1 (s1): stretch 2x `detected`; centre-crop of marked\|marked
|
||||
hstack back to native `detected`; 0.20x pixels on a native-size white
|
||||
canvas `not_detected` (the same 0.20x file uploaded alone was `detected`).
|
||||
|
||||
## L1 is not a mark task
|
||||
|
||||
A missing `watermarked.unbound` assertion is not a clean negative. On
|
||||
2026-08-16 the official verifier called 65 of 94 OpenAI rows without that
|
||||
assertion `detected` (69%), with 9 of 9 interleaved health positives
|
||||
detected so the endpoint was answering. Google's own API emits watermarked
|
||||
images with no assertion, per its documentation. Corpus AUCs that treat
|
||||
"no assertion" as unmarked rest on a negative class that is roughly
|
||||
two-thirds positive.
|
||||
|
||||
Control-only 283 rows: 203 detected, 80 not_detected. Forward-temporal
|
||||
nested ridge on 124 pixel features: AUC 0.649 / 0.641. Geometry-only:
|
||||
0.781 / 0.826. All 21 `claim_generator=ChatGPT` controls are
|
||||
`not_detected`. C2PA `opened`: 24/24 `not_detected`. Inside 1254x1254,
|
||||
mean-feature permutation p=0.45.
|
||||
|
||||
OpenAI-supervised ridge on 16-32 band-passed `image_features`:
|
||||
forward-temporal AUC 0.53 vs L1 `not_detected`, 0.95 vs COCO, 0.97 vs COCO
|
||||
after a 2 px crop. The student learns OpenAI-versus-photo and still does
|
||||
not see the oracle mark contrast.
|
||||
|
||||
## Product remainder for the watermark
|
||||
|
||||
Signed provenance (`identify`) and `verify-openai-synthid` (remote, explicit
|
||||
upload). A research lattice miss is not a clean SynthID negative.
|
||||
@@ -0,0 +1,243 @@
|
||||
# SynthID mark removal research
|
||||
|
||||
> Research archive for pixel-only SynthID removal, not diffusion
|
||||
> regeneration. Not a statement of current product capability. Shipped
|
||||
> invisible removal is lossy regeneration:
|
||||
> [known limitations](known-limitations.md).
|
||||
>
|
||||
> Sister pages: [SynthID local detector](synthid-detector-research.md),
|
||||
> [SynthID source classifiers](synthid-classifiers.md),
|
||||
> [mechanism reference](synthid.md).
|
||||
|
||||
## Result
|
||||
|
||||
The quality-preserving OpenAI SynthID remover hunt closed 2026-08-20.
|
||||
Bayer, VNG demosaic, upscale-then-Bayer, barrel distortion, scanline
|
||||
jitter, and a 2 px shift closed 2026-08-22 on s1/s2 and 2026-08-23 on s3
|
||||
and fish: they leave the official oracle `detected`.
|
||||
|
||||
Working residual kills on photographs cost about 19-26 dB:
|
||||
|
||||
- 16-32 px cartesian phase scramble (s1 24.6-24.8 dB, fish 23.2 dB, s2 19.0 dB)
|
||||
- Fourier-angle scramble of the same annulus (s1 24.1 dB)
|
||||
- Radial-phase scramble of the same annulus (s1 25.0 dB)
|
||||
- Y-only 16-32 scramble (s1 24.6 dB); Cb/Cr-only do not kill
|
||||
- File named polar-1632, actually cartesian (s1 25.6 dB)
|
||||
- Replace 16-32 with a COCO photo's 16-32 (s1 25.2 dB)
|
||||
- Gaussian blur sigma 7 (23.8 dB), holds 3/3 versus sigma 6
|
||||
- Additive 16-32 jam only at a=24 / 18.6 dB, worse than scramble
|
||||
|
||||
Baker-map, 8-seam carve, Poisson, nested LSB, palette64, and ICC rewrite
|
||||
do not kill at a better PSNR.
|
||||
|
||||
JPEG q5, noise sigma 16, grayscale, rot90, flip, 5°, downscale 0.20x,
|
||||
median 7, posterize 4, VAE round-trip, and white pad to 40% linear stay
|
||||
`detected`. Elastic warp is not a stable kill (s3 still `detected` at
|
||||
22.0 dB).
|
||||
|
||||
The product remainder is diffusion regeneration (`qwen-zimage` /
|
||||
`sdxl-zimage`), which does not decode and delete a payload. Defeating the
|
||||
verifier does not restore forensic deniability; see
|
||||
[synthid.md](synthid.md#23-removal-attacks-and-forensic-detectability).
|
||||
|
||||
Official `not_detected` on collage, photo-pad, and two-panel layouts is a
|
||||
presentation gate, not residual damage. Those rungs are in
|
||||
[detector research](synthid-detector-research.md).
|
||||
|
||||
## Closed quiet removers
|
||||
|
||||
| Attack | Close | Notes |
|
||||
| --- | --- | --- |
|
||||
| Quality-preserving photo remover | 2026-08-20 | Every residual `not_detected` that is not a collage is below usable quality |
|
||||
| Additive in-band jam as a quiet remover | 2026-08-21 | Fish flips only at 18.6 dB; 4-8 px at the same PSNR stays `detected` |
|
||||
| Bayer bilinear / VNG / upscale-Bayer | 2026-08-23 | s1/s2/s3 and fish still `detected`. s3 bilinear 37.1 dB, fish 34.7 dB. VNG on s2 is dirtier than scramble (18.5 vs 19.0 dB) and the mark remains |
|
||||
| Barrel k1=0.06, scanline ±0.8 px, and shift 2 px | 2026-08-23 | s1/s2/s3/fish `detected` even at 14-20 dB barrel. Fish scanline 32.5 dB and shift 22.7 dB still `detected` |
|
||||
| TrustMark-style micro-warp | 2026-08-21 | 0.25 px / 32 dB still 100% detect on TrustMark P; OpenAI elastic ~21 dB unreplicated |
|
||||
| OKLab random-codeword replacement as a quiet wipe | 2026-08-15 | On four public Google-oracle positives, replacing the period-16 tile at 0.95 dropped the *local* lattice score below 0.173 at ~54 dB. Gemini pixel verify never accepted the candidates (`Connecting to Verify AI`). Local-score intervention, not an official wipe |
|
||||
| PRC-style OKLab a/b noise plus spectral peak cleaning | 2026-08-15 | Independent high-pass OKLab noise at 32.6 dB moved the median local score by only `-0.0219`. Comb suppression moved one of four below threshold at 58 dB. Not a quiet official kill |
|
||||
| Rinne414 fixed-template subtraction | 2026-08-24 | The pinned pre-May GPT-Image2 residual was resized onto a current verified OpenAI image and subtracted at 0.25x, 0.5x, 1x, and 2x. Official SynthID stayed `detected` at 43.13, 37.49, 31.60, and 25.72 dB after metadata stripping. Even the dirtiest rung is no better than the existing 24-25 dB annulus-phase scramble. The template can spoof its own public correlator but does not cancel the current mark |
|
||||
| `newideas99` CNN gradient | 2026-08-24 | A 3x3 crop-grid objective spread the white-box perturbation over the full marked frame. The frozen ensemble fell from 0.99995 to 0.25956 at ±2/255 (45.92 dB) and to effectively zero at ±4/255 (40.30 dB). Official OpenAI SynthID stayed `detected` at ±1, ±2, ±4, and ±8/255, through 34.30 dB. The gradient attacks the surrogate's shortcut, not the production mark |
|
||||
| `reverse-SynthID` V4 Round-06 | 2026-08-24 | The advertised `final`/`nuke` path is SD-VAE regeneration plus elastic and affine warps, resize squeeze, color change, residual FFT subtraction, and a JPEG/noise chain, with PSNR floors of only 14/11 dB. The repository claims 20 manual Gemini-app successes but contains no manifest, tally, or per-image verdicts. Its four bundled older cleaned pairs are 45.7-50.2 dB, yet the reproduced V4 score rises on three and is nearly unchanged on the fourth. No callable Google pixel oracle was available for an independent Round-06 verdict. This is an unverified lossy regeneration/distortion stack, not a quiet pattern cancellation |
|
||||
| [`0xROOTPLS/DeSynth`](https://github.com/0xROOTPLS/DeSynth/tree/96db920731c2a3d04bf13163a5077b9a67706d1b) | 2026-08-24 | Qwen Image img2img at strength 0.25, followed by a Gaussian frequency split that restores the source high frequencies above sigma 1.95. The repository's public OpenAI original, default output, and edge-guided output all returned official `detected` in a current repeat after metadata stripping. The published `not found` claim does not reproduce against the current OpenAI oracle |
|
||||
| [`froggeric/gemini-watermark-and-synthid-remover`](https://github.com/froggeric/gemini-watermark-and-synthid-remover/tree/5918384ce403968de0560cefd889e50eba0163bc) | 2026-08-24 | SDXL img2img with a documented manual Google-verifier ladder. The author reports 7/8 clears at strength 0.08 and 9/9, including a double mark, at strength 0.10 with five effective denoise steps and PSNR 29-41 dB. The exact nine before/after verdict artifacts are not tracked, so this is useful external regeneration corroboration, not an independently reproduced oracle result |
|
||||
| [`atomantic/PortOS`](https://github.com/atomantic/PortOS/tree/b11a93e110262925c64a1b145a154ca87b340055) adversarial-jamming experiment | 2026-08-24 | Its own one-image manual OpenAI run found that quality-preserving phase noise, band noise, blur, and 0.70 resize squeeze stayed detected. Only visibly destructive phase perturbation cleared. A 0.85-0.90 resize caused repeated detector timeouts, which the repository correctly keeps separate from `not_detected`. This independently closes high-fidelity additive/phase jamming, but the source artifacts are not published |
|
||||
| Generic regeneration claims | 2026-08-24 | `mertizci/noai-watermark`, `BovineOverlord/Loyal-Bear`, `obaskly/NeuralBleach`, and `tymongumienik/unwatermark` are SD/CtrlRegen/ControlNet redraws. Their algorithms are already covered by the product's regeneration remainder and the CtrlRegen literature. README verdicts without a frozen manifest do not establish a quieter or more reliable operating point |
|
||||
| Add unmarked context (frame, pad, hstack, collage) | 2026-08-19 | Official `not_detected` is a presentation gate. White pad 40% linear stays `detected`; photo fill at the same geometry is `not_detected`. See detector identity/layout |
|
||||
| `cebeuq/Synthid-Bypass` diffusion reconstruction | 2026-08-13 | 12 public pairs. Lattice and Gemini Verify go quiet on the *after* images because the image was redrawn, not because a payload was deleted. Faces and text are a quality claim for regeneration, not a local wipe |
|
||||
|
||||
Oracle: `verify_openai_synthid` after AI-metadata strip. Seeds s1, s2, s3
|
||||
as in [detector research](synthid-detector-research.md). Raw files:
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/`.
|
||||
|
||||
## Band that actually carries the mark
|
||||
|
||||
Phase-randomize one octave at a time, preserve energy (E2, s1, replicated
|
||||
p16_32 on s2 and s3):
|
||||
|
||||
| Octave (period px) | Verdict |
|
||||
| --- | --- |
|
||||
| 2-4 | detected |
|
||||
| 4-8 | detected |
|
||||
| 8-16 | detected |
|
||||
| 16-32 | not_detected |
|
||||
| 32-64 | detected |
|
||||
| 64-128 | detected |
|
||||
|
||||
Destroying only 16-32 px periods silences the official decoder. The
|
||||
lattice band 8-16 px does not. On a marked gray flat the same scramble is
|
||||
51.1 dB `not_detected` because the band is almost empty. On a photograph
|
||||
the band holds the scene (24.3 dB). Replacing native 16-32 with a
|
||||
2x-pyramid prediction is 48.4 dB and still `detected`.
|
||||
|
||||
Fish additive jam, luma RMS `a`:
|
||||
|
||||
| Band | a | PSNR | Verdict |
|
||||
| --- | ---: | ---: | --- |
|
||||
| 16-32 | 0.06-4.0 | 51.2-33.6 | detected |
|
||||
| 16-32 | 8 | 27.7 | detected |
|
||||
| 16-32 | 16 | 21.9 | detected |
|
||||
| 16-32 | 24 | 18.6 | not_detected |
|
||||
| 4-8 | 16 | 21.9 | detected |
|
||||
| 4-8 | 24 | 18.6 | detected |
|
||||
|
||||
Preregistered close `a*_photo < 0.53` failed (`a* > 16`).
|
||||
|
||||
## Bayer and geometry (2026-08-22)
|
||||
|
||||
Frozen one-pass batch. Lattice scores on these OpenAI sources are already
|
||||
`indeterminate` and are not the label.
|
||||
|
||||
| Attack | s1 | s2 | s3 | fish |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| source | detected | detected | detected | detected |
|
||||
| Bayer bilinear | 32.7 detected | 28.4 detected | 37.1 detected | 34.7 detected |
|
||||
| Bayer VNG | 25.0 detected | 18.5 detected | 26.4 detected | 25.4 detected |
|
||||
| upscale 1.15 then Bayer | 33.9 detected | 29.8 detected | 38.4 detected | 36.0 detected |
|
||||
| barrel k1=0.06 | 19.9 detected | 14.2 detected | 20.2 detected | 18.5 detected |
|
||||
| scanline ±0.8 px | 31.5 detected | 26.7 detected | 35.7 detected | 32.5 detected |
|
||||
| shift 2 px | 23.1 detected | 15.9 detected | 24.7 detected | 22.7 detected |
|
||||
| 16-32 phase scramble | 24.6 not_detected | 19.0 not_detected | 25.2 not_detected | 23.2 not_detected |
|
||||
|
||||
Camera pipeline and mild geometry do not hit the decoder basis. A 2 px
|
||||
shift, which kills `pipeline_lattice`, left s1/s2/s3/fish `detected`.
|
||||
Scramble remains the residual kill on 4/4 sources.
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/bayer-geometry-2026-08-22/`.
|
||||
Manifest `status` is `oracle_complete` (2026-08-22 22:07).
|
||||
|
||||
## Attack ladders (OpenAI oracle)
|
||||
|
||||
Pilot 2026-08-17 (n=3): JPEG q40, 0.5x, noise8+jpeg85, 0.35x+jpeg50 all
|
||||
`detected` (s2 0.35x+jpeg50 at 20.1 dB still detected).
|
||||
|
||||
Kill and flip (s1), then replication 3/3:
|
||||
|
||||
| Rung | s1 | s2 | s3 |
|
||||
| --- | --- | --- | --- |
|
||||
| blur sigma 6 | detected | detected | detected |
|
||||
| blur sigma 7 | not_detected | not_detected | not_detected |
|
||||
| keep 0.50 linear | detected | detected | detected |
|
||||
| keep 0.40 linear | not_detected | not_detected | not_detected |
|
||||
| elastic ~24 dB | detected 24.6 | detected 17.9 | detected 26.1 |
|
||||
| elastic ~21 dB | not_detected 20.8 | not_detected 14.8 | detected 22.0 |
|
||||
| pad white 40% linear | detected | detected | detected |
|
||||
| hstack with photo | not_detected | not_detected | not_detected |
|
||||
| blend 25% marked | detected | detected | detected |
|
||||
| inset native 1:1 on 2x photo | not_detected | not_detected | not_detected |
|
||||
| JPEG q5 | detected | detected | detected |
|
||||
|
||||
Holds on 3/3: blur 6 vs 7, crop keep 0.50 vs 0.40, white pad 40%
|
||||
`detected`, hstack `not_detected`, blend 25% `detected`, JPEG q5
|
||||
`detected`. Elastic at ~21 dB does not hold. Crop keep-0.40 and hstack
|
||||
are presentation or support, not a quiet residual wipe.
|
||||
|
||||
Surround: photo fill at the white-pad geometries is `not_detected` on
|
||||
3/3; white pad at those sizes was `detected`. Unmarked photo strip flips
|
||||
between 25% width (`detected`) and 35% (`not_detected`). Photo frame 15%
|
||||
each side is `not_detected` on 3/3; 12% does not hold.
|
||||
|
||||
## Wild removers (oracle 2026-08-23)
|
||||
|
||||
Preregistered on s1, one pass, 25 s gap. Manifest:
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/wild-attacks-2026-08-23/oracle-manifest.json`.
|
||||
|
||||
| Attack | PSNR | Verdict | Note |
|
||||
| --- | ---: | --- | --- |
|
||||
| Replace 16-32 with a COCO photo's 16-32 | 25.2 | not_detected | Same annulus as cartesian scramble (24.6 dB). Substituting a camera band kills the decoder, 0.6 dB quieter than scramble on s1, not a new quality class |
|
||||
| File named polar-1632 | 25.6 | not_detected | Misnamed. The raster randomizes cartesian FFT phase in the 16-32 annulus, same family as scramble (24.6 dB). It is a scramble replicate, not a `(r, θ)` polar test |
|
||||
| 64-color median-cut palette | 35.6 | detected | Quieter than scramble and still marked. Posterize-4 already stayed `detected`; a smarter quantizer is not enough |
|
||||
| PIL RGB to CMYK to RGB | inf | not submitted | No-op on this PNG |
|
||||
|
||||
The 16-32 kill is luma phase in that annulus. Cartesian, Fourier-angle,
|
||||
and radial-phase all silence the decoder near 25 dB. A 90° sector of the
|
||||
same ring does not (27.5 dB `detected`). Cb-only and Cr-only 16-32
|
||||
scrambles stay `detected` at 44-45 dB. A foreign-scene transplant of the
|
||||
same band also silences the decoder near 25 dB. Palette, Baker-map, seam
|
||||
carve, Poisson, nested LSB, and ICC do not.
|
||||
|
||||
Follow-up 2026-08-23, s1, 25 s gap, `remaining-2026-08-23/oracle-manifest.json`:
|
||||
|
||||
| Attack | PSNR | Verdict |
|
||||
| --- | ---: | --- |
|
||||
| Fourier-angle 16-32 scramble | 24.1 | not_detected |
|
||||
| Cartesian 16-32 scramble (replicate) | 24.8 | not_detected |
|
||||
| Baker-map of the 16-32 band | 27.8 | detected |
|
||||
| Poisson noise | 30.7 | detected |
|
||||
| Nested LSB in blue | 55.9 | detected |
|
||||
| Seam carve 8 | 27.7 | detected |
|
||||
| ICC sRGB rewrite | inf | not submitted, no-op |
|
||||
|
||||
Non-local codecs, 2026-08-23, s1, 25 s gap:
|
||||
|
||||
| Attack | PSNR | Verdict |
|
||||
| --- | ---: | --- |
|
||||
| HEIF q80 | 46.3 | detected |
|
||||
| HEIF q50 | 39.3 | detected |
|
||||
| AV1 CRF 32 | 37.2 | detected |
|
||||
| Print-scan simulation | 24.95 | detected |
|
||||
|
||||
Physical print-scan is still blocked unattended (Brother DCP-L2520DW idle and
|
||||
accepting, no `scanimage`, no ImageCapture pyobjc). Face-gated scramble is
|
||||
unnecessary: Haar on s1 put *more* 16-32 energy on faces. Generic 25 dB is
|
||||
not the kill: this simulation stays `detected` at the PSNR where 16-32
|
||||
phase scramble does not.
|
||||
|
||||
Waveform-shell splits, 2026-08-23, s1, 25 s gap,
|
||||
`waveforms-shells-2026-08-23/oracle-manifest.json`:
|
||||
|
||||
| Attack | PSNR | Verdict |
|
||||
| --- | ---: | --- |
|
||||
| Y-only 16-32 scramble | 24.6 | not_detected |
|
||||
| Cb-only 16-32 scramble | 45.0 | detected |
|
||||
| Cr-only 16-32 scramble | 43.9 | detected |
|
||||
| 90° Fourier sector of 16-32 | 27.5 | detected |
|
||||
| Radial-phase-only 16-32 | 25.0 | not_detected |
|
||||
|
||||
## External literature (surveyed 2026-08-23)
|
||||
|
||||
Primary sources. Detector papers live in
|
||||
[detector research](synthid-detector-research.md). Forensic stealth of
|
||||
regeneration is already in [synthid.md](synthid.md#23-removal-attacks-and-forensic-detectability).
|
||||
|
||||
| Source | Attack | Against SynthID? | Map to this campaign |
|
||||
| --- | --- | --- | --- |
|
||||
| Zhao et al., [arXiv:2306.01953](https://arxiv.org/abs/2306.01953) (NeurIPS 2024) | Add noise, then denoise or regenerate (VAE / diffusion). Pixel-level invisible marks are provably removable. Semantic watermarks proposed as the alternative | Open post-hoc schemes, not production SynthID | This is the family our product uses (`qwen-zimage` / `sdxl-zimage`). Gowal trains SynthID-O against *weak* VAE regeneration. Our foreign-VAE round-trip at 22.3 dB stayed `detected`. Regeneration works when it redraws, not when it is a light codec |
|
||||
| Liu et al., [arXiv:2410.05470](https://arxiv.org/abs/2410.05470) (CtrlRegen, ICLR 2025) | Controllable diffusion from clean noise, with a knob on how many noise steps to add | SOTA open watermarks | Same family. Goonatilake later finds CtrlRegen+ the *most* forensically detectable remover (AUROC 0.9999) |
|
||||
| Kassis and Hengartner, [arXiv:2405.08363](https://arxiv.org/abs/2405.08363) (UnMarker, IEEE S&P 2025) | No decoder feedback. Two adversarial spectral optimizations. Breaks even some semantic watermarks (best remaining detection 43%) | Not production SynthID | Spectral disruption without an oracle is the honest analog of our 16-32 scramble, except UnMarker is optimized and we used a one-octave phase shuffle. Goonatilake: UnMarker TPR 98.28% at 0.1% FPR as a *forensic* leftover |
|
||||
| Tallam et al., [arXiv:2505.08234](https://arxiv.org/abs/2505.08234) (SemanticRegen) | Partial, label-free regeneration of main objects | Tree-Ring, StegaStamp, StableSig, DWT/DCT. Not SynthID | Partial redraw. Our collage / photo-pad `not_detected` is a presentation gate, not this attack |
|
||||
| Cao et al., [arXiv:2608.10166](https://arxiv.org/abs/2608.10166) (MarkNull, USENIX Security 2026) | On-manifold latent decorrelation via a public diffusion proxy. Claims 100% on 20 Imagen-3 Gemini-verify images. PSNR 25.36 dB, SSIM 0.80 | Small Gemini-verify set | Independent evidence that a no-box latent reconstruction can confuse Gemini. Does not meet this project's 40 dB / 0.99 SSIM release gate. Still generation, not a pixel-only wipe |
|
||||
| Goonatilake and Ateniese, [arXiv:2605.09203](https://arxiv.org/abs/2605.09203) | Six removers all leave a forensic residue a ResNet-50 sees at >98% TPR @ 1% FPR | Applies to UnMarker, Zhao's WatermarkAttacker, CtrlRegen+ | Defeating `verify-openai-synthid` is not deniability. This is the product remainder |
|
||||
| An et al., [arXiv:2401.08573](https://arxiv.org/abs/2401.08573) (WAVES, ICML 2024) | 26 attacks on StegaStamp, Stable Signature, Tree-Ring. Regeneration, not JPEG, is the attack that matters. StegaStamp TPR@1%FPR 1.00 to 0.01; Tree-Ring 0.99 to 0.12 | Open watermarks | Protocol. Our blur-sigma-7 and 16-32 scramble are closer to WAVES "distortion" than to regeneration |
|
||||
| Wen et al., [arXiv:2305.20030](https://arxiv.org/abs/2305.20030) (Tree-Ring) | In-generation Fourier pattern in initial noise | Not SynthID | Kill is DDIM inversion plus latent wipe, not a pixel scramble. Lin and Juarez [arXiv:2506.10502](https://arxiv.org/abs/2506.10502) (USENIX 2025) remove it from public knowledge |
|
||||
| Fernandez et al., Stable Signature, ICCV 2023 | Fine-tuned VAE decoder | Not SynthID | Regeneration that replaces the decoder is in-family. Our foreign VAE at 22.3 dB did not kill OpenAI SynthID, which is the expected mismatch |
|
||||
| Google to [The Verge, 2026-04-14](https://www.theverge.com/ai-artificial-intelligence/911579/google-synthid-ai-watermarking-system-reverse-engineered) | reverse-SynthID does not systematically remove SynthID | Google spokesperson on the spectral bypass | Agrees with our V4 audit and with Bypass: reconstruction or decoder confusion, not a payload delete |
|
||||
|
||||
Quiet pixel-only removal is not a published result on production
|
||||
OpenAI or Google SynthID. Every paper that "beats" a watermark either
|
||||
redraws the image, attacks an open encoder, or reports PSNR in the same
|
||||
19-25 dB band as our 16-32 scramble.
|
||||
|
||||
## Product remainder for removal
|
||||
|
||||
Invisible removal in this package regenerates through `qwen-zimage` or
|
||||
`sdxl-zimage`. It is lossy. There is no shipped pixel-only OpenAI or
|
||||
Google SynthID wipe. Do not add Bayer or geometry as remover arms.
|
||||
Reference in new issue
Block a user