From 07b07634cf522de2bf9d86f049edc630e6223975 Mon Sep 17 00:00:00 2001 From: Victor Kuznetsov Date: Tue, 1 Sep 2026 11:37:32 -0700 Subject: [PATCH] Merge the docs/arxiv-paper-review research line into main (#95) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add rigorous SynthID research and evaluation harness * Record SynthID cross-epoch validation results * Reset SynthID detector hypotheses to paired training * Add paired wavelet and spectral SynthID probes * Record closed-corpus SynthID detector ablations * Add registered SynthID phase probing * Add periodic SynthID tile probing * Add calibrated SynthID pixel detector * Harden scale-registered SynthID detection * Record SynthID phase and channel rescue findings * Add production SynthID routing and OpenAI verification * Reframe periodic pixel route as pipeline lattice; confirm and harden detection The frozen periodic experts read an origin-anchored generation-pipeline lattice destroyed by a crop off the tile grid, not the crop-robust SynthID mark. Route the pixel result as an experimental pipeline_lattice signal kept out of the watermark inventory, and carry the crop sensitivity in every verdict envelope. Add split-patch phase/amplitude/codeword confirmation for registered-v3, affine-lattice and cyclostationary research probes, and timeout/retry/error-taxonomy hardening for the official OpenAI verification path. * Record pipeline-lattice reframing and registered-v3 phase lock Document that the pixel route detects an origin-anchored lattice rather than the watermark, and record the registered-v3 two-pixel phase-lock measurement: a 2px diagonal crop killed all 36 tested detections (28 foreign-generator, 8 Google) with recovery only at offsets that are multiples of four. Update README, CLI, Python API, supported-signals, known-limitations, module-internals, the SynthID reference, and the detector research plan. * Split SynthID campaign docs by detector, classifiers, and removal Keep Model 1 as a research AI-versus-camera result outside the library. Retire the mixed oracle-ladders page to a routing hub and mark the plan file as a chronological archive. * Keep 2026-08-22 SynthID campaign results in research docs only Drop Bayer, Model 1, and lattice re-check numbers from user guides and module internals. The library surface is unchanged. * Unplug lattice detector from package, expand research docs * Remove lattice detector sources from the public package * Document SynthID GitHub sweep findings * Document general AI classifier sweep * Document modern-negative taxonomy audit and classifier doc split Split general AI-classifier research into ai-generated-image-classifiers.md and record the modern-negative campaign: Unsplash/Openverse date-clean cells, contamination controls, quarter-hard paired result, closed linear veto and taxonomy-continuation arms. pre-commit: 1) maintain.sh - exit 1, known uv-secure lightning advisory with no upstream fix (same triage as main); core checks separately green (ruff, format, pyright, 1564 tests); 2) /simplify - docs-only single pass, no findings; 3) docs sync - split references updated everywhere, none stale; 4) CLAUDE.md - routing line already updated, compliant * Record wild-AI audit, Meta provider class, and stock-negative expansion The wild vendor-flagged AI cell (300 stratified rows) puts Model 1 recall at 69.7% on unknown-renderer stock AI; the stock-negative harvest triples the modern fashion/product cells and confirms the combined-pool veto control; the Meta muse-image corpus doubles to 132 rows with its margin sweep; a per-channel cv2 reference fixes the latent fold test under cv2 4.10.0. pre-commit: 1) maintain.sh - exit 1, known uv-secure lightning advisory with no upstream fix; core checks separately green (ruff, format, pyright, 1665 tests); 2) /simplify - docs-only single pass, no findings; 3) docs sync - new run references point at the gitignored research store, none stale; 4) CLAUDE.md - compact, no changes needed * Close representation-change campaign: both candidates falsified Pre-registered gates (AI-test >=1772, fresh OI <=50, Kodak 0, FLUX >=288, EvalGEN >=95) rejected DINOv2-L ridge head (1427/1847, 162/3000, 1/24, 29/300, 5/100) and the paired-reconstruction FFT tower (14/1847, 12/3000, 0/24, 0/300, 0/100; contract learned at 0.949 pairwise separation but no transfer to production generators). Campaign verdict: general classifier beyond local scale as pursued; product stays on documented remainder. pre-commit: 1) maintain.sh - exit 1, known uv-secure lightning advisory with no upstream fix; core checks green minutes ago (ruff, format, pyright, 1665 tests); 2) /simplify - docs-only single pass, no findings; 3) docs sync - verdict mirrored in data/research/REPRESENTATION-CAMPAIGN.md, no stale refs; 4) CLAUDE.md - no changes * Record full failure map, H2 router falsification, and H3 process detector closure 20 measured negative domains with batch-2 (memes, whiteboards, tickets, book pages). H2 domain router falsified: 97.9% of AI-test looks structured, every family 94-100%. H3 process detector on 369 contrastive pairs from three generators (OpenAI gpt-image-1-mini, Meta muse-image-1.0, Gemini flash-image) achieves 0.909 pairwise AUC on training pairs but transfers at AUC 0.55-0.60 to production AI; human receipts score higher than any AI family. The generator-specific spectral trace does not generalize across three generators, closing the process-trace family in its current form. pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1665 tests); 2) /simplify - docs + research scripts, single pass, no findings; 3) docs sync - all artifacts in data/research/, no stale refs; 4) CLAUDE.md - no changes * Record H4-H6 research hypotheses: MLP probe, spectral invariant, TTA voting H4 (MLP on frozen CLIP): 98.6% AI-test recall but 15.8% photo FPR — same trade-off curve as quarter-hard. H5 (spectral invariant from CVPR 2025): 2.4% recall, the 1/f^alpha profile does not discriminate current generators. H6 (TTA voting): fixes museum FP but not documents/UI/fashion, loses 9% recall. All three falsified against frozen gates; ten total closed hypotheses now document the CLIP-L representation limit from every angle. Also records the provider cascade across all 20 domains (69.8% stray on documents mirrors the detector failure map) and the full Y-phase campaign PSNR data awaiting oracle rate-limit reset. pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1665 tests); 2) /simplify - docs-only single pass, no findings; 3) docs sync - all artifacts in data/research/, no stale refs; 4) CLAUDE.md - no changes * Fix Y-phase frequency convention, record H4-H6 and campaign results The initial Y-phase campaign targeted FFT bins 16-32 instead of spatial periods 16-32 px (1/32 to 1/16 cycles/px). After the fix, the full-octave Y-phase scramble reproduces the known kill (not_detected at 20.5 dB on s1 at native 1122x1402). Partial sub-bands are visually invisible (42.5 dB) but stay detected: the mark is distributed across the full octave, not concentrated. Point intervention is not viable within this frequency band. H4 MLP probe: 98.6% recall, 15.8% FPR (same trade-off wall). H5 spectral invariant: 2.4% recall (1/f^alpha does not discriminate). H6 TTA voting: fixes museum FP but not documents/UI/fashion. pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (1665 tests, verified minutes ago); 2) /simplify - docs-only; 3) docs sync - all artifacts synced; 4) CLAUDE.md - no changes * Add TC260 fifth class and per-class margin calibration to provider cascade Five-head cascade (openai/google/tc260/meta/no_ai): openai 68.8%, google 76.6%, tc260 63.8%, meta 71.8% mean recall at uniform margin 0.50. Per-class margins calibrated against 1.67% photo budget: openai 0.31, google 0.29, tc260 0.39, meta 0.55. TC260 test recall 71.4% at calibrated margins on 196 held-out images. Chinese-AIGC is a viable pixel class. pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (verified within the hour); 2) /simplify - research script, single pass; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes * Record universal detector v3-v8: diverse training shifts the curve Fresh MLP on frozen CLIP embeddings with 49 negative domains (12,258 images) and 5,043 AI positives achieves 93.0% AI-test recall with 39/49 domains at zero false positives. Digital art FP dropped from 99.3% (Model 1) to 24.8%, fashion from 62-71% to 12%, UI from 99.3% to 12%. Photo FPR 0.7%. The improvement comes from training data diversity on frozen embeddings, not representation change (v5 fine-tune proved this by undoing the gains). Also adds .local-eval to ruff exclude so the gate covers only tracked files. pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - docs + research; 3) docs sync - all artifacts in data/research/; 4) CLAUDE.md - no changes * Add three-tier ensemble detector: DEFINITELY/POSSIBLY/LIKELY_HUMAN Combines Model 1 and v8 into calibrated confidence tiers. DEFINITELY_AI (both models agree) achieves 79.6% recall at 0.50% photo FPR with 46/49 domains at zero FP. The union reaches 93.3% recall at 1.80% photo FPR with Kodak 0/24. First detector working across ALL content types with per-tier calibrated error rates. pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - docs + ensemble; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes * Add provider v2: diversity training eliminates document leakage Same diversity-training insight applied to the provider cascade. Adding 3,769 structured negative features to training drops document/receipt leakage from 69.8% to 5.1%, UI from 10% to 4.3%, fashion from 5-15% to 0-3%. AI recall improves: OpenAI 84% (was 75%), Google 88.8% (was 79%). Photo leakage 1.4%. Worst regression: memes 47.9% (was 27%). pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - provider v2; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes * Add universal v11: full taxonomy, 136 domains, 124 at zero FP Master data plan mapped 148 visual domains. Massive collection (3,343 images, 73 domains from Openverse) plus gap-filler (724 images from Safebooru anime/manga) expanded training to 19,768 images across 136 negative domains. Universal v11 (MLP-512 + Model 1 ensemble): DEFINITELY 90.3% recall at 0.50% photo FPR, ANY 95.6% recall, 124/134 domains at exactly 0% FP, worst FP 13.8% (digital art). New domains opened: anime, manga, chibi, mecha, vintage photos, banknotes, newspapers, magazines, movie posters, antique maps, watercolor/oil originals, patterns, cosmetics, real estate, 3D renders, and 60+ more. pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - docs + research; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes * Confirm representation limit: v12 fine-tune destroys structured FP despite 6x data Fine-tuning CLIP tower on 19,518 images achieves 95.1% standalone recall but returns to 100% FP on art/UI and 14/59 zero-FP domains (vs 124/134 for frozen v11). This is a representation property, not a data issue. Frozen CLIP + MLP (v11) is the only working architecture. Also records provider v3 trained on all negative domains. pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - docs + research; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes * Document v13 plateau: 128/136 zero FP, frozen-CLIP ceiling confirmed v13 with top-plan expansion (1,169 new images) improves domain coverage to 128/136 at zero FP and reduces fashion FP to 5.3%, but DEFINITELY recall plateaus at 89.6% and art FP at 15.9%. The frozen CLIP + MLP architecture has reached its ceiling. pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (verified minutes ago, 1731 tests); 2) /simplify - docs-only; 3) docs sync - v13 recorded; 4) CLAUDE.md - no changes * Document the 2026-08-31 photo-classifier freeze. Record the AI-versus-camera contract, Meta v3 hold-out, Flickr UI replacement, and that receipts stay out of the supported domain. * Scrub Spaces catalog details from classifier research docs * Record receipt layout-gate probe for the universal detector --- .claude/rules/development.md | 5 + .env.example | 4 + CLAUDE.md | 2 +- README.md | 23 + data/README.md | 6 + data/synthid/README.md | 5 +- data/synthid/research-manifest.md | 91 + docs/ai-generated-image-classifiers.md | 18 +- docs/cli.md | 41 +- docs/controlnet-removal-pipeline-research.md | 19 +- docs/index.md | 6 +- docs/installation.md | 16 +- docs/known-limitations.md | 55 +- docs/module-internals.md | 377 +- docs/python-api.md | 21 +- docs/supported-signals.md | 25 +- docs/synthid-detector-removal-plan.md | 6409 +++++++++++++++++ docs/synthid-oracle-ladders.md | 54 + docs/synthid.md | 528 +- docs/verification-plan.md | 9 +- pyproject.toml | 15 +- scripts/synthid_adaptive_carrier_suppress.py | 197 + scripts/synthid_affine_lattice_probe.py | 1636 +++++ scripts/synthid_color_space_probe.py | 353 + scripts/synthid_conformal_cascade.py | 352 + scripts/synthid_confound_probe.py | 478 ++ scripts/synthid_consensus_probe.py | 259 + scripts/synthid_cyclostationary_probe.py | 228 + scripts/synthid_ensemble_attack.py | 223 + scripts/synthid_ensemble_detector.py | 188 + scripts/synthid_fragment_attack.py | 243 + scripts/synthid_hybrid_attack.py | 140 + scripts/synthid_oracle_batch.py | 467 ++ scripts/synthid_periodic_tile.py | 23 + scripts/synthid_periodic_tile_ablation.py | 288 + scripts/synthid_periodic_tile_probe.py | 273 + scripts/synthid_phase_carrier.py | 384 + scripts/synthid_phase_registration.py | 92 + scripts/synthid_pixel_attack.py | 199 + scripts/synthid_research_inventory.py | 170 + scripts/synthid_research_manifest.py | 347 + scripts/synthid_routed_detector.py | 69 + scripts/synthid_routed_expert_bank.py | 151 + scripts/synthid_runtime/__init__.py | 5 + .../synthid_runtime/_synthid_confirmation.py | 288 + .../synthid_runtime/_synthid_registered.py | 664 ++ scripts/synthid_runtime/synthid_detector.py | 542 ++ .../synthid_periodic_tile_2048_v1.npz | Bin 0 -> 7501 bytes scripts/synthid_runtime_expert_scores.py | 119 + scripts/synthid_spectral_probe.py | 524 ++ scripts/synthid_tile_attack.py | 147 + scripts/synthid_v3_carrier_subtract.py | 148 + scripts/synthid_v3_codebook_probe.py | 344 + src/remove_ai_watermarks/__init__.py | 13 + src/remove_ai_watermarks/assets/__init__.py | 2 +- src/remove_ai_watermarks/cli.py | 75 +- src/remove_ai_watermarks/metadata.py | 5 +- src/remove_ai_watermarks/openai_provenance.py | 351 + tests/test_api.py | 11 + tests/test_cli.py | 83 + tests/test_openai_provenance.py | 429 ++ tests/test_packaging.py | 3 +- .../test_synthid_adaptive_carrier_suppress.py | 77 + tests/test_synthid_affine_lattice_probe.py | 398 + tests/test_synthid_color_space_probe.py | 105 + tests/test_synthid_confirmation.py | 83 + tests/test_synthid_conformal_cascade.py | 240 + tests/test_synthid_confound_probe.py | 243 + tests/test_synthid_consensus_probe.py | 73 + tests/test_synthid_cyclostationary_probe.py | 65 + tests/test_synthid_detector.py | 787 ++ tests/test_synthid_ensemble_attack.py | 90 + tests/test_synthid_ensemble_detector.py | 86 + tests/test_synthid_fragment_attack.py | 92 + tests/test_synthid_hybrid_attack.py | 50 + tests/test_synthid_oracle_batch.py | 212 + tests/test_synthid_periodic_tile_ablation.py | 122 + tests/test_synthid_periodic_tile_probe.py | 135 + tests/test_synthid_phase_carrier.py | 165 + tests/test_synthid_pixel_attack.py | 62 + tests/test_synthid_research_inventory.py | 107 + tests/test_synthid_research_manifest.py | 252 + tests/test_synthid_routed_detector.py | 78 + tests/test_synthid_routed_expert_bank.py | 138 + tests/test_synthid_runtime_expert_scores.py | 97 + tests/test_synthid_spectral_probe.py | 182 + tests/test_synthid_tile_attack.py | 68 + tests/test_synthid_v3_carrier_subtract.py | 66 + tests/test_synthid_v3_codebook_probe.py | 116 + uv.lock | 169 +- 90 files changed, 22498 insertions(+), 102 deletions(-) create mode 100644 data/synthid/research-manifest.md create mode 100644 docs/synthid-detector-removal-plan.md create mode 100644 docs/synthid-oracle-ladders.md create mode 100644 scripts/synthid_adaptive_carrier_suppress.py create mode 100644 scripts/synthid_affine_lattice_probe.py create mode 100644 scripts/synthid_color_space_probe.py create mode 100644 scripts/synthid_conformal_cascade.py create mode 100644 scripts/synthid_confound_probe.py create mode 100644 scripts/synthid_consensus_probe.py create mode 100644 scripts/synthid_cyclostationary_probe.py create mode 100644 scripts/synthid_ensemble_attack.py create mode 100644 scripts/synthid_ensemble_detector.py create mode 100644 scripts/synthid_fragment_attack.py create mode 100644 scripts/synthid_hybrid_attack.py create mode 100644 scripts/synthid_oracle_batch.py create mode 100644 scripts/synthid_periodic_tile.py create mode 100644 scripts/synthid_periodic_tile_ablation.py create mode 100644 scripts/synthid_periodic_tile_probe.py create mode 100644 scripts/synthid_phase_carrier.py create mode 100644 scripts/synthid_phase_registration.py create mode 100644 scripts/synthid_pixel_attack.py create mode 100644 scripts/synthid_research_inventory.py create mode 100644 scripts/synthid_research_manifest.py create mode 100644 scripts/synthid_routed_detector.py create mode 100644 scripts/synthid_routed_expert_bank.py create mode 100644 scripts/synthid_runtime/__init__.py create mode 100644 scripts/synthid_runtime/_synthid_confirmation.py create mode 100644 scripts/synthid_runtime/_synthid_registered.py create mode 100644 scripts/synthid_runtime/synthid_detector.py create mode 100644 scripts/synthid_runtime/synthid_periodic_tile_2048_v1.npz create mode 100644 scripts/synthid_runtime_expert_scores.py create mode 100644 scripts/synthid_spectral_probe.py create mode 100644 scripts/synthid_tile_attack.py create mode 100644 scripts/synthid_v3_carrier_subtract.py create mode 100644 scripts/synthid_v3_codebook_probe.py create mode 100644 src/remove_ai_watermarks/openai_provenance.py create mode 100644 tests/test_openai_provenance.py create mode 100644 tests/test_synthid_adaptive_carrier_suppress.py create mode 100644 tests/test_synthid_affine_lattice_probe.py create mode 100644 tests/test_synthid_color_space_probe.py create mode 100644 tests/test_synthid_confirmation.py create mode 100644 tests/test_synthid_conformal_cascade.py create mode 100644 tests/test_synthid_confound_probe.py create mode 100644 tests/test_synthid_consensus_probe.py create mode 100644 tests/test_synthid_cyclostationary_probe.py create mode 100644 tests/test_synthid_detector.py create mode 100644 tests/test_synthid_ensemble_attack.py create mode 100644 tests/test_synthid_ensemble_detector.py create mode 100644 tests/test_synthid_fragment_attack.py create mode 100644 tests/test_synthid_hybrid_attack.py create mode 100644 tests/test_synthid_oracle_batch.py create mode 100644 tests/test_synthid_periodic_tile_ablation.py create mode 100644 tests/test_synthid_periodic_tile_probe.py create mode 100644 tests/test_synthid_phase_carrier.py create mode 100644 tests/test_synthid_pixel_attack.py create mode 100644 tests/test_synthid_research_inventory.py create mode 100644 tests/test_synthid_research_manifest.py create mode 100644 tests/test_synthid_routed_detector.py create mode 100644 tests/test_synthid_routed_expert_bank.py create mode 100644 tests/test_synthid_runtime_expert_scores.py create mode 100644 tests/test_synthid_spectral_probe.py create mode 100644 tests/test_synthid_tile_attack.py create mode 100644 tests/test_synthid_v3_carrier_subtract.py create mode 100644 tests/test_synthid_v3_codebook_probe.py diff --git a/.claude/rules/development.md b/.claude/rules/development.md index 4eda3e3..2994149 100644 --- a/.claude/rules/development.md +++ b/.claude/rules/development.md @@ -106,6 +106,11 @@ rules follow, and both were broken in practice before they were written down: - Detection and the removal mask must read ONE sweep. The winning box travels on `TextMarkDetection.match_box` and the registry threads the detection into the mask builder; a mask path that re-runs its own sweep is how the two drift apart. +- When the research lattice runtime in `scripts/synthid_runtime/` routes by image + geometry, preserve the returned `SynthIDDetection.detector` in score manifests + and downstream routers. Record an inactive expert as explicitly unsupported; + never attribute a routed large-image score to the fixed expert. This expert is + not part of the public package. The C2PA manifest-store JSON is NOT stable across reads: the reader regenerates manifest URNs and instance ids. Compare the derived `c2pa_info`, never the raw store. diff --git a/.env.example b/.env.example index 3f7e0e8..8cd9964 100644 --- a/.env.example +++ b/.env.example @@ -1,3 +1,7 @@ # HuggingFace token (optional; only needed for gated/private models) # Get yours at: https://huggingface.co/settings/tokens # HF_TOKEN= + +# OpenAI API key (optional; only for explicit verify-openai-synthid uploads) +# Create and manage keys at: https://platform.openai.com/api-keys +# OPENAI_API_KEY= diff --git a/CLAUDE.md b/CLAUDE.md index e14d482..edf46ec 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,7 +39,7 @@ Before a release, read [`docs/release-and-distribution.md`](docs/release-and-dis [`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem. -Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Pixel photo classification is [`docs/photo-classify.md`](docs/photo-classify.md); it is not `identify`. Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md). +Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Pixel photo classification is [`docs/photo-classify.md`](docs/photo-classify.md); it is not `identify`. Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md). The pre-split chronological archive is [`docs/synthid-detector-removal-plan.md`](docs/synthid-detector-removal-plan.md). ## Data safety diff --git a/README.md b/README.md index 8e007e6..ce9cde9 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,7 @@ removal. | --- | --- | --- | | Find provenance signals and watermarks | `identify` | No | | Classify a photograph from pixels (opt-in, not provenance) | `classify` | No | +| Verify supported OpenAI SynthID from pixels with the official remote API | `verify-openai-synthid` | No | | Remove known visible AI marks | `visible` | No | | Erase a region you select | `erase` | No | | Strip AI metadata | `metadata` | No | @@ -56,6 +57,7 @@ its linked C2PA manifest; metadata stripping alone removes only the manifest. | --- | --- | | Metadata inspection and stripping | `remove-ai-watermarks` | | Photograph AI-versus-camera classification | `remove-ai-watermarks[classify]` | +| Official remote OpenAI SynthID verification | `remove-ai-watermarks[verify]` | | Visible detection and removal | `remove-ai-watermarks[visible]` | | Visible video processing | `remove-ai-watermarks[video]` | | Video SynthID removal | `remove-ai-watermarks[video,diffusion]` | @@ -92,6 +94,24 @@ remove-ai-watermarks classify image.png Guide: [photo pixel classification](docs/photo-classify.md). +Signed provenance is the supported route for SynthID and `identify` reads it. +There is no local SynthID pixel detector in the package. Research on a +periodic lattice expert is in `scripts/synthid_runtime/` and +[synthid-detector-research.md](docs/synthid-detector-research.md). + +For supported OpenAI images, the optional official verifier provides a pixel +watermark verdict: + +```bash +uv tool install --force "remove-ai-watermarks[verify]" +remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload +``` + +The command removes AI provenance metadata from a temporary copy, verifies that +the decoded pixels are unchanged, uploads only that copy to OpenAI, and consumes +only the independent SynthID response. It never runs implicitly from `identify`. +An API key, endpoint access, and explicit upload acknowledgement are required. + For visible watermark removal, install the pixel dependencies: ```bash @@ -381,6 +401,9 @@ import remove_ai_watermarks as raiw result, removed = raiw.remove_visible("watermarked.png", "clean.png") print(removed) +openai_synthid = raiw.verify_openai_synthid("image.png", acknowledge_upload=True) +print(openai_synthid.status) + provenance = raiw.identify_video("input.mp4") report = raiw.inspect_video_metadata("input.mp4") complete = raiw.remove_video_all("input.mp4", "clean.mp4") diff --git a/data/README.md b/data/README.md index db530e5..d4549e0 100644 --- a/data/README.md +++ b/data/README.md @@ -26,6 +26,12 @@ data/ ## Storage rules +Large local-only research data (the 211 GB Spaces corpus and the frozen +checkpoints, research corpora, and campaign reports) lives under gitignored +paths documented in `data/research/README.md` (ignored via `.git/info/exclude`, +never committed). Worktree `.local-eval/` directories are scratch copies; the +durable store is `data/research/` in the main checkout. + 1. Store each binary image once. Evaluation manifests and documentation point to its canonical location. 2. Put executable test fixtures in `fixtures/`. diff --git a/data/synthid/README.md b/data/synthid/README.md index 565a2ab..1a11286 100644 --- a/data/synthid/README.md +++ b/data/synthid/README.md @@ -54,7 +54,10 @@ versions, or runtime settings. Ground-truth quality, strongest first: - `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models). -- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images). +- `openai-verify` — checked through the OpenAI web verifier. This is the + historical manifest label for existing rows. OpenAI now also documents a + Content Provenance API with a separate SynthID result; add a distinct manifest + value before recording API-derived labels rather than silently reusing this one. - `synthid-portal` — checked via Google's SynthID Detector portal. - `c2pa-metadata` — supported provenance evidence (Google AI C2PA, or OpenAI C2PA with an explicit `c2pa.watermarked.*` action). Weaker than a provider diff --git a/data/synthid/research-manifest.md b/data/synthid/research-manifest.md new file mode 100644 index 0000000..1e1ac27 --- /dev/null +++ b/data/synthid/research-manifest.md @@ -0,0 +1,91 @@ +# Private SynthID research manifest + +The research manifest is a local CSV for detector and pixel-only removal +experiments. It is intentionally separate from `manifest.csv`, which describes +the small public regression corpus. + +Store the CSV and its media below `.local-eval/synthid/`. Keep one manifest per +detector target, for example `google-manifest.csv` and `openai-manifest.csv`. +Separate manifests may reference the same external negative bytes without +duplicating an artifact inside either provider's split graph. Do not commit +private media, corpus sizes, oracle sessions, or provider-access details. + +Before assigning labels, build a label-free inventory. It records byte and +decoded-pixel hashes, exact duplicates, geometry, and format, but deliberately +has no provider, oracle, outcome, or split columns: + +```bash +uv run python scripts/synthid_research_inventory.py \ + --root .local-eval/synthid negatives google openai \ + --inventory-out .local-eval/synthid/inventory.csv +``` + +An inventory row becomes a manifest row only after its source, detector target, +label evidence, group lineage, and split are established independently. A +directory name is not evidence and must not be promoted mechanically. + +Audit each local manifest with: + +```bash +uv run python scripts/synthid_research_manifest.py \ + .local-eval/synthid/google-manifest.csv --verify-files +``` + +Once a provider manifest has ordinary positives and negatives in train, +validation, and locked-test splits, run the D1 confound challenge: + +```bash +uv run --extra pixels python scripts/synthid_confound_probe.py \ + .local-eval/synthid/google-manifest.csv \ + --target-provider google \ + --report-out .local-eval/synthid/google-d1-confounds.json +``` + +D1 excludes candidate, sham, and source-control rows. Its `container`, +`thumbnail`, and `canonical` baselines intentionally measure how well export +fields, geometry, and coarse generator/content style can imitate detection. A +report is evidence-ready only when the locked test includes a same-provider hard +negative and the temporal split contains both labels. Evidence readiness does +not mean the D1 gate passed; a candidate signal still has to beat the frozen +canonical baseline on those controls. + +## Columns + +| Column | Meaning | +| --- | --- | +| `artifact_sha256` | SHA-256 of the exact file submitted or measured. | +| `pixel_sha256` | SHA-256 of decoded RGB bytes, used to catch lossless duplicates. | +| `artifact_path` | Safe path relative to the manifest. | +| `parent_sha256` | Exact parent artifact for a derivative, empty for an original. | +| `group_id` | Leakage boundary shared by an original and all semantic or transformed siblings. | +| `target_provider` | Detector target, `openai` or `google`. | +| `source_provider` | Actual source family: `openai`, `google`, `camera`, `other_ai`, `synthetic`, or `editor`. | +| `surface` | Product or export surface. | +| `model_epoch` | Model family plus a dated epoch when the exact version is unavailable. | +| `generation_session` | Groups outputs created in one provider session. | +| `content_stratum` | Predeclared content class. | +| `width`, `height`, `format` | Decoded geometry and file format. | +| `transform` | `original` or the reproducible transform applied to `parent_sha256`. | +| `split` | `discovery`, `train`, `validation`, `test`, or `temporal`. | +| `c2pa_outcome` | C2PA result, recorded independently from the pixel signal. | +| `synthid_outcome` | `detected`, `not_detected`, `indeterminate`, `refused`, or `not_checked`. | +| `verified_via` | Matching provider oracle, external source evidence, or `none`. | +| `evidence_reference` | Stable URL or local evidence-record reference required for `source-evidence`; never infer it from a directory name. | +| `oracle_session` | Groups allowed checks made in one session. | +| `oracle_role` | `ordinary`, `source_control`, `candidate`, or `sham`. | +| `captured_at`, `oracle_checked_at` | Timezone-aware ISO-8601 timestamps. | +| `notes` | Verbatim context that does not fit a structured column. | + +`source-evidence` may establish an ordinary external negative, but never a +positive or a same-provider negative. Provider positives and same-provider hard +negatives require the matching provider verifier. An `indeterminate`, `refused`, +or unchecked row may remain in `discovery`; it cannot enter a train, validation, +test, or temporal split. Every `source-evidence` row must retain the source URL +or evidence-record reference that establishes the claim. + +The auditor also rejects duplicate artifact hashes, identical decoded pixels in +different groups, derivatives without parents, cross-provider parentage, +lineage cycles, and any group crossing split boundaries. A `not_detected` +candidate or sham is valid only when the same provider, group, and oracle +session contains a detected `source_control`. A session that misses its control +cannot establish removal, even when its candidate response is negative. diff --git a/docs/ai-generated-image-classifiers.md b/docs/ai-generated-image-classifiers.md index 8377997..7fedad6 100644 --- a/docs/ai-generated-image-classifiers.md +++ b/docs/ai-generated-image-classifiers.md @@ -2,7 +2,7 @@ > Research archive for metadata-free `ai_generated` versus > `not_ai_generated` classifiers. These are not SynthID detectors and are not -> `identify` verdicts. The 2026-08-31 freeze is consumed only by explicit + `identify` verdicts. The 2026-08-31 freeze is consumed only by explicit > `classify` / `classify_pixels`. Current behavior: > [photo pixel classification](photo-classify.md), > [supported signals](supported-signals.md), @@ -21,7 +21,7 @@ origin-locked residual bank are different features for different jobs. ## Frozen result, 2026-08-31 The freeze is **AI versus camera**, not a universal detector, and it is not -wired into `identify`. The library consumes it only through explicit +ired into `identify`. The library consumes it only through explicit `classify_pixels` / CLI `classify`. Detector: CLIP-L-ft ridge AND freeze MLP. Provider: 124-d focal heads, only after DEFINITELY. Library guide: [photo-classify.md](photo-classify.md). Hub card: @@ -817,6 +817,20 @@ calibrated abstain band presented as unknown rather than no_ai, accepting a bounded recall loss decided as a product tradeoff, not a model fix. Artifacts: `receipt-gate-2026-08-27/report.json`. +The 2026-08-31 layout probe (OpenCV stacked text-lines, no CLIP, no +retraining) splits that failure into two domains. Photoreal AI receipts +are easy layout positives (16 and 26 lines on the two probes, gated at +every cut from 3 to 15). Photographed thermal CORD is the hard cell +(median 6 lines). Photos and AI-test sit at median 1 line. A +high-precision cut `n_lines >= 11` covers both AI-receipt probes, 14.1% +of CORD, 0.3% of a 300-photo sample, 0/24 Kodak, and 3.7% of a 300-image +AI-test sample, versus the CLIP receipt-gate's 14.4% AI-test loss. Paddle +OCR on the two AI receipts reads brand and totals and garbles the +surrounding Spanish; that is a specialist signal, not a Model 1 feature. +Do not add receipts to CLIP-L train. Next: a layout-first router into an +OCR specialist, with thermal CORD still uncovered at a photo-safe cut. +Artifacts: `receipt-layout-gate-2026-08-31/report.json`. + Batch-1 then filled four more unmeasured domains under the same sieves (428→421 rows after phantom repair: UI screenshots, 150; Danbooru community digital art, 145; software-rendered matplotlib charts, 82; OpenStreetMap diff --git a/docs/cli.md b/docs/cli.md index 6cd09b4..460beda 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -15,6 +15,7 @@ defaults. This page focuses on choosing the right command. | --- | --- | | `metadata` and metadata-only `identify` | Default package | | `classify` | `remove-ai-watermarks[classify]` | +| `verify-openai-synthid` | `remove-ai-watermarks[verify]`, API access, and `OPENAI_API_KEY` | | Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) | | Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` | | Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` | @@ -63,8 +64,44 @@ Metadata only inspection: remove-ai-watermarks identify image.png --no-visible ``` -Despite the historical option name, `--no-visible` skips both visible and open -invisible pixel detectors. Metadata inspection still runs. +Despite the historical option name, `--no-visible` skips all pixel detectors, +including visible marks, open DWT-DCT, and TrustMark. Metadata inspection still +runs. + +## Verify OpenAI SynthID from pixels + +```bash +uv tool install --force "remove-ai-watermarks[verify]" +remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload +remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload --json +``` + +This is an explicit remote check against OpenAI's official Content Provenance +API, not the incomplete local OpenAI carrier research model. Before upload, the +command writes a temporary copy with AI provenance metadata removed and aborts +unless the decoded RGBA pixels are identical to the source. It then reads only +the API's independent `synthid` entry; a C2PA-only response cannot become a +SynthID detection. The source is never modified. + +The API supports PNG, JPEG, and WebP files up to 50 MiB. The command requires +`OPENAI_API_KEY` and an organization with endpoint access. Because the sanitized +raster is uploaded to OpenAI and the endpoint is not eligible for Zero Data +Retention, `--acknowledge-upload` is mandatory. This command is never called by +`identify`. `not_detected` means only that OpenAI's verifier did not recognize a +supported watermark in this file; it is not proof of human authorship. + +The built-in client bounds the request at 120 seconds and disables automatic +SDK retries, so one acknowledgement cannot silently upload the media multiple +times. A timeout, disconnect, malformed response, access failure, or rate limit +is an error, never a negative watermark verdict. API failures expose status, +error code, request id, `Retry-After`, and whether an explicit caller-controlled +retry is appropriate through `OpenAIProvenanceError`; the verifier itself never +retries an upload. +The JSON result uses the same provider-scope, backend, pixel-preservation, and +metadata-use audit fields as `verify_openai_synthid`. + +The Python API enforces the same boundary with the required explicit intent +flag `verify_openai_synthid(path, acknowledge_upload=True)`. ## Classify a photograph from pixels diff --git a/docs/controlnet-removal-pipeline-research.md b/docs/controlnet-removal-pipeline-research.md index 4b56600..41259ab 100644 --- a/docs/controlnet-removal-pipeline-research.md +++ b/docs/controlnet-removal-pipeline-research.md @@ -79,8 +79,9 @@ enough yet barely touches flat fills. So the survivors FLIP by content type — choice alone does not guarantee removal. **2. Seed non-determinism near threshold.** img2img uses a random seed unless `--seed` -is passed, and there is no local SynthID detector to self-verify. The bracelet survived -controlnet @0.15 in one run and CLEARED @0.15 in another (same pipeline+strength+res). +is passed, and these geometries are outside the current local detector's scope. +The bracelet survived controlnet @0.15 in one run and CLEARED @0.15 in another +(same pipeline+strength+res). So a single clean run does NOT establish a strength as safe — characterizing a reliable floor needs a seed-repeatability sweep (N runs, varied seed), not one pass. @@ -144,8 +145,8 @@ Gemini app; the two payloads are vendor-specific and never cross-checked): but never recovered original identity precisely — every setting traded one problem for another. See `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up" for the full sweep. -- **No local SynthID detector exists** → the service can't self-verify; bake in strength - margin and periodic oracle spot-checks. +- **No applicable local detector exists for these geometries** → the service + can't self-verify; bake in strength margin and periodic oracle spot-checks. - **Lesson:** visual-quality / face-identity recovery does NOT prove removal — only the oracle does, across MULTIPLE content types; never conclude from a partial result (the photoreal-only data first read as "controlnet shields, default removes"; the flat-graphic @@ -298,8 +299,9 @@ attention-slicing; ~1-2 min/image, so a coarse sweep is a sub-hour background ru is needed ONLY for the separate native-large-Gemini (2816 px) case, which OOMs even without a ControlNet (that requires a GPU task). The genuine external dependency is NOT compute but the **manual SynthID oracle**: -there is no local SynthID detector, so removal is verified by hand in the Gemini app -("Verify with SynthID") per image, regardless of where the diffusion runs. +these geometries are outside the current local detector's scope, so removal is +verified by hand in the Gemini app ("Verify with SynthID") per image, regardless +of where the diffusion runs. Runner: **`scripts/controlnet_sweep.py`** (built 2026-06-02) implements exactly this sweep — SDXL base 1.0 + an SDXL-native ControlNet img2img, one output per (control x strength x scale) @@ -441,8 +443,9 @@ shielding risk; defer to a v2 after the single-canny path is dialed in. **Hard caveat:** every change that increases preservation (higher scale, denser canny, fuller window, softer edges) marginally REDUCES effective regeneration and so raises the chance the watermark -survives -- exactly the shielding failure mode. There is no local SynthID detector, so each tuning -change must be re-confirmed on the oracle. These are img2img-context recommendations derived from +survives -- exactly the shielding failure mode. These geometries are outside the +current local detector's scope, so each tuning change must be re-confirmed on +the oracle. These are img2img-context recommendations derived from generation-context sources plus our own measurements; treat the playbook as hypotheses to verify, not settled defaults. diff --git a/docs/index.md b/docs/index.md index ed34af7..d6d89ca 100644 --- a/docs/index.md +++ b/docs/index.md @@ -26,7 +26,7 @@ to run the tool. Use the maintainer references only when changing the code. | [Verification plan](verification-plan.md) | Verification methods, completed measurements, and remaining validation gaps. | | [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, photo-classify Hub model, ComfyUI Registry, and the release workflow. | | [Watermarking landscape](watermarking-landscape.md) | Vendor signals and detection approaches. | -| [SynthID technical reference](synthid.md) | Mechanism, detector access, robustness, and implications for this project. | +| [SynthID technical reference](synthid.md) | Mechanism, provenance, robustness, regeneration. | ## Research archive @@ -39,11 +39,13 @@ The current behavior is defined by the code, tests, README, and user guides. - [Doubao reverse-alpha research](research-doubao-distillation.md) - [AI-generated image classifiers](ai-generated-image-classifiers.md) (photo AI-versus-camera freeze consumed by `classify_pixels`, not `identify`) - [Photo classifier Hugging Face card](photo-classify-hf/README.md) (Hub publication text of the 2026-08-31 freeze) -- [SynthID source classifiers](synthid-classifiers.md) +- [SynthID source classifiers](synthid-classifiers.md) (metadata-free OpenAI/Gemini source finding and provider-lineage experiments) - [SynthID local detector research](synthid-detector-research.md) - [SynthID mark removal research](synthid-removal-research.md) - [SynthID identity research](synthid-robust-identity-research.md) - [SynthID identity follow-up](synthid-robust-identity-research-2026-06-08.md) - [Video SynthID quality research](video-synthid-quality-research.md) +- [OpenAI SynthID oracle ladders](synthid-oracle-ladders.md) (routing hub) +- [SynthID detector and removal plan](synthid-detector-removal-plan.md) (chronological mixed archive) - [Text protection research](text-protection-research.md) - [Chroma1 engine research](chroma1-engine-research.md) diff --git a/docs/installation.md b/docs/installation.md index 6a05aec..87ae329 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -106,13 +106,14 @@ application actually uses: | Extra | Capability | Automatically includes | Torch or model download | | --- | --- | --- | --- | -| `pixels` | Shared BGR array and image-processing runtime | NumPy, headless OpenCV | No | +| `pixels` | Shared BGR runtime and calibrated-size SynthID carrier detection | NumPy, headless OpenCV | No | | `heif` | HEIC, HEIF, and AVIF pixel decoding | pillow-heif | No | | `visible` | Visible mark detection, OpenCV inpainting, and manual erasing | `pixels` | No | | `video` | Visible video identification/removal and timestamp preservation | `visible`, PyAV | No | | `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No | | `trustmark` | Adobe TrustMark detection on Python 3.11-3.12 | trustmark | Yes | | `classify` | Metadata-free photo AI-versus-camera classifier plus gated provider | `pixels`, Torch, Transformers | Yes | +| `verify` | Official remote OpenAI SynthID verification | OpenAI SDK | No | | `diffusion` | Torch and Diffusers runtime; video SynthID regeneration | `pixels`, Torch, Diffusers | Yes | | `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch | | `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch | @@ -139,9 +140,10 @@ flowchart LR heif trustmark classify --> pixels + verify ``` -`heif`, `trustmark`, and `text-draft` are independent branches. Combine them +`heif`, `trustmark`, `verify`, and `text-draft` are independent branches. Combine them explicitly with another feature when required. `text-draft` is excluded from `all` because it proposes unverified OCR annotations and is not a production removal path. TrustMark requires NumPy 1.x, which has no @@ -164,6 +166,9 @@ uv tool install --force "remove-ai-watermarks[video]" # DWT-DCT and TrustMark detection without diffusion removal uv tool install --force "remove-ai-watermarks[detect,trustmark]" +# Official OpenAI SynthID verification +uv tool install --force "remove-ai-watermarks[verify]" + # Every production capability compatible with this Python uv tool install --force "remove-ai-watermarks[all]" @@ -176,6 +181,13 @@ do not install libheif. `detect` uses the in-tree torch-free decoder and does not install the upstream `invisible-watermark` package. Optional models download their weights on first use. +The `verify` extra makes an explicit remote request. The +`verify-openai-synthid` command first removes AI provenance metadata from a +temporary copy, checks that its decoded pixels are unchanged, and then uploads +that copy to OpenAI. It needs `OPENAI_API_KEY`; the command never runs from +`identify` and refuses to upload without `--acknowledge-upload`. The Python API +requires the equivalent explicit `acknowledge_upload=True` argument. + The old `gpu` and `remove` aliases are intentionally not provided. Use `diffusion` and `visible` respectively. diff --git a/docs/known-limitations.md b/docs/known-limitations.md index 35bc453..323df8b 100644 --- a/docs/known-limitations.md +++ b/docs/known-limitations.md @@ -111,12 +111,14 @@ rates are kept in [`text-protection-research.md`](text-protection-research.md) and the [`fidelity` evaluation record](../data/evaluations/fidelity/README.md). -### Removal cannot be verified locally for proprietary SynthID +### There is no local SynthID pixel detector in the package -The project has no public local SynthID pixel decoder. It recognizes presence -from supported provenance metadata: Google AI C2PA under Google's all-media -policy, or current OpenAI C2PA with an explicit watermark action. A missing -provenance signal is not a negative pixel verdict. +Google does not publish the proprietary SynthID payload decoder, and the +package does not ship one. Signed provenance is the supported route: +Google AI C2PA or current OpenAI C2PA with an explicit watermark action. +`verify-openai-synthid` is the official remote pixel check for OpenAI. +Research on a periodic lattice expert is in +[synthid-detector-research.md](synthid-detector-research.md). For important outputs: @@ -457,9 +459,46 @@ with plain `torch.load`. Bump lightning and cut a patch release when a fix ships. The calibration history is in [module internals](module-internals.md#metadata-and-provenance). -External AI versus real image classifiers are out of scope. The project -identifies concrete local provenance signals instead of shipping a generic -statistical classifier. +A generic metadata-free AI-generated-image classifier is not shipped. It is a +separate open research task from provenance detection: the current Model 1 +result is AI-versus-camera and still confuses some conventional graphics, CGI, +product cutouts, and scans with generation. The OpenAI/Gemini source finder is +narrower again and cannot substitute for the general classifier. The shipped +product identifies concrete local provenance signals instead of presenting +either research classifier as a supported verdict. + +Frozen transfers of Community Forensics, SPAI, SAFE, RINE, Nonescape Mini, +Dual Data Alignment, PGC, and DGS-Net did not close this gap at the required low +false-positive rate. DDA supplies a complementary representation but its +independent errors make simple hybrids worse. PGC SD1.4 is strong on OpenAI, +but its published output and the useful global/residual ablations misclassify +most or all of the Kodak scan set; removing that branch removes the OpenAI +gain. DGS-Net is weak at the same operating point and also adds independent +photo errors. A source-disjoint public-domain museum audit confirmed a separate +human-art failure: Model 1 called 355/1,050 drawings, prints, manuscripts, +paintings, textiles, and studio-object or historical-photo rows AI-generated. +Its error was 0/150 on the historical-photo control, so ordinary camera accuracy +does not bound accuracy on other human-created visual media. A domain veto +removed the museum errors in development but failed the existing AI-test and +fresh-photo core. A second source-disjoint modern-negative audit measured the +commercial cells: across 421 date-clean Unsplash and CC-licensed Flickr images, +Model 1 accepted 16.4% as AI, concentrated in logo/graphic design (38.3%), +retouched fashion photography (34.1% on the Unsplash cell; 2/40 on the Flickr +replication), and product cutouts (16.0%), while composites +stayed clean at 0/50 and long exposure near clean at 11/146 after a +two-part contamination control (a 0/450 provenance-metadata scan and a +pre-2022-08 date bound that dropped 29 rows, correcting one contaminated +Flickr fashion cell). The failure pattern therefore spans both historical art +and modern stylized graphics, not photographic exposure technique. A +source-disjoint linear veto fit on the date-clean modern cells was then +rejected one gate earlier: no searched operating point repaired even one +modern dev error without dropping below the frozen EvalGEN and FLUX floors, +because modern human graphics and AI generations overlap almost one-to-one in +the frozen CLIP embedding, at roughly 1.3 to 1.5 AI positives lost per negative +repaired. Model 1 also lacks a time/device-disjoint negative contract +for modern computational photography. The measured public protocol, GitHub +survey, and rejected fusions are recorded in +[AI-generated image classifier research](ai-generated-image-classifiers.md#general-ai-classifier-github-sweep-2026-08-25). ## Output and traceability diff --git a/docs/module-internals.md b/docs/module-internals.md index d0b9834..53a9f72 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -539,7 +539,382 @@ metadata extraction from verdict logic: both extractors reach the same answer. It did not, and the record path silently reported no SynthID for images the file path flagged. - `identify` preserves the path-based API and adds the optional registered - visible-mark and open invisible-watermark decoders after extraction. + visible-mark and open invisible-watermark detectors after extraction. + +### SynthID periodic carrier detector (research only) + +The local lattice expert is not part of the public package. Runtime code lives +in [`scripts/synthid_runtime/`](../scripts/synthid_runtime/) and the campaign +log is [`synthid-detector-research.md`](synthid-detector-research.md). The +notes below are the calibration history of that research expert. + +[`synthid_detector.py`](../scripts/synthid_runtime/synthid_detector.py) is the +runtime form of the frozen 2048x2048 periodic-tile experiment. It folds a +Gaussian high-pass residual modulo 16x16 within a calibrated pixel-count range +and compares the normalized RGB tile with the bundled float64 template +`scripts/synthid_runtime/synthid_periodic_tile_2048_v1.npz`. Exact multiples use the original +reshape-and-mean path; other sizes use count-correct modulo folding, +without resize. Channels are filtered and folded sequentially, and partial edge +blocks are accumulated without a full-frame padding buffer so the 18-megapixel +ceiling does not require multiple three-channel float workspaces. The model hash +is pinned by a test, and the unchanged operating threshold is +`0.17357069773071196` through 10 megapixels. + +That fixed threshold is not production-qualified. A later source-fresh +Open Images test-split challenge produced 5 crossings among the 211 images in +its supported geometry. A precision-first `0.28` replacement is frozen as a +research candidate after retaining all 12 available source-diverse native +positives and rejecting those five crossings. It then failed the second +untouched holdout at 1/213, with score `0.322542963`. The replacement is +rejected and fixed-v2 remains an explicit diagnostic only. + +The direct API returns `detected`, `indeterminate`, or `unsupported`. Passing +`register_scale=False` selects fixed-v2 from 1,000,000 through 10,000,000 +decoded pixels as an explicit diagnostic. Its frozen threshold accepted +none of 5,000 public COCO views balanced across every observed target geometry, +and none of a separate 5,000-view challenge over 256 generated geometries +covering every pair of modulo-16 edge remainders. The original 2048x2048 +verdicts and exact scores remain unchanged. Runtime matches do not attribute a +provider. `identify` adds only positive matches as high-confidence +evidence and never turns a local negative into a clean verdict. +The result envelope also names the signal family, provider scope, backend, +whether metadata contributed to the verdict, whether pixels were preserved, +and an explicit reason for unsupported or indeterminate results. These fields +are shared with the official OpenAI verifier's JSON boundary. + +The production router selects `synthid-periodic-tile-large-v1` above 10 through 18 +megapixels when both dimensions are at least 2,048 pixels. It evaluates all +phase-aligned 2,048-square windows and combines the minimum fixed-template, +Red-minus-Green, and Blue-minus-Yellow spatial correlations with the most +negative Blue-minus-Yellow mid-band correlation. The 3072x5504 portrait +geometry also applies a Green mid-band alias veto. Each component is normalized +to its frozen gate and the public threshold is `1.0`. + +All 37 inferred large candidates cross the rule, and all seven metadata-free, +pixel-identical candidates checked by the official Gemini verifier were +detected. The constants rejected all 17,417 exposed external controls. A +post-freeze production-path challenge then rejected all 2,637 decoded-pixel- +unique controls drawn from 2,000 COCO images excluded from the earlier large +color-phase challenge and 637 deduplicated Picsum controls. Four large +geometries and four resampling kernels were balanced; the maximum score was +`0.0592777965`. The source collections were not freshly acquired, so this is a +feature-unseen holdout rather than a fresh-source estimate. + +A separate post-freeze Open Images download yielded 41 completed, +decoded-pixel-unique controls after excluding incomplete `.aria2` files and all +prior Open Images hashes. The frozen production path accepted 0/41 and reached +a maximum score of `0.4083013324`. This source-fresh audit is too small to +replace the main holdout interval but checks the acquisition boundary. + +The same seven official positives were then re-encoded at unchanged dimensions. +JPEG-95 and JPEG-90 each reduced detection from 7/7 native files to 0/7. The +large operating point is therefore native-pixel and lossless-copy support, not +a codec-robust claim. + +Arbitrary geometry is not the same as arbitrary spatial resampling. On a +stratified 80-image fixed-positive sample, one-step resizes at seven nonidentity +scales from 0.5 through 1.5 reduced the unchanged 16x16 detector from 80 accepted +sources to zero at every scale. Restoring the original geometry recovered 58-80 +sources, showing that the carrier period scaled with the pixels. A discovery +bank that scaled the template to integer periods 8, 10, 12, 14, 18, 20, and 24 +was promising: a threshold frozen above 3,000 resized COCO controls accepted no +view in a 2,000-control final partition and accepted 672 of 800 source-disjoint +provider positives. It also stayed below threshold on the tracked OpenAI and +Adobe controls. The branch remains research-only because noninteger periods at +scales 0.8, 0.9, 1.1, 1.2, and 1.333 collapsed, while separate per-period +thresholds accepted five final controls. The runtime therefore keeps only the +fixed 16-pixel lattice. + +A follow-up fractional-period probe sampled the 30 strongest template harmonics +over a continuous 7.5-24.5 period range. The correct period appeared within +0.05 pixels among the top three candidates for 58 of 60 transformed positives. +Testing nine neighboring reconstructed geometries then recovered 44 of 60 at +the native threshold, compared with an upper bound of 48 when the true source +geometry was supplied. The complete search still failed its small frozen +control split: a threshold above 250 development controls accepted two of 150 +final controls. Multiplying the canonical score by spectral-period confidence +also accepted two. That baseline was rejected rather than shipped at its +discovery threshold. + +The research-only +[`synthid_affine_lattice_probe.py`](../scripts/synthid_affine_lattice_probe.py) +adds split-confirm synchronization. It estimates complex harmonic coherence on +one checkerboard of patches, confirms the selected period on the other, and +reports amplitude-aware confirmation, a locally content-whitened multichannel +code match, and phase-preserving and cyclically registered template scores. A +0.1-pixel grid recovered the expected +12.8, 14.4, 17.6, and 19.2 periods in 20/20 transformed views from five official +positive parents. A provisional conjunction accepted none of 800 corresponding +views from 200 oracle-negative parents, none of 469 OpenAI-labeled rows, and +three of 276 broad non-Google rows whose TC260 or Samsung provenance prevents +treating them as clean oracle negatives. This probe is not runtime routing: +positive diversity is still inadequate, period 8 remains rejected, and the +amplitude stage currently handles isotropic scale at zero rotation only. Full +protocol and caveats are in the detector research plan. + +The whitened match uses neighboring noncarrier bins to estimate complex +Green/opponent-color covariance around every selected harmonic. It corrected a +period-24 alias on one native official positive, giving the broad native search +the correct period 16 on all five ordinary-size positives. It was retained only +as a candidate reranker: its positive-to-negative margin was smaller than the +existing spatial-template margin in both the native pilot and a locked 0.8 +resize challenge. Two of the three TC260/Samsung-provenance challenge crossings +also matched the full whitened code more strongly than the weakest official +large positive, so the pixel result identifies a compatible signal family, not +a provider. + +The same probe also exposes a payload-agnostic H5 confirmation. It estimates a +complex harmonic vector independently on the two checkerboards and measures +their fixed relative-phase inner product; the codeword need not match the known +template. Free cyclic-shift selection was rejected because natural phase aliases +overlapped the controls. The fixed all-harmonic statistic separated five native +official positives from 130 controls with margin `0.1619`, and five 0.8-resized +views from 200 fresh-parent controls with margin `0.1541`; all seven large +official positives also passed the observed gap. It remains research-only +because these tests supplied the period, reuse one negative source family, and +contain only 12 independent positive parents. + +The separate research-only +[`synthid_cyclostationary_probe.py`](../scripts/synthid_cyclostationary_probe.py) +measures full complex cross-channel spectral correlation at carrier shifts +against neighboring-shift same-image nulls. It recognizes synthetic amplitude +modulation and rejects both independent noise and a purely additive carrier. +The real native pilot overlapped completely: five official-positive joint +contrasts ranged from `-0.0027` to `0.0337`, while 130 controls reached `0.0593`. +The cyclostationary branch is rejected rather than tuned on new frequency bands. + +The affine-lattice research CLI also supports symmetric in-memory JPEG and +top-left crop challenges. At fixed native period 16, codec-specific +phase-preserving template thresholds retained all five repeated positive views +and accepted none of 80 locked controls at JPEG qualities 95, 90, and 80. The +locked margins declined from `0.1054` to `0.0257`; whitened and unknown-codeword +scores were rejected for lossy JPEG. A separate registered crop conjunction +retained all five views and accepted none of 80 controls after both 5% and 10% +top-left crops, with registered-template margins above `0.22`. These are +research pilots, not runtime experts: the period was supplied, positive parents +were reused, and only one codec implementation and one negative family were +tested. + +The `register_scale=True` path is the calibrated ordinary-size production +expert. It +keeps the harmonic search, adds three-level scale-space consistency and quadrant +agreement. Its calibrated decision requires the canonical winner to be the +strongest spectral-period candidate, its three-way amplitude to cross the +threshold for that period bin, and both independent high-frequency template +bands to correlate by at least `0.075`. The public registered score is the +minimum normalized margin across those gates, so its decision threshold is +`1.0`. + +The earlier single-threshold version produced 68 false positives among 11,273 +symmetrically downscaled Spaces controls. A top-candidate plus period-bin version +removed those but still produced 6 false positives among 1,000 source-independent +Open Images controls. The high-band gate removed them and then accepted none of +499 previously untouched Open Images reserve controls. The resulting rule +retained 229 of 355 source-disjoint transformed positives: 0/65 at scale 0.5 and +229/290 from scale 0.65 through 1.5. The explicit period-8 rescue is rejected +because resize lattices fully overlap its positive distribution. + +A subsequently frozen 1,000-image Open Images reserve accepted zero in +registered mode. The fixed expert supported only 81 of those geometries and +accepted seven, so fixed and registered results cannot safely be unioned. In +overlapping geometry the registered decision remains the validated path; +fixed-only evidence is a diagnostic rather than a universal-cascade positive. + +A later source-fresh 3,000-image Open Images test-split challenge superseded the +registered-v2 precision claim: it accepted 4 of 2,997 supported controls. The +registered-v3 confirmation uses disjoint patch groups for phase, cyclic +amplitude, and held-out-codeword evidence. It rejected all four crossings, +retained 147 of 148 base-accepted source-diverse positives, and retained all 359 +base-accepted views in a dense 0.65-1.50 transform matrix over 12 independent +parents. Frozen unchanged, it then accepted 0/2,996 controls from a second +nonoverlapping Open Images cohort and 0/2,366 supported controls from a +3,000-image COCO second-family challenge. Registered-v3 is now the default +ordinary-size positive route. The exact gates, acquisition hashes, and rejected +weak-signal rescue are recorded in the detector research plan. + +Every one of those control rates is photographic. Against 223 corpus images +whose C2PA names a non-Google generator, the unchanged entry point accepted 29 +(`0.130`, Adobe Firefly `0.241`, highest foreign score `3.01`), all from +registered-v3. The branch reads a lattice shared across generation pipelines, +which is why it must not be reported as a watermark. The public `identify` +path no longer calls this expert. + +The branch is also phase-locked to the image origin, exactly like the large +expert. A two-pixel diagonal crop killed all 28 in-geometry foreign detections +and all 8 detected Google provenance positives (maximum remaining scores +`0.779` and `0.311` against the `1.0` threshold); the signal recovers only at +offsets that are multiples of four. Registered-v3 therefore detects the same +crop-destroyed generation-pipeline lattice as `large-v1`, on ordinary sizes. + +When registered-v3 abstains, opponent-registered-v1 searches the same frozen +template in Red-minus-Green and Blue-minus-Yellow space. It reranks three +separated scale candidates with fixed RGB and two spatial opponent-color gates, +then accepts only periods 7.9-12.0 on 1-10 megapixel rasters whose sides are at +least 768 pixels. Period-8 candidates additionally require Red-Green and +Blue-Yellow 8-pixel edge ratios no greater than `1.05`; this vetoes the +deterministic JPEG block lattice without using container metadata. The final +rule recovered 49/49 lossless 0.5x-0.75x views from seven official-positive +parents. It rejected all 1,790 measured period-8 codec crossings, while 350 +identically resized controls had no base crossing and the earlier period-band +rule accepted 0/1,000 post-freeze Picsum controls. Period 12.8 remains excluded, +and lossy JPEG/WebP views remain inconclusive. + +The runtime precedence is registered-v3, the bounded opponent fallback, then +large-v1 above 10 megapixels. Passing `register_scale=False` selects the legacy +fixed diagnostic explicitly. The research bank in +`scripts/synthid_routed_expert_bank.py` keeps all three observations for audits. +Neither runtime nor research routing returns a clean-image verdict. + +An unchanged registered challenge from 10 to 18 megapixels retained only 1 of +37 Google candidates and zero of 89 non-Google controls. Twenty-eight positives +cleared amplitude, 21 had matching spatial and spectral periods, but only three +cleared high-band agreement. The 10-megapixel ceiling therefore remains. +Phase-aligned 2,048-square fixed windows did not provide a fallback: median +consensus retained 36 positives and accepted 10 controls, while all-window +consensus retained 36 and accepted eight. +One frozen full-frame pre-resize to eight megapixels also retained only the same +1 of 37 positives and zero controls; just three positives cleared high-band +agreement. Large images therefore cannot be routed through a canonical-size +registered fallback; the later native opponent-color branch is separate. +The remaining phase-aligned window variants closed this branch: a single center +2,048-square registered window retained 1 of 37 positives and zero of 89 +controls, while accepting any phase-aligned 2,048-square window retained 2 of +37 and zero controls. The latter control maximum was already 0.968 against the +1.0 decision threshold. Neither the coverage nor the exposed specificity +margin supports a registered-window expert; these results do not apply to the +later native opponent-color branch. + +A separate half-scale patch-consensus branch initially looked promising. Its +64-pixel, 90th-percentile patch statistic retained 33 of 49 validation positives +and zero of 166 controls, then 27 of 52 locked-test positives and zero of 140 +controls. The frozen broad challenge rejected it: 61 of 10,906 symmetrically +half-scaled Spaces controls and 22 of 1,000 independent Open Images controls +crossed the unchanged threshold. A post-challenge Green negative-phase spread +gate removed those errors but retained only 2 of 234 later Google rows and zero +of 689 later controls. Half-scale support therefore remains explicitly absent. + +An independent reproduction of the public `aloshdenny/reverse-SynthID` V4 +cross-color codebook did not provide another expert. Its best-of-two-model phase +rule retained 141 of 355 positives but accepted 191 of 499 controls and 386 of +a fresh 1,000-control reserve; AUC was 0.517. The repository's stricter +aspect-ratio routing reduced coverage but preserved the same approximately 38% +positive and control acceptance within supported rows. Its older V3 phase score +had AUC 0.473, and two prespecified amplitude-aware V4 rerankings reached only +0.521 and 0.524 AUC. These external rules are research baselines only. Their useful +contribution is the solid-reference phase-consensus construction, not either +published single-image threshold. Full methodology and the DALL-E reference-set +confound are recorded in the detector research plan. +A later exact repeat on the hash-disjoint v7 challenge confirmed the same +failure: the published cut accepted 77/200 Google and 76/200 OpenAI, so it did +not separate the two target providers. +A direct exact-1024 reproduction of the same V4 artifact confirmed rather than +rescued that verdict: the union of its two published profiles accepted 177 of +443 Google rows and 60 of 162 controls. The artifact's floor left only about two +active bins in the inspected profile/channel, explaining why its score acts +as a weak generic phase-coherence test rather than a specific codebook match. + +An exact-1086x1448 OpenAI follow-up also rejected the solid-reference transfer. +A low-texture 256-bin phase codebook achieved native and JPEG-95 test AUCs of +0.512, and independent train halves shared only two exact bins. The paper's +content-dependent alternative was tested with 60 OKLab masking-alignment +features. Its native/JPEG minimum reached test AUC 0.593 and 15/562 positives +versus 1/228 controls at the frozen boundary, but deterministic half-fits shared +no positive decision and had coefficient cosine 0.148. Neither branch is a +runtime OpenAI expert. + +An exact-1254 assertion-contrast search selected an sRGB period-8 Blue-channel +direction at 45 of 259 validation asserted rows and zero of 16 same- or +other-provider validation rows. It failed its locked specificity check at 39 of +233 asserted rows and one of 18 controls. Across the complete exact-geometry +audit it accepted 185 of 992 asserted rows, one of 55 same-provider rows without +an assertion, and one of 13 other-provider rows. Raising the threshold above +the wider controls reduced recall to a small minority, so this branch remains +research-only. + +Three attempted veto or missing-codeword rescues also failed. The official +InvisMark decoder checkpoint passed its own embedded-watermark self-test at +0.997 confidence and 0.97 bit accuracy, but all OpenAI and provider-control +cohorts clustered around 0.18 confidence with no useful separation. DTCWT +modulus before periodic folding retained 5 of 158 locked-test OpenAI rows and +zero of 135 controls, then zero of 40 fresh OpenAI rows. A 16-codeword whitened +period-8 model retained 5 of 158 and zero controls, then 1 of 40 fresh OpenAI +rows and 1 of 50 fresh controls. They neither explain the known oracle-positive +miss nor justify runtime routing. + +An image-level patch-bag model over nine normalized high-pass patches was the +strongest later local OpenAI classifier. It accepted 324/1,364 development-test +OpenAI rows with 4/3,373 heterogeneous controls, then transferred to 120/486 +OpenAI rows, 8/504 general controls, and 5/33 same-provider non-assertion rows +in the later temporal challenge. Assertion enrichment over the same-provider +stratum was not independently significant (`p = 0.151`, one-sided exact test). +JPEG-95 reduced the unchanged conjunction to +1/486 positives and zero controls. The native temporal hits were complementary +to the signed period-8 hits, but the same-provider rate and codec collapse +identify another export noiseprint rather than a runtime SynthID expert. + +The separate OpenAI period-8 DTCWT component is persistent rather than tied to +one short rollout: exact-generator asserted hits were 3/16, 46/365, and 26/200 +from May through July, with 75/581 overall versus 1/52 same-generator rows +without an assertion. Its native/JPEG minimum score reached 0.721 AUC between +those indeterminate strata. Sorting all 64 cyclic correlation scores removed +absolute phase but also removed locked-test discrimination at 0/158 positives +and 0/135 controls. This remains research evidence for a weak signed carrier, +not a runtime OpenAI detector. + +A four-family open-proxy challenge also failed to justify a generic neural +watermark expert. A fixed residual frontend and cross-family residual mixing +were trained on three of TrustMark P, VideoSeal, DWT-DCT, and WAM while the +fourth encoder and its test sources remained unseen. Held-out AUCs ranged from +0.437 to 0.562. Equal-power phase-scrambled hard negatives prevented simple +spectral-energy shortcuts, but did not produce architecture transfer. A +separate translation-invariant Gemini bicoherence search selected none of 20 +development positives and finished at 0/50 positives, 0/199 controls, and AUC +0.374. Neither branch is part of runtime routing; full split and oracle details +are in the detector research plan. + +The separately measured registered geometry range remains 250,000 through +10,000,000 decoded pixels with both sides at least 256 pixels. The default path +and `identify` use registered-v3, then the narrower opponent-registered-v1 +fallback in its 1-10 megapixel domain, and large-v1 above 10 megapixels. A +20-image real-corpus drift check was byte-identical after the earlier v2 integration. +The calibration history and caveats are in the linked detector research plan. + +### Official OpenAI SynthID verifier + +[`openai_provenance.py`](../src/remove_ai_watermarks/openai_provenance.py) +provides the explicit remote production backend exposed as +`verify-openai-synthid`. It is intentionally separate from `identify`, because +one invocation uploads a sanitized raster to OpenAI. The CLI requires +`--acknowledge-upload`, and the optional OpenAI SDK lives in the independent +`verify` extra. + +The backend accepts only PNG, JPEG, and WebP. It computes a decoded RGBA pixel +fingerprint, removes AI provenance metadata into a temporary file through +`metadata.strip_and_verify`, recomputes the fingerprint, and aborts before any +request if metadata survived, the format changed, the pixels changed, or the +sanitized file exceeds the endpoint's 50 MiB limit. It then sends exactly one +multipart file to `content_provenance_checks.create` and parses exactly one +`type == "synthid"` result. The independent C2PA entry is never returned or +used as fallback evidence. Missing, duplicate, or unknown SynthID outcomes are +errors rather than negative detections. + +The default SDK client has a 120-second request timeout and zero automatic +retries. One upload acknowledgement therefore authorizes at most one media +transmission rather than inheriting the SDK's retry default. Request logs keep +the endpoint, temporary basename, media type, byte count, timeout, retry policy, +duration, HTTP status, error code, and request id when available, but omit the +source path, image bytes, credentials, and decoded-pixel fingerprint. + +`OpenAIProvenanceError` preserves the status, API error code, request id, +`Retry-After` value, and a transient-only `retryable` flag. The library does not +automatically act on that flag: an explicit caller invocation is required for +every additional upload. Transport and schema failures remain errors rather +than becoming `not_detected` or a local detector result. + +The result remains provider-scoped and positive-evidence-only. `not_detected` +does not mean human-created, and the official endpoint's published prohibition +on repeated reverse-engineering or evasion queries prevents using this backend +as an adaptive training or removal oracle. ### Portable metadata record diff --git a/docs/python-api.md b/docs/python-api.md index 464f99e..6ccbc29 100644 --- a/docs/python-api.md +++ b/docs/python-api.md @@ -12,6 +12,23 @@ path that still runs on CPU and combines `video` and `diffusion`. Add `heif` independently when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See the complete [feature-extra matrix](installation.md#feature-extras). +## Verify OpenAI SynthID + +```python +import remove_ai_watermarks as raiw + +result = raiw.verify_openai_synthid("input.png", acknowledge_upload=True) +print(result.status) +``` + +Official OpenAI JSON results expose `signal_family`, +`provider_scope`, `backend`, `metadata_used_for_verdict`, and +`pixels_preserved`. Remote transport and response failures raise `OpenAIProvenanceError`. Its +`status_code`, `error_code`, `request_id`, `retry_after`, and `retryable` +attributes let a caller implement bounded backoff or a circuit breaker without +turning an API outage into a false `not_detected` result. One function call still +performs at most one upload. + ## Remove visible marks Install `remove-ai-watermarks[visible]` before using the visible-removal API. @@ -372,8 +389,8 @@ Timings and spatial artifacts are opt-in. Artifacts include image-identifying da such as a thumbnail and perceptual hash; aggregate feature families do not. `identify_from_evidence` does not reopen the source file by default: it evaluates -metadata only, and registered visible marks and pixel-backed invisible watermarks -remain in the path-based `identify` call. +metadata only, and the pixel-backed checks remain in the path-based `identify` +call: registered visible marks and open invisible-watermark decoders. Pass `image_path` together with `check_visible` or `check_invisible` to add those pixel detectors on top of the SAME evidence. That is how a caller asking one file diff --git a/docs/supported-signals.md b/docs/supported-signals.md index 3ad6cbe..00367fe 100644 --- a/docs/supported-signals.md +++ b/docs/supported-signals.md @@ -141,12 +141,23 @@ Current pipeline values, all CUDA-only: The `controlnet`, `sdxl`, `qwen` and `default` values were removed. A retired name is rejected at parse time rather than remapped onto a surviving profile. -SynthID does not have a public local pixel decoder in this project. The tool -recognizes presence from supported provenance: Google AI C2PA under Google's -all-media watermark policy, and current OpenAI C2PA carrying an explicit -`c2pa.watermarked.*` action. Legacy OpenAI C2PA without that action does not -assert SynthID. After provenance metadata is removed, a local negative result -is still inconclusive. +Google does not publish the SynthID payload decoder. This package does not +ship a local pixel detector for that watermark. Research on a periodic +lattice expert is in [synthid-detector-research.md](synthid-detector-research.md) +and `scripts/synthid_runtime/`. + +The tool recognizes presence from supported provenance: Google AI C2PA +under Google's all-media watermark policy, and current OpenAI C2PA carrying an +explicit `c2pa.watermarked.*` action. Legacy OpenAI C2PA without that action +does not assert SynthID. + +The optional `verify-openai-synthid` command is a separate official remote +verifier for supported OpenAI watermarks. It strips AI provenance metadata from +a temporary PNG, JPEG, or WebP copy, proves that decoded RGBA pixels are +unchanged, and uses only the API's SynthID result. It is therefore independent +of C2PA for its decision, but it is not local: the sanitized raster is uploaded +to OpenAI after explicit acknowledgement. It is intentionally excluded from +`identify` and its negative result remains inconclusive. Microsoft Paint can name `com.microsoft.invismark.1` in a C2PA soft-binding assertion. Inspection reports both that exact algorithm and its signed `value`, @@ -219,7 +230,7 @@ not a universal clean verdict. | Provider or family | Visible | Invisible path | Metadata or provenance | | --- | --- | --- | --- | -| Google Gemini | Sparkle | Diffusion regeneration for SynthID | C2PA and related source signals | +| Google Gemini | Sparkle | Diffusion regeneration | C2PA and related source signals | | Google Veo video | Veo diamond and legacy text | Oracle-certified VAE removal for SynthID | C2PA and related source signals | | OpenAI image generators | None registered | Diffusion regeneration for supported invisible signals | C2PA and generator provenance | | Meta Muse Image | None on Muse output (legacy `Imagined with AI` unregistered) | Diffusion regeneration for Content Seal, oracle-verified on the default profile | XMP IPTC `trainedAlgorithmicMedia` companion tag; no local Content Seal decoder | diff --git a/docs/synthid-detector-removal-plan.md b/docs/synthid-detector-removal-plan.md new file mode 100644 index 0000000..8b29350 --- /dev/null +++ b/docs/synthid-detector-removal-plan.md @@ -0,0 +1,6409 @@ +# SynthID detector and pixel-only removal research plan + +> Chronological mixed archive of detector, classifier, and removal work. +> Not a statement of current product capability. Read the split pages +> first: +> +> - [SynthID local detector research](synthid-detector-research.md) +> - [AI-generated image classifiers](ai-generated-image-classifiers.md) +> - [SynthID source classifiers](synthid-classifiers.md) +> - [SynthID mark removal research](synthid-removal-research.md) +> +> Shipped behavior remains in [supported signals](supported-signals.md), +> [known limitations](known-limitations.md), and +> [module internals](module-internals.md). Keep this file for dated H-gates, +> corpora, and session notes. Do not add new campaign results here; put +> them on the matching split page. + +## Objective + +Build two independent provider tracks with four capabilities: + +1. a local, metadata-independent detector for the OpenAI SynthID image signal; +2. a local, metadata-independent detector for the Google SynthID image signal; +3. a pixel-only remover for the OpenAI signal that does not use diffusion, + VAE reconstruction, semantic regeneration, or generative inpainting; +4. an independently calibrated pixel-only remover for the Google signal under + the same constraints. + +OpenAI is the first research track because it has a documented remote verifier. +Google follows with the same experimental protocol but its own corpus, labels, +model, thresholds, and oracle. No carrier, feature, score, or operating point +transfers between providers until a held-out experiment demonstrates that it +does. + +The detector target is signal presence, not payload recovery, provider +classification, or general AI-image classification. The removal target is a +minimal pixel residual that makes a source-positive image negative in the +matching provider oracle while preserving the image's geometry and semantics. + +## Non-negotiable evidence rules + +1. **Detector before remover.** A remover may be prototyped against synthetic + carriers, but no real-image removal claim is made until the local detector + passes its own held-out gate. +2. **Provider-specific ground truth.** OpenAI and Google labels come from their + matching verifier. C2PA is recorded separately and is never the pixel label. +3. **No metadata or export leakage.** Training and evaluation operate on decoded + pixels after controlled metadata stripping and matched re-encoding. Geometry, + filename, file size, chunks, encoder settings, and source directories cannot + be model inputs. +4. **Signal identifiability is a gate.** A classifier trained only on provider + positives and unrelated negatives can learn the provider's generator or + export fingerprint. It is called a provider classifier, not a SynthID + detector, until at least one causal control succeeds. +5. **The oracle is held out from optimization.** Candidate algorithms and + hyperparameters are selected locally. Oracle batches are immutable and + registered before submission. A remote binary verdict is never used as an + online loss or hill-climbing signal. +6. **A local score decrease is not removal evidence.** Removal requires a + source-positive, output-negative result from the matching provider oracle. +7. **One hypothesis does not become several signals.** Correlated spatial, + spectral, and color statistics derived from one residual are reported as one + line of evidence unless independent controls separate them. +8. **Every result is reproducible.** Input and output hashes, code revision, + model artifact hash, preprocessing, transform lineage, seed, score, threshold, + quality metrics, oracle result, session, and timestamp are retained. + +## Oracle boundary + +OpenAI documents a synchronous Content Provenance API at +`POST /v1/content_provenance_checks`. For images it returns separate `c2pa` and +`synthid` entries and supports PNG, JPEG, and WebP. It remains a remote, +OpenAI-scoped verifier, not a released local decoder. The same documentation +explicitly says not to use repeated queries to reverse-engineer, remove, or +evade a watermark. Adaptive detector or remover research against that endpoint +therefore requires explicit OpenAI authorization or a separate research oracle +whose terms permit the work. Without that authorization, the OpenAI track may +develop local hypotheses but stops before oracle-driven calibration and removal +certification. + +Google's Gemini verification flow is a Google-scoped oracle. Its documented +result can be detected, not detected, or unclear, and the consumer flow has a +small rolling quota. An unclear result is `indeterminate`, never a negative. + +For every permitted oracle batch: + +- verify the untouched source first; +- submit one file per request; +- record C2PA and SynthID outcomes independently when both are returned; +- preserve the exact submitted bytes and SHA-256 outside the public repository; +- retry only a transient failure on the same bytes; +- record `detected`, `not_detected`, `indeterminate`, or `refused` verbatim; +- submit a matched transform-only control before attributing a negative result + to an experimental edit; +- reserve a final temporal holdout that no feature, threshold, or remover has + seen. + +## Data design + +### Corpus layers + +Each provider gets a separate corpus with five layers: + +| Layer | Purpose | Required controls | +| --- | --- | --- | +| Verified positives | Learn and evaluate the real signal | Matching provider oracle, original bytes | +| Same-provider hard negatives | Separate watermark from generator identity | Same surface or model, oracle-negative | +| External hard negatives | Measure false positives | Cameras, scans, edited photos, other generators, synthetic graphics | +| Low-texture probes | Expose weak shared structure | Solid colors, gradients, ramps, checkerboards, sparse edges | +| Causal pairs | Attribute a residual to the watermark | Same underlying pixels, positive and confirmed negative | + +The causal-pair layer is the most valuable and the hardest to obtain. An +authorized encoder-off pair is ideal. A provider output and a pixel-only +processed version become a usable pair only after the original is positive and +the processed bytes are negative in the provider oracle. Public third-party +pairs are discovery material until their provenance and both labels are +independently verified. + +A negative created by a remover trained against detector A cannot train or +validate detector A in the same experiment. Reserve it for detector B or a later +model epoch after closing the originating experiment. Otherwise detector and +remover can certify each other's shared blind spot. + +If no same-provider hard negatives or causal pairs can be obtained, learned +real-image models stay explicitly labeled as provider classifiers. Spectral +repeatability on solid fills alone does not clear this gate. + +### Strata + +Record and split by: + +- provider, product surface, model family, and generation date window; +- native width, height, aspect ratio, file format, and color mode; +- photoreal, face, text-heavy, flat graphic, illustration, low texture, and + high texture content; +- untouched, metadata-stripped, lossless-normalized, JPEG/WebP, resized, + cropped, and color-adjusted lineage; +- generation session and prompt family; +- parent hash for every derivative. + +Store research media under `.local-eval/synthid/`, not in the public repository. +Only cleared fixtures may enter `data/synthid/`. Tracked files contain schemas, +scripts, synthetic fixtures, aggregate verdicts, and non-sensitive hashes. + +### Split discipline + +Deduplicate by decoded-pixel hash and perceptual similarity before splitting. +Keep every derivative, prompt sibling, and semantic near-duplicate in one hash +group. Split by group into train, validation, locked test, and a later temporal +test collected after the detector is frozen. A random image-level split is +invalid because it leaks transform and generation-family fingerprints. + +The negative set must be large enough to support the claimed operating point. +At zero observed false positives, roughly 3,000 independent negatives are +needed merely to put the one-sided 95% upper bound near 0.1% by the rule of +three. The final evaluation should prefer at least 10,000 hard and ordinary +negatives per provider, report confidence intervals, and report every negative +stratum separately rather than hiding a weak stratum in an aggregate. + +## Detector program + +### Experiment D0: oracle and corpus integrity + +Goal: prove that labels and bytes mean what the manifest says. + +- Separate SynthID from C2PA in every response. +- Confirm that metadata stripping changes C2PA but does not silently define the + SynthID label. +- Confirm every positive used for evaluation on its matching oracle. +- Mutation-test manifest ingestion with swapped provider, duplicate bytes, + derivative leakage, an `indeterminate` result mislabeled as negative, and a + changed file after hashing. +- Measure permitted same-byte verifier reproducibility on a small preregistered + sample. Do not turn retries into adaptive querying. + +Gate: no ambiguous label path, no cross-provider oracle substitution, and no +train/test family leakage. + +### Experiment D1: export and generator confounds + +Goal: determine how accurately SynthID can appear to be detected when no +watermark-specific evidence is available. + +Train deliberately confounded baselines on file/container fields, dimensions, +RGB thumbnails, and generator-vs-camera content. Then repeat after canonical +decode, metadata removal, resolution matching, and hard-negative balancing. + +Gate: any proposed signal feature must beat the canonicalized confound baseline +on same-provider hard negatives and on a temporal holdout. Otherwise the result +is generator attribution. + +### Experiment D2: low-texture carrier discovery + +Goal: test whether a repeatable carrier component is observable when scene +texture is suppressed. + +Extend the existing `scripts/synthid_pixel_probe.py` measurements from grayscale +NCC to: + +- per-channel and opponent-color residuals; +- two-dimensional FFT magnitude and circular phase coherence; +- wavelet bands and multi-scale autocorrelation; +- resolution and aspect-ratio registration; +- cross-color, cross-session, cross-model, and cross-date agreement; +- matched clean synthetic fills and same-provider oracle-negative probes. + +Use leave-one-color, leave-one-session, and leave-one-resolution-out tests. +Fixed bins discovered and evaluated on the same images are descriptive only. + +Gate: a template learned on one subset must detect held-out positive probes above +matched negatives and survive a later collection window. Failure kills the +fixed-carrier branch but not the content-dependent branch. + +### Experiment D3: classical real-image detector + +Goal: establish the strongest interpretable baseline before a neural model. + +Candidate features include locally normalized high-pass residuals, FFT and +wavelet energy ratios, circular phase coherence, color-channel agreement, +block periodicity, and correlations against provider/resolution templates. +Fit regularized logistic regression and a shallow tree model. Calibration uses +only validation data. + +Report TPR at 0.1% FPR as the primary metric, with bootstrap confidence +intervals. AUROC, average precision, and an arbitrary accuracy percentage are +secondary. Also report worst-stratum FPR, temporal-holdout TPR, and score drift. + +Gate: advance the classical detector only if the locked test shows a stable +watermark-specific advantage over confound baselines. Do not choose a threshold +from the locked test. + +### Mechanism hypotheses for D3-D5 + +The next detector epoch tests these hypotheses as one preregistered program: + +1. **Counterfactual labels.** Train and calibrate on source-matched + clean/watermarked examples, ideally the same underlying image before and + after encoding. A result that vanishes when source, date, codec, and + dimensions are balanced is a provider fingerprint, not watermark evidence. +2. **Canonical full-field evidence.** Preserve a 512x512 image-level field, + absolute residual amplitude, and chroma alongside normalized residuals. + Compare it against the frozen patch baseline on source-disjoint pairs. + Failure to improve paired low-FPR detection rejects the added global context. +3. **Registration.** Score several bicubic canonical views spanning small scale, + crop, and offset changes, then aggregate before calibration. This tests + whether synchronization is distributed across the image rather than fixed to + one global phase or local grid. +4. **Independent detection head.** Learn one presence logit directly. Treat + payload-like or phase-consistency heads as auxiliary evidence, not as the + presence decision. Their inclusion must improve a held-out paired test, not + only attack-surface scores. +5. **Symmetric transformation channel.** Apply the identical sampled codec, + resize, crop, color, noise, or overlay operation to both members of a pair. A + transform is admitted only when the reference decoder, where available, + confirms that the transformed positive remains valid. +6. **Encoder versions.** Compare one universal model with version- or + epoch-specific experts on a cross-version transfer matrix. A version split is + useful only if it improves held-out likelihood without source metadata at + inference. +7. **Content-dependent watermarkability.** Measure flat, low-contrast, + monochrome, logo, and pixel-art strata separately. Allow a + content-conditioned expert or abstention instead of forcing one global + operating point. +8. **Two-sided calibration.** Calibrate empirical evidence for both `not + watermarked` and `watermarked`, returning positive, negative, or abstain. + External generator corpora enter afterward as an untouched false-positive + challenge, not as a substitute for counterfactual negatives. + +### Experiment D4: learned residual detector + +Goal: learn watermark presence from counterfactual image-level evidence that a +fixed template misses. + +The primary model uses a full canonical field with raw RGB or luminance/chroma, +absolute-amplitude fine and coarse residuals, and optional stationary-wavelet +and complex-frequency branches. Every optional representation is encoded +separately and fused late; early channel concatenation is not a valid ablation. +Locally normalized patch evidence remains a frozen baseline, not the primary +input. Multi-view registration is aggregated into one image-level presence +logit. Payload-like, phase, localization, and content-watermarkability heads are +auxiliary and must prove an incremental held-out benefit. + +Use group-aware pair sampling and apply every sampled transformation +symmetrically to the clean and watermarked members. Never expose metadata, +paths, native dimensions without normalization, or encoder-specific byte +patterns to the network. If causal pairs remain unavailable, do not advance a +learned model from provider classification to this experiment. + +Train provider-specific models first. A shared backbone with provider-specific +heads is a later ablation, not the default architecture. Keep a second detector +family completely outside remover training so it can reveal surrogate overfit. +Compare a universal head with version-specific experts, and calibrate the +selected model with two-sided empirical evidence so ambiguous inputs abstain. + +Gate for a detector release candidate: + +- empirical FPR at or below 0.1% on the locked negative set; +- one-sided 95% TPR lower bound at or above 90% on untouched positives; +- no declared hard-negative stratum above 0.5% FPR; +- useful TPR on the temporal holdout and transformation suite; +- meaningful discrimination on causal pairs or same-provider hard negatives; +- calibrated `abstain` behavior outside supported providers and strata. + +These are research gates, not promises that the proprietary decoder's operating +point has been reproduced. + +### Experiment D5: robustness and drift + +Evaluate identity, JPEG and WebP, resize, crop, padding, rotation, color changes, +blur, noise, overlays, screenshots, and combinations. Preserve a matched +transform-only positive control for every attack family. Report both average +and worst-transform TPR at the frozen threshold. + +Repeat a small fixed collection after provider model or surface changes. A +shift in score distribution opens a new model epoch; it does not silently +recalibrate the old threshold. + +## Localization program + +A whole-image detector is not automatically a useful removal loss. Test whether +its evidence is spatially causal with three independent methods: + +- tile occlusion and replacement with distortion-matched controls; +- detector-gradient attribution for the learned model; +- phase or template energy mapped back to spatial blocks. + +For each source, create top-ranked, random, and bottom-ranked edits with equal +pixel norm and the same codec path. The oracle batch is registered before any +results return. + +Gate: top-ranked edits must reduce matching-oracle detection more often than +random edits under paired analysis. Aggregate per-image differences and confirm +the direction with a sign test. If localization does not transfer, do not build +a region remover around it. + +## Pixel-only removal program + +All candidates preserve image dimensions and avoid a generative decoder. Run +the following ladder in order. + +### Experiment R1: analytical carrier subtraction + +Estimate provider, model-epoch, and geometry-specific residual components from +verified probes or causal pairs. Test: + +- complex FFT projection with conjugate symmetry; +- wavelet-band projection; +- per-channel and opponent-color residual subtraction; +- spatially varying strength based on local texture and detector attribution. + +Sweep signed amplitude, not only attenuation. Include a sham edit with identical +pixel norm outside candidate bins. This is the highest-value path because a real +shared post-hoc residual could be removed at very high fidelity. + +### Experiment R2: constrained per-image optimization + +Optimize pixels against an ensemble of frozen local detectors. The objective +combines detector margin with L-infinity and L2 bounds, LPIPS or DISTS, MS-SSIM, +edge consistency, OCR preservation, and face-embedding preservation where +applicable. Expectation over transformations covers lossless export, JPEG/WebP, +small resize, and color conversion so the result is not a fragile local +adversarial example. + +The optimization may read only local detectors. The provider oracle evaluates +a frozen candidate batch afterward and never supplies gradients, search +direction, or per-step feedback. + +### Experiment R3: feed-forward residual remover + +If R2 transfers to the provider oracle, distill successful minimal residuals +into a compact image-to-residual network. Constrain the output amplitude and +frequency distribution explicitly. Train on one detector ensemble and select +on the held-out detector family. Preserve the per-image optimizer as the +reference implementation. + +### Experiment R4: detector-remover co-evaluation + +Evaluate four combinations separately: + +1. local detector positive, provider oracle positive; +2. local detector negative, provider oracle positive; +3. local detector positive, provider oracle negative; +4. both negative. + +Case 2 is the critical surrogate-overfit failure. Add every permitted example +to a future hard set only after the current experiment is closed; never tune and +report on the same oracle failure. + +## Removal success gate + +A provider-specific remover is a release candidate only when a locked, +source-positive evaluation shows all of the following: + +- at least 90% matching-oracle `not_detected` results, with `indeterminate` + counted as failure; +- a positive matched transform-only control for every evaluated source; +- exact-size output with no semantic regeneration stage; +- median PSNR at least 40 dB and fifth-percentile PSNR at least 35 dB; +- median SSIM at least 0.99, plus LPIPS or DISTS reported rather than optimized + silently; +- no OCR regression on text strata and no material face-identity regression on + face strata under preregistered thresholds; +- no worse oracle-negative rate after a standard downstream JPEG/WebP/resize + suite; +- byte-identical pass-through for detector-negative inputs by default; +- a significant paired advantage over random and norm-matched sham edits; +- no claim of forensic cleanliness without a separately trained removal-artifact + detector and a held-out evaluation. + +If pixel-only methods fail this gate, retain diffusion regeneration as the +explicit fallback. Do not combine its success with pixel-only results. + +## Provider sequence + +### OpenAI first + +1. Resolve the oracle authorization gate. +2. Build the corpus schema and confound challenge. +3. Verify or reject the shared-carrier hypothesis on low-texture probes. +4. Train classical and learned detectors. +5. Freeze the detector and temporal test. +6. Run R1, then R2, then R3 only after each preceding gate passes. +7. Submit one preregistered final oracle batch. + +OpenAI work establishes the experimental machinery, not parameters for Google. + +### Google second + +Repeat the full sequence with Google-native positives, Google hard negatives, +and the Gemini oracle. Spend the small manual oracle budget on controls and +decisive boundary points, not uniform sweeps. Use local detector uncertainty to +choose a batch before submission, then freeze it. Test Gemini app and AI Studio +surfaces as separate strata because their export and metadata paths differ. + +## Implementation order + +The research harness remains outside the public API until the gates pass. + +1. **Implemented:** use the research corpus schema and auditor documented in + [`data/synthid/research-manifest.md`](../data/synthid/research-manifest.md) to + record provider, surface, model epoch, session, content stratum, parent hash, + transform lineage, separate C2PA and SynthID outcomes, oracle session, and + artifact hashes. +2. **Implemented:** build a label-free local inventory before promotion so byte-identical files, + decoded-pixel duplicates, and unsupported formats are visible without + inferring evidence from directory names. +3. **Implemented:** add a corpus auditor that rejects hash-group leakage, missing parent links, + ambiguous labels, and unsupported oracle-provider pairs. +4. **Harness implemented; evidence run pending:** run the manifest-driven D1 + challenge over container, thumbnail, and canonical decoded-content + baselines. Freeze its validation threshold and report same-provider negative + cohorts separately. +5. Generalize `scripts/synthid_pixel_probe.py` into reusable feature extraction + while preserving its current synthetic tests. +6. Add reproducible train/evaluate commands whose output is a versioned model + card and metrics snapshot, never an unversioned console claim. +7. Add analytical and optimization removal harnesses with norm-matched controls. +8. Reuse the existing fidelity scripts for PSNR, SSIM, OCR, face, and edge + measurements, adding only missing metrics. +9. Package provider-specific detector weights behind an optional dependency only + after the detector gate passes. +10. Add a runtime remover and CLI surface only after the removal gate passes. + +Pure feature, manifest, split, threshold, and residual-constraint logic must be +unit-tested without model downloads. Real model and oracle runs stay explicit +research jobs. + +The inventory, manifest auditor, and D1 confound harness now exist as local +research tools. D1 has not produced a real-corpus metric yet because existing +artifacts have not been promoted into evidence-bearing provider manifests. This +is an evidence gap, not permission to infer labels from their paths. The next +real D1 run begins only after ordinary rows cover train, validation, locked test, +same-provider hard negatives, and a two-class temporal holdout. + +## Hypothesis register + +What is open, what is closed, and what would move each one. A status changes only +against recorded evidence in the log below, never against an argument. `Refuted` +and `supported` name a scope: a hypothesis refuted for one provider or for one +question stays open for the others, and the scope column is the load-bearing part. + +Statuses: `open` (never tested), `partial` (tested in one scope only), `supported`, +`refuted`, `running`. + +### The labels + +Everything else in this register is conditional on these. They were assumed for +months and tested for the first time on 2026-08-15. + +| # | Hypothesis | Status | What moves it | +| --- | --- | --- | --- | +| L1 | A missing `watermarked.unbound` assertion means no watermark was embedded | `refuted` for both providers | Settled 2026-08-16. OpenAI: the official verifier called 65 of 94 non-asserted rows `detected`, a contamination rate of `69%`, with 9 of 9 interleaved health positives detected so the verifier was demonstrably answering throughout. Google: its own API emits watermarked images, per documentation that states so outright, carrying no assertion. Every corpus-labelled AUC in this document rests on a negative class that is roughly two-thirds positive | +| L2 | A watermark assertion means a pixel watermark is actually present | `partial` | Ten source controls returned `detected` across both providers. Ten of twelve thousand is a smoke test, not a rate. As of 2026-08-16 the verified-positive count is past thirty: Google web-verifier source arms across three accounts, the OpenAI provenance-API batches v1 and v2 (every asserted source detected), and 9 of 9 interleaved health positives. Still no denominator, so still not a rate. Extending the same runner to a frozen sample of asserted rows would give one. Update 2026-08-19: the first health miss (21/22) resolved as a bad label, not a verifier flip; the row `originals/2026-06-25/198ae69e...` is classified `openai/asserted` yet carries `claim_generator: Google C2PA Core Generator Library` and Google-side geometry, and the verifier's `not_detected` is the correct answer for it. The health design caught a corpus labeling error, which is the first direct evidence that assertion rows also carry at least one mislabeled member | +| L3 | Date and model version do not carry the label | `supported` | The matched sets fix size, generator and codec but not date, and 57 of 70 Google controls fall in one month. Tested directly on 2026-08-16: a date-only classifier scores AUC 0.524/0.491, so date carries nothing and the covariate re-run is no longer load-bearing | + +### Where the signal lives + +| # | Hypothesis | Status | What moves it | +| --- | --- | --- | --- | +| S1 | The periodic comb carries the watermark | `refuted` | Settled 2026-08-16. The structure every periodic expert reads is destroyed by a seven-pixel crop while the published mark retains 99.97% TPR under aggressive crop and resize, so it is a generation-pipeline lattice anchored at the image origin, not the watermark | +| S2 | The signal is the broadband content between the comb lines | `partial` | First direct oracle arm 2026-08-15: the equal-RMS broadband randomization (54.27 dB, local score unchanged by construction) remained `detected` by the Google verifier, as did every other arm of that account-isolated group. One image group at one perturbation class is a bound, not a sweep, but the between-lines content was randomized without moving the verdict, which is evidence against this scope. The local readout is blind between the comb lines by construction, so no local rerun can substitute; a wider amplitude ladder needs new provider quota | +| S3 | The signal is the shape of the normalized mid-band spectrum | `supported` for OpenAI, `open` for Google | OpenAI confirmed under nested family and lambda selection on 2026-08-15: `0.510` shipped to `0.605`, CI [+0.045, +0.147], above the permutation null. The Google branch was tested the same way on 2026-08-16 and did not improve: `0.678` against `0.689` shipped, CI [-0.071, +0.046]; at 63 controls a 0.02-0.03 gain is unprovable there in principle, so Google stays control-limited | +| S4 | The carrier is content-dependent, so no fixed key-derived pattern exists | `refuted` for Google, `supported` for OpenAI | Settled 2026-08-15 by cross-image tile correlation. Google carries one fixed phase-coherent pattern, present in the controls at about half the amplitude; OpenAI carries none at chance level. What remains open is whether the Google controls are unwatermarked at all, which is L1 | +| S5 | The signal needs a learned nonlinear representation, not hand statistics | `refuted` at the tested capacity | First slice 2026-08-17: a 4-layer conv on opponent-residual patches, translation-invariant by construction (unaligned random patch offsets), leave-one-era-out. Against photographs it reaches 0.6-0.94, against foreign generators 0.44-0.56 with reversals to 0.11-0.15 on fresh eras, at a permutation null of 0.54-0.59: it learns "AI render versus photograph", nothing vendor-specific. A larger model remains free to try but inherits the same leave-one-era-out and leave-one-vendor acceptance gate | +| S6 | Some hand-crafted statistic of the patch bank separates in every era, vendor and codec cell | `refuted` | Settled 2026-08-17 by the full-grid audit: of 124 features, zero clear minimum AUC 0.55 over 27 cells (best worst-cell 0.481), and a corpus-fitted ridge transfers to fresh eras at 0.388 native, 0.178 after JPEG-95 | + +### Robustness + +A detector that fails these is not shippable whatever its AUC. + +| # | Hypothesis | Status | What moves it | +| --- | --- | --- | --- | +| R0 | A candidate expert survives a crop that is not tile-aligned | `refuted` for every expert built so far, including the shipped registered-v3 default | A seven-pixel offset takes the red-green agreement from `0.962` to `-0.111` at every size on large-v1, and on 2026-08-16 a two-pixel diagonal crop killed every one of the 36 registered-v3 detections across 28 foreign-generator acceptances and 8 Google provenance positives. This is now a precondition for calibration, not a later robustness check | + +| # | Hypothesis | Status | What moves it | +| --- | --- | --- | --- | +| R1 | The patch-local feature survives JPEG, resize and crop | `partial`, and its numbers await clean labels | Re-score the matched set under each transform. The earlier patch-bag model kept one positive in 486 after JPEG-95, so this is a gate and not an improvement | +| R2 | The OpenAI result is not an artifact of PNG encoding or one geometry | `supported`, on contaminated labels | Repeat the evaluation inside a single exact size and a single month | + +### Scope and universality + +| # | Hypothesis | Status | What moves it | +| --- | --- | --- | --- | +| U1 | One feature space serves both providers | `refuted` at the current features | Transfer scores `0.535` and `0.469`. A shared representation, or an honest per-provider router with abstention | +| U2 | A watermark detector does not fire on non-watermarked AI images from other generators | `refuted` | Measured 2026-08-15 at feature level (the Google shipped feature accepts 34% of Adobe Firefly output) and 2026-08-16 through the production entry point: 29/223 signed non-Google generator images accepted, `0.130`, Firefly `0.241`. Photographic controls cannot bound this rate; see the two empirical entries below | +| U3 | The detector works on the distribution it will actually meet | `open` | Both classes are drawn from files that still carry C2PA. A re-saved image carries none, and that is the case a user brings | + +### Method validation + +These do not test SynthID. They test whether our method can find a watermark it +is known to contain, which is what separates "the features are weak" from "the +data is too small". + +| # | Hypothesis | Status | What moves it | +| --- | --- | --- | --- | +| M1 | The pipeline recovers a watermark we embedded ourselves | `supported` for a fixed-structure mark at SynthID's amplitude | Open watermarkers, VideoSeal, Watermark Anything, Stable Signature, give unlimited matched pairs. A pipeline that misses those has a method problem, not a data problem. The 2026-08-15 dwtDct ladder adds the boundary: full strength AUC `0.990`, quarter strength (RMS `0.563`, our SynthID measurement range) `0.936`, so the pipeline is sound at the amplitudes we work at. The M1b/M1c ablation the same day bounds every hand-statistic branch: at equal RMS a fixed mark scores `0.670`, a keyed mark `0.669`, and a keyed perceptually masked mark `0.546`, and demasking does not recover it (`0.538`). Masking, not keying or content dependence, is what defeats hand features | +| M2 | Matched pairs can be minted rather than mined | `refuted` | Imagen, whose `addWatermark` the plan was built on, is absent from the model garden and every id 404s. The `gemini-*-image` models that replaced it reject the parameter, and the documentation states that all generated images carry a SynthID watermark. No current Google path can emit an unwatermarked image. Re-verified independently on 2026-08-16: `imagen-4.0-generate-001` and `imagen-3.0-generate-002` return 404 in us-central1 and us-east1 on both GCP projects, `gemini-2.5-flash-image` rejects `addWatermark` at request parsing (`Cannot find field`), and the current Gemini API documentation states for both the Gemini-image and Imagen paths that all generated images include a SynthID watermark, with no disable parameter documented. Settled on live API and documentation. Extended 2026-08-17 by the attack routes on the OpenAI side: pixel attacks to PSNR 20.1 (JPEG q40, 0.5 resize, sigma-8 noise plus JPEG q85, 0.35 resize plus JPEG q50) and a foreign-VAE decoder-substitution round-trip to PSNR 22.3 all left the official verifier answering detected, so attacking a verified positive cannot mint a verified negative on either provider by any mechanism tried | +| M3 | `folded_template_score` is correct | `supported` | Settled 2026-08-15: an independent reimplementation of the fold scored over the same sample agrees with the shipped implementation to `3.7e-4`, the residual traced to float32 in the cv2 blur. Removal and detection results still share the one implementation, but it now matches an independent one | + +### The shipped decision + +| # | Hypothesis | Status | What moves it | +| --- | --- | --- | --- | +| D1 | Some usable fraction of images can be decided at a precision-first threshold | `refuted` at one percent control acceptance | Asked and measured 2026-08-16: at one percent control acceptance the decidable positive fraction is 0-4.5%. A precision-first threshold that decides almost nothing is not a usable product surface | + +## Empirical log + +### 2026-08-09: D1 confound pilot and codec challenge + +The manifest-driven D1 harness was run on the frozen Google pilot containing +five oracle-positive images and 330 deduplicated exact-geometry external +photographs. The manifest passed byte, decoded-pixel, lineage, and split +verification. It is not evidence-ready because the locked test contains no +same-provider hard negative. + +The container-only baseline separated the labels perfectly because the pilot +still exposes format and export-geometry differences. That result is a measured +confound, not watermark evidence. The canonical 8x8 decoded-content baseline +was much weaker: locked-test AUC was 0.620 and temporal AUC was 0.533, and its +validation-frozen threshold detected neither held-out positive. The existing +positive-only RGB plus HSV S/V ensemble still detected all five positives and +emitted no positive verdict on the 330 frozen external images. + +A codec challenge re-encoded every positive without changing geometry. The +ensemble remained positive on five of five JPEG-95 outputs and five of five +WebP-95 outputs; JPEG-90 retained three of five. This rejects a bare PNG versus +JPEG container explanation, but does not exclude a generator or source-pipeline +correlate. With only five positives, the one-sided 95% lower bound on TPR is +54.9%. With zero positives among 330 external images, the one-sided 95% upper +bound on FPR is 0.904%, still nine times the 0.1% detector target. Four positives +also participated in model fitting, leaving only one independent temporal +positive. The next valid detector claim still requires new oracle-positive +images and ordinary same-provider oracle-negative controls. + +A post-freeze source-provenance challenge then added three exact-geometry +Google originals that had not influenced fitting or threshold selection. Each +carried the same signed Google LLC C2PA issuer, trained-algorithmic-media source +type, and explicit SynthID-present assertion as the detected temporal control. +The ensemble abstained on all three: two lacked active carrier support in both +branches, and the third passed RGB evidence but missed the HSV S/V evidence and +support gates. These are provider-signed embedding assertions rather than +matching-oracle pixel labels, but zero positives in three new same-geometry +images falsifies the current ensemble as a general Google SynthID detector. The +measured phase family is retained only as an epoch- or surface-specific +correlate pending a broader oracle-labeled corpus. + +A follow-up cross-epoch check found that the correlate is not entirely confined +to the original five images. RGB and HSV models fitted only on the three later +images ranked each of the five earlier images above all 329 external negatives +in both color spaces (AUC 1.0), but thresholds derived from the later fitting +scores transferred poorly: the strict RGB-plus-HSV decision retained only one +of five earlier positives at zero false positives on the 279-image holdout. An +eight-fold leave-one-positive-out refit across both source groups then detected +six of eight excluded positives with the same RGB-plus-HSV conjunction and each +branch frozen just above its 50-image calibration maximum. RGB alone put all +eight excluded positives above that maximum, while HSV missed the geometric and +low-texture images; the per-fold RGB operating points also produced between one +and eight false positives on 279 held-out negatives. +This is evidence for a transferable but content- and epoch-sensitive Google +pixel correlate, not a shippable detector. The experiment still lacks ordinary +same-provider oracle negatives, and its external-negative set is too small for +the 0.1% false-positive target. + +### 2026-08-09: fixed spectral-template baseline rejected + +An exploratory Google template was reconstructed from four public, purported +clean/marked pairs. Their provenance and oracle status could not be established, +so they were used for discovery only. A scalar phase-consensus score and its +threshold were selected on those pairs, four older Google-oracle positives, and +30 external negatives. + +The frozen threshold then produced six false positives on a new 100-image +external holdout. It also detected only one of four newly generated Gemini +images collected after threshold selection. Duplicate-image removal cannot +reduce the false-positive rate enough to approach the 0.1% detector gate, and +the temporal result is far below the required sensitivity. The baseline is +therefore rejected, not recalibrated on the holdout. + +This result rules out the fixed phase template as a detector or removal loss. +It does not rule out a content-adaptive or model-epoch-specific signal. The +next detector must learn from independently labeled provider data, must retain +the failed holdout unchanged, and must demonstrate discrimination from export +format and generator identity. + +### 2026-08-09: cross-color low-texture consensus rejected + +A polarity-invariant consensus template was trained on black, white, and red +Gemini low-texture probes and frozen before evaluation. Its median score fell +from 0.574 on the training groups to 0.354 on held-out probes of the same +colors and 0.011 on unseen blue, green, and gray probes. External negatives had +a median score of 0.0047. The separation therefore depended on the training +colors and did not generalize across the intended low-texture stratum. + +This branch is rejected as a current Google detector. It remains useful as a +negative control demonstrating why high training coherence is not evidence of +a shared carrier. + +### 2026-08-09: external V3 codebook detector pilot + +The numeric format-v2 V3 artifact from `reverse-SynthID` was loaded with pickle +disabled and evaluated independently; no third-party code was imported or +executed. A 256-bin phase score with a discovery-frozen threshold of 0.5 +detected five of five Google-oracle-positive images and produced zero false +positives on 194 external images collected before and after threshold freeze. +The pilot includes four older positives and one newly generated temporal +positive. It is promising discovery evidence, not a released detector: + +- zero errors on 194 negatives cannot support a 0.1% FPR claim; +- the set lacks enough same-provider, oracle-negative hard controls; +- the phase profile is a third-party artifact whose positive provenance is not + independently reproducible from our current corpus; +- a secondary `active_weight_fraction` shortcut failed on an exact-resolution + holdout, firing on 17 of 50 new negatives, and is rejected as a + native-resolution/content confound. + +The next detector milestone is to reproduce the phase carrier from our own +deduplicated positives, expand the negative set to at least the rule-of-three +minimum, and add same-provider hard negatives before fitting or publishing an +operating point. + +### 2026-08-13: external V4 cross-color codebook audit + +The later `aloshdenny/reverse-SynthID` V4 artifact was audited independently at +commit `b110836`. The audit read only numeric NPY members from its NPZ ZIP with +pickle disabled and reproduced the published native-resolution phase score; +no third-party Python or pickle payload was executed. Its bundled +`sample_watermarked.png` crossed the documented `phase_match > 0.52` threshold +at `0.535`, confirming that the reproduction follows the intended path. + +The operating point did not transfer. Taking the better of the two published +model profiles detected 141 of 355 Google positives and 191 of 499 controls. +On a later frozen 1,000-image Open Images reserve it accepted 386 controls. The +phase score had AUC 0.517 on the paired corpus. Reproducing the July 2026 GUI's +0.3%-aspect-ratio gate reduced coverage without adding discrimination: it +supported 121 positives and detected 46, while supporting 343 controls and +detecting 132; it also detected 284 of 685 supported fresh controls. + +The failure follows directly from the metric. Per-bin phase similarity is +`1 - abs(wrapped_difference) / pi`, whose expectation under independent uniform +phase is 0.5, only 0.02 below the threshold. The detector selects bins by +cross-color phase coherence alone and does not require image-side carrier +amplitude or a content-baseline excess. Its selected bins span nearly the full +spectrum, with a typical spatial period around 2.5 pixels. Cross-color +coherence is useful for discovering candidates from genuinely low-texture +references, but this single-image score is not a calibrated detector. +Two prespecified rerankings did not rescue it. Restricting carrier periods to +4-128 pixels and ranking by either coherence times reference magnitude or the +stored carrier weight times reference magnitude produced best-of-model AUCs of +0.521 and 0.524. They were evaluated as continuous scores without selecting a +new test-set threshold and are also rejected. + +The repository's older V3 phases also failed this external challenge. Their +best dark/white phase score had AUC 0.473; the documented 0.78 phase threshold +accepted 5 of 355 positives, 5 of 499 controls, and 6 of the later 1,000 fresh +controls. The repository's current bundled sample and four validation sources +all remained below that phase threshold. The README's accuracy claim is +therefore not accepted as evidence for the current corpora. + +The linked Hugging Face dataset cannot repair OpenAI calibration as published. +Its DALL-E 3 `black` and `white` buckets are not solid-color probes: per-image +mean luminance ranged from 11.66 to 143.74 and 124.94 to 235.66 respectively. +Across ten 1024-square images per bucket, only 39 and 32 RGB frequency bins had +phase coherence above 0.9, versus 20,443 and 100,252 for the corresponding +Gemini 3.1 buckets. Only four DALL-E bins cleared 0.9 in both colors and all +four were trivial DC or self-conjugate-axis coordinates; Gemini had 11,478 +nontrivial joint bins and 3,341 also had cross-color absolute-cosine agreement +above 0.9. Those DALL-E folders measure heterogeneous content phase, not an +OpenAI solid-reference codebook. The official SynthID-Image paper also says +nearly uniform corner cases may be excluded from watermarking, so generating +flatter references does not guarantee a stronger watermark. A useful next +external experiment requires newly collected low-content OpenAI outputs whose +pixel watermark is separately confirmed, plus same-provider oracle-negative +controls, before freezing any phase bins. + +### 2026-08-09: independently fitted phase-carrier pilot + +An exact-geometry phase model was then fitted from four of our own +Google-oracle-positive images. The first unconstrained search was a clear +multiple-comparisons failure: training scores were 0.983-0.991, while the +newly confirmed fern positive scored 0.061 and an exact-resolution negative +reached 0.105. Leave-one-out discovery detected only two of four older +positives. That unrestricted model is rejected. + +A second model used only the 256 frequency coordinates proposed by the numeric +V3 artifact, while relearning phase, weight, and expected magnitude entirely +from our four positives. This restricted model scored the fern validation +positive at 0.740 and the four fitting positives at 0.936-0.963. A threshold of +0.43 was frozen after 50 validation negatives whose maximum was 0.305. + +The phase-only epoch failed its first new holdout with three false positives in +121 exact-size images. All three had active carrier weight between 0.07 and +0.14, so the failed epoch was closed and a new rule was frozen: phase score at +least 0.43 and active weight at least 0.50. That second epoch produced zero +false positives on 279 newly downloaded exact-size images. + +A separate 3,000-image COCO check also produced zero joint-rule positives, but +it is not counted toward the supported-stratum FPR bound. Every COCO image had +been upscaled to the model geometry and every active weight was at most 0.201. +The rule therefore abstained because the inputs were outside its measured +carrier-support distribution. Treat active weight as an `abstain` gate, not as +negative SynthID evidence. + +This is the first positive local-detector pilot derived from our labels, but it +does not clear the detector gate. It still depends on third-party candidate +coordinates, has only five confirmed positives including the fitting images, +has only 279 fresh negatives inside the current support stratum, and has no +same-provider hard negatives. The next epoch requires a larger independently +labeled positive set, at least 3,000 native-support negatives, same-provider +oracle negatives, and a new temporal positive that has not influenced feature +or threshold selection. + +### 2026-08-09: color-space spectral comparison + +The restricted phase-carrier experiment was repeated in RGB, full-range +YCbCr, YCoCg, an orthogonal opponent basis, CIE Lab, and HSV. Every branch +started from the same 102 spatial frequencies: the top 256 external codebook +coordinates contained 102 unique `(row, column)` pairs, which were expanded +over all three components. Each branch then independently selected 256 of the +306 component-frequency candidates and relearned phase, expected magnitude, +and weight from the same four oracle-positive fitting images. The score was +the phase score multiplied by active carrier weight, so weak spectral support +reduced rather than merely qualified the evidence. + +Thresholds were frozen from the fern validation positive and 50 exact-size +validation negatives before scoring the 279-image comparison set. The +comparison set was locked for this color-space branch, although it had already +served as the second epoch's RGB negative set and is therefore not a globally +virgin corpus. + +| Space | Validation gap | Comparison negative max | Fern minus negative max | False positives | +|---|---:|---:|---:|---:| +| RGB | 0.509 | 0.097 | 0.459 | 0/279 | +| YCbCr | 0.330 | 0.193 | 0.312 | 0/279 | +| YCoCg | 0.437 | 0.163 | 0.390 | 0/279 | +| Opponent | 0.351 | 0.167 | 0.339 | 0/279 | +| Lab | 0.408 | 0.126 | 0.382 | 0/279 | +| HSV | 0.530 | 0.084 | 0.495 | 0/279 | + +HSV had the best observed worst-negative margin, narrowly ahead of RGB, but +did not generally shift paired negatives below RGB. After normalizing each +space by its fern score, the median paired HSV-minus-RGB difference was 0.0015 +and a two-sided sign test gave `p=0.632` (144 higher, 135 lower). The result is +therefore a tail observation, not evidence that HSV dominates RGB. + +Channel decomposition localized the useful effect. HSV hue contributed only +0.012 of fern evidence and its fern-minus-holdout-maximum channel gap was +-0.024; saturation and value contributed 0.333 and 0.233, with positive gaps +of 0.273 and 0.194. In YCbCr, YCoCg, opponent, and Lab, the luminance-like +channel supplied the largest and most stable separation, while chroma-channel +gaps were much smaller. This supports a carrier expressed across intensity and +contrast structure, with useful nonlinear saturation/value projections, not a +hue-specific watermark. + +This experiment does not raise the detector's evidence grade. It still has +only one held-out confirmed positive, reuses the same third-party spatial +frequency prior, lacks same-provider hard negatives, and evaluates only one +native geometry. A future HSV S+V ablation must freeze its own operating point +and use a new negative epoch and new oracle-positive images; dropping hue now +would be a post-holdout model choice. + +### 2026-08-09: pixel-only R1 pilot did not transfer + +Three non-generative candidate families were frozen locally before Gemini +verification: + +1. local visible-region inpainting changed 0.087% of pixels at 47.64 dB PSNR + and 0.99963 SSIM; +2. sparse complex-spectrum subtraction used 256, 4,096, or 16,384 numeric V3 + bins while preserving native 2816x1536 geometry; +3. deterministic spatial fragmentation combined smooth local warps, one global + resampling operation, mild color changes, and codec round-trips. + +Local inpainting left the 256-bin phase score effectively unchanged. Healthy +Gemini sessions continued to detect both the target and its spatially shifted +sham, refuting an earlier one-off `not detected` response. + +The widest spectral candidate reduced the frozen 256-bin phase score from +0.639 to 0.124 while retaining 51.94 dB PSNR and 0.99854 SSIM. A +pixel-identical, metadata-normalized source control was detected in two healthy +oracle sessions. The 16,384-bin candidate remained detected in one of them and +the second hit quota before returning a verdict. The 4,096-bin candidates also +remained detected. Therefore the external phase carrier is correlated with the +Google signal but is not a sufficient removal loss. + +One additional profile returned `not detected` for the 16,384-bin candidate, +then also returned `not detected` for the untouched source. That entire session +is invalidated rather than counted as removal success. This failure adds a new +oracle-health rule: every account/session batch starts with the same +pixel-identical positive control; candidate verdicts are discarded when that +control is not positive. Quota and tool-unavailable responses remain +`indeterminate`. + +The frozen fragmentation batch is not yet oracle-resolved because healthy +profiles exhausted their rolling quota. Its balanced and strong variants also +fall below the release fidelity gate locally, so even a future negative oracle +result would be a mechanism-discovery result, not a releasable remover. + +A follow-up bounded-warp batch replaced RMS-normalized displacement with strict +per-axis maxima of 1.0, 1.8, and 2.8 pixels. The 1.0-pixel variant retained +37.79 dB PSNR and 0.9784 SSIM but left the local phase score essentially +unchanged at 0.643. The 2.8-pixel variant reduced the score only to 0.503 while +falling to 28.21 dB and 0.8570 SSIM. A multi-transform bounded variant reached +0.159 locally only at 21.97 dB and 0.6749 SSIM. Spatial fragmentation therefore +does not approach the release fidelity gate in this implementation and remains +a preregistered mechanism probe for the next healthy oracle window, not a +product candidate. + +### 2026-08-09: positive-only RGB plus S/V detector epoch + +The color-space result was converted into a deliberately one-sided research +detector. It requires exact 2816x1536 geometry, RGB evidence of at least 0.302, +HSV saturation-plus-value evidence of at least 0.302, and active carrier +support of at least 0.50 in both branches. It emits `positive` only when every +condition passes and emits `abstain` otherwise; it never claims that SynthID is +absent. + +Leave-one-positive-out fitting passed all four older Google positives. Their +minimum held-out RGB evidence was 0.527, minimum held-out S+V evidence was +0.452, and minimum active support was 0.600. The final frozen models detected +all five available oracle-positive controls. A new Picsum epoch downloaded +IDs 600-999; after exact-geometry validation and content-hash deduplication +against all earlier epochs, 330 images remained. None produced a positive +verdict. However, 328 of 330 abstained for insufficient carrier support and +only two reached both support gates before falling below the evidence +thresholds. This validates a positive-only signal catcher inside one geometry, +not a two-sided detector or a general FPR claim. + +### 2026-08-09: ensemble phase projection is not a sufficient removal loss + +An analytical pixel-only attack removed the positive complex projection onto +the learned RGB phases and HSV S/V phases. One iteration at strength 0.5 +cleared the local ensemble at 56.03 dB PSNR, 0.99909 SSIM, and a maximum pixel +change of two levels. A norm-matched random-noise control remained locally +positive. + +The provider result rejected this loss as sufficient. In a Gemini session +where the untouched fern control returned `SynthID Status: Detected`, all +three preregistered projection candidates at strengths 0.5, 0.75, and 1.0 also +returned detected. The strongest candidate had negative local RGB and S+V +evidence at 51.97 dB PSNR. The ensemble therefore tracks a real correlate but +does not cover all evidence used by Google's decoder. + +### 2026-08-09: 16x32 periodic-tile carrier hypothesis + +The failed global projection exposed a more specific spatial structure. Of +the top 256 external V3 bins at 1536x2816, 208 (81.25%) lie exactly on an FFT +lattice spaced by 96 rows and 88 columns. This corresponds to a spatial cell +of `1536 / 96 = 16` rows by `2816 / 88 = 32` columns. Instead of subtracting a +small list of harmonics, a new pixel-only probe high-pass filters the image, +folds the residual modulo 16x32 over 8,448 repeats, and subtracts the complete +zero-mean folded template. + +On the fern positive with its visible sparkle already removed, the mildest +locally clearing candidate used Gaussian sigma 0.6 and strength 1.5. The +phase-aligned template cleared the ensemble at 57.03 dB PSNR and 0.99911 SSIM; +the same template shifted by one row and one column remained positive at +57.02 dB and 0.99912 SSIM. Two additional aligned/shifted pairs reproduced the +direction at 55.96 and 54.19 dB. + +The per-image estimator was then run on the four older positives with a frozen +strength ladder. The minimum locally clearing strength was 2.0 for three +images and 1.5 for one. All four phase-aligned outputs abstained, while all +four same-strength one-pixel-shifted controls remained positive. Selected +outputs retained at least 55.67 dB PSNR and 0.99874 SSIM. This is the strongest +local causal evidence so far because effect direction follows exact tile +phase at matched distortion across five contents. + +It is not a removal result. In a healthy Google verifier session, the fern +source control was detected and the mild aligned sigma-0.6, strength-1.5 +candidate was also detected. The stronger local variants therefore remain +unresolved rather than inheriting a negative label from the local ensemble. +Cross-resolution tile discovery and same-provider hard negatives also remain +open. + +### 2026-08-09: provider-specific phase losses failed both remote verifiers + +The Google result above and a separate OpenAI experiment closed the first +analytical-loss epoch. The OpenAI branch fitted an 8-pixel phase pattern from +OpenAI low-texture outputs and produced a 46.70 dB candidate whose local phase +score changed sign. A single frozen submission to OpenAI Verify still returned +`SynthID detected`; C2PA was absent because the candidate had been re-encoded. +This confirms that metadata removal and a local phase-score reversal do not +establish pixel-watermark removal. + +The experiments are not cross-provider replications. Google images were sent +only to Google Verify AI and the OpenAI candidate only to OpenAI Verify. No +carrier, threshold, response, or verdict was transferred between tracks. +OpenAI's published verifier guidance prohibits repeated queries for watermark +reverse engineering or evasion, so that one failed query did not start an +adaptive loop. Further OpenAI candidates remain local hypotheses unless an +authorized research oracle becomes available. + +### 2026-08-09: JPEG-robust periodic EOT surrogate pilot + +Small residual CNNs were trained separately for each provider as differentiable +attack surfaces. They are provider classifiers, not SynthID detectors: their +corpora lack current same-provider oracle-negative controls, and several +external negatives approach the positive range. Two same-architecture models +were used for optimization and a third model with a different first kernel and +channel schedule was kept outside the attack. + +For Google, all three models scored the temporal fern positive at a 75th-patch +percentile between 0.938 and 0.962. A patch-local L-infinity-8 attack initially +cleared the models and survived JPEG, but visual inspection exposed rectangular +grid boundaries; that candidate was rejected despite favorable scalar metrics. +A replacement optimized one 16x32 residual cell repeated over the full image, +with differentiable JPEG-95 and JPEG-90 branches. At half strength it retained +36.80 dB PSNR and 0.9241 SSIM against the icon-clean source. The three PNG +scores fell to 0.024, 0.047, and 0.016; after an actual JPEG-90 round trip they +were 0.182, 0.243, and 0.016. The independent RGB and HSV S+V phase ensemble +also abstained, and remained below threshold after JPEG-90, JPEG-95, WebP-95, +and a 0.98 resize round trip. + +For OpenAI, the held-out complex market image scored 0.936, 0.929, and 0.813 +on its three provider classifiers. An independently optimized 8x8 periodic EOT +residual was then attenuated in low-texture regions. The selected local +candidate retained 34.97 dB PSNR and 0.9356 SSIM. Its PNG scores were 0.030, +0.053, and 0.099; after actual JPEG-90 they were 0.042, 0.063, and 0.178. +WebP-95 behaved similarly, while a 0.98 resize raised the independent model to +0.692 and remains a measured weakness. A second optimization that included a +resize branch improved the two optimized models but transferred worse to the +independent model, so it was rejected. + +The periodic residuals were then tested for cross-image transfer without +further optimization. The OpenAI 8x8 cell learned from the market image was +applied with the same low-texture attenuation to 11 other OpenAI positives. +Every image moved downward on all three models, both as PNG and after an actual +JPEG-90 round trip. Median PNG scores changed from 0.971, 0.970, and 0.931 to +0.016, 0.033, and 0.036. Each model had 11 of 11 paired decreases, giving a +two-sided exact sign-test `p=0.00098`; per-image PSNR ranged from 33.50 to +35.99 dB. Two complex scenes retained materially higher scores on the +independent model, so the transfer is strong but not uniform. + +The Google 16x32 EOT cell was likewise applied to the four older images after +their own locally clearing phase-tile subtraction. All four remained phase +ensemble abstentions, and every CNN score decreased in the lossless outputs. +JPEG-90 preserved the direction but restored the two optimized models to +roughly 0.58-0.78 on the hardest images. Fidelity was approximately 37.25 dB +PSNR. With only four paired images this is descriptive transfer evidence, not a +statistical or oracle gate. + +These are positive local robustness results, not watermark-removal results. +Neither candidate meets the release fidelity gate, neither CNN ensemble has +cleared the detector-identifiability gate, and neither selected EOT candidate +has a matching-provider negative oracle verdict. A three-file Google batch is +hash-frozen outside the repository for the next healthy quota window: the +positive source, the stronger phase-only ablation, and the combined periodic +EOT candidate. Results will be accepted only in that fixed order without +between-query adaptation. Two later attempts detected the positive source, then +returned a usage-limit response for the phase-only ablation. In the latest +attempt the verifier requested a retry after 17 hours. Both responses are +`indeterminate`, and the combined candidate was not submitted out of order. The +OpenAI candidate is not scheduled for the public verifier under the current +usage restriction. + +### 2026-08-09: OpenAI fidelity and architecture-transfer frontier + +Further local experiments tested whether the OpenAI periodic result could move +above the release fidelity gate without losing transport robustness. Chroma-only +projection reached 47.43 dB PSNR, but an actual JPEG round trip removed the +perturbation and restored the two classifier scores to approximately 0.9. Mixed +luminance/chroma projections and stronger texture masks did not improve that +frontier. A regularized Adam attack reached 41.21 dB on the source image, but +JPEG-90 restored the optimized classifiers to 0.90 and 0.91, and the independent +architecture scored 0.89. This rejected post-hoc color projection and simple +regularization as fidelity remedies. + +A second experiment put actual OpenCV JPEG and WebP results in the forward pass +and used a straight-through gradient. At 40.08 dB, actual JPEG-90 still scored +0.89 and 0.91 on the optimized models. Removing the luminance penalty improved +the first model to 0.76 but left the second at 0.88 and reduced fidelity to +38.78 dB. The exact-codec forward pass therefore corrected an invalid transport +approximation, but did not produce a new Pareto candidate. + +An architecture-diverse fourth classifier used fine RGB residuals, broader RGB +residuals, and luminance/opponent-color residuals. The market image remained +outside its training split. Its 75th-patch percentile ranged from 0.983 to 0.988 +on four held-out OpenAI positives; the median across 120 held-out COCO negatives +was 0.041 and the maximum was 0.521. This is useful attack-surface diversity, not +a certified detector, because no current same-provider oracle-negative corpus is +available. + +Adding that model to periodic optimization did not improve architecture transfer. +The preregistered local winner retained 34.86 dB and scored 0.057, 0.197, and +0.021 after JPEG-90 on the three optimized models, but the independent model +scored 0.573. A per-image half-resolution perturbation produced a higher-fidelity +41.86 dB candidate; JPEG-90 scores on its optimized models were 0.587, 0.572, +and 0.471, while the independent model remained at 0.875. Optimizing only the +two original models produced the same failure on both independent architectures. + +The measured local frontier is therefore unchanged. The 34.97 dB periodic +candidate remains the strongest transport-robust and cross-architecture result; +the candidates above 40 dB are architecture-specific and cannot support a +removal claim. No additional public OpenAI verifier requests were made during +these experiments. + +### 2026-08-09: mechanism reset and paired open-method control + +The first detector epochs optimized the wrong statistical problem. The +[technical reference](synthid.md#11-post-hoc-model-independent-design) now +records the paper's paired training target, symmetric transformation channel, +independent detection logit, and two-sided calibration. Applied to the initial +pilots, those details reject three implicit assumptions: that a carrier phase +can stand in for the detection score, that mean patch classification is an +adequate image-level decoder, and that a sigmoid threshold calibrated on +unrelated negatives is a deployable decision rule. + +A bounded open-method control tested the architecture concern without claiming +that another watermark reproduces SynthID. TrustMark P was used only because +its public encoder creates causal clean/watermarked pairs and its public decoder +can validate transformed labels. Ninety public COCO images were split by source +image into 60 training, 15 calibration, and 15 test pairs. Each pair received +the same identity, JPEG-90, 0.8 resize round trip, or 5% crop transformation. +The official decoder detected 14/15, 15/15, 14/15, and 15/15 transformed marked +test images respectively, with zero detections on the corresponding clean +images. + +The existing normalized residual patch recipe reached only 0.707 identity AUC +and 0.668 aggregate AUC on the held-out pairs. A small full-field model that +retained RGB amplitude plus fine and coarse residuals reached 0.947 identity AUC +and 0.841 aggregate AUC. At a threshold above every calibration negative, their +aggregate TPRs were 5% and 20% respectively. Both models failed to reproduce +the official decoder's JPEG robustness. This small proxy is not a TrustMark +benchmark and says nothing directly about SynthID accuracy. It does falsify the +claim that the current patch-normalized architecture and training recipe are an +adequate generic neural-watermark detector. + +The resulting eight falsifiable hypotheses now define the canonical +[D3-D5 mechanism program](#mechanism-hypotheses-for-d3-d5). In particular, +external generator corpora remain a final false-positive challenge and do not +substitute for counterfactual negatives. + +Until the counterfactual-label and full-field gates pass, the current residual +CNN remains a useful vendor-triage stage for a future cascade, but it is not a +SynthID detector and should not be optimized as the final decision surface. + +### 2026-08-09: paired wavelet and spectral ablation + +The paired spectral harness was extended with three-level undecimated `db2` +wavelets, complex Fourier phase coherence and power, cepstral peaks, and a +cyclic clean/marked permutation control. The implementation streams wavelet and +spectrum field accumulators rather than stacking transformed fields across +pairs. + +On 60 public TrustMark P training pairs at canonical size 256, the true +residuals had mean inter-pair RGB NCC of 0.097-0.121. Cyclically mismatching each +clean image with the next marked image reduced all three channels to about +-0.017. The strongest coherent Fourier cluster occurred around vertical offsets +9-11 and was strongest in the blue channel. Wavelet repeatability was greatest +in horizontal detail bands at levels 1 and 2. These are causal residual +measurements for the open proxy, not evidence about SynthID's carrier. + +The representations then faced a source-disjoint single-image test with 60 +training, 15 calibration, and 15 test pairs under identity, JPEG-90, 0.8 resize, +and 5% crop transformations. A fixed spectral-template score reached only 0.618 +identity AUC and collapsed to 0.498 under crop. A 318-feature wavelet/spectral +summary reached 0.653 aggregate AUC and zero TPR at the threshold above every +calibration negative. + +Full-field neural ablations at size 128 gave the following aggregate results: + +| Input | AUC | Paired wins | Test FPR | Test TPR | +| --- | ---: | ---: | ---: | ---: | +| Spatial RGB plus residuals | 0.723 | 57/60 | 0% | 11.7% | +| SWT maps only | 0.671 | 59/60 | 0% | 0% | +| Complex FFT maps only | 0.516 | 41/60 | 0% | 0% | +| Spatial plus SWT, late fusion | 0.665 | 59/60 | 5.0% | 8.3% | +| Spatial plus SWT plus FFT, early fusion | 0.639 | 57/60 | 1.7% | 1.7% | +| Spatial plus SWT plus FFT, late fusion | 0.698 | 54/60 | 0% | 0% | + +The table uses one threshold above every calibration negative; test FPR is +reported separately because a clean calibration result does not guarantee a +clean locked test. + +The SWT branch contains weak, unusually consistent paired evidence, but it did +not improve the spatial model's source-disjoint discrimination in this small +proxy. The FFT map branch was indistinguishable from chance. Therefore neither +representation advances into the primary detector by default. SWT remains a +late-fusion ablation for a larger paired corpus; complex spectral analysis +remains a residual-discovery diagnostic unless a future held-out test reverses +this result. + +### 2026-08-09: closed-corpus architecture and score ablations + +A temporally split closed three-class corpus then tested whether the open-proxy +architecture result transferred to real external-source images. These labels +establish source classes, not watermark presence; they can measure +generalization and false positives but cannot substitute for counterfactual +clean/marked labels. + +The frozen normalized-patch baseline was repeated over three training seeds. +Individual original-image AUCs ranged from 0.864 to 0.879. Averaging the three +models increased AUC to 0.884, but a threshold above every validation negative +still produced one false positive among 259 test negatives. At that operating +point the two positive-source TPRs were only 2.45% and 8.97%. Calibrating near +1% validation FPR yielded 1.54% test FPR and TPRs of 6.68% and 18.16%. +Accepted positives had 98.8-100% vendor accuracy, so vendor attribution is +useful only after an independently reliable presence decision. + +A naive full-field spatial model reached only 0.599 AUC. Adding a separately +encoded stationary-wavelet branch and late fusion reduced AUC to 0.597. Both +had effectively zero strict TPR. The result does not contradict the causal +open proxy: it shows that global average pooling over source-labeled images +does not isolate the transferable signal and that SWT does not repair the +source shift. + +Post-hoc image aggregation also failed to create an operating point. Averaging +presence logits raised AUC from 0.884 to 0.890, but increased strict test FPR to +0.77% while reducing TPR. Median, lower-quartile, upper-quartile, and top-tail +aggregation either preserved the overlap or suppressed almost every positive. +A separately trained binary presence head plus vendor head reached 0.880 +ensemble AUC, 0.77% strict test FPR, and TPRs of 1.11% and 7.26%. Decoupling the +heads is architecturally faithful but insufficient without faithful labels. + +These ablations close spectral feature engineering, naive full-field pooling, +and score aggregation as the next detector step. The next valid training epoch +requires same-content counterfactual pairs or independent watermark oracle +labels, matched transformations, and an image-level detection loss. External +generator corpora, including difficult non-target providers, remain hard +negative and FPR-challenge sets only. + +### 2026-08-10: low-content controls and registered phase carrier + +A same-resolution low-content matrix compared independently generated solid +outputs from two target model families against three per-image controls: exact +mean fill, amplitude-matched Gaussian noise, and a phase-randomized residual +with preserved Fourier magnitude. Raw stationary-wavelet summaries transferred +between the two target families with AUCs of 0.982 and 1.000, and reached 0.973 +when blue and green were held out by color. This was not watermark evidence. +The frozen classifier accepted every one of 1,869 external negatives because +it had learned the distinction between real generator texture and artificial +controls. Removing absolute wavelet energy reduced external-negative +acceptance only to 61.6%, with similar 58.1-67.3% acceptance across all three +source classes. Both low-content wavelet branches are rejected as presence +detectors until real non-target solid outputs provide matched negatives. + +The numeric V3 audit loader was then extended to support both dense and sparse +format-v2 profiles without pickle. Exact-profile evaluation exposed a sharp +encoder-version boundary. The 1024x1024 profile accepted none of 231 target +provider images and none of 26 exact-geometry negatives. The 1536x2816 profile +accepted 30 of 55 target-provider images, including all four temporal-test +images, while rejecting the one exact-geometry negative available in the +closed corpus. The independently fitted phase model accepted 24 of those 55 +and also accepted all four temporal-test images. This is positive evidence for +a geometry- and epoch-specific carrier, not a universal SynthID decoder. + +On the four temporal-test positives, the fixed V3 score survived JPEG-95 and a +75% downscale on all four images, survived JPEG-85 on two, and failed after a +5% center crop or a one-pixel translation on all four. Bounded analytical +translation registration recovered all four shifted images and selected the +known `(-1, -1)` offset. Searching up to 16 pixels produced no positives among +50 exact-resolution and 144 canonicalized frozen negatives. The shared +registration implementation now serves both the numeric V3 probe and the +independently fitted phase model. + +A discovery-only scale-and-translation view search recovered all four 5% +cropped temporal images with the independently fitted model after lowering the +active-support gate from 0.50 to 0.40. It produced zero positives on the 194 +frozen negatives and on the same preregistered 3,000-image COCO challenge used +by the identity scorer. The latter result has a zero-error one-sided 95% bound +of 0.0998% only for that abstention challenge: every COCO image remained +outside carrier support, with a maximum active fraction of 0.201. The scale +rule is not frozen because its support threshold was selected after inspecting +the crop examples. It requires a new temporal positive holdout before it can +join the detector rule. + +The current actionable research candidate remains a positive-only, +provider-specific expert for the supported 1536x2816 carrier epoch. Identity +and bounded translation views use the frozen phase and support thresholds; +unsupported geometry, insufficient carrier magnitude, and ambiguous phase +return `abstain`. Vendor attribution may select the expert that supplied +accepted evidence, but it must not turn an abstention into a provider label. +The next calibration gate still requires at least 3,000 native-support +negatives, same-provider oracle negatives, matched non-target solid outputs, +and a new temporal positive that influenced neither profile nor threshold. + +### 2026-08-10: 2048 periodic-tile detector + +The phase hypothesis transferred to a second native geometry when the model +and operating point were kept separate. A 256-peak 2048x2048 model learned from +111 train positives used the already frozen phase threshold of 0.43, active +support threshold of 0.40, and a translation search of plus or minus four +pixels. It accepted 31 of 35 validation positives and 27 of 30 locked-test +positives while accepting none of 49 validation or 38 test negatives. +Translation registration recovered the shifted positives without changing +those identity counts. A threshold recalibrated from only 49 validation +negatives reached 30 of 30 test positives but accepted one test negative, so it +was rejected in favor of the transferred rule. + +The wider native-geometry challenge exposed the remaining uncertainty. The +frozen 2048 rule accepted two of 182 earlier external-provider images, for two +accepted source negatives among all 269 native negatives. Both cases passed at +zero translation with high phase and support, and both also passed an +independently learned HSV phase branch. They are operational false positives +under source labels, but source provenance does not establish watermark +absence. They may instead expose a shared encoder or upstream backend. Without +an independent watermark oracle they cannot be relabeled either way. The same +experiment rejected the 1024x1024 and 768x1376 experts: they accepted 9 of 26 +and 4 of 9 native source negatives, respectively. + +The 2048 carrier has a concrete periodic mechanism. Its 256 peaks reduced to +108 unique spatial frequencies. Translating the frequency coordinates by 128 +rows preserved 56 coordinates, while the maximum overlap in each of 1,000 +uniform random controls was two. The permutation estimate was 0.001, and the +128-bin spacing implies a 16x16 spatial tile. A separate detector therefore +folded a high-pass residual modulo 16x16, averaged 16,384 repetitions, and +correlated the normalized tile against a train-positive template. After +float64 serialization and validation-only threshold calibration, the fixed +tile accepted 34 of 35 validation and 29 of 30 test positives, none of the 49 +calibration or 38 held-out test negatives, and the same two of 182 earlier +source negatives. This second representation supports a real periodic carrier +rather than an arbitrary set of FFT peaks, but it does not resolve the two +labels. + +The fixed tile also accepted none of a preregistered 3,000-image general-image +challenge after deterministic 2048x2048 canonicalization. The maximum +normalized correlation was 0.106 against a threshold of 0.174. The zero-error +one-sided 95% upper bound is 0.0998% for that challenge. Unlike the earlier +sparse-phase result, the tile score evaluates every image without a weak +carrier-support abstention. The challenge is still not native provider data +and cannot replace the required oracle-negative calibration. + +A later exact-1024 content audit found that perceptual siblings can cross an +image-level fit split even when file and decoded-pixel hashes differ. The local +2048 reports retain aggregate counts and model hashes but not the 111 fitting +paths, so the old train/validation/test split cannot now be audited for that +specific leakage mode. This does not invalidate the independently collected +general-image challenges or justify changing the frozen runtime model, but the +reported positive rates remain conditional on the historical split. Any next +model calibration must retain content-group membership and prove that no group +crosses train, validation, test, or temporal partitions. + +Symmetric attack evaluation established the robustness boundary. The fixed +tile accepted 29 of 30 original test positives, all 30 after a 75% downscale +round trip, 21 after JPEG-95, three after JPEG-85, and none after a 5% crop, +with no accepted held-out negatives under the identity threshold. A +validation-calibrated JPEG-95 tile threshold recovered 27 of 30 but accepted +one of 38 test negatives. Requiring both codec-conditioned tile and phase +scores reduced JPEG-95 to 16 of 30 with no held-out-negative acceptance, but +still accepted one of the 182 earlier source negatives. The corresponding +JPEG-85 consensus accepted 5 of 30 positives and none of all 269 native source +negatives. Scale-and-translation phase search recovered 15 of 35 validation +and 14 of 30 test crops with no held-out-negative acceptance, but remains +discovery-only because the test transformation had already influenced the +branch. Low-frequency peak subsets and transform-augmented phase training +improved JPEG sensitivity only by raising validation false positives to +2-10%, so both were rejected. + +The reproducible implementation is `scripts/synthid_periodic_tile_probe.py`. +It stores the normalized template in float64 and calibrates only after loading +the serialized artifact; an earlier float32 experiment moved a boundary score +by approximately 2.5e-10 and demonstrated why calibration-before-serialization +is invalid. The resulting research detector is positive-only and limited to a +confirmed 2048x2048 carrier epoch. An accepted expert may suggest the encoder +family, but the two cross-source carrier matches prohibit a stronger vendor +claim until an oracle distinguishes direct provider output from shared-backend +output. + +A pixel-space ablation then tested whether the frozen tile merely predicted the +local scores or controlled them. At twice the train-median tile norm, aligned +subtraction changed the fixed-tile decision from 29 of 30 accepted originals to +zero and the independently fitted sparse-phase decision from 27 of 30 to zero. +The median fidelity was 53.74 dB PSNR and 0.99681 SSIM. Cyclically shifting the +same tile by one row and column left one phase acceptance, while a seeded +zero-mean random tile orthogonal to the learned template left 13; their median +PSNRs were within 0.13 dB of the aligned edit. For both local representations, +the aligned edit reduced the score more than either control on all 30 paired +images, with a two-sided sign-test p-value of 1.86e-9 for each comparison. +The same aligned edit reversed both local decisions on each of the two disputed +cross-source matches, while the orthogonal control left both phase decisions +accepted. That two-item result is descriptive but makes an accidental threshold +crossing less likely; it still cannot distinguish a shared carrier from direct +provider output. + +This is the strongest local causal evidence for the 16x16 mechanism, but the +strength was selected after inspecting this locked test and is therefore +discovery-only. The shifted control also suppressed the phase representation +substantially, so local score reversal cannot certify signal removal. A matching +provider oracle must still compare untouched sources, aligned candidates, and +norm-matched controls before any removal claim. The reproducible local harness +is `scripts/synthid_periodic_tile_ablation.py`; its report records input and +model hashes without writing derivative images. + +### 2026-08-12: fast adaptive carrier projection + +The fixed twice-median subtraction was intentionally conservative but spent +substantially more pixel norm than many images needed. A follow-up tested a +one-dimensional per-image solver. For source pixels `x`, frozen unit carrier +`t`, and scalar amplitude `a`, the candidate is +`clip(round(x - a * tile(t)))`. A bounded scalar search selects the smallest +`a` that reaches the local target. This is a direct pixel edit: it uses no +generative decoder, does not resize the image, and searches one scalar rather +than optimizing a full-resolution tensor. + +The first challenge reused the 30 historical 2048x2048 causal-ablation +sources. Three were not simultaneously positive under the fixed-tile and +independently fitted phase rules and were excluded before counting a reversal. +All 27 conjunctive positives cleared both local rules after adaptive +subtraction. Median PSNR was 59.83 dB and median SSIM was 0.99890; the minima +were 55.19 dB and 0.99705. Repeating the same selected amplitude with a +one-row, one-column cyclic shift cleared zero of 27. Sixteen original images +remained conjunctively positive after JPEG-95. The aligned candidates cleared +all 16 after the same round trip, while the shifted controls cleared one. This +improves materially on the earlier fixed-strength median of 53.74 dB without +weakening the local causal control. + +A three-image pilot also compared subtracting the complete per-image folded +high-pass residual and a regularized inverse-high-pass reconstruction. Both +could clear the two local scores, but their selected candidates retained only +about 54-57 dB PSNR, compared with 59-62 dB for the frozen carrier projection +on the same images. Removing the whole folded residual therefore spends pixel +norm on content-correlated periodic structure and was rejected in favor of the +narrow carrier direction. + +The geometry challenge then selected three `first_only` carrier positives from +each of the 20 most frequent native Spaces geometries. The edit repeats the +same 16x16 carrier past the image boundaries and crops the repetition to the +decoded dimensions, so it does not require dimensions divisible by 16. A zero +score target cleared 60 of 60 aligned candidates and 5 of 60 matched shifted +controls. Median PSNR and SSIM were 60.37 dB and 0.99919; the minima were +53.56 dB and 0.99581. These samples come from an already inspected detector +corpus, so they measure mechanism transfer across geometry rather than fresh +detector or removal generalization. + +The zero target left one WebP-95 and eight resize survivors among transformed +sources that were still locally positive. A frozen follow-up used a target +score of -0.25. It cleared all 60 native candidates at median 57.70 dB PSNR and +0.99863 SSIM, with minima of 52.38 dB and 0.99443. Conditional on the matching +transformed source remaining above the fixed carrier threshold, the aligned +candidate cleared 32 of 32 JPEG-95, 13 of 13 JPEG-90, 8 of 8 JPEG-85, 15 of 15 +WebP-95, and 54 of 54 0.75x resize round trips. The corresponding one-pixel +shifted controls cleared 7, 1, 1, 5, and 17. The increased shifted-control rate +is the measured price of adding transform margin. + +The frozen -0.25 recipe was finally run once on one deterministic first-carrier +positive from every represented native Spaces geometry. All 647 images at 647 +distinct decoded sizes reached the target, with no maximum-amplitude failure. +Median fidelity was 57.67 dB PSNR and 0.99867 SSIM; the worst case was 51.78 dB +and 0.99400. Of 323 source images whose carrier survived JPEG-95, 323 aligned +candidates and 58 shifted controls cleared after the same round trip. Natively, +153 of 647 shifted controls also cleared at this deeper margin. The consistent +aligned result supports geometry-independent control, while the increased +shifted rate shows that the conservative edit perturbs a wider neighborhood of +the periodic carrier. Because the all-geometry corpus had already influenced +detector research, it is a coverage challenge rather than a fresh holdout. + +This is a fast local carrier suppressor, not a verified SynthID remover. The +earlier Google oracle remained positive after analytical edits that already +looked favorable to local models, and the adaptive scalar recipe has not been +submitted to a matching-provider oracle. Provider terms and the project oracle +policy also prohibit turning verifier responses into an adaptive search loop. +The next valid test is one preregistered source/re-encode/aligned/shifted batch +at a healthy authorized oracle, with the amplitude rule and -0.25 target frozen +before submission. Until that batch returns a matching-provider negative result, +no production command exposes this method as removal. + +The reproducible implementation is +`scripts/synthid_adaptive_carrier_suppress.py`. It writes only lossless PNG, +refuses overwrite and locally negative input, and records source/output hashes, +the selected scalar amplitude, local scores, fidelity metrics, and runtime in a +sidecar JSON report. Unit tests cover non-divisible geometry, target attainment, +an unreachable target, and the negative-input guard. A real CLI smoke test on a +Spaces source reproduced its reported score exactly through the runtime detector +after saving. It remains a research script rather than a routed package command. + +### 2026-08-10: confirmatory oracle-batch readiness + +The corpus audit found no new 2048x2048 positive collected after the tile rule +and subtraction strength were frozen. Existing test images cannot be renamed as +a temporal holdout, so the confirmatory oracle batch remains blocked on new +provider output at the supported native geometry. + +The batch mechanics are ready in `scripts/synthid_oracle_batch.py`. For each new +source, it preregisters five requests in a fixed order: untouched source, +lossless re-encode, aligned subtraction, cyclic-shifted subtraction, and an +orthogonal norm-matched tile. It refuses to write media inside the repository, +records artifact and decoded-pixel hashes, pins both model artifacts and the +matching provider, and detects any post-registration byte change. Oracle results +live in a separate manifest-bound file so the preregistration remains immutable; +the evaluator requires complete ordered results, verbatim responses, +timezone-aware timestamps, and separate SynthID and C2PA verdicts. + +A causal success requires the source, re-encode, shifted, and orthogonal views +all to remain `detected` while only the aligned view becomes `not_detected`. +`Indeterminate` and refusal remain indeterminate. A one-source dry run on an old +test positive verified all five artifacts and the local scoring path, but is +pipeline validation only and contributes no confirmatory evidence. + +### 2026-08-10: first confirmatory oracle attempt + +Four new 2048x2048 Gemini outputs were generated after the local rule and +strength were frozen, then registered as a 20-request batch. The manifest and +all 20 artifacts passed hash verification before submission. The first source +group produced these ordered results: + +| Role | Local tile/phase decision | Provider result | +| --- | --- | --- | +| source | detected / detected | SynthID indeterminate; Google C2PA present | +| re-encode control | detected / detected | Google AI signal detected | +| aligned subtraction | detected / not detected | Google AI signal detected | +| shifted control | detected / detected | Google AI signal detected | +| orthogonal control | detected / detected | refused: verification quota exceeded | + +The quota response instructed the account to wait 24 hours. No adaptive retry +was made, and the remaining 15 preregistered requests were not submitted. The +batch cannot receive its preregistered aggregate verdict until complete ordered +results exist. + +Even incomplete, this attempt rejects the frozen removal recipe for the first +temporal source: aligned subtraction crossed the phase threshold but remained +above the tile threshold and did not clear the provider oracle. The remaining +sources can still measure transfer and disagreement between the two local +experts, but they cannot turn this first aligned result into a universal +pixel-only removal success. + +### 2026-08-10: positive-only runtime detector + +Removal is deferred while the transferred tile signal is exposed as a bounded +detector. `src/remove_ai_watermarks/synthid_detector.py` loads the frozen model +as a bundled pickle-free runtime asset and returns `detected`, `not_detected`, +or `unsupported` without resizing input. The direct API and +`detect-synthid` CLI initially covered only native 2048x2048 images; `identify` +consumes a positive match as high-confidence evidence but never treats a +negative or unsupported result as proof of absence. + +The operating point and model are unchanged from the locked experiment. The +detector accepted all four post-freeze Gemini outputs, including the source +whose frozen phase score missed, while retaining the prior 29-of-30 locked-test +sensitivity and zero accepted validation/test negatives. The unresolved two +external-source matches remain the reason a runtime positive names the carrier +but does not attribute a provider. + +### 2026-08-10: calibrated native-geometry extension + +The frozen 16x16 template and threshold transfer across the common native +geometries in the provider-positive evidence. Non-divisible image dimensions +use direct modulo folding with per-cell sample counts; divisible dimensions, +including 2048x2048, retain the original numerical path. All prior 2048x2048 +records were replayed and matched exactly, including floating-point scores. + +The runtime registry was expanded only after the unchanged threshold accepted +none of 60,000 public COCO challenge views, 3,000 at each of 20 target +geometries. This is a geometry extension of the same positive-only carrier +expert, not an OpenAI pixel detector or a proprietary payload decoder. + +### 2026-08-11: calibrated image-size range + +The fixed template was then evaluated on every provider-positive image in the +local evidence set rather than only its common geometries. It accepted 3,928 of +4,698 images across 757 exact geometries. Sensitivity separated by pixel count: +1,987 of 2,021 images at or above three megapixels crossed the threshold, while +1,940 of 2,672 images from one through two megapixels did. This establishes a +carrier-family boundary, not universal SynthID recall: explicit C2PA watermark +actions also occur below threshold, and three strong carriers use a different +cyclic phase. + +Two public-image geometry challenges tested whether geometry itself creates +false matches. The first balanced 5,000 COCO images across all 757 observed +geometries, with every geometry present in both development and final partitions; +the maximum fixed score was 0.12549 and none crossed the unchanged 0.17357 +threshold. The second transformed the same 5,000 source images at 256 generated +geometries from one through 18 megapixels, covering every width/height remainder +pair modulo 16. Its maximum was 0.16387 and again none crossed the threshold. +Runtime support therefore uses that challenged pixel-count interval instead of +an enumerated geometry registry, still without resizing. + +A cyclic-registration branch was rejected after the public-image challenge. +At a low threshold it recovered 130 fixed-phase misses but false-fired on +tracked OpenAI and Adobe controls. Raising the threshold above all development +controls recovered only 23 in-range provider positives while adding 84 +cross-source matches beyond the fixed detector. Three strong shifted Google +carriers remain a documented miss rather than weakening the runtime rule. + +Top-level source labels were not treated as watermark-negative truth. Of 237 +non-Google-labeled corpus images crossing the fixed threshold, 81 retained +SynthID provenance. High-scoring Adobe manifests explicitly named Gemini Flash +as the remote provider, and perceptual matching found direct Google derivatives +under editing or relabeling categories. The remaining source labels cannot prove +absence of the carrier, so they constrain provider attribution but do not justify +raising the pixel threshold. + +### 2026-08-11: spatial-resampling boundary + +Arbitrary decoded geometry did not imply arbitrary resize robustness. A +stratified sample of 80 fixed-positive provider images was rescaled in memory at +0.5, 0.625, 0.75, 0.875, 1.125, 1.25, and 1.5. The unchanged 16x16 detector +accepted none of the 80 images at every nonidentity scale. Restoring each view +to its source dimensions recovered 58-80, which isolates the failure to the +scaled sampling lattice rather than simple information destruction. + +Scaling the template and modulo fold to the corresponding integer periods 8, +10, 12, 14, 18, 20, and 24 recovered 73-80 of the discovery images at the +native threshold. Searching all eight periods raised control scores, so a +single threshold was frozen above the maximum of 3,000 deterministically +resized COCO development views. It accepted none of 2,000 final controls and +672 of 800 source-disjoint provider positives, with 77-88 accepted at each +scale. Six tracked OpenAI and Adobe controls remained below the frozen threshold. + +This did not generalize to arbitrary scaling. Factors 0.8, 0.9, 1.1, 1.2, and +1.333 create noninteger carrier periods; rounding the period accepted zero of +80 images in the fixed-phase branch at every factor, and cyclic registration +accepted only 1-16. Period-specific thresholds improved sensitivity but accepted +five of the 2,000 final controls. The integer-period bank is therefore retained +as a research result, not added to the runtime. A universal resize branch needs +fractional-period registration or reliable reconstruction of the pre-resize +geometry, followed by a new multiple-search calibration. + +A discovery follow-up tested that fractional-period direction with the 30 +strongest template harmonics. A continuous 7.5-24.5 spectral search placed the +true period within 0.05 pixels among its top three candidates for 58 of 60 +transformed positives. Searching nine neighboring integer reconstruction +geometries recovered 44 of 60 at the native threshold; even the true source +geometry recovered only 48, so resampling damage sets a separate upper bound. +The full search then failed a frozen 250-development/150-final COCO challenge: +the calibration threshold accepted two final controls. A preselected product +of canonical correlation and spectral confidence increased positive +acceptance from 49 to 52 of 60 at its development threshold but accepted the +same two final controls. This fractional FFT baseline is rejected for runtime +use; better period estimation alone does not solve multiple-search specificity. + +### 2026-08-11: multiscale calibration and opt-in registered detector + +The fractional search was revisited as a multiple-search calibration problem, +not as period estimation alone. A source-disjoint challenge used 200 confirmed +fixed-carrier positives, 40 independent sources at each scale 0.8, 0.9, 1.1, +1.2, and 1.333. Controls were 4,900 unique COCO images resized into the positive +pixel-count domain, split into 2,900 development and 2,000 final items. The ten +frozen candidates combined exact fractional canonical correlation, quadrant +agreement, and a three-level scale-space score through mean or minimum rules. +Pure box-counting dimensions, multifractal width, lacunarity, roughness, +wavelet-energy slope, and a generic log-Mellin response had already failed the +smaller pilot and were not promoted to the large challenge. + +At the development-only threshold, the three-way mean accepted 177 of 200 +positives versus 163 for canonical correlation alone, with 16 positives rescued +and two lost in the paired decision. Each produced one final false positive, on +different source images, so the development threshold was rejected. Recalibrating +above all 4,900 COCO controls left 176 of 200 three-way positives and 160 of 200 +canonical-only positives. This recalibration is exploratory rather than a new +validation because the final controls had then been inspected. + +A second source-disjoint corpus froze those thresholds before scoring 105 new +positives across scales 0.5, 0.65, 0.8, 1.0, 1.2, 1.333, and 1.5, plus all 879 +content-deduplicated Kodak and Picsum controls available locally. Natural +2816x1536 controls had a heavier null tail than upscaled COCO: the three-way +mean accepted 82 of 105 positives and six controls at the COCO threshold, +whereas canonical-only accepted 79 positives and ten controls. A threshold +above all 5,779 controls retained 253 of the first 305 positives for the +three-way mean. Four coarse pixel-count bins recovered only six more positives, +so size normalization did not explain the main 0.5x miss. + +At 0.5x, a fixed period-8 branch raised canonical correlation for seven of the +15 discovery positives but also won the unconstrained search on 629 of 879 +controls. Searching it unconditionally merely raised the null threshold. The +frozen remedy requires period-8 native correlation of at least 0.35 and takes +the maximum of the old and period-8 three-way scores. It added four positives +without an accepted control in the discovery corpus. On the earlier 4,900 +controls, the frozen gate never opened and none crossed the final three-way +threshold `0.2460603834083705`. + +The improvement itself was then tested on 50 additional source-disjoint 0.5x +positives that influenced neither gate nor threshold. The ungated three-way +score accepted 6; the frozen gate accepted 17, rescuing 11 and losing none. The +exact paired sign test was `p = 0.0009765625`. These results support an opt-in +scale-registered detector over 250,000 through 10,000,000 decoded pixels. The +runtime now requires both sides to be at least 256 pixels because its frozen +confirmation needs complete disjoint 256-pixel patch groups. Narrower inputs +need a separately calibrated adaptive-patch expert. These results do not establish a +universal SynthID decoder: the control maximum helped set the operating point, +33 of 50 new 0.5x positives still missed, and crop, strong codec changes, other +carrier epochs, and provider attribution remain outside the claim. The fast +native fold remains the default and the `identify` path. + +That period-8 conclusion was overturned by a later symmetric negative test. +Every one of the 11,506 known non-Google Spaces controls was downscaled by 0.5, +matching the positive transformation; 11,273 outputs remained in the registered +geometry range. The direct analytic 8-by-8 template was dominated by shared +resize structure: 80.8% of its energy projected onto a 2-by-2 lattice and 94.0% +onto a 4-by-4 lattice. The production period-8 gate opened on 73 controls, and +all 73 crossed the registered threshold. They spanned hosted jobs, Firefly, +China AIGC-labeled generators, Meta-tagged images, OpenAI, and local tools. No +tested secondary condition separated the 11 rescued positives from those 73 +controls. The explicit period-8 rescue is therefore rejected and not part of +the runtime detector. The 0.5x rescue result is retained here as an incident +record against asymmetric transformed-positive evaluation. + +The same symmetric corpus also invalidated the main single-threshold registered +rule: 68 of 11,273 supported controls crossed `0.2460603834083705`. Fixed +thresholds for nine recovered-period bins eliminated those errors while +retaining 238 of 355 transformed positives, but one of 4,900 COCO controls then +crossed through the third spectral candidate. Requiring the canonical winner to +match the strongest spectral candidate removed that case and all 68 Spaces +errors, with 239 of 355 positives retained. This version was not shipped: a +fresh, content-deduplicated 1,000-image Open Images validation corpus produced +six false positives. + +All six Open Images errors concentrated their apparent match in one +middle-frequency ring. Their correlations disappeared or changed sign in the +two higher-frequency rings of the 16-by-16 carrier, unlike the accepted Gemini +positives. Adding a frozen minimum correlation of `0.075` across those two rings +removed the six development errors and retained 229 of 355 positives. A +previously untouched reserve of 499 supported Open Images controls then produced +five amplitude-gate candidates and zero final positives; their maximum +high-band score was `0.004682175209264222`. + +The retained positives by scale were 0/65 at 0.5, 8/15 at 0.65, 53/55 at 0.8, +33/40 at 0.9, 13/15 at 1.0, 30/40 at 1.1, 38/55 at 1.2, 40/55 at 1.333, and +14/15 at 1.5. A physically transformed template recovered strong correlations +for some period-8 positives, but 975 top-candidate Spaces controls fully +overlapped every tested amplitude, spectral-band, and joint feature. No +zero-control operating point retained a period-8 positive. The calibrated +runtime therefore uses the top-candidate, period-bin, and high-band gates and +makes no 0.5x detection claim. + +An exploratory amplitude-times-high-band rescue later tested whether the two +existing gates could trade evidence instead of both passing independently. A +threshold frozen above all 341 top-1 development controls recovered 11 of the +126 current false negatives and accepted none of a separate 499-image Open +Images reserve; the reserve maximum was 0.100583 against the 0.118168 +threshold. The decisive symmetric challenge rejected it: 19 of all 11,506 +Spaces controls after 0.5x resizing crossed the frozen threshold, with a +maximum of 0.179989. The errors spanned OpenAI, Firefly, Microsoft, hosted +jobs, and other generators. Raising the threshold post hoc above that maximum +would recover only seven false negatives, none at scale 0.65 or 0.8. The joint +rescue is rejected; the independent amplitude and high-band gates remain the +measured operating point. + +### 2026-08-13: public bypass corpus and inverse-carrier audit + +The public `cebeuq/Synthid-Bypass` V2 comparison set supplied 12 external +before/after pairs produced by a diffusion reconstruction workflow. The fixed +runtime carrier accepted 10 of 12 `before` images and none of the 12 `after` +images. The two misses were not assumed negative: Gemini Verify AI reported +SynthID on `before/03.webp` and `before/07.png`, and no SynthID on both paired +outputs. Outputs `01` through `06` were also oracle-negative; the verifier's +daily limit stopped the same-session sweep at `08`. + +`07.png` is a low-SNR instance of the shipped positive direction. Its full-frame +score was `0.1248368819`; its signed RGB correlations were 0.073, 0.183, and +0.111, and all nine aligned region scores were positive. A proposed rescue at +`score >= 0.12` with Green dominance and spatial agreement separated it from +the two highest transformed-COCO controls, but failed a fresh exact-1024 Open +Images challenge. A 2009 natural photograph scored `0.2123730698`, had stronger +Green dominance and nine-region agreement, and Gemini returned `SynthID not +detected`. The downloaded bytes matched a fresh fetch from the official Open +Images validation bucket. This is a confirmed runtime false positive, not a +provider-label inference. + +The failure is broader than that one JPEG. Symmetrically resizing the 24 Kodak +PNG controls to 1536 by 2752 with Lanczos produced fixed scores as high as +`0.3768339306`. Those synthetic views were not used as independent clean +samples, but they prove that rational resize lattices can create a stronger +signed 16-by-16 match than the runtime threshold. The existing 5,000-view +observed-geometry and 5,000-view generated-geometry COCO challenges remain +valid for their exact source-and-transform assignments; they do not justify a +universal transformation claim. No threshold relaxation for `07` is allowed, +and the current direct detector needs a fresh source-and-transform-grouped +specificity epoch before its support range can be broadened. + +`03.webp` tested a different hypothesis. The two content-group-separated +alternate-Gemini experts both rejected it. Its shipped-template score was +instead `-0.3263447029`, all three channel scores were below `-0.28`, and every +one of nine aligned regions was negative. The paired oracle-negative output +weakened to `-0.1531420136`. Neither 5,000-view COCO geometry challenge nor the +fresh Open Images blocks contained a score at or below `-0.20`, making the +opposite-polarity response a plausible additional codeword or provider state. +It is not specific enough to ship: 40 of 11,506 heterogeneous non-Google-labeled +AI controls crossed `-0.20`, with examples from OpenAI, Microsoft, multiple +other generators, and unassigned made-with-AI files. An inverse-template expert +therefore remains a research hypothesis until matching-provider oracle labels +and a new untouched natural and AI control split exist. + +A follow-up decomposed the two high-frequency rings instead of taking their +minimum. The 19 joint-rule false positives were broad-band, with median +`min(ring) / max(ring)` of 0.84, while the registered false negatives had a +median of 0.27. A development-selected rescue therefore multiplied normalized +amplitude by the stronger ring and required ring balance at or below 0.5. A +threshold above 341 development controls recovered 11 of 65 top-1 false +negatives and accepted none of the 499-image Open Images reserve. The full +symmetric challenge again rejected it: 13 of 11,506 resized Spaces controls +passed, with a maximum score of 0.694818 against the 0.308683 threshold. Strong +single-ring lattices occurred in Firefly, OpenAI, hosted jobs, China-labeled +generators, Canva, and other controls. Radial imbalance is therefore another +general renderer or resize feature, not a SynthID-specific rescue. + +A multi-codeword hypothesis then projected the shared 16-by-16 template out of +each normalized residual tile and fitted source-disjoint spherical codebooks to +the remaining 768-dimensional vectors. Five prototypes selected on 300 +positives retained 366 of 600 held-out positives. Three prototypes were highly +reproducible across an independent 300-positive fit, with matched correlations +of 0.88 through 0.95, while the additional states were less stable. A threshold +above 2,500 development natural-image controls accepted none of a separate +2,500-image reserve and selected 225 of 770 metadata-labeled Google images that +the shared-template threshold missed. The broader challenge decisively rejected +the feature: 828 of 11,506 metadata-labeled non-Google Spaces controls crossed +the frozen threshold. Errors occurred across OpenAI, Microsoft, Firefly, +China-labeled generators, hosted jobs, Stable Diffusion, and other sources. A +post-hoc intersection above every control that also passed the registered +min(R, B) gate retained 138 Google candidates, but none passed that independent +carrier gate. The residual clusters are reproducible generator or resampling +structure, not a specific additional SynthID codebook, and are excluded from +runtime. + +A related spatial-payload pilot stopped averaging all 16-by-16 blocks and +measured carrier and codebook responses block by block. The positive carrier +response was spatially coherent: median adjacent-block correlation was 0.61 on +both axes, compared with 0.20 and 0.24 in a heterogeneous Spaces sample and +0.04 and 0.06 among codebook false positives. The development-selected rescue +was instead the standard deviation of the best codebook response across +blocks. It retained 11 of 60 reserve positives and none of 180 reserve controls, +but accepted six of 1,000 fresh heterogeneous controls. Those errors included +OpenAI, hosted, and China-labeled generators; raising the threshold above their +maximum retained only four of the 120 pilot positives. Per-block carrier or +codeword coherence is therefore diagnostic but not a usable rescue. + +A geometry-stratified follow-up found a second reproducible native 16-by-16 +carrier in the fixed-template misses. Templates fitted independently from two +halves of the 1024-by-1024 misses correlated at 0.869, and two independent fits +from the non-1024 misses correlated at 0.956. The latter correlated 0.82-0.84 +with the 1024 template but only about 0.28 with the shipped carrier. This rules +out a single-resolution averaging accident and supports a distinct carrier +direction or state. + +The initial 1024 template retained 8 of 91 held-out misses and accepted none of +81 exact-geometry reserve controls or 500 natural-image reserve controls. Seven +of those eight positives survived JPEG 95, six survived JPEG 85 and 70, and +three survived a 0.75x resize round trip; WebP 95 retained none. A threshold +frozen above the exact-geometry development controls accepted 25 of 11,506 +heterogeneous controls when applied at arbitrary native geometries, so an +unrestricted second native template is rejected. Canonicalizing every control +to 1024 by 1024 eliminated all 11,506 errors, and the same path accepted only +the native 1024-by-1024 Google misses. The signal has a fixed native period, +not a size-normalized frequency. + +A stricter chronological fit used only 1024-by-1024 misses through mid-June and +tested July misses. Its template correlated 0.839 with the independently fitted +July template, retained 5 of 95 temporal misses, and accepted none of 81 reserve +exact-geometry controls. At that frozen threshold, canonicalization accepted 18 +of all 770 fixed-template Google misses and none of all 11,506 heterogeneous +controls; every accepted miss was natively 1024 by 1024. Requiring agreement +between two independently fitted early templates retained three temporal +misses and no reserve controls. This is credible evidence for a second carrier +direction within the Google cohort, but the positive and exact-geometry control +counts are still too small for a runtime operating point. The later external +natural-image challenge below further shows that this repeatability is not yet +specific enough for blind detection. + +Projecting the shipped carrier out of every normalized tile did not remove the +second direction. The early orthogonal template was numerically orthogonal to the +shipped template, correlated 0.834 with an independently fitted July orthogonal +template, and retained the same 5 of 95 temporal misses with none of 81 reserve +exact-geometry controls. Its threshold was frozen above the development controls +before the decisive 1024-by-1024 canonical challenge; none of all 11,506 +heterogeneous controls crossed it, and the maximum remained just below the +frozen boundary. The second direction is therefore not a weak projection of the +shipped carrier. This strengthens the research finding but does not enlarge the +small temporal-positive or exact-geometry control reserves required for runtime. + +Mapping both frozen scores over all 4,698 Google-labeled images changed the +interpretation from an epoch replacement to coexisting carrier states. The map +contained 3,825 first-only, 69 second-only, 103 both, and 701 neither cases; +the two scores correlated at -0.563. Both directions occurred throughout the +collection interval and across multiple native geometries, while the presence +of both in 103 images rules out a simple dated switch from one template to the +other. The second native threshold remains exploratory outside exact +1024-by-1024 inputs because 8 of 11,506 heterogeneous native controls crossed +the orthogonal version's threshold; only the canonical exact-geometry challenge +retained the zero-of-11,506 result. + +Freezing a stricter native threshold above the maximum of 5,753 alternating +development controls left zero crossings in the other 5,753 controls and 26 +Google images missed by the first carrier above threshold. That retrospective +split is promising but not a certified operating point: the corpus had already +been inspected while developing the second carrier, so a fresh source- and +time-disjoint challenge is still required. + +An external natural-image challenge then rejected that strict native threshold. +It accepted none of 1,500 unique Open Images controls and none of 12 Wikimedia +controls, but one of 637 content-deduplicated Picsum controls. The original +Picsum directory contained 1,000 filenames but only 637 unique hashes; the one +crossing appeared twice under different ids and is counted once. Its RGB and +quadrant contributions were highly uneven, unlike the small temporal-positive +set, but that post-error observation cannot retroactively define a gate. The +strict native threshold remains excluded from runtime. + +The original exact-1024 threshold also failed on those external controls: four +of 1,500 Open Images and four of 637 unique Picsum images crossed it, including +native 1024-by-1024 cases. The earlier zero-of-11,506 canonical result was +therefore specific to that heterogeneous control composition, not evidence that +exact geometry alone makes the carrier specific. Requiring agreement between +the two independently fitted early templates did not close the gap; it retained +three temporal positives but accepted one Open Images and two unique Picsum +controls. The second direction remains a reproducible Google-cohort signal, not +a blind detector. + +Requiring the orthogonal score to survive a JPEG-95 round trip was not a +specificity gate either. It retained four of the five temporal positives but +also retained all four Open Images and all four unique Picsum crossings at the +original threshold. The natural-image confound is itself transport-stable. +Together with the failed low-rank, two-template, channel-balance, quadrant, and +two-carrier-plane variants, this exhausts mean-template consensus for the +second direction on the current data. + +The orthogonal carrier is not a cyclic shift or color permutation of the +shipped template. Its best absolute cyclic correlation was 0.212, Fourier +magnitude correlation was 0.480, and weighted phase lock was 0.200. Dominant +frequencies were axial, led by `(0, +/-8)`, `(0, +/-3)`, and `(0, +/-5)` rather +than the first direction's broader structure. A horizontal-axis-only ablation +retained all five temporal detections and no reserve exact-geometry control, but +then accepted three of the 11,506 canonicalized heterogeneous controls. The +errors included an OpenAI-labeled image and two watermark-remover outputs. The +full orthogonal template is more specific than its strongest axial component. + +A final low-rank variant tested whether that direction contains several payload +states rather than one mean carrier. SVD bases of ranks 1, 2, 3, 4, 6, 8, 12, +and 16 were fitted only on early 1024-by-1024 misses. Rank was selected on a +separate June interval and a separate control third, then reported on July +misses and the final control third. The selected rank 12 retained 2 of 38 +validation misses and 4 of 95 test misses, with no control error. Rank 2 happened +to retain 6 test misses but only one validation miss and therefore could not be +selected without post-test tuning. A multi-state subspace did not improve the +evidence-grade operating point. + +Deflation then tested whether a third stable linear carrier remained. After +projecting both the shipped and second orthogonal templates out of every +normalized tile, early and July residual averages correlated only 0.0003. The +early residual template retained none of 95 temporal misses and none of 81 +reserve exact-geometry controls. Within this 1024-by-1024 temporal corpus, the +linear periodic model therefore supports two reproducible carrier directions, +not an open-ended sequence of mean templates. + +Treating those directions as a single two-dimensional carrier plane did not +provide a safer weak-signal rescue. A positive-quadrant Euclidean norm used the +two independently frozen carrier thresholds as axis scales, then froze its own +boundary above 81 exact-geometry development controls. It rescued none of the +Google images below both component thresholds and accepted one of 81 reserve +controls. The two directions must retain separate operating points; combined +subthreshold energy is rejected. + +The five temporal positives above the original second-carrier threshold also +had unusually balanced normalized R, G, and B contributions: their worst +channel ranged from 0.297 to 0.320, while seven of eight native control errors +were below 0.297 and were commonly blue-heavy with a weak green contribution. +A clean chronological validation did not justify turning that observation into +a gate. A template fitted on half of the early positives produced no accepted +examples in the other early half at the threshold frozen from development +controls, despite accepting six July examples and no reserve control. The +channel-balance rule is therefore a post-test hypothesis, not detector logic. + +A phase-only follow-up removed observed Fourier magnitude and averaged phase +agreement using only the early template's fixed spectral-energy weights. The +phase threshold alone retained 2 of 31 early validation misses and 10 of 95 +July misses, but accepted one of 81 reserve exact-geometry controls. Requiring +both phase agreement and the independently calibrated linear projection +removed that control error and all external natural-image errors. The frozen +conjunction retained 1 of 31 early validation misses and 3 of 95 July misses, +accepted none of 162 heterogeneous exact-1024 controls, none of 1,500 unique +Open Images, none of 637 unique Picsum images, and none of 12 Wikimedia images. +Applied outside its declared geometry it accepted 3 of 11,506 heterogeneous +native controls, all at non-1024 sizes. This is the first independently +challenged phase-and-amplitude candidate for the second direction, but three +temporal positives are not enough to ship a runtime branch. + +Across all 443 exact-1024 Google images, the frozen conjunction found 11 that +the shipped carrier missed and 16 that both branches accepted; 244 were found +only by the shipped carrier and 172 by neither. The candidate therefore adds +signal rather than merely confirming the first direction, but its incremental +native recall is still small. + +Swapping the two early fit halves and recalibrating both thresholds retained 4 +of 31 opposite-half validation misses and 4 of 95 July misses, again with none +of 81 reserve exact-geometry controls passing the conjunction. This supports +fit-split signal reproducibility, but the swapped operating point accepted one +of 92 unique exact-1024 Open Images controls in its external challenge; none of +637 unique exact-1024 Picsum controls crossed. A post-hoc consensus of the two +fit halves would require another untouched control set, so it is not selected +from these exposed results. The candidate remains research-only. + +An initial two-expert rule required both independently fitted templates to pass +their own separately calibrated linear and phase thresholds. The next +nonoverlapping Open Images validation-id block was downloaded without scoring; +2,000 candidates yielded 100 unique native 1024-by-1024 controls. The first +expert accepted none, the second accepted one, and their conjunction accepted +none. That rule appeared to add 5 detections across the 443 exact-1024 Google +files, including 2 of 95 July fixed-carrier misses. + +A subsequent content-group audit invalidated those recall numbers. The 443 +files contained three exact decoded-pixel duplicate pairs and ten +perceptual-hash groups at a conservative 16-of-256-bit screening distance. One +early near-duplicate pair had been split between the two expert fits, so the +experts were not content-independent. After allocating whole groups to one +side, selecting one representative per group, and recalibrating both experts +from the same untouched development controls produced templates correlated at +0.720, and the consensus retained only 6 of 443 Google files. It added 2 files +missed by the shipped carrier and retained only 1 of 95 July misses. It still +accepted no conjunction among 81 reserve exact-geometry +controls, 92 exact-1024 Open Images, 637 exact-1024 Picsum images, or the 100 +new Open Images controls. The corrected result is specific but far too small +to ship. + +Removing the exact-geometry restriction invalidated even that corrected +consensus. Across all 11,506 heterogeneous native controls, the group-separated +experts accepted 5 and 6 respectively, and their conjunction accepted 2. Both +consensus errors were at non-1024 geometries. The result is explicitly an +exact-1024 research candidate and does not supply a universal-size detector. + +A post-hoc margin above both arbitrary-geometry collisions retained the two +corrected incremental exact-1024 positives. Applying that frozen margin to all +4,698 Google files accepted 28 and added 16 files missed by the shipped carrier +across 11 native geometries. A full-corpus perceptual audit assigned those 16 +files to 16 separate content groups, none containing a shipped-carrier-positive +variant. The same margin accepted none of 1,500 Open Images, 637 unique Picsum, +or 12 Wikimedia native views. This is the first concrete all-size hypothesis +for the second direction, but the margin was selected after all 11,506 +heterogeneous controls were exposed and the natural corpora had already +challenged related experts. It requires a new content-deduplicated AI-control +corpus before it can be treated as validation or runtime logic. + +The next source-independent model-cohort challenge sharply bounded that +hypothesis. With no exact-byte overlap against the Spaces inventory, the +original group-separated conjunction accepted 2 of 589 public Gemini 3.1 Flash +Image Preview images, none of 520 Nano Banana Pro Preview, and none of 280 +DALL-E 3 images. The two Gemini hits were visually distinct diverse images at +1408 by 768. Each Google cohort also included solid-color and gradient probes +over several native geometries. The post-hoc 1.033 strict margin rejected every +cohort image: Gemini reached 1.011, Nano Banana 0.941, and DALL-E 3 0.514. The +base rule therefore has weak transfer to a current Gemini cohort but still +collides with 2 of 11,506 arbitrary-size controls; the strict rule removes both +the controls and the current-Gemini transfer only through a post-test threshold. +Neither is a validated universal Google-model or cross-provider SynthID +detector. The 16 Spaces all-size hits and two public Gemini base hits are useful +hard positives for epoch analysis, not justification for implementation. + +The near-duplicate audit also provided a small mechanistic diagnostic. In all +three perceptually matched pairs where only one variant crossed the shipped +carrier, subtracting the shipped variant from the non-shipped variant reduced +the shipped-template direction and increased both independently refitted +second-template directions. The normalized mean paired difference correlated +-0.568 with the shipped template and 0.252 and 0.264 with the two second +templates. Three pairs are insufficient for inference; the two-sided sign-test +result is 0.25. The paired result supports a carrier-state interpretation but +is not detector validation. + +All three temporal detections from the original first-expert conjunction survived JPEG round trips +at qualities 95, 85, and 70. None survived WebP 95 or a 0.75x down-and-up resize +round trip. A symmetric transform challenge prevented promoting that positive +retention to a JPEG claim: one of 162 heterogeneous exact-1024 controls crossed +the unchanged rule after JPEG 85. The 92 exact-1024 Open Images and 637 unique +exact-1024 Picsum controls stayed below threshold in every view. The candidate +was native-only; fixed-lattice JPEG retention is diagnostic, not a certified +transport operating point. The transform result applies only to that +first-expert rule. The corrected group-separated experts have not been +transport-calibrated, so it supplies no robustness claim for them. + +The corrected carrier was then tested causally. The normalized sum of the two +content-group-separated expert directions correlated 0.927 with each expert. +Subtracting it from all 16 strict incremental hits cleared every linear and +phase component on every image; the same amplitudes with a one-pixel cyclic +shift cleared none. Selected amplitudes were 5.0 through 7.75 integer levels, +with median 6.5. Fidelity ranged from 58.92 through 69.97 dB PSNR and 0.99961 +through 0.99996 SSIM. The carrier itself was transform-fragile: only one of 16 +sources remained strict after JPEG 95 or 90, and the aligned edit cleared that +one in both views while the shifted edit did not. No source remained strict +after JPEG 85, WebP 95, or a 0.75x resize round trip. + +A joint ablation then started from all 28 strict second-carrier hits. It first +suppressed the shipped carrier to the frozen -0.25 target where required, then +suppressed the second direction until all four expert components were below +their base thresholds. The original cohort contained 12 first-carrier and 28 +strict second-carrier detections; the aligned candidates contained zero of +either. The shifted controls left one first-carrier and seven strict +second-carrier detections. Median fidelity was 61.69 dB PSNR and 0.99984 SSIM; +the minima were 58.15 dB and 0.99910. The two edits did not reactivate one +another. Together with the failed third-carrier fit, the current linear native +16x16 hypothesis is locally exhausted as two jointly controllable states. +These are detector-score interventions, not Google-oracle removals. + +### 2026-08-12: Registered color and phase-lock challenge + +A crop-specific research branch tested whether the recovered 16x16 carrier is +better represented in a perceptual color space or a shift-tolerant directional +transform. Every branch reused the direct RGB period-and-phase registration, +fitted on the first 30 positives, selected on the next 20 positives and 200 +controls, and reported the remaining 50 positives and 299 controls separately. +Only records whose recovered period was 15.5 through 16.5 were eligible. + +The frozen RGB template's channel norms were `0.887:1.000:0.930` for R:G:B, +including `0.903:1.000:0.916` above FFT radius 4.5. An SVD assigned 79.80% of +template energy to a nearly equal-channel component, 18.39% to a +green-vs-magenta component, and 1.81% to a red-vs-blue component. The common +component had strong diagonal energy, while 98.6% of the coarse +green-opponent orientation energy was axis-aligned. This measured carrier does +not support a fixed `0.85:1.00:0.70` channel rule or an exclusively diagonal +decoder. + +Nonlinear cube-root LMS coordinates from the OKLab transform exposed the +signal most strongly in the direct color-fold comparison, but the +development-selected full-vector candidate retained 32 of 50 final positives +and accepted six of 299 controls. This is evidence that nonlinear color +projections can improve carrier SNR, not evidence that the deployed embedder +uses Lab or OKLab internally. + +A three-level DTCWT probe retained all six orientations rather than only the +`+/-45` degree pair. Its development-selected near-vertical green energy +candidate retained 37 of 50 final positives and accepted eight controls. The +green-opponent near-vertical candidate retained 39 positives and accepted one +control. Separating energy, normalized complex correlation, and phase-only +coherence changed the development choice to all-orientation green-opponent +phase coherence, which retained 37 positives and accepted four controls. The +directional representation therefore exposes real carrier structure but fails +the low-false-positive gate. + +An explicit FFT probe then froze 8, 16, 32, or 64 carrier bins from the fit +positives and compared signed cosine phase coherence with the proposed +`abs(cos(delta_phase)) > 0.90` rule. The absolute rule never produced a usable +operating point; treating phase and antiphase as equivalent increased the null +overlap. With 64 bins, the development-selected green-channel signed cosine +retained 26 of 50 final positives and initially accepted none of 299 controls. +That result was underpowered because only 39 of 499 controls selected the +eligible period-16 branch. + +The green phase candidate and its threshold were frozen before a full +period-and-phase search over 1,000 COCO controls. It accepted five controls. +Among the 129 eligible controls, the maximum score was `0.6910271977636098`, +above the frozen `0.5443545641095117` threshold. Raising the threshold above +that maximum retained only eight of 100 positives. Explicit registered phase +lock is therefore a useful diagnostic of this carrier family but is rejected +as a standalone detector. Because it shares the RGB registration stage, it is +also not independent confirmation of the carrier. + +### 2026-08-12: Green phase perturbation and blind SWT removal + +A second removal probe tested the specific claim that changing only phase at +the recovered Green-channel carrier bins can preserve spectral magnitude and +image fidelity while defeating detection. It used the 100-image public Gemini +set at native period 16 and modified conjugate FFT pairs together so the +inverse remained real. Before pixel quantization, the maximum absolute change +in FFT magnitude was `5.24e-10` for the 32-pair inversion. After clipping and +rounding to 8-bit pixels, the maximum magnitude change relative to the largest +source coefficient was 0.175%. + +The native periodic detector accepted 59 of the 100 sources. Inverting the 32 +strongest unique Green-channel carrier pairs left one of those 59 positive. +Across all 100 images, PSNR was at least 43.36 dB and SSIM at least 0.99048; +the medians were 52.79 dB and 0.99739. Median signed phase coherence moved from +0.485 to -0.371 and the median full RGB tile score moved from 0.230 to 0.041. +This reproduces the proposed fidelity and local-score effect against the +recovered periodic detector. + +It does not establish SynthID removal. The current test targets the same +carrier used by the local score, and earlier wider complex-spectrum and +phase-projection candidates already cleared local phase models at 51.94-56.03 +dB while remaining detected in healthy Google verifier sessions. The 32-pair +Green candidate is therefore a preregistered oracle candidate, not a remover. +The official SynthID-Image description also identifies a post-hoc neural, +content-dependent encoder-decoder rather than a fixed Fourier codebook; it +does not document phase bins as its decision surface. + +The preregistered Google-oracle challenge then used three of the strongest +local positives. Each fixed group contained the original, its 32-pair Green +phase inversion, and an RMS-matched random-noise sham. All three phase +candidates fell below the local `0.17357069773071196` threshold, with PSNR +from 55.41 to 57.86 dB and SSIM from 0.99709 to 0.99807. The three shams +remained locally positive. Nine files were submitted to Gemini's `Verify with +SynthID` tool in separate chats with the unchanged prompt. Eight verdicts +persisted and all eight reported Google AI signals. This includes all three +phase candidates, both recoverable source controls, and all three shams. The +second source request was submitted, but its chat did not persist and no +verdict could be recovered; it was not retried outside the fixed upload budget. + +This falsifies the operative removal claim: Green-carrier phase inversion +clears the local periodic correlate on three of three images but clears the +Google oracle on zero of three. It is evidence that the local template observes +one stable component or correlate of SynthID, not that these FFT coefficients +form the proprietary decoder's sufficient decision surface. Do not productize +the Green phase inversion or report its local score change as SynthID removal. + +The three oracle-confirmed phase candidates were then treated as adversarial +hard positives for detector analysis, without using them to relax the existing +runtime thresholds. Both runtime modes missed all three. Their native scores +were 0.126-0.160 against the 0.174 threshold; their registered decision scores +were 0, 0, and 0.553 against the threshold of 1. A phase-invariant magnitude +profile was therefore tested as a possible rescue. The development-selected +all-bin RGB profile retained 39 of 50 final positives but accepted one of 299 +final Open Images controls and still missed all three hard positives. + +A frozen-template, top-32 Blue-magnitude profile did accept all three hard +positives and initially accepted none of 499 Open Images controls, but this was +a post-hoc hard-positive-selected branch with low ordinary sensitivity: 17 of +70 development and final positives. Its frozen threshold accepted 39 of 1,000 +fresh COCO controls. Combining it with a native-score floor of 0.10 happened to +accept none of all 3,000 COCO controls while retaining the three hard positives, +but failed the next independent challenge: 90 of 11,506 metadata-labeled +non-Google controls passed both gates. Accepted controls included Adobe +Firefly, local diffusion, Samsung, Microsoft, and several other generator +families. The phase-invariant magnitude rescue is therefore rejected as a +provider-specific detector; it measures a shared periodic rendering family. + +Channel separation exposed a narrower attack-specific rescue. The Green-only +phase inversion leaves Red and Blue pixels unchanged, but the runtime RGB score +normalizes all three channels jointly. On the three oracle-confirmed hard +positives, separate signed correlations remained 0.781-0.804 in Red and +0.798-0.825 in Blue while Green moved to -0.400 through -0.645. A Red threshold +of `0.3770237083693908`, frozen above 200 Open Images calibration controls, +retained 16 of 70 development and final ordinary positives and accepted none +of the remaining 299 Open Images controls. It also accepted all three oracle +hard positives and none of 3,000 fresh COCO controls. + +Red correlation alone remained a generic periodic-renderer correlate: it +accepted 96 of 11,506 metadata-labeled non-Google controls. Every one of those +96 had positive Green correlation. The attack-specific conjunction of Red at +or above the frozen threshold and negative Green accepted none of 15,005 total +controls across three independent corpora, while retaining all three +oracle-confirmed phase candidates. Across the 100-image synthetic inversion +probe, it recovered 19 of the 58 locally detected sources that the attack had +moved below the RGB threshold. One additional attacked image still passed the +existing detector. + +This channel-discordance rule is the first zero-control rescue for the +oracle-confirmed hard positives, but it is not added to runtime. Its form was +motivated after inspecting the attack, only three examples carry external +oracle labels, and its synthetic rescue sensitivity is 19 of 58. + +A follow-up challenge registered that frozen rule over all 256 modulo-16 phase +offsets and six fixed carrier periods: 8, 10.4, 12, 16, 20, and 24 pixels. The +Red-only form accepted one of 3,000 COCO controls during phase registration; +period search added no further acceptance. The accepted control scored 0.395 +in Red but only 0.267 in Blue. This rejects Red-only registered rescue at the +existing threshold. + +The previously frozen `min(Red, Blue) >= 0.3770237083693908` candidate, still +combined with negative Green, accepted none of the same 3,000 controls under +phase-only or phase-and-period registration. Its maximum registered control +score was 0.342. A symmetric transform stress test then evaluated all twelve +identity, crop, JPEG, and resize views for 100 fixed Open Images controls. None +of the 1,200 inputs was accepted; the maximum was 0.332. + +The stricter rule retained all three oracle-confirmed hard positives at native +scale and after offsets of 1x1 and 7x11 pixels or a centered 5% crop. Period +registration retained all three after 0.65x, 0.75x, 1.25x, and 1.5x resizing. +It retained none after 0.5x resizing or JPEG round trips at qualities 95, 85, +and 70. Every source and RMS-noise sham remained negative in all twelve views. +The small-control zero was not sufficient for the complete period bank. A +later five-period search over all 11,506 metadata-labeled non-Google controls +accepted 16 at the original 0.377 threshold: two at period 20 and 14 at period +24. The maximum `min(Red, Blue)` score was 0.435813. This rejects one common +threshold across periods 10.4 through 24. + +A conservative high-score branch was then frozen strictly above that measured +maximum. It retained all three hard positives at periods 12, 16, 20, and 24, +corresponding to 0.75x through 1.5x resizing, identity, and the three crop +views. A symmetric full-corpus resize challenge accepted none of 11,506 +controls at 0.75x, 1.25x, or 1.5x; the respective maxima were 0.431352, +0.393949, and 0.372036. The 0.65x path needed separate treatment. Its native +period-10.4 null maximum was 0.361367, below all three hard positives, but a +symmetric 0.65x resize round trip raised one control to 0.397396, above the +weakest hard positive at 0.394911. + +That collision motivated an exploratory low-period channel-ratio gate: +`min(Red, Blue) >= 0.3770237083693908`, negative Green, and +`Blue >= 1.1 * Red`, only at period 10.4. It retained all three 0.65x hard +positives. Among all 11,506 controls, the largest score satisfying the ratio +gate was 0.355165 natively and 0.354955 after the symmetric 0.65x round trip, +so neither challenge accepted a control. The combined exploratory rule also +retained all three hard positives in the identity, three crop, 0.65x, 0.75x, +1.25x, and 1.5x views, while every corresponding source and RMS-noise sham +remained negative. + +Applying the combined rule without further tuning to the 100-image synthetic +Green-inversion cohort exposed its low sensitivity. Among the 58 attacks that +moved a natively detected source below the native detector threshold, the rule +recovered 4 at 0.65x, 5 at 0.75x, 16 at identity, 11 at 1.25x, and 10 at 1.5x. +The three oracle source images are members of that cohort; only one additional +synthetic source passed the 0.65x branch. The rule is therefore a +high-specificity rescue for a strong residual R/B carrier, not a general +solution to Green-phase perturbation. + +A source-split Fisher projection then tested whether a learned three-channel +direction could improve that sensitivity. Red, Green, and Blue correlations +were standardized separately by scale, and the projection was fitted from 34 +training attacks and 5,719 training controls. Per-scale thresholds were frozen +above every training-control score. They accepted 11 of 23,148 views from the +5,787 test controls. On the seven source-held-out attacks, the projection +recovered 0 at 0.65x, 0 at 0.75x, 2 at 1.25x, and 2 at 1.5x. It is both less +specific and less sensitive than the explicit rule and is rejected. + +Spatial consistency at 0.65x was also insufficient. At the globally selected +period-10.4 phase, a 2x2 and 4x4 aligned grid measured local R/B strength, +negative-G coverage, and `Blue / Red` coverage on all 537 controls in the +global-score tail at or above 0.20 and on all 58 synthetic native-detector +misses. Every individual feature overlapped the control tail. Within a small +fixed grid of global floors and integer coverage counts, the best zero-control +conjunction recovered four attacks, no more than the simpler global ratio +rule. Lowering the global floor to 0.30 recovered five but accepted two +controls. The spatial branch is rejected as non-improving. + +This establishes bounded transform behavior for the attack diagnostic, not a +universal detector. The 0.65x ratio was selected after seeing the full-corpus +collision, the conservative threshold was derived from an exposed control +corpus, the period bank is discrete, only three recovered images have oracle +labels, and JPEG sensitivity is zero in this probe. The next valid gate remains +new oracle-confirmed Green-perturbed positives plus a preregistered untouched +control corpus. The branch stays outside runtime until that gate passes. + +An exploratory channel-contrast extension then tested whether the magnitude of +the negative Green response could recover the failed JPEG and 0.5x views. The +metrics were fixed as `min(Red, Blue) - Green` and +`mean(Red, Blue) - Green`; 50 Open Images controls across all twelve transforms +set thresholds before the remaining 50 controls were scored. Calibration over +600 views required thresholds of 0.655 and 0.695. Neither metric accepted an +evaluation control, but both still recovered zero of three JPEG cases at every +quality and only one of three 0.5x cases. A relaxed R/B-only threshold produced +the same result. Negative-G magnitude therefore adds no useful transform +coverage at the measured low-false-positive boundary and is rejected. + +Codec-specific templates were then fitted from 50 other public positives after +JPEG 95, 85, and 70 round trips, excluding all three oracle hard-positive +sources. Neither a quality-matched template nor a three-quality bank recovered +any of the nine JPEG hard-positive views. They accepted respectively three and +two of 150 evaluation-control views. The templates primarily captured the JPEG +8x8 grid and were rejected. + +Filtering the unchanged 16x16 template to radial frequency 0.5 through 4.5 +exposed a real JPEG-surviving component. At a threshold above the first 300 +Open Images JPEG views it retained all three hard positives at qualities 95, +85, and 70. The next frozen challenge rejected it: phase search accepted 35 +native, 50 JPEG-95, 48 JPEG-85, and 46 JPEG-70 views among 3,000 COCO images. +Cross-band phase agreement did not separate the nine hard-positive views from +those 179 false positives. + +A spatial gate required negative Green in all four quadrants and a median +quadrant R/B score above the COCO false-positive maximum. It retained all three +hard positives at JPEG 95 and 85 but none at JPEG 70. On 11,506 +metadata-labeled non-Google controls it nevertheless accepted 35 native, 66 +JPEG-95, and 88 JPEG-85 views. The errors included genuine camera images and +multiple unrelated generators. Scoring only the unchanged phase also failed, +accepting 38 native and 210 JPEG views among 3,000 COCO images; its maximum +control score of 0.673 exceeded the 0.464-0.609 hard-positive range. The +JPEG-stable band is a general low-frequency or codec lattice, not a +SynthID-specific rescue, and all of these branches are rejected. + +The 0.5x boundary was challenged separately with a physically reduced 8x8 +template. Projecting the dominant 2x2 lattice out of both observation and +template before scoring R/B with negative Green removed the known nuisance but +not the overlap. Across all 11,506 symmetrically downscaled controls, the +zero-control threshold was 0.396. The three oracle hard positives scored only +0.277-0.288. Channel separation therefore cannot rescue the rejected period-8 +branch. + +An untouched 2026-08-07 through 2026-08-10 temporal cohort then challenged the +runtime template and thresholds. Cohorts were recomputed from explicit +provenance because a provider-only split incorrectly placed mixed Google/OpenAI +lineage in the negative set. The frozen native detector accepted 183 of 252 +files with explicit Google LLC SynthID provenance and 3 of 488 with explicit +OpenAI-only SynthID provenance. Scale registration accepted 130 and 1, +respectively. Among 170 files with no SynthID provenance assertion, native and +registered modes accepted eight and two. These operational errors are not +provider-oracle negatives and cannot establish the true false-positive rate. + +The native detector remained positive on 153 of 252 explicit-Google images +after a symmetric 0.75x resize round trip, 66 after JPEG-95, and 54 after +WebP-95. A 5% center-crop round trip retained none. This fresh temporal result +confirms a real, partly codec-tolerant Google pixel signal across many stored +sizes, while rejecting the stronger claim that the fixed-phase runtime score +is a universal SynthID decoder. The post-hoc strict second carrier did not +generalize: it accepted only 2 of 234 Google-platform positives and none of 689 +provider-split controls. Applying the already frozen scale-registration path +after the crop recovered only 8 of 252 explicit-Google files and accepted none +of 488 explicit OpenAI-only files. The existing scale search does not solve the +crop phase boundary. + +A diagnostic joint-scale-and-phase probe then reversed the known 0.95 crop +scale and scored the carrier at the cyclic phase predicted from the removed +border. It reached 0.859 AUC for explicit Google provenance against all other +temporal cohorts. A post-hoc threshold above all 671 other rows retained 37 of +252 explicit-Google rows. Exhaustively searching all 256 tile phases was weaker +at 0.839 AUC and retained 41 rows above its own zero-control threshold. The +fixed geometric phase supplies a plausible crop mechanism and a candidate for +the next preregistered temporal challenge, not a calibrated runtime mode: both +the transform and reported operating point have now observed this cohort. + +A blind three-level `db2` SWT probe then soft-thresholded only the Green +horizontal, vertical, or diagonal detail bands at 0.25, 0.5, 1, and 2 times the +band's robust noise estimate. Diagonal HH thresholding left all 59 original +positives detected at every strength. Vertical thresholding cleared four at +strength 1 and five at strength 2, but the respective minimum fidelity fell to +32.57 dB and 0.96971 SSIM, then 28.52 dB and 0.93646. At strength 1, vertical +score reduction exceeded diagonal reduction on 97 of 100 paired images +(`p = 2.63e-25`, exact two-sided sign test). Blind HH soft-thresholding is +therefore rejected; the stronger vertical response is consistent with the +separately measured axis-aligned green-opponent carrier component. + +### 2026-08-13: patent architecture and amplitude confound audit + +The official paper intentionally leaves the neural network architecture +unspecified, but a related DeepMind patent family with overlapping authors gives +more specific architectural evidence. Its image example constructs a +content-dependent residual as `x' = x + g(x)`, describes a U-Net-like encoder +and convolutional decoder, injects a message or secret into intermediate layers, +and trains clean and watermarked pairs under the same sampled transformations. +It also permits a bank of paired networks in which one decoder is deliberately +unable to recognize another pair's mark. These patent alternatives are not +proof of the exact production implementation, but they invalidate the working +assumption that SynthID must reduce to one provider-independent, fixed +spread-spectrum key. Multiple Google states and a distinct OpenAI carrier are +compatible with one technology family. + +This changes the role of the shipped 16-by-16 template. It is a validated linear +expert for one observable encoder state, not a surrogate for the official +nonlinear detection logit. A universal local detector should be designed as a +versioned union of independently calibrated experts, with family-wise +false-positive control and an explicit abstention region. New experts still +need source-matched clean counterfactuals or provider-oracle labels; metadata +absence is not a negative label. + +A symmetric transform-stability experiment challenged a simpler rescue. The +weak oracle-positive `07.png` scored 0.1248 natively, 0.0288 after JPEG 95, +-0.0023 after JPEG 85, 0.1179 after a 0.99x Lanczos round trip, and 0.0791 after +an eight-pixel crop round trip. The oracle-negative Open Images chestnut scored +0.2124, 0.1322, 0.0482, 0.2274, and 0.0601 under the same views. Both signals +degrade similarly under codec and crop operations. Multi-view stability does +not distinguish the weak positive from this natural lattice confound and is +rejected as a rescue gate. + +Raw folded-tile amplitude was also tested because `07.png` had a norm of 6.924, +compared with 3.017 for the chestnut and 0.874 for the strongest tested resized +Kodak confound. The full corpus rejects a general amplitude gate. Among 4,698 +Google rows, the current score threshold accepts 3,928, but accepted norms run +as low as 0.712. A post-hoc branch at `score >= 0.121` and `norm >= 5` would add +190 rows below the current threshold, yet an observed-geometry development +control already reaches score 0.120911 at norm 15.339. The margin to `07.png` +is under 0.004 and there is no untouched validation set. Amplitude is useful as +a support diagnostic, but neither a veto nor this low-score rescue is eligible +for runtime. + +The next defensible Google experiment is consequently a preregistered +multi-expert cascade: preserve the current carrier unchanged; train candidate +experts only on source-grouped paired or oracle-labeled data; calibrate every +expert and their union on untouched natural, resize, codec, and provider +controls; and return `unknown` when neither the watermarked nor unwatermarked +hypothesis clears its calibrated boundary. + +The newly published MarkNull work supplies an independent no-box removal +baseline. It reports 100% attack success on 20 Imagen-3 images checked through +Gemini's verification flow. The attack uses a clean Stable Diffusion proxy to +invert each image, optimize a bounded latent against a noise-latent alignment +loss plus LPIPS, MSE, and SSIM, and decode it again. Its milder SynthID setting +still reports only 25.36 dB PSNR and 0.80 SSIM. It therefore strengthens the +case for latent regeneration as a detector-independent fallback, but it does +not meet this project's pixel-only or fidelity gates. + +The associated [public repository](https://github.com/JieJayCao/MarkNull) +contributed 50 metadata-free 512-by-512 Gemini/Imagen source candidates, but no +attacked SynthID pairs in the inspected Git tree. The paper reports only 20 +oracle-evaluated sources and does not map those verdicts to filenames, so all 50 +remain source-asserted candidates rather than individual oracle labels. Both +shipped experts detected zero of 50. More precisely, the fixed runtime expert +reported unsupported because 512 by 512 is below its calibrated pixel-count +floor. Its raw diagnostic score ranged from -0.1044 to 0.1058. Scale +registration supported all 50 and accepted none, with its highest decision +statistic at 0.9574 below the frozen 1.0 threshold. The shared scorer in +`scripts/synthid_runtime_expert_scores.py` now records this distinction and +exports both observations from one pixel decode. + +An external-source split nevertheless exposed a highly reproducible period-4 +component. Templates from the first and second 25 images correlated at 0.958. +A template fitted only on the first half accepted 22 of 25 held-out candidates +at a threshold above 250 canonicalized COCO development controls and accepted +zero of a separate 250 COCO controls. The broader challenge rejected watermark +specificity: it accepted two of 300 heterogeneous AI controls and five of 282 +content-deduplicated internal negative-test controls at the same frozen +threshold. + +The mechanism diagnostics also identify the confound. The period-4 mean placed +82.4% of its non-DC spectral power in the horizontal Nyquist bin, and stable +correlations at larger even periods vanished after projecting out all period-4 +harmonics. Its strongest channel was Blue rather than Green, and its best +phase-aware correlation with a fourfold-reduced shipped carrier was only 0.375. +This is an axial renderer or resampling signature, not evidence for another +validated diagonal SynthID carrier. It is retained as a hard-negative feature +and excluded from the runtime expert bank. + +A schema-versioned conformal cascade now implements the required aggregation in +`scripts/synthid_conformal_cascade.py`. Every configured expert must emit an +explicit supported score or unsupported result. For presence, the smallest +empirical clean-null upper-tail p-value receives a Bonferroni correction over +the supported expert bank. For absence, the largest lower-tail p-value against +each expert's positive calibration is used, so every possible encoder state +must be rejected. `not_detected` is disabled unless the calibration manifest +explicitly declares complete version coverage and every expert supports the +query. Input and output manifests are hash-pinned for reproducibility. + +The first source-disjoint smoke test used only the unchanged fixed 16-by-16 +expert. Alternating content hashes supplied 2,347 source-asserted Google +calibration rows and 2,346 evaluation rows. The existing observed-geometry COCO +partitions supplied 3,000 development controls and 2,000 final controls. At a +two-sided alpha of 0.001 with incomplete-version coverage, the cascade detected +2,084 of 2,346 Google rows and one of 2,000 final natural controls; all other +rows abstained and none returned `not_detected`. The corresponding frozen +runtime threshold detected 1,954 Google rows and zero final controls. The +conformal point therefore recovers 130 additional source-asserted positives at +the cost of one natural false positive. It is statistically compatible with +the target 0.1% FPR, but it is not a runtime replacement: the corpus has already +influenced detector development, Google labels are provenance assertions rather +than per-file oracle verdicts, and a multi-expert union still needs a fresh +family-wise challenge. + +The fixed and scale-registered experts were then evaluated together on their +frozen source-disjoint resize challenge. Current production scores were +recomputed from pixels for all 355 transformed Google candidates and 499 +untouched Open Images controls. Sorting each label by source identifier and +alternating rows produced 178 positive and 250 negative registered-calibration +examples, followed by 177 positive and 249 negative evaluation examples. The +fixed calibration used 4,516 Google sources after excluding every evaluation +source and 2,980 observed-geometry development controls. At the unchanged +runtime thresholds, fixed accepted 8 evaluation positives, registered accepted +116, and their union accepted 117 of 177 with zero of 249 controls. On the full +descriptive corpus the corresponding counts were 15, 229, and 231 of 355, again +with zero of 499 controls. Registered scale search therefore supplies 109 +source-disjoint evaluation positives that the fixed expert misses; the fixed +expert adds one that registered misses. + +Applying the empirical cascade at alpha 0.01 was less specific. It accepted 122 +of 177 evaluation positives but also five of 249 controls. All five control hits +came from the registered branch below its frozen decision threshold. The +250-control calibration provides only 0.00398 p-value granularity, and lowering +the operating point merely traded five additional positives for five false +positives. This conformal operating point is rejected. The implementation is +retained as versioned calibration infrastructure, while the independently +frozen production thresholds remain the valid two-expert decisions. + +A wider calibration reused the already saved component reports for 11,273 +supported, symmetrically resized Spaces controls. The stored high-band value +made the current registered decision score exactly reconstructible throughout +the relevant upper tail; rows screened below the historical 0.1182 amplitude +ratio cannot affect this tail. At alpha 0.001, the two-expert cascade accepted +119 of 177 evaluation positives and zero of 249 controls. The two additions +over the runtime union had registered scores 0.9830 and 0.9731, with corrected +clean-null p-values of 0.000710. This initially appeared to be a non-failing +multi-expert calibration point, but the positive challenge had already been +exposed. The alpha, expert bank, and thresholds were frozen before the fresh +test below. + +The decisive reserve used the next 1,500 sorted Open Images validation IDs, +3,500 through 4,999, frozen before scoring. Download and geometry checks yielded +1,000 registered-supported controls with no decoded-pixel hash overlap against +the prior 1,499 Open Images controls. Registered scale search accepted zero. +The fixed runtime threshold accepted 7 of the 81 controls in its supported +geometry, or 7 of 1,000 overall, so a simple OR of the two experts also accepted +seven. The frozen alpha-0.001 cascade was worse: it accepted nine controls, +eight through the fixed tail and one registered-only row at score 0.8690. +The candidate is rejected. In overlapping geometry, fixed-only evidence cannot +be added to registered evidence without a new specificity mechanism; keeping +the fixed detector available as a separately scoped diagnostic does not make +their union safe. + +The fixed-only geometry does not provide a fallback. Among 89 saved non-Google +Spaces controls from 10 to 18 megapixels, where registered mode is unsupported +but fixed mode is supported, 10 crossed the fixed threshold. The errors span +multiple generator families. A geometry router therefore cannot extend recall +by selecting fixed whenever registered is unavailable. + +`scripts/synthid_routed_expert_bank.py` implements the surviving conservative +policy without changing the standalone detectors. When registered mode is +supported, only its frozen threshold can produce `detected`; a fixed-only +crossing is retained in the report as `fixed_only_ambiguous`. When registered +mode is unsupported, fixed evidence is likewise reported but the bank abstains. +The bank never returns absence because encoder-version coverage is incomplete. +On all 355 transformed positives and 499 reserve controls it detected 229 and +zero. On the fresh 1,000-image reserve it detected zero and explicitly routed +the seven fixed crossings to abstention. This is a safe expert-bank baseline, +not a universal SynthID decoder: scale 0.5, unmeasured encoder states, and the +separate OpenAI family still require additional validated experts. + +Directly enlarging registered mode's pixel ceiling also failed. The unchanged +registered components were computed on every saved 10-to-18-megapixel example: +37 Google candidates and 89 non-Google Spaces controls. The frozen threshold +accepted one positive and zero controls. Amplitude passed for 28 positives and +the selected spatial and spectral periods agreed for 21, but only three cleared +the high-band gate. Large full-frame spectra therefore lose almost all recall; +the absence of control errors does not justify widening the declared geometry. +Phase-aligned window aggregation inside the calibrated range is the next +bounded large-image hypothesis. That hypothesis was then challenged with every +phase-aligned 2,048-square window. Median-window scoring retained 36 of 37 +positives but also 10 of 89 controls; requiring every window to pass retained +36 positives and eight controls. The false lattices are spatially coherent, so +window consensus cannot validate a large-image fixed fallback. Large images +outside registered scope remain abstentions. + +A final frozen full-frame alternative isotropically reduced each large image to +eight megapixels with area interpolation before applying the unchanged +registered detector. This maps a native period 16 into approximately 10.7-14.3 +pixels, inside the calibrated period range, without discarding spatial regions. +It still accepted only 1 of 37 positives and zero of 89 controls, and the one +hit was the same 4096-square image detected natively. Amplitude passed for 26 +positives, period agreement for 16, and high-band agreement for only three. +Pre-resizing therefore does not recover the missing evidence and adds harmonic +ambiguity; no target-size sweep is justified on this exposed cohort. + +The scale-0.5 failure was also rechecked without the registered detector's +sequential amplitude gate. High-band correlation was computed for all 65 +period-8 positives and all 499 reserve controls. Positive and control medians +were 0.0184 and 0.0213, with ranges -0.2381 to 0.1930 and -0.1734 to 0.2148. +Normalized amplitude also overlapped: positive median and maximum were 0.4611 +and 0.9259, compared with 0.3824 and 1.6296 for controls. Requiring amplitude +and high-band evidence jointly therefore cannot recover this scale at the +frozen specificity. The 0.5x case remains an explicit blind spot rather than an +eligible lower-threshold branch. + +### 2026-08-10: OpenAI periodic-carrier challenge + +The OpenAI track repeated the Google carrier method without runtime provenance +features. C2PA selected the research labels, but every candidate score consumed +only decoded RGB pixels. The temporally split corpus contained 4,615 OpenAI +training positives, 1,280 validation positives, 1,364 development-test +positives, 8,668 difficult external-source controls, and 3,000 COCO controls. +The development test is not called a locked holdout because earlier +representations had already exposed it. + +High-pass residuals were folded modulo 8x8, 16x16, and 32x32 and normalized as +in the Google fixed-tile expert. A single 16x16 OpenAI template accepted only +one of 1,280 validation positives at the threshold above all 2,287 validation +controls. A 16-template geometry bank improved that to 12 positives but +accepted one of 3,372 development-test controls. The corresponding +development-test sensitivity was 9 of 1,364. The frozen Google template did +not transfer: at its unchanged threshold it accepted 14 validation positives +and 572 validation controls. + +The apparent native-size signal was a confound. At 1024x1536, 91.6% of the +learned template energy projected onto a 2x2 periodic lattice, compared with +5.6% for the frozen Google template. Firefly, China AIGC-labeled, local +diffusion, and unidentified hosted-generator controls reached correlations of +0.92-0.98 against the uncorrected OpenAI template. This is operational evidence +for a shared decoder or upscaler lattice, not an OpenAI watermark carrier. + +Projecting the 2x2 nuisance lattice out of every 16x16 fold isolated a weaker +epoch-specific residual. The 1024x1536 expert accepted 5 of 55 validation and +10 of 51 development-test positives while accepting none of the full +validation or development-test controls. Nine of the ten development-test +matches occurred from July 12 through July 14; later dates contributed only +one of 27. The 1536x1024 expert accepted 6 of 37 validation and 2 of 51 +development-test positives with no development-test controls. Amplitude, +active-support, channel-consensus, payload-invariant power, and spherical +multi-template branches did not improve this frontier without false positives. + +The OpenAI branch therefore remains research-only and is not added to the +runtime detector. The next valid training epoch requires same-content +OpenAI-oracle `detected` and `not_detected` pairs, or provider-authorized decoder +labels, followed by a fresh temporal holdout. Source provenance alone cannot +distinguish a true watermark carrier from the measured generator lattice. + +### 2026-08-12: OpenAI opponent-color carrier challenge + +The RGB periodic and learned branches did not test the hypothesis that a weak +carrier is clearer after the nonlinear LMS transform used by OKLab. A new +validation-only screen compared sRGB, linear RGB, cube-root LMS, OKLab, and an +orthonormal linear opponent basis. For each representation it tested periods +4, 8, and 16 plus whole-vector, single-channel, and chroma-vector folds. Every +candidate fitted a positive-minus-control direction independently in the native +and JPEG-95 views. A positive had to cross both thresholds, with each threshold +strictly above every validation control in the corresponding view. + +The development corpus was limited to the three OpenAI native geometries: +1024x1024, 1024x1536, and 1536x1024. It contained 283/140/158 OpenAI +train/validation/test images and 252/145/135 heterogeneous controls. The best +validation candidate was the third cube-root LMS component at period 8. It +accepted 19 of 140 validation positives. The frozen locked test accepted 16 of +158 positives and none of 135 controls in the conjunctive native-plus-JPEG +rule. Native alone accepted 43 positives and one control; JPEG-95 alone accepted +22 positives and a different control. The native and JPEG templates correlated +at 0.890. + +The 16 test hits were not a single date, geometry, or content family. They +spanned 11 collection dates; the 1024x1024, 1024x1536, and 1536x1024 cohorts +contributed 2, 8, and 6 hits. A 256-bit DCT-hash audit found two multi-member +perceptual groups in the complete 581-image OpenAI native cohort, but neither +connected an accepted test image to another split. Thresholding against the +union of heterogeneous validation controls and 140 matched 0.75x-down-and-up +OpenAI views did not change the operating point. The 158 matched test views +accepted none while their 16 corresponding source views remained accepted. + +The transform boundary is mixed. All 16 source hits remained conjunctively +accepted after WebP-95; the complete test set then contained 26 accepted OpenAI +images and one accepted control. JPEG-90 retained four of the 16. Crop and 0.75x +resize round trips retained none. Searching all 64 cyclic period-8 phases did +not restore geometric robustness: resize retained 2 of 158 positives and one +of 135 controls, while crop retained no positive and one control. The loss is +not merely an unknown modulo origin. + +The candidate was also scored without refitting on all 52 action-derived +same-provider controls at these geometries. It accepted one of 52 and none of +the 12 controls explicitly naming `gpt-image 2.0`. The exact-version subset is +only 8/2/2 train/validation/test by the native geometry restriction, and the +absence of `c2pa.watermarked.unbound` remains an action-derived candidate label, +not a provider-oracle negative. These counts cannot establish an OpenAI-specific +false-positive rate. + +The dominant exporter lattice was removed explicitly. The raw period-8 +template placed 84-90% of its energy in period-2 or period-4 subspaces. For +each image, projecting out the complete period-4 component before refitting +reduced native/JPEG template correlation to 0.564 but still accepted 18 of 140 +validation and 15 of 158 test positives with zero test controls. The useful +residual therefore does not collapse to the previously documented 2x2/4x4 +generator lattice. + +A direct pixel ablation tested causality against the frozen local score. The +normalized native period-8 direction was subtracted only from the sRGB Blue +channel, with a scalar search for the smallest amplitude that cleared both the +native and JPEG-95 thresholds. All 16 locked-test sources cleared. Repeating +the same selected amplitude at the least-correlated cyclic phase cleared none +of 16. Median amplitude was two Blue-channel integer levels, median PSNR was +52.24 dB, and median SSIM was 0.99639. The minima were 40.62 dB and 0.97184, +so even as a local ablation this is not yet a production-quality remover. +More importantly, score causality does not establish that the candidate is the +OpenAI SynthID decoder surface. + +An all-size map then evaluated the same fixed native period through partial +modulo folds, so dimensions did not need to be divisible by eight. It accepted +94 of all 7,259 OpenAI-labeled files across 13 of 605 geometries and 9 of 8,670 +heterogeneous controls across 7 of 1,534 geometries. The three fit geometries +contributed 76 positives. The largest exported geometries, including 2,265 +files at 1086x1448 and 992 at 1254x1254, contributed none. Two corpus rows were +excluded because their decoded geometry no longer matched the indexed metadata. + +A scale search was frozen to integer periods 4 through 16 before examining the +all-size results. Its score was the maximum, across periods, of the minimum +native and JPEG-95 correlations. The common threshold was set above every +period-searched validation control, thereby charging the search itself against +specificity. It accepted 13 of 1,854 validation positives and none of 2,227 +validation controls. On test it accepted 14 of 1,839 positives and one of 2,162 +controls. Every accepted OpenAI image selected period 8; no other period added +a hit. The carrier does not scale continuously with stored resolution. + +This is the strongest local OpenAI periodic candidate so far: it is codec +tolerant, survives explicit subperiod deflation, has a phase-specific causal +ablation, and is not explained by date, native geometry, or hash-group leakage. +It is still research-only. It has low sensitivity, fails geometric transforms, +has one all-size test-control collision, and lacks a matching-provider oracle. +The next independent branch should look for a resampling-invariant, +content-dependent residual on source-grouped data rather than broadening the +same period search. No runtime OpenAI pixel expert or remover is justified. + +The next branch removed phase from the folded LMS spectrum and required three +views: native pixels, JPEG 95, and a 0.75x down-and-up resize. It screened +magnitude, power, and log-magnitude profiles at periods 4 through 16 on +validation only. The selected period-16 power candidate used a separately +fitted template for each view. It accepted 8 of 140 validation positives and 8 +of 158 locked-test positives, with no native-geometry control accepted in +either split. On the complete wide challenge it accepted none of 2,227 +validation and 2,162 test Spaces controls, one of all 261 action-derived +same-provider controls, and none of the 118 exact-version `gpt-image 2.0` +controls. Five of its eight test hits overlapped the signed period-8 candidate; +three were additional. + +Subperiod deflation bounded that result. Removing the exact period-8 Fourier +support from each period-16 spectrum improved native-to-resize template +correlation from 0.061 to 0.925, but the locked test then retained five +positives and accepted two of 135 controls. Removing period-4 support retained +no test positive. The transform-tolerant energy is therefore part of the same +harmonic family rather than a clearly independent carrier. + +Finally, a stricter experiment required one shared phase-free template across +all three views. Validation selected period-15 magnitude, whose independently +measured view directions correlated 0.992 for native versus JPEG and 0.900 for +native versus resize. It retained 7 of 158 native-geometry locked-test +positives and none of 135 controls. That apparent invariance failed on the +wide geometry challenge: the same thresholds accepted 11 of 2,227 validation +and 14 of 2,162 test Spaces controls, plus seven of 261 same-provider controls +and two of 118 exact-version controls. It is a general renderer or resampling +spectrum and is rejected. + +The separately fitted period-16 three-view ensemble remains a useful research +diagnostic because it survived the broader control challenge, but it is not a +single codebook, has only eight native-geometry test positives, and derives +most of its signal from the same subperiod family. It does not rescue the +universal-detector claim or justify runtime code. + +A non-learned fractal hypothesis was tested last. For each image it compared +the phase-free cube-root LMS spectrum with the same image after JPEG 95 and +after a physical 0.75x reduction. Candidate native periods 8, 12, 16, 20, and +24 were paired with scaled periods 6, 9, 12, 15, and 18. The score was the +minimum of native-to-JPEG and native-to-scaled spectral cosine, so no provider +template or fixed phase participated. Validation selected period 12-to-9 +log-magnitude and retained 3 of 140 positives above all 145 controls. The +locked test accepted none of 158 OpenAI positives and one of 135 controls; the +OpenAI median was 0.515 versus 0.555 for controls. Simple phase-free spectral +self-similarity is not an OpenAI watermark feature and the fractal branch is +rejected. + +The scale hypothesis was then challenged against the common non-native export +geometries. A nearest-native inverse-raster surrogate recovered none of 1,321 +test OpenAI images and accepted two of 1,485 symmetrically resampled control +views. That mapping was intentionally treated as incomplete because it changed +aspect ratio for important cohorts. The largest cohort, 1086x1448, is instead +consistent with an isotropic `1/sqrt(2)` reduction from 1536x2048. Restoring +that exact hypothesized source raster still recovered none of 579 validation +or 562 test positives. A direct fractional-phase fold avoided inverse +interpolation and searched two raster-coordinate conventions plus all 64 cyclic +origins, charging the complete registration search against validation +controls. It retained 2 of 579 validation and 4 of 562 test positives while +accepting 2 of 135 test controls. Its median score was lower for OpenAI than +for controls. The fractional-carrier explanation for this dominant export +geometry is rejected. + +A nonlinear cyclostationary screen next tested whether content-dependent or +payload-varying signs cancel the cross-image mean while leaving a periodic +variance. It compared folded magnitude, energy, locally normalized energy, +and cross-channel products in cube-root LMS and OKLab. Validation selected the +absolute residual of the third cube-root LMS component at period 4. It retained +33 of 140 validation and 43 of 158 locked-test positives, but also accepted 2 +of 135 locked-test controls. The frozen wide challenge confirmed a renderer +confound: it accepted 48 of 1,854 validation and 52 of 1,839 test OpenAI images, +against 7 of 2,227 validation and 5 of 2,162 test Spaces controls. It also +accepted 10 of 261 action-derived same-provider controls, including 5 of 118 +that explicitly name `gpt-image 2.0`. Requiring spatial agreement did not +repair specificity; the validation-selected median over four quadrants kept +41 of 158 test positives and the same two native-geometry test controls. + +Finally, a second-order cyclic screen folded spatial products of the residual +with lags 1, 2, 4, and 8 along axial and diagonal directions. It evaluated 360 +fixed LMS/OKLab, channel, period, lag, and normalization combinations. The +validation winner used the second cube-root LMS component, period 16, and a +vertical four-pixel lag. It retained 36 of 140 validation and 43 of 158 test +positives, but accepted 4 of 135 locked-test controls. Periodic magnitude, +patch consistency, and lag autocorrelation therefore expose reproducible +OpenAI renderer structure but do not meet the SynthID specificity gate. + +These negative results bound the current manual feature search. Signed phase, +phase-free power, subperiods, integer and fractional scale, inverse raster +registration, spectral self-similarity, nonlinear periodic variance, spatial +consistency, and second-order lag products have all either lost sensitivity or +crossed held-out controls. Adding further hand-selected transforms to the same +exposed corpus would increase adaptive overfitting rather than confidence. A +defensible next OpenAI epoch requires provider-authorized oracle labels or +clean same-content counterfactual pairs and a fresh temporal holdout. + +An untouched 2026-08-07 through 2026-08-10 temporal cohort supplied that +holdout for validation, but not the missing oracle negatives. It contained 486 +explicit OpenAI-provenance positives and 504 heterogeneous controls. Only seven +additional files were pure OpenAI C2PA candidates without a SynthID assertion; +the remaining same-platform candidates contained inferred or mixed Google and +OpenAI watermark provenance and were not treated as clean negatives. The frozen +signed period-8 candidate accepted 5 of 486 positives and 1 of 504 controls. +The phase-free period-16 candidate accepted 2 of 40 supported positives and 1 +of 50 supported controls. The nonlinear period-4 candidate accepted 14 of 486 +positives and 8 of 504 controls. The manual carrier branches fail temporal +transfer. + +### 2026-08-10: OpenAI content-dependent decoder challenge + +The periodic result did not exhaust the neural-watermark hypothesis. OpenAI's +ChatGPT Images 2.0 system card describes its watermark as content-specific, and +the SynthID-Image paper says the encoder is content-dependent by design. The +paper also separates watermark detection from payload recovery and evaluates +the external SynthID-O model at its preferred 512x512 input size. These facts +support a learned full-field decoder, but they do not establish that OpenAI and +Google deploy identical model weights, payloads, or pixel spectra. + +A second corpus audit found 261 same-provider candidate controls. Their signed +provenance contained `created` and `converted` actions without the +`c2pa.watermarked.unbound` action, but a subsequent software-agent audit showed +that they do not all come from the positive generation epoch. Only 118 name +`gpt-image 2.0`: 97 training, 16 validation, and five development-test images. +Another 108 name an earlier version and 35 do not expose a usable version. The +full 261-image split remains 217/26/18. These are useful confound controls, but +even the exact-version subset is action-derived rather than a provider-oracle +negative label. + +Two pixel-only native-resolution experts were trained with the mixed-version +control set. A +local residual CNN accepted 132 of 1,280 validation positives and 149 of 1,364 +development-test positives, with zero same-provider or COCO controls in both +splits and two of 1,373 difficult development-test controls. A separate +spectral, autocorrelation, bit-plane, and residual-error expert accepted 175 +validation and 167 development-test positives with no accepted controls. A +validation-selected linear combination accepted 243 positives in each split, +with no accepted validation or development-test control. The zero-error +development-test bound across 3,391 controls is 0.0883% at one-sided 95% +confidence, but the same-provider subset contains only 18 images and therefore +does not independently establish provider specificity. + +The exact-version audit did not invalidate the native score counts, but it +weakened their attribution. None of the 16 validation or five development-test +`gpt-image 2.0` controls crossed the residual, forensic, ensemble, canonical, +or JPEG-equalized thresholds. That is directionally useful, but five temporal +test controls cannot establish deployment specificity. Perceptual-hash and +local CLIP searches compared all 118 exact-version controls with 7,259 +positives. They recovered a few near-content matches but no clean before/after +watermark pair. Sub-pixel affine alignment left content, edge, and photometric +changes in the residual, while 8x8, 16x16, and 32x32 folded residual +correlations varied in sign and magnitude. Those candidates cannot be used as +causal watermark differences. + +Transformation challenges rejected the combined native expert as a SynthID +decoder. Of its 243 accepted development-test positives, 121 remained accepted +after a 5% crop round trip, six after a 75% resize round trip, and none after a +JPEG-95 round trip. No transformed same-provider control was accepted. The +codec fragility is inconsistent with treating the score as the documented +robust watermark without further evidence. + +The 512x512 branch also failed the held-out specificity gate. Its selected +model accepted 238 validation and 194 development-test positives, but also +accepted two of 18 same-provider, four of 1,373 difficult, and nine of 2,000 +COCO development-test controls. Finally, a CNN trained and evaluated only +after applying the same JPEG-95 round trip to every positive and control +accepted eight validation positives and six development-test positives while +accepting five of 2,000 development-test COCO controls. Equalizing the codec +therefore removed nearly all of the apparent signal instead of revealing a +robust hidden decoder feature. + +An independent three-CNN OpenAI surrogate was also evaluated from its +published weights, without adopting its code or claims. At the published 0.5 +threshold its ensemble marked 82.0% of COCO validation controls and 83.4% of +COCO development-test controls as watermarked. It also marked 56.3% of +difficult development-test controls and 15 of 16 exact-version OpenAI +validation controls, compared with 87.3% of OpenAI validation positives. The +repository does not publish its training corpus or logs, and its implementation +applies supposedly shared pair augmentation independently. This result rejects +the model as a SynthID decoder and demonstrates why same-content-looking +regenerations are not sufficient negative labels. + +A final transform-augmented full-field branch tested the closest practical +approximation to the documented neural decoder. A 512x512 CNN consumed both +RGB and a fixed high-pass residual and trained with random crop/resize, +JPEG 70-100, horizontal flip, and mild color transforms. Training sampling +balanced watermarked positives, difficult controls, and the exact-version +OpenAI candidate controls. Selection required a score above every control for +both native and JPEG-95 validation views. All four fixed training epochs +accepted zero of 500 selection positives. On the full validation set the +frozen model accepted zero of 1,280 OpenAI positives in native, JPEG-95, +75%-resize, and 5%-crop views, while accepting up to two unrelated controls. +On development test it accepted zero of 1,364 OpenAI positives in native, +JPEG-95, and resize views. The crop view accepted one OpenAI positive and one +of 1,373 difficult controls. Transform augmentation therefore did not recover +a usable weaker OpenAI signal. + +The measured OpenAI-native separation is best classified as an export or +rendering noiseprint, not a validated SynthID detector. None of these models is +added to runtime. Transform-augmented training has now also failed. A +defensible next epoch needs provider-authorized labels or clean same-content +before/after watermark pairs, plus a fresh temporal holdout that has not +influenced representation or threshold selection. + +The same untouched August cohort then challenged the frozen 512-square forensic +model and its unchanged threshold. It accepted 82 of 486 explicit OpenAI +positives, one of 504 heterogeneous controls, and one of seven pure OpenAI C2PA +candidate controls. The aggregate score separation was real, but transformation +testing resolved its interpretation: all 82 native detections disappeared after +both JPEG-95 and WebP-95 round trips. A 0.75x resize retained 46, and a 5% crop +retained 23. The model therefore generalizes as a fragile OpenAI export +noiseprint, not as the documented robust SynthID signal. + +The provider-key alternative was then tested directly. OpenAI training folds +were clustered into 1, 2, 4, 8, 16, or 32 spherical period-8 cube-root-LMS +directions, and the cluster count was selected on validation only. Eight +directions raised the development-test result from 16 to 27 of 158, with one +of 135 heterogeneous controls and two of 52 action-derived same-provider +controls accepted. This did not reveal eight independent codewords: the two +dominant clusters contained 234 of 283 training rows, and their templates +correlated at 0.982 natively and 0.924 after JPEG-95. + +The frozen multi-direction model accepted none of 443 Google native-geometry +files, but accepted 5 of 39 Microsoft Designer files and 123 of 674 OpenAI- +platform files. All five Microsoft hits had joint `Microsoft, OpenAI` +provenance, one explicitly asserted an OpenAI SynthID watermark, and all five +selected the same dominant direction as 111 OpenAI hits. On the +fresh August native-geometry cohort, SHA-256 grouping reduced seven OpenAI row +hits to five unique content groups; two unique Microsoft controls crossed the +same dominant direction. A sign-invariant carrier-subspace fit retained only 1 +of 158 development-test positives and none of 40 fresh positives. A final +OpenAI direction orthogonal to Google, Microsoft, and other-provider train +means likewise retained 1 of 158 and none of 40. The measured period-8 family +is distinct from Google but is shared with a Microsoft/OpenAI lineage. + +Regrouping by the signed watermark assertion found 115 of 589 explicit OpenAI- +SynthID files accepted, against 13 of 117 OpenAI-lineage files without an +assertion, none of 443 Google files, and one of 89 other-provider files. The +aggregate one-sided exact-test result was `p = 0.0179`; the already exposed +development-test split was not independently significant at 27 of 159 versus +2 of 25 (`p = 0.202`). Missing watermark action is not an oracle-negative +label, so this is compatible with a provider-keyed family but does not prove +that the measured component is the watermark. + +The frozen score did exhibit real codec persistence. WebP-95 retained 24 of 27 +native development-test detections and added nine other positives, with the +same one of 135 controls retained. JPEG-90 retained eight of 27 and added no +positive, with zero controls. A 0.75x resize and 5% crop retained none. The +result is stronger than the full-image noiseprint and supports a genuine +OpenAI/Microsoft raster-phase component, but its geometric fragility excludes +it as the documented complete robust SynthID decoder surface. + +The broader SynthID architecture permits provider-specific codes without +requiring one shared pixel template. The SynthID-Image paper separates binary +detection from payload recovery, identifies payloads as a mechanism for +distinguishing customers of one service, and assigns watermark versions to the +encoder. OpenAI and Google can therefore use the same watermark family while +deploying different encoder versions or payload distributions. + +The narrower same-carrier/different-phase prediction was challenged directly. +Period-8 cube-root LMS folded residuals were collected from 581 byte-unique +OpenAI-asserted files, 443 Google-asserted files, 93 OpenAI-lineage files with +no watermark assertion, and 82 other-provider controls. Both provider +contrasts were internally stable across deterministic halves. Raw phase-free +power means had cross-provider cosine 0.480, but this shared imaging background +disappeared after subtracting the respective controls: positive power support +had cosine 0.0123, bootstrap median 0.0232 with a 95% interval of 0.0009 to +0.0856, and no overlap among the 12 strongest coordinates. The three +per-channel cosines were all at most 0.0155. The signed contrasts were negative, +not phase-locked. The evidence therefore rejects a simple Google template with +an OpenAI phase or payload substitution. It remains compatible with a distinct, +content-dependent SynthID encoder version and motivates separate learned +experts rather than further rotation of the Google carrier. + +A fixed three-query OpenAI Verify validation then established attribution while +also falsifying completeness. An asserted OpenAI image accepted by the frozen +eight-direction period-8 candidate was submitted both in its original form and +as a pixel-identical PNG re-encoded without metadata. Both returned `SynthID +detected` and `Content Credentials not detected`. A second asserted OpenAI +image rejected by the local candidate, with native/JPEG scores 0.709/0.657 +against thresholds 0.918/0.923, was also re-encoded without metadata; it +returned the same pixel-only SynthID verdict. The period-8 family is therefore +a genuine OpenAI SynthID component with known oracle-positive false negatives, +not merely an export fingerprint and not a complete detector. No adaptive +oracle sweep followed: OpenAI's Content Provenance documentation explicitly +disallows repeated queries to reverse-engineer, remove, or evade a watermark. +The next local epoch must treat period-8 as one expert and discover complementary +content-dependent experts using frozen corpus splits, without oracle-guided +feature selection. + +Relaxing the eight-direction model to a separate validation-control threshold +for every direction increased validation recall to 36 of 140 with zero of 145 +controls, but it did not preserve specificity. The locked development test +accepted 46 of 158 OpenAI rows and 4 of 135 controls. A wider provider challenge +accepted 180 of 603 explicit OpenAI-SynthID rows, 15 of 80 OpenAI-lineage rows +without an assertion, 6 of 443 Google rows, and 7 of 112 other-provider rows. +The fresh temporal result was 13 of 40 OpenAI rows, 4 of 50 general controls, +and 1 of 3 same-provider controls. The oracle-positive local false negative did +cross direction 4 under this relaxed calibration, but that direction survived +only 1 of 4 leave-one-positive-out refits; the oracle-positive row itself then +failed its JPEG threshold. Direction-specific thresholds therefore expose +unstable minority clusters rather than additional validated codewords. + +A train-only bilinear follow-up tested whether a content-dependent or keyed +phase varies between images while retaining native-to-JPEG covariance within +each image. It subtracted the control within-class cross-covariance from the +OpenAI cross-covariance, selected the positive-eigenvalue rank on validation, +and kept the threshold above every validation control. Rank 1 retained only 1 +of 158 locked-test OpenAI rows and zero of 135 controls, then zero of 40 fresh +OpenAI rows and zero of 53 fresh controls. A single codec-stable variable-phase +subspace is therefore rejected as the missing complementary expert. + +Taking local spectral magnitude before spatial averaging produced a superficially +stronger complementary signal. Eighteen period, block-normalization, and spectral +variants were screened on validation; period-8 unit-block log-magnitude retained +8 of 158 locked-test OpenAI rows and zero of 135 controls, then 3 of 40 fresh +OpenAI rows and zero of 53 fresh controls. None of the three fresh hits overlapped +the frozen global eight-direction hits, and only one overlapped the unsafe +per-direction union. Both oracle-positive images, including the known local +false negative, remained below this expert's thresholds. + +The full provider challenge rejected watermark attribution despite that clean +development transfer. The local-spectrum expert accepted 17 of 603 explicit +OpenAI-SynthID rows, 5 of 80 OpenAI-lineage rows without an assertion, zero of +443 Google rows, and 2 of 112 other-provider rows; both other-provider hits were +Microsoft. Explicit watermark assertion was not enriched over same-lineage +non-assertion (`p = 0.966`, one-sided exact test in the required direction). +Recalibrating both frozen view thresholds above the 32 same-lineage validation +maxima reduced locked-test recall from 8 of 158 to zero. Local pre-aggregation +magnitude therefore reveals another OpenAI/Microsoft raster component, not a +defensible complementary SynthID expert. + +The proposed directional-wavelet branch was also tested directly on OpenAI. +A three-level DTCWT over the period-8 cube-root-LMS residual screened all six +orientations, the diagonal and axis-near groups, individual levels, and complex, +magnitude, and phase statistics. Validation selected all-orientation normalized +complex correlation at 19 of 140 OpenAI rows and zero of 145 controls. It +retained 16 of 158 locked-test OpenAI rows and zero of 135 controls, followed +by 2 of 40 fresh OpenAI rows and zero of 53 fresh controls. + +This is a stronger representation of the known phase carrier, not a new expert. +Both fresh hits were already accepted by the frozen eight-direction model, the +known oracle-positive local false negative remained negative, and the other +oracle-positive row missed only the DTCWT JPEG threshold at 0.894 versus 0.897. +The provider challenge accepted 77 of 603 explicit OpenAI-SynthID rows, 3 of 80 +OpenAI-lineage non-assertions, zero of 443 Google rows, and 2 of 112 other rows, +both Microsoft. The explicit-assertion enrichment was real (`p = 0.00879`), but +72 of the 77 hits overlapped the signed phase expert and their union increased +its explicit count only from 115 to 120. Same-lineage calibration retained 4 +of 158 locked-test and 1 of 40 fresh OpenAI rows, while one Microsoft test row +still crossed. + +The transform challenge then falsified operational shift invariance. Cyclic +rolls by `(1, 1)` and `(3, 5)`, a 0.75x resize round trip, and a 5% crop round +trip each retained zero of the 16 baseline detections. JPEG-90 retained four. +WebP-95 retained 15, accepted 25 OpenAI rows in total, and introduced one of +135 controls. DTCWT therefore improves codec tolerance for the already measured +raster phase but does not solve registration, scale, arbitrary resolution, or +the missing OpenAI payload variants. + +Selecting the best DTCWT magnitude and phase candidates within their own +families did not change that conclusion. The off-diagonal magnitude candidate +retained 10 of 158 locked-test OpenAI rows and zero of 135 controls, then 3 of +40 fresh OpenAI rows and zero of 53 controls. All three fresh hits were already +signed-phase hits. The near-vertical phase candidate retained 9 of 158 and zero +of 135, but failed temporally at 1 of 40 OpenAI rows versus 3 of 50 general +controls. Magnitude also retained zero of its ten baseline detections after a +one-pixel roll, `(3, 5)` roll, resize, or crop; JPEG-90 retained one and WebP-95 +retained two. Taking magnitude only after periodic complex folding preserves +the phase-origin dependency and does not realize DTCWT's intended shift +invariance. + +A final DTCWT screen removed periodic folding entirely and summarized global +energy over six orientations and three levels. Nine validation-screened +translation-insensitive magnitude, power, orientation, and robust-quantile +statistics selected median magnitude proportions within each level. The locked +test accepted only 5 of 158 OpenAI rows and 1 of 135 controls; the fresh holdout +accepted zero of 40 OpenAI rows and zero of 53 controls. DTCWT therefore presents +a measured tradeoff in this corpus: preserving the periodic complex field +retains the real OpenAI carrier but its phase origin, while pooling enough to +be translation insensitive removes the transferable separation. + +A metadata-stratified audit then tested whether the 77 provider-challenge hits +were confined to one encoder rollout or geometry. Within the exact `OpenAI +Media Service API` cohort, the frozen DTCWT expert accepted 75 of 581 asserted +rows and 1 of 52 same-generator rows without an assertion. Asserted-row rates +were 3/16 in May, 46/365 in June, and 26/200 in July, and 6/47, 44/320, and +25/214 across the three native geometries. The native/JPEG minimum score had +0.721 AUC against the same-generator non-assertion stratum. The component is +therefore continuous across time and geometry rather than a short-lived export +cohort, although missing assertion remains an indeterminate label rather than +a clean watermark negative. + +A prespecified shift-invariant follow-up sorted all 64 cyclic period-8 +template correlations within each image. This preserves the shape of the +correlation orbit while discarding its absolute phase; one train-only contrast +was fitted for native and JPEG-95 views, with both thresholds placed above all +145 validation controls. The individual views retained 5 and 7 of 140 +validation positives, but their conjunction retained none of 158 locked-test +positives and none of 135 controls. JPEG, WebP, resize, and crop could not +rescue a baseline detection because there was none. Thus the transferable +OpenAI evidence is carried by signed raster phase, not merely by a +shift-invariant matched-filter peak shape. + +Content-hash grouping also corrected the temporal denominators. The 486 OpenAI +rows represented 417 unique byte groups, and the 234 Google rows represented +211. The signed OpenAI period-8 result became 4 of 417 unique positives versus +1 of 454 general-control groups; the full-image noiseprint became 68 of 417; +the Google runtime result became 153 of 211. Repeated uploads did not change the +decision, but row-level counts must not be treated as independent evidence. + +### 2026-08-13: OpenAI low-texture phase and masking-alignment challenge + +The dominant 1086x1448 export geometry supplied 2,265 byte-unique images with +an explicit OpenAI SynthID assertion, 830 general controls at the same geometry, +and 26 OpenAI-lineage files without a watermark assertion. The exact-geometry +split contained 1,124/579/562 asserted OpenAI train/validation/test rows and +399/203/228 general controls. The lineage rows remained an indeterminate +stratum rather than being relabeled as negatives. + +A train-only cross-image phase model selected 256 RGB bins from the 20 +lowest-texture references in each of three luminance groups. It used +polarity-invariant axial phase and scored both native pixels and a JPEG-95 view +after canonicalizing to 512 square. Independent train halves shared only two +of 256 exact channel-frequency bins and three spatial coordinates. The full +model produced native AUC 0.526 on validation and 0.512 on test; JPEG-95 AUCs +were 0.521 and 0.512. A threshold above every validation control accepted none +of 562 test positives, while a validation 1% FPR threshold accepted four +positives and two of 228 test controls. Texture quartiles did not reproduce a +useful exception. The common absolute-phase hypothesis is rejected for this +OpenAI geometry. + +The official architecture points to a different mechanism: watermark +generation is deliberately content-dependent to resist watermark exchange, +and nearly uniform content can be filtered rather than watermarked. A second +frozen probe therefore discarded absolute phase and global residual amplitude. +It measured 60 OKLab relations between blockwise high-frequency energy and +local texture, contrast, and chroma at two filter scales and two block sizes. +One train-only shrinkage-Fisher direction was scored in native and JPEG-95 +views, and the joint decision used their minimum above every validation +control. Joint AUC was 0.563 on validation and 0.593 on test. The frozen test +count was 15 of 562 asserted OpenAI rows and one of 228 controls. + +That weak enrichment was not a stable expert. Directions fitted on deterministic +train halves had raw-coordinate cosine 0.148. One half accepted 13 test +positives and two controls, the other accepted none; their test score +correlations were 0.335 on positives and 0.127 on controls, with no overlapping +positive decision. The branch is rejected as an unstable content or export +correlate. Together these experiments reinforce the existing boundary: a new +OpenAI epoch needs pixel-oracle labels or clean same-content counterfactuals, +not further adaptive feature selection on provenance-labeled exports. + +### 2026-08-13: multi-scheme domain generalization and Gemini bicoherence + +A deliberately broader experiment tested whether SynthID could inherit a +generic post-hoc watermark signature instead of a provider carrier. Ninety +source-matched COCO pairs were prepared for each of TrustMark P, Meta VideoSeal +Y-256, classical 64-bit DWT-DCT, and Meta WAM-MIT, always with random payloads. +The official VideoSeal decoder recovered a mean 99.6% of bits across identity, +JPEG-90, 0.8x resize round trip, and 5% crop round trip; its mean pair PSNR was +45.04 dB. WAM recovered 99.5-99.6% across the same views at mean PSNR 38.44 dB. +DWT-DCT supplied an architecture-diverse control, but only identity and resize +were retained as positives because its official decode collapsed to chance +after JPEG and crop. + +A sign-symmetric 3,900-feature spatial rich model first learned residual +co-occurrences in luminance, opponent-color, and Lab channels. Its linear and +quadratic paired-delta variants recognized familiar schemes but did not transfer +to a held-out scheme. A stronger neural follow-up used a fixed residual bank, +worst-family training, cross-family residual transplantation, random sign, +phase, and amplitude, plus phase-scrambled equal-power residuals as hard +negatives. Each of four runs withheld one complete encoder family and all test +source images. Held-out aggregate AUCs were 0.562 for TrustMark, 0.474 for +VideoSeal, 0.497 for DWT-DCT, and 0.437 for WAM. The corresponding strict +true-positive rates were 5.0%, 3.3%, 0%, and 0%. This rejects the available +open-proxy path to a universal single-image detector: even causal paired data +from three distinct schemes did not generalize to a fourth. + +A separate Gemini experiment tested a translation-invariant higher-order +signature. Block bicoherence measured normalized phase coupling at period-16 +triplets satisfying `f1 + f2 = f3`; unlike ordinary phase correlation, the +global image-shift phase cancels from this product. Six Green, cube-root LMS, +and OKLab representations, two residual scales, two block sizes, and three +train-only harmonic counts produced 72 candidates. The data were separated +into harmonic-fit, candidate-selection, threshold-calibration, and final +splits: 30/20/50 Gemini positives and 100/100/100/199 Open Images controls. +The best selection candidate accepted none of 20 positives at the selection +control maximum and had AUC 0.523. After independent recalibration, the final +result was 0 of 50 Gemini positives and 0 of 199 controls, with AUC 0.374. +Period-16 bicoherence is therefore rejected as a scale or origin rescue. The +measured Gemini carrier remains a linear, registered periodic tile rather than +a transferable harmonic cascade. + +### 2026-08-14: remaining Gemini resolution challenges + +The 0.5x gap was tested with a local-consensus detector rather than another +global period-8 score. Each half-scaled image was divided into 64-pixel patches; +the 90th percentile of full-template patch correlations was selected without +using the locked test. It accepted 33 of 49 validation positives and zero of 166 +validation controls at the frozen 0.552176 threshold, then 27 of 52 locked-test +positives and zero of 140 controls. This looked like a useful low-resolution +expert but did not survive scale-matched specificity testing. + +The frozen challenge symmetrically resized every remaining Spaces control and a +separate 1,000-image Open Images reserve to 0.5x. The expert accepted 61 of +10,906 Spaces controls and 22 of 1,000 Open Images controls. All 83 errors were +rescored with the prespecified feature inventory. A negative-Green patch-spread +gate could remove them post hoc, but retained only 12 of 106 base-positive +development hits. On the later August 7-10 cohort the original patch expert +accepted 67 of 234 Google-provenance rows and 8 of 689 controls, while the new +conjunction retained only 2 Google rows and zero controls. The broad challenge +rejects the base expert and the temporal result rejects the post-hoc repair. +No 0.5x claim is added. + +The large-image window family was completed separately. One centered, +phase-aligned 2,048-square registered window accepted 1 of 37 candidates above +10 megapixels and zero of 89 controls. Scoring every phase-aligned 2,048-square +window and accepting the maximum raised recall to only 2 of 37, still with zero +controls; the maximum control decision score was already 0.9677 against the 1.0 +runtime boundary. Together with the previously rejected fixed-window consensus +and full-frame resize, these results leave the 10-megapixel ceiling intact. + +Near-ceiling windows, color spaces, and JPEG-paired phase directions did not +provide a missing large-image expert. Three aligned windows of approximately +10 megapixels retained 1 of 37 candidates and zero of 89 controls. A native +fixed-score plus Green mid-band rule retained 15 of 19 candidates in four exact +recurring geometries and accepted one of 11,417 resized Spaces controls, but +collapsed to 0 of 19 after JPEG 90. A codebook fitted to the minimum native/JPEG +period-16 correlation selected RGB Green mid-band on the pre-July split. It +retained 23 of 26 candidates and zero of 40 controls there, then 10 of 11 and 7 +of 49 on July. Lab and OKLab candidates failed on the same hard control. The +apparent native signal is a codec or upscaler lattice rather than a robust +watermark invariant. + +Those 37 candidate labels came from Google C2PA and an inference that Google +media is likely watermarked, not an individual Google pixel-decoder result. +They cannot support a production recall claim. The next large-image step is an +immutable SHA-to-verdict Gemini batch, followed by fitting only on +oracle-detected files. The first live attempt on 2026-08-14 was stopped by +Google reCAPTCHA before the first new file submission; no CAPTCHA bypass was +attempted. + +### 2026-08-14: official large oracle and opponent-color runtime branch + +The frozen Gemini batch was subsequently submitted without bypassing CAPTCHA. +Every submitted file had AI provenance removed while preserving the decoded +RGBA fingerprint. Google returned a pixel-positive verdict for seven of seven +usable submissions across 3072x5504, 5504x3072, 5632x3072, 6336x2688, +3584x4800, 5016x3328, and 4096x4096. An eighth case was indeterminate after the +verifier quota ended. This is direct evidence for seven positives, not a label +upgrade for the remaining 30 C2PA-inferred candidates. + +Revisiting the native period-16 fold by opponent color exposed the missing +separation. Google candidates preserve strong Red-minus-Green and +Blue-minus-Yellow phase agreement across every phase-aligned 2048-square +window, while natural resize aliases do not preserve the complete conjunction. +The frozen large-v1 rule requires minimum fixed score `0.14`, minimum +Red-minus-Green spatial correlation `0.90`, minimum Blue-minus-Yellow spatial +correlation `0.70`, and a minimum signed Blue-minus-Yellow mid-band correlation +at or below `-0.15`. The 3072x5504 portrait geometry additionally requires the +maximum Green mid-band correlation to stay at or below `0.06`. Its public score +is the minimum normalized gate margin and crosses at `1.0`. + +The rule retained all 37 inferred candidates, including all seven official +pixel positives. It accepted none of 17,417 exposed COCO, Open Images, and +Spaces controls after the portrait alias gate. Because those controls informed +the final gate, they do not certify the operating point. The constants were +then challenged through the actual runtime on a frozen, decoded-pixel-deduped +local holdout: 2,000 COCO val2017 images excluded from the earlier 3,000-image +large color-phase challenge plus 637 unique Picsum controls deduplicated from +1,000 files. Four large geometries and four resampling kernels were balanced. +The result was 0 of 2,637 detections, with maximum score `0.0592777965`; the 95% +zero-error upper bound is about 0.114%. These source collections existed before +the freeze and supported other experiments, so the result is feature-unseen +rather than a fresh-acquisition holdout. + +The post-freeze Open Images acquisition was too slow to complete the planned +3,000-image challenge, but its completed subset provides an independent source +audit. Incomplete `.aria2` files, all prior Open Images decoded-pixel hashes, +and duplicates were excluded before scoring, leaving 41 controls whose IDs and +URLs had been frozen after the constants. The unchanged runtime accepted 0/41; +the maximum score was `0.4083013324`. This small audit supports the source +boundary but does not replace the 2,637-control feature-unseen interval. + +A prespecified post-freeze codec stress test re-encoded the seven official +positives at unchanged dimensions. Native detection was 7/7, while JPEG-95 and +JPEG-90 were each 0/7. Thresholds were not refitted. Large-v1 is therefore a +native-pixel and lossless-copy detector, not a lossy-transcode detector. + +The old fixed-only rule still crosses on four locally saved rows representing +three unique pixel groups. One group is an Adobe file whose C2PA explicitly +reports Google SynthID present. The other two remain queued for an official +Gemini verdict and are not counted as clean negatives. The runtime branch is +therefore positive-only and does not turn a miss into an absence claim. + +Production routing now selects the fixed expert from 1 through 10 megapixels +and the large expert above 10 through 18 megapixels when both sides are at least +2048 pixels. The score exporter emits fixed, registered, and large observations +separately, preventing the large result from being mislabeled as a fixed score. +The router accepts a registered or large threshold crossing only within that +expert's measured scope and otherwise abstains. + +### 2026-08-14: final OpenAI local rescue checks + +An assertion-contrast probe at the dominant exact 1254-square geometry tested +RGB, OKLab, and cube-root-LMS period-8 channel directions without treating +missing assertion as a clean label. Validation selected `srgb:p8:c2`, accepting +45 of 259 asserted rows and zero of 13 same-provider plus three other-provider +rows. The locked test accepted 39 of 233 asserted rows, zero of 15 same-provider +rows, and one of three other-provider rows. The all-row audit accepted 185 of +992 asserted rows, one of 55 same-provider rows without an assertion, and one +of 13 other-provider rows. A safe boundary above either wider control maximum +retained only a small minority of the asserted cohort. Exact geometry therefore +does not turn the weak OpenAI component into a production detector. + +An independent published-decoder veto was then tested with the official +InvisMark checkpoint. The downloaded checkpoint hash was fixed before use, and +its bundled self-test reached 0.997 mean confidence with 0.97 recovered-bit +accuracy. In contrast, OpenAI DTCWT hits, OpenAI misses, exact-1254 hits and +misses, same-provider controls, Google, Microsoft, Canva, and unrelated controls +all had median confidence near 0.18. The decoder is working, but its watermark +family is absent from these cohorts; it cannot distinguish SynthID-bearing +OpenAI pixels from renderer confounds. + +Two representation-level missing-codeword hypotheses were also rejected. +DTCWT modulus computed before spatial period folding selected off-diagonal +phase coherence and retained 5 of 158 locked-test OpenAI rows versus zero of 135 +controls, but zero of 40 fresh OpenAI rows. Whitening the period-8 feature space +and clustering up to 16 codewords retained 5 of 158 and zero controls, then one +of 40 fresh OpenAI rows and one of 50 fresh controls. Neither captured the known +oracle-positive false negative. These were the remaining prespecified phase, +wavelet, codeword, and external-decoder variants; none earns a local runtime +route. + +### 2026-08-15: OpenAI high-pass patch-bag challenge + +One prepared but previously unexecuted content-dependent hypothesis aggregated +the distribution of nine normalized high-pass patch embeddings per image. Its +input contains luminance, Red-minus-Green, and Blue-minus-Yellow residuals but +no filenames, metadata, dimensions, or RGB thumbnails. Mean, standard +deviation, minimum, and maximum pooling allow spatially varying evidence without +requiring one global phase codeword. + +The validation-selected epoch accepted 285 of 1,280 OpenAI rows above every +validation control. With weights and threshold frozen, the already exposed +development test accepted 324 of 1,364 OpenAI rows, 3 of 1,373 difficult +controls, and 1 of 2,000 COCO controls. The later August temporal cohort was +not used by this model or threshold: native scoring accepted 120 of 486 OpenAI +rows, 8 of 504 general controls, and 5 of 33 same-provider rows without a +watermark assertion. Assertion enrichment over the same-provider stratum was +not independently significant (`p = 0.151`, one-sided exact test). After +decoded-pixel grouping the counts were 107/417, 4/453, and 4/30. Requiring the +same unchanged score after JPEG-95 retained only +1/486 OpenAI rows and no controls. + +None of the 120 native OpenAI hits overlapped the frozen signed period-8 hits; +four overlapped the nonlinear period-4 candidate. This supplies new evidence +for multiple complementary OpenAI raster components. It does not supply a +production watermark decoder: same-provider non-assertion rows crossed at a +high rate, general-control specificity failed, and ordinary JPEG destroyed all +but one positive. A post-hoc threshold above the temporal control maximum would +retain 19 unique OpenAI rows, but that threshold has observed the holdout and +is not a new expert. The official OpenAI verifier remains the provider-wide +pixel route. + +### 2026-08-14: production OpenAI verifier boundary + +OpenAI's official Content Provenance API now supplies the production-grade +OpenAI pixel verdict that the local experiments could not justify. The runtime +integration is deliberately a separate `verify-openai-synthid` command, never +an implicit `identify` call. It requires the independent `verify` extra, +`OPENAI_API_KEY`, endpoint access, and `--acknowledge-upload`. + +The implementation establishes metadata independence before the request. It +computes a decoded RGBA fingerprint, strips AI provenance metadata to a +temporary copy, verifies that no AI markers survived and that the format and +pixel fingerprint stayed identical, enforces the documented 50 MiB limit, and +uploads only the temporary PNG, JPEG, or WebP. It parses exactly one `synthid` +entry and deliberately ignores the independent C2PA outcome. C2PA-only +positives, missing or duplicate SynthID fields, altered pixels, surviving +metadata, unsupported formats, and 400/404/429 failures are covered by mocked +tests. No credentialed API request was made during implementation because no +API key was available. + +A separate live smoke used OpenAI's public web verifier after running the exact +production metadata-stripping and decoded-RGBA equality checks. Two OpenAI +images at opposite 3:2 orientations returned `SynthID detected` and `Content +Credentials not detected`. A Google SynthID oracle-positive control and a COCO +natural control returned OpenAI `SynthID not detected`, again with Content +Credentials absent. This confirms pixel-only, provider-specific semantics for +the integration, but four fixed cases do not estimate statistical error rates +and do not constitute a live SDK/API transport test. + +On 2026-08-15, one separately preselected OpenAI-positive PNG completed the +credentialed SDK path after the same metadata removal and decoded-pixel equality +checks. The API returned an independent SynthID `detected` result with nullable +model and generation-time fields, and the command completed without a retry. +This closes the first positive PNG transport smoke only. It does not qualify +JPEG, WebP, negative cases, operational rate limits, retention behavior, or +local-detector accuracy. + +This closes the production OpenAI detection surface through an official remote +backend, not by relabeling the incomplete local period-8 expert. It does not +expand the research oracle authority. The endpoint is not eligible for Zero +Data Retention, and its documentation prohibits repeated reverse-engineering or +evasion queries, so it cannot supply adaptive labels for detector or remover +optimization without separate authorization. + +### 2026-08-15: periodic codeword replacement and its measurement limit + +Suppressing the estimated periodic residual with independent noise spends +quality without reaching the decoder: at a strong budget the OKLab `a/b` +branch moved the Gemini score by a median of `-0.022` at `32.6 dB`, because +independent per-pixel noise averages out under the fold. Replacing the +estimated period-16 residual with a magnitude-matched random-phase codeword +instead moved all four Gemini positives below the frozen local threshold, with +a median score of `0.050` against `0.533` at `53.5 dB` and SSIM `0.9982`. +Adding non-periodic noise on top only cost `7 dB` and changed nothing. + +Both providers show a threshold, not a slope. Gemini keeps all four positives +at replacement strength `0.75` and loses all four at `1.0`; the OpenAI period-8 +proxy keeps `9/16` at `0.25`, `1/16` at `0.50`, and none at `0.75`. Random +injection without the subtraction left `1/16`, so the mechanism is +`subtract + randomize`, not added energy. + +Radial banding of the folded residual put the Gemini carrier in the middle +band: `0.125-0.40 cycles/pixel` gives a median `-0.209` at `56.5 dB`, against +`-0.009` above `0.40`. The OpenAI proxy weights the upper band instead. + +The follow-up question, whether the signal lives in the comb lines or in the +broadband content between them, cannot be answered by the local score at all. +`fold_residual_template` averages every tile, which is a comb filter: only +frequencies that are integer multiples of `1/16` cycles per pixel survive, and +everything between the lines is annihilated before the template is applied. +`probe_comb_vs_broadband.py` measures the consequence rather than asserting it. +Inside the same `0.125-0.40` annulus and at an identical residual RMS, +randomizing only the comb bins moved the four Gemini scores by `-0.177` to +`-0.519`, while randomizing only the bins between them moved them by `-0.0007` +to `-0.0012`. A local null on the broadband branch is therefore guaranteed by +the readout and is not evidence about the watermark. + +That makes the two branches an oracle question. A frozen batch of four such +triples is recorded in +`.local-eval/synthid/prc-oklab-attack-2026-08-15/comb-vs-broadband-oracle-manifest.json`. +Its first attempt established a control rule that the earlier runs had missed: +the tracked originals carry a C2PA `caBX` chunk, and Google answered from it +without ever reaching the pixel path, so an original is not a valid control for +pixel-only candidates. Every arm is now re-encoded from decoded RGB with the +chunk dropped and decoded pixels asserted identical. The re-encoded source then +returned a pixel-path positive, which qualifies the control. The broadband +candidate returned a temporary service limit with a two-hour retry, so no +candidate verdict exists yet; an unavailable answer is not a negative. + +### 2026-08-15: the local score is not the decoder margin + +One group was then completed inside a single account, so that account tier and +request routing stay fixed across its arms; spreading arms of one comparison +across accounts would make a negative indistinguishable from a different +backend. All four arms returned a Google positive: + +| Arm | Local score | PSNR | Google | +| --- | --- | --- | --- | +| source | `0.416` | - | detected | +| comb, equal RMS | `0.239` | `53.97 dB` | detected | +| broadband, equal RMS | `0.415` | `54.27 dB` | detected | +| full period-16 replacement at `0.95` | `0.087` | `52.56 dB` | detected | + +The last row is the result that matters. That candidate sits well below the +frozen local threshold of `0.174`, which is the operating point the removal gate +would have used, and Google still reports the watermark. A local score at or +near zero therefore certifies nothing about the provider decoder, and no removal +claim may rest on it. The equal-RMS pair adds that neither a comb-targeted nor a +broadband-targeted perturbation at roughly `54 dB` moves the verdict, so this +run does not locate the signal either; it bounds what the local readout can say. + +A replication attempt on a second image from a third account never produced a +verdict: that account hit the service limit on its first request, while another +had just served four. The limit therefore does not partition cleanly per +account, and holding several accounts does not buy proportionally more oracle +capacity. Plan oracle work around one window at a time and spend it on the arms +that decide something. + +Querying stopped here rather than escalating the perturbation budget until the +verdict flipped. Searching for the breaking point by repeated queries is exactly +the adaptive use of a provider oracle that this plan rules out, and the sham +control was not spent because it only interprets a negative, and no arm returned +one. + +### 2026-08-15: the same answer from the OpenAI verifier + +The equivalent frozen batch went through the official OpenAI endpoint on +`verify-openai-synthid`, two OpenAI positives in a source and `replace-chroma` +pair each, four calls, one pass. Every arm returned `detected` with metadata +stripped, decoded pixels preserved, and the C2PA outcome ignored. The candidates +sit at `54.13 dB` and `53.97 dB` with residual RMS near `0.50`, the same +perturbation class that empties the local proxy. + +Two providers, two independent verifiers, one conclusion: a periodic-codeword +replacement that drives the local score to zero does not remove what either +decoder reads. The removal program has no evidence of pixel-only removal for +either provider, and the local score cannot serve as its gate. + +A third OpenAI positive was excluded before any call. Every periodic variant on +it produced a residual RMS of exactly zero and a periodic energy ratio of +exactly `1.0`, so the attack changed nothing: it can only redistribute periodic +energy it estimated, and it estimated none. That file is nevertheless recorded +in the manifest as `openai-verify: SynthID detected`. An estimator that finds no +periodic residual in an image the provider verifier flags is direct evidence +that the period-16 and period-8 grids do not carry the whole signal, and a +residual RMS of exactly zero is the signature to check for before reading any +attack result as a null. + +This batch was a single verification pass over frozen files. It is not an +adaptive label source, and the endpoint's prohibition on repeated +reverse-engineering or evasion queries stands: no candidate was adjusted and +re-submitted, and no budget was escalated against the API. + +A wider batch followed the same rule, sixteen pairs frozen in +`openai-oracle-batch-v2-manifest.json` before the first call. Their perturbation +budgets are not chosen, they fall out of each image's own estimated residual, +and they span `37.9` to `57.5 dB`. The API rate limit stopped the run after +fifteen calls with a one-hour retry, leaving seven complete pairs plus one +source. Every arm returned `detected`, including candidates as aggressive as +`44.2 dB` at residual RMS `1.58`, roughly three times the perturbation that +empties the local proxy. + +The running total across both providers is nine source and candidate pairs and +zero verdict changes. The strongest frozen candidates, near `38 dB`, are still +queued behind the rate limit; completing them finishes one pass rather than +starting a new attempt. + +### 2026-08-15: what the periodic features actually separate + +The detector question needs no oracle at all. Both providers ship images from +the same pipeline with and without a watermark assertion, identical claim +generator and action list except for `watermarked.unbound`, so signed provenance +supplies the control group directly. `probe_render_fingerprint.py` scores five +groups with both shipped features. + +| Group | n | Google feature accepts | OpenAI feature accepts | +| --- | --- | --- | --- | +| google watermarked | 300 | 83.3% | 0.0% | +| google, same pipeline, no assertion | 70 | 58.6% | 0.0% | +| openai watermarked | 300 | 2.3% | 2.3% | +| openai, same pipeline, no assertion | 287 | 0.7% | 4.5% | +| unrelated control | 299 | 7.4% | 1.0% | + +Two results, and they land differently. + +The OpenAI period-8 expert has no detection power on the general corpus. It +accepts `2.3%` of watermarked rows and `4.5%` of same-pipeline rows without an +assertion, so the difference is absent and its sign is inverted +(`p = 0.956` one-sided for the intended direction). The sixteen rows the frozen +proxy "clears" are a selected subset, not evidence of a working expert, and this +branch must not reach a shipped universal detector. + +The Google period-16 feature does separate, but mostly by provider rather than by +watermark. It accepts `83.3%` of watermarked Google rows against `7.4%` of +unrelated controls, yet still accepts `58.6%` of same-pipeline rows carrying no +watermark assertion. The watermarked-versus-no-assertion gap is real +(`p = 1.6e-05`, medians `0.513` against `0.206`), so the feature is not purely a +render fingerprint, but a majority of its accepts on that generator's own +non-asserted output makes it a provider detector with a watermark-correlated +component, not a watermark detector. + +One caveat bounds both readings: a missing C2PA watermark assertion is not proof +that no watermark was embedded. The no-assertion group is a lower bound on the +false-positive rate, not a measured one. + +Neither feature transfers across providers, `0.0%` and `2.3%` in the off-diagonal +cells. That, and not the removal result, is the obstacle to a universal +detector: there is currently no shipped feature that reads a watermark rather +than a generator. + +### 2026-08-15: a patch-local feature that beats the OpenAI branch + +The evaluation set holds the confounds fixed rather than trusting them: one +claim generator per provider, PNG only, and every control matched to positives of +the exact same pixel dimensions, so size, codec and generator cannot carry the +label. Folds are taken over matched strata, content hashes are deduplicated +before assignment, and both the ridge strength and the feature family are chosen +inside the training folds. That yields `64` Google groups over `301` images and +`231` OpenAI groups over `1,081`. + +The features are patch-local and every one is a ratio: normalized radial band +fractions, comb-to-background contrast at both periods, normalized periodic +autocorrelation, and the residual's own small-lag autocovariance grid, in +luminance and two opponent planes. Nothing is folded, so nothing is comb-blind. + +| Provider | Shipped AUC | Patch-local AUC | Delta, 95% CI over groups | +| --- | --- | --- | --- | +| openai | 0.510 | 0.605 | `+0.095` `[+0.045, +0.147]` | +| google | 0.665 | 0.629 | `-0.036` `[-0.122, +0.050]` | + +The OpenAI result is the first local expert on that provider with measurable +detection power, against a shipped branch that sits exactly at chance. Twenty +group-preserving label permutations put the null at `0.492 ± 0.020` with a +maximum of `0.525`, so the new score clears its own null by more than five +standard deviations and by more than the largest permutation. + +The Google branch is not improved and the shipped feature stands. A full-sample +ablation showed the band subset alone reaching `0.676` there, above the shipped +`0.665`, but nested selection did not reproduce it, so that number was selection +noise on the evaluation set and is recorded here only as the trap it is. + +Two limits bound the claim. Sensitivity at ten percent control acceptance is +`0.147` against the shipped `0.143`, so the gain lives in the middle of the ROC +and not at the precision-first operating point a shipped verdict needs. And a +model fitted on one provider scores the other at `0.535` and `0.469`, so this is +a better provider-specific expert, not a universal detector. + +An ablation locates the OpenAI signal in the normalized spectral bands, `0.608` +alone against `0.571` for the autocovariance grid, `0.508` for comb contrast and +`0.460` for the periodic lags. The two periodic families, which are what the +shipped experts measure, contribute nothing on that provider. The across-patch +spread block currently covers only the luminance channel, which is a defect in +the feature vector rather than a finding; it scores at chance either way. + +### 2026-08-15: the Google branch is control-limited, not feature-limited + +A second attempt targeted three named causes rather than the score. The strata +were deepened to twelve positives each, `533` against `63` controls. The fold +assignment was repeated over seven seeds with the out-of-fold ranks averaged, +because a single assignment over 64 groups made the inner selection noisier than +the effect it selects on. And the shipped score was offered to that selection as +one candidate among the feature families, so a result below the baseline means +the selection failed rather than that the baseline is unbeatable. + +It did not improve. Global AUC came to `0.678` against the shipped `0.689`, a +delta of `-0.010` with a group-bootstrapped interval of `[-0.071, +0.046]`. + +Sensitivity at ten percent control acceptance did rise, `0.240` to `0.296`, and +that is the single number a precision-first reading would have quoted. It is an +artifact of one threshold. Partial AUC over the whole region below ten percent +control acceptance moves the other way, `0.164` shipped against `0.092`, so the +model is worse everywhere in the operating region except at the exact point the +first metric sampled. One threshold is not an operating point. + +The binding constraint is the control count. Sixty-three controls put the +bootstrap interval on any delta near `±0.06`, so an improvement of two or three +points cannot be demonstrated here whatever the features do, and further model +search on this set is selection on the evaluation data rather than progress. +Unblocking the Google branch means more images from that generator known to +carry no watermark, not more features. + +That is what a provider verifier would supply, and Google has no open route to +one. Three candidates were checked against their own pages on 2026-08-15; the +fetched evidence is archived under +`.local-eval/synthid/provider-access-2026-08-15/`. + +**Vertex `WatermarkVerificationModel` is not a route.** The class survives in +`vertexai.preview.vision_models` with `verify_image` returning a `decision` +field, but the whole generative-AI preview module is marked deprecated as of +2025-06-24 with removal scheduled 2026-06-24, a date already past, superseded by +the Google Gen AI SDK. The documentation page +`vertex-ai/generative-ai/docs/image/verify-watermark` returns HTTP 404, including +its locale variants and the legacy `cloud.google.com` redirect. Whether the +backing endpoint still serves is unverified, and no successor page was found. +An earlier revision of this document named this the programmatic route; that was +wrong. + +**Google Cloud's AI Content Detection API is real but answers a different +question.** It is documented at +`gemini-enterprise-agent-platform/models/ai-content-detection` (last updated +2026-08-13) as Private Preview behind an allowlist intake form whose own title is +`go/detect-ai-content-allowlist`. The page never uses the words SynthID or +watermark. It describes a classifier that analyzes "pixel-level artifacts, noise +patterns, and spectral anomalies" and states that "Support for C2PA metadata +detection is not included". So it decodes no watermark and no manifest: it is a +probabilistic AI-generated-image classifier. It cannot supply the ground-truth +negatives the Google branch is short of, though it would serve as an independent +comparison baseline for our own detector. + +**The DeepMind SynthID Detector reads the watermark but is closed to us.** It is +a portal, not an API, and its early-tester form ANDs three requirements, the +first being "You are an active journalist or verification professional", with a +news-publisher corporate email and required News Publisher and Publisher URL +fields. The words research, academic and developer do not appear on the form at +all. Research access is excluded by construction rather than by refusal. + +The Gemini app therefore stays the only working route, at roughly two checks per +account per two hours. The Google branch's control shortage has no near-term +provider-labelled fix. + +### 2026-08-15: the two providers are architecturally different (S4) + +Cross-image correlation of the folded residual, unit-normalized, over the matched +sets plus 200 unrelated controls. Three pairings are needed, not one: a high +positive-to-positive correlation alone cannot tell a watermark carrier from a +generator fingerprint, because the controls come off the same generator. + +| Provider | Representation | pos-pos | ctl-ctl | pos-ctl | pos-unrelated | chance | +| --- | --- | --- | --- | --- | --- | --- | +| google | tile16 | `+0.326` | `+0.158` | `+0.224` | `+0.023` | `0.036` | +| google | tile8 | `+0.420` | `+0.223` | `+0.304` | `+0.035` | `0.072` | +| openai | tile16 | `+0.032` | `+0.024` | `+0.027` | `+0.008` | `0.036` | +| openai | tile8 | `+0.035` | `+0.031` | `+0.033` | `+0.009` | `0.072` | + +Google images share a fixed, phase-coherent pattern at both geometries, an order +of magnitude above chance, and unrelated images do not carry it at all. OpenAI +images share nothing: every cell sits at or below the chance floor, which is what +a content-dependent post-hoc neural encoder produces and what the SynthID-Image +paper describes. The comb experts work on one provider and not the other because +the providers are not doing the same thing. + +The Google numbers say something sharper than "positives correlate more". A +single shared pattern present in two classes at different amplitudes gives a +cross-correlation at the geometric mean of the two within-class values, whereas +two distinct components would fall below it. Observed against predicted: +`0.224` against `0.227` for tile16, `0.304` against `0.306` for tile8, ratios of +`0.985` and `0.993`. So it is one pattern, not a watermark carrier layered on a +separate generator fingerprint, and the controls carry that same pattern at +roughly half the amplitude of the positives. + +Three readings survive that, and L1 decides between them: the controls are +watermarked after all and the label is wrong; the mark modulates the amplitude of +a fixed carrier the generator always emits; or the carrier is a generator +artifact whose strength happens to track the assertion. The first would also +explain the Google feature's 58.6% acceptance on controls without any of it being +a false positive. + +The magnitude-only rows are reported in `shared-carrier-report.json` but carry +little: discarding phase leaves the generic image spectrum, and unrelated +photographs already correlate at `+0.348` there, so the floor is too high for the +measure to separate anything. + +### 2026-08-15: the Google tile space is exhausted + +S4's geometry suggests an estimator. If the classes differ by the amplitude of one +shared pattern, a discriminant whitened by the within-class covariance should beat +a plain cosine against a template, which weights all 768 tile dimensions alike. +`detector_tile_lda.py` fits exactly that, in the same space, with the family and +penalty chosen inside training folds and seven repeated fold assignments. + +It does not. Google scores `0.644` against the shipped `0.689`, a delta of +`-0.044` with interval `[-0.090, +0.004]`. + +The failure follows from the finding rather than contradicting it. When two +classes carry ONE pattern at different amplitudes, the optimal statistic IS the +projection onto that pattern, which the cosine already computes; whitening adds +estimation variance over 64 controls and buys no direction that was not there. +Read together with the earlier 124-dimensional attempt, this closes the question +the two of them were confounding: the Google shortfall is not the metric and not +the feature bag. The folded tile simply contains one usable degree of freedom, +and the shipped score already spends it. + +Any Google improvement must therefore come from outside the fold, from the +content the comb filter destroys before the template is applied. That is S2, and +S2 needs the provider oracle. + +One reporting caveat: this run scored both providers against +`folded_template_score`, which for OpenAI is the Google template applied to +foreign images and not that provider's shipped expert. The OpenAI delta printed +in `detector-tile-lda.json` is therefore against the wrong baseline. Against the +right one the tile discriminant reaches `0.565` versus the period-8 expert's +`0.510`, which is still below the `0.605` the patch-local features already reach, +so it is not an improvement on either provider. + +### 2026-08-15: the implementation is sound, the date is innocent, the product is not + +Four register entries closed offline in one pass. + +**M3, the shared implementation, holds.** Every removal and detection result runs +through `folded_template_score`, so a defect there would produce both at once and +look like two findings agreeing. `probe_m3_independent_fold.py` re-derives the +statistic from its definition over 40 real images, folding by explicit modulo +accumulation and blurring with a hand-rolled separable Gaussian rather than the +OpenCV call, so a shared bug has nowhere to hide. Maximum score gap `3.7e-4`, +median `1.3e-4`, against scores spanning `0.01` to `0.85`; the residue is the +shipped path's float32 blur. The implementation is not the explanation. + +**L3, the date confound, is refuted.** Month alone predicts the label at `0.524` +on Google and `0.491` on OpenAI, both chance. Inside a single month the Google +signal is not merely intact but stronger than pooled: `0.723` in 2026-06 over 44 +controls and 336 positives, against `0.689` pooled. The OpenAI shipped expert +stays at chance within every month, as it is everywhere. + +**R2 says the OpenAI result is not a geometry artifact, and adds a caution about +Google.** Restricted to one exact pixel size and pooled over 13 strata weighted by +pairs, the patch-local feature scores `0.577` against the shipped `0.478`, an edge +of `+0.099` that matches the `+0.095` measured pooled. So the improvement survives +the strongest available geometry control. Individual strata swing from `0.381` to +`0.799`, which is what 5 to 55 controls per stratum buys and is why the pooled +figure is the one to quote. Google has only three usable strata and one dominates +them, so its within-size figures, `0.767` patch and `0.763` shipped, are recorded +as underpowered rather than as an improvement. + +**D1 is the finding that matters for the product, and it is bad.** Held at one +percent control acceptance, which with 63 and 229 controls means clearing every +control in the set, the fraction of positives that can be called at all is: +Google `0.045` shipped and `0.000` patch-local; OpenAI `0.025` shipped and +`0.018` patch-local. Every headline AUC in this document lives in the middle of +the ROC. At a precision-first threshold neither provider's detector can call more +than a twentieth of true positives, and on Google the newer feature calls none. + +That reframes the program. The gap between `0.605` AUC and a shippable verdict is +not a matter of a few more points; nothing currently supports a positive claim at +a defensible false-positive rate. Either the operating point moves, which means +accepting a control acceptance far above one percent and saying so in the product +surface, or the representation has to change enough to move the tail rather than +the middle. + +### 2026-08-15: U2, the detectors say "synthetic", not "watermarked" + +Until now every control was a photograph or same-provider output. Neither can +catch the failure that decides whether a positive verdict may be reported as a +provenance claim: firing on an AI-generated image from a generator that embeds no +SynthID. The corpus carries 223 such images with their own signed provenance, so +the label is read from a manifest rather than inferred from a path. + +Acceptance rates, each feature at its own threshold. The patch-local models have +no published threshold, so each is cut where one in ten of its own provider's +legitimate controls is already accepted. + +| Generator | n | Google shipped | OpenAI shipped | patch Google | patch OpenAI | +| --- | --- | --- | --- | --- | --- | +| Adobe Firefly | 86 | 0.34 | 0.06 | 0.51 | 0.24 | +| Microsoft Designer | 20 | 0.20 | 0.00 | 0.40 | 0.25 | +| Microsoft Responsible AI 1.0 | 55 | 0.00 | 0.24 | 0.31 | 0.20 | +| PixelBin.io | 35 | 0.31 | 0.00 | 0.37 | 0.09 | +| fal-ai/seedvr | 10 | 0.00 | 0.00 | 0.80 | 0.00 | + +The Google shipped feature accepts 7.4% of unrelated photographs and 83.3% of +watermarked Google images. It accepts 34% of Adobe Firefly output, which carries +no SynthID. The patch-local models are worse: cut at a level that admits one in +ten legitimate same-provider controls, they admit half of Firefly and four fifths +of fal-ai. + +No threshold repairs this. A detector that fires this often on foreign synthetic +images is measuring generation, not the watermark, and a positive verdict from it +cannot be reported as "SynthID detected" without asserting a provenance the +evidence does not support. That is a correctness bound on the product surface, +not a performance target: it constrains what a positive may be called, whatever +the AUC. + +It also bounds the reading of every earlier number here. The Google feature's +`0.665` to `0.689` AUC against same-provider controls was already suspected of +tracking the generator; these rows show it tracks synthesis more broadly than +that, across vendors with unrelated pipelines. + +### 2026-08-15: a high number that must not be quoted + +Following U2, the features were asked to separate watermarked images from +AI-generated images of other vendors, whose own provenance says they carry no +SynthID. Folds keep each vendor whole, so no fold trains and tests on the same +vendor. + +| Provider | vs foreign generators | vs same-provider controls | +| --- | --- | --- | +| google | `0.917` | `0.636` | +| openai | `0.806` | `0.610` | + +The high column is confounded and is recorded here so that it is not later +mistaken for a result. Separating a Google image from an Adobe Firefly image is +easy because the generators differ, and the same features already score `0.917` +and `0.806` at telling providers apart in every other experiment in this +document. The comparison cannot isolate the watermark, because changing the +control set also changed the generator. + +The design flaw is not fixable by choosing better foreign vendors: any control +drawn from a different generator carries generator identity along with the +absence of the mark. The only control that varies the watermark while holding +the generator fixed is same-generator output without the mark, which is the +scarce set L1 is currently validating and the set M2 proposes to mint. Every +unconfounded discrimination this program has measured sits at `0.61` to `0.69`, +and that band is the honest state of the detector. + +### 2026-08-15: M1, the pipeline is not the problem + +Every negative result here is ambiguous between two causes: the features cannot +see watermarks, or SynthID is harder than the data allows. Embedding a watermark +ourselves separates them. The ground truth is not a provenance assertion that may +be wrong, and each carrier is its own control, so image and mark never straddle a +fold. + +`invisible-watermark`'s `dwtDct` mark was embedded into 220 unrelated carriers at +three amplitudes, the library's own and two attenuations, because a mark loud +enough for a naive detector proves nothing about a signal at SynthID's level. + +| Strength | AUC | TPR at 10% FPR | median residual RMS | +| --- | --- | --- | --- | +| full | `0.990` | `0.995` | `2.28` | +| half | `0.974` | `0.977` | `1.17` | +| quarter | `0.936` | `0.895` | `0.563` | + +The last row is the one that matters. A residual RMS of `0.563` is the amplitude +band this program has been measuring SynthID in all along: the estimated periodic +residuals and every attack candidate sit between `0.45` and `0.6`. At that +amplitude the same feature extractor, the same folds and the same estimator reach +`0.936`, against `0.53` to `0.69` on real provider data. + +So the machinery is sound at the relevant signal level, and the shortfall on real +data is a property of the task rather than of the code. Two candidate causes +survive, and they are not exclusive: SynthID is structured unlike a fixed-carrier +mark, which S4 already showed directly for OpenAI, and the labels are heavily +contaminated, which L1 is showing now. + +The bound on this result: `dwtDct` applies one payload by one algorithm to every +carrier, so all marked images share a systematic perturbation. That is an easier +target than a content-adaptive neural encoder. M1 establishes that the pipeline +finds a fixed-structure watermark at SynthID's amplitude, not that it would find +any watermark. Repeating it with a neural watermarker is the next step, and it +would close the remaining ambiguity. + +### 2026-08-15: what property of a mark actually defeats these features + +M1 left one ambiguity: it found a fixed-structure mark and missed SynthID, and +content-adaptivity was the obvious suspect. Synthetic marks separate the +properties, all scaled to the same residual RMS so amplitude explains nothing. + +| Mark | AUC | median residual RMS | +| --- | --- | --- | +| fixed, one pattern for every carrier | `0.670` | `0.626` | +| keyed, a different pattern per carrier, same band | `0.669` | `0.626` | +| keyed and perceptually masked by local texture | `0.546` | `0.577` | + +Content-adaptivity is not the obstacle. Fixed and keyed are indistinguishable, +`0.670` against `0.669`, because the band features read the shared spectral +signature rather than the pattern. Perceptual masking is the obstacle, and it +costs three quarters of the separation. + +That places every number in this document on one scale. At a residual RMS near +`0.6`: `dwtDct` scores `0.936`, being structured and unmasked; diffuse band noise +scores `0.670`; masked diffuse noise scores `0.546`. Real SynthID scores `0.53` +to `0.69`. The provider mark behaves like a masked, diffuse, mid-band +perturbation, which is what a competent perceptual encoder is built to be, and +our `0.605` is at the ceiling of what global hand-crafted statistics reach +against one, not short of it. + +Undoing the mask does not help. Dividing the residual by the same local-activity +estimate before extraction scores `0.538` against `0.546` ordinary. The reason is +structural rather than a tuning failure: the band features are energy fractions +and therefore scale-invariant, so a mask that is nearly constant across a patch +already cancels. The loss comes from the correlation masking creates, not the +scaling. A mark whose amplitude tracks local texture is strongest exactly where +the image's own residual is strongest, so its local signal-to-noise is flat and +equal to its global value, and no region remains where it stands out. Statistics +of this family live on such regions. + +The consequence for the program is concrete. Beating a perceptually masked mark +requires knowing its pattern, which is a decoder with the key, not a better +description of the residual. Absent the key, a local detector is bounded near +where ours already sits, and the remaining levers are the label quality that L1 +is measuring and a learned decoder trained on genuinely clean pairs. + +A defect found and fixed in passing, recorded because the first version of this +test returned a plausible number: the demasking probe originally rebuilt an RGB +image from the corrected residual with a hand-written inverse of the opponent +transform whose round-trip error was 15 grey levels, against marks of half a +level. It scored `0.529` and looked like a clean negative. The residual is now +handed to the extractor directly, and the probe asserts that a supplied residual +reproduces the extractor's own features exactly before it measures anything. + +### 2026-08-15: the Google label rule is refuted, and the Google feature is blind + +The M2 pilot was meant to mint unwatermarked pairs with Imagen's `addWatermark`. +It failed at the first step and produced something more decisive than it was +designed to. + +Imagen is no longer in the model garden. All of `imagen-4.0-generate-001`, +`imagen-3.0-generate-002` and `imagegeneration@006` return 404, and a listing of +the 127 available publisher models contains no `imagen-*` entry. Image generation +now runs on `gemini-2.5-flash-image` and the `gemini-3.x-*-image` family, and +those reject the parameter outright: `Unknown name "addWatermark" at +'generation_config': Cannot find field`. There is no watermark toggle on any +current Google generation path, so M2 as designed cannot be built. + +The Gemini API documentation states the policy directly, twice, rather than +leaving it to inference: "All generated images include a SynthID watermark." Its +optional-configuration section lists output format, aspect ratio and image size, +and nothing else. Generated images are therefore watermarked by construction. + +Two results follow, and both are read off documentation plus four generated +images rather than off an oracle. + +**The Google feature does not see SynthID on current output.** Four freshly +generated images scored `0.018`, `0.103`, `0.054` and `0.071` on +`folded_template_score`, a median of `0.063`, against a corpus median of `0.51` +for watermarked rows and `0.01` for unrelated photographs. The feature is not +over-firing on unwatermarked images; it fails to fire on watermarked ones. Taken +with the U2 result, it detects a generator era rather than a watermark, and that +is now demonstrated rather than suspected. + +**L1's refutation extends to Google.** Every one of those four images carries a +C2PA manifest from the same claim generator as the corpus positives, "Google C2PA +Core Generator Library", and none of them asserts `watermarked.unbound`; their +action lists are `c2pa.created` and `c2pa.edited` only. Google's own API emits a +watermarked image with no watermark assertion. The rule that a missing assertion +means a missing watermark is therefore refuted for both providers, and the 70 +Google controls are unreliable in exactly the way the 287 OpenAI controls were +measured to be. Every Google AUC in this document rests on that label. + +A method note. The first attempt to settle this empirically was going to upload a +generated image to the Gemini app verifier. The composer renamed the attachment +from `.png` to `.jpg`: the interface re-encodes uploads. A `not_detected` verdict +after re-encoding would have been uninterpretable, and the query was abandoned +rather than spent. Today's earlier oracle runs went through the same interface +and returned `detected`, so the mark survived re-encoding there, but no negative +verdict obtained through that surface should be trusted without controlling for +it. + +### 2026-08-15: what the corpus labels actually mean, and what that invalidates + +Earlier entries in this log, written the same day, describe both providers' +label as the `watermarked.unbound` action. That is wrong for Google and the error +propagated into every Google experiment here. The corpus was counted directly. + +`watermarked.unbound` appears in `0` of `4,780` Google rows and `7,427` of +`7,731` OpenAI rows. It is an OpenAI field. Google's `synthid_from_provenance` +flag is driven instead by `synthid_watermark`, "present according to Google LLC +provenance", which is set from the presence of a Google generation claim. + +The two classes therefore differ by something other than a watermark: + +| | Google positives, 4,710 | Google "controls", 70 | +| --- | --- | --- | +| claim generator | Google C2PA Core Generator Library | Google C2PA Core Generator Library | +| issuer | Google LLC | Google LLC | +| actions | `created, converted, edited, opened` and similar, always with `created` | `converted, edited, opened`, never `created` | +| platform | Google (Gemini / Imagen) | C2PA signer: Google LLC (no known AI generator named) | + +The controls are images Google **edited or converted but did not create**. So +every Google number in this document, the `0.665` to `0.689` shipped AUC, the +patch-local comparisons, the within-month and within-size splits, measured +Google-generated against Google-edited. That is a synthetic-versus-photographic +task, not a watermark task, and it explains U2 without any further hypothesis: +a classifier trained that way fires on Adobe Firefly because Firefly output is +synthetic. + +There is no Google watermark control set in this corpus and there never was. + +Worse for the program, one cannot be collected. Google's documentation states +that all images generated by its image models carry a SynthID watermark, and the +`addWatermark` toggle that could produce an exception exists only on Imagen, +which returns 404 for every documented model id on an accessible project. The +same documentation adds a restriction that rules out the cleanest experiment even +with access: "You can use the `seed` field to get deterministic output only when +this field is set to `false`", so an identical image with and without the mark +cannot be generated at all. + +For OpenAI the label is the right field but the negative class is contaminated, +measured at 20 of the first 28 non-asserted rows carrying a watermark. + +What survives from the day's work is what did not depend on these labels: M1 and +M1b, which used marks embedded here with known ground truth, and M3. The rest +needs redoing on a task that is stated correctly first. + +### 2026-08-15: the periodic lattice is JPEG, not SynthID + +Classifier work was abandoned here for signal characterization. A discriminative +model finds whatever separates its classes, and this program has now measured +that what separates them is the generator and its era, so every trained number +carries that confound inside it. The question was changed to what the watermark +does to pixels. + +Without a clean copy of the same image the perturbation is isolated by averaging: +a content-independent component accumulates over hundreds of images while content +cancels. Three groups make the comparison specific. Photographs carry neither +mark nor synthesis. Foreign-generator images are synthetic and carry no SynthID, +so subtracting them removes what is generic to generation and leaves what is +particular to Google. + +The difference looked, at first, like a discovery. Google minus foreign +concentrates on the period-8 lattice far beyond chance: the lattice holds `0.024%` +of bins and takes `2.7%` of the difference in luminance, `10.8%` in red-green and +`7.8%` in blue-yellow, enrichments of `112x`, `441x` and `318x`. The strongest +individual bins sit at one eighth and three eighths of a cycle per pixel, on the +same lattice. + +Period 8 is also the JPEG block size, and the control settles it. Measuring each +group's own lattice concentration rather than a difference: + +| Group | L | R-G | B-Y | +| --- | --- | --- | --- | +| corpus Google | `17.8` | `117.4` | `70.1` | +| foreign generators | `18.9` | `73.1` | `68.9` | +| photographs | `9.1` | `30.4` | `25.5` | +| fresh Google from the API | `3.4` | `9.5` | `6.2` | + +The fresh images are watermarked by documented policy and have never been through +a lossy codec, and they carry the least lattice structure of any group. If the +lattice were the watermark they would carry the most. Foreign generators carry +nearly as much as Google. The ordering tracks compression history. + +So the periodic lattice this program has been measuring, attacking and building +experts on since the beginning is block-transform structure, not SynthID. That +one fact retro-explains the rest without further hypotheses: why the shipped +detector scores `0.51` on corpus positives and `0.063` on fresh output from the +same provider, why it fires on Adobe Firefly, and why S4 found a shared fixed +pattern among Google images that the controls also carried. + +It also reinterprets the removal programme. Nine oracle pairs changed no verdict +at up to three times the perturbation that empties the local score, and the +reason is now plain: the attacks removed and replaced compression structure. The +watermark was never touched, so there was nothing for a provider decoder to lose. + +### 2026-08-16: the large expert detects fresh out-of-distribution output + +The previous entry concluded that the periodic structure this programme has been +measuring is block-transform artifact rather than SynthID. That conclusion was +drawn from `1024`-square API images and generalized too far. It is corrected +here. + +Reading the earlier log supplied the constraint that prompted the check. The +2026-08-14 entry records `large-v1` validated against seven official Google pixel +verdicts and accepting none of `2,637` feature-unseen controls, and it records a +codec stress test in which JPEG-95 and JPEG-90 each took detection from seven of +seven to zero of seven. A compression artifact would be reinforced by +re-encoding, not destroyed by it. + +`gemini-3.1-flash-image` on the global endpoint accepts `imageConfig.imageSize` +and returns `5632x3072`, `17.3` megapixels, inside the large expert's routed +scope and one of the geometries it was calibrated against. Eleven such images +were generated, never written through a lossy codec, and watermarked by the +documented policy that all generated images carry SynthID. + +The shipped runtime detected eight of eleven. The crossing scores cluster just +above the gate at `1.036` to `1.061`; the three misses returned `0.637`, `0.568` +and `0.184` and were reported `indeterminate` rather than negative. + +That model did not exist when the expert was calibrated, and these files have no +codec history, so this is generalization across both model generation and +pipeline. `large-v1` is a working local SynthID detector for large images at +roughly `73%` sensitivity on out-of-distribution positives. + +The corrected reading of the two experts: + +- the **large** expert measures something real that survives a new model + generation and the absence of compression; +- the **fixed** period-16 expert scores a median of `0.063` on fresh `1024`-square + output against `0.51` on the corpus, and the corpus small-image chroma lattice + is ten times the fresh one at matched size with luminance nearly equal, which + is the signature of 4:2:0 chroma subsampling. Its corpus performance is + therefore still best explained by compression history, but that explanation is + now specific to the small-image expert rather than to the periodic family. + +What remains genuinely open is whether the fixed expert is confounded or merely +out of regime: `1024` squares may be below the size at which this signal is +recoverable at all, which the large expert's scope already hints at. + +### 2026-08-16: a pre-registered test that failed, and why it could not have worked + +The mid-band chroma lead from nine images was carried into a confirmation run on +fresh data the hypothesis had not seen. It is recorded here because a +pre-registered test is worth nothing if only its successes are written down. + +Against 223 foreign-generated controls, band 4 in the opponent channels came back +at `z = -8.33` and `z = -5.69` on 150 fresh OpenAI images: enormous, and pointing +the opposite way to the lead. The exploratory scan behaves the same, with bands +0, 1 and 6 and the lag-one autocorrelation all separating at `|z| > 6`. + +Two reasons this is not evidence about a watermark. The lead came from Google +images and only the OpenAI arm reached usable size, so the populations differ. +More fundamentally, the control differs in generator, so a large `z` here says +`gpt-image` at low quality has different spectral statistics from Firefly and +Microsoft output, which was never in doubt. This is the same confound identified +in the foreign-generator classification test, and averaging does not escape it: +without an unwatermarked control from the same generator, no amount of `n` +converts a generator difference into a watermark difference. + +The Google arm stands at ten `1024`-square images, too few to test, and the same +objection would apply to it. + +The practical consequence points at the day's other result. The signal that is +demonstrably present is the one `large-v1` reads at 17 megapixels, and every +OpenAI image collected here is `1024` square, which on the Google side is the +regime where nothing is recoverable. The next OpenAI attempt should be made at +large geometry rather than with more small images. + +### 2026-08-16: a seven-pixel shift, and what `large-v1` actually reads + +The crop ladder was run to find the size at which the signal stops being +recoverable. It found no such size, and answered a different question instead. + +Cropping preserves pixels and lattice phase where resampling would not, so crops +of the eleven fresh 17-megapixel positives isolate size from every other +variable. Two ladders were taken, one from the origin and one from an offset of +seven pixels, which is deliberately not a multiple of the tile period. Components +are medians over the eleven images, folded whole rather than per window. + +| Crop | Aligned fixed | Aligned R-G | Shifted fixed | Shifted R-G | +| --- | --- | --- | --- | --- | +| 5632x3072, 17.3 MP | `0.338` | `0.962` | - | - | +| 4096x2560, 10.5 MP | `0.342` | `0.962` | `0.044` | `-0.111` | +| 2048x2048, 4.2 MP | `0.335` | `0.963` | `0.044` | `-0.104` | +| 1024x1024, 1.0 MP | `0.366` | `0.959` | `0.039` | `-0.081` | +| 512x512, 0.3 MP | `0.350` | `0.937` | `0.046` | `-0.068` | + +Two facts, and the second settles the programme's central question. + +The structure does not decay with size at all. It is as strong in a +512-square crop as in the full 17 megapixels, so the fixed expert's blindness on +small images was never a size limit. + +And it is phase-locked to the image origin. A seven-pixel shift takes the +red-green agreement from `0.962` to `-0.111` at every size. + +That is disqualifying. The SynthID-Image paper reports `99.97%` TPR in its +"Spatial worst" category, aggressive crop combined with resize, evaluated at +512 squares, and Google's own page describes the mark as "designed to stand up to +modifications like cropping". A structure destroyed by a seven-pixel crop cannot +be a watermark whose published evaluation survives far worse. + +So `large-v1` reads a generation-pipeline artifact: origin-anchored, +size-invariant, chroma-dominant, at the tile period. A tiled upsampling or +decoding grid fixed to the image corner has exactly those properties. Every +observation about it now follows without further hypotheses: seven official +positives that all came from one pipeline family, none of 2,637 photographic +controls which have no origin-locked lattice, eight of eleven fresh +`gemini-3.1-flash-image` outputs from the same family, `0.063` on +`gemini-2.5-flash-image` output from a different one, and destruction by JPEG, +which imposes its own grid and breaks the phase agreement. + +A method note that should outlive this entry: the shift test costs one line and +would have settled this in a day at any point in the last three months. Any +candidate local expert should be required to survive a non-aligned crop before +it is calibrated, let alone shipped, because the mark it claims to detect +provably does. + +Also recorded, since it produced a contradiction worth explaining: the table +above folds each crop whole, while `large_image_components` minimises its +components over phase-aligned 2,048 windows. The two are not the same statistic, +which is why the shipped runtime detected eight of eleven full images that this +table's gate arithmetic rejects. The aligned-versus-shifted contrast does not +depend on that difference. + +### 2026-08-16: specificity measured against what the route can be confused by + +Every control rate this project publishes was measured on photographs: `0/2,637` +for the large branch, `0/5,993` and `0/2,366` for registered-v3, `0/41` on a +post-freeze acquisition. Photographs are the easy negative, because they carry no +origin-anchored lattice at all. Once the route is named a generation-pipeline +detector, the negative that decides its worth is other generators' output. + +The production entry point was run over the 223 corpus images whose C2PA claim +generator is a non-Google vendor, so the cohort's composition is signed rather +than assumed. + +| Claim generator | Detected | n | Rate | +| --- | --- | --- | --- | +| Adobe Firefly | 21 | 87 | `0.241` | +| Microsoft Designer | 3 | 20 | `0.150` | +| PixelBin.io | 5 | 36 | `0.139` | +| Microsoft Responsible AI Provenance | 0 | 14 | `0.000` | +| fal-ai/seedvr | 0 | 10 | `0.000` | +| Microsoft Responsible AI 1.0 | 0 | 56 | `0.000` | +| **total** | **29** | **223** | **`0.130`** | + +The shipped runtime accepts about one in eight images from other generators, with +a one-sided 95% lower bound of `0.095`, and reaches `0.241` on Adobe Firefly. The +highest foreign score was `3.01` against a threshold of `1.0`, so these are not +marginal crossings. Every acceptance came from registered-v3; the large branch +was either unsupported or indeterminate on this cohort. + +That is the number a user meets. A tool that answers "is this AI-generated +content watermarked" will be pointed at AI-generated content, not at landscapes, +and against that population the published photographic control rates do not +describe its behaviour. + +One measurement in the same run is reported but must not be read as a +false-positive rate. A cohort of 300 corpus rows with neither provider nor claim +generator was accepted at `0.090`. Those rows are unlabelled rather than verified +photographs and may contain AI images stripped of metadata, so the number is +uninterpretable in either direction. The COCO and Open Images collections behind +the published figures are not on disk, so the `0/2,637` result could not be +reproduced here and is neither confirmed nor contradicted. + +### 2026-08-16: registered-v3 is phase-locked too, so the whole runtime is + +Every acceptance in the entry above came from registered-v3, the default +ordinary-size route, and the seven-pixel shift test had only ever been run on +`large-v1` and the large branch's components. The same test was therefore run +through the production entry point on the ordinary-size expert: the 29 +foreign-generator detections, plus 12 Google provenance-signed positives chosen +inside the registered geometry, each scored at diagonal crop offsets 0, 2, 4, +and 8 with a 32-pixel margin on every side. + +| Cohort | n | Detected at 0 | Detected at 2 | Detected at 4 | Detected at 8 | +| --- | --- | --- | --- | --- | --- | +| Foreign-generator detections | 28 in cropped geometry | 28 | 0 | 27 | 25 | +| Google provenance positives | 12 (8 detected at 0) | 8 | 0 | 8 | 8 | + +One foreign image left the registered geometry under the crop and could not be +shifted. Every one of the 36 detections that remained in geometry died at a +two-pixel diagonal offset: maximum foreign score `0.779` and maximum Google +score `0.311` against the `1.0` threshold, with medians of `0.0`. Four of the +twelve Google positives were indeterminate at every offset, a sensitivity miss +rather than a phase effect. + +The recovery at offsets four and eight is itself the signature. Forcing the +period-16 fold on one Adobe Firefly detection gives `0.808` correlation at +offset zero, `-0.008` at offset one, about `0.14` at offset two, and back above +`0.77` at offset four: the matched structure repeats every four diagonal pixels +and is destroyed by anything in between. Period selection breaks as well; the +canonical winner jumps from `16.000` at offset zero to `9.590` at offset two and +the base statistic falls from `2.47` to `-0.58`. + +A mark designed to survive cropping would score the same under a two-pixel +crop; the published SynthID evaluation retains 99.97% TPR under aggressive crop +and resize. Registered-v3 therefore reads the same origin-anchored, +crop-destroyed lattice established for `large-v1` in the seven-pixel entry, and +reads it in both directions: on the foreign generators it fires because those +pipelines also leave a sixteen-pixel origin-anchored structure that partially +matches the frozen Google template, and on the Google positives it reads the +same class of structure rather than a robust mark. + +The S1 conclusion now covers every shipped periodic expert, not only the large +branch: the entire local pixel route detects generation-pipeline lattice, and +`identify`'s experimental `pipeline_lattice` signal is the honest name for all +of it. Any future expert must pass the two-pixel shift test before calibration, +per the R0 precondition. + +### 2026-08-16: hypothesis register audit + +A full pass over the register against the recorded evidence, closing what the +log already settles: + +- M2 re-verified independently and settled. `imagen-4.0-generate-001` and + `imagen-3.0-generate-002` return 404 in us-central1 and us-east1 on both GCP + projects (`gen-lang-client-0926942364` and `raiw-cws-publish`); + `gemini-2.5-flash-image` rejects `addWatermark` at request parsing + (`Unknown name "addWatermark" at 'generation_config': Cannot find field`), + and the current Gemini API documentation states for both the Gemini-image + ("Nano Banana") and Imagen paths that all generated images include a SynthID + watermark, with no disable parameter documented. No current Google path can + emit an unwatermarked image. +- S2 moves from `open` to `partial`: the equal-RMS broadband arm of the + account-isolated Google oracle group remained `detected`, first direct + evidence against the between-lines scope at the tested perturbation class. +- S3 gains the Google nested test, which did not improve on the shipped score + (`0.678` vs `0.689`), so the Google branch stays control-limited. +- L3 is settled by a direct date-only classifier (AUC 0.524/0.491). +- M3 is settled by the independent fold reimplementation (`3.7e-4`). +- D1 was asked and measured: 0-4.5% of positives decidable at one percent + control acceptance. +- The ranked backlog gains an execution-state table: H1 executed and closed as + a watermark route (it locks onto the S1-refuted pipeline lattice), H2 + rejected, H3 partially executed, H4 rejected as a standalone detector, H5 + advanced as research confirmation only, H6/H7 blocked on the oracle-positive + corpus, H8 not started. + +Nothing in this pass reopens a closed hypothesis, and no status moved on an +argument; every change cites a recorded measurement. + +### 2026-08-17: era, vendor and codec audit of the patch-local bank + +The power-spectrum bank cannot be blinded by translation, so the plain crop +test that killed the phase-locked experts certifies nothing for it. The axes +that can still separate a mark candidate from a pipeline signature are era +(the statistic must fire on corpus-era June-July positives, fresh +`gemini-2.5-flash-image` positives, and fresh `gemini-3.1-flash-image` 4K +positives alike, since the documented policy watermarks all three), vendor +(silence on C2PA-signed Firefly, Microsoft, PixelBin and fal-ai output), and +codec (survival of JPEG-95 applied to both classes, which the published mark +survives). `probe_era_vendor_codec.py` scored all 124 features over that grid: +positives 533 corpus rows plus 10 fresh-2.5 plus 11 fresh-4K, negatives 218 +foreign-generator images, 300 seeded COCO photographs and 100 exact-1024 Open +Images photographs, three conditions each. + +Result: zero features clear a minimum AUC of 0.55 over the 27 cells, zero +clear 0.6, 0.65 or 0.7. The best worst-cell is 0.481. The strongest cells are +the familiar ones, for example feature 47 at 0.9997 on corpus-native against +photographs, and every such feature collapses below chance in some other era +or codec cell. A ridge combination fitted on corpus-era positives transfers to +the fresh eras at AUC 0.388 native and 0.178 after JPEG-95; fitted on fresh +2.5-flash it transfers back to the corpus era at 0.243. The shipped folded +baseline repeats the phase-lock story per era: corpus 0.8845 native against +foreign generators falls to 0.4479 under a random 1-15 px crop, fresh 4K falls +from 0.859 to 0.4508. The one weak exception is the ten fresh 2.5-flash images +at 0.6114 under crop, which is also the set whose negatives are JPEG-stored +while the positives are PNG, so the codec asymmetry is untested there and the +sample is ten images. + +What this settles: every separating statistic in the hand-crafted bank is +era-, vendor- or codec-bound, for the Google side measured directly and for the +OpenAI side already implied by the reversed pre-registered band test of +2026-08-16. Combined with the M1b masked-mark ceiling this closes the +hand-crafted branch as a SynthID route for both providers. The one untested +local route left is S5, a learned nonlinear model on residual patches, which +now carries the additional requirement demonstrated here: leave-one-era-out +transfer is the acceptance gate, because within-era separation is proven to be +pipeline separation. + +### 2026-08-17: S5 first slice, learned patch detector under the era gate + +`probe_s5_patch_conv.py` trains a 4-layer convolutional model on opponent-plane +Gaussian residual patches, eight unaligned random 128-px patches per image, so +no phase anchor to the image origin can exist by construction. Positives span +the eras available on disk: 535 corpus-era Google rows, 10 fresh +`gemini-2.5-flash-image`, 11 fresh `gemini-3.1-flash-image` 4K, 853 corpus-era +OpenAI rows and 151 fresh `gpt-image-1`. Negatives are 223 C2PA-signed foreign +generator images and 400 photographs (300 seeded COCO, 100 exact-1024 Open +Images). Each provider's model is trained on all eras but one and evaluated on +the held-out era, native, after JPEG-95 on both classes, and after a two-pixel +shift; the shift column equals the native one as designed, confirming the +construction. Three image-level label permutations put the null at 0.552, +0.553, 0.593 for Google and 0.540, 0.566, 0.554 for OpenAI. + +The verdict repeats the U2 result for a learned model. Against photographs the +held-out eras score 0.711, 0.599 and 0.843 for Google and 0.791 and 0.767 for +OpenAI. Against foreign generators the same models score 0.442, 0.153 and +0.486 for Google and 0.455 and 0.153 for OpenAI: at or below the permutation +null, with the fresh eras driven below chance, meaning the model ranks +unwatermarked competitor output above freshly minted watermarked positives. +A Google-versus-photographs model that never saw a foreign generator accepts +94.6 to 100 percent of every foreign vendor at the photograph median, and +scores fal-ai output above Google's own fresh 4K era (AUC 0.946 against it). +One outlier cell, OpenAI fresh after JPEG-95 at 0.942 against photographs, +carries a storage asymmetry: the positives are stored PNG and the photograph +negatives JPEG, and re-encoding both does not equalize that history; it is +recorded as an anomaly, not a signal. + +What this settles: the learned route at this capacity learns "AI render +versus photograph" and nothing vendor-specific. With S6 refuting every +hand-crafted feature across the same grid, R0 refuting the phase-locked +experts, and M1b bounding what per-image statistics can do against a properly +masked mark, three independent directions now agree: a per-image local +SynthID detector is not reachable from these pixels by statistical means at +any scale we can validate. Provider mark detection requires the provider +decoder, which exists as an API for OpenAI and as an access-gated service for +Google. The separable local signal that remains is the AI-render versus +photograph distinction, whose honest product framing is generation detection, +not watermark detection. + +### 2026-08-17: matched-pair manufacturing and the distillation screen + +Two measurements moved the "build our own detector" question: + +The attack ladder pilot (`build_attack_ladder_batch.py` + +`run_attack_ladder_batch.py`, batch `attack-ladder-pilot-2026-08-17`, fifteen +arms, every arm submitted exactly once) took three verifier-confirmed OpenAI +positives through four attack mechanisms: JPEG q40, 0.5 bilinear resize, +sigma-8 noise plus JPEG q85, and 0.35 resize plus JPEG q50. Every single arm +returned `detected`, down to PSNR 20.1 dB and RMS 25.1. The OpenAI mark +outlives distortions that destroy the image as a product, and MarkNull-severity +attacks (which unwatermark Imagen-3 at PSNR 25.4 in their paper) do not flip +this verifier. Matched pairs cannot be manufactured by attacking positives: +the flip point sits beyond usable image quality, so same-content label +contrast is unreachable on this provider by the attack route. All three +re-verified sources stayed `detected`, so the run doubles as a health check. + +The distillation screen (`probe_distillation_screen.py`) took the other +oracle-labeled contrast on disk, the 93 L1 control verdicts (65 detected, 28 +not detected; metadata says "no watermark" for all of them, the oracle +supersedes it), and asked whether any local score reproduces those verdicts. +The shipped folded score reaches AUC 0.613 against a permutation null of +0.485 +/- 0.062, roughly two sigma. The best of 126 candidate scores reaches +0.649 before selection correction. A ridge fitted on the test rows themselves +(an explicit upper bound) reaches 0.693, while a ridge fitted on fresh mints +against photographs and foreign generators transfers at 0.409, below chance: +the mark-specific component does not survive provider-external training. The +confound audit shows the two verdict groups matched on median size (1.573 MP +both) and mixed across months and generators. Reading: a weak mark-specific +trace exists in the natural no-assertion region, with a per-score ceiling +around 0.61-0.65 at n=93. Tripling the labeled rows is the cheapest way to +either confirm that ceiling or climb it, so the L1 runner resumed on its +frozen 315-row plan the same day. + +The distillation harness (`probe_distillation_train.py`, feature cache keyed +by content hash over all 329 factory-reachable rows, temporal folds by +collection month, nested-penalty ridge, max-statistic null) ran its dry pass +at the labels available that evening: 102 hard-contrast rows give temporal-CV +AUC 0.632, the 110-row pooled contrast 0.652, and one feature of 124 exceeds +the max-statistic null by two sigma (0.724 against 0.657 +/- 0.033). The +harness is the acceptance instrument for the growing label set; the decision +it will support at n near 280 is whether a weak-but-honest local component +ships under an `indeterminate`-heavy contract or the ceiling is declared. + +### 2026-08-17 evening: decoder substitution and the state of Google access + +Access re-checks first, both negative. The Vertex publisher model +`imagewatermarkdetector@001` returns "not servable in region us-central1" and +404 "not found or your project does not have access" in us-east4, +europe-west4 and asia-northeast1, on live credentials; nothing has opened +since the 2026-08-15 verification sweep. The hypothesized open release of +official image watermark weights ("synthid-suite") does not exist: GitHub, +the google-deepmind organization and the Hugging Face registries expose only +`synthid-text`. No official image detector or marker weights are public, so +no local distillation of the official encoder is possible either. + +Decoder substitution was then tested as a matched-pair factory, on the +MarkNull mechanism reading that the mark lives in the decoder's output +distribution. `build_vae_pair_batches.py` froze five oracle-verified OpenAI +sources plus their float32 encode/decode round-trips through +`stabilityai/sd-vae-ft-mse` (latent mode, no noise, edge padding to a +multiple of eight), PSNR 22.3-32.0 dB, and `run_vae_pairs_openai.py` +submitted the five treated arms exactly once: every arm returned `detected`. +The OpenAI mark is not decoder-bound. It has now survived, with verdicts +unchanged, JPEG q40, 0.5 resize, sigma-8 noise plus JPEG q85, 0.35 resize +plus JPEG q50 down to PSNR 20.1, and a foreign-VAE reconstruction down to +PSNR 22.3. Every same-content pair-factory route is closed on this provider; +natural verdicts remain the only label source, which is the L1 extension +already running. + +The matching Google batch `vae-pairs-google-2026-08-17` (three fresh +policy-watermarked generations, both arms, six submissions) is frozen for the +Gemini web verifier. MarkNull reports its SD-proxy regeneration flipping +Gemini verdicts at 25.4 dB, so this batch decides whether a plain +reconstruction (no latent optimization) already breaks the Google mark, and +with it whether the local negative factory is viable for Google. It requires +a browser session and is the next oracle action when one is available. + +Resolved 2026-08-18 as `vae-pairs-google-v2-2026-08-18`, submitted manually by +the user through the Gemini web verifier. The first v1 attempt exposed a +known trap live: the source arm answered through C2PA Content Credentials +(metadata), and the vae arm returned visual analysis with no watermark check; +neither is a pixel verdict, and both were discarded. The v2 batch re-encoded +every arm through PIL so all arms carry identical chunk sets with the sources' +decoded pixels asserted identical, and added two attribution controls. All +eight arms returned explicit SynthID pixel verdicts: the three stripped +sources `detected`, the three VAE round-trips `detected` at 26.6-32.9 dB, the +COCO photograph `not detected`, and the C2PA-signed Adobe Firefly image `not +detected`. Two conclusions. First, the Google mark survives foreign-decoder +substitution exactly as the OpenAI mark does, so the plain-reconstruction +negative factory is dead on both providers; only MarkNull-style latent +optimization remains untested, and it destroys product quality (25.4 dB, +SSIM 0.80). Second, the controls give the web pixel verdict its first +attribution validation: it stays silent on another generator's AI image, so +within this session it read the Google mark or Google pipeline, not generic +AI forensics. That makes the earlier comb-vs-broadband and replace-095 +verdicts, all answered on Google-generated pixels, retroactively cleaner as +mark-presence evidence than they were when recorded. + +### 2026-08-18/19: the three-metadata-dataset program and the shared-component attribution + +The negative-factory hypotheses being dead, the working program switched to +estimation on positives, organized as three datasets built only from +unambiguous metadata: G, Google `created` rows (4,710, watermarked by +documented policy); O, OpenAI `watermarked.unbound` rows (7,444, all oracle- +sampled rows detected); N, images that cannot carry SynthID (C2PA-signed +foreign generators plus C2PA-signed smartphone cameras; the 2026-08-18 web +control answered `not_detected` on a Firefly member). The 287 OpenAI +no-assertion rows and 70 Google non-`created` rows stay excluded (L1: 69% of +no-assertion rows are actually watermarked), and the 19,792 rows with unknown +provenance are not used as negatives. + +`probe_shared_component.py` estimates, per class, the mean origin-aligned +opponent-residual pattern over up to six unit-RMS 256-px patches per image, +with split-half convergence bounds and photographs as the estimator null +(0.002 +/- 0.002, i.e. zero). Results: every AI class carries a strong shared +component (Google corpus 0.922 at n=535, OpenAI corpus 0.723, OpenAI fresh +0.978 at n=151, foreign generators 0.597, fresh Google 0.30-0.52 at n=10-11), +photographs carry none. Corpus-era components are compression-dominated (50-98% +of power on the period-8 comb); fresh-era components are not (4-10% comb). + +The attribution of the shared component settles the mark question. In chroma, +Google eras correlate pairwise (R-G 0.29-0.59 full-field, 0.25-0.48 after +removing the JPEG comb) while OpenAI eras correlate at zero and OpenAI versus +Google at zero: the Google component is provider-specific but era-crossing. +But it is not the mark. Per-generator decomposition against the fresh 3.1-flash +mean shows Adobe Firefly at 0.572 and PixelBin at 0.406 full-band coherence +while Microsoft generators sit at 0.03-0.19 and fal-ai at 0.05; band-resolved, +Firefly tracks Google in every band including the high-frequency chroma bands +where the two fresh Google eras correlate (0.87 in the period-4 render band, +0.30-0.63 above 0.30 cyc/px). The 2026-08-18 oracle control verified Firefly +carries no SynthID. Therefore the era-crossing shared chroma component is a +renderer-architecture lineage shared by Google's decoders with Firefly and +PixelBin, absent from Microsoft, fal-ai and OpenAI, and the era genealogy is +visible directly (corpus and 3.1-flash share the period-4 band at 0.92 while +2.5-flash does not, matching the model lineages). No frequency band exists +where Google eras correlate and Firefly does not. A one-pixel diagonal shift +erases the component completely (0.48 to -0.05), the origin lock already +established for the lattice. + +This closes the fixed-signature program: every image-independent signature +measurable from positives is renderer genealogy, and the mark itself presents +no fixed cross-image signature at all, consistent with a content-dependent +masked encoder (the M1b ceiling and the DeepMind patent model) and with S4's +zero carrier finding for OpenAI. The era-mean patterns are saved as amplified +PNGs (`shared-component/pattern-*.png`) so the pipeline signature is now +literally visible. The remaining measurable routes are the OpenAI natural- +verdict distillation (L1 at 227/315) and same-content differential analysis +inside near-duplicate groups, where a `created`+`edited` pair of the same +content is the one remaining place a mark-versus-mark contrast could exist; +the group miner `probe_near_duplicates.py` records mixed-assertion groups +explicitly for that. + +The near-duplicate miner completed over all 12,511 provider rows: 170 groups +at Hamming 12, 21 cross-provider, 5 with mixed assertion, largest group 9. +Every cross-provider pair is a re-render (all 21 differ in raster size; the +closest, 896x1200 versus 895x1200, aligns at only 23.7 dB PSNR), so no +same-raster cross-provider pair exists and the cross-provider groups cannot +serve as same-content mark contrasts; they remain renderer-attribution +controls. The local half of decoder substitution also finished (`decoder- +substitution.json`, 287 photographs and 217 foreign round-trips completing +the earlier sets): zero of 124 paired-delta features clear the pre-registered +0.70 acceptance against both negative families (best 0.644), and the OpenAI +deltas point the wrong way entirely (0.23-0.30). This agrees with the oracle +outcome rather than contradicting it: since the mark survives the round-trip, +no mark-destruction signature exists to find, and the null result is the +expected shape. + +A first product-shaped use of the renderer signature was measured the same +day: correlating an image's origin-aligned chroma residual against the frozen +era-mean signatures (matched filter over the three Google era patterns) gives +AUC 0.914 against foreign generators and 1.000 against OpenAI corpus images, +with 0.68 TPR at 10% FPR against foreign. The highest-scoring foreign images +are exclusively Adobe Firefly and PixelBin, the two generators that share the +Google renderer lineage, so the honest claim is "diffusion-render-family +generation detector" (Google, Firefly, PixelBin), not provider attribution, +and it inherits the one-pixel origin lock. It is a candidate second pixel +signal beside the pipeline lattice, to be frozen under the R0 gate before any +product integration. + +The local half of the same probe (`probe_decoder_substitution.py`) runs the +round-trip over fresh and corpus Google, corpus and fresh OpenAI, foreign +generators and photographs, with a pre-registered acceptance rule: a paired +feature delta separating Google from BOTH photographs and foreign generators +at AUC 0.70 with a permutation null two sigma below. Separation from +photographs alone repeats the render-versus-photo result and is not +evidence. Its outcome is in `decoder-substitution.json`: zero of 124 features +clear 0.70 against both negative families (best `google_min` 0.644). + +### 2026-08-19: aoxo/reverse-synthid scored against the frozen era bank + +The Hugging Face dump `aoxo/reverse-synthid` was downloaded in full to +`.local-eval/synthid/reverse-synthid-hf` (2,197 files, 1,922 images) after +the public `reverse-SynthID` repository had already been audited as a +chance-level detector of our lattice, not of the mark. The question left +open was whether their solid-color and low-content images were usable as +independent copies of the Google-era signatures. `probe_aoxo_reverse_synthid.py` +reuses the shared-component residual, the origin-aligned 256-px unit-RMS +patches, and the frozen three-era matched filter. It does not call a +provider API. + +Their Google-family means are the same renderer genealogy we already have. +Chroma R-G of the family mean versus `google_corpus` is 0.805 on +`gemini-3.1-flash-image-preview` (n=589, split-half 0.958), 0.905 on +`nano-banana-pro-preview` (n=518, split-half 0.938), 0.941 on `gemini_random` +(n=88), and 0.898 on `gemini_black_nb_pro` (n=91, split-half 0.964). The +same means sit at 0.48-0.56 versus `google_fresh31` and near zero versus +both OpenAI eras and photographs. A two-pixel crop on a `gemini_black_nb_pro` +2048-px black kills the match (`0.408` to `-0.063` versus corpus). The +amplified family mean is the period-8 vertical lattice. DALL-E 3 is a clean +negative for that filter: 0/280 images above the frozen foreign 10% FPR cut +(0.122), family-mean chroma 0.105 versus corpus, and the amplified mean is +unstructured noise. + +Two of their folders are a different shared object and must not be mixed +into the Google bank. `gemini_black` and `gemini_white` have split-half +0.980-0.983, so they do share something, but chroma versus `google_corpus` +is 0.046 and `-0.140`. They correlate with each other at 0.527 and with the +Google-lineage aoxo families at 0.14 or below. The production folded +template accepts 0/20 in each folder. + +The production lattice detector follows raster size, not folder label. It +accepts 20/20 `gemini_black_nb_pro` (2048 px, median 0.593) and 20/20 +`gemini_random` (2816x1536, median 0.361), and rejects 20/20 of the leading +1024-px solids in the structured Gemini 3.1 and nano-banana folders. That is +the existing large-native scope, not a new carrier. + +Adding the four matching aoxo means as extra matched-filter templates makes +the generation detector worse on the frozen holdout. Against foreign +generators, AUC falls from 0.878 to 0.857 and TPR at 10% FPR from 0.471 to +0.445 (n=535 vs 223). Against the DALL-E bank the same change drops TPR at +10% FPR from 0.991 to 0.907. Foreign and DALL-E scores rise more than Google +scores do. The frozen three-era bank is therefore left unchanged. +`shared-component/era-means-extended.npz` keeps the aoxo means as labeled +reference copies, including `aoxo_dalle3_solid` as a negative, and is not an +input to the detector. Per-family numbers live in +`aoxo-reverse-synthid/aoxo-reverse-synthid.json`. + +Substitution was checked next, because addition can only raise scores. +Replacing `google_fresh31` with any aoxo Google-family mean does not beat +`google_corpus` alone. On the full in-sample holdout the three-era max is +already worse than the corpus mean by itself (full-field AUC 0.878 vs 0.892 +against foreign). The leaked in-sample number is only slightly optimistic: +ten split-half re-estimates of the corpus mean, scored on the unused half, +give AUC 0.880 +/- 0.003 and TPR at 10% FPR 0.480 +/- 0.017. Chroma R-G +scoring is the one change that moves the needle on that honest split +(0.904 +/- 0.003, TPR 0.597 +/- 0.020). It is a scoring change, not a new +signature, and it is not frozen into the detector here. + +The dump's actual use is as held-out positives the corpus mean never saw. +Against foreign generators, chroma correlation with the frozen corpus mean +separates `gemini_black_nb_pro` at AUC 0.990 / TPR 0.989 (n=91) and +`gemini_random` at 0.935 / 0.750 (n=88). The structured 1024-px solid +folders are the same lineage at much lower per-image SNR: `nano_banana_solid` +0.799 / 0.216 and `gemini31_solid` 0.801 / 0.153. `gemini_black` is chance +(0.594 / 0.000) and `gemini_white` is anti-aligned (0.315). Those two +folders are near-black and near-white Gemini dumps with a private shared +texture (split-half 0.98, pixel std about 3.5), not the period-8 lattice. +Numbers: `aoxo-reverse-synthid/generation-detector-aoxo-holdout.json` and +`generation-detector-aoxo-substitute.json`. + +Deflation asks the remaining averaging question: after that lattice is +projected out, does a second shared component look like a mark. +Projecting a set's own mean out of itself forces split-half near `-1` and +is discarded as a tautology. The independent case is the one that counts. +Removing the frozen corpus mean from the aoxo Google families, or removing +the aoxo `gemini_black_nb_pro` mean from the corpus, leaves a leftover +chroma correlation of 0.74 between `nano_banana_solid` and +`gemini31_solid`. That leftover dies at a one-pixel roll (`0.742` to +`-0.005`) and returns at shifts 4/8/16 (`0.61` / `0.47` / `0.49`). It is +uncorrelated with photographs (`-0.01`), DALL-E solids (`0.11`), and the +OpenAI corpus (`0.09`). So rank-1 removal peels the JPEG-heavy corpus +lattice and uncovers a cleaner origin-locked period of the same renderer +family, which is why extra aoxo templates raised foreign scores instead of +finding a mark. OpenAI split-half is unchanged by Google-lattice removal +(0.723 / 0.978), as required if that component is a different renderer. +`aoxo-deflate.json` and `aoxo-deflate-leftover.json`. + +### 2026-08-19: remaining local hypotheses + +Four locally testable leftovers were closed by +`probe_remaining_hypotheses.py`. None of them is a mark. + +**Rank deflation.** The leftover chroma between held-out `nano_banana_solid` +and `gemini31_solid` after removing the top K corpus chroma PCs stays at +0.57 from K=2 through K=16 and still dies at a one-pixel roll (0.045). It +is not in the corpus JPEG-era subspace. Fitting the PCs on `nano_banana_solid` +itself does remove it from `gemini31_solid` (0.70 at rank 1, 0.18 at rank 5, +0.12 at rank 8), so the leftover is a low-rank 3.1-family lattice the mixed +corpus does not span. Photographs stay at 0.01. Extra ranks do not uncover a +shift-robust residue. + +**Mixed-assertion near-duplicates.** All five groups were opened. Three are +cross-provider re-renders at different rasters and cannot isolate a mark. +The two same-provider OpenAI pairs are same-raster: 1254 px at 29.6 dB +(RMS 8.4) and 1054x1492 at 22.1 dB (RMS 20.0). The pixel difference of +either pair is uncorrelated with the Google lattice, the OpenAI corpus mean, +and the OpenAI fresh mean (best 0.066). Both members of the 29.6 dB pair +match the OpenAI corpus fingerprint equally (0.079 vs 0.081). Assertion +metadata does not flip a shared residual, which matches the L1 finding that +most OpenAI no-assertion rows are still watermarked. + +**OpenAI shared component.** The corpus-era OpenAI mean is origin-locked and +periodic: chroma self-correlation 1.0 at shift 0, `-0.446` at 1 px, then +0.57-0.68 at 2/4/8/16. Cross-era correlation with fresh `gpt-image-1` is +~0 at every shift. The fresh mean itself is origin-locked but not the same +period-8 object (self-shift 1 px `-0.083`, later shifts ~0). Both sit at +zero against photographs. This is an era-specific renderer fingerprint of +the same class as Google's, not a stable mark. + +**aoxo `gemini_black` / `gemini_white`.** Their private shared object is also +origin-locked and periodic (black vs white 0.527 at 0 px, `-0.502` at 1 px, +0.51-0.53 at 2/4/8/16) and is not the Google corpus lattice (0.046 / +`-0.140`) nor `gemini_black_nb_pro` (0.139). It is a third phase-locked +texture, not a crop-robust mark. + +Local averaging, deflation, near-duplicate contrast, and the OpenAI shared +component are therefore exhausted as mark routes. What remains needs a +provider oracle or a product decision: L1 distillation (OpenAI verifier), +H6/H7 (oracle-positive training pairs), freezing chroma scoring under R0 as +a generation detector, and H8 as a multi-image product mode. H3 is not a +new carrier; a GLRT on the same origin-locked lattice would rediscover S1. +Numbers: `aoxo-reverse-synthid/remaining-hypotheses.json`. + +### 2026-08-19: new search class, lattice death + +Hand-crafted templates and cross-image averages are exhausted because they +read origin-locked renderer structure. The mark, by publication and by our +oracle ladder, survives JPEG-95 and a few-pixel crop; the lattice does not. +The next class is therefore: destroy the lattice first, then ask whether any +statistic still separates Google from both Firefly and Microsoft. + +`probe_lattice_death.py` applies JPEG-95 and an 8-24 px crop, then scores the +folded template, chroma correlation with the corpus mean, and the 124-feature +bank. The lattice dies as required: chroma AUC falls to 0.49-0.56 against +every family, and the folded template accept rate on Google falls from 0.88 +to 0.04. The feature bank does not die. Best-of-124 AUC is 0.796 against +Firefly and 0.792 against Microsoft (n=149 vs 78/85). A 200-draw max-stat +null on the Firefly comparison has mean 0.600, 95th percentile 0.633, and +maximum 0.684; the observed 0.802 never occurs. Size and mean luma are not +the confound (AUC 0.51 and 0.35). + +The shared winner is feature 106: median B-Y residual autocovariance at lag +(2, 3). It is a short-range opponent covariance, not a period-8 comb. On a +held-out scoring of that single feature the era gate fails. Corpus Google +median `-0.015`; fresh `gemini-3.1` `-0.011` (same side); fresh +`gemini-2.5-flash` `-0.073` (AUC 0.946 against the corpus, n=10). DALL-E 3 +and fresh OpenAI sit with the other generators (0.815 and 0.796). PixelBin, +the other shared-lineage renderer, is the nearest foreign (0.663). A mark +required by policy on every Gemini era cannot miss 2.5-flash. The leftover +is therefore a crop-and-JPEG-stable renderer/era texture, a second pipeline +signature beside the lattice, not SynthID. + +`lattice-death.json` and `feature-106-holdout.json`. + +A linear follow-up asks whether the rest of the bank can recover the era +that feature 106 missed. Ridge on the 124 JPEG-plus-crop features, trained +on 99 corpus Google versus 164 Firefly+Microsoft rows (lambda 100 chosen +on an inner hold), reaches AUC 0.852 on the held-out 50 corpus Google +images against that foreign pool. It does not transfer: fresh 2.5-flash +0.306, fresh 3.1 0.403, DALL-E 3 0.485, fresh OpenAI 0.418. It does fire +on aoxo 3.1 solids (nano 0.962, gemini31 0.978), so what it learned is +corpus-era plus low-content 3.1 export texture, not a policy-wide mark. +`post-lattice-ridge.json`. + +### 2026-08-19: L1 finished + +The OpenAI control-validation runner completed the 315-row plan. Controls: +203 `detected` / 285 verdicts (contamination 0.712). Health positives: 27/28 +`detected`, one miss, the same corpus-label error recorded earlier. Two +control rows remain `error` from an earlier unreachable-API window and were +not retried, because a request that may have reached the model is not a +second submission. The new key was accepted; a 429 after a burst slept 61 +minutes and then drained the tail. + +On provider-clean labels the frozen patch-local bank still does not separate +(`patch_local_auc` 0.572, shipped OpenAI 0.439, n=283). Distillation on the +full L1 set does move: temporal-CV ridge AUC 0.720 on the hard no-assertion +contrast (311 rows) and 0.728 pooled, with 8 and 6 univariate features +above a 2-sigma max-stat null (best 0.674 vs 0.595 +/- 0.016). In-sample +ridge permutation stays at 0.78, so the in-sample fit is still overfit; the +0.72 is the out-of-fold number. That is above the earlier 0.63 at n~110, and +still not a production expert. `openai-control-validation.json`, +`l1-snapshots.json`, `distillation-train.json`. + +### 2026-08-19/20: OpenAI identity ladder + +The 2026-08-17 attack ladder left crop, rotation, flip and grayscale unasked. +`identity-ladder-2026-08-19` takes the already-verified s1 positive and submits +ten frozen rungs once each. All ten returned `detected`: + +| Rung | Output | PSNR (resized-back) | Oracle | +| --- | --- | --- | --- | +| JPEG q80 | 1024x1536 | 37.3 | detected | +| JPEG q20 | 1024x1536 | 31.4 | detected | +| crop 2 px top-left | 1022x1534 | 26.0 | detected | +| crop 8 px all sides | 1008x1520 | 21.0 | detected | +| center keep 75% | 768x1152 | 11.8 | detected | +| center keep 50% | 512x768 | 10.3 | detected | +| rotate 90 deg | 1536x1024 | 10.9 | detected | +| rotate 5 deg, expand | 1154x1620 | 13.6 | detected | +| horizontal flip | 1024x1536 | 10.7 | detected | +| grayscale | 1024x1536 | 26.1 | detected | + +Together with the earlier 3x4 ladder (JPEG q40, 0.5x, noise+JPEG q85, +0.35x+JPEG q50, all detected, n=3) the official OpenAI mark, on this seed, +survives every identity-class transform that kills the pipeline lattice. +The two-pixel crop that zeroes the origin-locked chroma template is not a +mark failure. A half-image center crop is not a mark failure. Grayscale is +not a mark failure, so the decoder is not chroma-only. 90-degree rotation +and a flip are not mark failures, so the decoder is not phase-locked to the +original raster axes. + +This is n=1 for the new rungs. It is enough to reject several detector +hypotheses and to constrain the embedding: a spatially redundant residual, +readable from luminance, trained with geometric and JPEG augmentations, not +a border, not a fixed tile, not the period-8 lattice. Local detection then +has to be a crop-equivariant learned decoder, which is the M1b ceiling, not +a better comb statistic. `identity-ladder-2026-08-19/manifest.json`. + +### 2026-08-20: how the OpenAI mark is applied + +The survival envelope plus the paper/patent plus the VAE pairs constrain the +embedding method. It is not a new measurement; it is the intersection of the +ones already taken. + +The encoder is post-hoc and additive in pixel space: after the generator has +emitted an image, a separate network writes `x' = x + g(x)` and a separate +network reads it back. That is the paper's definition and the patent's image +example. Three oracle facts say the same thing in measurements. Metadata +stripping does not change the verdict, so the mark is not a C2PA field. A +VAE decoder substitution (`sd-vae-ft-mse`, 22.3-32.0 dB, 5/5 detected) does +not remove it, so the mark is not a latent of that decoder. Replacing the +generator would not remove the capability, which is why Google and OpenAI can +share a SynthID family name and still fail each other's official decoder. + +Where it is written. Grayscale (PIL `L`, then RGB) stayed `detected`, so the +decoder reads luminance: stacking `L,L,L` zeroes chroma, and a chroma-only +carrier would have gone silent. JPEG q20 stayed `detected`, so the carrier +is not the high-frequency DCT tail that classical DWT-DCT uses. A center +crop keeping half the area stayed `detected`, so the residual is spatially +redundant rather than a border or a single tile. A two-pixel crop stayed +`detected` while our origin-locked lattice dies, so the written residual is +not that lattice. 90-degree rotation and a horizontal flip stayed `detected`, +so `g` is not a phase-locked overlay on the original raster axes. + +How `g` is shaped. M1b at matched RMS put a fixed mid-band mark at AUC 0.670, +a keyed mid-band mark at 0.669, and a keyed perceptually masked mark at 0.546. +Masking, not keying, is what hides it from hand statistics: amplitude tracks +local texture, local SNR is flat, and there is no quiet region to average. +The patent's U-Net with optional key injection is the architecture that +produces exactly that residual. Content dependence (`g(x)` rather than a +fixed `g`) is why averaging thousands of OpenAI images never yielded a mark +template, only an era-specific renderer fingerprint. + +The working model of application is therefore: a provider-specific neural +encoder, run after generation, adds a perceptually masked luminance residual +in the mid-band, redundant across the frame, trained to survive JPEG, crop, +resize, flip and small/90-degree rotation. The matching decoder holds the +provider key as weights. Nothing in the local pixel statistics we can write +by hand is that decoder. + +### 2026-08-20: census of embedding methods + +Every plausible way to write a SynthID-class mark, sorted by what the +measurements already forbid. Identity-ladder rungs (crop, rotation, flip, +grayscale, JPEG q20) are n=1 on OpenAI s1. Attack-ladder and VAE pairs are +n=3 and n=5. Google VAE is n=3. A method that survives those tests is not +proved; a method that dies on them is closed. + +**Ruled out** + +| Method | Why it is closed | +| --- | --- | +| C2PA, EXIF, filename, chunks, or any sidecar | Official OpenAI path strips AI metadata, proves pixels unchanged, still `detected` | +| LSB / bit-plane steganography | JPEG q20 and `sd-vae-ft-mse` both destroy LSBs; both left the mark `detected` | +| Classical DWT-DCT / `imwatermark` bins | That family dies near JPEG q90; SynthID stayed `detected` at q20 and at 0.35x+q50 | +| Chroma-only residual | Grayscale zeroes R-G and B-Y; oracle still `detected` | +| High-frequency-only tail | JPEG q20 and VAE keep mid-band and discard the tail; mark remained | +| Border, corner, or single-tile stamp | Center keep-50% still `detected` | +| Origin-locked periodic lattice (our S4/S1 object) | 2 px crop kills that lattice and not the oracle; Firefly shares the lattice and has no SynthID | +| Axis-locked overlay on the original raster | 90 deg rotation and horizontal flip still `detected` | +| Mark stored only in one generator's VAE latent | Decoder substitution through `sd-vae-ft-mse` left 5/5 OpenAI and 3/3 Google `detected` | +| In-model watermark baked into diffusion weights | Paper: post-hoc `f: X -> X` on an already generated image; swapping the generator does not remove the capability | +| One shared decoder for Google and OpenAI | OpenAI verifier returned `not_detected` on a Google positive and `detected` on OpenAI positives | +| Fixed additive template shared across images | OpenAI corpus vs fresh split-half cross-era ~0; averaging recovered renderer genealogy, not a mark | +| Keying alone as the reason hand features fail | M1b: fixed vs keyed mid-band 0.670 vs 0.669; masked keyed 0.546 | + +**Still possible, not forced** + +| Method | What would still fit | +| --- | --- | +| Additive `x' = x + g(x)` vs a learned invertible map | Additive is the patent example; a small invertible network is not ruled out if it is JPEG-and-crop stable | +| Luma-only vs luma-plus-chroma | Luma is required; extra chroma is allowed because grayscale kept the luma copy | +| Mid-band spatial residual vs mid-band learned latent of the watermarker itself | VAE test only kills *that* VAE; the encoder may still think in its own latent | +| U-Net vs other conv/attention generators | Patent describes U-Net; any local-and-global residual net that masks to texture would survive the same tests | +| Resize to a trained size, mark, resize back | Patent alternative; compatible with scale survival; decoder would canonicalize or be scale-trained | +| Ensemble of encoder/decoder pairs per provider or epoch | Patent continuation; explains Google vs OpenAI silence and era-specific keys without changing the method class | +| Per-image payload / secret injected into `g` | Presence detection does not need payload recovery; optional in the patent | +| Training augmentations beyond the ones we hit | Elastic warp, extreme downscale, JPEG q5, 10% area crop are untested; they may be outside the trained set | +| Slightly different Google vs OpenAI implementations of the same class | Same method, different weights, possibly different trained-size or mask strength | + +**Forced by the intersection** + +| Property | Forced by | +| --- | --- | +| Written in decoded pixels, after generation | Metadata strip; paper definition | +| Readable from luminance | Grayscale `detected` | +| Spatially redundant across the frame | Keep-50% `detected` | +| Not phase-locked to the source origin or axes | 2 px crop, rot90, flip `detected` | +| Mid-band, JPEG-stable | q20 and 0.35x+q50 `detected` | +| Provider-specific decoder weights | Cross-provider `not_detected` | +| Content-dependent residual, perceptually masked | No shared template under averaging; M1b mask gap | + +The method class is therefore one: a provider-keyed neural residual in luma +mid-band, post-hoc, redundant, geometry-trained. Variants inside that class +(U-Net vs other nets, optional payload, resize-to-canonical, ensemble of +pairs) remain open and do not change the detector architecture. Everything +outside that class is closed. + +### 2026-08-20: OpenAI kill ladder, first not_detected + +Identity and attack ladders never silenced the official decoder. The kill +ladder on the same s1 seed asks which frozen destruction first does. + +| Rung | Output | PSNR | Oracle | +| --- | --- | --- | --- | +| JPEG q10 | 1024x1536 | 29.1 | detected | +| JPEG q5 | 1024x1536 | 26.3 | detected | +| center keep 25% | 256x384 | 9.9 | not_detected | +| center keep 10% | 102x153 | 10.5 | not_detected | +| Gaussian blur sigma 3 | 1024x1536 | 26.9 | detected | +| Gaussian blur sigma 8 | 1024x1536 | 23.3 | not_detected | +| scale 0.20 | 204x307 | 27.0 | detected | +| median 7 | 1024x1536 | 28.6 | detected | +| elastic alpha 12 | 1024x1536 | 40.7 | detected | +| posterize 4 bits | 1024x1536 | 29.5 | detected | + +The flip is not pixel count. Scale 0.20 has fewer pixels than keep-25% and +stayed `detected`, because it still holds the whole frame. Keep-50% was +`detected` on the identity ladder; keep-25% is the first crop miss. Blur +sigma 8 is the first full-frame miss; JPEG q5 is not. Mild elastic (40.7 dB) +is not reverse-SynthID's 18-24 dB warp and did not flip. + +So the official decoder needs a large fraction of the original scene and is +more sensitive to spatial low-pass than to DCT quantization. That is not a +remover: blur-8 at 23 dB and a quarter-frame crop destroy the picture. It is +the first oracle-negative envelope on OpenAI. n=1 seed. +`kill-ladder-2026-08-20/manifest.json`. + +The in-between batch `flip-ladder-2026-08-20` on the same seed: + +| Rung | Output | PSNR | Oracle | +| --- | --- | --- | --- | +| blur sigma 5 | 1024x1536 | 25.1 | detected | +| blur sigma 6 | 1024x1536 | 24.4 | detected | +| blur sigma 7 | 1024x1536 | 23.8 | not_detected | +| center keep 40% linear | 409x614 | 10.5 | not_detected | +| center keep 33% linear | 337x506 | 10.1 | not_detected | +| elastic ~24.6 dB | 1024x1536 | 24.6 | detected | +| elastic ~20.8 dB | 1024x1536 | 20.8 | not_detected | + +Flip points, still n=1: Gaussian blur between 6 and 7 (24.4 dB vs 23.8 dB); +crop between keep-50% linear (detected on the identity ladder, 512x768) and +keep-40% (409x614); elastic between 24.6 dB and 20.8 dB. JPEG q5 remains +`detected`. No rung that silences the decoder is a usable picture. `flip-ladder-2026-08-20/manifest.json`. + +### 2026-08-20: OpenAI add ladder + +The crop miss could have been "too few marked pixels in the upload" or +"too little of the original scene". Adding unmarked pixels distinguishes +them. Same s1 seed, donor COCO `000000000139`. + +| Rung | Upload | Oracle | +| --- | --- | --- | +| pad white, original 70% linear | 1463x2194 | detected | +| pad white, original 40% linear | 2560x3840 | detected | +| inset on photo, 70% of 1600 canvas | 1600x1600 | not_detected | +| inset on photo, 40% of 1600 canvas | 1600x1600 | not_detected | +| blend 50% with photo | 1024x1536 | detected | +| blend 25% marked / 75% photo | 1024x1536 | detected | +| horizontal concat with photo | 2048x1536 | not_detected | +| Gaussian noise sigma 16 | 1024x1536 | detected | + +White padding to 40% linear stays `detected`, while cropping the same seed +to 40% linear was `not_detected`. The decoder is not counting marked pixels +in the upload; it is reading coverage of the original scene. Uniform pad is +ignored. A second unmarked photo beside the intact marked half (`hstack`) +silences it, as does pasting a resized copy onto a photo (`inset`, with an +aspect-ratio confound). Mixing unmarked pixels into every location at 75% +still `detected`, so amplitude 0.25 of the residual is enough when it is +present everywhere. n=1. `add-ladder-2026-08-20/manifest.json`. + +All OpenAI oracle ladder numbers are copied into tracked +[`docs/synthid-removal-research.md`](synthid-removal-research.md) so they survive +the gitignored eval directory. + +Replication on s2 and s3 of the s1 flip points is complete. Blur sigma 6 vs 7 +and crop keep 0.50 vs 0.40 hold on 3/3 seeds. White pad 40% stays `detected` +and hstack stays `not_detected` on 3/3. Blend 25% stays `detected` on 3/3. +Native-aspect inset onto a 2x photo canvas is `not_detected` on 3/3 (s1 added). +Elastic at ~21 dB does not generalize: s3 remains `detected` at 22.0 dB while +s1 and s2 miss. The durable crop and blur flips are therefore the ones to +keep; elastic PSNR is not a stable kill threshold. + +Surround ladder on s1: photo fill at the white-pad geometries is +`not_detected` at both 70% and 40% linear, so extra canvas only kills when +the pad is a photograph. A side strip of unmarked photo stays `detected` at +10% and 25% of width and becomes `not_detected` at 35% and 40%. The decoder +treats a large unmarked scene in the same upload as a whole-frame negative, +while uniform white is ignored. Numbers in +[`docs/synthid-removal-research.md`](synthid-removal-research.md). + +Photo-pad vs white-pad holds on 3/3. The hstack unmarked 25% vs 35% split +holds on 3/3. A photo frame around s1 stays `detected` at 5/8/10/12% each +side and becomes `not_detected` at 15%. Frame 15% holds on s2 and s3. +Frame 12% does not: s2 is already `not_detected`. + +### 2026-08-20: close the local OpenAI mark hunt + +Decision: stop searching for a local OpenAI SynthID detector or a +quality-preserving pixel remover on this evidence. The official decoder's +flip points are now replicated (blur 6 vs 7, crop keep 0.50 vs 0.40, photo +surround, hstack 25 vs 35, JPEG q5 still detected, all 3/3 except elastic +PSNR and frame 12%). Every `not_detected` that is not a collage is below +usable quality. Distillation on L1 labels remains 0.72 temporal-CV against a +0.78 in-sample permutation, so it is not an expert. + +What the product already ships is the honest remainder: signed provenance for +the watermark, and `detect-synthid` / `identify`'s `pipeline_lattice` for the +origin-locked generation lattice, labeled as not SynthID. The unshipped +chroma matched-filter (AUC about 0.90 vs foreign on the frozen holdout) is a +second generation-family candidate and may only enter runtime under the R0 +gate, never as a watermark claim. + +Oracle numbers: [`docs/synthid-detector-research.md`](synthid-detector-research.md) +and [`docs/synthid-removal-research.md`](synthid-removal-research.md). + +Open questions that are not a repeat of closed hunts, now that the official +OpenAI decoder can be queried: + +1. Self-concat. `hstack` of marked|unmarked is `not_detected` on 3/3. + `hstack` of marked|marked (same seed, two copies) is untested. If that + stays `detected`, the kill is a second scene, not extra canvas. If it + misses, any two-panel layout drops the score. +2. Surrogate. The ladders plus L1 are a small labeled set of (image, + transform, official verdict). A student that copies those verdicts would + let attack search run offline. It is not a mark decoder until it beats + the 0.72 L1 temporal-CV and the 0.78 in-sample permutation on held-out + seeds. +3. Luma-only blur between sigma 6 and 7 at PSNR above 30 dB. Closed + 2026-08-22 as a quiet-remover class together with Bayer, VNG, barrel, and + scanline: none flipped the OpenAI oracle below scramble cost. The only + full-frame residual kill remains ~24 dB. +4. The chroma matched-filter remains a generation-family candidate under R0, + never a watermark claim. + +Item 1 is the cheapest untested discriminator. Item 2 is the only path to a +local stand-in for the official decoder. Item 3 is the last quality-removal +check worth an oracle burst. + +### 2026-08-20: cardinal directions, including the text analog + +SynthID-Text is not a recipe for the image encoder. Text is biased at +sampling time among near-synonyms; the published image system is a post-hoc +map `f: X -> X`. The useful borrow is the *detector* shape: a keyed weak +statistic pooled over many tokens, so you need a long enough span and +unmarked tokens vote against. That is compatible with what the OpenAI image +oracle already did (keep-0.40 misses, whole-frame 0.20x hits, white pad +ignored, photo surround votes no). It does not revive averaging or a fixed +tile. + +Directions that are actually untested, not a new comb: + +- Token analog. Submit the four quadrants of a known-positive separately. + 4/4 `detected` means a locally redundant residual. 0/4 `detected` while + the full frame hits means a global pool. Mixed means content-dependent + support. Cheap, four uploads, not a transform we have run. +- Layout scramble. Cut the same positive into a grid of tiles, permute, + reassemble, upload. Local-and-pool marks can survive; layout-locked or + object-aligned `g(x)` should drop. Orthogonal to blur and crop. +- marked|marked concat. Distinguishes "second scene" from "any two-panel + layout". +- Self-keyed residual. Text hashes the prefix; an image analog hashes a + thumbnail and chips the residual. Averaging across images then must fail, + which it did. The untested check is whether an image's high-pass residual + is a stable function of its own low-pass thumbnail across seeds. A yes + would be a local handle without the provider key. A no leaves only the + official decoder. +- Surrogate of the oracle, not of the mark. Train on the labeled ladder plus + L1. Risk: it learns collage/blur/crop, not SynthID. Gate it on held-out + seeds and on JPEG q5 still calling `detected`. + +Tree-Ring-style latent Fourier marks are already a poor fit: VAE +substitution left the OpenAI mark up. Do not reopen band templates, aoxo, or +elastic PSNR as a threshold. + +Token-ladder on s1, 2026-08-20: all four quadrants `detected` at 512x768. +The mark is not a single global code that needs the whole frame; a corner +half is enough, matching keep-0.50. `hstack` and `vstack` of two copies of +the same marked image are both `not_detected`. Photo surround was therefore +misread: the decoder also fails when both halves are marked. A two-panel +layout is the kill, not unmarked pixels. 4x4 scramble stays `detected`; +8x8 scramble misses, so the local support is larger than ~128 px tiles and +can live in ~256 px tiles. Together with scale-0.20 still detecting the +full scene, the official decoder looks like a pool over photograph-shaped +regions of sufficient span, plus a gate that rejects obvious collages. + +An independent Claude Code pass (2026-08-20, tools disabled) argued the +collage gate may be an artifact of **canonical-canvas resampling**: the +decoder resizes the whole upload, so a 0.20x full scene is restored, while +a photo-pad or two-panel layout leaves the mark at half canonical scale. +That one mechanism would reclassify most collage `not_detected` as +preprocessing, leaving blur-7 as the only confirmed residual kill. Cheapest +falsifier: submit 0.20x as-is (known detected) versus the same 0.20x pixels +centered on a white canvas of the original size. Identical marked pixels, +only canvas scale changes. If the padded copy misses, scale-on-canvas is +real. If it still detects, the collage/photo-content gate stands. +A second independent pass the same day used Claude Code Opus with full tools +and Codex `gpt-5.6-sol` at high reasoning. Full texts: +`.local-eval/synthid/prc-oklab-attack-2026-08-15/agent-claude-plan.md` and +`agent-codex-plan.md`. + +Merged ordered plan from those two independent reviews, OpenAI first: + +0. Repair L1 evaluation (0 calls). Codex: the 0.72 is leave-one-date-out on + ~4-image folds, includes health positives, and the 0.78 permutation is + in-sample. Forward-temporal control-only reanalysis is about 0.64-0.65. + Use that as the baseline, not 0.72 vs 0.78. +1. The 0.20x-on-native-white discriminator was run anyway on s1. The 0.20x + file alone was `detected`; the same pixels on a 1024x1536 white canvas are + `not_detected`. Stretch 2x width is `detected`. Centre-crop of marked|marked + back to native size is `detected`. Pure aspect and pure duplication at + native size are not the hstack kill. Scale of the marked region on the + uploaded canvas is real. It does not replace pad_photo vs pad_white, which + still shows unmarked photographic content as a second mechanism. +2. L1 repair: control-only forward-temporal nested ridge is 0.649 / 0.641, + not 0.72. June-15 cut beats a 40-run nested permutation at p=0; June-30 + cut p=0.075. +3. E3 leave-one-encoder-out on 80 COCO images: ridge and a 4-layer conv both + fail to transfer to TrustMark (AUC 0.51 / 0.51). Ridge transfers only to + dwtDct at 0.675. After VAE laundry of both classes: all three held-out + families are chance or worse (0.41 / 0.49 / 0.47). The 0.65 gate fails. + A local blind detector of this mark class is not learnable here at this + capacity and data. + +Independent debate 2026-08-20 (Opus full tools vs Codex gpt-5.6-sol high): +Codex votes CLOSED for the whole local-blind program. Opus splits: blind +transfer CLOSED, Google CLOSED, OpenAI OPEN by one experiment (E4: graded +phase rotation in 16-32 px, stay open only if some theta < pi kills the +oracle while a keyless paired AUC on that perturbation stays below 0.75). +Both texts: `agent-debate-claude.md`, `agent-debate-codex.md` under the eval +directory. Third-voice merge: honor E3 as closing surrogate-encoder transfer; +do not reopen templates. The only remaining OpenAI experiment that is not +shopping is a *short* E4: three seeds times theta in {pi/4, pi/8} in the +16-32 annulus, with a matched 64-128 control at the same theta. If nothing +below full scramble flips 2/3 seeds, close. If it flips, the pair layer is +real and D4 may be attempted once. That is 12 calls, not 57. + +Unbounded open pass 2026-08-20 (Opus and Codex gpt-5.6-sol high, full tools, +local measurements, no oracle). Texts: `agent-open-claude.md`, +`agent-open-codex.md`. + +Opus measured TrustMark mark/content power in the carrier octave at -33.9 dB +and showed a keyless energy detector's AUC is theoretically ~0.51 at s1 size +and ~0.54 at 4K; campaign keyless numbers (0.53-0.57) sit on that ceiling. +E4 pair-by-band-destruction is then dominated by the attack (+24 dB over the +mark at the gentlest killing theta). Micro-warp desync failed. An external +HF "gpt-image-synthid-detector" scores 0.50 vs COCO (positive on 80% of +photos) but 0.69 on L1 official labels; the 0.70 plateau is real and unnamed. + +Codex measured that an 11-number geometry vector (size, aspect, modulo 64) +gets forward-temporal AUC 0.79-0.80 on the same 283 L1 controls where pixel +ridge is 0.64. Raster geometry currently beats every watermark representation +as a predictor of the official verdict. TrustMark-P analog: survives 0.20x +12/12, dies on crop and scramble; its decoder center-crops to 224, so lethal +octaves can be decoder coordinates. + +Merged next action if the local hunt continues at all: zero-API, +geometry-conditioned L1 (Codex) plus naming the 0.70 (Opus). Do not run E4. +Do not train a CNN on raw L1 bits. + +Ran 2026-08-20. ChatGPT rows are all `not_detected` (21/21). After dropping +them, geometry still beats pixels (0.72/0.78 vs 0.63/0.67). Largest mixed +API size 1254x1254 is chance (p=0.45). Self-keyed 16-32 map is 0.41 vs L1-neg +and 0.50 vs COCO. L1 official bits remain unidentifiable as a mark task on +this corpus. Do not train a CNN on them. + +Detector debate 2026-08-21 after `gpt-image-2` pairs (Opus, Codex gpt-5.6-sol +high, this session). Notes: `agent-detector-claude.md`, `agent-detector-codex.md`. +Opus: photo matched-filter ceiling from the flat 16-32 amplitude (0.11 RMS) vs +in-band photo content (12.2 RMS) is d'~0.93 / AUC~0.75 even with a perfect +template; P5 needs d'>=4.37, a 13.4 dB deficit *unless* the photo-domain mark is +louder than the flat measurement. Next: 16-32 jamming titration on the fish edit +and the gray-128 flat (~14 oracle calls) to measure that amplitude. Codex: do not +reopen keyless search; train a frozen wavelet CNN only on procedural +flat-to-texture pairs, lock >=128 photos, require paired AUC LCB>0.65 plus a +five-file causal panel. Merge: titration first. A student cannot beat a matched +filter if the amplitude is as quiet as the flats; minting 128 photos before that +number is architecture shopping. Retract L1 matched-filter AUC 0.701 as size +confound. + +Titration ran 2026-08-21. Gray arm is not a coding-gain calibrator (any extra +energy, including 4-8 px, is `not_detected`). Fish 16-32 additive jam stays +`detected` through a=16 / 21.9 dB and flips at a=24 / 18.6 dB; 4-8 px at a=16 +stays `detected`. 4-8 px jam at a=24 / 18.6 dB stays `detected`, so the 16-32 flip is +band-specific. Phase scramble of 16-32 remains the cheaper kill at 24.3 dB. +The close `a*_photo < 0.53` failed. Do not mint a 128-photo student on that +number. Do not treat additive in-band noise as a photo remover. + +### 2026-08-21: source-matched pairs and titration + +Full numeric log: [`docs/synthid-detector-research.md`](synthid-detector-research.md) +and [`docs/synthid-removal-research.md`](synthid-removal-research.md). + +`gpt-image-2` can mint encoder-on/off pairs. Flats are clean (PSNR 38-49 dB, +oracle `not_detected` to `detected`) and share a highpass residual (LOO 0.645) +that does not transfer to photographs. The 16-32 carrier is causal on both +flats (scramble 51 dB `not_detected`) and the fish photo (scramble 24.3 dB +`not_detected`; pyramid wipe 48 dB still `detected`; additive jam flips only +at 18.6 dB, while equal-energy 4-8 px jam stays `detected`). + +That does not open a local photo detector: the flat stamp is not the photo +mark, L1 bits remain a geometry/export task, and the photo jam amplitude never +fell below the preregistered 0.53 RMS close. Product remainder unchanged. + +Future analog track, not a current experiment: a self-owned encoder in the +16-32 px midband (thumbnail-keyed phase, JND mask) as a research stand, never +as a Gemini/OpenAI detector. Future evasion track, only with oracle quota: a +graded phase shift in 16-32 at PSNR > 30 dB on s1-s3. Collage, photo-pad, and +small-on-canvas stay presentation bypasses, not removers. + +### 2026-08-22: Model 1 result, provider CLIP, lattice re-check, Bayer close + +Numeric log: [`docs/ai-generated-image-classifiers.md`](ai-generated-image-classifiers.md), +[`docs/synthid-classifiers.md`](synthid-classifiers.md), and +[`docs/synthid-removal-research.md`](synthid-removal-research.md). + +Model 1 is the AI-versus-camera research result, not a watermark detector. +Finetuned CLIP-L, 224 letterbox, last two vision blocks: Kodak 0/24, locked +Open Images fresh FPR 1.7%, AI-test TPR 93.0% (OpenAI 93.2, Gemini 90.5, +FLUX hold 92.7). False positives on Open Images are graphics, CGI, and +cutouts, not Gemini. It does not enter `identify`. CLIP-H and DINOv2-giant +are not the result. + +Provider CLIP (`openai` / `google` / `other`) and an OpenAI-plus-Gemini +union fail Firefly. The 124-d residual bank remains the Google-lineage +lattice, not an AI-or-not feature set. + +Production `pipeline_lattice` re-check (n=628): Google 56%, Firefly 18%, +PixelBin 14%, OpenAI 1%, photographs 0. A 2 px crop kills the positives. +Name stays generation-pipeline lattice. + +Bayer bilinear, Bayer VNG, upscale-then-Bayer, barrel, and scanline left +OpenAI SynthID `detected` on s1 and s2. 16-32 phase scramble still flipped +both. VNG on s2 is dirtier than scramble and the mark remains. Camera +pipeline and mild geometry are closed as quiet removers. s3 and the fish +photo were not submitted in this pass (OpenAI 429). +4. E2 per-octave phase scramble on s1, replicated p16_32 on s2 and s3: + destroying periods 16-32 px is `not_detected` 3/3; 2-4, 4-8, 8-16, 32-64, + 64-128 stay `detected`. The carrier octave is 16-32 px, not the lattice + band. Keyless energy and RGB phase-agreement in that octave on L1 are + AUC 0.53 and 0.57. +5. Google gets no independent hunt. If (3) transfers, validate on a small + Gemini-app batch against Firefly/PixelBin. If (3) fails, say so. + +Working decoder hypothesis, not proved: a net trained on single whole +photographs, probably after a canonical resize. That one story fits JPEG q5 +(still one photo), scale 0.20 (same scene), white letterbox (still one +photo), keep-0.50 vs 0.40 (enough vs too-tight crop of the scene), +photo-pad and any two-panel concat including marked|marked (not a single +photo), scramble 4x4 (still reads as one image) vs 8x8 (a mosaic). It is a +hypothesis about the *decoder*, not a new encoder. Falsify it with a +concat that still looks like one photograph, or a mosaic that still looks +like one. + +Ontology split, forced by marked|marked missing while every quadrant of the +same seed detects: an official `not_detected` is not one event. Collage, +photo-pad, two-panel self-concat, and 8x8 scramble can be a *presentation +gate* (this upload is not a single photograph). Blur sigma 7, keep-0.40, and +JPEG-surviving full-frame views sit on the single-photo manifold and are the +only current candidates for actually damaging the residual. Crop-keep-0.40 +may be either: too little scene for the residual net, or a zoom the +preprocessor treats as a different image. Do not train a surrogate on mixed +collage misses if the goal is a mark student; those labels are the gate. + +## Production hardening roadmap + +The next milestone is a precision-first production detector system, not one +unqualified binary classifier. "Bulletproof" means that every positive verdict +is tied to a frozen, independently challenged expert; every negative verdict is +limited to a sensitivity-certified scope; and every other case is an explicit +abstention. It does not mean claiming that an undocumented watermark can be +recognized after every possible edit or future encoder change. + +### Current starting point + +| Route | Evidence already obtained | Production gap | +| --- | --- | --- | +| Gemini fixed native, 1-10 MP | Frozen period-16 template; 0/5,000 observed-geometry COCO controls and 0/5,000 generated-geometry controls | Too few matching-oracle positives to estimate recall by geometry, epoch, and product surface | +| Gemini large native, 10-18 MP | 7/7 official pixel positives; 0/2,637 feature-unseen controls; 0/41 post-freeze Open Images controls | Positive sample is too small; JPEG-95 and JPEG-90 each reduce 7/7 to 0/7; the fresh negative acquisition is incomplete | +| Gemini registered cascade, 0.25-10 MP | Registered-v3 plus bounded opponent fallback; the fallback recovered 49/49 lossless 0.5x-0.75x views from seven official positives, rejected 1,790 measured period-8 codec crossings, and retained the earlier 0/1,000 post-freeze control result | Provenance labels are not enough for a provider-wide recall claim; lossy retranscodes are rejected; additional encoder epochs remain abstentions | +| OpenAI official remote | Metadata-free, pixel-identical web smoke: 2/2 OpenAI positives detected, while one Google positive and one COCO control returned not detected; one preselected OpenAI-positive PNG also returned SynthID detected through the credentialed SDK path; mocked transport and parser tests pass | JPEG, WebP, approved negative transport cases, operational limits, privacy behavior, and live error paths are not yet certified | +| OpenAI local | Period-8, DTCWT, codeword, external-decoder, and generic-watermark transfer branches all failed their locked or fresh checks | No local route may ship without authorized pixel-oracle labels or source-matched counterfactual pairs | + +These rows are separate claims. The OpenAI official verifier must never be used +as a fallback label for Gemini, and the local Gemini carrier must not be +reported as OpenAI or even as Google-exclusive until provider exclusivity is +measured against other SynthID deployments. + +### P0: freeze the product contract before collecting more scores + +Introduce one provider-neutral result envelope around the existing backends: + +- `status`: `detected`, `not_detected`, `unsupported`, `indeterminate`, or + `error`; +- `signal_family`, `provider_scope`, `backend`, `detector_version`, and + `calibration_version`; +- decoded geometry, input format, score, threshold, and the exact support reason; +- `pixels_preserved` and `metadata_used_for_verdict` audit fields; +- a calibration-manifest digest and an optional decoded-pixel audit digest that + stays in the local qualification manifest and is never emitted in normal user + output or centralized telemetry; +- explicit caveats for lossy input, spatial resampling, unknown encoder epoch, + and remote upload. + +At the product boundary, a miss from the current positive-only Gemini experts +maps to `indeterminate`, not to a clean-image claim. `not_detected` is reserved +for an official provider verdict or a future local scope whose sensitivity gate +has passed. `unsupported`, remote failure, malformed response, quota exhaustion, +and verifier refusal remain distinct states and never collapse into a negative. + +The expert registry owns support predicates and precedence. Fixed, registered, +large, and future codec experts keep separate identities and calibration +artifacts. A router may select one predeclared expert from input scope, but may +not OR together overlapping scores or relabel the default expert after seeing +its value. Offline commands never invoke a remote verifier implicitly. + +The first P0 runtime slice is now implemented. Local results return +`detected`, `indeterminate`, or `unsupported`, include an explicit support or +miss reason, and expose `signal_family`, `provider_scope`, `backend`, +`metadata_used_for_verdict`, and `pixels_preserved`. The official OpenAI result +uses the same audit fields while retaining the provider's documented +`detected`/`not_detected` outcomes. Calibration-manifest digests and a unified +error envelope remain qualification work rather than being synthesized from +the historical research files. + +### P1: build the immutable qualification corpus + +All qualification media stay outside the public repository. The tracked +artifacts are schemas, hashes, split manifests, aggregate reports, and cleared +synthetic fixtures. + +For Gemini, collect matching-oracle labels before any new threshold fitting: + +- at least 200 decoded-pixel-unique native positives for the fixed route, + balanced across five pixel-count bins, landscape, portrait, and square + geometry, at least three collection windows, and every available Gemini image + surface; +- at least 100 official native positives for the large route, balanced across + 10-12, 12-15, and 15-18 MP, the three aspect families, and at least three + collection windows; +- at least 200 source-disjoint transformed positives before reconsidering the + registered route, with the untouched parent verified first and every + derivative grouped with that parent; +- at least 10,000 freshly frozen decoded-pixel-unique negatives per claimed + local route. Each route needs cameras, scans, screenshots, flat graphics, + synthetic textures, other generators, other watermark families, resampling + aliases, and same-provider oracle-negative examples. No source family may + dominate the aggregate result. + +Complete the pending 3,000-image post-freeze Open Images large challenge rather +than treating the current 41 completed files as a final estimate. Deduplicate +every item against all prior train, development, threshold, and challenge +corpora by decoded hash and perceptual group. + +For OpenAI, prepare a preregistered transport-qualification set with previously +established ordinary-use verdicts across PNG, JPEG, WebP, portrait, landscape, +square, alpha, maximum practical dimensions, and multiple model/date windows. +Run it through the SDK only within the published usage policy and the account's +authority. If the required batch volume or retention terms cannot be approved, +the remote backend remains opt-in preview rather than being called fully +qualified. These cases validate transport and response semantics; they are not +training labels for a local detector. + +Every split is frozen by parent group before scoring. Mutation tests must prove +that swapped providers, changed bytes, duplicate pixels, derivative leakage, +indeterminate-as-negative conversion, and a wrong expert identity invalidate +the run. + +### P2: finish the Gemini local expert bank + +Rebaseline the fixed and large routes only on oracle-confirmed positives and +fresh negatives. Do not tune the existing holdouts again. Each failed gate +creates a new detector version and a new untouched test partition rather than a +patch to the same test set. + +The all-resolution program is a registered bank of measured carrier periods, +not an assumption that arbitrary dimensions imply arbitrary scale. Test these +branches in order: + +1. native period-16 fixed and large experts, with no threshold changes unless + the new locked corpus rejects them; +2. integer-period experts for periods 8, 10, 12, 14, 18, 20, and 24, each with + its own search-corrected null calibration; +3. a fractional-period expert that estimates period on one image region and + scores a disjoint region, preventing the same noise peak from selecting and + proving itself; +4. codec-specific experts trained and evaluated symmetrically on JPEG and WebP + pixels, never by requiring a native file to survive a second lossy encoding; +5. crop and screenshot experts only if phase registration and boundary search + pass a fresh multiple-search challenge. + +Candidate features remain restricted to watermark-mechanism evidence: +registered residual tiles, signed opponent-color phase, cross-region phase +agreement, circular phase statistics, and directional wavelet residuals. +Scene semantics, thumbnails, filenames, container size, metadata, and provider +logos are forbidden. A learned model may be introduced only after a classical +expert passes the same source-matched gate, and it must consume high-pass +residuals rather than RGB content. + +For each new expert, fit on train, choose features on validation, calibrate on a +separate null partition, and evaluate once on locked and later temporal tests. +Report per-parent paired differences and confirm claimed improvements with a +sign test. A larger aggregate AUC does not compensate for a failed hard-negative +stratum. + +### Ranked detector hypothesis backlog + +The evidence gap blocks threshold promotion, not mechanism research. The +following hypotheses are materially different from the already rejected extra +color spaces, raw FFT magnitude, standalone SWT/DTCWT, multifractal summaries, +folded variance, spatial lag products, and provider-versus-control CNNs. They +are ranked by expected information gain and must still follow the P1 split +discipline. + +Execution state as of the 2026-08-16 audit (the entries below keep their +original wording as the research designs; this block records what happened): + +| # | State | Outcome | +| --- | --- | --- | +| H1 | Executed 2026-08-14 | Split-confirm lattice synchronization works: 20/20 transformed positive views at exact expected periods, 0/800 locked negative views. But the lattice it locks onto is the origin-anchored generation-pipeline lattice refuted as the watermark under S1/R0, and the broad non-Google challenge accepted 3/276. Closed as a watermark-detection route; remains the mechanism baseline | +| H2 | Rejected 2026-08-14 | The complex spectral-correlation statistic did not beat its own phase-preserving mutation control; rejected without consuming another locked scale partition | +| H3 | Partially executed | Split-confirm selection and the content-matched same-image null ran inside the H1/H4/H5 probes; the full GLRT with empirical p-values folded into the conformal cascade is still open | +| H4 | Rejected as a standalone detector 2026-08-14 | Content-whitened matched filter added no boundary over the phase-preserving spatial template; survives only as an H1 synchronization refinement, calibrated inside that branch | +| H5 | Advanced as research confirmation | The unknown-payload patch-agreement statistic retained positive margins under JPEG and crop; explicitly not a runtime threshold, and it inherits the S1 scope question because it reads the same periodic structure | +| H6 | Blocked | Requires the matching-oracle positive corpus (P1 prerequisites, P3 boundary for OpenAI); not started | +| H7 | Blocked | Same corpus dependency as H6; not started | +| H8 | Not started | A separate batch product mode, not a substitute for the single-image detector | + +#### H1: recover the complete reciprocal lattice + +The current scale registration estimates mainly one carrier period. A resize, +rotation, shear, screenshot, or mild perspective warp transforms a two- +dimensional reciprocal lattice, not one scalar period. Fit two lattice basis +vectors from harmonic peak pairs with a robust consensus estimator, then +demodulate or resample the residual into canonical carrier coordinates. + +Estimate the lattice on one fixed checkerboard of spatial blocks and score the +held-out blocks. This prevents content peaks used for synchronization from also +proving detection. Compare translation-only, isotropic-scale, affine, and mild +projective models in that order, charging every model and harmonic candidate to +the same maximum-statistic calibration. + +Gate: improve paired sensitivity on resized, rotated, and screenshot positives +without raising the frozen hard-negative upper bound. If affine registration +selects natural resize lattices as often as watermark lattices, reject the +branch rather than adding a semantic veto. + +#### H2: complex cross-spectral cyclostationary coherence + +The rejected OpenAI cyclostationary screens measured folded magnitude, +periodic variance, and spatial lag products. They did not test the complex +spectral correlation density +`F(k + alpha/2) * conjugate(F(k - alpha/2))` across disjoint spatial blocks. +A spread-spectrum periodic modulation can couple neighboring frequencies even +when payload signs cancel the cross-image mean. + +Measure normalized complex coherence only at preregistered cyclic frequencies +and compare it with same-radius off-carrier frequency pairs. Preserve phase and +cross-channel covariance instead of averaging magnitudes. Fit frequencies on +train, choose one statistic on validation, and confirm on locked images and +fresh same-provider controls. + +Gate: the carrier-to-off-carrier contrast must remain positive within images, +across temporal windows, and after symmetric codec transforms. A difference +that appears only between providers is a renderer statistic and is rejected. + +#### H3: select-confirm GLRT with a same-image null + +Large external control sets remain necessary, but each image can also supply a +content-matched null. Estimate period, phase, orientation, and channel direction +on one region set. On disjoint regions, compare the selected carrier against +neighboring off-lattice bins, orthogonal orientations, phase-scrambled +templates, and nonselected lattice hypotheses. Convert the selected-versus-null +rank into an empirical p-value, then calibrate the final maximum over experts +with the existing conformal cascade. + +This is stronger than scoring every candidate on the same pixels: selection +noise cannot confirm itself, and textured images define their own spectral +background. + +Gate: p-values must be approximately super-uniform on every locked negative +stratum. A positive-only score improvement without null calibration does not +advance. + +#### H4: content-whitened multichannel matched filter + +The current template correlations normalize global energy but do not model the +image-specific complex noise covariance around carrier bins. Estimate local +power spectral density and the full RGB or LMS cross-channel covariance from +neighboring noncarrier frequencies, whiten the residual, and then apply the +frozen complex carrier template. Learn at most one generalized-eigenvector +channel projection per expert on training data; Green, Lab, and OKLab are +baselines rather than separately selected hypotheses. + +Gate: whitening must improve within-parent carrier margin in high-texture +positives and reduce the heavy null tail on natural resampling aliases. It must +not depend on scene category or expose RGB thumbnails to a learned model. + +#### H5: repeated unknown-payload consistency + +Cross-image phase lock can fail when every image carries a different payload or +key. The watermark may still repeat one internally consistent phase or sign +codeword across distant patches of the same image. Estimate a short complex +phase/sign vector independently in each nonoverlapping patch, allow one global +cyclic phase and polarity nuisance parameter, and test patch-to-patch agreement. +Do not require the codeword to match another image. + +This generalizes the existing quadrant minimum: it tests a structured vector +and its parity or low-rank consistency rather than one scalar correlation. Use +synthetic random codewords and phase-scrambled equal-power fields to mutation- +test that the statistic recognizes repetition, not merely periodic energy. + +Gate: agreement must transfer to new source images and remain absent from +periodic graphics, demosaicing grids, JPEG blocks, and renderer lattices. If no +same-image consistency exists in low-texture oracle positives, reject the +payload-repetition model. + +#### H6: calibrated encoder-state mixtures + +One provider may deploy several encoder keys, payload families, strengths, or +model epochs. Fit a small mixture only on matching-oracle positives or causal +pairs, freeze every component before evaluation, and expose each component as +a separately identified expert. The union uses search-corrected conformal +p-values, not per-component thresholds chosen from the test set. + +The earlier OpenAI multi-direction codebook does not justify this branch: it +was trained on assertion-labeled exports and failed fresh specificity. Reopen +the hypothesis only with the P1 OpenAI prerequisites. For Gemini, test whether +the known first and orthogonal carrier directions remain separate on official +pixel positives. + +Gate: a mixture must rescue locked positives without increasing the familywise +false-positive bound, and every retained component must recur in a later time +window. Singletons and post-test clusters are discarded. + +#### H7: residual-only equivariant network + +If H1-H5 expose a causal statistic but a hand-built aggregation loses recall, +train a compact network on high-pass residual fields with explicit translation, +rotation, scale, and codec transformations. Give it complex carrier maps or +steerable-wavelet responses, not RGB content. Enforce source-matched pairs, +leave one encoder state and one transform family out, and keep the classical +expert as an independent confirmer. + +Gate: the network must improve the paired locked test and the unseen-transform +test at the same low-FPR boundary. Saliency must remain on residual carrier +structure; a content or border shortcut kills the branch. + +#### H8: optional multi-image evidence + +A user may possess a set of outputs from one generation session. A separate +batch detector can aggregate weak phase or codeword evidence across diverse +images after content-hash grouping. It may detect a shared encoder state that +is too weak in one image, but it must return batch-level confidence and must +not relabel every member as individually detected. + +Gate: define a minimum number of content-diverse parent groups, evaluate whole +sessions as the independent unit, and challenge same-renderer negative batches. +If gains disappear after session and content balancing, the result is provider +attribution rather than watermark detection. + +### Hypothesis execution policy + +Run H1, H3, and H4 first for Gemini because they directly target the measured +resize, texture, and large-image failure modes. Run H2 and H5 as the next +mechanism probes because they can detect payload-varying structure without a +shared cross-image template. H6 and H7 require the new oracle-positive corpus; +for OpenAI they additionally require the authorization or causal-pair boundary +in P3. H8 is a separate product mode, not a substitute for single-image +qualification. + +At most one hypothesis family may consume a locked partition. A failed locked +test closes that partition for model selection and opens a new version only +after a fresh holdout exists. This is the cost of being creative without +turning repeated hypothesis search into an invisible false-positive generator. + +### 2026-08-14: split-confirm lattice synchronization result + +The first H1/H3 prototype is implemented in +[`synthid_affine_lattice_probe.py`](../scripts/synthid_affine_lattice_probe.py). +It samples strong complex harmonics from 16 nonoverlapping checkerboard patches, +selects period candidates on eight patches, and confirms the chosen lattice on +the other eight after correcting every patch for its global origin. Five +separated zero-rotation period candidates then enter an amplitude-aware +reranker. Cyclic phase and period are selected from one patch group and scored +on the other. The final full-image template correlation preserves the observed +phase; a separately reported cyclically registered score is diagnostic only. + +The separation matters. A broad coherence search at 0.25-pixel spacing selected +harmonic aliases such as 10.5 instead of the true 12.8-pixel period. Refining the +grid to 0.1 pixels recovered the exact expected periods in all 20 transformed +positive views: 12.8, 14.4, 17.6, and 19.2 pixels for scale factors 0.8, 0.9, +1.1, and 1.2. A candidate-count mutation from five to one selected the same +periods and produced the same template scores in all 20 views. The grid +resolution, not the amplitude reranker, caused this recovery; amplitude remains +an independent confirmation statistic. Cyclic registration was also rejected +for the ordinary scale route. It increased scores on +resize-interpolation lattices, whereas the phase-preserving correlation kept +those aliases negative. + +One provisional conjunction used the already frozen fixed-template threshold +`0.17357069773071196`, split-confirm lattice coherence of at least `0.52`, and +split-confirm amplitude of at least `0.20`. These latter two constants are +research thresholds, not runtime calibration. Results were: + +| Challenge | Positive result | Negative or cross-provider result | +| --- | --- | --- | +| Native matching-oracle pilot | 5/5 | 0/130 matching-oracle Picsum controls | +| Four-scale development matrix | 20/20 transformed views from 5 parents | 0/200 views from 50 oracle-negative parents | +| Four-scale locked matrix | 20/20 transformed views from the same 5 positive parents | 0/800 views from 200 previously unused oracle-negative parents | +| Native large official positives | 7/7 | Not a new negative challenge; the frozen large-v1 controls remain the applicable evidence | +| Source-disjoint provider-labeled test | 295/455 Google-labeled candidates | 0/469 OpenAI-labeled candidates | +| Broad non-Google challenge | Not applicable | 3/276 scored rows, plus 6 unsupported small images | + +The four transformed views of one parent are correlated observations, so the +scale rows establish transform consistency for 5 positive and 250 negative +parents, not independent sample sizes of 20 and 1,000. Across the locked scale +matrix, the phase-preserving template score had positive minimum `0.3603` and +negative maximum `0.1716`; split coherence had positive minimum `0.5344` in the +development matrix and locked negative maximum `0.5028`. All 12 independent +matching-oracle native positives, five ordinary-size and seven large, selected +period 16 and crossed the provisional conjunction. + +The three broad-challenge crossings are not established false positives. Two +carry TC260 AIGC provenance and one carries Samsung Galaxy C2PA provenance; +all three independently crossed the earlier fixed carrier detector. They show +that the measured carrier family is not safe for provider attribution. They may +be another compatible watermark deployment, Google-backed editing inside a +different product, or a shared renderer lattice. Only a matching pixel oracle +or source-matched counterfactual can distinguish those explanations. + +The cross-provider result is nevertheless useful: the Google codebook plus the +new synchronization path accepted none of 469 OpenAI-labeled test images. This +supports a different OpenAI payload or carrier family rather than a uniformly +weaker copy of the measured Google code. It does not provide a local OpenAI +detector; the official OpenAI pixel verifier remains the only production route. + +Two related hypotheses were rejected. Phase-only cyclic codeword agreement +retained hard non-Google carrier rows, so amplitude cannot be discarded. A +period-8 decoder made by antialiasing the 16-by-16 codebook into 8-by-8 templates +with area, linear, cubic, or Lanczos kernels also failed: its best Google-versus- +control AUC was `0.567`, and split amplitude remained near chance. Period 8 is +therefore still an unsupported state, not a threshold adjustment. + +This result promotes split-confirm synchronization to the next Gemini research +candidate, but not to runtime. The positive oracle set remains small, the +provider-labeled corpus is not a clean oracle, and the amplitude reranker +currently supports isotropic scale at zero rotation only. Rotation, crop, +translation, perspective, JPEG/WebP, independent temporal positives, and a +fresh clean negative source must pass before a production expert is calibrated. + +### 2026-08-14: content-whitened matched-filter result + +H4 was implemented as a complex multichannel matched filter on the 16 strongest +template harmonics. For each harmonic and patch, a ring of neighboring +noncarrier bins estimates the image-specific complex covariance of Green, +Red-minus-Green, and Blue-minus-Yellow. A fixed ridge regularizes that covariance +before the template and observed carrier vectors are whitened. Period is chosen +from the five H1 candidates on one checkerboard of patches; the reported score +is the smaller whitened cosine on the selection and disjoint confirmation +checkerboards. This retains the P1 separation and charges the candidate search +to the observed negative tail. + +The implementation passed synthetic carrier, independent-noise, resize, and +cyclic-phase mutations. It also found a real synchronization failure that the +amplitude reranker missed: under a broad native search, one of five official +ordinary-size positives selected a period-24 harmonic alias. Choosing among the +same frozen candidate set by whitened selection score changed that result to +period 16, and all five ordinary-size positives then selected period 16. + +The matched-filter score itself did not pass its preregistered improvement gate: + +| Challenge | Whitened positives | Whitened negatives | Whitened margin | Existing template margin | +| --- | --- | --- | --- | --- | +| Native pilot | 5/5, range `0.7006-0.8243` | 130 controls, maximum `0.4797` | `0.2209` | `0.2650` | +| Locked 0.8 resize | 5/5, range `0.4268-0.5994` | 200 fresh-parent controls, maximum `0.2165` | `0.2103` | `0.2420` | +| Native large positives | 7/7, range `0.5401-0.6017` | No new negative partition consumed | Not estimated | Not estimated | + +The different score scales make the absolute margins diagnostic rather than a +shared operating threshold, but both locked comparisons point in the same +direction: whitening separated the measured cohorts yet did not improve the +boundary over the phase-preserving spatial template. H4 is therefore rejected +as a replacement detection statistic. Its candidate-selection role remains a +useful H1 synchronization refinement and must be calibrated as part of that +search, not added as another post-hoc threshold. + +The full complex code also sharpened the interpretation of the non-Google +challenge. The three earlier TC260- or Samsung-provenance crossings scored +`0.4764`, `0.6666`, and `0.7746`; two match the whitened code more strongly than +the weakest official large positive. They cannot be dismissed as generic +period-16 image structure. They remain compatible watermark-family detections, +Google-backed edits, or a shared implementation until a pixel oracle or causal +counterfactual labels them. Provider attribution from this code is still +invalid. + +Finally, the ten OpenAI-labeled rows with the largest earlier Google-template +scores reached at most `0.2234` under the whitened Google code. This selected +challenge is not a sensitivity estimate and does not create an OpenAI detector, +but it is inconsistent with the hypothesis that those hardest OpenAI rows carry +the same code at merely lower amplitude. The next local OpenAI experiment still +requires authorized oracle labels or watermark-on/off pairs from which to learn +its own carrier and codebook. + +### 2026-08-14: repeated unknown-codeword result + +H5 was tested without requiring the observed phase vector to match the known +Google template. The codeword is estimated from one checkerboard of patches; +the score is the magnitude of its complex weighted inner product with the +independently estimated codeword from the other checkerboard. The 16 harmonic +coordinates remain preregistered from the carrier template, but the phase and +sign values are learned within each image. A global phase or polarity cancels +in the cross-product. + +The initially proposed free cyclic-shift nuisance was rejected. Selecting a +shift on alternating harmonics and confirming it on the other harmonics gave +official-positive scores of `0.3651-0.6739`, but the 130-control maximum was +`0.4698`. A fixed zero-relative-shift variant also overlapped. Natural +periodic phase supplied enough shift aliases that search-corrected confirmation +did not separate the cohorts. + +After global patch-origin correction, no relative shift is expected. The final +preregistered variant therefore uses all 16 harmonics, performs no shift or +harmonic selection, and confirms only across the two patch groups. It produced: + +| Challenge | Official or oracle-positive range | Control maximum | Margin | +| --- | --- | --- | --- | +| Native fixed period 16 | 5/5, `0.4498-0.7291` | 130 controls, `0.2879` | `0.1619` | +| Locked 0.8 resize at period 12.8 | 5/5, `0.3922-0.6713` | 200 fresh-parent controls, `0.2381` | `0.1541` | +| Native large fixed period 16 | 7/7, `0.4044-0.6485` | No new negative partition consumed | Not estimated | + +Synthetic mutation tests confirmed the intended invariance: the score accepted +the repeated random carrier, rejected independent equal-power noise, survived +isotropic resize after period canonicalization, and stayed high after a global +cyclic image translation even though the known phase-preserving template score +collapsed. + +The three TC260- or Samsung-provenance compatible carriers scored `0.3994`, +`0.4541`, and `0.6052`. Their within-image codeword repeat is therefore also +inconsistent with an accidental single global Fourier spike. This strengthens +the compatible-signal-family interpretation without identifying a provider. +The ten OpenAI-labeled rows selected for their earlier Google-template scores +reached at most `0.2568` at fixed period 16, below the native-control maximum. +That selected result is not a full OpenAI screen, but supplies no evidence that +the same period-16 harmonic support carries a different repeating OpenAI +payload. + +H5 advances as an independent research confirmation, not a runtime threshold. +The current evaluation supplies the correct period rather than charging the +full period search to this statistic, has only 12 independent official positive +parents, and reuses the available Picsum negative source. It must next survive +the broad H1 candidate search, the remaining scale and codec matrix, a second +clean negative family, and independent temporal positives. A free-shift version +is not reopened for the ordinary scale route; translation and crop belong in a +separately calibrated expert. + +### 2026-08-14: complex cyclostationary result + +H2 was implemented as the normalized complex spectral-correlation matrix +`F(k + alpha) * conjugate(F(k))` across all nine Green/opponent-color channel +pairs. The eight strongest preregistered template harmonics supplied `alpha`. +For every carrier shift, six one-bin neighboring shifts formed a same-radius +same-image null. Complex matrices were averaged separately over the selection +and confirmation checkerboards before taking their Frobenius norms, so content +phase had to repeat across distant patches to count. + +Synthetic mutations verified that this was a genuine cyclostationary test. An +amplitude-modulated equal-power random field produced positive carrier-minus- +neighbor contrast on both checkerboards; independent noise and an additive +periodic carrier did not. The latter mutation distinguishes H2 from the H1/H5 +additive-codeword family. + +The native oracle pilot failed decisively: + +| Statistic | Five official positives | 130 controls | Positive minimum minus control maximum | +| --- | --- | --- | --- | +| Selection contrast | `-0.0027-0.0342` | maximum `0.0593` | `-0.0620` | +| Confirmation contrast | `0.0067-0.0337` | maximum `0.0608` | `-0.0542` | +| Joint contrast | `-0.0027-0.0337` | maximum `0.0593` | `-0.0620` | + +H2 is therefore rejected without consuming another locked scale partition or +searching frequency bands, harmonic subsets, color spaces, or null offsets on +the observed controls. The measured Google signal supports a repeated additive +or content-adaptive codeword, not a detectable global multiplicative +cyclostationary modulation. This result does not constrain an unknown OpenAI +carrier, but the OpenAI branch still lacks the oracle evidence required to fit +one without provider leakage. + +### 2026-08-14: fixed-period codec and crop pilots + +The H1/H4/H5 probe gained explicit in-memory JPEG and top-left crop transforms +so the same decoded parent pixels enter every paired comparison. These pilots +use the five ordinary-size official positive parents and split the 130 native +controls into 50 development and 80 locked controls. The transformed views are +not new independent positive evidence. + +Lossy JPEG separates the useful statistics. Whitened and unknown-codeword +scores no longer maintain a positive development margin even at quality 95, so +they are unsupported for the lossy-codec route. The phase-preserving spatial +template remains usable at the known native period. Thresholds were frozen +after the 50-control development split and then applied unchanged: + +| Transform | Frozen template threshold | Repeated positive views | Locked controls | Locked score margin | +| --- | --- | --- | --- | --- | +| JPEG 95 | `0.12` | 5/5 | 0/80 | `0.1054` | +| JPEG 90 | `0.09` | 5/5 | 0/80 | `0.0727` | +| JPEG 80 | `0.065` | 5/5 | 0/80 | `0.0257` | + +The declining margin is monotonic and leaves JPEG 80 especially fragile. These +constants are research thresholds, not a runtime calibration: they use only +five positive parents, one negative family, fixed period 16, and one OpenCV +JPEG implementation. JPEG 100, chroma-subsampling variants, WebP, a second +encoder, codec-after-resize combinations, and independent positives remain +unqualified. + +Cropping shows the opposite phase behavior. Removing 5% or 10% from the top +and left collapses the unregistered template score, while cyclic phase +registration recovers the carrier. A crop-only conjunction was frozen at +registered template at least `0.20`, split coherence at least `0.55`, and split +amplitude at least `0.20`: + +| Transform | Repeated positive views | Locked controls | Registered-template locked margin | +| --- | --- | --- | --- | +| 5% top-left crop | 5/5 | 0/80 | `0.2275` | +| 10% top-left crop | 5/5 | 0/80 | `0.2378` | + +The unknown-codeword H5 statistic independently retained positive margins of +`0.0872` and `0.0839` in those two cells. This supports a separate native +period-16 crop/translation expert. It does not reopen cyclic registration in +the ordinary scale route, where the same nuisance parameter was already shown +to amplify interpolation aliases. Arbitrary crop sides and offsets, one-pixel +translations, combined crop-plus-codec edits, rotation, screenshots, broad +negative sources, and new positive parents remain before qualification. + +### 2026-08-14: fresh-source failure and frozen v3 confirmation candidate + +The unchanged runtime experts were challenged on the first 3,000 decoded- +pixel-unique files from the official Open Images test split. The image-id list +was fixed before download and has SHA-256 +`9b83bc221dd70823f39a9ad62b16c1dcc1f9c449f17c9ea3f6c9a51b581e0c1a`; +the ordered pixel-file hash list has SHA-256 +`2c49ccdec79ec662f202670b9e4c5ee0dfdd80d5747929dd09795d945b3a22a6`. +This source-fresh challenge rejected both existing precision claims. The fixed +expert accepted 5 of its 211 supported images at its shipped `0.1735706977` +threshold. The registered expert accepted 4 of 2,997 supported images at its +shipped `1.0` threshold. These are false-positive measurements, not clean-image +labels inferred from detector misses. + +The fixed v3 research candidate is frozen at `0.28`. It is above the largest +fresh crossing, `0.2592123`, and below the weakest of 12 source-diverse native +Google positives, `0.2978951`. That separation selected the threshold, so the +3,000-image set is development evidence for v3 and cannot validate it. The +candidate must remain unchanged on a second untouched source split. + +The registered v3 research candidate preserves the unchanged registered base +decision and adds an independently split phase confirmation. Selection and +confirmation use disjoint checkerboards of patches. The confirmation requires +period at least `10.0`, joint harmonic coherence at least `0.30`, and nonnegative +joint cyclic amplitude. Three narrow period ranges add the following measured +alias controls: + +| Registered period | Additional frozen gate | +| --- | --- | +| `18.0 <= p < 18.6` | held-out unknown-codeword confirmation at least `0.13` | +| `18.6 <= p < 20.0` | joint harmonic coherence at least `0.40` | +| `20.0 <= p < 22.0` | held-out unknown-codeword confirmation at least `0.02` | + +The rule rejected all four fresh registered crossings while retaining 147 of +148 base-accepted positives from 226 source-diverse transformed parents. A +separate dense transform matrix applied 35 scales from 0.65 through 1.50 in +0.025 increments to 12 native positive parents; it retained all 359 views that +the unchanged registered base accepted. At the hard-negative period `18.28`, +all 11 base-accepted positive views passed, while the natural-image crossing +failed the held-out-codeword gate. Derivatives are not counted as independent +positive evidence. The exact candidate constants live in +`scripts/synthid_affine_lattice_probe.py` and are frozen before the second +negative holdout is acquired. + +An explicit period-8 rescue is still rejected. A six-gate candidate selected +on earlier controls accepted 7 of these 3,000 fresh controls, so scale 0.5 has +no qualified local route. + +### 2026-08-14: registered-v3 holdout qualification and weak-signal rescue + +The registered confirmation rule above was frozen unchanged before a second +3,000-file Open Images test-split cohort was decoded. Its image-id list has +SHA-256 `91e1f19f559a432f61c5eb1f16797d40d706170c43094bc7d4cb9506309b662a`; +the ordered pixel-file hash list has SHA-256 +`651f959689e7043989ded09fa4aece5cea2592e7b69e371cc97c35f581d9f57d`. +All 3,000 decoded hashes were unique and none overlapped the first cohort. The +unchanged registered-v2 base produced no threshold crossings among 2,996 +supported images, so registered-v3 also produced 0/2,996. Together, the two +Open Images cohorts give registered-v3 0/5,993 false positives after the +confirmation rule was applied, with a one-sided 95% zero-error upper bound of +about 0.050%. They are source-disjoint samples from one source family, not two +independent acquisition mechanisms. + +A second-family challenge used the first 3,000 sorted COCO val2017 files. The +newline-delimited file-name list has SHA-256 +`7a2e5f47f6c775f0de839b8270ee0874526a8d0984c66cd7940e1a04fc89f4f5`; +the ordered per-file SHA-256 manifest has SHA-256 +`b1b0c73891b6d5b49a7817c9bd0a47309a424e437ba36fabed7bd9077dfeda54`. +All 3,000 pixel hashes were unique. The unchanged registered base produced +0/2,366 crossings, with maximum score `0.932964706`; registered-v3 therefore +also produced zero. COCO predates this confirmation experiment and is not a +fresh acquisition, but it is a distinct natural-image source family. Across +both Open Images cohorts and this COCO challenge, registered-v3 has 0/8,359 +supported-control crossings. This qualifies the precision-first positive route; +it does not turn a miss into proof of absence. + +The fixed `0.28` candidate failed the second holdout and is rejected. It +accepted one of 213 supported images, at score `0.322542963`. The runtime keeps +fixed-v2 only as an explicitly requested diagnostic and never unions it with a +production positive route. + +Split confirmation was also tested as a recall rescue below the registered base +threshold. The first relaxed rule recovered 16 of 76 source-diverse base misses +but accepted 94 of the 312 most dangerous first-cohort controls, so repeated +phase alone is decisively insufficient. A stricter H4/H5 conjunction required +matching spatial and spectral periods, base amplitude margin at least `0.4`, +high-band margin at least `0.3`, the frozen confirmation gates, and a minimum of +content-whitened template match, all-harmonic repeated-codeword match, and +split cyclic amplitude at least `0.225`. The threshold is above the first- +cohort maximum `0.222382188`. It added only 6 of 224 scored source-diverse +positives and 8 of 432 dense-scale views, while accepting none of the second- +cohort candidates and none of the scored COCO candidates. That small recall +gain does not justify another runtime branch: it was selected on exposed +development positives, adds a substantially more expensive whitening pass, and +lacks new positive parents. It remains a recorded research candidate rather +than being hidden inside registered-v3. + +The runtime implementation was then checked on the transformed pixels rather +than the original source paths. Of 226 source-diverse transform assignments, +224 remained large enough for the split-patch research scorer. Registered-v3 +detected 147/224 overall: 0/40 at 0.5x and 147/184 (79.9%) from 0.65x through +1.5x. The v2 base accepted 148 of the latter 184 and v3 confirmation rejected +only one base crossing. Failure attribution on the 184 supported non-half-scale +views was 10 period-winner mismatches, 17 amplitude failures, 9 high-band +failures, one confirmation failure, and 147 detections. This production-parity +measurement replaces the earlier accidental source-path check; transform +labels attached to untransformed files are not sensitivity evidence. + +H3 then supplied a genuinely content-matched null. For each registered period, +the 16 fixed neighboring periods at offsets from plus or minus 0.35 through 2.0 +pixels are scored on the same two checkerboards. The candidate statistic is the +smaller of the selection- and confirmation-group target coherence minus that +group's largest off-period coherence. Combining the two Open Images development +cohorts froze a recovery rule at registered base score at least `0.8`, the +existing v3 confirmation gates, and same-image joint excess at least +`0.05695968150082637`. It rescued 7 of 36 sub-threshold transformed views and +accepted 0/262 prescreened candidates from the first 3,000 COCO controls. The +untouched final 2,000 COCO files contained 1,598 registered-supported images +and seven near-threshold candidates. Four candidates were too small to form two +independent 256-pixel patch groups; all three scored candidates had joint excess +at most zero. The frozen rule and registered-v3 both accepted 0/1,598. This +passes the new negative-side test without changing the threshold. The candidate +remains outside runtime because all seven recovered positives influenced +feature selection and no new positive parent has yet confirmed the gain. + +The same null did not justify relaxing the registered requirement that the +spatial reconstruction winner equal the strongest spectral-period candidate. +A full rescore of the two 3,000-file Open Images cohorts retained 469 plausible +mismatch candidates after amplitude and high-band prefilters; 212 also passed +the frozen split confirmation. Their same-image joint-excess maximum was +`0.1762448251`. Only five of the ten non-half-scale positive mismatch records +passed the same mechanism prefilters, and their maximum was lower at +`0.1464860382`. No zero-control same-image threshold retained a positive. A +post-hoc product of amplitude margin, high-band margin, and coherence could +retain two positives above the observed control maximum, but it was selected +from many combinations and offers too little gain to open another holdout. +Spatial/spectral winner equality therefore remains mandatory. + +### 2026-08-14: content-adaptive patch-phase recovery candidate + +The official SynthID-Image paper describes a content-dependent learned +watermark pattern, so a single whole-image template cannot be assumed to be the +complete decoder. A new research statistic therefore treats the frozen tile as +a synchronization basis rather than as a fixed global payload. At the +registered period, each 256-pixel patch is folded independently, every cyclic +tile shift is standardized against that patch's own shift distribution, and +the strongest half of the selection checkerboard chooses one phase. The score +is the smaller of that phase's trimmed standardized response on the selection +and disjoint confirmation checkerboards. Filenames, metadata, and scene labels +do not enter the statistic. + +The research-only recovery rule was frozen at matching spatial and spectral +periods, the existing split confirmation, both registered gate margins at +least `0.45`, one gate margin at least `1.0`, and joint patch-shift score at +least `2.5`. It recovered 5 of 18 scored amplitude/high-band failures from the +224-view transformed-positive audit. Added to registered-v3 alone, that would +move the measured non-half-scale result from 147/184 to 152/184 (82.6%). It +accepted none of 88 gate-failure candidates in the first 3,000-image Open +Images development cohort and none of 334 previously scored control +candidates; those sets overlap and are not added as independent denominators. +The rule and threshold were recorded before reading a 350-image Picsum +feature-holdout. All 350 decoded rasters were unique. The frozen rule accepted +none; only two controls reached its mechanism pre-gates, and their maximum +joint patch-shift score was `0.7029283` against the `2.5` threshold. The +zero-error one-sided 95% upper bound is about 0.85% for this one-source +feature-holdout. The candidate remains outside runtime until new positive +parents pass unchanged. + +That positive test then failed on a genuinely different carrier cohort. Seven +large Gemini images with independent official pixel-positive verdicts, none of +whose source paths appeared in the 224-view transform audit, were resized by +0.65, 0.70, and 0.75. Registered-v3 accepted 0/21 and the frozen patch-phase +recovery added 0/21. The base amplitude was often strong, but the RGB high-band +margin became negative on most views and several period searches selected an +octave alias. The branch is therefore a real recall improvement for the +measured registered carrier, but not a universal cross-epoch solution. A +scale-registered form of the independently validated large-v1 opponent-color +signal is the next hypothesis; weakening the existing high-band gate is not. + +That hypothesis recovered the entire failed cohort. A signed spectral curve in +the Red-minus-Green and Blue-minus-Yellow planes searched periods 10.0 through +12.5 in 0.05-pixel steps, retained three separated candidates, canonicalized +each to the frozen period-16 tile, and reranked by the minimum of fixed RGB, +Red-minus-Green spatial, and Blue-minus-Yellow spatial agreement. Thresholds +were frozen at `0.16`, `0.60`, and `0.55`, respectively. Automatic period +selection recovered the exact injected period on all 21 views and accepted +21/21, versus registered-v3's 0/21. It accepted 0/21 symmetrically resized +views from seven large Open Images controls; the maximum control decision +margin was `0.9462124`. These positives and controls selected the gates, so +this is development evidence. The candidate was frozen before a 350-image +Picsum feature-holdout. It accepted 0/350 decoded-pixel-unique controls there; +the maximum decision margin was `0.7778322`. This negative-side result passed +unchanged. + +The broader challenge rejected that unrestricted form. Extending the search to +periods 7.5-14.5 recovered 63/63 lossless views from the same seven official +parents at nine scales from 0.5 through 0.9, but accepted 605/2,997 and +587/2,996 controls in two Open Images cohorts. The crossings concentrated at +periods 8 and 12.8 and tracked JPEG sampling. A 3,964-image COCO challenge added +one period-10 crossing below one megapixel, while 3,772 heterogeneous +non-Google/non-OpenAI Spaces controls added 28 crossings, again only at periods +8 and 12.8. The unrestricted branch is a codec/resize-lattice detector and is +rejected. + +A symmetric codec test separated lossless scale recovery from recompression. +At scales 0.5, 0.625, and 0.75 the raw views passed 21/21, while JPEG-95, +JPEG-85, and WebP-95 passed 0/63. At the supplied target period, half-scale JPEG +views inverted the opponent-color polarity and the other lossy views largely +lost spatial agreement. This is not a period-search-only failure. + +The production candidate was then frozen at 1-10 megapixels, minimum side 768, +selected period 8.1-12.0, and the original `0.16`, `0.60`, and `0.55` gates. +An initially added spectral/spatial period-agreement condition was removed when +the real API path showed that a harmless 11.0 spectral alias reduced 0.55x +recall from 7/7 to 1/7; its only reserve-control counterpart was below the +already frozen geometry floor. The revised rule was fixed before a separate +1,000-image Picsum holdout. It accepted 0/1,000, with maximum unrestricted +margin `0.7953734`, and the production `detect_synthid` seam detected 42/42 +lossless views across scales 0.55-0.75. This bounded expert now runs after +registered-v3 abstains as `synthid-periodic-tile-opponent-registered-v1`. + +### 2026-08-15: period-8 codec veto and lossless 0.5x recovery + +The original period exclusion conflated two observations: the 0.5x watermark +and the native JPEG block lattice both select period 8. Direct template polarity +did not resolve the confound. JPEG-95 and JPEG-85 half-scale positives produced +a strong inverted opponent template, but random JPEG 4:2:0 controls produced +the same profile. A matched resize-plus-JPEG residual experiment used 175 +control parents for development, 175 disjoint parents for holdout, three JPEG +qualities per parent, and leave-one-positive-parent-out checks. The best ridge +variant accepted 19/21 fitted positive views but also 86/525 holdout controls; +the variants with fewer control errors lost most or all positive parents. + +A nonlinear block-DCT follow-up evaluated 80 target-derived statistics in RGB, +Green, opponent-color, and YCbCr views on the same matched transform. Its best +zero-holdout-error statistic retained only 2/21 JPEG positives. The highest-AUC +zero-error opponent statistic reached about `0.735` but retained 0/21 at the +development maximum. Neither JPEG recovery hypothesis enters runtime. + +The successful discriminator is instead a codec veto around the unchanged +opponent carrier gates. For each decoded opponent plane, it measures the mean +absolute edge energy on native 8-pixel boundaries relative to the median of +non-block phases. All seven lossless 0.5x positives had both Red-Green and +Blue-Yellow ratios no greater than the frozen `1.05` ceiling. The ceiling +rejected 600/600 development period-8 codec crossings, then 582/582 crossings +in the nonoverlapping Open Images holdout, 601/601 in the feature-unseen frozen +reserve, and 7/7 in the heterogeneous Spaces challenge. An additional 350 +controls transformed through the identical lossless 0.5x resize produced zero +base crossings before the veto. + +The production fallback now accepts periods 7.9-12.0 and requires the veto +through period 8.1. The public API detects 7/7 scale-0.5 views and 49/49 across +the measured 0.5-0.75 lossless matrix from seven official pixel-positive +parents. This is a bounded lossless extension, not evidence of JPEG robustness +or arbitrary scale coverage. + +The provider challenge is evidence for a shared family, not attribution. The +unrestricted statistic accepted 95/7,254 OpenAI files with an explicit SynthID +assertion, all at periods 10.70-10.80, and 4/287 same-lineage files without an +assertion. The latter label is not an oracle-negative watermark verdict. The +95 decoded rasters were unique and had zero overlap with the 94 period-8 +candidate hits, producing a 189/7,259 union. However, JPEG-95 and WebP-95 each +reduced the new component from 95/95 to 0/95. It remains compatible with either +a complementary carrier epoch or a geometry-linked export lattice. The runtime +result remains provider-neutral, and the official OpenAI verifier is still the +only qualified provider-wide OpenAI route. + +### P3: productionize OpenAI without pretending the local route works + +The near-term production route is the official pixel verifier. Remaining live +and deployment qualification: + +- complete the approved SDK transport set beyond the finished positive PNG + smoke, covering JPEG, WebP, and pre-established positive and negative + expectations with pixel-identical, metadata-free files; +- malformed file, 50 MiB boundary, timeout, disconnect, 400, 404, 429, 5xx, + unexpected response, duplicate SynthID result, and missing SynthID result + qualification against recorded or contract-safe fixtures; +- deployment-level bounded backoff and a circuit breaker for transient API + failures, without representing transport failure as a watermark verdict; +- confirmation of data-retention, access-control, audit-log, and user-consent + requirements for the deployment environment; +- secret-safe logs that retain request ids, byte counts, timings, and status but + never image bytes, credentials, user paths, or decoded-pixel hashes. + +The local transport now fixes both the default SDK and per-request timeout at +120 seconds, sets automatic retries to zero so one acknowledgement cannot +retransmit media, and omits the decoded-pixel hash from request logs. Contract +tests cover exact and over-limit file sizes, caller cancellation, malformed +responses, all documented HTTP classes, timeout, and connection failure. A +structured error preserves status, API error code, request id, `Retry-After`, +and whether an explicit retry is appropriate. One credentialed positive PNG +smoke has now passed. Remaining format qualification, deployment retention +approval, and an operational circuit breaker remain environment-level work; +the research harness itself does not perform live uploads. + +A local OpenAI detector reopens only after one of two prerequisites exists: +written authorization to obtain non-adaptive pixel-verifier labels at research +scale, or source-matched watermark-on/watermark-off pairs. The first local +epoch then starts from those causal residuals and uses same-provider negatives. +No additional model is selected on C2PA assertions, filenames, OpenAI export +geometry, or the already exhausted provenance-labeled cohorts. + +### Updated local-detector execution order + +The local program now advances through evidence gaps rather than adding more +unbounded feature searches. + +1. Freeze the current routed Gemini bank, calibration artifacts, parent-group + manifests, and support reasons as the reproducible baseline. Finish the + active fine/anisotropic registration experiment against a fresh negative + holdout and either promote it as a separately versioned expert or reject it + before opening another synchronization branch. +2. Run the production router, not isolated expert functions, over the complete + available size, scale, aspect, orientation, and codec matrix. Group results + by parent and cluster misses and abstentions by measured carrier state. This + produces the actual coverage backlog. +3. Address Gemini clusters one at a time. Use full reciprocal-lattice + registration first, select-confirm same-image null and repeated unknown + codeword confirmation second, and codec-specific residual experts third. + Every branch selects on one region group, confirms on disjoint regions, and + receives its own multiple-search null calibration. +4. Do not train a local OpenAI classifier from renderer style or provenance + assertions. Existing verifier-confirmed cases may validate whether a + candidate reaches the known pixel signal, but the training corpus requires + source-matched watermark-on/off pairs or written authorization for a + non-adaptive research label set. Same-provider hard negatives are mandatory. +5. Once that OpenAI prerequisite is met, learn causal residual differences, + cluster stable carrier states across model and time windows, and expose each + recurring state as a separate versioned expert. Test reciprocal-lattice, + multichannel complex matched-filter, and within-image unknown-codeword + families before any residual-only learned model. +6. Qualify both provider banks through one locked and later temporal matrix. + A positive names only the provider scope proved by exclusivity data; a miss + is `not_detected` only inside sensitivity-certified strata and otherwise + remains `indeterminate` or `unsupported`. + +### P4: run one symmetric robustness matrix + +Every promoted expert and both official-verifier integrations receive the same +parent-grouped challenge: + +| Axis | Frozen views | +| --- | --- | +| Codec | lossless PNG, source JPEG, JPEG 100/95/90/80, WebP lossless/95/80, 4:4:4 and 4:2:0 where controllable | +| Scale | 0.5, 0.625, 0.75, 0.8, 0.9, 1.0, 1.1, 1.2, 1.333, 1.5, and 2.0 | +| Geometry | supported size-bin boundaries, modulo-period edges, portrait, landscape, square, and extreme aspect ratios | +| Spatial edit | 1-10% crop, one-pixel and multi-pixel translations, screenshot round trip, and 90/180/270-degree rotation | +| Pixel edit | mild blur, sharpen, denoise, gamma, contrast, saturation, and color-profile conversion | +| Content | photo, face, text-heavy, illustration, flat graphic, low texture, high texture, and alpha | + +Transforms are applied identically to positives and controls. Scores and +verdicts are aggregated by parent, not by treating eleven derivatives as eleven +independent images. Unsupported cells are recorded as such; they do not lower +the denominator silently. Robustness work stops when the signal is no longer +identifiable at the required false-positive boundary. + +### P5: use release-grade statistical gates + +Freeze the following gates before the new corpus is scored: + +- the one-sided 95% upper confidence bound for false-positive rate must be at + most 0.1% overall and at most 0.5% in every critical negative stratum; +- the one-sided 95% lower confidence bound for sensitivity must be at least 90% + in every scope advertised as detection-supported; +- a local `not_detected` absence claim needs a much stronger 99% sensitivity + lower bound in every advertised stratum; otherwise a miss remains + `indeterminate`; +- temporal, model-family, surface, geometry, and codec results are reported + separately, with no claim wider than the weakest passing stratum; +- every selected threshold must remain unchanged on a newly acquired temporal + holdout and on a second negative source family; +- provider attribution requires cross-provider exclusivity data. Until that + passes, the local result names the signal family but not its generator. + +Confidence intervals, operating thresholds, model hashes, calibration hashes, +split hashes, and code revision go into one machine-readable qualification +report. The report fails closed on missing records, duplicate parent groups, +NaN or infinite scores, unknown detector ids, or a changed artifact hash. + +### P6: harden runtime and operations + +Before release: + +- benchmark wall time and peak memory at every supported size boundary; keep + large-image folding and template subtraction stripe-bounded, and freeze + numeric p95 latency, peak-memory, and concurrency budgets for the target + deployment before qualification; +- fuzz corrupt, truncated, decompression-bomb, unusual color-mode, alpha, and + oversized images; errors must be bounded and must not become negatives; +- prove deterministic scores across supported platforms and dependency pins, or + version platform-specific calibration explicitly; +- run shadow mode before user-visible verdicts, collecting only consented, + privacy-safe aggregate score histograms, support rates, latency, errors, and + detector-version counts; +- alert on score-distribution drift, support-rate changes, remote outcome drift, + error-rate spikes, and latency regressions; +- keep the previous detector and calibration artifacts available for immediate + rollback, and make every result identify which version produced it; +- require the applicable dependency lock, security scan, Ruff, formatting, + Pyright, unit, integration, mutation, packaging, and full corpus gates to be + green. An untriaged production dependency advisory blocks release. + +### Execution order and stop/go checkpoints + +| Milestone | Deliverable | Go condition | +| --- | --- | --- | +| M1 | Unified result contract and expert registry | No miss, error, unsupported input, or remote failure can be presented as clean | +| M2 | Immutable Gemini and OpenAI qualification manifests | Parent-group split audit and all ingestion mutations pass | +| M3 | Gemini fixed and large requalification | P5 precision and sensitivity gates pass on locked and temporal data | +| M4 | OpenAI SDK/API qualification | Approved live transport matrix passes, privacy boundary is accepted, operational failures abstain | +| M5 | Optional registered, codec, and crop experts | Each expert independently passes P5; failed cells stay unsupported | +| M6 | Shadow deployment | No unexplained drift or SLO regression through the prespecified observation window | +| M7 | Production release candidate | All code, corpus, security, documentation, rollback, and versioning gates pass | + +The immediate work queue is M1, freezing and auditing the current routed +baseline, completing the active Gemini fine/anisotropic candidate decision, +finishing the 3,000-image fresh Gemini large negative acquisition, and running +the router-level all-resolution matrix. In parallel at the planning and data +provenance level, identify a permitted source of OpenAI watermark-on/off pairs +or obtain written authorization for a non-adaptive research label set. The +first credentialed OpenAI positive PNG smoke is complete; further live API +calls are transport qualification, not local-detector training. New feature +search starts only from a measured coverage cluster and consumes a fresh +holdout once. + +### 2026-08-15: OKLab random-codeword replacement probe + +A pixel-only probe tested the hypothesis that the measured periodic residual is +only the carrier for a repeated coded message. It compared independent +high-pass OKLab noise, periodic-comb suppression, random periodic injection, +and complete replacement of the estimated period-16 residual with a +magnitude-matched random-phase tile. The four inputs were the cleared public +Google-oracle-positive originals already in the tracked research corpus. + +Independent noise was inefficient. At the stronger tested allocation, with +three times more RMS assigned to each chroma channel than to luminance, the +median fixed-carrier score changed by only `-0.0219` while median PSNR fell to +`32.56 dB`. Comb suppression alone changed the median score by `-0.1962` at +`58.05 dB`, but moved only one of four inputs below the unchanged detector +threshold. + +Replacing the complete tile was qualitatively different. At replacement +strength `0.95`, all four fixed-carrier scores moved below the frozen +`0.17357069773071196` threshold, from source scores `0.7322`, `0.6503`, +`0.3300`, and `0.4161` to `0.097`, `0.031`, `0.158`, and `0.087`. Median PSNR +was `53.85 dB`, minimum PSNR was `52.56 dB`, and minimum grayscale SSIM was +`0.99811`. All four paired score changes had the same direction, but four +parents give a two-sided sign-test result of only `p = 0.125`. The probe was +designed after the carrier family was known and is a causal local-score +intervention, not an independent detector or removal validation. + +A preregistered provider check stopped at its source control. Gemini returned +the expected Google C2PA record but its pixel-verification stage remained at +`Connecting to Verify AI`; no candidate was submitted after the unhealthy +control. The next healthy oracle window must use the unchanged source control, +the frozen `0.95` replacement, and a norm-matched sham. Until that paired batch +returns a source-positive and candidate-negative pixel verdict, random-codeword +replacement remains a stronger mechanism hypothesis rather than a SynthID +remover. + +A follow-up radial ablation separated the replacement tile into full-image +frequencies measured in cycles per pixel. On the same four public positives, +randomizing only `0.125-0.40` reduced the fixed score by a median `0.2093` at +`56.48 dB` median PSNR. The `0-0.125` band had no median effect, and frequencies +above `0.40` changed the median score by only `-0.0086`. Lower-middle and +upper-middle bands each contributed, with median changes of `-0.0594` and +`-0.0508`. This locates the measured Gemini carrier's most efficient causal +surface in the middle band, not at the highest frequencies. + +The ablation still operates on a folded periodic tile, so every edited +coefficient remains a comb harmonic. It does not distinguish a sparse set of +carrier lines from broadband coded noise between those lines. The next +mechanism test must use continuous annular bands in local patch spectra, compare +spatially independent and cross-patch-coherent perturbations at matched OKLab +distance, and preserve a fixed random seed before any provider verdict. + +## Decision record + +The program has four possible honest outcomes per provider: + +| Outcome | Product consequence | +| --- | --- | +| Causal signal and detector both generalize | Continue to pixel-only removal | +| Detector works but causal attribution fails | Ship no SynthID detector claim; retain as provenance research | +| Detector generalizes but pixel-only removal does not transfer | Keep local detection, retain regeneration fallback | +| Pixel-only removal clears the oracle with quality gates | Productize provider-specific detector and remover | + +Stopping at a failed gate is a result. It prevents a local surrogate, export +fingerprint, or quality metric from being mistaken for control over SynthID. + +### Decision taken 2026-08-16 + +The second row is the outcome that occurred, on the evidence in this log. No +local expert in this program detects SynthID. `large-v1` detects the generation +pipeline: its statistic is destroyed by a seven-pixel crop, while the published +evaluation of the mark retains `99.97%` TPR under aggressive crop and resize. +The direction agreed is therefore to keep what works under its true name and to +build toward provenance rather than toward a watermark claim. + +**Ship what is real, renamed.** `large-v1` identifies large Google-pipeline +images with a measured `0/2,637` control rate and `8/11` on fresh +out-of-distribution positives. It may be shipped as generator or pipeline +identification, never as watermark detection, and its two failure modes belong +in the product surface rather than in a footnote: it breaks on a crop that is +not tile-aligned, and it breaks when the generation pipeline changes, as +`gemini-2.5-flash-image` output already demonstrates at `0.063`. + +**Develop toward provenance.** Generator, pipeline and epoch identification is a +task with abundant data, reliable labels and a measurable gate. U2 shows it is +not solved, since the current features accept between a fifth and a half of +foreign synthetic images, but it is the task the evidence actually supports. + +**Keep the watermark route open, blocked on access rather than on effort.** A +crop-robust detector of a keyed, perceptually masked mark needs a learned decoder +trained on clean pairs. Google emits no unwatermarked image from any accessible +path, and OpenAI's verifier may not be used to label a training set. If either +constraint lifts, this route reopens; until then it is not an engineering +problem. + +**One rule adopted from the failure.** Every candidate local expert must survive +a non-tile-aligned crop before it is calibrated. The test costs one line, it is +the property the mark provably has, and its absence is why a pipeline artifact +was carried for three months as a watermark detector. + +## Historical first milestone + +The first local-research milestone was defined to produce evidence rather than +shipping code. Its scope was: + +1. the private-corpus schema and auditor; +2. an OpenAI authorization decision for use of the remote provenance verifier; +3. an independently verified status for candidate causal pairs; +4. a canonicalized OpenAI pilot set with hard negatives; +5. the D1 confound report; +6. the D2 low-texture carrier report with leave-one-group-out results; +7. a go or no-go decision for real-image detector training. + +The completed experiments produced a no-go decision for a local universal +OpenAI detector. The later official remote backend is a separate production +route and does not retroactively turn provenance-labeled exports into pixel +oracle training data. + +## Primary sources + +- OpenAI, [Content provenance](https://developers.openai.com/api/docs/guides/content-provenance). +- OpenAI, [ChatGPT Images 2.0 system card](https://deploymentsafety.openai.com/chatgpt-images-2-0/automated-evaluations-and-adversarial-testing). +- Google, [Verify AI-generated images, videos, and audio](https://support.google.com/gemini/answer/16722517?hl=en). +- Gowal et al., [SynthID-Image: Image watermarking at internet scale](https://arxiv.org/abs/2510.09263). +- Meta, [VideoSeal](https://github.com/facebookresearch/videoseal). +- Meta, [Watermark Anything](https://github.com/facebookresearch/watermark-anything). diff --git a/docs/synthid-oracle-ladders.md b/docs/synthid-oracle-ladders.md new file mode 100644 index 0000000..e8cc17e --- /dev/null +++ b/docs/synthid-oracle-ladders.md @@ -0,0 +1,54 @@ +# OpenAI SynthID oracle ladders + +> This page is a routing hub. The mixed 2026-08 campaign log was split by +> purpose on 2026-08-22. Numeric tables now live on the page for that +> purpose. Raw images stay gitignored under +> `.local-eval/synthid/prc-oklab-attack-2026-08-15/`. +> +> Seeds s1, s2, s3 are listed in +> [SynthID local detector research](synthid-detector-research.md). + +| Page | Use it for | +| --- | --- | +| [SynthID local detector research](synthid-detector-research.md) | Hunt for a keyless local mark detector. Open. Closed routes on that page. | +| [AI-generated image classifiers](ai-generated-image-classifiers.md) | Primary metadata-free AI-generation task, Model 1 partial result, and frozen-checkpoint sweep. | +| [SynthID source classifiers](synthid-classifiers.md) | OpenAI/Gemini source finding and `pipeline_lattice` as google-lineage. | +| [SynthID mark removal research](synthid-removal-research.md) | Quiet-remover hunt. Closed except ~19-24 dB 16-32 scramble and blur sigma 7. | +| [Mechanism reference](synthid.md) | How SynthID works, provenance, robustness, regeneration. | +| [Chronological plan archive](synthid-detector-removal-plan.md) | Dated H-gates, corpora, and session notes in original order. | + +## Where former sections went + +| Former heading | Now | +| --- | --- | +| 2026-08-21 pairs, flats, L1 labels | [detector](synthid-detector-research.md) | +| Identity, token/layout, tomography, preprocess E1 | [detector](synthid-detector-research.md) (presentation gate) | +| L1 repair, L1 geometry, camera vs edit pair, E3 | [detector](synthid-detector-research.md) | +| CLIP-L photo vs AI, provider CLIP, union | [AI classifiers](ai-generated-image-classifiers.md), [source classifiers](synthid-classifiers.md) | +| 124-d three-class and binary AI | [source classifiers](synthid-classifiers.md) | +| `pipeline_lattice` re-check, Spaces census | [source classifiers](synthid-classifiers.md) | +| Attack / kill / flip / add / surround ladders | [removal](synthid-removal-research.md) | +| 16-32 titration, E2 scramble, Bayer and geometry | [removal](synthid-removal-research.md) | +| S4 provider split, M2 Imagen `addWatermark`, reverse-SynthID, Bypass | [detector](synthid-detector-research.md) | +| Photo `d'` budget 13.4 dB, 128-photo student, 16-32 residual without flat `G` | [detector](synthid-detector-research.md) | +| OKLab codeword replacement, add-context as presentation | [removal](synthid-removal-research.md) | +| Three-class OpenAI / Gemini / photo ask | [source classifiers](synthid-classifiers.md) | +| Wild hypotheses 2026-08-23 (sort, CDMA, bispectrum, PRNU, affine 16-32 NCC no lock) | [detector](synthid-detector-research.md) | +| Wild oracle 2026-08-23: misnamed polar-1632 is cartesian scramble replicate `not_detected`; band-transplant `not_detected`; palette64 `detected` | [removal](synthid-removal-research.md) | +| External literature 2026-08-23 (Gowal, AWPD, PRC, Zhao, UnMarker, CtrlRegen, MarkNull, reverse-SynthID) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md), [AI classifiers](ai-generated-image-classifiers.md) | +| Adjacent literature, not SynthID (Cox, HiDDeN, StegaStamp, Tree-Ring, Ojha CLIP, Corvi Fourier, DCCT CFA, PRNU PCE) | [detector](synthid-detector-research.md), [AI classifiers](ai-generated-image-classifiers.md), [removal](synthid-removal-research.md) | +| Image investigation and data hiding (LSB, UNIWARD, Baluja, HiNet, Gaussian Shading, ELA, JPEG ghosts, DIRE, CNNDetect) | [detector](synthid-detector-research.md), [AI classifiers](ai-generated-image-classifiers.md) | +| Waveforms in a picture (DFT, Fourier-Mellin, Zernike, chirps, Fresnel, DRPE, cyclostationary) | [detector](synthid-detector-research.md) | +| Remaining hypotheses 2026-08-23 (CFA, JPEG ghost, Paeth, face-gate, Baker, Poisson, nested LSB, angular scramble) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md), [AI classifiers](ai-generated-image-classifiers.md) | +| Non-local 2026-08-23 (HEIF/AV1 survive, print-scan sim 24.95 dB still detected, gpt-image-1.5/mini encoder-off, dated gpt-image-2 same stamp, DIRE DDIM texture) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md), [AI classifiers](ai-generated-image-classifiers.md) | +| Waveform shells 2026-08-23 (Mellin/Zernike/PCET/Fresnel no photo lock; Y scramble kills, chroma and 90° sector do not; radial-phase kills) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md) | +| Public decoder sweep 2026-08-23/24 (`Rinne414` fixed templates 0/400 on current OpenAI/Google images; injected template spoofs the public correlator, but 0.25-2x subtraction stays official `detected` at 43.13-25.72 dB) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md) | +| Public CNN sweep 2026-08-24 (`newideas99` retains 172/200 OpenAI but accepts 110/200 Google; a whole-frame gradient makes the ensemble effectively zero but stays official `detected` through ±8/255 / 34.30 dB) | [detector](synthid-detector-research.md), [source classifiers](synthid-classifiers.md), [removal](synthid-removal-research.md) | +| `reverse-SynthID` V4 repeat 2026-08-24 (77/200 Google and 76/200 OpenAI at the published cut; Round-06 manual Gemini verdicts are not published as per-image evidence) | [detector](synthid-detector-research.md), [source classifiers](synthid-classifiers.md), [removal](synthid-removal-research.md) | +| Broad GitHub sweep 2026-08-24 (133 unique primary repositories; REGRET and the vordme flat-field SVM fail the strict v7 transfer; literal ports and generic heuristics add no signal) | [detector](synthid-detector-research.md), [source classifiers](synthid-classifiers.md) | +| Additional public removal sweep 2026-08-24 (DeSynth stays official `detected`; froggeric and other ControlNet/diffusion projects corroborate lossy regeneration; PortOS closes high-fidelity phase/noise jamming) | [removal](synthid-removal-research.md) | +| Google verifier surfaces 2026-08-24 (retired Vertex `imageverification@001`; live but unlisted and inaccessible `synthid.googleapis.com/upload/v1:verifyContent`) | [detector](synthid-detector-research.md) | +| Metadata-free source hunt 2026-08-24 (1% cascades rejected; frozen multiscale fusion found 276/600 OpenAI-or-Google targets, 0/1,000 photographs and 1/25 TC260 on blind v4) | [source classifiers](synthid-classifiers.md) | +| Original-export source hybrids 2026-08-24 (frozen v7 215/400 exact; post-hoc v8 292/400 and per-codec v11 316/400 exact) | [source classifiers](synthid-classifiers.md) | +| Frozen general-detector transfer 2026-08-24 (Community Forensics and SPAI do not replace or safely fuse with Model 1 at the 1% calibration FPR; B-Free weights unavailable and license noncommercial) | [AI classifiers](ai-generated-image-classifiers.md#frozen-public-checkpoint-transfer-2026-08-24) | +| General-classifier GitHub sweep 2026-08-25 (2,006 unique repositories found; DDA and PGC expose complementary frozen signals, but DDA raises independent FPR and PGC's OpenAI gain confounds 24/24 Kodak scans; DGS-Net, SAFE, RINE, and Nonescape Mini rejected) | [AI classifiers](ai-generated-image-classifiers.md#general-ai-classifier-github-sweep-2026-08-25) | diff --git a/docs/synthid.md b/docs/synthid.md index 0f949bd..ac4a904 100644 --- a/docs/synthid.md +++ b/docs/synthid.md @@ -1,8 +1,12 @@ # SynthID: technical reference -> Technical research reference. Current package behavior is defined by the +> Technical research reference for the mark itself and for what this package +> ships. Current package behavior is defined by the > [supported signals](supported-signals.md), [known limitations](known-limitations.md), -> and [module internals](module-internals.md). Dated measurements below are +> and [module internals](module-internals.md). Campaign results are split: +> [local detector](synthid-detector-research.md), +> [SynthID source classifiers](synthid-classifiers.md), +> [mark removal](synthid-removal-research.md). Dated measurements below are > historical evidence and should not be read as current CLI defaults. This document covers how Google SynthID for images works mechanically, what it @@ -42,11 +46,58 @@ after generation, by a separate neural network. This means: - Replacing the generative model does not remove the watermarking capability. - The encoder/decoder pair can be updated independently of the generative model. +The detector's training target is narrower than provider or AI-image +classification. Equation 1 trains on watermarked `f(x)` as positive and the +corresponding unwatermarked distribution `x` as negative, with both drawn from +the same target image distribution. Equation 2 applies the same sampled +semantics-preserving transformation to both sides. Unrelated generators and +camera images are useful later for false-positive evaluation, but they do not +replace source-matched clean examples during signal-identifiability training. + +The decoder produces a dedicated detection logit whose threshold is calibrated +after training for a target false-positive rate. Payload recovery is a separate +problem, not the definition of the presence score. The paper additionally +describes two-sided conformal calibration for the `not watermarked` and +`watermarked` hypotheses, which permits a detector to abstain instead of forcing +an unsupported binary verdict. + The paper does not disclose the internal architecture of the encoder/decoder networks (layer types, capacity). The external variant SynthID-O is available to partners; the production internal variant is not published. -### 1.2 How it differs from classical DWT-DCT watermarks +### 1.2 Patent-backed architecture clues + +A related [DeepMind patent family](https://patents.google.com/patent/US12094474B1/en), +filed in 2023 and naming several authors of the SynthID-Image paper, describes +the likely architectural design space in more detail. Patent alternatives are +not proof that every option is deployed in production SynthID, so the following +items are constraints and research clues rather than implementation claims. + +The patent's image example forms a content-dependent residual, `x' = x + g(x)`. +It describes a U-Net-like watermark generator with convolution, attention, and +skip connections; a separately trained convolutional decoder; optional message +or secret-key injection into intermediate layers; and paired training on clean +and watermarked images under sampled differentiable transformations. It also +allows an image to be resized to a trained target size for watermarking and then +resized back to its original dimensions. + +Most importantly for blind detection, a +[continuation patent](https://patents.google.com/patent/US20250149048A1/en) +describes groups or ensembles of paired encoder/decoder networks. A decoder can +be trained not to recognize marks from another pair, while deployment can +select one pair or combine several outputs. This supplies a concrete mechanism +for coexisting watermark versions, provider-specific states, or distinct +codewords without requiring one universal pixel template. + +The fixed periodic carriers measured in this project should therefore be +treated as linear experts for particular observable states, not as the +definition of SynthID. A broader detector needs independently validated experts +for additional encoder states and transformations, followed by joint +false-positive calibration. The paper's separate detection logit and two-sided +conformal decision provide the correct target behavior: present, absent, or +abstain when neither hypothesis is supported. + +### 1.3 How it differs from classical DWT-DCT watermarks The open watermarks used by Stable Diffusion / SDXL / FLUX (via the `imwatermark` library) use classical **DWT-DCT** frequency-domain embedding: a @@ -70,7 +121,7 @@ embeds in fixed, predictable frequency bins, making it brittle to any operation that hits those bins (e.g., JPEG re-quantization wipes it cleanly at quality <= 90). -### 1.3 Payload capacity +### 1.4 Payload capacity SynthID-O (the external/partnership variant) encodes: @@ -90,7 +141,7 @@ paper separates watermark **detection** (is this watermarked?) from payload **recovery** (what does the payload say?): the detection path is what oracles like the Gemini app's "Verify with SynthID" exercise. -### 1.4 Where in the pipeline it lives +### 1.5 Where in the pipeline it lives ``` [Diffusion model] @@ -176,6 +227,232 @@ conversion (the body text of Section 10 is absent from the HTML). **What is known empirically from our own oracle-verified testing.** +A 2026-08-09 non-generative pilot found a promising Google phase-correlate, +but did not establish a releasable local detector or pixel-only remover. +The 2026-08-20/22 OpenAI campaign is recorded in +[synthid-detector-research.md](synthid-detector-research.md), +[synthid-classifiers.md](synthid-classifiers.md), and +[synthid-removal-research.md](synthid-removal-research.md). +JPEG q5 and 16-32 px phase structure survive as the official mark; a +quality-preserving local remover and a local SynthID detector for +photographs were not found. `gpt-image-2` source-matched flat pairs exist; +their residual does not transfer to photographs. The +best independently fitted spectral model relearned phase and magnitude from four of our +positives while using third-party candidate coordinates; its second frozen +epoch had zero false positives on 279 new exact-size external images and +detected the one new confirmed positive used for validation. An additional +3,000 upscaled photographs all fell outside the measured active-carrier support +and therefore count as abstentions, not negatives. The corpus is still far too +small and lacks same-provider hard negatives needed for a 0.1% FPR claim. +Repeating the spectral fit in six color spaces produced zero false positives +on the same 279-image comparison set for every branch. HSV had the best +observed worst-negative margin, but paired normalized negative scores did not +show a general improvement over RGB. Its apparent benefit came from saturation +and value; hue failed its channel-level separation check. Luminance-like +channels dominated YCbCr, YCoCg, opponent, and Lab. This narrows the next +hypothesis to intensity/contrast and HSV S+V projections rather than a +hue-specific carrier, but it does not add independent positives or certify an +operating point. +The resulting positive-only RGB plus S/V ensemble passed four +leave-one-positive-out checks, detected all five available positive controls, +and emitted no positive verdict on 330 newly collected exact-size images. +Almost all external images lacked sufficient measured carrier support and +therefore remained abstentions rather than proven negatives. +A later post-freeze challenge used three additional exact-size Google originals +with signed Google LLC C2PA and explicit SynthID-present assertions. The +ensemble abstained on all three. They do not replace matching-oracle pixel +labels, but this zero-of-three source-provenance result rejects the frozen +ensemble as a general Google detector and narrows it to an epoch- or +surface-specific correlate. + +A multi-epoch leave-one-positive-out refit recovered six of the eight total +Google positives under a strict RGB-plus-HSV decision; RGB alone ranked every +excluded positive above the 50-image calibration maximum but produced up to +eight false positives on 279 held-out external negatives. The signal therefore +transfers across the two source groups, but remains content-sensitive and +cannot be separated from provider or generation-pipeline attribution without +same-provider oracle-negative controls. + +Directly projecting out the ensemble phases cleared the local detector above +51 dB PSNR, but three frozen candidates remained detected by Gemini in a +healthy control session. A subsequent spatial analysis found that 81.25% of +the top 256 carrier bins lie on a lattice corresponding to a repeating 16x32 +pixel cell. Modulo-folding and subtracting the complete high-pass tile gave +phase-specific matched-control separation across all five Google positives at +at least 55.67 dB PSNR: aligned outputs cleared the local ensemble, while +one-pixel-shifted controls remained positive. The mild locally clearing tile +candidate nevertheless remained detected in a healthy Google Verify AI +session, so the tile correlate is not a sufficient removal loss. +Sparse subtraction reduced the local score sharply at more than 50 dB PSNR, +yet healthy Gemini sessions still detected SynthID. A one-off negative was +discarded because the same session also missed the source-positive control. +An OpenAI-specific 8-pixel phase candidate likewise changed its local score at +46.70 dB PSNR but remained `SynthID detected` in one frozen OpenAI Verify +check. No Google threshold or carrier was used in that experiment, and no +further public OpenAI checks were made because the documented verifier guidance +rules out repeated watermark-removal queries. + +A later local EOT pilot combined periodic full-frame residuals with JPEG-aware +optimization against provider-specific CNN surrogates. It produced candidates +that stayed below three local models after actual JPEG round trips, but the +models remain provider classifiers without same-provider oracle-negative +controls. The selected Google candidate retained 36.80 dB PSNR and 0.9241 +SSIM; the selected OpenAI candidate retained 34.97 dB and 0.9356. Both miss the +research fidelity gate. The OpenAI periodic residual also reduced all three +local model scores on 11 of 11 additional images before and after JPEG-90, but +neither provider candidate has a negative matching-provider oracle verdict, so +neither is an established remover. + +A later paired open-method control tested stationary wavelet and complex +spectral representations without treating another watermark as SynthID. On a +source-disjoint TrustMark P proxy, the SWT branch moved 59 of 60 transformed +clean/watermarked pairs in the correct direction but did not improve the +spatial model's aggregate AUC. Complex FFT maps were indistinguishable from +chance, and fixed phase scoring collapsed under crop. Wavelets therefore remain +a gated late-fusion ablation, while complex spectral analysis remains a +residual-discovery tool rather than a presence score. + +Expanding that proxy to four causally paired watermark families did not produce +a universal residual detector. TrustMark P, VideoSeal Y-256, DWT-DCT, and WAM +each used 90 source-matched images and random payloads; official decoders +validated which transformed positives still carried readable marks. A fixed +steganalysis residual bank and a neural model trained with cross-family +residual mixing, sign and phase augmentation, and equal-power phase-scrambled +hard negatives were evaluated leave-one-family-out. The four unseen-family +AUCs were 0.562, 0.474, 0.497, and 0.437. A network can therefore learn each +available encoder without learning a transferable class called "neural +watermark." This open-proxy route is not evidence for Google or OpenAI SynthID. + +The same conclusion held on a larger temporally split closed corpus. A +three-seed normalized-patch ensemble reached 0.884 AUC, but its threshold above +all validation negatives produced a test false positive and detected only +2.45% and 8.97% of the two positive-source classes. Naive full-field pooling, +late-fused SWT, robust patch aggregation, and a separate binary detection head +did not improve that low-FPR operating point. Vendor classification was highly +accurate among accepted positives, but it cannot rescue an unreliable presence +decision. Source labels therefore remain suitable for vendor triage and hard +negative evaluation, not for establishing a SynthID detector without +counterfactual or oracle watermark labels. + +A later low-content matrix also rejected wavelet energy and normalized +wavelet-shape classifiers: they separated real target outputs from artificial +flat, Gaussian, and phase-random controls, then accepted 61.6-100% of real +external negatives. The surviving branch is narrower. A provider-specific +1536x2816 phase carrier detected all four temporal-test positives, bounded +translation registration recovered all four one-pixel shifts, and the joint +phase/support rule produced zero positives on 194 frozen negatives. A +scale-and-translation discovery rule also produced zero positives on a +preregistered 3,000-image COCO challenge, but every COCO image was outside +carrier support and the scale threshold was selected post hoc. The result is +therefore a positive-only, geometry- and epoch-specific expert with abstention, +not a universal SynthID detector. Exact measurements and remaining calibration +gates are in the +[`detector and removal research plan`](synthid-detector-removal-plan.md#2026-08-10-low-content-controls-and-registered-phase-carrier). + +The next native-geometry experiment isolated a stronger mechanism. At +2048x2048, 108 selected spatial frequencies formed a 128-bin lattice, implying +a 16x16 periodic residual tile. Folding and averaging 16,384 tile repetitions +produced a spatial detector that accepted 29 of 30 test positives, none of 49 +calibration negatives, and none of 38 held-out test negatives. It also accepted +the same two of 182 earlier external-source images as the independent RGB and +HSV phase branches. Those cases count against operational source-label FPR, +but may contain the same carrier through an upstream encoder; only an oracle +can distinguish the two explanations. A normalized tile challenge accepted +none of 3,000 general images, while symmetric attacks showed strong resize but +limited JPEG and crop robustness. The pickle-free research implementation is +`scripts/synthid_periodic_tile_probe.py`; exact evidence and caveats are in the +[`2048 periodic-tile experiment`](synthid-detector-removal-plan.md#2026-08-10-2048-periodic-tile-detector). + +The historical 2048 reports retain aggregate counts and model hashes, but not +the 111 fitting paths. A later exact-1024 audit demonstrated that perceptual +siblings can cross an image-level split despite different file and decoded-pixel +hashes, so the old 2048 train/validation/test split cannot be retrospectively +cleared of that leakage mode. Its external control challenges remain valid; +its positive rates are conditional on the historical split. The next model +calibration must preserve content-group membership across every partition. + +An aligned-subtraction ablation strengthened the mechanism finding without +clearing the oracle gate. At a discovery-selected amplitude, it reversed both +the fixed-tile and independently fitted phase decisions on all 30 test images +at a median 53.74 dB PSNR and 0.99681 SSIM. The aligned edit reduced both scores +more than cyclic-shifted and orthogonal random tile controls on every paired +image. A shifted tile nevertheless suppressed the phase score enough to leave +only one accepted image, so these local reversals remain surrogate evidence, +not verified SynthID removal. The exact controls and caveats are recorded in +the linked experiment section. + +A later adaptive projection reduced that local cost substantially. It repeats +the frozen unit-norm 16x16 carrier over the decoded image and uses a bounded +scalar search for the smallest subtraction amplitude that reaches a requested +fixed-tile score. There is no generative model, spatial optimizer, or change of +image dimensions. Among the 30 historical 2048x2048 ablation sources, 27 were +positive under both the fixed-tile and independent phase rules before editing. +Searching only the scalar amplitude cleared both rules on all 27 at a median +59.83 dB PSNR and 0.99890 SSIM; the minima were 55.19 dB and 0.99705. Applying +the same selected amplitudes to a one-pixel cyclic shift of the carrier cleared +none of the 27. Of the 16 sources whose carrier survived an otherwise identical +JPEG-95 round trip, aligned projection cleared all 16 after JPEG-95 while the +shifted control cleared one. + +The same native-period edit works without resizing when dimensions are not +multiples of 16: repeat the carrier past both image boundaries and crop it to +the decoded geometry. A deterministic development challenge selected three +first-carrier-positive Spaces images from each of the 20 most frequent native +geometries, for 60 images total. Searching to a zero fixed-tile target cleared +60 of 60, compared with 5 of 60 norm-matched one-pixel-shifted controls. Median +fidelity was 60.37 dB PSNR and 0.99919 SSIM; the minima were 53.56 dB and +0.99581. A more conservative target score of -0.25 retained 60 of 60 native +local clearances at median 57.70 dB PSNR and 0.99863 SSIM, with minima of +52.38 dB and 0.99443. + +The frozen -0.25 rule was then evaluated once on one deterministic first-carrier +positive from every native geometry represented by that Spaces subset: 647 +images at 647 distinct decoded sizes. It reached the local target on all 647, +with no maximum-amplitude failure. Median fidelity was 57.67 dB PSNR and +0.99867 SSIM; the worst case was 51.78 dB and 0.99400. Among the 323 images +whose source carrier survived JPEG-95, the aligned candidate cleared 323 of +323 after the same codec round trip, compared with 58 of 323 one-pixel-shifted +controls. At the deeper transform margin, shifted controls also clear more +often natively, 153 of 647, so exact carrier phase remains causal but the edit +is no longer phase-exclusive on every content. The all-geometry corpus was +already inspected during detector development and contributes mechanism +coverage, not an independent generalization split. + +The conservative target also survived every tested transform for which the +same transformed source remained locally positive: 32 of 32 JPEG-95, 13 of 13 +JPEG-90, 8 of 8 JPEG-85, 15 of 15 WebP-95, and 54 of 54 0.75x resize round +trips. The shifted controls cleared 7, 1, 1, 5, and 17 of those respective +source-positive subsets. These results establish fast, geometry-independent +control over the recovered carrier and quantify the quality-versus-transport +margin. They do not establish SynthID removal. The previous provider check +showed that Google can remain positive after a locally favorable analytical +edit, and this adaptive recipe has not received a matching-provider negative +oracle verdict. It therefore remains a research candidate rather than a public +removal path. + +The reproducible local tool is +`scripts/synthid_adaptive_carrier_suppress.py`. It accepts one locally positive +image, writes a lossless PNG without overwriting existing files, and records +the input/output hashes, target, selected amplitude, scores, fidelity, and +runtime in JSON. Its default -0.25 target is the transform-margin setting above. +The tool intentionally remains outside the public package CLI and refuses +locally negative inputs; its output status names carrier suppression rather +than provider-verified removal. + +The immutable oracle-batch and result evaluator are implemented. On 2026-08-10, +four new 2048x2048 Gemini images were generated after the rule was frozen and +registered as a 20-request confirmatory batch. The first source group exhausted +the account's verification quota after five requests. The untouched source +returned Google C2PA Content Credentials without a separate SynthID verdict; +the lossless re-encode, aligned subtraction, and cyclic-shifted control all +still returned a Google AI signal. The orthogonal control was refused because +the quota had been exceeded, and the remaining 15 requests were not submitted. +This incomplete run is already negative evidence for the frozen pixel-only +recipe: the aligned candidate cleared the phase detector but remained positive +under the tile detector and the provider oracle. The local carrier expert +therefore ships only as a positive-only, exact-geometry detector; it is not a +universal SynthID detector or a remover. + A controlled study (June 2026, clean v0.8.6 with text/face protection OFF, native resolution on this repo's default SDXL pipeline) measured the minimum img2img strength that removes the SynthID pixel watermark, verified per image on @@ -186,14 +463,15 @@ Generated cleaned outputs are not committed; the table below is the durable record of the historical oracle verdicts. One third-party image from issue #14 was oracle-verified but is not committed. -**Oracle validation order: start with OpenAI.** When validating removal across -vendors, run the OpenAI arm first. `openai.com/verify` is more accessible than the -Gemini app -- fewer per-check restrictions, so it gives the fastest signal and is -the strongest candidate for automation (Playwright / Chrome MCP driving -`openai.com/verify`); the Gemini "Verify with SynthID" flow is more manual. This is -an ordering/throughput choice, not a substitution: each oracle only reads its own -vendor's SynthID (`openai.com/verify` is OpenAI-scoped), so Google content still -needs the Gemini app. +**Historical oracle validation order: start with OpenAI.** The June study used +the OpenAI web verifier first because it was more accessible than the Gemini app. +OpenAI now documents a Content Provenance API, but its usage guidance explicitly +rules out repeated queries for reverse engineering or watermark removal. New +adaptive experiments require separate authorization and must follow the oracle +boundary in the +[`detector and removal research plan`](synthid-detector-removal-plan.md). This is +not a cross-provider substitution: each oracle reads only its own vendor's +SynthID, so Google content still needs the Gemini flow. | Vendor | Images | Resolution(s) | Pipeline | Removed at | |--------|--------|---------------|----------|------------| @@ -265,11 +543,29 @@ notable because it is also the most powerful removal attack. The paper notes that diffusion regeneration "leaves a strong reconstruction signature from the diffusion prior." +A newer [MarkNull study](https://arxiv.org/abs/2608.10166), accepted at USENIX +Security 2026, evaluates a no-box latent-space attack on 20 Imagen-3 images and +checks the outputs with Gemini's SynthID verification flow. MarkNull, its +amortized variant, and most advanced regeneration or adversarial baselines +reached 100% attack success in that small evaluation; a simple VAE round trip +reached 0%. The per-image method inverts the input with a clean public diffusion +proxy, optimizes its latent to decorrelate the reconstructed initial noise, and +constrains LPIPS, MSE, and SSIM before decoding it again. + +This is important independent evidence that removal need not know SynthID's +carrier or decoder, but it is still a generative latent reconstruction, not the +pixel-only analytical path pursued here. The paper's milder SynthID setting +reports PSNR 25.36 dB and SSIM 0.80 at 100% attack success. Those values do not +meet this project's 40 dB median PSNR and 0.99 median SSIM release gate, despite +the paper's favorable composite quality score. MarkNull-A's reported 0.50-second +runtime and 6.3 GB VRAM are attractive as a future optional fallback, but they +do not establish a fast, visually lossless pixel-only remover. + --- ## 3. Detectability and verifier access -### 3.1 No public local detector +### 3.1 No public payload decoder The SynthID decoder is proprietary and not released: @@ -278,18 +574,78 @@ The SynthID decoder is proprietary and not released: > available to trusted testers." > -- Gowal et al., arXiv:2510.09263 -There is no public API, no released decoder weights, and no reproducible -algorithm for local detection. The verification service (SynthID Detector) is: +There are no released payload-decoder weights or public algorithm. Google +provides verification in Gemini and a limited SynthID Detector +portal. OpenAI now documents a synchronous Content Provenance API whose image +response contains separate C2PA and SynthID outcomes. That API is a remote, +OpenAI-scoped verifier, not a local decoder. Its documentation also says not to +use repeated queries to reverse-engineer, remove, or evade a watermark, so an +adaptive research loop requires separate authorization. + +Google's SynthID Detector service is: > "a verification portal" in early testing with "journalists and media > professionals" on a waitlist > -- deepmind.google/models/synthid/ -The external variant SynthID-O is available "through partnerships" only. Our -tool cannot locally detect SynthID presence or absence -- this is by design, -not a gap we can fill. +The external variant SynthID-O is available "through partnerships" only. This +package does not ship a local payload decoder. Research on a periodic lattice +expert is in [synthid-detector-research.md](synthid-detector-research.md). +A 2026-08-23 literature map (Gowal, AWPD/FSNet, PRC, reverse-SynthID) is +in that page's external-literature section. The product routes are signed +provenance and `verify-openai-synthid`. -### 3.2 How our tool recognizes SynthID from provenance +### 3.2 How our tool detects the supported carrier + +This heading is kept because older README and CLI links pointed here. The +package does not ship a local SynthID pixel detector. The periodic-lattice +expert is research-only under `scripts/synthid_runtime/` and +[synthid-detector-research.md](synthid-detector-research.md). It is not a +payload decoder and is not called from `identify` or the CLI. + +The product routes for SynthID are signed provenance, documented in the next +sections, and `verify-openai-synthid`. + +### 3.3 Official OpenAI pixel verification + +`remove-ai-watermarks verify-openai-synthid image.png +--acknowledge-upload` uses OpenAI's official Content Provenance API for the +OpenAI half of the production detector. This is not the weak local period-8 +research expert and it does not use C2PA as a proxy. + +Before making one request, the command strips AI provenance metadata into a +temporary PNG, JPEG, or WebP file and compares hashes of the decoded RGBA raster +before and after. It refuses the upload if any AI marker survived, if the format +or pixels changed, or if the sanitized file exceeds 50 MiB. It then reads +exactly one independent `type == "synthid"` response entry and ignores the C2PA +entry. The source is not modified. Tests deliberately cover C2PA-only positive +responses, pixel mutation, surviving metadata, malformed response shapes, and +documented access and rate-limit failures. + +The default SDK client fixes a 120-second request timeout and disables +automatic retries. One explicit acknowledgement therefore cannot silently +transmit the sanitized raster more than once. Timeout and connection failures +are errors, and request logs omit source paths and decoded-pixel hashes. + +A live 2026-08-14 web-verifier smoke used the same sanitization invariant. Two +metadata-stripped, pixel-identical OpenAI images at 1536 by 1024 and 1024 by +1536 both returned `SynthID detected` with `Content Credentials not detected`. +An oracle-positive Google SynthID image and a COCO photograph, sanitized through +the same path, both returned OpenAI `SynthID not detected` and no Content +Credentials. This directly validates pixel-only and provider-specific behavior +for the production design. Four files are only a functional smoke test, not a +false-positive calibration, and the credentialed SDK endpoint itself was not +called because no API key was available. + +This backend is deliberately excluded from `identify`: verification uploads +the sanitized raster to OpenAI, the endpoint is not eligible for Zero Data +Retention, and explicit acknowledgement is required. It is suitable for +individual supported OpenAI provenance checks, not adaptive detector fitting or +removal search. The API documentation prohibits repeated queries for watermark +reverse engineering or evasion. As with every detector in this project, +`not_detected` is absence of recognized evidence, not proof of human authorship. + +### 3.4 How our tool recognizes SynthID from provenance We recognize SynthID indirectly from supported C2PA evidence; this is not a pixel watermark decode. Google states that all media generated by its tools is @@ -310,28 +666,26 @@ This is why: - A quiet `identify` output is not proof that SynthID was removed -- it only means the metadata signal is gone. -### 3.3 Oracle scope: each vendor detects only their own +### 3.5 Oracle scope: each vendor detects only their own -From openai.com/research/verify (verbatim, verified 2026-05-31): - -> "OpenAI generation signals will only be detected if the image was generated -> with our tools." -> "Content could also still be AI-generated by another company's model, which -> the tool currently does not detect." +OpenAI's current Content Provenance API documentation says it checks supported +OpenAI signals and is not a general-purpose AI detector. Google's current Gemini +documentation likewise says Gemini recognizes SynthID from Google AI tools, +not other companies' payloads. SynthID technology is used by multiple vendors, but each verifier is keyed to its own payload: -| Oracle | Detects | Does NOT detect | -|-------------------------------|------------------|-------------------------| -| Gemini app "Verify with SynthID" | Google SynthID | OpenAI SynthID | -| openai.com/research/verify | OpenAI SynthID | Google SynthID | +| Oracle | Detects | Does not detect | +| --- | --- | --- | +| Gemini app verification | Google SynthID | OpenAI SynthID | +| OpenAI Content Provenance API or web verifier | Supported OpenAI SynthID | Google SynthID | A Google-SynthID image reads clean on openai.com/verify. An OpenAI image reads clean in the Gemini oracle. They are different payloads within the same framework. -### 3.4 Video verification and attack harness +### 3.6 Video verification and attack harness Gemini's built-in verification flow reports whether and where it detects Google SynthID in a video. This remains a proprietary oracle: invoke `@synthid`, use @@ -560,7 +914,8 @@ photoreal, `sdxl` on flat graphics, the §5.1 content-x-pipeline table), BUT on hard case (flat fills) `sdxl` is the WEAKER remover (plain img2img barely perturbs a flat region at low strength), so it needs AT LEAST controlnet's strength -- the certified floor is therefore the right floor for `sdxl` too. This is a MARGIN argument -for `sdxl`, not a separate certification (no local SynthID detector to self-verify). +for `sdxl`, not a separate certification (the tested geometries are outside the +current local detector's scope). The higher strength costs little quality where it matters, because `controlnet` is now the default pipeline, so `sdxl` is reached only via an explicit `--pipeline sdxl` (a deliberate opt-down), where over-regeneration has no faces/text to damage. @@ -620,8 +975,10 @@ Two constraints on reading this: subjects.** JPEG re-encoding removes C2PA metadata but does NOT remove the SynthID pixel watermark (verified June 2026 on issue #14 pic3). Do not dismiss these as "not faithful originals" for SynthID-removal tests. -- **The correct oracle for OpenAI images is openai.com/verify**, not the Gemini - app. The two oracles detect different payloads. +- **The correct oracle for OpenAI images is an authorized OpenAI provenance + verifier**, not the Gemini app. OpenAI now documents both a web tool and a + Content Provenance API; the API's published use restrictions still apply. + The OpenAI and Google oracles detect different payloads. - **A quiet `identify` output after processing is not proof of removal.** It means the provenance evidence is gone. The pixel watermark state is unknown without an oracle check. @@ -669,7 +1026,7 @@ random (unset) seed differed between runs. So **0.15 is the borderline floor for controlnet photoreal, not a robust guarantee**: at the threshold the same image+settings can pass or fail run-to-run. img2img runs with `seed=None` (random) unless `--seed` is passed, so a removal SERVICE gets a coin-flip near threshold and -has no local SynthID detector to self-verify. +has no applicable local SynthID detector at these geometries to self-verify. **Controlnet strength ladder on the two photoreal images (oracle, `--auto`, `--max-resolution 1536`):** @@ -684,7 +1041,7 @@ has no local SynthID detector to self-verify. non-deterministic borderline); both photoreal survivors cleared at 0.20. Honest caveat: 0.20 is one confirming run WITH margin, not an N-run repeatability proof -- for a removal service, add a little more margin or validate repeatability, since -there is no local SynthID detector to self-check. **Implications:** (1) the +these geometries are outside the current local detector's scope. **Implications:** (1) the content×pipeline table above conflates a borderline/non-deterministic 0.15 result with deterministic content behavior -- the photoreal-survives-controlnet effect is solid at 0.10 but at 0.15 it is near-threshold noise; (2) for reliable removal pick @@ -756,8 +1113,8 @@ reproducible verification requires a fixed seed. Forensic Stealth in Generative-AI Watermark Removal.** arXiv:2605.09203. https://arxiv.org/abs/2605.09203 -5. OpenAI. **Verify tool for AI-generated images.** openai.com/research/verify. - Accessed 2026-05-31. +5. OpenAI. **Content provenance.** + https://developers.openai.com/api/docs/guides/content-provenance 6. Google. **Verify AI-generated images, videos, and audio.** https://support.google.com/gemini/answer/16722517 @@ -770,6 +1127,99 @@ reproducible verification requires a fixed seed. Watermarking.** arXiv:2505.21620. https://arxiv.org/abs/2505.21620 +9. OpenAI. **ChatGPT Images 2.0 system card.** + https://deploymentsafety.openai.com/chatgpt-images-2-0/automated-evaluations-and-adversarial-testing + +10. Cao et al. (2026). **MarkNull: Model-Agnostic Watermark Removal in + AI-Generated Images via On-Manifold Latent Manipulation.** USENIX Security + 2026, arXiv:2608.10166. https://arxiv.org/abs/2608.10166 + +11. Ao, Du, Wang, Chen, Lu (2026). **AWPD: Frequency Shield Network for + Agnostic Watermark Presence Detection.** arXiv:2603.06723. + https://arxiv.org/abs/2603.06723 + +12. Gunn, Zhao, Song (2025). **An Undetectable Watermark for Generative + Image Models.** ICLR 2025, arXiv:2410.07369. + https://arxiv.org/abs/2410.07369 + +13. Francati, Goonatilake, Pawar, Venturi, Ateniese (2026). **The Coding + Limits of Robust Watermarking for Generative Models.** IEEE EuroS&P 2026, + arXiv:2509.10577. https://arxiv.org/abs/2509.10577 + +14. Kassis and Hengartner (2025). **UnMarker: A Universal Attack on + Defensive Image Watermarking.** IEEE S&P 2025, arXiv:2405.08363. + https://arxiv.org/abs/2405.08363 + +15. Liu et al. (2025). **Image Watermarks are Removable Using Controllable + Regeneration from Clean Noise.** ICLR 2025, arXiv:2410.05470. + https://arxiv.org/abs/2410.05470 + +16. An et al. (2024). **WAVES: Benchmarking the Robustness of Image + Watermarks.** ICML 2024, arXiv:2401.08573. + https://arxiv.org/abs/2401.08573 + +17. Bui, Agarwal, Collomosse (2023). **TrustMark: Universal Watermarking + for Arbitrary Resolution Images.** arXiv:2311.18297. + https://arxiv.org/abs/2311.18297 + +18. Dathathri et al. (2024). **Scalable watermarking for identifying large + language model outputs.** Nature 634, 818-823. + https://www.nature.com/articles/s41586-024-08025-4 + +19. DeepMind. **US12094474B1** and continuation **US20250149048A1**. + https://patents.google.com/patent/US12094474B1/en + +20. Weatherbed (2026-04-14). **Has Google's AI watermarking system been + reverse-engineered?** The Verge. + https://www.theverge.com/ai-artificial-intelligence/911579/google-synthid-ai-watermarking-system-reverse-engineered + +21. Cox, Kilian, Leighton, Shamoon (1997). **Secure spread spectrum + watermarking for multimedia.** IEEE Trans. Image Process. 6(12). + +22. Wen, Kirchenbauer, Geiping, Goldstein (2023). **Tree-Ring Watermarks.** + NeurIPS 2023, arXiv:2305.20030. https://arxiv.org/abs/2305.20030 + +23. Ojha, Li, Lee (2023). **Towards Universal Fake Image Detectors that + Generalize Across Generative Models.** CVPR 2023, arXiv:2302.10174. + https://arxiv.org/abs/2302.10174 + +24. Corvi, Cozzolino, Poggi, Nagano, Verdoliva (2023). **Intriguing + properties of synthetic images.** CVPRW 2023, arXiv:2304.06408. + https://arxiv.org/abs/2304.06408 + +25. Zhong, Xu, Zou (2026). **Color Matters: Demosaicing-Guided Color + Correlation Training.** arXiv:2601.22778. + https://arxiv.org/abs/2601.22778 + +26. Baluja (2017). **Hiding Images in Plain Sight: Deep Steganography.** + NeurIPS 2017. + +27. Holub, Fridrich, Denemark (2014). **Universal distortion function for + steganography in an arbitrary domain.** EURASIP J. Information Security. + +28. Jing et al. (2021). **HiNet: Deep Image Hiding by Invertible Network.** + ICCV 2021. + +29. Yang et al. (2024). **Gaussian Shading: Provable Performance-Lossless + Image Watermarking for Diffusion Models.** CVPR 2024, arXiv:2404.04956. + https://arxiv.org/abs/2404.04956 + +30. Farid (2009). **Exposing digital forgeries from JPEG ghosts.** IEEE + Trans. Information Forensics and Security 4(1). + +31. Wang et al. (2023). **DIRE for Diffusion-Generated Image Detection.** + ICCV 2023, arXiv:2303.09295. https://arxiv.org/abs/2303.09295 + +32. Wang, Wang, Zhang, Owens, Efros (2020). **CNN-generated images are + surprisingly easy to spot... for now.** CVPR 2020, arXiv:1912.11035. + https://arxiv.org/abs/1912.11035 + +33. Ó Ruanaidh and Pun (1998). **Rotation, scale and translation invariant + spread spectrum digital image watermarking.** Signal Processing 66(3). + +34. Réfrégier and Javidi (1995). **Optical image encryption based on input + plane and Fourier plane random encoding.** Optics Letters 20(7). + **Google floor on qwen-zimage (0.27.2, remeasured 2026-08-25).** The bracket above did not survive the full production path: on the 4.33 MP CJK-sign fixture (visible-stage sparkle removal -> qwen-zimage seed 0 at the curve's 0.154 top -> resize-back -> diff --git a/docs/verification-plan.md b/docs/verification-plan.md index 4f531ac..1053daa 100644 --- a/docs/verification-plan.md +++ b/docs/verification-plan.md @@ -184,7 +184,7 @@ Three harness rules are load-bearing and must not be relaxed: score a mark only crop's adjudication scope; take provenance from metadata, never from labels; and never report recall from the detector-sampled set. -## Tier D -- external oracles (manual, not automatable here) +## Tier D -- external oracles Proprietary watermark removal cannot be verified locally by design -- no public decoder exists. Each vendor has its own oracle and it covers only that vendor's content: @@ -195,6 +195,13 @@ needs a pixel-identical metadata-stripped control: the control must lose `C2PA` remaining `Watermark`-positive before a candidate's negative result can be attributed to pixel regeneration. A quiet metadata proxy is **not** proof the pixel watermark is gone. +OpenAI's API documentation says not to use repeated queries to reverse-engineer, remove, +or evade a watermark. Using it as an adaptive research oracle therefore requires explicit +authorization. Without that authorization it must not become a training loss, search loop, +or automated removal gate. The provider-specific detector and pixel-only removal research +protocol is in [`synthid-detector-removal-plan.md`](synthid-detector-removal-plan.md). + + Scope honestly: this tier certifies strength floors on a handful of images per vendor, and that is all it can do. See `docs/synthid.md`. diff --git a/pyproject.toml b/pyproject.toml index 91cf5a0..e6a7687 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -151,6 +151,11 @@ classify = [ "tokenizers>=0.22,<0.23", "huggingface-hub>=0.20.0", ] +# Official remote OpenAI SynthID verification. The command strips AI provenance +# metadata and proves pixel identity before upload; it is never called implicitly. +verify = [ + "openai>=2.52.0", +] # Universal region eraser backend -- big-LaMa via onnxruntime (Carve/LaMa-ONNX, # Apache-2.0). CPU, no torch. Model (~200 MB) is downloaded on first use and # cached by huggingface_hub; it is never bundled in this repo. The default cv2 @@ -183,7 +188,7 @@ dev = [ ] # ``qwen-zimage`` already pulls ``diffusion``; naming both would suggest diffusion is # independently sufficient for a removal, which it is not. -all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan,classify]"] +all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan,classify,verify]"] [project.scripts] remove-ai-watermarks = "remove_ai_watermarks.cli:main" @@ -216,13 +221,13 @@ exclude = [ [tool.pytest.ini_options] testpaths = ["tests"] -pythonpath = ["src"] +pythonpath = ["src", "scripts"] addopts = "-v --tb=short" [tool.ruff] target-version = "py311" line-length = 120 -exclude = ["_refs"] +exclude = ["_refs", ".local-eval"] extend-exclude = ["*.md"] [tool.ruff.lint] @@ -233,7 +238,7 @@ ignore = [ ] [tool.ruff.lint.per-file-ignores] -"scripts/*.py" = ["G004", "S108", "S310", "T20"] +"scripts/**/*.py" = ["G004", "S108", "S310", "T20"] "tests/*.py" = ["ANN", "S101", "S105", "S106", "S108"] "src/remove_ai_watermarks/_internal/watermark_remover.py" = ["S603", "S606", "S607"] # nvidia-smi capability probe "src/remove_ai_watermarks/_internal/c2pa.py" = ["S110"] # try-except-pass for corrupt file handling @@ -245,7 +250,7 @@ indent-style = "space" [tool.pyright] pythonVersion = "3.11" typeCheckingMode = "strict" -exclude = ["_refs"] +exclude = ["_refs", ".local-eval"] [[tool.pyright.executionEnvironments]] root = "tests" diff --git a/scripts/synthid_adaptive_carrier_suppress.py b/scripts/synthid_adaptive_carrier_suppress.py new file mode 100644 index 0000000..8eae6a6 --- /dev/null +++ b/scripts/synthid_adaptive_carrier_suppress.py @@ -0,0 +1,197 @@ +"""Suppress the recovered periodic carrier without image regeneration. + +This research tool controls the project's local fixed-template score. A local +score reversal is not evidence that a provider SynthID verifier will stop +detecting the image. +""" + +from __future__ import annotations + +import json +import logging +import math +import sys +import time +from pathlib import Path +from typing import TYPE_CHECKING, Any + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +import click +from PIL import Image +from synthid_pixel_attack import load_rgb, measure # pyright: ignore[reportUnknownVariableType] +from synthid_research_manifest import artifact_sha256 +from synthid_runtime.synthid_detector import ( + TILE_THRESHOLD, + _geometry_supported, # pyright: ignore[reportPrivateUsage] + _load_template, # pyright: ignore[reportPrivateUsage] + folded_template_score, +) +from synthid_tile_attack import subtract_tiled_template + +if TYPE_CHECKING: + from numpy.typing import NDArray + +log = logging.getLogger(__name__) + + +def apply_template(pixels: NDArray[Any], template: NDArray[Any], *, amplitude: float) -> NDArray[Any]: + """Subtract AMPLITUDE times periodic TEMPLATE from arbitrary RGB PIXELS.""" + return subtract_tiled_template(pixels, template, strength=amplitude) + + +def carrier_score(pixels: NDArray[Any], template: NDArray[Any], sigma: float) -> float: + """Return the local fixed-template carrier score for PIXELS.""" + score, _folded = folded_template_score(pixels, template, sigma) + return score + + +def find_minimum_amplitude( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, + *, + target_score: float, + maximum_amplitude: float, + iterations: int, +) -> tuple[float, NDArray[Any], float]: + """Return the smallest searched amplitude whose score reaches TARGET_SCORE.""" + if not math.isfinite(target_score): + raise ValueError("target score must be finite") + if not math.isfinite(maximum_amplitude) or maximum_amplitude <= 0.0: + raise ValueError("maximum amplitude must be finite and positive") + if iterations < 1: + raise ValueError("iterations must be positive") + maximum_pixels = apply_template(pixels, template, amplitude=maximum_amplitude) + maximum_score = carrier_score(maximum_pixels, template, sigma) + if maximum_score > target_score: + raise ValueError( + f"maximum amplitude {maximum_amplitude:g} reached score {maximum_score:.6f}, " + f"above target {target_score:.6f}" + ) + + low = 0.0 + high = maximum_amplitude + best_pixels = maximum_pixels + best_score = maximum_score + for _iteration in range(iterations): + middle = (low + high) / 2.0 + candidate = apply_template(pixels, template, amplitude=middle) + candidate_score = carrier_score(candidate, template, sigma) + if candidate_score <= target_score: + high = middle + best_pixels = candidate + best_score = candidate_score + else: + low = middle + return high, best_pixels, best_score + + +def suppress_carrier( + pixels: NDArray[Any], + *, + target_score: float = -0.25, + maximum_amplitude: float = 40.0, + iterations: int = 8, +) -> tuple[NDArray[Any], dict[str, float | int | str]]: + """Suppress a locally detected carrier and return pixels plus measurements.""" + height, width = pixels.shape[:2] + if not _geometry_supported(width, height): + raise ValueError(f"unsupported decoded geometry: {width}x{height}") + if target_score >= TILE_THRESHOLD: + raise ValueError(f"target score must be below the detector threshold {TILE_THRESHOLD:.6f}") + template, sigma, _model_height, _model_width, tile_height, tile_width = _load_template() + original_score = carrier_score(pixels, template, sigma) + if original_score < TILE_THRESHOLD: + raise ValueError( + f"local carrier is not detected: score {original_score:.6f} is below threshold {TILE_THRESHOLD:.6f}" + ) + + started = time.perf_counter() + amplitude, candidate, candidate_score = find_minimum_amplitude( + pixels, + template, + sigma, + target_score=target_score, + maximum_amplitude=maximum_amplitude, + iterations=iterations, + ) + quality = measure(pixels, candidate, name="adaptive-carrier", path=Path("")) + return candidate, { + "status": "local_carrier_suppressed", + "detector_scope": "local fixed-template carrier, not provider-verified SynthID removal", + "width": width, + "height": height, + "tile_height": tile_height, + "tile_width": tile_width, + "threshold": TILE_THRESHOLD, + "target_score": target_score, + "original_score": original_score, + "candidate_score": candidate_score, + "amplitude": amplitude, + "maximum_amplitude": maximum_amplitude, + "iterations": iterations, + "residual_rms": quality.residual_rms, + "psnr_db": quality.psnr_db, + "ssim": quality.ssim, + "changed_pixel_fraction": quality.changed_pixel_fraction, + "elapsed_seconds": time.perf_counter() - started, + } + + +@click.command() +@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("output", type=click.Path(dir_okay=False, path_type=Path)) +@click.option("--target-score", type=float, default=-0.25, show_default=True) +@click.option("--maximum-amplitude", type=click.FloatRange(min=0.0, min_open=True), default=40.0, show_default=True) +@click.option("--iterations", type=click.IntRange(min=1), default=8, show_default=True) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path)) +def main( + source: Path, + output: Path, + target_score: float, + maximum_amplitude: float, + iterations: int, + report_out: Path | None, +) -> None: + """Write a lossless PNG with the recovered local carrier suppressed.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + if output.suffix.lower() != ".png": + raise click.BadParameter("output must use the .png extension", param_hint="output") + report_path = report_out or output.with_suffix(".json") + for path in (output, report_path): + if path.exists(): + raise click.ClickException(f"refusing to overwrite existing file: {path}") + try: + candidate, report = suppress_carrier( + load_rgb(source), # pyright: ignore[reportUnknownArgumentType] + target_score=target_score, + maximum_amplitude=maximum_amplitude, + iterations=iterations, + ) + except ValueError as error: + raise click.ClickException(str(error)) from error + output.parent.mkdir(parents=True, exist_ok=True) + report_path.parent.mkdir(parents=True, exist_ok=True) + Image.fromarray(candidate, mode="RGB").save(output, format="PNG", compress_level=9) + report.update( + { + "source": str(source.resolve()), + "source_sha256": artifact_sha256(source), + "output": str(output.resolve()), + "output_sha256": artifact_sha256(output), + } + ) + report_path.write_text(json.dumps(report, indent=2, allow_nan=False) + "\n", encoding="utf-8") + log.info( + "Suppressed local carrier %.6f -> %.6f at %.2f dB PSNR; wrote %s", + report["original_score"], + report["candidate_score"], + report["psnr_db"], + output, + ) + log.info("Research caveat: this is not provider-verified SynthID removal") + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_affine_lattice_probe.py b/scripts/synthid_affine_lattice_probe.py new file mode 100644 index 0000000..683649c --- /dev/null +++ b/scripts/synthid_affine_lattice_probe.py @@ -0,0 +1,1636 @@ +"""Probe a periodic carrier lattice and codeword with split confirmation. + +The synchronization parameters are selected on one checkerboard of image +patches and scored on the disjoint checkerboard. The statistics measure complex +phase coherence and a content-whitened multichannel template match after +correcting every patch for its global origin. They do not use filenames, +metadata, or scene-class features. +""" + +from __future__ import annotations + +import json +import logging +import math +import sys +from dataclasses import asdict, dataclass, replace +from pathlib import Path +from typing import TYPE_CHECKING, Any + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +import click +import cv2 +import numpy as np +from synthid_pixel_attack import jpeg_round_trip, load_rgb +from synthid_runtime._synthid_confirmation import ( + registered_confirmation_passes as runtime_registered_confirmation_passes, +) +from synthid_runtime._synthid_registered import RegisteredComponents, registered_components +from synthid_runtime.synthid_detector import fold_residual_template, folded_template_score, unit_tile + +if TYPE_CHECKING: + from numpy.typing import NDArray + +log = logging.getLogger(__name__) + +FIXED_CANDIDATE_THRESHOLD = 0.28 +PATCH_SHIFT_MIN_MARGIN = 0.45 +PATCH_SHIFT_STRONG_MARGIN = 1.0 +PATCH_SHIFT_MIN_Z = 2.5 +OPPONENT_REGISTERED_FIXED_MIN = 0.16 +OPPONENT_REGISTERED_RED_GREEN_MIN = 0.60 +OPPONENT_REGISTERED_BLUE_YELLOW_MIN = 0.55 +SAME_IMAGE_NULL_OFFSETS = ( + -2.0, + -1.75, + -1.5, + -1.25, + -1.0, + -0.75, + -0.5, + -0.35, + 0.35, + 0.5, + 0.75, + 1.0, + 1.25, + 1.5, + 1.75, + 2.0, +) + + +def _webp_round_trip(pixels: NDArray[Any], quality: int) -> NDArray[Any]: + """Apply one in-memory WebP encode/decode while returning RGB pixels.""" + if quality < 1 or quality > 101: + raise ValueError("WebP quality must be between 1 and 101") + success, encoded = cv2.imencode( + ".webp", + cv2.cvtColor(pixels, cv2.COLOR_RGB2BGR), + [cv2.IMWRITE_WEBP_QUALITY, quality], + ) + if not success: + raise RuntimeError("WebP encoding failed") + decoded = cv2.imdecode(encoded, cv2.IMREAD_COLOR) + if decoded is None: + raise RuntimeError("WebP decoding failed") + return cv2.cvtColor(decoded, cv2.COLOR_BGR2RGB) + + +@dataclass(frozen=True) +class LatticeScore: + """One select-confirm reciprocal-lattice observation.""" + + selected_period: float + selected_rotation_degrees: float + selected_orientation_degrees: int + selected_horizontal_reflection: bool + selected_deskew_degrees: float + deskew_candidate_count: int + deskew_selection_coherence: float + deskew_direct_selection_match: float + deskew_direct_confirmation_match: float + deskew_direct_joint_match: float + peak_period: float + peak_rotation_degrees: float + peak_selection_coherence: float + selection_coherence: float + selection_candidate_p95: float + selection_candidate_p99: float + selection_excess_p95: float + selection_excess_p99: float + confirmation_coherence: float + confirmation_candidate_p95: float + confirmation_candidate_p99: float + confirmation_excess_p95: float + confirmation_excess_p99: float + joint_coherence: float + joint_excess_p99: float + selected_shift_y: int + selected_shift_x: int + selection_codeword: float + confirmation_codeword: float + confirmation_codeword_shift_p95: float + confirmation_codeword_shift_p99: float + joint_codeword: float + unknown_codeword_shift_y: int + unknown_codeword_shift_x: int + unknown_codeword_selection: float + unknown_codeword_confirmation: float + unknown_codeword_fixed_confirmation: float + unknown_codeword_fixed_all: float + unknown_codeword_confirmation_p95: float + unknown_codeword_confirmation_p99: float + unknown_codeword_excess_p99: float + amplitude_shift_y: int + amplitude_shift_x: int + selection_amplitude: float + confirmation_amplitude: float + joint_amplitude: float + selection_whitened_match: float + confirmation_whitened_match: float + joint_whitened_match: float + amplitude_candidate_count: int + amplitude_rerank_count: int + canonical_template_score: float + canonical_registered_template_score: float + selection_patches: int + confirmation_patches: int + + +@dataclass(frozen=True) +class _AmplitudeScore: + """Amplitude-aware select-confirm score for one period candidate.""" + + period: float + selection: float + confirmation: float + shift_y: int + shift_x: int + unaligned_full: float + aligned_full: float + + +@dataclass(frozen=True) +class SameImageNullScore: + """Target-period coherence relative to neighboring periods in one image.""" + + target_period: float + target_selection_coherence: float + target_confirmation_coherence: float + joint_coherence: float + selection_off_period_max: float + confirmation_off_period_max: float + joint_excess: float + off_period_count: int + + +@dataclass(frozen=True) +class PatchShiftConsensusScore: + """Robust per-patch agreement on one cyclic carrier phase.""" + + period: float + selected_shift_y: int + selected_shift_x: int + selection_trimmed_z: float + confirmation_trimmed_z: float + joint_trimmed_z: float + selection_support_fraction: float + confirmation_support_fraction: float + joint_support_fraction: float + selection_patches: int + confirmation_patches: int + + +@dataclass(frozen=True) +class OpponentRegisteredScore: + """Scale-registered opponent-color carrier observation.""" + + selected_period: float + spectral_period: float + spectral_score: float + fixed_score: float + red_green_spatial: float + blue_yellow_spatial: float + candidate_count: int + + @property + def decision_score(self) -> float: + """Return the minimum normalized research gate margin.""" + return min( + self.fixed_score / OPPONENT_REGISTERED_FIXED_MIN, + self.red_green_spatial / OPPONENT_REGISTERED_RED_GREEN_MIN, + self.blue_yellow_spatial / OPPONENT_REGISTERED_BLUE_YELLOW_MIN, + ) + + +def fixed_candidate_passes(score: float) -> bool: + """Return the frozen precision-first fixed-period candidate verdict.""" + return math.isfinite(score) and score >= FIXED_CANDIDATE_THRESHOLD + + +def registered_confirmation_passes(score: LatticeScore) -> bool: + """Return the frozen period-aware split-confirmation verdict.""" + return runtime_registered_confirmation_passes( + score.selected_period, + score.joint_coherence, + score.joint_amplitude, + score.unknown_codeword_fixed_confirmation, + ) + + +def patch_shift_recovery_passes( + *, + amplitude_margin: float, + high_band_margin: float, + periods_agree: bool, + confirmation_passes: bool, + joint_trimmed_z: float, +) -> bool: + """Apply the frozen research-only content-adaptive recovery rule.""" + margins = (amplitude_margin, high_band_margin) + return ( + all(math.isfinite(value) for value in (*margins, joint_trimmed_z)) + and periods_agree + and confirmation_passes + and min(margins) >= PATCH_SHIFT_MIN_MARGIN + and max(margins) >= PATCH_SHIFT_STRONG_MARGIN + and joint_trimmed_z >= PATCH_SHIFT_MIN_Z + ) + + +def _opponent_channels(values: NDArray[Any]) -> NDArray[Any]: + """Return Green, Red-minus-Green, and Blue-minus-Yellow channels.""" + red = values[:, :, 0] + green = values[:, :, 1] + blue = values[:, :, 2] + return np.stack((green, red - green, blue - 0.5 * (red + green)), axis=2) + + +def _opponent_color_pair(values: NDArray[Any]) -> NDArray[Any]: + """Return the large-carrier Red-Green and Blue-Yellow planes.""" + red = values[:, :, 0] + green = values[:, :, 1] + blue = values[:, :, 2] + return np.stack((red - green, blue - 0.5 * (red + green)), axis=2) + + +def template_harmonics(template: NDArray[Any], count: int = 16) -> tuple[NDArray[Any], NDArray[Any], NDArray[Any]]: + """Return strong unique half-plane harmonics and their channel weights.""" + if template.ndim != 3 or template.shape[2] != 3: + raise ValueError("template must have shape (height, width, 3)") + if count < 1: + raise ValueError("count must be positive") + opponent = _opponent_channels(np.asarray(template, dtype=np.float64)) + spectrum = np.fft.fft2(opponent, axes=(0, 1)) + height, width = template.shape[:2] + candidates: list[tuple[float, int, int, int, int]] = [] + for row in range(height): + signed_row = row if row <= height // 2 else row - height + for column in range(width): + signed_column = column if column <= width // 2 else column - width + if signed_row < 0 or (signed_row == 0 and signed_column <= 0): + continue + power = float(np.sum(np.abs(spectrum[row, column]) ** 2)) + candidates.append((power, signed_row, signed_column, row, column)) + candidates.sort(reverse=True) + selected = candidates[:count] + harmonics = np.asarray([(row, column) for _power, row, column, _y, _x in selected], dtype=np.float64) + coefficients = np.asarray([spectrum[y, x] for _power, _row, _column, y, x in selected]) + weights = np.abs(coefficients) + weight_sum = float(np.sum(weights)) + if weight_sum <= 0.0: + raise ValueError("template has no nonzero periodic harmonics") + coefficient_units = np.divide( + coefficients, + np.abs(coefficients), + out=np.zeros_like(coefficients), + where=np.abs(coefficients) > 1e-12, + ) + return harmonics, weights / weight_sum, coefficient_units + + +def _patch_origins(height: int, width: int, patch_size: int, grid_size: int) -> list[tuple[int, int, int]]: + if patch_size < 32 or height < patch_size or width < patch_size: + raise ValueError("image must contain at least one patch of the requested size") + if grid_size < 2: + raise ValueError("grid size must be at least two") + y_values = np.linspace(0, height - patch_size, min(grid_size, height // patch_size), dtype=np.int64) + x_values = np.linspace(0, width - patch_size, min(grid_size, width // patch_size), dtype=np.int64) + origins = [] + for y_index, y in enumerate(np.unique(y_values)): + for x_index, x in enumerate(np.unique(x_values)): + origins.append((int(y), int(x), (y_index + x_index) % 2)) + if {group for _y, _x, group in origins} != {0, 1}: + raise ValueError("image geometry does not provide two patch groups") + return origins + + +def _bilinear_sample( + spectrum: NDArray[Any], + y: NDArray[Any], + x: NDArray[Any], +) -> NDArray[Any]: + height, width = spectrum.shape + y_floor = np.floor(y) + x_floor = np.floor(x) + y0 = y_floor.astype(np.int64) % height + x0 = x_floor.astype(np.int64) % width + y1 = (y0 + 1) % height + x1 = (x0 + 1) % width + dy = y - y_floor + dx = x - x_floor + return ( + spectrum[y0, x0] * (1.0 - dy) * (1.0 - dx) + + spectrum[y1, x0] * dy * (1.0 - dx) + + spectrum[y0, x1] * (1.0 - dy) * dx + + spectrum[y1, x1] * dy * dx + ) + + +def _candidate_frequencies( + periods: NDArray[Any], + rotations_degrees: NDArray[Any], + harmonics: NDArray[Any], +) -> tuple[NDArray[Any], NDArray[Any], NDArray[Any], NDArray[Any]]: + period_grid, rotation_grid = np.meshgrid(periods, rotations_degrees, indexing="ij") + flat_periods = period_grid.ravel() + flat_rotations = rotation_grid.ravel() + angles = np.deg2rad(flat_rotations) + cosine = np.cos(angles)[:, None] + sine = np.sin(angles)[:, None] + base_y = harmonics[None, :, 0] / flat_periods[:, None] + base_x = harmonics[None, :, 1] / flat_periods[:, None] + frequencies_y = sine * base_x + cosine * base_y + frequencies_x = cosine * base_x - sine * base_y + return flat_periods, flat_rotations, frequencies_y, frequencies_x + + +def _patch_unit_values( + pixels: NDArray[Any], + origin_y: int, + origin_x: int, + patch_size: int, + frequencies_y: NDArray[Any], + frequencies_x: NDArray[Any], +) -> NDArray[Any]: + patch = np.asarray( + pixels[origin_y : origin_y + patch_size, origin_x : origin_x + patch_size], + dtype=np.float32, + ) + channels = _opponent_channels(patch) + window_1d = np.hanning(patch_size).astype(np.float32) + window = window_1d[:, None] * window_1d[None, :] + sampled = np.empty((*frequencies_y.shape, 3), dtype=np.complex128) + sample_y = frequencies_y * patch_size + sample_x = frequencies_x * patch_size + for channel in range(3): + residual = channels[:, :, channel] + residual = residual - cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=1.0, + sigmaY=1.0, + borderType=cv2.BORDER_REFLECT_101, + ) + spectrum = np.fft.fft2(residual * window) + sampled[:, :, channel] = _bilinear_sample(spectrum, sample_y, sample_x) + phase_correction = np.exp(-2j * math.pi * (frequencies_y * origin_y + frequencies_x * origin_x)) + sampled *= phase_correction[:, :, None] + magnitudes = np.abs(sampled) + return np.divide(sampled, magnitudes, out=np.zeros_like(sampled), where=magnitudes > 1e-12) + + +def _coherence(unit_values: list[NDArray[Any]], weights: NDArray[Any]) -> NDArray[Any]: + if not unit_values: + raise ValueError("coherence needs at least one patch") + values = np.stack(unit_values) + coherence = np.abs(np.mean(values, axis=0)) + return np.sum(coherence * weights[None, :, :], axis=(1, 2)) + + +def score_same_image_period_null( + pixels: NDArray[Any], + template: NDArray[Any], + target_period: float, + *, + period_min: float = 7.5, + period_max: float = 24.5, + patch_size: int = 256, + grid_size: int = 4, + harmonic_count: int = 16, +) -> SameImageNullScore: + """Compare a fixed target period with preregistered neighboring periods.""" + if not math.isfinite(target_period) or not period_min <= target_period <= period_max: + raise ValueError("target period must be finite and inside the null range") + off_periods = np.asarray( + [ + target_period + offset + for offset in SAME_IMAGE_NULL_OFFSETS + if period_min <= target_period + offset <= period_max + ], + dtype=np.float64, + ) + if not len(off_periods): + raise ValueError("same-image period null needs at least one neighboring period") + periods = np.concatenate((np.asarray([target_period]), off_periods)) + harmonics, weights, _coefficient_units = template_harmonics(template, harmonic_count) + _periods, _rotations, frequencies_y, frequencies_x = _candidate_frequencies( + periods, + np.asarray([0.0]), + harmonics, + ) + grouped_values: dict[int, list[NDArray[Any]]] = {0: [], 1: []} + for origin_y, origin_x, group in _patch_origins(*pixels.shape[:2], patch_size, grid_size): + grouped_values[group].append( + _patch_unit_values( + pixels, + origin_y, + origin_x, + patch_size, + frequencies_y, + frequencies_x, + ) + ) + selection = _coherence(grouped_values[0], weights) + confirmation = _coherence(grouped_values[1], weights) + selection_off_max = float(np.max(selection[1:])) + confirmation_off_max = float(np.max(confirmation[1:])) + target_selection = float(selection[0]) + target_confirmation = float(confirmation[0]) + return SameImageNullScore( + target_period=target_period, + target_selection_coherence=target_selection, + target_confirmation_coherence=target_confirmation, + joint_coherence=min(target_selection, target_confirmation), + selection_off_period_max=selection_off_max, + confirmation_off_period_max=confirmation_off_max, + joint_excess=min( + target_selection - selection_off_max, + target_confirmation - confirmation_off_max, + ), + off_period_count=len(off_periods), + ) + + +def _codeword_scores( + selection_values: list[NDArray[Any]], + confirmation_values: list[NDArray[Any]], + selected_index: int, + harmonics: NDArray[Any], + coefficient_units: NDArray[Any], + weights: NDArray[Any], + tile_size: int, +) -> tuple[int, int, float, float, float, float]: + selection_mean = np.mean(np.stack(selection_values), axis=0)[selected_index] + confirmation_mean = np.mean(np.stack(confirmation_values), axis=0)[selected_index] + shift_y, shift_x = np.meshgrid(np.arange(tile_size), np.arange(tile_size), indexing="ij") + flat_y = shift_y.ravel() + flat_x = shift_x.ravel() + phase = np.exp( + -2j * math.pi * (flat_y[:, None] * harmonics[None, :, 0] + flat_x[:, None] * harmonics[None, :, 1]) / tile_size + ) + shifted_codewords = coefficient_units[None, :, :] * phase[:, :, None] + + def scores(values: NDArray[Any]) -> NDArray[Any]: + agreement = np.real(values[None, :, :] * np.conj(shifted_codewords)) + return np.sum(agreement * weights[None, :, :], axis=(1, 2)) + + selection_scores = scores(selection_mean) + confirmation_scores = scores(confirmation_mean) + selected_shift = int(np.argmax(selection_scores)) + return ( + int(flat_y[selected_shift]), + int(flat_x[selected_shift]), + float(selection_scores[selected_shift]), + float(confirmation_scores[selected_shift]), + float(np.quantile(confirmation_scores, 0.95)), + float(np.quantile(confirmation_scores, 0.99)), + ) + + +def _unknown_codeword_scores( + selection_values: list[NDArray[Any]], + confirmation_values: list[NDArray[Any]], + selected_index: int, + harmonics: NDArray[Any], + weights: NDArray[Any], + tile_size: int, +) -> tuple[int, int, float, float, float, float, float, float]: + """Fit an unknown patch codeword and confirm it on held-out harmonics.""" + if len(harmonics) < 4: + raise ValueError("unknown-codeword confirmation needs at least four harmonics") + selection_mean = np.mean(np.stack(selection_values), axis=0)[selected_index] + confirmation_mean = np.mean(np.stack(confirmation_values), axis=0)[selected_index] + cross_codeword = confirmation_mean * np.conj(selection_mean) + + shift_y, shift_x = np.meshgrid(np.arange(tile_size), np.arange(tile_size), indexing="ij") + flat_y = shift_y.ravel() + flat_x = shift_x.ravel() + phase = np.exp( + -2j * math.pi * (flat_y[:, None] * harmonics[None, :, 0] + flat_x[:, None] * harmonics[None, :, 1]) / tile_size + ) + shifted_cross = cross_codeword[None, :, :] * phase[:, :, None] + selection_mask = np.arange(len(harmonics)) % 2 == 0 + confirmation_mask = ~selection_mask + + def scores(mask: NDArray[Any]) -> NDArray[Any]: + masked_weights = weights[mask] + normalizer = float(np.sum(masked_weights)) + if normalizer <= 0.0: + raise ValueError("unknown-codeword harmonic split has no template weight") + return np.abs(np.sum(shifted_cross[:, mask, :] * masked_weights[None, :, :], axis=(1, 2))) / normalizer + + selection_scores = scores(selection_mask) + confirmation_scores = scores(confirmation_mask) + selected_shift = int(np.argmax(selection_scores)) + fixed_all = float(np.abs(np.sum(cross_codeword * weights))) + return ( + int(flat_y[selected_shift]), + int(flat_x[selected_shift]), + float(selection_scores[selected_shift]), + float(confirmation_scores[selected_shift]), + float(confirmation_scores[0]), + fixed_all, + float(np.quantile(confirmation_scores, 0.95)), + float(np.quantile(confirmation_scores, 0.99)), + ) + + +def _period_candidate_indices( + periods: NDArray[Any], + rotations_degrees: NDArray[Any], + selection_scores: NDArray[Any], + count: int, +) -> list[int]: + """Return separated zero-rotation candidates ranked on selection patches.""" + candidates: list[int] = [] + eligible = np.flatnonzero(np.isclose(rotations_degrees, 0.0, atol=1e-12)) + for index in eligible[np.argsort(selection_scores[eligible])[::-1]]: + if any(abs(float(periods[index] - periods[prior])) < 0.5 for prior in candidates): + continue + candidates.append(int(index)) + if len(candidates) == count: + break + if not candidates: + raise ValueError("amplitude confirmation requires a zero-rotation candidate") + return candidates + + +def _period_alias_candidate_indices( + periods: NDArray[Any], + rotations_degrees: NDArray[Any], + base_indices: list[int], +) -> list[int]: + """Expand coarse periods with octave aliases and adjacent grid bins.""" + eligible = np.flatnonzero(np.isclose(rotations_degrees, 0.0, atol=1e-12)) + eligible_periods = periods[eligible] + candidates: list[int] = [] + seen: set[int] = set() + for base_index in base_indices: + for ratio in (1.0, 0.5, 2.0): + target = float(periods[base_index] * ratio) + if target < float(np.min(eligible_periods)) or target > float(np.max(eligible_periods)): + continue + center = int(np.argmin(np.abs(eligible_periods - target))) + for position in range(max(0, center - 1), min(len(eligible), center + 2)): + index = int(eligible[position]) + if index not in seen: + seen.add(index) + candidates.append(index) + return candidates + + +def _cyclic_correlations(template: NDArray[Any], tile: NDArray[Any]) -> NDArray[Any]: + template_spectrum = np.fft.fft2(template, axes=(0, 1)) + tile_spectrum = np.fft.fft2(tile, axes=(0, 1)) + return np.fft.ifft2(np.sum(template_spectrum * np.conj(tile_spectrum), axis=2)).real + + +def _canonical_pixels(pixels: NDArray[Any], template: NDArray[Any], period: float) -> NDArray[Any]: + width = max(template.shape[1], round(pixels.shape[1] * template.shape[1] / period)) + height = max(template.shape[0], round(pixels.shape[0] * template.shape[0] / period)) + if (height, width) == pixels.shape[:2]: + return pixels + interpolation = cv2.INTER_AREA if width < pixels.shape[1] else cv2.INTER_CUBIC + return np.asarray(cv2.resize(pixels, (width, height), interpolation=interpolation)) + + +def _real_correlation(left: NDArray[Any], right: NDArray[Any]) -> float: + denominator = float(np.linalg.norm(left) * np.linalg.norm(right)) + return float(np.real(np.vdot(right, left)) / denominator) if denominator > 0.0 else 0.0 + + +def _opponent_period_curve( + pixels: NDArray[Any], + template: NDArray[Any], + periods: NDArray[Any], + *, + harmonic_count: int, +) -> NDArray[Any]: + """Return signed opponent-color template coherence over PERIODS.""" + template_opponent = _opponent_color_pair(np.asarray(template, dtype=np.float64)) + template_spectrum = np.fft.fft2(template_opponent, axes=(0, 1)) + power = np.sum(np.abs(template_spectrum) ** 2, axis=2) + power[0, 0] = 0.0 + indices = np.argsort(power.ravel())[::-1][:harmonic_count] + rows, columns = np.unravel_index(indices, power.shape) + height, width = template.shape[:2] + signed_rows = np.where(rows <= height // 2, rows, rows - height) + signed_columns = np.where(columns <= width // 2, columns, columns - width) + harmonics = np.column_stack((signed_rows, signed_columns)).astype(np.float64) + coefficients = template_spectrum[rows, columns] + + image_height, image_width = pixels.shape[:2] + sample_y = periods[:, None] ** -1 * harmonics[None, :, 0] * image_height + sample_x = periods[:, None] ** -1 * harmonics[None, :, 1] * image_width + sampled = np.empty((len(periods), len(harmonics), 2), dtype=np.complex128) + image_opponent = _opponent_color_pair(np.asarray(pixels, dtype=np.float32)) + for channel in range(2): + residual = image_opponent[:, :, channel] + residual -= cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=1.0, + sigmaY=1.0, + borderType=cv2.BORDER_REFLECT_101, + ) + sampled[:, :, channel] = _bilinear_sample( + np.fft.fft2(residual), + sample_y % image_height, + sample_x % image_width, + ) + numerator = np.real(np.sum(np.conj(coefficients)[None, :, :] * sampled, axis=(1, 2))) + denominator = np.linalg.norm(coefficients) * np.linalg.norm(sampled, axis=(1, 2)) + return np.divide(numerator, denominator, out=np.zeros_like(numerator), where=denominator > 0.0) + + +def score_opponent_registered( + pixels: NDArray[Any], + template: NDArray[Any], + *, + periods: NDArray[Any], + harmonic_count: int = 30, + candidate_count: int = 5, + denoise_sigma: float = 1.0, +) -> OpponentRegisteredScore: + """Search scale using the large-carrier opponent-color representation.""" + if pixels.ndim != 3 or pixels.shape[2] != 3: + raise ValueError("pixels must have shape (height, width, 3)") + if periods.ndim != 1 or not len(periods) or np.any(~np.isfinite(periods)) or np.any(periods <= 0.0): + raise ValueError("periods must be a nonempty positive finite vector") + if harmonic_count < 1 or candidate_count < 1: + raise ValueError("harmonic and candidate counts must be positive") + curve = _opponent_period_curve( + pixels, + template, + periods, + harmonic_count=harmonic_count, + ) + rotations = np.zeros_like(periods) + candidate_indices = _period_candidate_indices(periods, rotations, curve, candidate_count) + observations: list[OpponentRegisteredScore] = [] + for index in candidate_indices: + period = float(periods[index]) + canonical = _canonical_pixels(pixels, template, period) + fixed_score, folded = folded_template_score(canonical, template, denoise_sigma) + folded_opponent = _opponent_color_pair(folded) + template_opponent = _opponent_color_pair(template) + observations.append( + OpponentRegisteredScore( + selected_period=period, + spectral_period=float(periods[int(np.argmax(curve))]), + spectral_score=float(curve[index]), + fixed_score=fixed_score, + red_green_spatial=_real_correlation(folded_opponent[:, :, 0], template_opponent[:, :, 0]), + blue_yellow_spatial=_real_correlation(folded_opponent[:, :, 1], template_opponent[:, :, 1]), + candidate_count=len(candidate_indices), + ) + ) + return max(observations, key=lambda score: score.decision_score) + + +def _content_whitened_patch_score( + pixels: NDArray[Any], + template: NDArray[Any], + harmonics: NDArray[Any], + *, + denoise_sigma: float, + noise_radius: int = 4, + guard_radius: int = 1, + ridge: float = 0.1, +) -> float: + """Return a complex matched-filter cosine after local color whitening.""" + patch_height, patch_width = pixels.shape[:2] + tile_height, tile_width = template.shape[:2] + if patch_height % tile_height or patch_width % tile_width: + raise ValueError("whitened patch dimensions must be multiples of the template dimensions") + if noise_radius <= guard_radius or guard_radius < 0: + raise ValueError("noise radius must exceed the nonnegative guard radius") + if not math.isfinite(ridge) or ridge <= 0.0: + raise ValueError("whitening ridge must be finite and positive") + + channels = _opponent_channels(np.asarray(pixels, dtype=np.float64)) + for channel in range(channels.shape[2]): + residual = channels[:, :, channel] + channels[:, :, channel] = residual - cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=denoise_sigma, + sigmaY=denoise_sigma, + borderType=cv2.BORDER_REFLECT_101, + ) + spectrum = np.fft.fft2(channels, axes=(0, 1)) + template_spectrum = np.fft.fft2(_opponent_channels(np.asarray(template, dtype=np.float64)), axes=(0, 1)) + + numerator = 0.0 + template_energy = 0.0 + observation_energy = 0.0 + for signed_row_value, signed_column_value in harmonics: + signed_row = round(float(signed_row_value)) + signed_column = round(float(signed_column_value)) + bin_y = (signed_row * patch_height // tile_height) % patch_height + bin_x = (signed_column * patch_width // tile_width) % patch_width + noise = np.asarray( + [ + spectrum[(bin_y + offset_y) % patch_height, (bin_x + offset_x) % patch_width] + for offset_y in range(-noise_radius, noise_radius + 1) + for offset_x in range(-noise_radius, noise_radius + 1) + if max(abs(offset_y), abs(offset_x)) > guard_radius + ] + ) + covariance = noise.T @ np.conj(noise) / len(noise) + local_power = float(np.trace(covariance).real / covariance.shape[0]) + covariance += np.eye(covariance.shape[0]) * (ridge * local_power + 1e-12) + template_value = template_spectrum[signed_row % tile_height, signed_column % tile_width] + observation = spectrum[bin_y, bin_x] + whitened_template = np.linalg.solve(covariance, template_value) + whitened_observation = np.linalg.solve(covariance, observation) + numerator += float(np.vdot(template_value, whitened_observation).real) + template_energy += float(np.vdot(template_value, whitened_template).real) + observation_energy += float(np.vdot(observation, whitened_observation).real) + denominator = math.sqrt(max(0.0, template_energy * observation_energy)) + return numerator / denominator if denominator > 1e-12 else 0.0 + + +def _content_whitened_score( + pixels: NDArray[Any], + template: NDArray[Any], + harmonics: NDArray[Any], + *, + period: float, + patch_size: int, + grid_size: int, + denoise_sigma: float, +) -> tuple[float, float]: + """Score a selected period on disjoint patch groups after local whitening.""" + canonical = _canonical_pixels(pixels, template, period) + tile_height, tile_width = template.shape[:2] + grouped_scores: dict[int, list[float]] = {0: [], 1: []} + for origin_y, origin_x, group in _patch_origins(*canonical.shape[:2], patch_size, grid_size): + aligned_y = (origin_y // tile_height) * tile_height + aligned_x = (origin_x // tile_width) * tile_width + patch = canonical[aligned_y : aligned_y + patch_size, aligned_x : aligned_x + patch_size] + grouped_scores[group].append( + _content_whitened_patch_score( + patch, + template, + harmonics, + denoise_sigma=denoise_sigma, + ) + ) + return float(np.mean(grouped_scores[0])), float(np.mean(grouped_scores[1])) + + +def _amplitude_score( + pixels: NDArray[Any], + template: NDArray[Any], + *, + period: float, + patch_size: int, + grid_size: int, + denoise_sigma: float, +) -> _AmplitudeScore: + """Select cyclic phase on one patch group and confirm it on the other.""" + canonical = _canonical_pixels(pixels, template, period) + grouped_units: dict[int, list[NDArray[Any]]] = {0: [], 1: []} + tile_height, tile_width = template.shape[:2] + for origin_y, origin_x, group in _patch_origins(*canonical.shape[:2], patch_size, grid_size): + aligned_y = (origin_y // tile_height) * tile_height + aligned_x = (origin_x // tile_width) * tile_width + folded = fold_residual_template( + canonical[aligned_y : aligned_y + patch_size, aligned_x : aligned_x + patch_size], + tile_height=tile_height, + tile_width=tile_width, + denoise_sigma=denoise_sigma, + ) + unit, _norm = unit_tile(folded) + grouped_units[group].append(unit) + selection_tile, _selection_norm = unit_tile(np.mean(grouped_units[0], axis=0)) + confirmation_tile, _confirmation_norm = unit_tile(np.mean(grouped_units[1], axis=0)) + selection_correlations = _cyclic_correlations(template, selection_tile) + confirmation_correlations = _cyclic_correlations(template, confirmation_tile) + selected_shift = int(np.argmax(selection_correlations)) + shift_y, shift_x = np.unravel_index(selected_shift, selection_correlations.shape) + unaligned_full, full_folded = folded_template_score(canonical, template, denoise_sigma) + aligned_full, _aligned_norm = unit_tile(np.roll(full_folded, shift=(shift_y, shift_x), axis=(0, 1))) + return _AmplitudeScore( + period=period, + selection=float(selection_correlations[shift_y, shift_x]), + confirmation=float(confirmation_correlations[shift_y, shift_x]), + shift_y=int(shift_y), + shift_x=int(shift_x), + unaligned_full=unaligned_full, + aligned_full=float(np.sum(template * aligned_full)), + ) + + +def _robust_shift_z(correlations: NDArray[Any]) -> NDArray[Any]: + """Standardize each patch against its own cyclic-shift null.""" + flattened = correlations.reshape(correlations.shape[0], -1) + center = np.median(flattened, axis=1, keepdims=True) + mad = np.median(np.abs(flattened - center), axis=1, keepdims=True) + scale = 1.4826 * mad + fallback = np.std(flattened, axis=1, keepdims=True) + scale = np.where(scale > 1e-12, scale, fallback) + standardized = np.divide( + flattened - center, + scale, + out=np.zeros_like(flattened), + where=scale > 1e-12, + ) + return standardized.reshape(correlations.shape) + + +def _top_half_mean(values: NDArray[Any], *, axis: int) -> NDArray[Any]: + """Average the strongest half without letting one patch dominate.""" + count = values.shape[axis] + retained = max(1, (count + 1) // 2) + partitioned = np.partition(values, count - retained, axis=axis) + indices = np.arange(count - retained, count) + return np.mean(np.take(partitioned, indices, axis=axis), axis=axis) + + +def _shift_support_fraction(correlations: NDArray[Any], shift_y: int, shift_x: int) -> float: + """Return the patch fraction placing one shift in its upper five percent.""" + selected = correlations[:, shift_y, shift_x] + flattened = correlations.reshape(correlations.shape[0], -1) + less = np.sum(flattened < selected[:, None], axis=1) + equal = np.sum(flattened == selected[:, None], axis=1) + percentile = (less + 0.5 * equal) / flattened.shape[1] + return float(np.mean(percentile >= 0.95)) + + +def score_patch_shift_consensus( + pixels: NDArray[Any], + template: NDArray[Any], + period: float, + *, + patch_size: int = 256, + grid_size: int = 4, + denoise_sigma: float = 1.0, +) -> PatchShiftConsensusScore: + """Select a robust cyclic phase and confirm it on disjoint patches. + + This probe targets a content-adaptive encoder that may place the shared + detection carrier strongly in only part of an image. Every patch is + normalized against its own 2-D cyclic-shift distribution before the + strongest half are pooled. It is a research statistic, not a detector + threshold. + """ + if not math.isfinite(period) or period <= 0.0: + raise ValueError("period must be finite and positive") + canonical = _canonical_pixels(pixels, template, period) + tile_height, tile_width = template.shape[:2] + grouped_correlations: dict[int, list[NDArray[Any]]] = {0: [], 1: []} + for origin_y, origin_x, group in _patch_origins(*canonical.shape[:2], patch_size, grid_size): + aligned_y = (origin_y // tile_height) * tile_height + aligned_x = (origin_x // tile_width) * tile_width + folded = fold_residual_template( + canonical[aligned_y : aligned_y + patch_size, aligned_x : aligned_x + patch_size], + tile_height=tile_height, + tile_width=tile_width, + denoise_sigma=denoise_sigma, + ) + unit, _norm = unit_tile(folded) + grouped_correlations[group].append(_cyclic_correlations(template, unit)) + + selection = np.stack(grouped_correlations[0]) + confirmation = np.stack(grouped_correlations[1]) + selection_z = _robust_shift_z(selection) + confirmation_z = _robust_shift_z(confirmation) + selection_trimmed = _top_half_mean(selection_z, axis=0) + selected_shift = int(np.argmax(selection_trimmed)) + shift_y, shift_x = np.unravel_index(selected_shift, selection_trimmed.shape) + selection_score = float(selection_trimmed[shift_y, shift_x]) + confirmation_score = float(_top_half_mean(confirmation_z[:, shift_y, shift_x], axis=0)) + selection_support = _shift_support_fraction(selection, int(shift_y), int(shift_x)) + confirmation_support = _shift_support_fraction(confirmation, int(shift_y), int(shift_x)) + return PatchShiftConsensusScore( + period=period, + selected_shift_y=int(shift_y), + selected_shift_x=int(shift_x), + selection_trimmed_z=selection_score, + confirmation_trimmed_z=confirmation_score, + joint_trimmed_z=min(selection_score, confirmation_score), + selection_support_fraction=selection_support, + confirmation_support_fraction=confirmation_support, + joint_support_fraction=min(selection_support, confirmation_support), + selection_patches=len(selection), + confirmation_patches=len(confirmation), + ) + + +def score_lattice( + pixels: NDArray[Any], + template: NDArray[Any], + *, + periods: NDArray[Any], + rotations_degrees: NDArray[Any], + patch_size: int = 256, + grid_size: int = 4, + harmonic_count: int = 16, + amplitude_candidate_count: int = 5, + denoise_sigma: float = 1.0, +) -> LatticeScore: + """Select a lattice on one patch group and confirm it on the other.""" + if pixels.ndim != 3 or pixels.shape[2] != 3: + raise ValueError("pixels must have shape (height, width, 3)") + if periods.ndim != 1 or not len(periods) or np.any(~np.isfinite(periods)) or np.any(periods <= 0.0): + raise ValueError("periods must be a nonempty positive finite vector") + if rotations_degrees.ndim != 1 or not len(rotations_degrees) or np.any(~np.isfinite(rotations_degrees)): + raise ValueError("rotations must be a nonempty finite vector") + if not math.isfinite(denoise_sigma) or denoise_sigma <= 0.0: + raise ValueError("denoise sigma must be finite and positive") + if amplitude_candidate_count < 1: + raise ValueError("amplitude candidate count must be positive") + harmonics, weights, coefficient_units = template_harmonics(template, harmonic_count) + flat_periods, flat_rotations, frequencies_y, frequencies_x = _candidate_frequencies( + periods, + rotations_degrees, + harmonics, + ) + grouped_values: dict[int, list[NDArray[Any]]] = {0: [], 1: []} + for origin_y, origin_x, group in _patch_origins(*pixels.shape[:2], patch_size, grid_size): + grouped_values[group].append( + _patch_unit_values( + pixels, + origin_y, + origin_x, + patch_size, + frequencies_y, + frequencies_x, + ) + ) + selection_scores = _coherence(grouped_values[0], weights) + confirmation_scores = _coherence(grouped_values[1], weights) + peak_index = int(np.argmax(selection_scores)) + coarse_candidate_indices = _period_candidate_indices( + flat_periods, + flat_rotations, + selection_scores, + amplitude_candidate_count, + ) + candidate_indices = _period_alias_candidate_indices( + flat_periods, + flat_rotations, + coarse_candidate_indices, + ) + whitened_scores = [ + _content_whitened_score( + pixels, + template, + harmonics, + period=float(flat_periods[index]), + patch_size=patch_size, + grid_size=grid_size, + denoise_sigma=denoise_sigma, + ) + for index in candidate_indices + ] + whitened_order = np.argsort([selection for selection, _confirmation in whitened_scores])[::-1] + rerank_candidates = [int(index) for index in whitened_order[:3]] + rerank_amplitudes = [ + _amplitude_score( + pixels, + template, + period=float(flat_periods[candidate_indices[index]]), + patch_size=patch_size, + grid_size=grid_size, + denoise_sigma=denoise_sigma, + ) + for index in rerank_candidates + ] + rerank_scores = [ + min(*whitened_scores[index], amplitude.selection, amplitude.confirmation) + for index, amplitude in zip(rerank_candidates, rerank_amplitudes, strict=True) + ] + rerank_winner = int(np.argmax(rerank_scores)) + selected_candidate = rerank_candidates[rerank_winner] + selected_index = candidate_indices[selected_candidate] + amplitude = rerank_amplitudes[rerank_winner] + selection_whitened, confirmation_whitened = whitened_scores[selected_candidate] + selection_p95 = float(np.quantile(selection_scores, 0.95)) + selection_p99 = float(np.quantile(selection_scores, 0.99)) + confirmation_p95 = float(np.quantile(confirmation_scores, 0.95)) + confirmation_p99 = float(np.quantile(confirmation_scores, 0.99)) + confirmation = float(confirmation_scores[selected_index]) + selection = float(selection_scores[selected_index]) + shift_y, shift_x, selection_codeword, confirmation_codeword, codeword_p95, codeword_p99 = _codeword_scores( + grouped_values[0], + grouped_values[1], + selected_index, + harmonics, + coefficient_units, + weights, + template.shape[0], + ) + ( + unknown_shift_y, + unknown_shift_x, + unknown_selection, + unknown_confirmation, + unknown_fixed_confirmation, + unknown_fixed_all, + unknown_p95, + unknown_p99, + ) = _unknown_codeword_scores( + grouped_values[0], + grouped_values[1], + selected_index, + harmonics, + weights, + template.shape[0], + ) + return LatticeScore( + selected_period=float(flat_periods[selected_index]), + selected_rotation_degrees=float(flat_rotations[selected_index]), + selected_orientation_degrees=0, + selected_horizontal_reflection=False, + selected_deskew_degrees=0.0, + deskew_candidate_count=0, + deskew_selection_coherence=0.0, + deskew_direct_selection_match=0.0, + deskew_direct_confirmation_match=0.0, + deskew_direct_joint_match=0.0, + peak_period=float(flat_periods[peak_index]), + peak_rotation_degrees=float(flat_rotations[peak_index]), + peak_selection_coherence=float(selection_scores[peak_index]), + selection_coherence=selection, + selection_candidate_p95=selection_p95, + selection_candidate_p99=selection_p99, + selection_excess_p95=selection - selection_p95, + selection_excess_p99=selection - selection_p99, + confirmation_coherence=confirmation, + confirmation_candidate_p95=confirmation_p95, + confirmation_candidate_p99=confirmation_p99, + confirmation_excess_p95=confirmation - confirmation_p95, + confirmation_excess_p99=confirmation - confirmation_p99, + joint_coherence=min(selection, confirmation), + joint_excess_p99=min(selection - selection_p99, confirmation - confirmation_p99), + selected_shift_y=shift_y, + selected_shift_x=shift_x, + selection_codeword=selection_codeword, + confirmation_codeword=confirmation_codeword, + confirmation_codeword_shift_p95=codeword_p95, + confirmation_codeword_shift_p99=codeword_p99, + joint_codeword=min(selection_codeword, confirmation_codeword), + unknown_codeword_shift_y=unknown_shift_y, + unknown_codeword_shift_x=unknown_shift_x, + unknown_codeword_selection=unknown_selection, + unknown_codeword_confirmation=unknown_confirmation, + unknown_codeword_fixed_confirmation=unknown_fixed_confirmation, + unknown_codeword_fixed_all=unknown_fixed_all, + unknown_codeword_confirmation_p95=unknown_p95, + unknown_codeword_confirmation_p99=unknown_p99, + unknown_codeword_excess_p99=unknown_confirmation - unknown_p99, + amplitude_shift_y=amplitude.shift_y, + amplitude_shift_x=amplitude.shift_x, + selection_amplitude=amplitude.selection, + confirmation_amplitude=amplitude.confirmation, + joint_amplitude=min(amplitude.selection, amplitude.confirmation), + selection_whitened_match=selection_whitened, + confirmation_whitened_match=confirmation_whitened, + joint_whitened_match=min(selection_whitened, confirmation_whitened), + amplitude_candidate_count=len(candidate_indices), + amplitude_rerank_count=len(rerank_candidates), + canonical_template_score=amplitude.unaligned_full, + canonical_registered_template_score=amplitude.aligned_full, + selection_patches=len(grouped_values[0]), + confirmation_patches=len(grouped_values[1]), + ) + + +def score_orientation_bank( + pixels: NDArray[Any], + template: NDArray[Any], + *, + periods: NDArray[Any], + rotations_degrees: NDArray[Any], + patch_size: int = 256, + grid_size: int = 4, + harmonic_count: int = 16, + amplitude_candidate_count: int = 5, + denoise_sigma: float = 1.0, +) -> LatticeScore: + """Select a right-angle orientation on selection-patch whitened match.""" + candidates = [ + score_lattice( + np.rot90(pixels, k=quarter_turns), + template, + periods=periods, + rotations_degrees=rotations_degrees, + patch_size=patch_size, + grid_size=grid_size, + harmonic_count=harmonic_count, + amplitude_candidate_count=amplitude_candidate_count, + denoise_sigma=denoise_sigma, + ) + for quarter_turns in range(4) + ] + selected_index = int(np.argmax([candidate.selection_whitened_match for candidate in candidates])) + return replace(candidates[selected_index], selected_orientation_degrees=selected_index * 90) + + +def score_dihedral_bank( + pixels: NDArray[Any], + template: NDArray[Any], + *, + periods: NDArray[Any], + rotations_degrees: NDArray[Any], + patch_size: int = 256, + grid_size: int = 4, + harmonic_count: int = 16, + amplitude_candidate_count: int = 5, + denoise_sigma: float = 1.0, +) -> LatticeScore: + """Select a rotation and optional reflection on selection patches.""" + candidates: list[LatticeScore] = [] + transforms: list[tuple[int, bool]] = [] + for reflected in (False, True): + reflected_pixels = np.fliplr(pixels) if reflected else pixels + for quarter_turns in range(4): + candidates.append( + score_lattice( + np.rot90(reflected_pixels, k=quarter_turns), + template, + periods=periods, + rotations_degrees=rotations_degrees, + patch_size=patch_size, + grid_size=grid_size, + harmonic_count=harmonic_count, + amplitude_candidate_count=amplitude_candidate_count, + denoise_sigma=denoise_sigma, + ) + ) + transforms.append((quarter_turns * 90, reflected)) + selected_index = int(np.argmax([candidate.selection_whitened_match for candidate in candidates])) + selected_orientation, selected_reflection = transforms[selected_index] + return replace( + candidates[selected_index], + selected_orientation_degrees=selected_orientation, + selected_horizontal_reflection=selected_reflection, + ) + + +def _rotate_fixed_canvas(pixels: NDArray[Any], angle_degrees: float) -> NDArray[Any]: + """Rotate RGB pixels around the image center without changing the canvas.""" + height, width = pixels.shape[:2] + transform = cv2.getRotationMatrix2D(((width - 1) / 2.0, (height - 1) / 2.0), angle_degrees, 1.0) + return np.asarray( + cv2.warpAffine( + pixels, + transform, + (width, height), + flags=cv2.INTER_CUBIC, + borderMode=cv2.BORDER_REFLECT_101, + ) + ) + + +def _select_deskew_angle( + pixels: NDArray[Any], + template: NDArray[Any], + periods: NDArray[Any], + deskew_degrees: NDArray[Any], + *, + patch_size: int, + grid_size: int, + harmonic_count: int, +) -> tuple[float, float, float]: + """Select one deskew angle by carrier coherence on selection patches.""" + harmonics, weights, _coefficient_units = template_harmonics(template, harmonic_count) + flat_periods, flat_rotations, frequencies_y, frequencies_x = _candidate_frequencies( + periods, + deskew_degrees, + harmonics, + ) + selection_values = [ + _patch_unit_values( + pixels, + origin_y, + origin_x, + patch_size, + frequencies_y, + frequencies_x, + ) + for origin_y, origin_x, group in _patch_origins(*pixels.shape[:2], patch_size, grid_size) + if group == 0 + ] + selection_scores = _coherence(selection_values, weights) + selected_index = int(np.argmax(selection_scores)) + return ( + float(flat_periods[selected_index]), + float(flat_rotations[selected_index]), + float(selection_scores[selected_index]), + ) + + +def _affine_whitened_patch_score( + pixels: NDArray[Any], + expected_template: NDArray[Any], + frequencies_y: NDArray[Any], + frequencies_x: NDArray[Any], + *, + origin_y: int, + origin_x: int, + denoise_sigma: float, + noise_radius: int = 4, + guard_radius: int = 1, + ridge: float = 0.1, +) -> float: + """Match an affine carrier directly, without resampling the image.""" + patch_size = pixels.shape[0] + channels = _opponent_channels(np.asarray(pixels, dtype=np.float64)) + for channel in range(channels.shape[2]): + residual = channels[:, :, channel] + channels[:, :, channel] = residual - cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=denoise_sigma, + sigmaY=denoise_sigma, + borderType=cv2.BORDER_REFLECT_101, + ) + window_1d = np.hanning(patch_size) + spectrum = np.fft.fft2(channels * (window_1d[:, None] * window_1d[None, :])[:, :, None], axes=(0, 1)) + offset_y, offset_x = zip( + *( + (y, x) + for y in range(-noise_radius, noise_radius + 1) + for x in range(-noise_radius, noise_radius + 1) + if max(abs(y), abs(x)) > guard_radius + ), + strict=True, + ) + offset_y_values = np.asarray(offset_y, dtype=np.float64) + offset_x_values = np.asarray(offset_x, dtype=np.float64) + sample_y = frequencies_y * patch_size + sample_x = frequencies_x * patch_size + phase_correction = np.exp(-2j * math.pi * (frequencies_y * origin_y + frequencies_x * origin_x)) + numerator = 0.0 + template_energy = 0.0 + observation_energy = 0.0 + for harmonic_index in range(len(frequencies_y)): + y = np.asarray([sample_y[harmonic_index]]) + x = np.asarray([sample_x[harmonic_index]]) + observation = np.asarray( + [_bilinear_sample(spectrum[:, :, channel], y, x)[0] for channel in range(channels.shape[2])] + ) + observation *= phase_correction[harmonic_index] + noise = np.stack( + [ + _bilinear_sample( + spectrum[:, :, channel], + y + offset_y_values, + x + offset_x_values, + ) + for channel in range(channels.shape[2]) + ], + axis=1, + ) + covariance = noise.T @ np.conj(noise) / len(noise) + local_power = float(np.trace(covariance).real / covariance.shape[0]) + covariance += np.eye(covariance.shape[0]) * (ridge * local_power + 1e-12) + template_value = expected_template[harmonic_index] + whitened_template = np.linalg.solve(covariance, template_value) + whitened_observation = np.linalg.solve(covariance, observation) + numerator += float(np.vdot(template_value, whitened_observation).real) + template_energy += float(np.vdot(template_value, whitened_template).real) + observation_energy += float(np.vdot(observation, whitened_observation).real) + denominator = math.sqrt(max(0.0, template_energy * observation_energy)) + return numerator / denominator if denominator > 1e-12 else 0.0 + + +def _affine_whitened_score( + pixels: NDArray[Any], + template: NDArray[Any], + harmonics: NDArray[Any], + *, + period: float, + deskew_degrees: float, + patch_size: int, + grid_size: int, + denoise_sigma: float, +) -> tuple[float, float]: + """Score a rotated carrier in the original pixels on disjoint patches.""" + _periods, _rotations, frequencies_y, frequencies_x = _candidate_frequencies( + np.asarray([period]), + np.asarray([deskew_degrees]), + harmonics, + ) + height, width = pixels.shape[:2] + input_rotation = -deskew_degrees + forward = cv2.getRotationMatrix2D( + ((width - 1) / 2.0, (height - 1) / 2.0), + input_rotation, + 1.0, + ) + inverse = cv2.invertAffineTransform(forward) + origin_x, origin_y = inverse[:, 2] + phase = np.exp(2j * math.pi * (harmonics[:, 0] * origin_y / period + harmonics[:, 1] * origin_x / period)) + template_spectrum = np.fft.fft2(_opponent_channels(np.asarray(template, dtype=np.float64)), axes=(0, 1)) + expected_template = ( + np.asarray( + [ + template_spectrum[round(float(row)) % template.shape[0], round(float(column)) % template.shape[1]] + for row, column in harmonics + ] + ) + * phase[:, None] + ) + grouped_scores: dict[int, list[float]] = {0: [], 1: []} + for patch_y, patch_x, group in _patch_origins(height, width, patch_size, grid_size): + patch = pixels[patch_y : patch_y + patch_size, patch_x : patch_x + patch_size] + grouped_scores[group].append( + _affine_whitened_patch_score( + patch, + expected_template, + frequencies_y[0], + frequencies_x[0], + origin_y=patch_y, + origin_x=patch_x, + denoise_sigma=denoise_sigma, + ) + ) + return float(np.mean(grouped_scores[0])), float(np.mean(grouped_scores[1])) + + +def score_deskew_bank( + pixels: NDArray[Any], + template: NDArray[Any], + *, + periods: NDArray[Any], + deskew_degrees: NDArray[Any], + patch_size: int = 256, + grid_size: int = 4, + harmonic_count: int = 16, + amplitude_candidate_count: int = 5, + denoise_sigma: float = 1.0, +) -> LatticeScore: + """Select a small-angle deskew operation on selection patches.""" + if deskew_degrees.ndim != 1 or not len(deskew_degrees) or np.any(~np.isfinite(deskew_degrees)): + raise ValueError("deskew angles must be a nonempty finite vector") + harmonics, _weights, _coefficient_units = template_harmonics(template, harmonic_count) + selected_period, selected_angle, selection_coherence = _select_deskew_angle( + pixels, + template, + periods, + deskew_degrees, + patch_size=patch_size, + grid_size=grid_size, + harmonic_count=harmonic_count, + ) + direct_selection, direct_confirmation = _affine_whitened_score( + pixels, + template, + harmonics, + period=selected_period, + deskew_degrees=selected_angle, + patch_size=patch_size, + grid_size=grid_size, + denoise_sigma=denoise_sigma, + ) + score = score_lattice( + _rotate_fixed_canvas(pixels, selected_angle), + template, + periods=periods, + rotations_degrees=np.asarray([0.0]), + patch_size=patch_size, + grid_size=grid_size, + harmonic_count=harmonic_count, + amplitude_candidate_count=amplitude_candidate_count, + denoise_sigma=denoise_sigma, + ) + return replace( + score, + selected_deskew_degrees=selected_angle, + deskew_candidate_count=len(periods) * len(deskew_degrees), + deskew_selection_coherence=selection_coherence, + deskew_direct_selection_match=direct_selection, + deskew_direct_confirmation_match=direct_confirmation, + deskew_direct_joint_match=min(direct_selection, direct_confirmation), + ) + + +def _load_template(path: Path) -> NDArray[Any]: + with np.load(path, allow_pickle=False) as artifact: + template = np.asarray(artifact["template"], dtype=np.float64) + if template.shape != (16, 16, 3) or not np.all(np.isfinite(template)): + raise ValueError("template artifact does not contain a finite 16x16 RGB template") + return template + + +@click.command() +@click.argument("template_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--period-min", type=click.FloatRange(min=1.0), default=7.5, show_default=True) +@click.option("--period-max", type=click.FloatRange(min=1.0), default=24.5, show_default=True) +@click.option("--period-step", type=click.FloatRange(min=0.01), default=0.1, show_default=True) +@click.option("--rotation-min", type=float, default=-3.0, show_default=True) +@click.option("--rotation-max", type=float, default=3.0, show_default=True) +@click.option("--rotation-step", type=click.FloatRange(min=0.01), default=0.25, show_default=True) +@click.option("--input-angle", type=float, default=0.0, show_default=True) +@click.option("--deskew-search/--no-deskew-search", default=False, show_default=True) +@click.option("--input-scale", type=click.FloatRange(min=0.01), default=1.0, show_default=True) +@click.option("--crop-fraction", type=click.FloatRange(min=0.0, max=0.49), default=0.0, show_default=True) +@click.option("--jpeg-quality", type=click.IntRange(min=1, max=100), default=None) +@click.option("--webp-quality", type=click.IntRange(min=1, max=101), default=None) +@click.option("--input-rotation", type=click.Choice(("0", "90", "180", "270")), default="0", show_default=True) +@click.option( + "--input-flip", + type=click.Choice(("none", "horizontal", "vertical", "both")), + default="none", + show_default=True, +) +@click.option("--orientation-search/--no-orientation-search", default=False, show_default=True) +@click.option("--dihedral-search/--no-dihedral-search", default=False, show_default=True) +@click.option("--registered-period/--no-registered-period", default=False, show_default=True) +@click.option("--same-image-null/--no-same-image-null", default=False, show_default=True) +@click.option("--patch-shift-consensus/--no-patch-shift-consensus", default=False, show_default=True) +@click.option("--opponent-registered/--no-opponent-registered", default=False, show_default=True) +@click.option("--amplitude-candidate-count", type=click.IntRange(min=1), default=5, show_default=True) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def main( + template_path: Path, + images: tuple[Path, ...], + period_min: float, + period_max: float, + period_step: float, + rotation_min: float, + rotation_max: float, + rotation_step: float, + input_angle: float, + deskew_search: bool, + input_scale: float, + crop_fraction: float, + jpeg_quality: int | None, + webp_quality: int | None, + input_rotation: str, + input_flip: str, + orientation_search: bool, + dihedral_search: bool, + registered_period: bool, + same_image_null: bool, + patch_shift_consensus: bool, + opponent_registered: bool, + amplitude_candidate_count: int, + report_out: Path, +) -> None: + """Score IMAGES with split-confirmed reciprocal-lattice coherence.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + if period_max < period_min or rotation_max < rotation_min: + raise click.BadParameter("search maximum must be at least its minimum") + if jpeg_quality is not None and webp_quality is not None: + raise click.UsageError("JPEG and WebP transforms are mutually exclusive") + enabled_geometric_searches = sum((orientation_search, dihedral_search, deskew_search)) + if enabled_geometric_searches > 1: + raise click.UsageError("orientation, dihedral, and deskew search are mutually exclusive") + if registered_period and enabled_geometric_searches: + raise click.UsageError("registered-period scoring cannot be combined with a geometric search") + if same_image_null and not registered_period: + raise click.UsageError("same-image-null scoring requires registered-period scoring") + if patch_shift_consensus and not registered_period: + raise click.UsageError("patch-shift consensus requires registered-period scoring") + periods = np.arange(period_min, period_max + period_step / 2.0, period_step) + rotations = np.arange(rotation_min, rotation_max + rotation_step / 2.0, rotation_step) + template = _load_template(template_path) + rows = [] + for path in images: + registered: RegisteredComponents | None = None + period_null: SameImageNullScore | None = None + patch_consensus: PatchShiftConsensusScore | None = None + opponent_score: OpponentRegisteredScore | None = None + try: + pixels = load_rgb(path) + if input_scale != 1.0: + scaled_width = max(1, round(pixels.shape[1] * input_scale)) + scaled_height = max(1, round(pixels.shape[0] * input_scale)) + interpolation = cv2.INTER_AREA if input_scale < 1.0 else cv2.INTER_CUBIC + pixels = cv2.resize(pixels, (scaled_width, scaled_height), interpolation=interpolation) + if crop_fraction: + crop_y = round(pixels.shape[0] * crop_fraction) + crop_x = round(pixels.shape[1] * crop_fraction) + pixels = pixels[crop_y:, crop_x:] + if jpeg_quality is not None: + pixels = jpeg_round_trip(pixels, jpeg_quality) + if webp_quality is not None: + pixels = _webp_round_trip(pixels, webp_quality) + if input_angle: + pixels = _rotate_fixed_canvas(pixels, input_angle) + quarter_turns = int(input_rotation) // 90 + if quarter_turns: + pixels = np.rot90(pixels, k=-quarter_turns) + if input_flip in {"horizontal", "both"}: + pixels = np.fliplr(pixels) + if input_flip in {"vertical", "both"}: + pixels = np.flipud(pixels) + image_periods = periods + image_rotations = rotations + if registered_period: + registered = registered_components(pixels, template, 1.0) + image_periods = np.asarray([registered.selected_period]) + image_rotations = np.asarray([0.0]) + if same_image_null: + period_null = score_same_image_period_null( + pixels, + template, + registered.selected_period, + ) + if patch_shift_consensus: + patch_consensus = score_patch_shift_consensus( + pixels, + template, + registered.selected_period, + ) + if opponent_registered: + opponent_score = score_opponent_registered( + pixels, + template, + periods=periods, + ) + if dihedral_search: + scorer = score_dihedral_bank + elif orientation_search: + scorer = score_orientation_bank + elif deskew_search: + score = score_deskew_bank( + pixels, + template, + periods=periods, + deskew_degrees=rotations, + amplitude_candidate_count=amplitude_candidate_count, + ) + scorer = None + else: + scorer = score_lattice + if scorer is not None: + score = scorer( + pixels, + template, + periods=image_periods, + rotations_degrees=image_rotations, + amplitude_candidate_count=amplitude_candidate_count, + ) + except (OSError, ValueError) as error: + rows.append({"path": str(path), "status": "unsupported", "reason": str(error)}) + log.warning("%s: unsupported: %s", path, error) + continue + row: dict[str, Any] = {"path": str(path), "status": "scored", "score": asdict(score)} + if registered is not None: + row["registered"] = {**asdict(registered), "decision_score": registered.decision_score} + row["registered_confirmation_passes"] = registered_confirmation_passes(score) + if period_null is not None: + row["same_image_null"] = asdict(period_null) + if patch_consensus is not None: + row["patch_shift_consensus"] = asdict(patch_consensus) + if opponent_score is not None: + row["opponent_registered"] = { + **asdict(opponent_score), + "decision_score": opponent_score.decision_score, + } + rows.append(row) + log.info( + "%s: period=%.3f rotation=%.3f lattice=%.6f whitened=%.6f template=%.6f", + path, + score.selected_period, + score.selected_rotation_degrees, + score.joint_coherence, + score.joint_whitened_match, + score.canonical_template_score, + ) + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text( + json.dumps( + { + "schema_version": 13, + "template": str(template_path), + "periods": [float(value) for value in periods], + "rotations_degrees": [float(value) for value in rotations], + "input_angle": input_angle, + "deskew_search": deskew_search, + "input_scale": input_scale, + "crop_fraction": crop_fraction, + "jpeg_quality": jpeg_quality, + "webp_quality": webp_quality, + "input_rotation": int(input_rotation), + "input_flip": input_flip, + "orientation_search": orientation_search, + "dihedral_search": dihedral_search, + "registered_period": registered_period, + "same_image_null": same_image_null, + "patch_shift_consensus": patch_shift_consensus, + "opponent_registered": opponent_registered, + "amplitude_candidate_count": amplitude_candidate_count, + "records": rows, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d lattice records: %s", len(rows), report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_color_space_probe.py b/scripts/synthid_color_space_probe.py new file mode 100644 index 0000000..3d451d0 --- /dev/null +++ b/scripts/synthid_color_space_probe.py @@ -0,0 +1,353 @@ +"""Compare an exact-geometry phase carrier across color spaces. + +Every model uses the same spatial-frequency candidates. Phases, expected +magnitudes, selected channels, and weights are learned independently from the +supplied positive images. The resulting scores are experimental evidence, not +a proprietary SynthID decoder or a certified production detector. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict, dataclass +from pathlib import Path + +import click +import cv2 +import numpy as np +from PIL import Image +from synthid_phase_carrier import _leave_one_out_coherence +from synthid_phase_registration import extract_frequency_values +from synthid_v3_codebook_probe import load_v3_model + +log = logging.getLogger(__name__) + +COLOR_SPACES = ("rgb", "ycbcr", "ycocg", "opponent", "lab", "hsv") +CHANNEL_NAMES = { + "rgb": ("R", "G", "B"), + "ycbcr": ("Y", "Cb", "Cr"), + "ycocg": ("Y", "Co", "Cg"), + "opponent": ("L", "R-G", "R+G-2B"), + "lab": ("L*", "a*", "b*"), + "hsv": ("H", "S", "V"), +} + + +@dataclass(frozen=True) +class ColorPhaseModel: + """Sparse phase carrier learned in one color space.""" + + color_space: str + height: int + width: int + rows: np.ndarray + columns: np.ndarray + channels: np.ndarray + phases: np.ndarray + weights: np.ndarray + expected_magnitudes: np.ndarray + + +@dataclass(frozen=True) +class ColorPhaseScore: + """Phase-carrier evidence and channel contributions for one image.""" + + path: str + color_space: str + phase_score: float + active_weight_fraction: float + evidence_score: float + channel_evidence: tuple[float, float, float] + selected_peak_counts: tuple[int, int, int] + peak_count: int + + +def transform_color_space(rgb: np.ndarray, color_space: str) -> np.ndarray: + """Transform float RGB values in [0, 255] into COLOR_SPACE.""" + if rgb.ndim != 3 or rgb.shape[2] != 3: + raise ValueError("rgb must have shape (height, width, 3)") + pixels = np.asarray(rgb, dtype=np.float64) + red, green, blue = np.moveaxis(pixels, 2, 0) + if color_space == "rgb": + transformed = pixels + elif color_space == "ycbcr": + transformed = np.stack( + ( + 0.299 * red + 0.587 * green + 0.114 * blue, + 128.0 - 0.168736 * red - 0.331264 * green + 0.5 * blue, + 128.0 + 0.5 * red - 0.418688 * green - 0.081312 * blue, + ), + axis=2, + ) + elif color_space == "ycocg": + transformed = np.stack( + ( + 0.25 * red + 0.5 * green + 0.25 * blue, + 0.5 * red - 0.5 * blue, + -0.25 * red + 0.5 * green - 0.25 * blue, + ), + axis=2, + ) + elif color_space == "opponent": + transformed = np.stack( + ( + (red + green + blue) / np.sqrt(3.0), + (red - green) / np.sqrt(2.0), + (red + green - 2.0 * blue) / np.sqrt(6.0), + ), + axis=2, + ) + elif color_space == "lab": + transformed = cv2.cvtColor((pixels / 255.0).astype(np.float32), cv2.COLOR_RGB2LAB).astype(np.float64) + elif color_space == "hsv": + transformed = cv2.cvtColor((pixels / 255.0).astype(np.float32), cv2.COLOR_RGB2HSV).astype(np.float64) + else: + raise ValueError(f"unsupported color space: {color_space}") + if not np.all(np.isfinite(transformed)): + raise ValueError(f"{color_space} transform produced non-finite values") + return transformed + + +def candidate_bins_from_codebook( + codebook_path: Path, + *, + height: int, + width: int, + source_peak_count: int = 256, +) -> np.ndarray: + """Expand the codebook's unique spatial coordinates over three channels.""" + prior = load_v3_model( + codebook_path, + height=height, + width=width, + peak_count=source_peak_count, + ) + spatial = np.unique(np.column_stack((prior.rows, prior.columns)), axis=0) + return np.asarray( + [(int(row), int(column), channel) for row, column in spatial for channel in range(3)], + dtype=np.int32, + ) + + +def _load_rgb(path: Path, *, height: int, width: int) -> np.ndarray: + """Load an exact-geometry image as float64 RGB.""" + with Image.open(path) as image: + rgb = image.convert("RGB") + if rgb.size != (width, height): + raise ValueError(f"{path}: geometry {rgb.width}x{rgb.height} does not match {width}x{height}") + return np.asarray(rgb, dtype=np.float64) + + +def discover_model( + paths: list[Path], + *, + color_space: str, + candidate_bins: np.ndarray, + peak_count: int = 256, +) -> ColorPhaseModel: + """Learn one color-space phase carrier from exact-geometry PATHS.""" + if len(paths) < 3: + raise ValueError("at least three positive images are required") + if color_space not in COLOR_SPACES: + raise ValueError(f"unsupported color space: {color_space}") + with Image.open(paths[0]) as first: + width, height = first.size + bins = np.asarray(candidate_bins, dtype=np.int32) + if bins.ndim != 2 or bins.shape[1] != 3: + raise ValueError("candidate_bins must have shape (count, 3)") + if len(bins) < peak_count: + raise ValueError(f"only {len(bins)} candidate bins for {peak_count} peaks") + if ( + np.any(bins[:, 0] < 0) + or np.any(bins[:, 0] >= height) + or np.any(bins[:, 1] <= 0) + or np.any(bins[:, 1] > width // 2) + or np.any(bins[:, 2] < 0) + or np.any(bins[:, 2] > 2) + ): + raise ValueError("candidate_bins contain out-of-range coordinates") + + image_values = np.empty((len(paths), len(bins)), dtype=np.complex128) + for index, path in enumerate(paths): + rgb = _load_rgb(path, height=height, width=width) + image_values[index] = extract_frequency_values( + transform_color_space(rgb, color_space), + bins[:, 0], + bins[:, 1], + bins[:, 2], + ) + magnitudes = np.abs(image_values) + units = np.divide(image_values, magnitudes, out=np.zeros_like(image_values), where=magnitudes != 0.0) + unit_sum = np.sum(units, axis=0) + count = float(len(paths)) + minimum_loo = np.ones(len(bins), dtype=np.float64) + for unit in units: + minimum_loo = np.minimum(minimum_loo, _leave_one_out_coherence(unit_sum, unit, count)) + expected_magnitudes = np.mean(magnitudes, axis=0) + selection = np.power(minimum_loo, 4.0) * np.log1p(expected_magnitudes) + chosen = np.argpartition(selection, -peak_count)[-peak_count:] + chosen = chosen[np.argsort(selection[chosen])[::-1]] + raw_weights = selection[chosen] + if np.sum(raw_weights) <= 0.0: + raise ValueError("candidate bins have no usable phase consensus") + selected = bins[chosen] + return ColorPhaseModel( + color_space=color_space, + height=height, + width=width, + rows=selected[:, 0].astype(np.int32), + columns=selected[:, 1].astype(np.int32), + channels=selected[:, 2].astype(np.int8), + phases=np.angle(unit_sum[chosen] / count).astype(np.float64), + weights=(raw_weights / np.sum(raw_weights)).astype(np.float64), + expected_magnitudes=expected_magnitudes[chosen].astype(np.float64), + ) + + +def score_image(path: Path, model: ColorPhaseModel) -> ColorPhaseScore: + """Score PATH against MODEL and expose additive channel evidence.""" + rgb = _load_rgb(path, height=model.height, width=model.width) + bins = np.column_stack((model.rows, model.columns, model.channels)) + values = extract_frequency_values( + transform_color_space(rgb, model.color_space), + bins[:, 0], + bins[:, 1], + bins[:, 2], + ) + magnitude_gate = np.minimum(np.abs(values) / (model.expected_magnitudes + 1e-12), 1.0) + contributions = model.weights * magnitude_gate * np.cos(np.angle(values) - model.phases) + active_weights = model.weights * magnitude_gate + active_weight = float(np.sum(active_weights)) + evidence = float(np.sum(contributions)) + phase_score = 0.0 if active_weight == 0.0 else evidence / active_weight + channel_evidence = tuple(float(np.sum(contributions[model.channels == channel])) for channel in range(3)) + selected_peak_counts = tuple(int(np.sum(model.channels == channel)) for channel in range(3)) + return ColorPhaseScore( + path=str(path), + color_space=model.color_space, + phase_score=phase_score, + active_weight_fraction=active_weight, + evidence_score=evidence, + channel_evidence=channel_evidence, + selected_peak_counts=selected_peak_counts, + peak_count=len(model.rows), + ) + + +def save_model(path: Path, model: ColorPhaseModel) -> None: + """Save MODEL as a pickle-free numeric NPZ.""" + path.parent.mkdir(parents=True, exist_ok=True) + np.savez_compressed( + path, + format_version=np.asarray(1, dtype=np.int32), + color_space=np.asarray(model.color_space), + height=np.asarray(model.height, dtype=np.int32), + width=np.asarray(model.width, dtype=np.int32), + rows=model.rows.astype(np.int32), + columns=model.columns.astype(np.int32), + channels=model.channels.astype(np.int8), + phases=model.phases.astype(np.float32), + weights=model.weights.astype(np.float32), + expected_magnitudes=model.expected_magnitudes.astype(np.float64), + ) + + +def load_model(path: Path) -> ColorPhaseModel: + """Load and validate one color-space phase-carrier artifact.""" + with np.load(path, allow_pickle=False) as artifact: + if int(artifact["format_version"]) != 1: + raise ValueError("unsupported color-phase model format version") + model = ColorPhaseModel( + color_space=str(artifact["color_space"]), + height=int(artifact["height"]), + width=int(artifact["width"]), + rows=np.asarray(artifact["rows"], dtype=np.int32), + columns=np.asarray(artifact["columns"], dtype=np.int32), + channels=np.asarray(artifact["channels"], dtype=np.int8), + phases=np.asarray(artifact["phases"], dtype=np.float64), + weights=np.asarray(artifact["weights"], dtype=np.float64), + expected_magnitudes=np.asarray(artifact["expected_magnitudes"], dtype=np.float64), + ) + count = len(model.rows) + arrays = (model.columns, model.channels, model.phases, model.weights, model.expected_magnitudes) + if model.color_space not in COLOR_SPACES: + raise ValueError("invalid model color space") + if model.height < 64 or model.width < 64 or count == 0 or any(array.shape != (count,) for array in arrays): + raise ValueError("invalid color-phase model shapes") + if np.any(model.rows < 0) or np.any(model.rows >= model.height): + raise ValueError("invalid color-phase row indices") + if np.any(model.columns <= 0) or np.any(model.columns > model.width // 2): + raise ValueError("invalid color-phase column indices") + if np.any(model.channels < 0) or np.any(model.channels > 2): + raise ValueError("invalid color-phase channel indices") + if not np.isclose(np.sum(model.weights), 1.0, atol=1e-5) or np.any(model.weights < 0.0): + raise ValueError("invalid color-phase weights") + if np.any(model.expected_magnitudes < 0.0): + raise ValueError("invalid expected magnitudes") + return model + + +@click.group() +def main() -> None: + """Discover and score phase carriers in multiple color spaces.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + + +@main.command() +@click.argument("codebook", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("positives", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--color-space", type=click.Choice(COLOR_SPACES), required=True) +@click.option("--source-peak-count", type=click.IntRange(min=1), default=256, show_default=True) +@click.option("--peak-count", type=click.IntRange(min=1), default=256, show_default=True) +@click.option("--model-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def discover( + codebook: Path, + positives: tuple[Path, ...], + color_space: str, + source_peak_count: int, + peak_count: int, + model_out: Path, +) -> None: + """Learn a color-space carrier from exact-geometry POSITIVES.""" + with Image.open(positives[0]) as first: + width, height = first.size + candidates = candidate_bins_from_codebook( + codebook, + height=height, + width=width, + source_peak_count=source_peak_count, + ) + model = discover_model( + list(positives), + color_space=color_space, + candidate_bins=candidates, + peak_count=peak_count, + ) + save_model(model_out, model) + log.info("Wrote %s color-phase model with %s candidates: %s", color_space, len(candidates), model_out) + + +@main.command() +@click.argument("model_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def score(model_path: Path, images: tuple[Path, ...], report_out: Path) -> None: + """Score exact-geometry IMAGES with MODEL_PATH.""" + model = load_model(model_path) + payload = { + "model": str(model_path), + "color_space": model.color_space, + "channel_names": CHANNEL_NAMES[model.color_space], + "height": model.height, + "width": model.width, + "peak_count": len(model.rows), + "scores": [asdict(score_image(image, model)) for image in images], + } + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") + log.info("Wrote %s color-phase score report: %s", model.color_space, report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_conformal_cascade.py b/scripts/synthid_conformal_cascade.py new file mode 100644 index 0000000..f68a846 --- /dev/null +++ b/scripts/synthid_conformal_cascade.py @@ -0,0 +1,352 @@ +"""Calibrate a versioned SynthID expert bank without forcing binary verdicts. + +This research utility combines already-computed pixel-only expert scores. It +does not inspect provenance, metadata, filenames, or provider labels at +inference. Expert support must be determined from predeclared geometry or model +scope, never from the observed score. + +The clean null is a union test: any supported expert may provide positive +evidence, so its smallest empirical upper-tail p-value receives a Bonferroni +correction. The watermarked hypothesis is itself a union over possible encoder +states and can be rejected only when every configured expert has complete +coverage and gives a small empirical lower-tail p-value. +""" + +from __future__ import annotations + +import bisect +import json +import logging +import math +from dataclasses import asdict, dataclass +from pathlib import Path +from typing import Literal, cast + +import click +from synthid_research_manifest import artifact_sha256 + +log = logging.getLogger(__name__) + +CascadeVerdict = Literal["detected", "not_detected", "abstain"] + + +@dataclass(frozen=True) +class ExpertCalibration: + """Frozen positive and negative score distributions for one expert.""" + + name: str + positive_scores: tuple[float, ...] + negative_scores: tuple[float, ...] + higher_is_positive: bool = True + + def __post_init__(self) -> None: + if not self.name: + raise ValueError("expert name must not be empty") + if not self.positive_scores or not self.negative_scores: + raise ValueError(f"expert {self.name!r} needs positive and negative calibration scores") + if not all(math.isfinite(score) for score in (*self.positive_scores, *self.negative_scores)): + raise ValueError(f"expert {self.name!r} contains a non-finite calibration score") + direction = 1.0 if self.higher_is_positive else -1.0 + object.__setattr__(self, "positive_scores", tuple(sorted(direction * score for score in self.positive_scores))) + object.__setattr__(self, "negative_scores", tuple(sorted(direction * score for score in self.negative_scores))) + + def orient(self, score: float) -> float: + """Return SCORE in the common higher-means-more-positive direction.""" + return score if self.higher_is_positive else -score + + +@dataclass(frozen=True) +class CascadeConfig: + """Calibration distributions and two-sided decision levels.""" + + experts: tuple[ExpertCalibration, ...] + positive_alpha: float + negative_alpha: float + coverage_complete: bool + scope: str + + def __post_init__(self) -> None: + if not self.experts: + raise ValueError("at least one expert is required") + names = [expert.name for expert in self.experts] + if len(set(names)) != len(names): + raise ValueError("expert names must be unique") + for label, value in (("positive_alpha", self.positive_alpha), ("negative_alpha", self.negative_alpha)): + if not 0.0 < value <= 1.0: + raise ValueError(f"{label} must be in (0, 1]") + if not self.scope: + raise ValueError("detector scope must not be empty") + + +@dataclass(frozen=True) +class ExpertObservation: + """One expert score, or an explicit unsupported result.""" + + name: str + supported: bool + score: float | None + + def __post_init__(self) -> None: + if not self.name: + raise ValueError("observation expert name must not be empty") + if self.supported: + if self.score is None or not math.isfinite(self.score): + raise ValueError(f"supported expert {self.name!r} needs a finite score") + elif self.score is not None: + raise ValueError(f"unsupported expert {self.name!r} must not provide a score") + + +@dataclass(frozen=True) +class ExpertEvidence: + """Two empirical p-values for one supported expert.""" + + name: str + score: float + clean_null_p_value: float + watermarked_p_value: float + positive_calibration_count: int + negative_calibration_count: int + + +@dataclass(frozen=True) +class CascadeResult: + """Auditable tri-state verdict for one observation record.""" + + verdict: CascadeVerdict + reason: str + clean_null_p_value: float | None + watermarked_p_value: float | None + supported_expert_count: int + configured_expert_count: int + coverage_complete: bool + evidence: tuple[ExpertEvidence, ...] + + +def _upper_tail_p_value(sorted_scores: tuple[float, ...], score: float) -> float: + """Smoothed empirical probability of a calibration score at least SCORE.""" + tail_count = len(sorted_scores) - bisect.bisect_left(sorted_scores, score) + return (tail_count + 1.0) / (len(sorted_scores) + 1.0) + + +def _lower_tail_p_value(sorted_scores: tuple[float, ...], score: float) -> float: + """Smoothed empirical probability of a calibration score at most SCORE.""" + tail_count = bisect.bisect_right(sorted_scores, score) + return (tail_count + 1.0) / (len(sorted_scores) + 1.0) + + +def classify_observations(config: CascadeConfig, observations: tuple[ExpertObservation, ...]) -> CascadeResult: + """Combine one explicit observation from every configured expert.""" + calibration_by_name = {expert.name: expert for expert in config.experts} + observation_by_name = {observation.name: observation for observation in observations} + if len(observation_by_name) != len(observations): + raise ValueError("observation expert names must be unique") + if observation_by_name.keys() != calibration_by_name.keys(): + missing = sorted(calibration_by_name.keys() - observation_by_name.keys()) + unknown = sorted(observation_by_name.keys() - calibration_by_name.keys()) + raise ValueError(f"observations must cover the configured bank; missing={missing}, unknown={unknown}") + + evidence: list[ExpertEvidence] = [] + for calibration in config.experts: + observation = observation_by_name[calibration.name] + if not observation.supported: + continue + if observation.score is None: + raise RuntimeError("validated supported observation lost its score") + oriented_score = calibration.orient(observation.score) + evidence.append( + ExpertEvidence( + name=calibration.name, + score=observation.score, + clean_null_p_value=_upper_tail_p_value(calibration.negative_scores, oriented_score), + watermarked_p_value=_lower_tail_p_value(calibration.positive_scores, oriented_score), + positive_calibration_count=len(calibration.positive_scores), + negative_calibration_count=len(calibration.negative_scores), + ) + ) + + if not evidence: + return CascadeResult( + verdict="abstain", + reason="unsupported", + clean_null_p_value=None, + watermarked_p_value=None, + supported_expert_count=0, + configured_expert_count=len(config.experts), + coverage_complete=config.coverage_complete, + evidence=(), + ) + + supported_count = len(evidence) + clean_null_p_value = min(1.0, supported_count * min(item.clean_null_p_value for item in evidence)) + watermarked_p_value = max(item.watermarked_p_value for item in evidence) + rejects_clean_null = clean_null_p_value <= config.positive_alpha + full_support = supported_count == len(config.experts) + rejects_watermarked = config.coverage_complete and full_support and watermarked_p_value <= config.negative_alpha + + if rejects_clean_null and rejects_watermarked: + verdict: CascadeVerdict = "abstain" + reason = "conflicting_evidence" + elif rejects_clean_null: + verdict = "detected" + reason = "watermarked_hypothesis_supported" + elif rejects_watermarked: + verdict = "not_detected" + reason = "unwatermarked_hypothesis_supported" + elif config.coverage_complete and not full_support: + verdict = "abstain" + reason = "incomplete_support" + elif not config.coverage_complete and watermarked_p_value <= config.negative_alpha: + verdict = "abstain" + reason = "incomplete_coverage" + else: + verdict = "abstain" + reason = "insufficient_evidence" + + return CascadeResult( + verdict=verdict, + reason=reason, + clean_null_p_value=clean_null_p_value, + watermarked_p_value=watermarked_p_value, + supported_expert_count=supported_count, + configured_expert_count=len(config.experts), + coverage_complete=config.coverage_complete, + evidence=tuple(evidence), + ) + + +def _mapping(value: object, label: str) -> dict[str, object]: + if not isinstance(value, dict): + raise ValueError(f"{label} must be an object") + return cast("dict[str, object]", value) + + +def _sequence(value: object, label: str) -> list[object]: + if not isinstance(value, list): + raise ValueError(f"{label} must be an array") + return cast("list[object]", value) + + +def _scores(value: object, label: str) -> tuple[float, ...]: + scores: list[float] = [] + for index, score in enumerate(_sequence(value, label)): + if isinstance(score, bool) or not isinstance(score, (int, float)): + raise ValueError(f"{label}[{index}] must be a number") + scores.append(float(score)) + return tuple(scores) + + +def _number(value: object, label: str) -> float: + if isinstance(value, bool) or not isinstance(value, (int, float)): + raise ValueError(f"{label} must be a number") + return float(value) + + +def _boolean(value: object, label: str) -> bool: + if not isinstance(value, bool): + raise ValueError(f"{label} must be a boolean") + return value + + +def _string(value: object, label: str) -> str: + if not isinstance(value, str) or not value: + raise ValueError(f"{label} must be a non-empty string") + return value + + +def load_config(path: Path) -> CascadeConfig: + """Load a schema-versioned calibration manifest.""" + payload = _mapping(json.loads(path.read_text(encoding="utf-8")), "calibration manifest") + if payload.get("schema_version") != 1: + raise ValueError("unsupported calibration manifest schema") + experts: list[ExpertCalibration] = [] + for index, raw_expert in enumerate(_sequence(payload.get("experts"), "experts")): + expert = _mapping(raw_expert, f"experts[{index}]") + experts.append( + ExpertCalibration( + name=_string(expert.get("name"), f"experts[{index}].name"), + positive_scores=_scores(expert.get("positive_scores"), f"experts[{index}].positive_scores"), + negative_scores=_scores(expert.get("negative_scores"), f"experts[{index}].negative_scores"), + higher_is_positive=_boolean( + expert.get("higher_is_positive", True), + f"experts[{index}].higher_is_positive", + ), + ) + ) + return CascadeConfig( + experts=tuple(experts), + positive_alpha=_number(payload.get("positive_alpha"), "positive_alpha"), + negative_alpha=_number(payload.get("negative_alpha"), "negative_alpha"), + coverage_complete=_boolean(payload.get("coverage_complete", False), "coverage_complete"), + scope=_string(payload.get("scope"), "scope"), + ) + + +def load_observation_records(path: Path) -> list[tuple[str, tuple[ExpertObservation, ...]]]: + """Load named score records with explicit support for every expert.""" + payload = _mapping(json.loads(path.read_text(encoding="utf-8")), "observation manifest") + if payload.get("schema_version") != 1: + raise ValueError("unsupported observation manifest schema") + records: list[tuple[str, tuple[ExpertObservation, ...]]] = [] + for record_index, raw_record in enumerate(_sequence(payload.get("records"), "records")): + record = _mapping(raw_record, f"records[{record_index}]") + record_id = _string(record.get("id"), f"records[{record_index}].id") + observations: list[ExpertObservation] = [] + for observation_index, raw_observation in enumerate( + _sequence(record.get("observations"), f"records[{record_index}].observations") + ): + observation = _mapping(raw_observation, f"records[{record_index}].observations[{observation_index}]") + raw_score = observation.get("score") + observations.append( + ExpertObservation( + name=_string( + observation.get("name"), + f"records[{record_index}].observations[{observation_index}].name", + ), + supported=_boolean( + observation.get("supported", False), + f"records[{record_index}].observations[{observation_index}].supported", + ), + score=None + if raw_score is None + else _number(raw_score, f"records[{record_index}].observations[{observation_index}].score"), + ) + ) + records.append((record_id, tuple(observations))) + return records + + +@click.command() +@click.argument("calibration_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("observation_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def main(calibration_path: Path, observation_path: Path, report_out: Path) -> None: + """Classify precomputed expert scores using CALIBRATION_PATH.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + config = load_config(calibration_path) + rows: list[dict[str, object]] = [] + verdict_counts: dict[CascadeVerdict, int] = {"detected": 0, "not_detected": 0, "abstain": 0} + for record_id, observations in load_observation_records(observation_path): + result = classify_observations(config, observations) + verdict_counts[result.verdict] += 1 + rows.append({"id": record_id, "result": asdict(result)}) + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text( + json.dumps( + { + "schema_version": 1, + "scope": config.scope, + "calibration_sha256": artifact_sha256(calibration_path), + "observation_sha256": artifact_sha256(observation_path), + "counts": verdict_counts, + "records": rows, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d conformal cascade verdicts: %s", len(rows), report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_confound_probe.py b/scripts/synthid_confound_probe.py new file mode 100644 index 0000000..8701f15 --- /dev/null +++ b/scripts/synthid_confound_probe.py @@ -0,0 +1,478 @@ +"""Measure how well non-watermark confounds predict provider SynthID labels. + +The experiment deliberately trains three provider-specific baselines: + +``container`` + File size, decoded geometry, aspect ratio, and format. +``thumbnail`` + Container features plus a small RGB thumbnail that can learn generator and + content style. +``canonical`` + Decoded, orientation-normalized RGB at a fixed geometry, with no container, + original-resolution, metadata, filename, or path features. + +These are challenge baselines, not SynthID detectors. A candidate detector must +beat the canonical baseline on same-provider hard negatives and a temporal +holdout before its result can be attributed to a watermark-specific signal. + +Usage: + uv run --extra pixels python scripts/synthid_confound_probe.py \ + .local-eval/synthid/manifest.csv --target-provider google \ + --report-out .local-eval/synthid/google-d1-confounds.json +""" + +from __future__ import annotations + +import csv +import json +import logging +from collections import Counter +from dataclasses import asdict, dataclass +from pathlib import Path + +import click +import numpy as np +from PIL import Image, ImageOps +from synthid_research_manifest import artifact_sha256, audit_manifest, resolve_artifact_path + +log = logging.getLogger(__name__) + +FEATURE_FAMILIES = ("container", "thumbnail", "canonical") +FINAL_SPLITS = ("train", "validation", "test", "temporal") +THUMBNAIL_SIZE = 8 +FORMAT_NAMES = ("png", "jpeg", "webp") +LOGISTIC_ITERATIONS = 2_000 +LOGISTIC_LEARNING_RATE = 0.2 +LOGISTIC_L2 = 0.01 + + +@dataclass(frozen=True) +class Example: + """One ordinary, provider-targeted manifest row eligible for D1.""" + + artifact_path: Path + artifact_sha256: str + group_id: str + split: str + label: int + negative_cohort: str | None + + +@dataclass(frozen=True) +class LogisticModel: + """Standardization and regularized logistic-regression parameters.""" + + mean: np.ndarray + scale: np.ndarray + weights: np.ndarray + bias: float + + +@dataclass(frozen=True) +class Metrics: + """Binary metrics at one validation-frozen threshold.""" + + count: int + positives: int + negatives: int + true_positives: int + false_positives: int + true_negatives: int + false_negatives: int + tpr: float | None + fpr: float | None + auc: float | None + + +def _safe_artifact_path(root: Path, value: str) -> Path: + """Resolve a manifest-relative path without accepting traversal.""" + candidate = resolve_artifact_path(root, value) + if candidate is None: + raise ValueError(f"unsafe artifact_path {value!r}") + return candidate + + +def _read_manifest(path: Path) -> list[dict[str, str]]: + """Read manifest rows after the caller has run the canonical auditor.""" + with path.open(newline="", encoding="utf-8") as stream: + return list(csv.DictReader(stream)) + + +def load_examples(manifest: Path, target_provider: str) -> list[Example]: + """Load ordinary final-label examples for one provider target. + + Candidate, sham, and source-control rows are deliberately excluded. A + remover-generated negative must not certify the detector that created it, + and repeated controls must not receive extra sample weight. + """ + root = manifest.parent + examples: list[Example] = [] + for row in _read_manifest(manifest): + if row.get("target_provider") != target_provider: + continue + if row.get("split") not in FINAL_SPLITS or row.get("oracle_role") != "ordinary": + continue + outcome = row.get("synthid_outcome") + if outcome not in {"detected", "not_detected"}: + continue + source_provider = row.get("source_provider", "") + if outcome == "detected" and source_provider != target_provider: + raise ValueError( + f"positive group {row.get('group_id')!r} targets {target_provider!r} " + f"but declares source_provider {source_provider!r}" + ) + negative_cohort: str | None = None + if outcome == "not_detected": + if source_provider == target_provider: + negative_cohort = "same_provider" + elif source_provider in {"openai", "google", "other_ai"}: + negative_cohort = "other_ai" + else: + negative_cohort = "external" + examples.append( + Example( + artifact_path=_safe_artifact_path(root, row.get("artifact_path", "")), + artifact_sha256=row.get("artifact_sha256", ""), + group_id=row.get("group_id", ""), + split=row.get("split", ""), + label=1 if outcome == "detected" else 0, + negative_cohort=negative_cohort, + ) + ) + if not examples: + raise ValueError(f"manifest has no eligible ordinary rows for target_provider={target_provider!r}") + return examples + + +def _decoded_rgb(path: Path) -> Image.Image: + """Return rendered RGB pixels with EXIF orientation applied once.""" + with Image.open(path) as image: + return ImageOps.exif_transpose(image).convert("RGB") + + +def _thumbnail_features(image: Image.Image) -> np.ndarray: + """Return a fixed-size RGB content fingerprint with no source geometry.""" + thumbnail = image.resize((THUMBNAIL_SIZE, THUMBNAIL_SIZE), Image.Resampling.LANCZOS) + return np.asarray(thumbnail, dtype=np.float64).reshape(-1) / 255.0 + + +def extract_feature_families(example: Example) -> dict[str, np.ndarray]: + """Extract every confounded feature family with one image decode.""" + with _decoded_rgb(example.artifact_path) as image: + width, height = image.size + thumbnail = _thumbnail_features(image) + file_size = example.artifact_path.stat().st_size + pixels = width * height + image_format = example.artifact_path.suffix.lower().lstrip(".") + if image_format == "jpg": + image_format = "jpeg" + container = np.asarray( + [ + np.log1p(width), + np.log1p(height), + np.log1p(pixels), + np.log1p(file_size), + width / height, + file_size / pixels, + *(1.0 if image_format == name else 0.0 for name in FORMAT_NAMES), + ], + dtype=np.float64, + ) + return { + "container": container, + "thumbnail": np.concatenate((container, thumbnail)), + "canonical": thumbnail, + } + + +def extract_features(example: Example, family: str) -> np.ndarray: + """Extract one deliberately confounded feature family.""" + if family not in FEATURE_FAMILIES: + raise ValueError(f"unsupported feature family {family!r}") + return extract_feature_families(example)[family] + + +def feature_matrix(examples: list[Example], family: str) -> np.ndarray: + """Extract a dense matrix in manifest order.""" + return np.stack([extract_features(example, family) for example in examples]) + + +def feature_matrices(examples: list[Example]) -> dict[str, np.ndarray]: + """Extract all dense feature matrices while decoding each artifact once.""" + rows = [extract_feature_families(example) for example in examples] + return {family: np.stack([row[family] for row in rows]) for family in FEATURE_FAMILIES} + + +def _balanced_sample_weights(labels: np.ndarray) -> np.ndarray: + """Give each class equal total weight regardless of corpus imbalance.""" + positives = int(np.sum(labels == 1)) + negatives = int(np.sum(labels == 0)) + if positives == 0 or negatives == 0: + raise ValueError("training requires at least one positive and one negative") + return np.where(labels == 1, 0.5 / positives, 0.5 / negatives) + + +def _sigmoid(values: np.ndarray) -> np.ndarray: + """Evaluate a numerically stable logistic sigmoid.""" + result = np.empty_like(values, dtype=np.float64) + positive = values >= 0 + result[positive] = 1.0 / (1.0 + np.exp(-values[positive])) + exponent = np.exp(values[~positive]) + result[~positive] = exponent / (1.0 + exponent) + return result + + +def fit_logistic( + features: np.ndarray, + labels: np.ndarray, + *, + iterations: int = LOGISTIC_ITERATIONS, + learning_rate: float = LOGISTIC_LEARNING_RATE, + l2: float = LOGISTIC_L2, +) -> LogisticModel: + """Fit deterministic class-balanced L2 logistic regression.""" + if features.ndim != 2 or labels.shape != (features.shape[0],): + raise ValueError("feature and label shapes are inconsistent") + if iterations < 1 or learning_rate <= 0.0 or l2 < 0.0: + raise ValueError("iterations and learning_rate must be positive; l2 must be nonnegative") + mean = np.mean(features, axis=0) + scale = np.std(features, axis=0) + scale = np.where(scale > 1e-12, scale, 1.0) + standardized = (features - mean) / scale + sample_weights = _balanced_sample_weights(labels) + weights = np.zeros(features.shape[1], dtype=np.float64) + bias = 0.0 + for _ in range(iterations): + probabilities = _sigmoid(standardized @ weights + bias) + error = (probabilities - labels) * sample_weights + gradient = standardized.T @ error + l2 * weights + weights -= learning_rate * gradient + bias -= learning_rate * float(np.sum(error)) + return LogisticModel(mean=mean, scale=scale, weights=weights, bias=bias) + + +def predict_scores(model: LogisticModel, features: np.ndarray) -> np.ndarray: + """Return positive-class probabilities for FEATURES.""" + standardized = (features - model.mean) / model.scale + return _sigmoid(standardized @ model.weights + model.bias) + + +def select_threshold(labels: np.ndarray, scores: np.ndarray, *, max_fpr: float) -> float: + """Choose the validation threshold with maximum TPR under MAX_FPR.""" + if labels.shape != scores.shape or labels.ndim != 1: + raise ValueError("validation labels and scores must be one-dimensional and aligned") + if not 0.0 <= max_fpr <= 1.0: + raise ValueError("max_fpr must be between zero and one") + if not np.any(labels == 1) or not np.any(labels == 0): + raise ValueError("threshold selection requires positive and negative validation examples") + candidates = [float(np.nextafter(np.max(scores), np.inf)), *sorted(set(map(float, scores)), reverse=True)] + best: tuple[float, float, float] | None = None + for threshold in candidates: + predicted = scores >= threshold + tpr = float(np.mean(predicted[labels == 1])) + fpr = float(np.mean(predicted[labels == 0])) + if fpr > max_fpr: + continue + candidate = (tpr, -fpr, threshold) + if best is None or candidate > best: + best = candidate + if best is None: + raise RuntimeError("threshold search found no feasible operating point") + return best[2] + + +def _auc(labels: np.ndarray, scores: np.ndarray) -> float | None: + """Return tie-aware ROC AUC, or None when one class is absent.""" + positive_count = int(np.sum(labels == 1)) + negative_count = int(np.sum(labels == 0)) + if positive_count == 0 or negative_count == 0: + return None + order = np.argsort(scores, kind="stable") + sorted_scores = scores[order] + ranks = np.empty(len(scores), dtype=np.float64) + start = 0 + while start < len(scores): + end = start + 1 + while end < len(scores) and sorted_scores[end] == sorted_scores[start]: + end += 1 + ranks[order[start:end]] = (start + 1 + end) / 2.0 + start = end + positive_rank_sum = float(np.sum(ranks[labels == 1])) + return (positive_rank_sum - positive_count * (positive_count + 1) / 2.0) / (positive_count * negative_count) + + +def calculate_metrics(labels: np.ndarray, scores: np.ndarray, threshold: float) -> Metrics: + """Calculate confusion counts, rates, and AUC at THRESHOLD.""" + predicted = scores >= threshold + positives = labels == 1 + negatives = ~positives + true_positives = int(np.sum(predicted & positives)) + false_positives = int(np.sum(predicted & negatives)) + true_negatives = int(np.sum(~predicted & negatives)) + false_negatives = int(np.sum(~predicted & positives)) + positive_count = int(np.sum(positives)) + negative_count = int(np.sum(negatives)) + return Metrics( + count=len(labels), + positives=positive_count, + negatives=negative_count, + true_positives=true_positives, + false_positives=false_positives, + true_negatives=true_negatives, + false_negatives=false_negatives, + tpr=true_positives / positive_count if positive_count else None, + fpr=false_positives / negative_count if negative_count else None, + auc=_auc(labels, scores), + ) + + +def _split_indices(examples: list[Example], split: str) -> np.ndarray: + """Return integer indices for SPLIT.""" + return np.asarray([index for index, example in enumerate(examples) if example.split == split], dtype=np.int64) + + +def _cohort_metrics( + examples: list[Example], scores: np.ndarray, threshold: float, split: str +) -> dict[str, dict[str, int | float | None]]: + """Report negative-only false-positive rates by provenance cohort.""" + result: dict[str, dict[str, int | float | None]] = {} + for cohort in ("same_provider", "other_ai", "external"): + indices = np.asarray( + [ + index + for index, example in enumerate(examples) + if example.split == split and example.negative_cohort == cohort + ], + dtype=np.int64, + ) + if not len(indices): + result[cohort] = {"count": 0, "false_positives": 0, "fpr": None} + continue + cohort_scores = scores[indices] + false_positives = int(np.sum(cohort_scores >= threshold)) + result[cohort] = { + "count": len(indices), + "false_positives": false_positives, + "fpr": false_positives / len(indices), + } + return result + + +def _require_split_classes(examples: list[Example], split: str) -> None: + """Require both labels in the train, validation, and locked test splits.""" + labels = {example.label for example in examples if example.split == split} + if labels != {0, 1}: + raise ValueError(f"split {split!r} must contain at least one ordinary positive and negative") + + +def run_experiment( + manifest: Path, + target_provider: str, + *, + max_fpr: float = 0.001, + verify_files: bool = True, +) -> dict[str, object]: + """Audit MANIFEST, train all confound baselines, and return a JSON-safe report.""" + errors = audit_manifest(manifest, verify_files=verify_files) + if errors: + preview = "; ".join(errors[:5]) + raise ValueError(f"manifest audit failed with {len(errors)} error(s): {preview}") + examples = load_examples(manifest, target_provider) + for split in ("train", "validation", "test"): + _require_split_classes(examples, split) + + labels = np.asarray([example.label for example in examples], dtype=np.int64) + split_counts = Counter(example.split for example in examples) + negative_counts = Counter(example.negative_cohort for example in examples if example.negative_cohort is not None) + train_indices = _split_indices(examples, "train") + validation_indices = _split_indices(examples, "validation") + report_families: dict[str, object] = {} + matrices = feature_matrices(examples) + + for family in FEATURE_FAMILIES: + features = matrices[family] + model = fit_logistic(features[train_indices], labels[train_indices]) + scores = predict_scores(model, features) + threshold = select_threshold(labels[validation_indices], scores[validation_indices], max_fpr=max_fpr) + split_metrics: dict[str, object] = {} + cohort_metrics: dict[str, object] = {} + for split in FINAL_SPLITS: + indices = _split_indices(examples, split) + if not len(indices): + split_metrics[split] = None + cohort_metrics[split] = { + cohort: {"count": 0, "false_positives": 0, "fpr": None} + for cohort in ("same_provider", "other_ai", "external") + } + continue + split_metrics[split] = asdict(calculate_metrics(labels[indices], scores[indices], threshold)) + cohort_metrics[split] = _cohort_metrics(examples, scores, threshold, split) + report_families[family] = { + "feature_count": features.shape[1], + "threshold": threshold, + "metrics": split_metrics, + "negative_cohorts": cohort_metrics, + } + + manifest_digest = artifact_sha256(manifest) + has_same_provider_test = any( + example.split == "test" and example.negative_cohort == "same_provider" for example in examples + ) + has_temporal_both_classes = {example.label for example in examples if example.split == "temporal"} == {0, 1} + return { + "schema_version": 1, + "experiment": "synthid-d1-confounds", + "target_provider": target_provider, + "manifest_sha256": manifest_digest, + "max_validation_fpr": max_fpr, + "eligible_examples": len(examples), + "split_counts": dict(sorted(split_counts.items())), + "negative_cohort_counts": dict(sorted(negative_counts.items())), + "evidence_ready": has_same_provider_test and has_temporal_both_classes, + "evidence_missing": [ + reason + for missing, reason in ( + (not has_same_provider_test, "locked test has no same-provider hard negative"), + (not has_temporal_both_classes, "temporal split does not contain both labels"), + ) + if missing + ], + "feature_schema": { + "families": list(FEATURE_FAMILIES), + "thumbnail_size": THUMBNAIL_SIZE, + "format_names": list(FORMAT_NAMES), + }, + "logistic_regression": { + "iterations": LOGISTIC_ITERATIONS, + "learning_rate": LOGISTIC_LEARNING_RATE, + "l2": LOGISTIC_L2, + "class_balancing": "equal total weight per class", + }, + "families": report_families, + } + + +@click.command() +@click.argument("manifest", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--target-provider", required=True, type=click.Choice(["openai", "google"])) +@click.option("--report-out", required=True, type=click.Path(dir_okay=False, path_type=Path)) +@click.option("--max-fpr", type=click.FloatRange(0.0, 1.0), default=0.001, show_default=True) +@click.option("--verify-files/--no-verify-files", default=True, show_default=True) +def main(manifest: Path, target_provider: str, report_out: Path, max_fpr: float, verify_files: bool) -> None: + """Run D1 confound baselines from a provider-specific research MANIFEST.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + try: + report = run_experiment(manifest, target_provider, max_fpr=max_fpr, verify_files=verify_files) + except (OSError, ValueError) as exc: + raise click.ClickException(str(exc)) from exc + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text(json.dumps(report, indent=2, sort_keys=True) + "\n", encoding="utf-8") + log.info("Wrote D1 confound report: %s", report_out) + if report["evidence_ready"]: + log.info("D1 report contains same-provider locked negatives and a two-class temporal holdout") + else: + log.warning("D1 report lacks required evidence: %s", "; ".join(report["evidence_missing"])) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_consensus_probe.py b/scripts/synthid_consensus_probe.py new file mode 100644 index 0000000..7d03fcf --- /dev/null +++ b/scripts/synthid_consensus_probe.py @@ -0,0 +1,259 @@ +"""Discover a polarity-invariant spectral carrier from low-texture groups. + +This research harness is intentionally separate from the shipping detector. It +uses repeated, independently generated low-texture images to find Fourier bins +whose phase is stable within a content group and whose phase axis is stable +across groups. Treat its output as a carrier hypothesis until it passes the +provider-oracle and hard-negative gates in the SynthID research plan. + +Usage: + uv run --extra pixels python scripts/synthid_consensus_probe.py discover \ + refs/black refs/white refs/red --limit 5 \ + --model-out .local-eval/synthid/consensus.npz + + uv run --extra pixels python scripts/synthid_consensus_probe.py score \ + .local-eval/synthid/consensus.npz images/*.png +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict, dataclass +from pathlib import Path + +import click +import numpy as np +from PIL import Image, ImageFilter + +log = logging.getLogger(__name__) + +IMAGE_SUFFIXES = {".jpeg", ".jpg", ".png", ".webp"} + + +@dataclass(frozen=True) +class ConsensusScore: + """One image's alignment with a frozen carrier hypothesis.""" + + path: str + score: float + active_weight_fraction: float + peak_count: int + + +@dataclass(frozen=True) +class ConsensusModel: + """A compact, pickle-free carrier hypothesis.""" + + size: int + peaks: np.ndarray + axial_phase: np.ndarray + weights: np.ndarray + expected_magnitude: np.ndarray + + +def _image_paths(directory: Path, limit: int | None) -> list[Path]: + """Return a deterministic list of supported images in DIRECTORY.""" + paths = sorted(path for path in directory.iterdir() if path.suffix.lower() in IMAGE_SUFFIXES) + if limit is not None: + paths = paths[:limit] + if not paths: + raise ValueError(f"no supported images in {directory}") + return paths + + +def _high_pass_rgb(path: Path, size: int, blur_radius: float) -> np.ndarray: + """Decode PATH, canonicalize its geometry, and remove local image content.""" + with Image.open(path) as source: + image = source.convert("RGB").resize((size, size), Image.Resampling.LANCZOS) + pixels = np.asarray(image, dtype=np.float64) + blurred = np.asarray(image.filter(ImageFilter.GaussianBlur(radius=blur_radius)), dtype=np.float64) + return pixels - blurred + + +def _spectrum(path: Path, size: int, blur_radius: float) -> np.ndarray: + """Return a centered channel-wise spectrum for PATH.""" + residual = _high_pass_rgb(path, size, blur_radius) + return np.fft.fftshift(np.fft.fft2(residual, axes=(0, 1)), axes=(0, 1)) + + +def _group_statistics(paths: list[Path], size: int, blur_radius: float) -> tuple[np.ndarray, np.ndarray]: + """Return phase coherence and mean magnitude for one reference group.""" + spectra = [_spectrum(path, size, blur_radius) for path in paths] + units = [spectrum / (np.abs(spectrum) + 1e-12) for spectrum in spectra] + mean_unit = np.mean(units, axis=0) + coherence = np.abs(mean_unit) + mean_magnitude = np.mean([np.abs(spectrum) for spectrum in spectra], axis=0) + phase = np.angle(mean_unit) + return coherence * np.exp(1j * phase), mean_magnitude + + +def _valid_half_plane(size: int, min_radius: float, max_radius_fraction: float) -> np.ndarray: + """Return the nonredundant Fourier region allowed for carrier selection.""" + center = size // 2 + yy, xx = np.ogrid[:size, :size] + dy = yy - center + dx = xx - center + radius = np.sqrt(np.square(dy) + np.square(dx)) + half_plane = (dy > 0) | ((dy == 0) & (dx > 0)) + off_axis = (dy != 0) & (dx != 0) + return half_plane & off_axis & (radius >= min_radius) & (radius <= size * max_radius_fraction) + + +def discover_model( + groups: list[list[Path]], + *, + size: int = 512, + blur_radius: float = 2.0, + peak_count: int = 256, + min_radius: float = 8.0, + max_radius_fraction: float = 0.4, +) -> ConsensusModel: + """Discover a polarity-invariant carrier from independent image GROUPS.""" + if len(groups) < 2: + raise ValueError("at least two reference groups are required") + if any(len(group) < 2 for group in groups): + raise ValueError("each reference group requires at least two images") + + group_units: list[np.ndarray] = [] + group_magnitudes: list[np.ndarray] = [] + for paths in groups: + unit, magnitude = _group_statistics(paths, size, blur_radius) + group_units.append(unit) + group_magnitudes.append(magnitude) + + stacked = np.stack(group_units, axis=0) + within_coherence = np.abs(stacked) + group_phase = np.angle(stacked) + axial_mean = np.mean(np.exp(2j * group_phase) * within_coherence, axis=0) + axial_coherence = np.abs(axial_mean) / (np.mean(within_coherence, axis=0) + 1e-12) + mean_within = np.mean(within_coherence, axis=0) + expected_magnitude = np.mean(group_magnitudes, axis=0) + + magnitude_scale = np.median(expected_magnitude, axis=(0, 1), keepdims=True) + 1e-12 + magnitude_score = np.log1p(expected_magnitude / magnitude_scale) + selection_score = np.square(mean_within) * np.square(axial_coherence) * magnitude_score + selection_score *= _valid_half_plane(size, min_radius, max_radius_fraction)[:, :, None] + + candidate_count = int(np.count_nonzero(selection_score)) + if candidate_count < peak_count: + raise ValueError(f"only {candidate_count} valid carrier candidates for {peak_count} peaks") + flat = selection_score.ravel() + indices = np.argpartition(flat, -peak_count)[-peak_count:] + indices = indices[np.argsort(flat[indices])[::-1]] + rows, columns, channels = np.unravel_index(indices, selection_score.shape) + peaks = np.column_stack((rows - size // 2, columns - size // 2, channels)).astype(np.int32) + + axial_phase = 0.5 * np.angle(axial_mean[rows, columns, channels]) + weights = selection_score[rows, columns, channels] + weights /= np.sum(weights) + magnitudes = expected_magnitude[rows, columns, channels] + return ConsensusModel( + size=size, + peaks=peaks, + axial_phase=axial_phase.astype(np.float64), + weights=weights.astype(np.float64), + expected_magnitude=magnitudes.astype(np.float64), + ) + + +def score_image(path: Path, model: ConsensusModel, *, blur_radius: float = 2.0) -> ConsensusScore: + """Score PATH against a frozen polarity-invariant carrier model.""" + spectrum = _spectrum(path, model.size, blur_radius) + center = model.size // 2 + rows = center + model.peaks[:, 0] + columns = center + model.peaks[:, 1] + channels = model.peaks[:, 2] + values = spectrum[rows, columns, channels] + phase_alignment = np.cos(2.0 * (np.angle(values) - model.axial_phase)) + magnitude_gate = np.minimum(np.abs(values) / (model.expected_magnitude + 1e-12), 1.0) + active_weights = model.weights * magnitude_gate + active_weight = float(np.sum(active_weights)) + score = 0.0 if active_weight == 0.0 else float(np.sum(active_weights * phase_alignment) / active_weight) + return ConsensusScore( + path=str(path), + score=score, + active_weight_fraction=active_weight, + peak_count=len(model.peaks), + ) + + +def save_model(path: Path, model: ConsensusModel) -> None: + """Save MODEL as a pickle-free NPZ artifact.""" + path.parent.mkdir(parents=True, exist_ok=True) + np.savez_compressed( + path, + size=np.asarray(model.size, dtype=np.int32), + peaks=model.peaks.astype(np.int32), + axial_phase=model.axial_phase.astype(np.float32), + weights=model.weights.astype(np.float32), + expected_magnitude=model.expected_magnitude.astype(np.float32), + ) + + +def load_model(path: Path) -> ConsensusModel: + """Load and validate a pickle-free carrier model.""" + with np.load(path, allow_pickle=False) as artifact: + model = ConsensusModel( + size=int(artifact["size"]), + peaks=np.asarray(artifact["peaks"], dtype=np.int32), + axial_phase=np.asarray(artifact["axial_phase"], dtype=np.float64), + weights=np.asarray(artifact["weights"], dtype=np.float64), + expected_magnitude=np.asarray(artifact["expected_magnitude"], dtype=np.float64), + ) + count = len(model.peaks) + if model.peaks.ndim != 2 or model.peaks.shape[1] != 3: + raise ValueError("invalid peak shape") + if any(array.shape != (count,) for array in (model.axial_phase, model.weights, model.expected_magnitude)): + raise ValueError("model arrays do not match peak count") + if model.size < 64 or np.any(np.abs(model.peaks[:, :2]) >= model.size // 2): + raise ValueError("invalid canonical size or peak coordinates") + if np.any(model.peaks[:, 2] < 0) or np.any(model.peaks[:, 2] > 2): + raise ValueError("invalid channel index") + if not np.isclose(np.sum(model.weights), 1.0, atol=1e-5): + raise ValueError("model weights must sum to one") + return model + + +@click.group() +def main() -> None: + """Run low-texture carrier discovery and scoring experiments.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + + +@main.command() +@click.argument("group_dirs", nargs=-1, required=True, type=click.Path(exists=True, file_okay=False, path_type=Path)) +@click.option("--limit", type=click.IntRange(min=2)) +@click.option("--size", type=click.IntRange(min=64), default=512, show_default=True) +@click.option("--peak-count", type=click.IntRange(min=1), default=256, show_default=True) +@click.option("--model-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def discover(group_dirs: tuple[Path, ...], limit: int | None, size: int, peak_count: int, model_out: Path) -> None: + """Discover a carrier from the images in each GROUP_DIRS directory.""" + groups = [_image_paths(directory, limit) for directory in group_dirs] + model = discover_model(groups, size=size, peak_count=peak_count) + save_model(model_out, model) + log.info("Wrote consensus model: %s", model_out) + + +@main.command() +@click.argument("model_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path)) +def score(model_path: Path, images: tuple[Path, ...], report_out: Path | None) -> None: + """Score IMAGES against MODEL_PATH.""" + model = load_model(model_path) + payload = { + "model": str(model_path), + "scores": [asdict(score_image(image, model)) for image in images], + } + rendered = json.dumps(payload, indent=2) + "\n" + if report_out is None: + log.info("%s", rendered.rstrip()) + return + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text(rendered, encoding="utf-8") + log.info("Wrote score report: %s", report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_cyclostationary_probe.py b/scripts/synthid_cyclostationary_probe.py new file mode 100644 index 0000000..6e4e9d1 --- /dev/null +++ b/scripts/synthid_cyclostationary_probe.py @@ -0,0 +1,228 @@ +"""Probe complex cross-spectral coupling at preregistered carrier shifts.""" + +from __future__ import annotations + +import json +import logging +import math +from dataclasses import asdict, dataclass +from pathlib import Path +from typing import TYPE_CHECKING, Any + +import click +import cv2 +import numpy as np +from synthid_affine_lattice_probe import ( + _canonical_pixels, + _opponent_channels, + _patch_origins, + template_harmonics, +) +from synthid_pixel_attack import load_rgb + +if TYPE_CHECKING: + from numpy.typing import NDArray + +log = logging.getLogger(__name__) + +_OFF_CARRIER_OFFSETS = ((1, 0), (-1, 0), (0, 1), (0, -1), (1, 1), (-1, -1)) + + +@dataclass(frozen=True) +class CyclostationaryScore: + """Complex carrier-versus-neighbor contrast on disjoint patch groups.""" + + selection_carrier: float + selection_off_carrier_median: float + selection_contrast: float + confirmation_carrier: float + confirmation_off_carrier_median: float + confirmation_contrast: float + joint_contrast: float + harmonic_count: int + selection_patches: int + confirmation_patches: int + + +def _patch_cyclic_matrices( + pixels: NDArray[Any], + harmonics: NDArray[Any], + *, + tile_size: int, + denoise_sigma: float, + band_min: float, + band_max: float, +) -> NDArray[Any]: + """Return normalized complex cross-channel matrices for one patch.""" + patch_size = pixels.shape[0] + if pixels.shape[:2] != (patch_size, patch_size) or patch_size % tile_size: + raise ValueError("cyclostationary patches must be square multiples of the tile size") + channels = _opponent_channels(np.asarray(pixels, dtype=np.float64)) + for channel in range(channels.shape[2]): + residual = channels[:, :, channel] + channels[:, :, channel] = residual - cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=denoise_sigma, + sigmaY=denoise_sigma, + borderType=cv2.BORDER_REFLECT_101, + ) + window_1d = np.hanning(patch_size) + window = window_1d[:, None] * window_1d[None, :] + spectrum = np.fft.fft2(channels * window[:, :, None], axes=(0, 1)) + frequency_y = np.fft.fftfreq(patch_size)[:, None] + frequency_x = np.fft.fftfreq(patch_size)[None, :] + radius = np.sqrt(frequency_y * frequency_y + frequency_x * frequency_x) + base_mask = (radius >= band_min) & (radius <= band_max) + + offset_values = ((0, 0), *_OFF_CARRIER_OFFSETS) + matrices = np.empty((len(harmonics), len(offset_values), 3, 3), dtype=np.complex128) + for harmonic_index, (signed_row_value, signed_column_value) in enumerate(harmonics): + alpha_y = round(float(signed_row_value) * patch_size / tile_size) + alpha_x = round(float(signed_column_value) * patch_size / tile_size) + for offset_index, (offset_y, offset_x) in enumerate(offset_values): + shift_y = alpha_y + offset_y + shift_x = alpha_x + offset_x + shifted = np.roll(spectrum, shift=(-shift_y, -shift_x), axis=(0, 1)) + mask = base_mask & np.roll(base_mask, shift=(-shift_y, -shift_x), axis=(0, 1)) + base_values = spectrum[mask] + shifted_values = shifted[mask] + normalizer = math.sqrt(float(np.sum(np.abs(base_values) ** 2)) * float(np.sum(np.abs(shifted_values) ** 2))) + if normalizer <= 1e-12: + matrices[harmonic_index, offset_index] = 0.0 + else: + matrices[harmonic_index, offset_index] = shifted_values.T @ np.conj(base_values) / normalizer + return matrices + + +def _group_score(values: list[NDArray[Any]], harmonic_weights: NDArray[Any]) -> tuple[float, float, float]: + if not values: + raise ValueError("cyclostationary score needs at least one patch") + mean_matrices = np.mean(np.stack(values), axis=0) + coherence = np.linalg.norm(mean_matrices, axis=(2, 3)) + carrier = float(np.sum(coherence[:, 0] * harmonic_weights)) + off_scores = [ + float(np.sum(coherence[:, offset_index] * harmonic_weights)) for offset_index in range(1, coherence.shape[1]) + ] + off_median = float(np.median(off_scores)) + return carrier, off_median, carrier - off_median + + +def score_cyclostationary( + pixels: NDArray[Any], + template: NDArray[Any], + *, + period: float, + patch_size: int = 256, + grid_size: int = 4, + harmonic_count: int = 8, + denoise_sigma: float = 1.0, + band_min: float = 0.05, + band_max: float = 0.35, +) -> CyclostationaryScore: + """Measure split-confirmed complex spectral coupling at one period.""" + if pixels.ndim != 3 or pixels.shape[2] != 3: + raise ValueError("pixels must have shape (height, width, 3)") + if not math.isfinite(period) or period <= 0.0: + raise ValueError("period must be finite and positive") + if not (0.0 < band_min < band_max < 0.5): + raise ValueError("frequency band must satisfy 0 < min < max < 0.5") + if not math.isfinite(denoise_sigma) or denoise_sigma <= 0.0: + raise ValueError("denoise sigma must be finite and positive") + + canonical = _canonical_pixels(pixels, template, period) + tile_size = template.shape[0] + harmonics, channel_weights, _coefficient_units = template_harmonics(template, harmonic_count) + harmonic_weights = np.linalg.norm(channel_weights, axis=1) + harmonic_weights /= np.sum(harmonic_weights) + grouped_values: dict[int, list[NDArray[Any]]] = {0: [], 1: []} + for origin_y, origin_x, group in _patch_origins(*canonical.shape[:2], patch_size, grid_size): + aligned_y = (origin_y // tile_size) * tile_size + aligned_x = (origin_x // tile_size) * tile_size + patch = canonical[aligned_y : aligned_y + patch_size, aligned_x : aligned_x + patch_size] + grouped_values[group].append( + _patch_cyclic_matrices( + patch, + harmonics, + tile_size=tile_size, + denoise_sigma=denoise_sigma, + band_min=band_min, + band_max=band_max, + ) + ) + selection_carrier, selection_null, selection_contrast = _group_score(grouped_values[0], harmonic_weights) + confirmation_carrier, confirmation_null, confirmation_contrast = _group_score(grouped_values[1], harmonic_weights) + return CyclostationaryScore( + selection_carrier=selection_carrier, + selection_off_carrier_median=selection_null, + selection_contrast=selection_contrast, + confirmation_carrier=confirmation_carrier, + confirmation_off_carrier_median=confirmation_null, + confirmation_contrast=confirmation_contrast, + joint_contrast=min(selection_contrast, confirmation_contrast), + harmonic_count=len(harmonics), + selection_patches=len(grouped_values[0]), + confirmation_patches=len(grouped_values[1]), + ) + + +def _load_template(path: Path) -> NDArray[Any]: + with np.load(path, allow_pickle=False) as artifact: + template = np.asarray(artifact["template"], dtype=np.float64) + if template.shape != (16, 16, 3) or not np.all(np.isfinite(template)): + raise ValueError("template artifact does not contain a finite 16x16 RGB template") + return template + + +@click.command() +@click.argument("template_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--period", type=click.FloatRange(min=1.0), required=True) +@click.option("--input-scale", type=click.FloatRange(min=0.01), default=1.0, show_default=True) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def main( + template_path: Path, + images: tuple[Path, ...], + period: float, + input_scale: float, + report_out: Path, +) -> None: + """Score IMAGES for complex cross-spectral carrier coupling.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + template = _load_template(template_path) + rows = [] + for path in images: + try: + pixels = load_rgb(path) + if input_scale != 1.0: + width = max(1, round(pixels.shape[1] * input_scale)) + height = max(1, round(pixels.shape[0] * input_scale)) + interpolation = cv2.INTER_AREA if input_scale < 1.0 else cv2.INTER_CUBIC + pixels = cv2.resize(pixels, (width, height), interpolation=interpolation) + score = score_cyclostationary(pixels, template, period=period) + except ValueError as error: + rows.append({"path": str(path), "status": "unsupported", "reason": str(error)}) + log.warning("%s: unsupported: %s", path, error) + continue + rows.append({"path": str(path), "status": "scored", "score": asdict(score)}) + log.info("%s: joint contrast=%.6f", path, score.joint_contrast) + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text( + json.dumps( + { + "schema_version": 1, + "template": str(template_path), + "period": period, + "input_scale": input_scale, + "records": rows, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d cyclostationary records: %s", len(rows), report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_ensemble_attack.py b/scripts/synthid_ensemble_attack.py new file mode 100644 index 0000000..5a46f58 --- /dev/null +++ b/scripts/synthid_ensemble_attack.py @@ -0,0 +1,223 @@ +"""Build pixel-only alternating-projection candidates for the ensemble detector. + +The attack removes only the positive complex-spectrum projection onto the +learned RGB phases and HSV saturation/value phases. It preserves geometry and +does not invoke a generative model. Clearing the local research detector is not +proof that a provider oracle will clear SynthID. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict +from pathlib import Path +from typing import TYPE_CHECKING + +import click +import cv2 +import numpy as np +from PIL import Image +from synthid_ensemble_detector import EnsembleConfig, detect_image, load_config, load_models +from synthid_pixel_attack import load_rgb, measure, norm_matched_noise + +if TYPE_CHECKING: + from synthid_color_space_probe import ColorPhaseModel + +log = logging.getLogger(__name__) + + +def _remove_positive_projection( + channel: np.ndarray, + *, + rows: np.ndarray, + columns: np.ndarray, + phases: np.ndarray, + strength: float, +) -> np.ndarray: + """Remove STRENGTH of each positive phase projection from CHANNEL.""" + if strength < 0.0 or strength > 1.0: + raise ValueError("strength must be between zero and one") + height, width = channel.shape + spectrum = np.fft.fft2(channel.astype(np.float64)) + for row, column, phase in zip(rows, columns, phases, strict=True): + row_index = int(row) + column_index = int(column) + direction = np.exp(1j * float(phase)) + value = spectrum[row_index, column_index] + projection = max(0.0, float(np.real(value * np.conj(direction)))) + delta = strength * projection * direction + conjugate_row = (-row_index) % height + conjugate_column = (-column_index) % width + spectrum[row_index, column_index] -= delta + if (conjugate_row, conjugate_column) == (row_index, column_index): + spectrum[row_index, column_index] = complex(spectrum[row_index, column_index].real, 0.0) + else: + spectrum[conjugate_row, conjugate_column] -= np.conj(delta) + return np.fft.ifft2(spectrum).real + + +def _project_model_channels( + pixels: np.ndarray, + model: ColorPhaseModel, + *, + included_channels: frozenset[int], + strength: float, +) -> np.ndarray: + """Apply positive-projection removal to selected MODEL channels.""" + result = pixels.astype(np.float64, copy=True) + for channel in included_channels: + positions = np.flatnonzero(model.channels == channel) + if len(positions) == 0: + continue + result[:, :, channel] = _remove_positive_projection( + result[:, :, channel], + rows=model.rows[positions], + columns=model.columns[positions], + phases=model.phases[positions], + strength=strength, + ) + return result + + +def alternating_projection( + pixels: np.ndarray, + rgb_model: ColorPhaseModel, + hsv_model: ColorPhaseModel, + *, + strength: float, + iterations: int, +) -> np.ndarray: + """Alternate RGB and HSV S/V phase projections without regeneration.""" + if iterations < 1: + raise ValueError("iterations must be positive") + expected_shape = (rgb_model.height, rgb_model.width, 3) + if pixels.shape != expected_shape or pixels.shape != (hsv_model.height, hsv_model.width, 3): + raise ValueError("pixel and model geometries do not match") + result = pixels.astype(np.float64) + for _ in range(iterations): + result = _project_model_channels( + result, + rgb_model, + included_channels=frozenset({0, 1, 2}), + strength=strength, + ) + rgb_unit = np.clip(result / 255.0, 0.0, 1.0).astype(np.float32) + hsv = cv2.cvtColor(rgb_unit, cv2.COLOR_RGB2HSV).astype(np.float64) + hsv = _project_model_channels( + hsv, + hsv_model, + included_channels=frozenset({1, 2}), + strength=strength, + ) + hsv[:, :, 0] = np.mod(hsv[:, :, 0], 360.0) + hsv[:, :, 1:] = np.clip(hsv[:, :, 1:], 0.0, 1.0) + result = cv2.cvtColor(hsv.astype(np.float32), cv2.COLOR_HSV2RGB).astype(np.float64) * 255.0 + return np.clip(np.rint(result), 0, 255).astype(np.uint8) + + +def parse_positive_floats(value: str) -> tuple[float, ...]: + """Parse strictly increasing strengths in the interval (0, 1].""" + try: + values = tuple(float(item.strip()) for item in value.split(",")) + except ValueError as error: + raise click.BadParameter("strengths must be comma-separated numbers") from error + if not values or any(not np.isfinite(item) or item <= 0.0 or item > 1.0 for item in values): + raise click.BadParameter("strengths must be finite and in the interval (0, 1]") + if tuple(sorted(set(values))) != values: + raise click.BadParameter("strengths must be unique and strictly increasing") + return values + + +def parse_positive_integers(value: str) -> tuple[int, ...]: + """Parse strictly increasing positive iteration counts.""" + try: + values = tuple(int(item.strip()) for item in value.split(",")) + except ValueError as error: + raise click.BadParameter("iterations must be comma-separated integers") from error + if not values or any(item < 1 for item in values): + raise click.BadParameter("iterations must be positive") + if tuple(sorted(set(values))) != values: + raise click.BadParameter("iterations must be unique and strictly increasing") + return values + + +def _load_source(path: Path, config: EnsembleConfig) -> np.ndarray: + """Load an exact-geometry RGB source.""" + pixels = load_rgb(path) + if pixels.shape != (config.height, config.width, 3): + raise ValueError("source geometry does not match detector config") + return pixels + + +@click.command() +@click.argument("config_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("output_dir", type=click.Path(file_okay=False, path_type=Path)) +@click.option("--strengths", default="0.25,0.5,0.75,1", show_default=True) +@click.option("--iterations", default="1,2,4", show_default=True) +def main(config_path: Path, source: Path, output_dir: Path, strengths: str, iterations: str) -> None: + """Write a frozen pixel-only alternating-projection batch for SOURCE.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + config = load_config(config_path) + rgb_model, hsv_model = load_models(config) + reference = _load_source(source, config) + strength_values = parse_positive_floats(strengths) + iteration_values = parse_positive_integers(iterations) + output_dir.mkdir(parents=True, exist_ok=True) + + variants: list[dict[str, object]] = [] + strongest = reference + for iteration_count in iteration_values: + for strength in strength_values: + pixels = alternating_projection( + reference, + rgb_model, + hsv_model, + strength=strength, + iterations=iteration_count, + ) + strength_name = f"{strength:g}".replace(".", "p") + name = f"project-s{strength_name}-i{iteration_count}" + path = output_dir / f"{name}.png" + Image.fromarray(pixels, mode="RGB").save(path) + variants.append( + { + **asdict(measure(reference, pixels, name=name, path=path)), + **asdict(detect_image(path, config, rgb_model, hsv_model)), + "strength": strength, + "iterations": iteration_count, + } + ) + strongest = pixels + + sham = norm_matched_noise(reference, strongest, seed=20260823) + sham_path = output_dir / "sham-strongest-rms.png" + Image.fromarray(sham, mode="RGB").save(sham_path) + variants.append( + { + **asdict(measure(reference, sham, name="sham-strongest-rms", path=sham_path)), + **asdict(detect_image(sham_path, config, rgb_model, hsv_model)), + "strength": None, + "iterations": None, + } + ) + + report_path = output_dir / "report.json" + report_path.write_text( + json.dumps( + { + "source": str(source), + "config": str(config_path), + "variants": variants, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d alternating-projection candidates: %s", len(variants), report_path) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_ensemble_detector.py b/scripts/synthid_ensemble_detector.py new file mode 100644 index 0000000..26c436d --- /dev/null +++ b/scripts/synthid_ensemble_detector.py @@ -0,0 +1,188 @@ +"""Run a positive-only exact-geometry SynthID research detector. + +The detector requires independently frozen RGB and HSV phase models. It emits +``positive`` only when both branches and both support gates pass; every other +case is ``abstain``. It never claims that SynthID is absent. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict, dataclass +from pathlib import Path + +import click +from PIL import Image +from synthid_color_space_probe import ColorPhaseModel, ColorPhaseScore, load_model, score_image +from synthid_research_manifest import artifact_sha256 + +log = logging.getLogger(__name__) + + +@dataclass(frozen=True) +class EnsembleConfig: + """Frozen paths, hashes, geometry, and positive thresholds.""" + + width: int + height: int + rgb_model_path: Path + rgb_model_sha256: str + rgb_evidence_threshold: float + rgb_active_threshold: float + hsv_model_path: Path + hsv_model_sha256: str + hsv_sv_evidence_threshold: float + hsv_active_threshold: float + + +@dataclass(frozen=True) +class EnsembleVerdict: + """Positive-only verdict with the evidence needed to audit it.""" + + path: str + verdict: str + reason: str + rgb_evidence: float | None + rgb_active_support: float | None + hsv_sv_evidence: float | None + hsv_active_support: float | None + + +def load_config(path: Path) -> EnsembleConfig: + """Load a frozen epoch manifest and verify both model artifacts.""" + payload = json.loads(path.read_text(encoding="utf-8")) + if payload.get("verdict_scope") != "positive-only exact-geometry research detector": + raise ValueError("config is not a positive-only exact-geometry detector") + rgb = payload["rgb_model"] + hsv = payload["hsv_model"] + geometry = payload["geometry"] + config = EnsembleConfig( + width=int(geometry["width"]), + height=int(geometry["height"]), + rgb_model_path=Path(rgb["path"]), + rgb_model_sha256=str(rgb["sha256"]), + rgb_evidence_threshold=float(rgb["evidence_threshold"]), + rgb_active_threshold=float(rgb["active_support_threshold"]), + hsv_model_path=Path(hsv["path"]), + hsv_model_sha256=str(hsv["sha256"]), + hsv_sv_evidence_threshold=float(hsv["sv_evidence_threshold"]), + hsv_active_threshold=float(hsv["active_support_threshold"]), + ) + if config.width < 64 or config.height < 64: + raise ValueError("invalid detector geometry") + for model_path, expected_hash in ( + (config.rgb_model_path, config.rgb_model_sha256), + (config.hsv_model_path, config.hsv_model_sha256), + ): + if not model_path.is_file(): + raise ValueError(f"model artifact does not exist: {model_path}") + if artifact_sha256(model_path) != expected_hash: + raise ValueError(f"model artifact hash mismatch: {model_path}") + return config + + +def load_models(config: EnsembleConfig) -> tuple[ColorPhaseModel, ColorPhaseModel]: + """Load and cross-check the RGB and HSV models in CONFIG.""" + rgb_model = load_model(config.rgb_model_path) + hsv_model = load_model(config.hsv_model_path) + if rgb_model.color_space != "rgb" or hsv_model.color_space != "hsv": + raise ValueError("detector requires one RGB model and one HSV model") + expected_geometry = (config.height, config.width) + if (rgb_model.height, rgb_model.width) != expected_geometry: + raise ValueError("RGB model geometry does not match config") + if (hsv_model.height, hsv_model.width) != expected_geometry: + raise ValueError("HSV model geometry does not match config") + return rgb_model, hsv_model + + +def classify_scores( + path: Path, + rgb_score: ColorPhaseScore, + hsv_score: ColorPhaseScore, + config: EnsembleConfig, +) -> EnsembleVerdict: + """Apply CONFIG's positive-only rule to precomputed branch scores.""" + hsv_sv_evidence = float(sum(hsv_score.channel_evidence[1:])) + rgb_support = rgb_score.active_weight_fraction >= config.rgb_active_threshold + hsv_support = hsv_score.active_weight_fraction >= config.hsv_active_threshold + rgb_pass = rgb_score.evidence_score >= config.rgb_evidence_threshold + hsv_pass = hsv_sv_evidence >= config.hsv_sv_evidence_threshold + if rgb_support and hsv_support and rgb_pass and hsv_pass: + verdict = "positive" + reason = "ensemble_pass" + elif not rgb_support or not hsv_support: + verdict = "abstain" + reason = "insufficient_support" + elif rgb_pass != hsv_pass: + verdict = "abstain" + reason = "branch_disagreement" + else: + verdict = "abstain" + reason = "below_positive_threshold" + return EnsembleVerdict( + path=str(path), + verdict=verdict, + reason=reason, + rgb_evidence=rgb_score.evidence_score, + rgb_active_support=rgb_score.active_weight_fraction, + hsv_sv_evidence=hsv_sv_evidence, + hsv_active_support=hsv_score.active_weight_fraction, + ) + + +def detect_image( + path: Path, + config: EnsembleConfig, + rgb_model: ColorPhaseModel, + hsv_model: ColorPhaseModel, +) -> EnsembleVerdict: + """Evaluate PATH or abstain when its geometry is unsupported.""" + with Image.open(path) as image: + if image.size != (config.width, config.height): + return EnsembleVerdict( + path=str(path), + verdict="abstain", + reason="unsupported_geometry", + rgb_evidence=None, + rgb_active_support=None, + hsv_sv_evidence=None, + hsv_active_support=None, + ) + return classify_scores( + path, + score_image(path, rgb_model), + score_image(path, hsv_model), + config, + ) + + +@click.command() +@click.argument("config_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def main(config_path: Path, images: tuple[Path, ...], report_out: Path) -> None: + """Score IMAGES with the frozen positive-only detector CONFIG_PATH.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + config = load_config(config_path) + rgb_model, hsv_model = load_models(config) + verdicts = [detect_image(image, config, rgb_model, hsv_model) for image in images] + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text( + json.dumps( + { + "config": str(config_path), + "positive_count": sum(verdict.verdict == "positive" for verdict in verdicts), + "abstain_count": sum(verdict.verdict == "abstain" for verdict in verdicts), + "verdicts": [asdict(verdict) for verdict in verdicts], + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d positive-only detector verdicts: %s", len(verdicts), report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_fragment_attack.py b/scripts/synthid_fragment_attack.py new file mode 100644 index 0000000..dca9a39 --- /dev/null +++ b/scripts/synthid_fragment_attack.py @@ -0,0 +1,243 @@ +"""Build non-generative spatial-fragmentation SynthID attack candidates. + +The candidates combine deterministic smooth local warps with mild global +resampling, color changes, and codec round-trips. These are pixel transforms, +not semantic reconstruction or generative inpainting. Provider-oracle results +must be evaluated in a frozen batch with a source-positive control. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict +from pathlib import Path + +import click +import cv2 +import numpy as np +from PIL import Image +from synthid_pixel_attack import ( + jpeg_round_trip, + load_rgb, + measure, + norm_matched_noise, + resize_squeeze, + smooth_warp, +) +from synthid_v3_codebook_probe import load_v3_model, score_image + +log = logging.getLogger(__name__) + + +def bounded_smooth_warp( + pixels: np.ndarray, + *, + max_displacement: float, + sigma: float, + seed: int, +) -> np.ndarray: + """Apply a smooth warp whose per-axis displacement is absolutely bounded.""" + if max_displacement < 0.0 or sigma <= 0.0: + raise ValueError("max_displacement must be nonnegative and sigma positive") + height, width = pixels.shape[:2] + rng = np.random.default_rng(seed) + fields: list[np.ndarray] = [] + for _ in range(2): + noise = rng.normal(size=(height, width)).astype(np.float32) + field = cv2.GaussianBlur(noise, (0, 0), sigmaX=sigma, sigmaY=sigma) + maximum = float(np.max(np.abs(field))) + fields.append(np.zeros_like(field) if maximum == 0.0 else field * (max_displacement / maximum)) + yy, xx = np.mgrid[:height, :width].astype(np.float32) + return cv2.remap( + pixels, + xx + fields[0], + yy + fields[1], + interpolation=cv2.INTER_LANCZOS4, + borderMode=cv2.BORDER_REFLECT_101, + ) + + +def affine_combo(pixels: np.ndarray, *, rotation_degrees: float, zoom: float) -> np.ndarray: + """Apply one centered rotation-and-zoom resampling operation.""" + if zoom < 0.0: + raise ValueError("zoom must be nonnegative") + height, width = pixels.shape[:2] + matrix = cv2.getRotationMatrix2D( + center=((width - 1) / 2.0, (height - 1) / 2.0), + angle=rotation_degrees, + scale=1.0 + zoom, + ) + return cv2.warpAffine( + pixels, + matrix, + (width, height), + flags=cv2.INTER_LANCZOS4, + borderMode=cv2.BORDER_REFLECT_101, + ) + + +def color_nudge( + pixels: np.ndarray, + *, + brightness: float, + contrast: float, + saturation: float, + hue_degrees: float, +) -> np.ndarray: + """Apply bounded global RGB contrast and HSV saturation/hue changes.""" + rgb = pixels.astype(np.float32) / 255.0 + rgb = np.clip((rgb - 0.5) * (1.0 + contrast) + 0.5 + brightness, 0.0, 1.0) + hsv = cv2.cvtColor(rgb, cv2.COLOR_RGB2HSV) + hsv[:, :, 0] = np.mod(hsv[:, :, 0] + hue_degrees, 360.0) + hsv[:, :, 1] = np.clip(hsv[:, :, 1] * (1.0 + saturation), 0.0, 1.0) + result = cv2.cvtColor(hsv, cv2.COLOR_HSV2RGB) + return np.clip(np.rint(result * 255.0), 0, 255).astype(np.uint8) + + +def jpeg_chain(pixels: np.ndarray, qualities: tuple[int, ...]) -> np.ndarray: + """Apply sequential JPEG round-trips at QUALITIES.""" + result = pixels + for quality in qualities: + result = jpeg_round_trip(result, quality) + return result + + +def build_candidates(source: np.ndarray) -> dict[str, np.ndarray]: + """Build the frozen spatial-fragmentation ladder for SOURCE.""" + candidates: dict[str, np.ndarray] = { + "control": source.copy(), + "elastic-075": smooth_warp(source, amplitude=0.75, sigma=56.0, seed=20260812), + "elastic-125": smooth_warp(source, amplitude=1.25, sigma=52.0, seed=20260813), + "bounded-100": bounded_smooth_warp( + source, + max_displacement=1.0, + sigma=56.0, + seed=20260817, + ), + "bounded-180": bounded_smooth_warp( + source, + max_displacement=1.8, + sigma=56.0, + seed=20260818, + ), + "bounded-280": bounded_smooth_warp( + source, + max_displacement=2.8, + sigma=44.0, + seed=20260819, + ), + } + + balanced = smooth_warp(source, amplitude=0.75, sigma=56.0, seed=20260814) + balanced = affine_combo(balanced, rotation_degrees=0.2, zoom=0.004) + balanced = resize_squeeze(balanced, 0.94) + balanced = color_nudge( + balanced, + brightness=0.004, + contrast=0.006, + saturation=-0.005, + hue_degrees=0.15, + ) + balanced = jpeg_chain(balanced, (94, 90)) + candidates["fragment-balanced"] = balanced + + strong = smooth_warp(source, amplitude=1.5, sigma=48.0, seed=20260815) + strong = affine_combo(strong, rotation_degrees=0.4, zoom=0.01) + strong = resize_squeeze(strong, 0.88) + strong = color_nudge( + strong, + brightness=0.008, + contrast=0.012, + saturation=-0.01, + hue_degrees=0.3, + ) + strong = jpeg_chain(strong, (92, 88)) + candidates["fragment-strong"] = strong + candidates["sham-strong-rms"] = norm_matched_noise(source, strong, seed=20260816) + + bounded_balanced = bounded_smooth_warp( + source, + max_displacement=1.8, + sigma=56.0, + seed=20260820, + ) + bounded_balanced = resize_squeeze(bounded_balanced, 0.98) + bounded_balanced = color_nudge( + bounded_balanced, + brightness=0.002, + contrast=0.003, + saturation=-0.003, + hue_degrees=0.1, + ) + bounded_balanced = jpeg_chain(bounded_balanced, (96,)) + candidates["bounded-fragment-balanced"] = bounded_balanced + + bounded_strong = bounded_smooth_warp( + source, + max_displacement=2.8, + sigma=44.0, + seed=20260821, + ) + bounded_strong = affine_combo(bounded_strong, rotation_degrees=0.2, zoom=0.004) + bounded_strong = resize_squeeze(bounded_strong, 0.94) + bounded_strong = color_nudge( + bounded_strong, + brightness=0.004, + contrast=0.006, + saturation=-0.005, + hue_degrees=0.15, + ) + bounded_strong = jpeg_chain(bounded_strong, (94, 90)) + candidates["bounded-fragment-strong"] = bounded_strong + candidates["sham-bounded-strong-rms"] = norm_matched_noise(source, bounded_strong, seed=20260822) + return candidates + + +@click.command() +@click.argument("codebook", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("output_dir", type=click.Path(file_okay=False, path_type=Path)) +@click.option("--height", type=click.IntRange(min=64), required=True) +@click.option("--width", type=click.IntRange(min=64), required=True) +@click.option("--peak-count", type=click.IntRange(min=1), default=256, show_default=True) +def main(codebook: Path, source: Path, output_dir: Path, height: int, width: int, peak_count: int) -> None: + """Write a frozen spatial-fragmentation batch for SOURCE.""" + reference = load_rgb(source) + if reference.shape != (height, width, 3): + raise click.BadParameter("source geometry does not match --height and --width") + model = load_v3_model(codebook, height=height, width=width, peak_count=peak_count) + output_dir.mkdir(parents=True, exist_ok=True) + + rows: list[dict[str, object]] = [] + for name, pixels in build_candidates(reference).items(): + path = output_dir / f"{name}.png" + Image.fromarray(pixels, mode="RGB").save(path) + rows.append( + { + **asdict(measure(reference, pixels, name=name, path=path)), + **asdict(score_image(path, model)), + } + ) + report_path = output_dir / "report.json" + report_path.write_text( + json.dumps( + { + "source": str(source), + "codebook": str(codebook), + "height": height, + "width": width, + "peak_count": peak_count, + "variants": rows, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d frozen fragmentation candidates: %s", len(rows), report_path) + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(message)s") + main() diff --git a/scripts/synthid_hybrid_attack.py b/scripts/synthid_hybrid_attack.py new file mode 100644 index 0000000..1bd649a --- /dev/null +++ b/scripts/synthid_hybrid_attack.py @@ -0,0 +1,140 @@ +"""Build non-generative hybrid phase-projection and fragmentation candidates. + +The matrix combines two independently measured mechanisms: sparse RGB/HSV +phase projection and spatially varying subpixel displacement. It is intended +for frozen provider-oracle batches with a visible-mark-removed source control. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict +from pathlib import Path +from typing import TYPE_CHECKING + +import click +from PIL import Image +from synthid_ensemble_attack import alternating_projection +from synthid_ensemble_detector import detect_image, load_config, load_models +from synthid_fragment_attack import bounded_smooth_warp, color_nudge, jpeg_chain +from synthid_pixel_attack import load_rgb, measure, norm_matched_noise, resize_squeeze, smooth_warp + +if TYPE_CHECKING: + import numpy as np + from synthid_color_space_probe import ColorPhaseModel + +log = logging.getLogger(__name__) + + +def build_candidates( + source: np.ndarray, + rgb_model: ColorPhaseModel, + hsv_model: ColorPhaseModel, +) -> dict[str, np.ndarray]: + """Return a frozen mechanism matrix derived from SOURCE.""" + projected_075 = alternating_projection( + source, + rgb_model, + hsv_model, + strength=0.75, + iterations=1, + ) + projected_100 = alternating_projection( + source, + rgb_model, + hsv_model, + strength=1.0, + iterations=1, + ) + bounded_100 = bounded_smooth_warp( + source, + max_displacement=1.0, + sigma=56.0, + seed=20260824, + ) + projected_bounded_100 = bounded_smooth_warp( + projected_075, + max_displacement=1.0, + sigma=56.0, + seed=20260824, + ) + + bounded_polish = resize_squeeze(projected_bounded_100, 0.98) + bounded_polish = color_nudge( + bounded_polish, + brightness=0.002, + contrast=0.003, + saturation=-0.003, + hue_degrees=0.1, + ) + bounded_polish = jpeg_chain(bounded_polish, (96,)) + + elastic_combo = smooth_warp(projected_100, amplitude=0.75, sigma=56.0, seed=20260825) + elastic_combo = resize_squeeze(elastic_combo, 0.98) + elastic_combo = jpeg_chain(elastic_combo, (96,)) + + return { + "projection-075": projected_075, + "bounded-100": bounded_100, + "projection-075-bounded-100": projected_bounded_100, + "projection-075-bounded-polish": bounded_polish, + "projection-100-elastic-075": elastic_combo, + } + + +@click.command() +@click.argument("config_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("output_dir", type=click.Path(file_okay=False, path_type=Path)) +def main(config_path: Path, source: Path, output_dir: Path) -> None: + """Write a frozen non-generative hybrid attack matrix for SOURCE.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + config = load_config(config_path) + rgb_model, hsv_model = load_models(config) + reference = load_rgb(source) + if reference.shape != (config.height, config.width, 3): + raise click.BadParameter("source geometry does not match detector config") + output_dir.mkdir(parents=True, exist_ok=True) + + variants: list[dict[str, object]] = [] + candidates = build_candidates(reference, rgb_model, hsv_model) + for name, pixels in candidates.items(): + path = output_dir / f"{name}.png" + Image.fromarray(pixels, mode="RGB").save(path) + variants.append( + { + **asdict(measure(reference, pixels, name=name, path=path)), + **asdict(detect_image(path, config, rgb_model, hsv_model)), + } + ) + + selected = candidates["projection-075-bounded-100"] + sham = norm_matched_noise(reference, selected, seed=20260826) + sham_path = output_dir / "sham-projection-bounded-rms.png" + Image.fromarray(sham, mode="RGB").save(sham_path) + variants.append( + { + **asdict(measure(reference, sham, name="sham-projection-bounded-rms", path=sham_path)), + **asdict(detect_image(sham_path, config, rgb_model, hsv_model)), + } + ) + + report_path = output_dir / "report.json" + report_path.write_text( + json.dumps( + { + "source": str(source), + "config": str(config_path), + "variants": variants, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d frozen hybrid candidates: %s", len(variants), report_path) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_oracle_batch.py b/scripts/synthid_oracle_batch.py new file mode 100644 index 0000000..34c5d78 --- /dev/null +++ b/scripts/synthid_oracle_batch.py @@ -0,0 +1,467 @@ +"""Build and verify an immutable confirmatory batch for a SynthID oracle. + +The batch contains a referenced untouched source plus four lossless PNG views: +an exact-pixel re-encode, aligned periodic-tile subtraction, a cyclically +shifted tile, and an orthogonal random tile. Building the batch performs no +network requests. Oracle results remain empty until a separately authorized +submission records them. +""" + +from __future__ import annotations + +import json +import logging +import math +from datetime import datetime +from pathlib import Path +from typing import TYPE_CHECKING + +import click +from PIL import Image +from synthid_periodic_tile_ablation import ( + candidate_quality, + control_templates, + phase_score_pixels, + tile_score_pixels, +) +from synthid_periodic_tile_probe import PeriodicTileModel +from synthid_periodic_tile_probe import load_model as load_tile_model +from synthid_phase_carrier import PhaseCarrierModel +from synthid_phase_carrier import load_model as load_phase_model +from synthid_pixel_attack import load_rgb +from synthid_research_manifest import artifact_sha256, pixel_fingerprint +from synthid_tile_attack import subtract_tiled_template + +if TYPE_CHECKING: + import numpy as np + +log = logging.getLogger(__name__) + +ROLE_ORDER = ("source", "reencode_control", "aligned", "shifted", "orthogonal_random") +DERIVATIVE_ROLES = ROLE_ORDER[1:] +FORMAT_VERSION = 1 +SYNTHID_RESULTS = {"detected", "not_detected", "indeterminate", "refused"} +C2PA_RESULTS = {"present", "absent", "indeterminate", "unavailable"} + + +def _inside(path: Path, parent: Path) -> bool: + """Return whether resolved PATH is inside resolved PARENT.""" + try: + path.resolve().relative_to(parent.resolve()) + except ValueError: + return False + return True + + +def _json_float(value: float) -> float | None: + """Return finite VALUE or None for JSON interoperability.""" + return value if math.isfinite(value) else None + + +def _write_png(path: Path, pixels: np.ndarray) -> None: + """Write exact RGB PIXELS as a deterministic lossless PNG.""" + path.parent.mkdir(parents=True, exist_ok=True) + if path.exists(): + raise ValueError(f"refusing to overwrite oracle artifact: {path}") + Image.fromarray(pixels, mode="RGB").save(path, format="PNG", compress_level=9) + + +def _score_row( + pixels: np.ndarray, + *, + source: np.ndarray, + tile_model: PeriodicTileModel, + phase_model: PhaseCarrierModel, + tile_threshold: float, + phase_threshold: float, + active_threshold: float, +) -> dict[str, object]: + """Return local frozen scores and paired fidelity for PIXELS.""" + tile_score = tile_score_pixels(pixels, tile_model) + phase_score, active_support = phase_score_pixels(pixels, phase_model) + quality = candidate_quality(source, pixels) + return { + "tile_score": tile_score, + "tile_accepted": tile_score >= tile_threshold, + "phase_score": phase_score, + "active_support": active_support, + "phase_accepted": phase_score >= phase_threshold and active_support >= active_threshold, + "residual_rms": quality["residual_rms"], + "psnr_db": _json_float(quality["psnr_db"]), + "ssim": quality["ssim"], + "changed_pixel_fraction": quality["changed_pixel_fraction"], + } + + +def _artifact_row( + path: Path, + *, + role: str, + source_id: str, + in_batch: bool, + batch_root: Path, + scores: dict[str, object], +) -> dict[str, object]: + """Return one hash-frozen manifest row for PATH.""" + pixel_sha256, width, height = pixel_fingerprint(path) + return { + "source_id": source_id, + "role": role, + "path": str(path.relative_to(batch_root) if in_batch else path.resolve()), + "in_batch": in_batch, + "artifact_sha256": artifact_sha256(path), + "pixel_sha256": pixel_sha256, + "width": width, + "height": height, + "synthid_result": None, + "c2pa_result": None, + "submitted_at": None, + **scores, + } + + +def build_batch( + sources: list[Path], + *, + output_dir: Path, + tile_model_path: Path, + phase_model_path: Path, + tile_threshold: float, + phase_threshold: float, + active_threshold: float, + strength: float, + seed: int, + provider: str, + repository_root: Path, +) -> Path: + """Build a preregistered, unsubmitted oracle batch and return its manifest.""" + if not sources: + raise ValueError("at least one source is required") + if strength <= 0.0 or not math.isfinite(strength): + raise ValueError("strength must be finite and positive") + if provider not in {"google", "openai"}: + raise ValueError("provider must be google or openai") + if _inside(output_dir, repository_root): + raise ValueError("oracle batches must be written outside the repository") + if output_dir.exists() and any(output_dir.iterdir()): + raise ValueError("output directory must not already contain files") + output_dir.mkdir(parents=True, exist_ok=True) + + tile_model = load_tile_model(tile_model_path) + phase_model = load_phase_model(phase_model_path) + if (tile_model.height, tile_model.width) != (phase_model.height, phase_model.width): + raise ValueError("tile and phase model geometries differ") + templates = control_templates(tile_model.template, seed=seed) + + rows: list[dict[str, object]] = [] + seen_source_hashes: set[str] = set() + for source_path in sources: + source_hash = artifact_sha256(source_path) + if source_hash in seen_source_hashes: + raise ValueError("duplicate source artifact") + seen_source_hashes.add(source_hash) + source = load_rgb(source_path) + if source.shape != (tile_model.height, tile_model.width, 3): + raise ValueError(f"{source_path}: geometry does not match the frozen models") + source_id = source_hash[:16] + score_args = { + "source": source, + "tile_model": tile_model, + "phase_model": phase_model, + "tile_threshold": tile_threshold, + "phase_threshold": phase_threshold, + "active_threshold": active_threshold, + } + source_scores = _score_row(source, **score_args) + rows.append( + _artifact_row( + source_path, + role="source", + source_id=source_id, + in_batch=False, + batch_root=output_dir, + scores=source_scores, + ) + ) + for role in DERIVATIVE_ROLES: + variant = ( + source + if role == "reencode_control" + else subtract_tiled_template( + source, + templates[role] * tile_model.expected_norm, + strength=strength, + ) + ) + output_path = output_dir / source_id / f"{role}.png" + _write_png(output_path, variant) + rows.append( + _artifact_row( + output_path, + role=role, + source_id=source_id, + in_batch=True, + batch_root=output_dir, + scores=(source_scores if role == "reencode_control" else _score_row(variant, **score_args)), + ) + ) + + manifest = { + "format_version": FORMAT_VERSION, + "status": "preregistered_unsubmitted", + "source_count": len(sources), + "request_count": len(rows), + "request_order": ROLE_ORDER, + "provider": provider, + "decision_rule": ( + "Count causal success only when source, reencode_control, shifted, and " + "orthogonal_random are detected in the matching provider SynthID oracle, " + "aligned is not_detected, and indeterminate is never treated as negative." + ), + "strength": strength, + "seed": seed, + "tile_threshold": tile_threshold, + "phase_threshold": phase_threshold, + "active_threshold": active_threshold, + "tile_model": str(tile_model_path.resolve()), + "tile_model_sha256": artifact_sha256(tile_model_path), + "phase_model": str(phase_model_path.resolve()), + "phase_model_sha256": artifact_sha256(phase_model_path), + "rows": rows, + } + manifest_path = output_dir / "manifest.json" + manifest_path.write_text(json.dumps(manifest, indent=2, allow_nan=False) + "\n", encoding="utf-8") + manifest_hash = artifact_sha256(manifest_path) + (output_dir / "manifest.sha256").write_text(manifest_hash + " manifest.json\n", encoding="utf-8") + results_template = { + "format_version": FORMAT_VERSION, + "manifest_sha256": manifest_hash, + "provider": provider, + "rows": [ + { + "source_id": row["source_id"], + "role": row["role"], + "artifact_sha256": row["artifact_sha256"], + "synthid_result": None, + "c2pa_result": None, + "raw_response": None, + "submitted_at": None, + } + for row in rows + ], + } + (output_dir / "results-template.json").write_text( + json.dumps(results_template, indent=2) + "\n", + encoding="utf-8", + ) + verify_batch(manifest_path, repository_root=repository_root) + return manifest_path + + +def verify_batch(manifest_path: Path, *, repository_root: Path) -> dict[str, object]: + """Verify manifest, model, source, and derivative hashes without mutation.""" + batch_root = manifest_path.parent + if _inside(batch_root, repository_root): + raise ValueError("oracle batches must remain outside the repository") + digest_path = batch_root / "manifest.sha256" + expected_manifest_hash = digest_path.read_text(encoding="utf-8").split()[0] + if artifact_sha256(manifest_path) != expected_manifest_hash: + raise ValueError("manifest hash mismatch") + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + if manifest.get("format_version") != FORMAT_VERSION: + raise ValueError("unsupported oracle-batch manifest version") + rows = manifest.get("rows") + if not isinstance(rows, list) or len(rows) != manifest.get("request_count"): + raise ValueError("manifest request count mismatch") + if artifact_sha256(Path(manifest["tile_model"])) != manifest["tile_model_sha256"]: + raise ValueError("tile model hash mismatch") + if artifact_sha256(Path(manifest["phase_model"])) != manifest["phase_model_sha256"]: + raise ValueError("phase model hash mismatch") + + groups: dict[str, list[str]] = {} + artifact_hashes: set[str] = set() + for row in rows: + source_id = str(row["source_id"]) + groups.setdefault(source_id, []).append(str(row["role"])) + path = batch_root / str(row["path"]) if row["in_batch"] else Path(str(row["path"])) + if artifact_sha256(path) != row["artifact_sha256"]: + raise ValueError(f"artifact hash mismatch for {source_id}/{row['role']}") + pixel_sha256, width, height = pixel_fingerprint(path) + if (pixel_sha256, width, height) != (row["pixel_sha256"], row["width"], row["height"]): + raise ValueError(f"pixel fingerprint mismatch for {source_id}/{row['role']}") + artifact_hashes.add(str(row["artifact_sha256"])) + if any(tuple(roles) != ROLE_ORDER for roles in groups.values()): + raise ValueError("each source must have the fixed request-role order") + if len(groups) != manifest.get("source_count"): + raise ValueError("manifest source count mismatch") + if len(artifact_hashes) < len(rows) - len(groups): + raise ValueError("unexpected duplicate derivative artifacts") + template = json.loads((batch_root / "results-template.json").read_text(encoding="utf-8")) + expected_identities = [(row["source_id"], row["role"], row["artifact_sha256"]) for row in rows] + template_identities = [(row["source_id"], row["role"], row["artifact_sha256"]) for row in template.get("rows", [])] + if template.get("manifest_sha256") != expected_manifest_hash or template.get("provider") != manifest["provider"]: + raise ValueError("results template does not identify the preregistered batch") + if template_identities != expected_identities: + raise ValueError("results template row identities differ from the manifest") + return manifest + + +def _parse_submitted_at(value: object) -> None: + """Reject timestamps that are absent or not timezone-aware ISO-8601.""" + if not isinstance(value, str): + raise ValueError("submitted_at must be a timezone-aware ISO-8601 timestamp") + try: + parsed = datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError as error: + raise ValueError("submitted_at must be a timezone-aware ISO-8601 timestamp") from error + if parsed.tzinfo is None: + raise ValueError("submitted_at must be a timezone-aware ISO-8601 timestamp") + + +def evaluate_results( + manifest_path: Path, + results_path: Path, + *, + repository_root: Path, +) -> dict[str, object]: + """Validate a complete result file and apply the preregistered decision rule.""" + manifest = verify_batch(manifest_path, repository_root=repository_root) + results = json.loads(results_path.read_text(encoding="utf-8")) + manifest_hash = artifact_sha256(manifest_path) + if results.get("format_version") != FORMAT_VERSION: + raise ValueError("unsupported oracle-results version") + if results.get("manifest_sha256") != manifest_hash or results.get("provider") != manifest["provider"]: + raise ValueError("results do not identify the preregistered batch") + manifest_rows = manifest["rows"] + result_rows = results.get("rows") + if not isinstance(result_rows, list) or len(result_rows) != len(manifest_rows): + raise ValueError("oracle results must cover every preregistered request") + + grouped: dict[str, dict[str, str]] = {} + for expected, result in zip(manifest_rows, result_rows, strict=True): + identity = (result.get("source_id"), result.get("role"), result.get("artifact_sha256")) + expected_identity = (expected["source_id"], expected["role"], expected["artifact_sha256"]) + if identity != expected_identity: + raise ValueError("oracle result order or artifact identity differs from the manifest") + synthid_result = result.get("synthid_result") + c2pa_result = result.get("c2pa_result") + if synthid_result not in SYNTHID_RESULTS: + raise ValueError("invalid or missing SynthID result") + if c2pa_result not in C2PA_RESULTS: + raise ValueError("invalid or missing C2PA result") + if not isinstance(result.get("raw_response"), str) or not result["raw_response"].strip(): + raise ValueError("raw_response must preserve the nonempty verbatim oracle result") + _parse_submitted_at(result.get("submitted_at")) + grouped.setdefault(str(expected["source_id"]), {})[str(expected["role"])] = str(synthid_result) + + source_results: list[dict[str, str]] = [] + for source_id, roles in grouped.items(): + if tuple(roles) != ROLE_ORDER: + raise ValueError("oracle results do not preserve the fixed role order") + values = set(roles.values()) + if values & {"indeterminate", "refused"}: + verdict = "indeterminate" + elif ( + roles["source"] == "detected" + and roles["reencode_control"] == "detected" + and roles["shifted"] == "detected" + and roles["orthogonal_random"] == "detected" + and roles["aligned"] == "not_detected" + ): + verdict = "causal_success" + elif any(roles[role] != "detected" for role in ("source", "reencode_control", "shifted", "orthogonal_random")): + verdict = "control_failed" + else: + verdict = "aligned_still_detected" + source_results.append({"source_id": source_id, "verdict": verdict}) + counts = { + verdict: sum(row["verdict"] == verdict for row in source_results) + for verdict in ("causal_success", "aligned_still_detected", "control_failed", "indeterminate") + } + return { + "format_version": FORMAT_VERSION, + "manifest_sha256": manifest_hash, + "provider": manifest["provider"], + "source_count": manifest["source_count"], + "counts": counts, + "sources": source_results, + } + + +@click.group() +def main() -> None: + """Build and verify an immutable confirmatory oracle batch.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + + +@main.command("build") +@click.argument("tile_model_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("phase_model_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("sources", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--output-dir", type=click.Path(file_okay=False, path_type=Path), required=True) +@click.option("--tile-threshold", type=float, required=True) +@click.option("--phase-threshold", type=float, required=True) +@click.option("--active-threshold", type=float, required=True) +@click.option("--strength", type=click.FloatRange(min=0.0, min_open=True), default=2.0, show_default=True) +@click.option("--seed", type=int, default=20260810, show_default=True) +@click.option("--provider", type=click.Choice(["google", "openai"]), required=True) +def build_command( + tile_model_path: Path, + phase_model_path: Path, + sources: tuple[Path, ...], + output_dir: Path, + tile_threshold: float, + phase_threshold: float, + active_threshold: float, + strength: float, + seed: int, + provider: str, +) -> None: + """Build a frozen batch from exact-geometry SOURCES.""" + manifest = build_batch( + list(sources), + output_dir=output_dir, + tile_model_path=tile_model_path, + phase_model_path=phase_model_path, + tile_threshold=tile_threshold, + phase_threshold=phase_threshold, + active_threshold=active_threshold, + strength=strength, + seed=seed, + provider=provider, + repository_root=Path(__file__).resolve().parent.parent, + ) + log.info("Wrote preregistered oracle batch: %s", manifest) + + +@main.command("verify") +@click.argument("manifest_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +def verify_command(manifest_path: Path) -> None: + """Verify an existing batch without changing it.""" + manifest = verify_batch(manifest_path, repository_root=Path(__file__).resolve().parent.parent) + log.info("Verified %d immutable oracle requests", manifest["request_count"]) + + +@main.command("evaluate") +@click.argument("manifest_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("results_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def evaluate_command(manifest_path: Path, results_path: Path, report_out: Path) -> None: + """Validate RESULTS_PATH and write the preregistered batch verdict.""" + repository_root = Path(__file__).resolve().parent.parent + if _inside(report_out, repository_root): + raise click.BadParameter("oracle result reports must be written outside the repository") + report = evaluate_results( + manifest_path, + results_path, + repository_root=repository_root, + ) + report_out.parent.mkdir(parents=True, exist_ok=True) + if report_out.exists(): + raise click.BadParameter("refusing to overwrite an oracle result report") + report_out.write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8") + log.info("Wrote oracle batch verdict: %s", report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_periodic_tile.py b/scripts/synthid_periodic_tile.py new file mode 100644 index 0000000..49b994b --- /dev/null +++ b/scripts/synthid_periodic_tile.py @@ -0,0 +1,23 @@ +"""Compatibility imports for shared periodic-residual helpers.""" + +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from synthid_runtime.synthid_detector import fold_residual_template, unit_tile + +__all__ = ["cyclic_tile_correlations", "fold_residual_template", "unit_tile"] + + +def cyclic_tile_correlations(template: np.ndarray, tile: np.ndarray) -> np.ndarray: + """Return correlations for every cyclic spatial shift of TILE.""" + if template.shape != tile.shape or template.ndim != 3: + raise ValueError("template and tile must have identical three-dimensional shapes") + template_spectrum = np.fft.fft2(template, axes=(0, 1)) + tile_spectrum = np.fft.fft2(tile, axes=(0, 1)) + return np.fft.ifft2(np.sum(template_spectrum * np.conj(tile_spectrum), axis=2)).real diff --git a/scripts/synthid_periodic_tile_ablation.py b/scripts/synthid_periodic_tile_ablation.py new file mode 100644 index 0000000..15a47b6 --- /dev/null +++ b/scripts/synthid_periodic_tile_ablation.py @@ -0,0 +1,288 @@ +"""Measure whether a frozen periodic tile causally controls local carrier scores. + +This harness compares subtraction of the learned tile with cyclically shifted +and orthogonal random tiles of the same norm. It measures local research +detectors only. A score reversal is not evidence that a provider oracle would +stop detecting SynthID. +""" + +from __future__ import annotations + +import json +import logging +import math +from collections import defaultdict +from pathlib import Path + +import click +import numpy as np +from synthid_periodic_tile import unit_tile +from synthid_periodic_tile_probe import PeriodicTileModel +from synthid_periodic_tile_probe import load_model as load_tile_model +from synthid_periodic_tile_probe import score_pixels as score_tile_pixels +from synthid_phase_carrier import PhaseCarrierModel, score_pixels +from synthid_phase_carrier import load_model as load_phase_model +from synthid_pixel_attack import load_rgb, measure +from synthid_research_manifest import artifact_sha256 +from synthid_tile_attack import parse_positive_floats, subtract_tiled_template + +log = logging.getLogger(__name__) + + +def exact_sign_test(negative: int, positive: int) -> float: + """Return an exact two-sided sign-test p-value after excluding ties.""" + count = negative + positive + if count == 0: + return 1.0 + tail = sum(math.comb(count, index) for index in range(min(negative, positive) + 1)) / 2**count + return min(1.0, 2.0 * tail) + + +def control_templates(template: np.ndarray, *, seed: int) -> dict[str, np.ndarray]: + """Return aligned, shifted, and norm-matched orthogonal control tiles.""" + rng = np.random.default_rng(seed) + random_tile = rng.normal(size=template.shape) + random_tile -= np.mean(random_tile, axis=(0, 1), keepdims=True) + random_tile -= np.sum(random_tile * template) * template + random_tile, norm = unit_tile(random_tile) + if norm == 0.0 or abs(float(np.sum(random_tile * template))) > 1e-12: + raise ValueError("could not construct an orthogonal random control") + return { + "aligned": template, + "shifted": np.roll(template, shift=(1, 1), axis=(0, 1)), + "orthogonal_random": random_tile, + } + + +def phase_score_pixels(pixels: np.ndarray, model: PhaseCarrierModel) -> tuple[float, float]: + """Return the unregistered phase score and active support for PIXELS.""" + result = score_pixels(pixels, model) + return result.score, result.active_weight_fraction + + +def tile_score_pixels(pixels: np.ndarray, model: PeriodicTileModel) -> float: + """Return the fixed-phase periodic-tile score for PIXELS.""" + return score_tile_pixels(pixels, model).score + + +def summarize(values: list[float]) -> dict[str, float]: + """Return bounded descriptive statistics for VALUES.""" + return { + "minimum": float(np.min(values)), + "median": float(np.median(values)), + "maximum": float(np.max(values)), + } + + +def direction_summary(values: list[float]) -> dict[str, float | int]: + """Return direction counts and a two-sided sign test for VALUES.""" + negative = sum(value < 0.0 for value in values) + positive = sum(value > 0.0 for value in values) + return { + "negative": negative, + "positive": positive, + "ties": len(values) - negative - positive, + "two_sided_sign_p": exact_sign_test(negative, positive), + } + + +def candidate_quality(reference: np.ndarray, candidate: np.ndarray) -> dict[str, float]: + """Return paired fidelity metrics for one equal-geometry candidate.""" + measurement = measure(reference, candidate, name="candidate", path=Path("")) + return { + "residual_rms": measurement.residual_rms, + "psnr_db": measurement.psnr_db, + "ssim": measurement.ssim, + "changed_pixel_fraction": measurement.changed_pixel_fraction, + } + + +def run_ablation( + sources: list[Path], + *, + tile_model: PeriodicTileModel, + phase_model: PhaseCarrierModel, + tile_threshold: float, + phase_threshold: float, + active_threshold: float, + strengths: tuple[float, ...], + phase_strength: float, + seed: int, +) -> dict[str, object]: + """Evaluate aligned subtraction and controls on exact-geometry SOURCES.""" + if not sources: + raise ValueError("at least one source is required") + if phase_strength not in strengths: + raise ValueError("phase strength must be one of the swept strengths") + if (tile_model.height, tile_model.width) != (phase_model.height, phase_model.width): + raise ValueError("tile and phase model geometries differ") + if not all(np.isfinite(value) for value in (tile_threshold, phase_threshold, active_threshold)): + raise ValueError("thresholds must be finite") + + templates = control_templates(tile_model.template, seed=seed) + rows: list[dict[str, object]] = [] + for source_path in sources: + source = load_rgb(source_path) + if source.shape != (tile_model.height, tile_model.width, 3): + raise ValueError(f"{source_path}: geometry does not match the models") + source_hash = artifact_sha256(source_path) + original_tile = tile_score_pixels(source, tile_model) + original_phase, original_support = phase_score_pixels(source, phase_model) + for strength in strengths: + for control, template in templates.items(): + candidate = subtract_tiled_template( + source, + template * tile_model.expected_norm, + strength=strength, + ) + tile_score = tile_score_pixels(candidate, tile_model) + row: dict[str, object] = { + "path": str(source_path), + "artifact_sha256": source_hash, + "control": control, + "strength": strength, + "original_tile_score": original_tile, + "tile_score": tile_score, + "tile_delta": tile_score - original_tile, + "tile_accepted": tile_score >= tile_threshold, + "original_phase_score": original_phase, + "original_active_support": original_support, + } + if strength == phase_strength: + phase_score, active_support = phase_score_pixels(candidate, phase_model) + row.update( + { + **candidate_quality(source, candidate), + "phase_score": phase_score, + "active_support": active_support, + "phase_delta": phase_score - original_phase, + "phase_accepted": phase_score >= phase_threshold and active_support >= active_threshold, + } + ) + rows.append(row) + + grouped: dict[tuple[str, float], list[dict[str, object]]] = defaultdict(list) + for row in rows: + grouped[(str(row["control"]), float(row["strength"]))].append(row) + tile_summaries: list[dict[str, object]] = [] + for (control, strength), group in sorted(grouped.items()): + deltas = [float(row["tile_delta"]) for row in group] + tile_summaries.append( + { + "control": control, + "strength": strength, + "accepted": sum(bool(row["tile_accepted"]) for row in group), + "delta": summarize(deltas), + "direction": direction_summary(deltas), + } + ) + + selected = [row for row in rows if float(row["strength"]) == phase_strength] + phase_summaries: list[dict[str, object]] = [] + for control in templates: + group = [row for row in selected if row["control"] == control] + phase_summaries.append( + { + "control": control, + "accepted": sum(bool(row["phase_accepted"]) for row in group), + "delta": summarize([float(row["phase_delta"]) for row in group]), + "active_support": summarize([float(row["active_support"]) for row in group]), + "psnr_db": summarize([float(row["psnr_db"]) for row in group]), + "ssim": summarize([float(row["ssim"]) for row in group]), + "changed_pixel_fraction": summarize([float(row["changed_pixel_fraction"]) for row in group]), + } + ) + + paired_comparisons: list[dict[str, object]] = [] + for control in ("shifted", "orthogonal_random"): + for metric in ("tile_delta", "phase_delta"): + aligned = { + str(row["artifact_sha256"]): float(row[metric]) for row in selected if row["control"] == "aligned" + } + comparison = { + str(row["artifact_sha256"]): float(row[metric]) for row in selected if row["control"] == control + } + differences = [aligned[key] - comparison[key] for key in sorted(aligned)] + paired_comparisons.append( + { + "aligned_minus": control, + "metric": metric, + "difference": summarize(differences), + "direction": direction_summary(differences), + } + ) + + return { + "source_count": len(sources), + "tile_threshold": tile_threshold, + "phase_threshold": phase_threshold, + "active_threshold": active_threshold, + "strengths": strengths, + "phase_strength": phase_strength, + "seed": seed, + "original": { + "tile_accepted": sum( + float(row["original_tile_score"]) >= tile_threshold for row in selected if row["control"] == "aligned" + ), + "phase_accepted": sum( + float(row["original_phase_score"]) >= phase_threshold + and float(row["original_active_support"]) >= active_threshold + for row in selected + if row["control"] == "aligned" + ), + }, + "tile_summaries": tile_summaries, + "phase_summaries": phase_summaries, + "paired_comparisons": paired_comparisons, + "items": rows, + } + + +@click.command() +@click.argument("tile_model_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("phase_model_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("sources", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--tile-threshold", type=float, required=True) +@click.option("--phase-threshold", type=float, required=True) +@click.option("--active-threshold", type=float, required=True) +@click.option("--strengths", default="1,1.5,2,3,4", show_default=True) +@click.option("--phase-strength", type=click.FloatRange(min=0.0, min_open=True), default=2.0, show_default=True) +@click.option("--seed", type=int, default=20260810, show_default=True) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def main( + tile_model_path: Path, + phase_model_path: Path, + sources: tuple[Path, ...], + tile_threshold: float, + phase_threshold: float, + active_threshold: float, + strengths: str, + phase_strength: float, + seed: int, + report_out: Path, +) -> None: + """Run a fixed periodic-tile causal ablation on exact-geometry SOURCES.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + strength_values = parse_positive_floats(strengths, option_name="strengths") + report = run_ablation( + list(sources), + tile_model=load_tile_model(tile_model_path), + phase_model=load_phase_model(phase_model_path), + tile_threshold=tile_threshold, + phase_threshold=phase_threshold, + active_threshold=active_threshold, + strengths=strength_values, + phase_strength=phase_strength, + seed=seed, + ) + report["tile_model"] = str(tile_model_path) + report["tile_model_sha256"] = artifact_sha256(tile_model_path) + report["phase_model"] = str(phase_model_path) + report["phase_model_sha256"] = artifact_sha256(phase_model_path) + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8") + log.info("Wrote periodic-tile causal ablation: %s", report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_periodic_tile_probe.py b/scripts/synthid_periodic_tile_probe.py new file mode 100644 index 0000000..c33d4c9 --- /dev/null +++ b/scripts/synthid_periodic_tile_probe.py @@ -0,0 +1,273 @@ +"""Discover and evaluate an exact-geometry periodic residual carrier. + +The model folds a high-pass residual modulo a fixed tile, averaging thousands +of spatial repetitions before normalized correlation. It is a positive-only +research signal, not a universal or certified SynthID decoder. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict, dataclass +from pathlib import Path + +import click +import numpy as np +from synthid_periodic_tile import cyclic_tile_correlations, fold_residual_template, unit_tile +from synthid_pixel_attack import load_rgb +from synthid_research_manifest import artifact_sha256 + +log = logging.getLogger(__name__) + + +@dataclass(frozen=True) +class PeriodicTileModel: + """One exact-geometry normalized periodic-residual template.""" + + height: int + width: int + tile_height: int + tile_width: int + denoise_sigma: float + template: np.ndarray + expected_norm: float + + +@dataclass(frozen=True) +class PeriodicTileScore: + """Normalized tile correlation and support for one image.""" + + path: str + score: float + active_support: float + row_shift: int + column_shift: int + repeat_count: int + + +def _load_rgb(path: Path, *, height: int, width: int) -> np.ndarray: + """Load PATH as exact-geometry uint8 RGB.""" + pixels = load_rgb(path) + if pixels.shape != (height, width, 3): + raise ValueError(f"{path}: geometry {pixels.shape[1]}x{pixels.shape[0]} does not match {width}x{height}") + return pixels + + +def discover_model( + paths: list[Path], + *, + tile_height: int, + tile_width: int, + denoise_sigma: float = 1.0, +) -> PeriodicTileModel: + """Learn a normalized periodic template from positive PATHS.""" + if len(paths) < 3: + raise ValueError("at least three positive images are required") + first_pixels = load_rgb(paths[0]) + height, width = first_pixels.shape[:2] + unit_sum = np.zeros((tile_height, tile_width, 3), dtype=np.float64) + norms: list[float] = [] + for index, path in enumerate(paths): + folded = fold_residual_template( + first_pixels if index == 0 else _load_rgb(path, height=height, width=width), + tile_height=tile_height, + tile_width=tile_width, + denoise_sigma=denoise_sigma, + ) + unit, norm = unit_tile(folded) + unit_sum += unit + norms.append(norm) + template, template_norm = unit_tile(unit_sum / len(paths)) + if template_norm == 0.0: + raise ValueError("positive images expose no periodic residual template") + return PeriodicTileModel( + height=height, + width=width, + tile_height=tile_height, + tile_width=tile_width, + denoise_sigma=denoise_sigma, + template=template, + expected_norm=float(np.median(norms)), + ) + + +def score_pixels( + pixels: np.ndarray, + model: PeriodicTileModel, + *, + register: bool = False, + path: str = "", +) -> PeriodicTileScore: + """Score exact-geometry RGB PIXELS, optionally searching cyclic tile shifts.""" + if pixels.shape != (model.height, model.width, 3): + raise ValueError("pixel geometry does not match periodic-tile model") + folded = fold_residual_template( + pixels, + tile_height=model.tile_height, + tile_width=model.tile_width, + denoise_sigma=model.denoise_sigma, + ) + unit, norm = unit_tile(folded) + if register: + correlations = cyclic_tile_correlations(model.template, unit) + row_shift, column_shift = np.unravel_index(int(np.argmax(correlations)), correlations.shape) + score = float(correlations[row_shift, column_shift]) + else: + score = float(np.sum(model.template * unit)) + row_shift = column_shift = 0 + return PeriodicTileScore( + path=path, + score=score, + active_support=min(norm / (model.expected_norm + 1e-12), 1.0), + row_shift=row_shift, + column_shift=column_shift, + repeat_count=(model.height // model.tile_height) * (model.width // model.tile_width), + ) + + +def score_image(path: Path, model: PeriodicTileModel, *, register: bool = False) -> PeriodicTileScore: + """Score PATH against MODEL, optionally searching cyclic tile shifts.""" + pixels = _load_rgb(path, height=model.height, width=model.width) + return score_pixels(pixels, model, register=register, path=str(path)) + + +def calibrate_threshold(paths: list[Path], model: PeriodicTileModel, *, register: bool = False) -> float: + """Return the first float above every negative score in PATHS.""" + if not paths: + raise ValueError("at least one calibration negative is required") + maximum = max(score_image(path, model, register=register).score for path in paths) + return float(np.nextafter(maximum, np.inf)) + + +def save_model(path: Path, model: PeriodicTileModel) -> None: + """Save MODEL as a pickle-free numeric artifact without precision loss.""" + path.parent.mkdir(parents=True, exist_ok=True) + np.savez_compressed( + path, + format_version=np.asarray(1, dtype=np.int32), + height=np.asarray(model.height, dtype=np.int32), + width=np.asarray(model.width, dtype=np.int32), + tile_height=np.asarray(model.tile_height, dtype=np.int32), + tile_width=np.asarray(model.tile_width, dtype=np.int32), + denoise_sigma=np.asarray(model.denoise_sigma, dtype=np.float64), + template=model.template.astype(np.float64), + expected_norm=np.asarray(model.expected_norm, dtype=np.float64), + ) + + +def load_model(path: Path) -> PeriodicTileModel: + """Load and validate one numeric periodic-tile model.""" + with np.load(path, allow_pickle=False) as artifact: + if int(artifact["format_version"]) != 1: + raise ValueError("unsupported periodic-tile model format version") + model = PeriodicTileModel( + height=int(artifact["height"]), + width=int(artifact["width"]), + tile_height=int(artifact["tile_height"]), + tile_width=int(artifact["tile_width"]), + denoise_sigma=float(artifact["denoise_sigma"]), + template=np.asarray(artifact["template"], dtype=np.float64), + expected_norm=float(artifact["expected_norm"]), + ) + if model.height < 1 or model.width < 1 or model.tile_height < 1 or model.tile_width < 1: + raise ValueError("invalid periodic-tile geometry") + if model.height % model.tile_height or model.width % model.tile_width: + raise ValueError("image geometry is not divisible by periodic-tile geometry") + if model.template.shape != (model.tile_height, model.tile_width, 3): + raise ValueError("invalid periodic-tile template shape") + if not np.all(np.isfinite(model.template)) or not np.isclose(np.linalg.norm(model.template), 1.0): + raise ValueError("invalid periodic-tile template") + if not np.isfinite(model.denoise_sigma) or model.denoise_sigma <= 0.0: + raise ValueError("invalid periodic-tile denoise sigma") + if not np.isfinite(model.expected_norm) or model.expected_norm <= 0.0: + raise ValueError("invalid periodic-tile expected norm") + return model + + +@click.group() +def main() -> None: + """Discover and evaluate an exact-geometry periodic carrier.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + + +@main.command() +@click.argument("positives", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--tile-height", type=click.IntRange(min=1), required=True) +@click.option("--tile-width", type=click.IntRange(min=1), required=True) +@click.option("--denoise-sigma", type=click.FloatRange(min=0.0, min_open=True), default=1.0, show_default=True) +@click.option("--model-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def discover( + positives: tuple[Path, ...], + tile_height: int, + tile_width: int, + denoise_sigma: float, + model_out: Path, +) -> None: + """Learn a periodic tile from exact-geometry POSITIVES.""" + save_model( + model_out, + discover_model( + list(positives), + tile_height=tile_height, + tile_width=tile_width, + denoise_sigma=denoise_sigma, + ), + ) + log.info("Wrote periodic-tile model: %s", model_out) + + +@main.command() +@click.argument("model_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("negatives", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--register", is_flag=True, help="Search every cyclic shift within the learned tile.") +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def calibrate(model_path: Path, negatives: tuple[Path, ...], register: bool, report_out: Path) -> None: + """Calibrate a zero-observed-error threshold on NEGATIVES.""" + model = load_model(model_path) + threshold = calibrate_threshold(list(negatives), model, register=register) + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text( + json.dumps( + { + "model": str(model_path), + "model_sha256": artifact_sha256(model_path), + "register": register, + "negative_count": len(negatives), + "threshold": threshold, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote periodic-tile calibration report: %s", report_out) + + +@main.command() +@click.argument("model_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--register", is_flag=True, help="Search every cyclic shift within the learned tile.") +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def score(model_path: Path, images: tuple[Path, ...], register: bool, report_out: Path) -> None: + """Score exact-geometry IMAGES with MODEL_PATH.""" + model = load_model(model_path) + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text( + json.dumps( + { + "model": str(model_path), + "model_sha256": artifact_sha256(model_path), + "register": register, + "scores": [asdict(score_image(image, model, register=register)) for image in images], + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote periodic-tile score report: %s", report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_phase_carrier.py b/scripts/synthid_phase_carrier.py new file mode 100644 index 0000000..4cf04c6 --- /dev/null +++ b/scripts/synthid_phase_carrier.py @@ -0,0 +1,384 @@ +"""Discover and evaluate an exact-geometry phase-carrier hypothesis. + +The model is learned only from supplied images and stored as numeric arrays in +a pickle-free NPZ. It is a research baseline, not a proprietary SynthID +decoder. A valid detector claim still requires provider labels, same-provider +hard negatives, group-aware splits, and a locked operating point. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict, dataclass +from pathlib import Path + +import click +import numpy as np +from PIL import Image +from synthid_phase_registration import MAX_TRANSLATION_SHIFT, extract_frequency_values, register_phase_translations + +log = logging.getLogger(__name__) + + +@dataclass(frozen=True) +class PhaseCarrierModel: + """Sparse exact-geometry phase carrier learned from positive images.""" + + height: int + width: int + rows: np.ndarray + columns: np.ndarray + channels: np.ndarray + phases: np.ndarray + weights: np.ndarray + expected_magnitudes: np.ndarray + + +@dataclass(frozen=True) +class PhaseCarrierScore: + """Alignment of one exact-geometry image with a phase-carrier model.""" + + path: str + score: float + active_weight_fraction: float + peak_count: int + + +@dataclass(frozen=True) +class RegisteredPhaseCarrierScore: + """Best phase-carrier score across a bounded translation search.""" + + path: str + score: float + active_weight_fraction: float + peak_count: int + row_shift: int + column_shift: int + + +def _load_rgb(path: Path, *, height: int, width: int, canonicalize_geometry: bool = False) -> np.ndarray: + """Load PATH as float64 RGB, optionally resizing to model geometry.""" + with Image.open(path) as image: + rgb = image.convert("RGB") + if rgb.size != (width, height): + if not canonicalize_geometry: + raise ValueError(f"{path}: geometry {rgb.width}x{rgb.height} does not match {width}x{height}") + rgb = rgb.resize((width, height), Image.Resampling.LANCZOS) + return np.asarray(rgb, dtype=np.float64) + + +def _valid_frequency_mask(height: int, width: int, min_radius: float) -> np.ndarray: + """Return eligible non-DC bins in an rFFT half-plane.""" + rows = np.arange(height) + signed_rows = np.where(rows > height // 2, rows - height, rows) + columns = np.arange(width // 2 + 1) + radius = np.sqrt(np.square(signed_rows[:, None]) + np.square(columns[None, :])) + return (radius >= min_radius) & (columns[None, :] > 0) + + +def _leave_one_out_coherence(unit_sum: np.ndarray, held_out_unit: np.ndarray, count: float) -> np.ndarray: + """Return phase coherence after removing HELD_OUT_UNIT from UNIT_SUM.""" + if count <= 1.0: + raise ValueError("leave-one-out coherence requires at least two samples") + return np.abs((unit_sum - held_out_unit) / (count - 1.0)) + + +def discover_model( + paths: list[Path], + *, + peak_count: int = 256, + min_radius: float = 15.0, + candidate_bins: np.ndarray | None = None, +) -> PhaseCarrierModel: + """Learn a sparse phase-consensus model from exact-geometry PATHS.""" + if len(paths) < 3: + raise ValueError("at least three positive images are required") + with Image.open(paths[0]) as first: + width, height = first.size + if min(height, width) < 64: + raise ValueError("images must be at least 64 pixels per side") + + half_width = width // 2 + 1 + unit_sum = np.zeros((height, half_width, 3), dtype=np.complex64) + magnitude_sum = np.zeros((height, half_width, 3), dtype=np.float64) + for path in paths: + pixels = _load_rgb(path, height=height, width=width) + for channel in range(3): + spectrum = np.fft.rfft2(pixels[:, :, channel]) + magnitude = np.abs(spectrum) + unit_sum[:, :, channel] += np.divide( + spectrum, + magnitude, + out=np.zeros_like(spectrum), + where=magnitude != 0.0, + ).astype(np.complex64) + magnitude_sum[:, :, channel] += magnitude + + count = float(len(paths)) + mean_unit = unit_sum / count + minimum_loo_coherence = np.ones((height, half_width, 3), dtype=np.float32) + for path in paths: + pixels = _load_rgb(path, height=height, width=width) + for channel in range(3): + spectrum = np.fft.rfft2(pixels[:, :, channel]) + magnitude = np.abs(spectrum) + unit = np.divide( + spectrum, + magnitude, + out=np.zeros_like(spectrum), + where=magnitude != 0.0, + ) + loo_coherence = _leave_one_out_coherence(unit_sum[:, :, channel], unit, count) + np.minimum(minimum_loo_coherence[:, :, channel], loo_coherence, out=minimum_loo_coherence[:, :, channel]) + expected_magnitude = magnitude_sum / count + selection = np.power(minimum_loo_coherence.astype(np.float64), 4.0) * np.log1p(expected_magnitude) + selection *= _valid_frequency_mask(height, width, min_radius)[:, :, None] + if candidate_bins is None: + candidate_indices = np.flatnonzero(selection) + else: + bins = np.asarray(candidate_bins, dtype=np.int64) + if bins.ndim != 2 or bins.shape[1] != 3: + raise ValueError("candidate_bins must have shape (count, 3)") + if ( + np.any(bins[:, 0] < 0) + or np.any(bins[:, 0] >= height) + or np.any(bins[:, 1] <= 0) + or np.any(bins[:, 1] > width // 2) + or np.any(bins[:, 2] < 0) + or np.any(bins[:, 2] > 2) + ): + raise ValueError("candidate_bins contain out-of-range coordinates") + candidate_indices = np.unique(np.ravel_multi_index(bins.T, selection.shape)) + candidate_indices = candidate_indices[selection.ravel()[candidate_indices] > 0.0] + candidate_count = len(candidate_indices) + if candidate_count < peak_count: + raise ValueError(f"only {candidate_count} eligible bins for {peak_count} peaks") + + flat = selection.ravel() + candidate_scores = flat[candidate_indices] + chosen = np.argpartition(candidate_scores, -peak_count)[-peak_count:] + indices = candidate_indices[chosen] + indices = indices[np.argsort(flat[indices])[::-1]] + rows, columns, channels = np.unravel_index(indices, selection.shape) + raw_weights = selection[rows, columns, channels] + return PhaseCarrierModel( + height=height, + width=width, + rows=rows.astype(np.int32), + columns=columns.astype(np.int32), + channels=channels.astype(np.int8), + phases=np.angle(mean_unit[rows, columns, channels]).astype(np.float64), + weights=(raw_weights / np.sum(raw_weights)).astype(np.float64), + expected_magnitudes=expected_magnitude[rows, columns, channels].astype(np.float64), + ) + + +def _frequency_values(pixels: np.ndarray, model: PhaseCarrierModel) -> np.ndarray: + """Return MODEL's selected complex coefficients from PIXELS.""" + return extract_frequency_values(pixels, model.rows, model.columns, model.channels) + + +def score_pixels(pixels: np.ndarray, model: PhaseCarrierModel, *, path: str = "") -> PhaseCarrierScore: + """Score exact-geometry RGB PIXELS against MODEL.""" + if pixels.shape != (model.height, model.width, 3): + raise ValueError("pixel geometry does not match phase-carrier model") + values = _frequency_values(np.asarray(pixels, dtype=np.float64), model) + magnitude_gate = np.minimum(np.abs(values) / (model.expected_magnitudes + 1e-12), 1.0) + active_weights = model.weights * magnitude_gate + active_weight = float(np.sum(active_weights)) + score = ( + 0.0 + if active_weight == 0.0 + else float(np.sum(active_weights * np.cos(np.angle(values) - model.phases)) / active_weight) + ) + return PhaseCarrierScore( + path=path, + score=score, + active_weight_fraction=active_weight, + peak_count=len(model.rows), + ) + + +def score_image( + path: Path, + model: PhaseCarrierModel, + *, + canonicalize_geometry: bool = False, +) -> PhaseCarrierScore: + """Score PATH against MODEL, with optional geometry canonicalization.""" + pixels = _load_rgb( + path, + height=model.height, + width=model.width, + canonicalize_geometry=canonicalize_geometry, + ) + return score_pixels(pixels, model, path=str(path)) + + +def score_translations( + path: Path, + model: PhaseCarrierModel, + *, + max_shift: int = 4, + canonicalize_geometry: bool = False, +) -> RegisteredPhaseCarrierScore: + """Return the best score after compensating bounded integer translations.""" + pixels = _load_rgb( + path, + height=model.height, + width=model.width, + canonicalize_geometry=canonicalize_geometry, + ) + registration = register_phase_translations( + _frequency_values(pixels, model), + phases=model.phases, + weights=model.weights, + expected_magnitudes=model.expected_magnitudes, + rows=model.rows, + columns=model.columns, + height=model.height, + width=model.width, + max_shift=max_shift, + ) + return RegisteredPhaseCarrierScore( + path=str(path), + score=registration.score, + active_weight_fraction=registration.active_weight_fraction, + peak_count=len(model.rows), + row_shift=registration.row_shift, + column_shift=registration.column_shift, + ) + + +def save_model(path: Path, model: PhaseCarrierModel) -> None: + """Save MODEL as a validated numeric NPZ artifact.""" + path.parent.mkdir(parents=True, exist_ok=True) + np.savez_compressed( + path, + format_version=np.asarray(1, dtype=np.int32), + height=np.asarray(model.height, dtype=np.int32), + width=np.asarray(model.width, dtype=np.int32), + rows=model.rows.astype(np.int32), + columns=model.columns.astype(np.int32), + channels=model.channels.astype(np.int8), + phases=model.phases.astype(np.float32), + weights=model.weights.astype(np.float32), + expected_magnitudes=model.expected_magnitudes.astype(np.float64), + ) + + +def load_model(path: Path) -> PhaseCarrierModel: + """Load and validate a numeric phase-carrier artifact.""" + with np.load(path, allow_pickle=False) as artifact: + if int(artifact["format_version"]) != 1: + raise ValueError("unsupported phase-carrier format version") + model = PhaseCarrierModel( + height=int(artifact["height"]), + width=int(artifact["width"]), + rows=np.asarray(artifact["rows"], dtype=np.int32), + columns=np.asarray(artifact["columns"], dtype=np.int32), + channels=np.asarray(artifact["channels"], dtype=np.int8), + phases=np.asarray(artifact["phases"], dtype=np.float64), + weights=np.asarray(artifact["weights"], dtype=np.float64), + expected_magnitudes=np.asarray(artifact["expected_magnitudes"], dtype=np.float64), + ) + count = len(model.rows) + arrays = (model.columns, model.channels, model.phases, model.weights, model.expected_magnitudes) + if model.height < 64 or model.width < 64 or any(array.shape != (count,) for array in arrays): + raise ValueError("invalid phase-carrier model shapes") + if count == 0 or np.any(model.rows < 0) or np.any(model.rows >= model.height): + raise ValueError("invalid phase-carrier row indices") + if np.any(model.columns <= 0) or np.any(model.columns > model.width // 2): + raise ValueError("invalid phase-carrier column indices") + if np.any(model.channels < 0) or np.any(model.channels > 2): + raise ValueError("invalid phase-carrier channel indices") + if not np.isclose(np.sum(model.weights), 1.0, atol=1e-5) or np.any(model.weights < 0.0): + raise ValueError("invalid phase-carrier weights") + return model + + +@click.group() +def main() -> None: + """Discover and evaluate an exact-geometry phase carrier.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + + +@main.command() +@click.argument("positives", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--peak-count", type=click.IntRange(min=1), default=256, show_default=True) +@click.option("--candidate-codebook", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--candidate-count", type=click.IntRange(min=1), default=16384, show_default=True) +@click.option("--model-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def discover( + positives: tuple[Path, ...], + peak_count: int, + candidate_codebook: Path | None, + candidate_count: int, + model_out: Path, +) -> None: + """Learn a phase carrier from exact-geometry POSITIVES.""" + candidate_bins: np.ndarray | None = None + if candidate_codebook is not None: + from synthid_v3_codebook_probe import load_v3_model + + with Image.open(positives[0]) as first: + width, height = first.size + prior = load_v3_model( + candidate_codebook, + height=height, + width=width, + peak_count=candidate_count, + ) + candidate_bins = np.column_stack((prior.rows, prior.columns, prior.channels)) + model = discover_model(list(positives), peak_count=peak_count, candidate_bins=candidate_bins) + save_model(model_out, model) + log.info("Wrote phase-carrier model: %s", model_out) + + +@main.command() +@click.argument("model_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +@click.option("--canonicalize-geometry", is_flag=True, help="Resize inputs to the model geometry before scoring.") +@click.option("--max-shift", type=click.IntRange(min=0, max=MAX_TRANSLATION_SHIFT), default=0, show_default=True) +def score( + model_path: Path, + images: tuple[Path, ...], + report_out: Path, + canonicalize_geometry: bool, + max_shift: int, +) -> None: + """Score IMAGES with MODEL_PATH.""" + model = load_model(model_path) + scores = ( + [asdict(score_image(image, model, canonicalize_geometry=canonicalize_geometry)) for image in images] + if max_shift == 0 + else [ + asdict( + score_translations( + image, + model, + max_shift=max_shift, + canonicalize_geometry=canonicalize_geometry, + ) + ) + for image in images + ] + ) + payload = { + "model": str(model_path), + "height": model.height, + "width": model.width, + "peak_count": len(model.rows), + "canonicalize_geometry": canonicalize_geometry, + "max_shift": max_shift, + "scores": scores, + } + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") + log.info("Wrote phase-carrier score report: %s", report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_phase_registration.py b/scripts/synthid_phase_registration.py new file mode 100644 index 0000000..6e4f1e3 --- /dev/null +++ b/scripts/synthid_phase_registration.py @@ -0,0 +1,92 @@ +"""Shared bounded translation registration for phase-carrier probes.""" + +from __future__ import annotations + +from dataclasses import dataclass + +import numpy as np + +MAX_TRANSLATION_SHIFT = 32 + + +@dataclass(frozen=True) +class TranslationRegistration: + """Best phase score and offset from one bounded translation search.""" + + score: float + active_weight_fraction: float + row_shift: int + column_shift: int + + +def phase_adjustment( + rows: np.ndarray, + columns: np.ndarray, + *, + height: int, + width: int, + row_shift: int | np.ndarray, + column_shift: int | np.ndarray, +) -> np.ndarray: + """Return the Fourier phase adjustment for one integer translation.""" + signed_rows = np.where(rows > height // 2, rows - height, rows) + return 2.0 * np.pi * (signed_rows * row_shift / height + columns * column_shift / width) + + +def extract_frequency_values( + pixels: np.ndarray, + rows: np.ndarray, + columns: np.ndarray, + channels: np.ndarray, +) -> np.ndarray: + """Extract sparse three-channel rFFT coefficients from PIXELS.""" + values = np.empty(len(rows), dtype=np.complex128) + for channel in range(3): + positions = np.flatnonzero(channels == channel) + if len(positions) == 0: + continue + spectrum = np.fft.rfft2(pixels[:, :, channel]) + values[positions] = spectrum[rows[positions], columns[positions]] + return values + + +def register_phase_translations( + values: np.ndarray, + *, + phases: np.ndarray, + weights: np.ndarray, + expected_magnitudes: np.ndarray, + rows: np.ndarray, + columns: np.ndarray, + height: int, + width: int, + max_shift: int, +) -> TranslationRegistration: + """Find the strongest phase alignment over bounded integer translations.""" + if not 0 <= max_shift <= MAX_TRANSLATION_SHIFT: + raise ValueError(f"max_shift must be between 0 and {MAX_TRANSLATION_SHIFT}") + magnitude_gate = np.minimum(np.abs(values) / (expected_magnitudes + 1e-12), 1.0) + active_weights = weights * magnitude_gate + active_weight = float(np.sum(active_weights)) + if active_weight == 0.0: + return TranslationRegistration(0.0, 0.0, 0, 0) + + shifts = np.arange(-max_shift, max_shift + 1) + adjustment = phase_adjustment( + rows[:, None, None], + columns[:, None, None], + height=height, + width=width, + row_shift=shifts[None, :, None], + column_shift=shifts[None, None, :], + ) + difference = np.angle(values) - phases + scores = np.sum(active_weights[:, None, None] * np.cos(difference[:, None, None] + adjustment), axis=0) + scores /= active_weight + row_index, column_index = np.unravel_index(int(np.argmax(scores)), scores.shape) + return TranslationRegistration( + score=float(scores[row_index, column_index]), + active_weight_fraction=active_weight, + row_shift=int(shifts[row_index]), + column_shift=int(shifts[column_index]), + ) diff --git a/scripts/synthid_pixel_attack.py b/scripts/synthid_pixel_attack.py new file mode 100644 index 0000000..b604043 --- /dev/null +++ b/scripts/synthid_pixel_attack.py @@ -0,0 +1,199 @@ +"""Build deterministic, pixel-only SynthID attack candidates and controls. + +The generated variants use quantization, resampling, and a smooth sub-pixel +warp. No generative model or image synthesis stage is involved. The command +also emits a norm-matched random-noise control so an oracle change cannot be +attributed to pixel distance alone. + +This is a research harness. A candidate is successful only when the matching +provider oracle changes from detected to not detected while the crop-only +control remains detected. +""" + +from __future__ import annotations + +import json +import logging +import math +from dataclasses import asdict, dataclass +from pathlib import Path + +import click +import cv2 +import numpy as np +from invisible_quality_audit import _ssim +from PIL import Image + +log = logging.getLogger(__name__) + + +@dataclass(frozen=True) +class FidelityMeasurement: + """Paired pixel metrics for one attack candidate.""" + + name: str + path: str + width: int + height: int + psnr_db: float + ssim: float + changed_pixel_fraction: float + residual_rms: float + residual_max: float + + +def load_rgb(path: Path) -> np.ndarray: + """Load PATH as uint8 RGB pixels.""" + with Image.open(path) as image: + return np.asarray(image.convert("RGB"), dtype=np.uint8) + + +def crop_visible_badge(pixels: np.ndarray, margin: int) -> np.ndarray: + """Remove the bottom and right margins that contain the visible badge.""" + height, width = pixels.shape[:2] + if margin < 0 or margin >= min(height, width): + raise ValueError("crop margin must be nonnegative and smaller than the image") + if margin == 0: + return pixels.copy() + return pixels[: height - margin, : width - margin].copy() + + +def quantize(pixels: np.ndarray, step: int) -> np.ndarray: + """Round RGB samples to the nearest multiple of STEP.""" + if step < 2 or step > 64: + raise ValueError("quantization step must be between 2 and 64") + values = np.rint(pixels.astype(np.float64) / step) * step + return np.clip(values, 0, 255).astype(np.uint8) + + +def smooth_warp(pixels: np.ndarray, *, amplitude: float, sigma: float, seed: int) -> np.ndarray: + """Apply a deterministic smooth sub-pixel displacement field.""" + if amplitude < 0.0 or sigma <= 0.0: + raise ValueError("warp amplitude must be nonnegative and sigma positive") + height, width = pixels.shape[:2] + rng = np.random.default_rng(seed) + fields: list[np.ndarray] = [] + for _ in range(2): + noise = rng.normal(size=(height, width)).astype(np.float32) + field = cv2.GaussianBlur(noise, (0, 0), sigmaX=sigma, sigmaY=sigma) + field_std = float(np.std(field)) + fields.append(np.zeros_like(field) if field_std == 0.0 else field * (amplitude / field_std)) + yy, xx = np.mgrid[:height, :width].astype(np.float32) + return cv2.remap( + pixels, + xx + fields[0], + yy + fields[1], + interpolation=cv2.INTER_LANCZOS4, + borderMode=cv2.BORDER_REFLECT_101, + ) + + +def resize_squeeze(pixels: np.ndarray, factor: float) -> np.ndarray: + """Downsample and restore the original geometry without synthesis.""" + if not 0.5 <= factor < 1.0: + raise ValueError("resize factor must be in [0.5, 1.0)") + height, width = pixels.shape[:2] + reduced = cv2.resize( + pixels, + (max(1, round(width * factor)), max(1, round(height * factor))), + interpolation=cv2.INTER_AREA, + ) + return cv2.resize(reduced, (width, height), interpolation=cv2.INTER_LANCZOS4) + + +def jpeg_round_trip(pixels: np.ndarray, quality: int) -> np.ndarray: + """Apply one in-memory JPEG encode/decode while returning RGB pixels.""" + if quality < 1 or quality > 100: + raise ValueError("JPEG quality must be between 1 and 100") + success, encoded = cv2.imencode( + ".jpg", + cv2.cvtColor(pixels, cv2.COLOR_RGB2BGR), + [cv2.IMWRITE_JPEG_QUALITY, quality], + ) + if not success: + raise RuntimeError("JPEG encoding failed") + decoded = cv2.imdecode(encoded, cv2.IMREAD_COLOR) + if decoded is None: + raise RuntimeError("JPEG decoding failed") + return cv2.cvtColor(decoded, cv2.COLOR_BGR2RGB) + + +def norm_matched_noise(reference: np.ndarray, target: np.ndarray, *, seed: int) -> np.ndarray: + """Return random RGB noise with approximately TARGET's residual RMS.""" + target_residual = target.astype(np.float64) - reference.astype(np.float64) + target_rms = float(np.sqrt(np.mean(np.square(target_residual)))) + rng = np.random.default_rng(seed) + noise = rng.normal(size=reference.shape) + noise *= target_rms / (float(np.sqrt(np.mean(np.square(noise)))) + 1e-12) + return np.clip(np.rint(reference.astype(np.float64) + noise), 0, 255).astype(np.uint8) + + +def measure(reference: np.ndarray, candidate: np.ndarray, *, name: str, path: Path) -> FidelityMeasurement: + """Measure paired fidelity between equal-shaped RGB arrays.""" + if reference.shape != candidate.shape: + raise ValueError("reference and candidate shapes differ") + residual = candidate.astype(np.float64) - reference.astype(np.float64) + mse = float(np.mean(np.square(residual))) + psnr = math.inf if mse == 0.0 else 20.0 * math.log10(255.0 / math.sqrt(mse)) + reference_gray = cv2.cvtColor(reference, cv2.COLOR_RGB2GRAY) + candidate_gray = cv2.cvtColor(candidate, cv2.COLOR_RGB2GRAY) + return FidelityMeasurement( + name=name, + path=str(path), + width=int(reference.shape[1]), + height=int(reference.shape[0]), + psnr_db=psnr, + ssim=float(_ssim(reference_gray, candidate_gray)), + changed_pixel_fraction=float(np.mean(np.any(residual != 0.0, axis=2))), + residual_rms=float(math.sqrt(mse)), + residual_max=float(np.max(np.abs(residual))), + ) + + +def build_candidates(source: np.ndarray) -> dict[str, np.ndarray]: + """Build the preregistered attack batch from cropped SOURCE pixels.""" + candidates: dict[str, np.ndarray] = { + "control-crop": source.copy(), + "quantize-2": quantize(source, 2), + "quantize-4": quantize(source, 4), + "quantize-8": quantize(source, 8), + "warp-035": smooth_warp(source, amplitude=0.35, sigma=48.0, seed=20260809), + } + combo = smooth_warp(source, amplitude=0.55, sigma=48.0, seed=20260810) + combo = resize_squeeze(combo, 0.96) + combo = quantize(combo, 4) + combo = jpeg_round_trip(combo, 96) + candidates["combo-mild"] = combo + candidates["sham-combo-rms"] = norm_matched_noise(source, combo, seed=20260811) + return candidates + + +@click.command() +@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("output_dir", type=click.Path(file_okay=False, path_type=Path)) +@click.option("--crop-margin", type=click.IntRange(min=0), default=160, show_default=True) +def main(source: Path, output_dir: Path, crop_margin: int) -> None: + """Write a frozen pixel-only attack batch for SOURCE into OUTPUT_DIR.""" + reference = crop_visible_badge(load_rgb(source), crop_margin) + output_dir.mkdir(parents=True, exist_ok=True) + measurements: list[FidelityMeasurement] = [] + for name, pixels in build_candidates(reference).items(): + path = output_dir / f"{name}.png" + Image.fromarray(pixels, mode="RGB").save(path) + measurements.append(measure(reference, pixels, name=name, path=path)) + report_path = output_dir / "fidelity.json" + payload = { + "source": str(source), + "crop_margin": crop_margin, + "variants": [asdict(row) for row in measurements], + } + report_path.write_text( + json.dumps(payload, indent=2) + "\n", + encoding="utf-8", + ) + log.info("Wrote %d candidates and fidelity report: %s", len(measurements), report_path) + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(message)s") + main() diff --git a/scripts/synthid_research_inventory.py b/scripts/synthid_research_inventory.py new file mode 100644 index 0000000..fd4cf71 --- /dev/null +++ b/scripts/synthid_research_inventory.py @@ -0,0 +1,170 @@ +"""Inventory local SynthID research images without assigning evidence labels. + +The inventory is deliberately weaker than the research manifest. It records +artifact hashes, decoded RGB hashes, geometry, format, and exact duplicates, +but contains no provider, SynthID outcome, oracle, or split fields. Promotion +from inventory to manifest therefore remains an explicit evidence decision. + +Usage: + uv run python scripts/synthid_research_inventory.py \ + --root .local-eval/synthid negatives google openai \ + --inventory-out .local-eval/synthid/inventory.csv +""" + +from __future__ import annotations + +import csv +import logging +from collections import Counter +from dataclasses import asdict, dataclass +from pathlib import Path + +import click +from synthid_research_manifest import artifact_sha256, decoded_image_fingerprint + +log = logging.getLogger(__name__) + +IMAGE_SUFFIXES = {".jpeg", ".jpg", ".png", ".webp"} +FIELDNAMES = ( + "artifact_sha256", + "pixel_sha256", + "artifact_path", + "width", + "height", + "format", + "exact_pixel_group", + "artifact_duplicate_of", + "pixel_duplicate_of", +) + + +@dataclass(frozen=True) +class InventoryRow: + """One decoded local image with exact-duplicate provenance.""" + + artifact_sha256: str + pixel_sha256: str + artifact_path: str + width: int + height: int + format: str + exact_pixel_group: str + artifact_duplicate_of: str + pixel_duplicate_of: str + + +def _inside_root(root: Path, path: Path) -> Path: + """Resolve PATH and reject anything outside ROOT.""" + resolved_root = root.resolve() + resolved = path.resolve() + try: + resolved.relative_to(resolved_root) + except ValueError as exc: + raise ValueError(f"source is outside inventory root: {path}") from exc + return resolved + + +def discover_images(root: Path, sources: tuple[Path, ...]) -> list[Path]: + """Return supported images below explicit in-root sources in stable order.""" + resolved_root = root.resolve() + discovered: set[Path] = set() + for source in sources: + candidate = source if source.is_absolute() else resolved_root / source + candidate = _inside_root(resolved_root, candidate) + if not candidate.exists(): + raise ValueError(f"inventory source does not exist: {source}") + if candidate.is_file(): + if candidate.suffix.lower() in IMAGE_SUFFIXES: + discovered.add(candidate) + continue + for path in candidate.rglob("*"): + if path.is_file() and path.suffix.lower() in IMAGE_SUFFIXES: + discovered.add(_inside_root(resolved_root, path)) + return sorted(discovered, key=lambda path: path.relative_to(resolved_root).as_posix()) + + +def build_inventory(root: Path, sources: tuple[Path, ...]) -> list[InventoryRow]: + """Hash and decode all selected images without inferring any labels.""" + resolved_root = root.resolve() + first_artifact: dict[str, str] = {} + first_pixels: dict[str, str] = {} + rows: list[InventoryRow] = [] + for path in discover_images(resolved_root, sources): + relative = path.relative_to(resolved_root).as_posix() + artifact_digest = artifact_sha256(path) + pixel_digest, width, height, image_format = decoded_image_fingerprint(path) + rows.append( + InventoryRow( + artifact_sha256=artifact_digest, + pixel_sha256=pixel_digest, + artifact_path=relative, + width=width, + height=height, + format=image_format, + exact_pixel_group=f"pixel-{pixel_digest[:16]}", + artifact_duplicate_of=first_artifact.get(artifact_digest, ""), + pixel_duplicate_of=first_pixels.get(pixel_digest, ""), + ) + ) + first_artifact.setdefault(artifact_digest, relative) + first_pixels.setdefault(pixel_digest, relative) + return rows + + +def write_inventory(path: Path, rows: list[InventoryRow], *, replace: bool = False) -> None: + """Write a complete inventory atomically enough to avoid partial decode results.""" + if path.exists() and not replace: + raise FileExistsError(f"inventory already exists: {path}; pass --replace to overwrite it") + path.parent.mkdir(parents=True, exist_ok=True) + temporary = path.with_name(f".{path.name}.tmp") + try: + with temporary.open("w", newline="", encoding="utf-8") as stream: + writer = csv.DictWriter(stream, fieldnames=FIELDNAMES) + writer.writeheader() + writer.writerows(asdict(row) for row in rows) + temporary.replace(path) + finally: + if temporary.exists(): + temporary.unlink() + + +def inventory_summary(rows: list[InventoryRow]) -> dict[str, object]: + """Return aggregate coverage without exposing media paths.""" + formats = Counter(row.format for row in rows) + geometries = Counter(f"{row.width}x{row.height}" for row in rows) + return { + "images": len(rows), + "unique_artifacts": len({row.artifact_sha256 for row in rows}), + "unique_pixels": len({row.pixel_sha256 for row in rows}), + "formats": dict(sorted(formats.items())), + "geometries": dict(sorted(geometries.items())), + } + + +@click.command() +@click.option("--root", required=True, type=click.Path(exists=True, file_okay=False, path_type=Path)) +@click.argument("sources", nargs=-1, required=True, type=click.Path(path_type=Path)) +@click.option("--inventory-out", required=True, type=click.Path(dir_okay=False, path_type=Path)) +@click.option("--replace", is_flag=True, help="Replace an existing generated inventory.") +def main(root: Path, sources: tuple[Path, ...], inventory_out: Path, replace: bool) -> None: + """Inventory image SOURCES below ROOT without assigning SynthID labels.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + try: + rows = build_inventory(root, sources) + if not rows: + raise ValueError("selected sources contain no supported images") + write_inventory(inventory_out, rows, replace=replace) + except (OSError, ValueError) as exc: + raise click.ClickException(str(exc)) from exc + summary = inventory_summary(rows) + log.info( + "Wrote inventory: %s images=%s unique_artifacts=%s unique_pixels=%s", + inventory_out, + summary["images"], + summary["unique_artifacts"], + summary["unique_pixels"], + ) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_research_manifest.py b/scripts/synthid_research_manifest.py new file mode 100644 index 0000000..c5b0779 --- /dev/null +++ b/scripts/synthid_research_manifest.py @@ -0,0 +1,347 @@ +"""Audit a private SynthID research manifest before training or evaluation. + +The research manifest is intentionally separate from ``data/synthid/manifest.csv``. +The latter records a small public regression corpus, while this schema tracks +private experiment lineage, provider-specific oracle evidence, and split groups. + +Usage: + uv run python scripts/synthid_research_manifest.py MANIFEST.csv + uv run python scripts/synthid_research_manifest.py MANIFEST.csv --verify-files +""" + +from __future__ import annotations + +import csv +import hashlib +import logging +import re +from collections import defaultdict +from datetime import datetime +from pathlib import Path + +import click +from PIL import Image + +log = logging.getLogger(__name__) + +FIELDNAMES = ( + "artifact_sha256", + "pixel_sha256", + "artifact_path", + "parent_sha256", + "group_id", + "target_provider", + "source_provider", + "surface", + "model_epoch", + "generation_session", + "content_stratum", + "width", + "height", + "format", + "transform", + "split", + "c2pa_outcome", + "synthid_outcome", + "verified_via", + "evidence_reference", + "oracle_session", + "oracle_role", + "captured_at", + "oracle_checked_at", + "notes", +) + +_SHA256 = re.compile(r"^[0-9a-f]{64}$") +_TARGET_PROVIDERS = {"openai", "google"} +_SOURCE_PROVIDERS = {"openai", "google", "camera", "other_ai", "synthetic", "editor"} +_SPLITS = {"discovery", "train", "validation", "test", "temporal"} +_C2PA_OUTCOMES = {"detected", "not_detected", "invalid", "not_present", "not_checked"} +_SYNTHID_OUTCOMES = {"detected", "not_detected", "indeterminate", "refused", "not_checked"} +_VERIFIERS = {"openai-api", "openai-web", "gemini-app", "synthid-portal", "source-evidence", "none"} +_FORMATS = {"png", "jpeg", "webp"} +_ORACLE_ROLES = {"ordinary", "source_control", "candidate", "sham"} +_FINAL_SPLITS = {"train", "validation", "test", "temporal"} +_MATCHING_VERIFIERS = { + "openai": {"openai-api", "openai-web"}, + "google": {"gemini-app", "synthid-portal"}, +} + + +def _read_rows(path: Path) -> tuple[list[dict[str, str]], list[str]]: + """Read a CSV and return rows plus header errors.""" + with path.open(newline="", encoding="utf-8") as stream: + reader = csv.DictReader(stream) + actual = tuple(reader.fieldnames or ()) + missing = [field for field in FIELDNAMES if field not in actual] + errors = [f"header: missing required field {field!r}" for field in missing] + return list(reader), errors + + +def _is_iso8601(value: str) -> bool: + """Return whether VALUE is a timezone-aware ISO-8601 timestamp.""" + if not value: + return False + try: + parsed = datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError: + return False + return parsed.tzinfo is not None + + +def _file_sha256(path: Path) -> str: + """Hash a file without loading it entirely into memory.""" + digest = hashlib.sha256() + with path.open("rb") as stream: + for chunk in iter(lambda: stream.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + +def decoded_image_fingerprint(path: Path) -> tuple[str, int, int, str]: + """Return the decoded-RGB digest, geometry, and source format.""" + with Image.open(path) as image: + image_format = (image.format or path.suffix.lstrip(".")).lower() + rgb = image.convert("RGB") + digest = hashlib.sha256(rgb.tobytes()).hexdigest() + return digest, rgb.width, rgb.height, "jpeg" if image_format == "jpg" else image_format + + +def _pixel_sha256(path: Path) -> tuple[str, int, int]: + """Hash canonical decoded RGB pixels and return hash, width, and height.""" + digest, width, height, _ = decoded_image_fingerprint(path) + return digest, width, height + + +def artifact_sha256(path: Path) -> str: + """Return the artifact digest used by research manifests and inventories.""" + return _file_sha256(path) + + +def pixel_fingerprint(path: Path) -> tuple[str, int, int]: + """Return the decoded-RGB digest and geometry used by manifest verification.""" + return _pixel_sha256(path) + + +def resolve_artifact_path(root: Path, value: str) -> Path | None: + """Resolve a manifest-relative artifact path without allowing traversal.""" + relative = Path(value) + if not value or relative.is_absolute() or ".." in relative.parts: + return None + candidate = (root / relative).resolve() + try: + candidate.relative_to(root.resolve()) + except ValueError: + return None + return candidate + + +def _row_errors(row: dict[str, str], index: int) -> list[str]: + """Validate one row without consulting other rows.""" + prefix = f"row {index}" + errors: list[str] = [] + + artifact_sha = row.get("artifact_sha256", "") + pixel_sha = row.get("pixel_sha256", "") + parent_sha = row.get("parent_sha256", "") + if not _SHA256.fullmatch(artifact_sha): + errors.append(f"{prefix}: invalid artifact_sha256") + if not _SHA256.fullmatch(pixel_sha): + errors.append(f"{prefix}: invalid pixel_sha256") + if parent_sha and not _SHA256.fullmatch(parent_sha): + errors.append(f"{prefix}: invalid parent_sha256") + + for field in ("group_id", "surface", "model_epoch", "generation_session", "content_stratum", "transform"): + if not row.get(field, "").strip(): + errors.append(f"{prefix}: {field} must not be empty") + + target = row.get("target_provider", "") + source = row.get("source_provider", "") + split = row.get("split", "") + c2pa = row.get("c2pa_outcome", "") + synthid = row.get("synthid_outcome", "") + verifier = row.get("verified_via", "") + oracle_role = row.get("oracle_role", "") + image_format = row.get("format", "").lower() + + if target not in _TARGET_PROVIDERS: + errors.append(f"{prefix}: unsupported target_provider {target!r}") + if source not in _SOURCE_PROVIDERS: + errors.append(f"{prefix}: unsupported source_provider {source!r}") + if split not in _SPLITS: + errors.append(f"{prefix}: unsupported split {split!r}") + if c2pa not in _C2PA_OUTCOMES: + errors.append(f"{prefix}: unsupported c2pa_outcome {c2pa!r}") + if synthid not in _SYNTHID_OUTCOMES: + errors.append(f"{prefix}: unsupported synthid_outcome {synthid!r}") + if verifier not in _VERIFIERS: + errors.append(f"{prefix}: unsupported verified_via {verifier!r}") + if image_format not in _FORMATS: + errors.append(f"{prefix}: unsupported format {image_format!r}") + if oracle_role not in _ORACLE_ROLES: + errors.append(f"{prefix}: unsupported oracle_role {oracle_role!r}") + + for dimension in ("width", "height"): + try: + if int(row.get(dimension, "")) <= 0: + raise ValueError + except ValueError: + errors.append(f"{prefix}: {dimension} must be a positive integer") + + if not _is_iso8601(row.get("captured_at", "")): + errors.append(f"{prefix}: captured_at must be timezone-aware ISO-8601") + + final_outcome = synthid in {"detected", "not_detected"} + if split in _FINAL_SPLITS and not final_outcome: + errors.append(f"{prefix}: split {split!r} requires a detected or not_detected SynthID outcome") + if final_outcome and not _is_iso8601(row.get("oracle_checked_at", "")): + errors.append(f"{prefix}: a final SynthID outcome requires oracle_checked_at") + + matching = _MATCHING_VERIFIERS.get(target, set()) + if synthid == "detected" and verifier not in matching: + errors.append(f"{prefix}: a detected {target!r} signal requires a matching provider verifier") + if synthid == "not_detected" and source == target and verifier not in matching: + errors.append(f"{prefix}: a same-provider negative requires a matching provider verifier") + if verifier in {"source-evidence", "none"} and synthid == "detected": + errors.append(f"{prefix}: {verifier!r} cannot establish a positive SynthID label") + if verifier == "source-evidence" and source == target: + errors.append(f"{prefix}: source-evidence cannot establish a same-provider negative") + if verifier == "source-evidence" and not row.get("evidence_reference", "").strip(): + errors.append(f"{prefix}: source-evidence requires evidence_reference") + if verifier in matching and not row.get("oracle_session", "").strip(): + errors.append(f"{prefix}: provider verification requires oracle_session") + if oracle_role == "source_control" and synthid != "detected": + errors.append(f"{prefix}: a source_control must have a detected SynthID outcome") + + transform = row.get("transform", "") + if transform == "original" and parent_sha: + errors.append(f"{prefix}: an original must not have parent_sha256") + if transform != "original" and not parent_sha: + errors.append(f"{prefix}: a derivative requires parent_sha256") + + return errors + + +def _lineage_errors(rows: list[dict[str, str]]) -> list[str]: + """Validate uniqueness, parent links, group splits, and lineage cycles.""" + errors: list[str] = [] + by_sha: dict[str, dict[str, str]] = {} + group_splits: defaultdict[str, set[str]] = defaultdict(set) + pixel_groups: defaultdict[str, set[str]] = defaultdict(set) + + for index, row in enumerate(rows, start=2): + artifact_sha = row.get("artifact_sha256", "") + if artifact_sha in by_sha: + errors.append(f"row {index}: duplicate artifact_sha256 {artifact_sha}") + else: + by_sha[artifact_sha] = row + group_splits[row.get("group_id", "")].add(row.get("split", "")) + pixel_groups[row.get("pixel_sha256", "")].add(row.get("group_id", "")) + + for index, row in enumerate(rows, start=2): + parent_sha = row.get("parent_sha256", "") + if not parent_sha: + continue + parent = by_sha.get(parent_sha) + if parent is None: + errors.append(f"row {index}: parent_sha256 is not present in the manifest") + continue + if parent.get("group_id") != row.get("group_id"): + errors.append(f"row {index}: derivative and parent must share group_id") + if parent.get("target_provider") != row.get("target_provider"): + errors.append(f"row {index}: derivative and parent must share target_provider") + + for group_id, splits in sorted(group_splits.items()): + if group_id and len(splits) > 1: + errors.append(f"group {group_id!r}: leaks across splits {sorted(splits)}") + for pixel_sha, groups in sorted(pixel_groups.items()): + if pixel_sha and len(groups) > 1: + errors.append(f"pixel_sha256 {pixel_sha}: appears in multiple groups {sorted(groups)}") + + for artifact_sha in by_sha: + seen: set[str] = set() + current_sha = artifact_sha + while current_sha: + if current_sha in seen: + errors.append(f"artifact_sha256 {artifact_sha}: lineage cycle detected") + break + seen.add(current_sha) + current = by_sha.get(current_sha) + if current is None: + break + current_sha = current.get("parent_sha256", "") + + return errors + + +def _oracle_session_errors(rows: list[dict[str, str]]) -> list[str]: + """Require a healthy source control before accepting removal outcomes.""" + errors: list[str] = [] + positive_controls = { + (row.get("oracle_session", ""), row.get("group_id", ""), row.get("target_provider", "")) + for row in rows + if row.get("oracle_role") == "source_control" and row.get("synthid_outcome") == "detected" + } + for index, row in enumerate(rows, start=2): + if row.get("oracle_role") not in {"candidate", "sham"} or row.get("synthid_outcome") != "not_detected": + continue + key = (row.get("oracle_session", ""), row.get("group_id", ""), row.get("target_provider", "")) + if key not in positive_controls: + errors.append( + f"row {index}: a not_detected {row.get('oracle_role')} requires a detected " + "source_control in the same oracle session, group, and provider" + ) + return errors + + +def audit_manifest(path: Path, *, verify_files: bool = False) -> list[str]: + """Return all manifest errors, including optional on-disk hash checks.""" + rows, errors = _read_rows(path) + if errors: + return errors + for index, row in enumerate(rows, start=2): + errors.extend(_row_errors(row, index)) + errors.extend(_lineage_errors(rows)) + errors.extend(_oracle_session_errors(rows)) + + if verify_files: + root = path.parent + for index, row in enumerate(rows, start=2): + artifact = resolve_artifact_path(root, row.get("artifact_path", "")) + if artifact is None: + errors.append(f"row {index}: artifact_path must be a safe manifest-relative path") + continue + if not artifact.is_file(): + errors.append(f"row {index}: artifact_path does not exist: {row.get('artifact_path', '')}") + continue + if _file_sha256(artifact) != row.get("artifact_sha256"): + errors.append(f"row {index}: artifact_sha256 does not match the file") + try: + pixel_sha, width, height = _pixel_sha256(artifact) + except Exception as exc: # Pillow intentionally accepts many user-controlled formats. + errors.append(f"row {index}: could not decode artifact: {exc}") + continue + if pixel_sha != row.get("pixel_sha256"): + errors.append(f"row {index}: pixel_sha256 does not match decoded RGB pixels") + if str(width) != row.get("width") or str(height) != row.get("height"): + errors.append(f"row {index}: dimensions do not match decoded pixels") + + return errors + + +@click.command() +@click.argument("manifest", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--verify-files", is_flag=True, help="Verify artifact bytes, decoded pixels, and dimensions.") +def main(manifest: Path, verify_files: bool) -> None: + """Audit MANIFEST for evidence, lineage, and split integrity.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + errors = audit_manifest(manifest, verify_files=verify_files) + if errors: + for error in errors: + log.error("ERROR: %s", error) + raise click.ClickException(f"manifest audit failed with {len(errors)} error(s)") + log.info("Manifest audit passed: %s", manifest) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_routed_detector.py b/scripts/synthid_routed_detector.py new file mode 100644 index 0000000..02081cb --- /dev/null +++ b/scripts/synthid_routed_detector.py @@ -0,0 +1,69 @@ +"""Score images and apply the conservative SynthID expert-bank router.""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict +from pathlib import Path +from typing import TypedDict + +import click +from synthid_conformal_cascade import ExpertObservation +from synthid_routed_expert_bank import classify_routed +from synthid_runtime_expert_scores import ExpertScore, score_path + +log = logging.getLogger(__name__) + + +class RoutedImage(TypedDict): + """One scored image and its conservative routed result.""" + + id: str + path: str + width: int + height: int + observations: list[ExpertScore] + result: dict[str, object] + + +def detect_path(path: Path) -> RoutedImage: + """Score and conservatively route one image PATH.""" + scored = score_path(path) + observations = tuple( + ExpertObservation( + name=observation["name"], + supported=observation["supported"], + score=observation["score"], + ) + for observation in scored["observations"] + ) + return {**scored, "result": asdict(classify_routed(observations))} + + +@click.command() +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def main(images: tuple[Path, ...], report_out: Path) -> None: + """Score and route IMAGES through the conservative pixel expert bank.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + records = [detect_path(path) for path in images] + detected = sum(record["result"].get("verdict") == "detected" for record in records) + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text( + json.dumps( + { + "schema_version": 1, + "counts": {"detected": detected, "abstain": len(records) - detected}, + "records": records, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d routed image verdicts: %s", len(records), report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_routed_expert_bank.py b/scripts/synthid_routed_expert_bank.py new file mode 100644 index 0000000..d7c4f2a --- /dev/null +++ b/scripts/synthid_routed_expert_bank.py @@ -0,0 +1,151 @@ +"""Route SynthID pixel experts without an unsafe union of overlapping positives. + +The registered expert owns its measured scale-search range and the large expert +owns its separately challenged native large-image range. A fixed-only crossing +remains auditable evidence but cannot produce a bank-level detection. The bank +never claims absence because encoder-version coverage is incomplete. +""" + +from __future__ import annotations + +import json +import logging +import sys +from dataclasses import asdict, dataclass +from pathlib import Path +from typing import Literal + +import click + +PROJECT_ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(PROJECT_ROOT / "scripts")) +sys.path.insert(0, str(PROJECT_ROOT / "src")) + +from synthid_conformal_cascade import ( # noqa: E402 + ExpertObservation, + load_observation_records, +) +from synthid_research_manifest import artifact_sha256 # noqa: E402 +from synthid_runtime import synthid_detector # noqa: E402 + +log = logging.getLogger(__name__) + +RoutedVerdict = Literal["detected", "abstain"] + + +@dataclass(frozen=True) +class RoutedBankResult: + """One conservative bank-level decision with every expert score retained.""" + + verdict: RoutedVerdict + reason: str + selected_expert: str | None + fixed_supported: bool + fixed_score: float | None + registered_supported: bool + registered_score: float | None + large_supported: bool + large_score: float | None + + +def classify_routed(observations: tuple[ExpertObservation, ...]) -> RoutedBankResult: + """Route explicit fixed, registered, and large observations without an OR rule.""" + by_name = {observation.name: observation for observation in observations} + if len(by_name) != len(observations): + raise ValueError("observation expert names must be unique") + expected = { + synthid_detector.DETECTOR_ID, + synthid_detector.REGISTERED_DETECTOR_ID, + synthid_detector.LARGE_DETECTOR_ID, + } + if by_name.keys() != expected: + missing = sorted(expected - by_name.keys()) + unknown = sorted(by_name.keys() - expected) + raise ValueError(f"observations must cover the routed bank; missing={missing}, unknown={unknown}") + + fixed = by_name[synthid_detector.DETECTOR_ID] + registered = by_name[synthid_detector.REGISTERED_DETECTOR_ID] + large = by_name[synthid_detector.LARGE_DETECTOR_ID] + if large.supported: + if large.score is None: + raise RuntimeError("validated large observation lost its score") + if large.score >= synthid_detector.LARGE_THRESHOLD: + verdict: RoutedVerdict = "detected" + reason = "large_threshold_crossed" + selected_expert: str | None = large.name + else: + verdict = "abstain" + reason = "large_below_threshold" + selected_expert = None + elif registered.supported: + if registered.score is None: + raise RuntimeError("validated registered observation lost its score") + if registered.score >= synthid_detector.REGISTERED_THRESHOLD: + verdict = "detected" + reason = "registered_threshold_crossed" + selected_expert = registered.name + else: + verdict = "abstain" + reason = ( + "fixed_only_ambiguous" + if fixed.supported and fixed.score is not None and fixed.score >= synthid_detector.TILE_THRESHOLD + else "registered_below_threshold" + ) + selected_expert = None + elif fixed.supported: + verdict = "abstain" + reason = ( + "fixed_only_geometry_uncalibrated" + if fixed.score is not None and fixed.score >= synthid_detector.TILE_THRESHOLD + else "registered_unsupported" + ) + selected_expert = None + else: + verdict = "abstain" + reason = "unsupported" + selected_expert = None + + return RoutedBankResult( + verdict=verdict, + reason=reason, + selected_expert=selected_expert, + fixed_supported=fixed.supported, + fixed_score=fixed.score, + registered_supported=registered.supported, + registered_score=registered.score, + large_supported=large.supported, + large_score=large.score, + ) + + +@click.command() +@click.argument("observation_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def main(observation_path: Path, report_out: Path) -> None: + """Route a three-expert pixel score manifest from OBSERVATION_PATH.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + counts: dict[RoutedVerdict, int] = {"detected": 0, "abstain": 0} + rows: list[dict[str, object]] = [] + for record_id, observations in load_observation_records(observation_path): + result = classify_routed(observations) + counts[result.verdict] += 1 + rows.append({"id": record_id, "result": asdict(result)}) + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text( + json.dumps( + { + "schema_version": 1, + "observation_sha256": artifact_sha256(observation_path), + "counts": counts, + "records": rows, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d routed expert-bank verdicts: %s", len(rows), report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_runtime/__init__.py b/scripts/synthid_runtime/__init__.py new file mode 100644 index 0000000..5b14168 --- /dev/null +++ b/scripts/synthid_runtime/__init__.py @@ -0,0 +1,5 @@ +"""Research-only periodic lattice detector, not a package export.""" + +from synthid_runtime.synthid_detector import SynthIDDetection, detect_synthid, is_available + +__all__ = ["SynthIDDetection", "detect_synthid", "is_available"] diff --git a/scripts/synthid_runtime/_synthid_confirmation.py b/scripts/synthid_runtime/_synthid_confirmation.py new file mode 100644 index 0000000..a0e159b --- /dev/null +++ b/scripts/synthid_runtime/_synthid_confirmation.py @@ -0,0 +1,288 @@ +"""Independent split-patch confirmation for the registered SynthID carrier.""" + +# The optional numeric libraries do not provide complete types for this path. +# pyright: reportMissingTypeStubs=false, reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false + +from __future__ import annotations + +import math +from dataclasses import dataclass +from typing import TYPE_CHECKING, Any + +import cv2 +import numpy as np + +from synthid_runtime.synthid_detector import fold_residual_template, unit_tile + +if TYPE_CHECKING: + from numpy.typing import NDArray + +MIN_PERIOD = 10.0 +MIN_COHERENCE = 0.30 +MIN_AMPLITUDE = 0.0 +H5_PERIOD = (18.0, 18.6) +H5_MIN = 0.13 +STRONG_COHERENCE_PERIOD = (18.6, 20.0) +STRONG_COHERENCE_MIN = 0.40 +WEAK_H5_PERIOD = (20.0, 22.0) +WEAK_H5_MIN = 0.02 +PATCH_SIZE = 256 +GRID_SIZE = 4 +HARMONIC_COUNT = 16 + + +@dataclass(frozen=True) +class RegisteredConfirmationComponents: + """Auditable split-patch confirmation components for one fixed period.""" + + period: float + joint_coherence: float + joint_amplitude: float + unknown_codeword_fixed_confirmation: float + selection_patches: int + confirmation_patches: int + + @property + def passes(self) -> bool: + """Whether every frozen period-aware confirmation gate passes.""" + return registered_confirmation_passes( + self.period, + self.joint_coherence, + self.joint_amplitude, + self.unknown_codeword_fixed_confirmation, + ) + + +def registered_confirmation_passes( + period: float, + joint_coherence: float, + joint_amplitude: float, + unknown_codeword_fixed_confirmation: float, +) -> bool: + """Apply the single frozen registered-carrier confirmation rule.""" + if period < MIN_PERIOD: + return False + if joint_coherence < MIN_COHERENCE or joint_amplitude < MIN_AMPLITUDE: + return False + if H5_PERIOD[0] <= period < H5_PERIOD[1]: + return unknown_codeword_fixed_confirmation >= H5_MIN + if STRONG_COHERENCE_PERIOD[0] <= period < STRONG_COHERENCE_PERIOD[1]: + return joint_coherence >= STRONG_COHERENCE_MIN + if WEAK_H5_PERIOD[0] <= period < WEAK_H5_PERIOD[1]: + return unknown_codeword_fixed_confirmation >= WEAK_H5_MIN + return True + + +def _opponent_channels(values: NDArray[Any]) -> NDArray[Any]: + red = values[:, :, 0] + green = values[:, :, 1] + blue = values[:, :, 2] + return np.stack((green, red - green, blue - 0.5 * (red + green)), axis=2) + + +def _template_harmonics(template: NDArray[Any]) -> tuple[NDArray[Any], NDArray[Any]]: + opponent = _opponent_channels(np.asarray(template, dtype=np.float64)) + spectrum = np.fft.fft2(opponent, axes=(0, 1)) + height, width = template.shape[:2] + candidates: list[tuple[float, int, int]] = [] + for row in range(height): + signed_row = row if row <= height // 2 else row - height + for column in range(width): + signed_column = column if column <= width // 2 else column - width + if signed_row < 0 or (signed_row == 0 and signed_column <= 0): + continue + power = float(np.sum(np.abs(spectrum[row, column]) ** 2)) + candidates.append((power, signed_row, signed_column)) + candidates.sort(reverse=True) + selected = candidates[:HARMONIC_COUNT] + harmonics = np.asarray([(row, column) for _power, row, column in selected], dtype=np.float64) + coefficients = np.asarray([spectrum[int(row) % height, int(column) % width] for row, column in harmonics]) + weights = np.abs(coefficients) + weight_sum = float(np.sum(weights)) + if weight_sum <= 0.0: + raise ValueError("template has no nonzero periodic harmonics") + return harmonics, weights / weight_sum + + +def _patch_origins(height: int, width: int) -> list[tuple[int, int, int]]: + if height < PATCH_SIZE or width < PATCH_SIZE: + raise ValueError("registered confirmation needs both image sides to be at least 256 pixels") + y_values = np.linspace(0, height - PATCH_SIZE, min(GRID_SIZE, height // PATCH_SIZE), dtype=np.int64) + x_values = np.linspace(0, width - PATCH_SIZE, min(GRID_SIZE, width // PATCH_SIZE), dtype=np.int64) + origins = [ + (int(y), int(x), (y_index + x_index) % 2) + for y_index, y in enumerate(np.unique(y_values)) + for x_index, x in enumerate(np.unique(x_values)) + ] + if {group for _y, _x, group in origins} != {0, 1}: + raise ValueError("registered confirmation needs two independent patch groups") + return origins + + +def _bilinear_sample(spectrum: NDArray[Any], y: NDArray[Any], x: NDArray[Any]) -> NDArray[Any]: + height, width = spectrum.shape + y_floor = np.floor(y) + x_floor = np.floor(x) + y0 = y_floor.astype(np.int64) % height + x0 = x_floor.astype(np.int64) % width + y1 = (y0 + 1) % height + x1 = (x0 + 1) % width + dy = y - y_floor + dx = x - x_floor + return ( + spectrum[y0, x0] * (1.0 - dy) * (1.0 - dx) + + spectrum[y1, x0] * dy * (1.0 - dx) + + spectrum[y0, x1] * (1.0 - dy) * dx + + spectrum[y1, x1] * dy * dx + ) + + +def _patch_unit_values( + pixels: NDArray[Any], + origin_y: int, + origin_x: int, + period: float, + harmonics: NDArray[Any], + denoise_sigma: float, +) -> NDArray[Any]: + patch = np.asarray( + pixels[origin_y : origin_y + PATCH_SIZE, origin_x : origin_x + PATCH_SIZE], + dtype=np.float32, + ) + channels = _opponent_channels(patch) + window_1d = np.hanning(PATCH_SIZE).astype(np.float32) + window = window_1d[:, None] * window_1d[None, :] + frequencies_y = harmonics[:, 0] / period + frequencies_x = harmonics[:, 1] / period + sample_y = frequencies_y * PATCH_SIZE + sample_x = frequencies_x * PATCH_SIZE + sampled = np.empty((len(harmonics), 3), dtype=np.complex128) + for channel in range(3): + residual = channels[:, :, channel] + residual -= cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=denoise_sigma, + sigmaY=denoise_sigma, + borderType=cv2.BORDER_REFLECT_101, + ) + sampled[:, channel] = _bilinear_sample(np.fft.fft2(residual * window), sample_y, sample_x) + sampled *= np.exp(-2j * math.pi * (frequencies_y * origin_y + frequencies_x * origin_x))[:, None] + magnitudes = np.abs(sampled) + return np.divide(sampled, magnitudes, out=np.zeros_like(sampled), where=magnitudes > 1e-12) + + +def _coherence(values: list[NDArray[Any]], weights: NDArray[Any]) -> float: + coherence = np.abs(np.mean(np.stack(values), axis=0)) + return float(np.sum(coherence * weights)) + + +def _unknown_codeword_fixed_confirmation( + selection_values: list[NDArray[Any]], + confirmation_values: list[NDArray[Any]], + weights: NDArray[Any], +) -> float: + cross_codeword = np.mean(np.stack(confirmation_values), axis=0) * np.conj( + np.mean(np.stack(selection_values), axis=0) + ) + confirmation_mask = np.arange(len(weights)) % 2 == 1 + masked_weights = weights[confirmation_mask] + return float(np.abs(np.sum(cross_codeword[confirmation_mask] * masked_weights)) / np.sum(masked_weights)) + + +def _canonical_pixels(pixels: NDArray[Any], template: NDArray[Any], period: float) -> NDArray[Any]: + width = max(template.shape[1], round(pixels.shape[1] * template.shape[1] / period)) + height = max(template.shape[0], round(pixels.shape[0] * template.shape[0] / period)) + if (height, width) == pixels.shape[:2]: + return pixels + interpolation = cv2.INTER_AREA if width < pixels.shape[1] else cv2.INTER_CUBIC + return np.asarray(cv2.resize(pixels, (width, height), interpolation=interpolation)) + + +def _cyclic_correlations(template: NDArray[Any], tile: NDArray[Any]) -> NDArray[Any]: + template_spectrum = np.fft.fft2(template, axes=(0, 1)) + tile_spectrum = np.fft.fft2(tile, axes=(0, 1)) + return np.fft.ifft2(np.sum(template_spectrum * np.conj(tile_spectrum), axis=2)).real + + +def _joint_amplitude( + pixels: NDArray[Any], + template: NDArray[Any], + period: float, + denoise_sigma: float, +) -> tuple[float, int, int]: + canonical = _canonical_pixels(pixels, template, period) + tile_height, tile_width = template.shape[:2] + grouped_units: dict[int, list[NDArray[Any]]] = {0: [], 1: []} + origins = _patch_origins(*canonical.shape[:2]) + for origin_y, origin_x, group in origins: + aligned_y = (origin_y // tile_height) * tile_height + aligned_x = (origin_x // tile_width) * tile_width + folded = fold_residual_template( + canonical[aligned_y : aligned_y + PATCH_SIZE, aligned_x : aligned_x + PATCH_SIZE], + tile_height=tile_height, + tile_width=tile_width, + denoise_sigma=denoise_sigma, + ) + unit, _norm = unit_tile(folded) + grouped_units[group].append(unit) + selection_tile, _selection_norm = unit_tile(np.mean(grouped_units[0], axis=0)) + confirmation_tile, _confirmation_norm = unit_tile(np.mean(grouped_units[1], axis=0)) + selection_correlations = _cyclic_correlations(template, selection_tile) + confirmation_correlations = _cyclic_correlations(template, confirmation_tile) + shift_y, shift_x = np.unravel_index(int(np.argmax(selection_correlations)), selection_correlations.shape) + return ( + min( + float(selection_correlations[shift_y, shift_x]), + float(confirmation_correlations[shift_y, shift_x]), + ), + len(grouped_units[0]), + len(grouped_units[1]), + ) + + +def registered_confirmation_components( + pixels: NDArray[Any], + template: NDArray[Any], + period: float, + denoise_sigma: float, +) -> RegisteredConfirmationComponents: + """Measure the frozen split-patch gates at one registered carrier period.""" + if pixels.ndim != 3 or pixels.shape[2] != 3: + raise ValueError("pixels must have shape (height, width, 3)") + if not math.isfinite(period) or period <= 0.0: + raise ValueError("registered period must be finite and positive") + harmonics, weights = _template_harmonics(template) + grouped_values: dict[int, list[NDArray[Any]]] = {0: [], 1: []} + for origin_y, origin_x, group in _patch_origins(*pixels.shape[:2]): + grouped_values[group].append( + _patch_unit_values( + pixels, + origin_y, + origin_x, + period, + harmonics, + denoise_sigma, + ) + ) + amplitude, selection_patches, confirmation_patches = _joint_amplitude( + pixels, + template, + period, + denoise_sigma, + ) + return RegisteredConfirmationComponents( + period=period, + joint_coherence=min( + _coherence(grouped_values[0], weights), + _coherence(grouped_values[1], weights), + ), + joint_amplitude=amplitude, + unknown_codeword_fixed_confirmation=_unknown_codeword_fixed_confirmation( + grouped_values[0], + grouped_values[1], + weights, + ), + selection_patches=selection_patches, + confirmation_patches=confirmation_patches, + ) diff --git a/scripts/synthid_runtime/_synthid_registered.py b/scripts/synthid_runtime/_synthid_registered.py new file mode 100644 index 0000000..1d02a12 --- /dev/null +++ b/scripts/synthid_runtime/_synthid_registered.py @@ -0,0 +1,664 @@ +"""Opt-in scale registration for the measured periodic SynthID carrier.""" + +# The optional numeric libraries do not provide complete types for this path. +# pyright: reportMissingTypeStubs=false, reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false + +from __future__ import annotations + +import itertools +import math +from dataclasses import dataclass +from typing import TYPE_CHECKING, Any + +import cv2 +import numpy as np + +from synthid_runtime._synthid_confirmation import ( + RegisteredConfirmationComponents, + registered_confirmation_components, +) +from synthid_runtime.synthid_detector import folded_template_score + +if TYPE_CHECKING: + from numpy.typing import NDArray + +_PYRAMID_SCALES = (0.75, 1.0, 1.25) +_SEARCH_PERIODS = np.linspace(5.0, 32.0, 541, dtype=np.float64) +_CANONICAL_PERIODS = np.linspace(7.5, 24.5, 1701, dtype=np.float64) +_OPPONENT_SEARCH_PERIODS = np.linspace(7.5, 14.5, 141, dtype=np.float64) +_FINE_OPPONENT_COARSE_PERIODS = np.linspace(7.5, 9.0, 31, dtype=np.float64) +_FINE_OPPONENT_PROBE_SIZE = 384 +_PERIOD_THRESHOLDS = ( + (7.5, 8.5, 0.3770629524888979), + (8.5, 10.0, 0.25174716660523494), + (10.0, 12.0, 0.284692023502354), + (12.0, 14.0, 0.19794247706938645), + (14.0, 16.0, 0.33930082812296375), + (16.0, 18.0, 0.28915284982686323), + (18.0, 20.0, 0.22885510746595789), + (20.0, 22.0, 0.24570317032768269), + (22.0, 24.5, 0.3142958338390489), +) +REGISTERED_HIGH_BAND_THRESHOLD = 0.075 +OPPONENT_REGISTERED_MIN_PERIOD = 7.9 +OPPONENT_REGISTERED_MAX_PERIOD = 12.0 +OPPONENT_REGISTERED_CODEC_VETO_MAX_PERIOD = 8.1 +OPPONENT_REGISTERED_MAX_P8_EDGE_RATIO = 1.05 +FINE_OPPONENT_REGISTERED_MIN_PERIOD = 7.5 +FINE_OPPONENT_REGISTERED_MAX_PERIOD = 9.0 +OPPONENT_REGISTERED_FIXED_MIN = 0.16 +OPPONENT_REGISTERED_RED_GREEN_MIN = 0.60 +OPPONENT_REGISTERED_BLUE_YELLOW_MIN = 0.55 + + +@dataclass(frozen=True) +class RegisteredComponents: + """Calibrated components of one scale-registered decision.""" + + raw_score: float + amplitude_threshold: float + selected_period: float + spectral_period: float + high_band_score: float + confirmation: RegisteredConfirmationComponents | None = None + + @property + def base_decision_score(self) -> float: + """Return the unchanged registered-v2 decision statistic.""" + if self.selected_period != self.spectral_period: + return 0.0 + return min( + self.raw_score / self.amplitude_threshold, + self.high_band_score / REGISTERED_HIGH_BAND_THRESHOLD, + ) + + @property + def decision_score(self) -> float: + """Return the base score only after split confirmation passes.""" + base_score = self.base_decision_score + if base_score < 1.0: + return base_score + if self.confirmation is None or not self.confirmation.passes: + return 0.0 + return base_score + + +@dataclass(frozen=True) +class OpponentRegisteredComponents: + """Auditable margins for the bounded opponent-color fallback.""" + + selected_period: float + spectral_period: float + spectral_score: float + fixed_score: float + red_green_spatial: float + blue_yellow_spatial: float + candidate_count: int + red_green_p8_edge_ratio: float | None + blue_yellow_p8_edge_ratio: float | None + + @property + def base_decision_score(self) -> float: + """Return the minimum normalized color-carrier margin.""" + return min( + self.fixed_score / OPPONENT_REGISTERED_FIXED_MIN, + self.red_green_spatial / OPPONENT_REGISTERED_RED_GREEN_MIN, + self.blue_yellow_spatial / OPPONENT_REGISTERED_BLUE_YELLOW_MIN, + ) + + @property + def decision_score(self) -> float: + """Return the margin only inside the independently challenged period band.""" + if not OPPONENT_REGISTERED_MIN_PERIOD <= self.selected_period <= OPPONENT_REGISTERED_MAX_PERIOD: + return 0.0 + if self.selected_period <= OPPONENT_REGISTERED_CODEC_VETO_MAX_PERIOD: + ratios = (self.red_green_p8_edge_ratio, self.blue_yellow_p8_edge_ratio) + if any(value is None or value > OPPONENT_REGISTERED_MAX_P8_EDGE_RATIO for value in ratios): + return 0.0 + return self.base_decision_score + + @property + def fine_decision_score(self) -> float: + """Return the margin for the separately calibrated fine-period expert.""" + if not FINE_OPPONENT_REGISTERED_MIN_PERIOD <= self.selected_period <= FINE_OPPONENT_REGISTERED_MAX_PERIOD: + return 0.0 + if self.selected_period <= OPPONENT_REGISTERED_CODEC_VETO_MAX_PERIOD: + ratios = (self.red_green_p8_edge_ratio, self.blue_yellow_p8_edge_ratio) + if any(value is None or value > OPPONENT_REGISTERED_MAX_P8_EDGE_RATIO for value in ratios): + return 0.0 + return self.base_decision_score + + +def _resize(pixels: NDArray[Any], width: int, height: int) -> NDArray[Any]: + interpolation = cv2.INTER_AREA if width < pixels.shape[1] else cv2.INTER_CUBIC + return np.asarray(cv2.resize(pixels, (width, height), interpolation=interpolation)) + + +def _template_frequency_features( + template: NDArray[Any], +) -> tuple[NDArray[Any], NDArray[Any], NDArray[Any]]: + spectrum = np.fft.fft2(template, axes=(0, 1)) + power = np.sum(np.abs(spectrum) ** 2, axis=2) + power[0, 0] = 0.0 + indices = np.argsort(power.ravel())[::-1][:30] + rows, columns = np.unravel_index(indices, power.shape) + height, width = template.shape[:2] + signed_rows = np.where(rows <= height // 2, rows, rows - height) + signed_columns = np.where(columns <= width // 2, columns, columns - width) + harmonics = np.column_stack((signed_rows, signed_columns)).astype(np.float64) + return harmonics, spectrum[rows, columns], spectrum + + +def _bilinear_sample( + spectrum: NDArray[Any], + y: NDArray[Any], + x: NDArray[Any], +) -> NDArray[Any]: + height, width = spectrum.shape + y_floor = np.floor(y) + x_floor = np.floor(x) + y0 = y_floor.astype(np.int64) % height + x0 = x_floor.astype(np.int64) % width + y1 = (y0 + 1) % height + x1 = (x0 + 1) % width + dy = y - y_floor + dx = x - x_floor + return ( + spectrum[y0, x0] * (1.0 - dy) * (1.0 - dx) + + spectrum[y1, x0] * dy * (1.0 - dx) + + spectrum[y0, x1] * (1.0 - dy) * dx + + spectrum[y1, x1] * dy * dx + ) + + +def _spectral_curve( + pixels: NDArray[Any], + periods: NDArray[Any], + harmonics: NDArray[Any], + coefficients: NDArray[Any], +) -> NDArray[Any]: + height, width = pixels.shape[:2] + y = (periods[:, None] ** -1) * harmonics[None, :, 0] * height + x = (periods[:, None] ** -1) * harmonics[None, :, 1] * width + sampled = np.empty((len(periods), len(harmonics), 3), dtype=np.complex128) + for channel in range(3): + residual = pixels[:, :, channel].astype(np.float32) + residual -= cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=1.0, + sigmaY=1.0, + borderType=cv2.BORDER_REFLECT_101, + ) + spectrum = np.fft.fft2(residual) + sampled[:, :, channel] = _bilinear_sample(spectrum, y % height, x % width) + numerator = np.real(np.sum(np.conj(coefficients)[None, :, :] * sampled, axis=(1, 2))) + denominator = np.linalg.norm(coefficients) * np.linalg.norm(sampled, axis=(1, 2)) + return np.divide( + numerator, + denominator, + out=np.zeros_like(numerator), + where=denominator > 0.0, + ) + + +def _period_candidates( + periods: NDArray[Any], + scores: NDArray[Any], + count: int = 3, +) -> list[float]: + candidates: list[float] = [] + for index in np.argsort(scores)[::-1]: + period = float(periods[index]) + if any(abs(period - existing_period) < 0.25 for existing_period in candidates): + continue + candidates.append(period) + if len(candidates) == count: + break + return candidates + + +def _period_threshold(period: float) -> float: + for index, (lower, upper, threshold) in enumerate(_PERIOD_THRESHOLDS): + if lower <= period < upper or (index == len(_PERIOD_THRESHOLDS) - 1 and period == upper): + return threshold + raise ValueError(f"registered period {period} is outside the calibrated range") + + +def _high_band_score( + folded: NDArray[Any], + template_spectrum: NDArray[Any], +) -> float: + folded_spectrum = np.fft.fft2(folded, axes=(0, 1)) + tile_height, tile_width = template_spectrum.shape[:2] + y_coordinates = np.minimum(np.arange(tile_height), tile_height - np.arange(tile_height)) + x_coordinates = np.minimum(np.arange(tile_width), tile_width - np.arange(tile_width)) + radius = np.sqrt(y_coordinates[:, None] ** 2 + x_coordinates[None, :] ** 2) + correlations = [] + for lower, upper in ((4.5, 6.5), (6.5, 12.0)): + mask = (radius >= lower) & (radius < upper) + selected_folded = folded_spectrum[mask] + selected_template = template_spectrum[mask] + denominator = np.linalg.norm(selected_folded) * np.linalg.norm(selected_template) + correlations.append( + float(np.real(np.vdot(selected_template, selected_folded)) / denominator) if denominator > 0.0 else 0.0 + ) + return min(correlations) + + +def _best_canonical( + pixels: NDArray[Any], + periods: list[float], + template: NDArray[Any], + sigma: float, +) -> tuple[float, NDArray[Any], NDArray[Any], float]: + best_score = -math.inf + best_canonical: NDArray[Any] | None = None + best_folded: NDArray[Any] | None = None + best_period: float | None = None + for period in periods: + predicted_width = round(pixels.shape[1] * template.shape[1] / period) + for delta in range(-4, 5): + width = predicted_width + delta + height = round(pixels.shape[0] * width / pixels.shape[1]) + canonical = _resize(pixels, width, height) + score, folded = folded_template_score(canonical, template, sigma) + if score > best_score: + best_score = score + best_canonical = canonical + best_folded = folded + best_period = period + if best_canonical is None or best_folded is None or best_period is None: + raise RuntimeError("scale registration produced no canonical view") + return float(best_score), best_canonical, best_folded, best_period + + +def _quadrant_median( + canonical: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> float: + tile_height, tile_width = template.shape[:2] + split_y = max(tile_height, (canonical.shape[0] // (2 * tile_height)) * tile_height) + split_x = max(tile_width, (canonical.shape[1] // (2 * tile_width)) * tile_width) + scores = [] + for region in ( + canonical[:split_y, :split_x], + canonical[:split_y, split_x:], + canonical[split_y:, :split_x], + canonical[split_y:, split_x:], + ): + score, _folded = folded_template_score(region, template, sigma) + scores.append(score) + return float(np.median(scores)) + + +def _pyramid_locked_mean( + pixels: NDArray[Any], + harmonics: NDArray[Any], + coefficients: NDArray[Any], + base_curve: NDArray[Any], +) -> float: + curves = [] + candidates = [] + for scale in _PYRAMID_SCALES: + if scale == 1.0: + curve = base_curve + else: + level = _resize( + pixels, + max(16, round(pixels.shape[1] * scale)), + max(16, round(pixels.shape[0] * scale)), + ) + curve = _spectral_curve(level, _SEARCH_PERIODS, harmonics, coefficients) + curves.append(curve) + candidates.append(_period_candidates(_SEARCH_PERIODS, curve)) + combinations = itertools.product(*candidates) + + def spread(combination: tuple[float, ...]) -> float: + normalized_periods = [ + candidate / scale + for candidate, scale in zip( + combination, + _PYRAMID_SCALES, + strict=True, + ) + ] + return float(np.std(np.log(normalized_periods))) + + best = min( + combinations, + key=spread, + ) + base_period = float(np.median([candidate / scale for candidate, scale in zip(best, _PYRAMID_SCALES, strict=True)])) + locked = [ + float(np.interp(base_period * scale, _SEARCH_PERIODS, curve)) + for curve, scale in zip(curves, _PYRAMID_SCALES, strict=True) + ] + return float(np.mean(locked)) + + +def _opponent_pair(values: NDArray[Any]) -> NDArray[Any]: + """Return Red-minus-Green and Blue-minus-Yellow color planes.""" + red = values[:, :, 0] + green = values[:, :, 1] + blue = values[:, :, 2] + return np.stack((red - green, blue - 0.5 * (red + green)), axis=2) + + +def _opponent_period_curve( + pixels: NDArray[Any], + template: NDArray[Any], + periods: NDArray[Any] = _OPPONENT_SEARCH_PERIODS, +) -> NDArray[Any]: + """Return signed opponent-color coherence across the frozen search grid.""" + template_opponent = _opponent_pair(np.asarray(template, dtype=np.float64)) + template_spectrum = np.fft.fft2(template_opponent, axes=(0, 1)) + power = np.sum(np.abs(template_spectrum) ** 2, axis=2) + power[0, 0] = 0.0 + indices = np.argsort(power.ravel())[::-1][:30] + rows, columns = np.unravel_index(indices, power.shape) + height, width = template.shape[:2] + signed_rows = np.where(rows <= height // 2, rows, rows - height) + signed_columns = np.where(columns <= width // 2, columns, columns - width) + harmonics = np.column_stack((signed_rows, signed_columns)).astype(np.float64) + coefficients = template_spectrum[rows, columns] + + image_height, image_width = pixels.shape[:2] + sample_y = periods[:, None] ** -1 * harmonics[None, :, 0] * image_height + sample_x = periods[:, None] ** -1 * harmonics[None, :, 1] * image_width + sampled = np.empty((len(periods), len(harmonics), 2), dtype=np.complex128) + image_opponent = _opponent_pair(np.asarray(pixels, dtype=np.float32)) + for channel in range(2): + residual = image_opponent[:, :, channel] + residual -= cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=1.0, + sigmaY=1.0, + borderType=cv2.BORDER_REFLECT_101, + ) + sampled[:, :, channel] = _bilinear_sample( + np.fft.fft2(residual), + sample_y % image_height, + sample_x % image_width, + ) + numerator = np.real(np.sum(np.conj(coefficients)[None, :, :] * sampled, axis=(1, 2))) + denominator = np.linalg.norm(coefficients) * np.linalg.norm(sampled, axis=(1, 2)) + return np.divide(numerator, denominator, out=np.zeros_like(numerator), where=denominator > 0.0) + + +def _opponent_period_candidates(scores: NDArray[Any], count: int = 3) -> list[int]: + """Return separated period indices in descending spectral-score order.""" + candidates: list[int] = [] + for index in np.argsort(scores)[::-1]: + period = float(_OPPONENT_SEARCH_PERIODS[index]) + if any(abs(period - float(_OPPONENT_SEARCH_PERIODS[prior])) < 0.5 for prior in candidates): + continue + candidates.append(int(index)) + if len(candidates) == count: + break + return candidates + + +def _canonical_at_period( + pixels: NDArray[Any], + template: NDArray[Any], + period: float, +) -> NDArray[Any]: + """Resample PIXELS so PERIOD maps to the frozen template period.""" + width = max(template.shape[1], round(pixels.shape[1] * template.shape[1] / period)) + height = max(template.shape[0], round(pixels.shape[0] * template.shape[0] / period)) + if (height, width) == pixels.shape[:2]: + return pixels + return _resize(pixels, width, height) + + +def _correlation(left: NDArray[Any], right: NDArray[Any]) -> float: + """Return the signed real cosine between equal-shaped arrays.""" + denominator = float(np.linalg.norm(left) * np.linalg.norm(right)) + return float(np.real(np.vdot(right, left)) / denominator) if denominator > 0.0 else 0.0 + + +def _period8_edge_ratio(values: NDArray[Any]) -> float: + """Measure native 8-pixel block edges relative to non-block phases.""" + phase_values = np.zeros(8, dtype=np.float64) + for axis in (0, 1): + differences = np.abs(np.diff(values, axis=axis)) + indices = np.arange(differences.shape[axis]) + for phase in range(8): + selected = indices[(indices + 1) % 8 == phase] + phase_values[phase] += 0.5 * float(np.take(differences, selected, axis=axis).mean()) + baseline = float(np.median(phase_values[[1, 2, 3, 5, 6, 7]])) + return float(phase_values[0] / baseline) if baseline > 1e-9 else math.inf + + +def _period8_opponent_edge_ratios(pixels: NDArray[Any]) -> tuple[float, float]: + """Return codec-grid ratios for the two opponent-color planes.""" + opponent = _opponent_pair(np.asarray(pixels, dtype=np.float32)) + return _period8_edge_ratio(opponent[:, :, 0]), _period8_edge_ratio(opponent[:, :, 1]) + + +def _opponent_components_at_period( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, + period: float, + *, + spectral_period: float, + spectral_score: float, + candidate_count: int, + period8_edge_ratios: tuple[float, float] | None = None, +) -> OpponentRegisteredComponents: + """Measure one period without selecting it from the image being scored.""" + canonical = _canonical_at_period(pixels, template, period) + fixed_score, folded = folded_template_score(canonical, template, sigma) + folded_opponent = _opponent_pair(folded) + template_opponent = _opponent_pair(template) + red_green_p8_edge_ratio, blue_yellow_p8_edge_ratio = period8_edge_ratios or (None, None) + return OpponentRegisteredComponents( + selected_period=period, + spectral_period=spectral_period, + spectral_score=spectral_score, + fixed_score=fixed_score, + red_green_spatial=_correlation(folded_opponent[:, :, 0], template_opponent[:, :, 0]), + blue_yellow_spatial=_correlation(folded_opponent[:, :, 1], template_opponent[:, :, 1]), + candidate_count=candidate_count, + red_green_p8_edge_ratio=red_green_p8_edge_ratio, + blue_yellow_p8_edge_ratio=blue_yellow_p8_edge_ratio, + ) + + +def opponent_registered_components( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> OpponentRegisteredComponents: + """Measure the bounded lossless-resize carrier in opponent-color space.""" + curve = _opponent_period_curve(pixels, template) + candidate_indices = _opponent_period_candidates(curve) + observations: list[OpponentRegisteredComponents] = [] + period8_edge_ratios: tuple[float, float] | None = None + for index in candidate_indices: + period = float(_OPPONENT_SEARCH_PERIODS[index]) + if period <= OPPONENT_REGISTERED_CODEC_VETO_MAX_PERIOD and period8_edge_ratios is None: + period8_edge_ratios = _period8_opponent_edge_ratios(pixels) + observations.append( + _opponent_components_at_period( + pixels, + template, + sigma, + period, + spectral_period=float(_OPPONENT_SEARCH_PERIODS[int(np.argmax(curve))]), + spectral_score=float(curve[index]), + candidate_count=len(candidate_indices), + period8_edge_ratios=period8_edge_ratios, + ) + ) + if not observations: + raise RuntimeError("opponent-color registration produced no candidates") + return max(observations, key=lambda observation: observation.base_decision_score) + + +def _fine_opponent_period_groups(curve: NDArray[Any]) -> list[list[float]]: + """Return fine period grids around separated absolute spectral peaks.""" + centers: list[float] = [] + for index in np.argsort(np.abs(curve))[::-1]: + period = float(_FINE_OPPONENT_COARSE_PERIODS[index]) + if any(abs(period - existing) < 0.2 for existing in centers): + continue + centers.append(period) + if len(centers) == 3: + break + return [ + sorted( + { + round(float(period), 2) + for period in np.arange(center - 0.36, center + 0.361, 0.01) + if FINE_OPPONENT_REGISTERED_MIN_PERIOD <= period <= FINE_OPPONENT_REGISTERED_MAX_PERIOD + } + ) + for center in centers + ] + + +def fine_opponent_registered_components( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> OpponentRegisteredComponents: + """Select and score the calibrated fine-period lossless-resize expert.""" + curve = _opponent_period_curve(pixels, template, _FINE_OPPONENT_COARSE_PERIODS) + spectral_index = int(np.argmax(np.abs(curve))) + spectral_period = float(_FINE_OPPONENT_COARSE_PERIODS[spectral_index]) + period_groups = _fine_opponent_period_groups(curve) + probe = pixels[ + : min(_FINE_OPPONENT_PROBE_SIZE, pixels.shape[0]), + : min(_FINE_OPPONENT_PROBE_SIZE, pixels.shape[1]), + ] + candidate_count = sum(len(group) for group in period_groups) + unique_periods = sorted({period for group in period_groups for period in group}) + probe_by_period = { + period: _opponent_components_at_period( + probe, + template, + sigma, + period, + spectral_period=spectral_period, + spectral_score=float(np.interp(period, _FINE_OPPONENT_COARSE_PERIODS, curve)), + candidate_count=candidate_count, + ) + for period in unique_periods + } + probe_groups = [[probe_by_period[period] for period in group] for group in period_groups] + probe_observations = [observation for group in probe_groups for observation in group] + finalist_periods = { + observation.selected_period + for group in probe_groups + for observation in sorted(group, key=lambda value: value.base_decision_score, reverse=True)[:2] + } + finalist_periods.update( + observation.selected_period + for observation in sorted( + probe_observations, + key=lambda value: value.base_decision_score, + reverse=True, + )[:5] + ) + period8_edge_ratios = ( + _period8_opponent_edge_ratios(pixels) + if any(period <= OPPONENT_REGISTERED_CODEC_VETO_MAX_PERIOD for period in finalist_periods) + else None + ) + observations = [ + _opponent_components_at_period( + pixels, + template, + sigma, + period, + spectral_period=spectral_period, + spectral_score=float(np.interp(period, _FINE_OPPONENT_COARSE_PERIODS, curve)), + candidate_count=len(probe_observations), + period8_edge_ratios=period8_edge_ratios, + ) + for period in sorted(finalist_periods) + ] + if not observations: + raise RuntimeError("fine opponent-color registration produced no candidates") + return max(observations, key=lambda observation: observation.base_decision_score) + + +def registered_components( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> RegisteredComponents: + """Measure a carrier after bounded scale registration.""" + harmonics, coefficients, template_spectrum = _template_frequency_features(template) + combined_periods = np.concatenate((_SEARCH_PERIODS, _CANONICAL_PERIODS)) + combined_curve = _spectral_curve(pixels, combined_periods, harmonics, coefficients) + base_curve = combined_curve[: len(_SEARCH_PERIODS)] + canonical_curve = combined_curve[len(_SEARCH_PERIODS) :] + candidates = _period_candidates(_CANONICAL_PERIODS, canonical_curve) + baseline, canonical, folded, selected_period = _best_canonical(pixels, candidates, template, sigma) + quadrant = _quadrant_median(canonical, template, sigma) + pyramid = _pyramid_locked_mean( + pixels, + harmonics, + coefficients, + base_curve, + ) + raw_score = float((baseline + quadrant + pyramid) / 3.0) + components = RegisteredComponents( + raw_score=raw_score, + amplitude_threshold=_period_threshold(selected_period), + selected_period=selected_period, + spectral_period=candidates[0], + high_band_score=_high_band_score(folded, template_spectrum), + ) + if components.base_decision_score < 1.0: + return components + try: + confirmation = registered_confirmation_components( + pixels, + template, + selected_period, + sigma, + ) + except ValueError: + return components + return RegisteredComponents( + raw_score=components.raw_score, + amplitude_threshold=components.amplitude_threshold, + selected_period=components.selected_period, + spectral_period=components.spectral_period, + high_band_score=components.high_band_score, + confirmation=confirmation, + ) + + +def registered_score( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> float: + """Return the calibrated registered decision statistic.""" + return registered_components(pixels, template, sigma).decision_score + + +def opponent_registered_score( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> float: + """Return the bounded opponent-color fallback decision statistic.""" + return opponent_registered_components(pixels, template, sigma).decision_score + + +def fine_opponent_registered_score( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> float: + """Return the separately calibrated fine-period decision statistic.""" + return fine_opponent_registered_components(pixels, template, sigma).fine_decision_score diff --git a/scripts/synthid_runtime/synthid_detector.py b/scripts/synthid_runtime/synthid_detector.py new file mode 100644 index 0000000..579687f --- /dev/null +++ b/scripts/synthid_runtime/synthid_detector.py @@ -0,0 +1,542 @@ +"""Detect the confirmed periodic SynthID image carrier at calibrated image sizes. + +This is a positive-only detector for one measured carrier epoch, not Google's +private payload decoder. A positive result is strong local evidence for the +carrier. An indeterminate result means only that the selected detector did not +find it; image sizes outside that mode's calibrated range are reported separately. + +Research runtime. Not exported by ``remove_ai_watermarks``. Needs numpy and OpenCV. +""" + +# The optional numeric libraries do not provide complete types for this path. +# pyright: reportMissingTypeStubs=false, reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false + +from __future__ import annotations + +from dataclasses import dataclass +from functools import lru_cache +from pathlib import Path +from typing import TYPE_CHECKING, Any, Literal + +if TYPE_CHECKING: + from numpy.typing import NDArray + +SynthIDDetectionStatus = Literal["detected", "indeterminate", "unsupported"] + +DETECTOR_ID = "synthid-periodic-tile-v2" +REGISTERED_DETECTOR_ID = "synthid-periodic-tile-registered-v3" +OPPONENT_REGISTERED_DETECTOR_ID = "synthid-periodic-tile-opponent-registered-v1" +FINE_OPPONENT_REGISTERED_DETECTOR_ID = "synthid-periodic-tile-opponent-fine-registered-v1" +LARGE_DETECTOR_ID = "synthid-periodic-tile-large-v1" +MODEL_FILENAME = "synthid_periodic_tile_2048_v1.npz" +# The template remains frozen at this model geometry. Runtime images are never +# resized. The supported pixel-count interval is the separately challenged domain: +# below it too few repetitions make the positive-only statistic unreliable, and +# above it resource use and specificity have not been calibrated. +MODEL_WIDTH = 2048 +MODEL_HEIGHT = 2048 +MIN_SUPPORTED_PIXELS = 1_000_000 +MAX_SUPPORTED_PIXELS = 18_000_000 +TILE_THRESHOLD = 0.17357069773071196 +REGISTERED_MIN_SUPPORTED_PIXELS = 250_000 +REGISTERED_MAX_SUPPORTED_PIXELS = 10_000_000 +# Registered-v3 can confirm a positive only when both disjoint checkerboard +# groups contain a complete frozen 256-pixel patch. Narrower geometries need a +# separately calibrated adaptive-patch expert and must not masquerade as misses. +REGISTERED_MIN_SIDE = 256 +# The registered score preserves the minimum normalized v2 margin only after +# independent split-patch phase, amplitude, and held-out codeword confirmation. +REGISTERED_THRESHOLD = 1.0 +# The opponent-color fallback is a narrower precision-first route for lossless +# scale changes. Smaller rasters retained a natural period-10 false positive. +OPPONENT_REGISTERED_THRESHOLD = 1.0 +OPPONENT_REGISTERED_MIN_PIXELS = 1_000_000 +OPPONENT_REGISTERED_MIN_SIDE = 768 +# Fine-period registration is separately frozen for the dense 0.47-0.55 +# lossless-resize challenge. Its more expensive selector is bounded to the +# geometry range covered by the locked and reserve negative sets. +FINE_OPPONENT_REGISTERED_THRESHOLD = 1.05 +FINE_OPPONENT_REGISTERED_MIN_PIXELS = 1_000_000 +FINE_OPPONENT_REGISTERED_MAX_PIXELS = 5_000_000 +FINE_OPPONENT_REGISTERED_MIN_SIDE = 768 +# The large-image score combines all-window fixed and spatial opponent gates +# with an any-window signed opponent mid-band gate. The one vulnerable portrait +# geometry has an additional Green mid-band upper gate. +LARGE_THRESHOLD = 1.0 +LARGE_MIN_PIXELS = 10_000_000 +LARGE_MAX_PIXELS = 18_000_000 +LARGE_WINDOW = 2_048 +LARGE_PHASE = 16 +LARGE_FIXED_SCORE_MIN = 0.14 +LARGE_RED_GREEN_SPATIAL_MIN = 0.90 +LARGE_BLUE_YELLOW_SPATIAL_MIN = 0.70 +LARGE_BLUE_YELLOW_MID_BAND_MAX = -0.15 +LARGE_PORTRAIT_GEOMETRY = (3_072, 5_504) +LARGE_PORTRAIT_GREEN_MID_BAND_MAX = 0.06 +INSTALL_HINT = "install numpy and opencv-python-headless" + + +@dataclass(frozen=True) +class SynthIDDetection: + """One local periodic-lattice verdict. + + The family this reports is NOT the watermark, and the field names say so. The + statistic is destroyed by a crop of seven pixels, while SynthID's published + evaluation retains 99.97% TPR under aggressive crop and resize, so what + crosses the threshold is a generation-pipeline lattice anchored at the image + origin. It identifies the pipeline, not the mark. The measurement is in + ``docs/synthid-detector-research.md`` and ``docs/synthid-classifiers.md``. + """ + + status: SynthIDDetectionStatus + width: int + height: int + score: float | None + threshold: float + detector: str = DETECTOR_ID + reason: str | None = None + signal_family: str = "generation-pipeline-lattice" + provider_scope: str = "provider-neutral" + backend: str = "local-pixel" + metadata_used_for_verdict: bool = False + pixels_preserved: bool = True + # Consumers cannot be expected to read a caveat in prose, so the two measured + # failure modes travel with every verdict. + tile_aligned_crop_required: bool = True + identifies_watermark: bool = False + + @property + def detected(self) -> bool: + """Whether the supported carrier crossed its frozen threshold.""" + return self.status == "detected" + + def to_dict(self) -> dict[str, str | int | float | bool | None]: + """Return a JSON-safe result without a local file path.""" + return { + "status": self.status, + "width": self.width, + "height": self.height, + "score": self.score, + "threshold": self.threshold, + "detector": self.detector, + "reason": self.reason, + "signal_family": self.signal_family, + "provider_scope": self.provider_scope, + "backend": self.backend, + "metadata_used_for_verdict": self.metadata_used_for_verdict, + "pixels_preserved": self.pixels_preserved, + "tile_aligned_crop_required": self.tile_aligned_crop_required, + "identifies_watermark": self.identifies_watermark, + } + + +@dataclass(frozen=True) +class LargeImageComponents: + """Auditable margins for the calibrated large-image carrier branch.""" + + width: int + height: int + minimum_fixed_score: float + minimum_red_green_spatial: float + minimum_blue_yellow_spatial: float + minimum_blue_yellow_mid_band: float + maximum_green_mid_band: float + + @property + def decision_score(self) -> float: + """Return the minimum normalized gate margin; one is the boundary.""" + margins = [ + self.minimum_fixed_score / LARGE_FIXED_SCORE_MIN, + self.minimum_red_green_spatial / LARGE_RED_GREEN_SPATIAL_MIN, + self.minimum_blue_yellow_spatial / LARGE_BLUE_YELLOW_SPATIAL_MIN, + self.minimum_blue_yellow_mid_band / LARGE_BLUE_YELLOW_MID_BAND_MAX, + ] + if (self.width, self.height) == LARGE_PORTRAIT_GEOMETRY: + margins.append(1.0 + LARGE_PORTRAIT_GREEN_MID_BAND_MAX - self.maximum_green_mid_band) + return min(margins) + + +def is_available() -> bool: + """True when numpy and OpenCV import.""" + import importlib.util + + return importlib.util.find_spec("cv2") is not None and importlib.util.find_spec("numpy") is not None + + +@lru_cache(maxsize=1) +def _load_template() -> tuple[NDArray[Any], float, int, int, int, int]: + """Load and validate the bundled pickle-free detector model.""" + import numpy as np + + model_path = Path(__file__).resolve().parent / MODEL_FILENAME + with np.load(model_path, allow_pickle=False) as artifact: + if int(artifact["format_version"]) != 1: + raise RuntimeError("unsupported SynthID detector model format") + height = int(artifact["height"]) + width = int(artifact["width"]) + tile_height = int(artifact["tile_height"]) + tile_width = int(artifact["tile_width"]) + denoise_sigma = float(artifact["denoise_sigma"]) + template = np.asarray(artifact["template"], dtype=np.float64) + if not _geometry_supported(width, height): + raise RuntimeError("bundled SynthID detector has unexpected geometry") + if template.shape != (tile_height, tile_width, 3): + raise RuntimeError("bundled SynthID detector has an invalid template shape") + if not np.all(np.isfinite(template)) or not np.isclose(np.linalg.norm(template), 1.0): + raise RuntimeError("bundled SynthID detector has an invalid template") + if not np.isfinite(denoise_sigma) or denoise_sigma <= 0.0: + raise RuntimeError("bundled SynthID detector has an invalid denoise sigma") + return template, denoise_sigma, height, width, tile_height, tile_width + + +def fold_residual_template( + pixels: NDArray[Any], + *, + tile_height: int, + tile_width: int, + denoise_sigma: float, +) -> NDArray[Any]: + """Estimate a zero-mean periodic residual template by modulo folding.""" + import cv2 + import numpy as np + + if pixels.ndim != 3 or pixels.shape[2] != 3: + raise ValueError("pixels must have shape (height, width, 3)") + if tile_height < 1 or tile_width < 1 or denoise_sigma <= 0.0: + raise ValueError("tile dimensions and denoise sigma must be positive") + height, width = pixels.shape[:2] + if height < tile_height or width < tile_width: + raise ValueError("image geometry must be at least as large as the tile geometry") + divisible = height % tile_height == 0 and width % tile_width == 0 + full_height = height - height % tile_height + full_width = width - width % tile_width + repeats_y = full_height // tile_height + repeats_x = full_width // tile_width + remaining_height = height - full_height + remaining_width = width - full_width + counts = np.full((tile_height, tile_width), repeats_y * repeats_x, dtype=np.int64) + counts[:remaining_height] += repeats_x + counts[:, :remaining_width] += repeats_y + counts[:remaining_height, :remaining_width] += 1 + + # OpenCV filters channels independently. Processing one channel at a time + # keeps the 18 MP upper bound from requiring two full three-channel float32 + # buffers in addition to the decoded image. + folded = np.empty((tile_height, tile_width, 3), dtype=np.float64) + for channel in range(3): + residual = pixels[:, :, channel].astype(np.float32) + residual -= cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=denoise_sigma, + sigmaY=denoise_sigma, + borderType=cv2.BORDER_REFLECT_101, + ) + if divisible: + folded[:, :, channel] = residual.reshape( + repeats_y, + tile_height, + repeats_x, + tile_width, + ).mean(axis=(0, 2), dtype=np.float64) + continue + folded_sum = ( + residual[:full_height, :full_width] + .reshape( + repeats_y, + tile_height, + repeats_x, + tile_width, + ) + .sum(axis=(0, 2), dtype=np.float64) + ) + if remaining_height: + bottom = residual[full_height:, :full_width].reshape( + remaining_height, + repeats_x, + tile_width, + ) + folded_sum[:remaining_height] += bottom.sum(axis=1, dtype=np.float64) + if remaining_width: + right = residual[:full_height, full_width:].reshape( + repeats_y, + tile_height, + remaining_width, + ) + folded_sum[:, :remaining_width] += right.sum(axis=0, dtype=np.float64) + if remaining_height and remaining_width: + folded_sum[:remaining_height, :remaining_width] += residual[ + full_height:, + full_width:, + ] + folded[:, :, channel] = folded_sum / counts + return folded - np.mean(folded, axis=(0, 1), keepdims=True) + + +def unit_tile(tile: NDArray[Any]) -> tuple[NDArray[Any], float]: + """Return TILE normalized by its L2 norm and the original norm.""" + import numpy as np + + norm = float(np.linalg.norm(tile)) + if norm == 0.0: + return np.zeros_like(tile, dtype=np.float64), 0.0 + return np.asarray(tile, dtype=np.float64) / norm, norm + + +def _image_size(image_path: Path) -> tuple[int, int]: + from PIL import Image + + with Image.open(image_path) as image: + return image.size + + +def _geometry_supported(width: int, height: int) -> bool: + """Whether the image has a calibrated number of periodic-tile samples.""" + pixels = width * height + return MIN_SUPPORTED_PIXELS <= pixels <= MAX_SUPPORTED_PIXELS + + +def _registered_geometry_supported(width: int, height: int) -> bool: + """Whether scale registration was challenged at this decoded size.""" + pixels = width * height + return ( + min(width, height) >= REGISTERED_MIN_SIDE + and REGISTERED_MIN_SUPPORTED_PIXELS <= pixels <= REGISTERED_MAX_SUPPORTED_PIXELS + ) + + +def _large_geometry_supported(width: int, height: int) -> bool: + """Whether fixed phase-aligned windows cover the calibrated large range.""" + pixels = width * height + return min(width, height) >= LARGE_WINDOW and LARGE_MIN_PIXELS < pixels <= LARGE_MAX_PIXELS + + +def _opponent_registered_geometry_supported(width: int, height: int) -> bool: + """Whether the opponent-color fallback passed its frozen geometry challenge.""" + pixels = width * height + return ( + min(width, height) >= OPPONENT_REGISTERED_MIN_SIDE + and OPPONENT_REGISTERED_MIN_PIXELS <= pixels <= REGISTERED_MAX_SUPPORTED_PIXELS + ) + + +def _fine_opponent_registered_geometry_supported(width: int, height: int) -> bool: + """Whether the fine-period selector passed its frozen geometry challenge.""" + pixels = width * height + return ( + min(width, height) >= FINE_OPPONENT_REGISTERED_MIN_SIDE + and FINE_OPPONENT_REGISTERED_MIN_PIXELS <= pixels <= FINE_OPPONENT_REGISTERED_MAX_PIXELS + ) + + +def folded_template_score( + pixels: NDArray[Any], + template: NDArray[Any], + denoise_sigma: float, +) -> tuple[float, NDArray[Any]]: + """Fold PIXELS at the model geometry and score the normalized tile.""" + tile_height, tile_width = template.shape[:2] + folded = fold_residual_template( + pixels, + tile_height=tile_height, + tile_width=tile_width, + denoise_sigma=denoise_sigma, + ) + normalized, _norm = unit_tile(folded) + return float((template * normalized).sum()), folded + + +def _large_window_starts(length: int) -> tuple[int, ...]: + """Return phase-aligned starts that cover both edges without resampling.""" + if length < LARGE_WINDOW: + raise ValueError("large-image sides must be at least 2,048 pixels") + last = ((length - LARGE_WINDOW) // LARGE_PHASE) * LARGE_PHASE + starts = list(range(0, last + 1, LARGE_WINDOW)) + if starts[-1] != last: + starts.append(last) + return tuple(starts) + + +def _correlation(left: NDArray[Any], right: NDArray[Any]) -> float: + import numpy as np + + denominator = float(np.linalg.norm(left) * np.linalg.norm(right)) + return float(np.real(np.vdot(right, left)) / denominator) if denominator > 0.0 else 0.0 + + +def _large_window_components( + folded: NDArray[Any], + template: NDArray[Any], +) -> tuple[float, float, float, float]: + """Measure the four color-phase features used by the large branch.""" + import numpy as np + + folded_red_green = folded[:, :, 0] - folded[:, :, 1] + template_red_green = template[:, :, 0] - template[:, :, 1] + folded_blue_yellow = folded[:, :, 2] - 0.5 * (folded[:, :, 0] + folded[:, :, 1]) + template_blue_yellow = template[:, :, 2] - 0.5 * (template[:, :, 0] + template[:, :, 1]) + + height, width = folded.shape[:2] + y_coordinates = np.minimum(np.arange(height), height - np.arange(height)) + x_coordinates = np.minimum(np.arange(width), width - np.arange(width)) + radius = np.sqrt(y_coordinates[:, None] ** 2 + x_coordinates[None, :] ** 2) + mid_band = (radius >= 4.5) & (radius < 6.5) + blue_yellow_mid = _correlation( + np.fft.fft2(folded_blue_yellow)[mid_band], + np.fft.fft2(template_blue_yellow)[mid_band], + ) + green_mid = _correlation( + np.fft.fft2(folded[:, :, 1])[mid_band], + np.fft.fft2(template[:, :, 1])[mid_band], + ) + return ( + _correlation(folded_red_green, template_red_green), + _correlation(folded_blue_yellow, template_blue_yellow), + blue_yellow_mid, + green_mid, + ) + + +def large_image_components( + pixels: NDArray[Any], + template: NDArray[Any], + denoise_sigma: float, +) -> LargeImageComponents: + """Score all phase-aligned 2,048-pixel windows of one large RGB image.""" + if pixels.ndim != 3 or pixels.shape[2] != 3: + raise ValueError("pixels must have shape (height, width, 3)") + height, width = pixels.shape[:2] + if not _large_geometry_supported(width, height): + raise ValueError("image geometry is outside the calibrated large-image range") + + minimum_fixed = float("inf") + minimum_red_green = float("inf") + minimum_blue_yellow = float("inf") + minimum_blue_yellow_mid = float("inf") + maximum_green_mid = -float("inf") + for y in _large_window_starts(height): + for x in _large_window_starts(width): + window = pixels[y : y + LARGE_WINDOW, x : x + LARGE_WINDOW] + fixed_score, folded = folded_template_score(window, template, denoise_sigma) + red_green, blue_yellow, blue_yellow_mid, green_mid = _large_window_components( + folded, + template, + ) + minimum_fixed = min(minimum_fixed, fixed_score) + minimum_red_green = min(minimum_red_green, red_green) + minimum_blue_yellow = min(minimum_blue_yellow, blue_yellow) + minimum_blue_yellow_mid = min(minimum_blue_yellow_mid, blue_yellow_mid) + maximum_green_mid = max(maximum_green_mid, green_mid) + return LargeImageComponents( + width=width, + height=height, + minimum_fixed_score=minimum_fixed, + minimum_red_green_spatial=minimum_red_green, + minimum_blue_yellow_spatial=minimum_blue_yellow, + minimum_blue_yellow_mid_band=minimum_blue_yellow_mid, + maximum_green_mid_band=maximum_green_mid, + ) + + +def detect_synthid( + image_path: str | Path, + *, + image: NDArray[Any] | None = None, + register_scale: bool | None = None, +) -> SynthIDDetection: + """Detect the supported periodic carrier in IMAGE_PATH. + + ``indeterminate`` means that the frozen periodic carrier did not cross its + calibrated threshold; it is not a clean-image guarantee. The default + production router uses scale registration through 10 megapixels and the + native large-image expert above that boundary. Set ``register_scale`` to + ``True`` to force registration or ``False`` to run the legacy fixed-period + diagnostic below the large-image boundary. + """ + path = Path(image_path) + if image is None: + width, height = _image_size(path) + else: + if image.ndim != 3 or image.shape[2] != 3: + raise ValueError("image must be a three-channel BGR array") + height, width = image.shape[:2] + large_mode = register_scale is not True and width * height > LARGE_MIN_PIXELS + registered_mode = register_scale is True or (register_scale is None and not large_mode) + if registered_mode: + geometry_supported = _registered_geometry_supported(width, height) + threshold = REGISTERED_THRESHOLD + detector_id = REGISTERED_DETECTOR_ID + unsupported_reason = ( + "registered-v3 requires 250,000-10,000,000 decoded pixels and both dimensions to be at least 256 pixels" + ) + elif large_mode: + geometry_supported = _large_geometry_supported(width, height) + threshold = LARGE_THRESHOLD + detector_id = LARGE_DETECTOR_ID + unsupported_reason = ( + "large-v1 requires more than 10,000,000 through 18,000,000 decoded pixels " + "and at least two phase-aligned 2048-pixel windows" + ) + else: + geometry_supported = _geometry_supported(width, height) + threshold = TILE_THRESHOLD + detector_id = DETECTOR_ID + unsupported_reason = "fixed-v2 requires 1,000,000-18,000,000 decoded pixels" + if not geometry_supported: + return SynthIDDetection( + status="unsupported", + width=width, + height=height, + score=None, + threshold=threshold, + detector=detector_id, + reason=unsupported_reason, + ) + if not is_available(): + raise RuntimeError(f"SynthID pixel detection needs numpy and OpenCV; {INSTALL_HINT}") + + import numpy as np + from PIL import Image + + template, sigma, *_model = _load_template() + if image is None: + with Image.open(path) as source: + pixels = np.asarray(source.convert("RGB"), dtype=np.uint8) + else: + pixels = np.asarray(image[:, :, ::-1], dtype=np.uint8) + if pixels.shape != (height, width, 3): + raise RuntimeError("decoded image geometry does not match its header") + if registered_mode: + from synthid_runtime._synthid_registered import ( + fine_opponent_registered_score, + opponent_registered_score, + registered_score, + ) + + score = registered_score(pixels, template, sigma) + if score < REGISTERED_THRESHOLD and _opponent_registered_geometry_supported(width, height): + opponent_score = opponent_registered_score(pixels, template, sigma) + if opponent_score >= OPPONENT_REGISTERED_THRESHOLD: + score = opponent_score + threshold = OPPONENT_REGISTERED_THRESHOLD + detector_id = OPPONENT_REGISTERED_DETECTOR_ID + if score < threshold and _fine_opponent_registered_geometry_supported(width, height): + fine_score = fine_opponent_registered_score(pixels, template, sigma) + if fine_score >= FINE_OPPONENT_REGISTERED_THRESHOLD: + score = fine_score + threshold = FINE_OPPONENT_REGISTERED_THRESHOLD + detector_id = FINE_OPPONENT_REGISTERED_DETECTOR_ID + elif large_mode: + score = large_image_components(pixels, template, sigma).decision_score + else: + score, _folded = folded_template_score(pixels, template, sigma) + detected = score >= threshold + return SynthIDDetection( + status="detected" if detected else "indeterminate", + width=width, + height=height, + score=score, + threshold=threshold, + detector=detector_id, + reason=None if detected else "the selected carrier expert did not cross every calibrated gate", + ) diff --git a/scripts/synthid_runtime/synthid_periodic_tile_2048_v1.npz b/scripts/synthid_runtime/synthid_periodic_tile_2048_v1.npz new file mode 100644 index 0000000000000000000000000000000000000000..33bbf4cb6817e65edc72c919065b278abdaca6b7 GIT binary patch literal 7501 zcmchcWl&t(nudWO!JP#6;2r`&1C55@78+^XT^nf#!Gc3@4epv?jZ4tr5ZtYC4Q>N< zPiF2pXXe)Ym^*9N-fQow_3rnrZ+*Ykqa^nPkqr(G?&-r#4L9sK6Mg^3z=I=zGqrQH zF>nUCnm9UH*x9n%+PlNU;lTMntbH6(|Cr!XcAssN;`H_?bt#pme+H)NVViTWz0mXhs!*i4CXHq~4d1gjOl!I_dk}QB$jYa^^E`^MY45##`B$yH| z48N10JtQ$Vu`n}t{2DP#+SwSr|K;|9=Kx7WZ-gT>|Wf0L~WHCZKvxQQZ!!Lx6x26Lw7-ji`;fxe#MaKj$pqwNVP<3SdsjaF(U8*rx-m^N|F?~a z%z2@Y{g=*^4e}5Eodn~t9ws*S)&|Zdze9X@{ttJvXaAVf;?G41BhI@|^CW@H*)oL^ z^t3!=Cg@CRGQ>W_J_{;5@-U#ptY z_)aI;OI_(SRYh4}-7hWErY5+!_o46gw@v(tu84GBL(JZHOA9;OP@$sB$q8o{+kWWI z)s)DEs@;GyzS?v&8(&l~LRyLZ9Et<4nzx4DDSd_JJ!f}Bx14yDVAI1G0DS?dMK$H$ zHY*h;So_UPTV(C+8RKoD~jq}Tn-7hxbCW&Uqc^s83WY+Q)^ z3C}IW?>bt0-41jHE}k-uW(qSM4EfjCSv3nJceY=wM_)!$CJocYWVDD2qJmu4i7lF6 zh%NXyMZ{^7aZadi$W@KXGq&&u!mPE?eBOSihi$jFRxWVeuX)iq8soXF4J3hw^5ox` zW4BJ$4x!}vNQsF-;WCeW@%vIeRgO=bn7Pz9TY zV^!BYOWmj`tmivd!r9t4JMty>CtvB0lM=3r0$IbXDX?y{fQZ$WI!v@A#5fHy4#slv zZ?YP;E_{ zI~-h*2U;%J{ER>PO-G_{B4_o$qpUZ2)`8Qwqk4EKZ)xDxP?S)!;KI^&G~#O}k!*dp z)CL061vejCHw+HdU&FP?9TB`VU%fGe(rhf=YwaD6UP@p|)_;~8pbpqv>s=22$$B9& zq3KaXMUtl@tnC*`(4PHsTX02HnDem0<5bCuTrch$q4s-d_&u*(9J^p|SBY|Yw2eQ; zMUBUp^uP+E43Uri!fq#Gwtd0}MKfCE>4x*qMk!C;y}EYC48V;R-Wb=%OZd1}LHi`O zS8cPWQL$k=J>b=DPweT(pC8-+AG;zy8)z}8E?IkHTE}Su*&5gUppJrR;gZM0rb>rf z77nVLtQHvSEp2d^U#>N+s#vJbk}SKbSco+m(Df?nO)xdWL!q-R+;1*-m^sLo33BEt z(`IU$a2s^O_AX>TEwf1G7k5}5%6h1M(Xl1^HPDi9w?_E{6Q-AZNysh6&#vnzDO&5P|Gnp+M!21LQ3ENv`HOYtEgVJnJ-t4x?#P1r>Q-s_s+Y90Rc@ z>%?(<`=c&YWk24YYAbdDG-H8TtRajYS)cYNp{ZE5s)(jF4b!w^l+{+-a$HfFa z5mgsEcAJLYqn+q0D}5-QrXmrI;6wE&DgSPe4(sbM<_;m#;Q#zgEO4V$!^0cOSHF%{ zViEHR$uC*OCHuT{Q~gXO)QpV~NysxLqY05CG*T;lVz6zxvy9YpKhSTgHM!^xQ|+cf z>9Vk&o9rbmsMssCzw4%#hn79^^YSMTtKsW-XD>GPg_R`*X$bghDIoUK&X6`2M&eeY z3Szl1w-Gh(slrXl0AdNpg1w<7EulnY5p(dwPv6G-2vr+CEhQ1>kx?Zs)b>|H;ZbfQ zT#8=%n(n`l#>vWs!7?0#{gdqb6|6Gtn@(5tk%(vTFn0H4yN!s77Y(PaJ7;8dX$d+) zv&M*&V5~Vv&#G_t4+hrp!BJmn*{voBfZ#4s*NNkOE2+joklhGuE}dbN=bFPo7J(y)hV|CDi^c)4ZTW& zDIS=6uPZImgac^$;6#8(hy zzE0s(VZYN9nW;;sFXzE^cyrw(*UcFOk-dF;6rb-f7>lC}8p@1)*HR}FXfTPMY9{an zRV39hL>4v-QwC5b5PGC;oJ7Upn%6_?pJUqy)@#%B4(~k=B3xYvF9Dl z%NI6SK5q$~DcO)|91x?l9?KT*_M;`V26&p=Cwwz!lSeL;CWn;|=zf_0NO-A!6FuBE zF7GP56FgHxGS@=h+cS5PB|X;sHmw4*9(RvGX!X6(z)1u%@<;DH>oRzu`$L(XXnNBUbZi? zE?a50*85)SBQ4Ck!CQc-*9pYNi=LicrS<`@CcDGppeUz?MPlk+7BQ@$7|6ocU^&Z} zHrNcPccF}?=hJpfc<-_Fx3+b~K|j5U=G+@*6A1H0n(p;vZg7PE3B++nxWnOXJ^i(t zGuMP({Ro3kS~zblJ7Hwb%VqG-pP<>%AHy0ddv6>#$i4d&k8evtkvvlRS72W-sOnIA zYdHK@DJ?|O$YRVZu&9n>_xsx2lqH{`VdbnC)Vi3=o<%Jc-FDMXc)x7qp-|B?>JXf0-sqkoqh2WsWW}mGk(Q>-{KFW_!C*=6)yS zlVZY;(cG1F3PD3s3Y>VLSE9)nqjG)$n|DR>FDx#Syuw%BmIp2H9f2Zh2--VaRMCUL zcJOl3nSHz*2(;^w3iqeXt;tKT3;SK4S@Yl0`P#+8Cz;GNjrvR<27EialFHXvN}s!p zs23nVRdi|I@pQ7*`3S77uUu9^E zXpW(V1<5cl((LM(5&C`z*bUZxE)avprF8 z^TG5WU0dKHu@rB>AM2Vlep?~cqeR0XjH#ejzmb-9^kJluL#&v6r1hIlV)ko6q)YWW z6t+~n<+9m&V4)uA4V$1u2t}@#MtgIG@tw!hc@cRv686BPV?m-xQQX!neIb3zc;DMF zh9kjHz_8VQ>0&OK7H6N82D=9r9JUS%tNa`>vR+cgu|k1{B+TN`T9QjHdXGGRs_TGHE!Ewbk^m%OZ>dLZW&^Ml)s>i*SKEqmF5uRZ$UVK zcHnt;*>}L2pHGNtmf!!n4nzbXI8ezBPvY25XlB|u5B3l7_9<+`-mA10;%QucjhTIe zoDSK;Xj^pj%r6hiU!6_!5vfXN6pQ|17F+6GrmyIHRJ9I=`J_n3#w~QuSJkd*IDDT5 zW4?fl((4uO6NXtde)?nI#o;QgVJ&i<6G+VKF}ye4=OU%u5}pK$JF1SSlFZg>yziHm z+yJu~8M6USg{{-b<8?_Y!wgezD$Xk;{9=+R=}Gv^TEfGkNBezDN%* zVfnbvg5(92zqF74>8dPKL||IhrhT+PqoeETMPHEcsf=b#o#CxTU3RTrqKek1a<>yA z96B57wc?!38d>Iron?9c&!h-|1D2>eppL`T@K+J# zY3X;90HGuO@~Q}aI8>)f~$IfD^w9I_zZs`qNa&~XKzH| zN=L)lmNjH6PFK&UQ#>Z??s;{orhUmckPXU7`7%KQ0rW*KK*~ZSD1d@Q2iqOSx`yZH zr`Cyb_MLM|$7FqGIn_@a;O{PQF%~UTr{J+B;lJ7mIXK@QsZ1BF@StgWy3omWS7s&u zgspk)S7)B(<_b(?^vrMb~7j*P+qjy3xxiL(b+DZUu3VkB;x}oArgd_@oQV z^7Owz&Xz}3ol{FR3q%`s{iCHB2*}Dse+!&4oN%_k_nFrJJbp`+%Zo&m zBN8zm!-z0)AL4Z?yu#IbwG~`mOqNkK?Xo>3hqrL9$ss>vLKRWXb4zihf8JiWQ48z@ z@Aa*B7k02#eGva93lwf9gBjiwnPgpa&`R<~dc7wdriJ=#VefX7X{_xc^t?TrwWy9k zayxN3gJ2f;TU#(g>w^%G1FH)iEgTNFt8|A0)(w|1XVU8q2T0j^OBS^mia2lj@p-d% zr3f}>Zxm=z)?{(jO!$zpTzMocEYJzhj0tDTH6tJhYlx&@@KH8nl zu76p52bNBnymk~soj)6F9FH=WXlqN0)x-RL6g1yS+N_p+30Z+;x=b%=aqBN}6JWan zx6yD{0yq>tgvi4tPF_n8P8a+cgve5_r&FBg|Z!4hl%BtQysdVTNj@5(L-w zB6bDWTuKacZ;PYG{oad}vy^h>xI2o0=-9LS_zlzawa1DfLVOPswG0?YlHS8qn((wj*rG>|R_EIUGEEGO)R=Odd zly*2)Ba?MM9i_P|M?X6l_-TxIVP&ubF-2<+L-qtb)kkr_-VGrB)AxFy&dW&tSB^lHFZ&6bU*EyqT;=|C;d*s!!@<97e80*g=17bkuhZ=ltsgq#N6heJ6LMtXx6U3cTx(nosrUq zbp{yR$MF~JdkIOzze0-i>?KSGS|9q3d*Ib!)Nl%YiUv!?Y+4#_V){e?1<=u6 z8@dLC`JS3{_d83;-Pt@9|7@@md=k#L#9i5Wor#W~{L{NP`1xi==m#B{;3^i1Z5nXl zyU_}0G7p^B$04jTNz2CD|l%pCIM z(>$OsyY3$@SmuN))51dm2M*V*)OVz%(j3YTvmwKa%Hl-dIob2d2{SU_+d}B4i7|SC zG&8oAlJ?c^%eKBIG-SYo2tm+>SlN-}`g0r4x#0%aS-&$nevKw~eyScylp&o@7yHY; zr0!wRlq$xIYP%Et;kfr|k>2rTU+M8RbsaFFmrw$?ArNG11~0VNz*dhV46W_vYWTa( zI|^3W^f<02C4@a1WH~;Bz9U=kDy)y;wancSc&<1iK$h9-dyv=qdo#)XMTxTxnCgP> zt|3vEo|E1TZ7h5`B?$30Ib+Fe!<)843EQ5X3tTQFgHl#IAJ11F_+c(%=CtWE-E)q6 z#oNl)xIcV|`B^=2D)yH~3Pvuh8*v&(WG-0sa%V9k z3%qh70hF`fuToReOyaNw@1iuxcC!Hzb1xS)e27PDwYSbdAYW_7*7 z@&F`waiJqN?f!;8vKv#1LdH(ZWT=%%Y93w^PthhU|!h5(Ba zcwO`g8zhQpapqkby|u&j7zn=&pUBfOBSD z*;MD_D$+_R77Dul_Y>J7DLCA8y3J=@;{<@kM3NxbvJxvpvu!EI@kg8hyj9e0H8NUG zWMM;Y<&chLJYXDMK)$a0U4am^d$Ep=NGjEK zOOZCFmj6XMH=Uy4G_ub=s1Q7jl8$VQ_7lBnJU?r!!#=bA)LEv%O-kwf9dc^@S}M;^ zHMDz_KjWhjh1r7NhxCuQzyxe>V&rUM46=P7TE9=~f8zqV|0`Tj`J?gr)h{^#xCdka zkHGfi&tUBlmp!60yg%$7rw{Ot&%a}~zv>@x*l#`BqvVhA0Y2p6?Z2SVztW-p7Cp=# z;4vPY|C8eURrbhye%oRHNf!Pf`wNKrS7NL`i3t$?5dV+HkC^9?-aKv%?w_>D58A&$ zpTAl^vYAKj^2lH?{v@Y+@?iZ>q~@>6$A)?AgTEX1lED4FeU#*okRSj5^x;~1D2XQN H ExpertScore: + return {"name": name, "supported": supported, "score": score} + + +def score_pixels(pixels: NDArray[np.uint8]) -> list[ExpertScore]: + """Return explicit fixed, registered, and large observations for RGB PIXELS.""" + if pixels.ndim != 3 or pixels.shape[2] != 3 or pixels.dtype != np.uint8: + raise ValueError("pixels must be an RGB uint8 array") + bgr_pixels = np.ascontiguousarray(pixels[:, :, ::-1]) + native = synthid_detector.detect_synthid("decoded-image", image=bgr_pixels, register_scale=False) + registered = synthid_detector.detect_synthid("decoded-image", image=bgr_pixels, register_scale=True) + fixed = _observation(FIXED_EXPERT_NAME, False, None) + large = _observation(LARGE_EXPERT_NAME, False, None) + native_observation = _observation(native.detector, native.status != "unsupported", native.score) + if native.detector == FIXED_EXPERT_NAME: + fixed = native_observation + elif native.detector == LARGE_EXPERT_NAME: + large = native_observation + else: + raise RuntimeError(f"unexpected default SynthID expert: {native.detector}") + return [ + fixed, + _observation(REGISTERED_EXPERT_NAME, registered.status != "unsupported", registered.score), + large, + ] + + +def score_path(path: Path) -> ScoredImage: + """Decode PATH once and return one hash-pinned observation record.""" + pixels = load_rgb(path) + height, width = pixels.shape[:2] + return { + "id": artifact_sha256(path), + "path": str(path), + "width": width, + "height": height, + "observations": score_pixels(pixels), + } + + +@click.command() +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def main(images: tuple[Path, ...], report_out: Path) -> None: + """Score IMAGES with every shipped pixel expert.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + records = [score_path(path) for path in images] + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text( + json.dumps( + { + "schema_version": 1, + "experts": [FIXED_EXPERT_NAME, REGISTERED_EXPERT_NAME, LARGE_EXPERT_NAME], + "records": records, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d three-expert score records: %s", len(records), report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_spectral_probe.py b/scripts/synthid_spectral_probe.py new file mode 100644 index 0000000..3859804 --- /dev/null +++ b/scripts/synthid_spectral_probe.py @@ -0,0 +1,524 @@ +"""Analyze wavelet and spectral structure from exact image pairs. + +This is a research harness, not a production SynthID detector. Pair provenance +and oracle labels remain external evidence. The harness deliberately separates +paired residual discovery from single-image fixed-template scoring, includes a +permuted-pair control, and stores arrays in NPZ without pickle. + +Usage: + uv run python scripts/synthid_spectral_probe.py discover \ + --pair clean.png marked.png --pair clean2.png marked2.png \ + --template-out .local-eval/synthid/template.npz \ + --report-out .local-eval/synthid/pair-report.json + + uv run python scripts/synthid_spectral_probe.py score \ + .local-eval/synthid/template.npz image.png other.png +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict, dataclass, field +from itertools import combinations +from pathlib import Path + +import click +import numpy as np +import pywt +from PIL import Image, ImageFilter + +log = logging.getLogger(__name__) + +WAVELET = "db2" +WAVELET_LEVELS = 3 +SPECTRAL_PEAKS = 32 +CEPSTRAL_PEAKS = 16 + + +@dataclass(frozen=True) +class PairMeasurement: + """Pixel-domain measurements for one exact clean/marked pair.""" + + clean: str + marked: str + width: int + height: int + psnr_db: float + changed_pixel_fraction: float + difference_min: float + difference_max: float + channel_mean: tuple[float, float, float] + channel_std: tuple[float, float, float] + + +@dataclass(frozen=True) +class ImageScore: + """Single-image phase alignment against a discovered template.""" + + path: str + phase_mean: float + phase_weighted: float + channel_phase_weighted: tuple[float, float, float] + top_two_channel_phase_12: float + peak_count: int + + +@dataclass +class _WaveletAccumulator: + """Streaming summary state for one wavelet band.""" + + coefficient_sum: np.ndarray + energy_sum: np.ndarray + coefficient_norm_sum: np.ndarray + energy_norm_sum: np.ndarray + rms_values: list[np.ndarray] = field(default_factory=list) + + +def load_rgb(path: Path) -> np.ndarray: + """Load PATH as float64 RGB pixels.""" + with Image.open(path) as image: + return np.asarray(image.convert("RGB"), dtype=np.float64) + + +def _resize_float(channel: np.ndarray, size: int) -> np.ndarray: + """Resize one floating-point channel without quantizing the residual.""" + image = Image.fromarray(channel.astype(np.float32), mode="F") + return np.asarray(image.resize((size, size), Image.Resampling.BILINEAR), dtype=np.float64) + + +def pair_residual(clean: Path, marked: Path, *, size: int = 512) -> tuple[np.ndarray, PairMeasurement]: + """Return a canonical RGB residual and measurements for an exact pair.""" + clean_rgb = load_rgb(clean) + marked_rgb = load_rgb(marked) + if clean_rgb.shape != marked_rgb.shape: + raise ValueError(f"pair shapes differ: {clean_rgb.shape} != {marked_rgb.shape}") + + difference = marked_rgb - clean_rgb + mse = float(np.mean(np.square(difference))) + psnr = float("inf") if mse == 0.0 else float(20.0 * np.log10(255.0 / np.sqrt(mse))) + residual = np.stack([_resize_float(difference[:, :, channel], size) for channel in range(3)], axis=2) + measurement = PairMeasurement( + clean=str(clean), + marked=str(marked), + width=int(clean_rgb.shape[1]), + height=int(clean_rgb.shape[0]), + psnr_db=psnr, + changed_pixel_fraction=float(np.mean(np.any(difference != 0.0, axis=2))), + difference_min=float(np.min(difference)), + difference_max=float(np.max(difference)), + channel_mean=tuple(float(value) for value in np.mean(difference, axis=(0, 1))), + channel_std=tuple(float(value) for value in np.std(difference, axis=(0, 1))), + ) + return residual, measurement + + +def normalized_channels(residual: np.ndarray) -> np.ndarray: + """Zero-center and unit-normalize each residual channel.""" + centered = residual - np.mean(residual, axis=(0, 1), keepdims=True) + norms = np.linalg.norm(centered, axis=(0, 1), keepdims=True) + return np.divide(centered, norms, out=np.zeros_like(centered), where=norms != 0.0) + + +def channel_ncc(first: np.ndarray, second: np.ndarray) -> tuple[float, float, float]: + """Return per-channel normalized cross-correlation.""" + first_norm = normalized_channels(first) + second_norm = normalized_channels(second) + values = np.sum(first_norm * second_norm, axis=(0, 1)) + return tuple(float(value) for value in values) + + +def pairwise_ncc_from_normalized_sum( + total: np.ndarray, + squared_norm_sum: np.ndarray, + count: int, +) -> tuple[float, float, float]: + """Recover mean pairwise NCC from the sum of normalized RGB arrays.""" + if count < 2: + return (0.0, 0.0, 0.0) + pair_count = count * (count - 1) / 2 + values = (np.sum(np.square(total), axis=(0, 1)) - squared_norm_sum) / (2.0 * pair_count) + return tuple(float(value) for value in values) + + +def stationary_wavelet_bands( + residual: np.ndarray, + *, + wavelet: str = WAVELET, + levels: int = WAVELET_LEVELS, +) -> dict[str, np.ndarray]: + """Return undecimated detail bands while preserving canonical coordinates.""" + if residual.ndim != 3 or residual.shape[2] != 3: + raise ValueError("residual must be an RGB array") + if levels < 1: + raise ValueError("levels must be positive") + divisor = 2**levels + if residual.shape[0] % divisor or residual.shape[1] % divisor: + raise ValueError(f"residual dimensions must be divisible by {divisor}") + + coefficients = pywt.swt2(residual, wavelet, level=levels, axes=(0, 1)) + bands: dict[str, np.ndarray] = {} + for level, (_approximation, details) in enumerate(reversed(coefficients), start=1): + for orientation, values in zip(("h", "v", "d"), details, strict=True): + bands[f"level_{level}_{orientation}"] = np.asarray(values, dtype=np.float64) + return bands + + +def wavelet_report(residuals: list[np.ndarray]) -> dict[str, object]: + """Summarize multi-scale wavelet energy and cross-pair repeatability.""" + if not residuals: + raise ValueError("at least one residual is required") + accumulators: dict[str, _WaveletAccumulator] = {} + for residual in residuals: + for name, values in stationary_wavelet_bands(residual).items(): + coefficient = normalized_channels(values) + energy = normalized_channels(np.square(values)) + accumulator = accumulators.get(name) + if accumulator is None: + accumulator = _WaveletAccumulator( + coefficient_sum=np.zeros_like(coefficient), + energy_sum=np.zeros_like(energy), + coefficient_norm_sum=np.zeros(3, dtype=np.float64), + energy_norm_sum=np.zeros(3, dtype=np.float64), + ) + accumulators[name] = accumulator + accumulator.coefficient_sum += coefficient + accumulator.energy_sum += energy + accumulator.coefficient_norm_sum += np.sum(np.square(coefficient), axis=(0, 1)) + accumulator.energy_norm_sum += np.sum(np.square(energy), axis=(0, 1)) + accumulator.rms_values.append(np.sqrt(np.mean(np.square(values), axis=(0, 1)))) + + bands: list[dict[str, object]] = [] + for name, accumulator in accumulators.items(): + rms = np.asarray(accumulator.rms_values) + bands.append( + { + "name": name, + "median_rms": [float(value) for value in np.median(rms, axis=0)], + "rms_cv": [ + float(value) + for value in np.divide( + np.std(rms, axis=0), + np.mean(rms, axis=0), + out=np.zeros(3, dtype=np.float64), + where=np.mean(rms, axis=0) > 1e-12, + ) + ], + "coefficient_ncc": pairwise_ncc_from_normalized_sum( + accumulator.coefficient_sum, + accumulator.coefficient_norm_sum, + len(residuals), + ), + "energy_map_ncc": pairwise_ncc_from_normalized_sum( + accumulator.energy_sum, + accumulator.energy_norm_sum, + len(residuals), + ), + } + ) + return {"wavelet": WAVELET, "levels": WAVELET_LEVELS, "bands": bands} + + +def _top_half_plane_offsets(values: np.ndarray, count: int, *, min_radius: float) -> list[tuple[int, int, int]]: + """Return top ROW/COLUMN/CHANNEL indices from one centered half-plane.""" + height, width, channels = values.shape + center_y, center_x = height // 2, width // 2 + yy, xx = np.ogrid[:height, :width] + radius = np.sqrt(np.square(yy - center_y) + np.square(xx - center_x)) + half_plane = (yy > center_y) | ((yy == center_y) & (xx >= center_x)) + valid = (radius >= min_radius) & half_plane + masked = np.where(valid[:, :, None], values, -np.inf) + limit = min(count, int(np.sum(np.isfinite(masked)))) + flat = np.argpartition(masked.ravel(), -limit)[-limit:] + flat = flat[np.argsort(masked.ravel()[flat])[::-1]] + return [tuple(int(value) for value in np.unravel_index(index, (height, width, channels))) for index in flat] + + +def spectral_report(residuals: list[np.ndarray]) -> dict[str, object]: + """Summarize complex phase coherence, power, and cepstral periodicity.""" + if not residuals: + raise ValueError("at least one residual is required") + unit_sum = np.zeros(residuals[0].shape, dtype=np.complex128) + power_sum = np.zeros(residuals[0].shape, dtype=np.float64) + for residual in residuals: + spectrum = np.fft.fftshift(np.fft.fft2(normalized_channels(residual), axes=(0, 1)), axes=(0, 1)) + magnitude = np.abs(spectrum) + unit_sum += np.divide(spectrum, magnitude, out=np.zeros_like(spectrum), where=magnitude > 1e-12) + power_sum += np.square(magnitude) + coherence = np.abs(unit_sum / len(residuals)) + power = power_sum / len(residuals) + weighted = coherence * np.sqrt(power) + height, width, _ = coherence.shape + center_y, center_x = height // 2, width // 2 + total_power = np.sum(power, axis=(0, 1)) + peaks = [] + for row, column, channel in _top_half_plane_offsets(weighted, SPECTRAL_PEAKS, min_radius=4.0): + peaks.append( + { + "dy": row - center_y, + "dx": column - center_x, + "channel": channel, + "phase_coherence": float(coherence[row, column, channel]), + "power_fraction": float(power[row, column, channel] / max(total_power[channel], 1e-12)), + } + ) + + mean_power = np.mean(power, axis=2) + cepstrum = np.abs(np.fft.fftshift(np.fft.ifft2(np.fft.ifftshift(np.log1p(mean_power))))) + cepstral_cube = cepstrum[:, :, None] + cepstral = [ + {"dy": row - center_y, "dx": column - center_x, "magnitude": float(cepstrum[row, column])} + for row, column, _channel in _top_half_plane_offsets(cepstral_cube, CEPSTRAL_PEAKS, min_radius=2.0) + ] + return { + "phase_coherence_median": float(np.median(coherence)), + "phase_coherence_p95": float(np.quantile(coherence, 0.95)), + "peaks": peaks, + "cepstral_peaks": cepstral, + } + + +def _canonical_rgb(path: Path, size: int) -> np.ndarray: + """Return floating-point RGB pixels at the canonical analysis size.""" + with Image.open(path) as source: + image = source.convert("RGB").resize((size, size), Image.Resampling.BILINEAR) + return np.asarray(image, dtype=np.float64) + + +def permutation_control( + residuals: list[np.ndarray], + measurements: list[PairMeasurement], +) -> dict[str, object] | None: + """Compare true residual repeatability with deliberately mismatched pairs.""" + if len(measurements) < 2: + return None + size = residuals[0].shape[0] + true_sum = np.zeros_like(residuals[0]) + mismatched_sum = np.zeros_like(residuals[0]) + true_norm_sum = np.zeros(3, dtype=np.float64) + mismatched_norm_sum = np.zeros(3, dtype=np.float64) + mismatched_rms: list[float] = [] + for residual in residuals: + normalized = normalized_channels(residual) + true_sum += normalized + true_norm_sum += np.sum(np.square(normalized), axis=(0, 1)) + for index, measurement in enumerate(measurements): + other = measurements[(index + 1) % len(measurements)] + clean = _canonical_rgb(Path(measurement.clean), size) + marked = _canonical_rgb(Path(other.marked), size) + mismatched = marked - clean + normalized = normalized_channels(mismatched) + mismatched_sum += normalized + mismatched_norm_sum += np.sum(np.square(normalized), axis=(0, 1)) + mismatched_rms.append(float(np.sqrt(np.mean(np.square(mismatched))))) + return { + "strategy": "cyclic marked-image permutation", + "true_pair_ncc": pairwise_ncc_from_normalized_sum(true_sum, true_norm_sum, len(residuals)), + "mismatched_pair_ncc": pairwise_ncc_from_normalized_sum(mismatched_sum, mismatched_norm_sum, len(residuals)), + "true_median_rms": float(np.median([np.sqrt(np.mean(np.square(residual))) for residual in residuals])), + "mismatched_median_rms": float(np.median(mismatched_rms)), + } + + +def build_template(residuals: list[np.ndarray]) -> np.ndarray: + """Average canonical residuals after per-channel normalization.""" + if not residuals: + raise ValueError("at least one residual is required") + shape = residuals[0].shape + if any(residual.shape != shape for residual in residuals): + raise ValueError("all canonical residuals must have the same shape") + return np.mean([normalized_channels(residual) for residual in residuals], axis=0) + + +def _template_fft(template: np.ndarray) -> np.ndarray: + """Return a centered two-dimensional FFT for each RGB channel.""" + return np.fft.fftshift(np.fft.fft2(template, axes=(0, 1)), axes=(0, 1)) + + +def select_peaks( + template: np.ndarray, + *, + count: int = 64, + min_radius: float = 8.0, + max_radius_fraction: float = 0.35, + min_distance: float = 3.0, +) -> np.ndarray: + """Select separated high-energy carrier bins from one Fourier half-plane.""" + if count <= 0: + raise ValueError("count must be positive") + height, width, channels = template.shape + if height != width or channels != 3: + raise ValueError("template must be a square RGB array") + center = height // 2 + spectrum = _template_fft(template) + magnitude = np.linalg.norm(spectrum, axis=2) + yy, xx = np.ogrid[:height, :width] + radius = np.sqrt(np.square(yy - center) + np.square(xx - center)) + valid = (radius >= min_radius) & (radius <= height * max_radius_fraction) + candidates = np.flatnonzero(valid) + order = candidates[np.argsort(magnitude.ravel()[candidates])[::-1]] + + selected: list[tuple[int, int]] = [] + for flat_index in order: + row, column = np.unravel_index(flat_index, magnitude.shape) + dy, dx = int(row - center), int(column - center) + if dy < 0 or (dy == 0 and dx < 0): + continue + if any((dy - old_dy) ** 2 + (dx - old_dx) ** 2 < min_distance**2 for old_dy, old_dx in selected): + continue + selected.append((dy, dx)) + if len(selected) == count: + break + if len(selected) != count: + raise ValueError(f"could select only {len(selected)} of {count} peaks") + return np.asarray(selected, dtype=np.int32) + + +def _high_pass_rgb(path: Path, size: int, blur_radius: float) -> np.ndarray: + """Decode, resize, and subtract a small Gaussian blur from RGB pixels.""" + with Image.open(path) as source: + image = source.convert("RGB").resize((size, size), Image.Resampling.LANCZOS) + pixels = np.asarray(image, dtype=np.float64) + blurred = np.asarray(image.filter(ImageFilter.GaussianBlur(radius=blur_radius)), dtype=np.float64) + return pixels - blurred + + +def score_image(path: Path, template: np.ndarray, peaks: np.ndarray, *, blur_radius: float = 2.0) -> ImageScore: + """Score one image by phase alignment at discovered carrier bins.""" + size = int(template.shape[0]) + image_fft = np.fft.fftshift(np.fft.fft2(_high_pass_rgb(path, size, blur_radius), axes=(0, 1)), axes=(0, 1)) + template_fft = _template_fft(template) + center = size // 2 + phase_values: list[np.ndarray] = [] + weights: list[np.ndarray] = [] + for dy, dx in peaks: + image_value = image_fft[center + int(dy), center + int(dx)] + template_value = template_fft[center + int(dy), center + int(dx)] + phase = np.real(image_value * np.conj(template_value)) / (np.abs(image_value) * np.abs(template_value) + 1e-12) + phase_values.append(phase) + weights.append(np.abs(template_value)) + phases = np.asarray(phase_values) + carrier_weights = np.asarray(weights) + channel_weighted = np.sum(phases * carrier_weights, axis=0) / np.sum(carrier_weights, axis=0) + consensus_count = min(12, len(peaks)) + consensus_channels = np.sum(phases[:consensus_count] * carrier_weights[:consensus_count], axis=0) / np.sum( + carrier_weights[:consensus_count], axis=0 + ) + top_two_channel_phase_12 = float(np.mean(np.sort(consensus_channels)[-2:])) + return ImageScore( + path=str(path), + phase_mean=float(np.mean(phases)), + phase_weighted=float(np.sum(phases * carrier_weights) / np.sum(carrier_weights)), + channel_phase_weighted=tuple(float(value) for value in channel_weighted), + top_two_channel_phase_12=top_two_channel_phase_12, + peak_count=len(peaks), + ) + + +def save_template(path: Path, template: np.ndarray, peaks: np.ndarray) -> None: + """Store a template in a pickle-free compressed NPZ artifact.""" + path.parent.mkdir(parents=True, exist_ok=True) + np.savez_compressed(path, template=template.astype(np.float32), peaks=peaks.astype(np.int32)) + + +def load_template(path: Path) -> tuple[np.ndarray, np.ndarray]: + """Load a template artifact without enabling pickle.""" + with np.load(path, allow_pickle=False) as artifact: + template = np.asarray(artifact["template"], dtype=np.float64) + peaks = np.asarray(artifact["peaks"], dtype=np.int32) + if template.ndim != 3 or template.shape[2] != 3 or template.shape[0] != template.shape[1]: + raise ValueError("invalid template shape") + if peaks.ndim != 2 or peaks.shape[1] != 2: + raise ValueError("invalid peak shape") + return template, peaks + + +def discovery_report( + residuals: list[np.ndarray], measurements: list[PairMeasurement], peaks: np.ndarray +) -> dict[str, object]: + """Build a JSON-safe paired wavelet and spectral discovery report.""" + pairwise = [ + { + "first": measurements[first].marked, + "second": measurements[second].marked, + "channel_ncc": channel_ncc(residuals[first], residuals[second]), + } + for first, second in combinations(range(len(residuals)), 2) + ] + return { + "pair_count": len(measurements), + "pairs": [asdict(measurement) for measurement in measurements], + "pairwise": pairwise, + "peaks": peaks.tolist(), + "wavelet": wavelet_report(residuals), + "spectral": spectral_report(residuals), + "permutation_control": permutation_control(residuals, measurements), + } + + +@click.group() +def main() -> None: + """Analyze paired residuals and score an experimental spectral template.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + + +@main.command() +@click.option( + "--pair", + "pairs", + type=( + click.Path(exists=True, dir_okay=False, path_type=Path), + click.Path(exists=True, dir_okay=False, path_type=Path), + ), + multiple=True, + required=True, + help="Exact CLEAN MARKED pair; repeat for multiple pairs.", +) +@click.option("--size", type=click.IntRange(min=64), default=512, show_default=True) +@click.option("--peak-count", type=click.IntRange(min=1), default=64, show_default=True) +@click.option("--template-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def discover( + pairs: tuple[tuple[Path, Path], ...], + size: int, + peak_count: int, + template_out: Path, + report_out: Path, +) -> None: + """Build a template and report from exact CLEAN MARKED pairs.""" + residuals: list[np.ndarray] = [] + measurements: list[PairMeasurement] = [] + for clean, marked in pairs: + residual, measurement = pair_residual(clean, marked, size=size) + residuals.append(residual) + measurements.append(measurement) + template = build_template(residuals) + peaks = select_peaks(template, count=peak_count) + save_template(template_out, template, peaks) + report = discovery_report(residuals, measurements, peaks) + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8") + log.info("Wrote template: %s", template_out) + log.info("Wrote report: %s", report_out) + + +@main.command() +@click.argument("template_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path)) +def score(template_path: Path, images: tuple[Path, ...], report_out: Path | None) -> None: + """Score IMAGES against TEMPLATE_PATH.""" + template, peaks = load_template(template_path) + scores = [asdict(score_image(image, template, peaks)) for image in images] + payload = {"template": str(template_path), "scores": scores} + rendered = json.dumps(payload, indent=2) + "\n" + if report_out is None: + log.info("%s", rendered.rstrip()) + return + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text(rendered, encoding="utf-8") + log.info("Wrote score report: %s", report_out) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_tile_attack.py b/scripts/synthid_tile_attack.py new file mode 100644 index 0000000..61a96b7 --- /dev/null +++ b/scripts/synthid_tile_attack.py @@ -0,0 +1,147 @@ +"""Estimate and subtract a periodic SynthID residual tile without regeneration. + +At 1536x2816, the dominant carrier bins lie on an FFT lattice spaced by 96 +rows and 88 columns, corresponding to a 16x32 spatial tile. Folding a +high-pass residual modulo that tile averages over 8448 repetitions and +suppresses non-periodic image content. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict +from pathlib import Path + +import click +import numpy as np +from PIL import Image +from synthid_ensemble_detector import detect_image, load_config, load_models +from synthid_periodic_tile import fold_residual_template +from synthid_pixel_attack import load_rgb, measure + +log = logging.getLogger(__name__) + + +def subtract_tiled_template(pixels: np.ndarray, template: np.ndarray, *, strength: float) -> np.ndarray: + """Subtract STRENGTH times TEMPLATE repeated over PIXELS.""" + if not np.isfinite(strength) or strength < 0.0: + raise ValueError("strength must be finite and nonnegative") + if pixels.ndim != 3 or pixels.shape[2] != 3: + raise ValueError("pixels must have shape (height, width, 3)") + if template.ndim != 3 or template.shape[2] != 3: + raise ValueError("template must have shape (tile height, tile width, 3)") + height, width = pixels.shape[:2] + tile_height, tile_width = template.shape[:2] + if tile_height == 0 or tile_width == 0: + raise ValueError("template dimensions must be positive") + + result = np.empty_like(pixels, dtype=np.uint8) + repeats_x = (width + tile_width - 1) // tile_width + for top in range(0, height, 256): + bottom = min(top + 256, height) + template_rows = template[np.arange(top, bottom) % tile_height] + repeated = np.tile(template_rows, (1, repeats_x, 1))[:, :width] + stripe = pixels[top:bottom].astype(np.float64) - strength * repeated + result[top:bottom] = np.clip(np.rint(stripe), 0, 255).astype(np.uint8) + return result + + +def parse_positive_floats(value: str, *, option_name: str) -> tuple[float, ...]: + """Parse a strictly increasing comma-separated positive-float sweep.""" + try: + values = tuple(float(item.strip()) for item in value.split(",")) + except ValueError as error: + raise click.BadParameter(f"{option_name} must be comma-separated numbers") from error + if not values or any(not np.isfinite(item) or item <= 0.0 for item in values): + raise click.BadParameter(f"{option_name} must be finite and positive") + if tuple(sorted(set(values))) != values: + raise click.BadParameter(f"{option_name} must be unique and strictly increasing") + return values + + +@click.command() +@click.argument("config_path", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("output_dir", type=click.Path(file_okay=False, path_type=Path)) +@click.option("--tile-height", type=click.IntRange(min=1), default=16, show_default=True) +@click.option("--tile-width", type=click.IntRange(min=1), default=32, show_default=True) +@click.option("--denoise-sigmas", default="0.6,1,1.5", show_default=True) +@click.option("--strengths", default="0.5,1,1.5,2", show_default=True) +def main( + config_path: Path, + source: Path, + output_dir: Path, + tile_height: int, + tile_width: int, + denoise_sigmas: str, + strengths: str, +) -> None: + """Write a frozen periodic-tile subtraction sweep for SOURCE.""" + logging.basicConfig(level=logging.INFO, format="%(message)s") + config = load_config(config_path) + rgb_model, hsv_model = load_models(config) + reference = load_rgb(source) + if reference.shape != (config.height, config.width, 3): + raise click.BadParameter("source geometry does not match detector config") + sigma_values = parse_positive_floats(denoise_sigmas, option_name="denoise sigmas") + strength_values = parse_positive_floats(strengths, option_name="strengths") + output_dir.mkdir(parents=True, exist_ok=True) + + variants: list[dict[str, object]] = [] + templates: list[dict[str, object]] = [] + for sigma in sigma_values: + template = fold_residual_template( + reference, + tile_height=tile_height, + tile_width=tile_width, + denoise_sigma=sigma, + ) + sigma_name = f"{sigma:g}".replace(".", "p") + templates.append( + { + "denoise_sigma": sigma, + "template_rms": float(np.sqrt(np.mean(np.square(template)))), + "template_max_abs": float(np.max(np.abs(template))), + } + ) + shifted = np.roll(template, shift=(1, 1), axis=(0, 1)) + for strength in strength_values: + strength_name = f"{strength:g}".replace(".", "p") + for control_name, selected_template in (("aligned", template), ("shifted", shifted)): + name = f"tile-{control_name}-sigma{sigma_name}-s{strength_name}" + pixels = subtract_tiled_template(reference, selected_template, strength=strength) + path = output_dir / f"{name}.png" + Image.fromarray(pixels, mode="RGB").save(path) + variants.append( + { + **asdict(measure(reference, pixels, name=name, path=path)), + **asdict(detect_image(path, config, rgb_model, hsv_model)), + "denoise_sigma": sigma, + "strength": strength, + "template_alignment": control_name, + } + ) + + report_path = output_dir / "report.json" + report_path.write_text( + json.dumps( + { + "source": str(source), + "config": str(config_path), + "tile_height": tile_height, + "tile_width": tile_width, + "repeat_count": (config.height // tile_height) * (config.width // tile_width), + "templates": templates, + "variants": variants, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d periodic-tile candidates: %s", len(variants), report_path) + + +if __name__ == "__main__": + main() diff --git a/scripts/synthid_v3_carrier_subtract.py b/scripts/synthid_v3_carrier_subtract.py new file mode 100644 index 0000000..335b541 --- /dev/null +++ b/scripts/synthid_v3_carrier_subtract.py @@ -0,0 +1,148 @@ +"""Build pixel-only V3 carrier-subtraction candidates and matched controls. + +The command uses a frozen numeric frequency profile as a local research +surrogate. It subtracts a sparse Hermitian spectrum, preserves image geometry, +and never invokes a generative model. A lower local score is not evidence that +the provider's SynthID verifier will change its decision. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict +from pathlib import Path + +import click +import numpy as np +from PIL import Image +from synthid_pixel_attack import load_rgb, measure, norm_matched_noise +from synthid_v3_codebook_probe import V3CarrierModel, load_v3_model, score_image + +log = logging.getLogger(__name__) + + +def load_exact_rgb(path: Path, model: V3CarrierModel) -> np.ndarray: + """Load PATH as RGB and reject geometry that differs from MODEL.""" + pixels = load_rgb(path) + if pixels.shape != (model.height, model.width, 3): + height, width = pixels.shape[:2] + raise ValueError(f"image geometry {width}x{height} does not match profile {model.width}x{model.height}") + return pixels + + +def subtract_carrier(pixels: np.ndarray, model: V3CarrierModel, *, strength: float) -> np.ndarray: + """Subtract STRENGTH times MODEL's sparse complex carrier from PIXELS.""" + if strength < 0.0: + raise ValueError("strength must be nonnegative") + expected_shape = (model.height, model.width, 3) + if pixels.shape != expected_shape: + raise ValueError(f"pixel shape {pixels.shape} does not match {expected_shape}") + + result = np.empty_like(pixels, dtype=np.float64) + for channel in range(3): + spectrum = np.fft.fft2(pixels[:, :, channel].astype(np.float64)) + positions = np.flatnonzero(model.channels == channel) + deltas: dict[tuple[int, int], complex] = {} + for position in positions: + row = int(model.rows[position]) + column = int(model.columns[position]) + delta = strength * model.expected_magnitudes[position] * np.exp(1j * model.phases[position]) + key = (row, column) + conjugate_key = ((-row) % model.height, (-column) % model.width) + deltas[key] = deltas.get(key, 0.0j) + delta + if conjugate_key == key: + deltas[key] = complex(deltas[key].real, 0.0) + else: + deltas[conjugate_key] = deltas.get(conjugate_key, 0.0j) + np.conj(delta) + for (row, column), delta in deltas.items(): + spectrum[row, column] -= delta + result[:, :, channel] = np.fft.ifft2(spectrum).real + return np.clip(np.rint(result), 0, 255).astype(np.uint8) + + +def parse_strengths(value: str) -> tuple[float, ...]: + """Parse a comma-separated, strictly increasing nonnegative sweep.""" + try: + strengths = tuple(float(item.strip()) for item in value.split(",")) + except ValueError as error: + raise click.BadParameter("strengths must be comma-separated numbers") from error + if not strengths or any(not np.isfinite(item) or item < 0.0 for item in strengths): + raise click.BadParameter("strengths must be finite and nonnegative") + if tuple(sorted(set(strengths))) != strengths: + raise click.BadParameter("strengths must be unique and strictly increasing") + return strengths + + +@click.command() +@click.argument("codebook", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("output_dir", type=click.Path(file_okay=False, path_type=Path)) +@click.option("--height", type=click.IntRange(min=64), required=True) +@click.option("--width", type=click.IntRange(min=64), required=True) +@click.option("--peak-count", type=click.IntRange(min=1), default=256, show_default=True) +@click.option("--strengths", default="0.25,0.5,1,1.5,2,4", show_default=True) +def main( + codebook: Path, + source: Path, + output_dir: Path, + height: int, + width: int, + peak_count: int, + strengths: str, +) -> None: + """Write a frozen analytical carrier-subtraction batch for SOURCE.""" + model = load_v3_model(codebook, height=height, width=width, peak_count=peak_count) + reference = load_exact_rgb(source, model) + sweep = parse_strengths(strengths) + output_dir.mkdir(parents=True, exist_ok=True) + + variants: list[dict[str, object]] = [] + strongest = reference + for strength in sweep: + pixels = subtract_carrier(reference, model, strength=strength) + name = f"subtract-{strength:g}".replace(".", "p") + path = output_dir / f"{name}.png" + Image.fromarray(pixels, mode="RGB").save(path) + variants.append( + { + **asdict(measure(reference, pixels, name=name, path=path)), + **asdict(score_image(path, model)), + "strength": strength, + } + ) + strongest = pixels + + sham = norm_matched_noise(reference, strongest, seed=20260809) + sham_path = output_dir / "sham-strongest-rms.png" + Image.fromarray(sham, mode="RGB").save(sham_path) + variants.append( + { + **asdict(measure(reference, sham, name="sham-strongest-rms", path=sham_path)), + **asdict(score_image(sham_path, model)), + "strength": None, + } + ) + + report_path = output_dir / "report.json" + report_path.write_text( + json.dumps( + { + "source": str(source), + "codebook": str(codebook), + "height": height, + "width": width, + "peak_count": peak_count, + "variants": variants, + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + log.info("Wrote %d frozen carrier candidates: %s", len(variants), report_path) + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(message)s") + main() diff --git a/scripts/synthid_v3_codebook_probe.py b/scripts/synthid_v3_codebook_probe.py new file mode 100644 index 0000000..bb65d97 --- /dev/null +++ b/scripts/synthid_v3_codebook_probe.py @@ -0,0 +1,344 @@ +"""Independently evaluate a numeric reverse-SynthID V3 NPZ codebook. + +The loader accepts only the documented dense or sparse numeric format-v2 arrays +and disables pickle. It does not import or execute third-party code. Scores are +exploratory: the external reference provenance and labels still require +independent oracle validation before this can support a SynthID detector claim. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import asdict, dataclass +from pathlib import Path + +import click +import numpy as np +from PIL import Image +from synthid_phase_registration import ( + MAX_TRANSLATION_SHIFT, + extract_frequency_values, + phase_adjustment, + register_phase_translations, +) + +log = logging.getLogger(__name__) +LOG_2 = np.log(2.0) + + +@dataclass(frozen=True) +class V3CarrierModel: + """Selected numeric bins from one exact-resolution V3 profile.""" + + height: int + width: int + rows: np.ndarray + columns: np.ndarray + channels: np.ndarray + phases: np.ndarray + weights: np.ndarray + expected_magnitudes: np.ndarray + + +@dataclass(frozen=True) +class V3Score: + """Phase-alignment scores for one image.""" + + path: str + phase_score: float + axial_phase_score: float + active_weight_fraction: float + peak_count: int + + +@dataclass(frozen=True) +class V3RegisteredScore: + """Best phase-alignment score across a bounded translation search.""" + + path: str + phase_score: float + axial_phase_score: float + active_weight_fraction: float + peak_count: int + row_shift: int + column_shift: int + + +@dataclass(frozen=True) +class _CarrierCandidates: + """Selected numeric carrier arrays from one V3 profile.""" + + weights: np.ndarray + rows: np.ndarray + columns: np.ndarray + channels: np.ndarray + phases: np.ndarray + magnitudes: np.ndarray + + +def _load_sparse_channel(artifact: np.lib.npyio.NpzFile, prefix: str, channel: int) -> tuple[np.ndarray, ...]: + """Load one sparse channel without reconstructing full image-sized arrays.""" + indices = np.asarray(artifact[f"{prefix}idx_{channel}"], dtype=np.uint32) + log_magnitudes = np.asarray(artifact[f"{prefix}mag_{channel}"]) + phases = np.asarray(artifact[f"{prefix}phase_{channel}"]) + coherence = np.asarray(artifact[f"{prefix}cons_{channel}"], dtype=np.float64) / 255.0 + if not (indices.shape == log_magnitudes.shape == phases.shape == coherence.shape): + raise ValueError("sparse profile arrays have inconsistent shapes") + return indices, log_magnitudes, phases, coherence + + +def _top_positions(selection: np.ndarray, tie_breaker: np.ndarray, peak_count: int) -> np.ndarray: + """Return deterministic descending positions for the strongest candidates.""" + if len(selection) < peak_count: + raise ValueError(f"profile exposes only {len(selection)} eligible bins") + cutoff = np.partition(selection, -peak_count)[-peak_count] + stronger = np.flatnonzero(selection > cutoff) + tied = np.flatnonzero(selection == cutoff) + remaining = peak_count - len(stronger) + if remaining < len(tied): + tied = tied[np.argpartition(tie_breaker[tied], -remaining)[-remaining:]] + positions = np.concatenate((stronger, tied)) + order = np.lexsort((-tie_breaker[positions], -selection[positions])) + return positions[order] + + +def _dense_candidates( + artifact: np.lib.npyio.NpzFile, + prefix: str, + *, + height: int, + width: int, + min_radius: float, + peak_count: int, +) -> _CarrierCandidates: + """Select candidate arrays from one dense numeric profile.""" + shape = (height, width // 2 + 1, 3) + log_magnitudes = np.asarray(artifact[f"{prefix}mag"]) + phases = np.asarray(artifact[f"{prefix}phase"]) + coherence = np.asarray(artifact[f"{prefix}cons"], dtype=np.float64) / 255.0 + if not (log_magnitudes.shape == phases.shape == coherence.shape == shape): + raise ValueError("dense profile arrays have inconsistent shapes") + + rows = np.arange(height) + signed_rows = np.where(rows > height // 2, rows - height, rows) + columns = np.arange(shape[1]) + radius = np.sqrt(np.square(signed_rows[:, None]) + np.square(columns[None, :])) + valid_spatial = (radius >= min_radius) & (columns[None, :] > 0) + valid = np.broadcast_to(valid_spatial[:, :, None], shape) + flat_valid = np.flatnonzero(valid) + selection = (np.square(coherence) * log_magnitudes * LOG_2).ravel()[flat_valid] + positions = _top_positions(selection, flat_valid, peak_count) + selected = flat_valid[positions] + selected_rows, selected_columns, selected_channels = np.unravel_index(selected, shape) + return _CarrierCandidates( + weights=selection[positions], + rows=selected_rows, + columns=selected_columns, + channels=selected_channels, + phases=np.asarray(phases.ravel()[selected], dtype=np.float64), + magnitudes=np.exp2(np.asarray(log_magnitudes.ravel()[selected], dtype=np.float64)) - 1.0, + ) + + +def _sparse_candidates( + artifact: np.lib.npyio.NpzFile, + prefix: str, + *, + height: int, + width: int, + min_radius: float, + peak_count: int, +) -> _CarrierCandidates: + """Select candidate arrays from one sparse numeric profile.""" + half_width = width // 2 + 1 + selections: list[np.ndarray] = [] + candidate_rows: list[np.ndarray] = [] + candidate_columns: list[np.ndarray] = [] + candidate_channels: list[np.ndarray] = [] + candidate_phases: list[np.ndarray] = [] + candidate_log_magnitudes: list[np.ndarray] = [] + for channel in range(3): + indices, log_magnitudes, phases, coherence = _load_sparse_channel(artifact, prefix, channel) + rows, columns = np.unravel_index(indices, (height, half_width)) + signed_rows = np.where(rows > height // 2, rows - height, rows) + radius = np.sqrt(np.square(signed_rows) + np.square(columns)) + valid = (radius >= min_radius) & (columns > 0) + selections.append((np.square(coherence) * log_magnitudes * LOG_2)[valid]) + candidate_rows.append(rows[valid]) + candidate_columns.append(columns[valid]) + candidate_channels.append(np.full(np.count_nonzero(valid), channel, dtype=np.int8)) + candidate_phases.append(phases[valid]) + candidate_log_magnitudes.append(log_magnitudes[valid]) + + selection = np.concatenate(selections) + rows = np.concatenate(candidate_rows) + columns = np.concatenate(candidate_columns) + channels = np.concatenate(candidate_channels) + tie_breaker = np.ravel_multi_index((rows, columns, channels), (height, half_width, 3)) + selected = _top_positions(selection, tie_breaker, peak_count) + return _CarrierCandidates( + weights=selection[selected], + rows=rows[selected], + columns=columns[selected], + channels=channels[selected], + phases=np.asarray(np.concatenate(candidate_phases)[selected], dtype=np.float64), + magnitudes=np.exp2(np.asarray(np.concatenate(candidate_log_magnitudes)[selected], dtype=np.float64)) - 1.0, + ) + + +def load_v3_model( + path: Path, + *, + height: int, + width: int, + peak_count: int = 256, + min_radius: float = 15.0, +) -> V3CarrierModel: + """Load top phase-consistent bins from a numeric V3 codebook profile.""" + prefix = f"{height}x{width}/" + with np.load(path, allow_pickle=False) as artifact: + if int(artifact["format_version"]) != 2: + raise ValueError("only numeric V3 format version 2 is supported") + loader = _sparse_candidates if bool(int(artifact[f"{prefix}sparse"])) else _dense_candidates + candidates = loader( + artifact, + prefix, + height=height, + width=width, + min_radius=min_radius, + peak_count=peak_count, + ) + return V3CarrierModel( + height=height, + width=width, + rows=np.asarray(candidates.rows, dtype=np.int32), + columns=np.asarray(candidates.columns, dtype=np.int32), + channels=np.asarray(candidates.channels, dtype=np.int8), + phases=candidates.phases, + weights=candidates.weights / np.sum(candidates.weights), + expected_magnitudes=candidates.magnitudes, + ) + + +def _load_profile_rgb(path: Path, model: V3CarrierModel) -> np.ndarray: + """Load PATH and resize only when it does not match the profile geometry.""" + with Image.open(path) as source: + image = source.convert("RGB") + if image.size != (model.width, model.height): + image = image.resize((model.width, model.height), Image.Resampling.LANCZOS) + return np.asarray(image, dtype=np.float64) + + +def _frequency_values(path: Path, model: V3CarrierModel) -> np.ndarray: + """Return the selected complex coefficients from PATH.""" + pixels = _load_profile_rgb(path, model) + return extract_frequency_values(pixels, model.rows, model.columns, model.channels) + + +def _score_values( + values: np.ndarray, + model: V3CarrierModel, + phase_offsets: np.ndarray | float = 0.0, +) -> tuple[float, float, float]: + """Return phase, axial-phase, and active-weight scores for VALUES.""" + phase_difference = np.angle(values) - model.phases + magnitude_gate = np.minimum(np.abs(values) / (model.expected_magnitudes + 1e-12), 1.0) + active_weights = model.weights * magnitude_gate + active_weight = float(np.sum(active_weights)) + if active_weight == 0.0: + phase_score = 0.0 + axial_score = 0.0 + else: + adjusted = phase_difference + phase_offsets + phase_score = float(np.sum(active_weights * np.cos(adjusted)) / active_weight) + axial_score = float(np.sum(active_weights * np.cos(2.0 * adjusted)) / active_weight) + return phase_score, axial_score, active_weight + + +def score_image(path: Path, model: V3CarrierModel) -> V3Score: + """Score PATH against selected V3 phase bins.""" + phase_score, axial_score, active_weight = _score_values(_frequency_values(path, model), model) + return V3Score( + path=str(path), + phase_score=phase_score, + axial_phase_score=axial_score, + active_weight_fraction=active_weight, + peak_count=len(model.rows), + ) + + +def score_translations(path: Path, model: V3CarrierModel, *, max_shift: int = 4) -> V3RegisteredScore: + """Return the best score after compensating bounded integer translations.""" + values = _frequency_values(path, model) + registration = register_phase_translations( + values, + phases=model.phases, + weights=model.weights, + expected_magnitudes=model.expected_magnitudes, + rows=model.rows, + columns=model.columns, + height=model.height, + width=model.width, + max_shift=max_shift, + ) + selected_adjustment = phase_adjustment( + model.rows, + model.columns, + height=model.height, + width=model.width, + row_shift=registration.row_shift, + column_shift=registration.column_shift, + ) + phase_score, axial_score, active_weight = _score_values(values, model, selected_adjustment) + return V3RegisteredScore( + path=str(path), + phase_score=phase_score, + axial_phase_score=axial_score, + active_weight_fraction=active_weight, + peak_count=len(model.rows), + row_shift=registration.row_shift, + column_shift=registration.column_shift, + ) + + +@click.command() +@click.argument("codebook", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--height", type=click.IntRange(min=64), required=True) +@click.option("--width", type=click.IntRange(min=64), required=True) +@click.option("--peak-count", type=click.IntRange(min=1), default=256, show_default=True) +@click.option("--max-shift", type=click.IntRange(min=0, max=MAX_TRANSLATION_SHIFT), default=0, show_default=True) +@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path), required=True) +def main( + codebook: Path, + images: tuple[Path, ...], + height: int, + width: int, + peak_count: int, + max_shift: int, + report_out: Path, +) -> None: + """Score IMAGES against one exact-resolution profile from CODEBOOK.""" + model = load_v3_model(codebook, height=height, width=width, peak_count=peak_count) + scores = ( + [asdict(score_image(image, model)) for image in images] + if max_shift == 0 + else [asdict(score_translations(image, model, max_shift=max_shift)) for image in images] + ) + payload = { + "codebook": str(codebook), + "height": height, + "width": width, + "peak_count": peak_count, + "max_shift": max_shift, + "scores": scores, + } + report_out.parent.mkdir(parents=True, exist_ok=True) + report_out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") + log.info("Wrote V3 score report: %s", report_out) + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(message)s") + main() diff --git a/src/remove_ai_watermarks/__init__.py b/src/remove_ai_watermarks/__init__.py index e3608e5..3f95d54 100644 --- a/src/remove_ai_watermarks/__init__.py +++ b/src/remove_ai_watermarks/__init__.py @@ -13,6 +13,7 @@ High-level API (lazy, so ``import remove_ai_watermarks`` stays cheap):: raiw.remove_video_metadata("in.mp4", "out.mp4") # verified metadata strip raiw.remove_video_invisible("in.mp4", "out.mp4") # oracle-certified SynthID removal raiw.remove_video_visible("in.mp4", "out.mp4") # stable visible video-mark removal + raiw.verify_openai_synthid("in.png", acknowledge_upload=True) # remote For a provenance verdict use the ``identify`` submodule:: @@ -38,6 +39,8 @@ __all__ = [ "BatchSummary", "InvisibleOptions", "MetadataStripIncomplete", + "OpenAIProvenanceError", + "OpenAISynthIDDetection", "RemoveAllResult", "__version__", "identify_video", @@ -50,6 +53,7 @@ __all__ = [ "remove_video_metadata", "remove_video_visible", "remove_visible", + "verify_openai_synthid", "visible_provenance", ] @@ -64,6 +68,11 @@ if TYPE_CHECKING: remove_visible, visible_provenance, ) + from remove_ai_watermarks.openai_provenance import ( + OpenAIProvenanceError, + OpenAISynthIDDetection, + verify_openai_synthid, + ) from remove_ai_watermarks.video import ( identify_video, inspect_video_metadata, @@ -103,4 +112,8 @@ def __getattr__(name: str) -> object: from remove_ai_watermarks import video return getattr(video, name) + if name in ("OpenAIProvenanceError", "OpenAISynthIDDetection", "verify_openai_synthid"): + from remove_ai_watermarks import openai_provenance + + return getattr(openai_provenance, name) raise AttributeError(f"module {__name__!r} has no attribute {name!r}") diff --git a/src/remove_ai_watermarks/assets/__init__.py b/src/remove_ai_watermarks/assets/__init__.py index a816eee..f480ba0 100644 --- a/src/remove_ai_watermarks/assets/__init__.py +++ b/src/remove_ai_watermarks/assets/__init__.py @@ -1 +1 @@ -"""Embedded assets for visible watermark removal.""" +"""Embedded runtime assets for watermark detection and removal.""" diff --git a/src/remove_ai_watermarks/cli.py b/src/remove_ai_watermarks/cli.py index b443a96..be976d3 100644 --- a/src/remove_ai_watermarks/cli.py +++ b/src/remove_ai_watermarks/cli.py @@ -326,8 +326,8 @@ _force_option = click.option( help=( "Run the diffusion scrub even when no invisible AI watermark is locally " "detectable. Default: skip it (regeneration only degrades a clean image; a " - "skip never claims the image is watermark-free -- a pixel SynthID is " - "undetectable once its metadata proxy is gone)." + "skip never claims the image is watermark-free -- this package has no local " + "SynthID pixel decoder)." ), ) _cpu_offload_option = click.option( @@ -505,18 +505,18 @@ def _no_invisible_signal_exit(source: Path) -> NoReturn: :func:`identify` finds no locally-detectable invisible AI signal, running it anyway would damage a clean image for nothing -- the dominant paid score-0 cause on no-watermark uploads. So skip it, but do NOT imply the image is - clean: a pixel SynthID is undetectable here once its metadata proxy is gone. - Write no output and exit :data:`EXIT_NO_INVISIBLE_SIGNAL`; ``--force`` runs - the scrub regardless. + clean: Google does not publish the SynthID payload decoder, and this package + does not ship one, so a mark can still be present after its metadata proxy + is gone. Write no output and exit :data:`EXIT_NO_INVISIBLE_SIGNAL`; + ``--force`` runs the scrub regardless. """ console.print( - " No invisible AI watermark detected (no C2PA/SynthID provenance, no open\n" + " No supported invisible AI watermark detected (no provenance or open\n" " watermark). Skipped the diffusion scrub -- regenerating the pixels would\n" " only degrade the image with nothing to remove, so no output was written.\n" - " This does NOT prove the image is clean: a pixel watermark such as SynthID\n" - " cannot be detected here once its metadata proxy is absent (it may have\n" - " been stripped earlier). If you know the image is AI-generated and want the\n" - " pixels regenerated regardless, re-run with --force:\n" + " This does NOT prove the image is clean: this package has no local SynthID\n" + " pixel decoder. If you know the image is AI-generated and want the pixels\n" + " regenerated regardless, re-run with --force:\n" f" remove-ai-watermarks invisible {source.name} --force" ) raise SystemExit(EXIT_NO_INVISIBLE_SIGNAL) @@ -1410,6 +1410,52 @@ def cmd_video_batch( raise SystemExit(1) +# ── Official OpenAI SynthID verification ── +@main.command("verify-openai-synthid") +@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option( + "--acknowledge-upload", + is_flag=True, + help="Confirm upload of a pixel-identical, AI-metadata-stripped copy to OpenAI.", +) +@click.option("--json", "as_json", is_flag=True, help="Emit the verifier result as JSON.") +def cmd_verify_openai_synthid(source: Path, acknowledge_upload: bool, as_json: bool) -> None: + """Use OpenAI's official verifier on pixels, independently of C2PA. + + The command strips AI provenance metadata from a temporary copy, proves the + decoded pixels are unchanged, and uploads that copy to OpenAI. It reads only + the SynthID result. The source file is never modified. + """ + if not acknowledge_upload: + raise click.ClickException( + "this command uploads a temporary pixel-identical copy to OpenAI; pass --acknowledge-upload to continue" + ) + from remove_ai_watermarks.openai_provenance import verify_openai_synthid + + source = _validate_image(source) + try: + result = verify_openai_synthid(source, acknowledge_upload=True) + except (OSError, RuntimeError, ValueError) as exc: + raise click.ClickException(str(exc)) from exc + + if as_json: + click.echo(json.dumps(result.to_dict(), indent=2)) + return + + _banner() + console.print(f"\n OpenAI SynthID pixel watermark: {result.status}") + console.print(" Detector: official OpenAI Content Provenance API") + if result.model is not None: + console.print(f" Model: {result.model}") + if result.generated_at is not None: + console.print(f" Generated at: {result.generated_at}") + console.print( + " Input: AI provenance metadata was stripped and decoded pixels were preserved.\n" + " Scope: supported OpenAI SynthID only. A not_detected result is not proof\n" + " that the image is human-created or contains no other watermark." + ) + + # ── Provenance identification ── @main.command("identify") @click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) @@ -1461,9 +1507,8 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo if report.is_ai_generated is None: console.print( " No locally-readable AI signal found. This is not the same as 'clean': " - "metadata is often stripped by re-encoding, screenshots, or upload, and SynthID-class " - "pixel watermarks (Gemini / Nano Banana / gpt-image) have no local detector. " - "See caveats below." + "metadata is often stripped by re-encoding, screenshots, or upload, and this " + "package has no local SynthID pixel decoder. See caveats below." ) if report.integrity_clashes: @@ -1601,8 +1646,8 @@ def cmd_all( stage_text = { ("invisible", "no-signal"): ( "Skipped (no invisible AI watermark detected; pixels left intact).\n" - " Not a clean-image guarantee: a pixel SynthID is undetectable once its\n" - " metadata proxy is gone. Re-run with --force to scrub regardless." + " Not a clean-image guarantee: this package has no local SynthID pixel\n" + " decoder. Re-run with --force to scrub regardless." ), ("invisible", "unavailable"): ( f"Warning: Skipped - GPU dependencies not installed.\n Install them with: pip install {INVISIBLE_EXTRA}" diff --git a/src/remove_ai_watermarks/metadata.py b/src/remove_ai_watermarks/metadata.py index 02beb63..78914f2 100644 --- a/src/remove_ai_watermarks/metadata.py +++ b/src/remove_ai_watermarks/metadata.py @@ -803,8 +803,9 @@ def synthid_source(image_path: Path, *, c2pa_info: dict[str, Any] | None = None) None. The evidence is readable only while the C2PA manifest is intact. Absence is - not proof: C2PA can be stripped while the pixel watermark survives, and the - pixel watermark itself is not locally detectable (proprietary decoder). + not proof: C2PA can be stripped while the pixel watermark survives. This + metadata helper does not call the separate, geometry-limited local carrier + detector. Args: image_path: Path to the image (PNG, JPEG, WebP, or ISOBMFF container). diff --git a/src/remove_ai_watermarks/openai_provenance.py b/src/remove_ai_watermarks/openai_provenance.py new file mode 100644 index 0000000..976ed5a --- /dev/null +++ b/src/remove_ai_watermarks/openai_provenance.py @@ -0,0 +1,351 @@ +"""Official OpenAI SynthID verification with metadata-independent input. + +The Content Provenance API returns C2PA and SynthID outcomes independently. +This module removes AI provenance metadata before upload, proves that the +decoded RGBA raster did not change, and then consumes only the SynthID result. +It is intentionally separate from :func:`identify`: calling it uploads one +sanitized raster to OpenAI and therefore always requires an explicit user +action. + +The OpenAI SDK is optional. Imports remain lazy so local and metadata-only +paths do not acquire a network client dependency. +""" + +from __future__ import annotations + +import hashlib +import importlib +import json +import logging +import tempfile +import time +from collections.abc import Callable, Mapping +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Literal, cast + +log = logging.getLogger(__name__) + +OpenAISynthIDStatus = Literal["detected", "not_detected"] + +DETECTOR_ID = "openai-content-provenance-synthid-v1" +INSTALL_HINT = "install the verification extra: uv add 'remove-ai-watermarks[verify]'" +MAX_UPLOAD_BYTES = 50 * 1024 * 1024 +REQUEST_TIMEOUT_SECONDS = 120.0 +# One acknowledgement authorizes one upload. The SDK otherwise retries some +# failures by default, which can transmit the same media more than once. +MAX_AUTOMATIC_RETRIES = 0 +_FORMAT_DETAILS = { + "JPEG": ("image/jpeg", ".jpg"), + "PNG": ("image/png", ".png"), + "WEBP": ("image/webp", ".webp"), +} + + +@dataclass(frozen=True) +class OpenAISynthIDDetection: + """One official OpenAI pixel-watermark verdict.""" + + status: OpenAISynthIDStatus + model: str | None + generated_at: str | None + api_created_at: int | None + detector: str = DETECTOR_ID + ai_metadata_stripped: bool = True + pixels_preserved: bool = True + signal_family: str = "synthid" + provider_scope: str = "openai" + backend: str = "official-openai-api" + metadata_used_for_verdict: bool = False + + @property + def detected(self) -> bool: + """Whether the official verifier recognized an OpenAI SynthID signal.""" + return self.status == "detected" + + def to_dict(self) -> dict[str, str | int | bool | None]: + """Return a JSON-safe result without a local path or C2PA outcome.""" + return { + "status": self.status, + "model": self.model, + "generated_at": self.generated_at, + "api_created_at": self.api_created_at, + "detector": self.detector, + "ai_metadata_stripped": self.ai_metadata_stripped, + "pixels_preserved": self.pixels_preserved, + "signal_family": self.signal_family, + "provider_scope": self.provider_scope, + "backend": self.backend, + "metadata_used_for_verdict": self.metadata_used_for_verdict, + } + + +class OpenAIProvenanceError(RuntimeError): + """An official verification failure, with retry and support context.""" + + def __init__( + self, + message: str, + *, + status_code: int | None = None, + error_code: str | None = None, + request_id: str | None = None, + retry_after: str | None = None, + retryable: bool = False, + ) -> None: + super().__init__(message) + self.status_code = status_code + self.error_code = error_code + self.request_id = request_id + self.retry_after = retry_after + self.retryable = retryable + + +def is_available() -> bool: + """True when the optional OpenAI SDK is installed.""" + from remove_ai_watermarks.optional_deps import module_available + + return module_available("openai") + + +def _pixel_fingerprint(path: Path) -> tuple[str, str]: + """Return the PIL format and a bounded-memory hash of decoded RGBA pixels.""" + from PIL import Image + + with Image.open(path) as image: + image.load() + image_format = image.format + if image_format not in _FORMAT_DETAILS: + supported = ", ".join(sorted(_FORMAT_DETAILS)) + actual = image_format or "unknown" + raise ValueError(f"OpenAI SynthID verification supports {supported} images; got {actual}") + + digest = hashlib.sha256() + digest.update(f"{image.width}x{image.height}:RGBA\0".encode()) + # Hash bands instead of materializing a second full-image byte string. + for top in range(0, image.height, 128): + bottom = min(top + 128, image.height) + digest.update(image.crop((0, top, image.width, bottom)).convert("RGBA").tobytes()) + return image_format, digest.hexdigest() + + +def _response_mapping(response: Any) -> Mapping[str, Any]: + """Normalize an SDK model or test double to the documented response mapping.""" + if isinstance(response, Mapping): + return cast("Mapping[str, Any]", response) + model_dump = getattr(response, "model_dump", None) + if callable(model_dump): + dumped = model_dump(mode="json") + if isinstance(dumped, Mapping): + return cast("Mapping[str, Any]", dumped) + raise RuntimeError("OpenAI Content Provenance returned an unexpected response type") + + +def _optional_string(entry: Mapping[str, Any], field: str) -> str | None: + value = entry.get(field) + if value is None or isinstance(value, str): + return value + raise RuntimeError(f"OpenAI SynthID result has an invalid {field!r} field") + + +def _parse_synthid_result(payload: Mapping[str, Any]) -> OpenAISynthIDDetection: + """Read exactly one SynthID entry and deliberately ignore C2PA entries.""" + if payload.get("object") != "content_provenance_check": + raise RuntimeError("OpenAI Content Provenance response has an unexpected 'object' field") + raw_results = payload.get("results") + if not isinstance(raw_results, list): + raise RuntimeError("OpenAI Content Provenance response has no results list") + results = cast("list[Any]", raw_results) + synthid_entries: list[Mapping[str, Any]] = [] + for raw_entry in results: + if not isinstance(raw_entry, Mapping): + raise RuntimeError("OpenAI Content Provenance response has an invalid result entry") + entry = cast("Mapping[str, Any]", raw_entry) + result_type = entry.get("type") + if not isinstance(result_type, str): + raise RuntimeError("OpenAI Content Provenance response has a result without a valid type") + if result_type == "synthid": + synthid_entries.append(entry) + if len(synthid_entries) != 1: + raise RuntimeError(f"OpenAI Content Provenance returned {len(synthid_entries)} SynthID results; expected one") + + synthid = synthid_entries[0] + outcome = synthid.get("outcome") + if outcome not in ("detected", "not_detected"): + raise RuntimeError(f"OpenAI SynthID result has an unsupported outcome: {outcome!r}") + created_at = payload.get("created_at") + if created_at is not None and (not isinstance(created_at, int) or isinstance(created_at, bool)): + raise RuntimeError("OpenAI Content Provenance response has an invalid 'created_at' field") + return OpenAISynthIDDetection( + status=outcome, + model=_optional_string(synthid, "model"), + generated_at=_optional_string(synthid, "generated_at"), + api_created_at=created_at, + ) + + +def _default_client() -> Any: + if not is_available(): + raise RuntimeError(f"OpenAI SynthID verification needs the OpenAI SDK; {INSTALL_HINT}") + openai_module = importlib.import_module("openai") + client_factory = cast("Callable[..., Any]", openai_module.OpenAI) + try: + client = client_factory( + timeout=REQUEST_TIMEOUT_SECONDS, + max_retries=MAX_AUTOMATIC_RETRIES, + ) + except Exception as exc: + raise RuntimeError(f"could not initialize the OpenAI client: {exc}") from exc + if not hasattr(client, "content_provenance_checks"): + raise RuntimeError(f"OpenAI SynthID verification needs openai>=2.52.0; {INSTALL_HINT}") + return client + + +def _string_attribute(value: Any) -> str | None: + return value if isinstance(value, str) and value else None + + +def _response_header(exc: Exception, name: str) -> str | None: + response = getattr(exc, "response", None) + headers = getattr(response, "headers", None) + if headers is None: + return None + try: + return _string_attribute(headers.get(name)) + except (AttributeError, TypeError): + return None + + +def _request_error(exc: Exception) -> OpenAIProvenanceError: + raw_status_code = getattr(exc, "status_code", None) + status_code = ( + raw_status_code if isinstance(raw_status_code, int) and not isinstance(raw_status_code, bool) else None + ) + error_code = _string_attribute(getattr(exc, "code", None)) + if error_code is None: + body = getattr(exc, "body", None) + # The SDK types the exception body as Any, so narrowing it leaves a + # Mapping with unknown parameters. The same cast the module already uses + # for response payloads keeps the gate clean here. + if isinstance(body, Mapping): + error_code = _string_attribute(cast("Mapping[str, Any]", body).get("code")) + request_id = _string_attribute(getattr(exc, "request_id", None)) or _response_header(exc, "x-request-id") + retry_after = _response_header(exc, "retry-after") + error_name = type(exc).__name__ + if error_name == "APITimeoutError": + detail = "the OpenAI Content Provenance request timed out" + elif error_name == "APIConnectionError": + detail = "the OpenAI Content Provenance API could not be reached" + elif status_code == 400: + detail = "OpenAI rejected the image as malformed, unsupported, or blocked" + elif status_code == 401: + detail = "OpenAI Content Provenance authentication failed" + elif status_code == 403: + detail = "the OpenAI project is not permitted to use Content Provenance" + elif status_code == 404: + detail = "the OpenAI organization does not have Content Provenance API access" + elif status_code == 429: + detail = "the OpenAI Content Provenance API rate limit was exceeded" + elif status_code is not None and status_code >= 500: + detail = "the OpenAI Content Provenance API returned a temporary server error" + else: + detail = f"OpenAI Content Provenance request failed: {exc}" + if retry_after is not None: + detail += f"; Retry-After: {retry_after}" + if error_code is not None: + detail += f"; error code: {error_code}" + if request_id is not None: + detail += f"; request id: {request_id}" + retryable = ( + error_name in ("APITimeoutError", "APIConnectionError") + or status_code == 429 + or (status_code is not None and status_code >= 500) + ) + return OpenAIProvenanceError( + detail, + status_code=status_code, + error_code=error_code, + request_id=request_id, + retry_after=retry_after, + retryable=retryable, + ) + + +def verify_openai_synthid( + image_path: str | Path, + *, + acknowledge_upload: bool = False, + client: Any | None = None, +) -> OpenAISynthIDDetection: + """Verify OpenAI SynthID after stripping AI metadata without changing pixels. + + This function performs one remote request and uploads a temporary sanitized + copy of the image. It never uses C2PA as a fallback and never interprets a + negative result as proof that the image is human-created. + """ + if not acknowledge_upload: + raise ValueError( + "OpenAI SynthID verification uploads a temporary pixel-identical copy; " + "pass acknowledge_upload=True to continue" + ) + source = Path(image_path) + source_format, source_fingerprint = _pixel_fingerprint(source) + media_type, suffix = _FORMAT_DETAILS[source_format] + + with tempfile.TemporaryDirectory(prefix="remove-ai-watermarks-openai-") as directory: + sanitized = Path(directory) / f"upload{suffix}" + from remove_ai_watermarks.metadata import strip_and_verify + + stripped, remaining = strip_and_verify(source, sanitized, keep_standard=True) + if remaining: + fields = ", ".join(sorted(remaining)) + raise RuntimeError(f"refusing upload because AI provenance metadata survived stripping: {fields}") + stripped_format, stripped_fingerprint = _pixel_fingerprint(stripped) + if stripped_format != source_format or stripped_fingerprint != source_fingerprint: + raise RuntimeError("refusing upload because metadata stripping changed the decoded pixels") + upload_bytes = stripped.stat().st_size + if upload_bytes > MAX_UPLOAD_BYTES: + raise ValueError("sanitized image exceeds the OpenAI Content Provenance 50 MiB upload limit") + + api_client = client if client is not None else _default_client() + if not hasattr(api_client, "content_provenance_checks"): + raise RuntimeError("OpenAI client does not expose content_provenance_checks; openai>=2.52.0 is required") + request_context = { + "endpoint": "/v1/content_provenance_checks", + "filename": sanitized.name, + "media_type": media_type, + "bytes": upload_bytes, + "automatic_retries": MAX_AUTOMATIC_RETRIES, + "timeout_seconds": REQUEST_TIMEOUT_SECONDS, + } + log.info("OpenAI Content Provenance request: %s", json.dumps(request_context, sort_keys=True)) + started_at = time.monotonic() + try: + with stripped.open("rb") as upload: + response = api_client.content_provenance_checks.create( + file=(sanitized.name, upload, media_type), + timeout=REQUEST_TIMEOUT_SECONDS, + ) + except Exception as exc: + failure_context = { + **request_context, + "duration_ms": round((time.monotonic() - started_at) * 1000), + "error_code": _string_attribute(getattr(exc, "code", None)), + "request_id": _string_attribute(getattr(exc, "request_id", None)) + or _response_header(exc, "x-request-id"), + "status_code": getattr(exc, "status_code", None), + } + log.exception( + "OpenAI Content Provenance request failed: %s", + json.dumps(failure_context, default=str, sort_keys=True), + ) + raise _request_error(exc) from exc + + payload = _response_mapping(response) + response_context = { + "duration_ms": round((time.monotonic() - started_at) * 1000), + "payload": payload, + "request_id": _string_attribute(getattr(response, "_request_id", None)), + } + log.info("OpenAI Content Provenance response: %s", json.dumps(response_context, default=str, sort_keys=True)) + return _parse_synthid_result(payload) diff --git a/tests/test_api.py b/tests/test_api.py index ba8d519..5cbfca4 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -18,13 +18,24 @@ CHATGPT = SAMPLES / "chatgpt-1.png" class TestTopLevelExports: def test_lazy_reexports_resolve(self): + from remove_ai_watermarks import openai_provenance + assert raiw.remove_visible is api.remove_visible assert raiw.visible_provenance is api.visible_provenance + assert raiw.verify_openai_synthid is openai_provenance.verify_openai_synthid + assert raiw.OpenAIProvenanceError is openai_provenance.OpenAIProvenanceError + assert raiw.OpenAISynthIDDetection is openai_provenance.OpenAISynthIDDetection def test_unknown_attribute_raises(self): with pytest.raises(AttributeError): _ = raiw.does_not_exist + def test_local_synthid_detector_is_not_a_package_export(self): + with pytest.raises(AttributeError): + _ = raiw.detect_synthid + with pytest.raises(AttributeError): + _ = raiw.SynthIDDetection + def test_bare_import_is_light(self): # importing the package must not pull the heavy cv2/torch stack (PEP 562 lazy). # Checked in a FRESH interpreter -- another test in this process may already diff --git a/tests/test_cli.py b/tests/test_cli.py index eca229f..ad1f00a 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -774,6 +774,89 @@ class TestIdentifyCommand: assert result.exit_code != 0 +class TestDetectSynthIDCommandRemoved: + def test_command_is_not_registered(self, runner): + result = runner.invoke(main, ["detect-synthid", "--help"]) + assert result.exit_code != 0 + assert "No such command" in result.output + + +class TestVerifyOpenAISynthIDCommand: + def test_help_names_upload_and_pixel_independence(self, runner): + result = runner.invoke(main, ["verify-openai-synthid", "--help"]) + + assert result.exit_code == 0 + assert "--acknowledge-upload" in result.output + assert "independently of C2PA" in result.output + + def test_upload_requires_explicit_acknowledgement(self, runner, tmp_clean_png, monkeypatch): + from remove_ai_watermarks import openai_provenance + + called = False + + def verify(_source, *, acknowledge_upload): + nonlocal called + assert acknowledge_upload is True + called = True + + monkeypatch.setattr(openai_provenance, "verify_openai_synthid", verify) + + result = runner.invoke(main, ["verify-openai-synthid", str(tmp_clean_png)]) + + assert result.exit_code != 0 + assert "pass --acknowledge-upload" in result.output + assert called is False + + def test_json_result_is_machine_readable(self, runner, tmp_clean_png, monkeypatch): + from remove_ai_watermarks import openai_provenance + + expected = openai_provenance.OpenAISynthIDDetection( + status="not_detected", + model=None, + generated_at=None, + api_created_at=1_778_000_000, + ) + monkeypatch.setattr( + openai_provenance, + "verify_openai_synthid", + lambda _source, *, acknowledge_upload: expected if acknowledge_upload else None, + ) + + result = runner.invoke( + main, + ["verify-openai-synthid", str(tmp_clean_png), "--acknowledge-upload", "--json"], + ) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload == expected.to_dict() + assert "c2pa" not in payload + + def test_text_result_preserves_negative_scope(self, runner, tmp_clean_png, monkeypatch): + from remove_ai_watermarks import openai_provenance + + expected = openai_provenance.OpenAISynthIDDetection( + status="not_detected", + model=None, + generated_at=None, + api_created_at=None, + ) + monkeypatch.setattr( + openai_provenance, + "verify_openai_synthid", + lambda _source, *, acknowledge_upload: expected if acknowledge_upload else None, + ) + + result = runner.invoke( + main, + ["verify-openai-synthid", str(tmp_clean_png), "--acknowledge-upload"], + ) + + assert result.exit_code == 0, result.output + assert "AI provenance metadata was stripped" in result.output + assert "not proof" in result.output + + class TestBatchCommand: """Tests for the 'batch' subcommand.""" diff --git a/tests/test_openai_provenance.py b/tests/test_openai_provenance.py new file mode 100644 index 0000000..77f9ee6 --- /dev/null +++ b/tests/test_openai_provenance.py @@ -0,0 +1,429 @@ +"""Contract tests for metadata-independent official OpenAI SynthID verification.""" + +from __future__ import annotations + +import io +from types import SimpleNamespace +from typing import TYPE_CHECKING, Any + +import pytest +from PIL import Image + +from remove_ai_watermarks import openai_provenance as provenance + +if TYPE_CHECKING: + from pathlib import Path + + +class _Checks: + def __init__(self, response: Any) -> None: + self.response = response + self.calls: list[tuple[str, bytes, str]] = [] + + def create(self, *, file: tuple[str, Any, str], timeout: float) -> Any: + assert timeout == provenance.REQUEST_TIMEOUT_SECONDS + filename, stream, media_type = file + self.calls.append((filename, stream.read(), media_type)) + return self.response + + +def _client(response: Any) -> tuple[Any, _Checks]: + checks = _Checks(response) + return SimpleNamespace(content_provenance_checks=checks), checks + + +def _response(*, synthid: str, c2pa: str = "not_detected") -> dict[str, Any]: + return { + "object": "content_provenance_check", + "created_at": 1_778_000_000, + "results": [ + { + "type": "c2pa", + "outcome": c2pa, + "validation_state": "trusted" if c2pa == "detected" else "not_present", + "issuer": "OpenAI OpCo, LLC" if c2pa == "detected" else None, + "model": "metadata-model" if c2pa == "detected" else None, + "generated_at": "2026-07-27T18:34:12Z" if c2pa == "detected" else None, + }, + { + "type": "synthid", + "outcome": synthid, + "model": "pixel-model" if synthid == "detected" else None, + "generated_at": "2026-07-28T18:34:12Z" if synthid == "detected" else None, + }, + ], + } + + +def _verify(image_path: Path, *, client: Any | None = None) -> provenance.OpenAISynthIDDetection: + return provenance.verify_openai_synthid(image_path, acknowledge_upload=True, client=client) + + +def test_upload_requires_explicit_library_acknowledgement(tmp_clean_png: Path) -> None: + with pytest.raises(ValueError, match="acknowledge_upload=True"): + provenance.verify_openai_synthid(tmp_clean_png) + + +def test_c2pa_only_response_is_not_a_synthid_detection(tmp_png_with_ai_metadata: Path) -> None: + client, checks = _client(_response(synthid="not_detected", c2pa="detected")) + + result = _verify(tmp_png_with_ai_metadata, client=client) + + assert result.status == "not_detected" + assert result.model is None + assert result.generated_at is None + assert result.ai_metadata_stripped is True + assert result.pixels_preserved is True + assert len(checks.calls) == 1 + filename, uploaded, media_type = checks.calls[0] + assert filename == "upload.png" + assert media_type == "image/png" + with Image.open(io.BytesIO(uploaded)) as image: + image.load() + assert image.convert("RGBA").getpixel((0, 0)) == (128, 128, 128, 255) + assert "parameters" not in image.info + assert "prompt" not in image.info + + +def test_detected_result_uses_only_synthid_fields(tmp_clean_png: Path) -> None: + client, _checks = _client(_response(synthid="detected", c2pa="not_detected")) + + result = _verify(tmp_clean_png, client=client) + + assert result.detected is True + assert result.model == "pixel-model" + assert result.generated_at == "2026-07-28T18:34:12Z" + assert result.api_created_at == 1_778_000_000 + assert "c2pa" not in result.to_dict() + assert result.to_dict()["metadata_used_for_verdict"] is False + assert result.to_dict()["provider_scope"] == "openai" + + +def test_sdk_model_response_is_normalized(tmp_clean_png: Path) -> None: + class SDKModel: + def model_dump(self, *, mode: str) -> dict[str, Any]: + assert mode == "json" + return _response(synthid="detected") + + client, _checks = _client(SDKModel()) + + result = _verify(tmp_clean_png, client=client) + + assert result.status == "detected" + + +def test_unexpected_response_object_is_an_error(tmp_clean_png: Path) -> None: + response = _response(synthid="detected") + response["object"] = "future_response" + client, _checks = _client(response) + + with pytest.raises(RuntimeError, match="unexpected 'object'"): + _verify(tmp_clean_png, client=client) + + +@pytest.mark.parametrize("entry", [None, {"outcome": "detected"}, {"type": 3, "outcome": "detected"}]) +def test_malformed_result_entry_is_an_error(tmp_clean_png: Path, entry: Any) -> None: + client, _checks = _client( + { + "object": "content_provenance_check", + "results": [entry], + } + ) + + with pytest.raises(RuntimeError, match=r"invalid result entry|valid type"): + _verify(tmp_clean_png, client=client) + + +@pytest.mark.parametrize( + ("image_format", "suffix", "media_type"), + [("PNG", ".png", "image/png"), ("JPEG", ".jpg", "image/jpeg"), ("WEBP", ".webp", "image/webp")], +) +def test_all_documented_image_formats_preserve_decoded_pixels( + tmp_path: Path, + image_format: str, + suffix: str, + media_type: str, +) -> None: + source = tmp_path / f"source{suffix}" + image = Image.new("RGB", (19, 17)) + image.putdata([((x * 13) % 256, (x * 29) % 256, (x * 47) % 256) for x in range(19 * 17)]) + image.save(source, format=image_format, quality=91) + with Image.open(source) as decoded: + expected = decoded.convert("RGBA").tobytes() + client, checks = _client(_response(synthid="not_detected")) + + _verify(source, client=client) + + filename, uploaded, actual_media_type = checks.calls[0] + assert filename == f"upload{suffix}" + assert actual_media_type == media_type + with Image.open(io.BytesIO(uploaded)) as decoded: + assert decoded.convert("RGBA").tobytes() == expected + + +@pytest.mark.parametrize("results", [[], [{"type": "c2pa", "outcome": "detected"}]]) +def test_missing_synthid_result_is_an_error(tmp_clean_png: Path, results: list[dict[str, str]]) -> None: + client, _checks = _client({"object": "content_provenance_check", "results": results}) + + with pytest.raises(RuntimeError, match="0 SynthID results"): + _verify(tmp_clean_png, client=client) + + +def test_duplicate_synthid_results_are_an_error(tmp_clean_png: Path) -> None: + client, _checks = _client( + { + "object": "content_provenance_check", + "results": [ + {"type": "synthid", "outcome": "detected"}, + {"type": "synthid", "outcome": "not_detected"}, + ], + } + ) + + with pytest.raises(RuntimeError, match="2 SynthID results"): + _verify(tmp_clean_png, client=client) + + +def test_pixel_mutation_aborts_before_remote_request( + monkeypatch: pytest.MonkeyPatch, + tmp_clean_png: Path, +) -> None: + from remove_ai_watermarks import metadata + + client, checks = _client(_response(synthid="detected")) + + def mutate(source: Path, output: Path, *, keep_standard: bool) -> tuple[Path, dict[str, str]]: + assert keep_standard is True + with Image.open(source) as image: + changed = image.convert("RGB") + changed.putpixel((0, 0), (0, 0, 0)) + changed.save(output) + return output, {} + + monkeypatch.setattr(metadata, "strip_and_verify", mutate) + + with pytest.raises(RuntimeError, match="changed the decoded pixels"): + _verify(tmp_clean_png, client=client) + assert checks.calls == [] + + +def test_surviving_ai_metadata_aborts_before_remote_request( + monkeypatch: pytest.MonkeyPatch, + tmp_clean_png: Path, +) -> None: + from remove_ai_watermarks import metadata + + client, checks = _client(_response(synthid="detected")) + + def survive(source: Path, output: Path, *, keep_standard: bool) -> tuple[Path, dict[str, str]]: + assert keep_standard is True + output.write_bytes(source.read_bytes()) + return output, {"C2PA": "present"} + + monkeypatch.setattr(metadata, "strip_and_verify", survive) + + with pytest.raises(RuntimeError, match="metadata survived"): + _verify(tmp_clean_png, client=client) + assert checks.calls == [] + + +def test_unsupported_image_format_is_rejected_before_remote_request(tmp_path: Path) -> None: + source = tmp_path / "image.bmp" + Image.new("RGB", (16, 16), color=(1, 2, 3)).save(source) + client, checks = _client(_response(synthid="detected")) + + with pytest.raises(ValueError, match="supports JPEG, PNG, WEBP"): + _verify(source, client=client) + assert checks.calls == [] + + +def test_upload_limit_is_checked_after_sanitizing( + monkeypatch: pytest.MonkeyPatch, + tmp_clean_png: Path, +) -> None: + client, checks = _client(_response(synthid="detected")) + monkeypatch.setattr(provenance, "MAX_UPLOAD_BYTES", 1) + + with pytest.raises(ValueError, match="50 MiB"): + _verify(tmp_clean_png, client=client) + assert checks.calls == [] + + +def test_upload_limit_allows_exact_boundary( + monkeypatch: pytest.MonkeyPatch, + tmp_clean_png: Path, +) -> None: + from remove_ai_watermarks import metadata + + client, checks = _client(_response(synthid="not_detected")) + + def copy_clean(source: Path, output: Path, *, keep_standard: bool) -> tuple[Path, dict[str, str]]: + assert keep_standard is True + output.write_bytes(source.read_bytes()) + return output, {} + + monkeypatch.setattr(metadata, "strip_and_verify", copy_clean) + monkeypatch.setattr(provenance, "MAX_UPLOAD_BYTES", tmp_clean_png.stat().st_size) + + result = _verify(tmp_clean_png, client=client) + + assert result.status == "not_detected" + assert len(checks.calls) == 1 + + +def test_missing_optional_sdk_has_install_hint( + monkeypatch: pytest.MonkeyPatch, + tmp_clean_png: Path, +) -> None: + monkeypatch.setattr(provenance, "is_available", lambda: False) + + with pytest.raises(RuntimeError, match=r"remove-ai-watermarks\[verify\]"): + _verify(tmp_clean_png) + + +def test_client_configuration_error_is_actionable( + monkeypatch: pytest.MonkeyPatch, + tmp_clean_png: Path, +) -> None: + def fail(**_kwargs: Any) -> None: + raise ValueError("OPENAI_API_KEY is missing") + + monkeypatch.setattr(provenance, "is_available", lambda: True) + monkeypatch.setattr(provenance.importlib, "import_module", lambda _name: SimpleNamespace(OpenAI=fail)) + + with pytest.raises(RuntimeError, match=r"could not initialize.*OPENAI_API_KEY"): + _verify(tmp_clean_png) + + +def test_default_client_bounds_one_acknowledged_upload(monkeypatch: pytest.MonkeyPatch) -> None: + calls: list[dict[str, Any]] = [] + expected = SimpleNamespace(content_provenance_checks=object()) + + def factory(**kwargs: Any) -> Any: + calls.append(kwargs) + return expected + + monkeypatch.setattr(provenance, "is_available", lambda: True) + monkeypatch.setattr(provenance.importlib, "import_module", lambda _name: SimpleNamespace(OpenAI=factory)) + + assert provenance._default_client() is expected + assert calls == [ + { + "timeout": provenance.REQUEST_TIMEOUT_SECONDS, + "max_retries": 0, + } + ] + + +@pytest.mark.parametrize( + ("status_code", "message"), + [ + (400, "rejected"), + (401, "authentication failed"), + (403, "not permitted"), + (404, "does not have"), + (429, "rate limit"), + (500, "temporary server error"), + ], +) +def test_documented_api_errors_are_actionable( + tmp_clean_png: Path, + status_code: int, + message: str, +) -> None: + class APIError(Exception): + pass + + error = APIError("details") + error.status_code = status_code # type: ignore[attr-defined] + + class FailingChecks: + def create(self, *, file: tuple[str, Any, str], timeout: float) -> None: + assert timeout == provenance.REQUEST_TIMEOUT_SECONDS + raise error + + client = SimpleNamespace(content_provenance_checks=FailingChecks()) + + with pytest.raises(RuntimeError, match=message): + _verify(tmp_clean_png, client=client) + + +@pytest.mark.parametrize( + ("error_name", "message"), + [("APITimeoutError", "timed out"), ("APIConnectionError", "could not be reached")], +) +def test_transport_errors_are_actionable( + tmp_clean_png: Path, + error_name: str, + message: str, +) -> None: + error_type = type(error_name, (Exception,), {}) + + class FailingChecks: + def create(self, *, file: tuple[str, Any, str], timeout: float) -> None: + assert timeout == provenance.REQUEST_TIMEOUT_SECONDS + raise error_type("details") + + client = SimpleNamespace(content_provenance_checks=FailingChecks()) + + with pytest.raises(RuntimeError, match=message): + _verify(tmp_clean_png, client=client) + + +def test_rate_limit_error_preserves_retry_context(tmp_clean_png: Path) -> None: + class RateLimitError(Exception): + status_code = 429 + code = "rate_limit_exceeded" + request_id = "req_test" + response = SimpleNamespace(headers={"retry-after": "7"}) + + class FailingChecks: + def create(self, *, file: tuple[str, Any, str], timeout: float) -> None: + assert timeout == provenance.REQUEST_TIMEOUT_SECONDS + raise RateLimitError("details") + + client = SimpleNamespace(content_provenance_checks=FailingChecks()) + + with pytest.raises(provenance.OpenAIProvenanceError, match="Retry-After: 7") as raised: + _verify(tmp_clean_png, client=client) + assert raised.value.status_code == 429 + assert raised.value.error_code == "rate_limit_exceeded" + assert raised.value.request_id == "req_test" + assert raised.value.retry_after == "7" + assert raised.value.retryable is True + + +def test_client_error_is_not_marked_retryable(tmp_clean_png: Path) -> None: + class BadRequestError(Exception): + status_code = 400 + code = "invalid_image" + + class FailingChecks: + def create(self, *, file: tuple[str, Any, str], timeout: float) -> None: + assert timeout == provenance.REQUEST_TIMEOUT_SECONDS + raise BadRequestError("details") + + client = SimpleNamespace(content_provenance_checks=FailingChecks()) + + with pytest.raises(provenance.OpenAIProvenanceError) as raised: + _verify(tmp_clean_png, client=client) + assert raised.value.status_code == 400 + assert raised.value.error_code == "invalid_image" + assert raised.value.retryable is False + + +def test_keyboard_interrupt_is_not_wrapped_or_retried(tmp_clean_png: Path) -> None: + class InterruptingChecks: + calls = 0 + + def create(self, *, file: tuple[str, Any, str], timeout: float) -> None: + assert timeout == provenance.REQUEST_TIMEOUT_SECONDS + self.calls += 1 + raise KeyboardInterrupt + + checks = InterruptingChecks() + client = SimpleNamespace(content_provenance_checks=checks) + + with pytest.raises(KeyboardInterrupt): + _verify(tmp_clean_png, client=client) + assert checks.calls == 1 diff --git a/tests/test_packaging.py b/tests/test_packaging.py index 5e86c10..50c28e2 100644 --- a/tests/test_packaging.py +++ b/tests/test_packaging.py @@ -61,6 +61,7 @@ def test_video_extra_owns_timestamp_dependency(): def test_file_format_and_detector_dependencies_are_independent(): assert "pillow-heif" in _requirement_names("heif") assert "pywavelets" in _requirement_names("detect") + assert "openai" in _requirement_names("verify") def test_trustmark_is_limited_to_its_numpy_compatible_python_range(): @@ -71,7 +72,7 @@ def test_trustmark_is_limited_to_its_numpy_compatible_python_range(): def test_extras_use_capability_names_without_legacy_aliases(): extras = set(metadata("remove-ai-watermarks").get_all("Provides-Extra") or []) - assert {"pixels", "heif", "visible", "video", "detect", "diffusion", "classify"} <= extras + assert {"pixels", "heif", "visible", "video", "detect", "diffusion", "classify", "verify"} <= extras assert {"gpu", "remove", "detect-pywavelets"}.isdisjoint(extras) diff --git a/tests/test_synthid_adaptive_carrier_suppress.py b/tests/test_synthid_adaptive_carrier_suppress.py new file mode 100644 index 0000000..0db0a65 --- /dev/null +++ b/tests/test_synthid_adaptive_carrier_suppress.py @@ -0,0 +1,77 @@ +"""Tests for the research adaptive periodic-carrier suppressor.""" + +from __future__ import annotations + +import sys +from pathlib import Path +from typing import TYPE_CHECKING, Any + +import numpy as np +import pytest + +if TYPE_CHECKING: + from numpy.typing import NDArray + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_adaptive_carrier_suppress as suppressor # pyright: ignore[reportMissingImports] + + +def repeated_template(height: int, width: int, amplitude: float) -> NDArray[Any]: + """Return a synthetic uint8 image carrying the bundled periodic template.""" + template, _sigma, *_model = suppressor._load_template() + repeats_y = (height + template.shape[0] - 1) // template.shape[0] + repeats_x = (width + template.shape[1] - 1) // template.shape[1] + carrier = np.tile(template, (repeats_y, repeats_x, 1))[:height, :width] + return np.clip(np.rint(128.0 + amplitude * carrier), 0, 255).astype(np.uint8) + + +def test_apply_template_handles_nondivisible_geometry() -> None: + template, _sigma, *_model = suppressor._load_template() + pixels = np.full((65, 67, 3), 128, dtype=np.uint8) + + candidate = suppressor.apply_template(pixels, template, amplitude=8.0) + + assert candidate.shape == pixels.shape + assert candidate.dtype == np.uint8 + assert np.any(candidate != pixels) + + +def test_find_minimum_amplitude_reaches_target() -> None: + template, sigma, *_model = suppressor._load_template() + pixels = repeated_template(128, 130, 80.0) + + amplitude, candidate, score = suppressor.find_minimum_amplitude( + pixels, + template, + sigma, + target_score=-0.25, + maximum_amplitude=160.0, + iterations=10, + ) + + assert 0.0 < amplitude <= 160.0 + assert score <= -0.25 + assert suppressor.carrier_score(candidate, template, sigma) == pytest.approx(score) + + +def test_find_minimum_amplitude_rejects_unreachable_target() -> None: + template, sigma, *_model = suppressor._load_template() + pixels = repeated_template(128, 128, 80.0) + + with pytest.raises(ValueError, match="maximum amplitude"): + suppressor.find_minimum_amplitude( + pixels, + template, + sigma, + target_score=-0.25, + maximum_amplitude=1.0, + iterations=8, + ) + + +def test_suppress_carrier_refuses_local_negative() -> None: + pixels = np.full((1000, 1000, 3), 128, dtype=np.uint8) + + with pytest.raises(ValueError, match="not detected"): + suppressor.suppress_carrier(pixels) diff --git a/tests/test_synthid_affine_lattice_probe.py b/tests/test_synthid_affine_lattice_probe.py new file mode 100644 index 0000000..397be0e --- /dev/null +++ b/tests/test_synthid_affine_lattice_probe.py @@ -0,0 +1,398 @@ +from __future__ import annotations + +import json +import sys +from dataclasses import replace +from pathlib import Path + +import cv2 +import numpy as np +import pytest +from click.testing import CliRunner +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_affine_lattice_probe as probe +from synthid_runtime._synthid_confirmation import RegisteredConfirmationComponents + + +def test_webp_lossless_round_trip_preserves_pixels() -> None: + rng = np.random.default_rng(20260817) + pixels = rng.integers(0, 256, (64, 64, 3), dtype=np.uint8) + + restored = probe._webp_round_trip(pixels, 101) + + assert np.array_equal(restored, pixels) + + +@pytest.fixture(scope="module") +def periodic_fixture() -> tuple[np.ndarray, np.ndarray]: + rng = np.random.default_rng(20260814) + template = rng.normal(0.0, 1.0, (16, 16, 3)) + template -= np.mean(template, axis=(0, 1), keepdims=True) + template /= np.linalg.norm(template) + coarse = rng.normal(0.0, 8.0, (16, 16, 3)).astype(np.float32) + background = cv2.resize(coarse, (1024, 1024), interpolation=cv2.INTER_CUBIC) + 128.0 + carrier = np.tile(template, (64, 64, 1)) * 3.0 + pixels = np.clip(np.rint(background + carrier), 0, 255).astype(np.uint8) + return pixels, template + + +def _score(pixels: np.ndarray, template: np.ndarray) -> probe.LatticeScore: + return probe.score_lattice( + pixels, + template, + periods=np.arange(12.0, 20.01, 0.25), + rotations_degrees=np.asarray([-1.0, 0.0, 1.0]), + patch_size=256, + grid_size=4, + harmonic_count=12, + ) + + +def test_period_alias_candidates_include_base_and_half_period_neighbors() -> None: + periods = np.arange(7.5, 24.501, 0.1) + rotations = np.zeros_like(periods) + base_index = int(np.argmin(np.abs(periods - 19.2))) + + candidates = probe._period_alias_candidate_indices(periods, rotations, [base_index]) + + assert [periods[index] for index in candidates] == pytest.approx([19.1, 19.2, 19.3, 9.5, 9.6, 9.7]) + + +def test_split_lattice_recovers_periodic_carrier(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + pixels, template = periodic_fixture + + result = _score(pixels, template) + + assert result.selected_period == pytest.approx(16.0, abs=0.25) + assert result.selected_rotation_degrees == 0.0 + assert result.confirmation_coherence > 0.9 + assert result.joint_coherence > 0.9 + assert result.joint_codeword > 0.8 + assert result.unknown_codeword_confirmation > 0.8 + assert result.unknown_codeword_fixed_confirmation > 0.8 + assert result.unknown_codeword_fixed_all > 0.8 + assert result.unknown_codeword_excess_p99 > 0.0 + assert result.joint_amplitude > 0.8 + assert result.joint_whitened_match > 0.8 + assert result.canonical_template_score > 0.8 + assert result.canonical_registered_template_score > 0.8 + assert result.confirmation_excess_p99 > 0.0 + assert result.selection_patches == result.confirmation_patches == 8 + + +def test_split_lattice_rejects_independent_noise(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + _pixels, template = periodic_fixture + rng = np.random.default_rng(20260815) + noise = rng.integers(0, 256, (1024, 1024, 3), dtype=np.uint8) + + result = _score(noise, template) + + assert result.confirmation_coherence < 0.8 + assert result.joint_coherence < 0.8 + assert result.joint_codeword < 0.8 + assert result.unknown_codeword_confirmation < 0.2 + assert result.unknown_codeword_fixed_confirmation < 0.2 + assert result.unknown_codeword_fixed_all < 0.2 + assert result.joint_amplitude < 0.2 + assert result.joint_whitened_match < 0.2 + assert result.canonical_template_score < 0.2 + assert result.canonical_registered_template_score < 0.2 + assert result.confirmation_excess_p99 < 0.0 + + +def test_split_lattice_tracks_resized_period(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + pixels, template = periodic_fixture + resized = cv2.resize(pixels, (819, 819), interpolation=cv2.INTER_CUBIC) + + result = probe.score_lattice( + resized, + template, + periods=np.arange(7.5, 24.501, 0.1), + rotations_degrees=np.asarray([0.0]), + patch_size=192, + grid_size=4, + harmonic_count=12, + ) + + assert result.selected_period == pytest.approx(12.8, abs=0.3) + assert result.confirmation_coherence > 0.8 + assert result.joint_amplitude > 0.8 + assert result.joint_whitened_match > 0.8 + assert result.unknown_codeword_confirmation > 0.8 + assert result.unknown_codeword_fixed_confirmation > 0.8 + assert result.unknown_codeword_fixed_all > 0.8 + + +def test_split_lattice_tracks_octave_aliased_resize(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + pixels, template = periodic_fixture + resized = cv2.resize(pixels, (614, 614), interpolation=cv2.INTER_AREA) + + result = probe.score_lattice( + resized, + template, + periods=np.arange(7.5, 24.501, 0.1), + rotations_degrees=np.asarray([0.0]), + patch_size=192, + grid_size=4, + harmonic_count=12, + ) + + assert result.selected_period == pytest.approx(9.6, abs=0.15) + assert result.canonical_template_score > 0.4 + + +def test_same_image_period_null_prefers_the_carrier_period( + periodic_fixture: tuple[np.ndarray, np.ndarray], +) -> None: + pixels, template = periodic_fixture + + correct = probe.score_same_image_period_null(pixels, template, 16.0, harmonic_count=12) + off_period = probe.score_same_image_period_null(pixels, template, 15.0, harmonic_count=12) + + assert correct.joint_excess > 0.2 + assert correct.joint_excess > off_period.joint_excess + assert correct.off_period_count == len(probe.SAME_IMAGE_NULL_OFFSETS) + + +def test_patch_shift_consensus_confirms_global_carrier_phase( + periodic_fixture: tuple[np.ndarray, np.ndarray], +) -> None: + pixels, template = periodic_fixture + rng = np.random.default_rng(20260818) + noise = rng.integers(0, 256, pixels.shape, dtype=np.uint8) + + carrier = probe.score_patch_shift_consensus(pixels, template, 16.0) + control = probe.score_patch_shift_consensus(noise, template, 16.0) + + assert carrier.joint_trimmed_z > control.joint_trimmed_z + assert carrier.joint_support_fraction == 1.0 + assert carrier.selection_patches == carrier.confirmation_patches == 8 + + +def test_patch_shift_recovery_uses_frozen_mechanism_gates() -> None: + baseline = { + "amplitude_margin": 0.8, + "high_band_margin": 1.0, + "periods_agree": True, + "confirmation_passes": True, + "joint_trimmed_z": 2.5, + } + + assert probe.patch_shift_recovery_passes(**baseline) + for field, failed_value in ( + ("amplitude_margin", 0.449), + ("high_band_margin", 0.449), + ("periods_agree", False), + ("confirmation_passes", False), + ("joint_trimmed_z", 2.499), + ): + candidate = {**baseline, field: failed_value} + assert not probe.patch_shift_recovery_passes(**candidate) + assert not probe.patch_shift_recovery_passes(**{**baseline, "amplitude_margin": 0.99, "high_band_margin": 0.99}) + + +def test_opponent_registration_recovers_resampled_carrier( + periodic_fixture: tuple[np.ndarray, np.ndarray], +) -> None: + pixels, template = periodic_fixture + resized = cv2.resize(pixels, (717, 717), interpolation=cv2.INTER_AREA) + rng = np.random.default_rng(20260819) + noise = rng.integers(0, 256, resized.shape, dtype=np.uint8) + periods = np.arange(10.0, 12.41, 0.05) + + carrier = probe.score_opponent_registered(resized, template, periods=periods) + control = probe.score_opponent_registered(noise, template, periods=periods) + + assert carrier.selected_period == pytest.approx(11.2, abs=0.1) + assert carrier.decision_score > 1.0 + assert control.decision_score < 1.0 + + +def test_split_lattice_aligns_cyclic_carrier_phase(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + pixels, template = periodic_fixture + shifted = np.roll(pixels, shift=(3, 5), axis=(0, 1)) + + result = _score(shifted, template) + + assert (result.selected_shift_y, result.selected_shift_x) == (3, 5) + assert (result.amplitude_shift_y, result.amplitude_shift_x) == (13, 11) + assert result.canonical_template_score < 0.2 + assert result.canonical_registered_template_score > 0.8 + assert result.joint_whitened_match < 0.4 + assert result.unknown_codeword_confirmation > 0.8 + assert result.unknown_codeword_fixed_confirmation > 0.8 + assert result.unknown_codeword_fixed_all > 0.8 + + +def test_split_lattice_recovers_cropped_carrier_phase(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + pixels, template = periodic_fixture + cropped = pixels[37:, 53:] + + result = probe.score_lattice( + cropped, + template, + periods=np.asarray([16.0]), + rotations_degrees=np.asarray([0.0]), + patch_size=256, + grid_size=4, + harmonic_count=12, + ) + + assert result.selected_period == pytest.approx(16.0, abs=0.25) + assert result.joint_coherence > 0.8 + assert result.unknown_codeword_fixed_all > 0.8 + assert result.canonical_template_score < 0.2 + assert result.canonical_registered_template_score > 0.8 + + +def test_orientation_bank_recovers_right_angle_rotation(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + pixels, template = periodic_fixture + rotated_clockwise = np.rot90(pixels, k=-1) + + result = probe.score_orientation_bank( + rotated_clockwise, + template, + periods=np.asarray([16.0]), + rotations_degrees=np.asarray([0.0]), + patch_size=256, + grid_size=4, + harmonic_count=12, + ) + + assert result.selected_orientation_degrees == 90 + assert result.joint_amplitude > 0.8 + assert result.canonical_template_score > 0.8 + + +def test_dihedral_bank_recovers_horizontal_reflection(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + pixels, template = periodic_fixture + + result = probe.score_dihedral_bank( + np.fliplr(pixels), + template, + periods=np.asarray([16.0]), + rotations_degrees=np.asarray([0.0]), + patch_size=256, + grid_size=4, + harmonic_count=12, + ) + + assert result.selected_orientation_degrees == 0 + assert result.selected_horizontal_reflection is True + assert result.joint_amplitude > 0.8 + assert result.canonical_template_score > 0.8 + + +def test_deskew_bank_recovers_small_rotation(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + pixels, template = periodic_fixture + rotated = probe._rotate_fixed_canvas(pixels, 1.5) + + result = probe.score_deskew_bank( + rotated, + template, + periods=np.asarray([16.0]), + deskew_degrees=np.asarray([-2.0, -1.5, -1.0]), + patch_size=256, + grid_size=4, + harmonic_count=12, + ) + + assert result.selected_deskew_degrees == -1.5 + assert result.joint_amplitude > 0.6 + assert result.canonical_template_score > 0.6 + assert result.deskew_direct_joint_match > 0.4 + + +def test_registered_period_mode_uses_runtime_selected_period( + periodic_fixture: tuple[np.ndarray, np.ndarray], + monkeypatch: pytest.MonkeyPatch, +) -> None: + pixels, template = periodic_fixture + runner = CliRunner() + with runner.isolated_filesystem(): + np.savez("template.npz", template=template) + Image.fromarray(pixels).save("image.png") + components = probe.RegisteredComponents( + raw_score=0.4, + amplitude_threshold=0.2, + selected_period=16.0, + spectral_period=16.0, + high_band_score=0.15, + confirmation=RegisteredConfirmationComponents( + period=16.0, + joint_coherence=0.5, + joint_amplitude=0.2, + unknown_codeword_fixed_confirmation=0.5, + selection_patches=8, + confirmation_patches=8, + ), + ) + monkeypatch.setattr(probe, "registered_components", lambda *_args: components) + + result = runner.invoke( + probe.main, + [ + "template.npz", + "image.png", + "--registered-period", + "--same-image-null", + "--patch-shift-consensus", + "--opponent-registered", + "--report-out", + "report.json", + ], + ) + + assert result.exit_code == 0, result.output + report = json.loads(Path("report.json").read_text(encoding="utf-8")) + assert report["registered_period"] is True + assert report["same_image_null"] is True + assert report["patch_shift_consensus"] is True + assert report["opponent_registered"] is True + assert report["records"][0]["registered"]["selected_period"] == 16.0 + assert report["records"][0]["registered"]["decision_score"] == 2.0 + assert report["records"][0]["score"]["selected_period"] == 16.0 + assert report["records"][0]["same_image_null"]["joint_excess"] > 0.2 + assert report["records"][0]["patch_shift_consensus"]["joint_support_fraction"] == 1.0 + assert report["records"][0]["opponent_registered"]["decision_score"] > 1.0 + + +def test_registered_confirmation_uses_frozen_period_aware_gates( + periodic_fixture: tuple[np.ndarray, np.ndarray], +) -> None: + pixels, template = periodic_fixture + baseline = _score(pixels, template) + generic = replace( + baseline, + selected_period=16.0, + joint_coherence=0.30, + joint_amplitude=0.0, + ) + + assert probe.registered_confirmation_passes(generic) + assert not probe.registered_confirmation_passes(replace(generic, selected_period=9.99)) + assert not probe.registered_confirmation_passes(replace(generic, joint_coherence=0.299)) + assert not probe.registered_confirmation_passes(replace(generic, joint_amplitude=-0.001)) + assert not probe.registered_confirmation_passes( + replace(generic, selected_period=18.28, unknown_codeword_fixed_confirmation=0.129) + ) + assert probe.registered_confirmation_passes( + replace(generic, selected_period=18.28, unknown_codeword_fixed_confirmation=0.13) + ) + assert not probe.registered_confirmation_passes(replace(generic, selected_period=19.14, joint_coherence=0.399)) + assert probe.registered_confirmation_passes(replace(generic, selected_period=19.14, joint_coherence=0.40)) + assert not probe.registered_confirmation_passes( + replace(generic, selected_period=21.31, unknown_codeword_fixed_confirmation=0.019) + ) + assert probe.registered_confirmation_passes( + replace(generic, selected_period=21.31, unknown_codeword_fixed_confirmation=0.02) + ) + + +def test_fixed_candidate_uses_frozen_precision_threshold() -> None: + assert not probe.fixed_candidate_passes(0.279999) + assert probe.fixed_candidate_passes(0.28) + assert not probe.fixed_candidate_passes(float("nan")) diff --git a/tests/test_synthid_color_space_probe.py b/tests/test_synthid_color_space_probe.py new file mode 100644 index 0000000..1f4c697 --- /dev/null +++ b/tests/test_synthid_color_space_probe.py @@ -0,0 +1,105 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_color_space_probe as probe + + +def _write_image(path: Path, *, phase: float | None, seed: int) -> None: + height = width = 64 + rng = np.random.default_rng(seed) + pixels = 100.0 + rng.normal(0.0, 3.0, size=(height, width, 3)) + if phase is not None: + yy, xx = np.mgrid[:height, :width] + pixels[:, :, 0] += 12.0 * np.cos(2.0 * np.pi * (7.0 * yy / height + 5.0 * xx / width) + phase) + pixels[:, :, 1] -= 8.0 * np.cos(2.0 * np.pi * (7.0 * yy / height + 5.0 * xx / width) + phase) + Image.fromarray(np.clip(np.rint(pixels), 0, 255).astype(np.uint8), mode="RGB").save(path) + + +@pytest.mark.parametrize("color_space", probe.COLOR_SPACES) +def test_color_transforms_are_finite_and_preserve_shape(color_space: str) -> None: + rgb = np.asarray([[[0.0, 128.0, 255.0], [64.0, 64.0, 64.0]]]) + + transformed = probe.transform_color_space(rgb, color_space) + + assert transformed.shape == rgb.shape + assert np.all(np.isfinite(transformed)) + + +def test_neutral_gray_has_neutral_chroma() -> None: + gray = np.full((1, 1, 3), 96.0) + + ycbcr = probe.transform_color_space(gray, "ycbcr") + ycocg = probe.transform_color_space(gray, "ycocg") + opponent = probe.transform_color_space(gray, "opponent") + lab = probe.transform_color_space(gray, "lab") + hsv = probe.transform_color_space(gray, "hsv") + + assert ycbcr[0, 0, 1:] == pytest.approx((128.0, 128.0), abs=1e-4) + assert ycocg[0, 0, 1:] == pytest.approx((0.0, 0.0), abs=1e-8) + assert opponent[0, 0, 1:] == pytest.approx((0.0, 0.0), abs=1e-8) + assert lab[0, 0, 1:] == pytest.approx((0.0, 0.0), abs=0.1) + assert hsv[0, 0, 1] == pytest.approx(0.0, abs=1e-8) + + +@pytest.mark.parametrize("color_space", ["rgb", "opponent"]) +def test_discovered_model_separates_shared_phase_from_noise(tmp_path: Path, color_space: str) -> None: + positives: list[Path] = [] + for index in range(4): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + heldout = tmp_path / "heldout.png" + negative = tmp_path / "negative.png" + _write_image(heldout, phase=0.4, seed=10) + _write_image(negative, phase=None, seed=11) + bins = np.asarray([(7, 5, channel) for channel in range(3)], dtype=np.int32) + + model = probe.discover_model( + positives, + color_space=color_space, + candidate_bins=bins, + peak_count=3, + ) + + assert probe.score_image(heldout, model).evidence_score > probe.score_image(negative, model).evidence_score + + +def test_model_round_trip_is_pickle_free(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + bins = np.asarray([(7, 5, channel) for channel in range(3)], dtype=np.int32) + model = probe.discover_model(positives, color_space="ycocg", candidate_bins=bins, peak_count=3) + artifact = tmp_path / "model.npz" + + probe.save_model(artifact, model) + loaded = probe.load_model(artifact) + + assert loaded.color_space == "ycocg" + assert np.array_equal(loaded.channels, model.channels) + assert np.isclose(np.sum(loaded.weights), 1.0) + + +def test_channel_evidence_adds_to_total(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + bins = np.asarray([(7, 5, channel) for channel in range(3)], dtype=np.int32) + model = probe.discover_model(positives, color_space="rgb", candidate_bins=bins, peak_count=3) + + score = probe.score_image(positives[0], model) + + assert sum(score.channel_evidence) == pytest.approx(score.evidence_score) + assert sum(score.selected_peak_counts) == score.peak_count diff --git a/tests/test_synthid_confirmation.py b/tests/test_synthid_confirmation.py new file mode 100644 index 0000000..70fd7c4 --- /dev/null +++ b/tests/test_synthid_confirmation.py @@ -0,0 +1,83 @@ +from __future__ import annotations + +import sys +from dataclasses import replace +from pathlib import Path + +import cv2 +import numpy as np +import pytest + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_affine_lattice_probe as research_probe +from synthid_runtime._synthid_confirmation import ( + RegisteredConfirmationComponents, + registered_confirmation_components, +) + + +@pytest.fixture(scope="module") +def periodic_fixture() -> tuple[np.ndarray, np.ndarray]: + rng = np.random.default_rng(20260818) + template = rng.normal(0.0, 1.0, (16, 16, 3)) + template -= np.mean(template, axis=(0, 1), keepdims=True) + template /= np.linalg.norm(template) + coarse = rng.normal(0.0, 8.0, (16, 16, 3)).astype(np.float32) + background = cv2.resize(coarse, (1024, 1024), interpolation=cv2.INTER_CUBIC) + 128.0 + carrier = np.tile(template, (64, 64, 1)) * 3.0 + pixels = np.clip(np.rint(background + carrier), 0, 255).astype(np.uint8) + return pixels, template + + +def test_runtime_components_match_frozen_research_seam( + periodic_fixture: tuple[np.ndarray, np.ndarray], +) -> None: + pixels, template = periodic_fixture + + runtime = registered_confirmation_components(pixels, template, 16.0, 1.0) + research = research_probe.score_lattice( + pixels, + template, + periods=np.asarray([16.0]), + rotations_degrees=np.asarray([0.0]), + ) + + assert runtime.period == research.selected_period + assert runtime.joint_coherence == pytest.approx(research.joint_coherence) + assert runtime.joint_amplitude == pytest.approx(research.joint_amplitude) + assert runtime.unknown_codeword_fixed_confirmation == pytest.approx(research.unknown_codeword_fixed_confirmation) + assert runtime.selection_patches == research.selection_patches + assert runtime.confirmation_patches == research.confirmation_patches + assert runtime.passes + + +def test_confirmation_rejects_independent_noise(periodic_fixture: tuple[np.ndarray, np.ndarray]) -> None: + _pixels, template = periodic_fixture + pixels = np.random.default_rng(20260819).integers(0, 256, (1024, 1024, 3), dtype=np.uint8) + + result = registered_confirmation_components(pixels, template, 16.0, 1.0) + + assert not result.passes + + +def test_period_aware_confirmation_boundaries() -> None: + baseline = RegisteredConfirmationComponents( + period=16.0, + joint_coherence=0.30, + joint_amplitude=0.0, + unknown_codeword_fixed_confirmation=0.5, + selection_patches=8, + confirmation_patches=8, + ) + + assert baseline.passes + assert not replace(baseline, period=9.99).passes + assert not replace(baseline, joint_coherence=0.299).passes + assert not replace(baseline, joint_amplitude=-0.001).passes + assert not replace(baseline, period=18.28, unknown_codeword_fixed_confirmation=0.129).passes + assert replace(baseline, period=18.28, unknown_codeword_fixed_confirmation=0.13).passes + assert not replace(baseline, period=19.14, joint_coherence=0.399).passes + assert replace(baseline, period=19.14, joint_coherence=0.40).passes + assert not replace(baseline, period=21.31, unknown_codeword_fixed_confirmation=0.019).passes + assert replace(baseline, period=21.31, unknown_codeword_fixed_confirmation=0.02).passes diff --git a/tests/test_synthid_conformal_cascade.py b/tests/test_synthid_conformal_cascade.py new file mode 100644 index 0000000..8965802 --- /dev/null +++ b/tests/test_synthid_conformal_cascade.py @@ -0,0 +1,240 @@ +from __future__ import annotations + +import json +import sys +from pathlib import Path + +import pytest +from click.testing import CliRunner + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_conformal_cascade as cascade + + +def _scores(value: float, count: int = 1999) -> tuple[float, ...]: + return (value,) * count + + +def _expert(name: str, *, higher_is_positive: bool = True) -> cascade.ExpertCalibration: + return cascade.ExpertCalibration( + name=name, + positive_scores=_scores(1.0), + negative_scores=_scores(0.0), + higher_is_positive=higher_is_positive, + ) + + +def _config(*experts: cascade.ExpertCalibration, coverage_complete: bool = False) -> cascade.CascadeConfig: + return cascade.CascadeConfig( + experts=experts, + positive_alpha=0.001, + negative_alpha=0.001, + coverage_complete=coverage_complete, + scope="synthetic test bank", + ) + + +def _observation(name: str, score: float | None, *, supported: bool = True) -> cascade.ExpertObservation: + return cascade.ExpertObservation(name=name, supported=supported, score=score) + + +def test_empirical_tail_p_values_include_ties_and_smoothing() -> None: + scores = (0.1, 0.2, 0.3) + + assert cascade._upper_tail_p_value(scores, 0.3) == 0.5 + assert cascade._upper_tail_p_value(scores, 0.31) == 0.25 + assert cascade._lower_tail_p_value(scores, 0.1) == 0.5 + assert cascade._lower_tail_p_value(scores, 0.09) == 0.25 + + +def test_any_expert_can_detect_with_familywise_correction() -> None: + config = _config(_expert("fixed"), _expert("registered")) + observations = (_observation("fixed", 2.0), _observation("registered", 0.5)) + + result = cascade.classify_observations(config, observations) + + assert result.verdict == "detected" + assert result.reason == "watermarked_hypothesis_supported" + assert result.clean_null_p_value == 0.001 + assert result.watermarked_p_value is not None + assert result.watermarked_p_value > config.negative_alpha + + +def test_familywise_correction_blocks_bank_wide_false_alarm() -> None: + config = _config(_expert("fixed"), _expert("registered"), _expert("version-3")) + + result = cascade.classify_observations( + config, + ( + _observation("fixed", 2.0), + _observation("registered", 0.5), + _observation("version-3", 0.5), + ), + ) + + assert result.verdict == "abstain" + assert result.reason == "insufficient_evidence" + assert result.clean_null_p_value == 0.0015 + + +def test_incomplete_version_coverage_never_claims_absence() -> None: + config = _config(_expert("fixed"), coverage_complete=False) + + result = cascade.classify_observations(config, (_observation("fixed", -1.0),)) + + assert result.verdict == "abstain" + assert result.reason == "incomplete_coverage" + assert result.watermarked_p_value == 0.0005 + + +def test_complete_bank_can_reject_every_watermarked_expert() -> None: + config = _config(_expert("fixed"), _expert("registered"), coverage_complete=True) + + result = cascade.classify_observations( + config, + (_observation("fixed", -1.0), _observation("registered", -1.0)), + ) + + assert result.verdict == "not_detected" + assert result.reason == "unwatermarked_hypothesis_supported" + assert result.watermarked_p_value == 0.0005 + + +def test_watermarked_union_survives_when_one_version_remains_plausible() -> None: + ambiguous = cascade.ExpertCalibration( + name="registered", + positive_scores=_scores(0.0), + negative_scores=_scores(0.0), + ) + config = _config(_expert("fixed"), ambiguous, coverage_complete=True) + + result = cascade.classify_observations( + config, + (_observation("fixed", -1.0), _observation("registered", 0.0)), + ) + + assert result.verdict == "abstain" + assert result.reason == "insufficient_evidence" + assert result.watermarked_p_value == 1.0 + + +def test_missing_geometry_support_prevents_negative_verdict() -> None: + config = _config(_expert("fixed"), _expert("registered"), coverage_complete=True) + + result = cascade.classify_observations( + config, + (_observation("fixed", -1.0), _observation("registered", None, supported=False)), + ) + + assert result.verdict == "abstain" + assert result.reason == "incomplete_support" + + +def test_out_of_distribution_gap_abstains_on_conflicting_evidence() -> None: + config = _config(_expert("fixed"), coverage_complete=True) + + result = cascade.classify_observations(config, (_observation("fixed", 0.5),)) + + assert result.verdict == "abstain" + assert result.reason == "conflicting_evidence" + assert result.clean_null_p_value == 0.0005 + assert result.watermarked_p_value == 0.0005 + + +def test_lower_scores_can_be_oriented_as_positive() -> None: + expert = cascade.ExpertCalibration( + name="inverse", + positive_scores=_scores(-1.0), + negative_scores=_scores(0.0), + higher_is_positive=False, + ) + + result = cascade.classify_observations(_config(expert), (_observation("inverse", -2.0),)) + + assert result.verdict == "detected" + + +def test_observations_must_explicitly_cover_the_expert_bank() -> None: + config = _config(_expert("fixed"), _expert("registered")) + + with pytest.raises(ValueError, match=r"missing=\['registered'\]"): + cascade.classify_observations(config, (_observation("fixed", 2.0),)) + + +def test_cli_writes_hash_pinned_tri_state_report(tmp_path: Path) -> None: + calibration_path = tmp_path / "calibration.json" + observation_path = tmp_path / "observations.json" + report_path = tmp_path / "report.json" + calibration_path.write_text( + json.dumps( + { + "schema_version": 1, + "scope": "synthetic CLI test", + "positive_alpha": 0.001, + "negative_alpha": 0.001, + "coverage_complete": False, + "experts": [ + { + "name": "fixed", + "higher_is_positive": True, + "positive_scores": list(_scores(1.0)), + "negative_scores": list(_scores(0.0)), + } + ], + } + ), + encoding="utf-8", + ) + observation_path.write_text( + json.dumps( + { + "schema_version": 1, + "records": [ + { + "id": "candidate-1", + "observations": [{"name": "fixed", "supported": True, "score": 2.0}], + } + ], + } + ), + encoding="utf-8", + ) + + result = CliRunner().invoke( + cascade.main, + [str(calibration_path), str(observation_path), "--report-out", str(report_path)], + ) + + assert result.exit_code == 0, result.output + report = json.loads(report_path.read_text(encoding="utf-8")) + assert report["scope"] == "synthetic CLI test" + assert report["counts"] == {"detected": 1, "not_detected": 0, "abstain": 0} + assert len(report["calibration_sha256"]) == 64 + assert report["records"][0]["result"]["verdict"] == "detected" + + +def test_loader_rejects_string_boolean_for_complete_coverage(tmp_path: Path) -> None: + calibration_path = tmp_path / "calibration.json" + calibration_path.write_text( + json.dumps( + { + "schema_version": 1, + "scope": "invalid test", + "positive_alpha": 0.001, + "negative_alpha": 0.001, + "coverage_complete": "false", + "experts": [ + { + "name": "fixed", + "positive_scores": [1.0], + "negative_scores": [0.0], + } + ], + } + ), + encoding="utf-8", + ) + + with pytest.raises(ValueError, match="coverage_complete must be a boolean"): + cascade.load_config(calibration_path) diff --git a/tests/test_synthid_confound_probe.py b/tests/test_synthid_confound_probe.py new file mode 100644 index 0000000..2ea5be3 --- /dev/null +++ b/tests/test_synthid_confound_probe.py @@ -0,0 +1,243 @@ +"""Tests for the manifest-driven SynthID D1 confound challenge.""" + +from __future__ import annotations + +import csv +import hashlib +import json +import sys +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image +from PIL.PngImagePlugin import PngInfo + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_confound_probe as probe +import synthid_research_manifest as manifest + + +def _write_image(path: Path, color: tuple[int, int, int], *, note: str | None = None) -> tuple[str, str]: + image = Image.new("RGB", (12, 10), color) + pnginfo = None + if note is not None: + pnginfo = PngInfo() + pnginfo.add_text("note", note) + image.save(path, pnginfo=pnginfo) + return hashlib.sha256(path.read_bytes()).hexdigest(), hashlib.sha256(image.tobytes()).hexdigest() + + +def _row( + artifact_sha: str, + pixel_sha: str, + artifact_path: str, + *, + group_id: str, + split: str, + outcome: str, + source_provider: str, + oracle_role: str = "ordinary", +) -> dict[str, str]: + matching_oracle = source_provider == "openai" + return { + "artifact_sha256": artifact_sha, + "pixel_sha256": pixel_sha, + "artifact_path": artifact_path, + "parent_sha256": "", + "group_id": group_id, + "target_provider": "openai", + "source_provider": source_provider, + "surface": "synthetic-test", + "model_epoch": "test-epoch", + "generation_session": f"session-{group_id}", + "content_stratum": "flat-graphic", + "width": "12", + "height": "10", + "format": "png", + "transform": "original", + "split": split, + "c2pa_outcome": "detected" if outcome == "detected" else "not_present", + "synthid_outcome": outcome, + "verified_via": "openai-api" if matching_oracle else "source-evidence", + "evidence_reference": "" if matching_oracle else f"https://example.test/{group_id}", + "oracle_session": f"oracle-{group_id}" if matching_oracle else "", + "oracle_role": oracle_role, + "captured_at": "2026-08-09T10:00:00Z", + "oracle_checked_at": "2026-08-09T10:05:00Z", + "notes": "synthetic confound fixture", + } + + +def _write_manifest(root: Path, rows: list[dict[str, str]]) -> Path: + path = root / "manifest.csv" + with path.open("w", newline="", encoding="utf-8") as stream: + writer = csv.DictWriter(stream, fieldnames=manifest.FIELDNAMES) + writer.writeheader() + writer.writerows(rows) + return path + + +def _corpus(root: Path, *, include_temporal: bool = True) -> Path: + rows: list[dict[str, str]] = [] + index = 0 + split_sizes = {"train": 4, "validation": 2, "test": 2} + if include_temporal: + split_sizes["temporal"] = 2 + for split, per_class in split_sizes.items(): + for label in (1, 0): + for offset in range(per_class): + index += 1 + color = (230, 10 + index, 20 + offset) if label else (20 + offset, 10 + index, 230) + path = root / f"image-{index}.png" + artifact_sha, pixel_sha = _write_image(path, color) + source_provider = "openai" if label or offset == 0 else "camera" + rows.append( + _row( + artifact_sha, + pixel_sha, + path.name, + group_id=f"group-{index}", + split=split, + outcome="detected" if label else "not_detected", + source_provider=source_provider, + ) + ) + return _write_manifest(root, rows) + + +def test_canonical_features_ignore_container_metadata(tmp_path: Path): + first = tmp_path / "first.png" + second = tmp_path / "second.png" + first_sha, _ = _write_image(first, (10, 20, 30), note="short") + second_sha, _ = _write_image(second, (10, 20, 30), note="a much longer metadata value") + first_example = probe.Example(first, first_sha, "one", "train", 1, None) + second_example = probe.Example(second, second_sha, "two", "train", 1, None) + + assert not np.array_equal( + probe.extract_features(first_example, "container"), + probe.extract_features(second_example, "container"), + ) + np.testing.assert_array_equal( + probe.extract_features(first_example, "canonical"), + probe.extract_features(second_example, "canonical"), + ) + + +def test_feature_matrices_decode_each_artifact_once(tmp_path: Path, monkeypatch: pytest.MonkeyPatch): + first = tmp_path / "first.png" + second = tmp_path / "second.png" + first_sha, _ = _write_image(first, (10, 20, 30)) + second_sha, _ = _write_image(second, (30, 20, 10)) + examples = [ + probe.Example(first, first_sha, "one", "train", 1, None), + probe.Example(second, second_sha, "two", "train", 0, "external"), + ] + original = probe._decoded_rgb + decode_count = 0 + + def counting_decode(path: Path): + nonlocal decode_count + decode_count += 1 + return original(path) + + monkeypatch.setattr(probe, "_decoded_rgb", counting_decode) + + matrices = probe.feature_matrices(examples) + + assert decode_count == len(examples) + assert set(matrices) == set(probe.FEATURE_FAMILIES) + assert all(matrix.shape[0] == len(examples) for matrix in matrices.values()) + + +def test_logistic_model_separates_simple_content_confound(): + features = np.asarray([[0.0], [0.1], [0.9], [1.0]], dtype=np.float64) + labels = np.asarray([0, 0, 1, 1], dtype=np.int64) + + model = probe.fit_logistic(features, labels) + scores = probe.predict_scores(model, features) + + assert max(scores[:2]) < min(scores[2:]) + + +def test_threshold_respects_validation_false_positive_limit(): + labels = np.asarray([1, 1, 0, 0], dtype=np.int64) + scores = np.asarray([0.9, 0.8, 0.7, 0.1], dtype=np.float64) + + threshold = probe.select_threshold(labels, scores, max_fpr=0.0) + metrics = probe.calculate_metrics(labels, scores, threshold) + + assert threshold == pytest.approx(0.8) + assert metrics.tpr == 1.0 + assert metrics.fpr == 0.0 + + +def test_run_experiment_reports_same_provider_and_temporal_gate(tmp_path: Path): + path = _corpus(tmp_path) + + report = probe.run_experiment(path, "openai") + + assert report["evidence_ready"] is True + assert report["evidence_missing"] == [] + canonical = report["families"]["canonical"] + assert canonical["metrics"]["test"]["auc"] == 1.0 + assert canonical["negative_cohorts"]["test"]["same_provider"]["count"] == 1 + assert canonical["metrics"]["temporal"]["positives"] == 2 + assert canonical["metrics"]["temporal"]["negatives"] == 2 + assert str(tmp_path) not in json.dumps(report) + + +def test_report_stays_explicit_when_temporal_holdout_is_missing(tmp_path: Path): + path = _corpus(tmp_path, include_temporal=False) + + report = probe.run_experiment(path, "openai") + + assert report["evidence_ready"] is False + assert report["evidence_missing"] == ["temporal split does not contain both labels"] + assert report["families"]["container"]["metrics"]["temporal"] is None + + +def test_candidate_rows_are_not_detector_examples(tmp_path: Path): + path = _corpus(tmp_path) + rows = probe._read_manifest(path) + candidate_path = tmp_path / "candidate.png" + artifact_sha, pixel_sha = _write_image(candidate_path, (90, 90, 90)) + rows.append( + _row( + artifact_sha, + pixel_sha, + candidate_path.name, + group_id="candidate-group", + split="train", + outcome="detected", + source_provider="openai", + oracle_role="candidate", + ) + ) + _write_manifest(tmp_path, rows) + + examples = probe.load_examples(path, "openai") + + assert all(example.artifact_path.name != candidate_path.name for example in examples) + + +def test_manifest_audit_blocks_group_leakage_before_training(tmp_path: Path): + path = _corpus(tmp_path) + rows = probe._read_manifest(path) + rows[-1]["group_id"] = rows[0]["group_id"] + _write_manifest(tmp_path, rows) + + with pytest.raises(ValueError, match="manifest audit failed"): + probe.run_experiment(path, "openai") + + +def test_positive_source_provider_must_match_target(tmp_path: Path): + path = _corpus(tmp_path) + rows = probe._read_manifest(path) + positive = next(row for row in rows if row["synthid_outcome"] == "detected") + positive["source_provider"] = "google" + _write_manifest(tmp_path, rows) + + with pytest.raises(ValueError, match="declares source_provider"): + probe.load_examples(path, "openai") diff --git a/tests/test_synthid_consensus_probe.py b/tests/test_synthid_consensus_probe.py new file mode 100644 index 0000000..b763339 --- /dev/null +++ b/tests/test_synthid_consensus_probe.py @@ -0,0 +1,73 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_consensus_probe as probe + + +def _write_pattern(path: Path, base: tuple[int, int, int], *, sign: float, marked: bool) -> None: + size = 128 + yy, xx = np.mgrid[:size, :size] + carrier = 2.0 * np.sin(2.0 * np.pi * (11.0 * yy + 7.0 * xx) / size) + carrier += 1.5 * np.sin(2.0 * np.pi * (17.0 * yy - 5.0 * xx) / size + 0.4) + pixels = np.broadcast_to(np.asarray(base, dtype=np.float64), (size, size, 3)).copy() + if marked: + pixels += sign * carrier[:, :, None] + Image.fromarray(np.clip(np.rint(pixels), 0, 255).astype(np.uint8), mode="RGB").save(path) + + +def _reference_groups(tmp_path: Path) -> tuple[list[list[Path]], Path, Path]: + groups: list[list[Path]] = [] + for group_index, base in enumerate(((40, 50, 60), (170, 180, 190))): + directory = tmp_path / f"group-{group_index}" + directory.mkdir() + paths: list[Path] = [] + for image_index in range(3): + path = directory / f"marked-{image_index}.png" + sign = -1.0 if group_index == 1 else 1.0 + _write_pattern(path, base, sign=sign, marked=True) + paths.append(path) + groups.append(paths) + positive = tmp_path / "positive.png" + negative = tmp_path / "negative.png" + _write_pattern(positive, (100, 110, 120), sign=-1.0, marked=True) + _write_pattern(negative, (100, 110, 120), sign=1.0, marked=False) + return groups, positive, negative + + +def test_discovers_polarity_invariant_carrier(tmp_path: Path) -> None: + groups, positive, negative = _reference_groups(tmp_path) + + model = probe.discover_model(groups, size=128, peak_count=16, min_radius=3.0) + positive_score = probe.score_image(positive, model) + negative_score = probe.score_image(negative, model) + + assert positive_score.score > 0.8 + assert positive_score.active_weight_fraction > 0.5 + assert negative_score.active_weight_fraction < 0.01 + + +def test_model_round_trip_disables_pickle(tmp_path: Path) -> None: + groups, positive, _ = _reference_groups(tmp_path) + model = probe.discover_model(groups, size=128, peak_count=8, min_radius=3.0) + artifact = tmp_path / "model.npz" + + probe.save_model(artifact, model) + loaded = probe.load_model(artifact) + + assert probe.score_image(positive, loaded).score == pytest.approx(probe.score_image(positive, model).score) + assert loaded.peaks.dtype == np.int32 + + +def test_requires_independent_groups(tmp_path: Path) -> None: + groups, _, _ = _reference_groups(tmp_path) + + with pytest.raises(ValueError, match="at least two"): + probe.discover_model(groups[:1], size=128, peak_count=8) diff --git a/tests/test_synthid_cyclostationary_probe.py b/tests/test_synthid_cyclostationary_probe.py new file mode 100644 index 0000000..2dd1eb7 --- /dev/null +++ b/tests/test_synthid_cyclostationary_probe.py @@ -0,0 +1,65 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_cyclostationary_probe as probe + + +def _template() -> np.ndarray: + _y, x = np.indices((16, 16)) + carrier = np.cos(2.0 * np.pi * 4.0 * x / 16.0) + template = np.stack((carrier, 0.8 * carrier, 0.6 * carrier), axis=2) + template -= np.mean(template, axis=(0, 1), keepdims=True) + return template / np.linalg.norm(template) + + +def test_detects_complex_spectral_coupling() -> None: + rng = np.random.default_rng(20260814) + base = rng.normal(0.0, 1.0, (1024, 1024, 3)) + _y, x = np.indices(base.shape[:2]) + modulation = 1.0 + 0.8 * np.cos(2.0 * np.pi * 4.0 * x / 16.0) + + result = probe.score_cyclostationary( + base * modulation[:, :, None], + _template(), + period=16.0, + harmonic_count=1, + ) + + assert result.selection_contrast > 0.1 + assert result.confirmation_contrast > 0.1 + assert result.joint_contrast > 0.1 + + +def test_rejects_independent_equal_power_noise() -> None: + rng = np.random.default_rng(20260815) + noise = rng.normal(0.0, 1.0, (1024, 1024, 3)) + + result = probe.score_cyclostationary( + noise, + _template(), + period=16.0, + harmonic_count=1, + ) + + assert result.joint_contrast < 0.01 + + +def test_does_not_confuse_additive_carrier_with_modulation() -> None: + rng = np.random.default_rng(20260816) + noise = rng.normal(0.0, 1.0, (1024, 1024, 3)) + additive = np.tile(_template(), (64, 64, 1)) * 2.0 + + result = probe.score_cyclostationary( + noise + additive, + _template(), + period=16.0, + harmonic_count=1, + ) + + assert result.joint_contrast < 0.01 diff --git a/tests/test_synthid_detector.py b/tests/test_synthid_detector.py new file mode 100644 index 0000000..f8e4e3d --- /dev/null +++ b/tests/test_synthid_detector.py @@ -0,0 +1,787 @@ +"""Runtime tests for the positive-only SynthID periodic carrier detector.""" + +from __future__ import annotations + +import hashlib +from pathlib import Path + +import numpy as np +import pytest +import synthid_runtime.synthid_detector as detector +from PIL import Image + + +@pytest.fixture(scope="module") +def supported_images(tmp_path_factory: pytest.TempPathFactory) -> tuple[Path, Path]: + """Create supported-geometry positive and negative synthetic fixtures.""" + directory = tmp_path_factory.mktemp("synthid-detector") + template, *_model = detector._load_template() + scaled_tile = np.rint(template / np.max(np.abs(template))) + marked = np.full((detector.MODEL_HEIGHT, detector.MODEL_WIDTH, 3), 128, dtype=np.float64) + marked += np.tile(scaled_tile, (128, 128, 1)) + + positive = directory / "positive.png" + negative = directory / "negative.png" + Image.fromarray(np.clip(np.rint(marked), 0, 255).astype(np.uint8), "RGB").save(positive) + Image.new("RGB", (detector.MODEL_WIDTH, detector.MODEL_HEIGHT), (128, 128, 128)).save(negative) + return positive, negative + + +@pytest.fixture(scope="module") +def registered_scale_positive(tmp_path_factory: pytest.TempPathFactory) -> Path: + """Create a strong period-12.8 carrier by shrinking a period-16 source.""" + import cv2 + + directory = tmp_path_factory.mktemp("synthid-registered") + template, *_model = detector._load_template() + scaled_tile = template / np.max(np.abs(template)) * 40.0 + source = np.tile(scaled_tile, (64, 64, 1)) + 128.0 + pixels = cv2.resize( + np.clip(np.rint(source), 0, 255).astype(np.uint8), + (819, 819), + interpolation=cv2.INTER_AREA, + ) + path = directory / "period-12.8-positive.png" + Image.fromarray(pixels, "RGB").save(path) + return path + + +@pytest.fixture(scope="module") +def opponent_registered_positive(tmp_path_factory: pytest.TempPathFactory) -> Path: + """Create a strong period-10 opponent-color fallback fixture.""" + import cv2 + + directory = tmp_path_factory.mktemp("synthid-opponent-registered") + template, *_model = detector._load_template() + scaled_tile = template / np.max(np.abs(template)) * 40.0 + source = np.tile(scaled_tile, (128, 128, 1)) + 128.0 + pixels = cv2.resize( + np.clip(np.rint(source), 0, 255).astype(np.uint8), + (1280, 1280), + interpolation=cv2.INTER_AREA, + ) + path = directory / "period-10-positive.png" + Image.fromarray(pixels, "RGB").save(path) + return path + + +@pytest.fixture(scope="module") +def opponent_period8_positive(tmp_path_factory: pytest.TempPathFactory) -> Path: + """Create a strong period-8 fallback fixture without native JPEG block edges.""" + import cv2 + + directory = tmp_path_factory.mktemp("synthid-opponent-period8") + template, *_model = detector._load_template() + scaled_tile = template / np.max(np.abs(template)) * 40.0 + source = np.tile(scaled_tile, (128, 128, 1)) + 128.0 + pixels = cv2.resize( + np.clip(np.rint(source), 0, 255).astype(np.uint8), + (1024, 1024), + interpolation=cv2.INTER_AREA, + ) + path = directory / "period-8-positive.png" + Image.fromarray(pixels, "RGB").save(path) + return path + + +@pytest.fixture(scope="module") +def fine_opponent_registered_positive(tmp_path_factory: pytest.TempPathFactory) -> Path: + """Create a strong period-7.68 carrier missed by the coarse period grid.""" + import cv2 + + directory = tmp_path_factory.mktemp("synthid-fine-opponent-registered") + template, *_model = detector._load_template() + scaled_tile = template / np.max(np.abs(template)) * 40.0 + source = np.tile(scaled_tile, (144, 144, 1)) + 128.0 + pixels = cv2.resize( + np.clip(np.rint(source), 0, 255).astype(np.uint8), + (1106, 1106), + interpolation=cv2.INTER_AREA, + ) + path = directory / "period-7.68-positive.png" + Image.fromarray(pixels, "RGB").save(path) + return path + + +def test_bundled_model_is_the_frozen_calibrated_artifact() -> None: + model = Path(detector.__file__).parent / detector.MODEL_FILENAME + + assert hashlib.sha256(model.read_bytes()).hexdigest() == ( + "ee7838da8542c206c3403284b68e98f0ac99429e82f262c1a438f50a638b488b" + ) + + +@pytest.mark.parametrize( + ("width", "height"), + [(1000, 1000), (1001, 1000), (3000, 6000), (768, 1364)], +) +def test_supported_geometry_uses_the_challenged_pixel_count_range(width: int, height: int) -> None: + assert detector._geometry_supported(width, height) + + +@pytest.mark.parametrize( + ("width", "height"), + [(999, 1000), (3001, 6000), (64, 32)], +) +def test_geometry_outside_the_challenged_pixel_count_range_is_unsupported( + width: int, + height: int, +) -> None: + assert not detector._geometry_supported(width, height) + + +@pytest.mark.parametrize( + ("width", "height", "supported"), + [ + (500, 500, True), + (4000, 2500, True), + (256, 977, True), + (499, 500, False), + (4001, 2500, False), + (255, 981, False), + (64, 3907, False), + (32, 7813, False), + ], +) +def test_registered_geometry_uses_its_measured_pixel_count_range( + width: int, + height: int, + supported: bool, +) -> None: + assert detector._registered_geometry_supported(width, height) is supported + + +@pytest.mark.parametrize( + ("width", "height", "supported"), + [ + (1000, 1000, True), + (4000, 2500, True), + (767, 1304, False), + (1000, 999, False), + (4001, 2500, False), + ], +) +def test_opponent_registered_geometry_uses_its_frozen_domain( + width: int, + height: int, + supported: bool, +) -> None: + assert detector._opponent_registered_geometry_supported(width, height) is supported + + +@pytest.mark.parametrize( + ("width", "height", "supported"), + [ + (1000, 1000, True), + (2500, 2000, True), + (767, 1304, False), + (1000, 999, False), + (2501, 2000, False), + ], +) +def test_fine_opponent_registered_geometry_uses_its_frozen_domain( + width: int, + height: int, + supported: bool, +) -> None: + assert detector._fine_opponent_registered_geometry_supported(width, height) is supported + + +@pytest.mark.parametrize( + ("width", "height", "supported"), + [ + (4883, 2048, True), + (3072, 5504, True), + (2048, 4882, False), + (2047, 6000, False), + (3001, 6000, False), + ], +) +def test_large_geometry_requires_multiple_calibrated_windows( + width: int, + height: int, + supported: bool, +) -> None: + assert detector._large_geometry_supported(width, height) is supported + + +def test_large_window_starts_cover_both_edges_on_carrier_phase() -> None: + starts = detector._large_window_starts(5504) + + assert starts == (0, 2048, 3456) + assert all(start % detector.LARGE_PHASE == 0 for start in starts) + assert starts[-1] + detector.LARGE_WINDOW == 5504 + + +def test_large_components_apply_the_portrait_alias_guard_only_to_its_geometry() -> None: + values = { + "minimum_fixed_score": 0.28, + "minimum_red_green_spatial": 0.95, + "minimum_blue_yellow_spatial": 0.85, + "minimum_blue_yellow_mid_band": -0.30, + "maximum_green_mid_band": 0.061, + } + portrait = detector.LargeImageComponents(width=3072, height=5504, **values) + landscape = detector.LargeImageComponents(width=5504, height=3072, **values) + + assert portrait.decision_score < detector.LARGE_THRESHOLD + assert landscape.decision_score > detector.LARGE_THRESHOLD + + +def test_large_red_green_gate_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, +) -> None: + width, height = 4883, 2048 + image = np.broadcast_to(np.zeros((1, 1, 3), dtype=np.uint8), (height, width, 3)) + components = detector.LargeImageComponents( + width=width, + height=height, + minimum_fixed_score=0.28, + minimum_red_green_spatial=detector.LARGE_RED_GREEN_SPATIAL_MIN, + minimum_blue_yellow_spatial=0.85, + minimum_blue_yellow_mid_band=-0.30, + maximum_green_mid_band=0.0, + ) + monkeypatch.setattr(detector, "is_available", lambda: True) + monkeypatch.setattr(detector, "_load_template", lambda: (np.zeros((16, 16, 3)), 1.0, 0, 0, 0, 0)) + monkeypatch.setattr(detector, "large_image_components", lambda *_args: components) + + baseline = detector.detect_synthid("unused.png", image=image) + monkeypatch.setattr( + detector, + "LARGE_RED_GREEN_SPATIAL_MIN", + float(np.nextafter(components.minimum_red_green_spatial, np.inf)), + ) + mutated = detector.detect_synthid("unused.png", image=image) + + assert baseline.status == "detected" + assert baseline.detector == detector.LARGE_DETECTOR_ID + assert mutated.status == "indeterminate" + + +def test_uncalibrated_narrow_large_geometry_is_unsupported() -> None: + image = np.broadcast_to(np.zeros((1, 1, 3), dtype=np.uint8), (11_000, 1000, 3)) + + result = detector.detect_synthid("unused.png", image=image) + + assert result.status == "unsupported" + assert result.detector == detector.LARGE_DETECTOR_ID + assert result.score is None + + +def test_registered_mode_rejects_a_side_too_short_for_quadrants(tmp_path: Path) -> None: + path = tmp_path / "too-narrow.png" + Image.new("RGB", (32, 7813), "white").save(path) + + result = detector.detect_synthid(path, register_scale=True) + + assert result.status == "unsupported" + assert result.score is None + assert result.detector == detector.REGISTERED_DETECTOR_ID + + +def test_detects_supported_periodic_carrier(supported_images: tuple[Path, Path]) -> None: + positive, _negative = supported_images + + result = detector.detect_synthid(positive, register_scale=False) + + assert result.status == "detected" + assert result.detected is True + assert result.score is not None + assert result.score > result.threshold + assert result.to_dict()["detector"] == detector.DETECTOR_ID + + +def test_detects_unregistered_non_divisible_geometry_in_size_range(tmp_path: Path) -> None: + width, height = 1001, 1000 + template, *_model = detector._load_template() + scaled_tile = np.rint(template / np.max(np.abs(template))) + repeats_y = (height + scaled_tile.shape[0] - 1) // scaled_tile.shape[0] + repeats_x = (width + scaled_tile.shape[1] - 1) // scaled_tile.shape[1] + carrier = np.tile(scaled_tile, (repeats_y, repeats_x, 1))[:height, :width] + pixels = np.clip(np.rint(carrier + 128.0), 0, 255).astype(np.uint8) + path = tmp_path / "non-divisible-positive.png" + Image.fromarray(pixels, "RGB").save(path) + + result = detector.detect_synthid(path, register_scale=False) + + assert result.status == "detected" + assert (result.width, result.height) == (width, height) + assert result.score is not None + assert result.score > result.threshold + + +def test_registered_mode_detects_a_rescaled_carrier(registered_scale_positive: Path) -> None: + fixed = detector.detect_synthid(registered_scale_positive, register_scale=False) + default = detector.detect_synthid(registered_scale_positive) + registered = detector.detect_synthid(registered_scale_positive, register_scale=True) + + assert fixed.status == "unsupported" + assert default == registered + assert registered.status == "detected" + assert registered.score is not None + assert registered.score > registered.threshold + assert registered.threshold == detector.REGISTERED_THRESHOLD + assert registered.detector == detector.REGISTERED_DETECTOR_ID + + +def test_registered_mode_falls_back_to_the_opponent_color_expert( + monkeypatch: pytest.MonkeyPatch, + opponent_registered_positive: Path, +) -> None: + import synthid_runtime._synthid_registered as registered_detector + + monkeypatch.setattr(registered_detector, "registered_score", lambda *_args: 0.0) + + result = detector.detect_synthid(opponent_registered_positive, register_scale=True) + + assert result.status == "detected" + assert result.detector == detector.OPPONENT_REGISTERED_DETECTOR_ID + assert result.score is not None + assert result.score >= result.threshold + + +def test_opponent_registered_threshold_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + opponent_registered_positive: Path, +) -> None: + import synthid_runtime._synthid_registered as registered_detector + + monkeypatch.setattr(registered_detector, "registered_score", lambda *_args: 0.0) + baseline = detector.detect_synthid(opponent_registered_positive, register_scale=True) + assert baseline.score is not None + assert baseline.detector == detector.OPPONENT_REGISTERED_DETECTOR_ID + monkeypatch.setattr( + detector, + "OPPONENT_REGISTERED_THRESHOLD", + float(np.nextafter(baseline.score, np.inf)), + ) + + mutated = detector.detect_synthid(opponent_registered_positive, register_scale=True) + + assert mutated.status == "indeterminate" + assert mutated.detector == detector.REGISTERED_DETECTOR_ID + + +def test_opponent_fallback_recovers_period8_without_codec_grid( + monkeypatch: pytest.MonkeyPatch, + opponent_period8_positive: Path, +) -> None: + import synthid_runtime._synthid_registered as registered_detector + + monkeypatch.setattr(registered_detector, "registered_score", lambda *_args: 0.0) + + result = detector.detect_synthid(opponent_period8_positive, register_scale=True) + + assert result.status == "detected" + assert result.detector == detector.OPPONENT_REGISTERED_DETECTOR_ID + + +def test_fine_opponent_fallback_recovers_off_grid_period( + monkeypatch: pytest.MonkeyPatch, + fine_opponent_registered_positive: Path, +) -> None: + import synthid_runtime._synthid_registered as registered_detector + + monkeypatch.setattr(registered_detector, "registered_score", lambda *_args: 0.0) + monkeypatch.setattr(registered_detector, "opponent_registered_score", lambda *_args: 0.0) + + result = detector.detect_synthid(fine_opponent_registered_positive, register_scale=True) + + assert result.status == "detected" + assert result.detector == detector.FINE_OPPONENT_REGISTERED_DETECTOR_ID + assert result.score is not None + assert result.score >= detector.FINE_OPPONENT_REGISTERED_THRESHOLD + + +def test_fine_opponent_threshold_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + fine_opponent_registered_positive: Path, +) -> None: + import synthid_runtime._synthid_registered as registered_detector + + monkeypatch.setattr(registered_detector, "registered_score", lambda *_args: 0.0) + monkeypatch.setattr(registered_detector, "opponent_registered_score", lambda *_args: 0.0) + baseline = detector.detect_synthid(fine_opponent_registered_positive, register_scale=True) + assert baseline.score is not None + assert baseline.detector == detector.FINE_OPPONENT_REGISTERED_DETECTOR_ID + monkeypatch.setattr( + detector, + "FINE_OPPONENT_REGISTERED_THRESHOLD", + float(np.nextafter(baseline.score, np.inf)), + ) + + mutated = detector.detect_synthid(fine_opponent_registered_positive, register_scale=True) + + assert mutated.status == "indeterminate" + assert mutated.detector == detector.REGISTERED_DETECTOR_ID + + +def test_fine_opponent_selector_recovers_the_fractional_period( + fine_opponent_registered_positive: Path, +) -> None: + import synthid_runtime._synthid_registered as registered_detector + + template, sigma, *_model = detector._load_template() + pixels = np.asarray(Image.open(fine_opponent_registered_positive).convert("RGB"), dtype=np.uint8) + components = registered_detector.fine_opponent_registered_components(pixels, template, sigma) + + assert components.selected_period == pytest.approx(7.68, abs=0.01) + assert components.fine_decision_score >= detector.FINE_OPPONENT_REGISTERED_THRESHOLD + assert components.candidate_count >= 100 + + +def test_period8_codec_veto_threshold_mutation_changes_real_components( + monkeypatch: pytest.MonkeyPatch, + opponent_period8_positive: Path, +) -> None: + import synthid_runtime._synthid_registered as registered_detector + + template, sigma, *_model = detector._load_template() + pixels = np.asarray(Image.open(opponent_period8_positive).convert("RGB"), dtype=np.uint8) + components = registered_detector.opponent_registered_components(pixels, template, sigma) + assert components.decision_score >= detector.OPPONENT_REGISTERED_THRESHOLD + assert components.red_green_p8_edge_ratio is not None + assert components.blue_yellow_p8_edge_ratio is not None + monkeypatch.setattr(registered_detector, "OPPONENT_REGISTERED_MAX_P8_EDGE_RATIO", 0.9) + + assert components.decision_score == 0.0 + + +def test_opponent_registered_period_band_and_codec_veto_are_required() -> None: + from synthid_runtime._synthid_registered import OpponentRegisteredComponents + + values = { + "spectral_score": 0.8, + "fixed_score": 0.32, + "red_green_spatial": 0.9, + "blue_yellow_spatial": 0.8, + "candidate_count": 3, + "red_green_p8_edge_ratio": None, + "blue_yellow_p8_edge_ratio": None, + } + matching = OpponentRegisteredComponents(10.0, 10.0, **values) + period8 = OpponentRegisteredComponents( + 8.0, + 8.0, + **{ + **values, + "red_green_p8_edge_ratio": 1.0, + "blue_yellow_p8_edge_ratio": 1.0, + }, + ) + codec_alias = OpponentRegisteredComponents( + 8.0, + 8.0, + **{ + **values, + "red_green_p8_edge_ratio": 1.2, + "blue_yellow_p8_edge_ratio": 1.2, + }, + ) + + assert matching.decision_score > detector.OPPONENT_REGISTERED_THRESHOLD + assert period8.decision_score > detector.OPPONENT_REGISTERED_THRESHOLD + assert codec_alias.base_decision_score > detector.OPPONENT_REGISTERED_THRESHOLD + assert codec_alias.decision_score == 0.0 + + +def test_registered_threshold_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + registered_scale_positive: Path, +) -> None: + baseline = detector.detect_synthid(registered_scale_positive, register_scale=True) + assert baseline.score is not None + mutated_threshold = float(np.nextafter(baseline.score, np.inf)) + monkeypatch.setattr(detector, "REGISTERED_THRESHOLD", mutated_threshold) + + mutated = detector.detect_synthid(registered_scale_positive, register_scale=True) + + assert mutated.status == "indeterminate" + assert mutated.threshold == mutated_threshold + + +def test_registered_period_thresholds_cover_the_bounded_search() -> None: + from synthid_runtime._synthid_registered import _period_threshold + + assert _period_threshold(7.5) == pytest.approx(0.3770629524888979) + assert _period_threshold(12.0) == pytest.approx(0.19794247706938645) + assert _period_threshold(24.5) == pytest.approx(0.3142958338390489) + with pytest.raises(ValueError, match="outside"): + _period_threshold(7.49) + + +def test_registered_amplitude_threshold_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + registered_scale_positive: Path, +) -> None: + import synthid_runtime._synthid_registered as registered_detector + + baseline = detector.detect_synthid(registered_scale_positive, register_scale=True) + assert baseline.status == "detected" + monkeypatch.setattr( + registered_detector, + "_PERIOD_THRESHOLDS", + ((7.5, 24.5, float("inf")),), + ) + + mutated = detector.detect_synthid(registered_scale_positive, register_scale=True) + + assert mutated.status == "indeterminate" + + +def test_registered_spectral_candidate_disagreement_blocks_decision() -> None: + from synthid_runtime._synthid_confirmation import RegisteredConfirmationComponents + from synthid_runtime._synthid_registered import RegisteredComponents + + confirmation = RegisteredConfirmationComponents(12.8, 0.5, 0.2, 0.5, 8, 8) + matching = RegisteredComponents(0.5, 0.25, 12.8, 12.8, 0.15, confirmation) + mismatching = RegisteredComponents(0.5, 0.25, 12.8, 12.9, 0.15, confirmation) + unconfirmed = RegisteredComponents(0.5, 0.25, 12.8, 12.8, 0.15) + + assert matching.decision_score == pytest.approx(2.0) + assert mismatching.decision_score == pytest.approx(0.0) + assert unconfirmed.base_decision_score == pytest.approx(2.0) + assert unconfirmed.decision_score == pytest.approx(0.0) + + +def test_registered_high_band_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + registered_scale_positive: Path, +) -> None: + import synthid_runtime._synthid_registered as registered_detector + + components = registered_detector.registered_components( + np.asarray(Image.open(registered_scale_positive).convert("RGB"), dtype=np.uint8), + detector._load_template()[0], + detector._load_template()[1], + ) + assert components.decision_score >= detector.REGISTERED_THRESHOLD + monkeypatch.setattr( + registered_detector, + "REGISTERED_HIGH_BAND_THRESHOLD", + float(np.nextafter(components.high_band_score, np.inf)), + ) + + mutated = detector.detect_synthid(registered_scale_positive, register_scale=True) + + assert mutated.status == "indeterminate" + + +def test_registered_confirmation_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + registered_scale_positive: Path, +) -> None: + import synthid_runtime._synthid_confirmation as confirmation_detector + import synthid_runtime._synthid_registered as registered_detector + + components = registered_detector.registered_components( + np.asarray(Image.open(registered_scale_positive).convert("RGB"), dtype=np.uint8), + detector._load_template()[0], + detector._load_template()[1], + ) + assert components.confirmation is not None + assert components.decision_score >= detector.REGISTERED_THRESHOLD + monkeypatch.setattr( + confirmation_detector, + "MIN_COHERENCE", + float(np.nextafter(components.confirmation.joint_coherence, np.inf)), + ) + + mutated = detector.detect_synthid(registered_scale_positive, register_scale=True) + + assert mutated.status == "indeterminate" + + +def test_supported_negative_does_not_claim_clean(supported_images: tuple[Path, Path]) -> None: + _positive, negative = supported_images + + result = detector.detect_synthid(negative, register_scale=False) + + assert result.status == "indeterminate" + assert result.detected is False + assert result.score == pytest.approx(0.0) + + +def test_threshold_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + supported_images: tuple[Path, Path], +) -> None: + positive, _negative = supported_images + baseline = detector.detect_synthid(positive, register_scale=False) + assert baseline.score is not None + assert baseline.status == "detected" + mutated_threshold = float(np.nextafter(baseline.score, np.inf)) + assert mutated_threshold > baseline.score + + monkeypatch.setattr(detector, "TILE_THRESHOLD", mutated_threshold) + mutated = detector.detect_synthid(positive, register_scale=False) + + assert mutated.status == "indeterminate" + assert mutated.threshold == mutated_threshold + + +def test_unsupported_geometry_is_distinct_from_negative(tmp_path: Path) -> None: + path = tmp_path / "small.png" + Image.new("RGB", (64, 32), "white").save(path) + + result = detector.detect_synthid(path, register_scale=False) + + assert result.status == "unsupported" + assert result.score is None + assert (result.width, result.height) == (64, 32) + assert result.reason is not None + assert result.to_dict()["metadata_used_for_verdict"] is False + assert result.to_dict()["provider_scope"] == "provider-neutral" + + +def test_shared_bgr_decode_matches_file_decode(supported_images: tuple[Path, Path]) -> None: + import cv2 + + positive, _negative = supported_images + bgr = cv2.imread(str(positive)) + assert bgr is not None + + from_file = detector.detect_synthid(positive, register_scale=False) + from_array = detector.detect_synthid(positive, image=bgr, register_scale=False) + + assert from_array == from_file + + +def test_supported_geometry_requires_pixel_dependencies( + monkeypatch: pytest.MonkeyPatch, + supported_images: tuple[Path, Path], +) -> None: + _positive, negative = supported_images + monkeypatch.setattr(detector, "is_available", lambda: False) + + with pytest.raises(RuntimeError, match="needs numpy and OpenCV"): + detector.detect_synthid(negative, register_scale=False) + + +def test_fold_accepts_non_divisible_geometry_without_resampling() -> None: + rng = np.random.default_rng(20260810) + tile = rng.normal(0.0, 8.0, size=(16, 16, 3)) + repeated = np.tile(tile, (19, 20, 1)) + 128.0 + + divisible = detector.fold_residual_template( + repeated, + tile_height=16, + tile_width=16, + denoise_sigma=1.0, + ) + non_divisible = detector.fold_residual_template( + repeated[:299, :317], + tile_height=16, + tile_width=16, + denoise_sigma=1.0, + ) + divisible_unit, _ = detector.unit_tile(divisible) + non_divisible_unit, _ = detector.unit_tile(non_divisible) + + assert non_divisible.shape == (16, 16, 3) + assert float(np.sum(divisible_unit * non_divisible_unit)) > 0.999 + + +def test_non_divisible_fold_matches_modulo_cell_means() -> None: + import cv2 + + rng = np.random.default_rng(44041) + pixels = rng.integers(0, 256, size=(53, 71, 3), dtype=np.uint8) + source = pixels.astype(np.float32) + # Mirror the module's documented per-channel blur: OpenCV's multi-channel + # GaussianBlur is not bit-identical to per-channel calls (observed 3e-5 on + # cv2 4.10.0), so a three-channel reference cannot satisfy atol=0. + residual = np.stack( + [ + source[:, :, channel] + - cv2.GaussianBlur( + source[:, :, channel].copy(), + (0, 0), + sigmaX=1.25, + sigmaY=1.25, + borderType=cv2.BORDER_REFLECT_101, + ) + for channel in range(3) + ], + axis=2, + ) + expected = np.empty((16, 16, 3), dtype=np.float64) + for tile_y in range(16): + for tile_x in range(16): + expected[tile_y, tile_x] = residual[tile_y::16, tile_x::16].mean( + axis=(0, 1), + dtype=np.float64, + ) + expected -= np.mean(expected, axis=(0, 1), keepdims=True) + + actual = detector.fold_residual_template( + pixels, + tile_height=16, + tile_width=16, + denoise_sigma=1.25, + ) + + np.testing.assert_allclose(actual, expected, rtol=0.0, atol=0.0) + + +def test_fold_rejects_tile_larger_than_image() -> None: + pixels = np.zeros((15, 16, 3), dtype=np.uint8) + + with pytest.raises(ValueError, match="at least as large"): + detector.fold_residual_template( + pixels, + tile_height=16, + tile_width=16, + denoise_sigma=1.0, + ) + + +def test_verdict_does_not_claim_the_watermark() -> None: + """The result must not assert SynthID, because the statistic is not SynthID. + + This was unguarded until 2026-08-16, and the claim had been wrong for months + without a single test noticing. The fields are pinned by value rather than by + presence so that a rename back to a watermark claim fails here. + """ + result = detector.SynthIDDetection( + status="detected", + width=4096, + height=2560, + score=1.0, + threshold=1.0, + ) + + payload = result.to_dict() + + assert payload["signal_family"] == "generation-pipeline-lattice" + assert payload["identifies_watermark"] is False + assert payload["tile_aligned_crop_required"] is True + assert "synthid" not in str(payload["signal_family"]).lower() + + +def test_the_statistic_is_locked_to_the_image_origin() -> None: + """A crop off the tile grid must destroy the score, and that must stay visible. + + SynthID's published evaluation retains 99.97% TPR under aggressive crop and + resize. This statistic loses everything to a seven-pixel shift, measured on + the real runtime at 4096x2560 where aligned crops scored up to 1.069 and + shifted ones reached -0.438. The property is asserted here so that any future + expert claiming to read the watermark has to survive the same shift first. + """ + template, sigma, *_model = detector._load_template() + tile = template / np.max(np.abs(template)) + pixels = np.full((1024, 1024, 3), 128.0) + pixels += 6.0 * np.tile(tile, (64, 64, 1)) + aligned = np.clip(np.rint(pixels), 0, 255).astype(np.uint8) + + aligned_score, _folded = detector.folded_template_score(aligned, template, sigma) + # Seven is deliberately coprime with the 16-pixel tile, so no residual phase survives. + shifted_score, _shifted_folded = detector.folded_template_score( + aligned[7:, 7:], + template, + sigma, + ) + + assert aligned_score > 0.5 + assert shifted_score < 0.1 * aligned_score diff --git a/tests/test_synthid_ensemble_attack.py b/tests/test_synthid_ensemble_attack.py new file mode 100644 index 0000000..aec1f4b --- /dev/null +++ b/tests/test_synthid_ensemble_attack.py @@ -0,0 +1,90 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_color_space_probe as probe +import synthid_ensemble_attack as attack + + +def _write_image(path: Path, *, phase: float, seed: int) -> None: + height = width = 64 + rng = np.random.default_rng(seed) + pixels = 100.0 + rng.normal(0.0, 2.0, size=(height, width, 3)) + yy, xx = np.mgrid[:height, :width] + wave = np.cos(2.0 * np.pi * (7.0 * yy / height + 5.0 * xx / width) + phase) + pixels[:, :, 0] += 14.0 * wave + pixels[:, :, 1] -= 10.0 * wave + Image.fromarray(np.clip(np.rint(pixels), 0, 255).astype(np.uint8), mode="RGB").save(path) + + +def test_projection_removes_only_positive_phase_component() -> None: + height = width = 64 + yy, xx = np.mgrid[:height, :width] + channel = np.cos(2.0 * np.pi * (7.0 * yy / height + 5.0 * xx / width) + 0.4) + before = np.fft.fft2(channel)[7, 5] + phase = np.angle(before) + + projected = attack._remove_positive_projection( + channel, + rows=np.asarray([7]), + columns=np.asarray([5]), + phases=np.asarray([phase]), + strength=1.0, + ) + after = np.fft.fft2(projected)[7, 5] + + assert np.real(after * np.exp(-1j * phase)) == pytest.approx(0.0, abs=1e-10) + assert np.max(np.abs(np.imag(np.fft.ifft2(np.fft.fft2(projected))))) < 1e-12 + + +def test_alternating_projection_reduces_rgb_and_sv_evidence(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + source = tmp_path / "source.png" + _write_image(source, phase=0.4, seed=10) + bins = np.asarray([(7, 5, channel) for channel in range(3)], dtype=np.int32) + rgb_model = probe.discover_model(positives, color_space="rgb", candidate_bins=bins, peak_count=3) + hsv_model = probe.discover_model(positives, color_space="hsv", candidate_bins=bins, peak_count=3) + before_rgb = probe.score_image(source, rgb_model) + before_hsv = probe.score_image(source, hsv_model) + + with Image.open(source) as image: + source_pixels = np.asarray(image.convert("RGB"), dtype=np.uint8) + projected = attack.alternating_projection( + source_pixels, + rgb_model, + hsv_model, + strength=1.0, + iterations=2, + ) + output = tmp_path / "projected.png" + Image.fromarray(projected, mode="RGB").save(output) + after_rgb = probe.score_image(output, rgb_model) + after_hsv = probe.score_image(output, hsv_model) + + assert after_rgb.evidence_score < before_rgb.evidence_score + assert sum(after_hsv.channel_evidence[1:]) < sum(before_hsv.channel_evidence[1:]) + + +@pytest.mark.parametrize("strength", [-0.1, 1.1]) +def test_projection_rejects_out_of_range_strength(strength: float) -> None: + channel = np.zeros((64, 64)) + + with pytest.raises(ValueError, match="between zero and one"): + attack._remove_positive_projection( + channel, + rows=np.asarray([7]), + columns=np.asarray([5]), + phases=np.asarray([0.0]), + strength=strength, + ) diff --git a/tests/test_synthid_ensemble_detector.py b/tests/test_synthid_ensemble_detector.py new file mode 100644 index 0000000..a0fb333 --- /dev/null +++ b/tests/test_synthid_ensemble_detector.py @@ -0,0 +1,86 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_ensemble_detector as detector +from synthid_color_space_probe import ColorPhaseScore + + +def _config(tmp_path: Path) -> detector.EnsembleConfig: + return detector.EnsembleConfig( + width=64, + height=64, + rgb_model_path=tmp_path / "rgb.npz", + rgb_model_sha256="0" * 64, + rgb_evidence_threshold=0.3, + rgb_active_threshold=0.5, + hsv_model_path=tmp_path / "hsv.npz", + hsv_model_sha256="1" * 64, + hsv_sv_evidence_threshold=0.3, + hsv_active_threshold=0.5, + ) + + +def _score( + *, color_space: str, evidence: float, active: float, channels: tuple[float, float, float] +) -> ColorPhaseScore: + return ColorPhaseScore( + path="fixture.png", + color_space=color_space, + phase_score=0.8, + active_weight_fraction=active, + evidence_score=evidence, + channel_evidence=channels, + selected_peak_counts=(80, 88, 88), + peak_count=256, + ) + + +def test_positive_requires_both_branches_and_support(tmp_path: Path) -> None: + config = _config(tmp_path) + rgb = _score(color_space="rgb", evidence=0.4, active=0.7, channels=(0.1, 0.1, 0.2)) + hsv = _score(color_space="hsv", evidence=0.45, active=0.8, channels=(0.05, 0.2, 0.2)) + + verdict = detector.classify_scores(Path("fixture.png"), rgb, hsv, config) + + assert verdict.verdict == "positive" + assert verdict.reason == "ensemble_pass" + + +def test_low_support_abstains_even_when_scores_pass(tmp_path: Path) -> None: + config = _config(tmp_path) + rgb = _score(color_space="rgb", evidence=0.4, active=0.49, channels=(0.1, 0.1, 0.2)) + hsv = _score(color_space="hsv", evidence=0.45, active=0.8, channels=(0.05, 0.2, 0.2)) + + verdict = detector.classify_scores(Path("fixture.png"), rgb, hsv, config) + + assert verdict.verdict == "abstain" + assert verdict.reason == "insufficient_support" + + +def test_branch_disagreement_abstains(tmp_path: Path) -> None: + config = _config(tmp_path) + rgb = _score(color_space="rgb", evidence=0.4, active=0.8, channels=(0.1, 0.1, 0.2)) + hsv = _score(color_space="hsv", evidence=0.2, active=0.8, channels=(0.02, 0.1, 0.08)) + + verdict = detector.classify_scores(Path("fixture.png"), rgb, hsv, config) + + assert verdict.verdict == "abstain" + assert verdict.reason == "branch_disagreement" + + +def test_unsupported_geometry_abstains_without_scoring(tmp_path: Path) -> None: + config = _config(tmp_path) + image_path = tmp_path / "small.png" + Image.new("RGB", (32, 32)).save(image_path) + + verdict = detector.detect_image(image_path, config, None, None) # type: ignore[arg-type] + + assert verdict.verdict == "abstain" + assert verdict.reason == "unsupported_geometry" + assert verdict.rgb_evidence is None diff --git a/tests/test_synthid_fragment_attack.py b/tests/test_synthid_fragment_attack.py new file mode 100644 index 0000000..dacaa6e --- /dev/null +++ b/tests/test_synthid_fragment_attack.py @@ -0,0 +1,92 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +import pytest + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_fragment_attack as attack + + +def _fixture() -> np.ndarray: + yy, xx = np.mgrid[:96, :128] + return ( + np.stack( + [ + 40.0 + 0.8 * xx, + 50.0 + 0.7 * yy, + 30.0 + 0.4 * xx + 0.3 * yy, + ], + axis=2, + ) + .clip(0, 255) + .astype(np.uint8) + ) + + +def test_affine_combo_preserves_geometry_and_is_deterministic() -> None: + source = _fixture() + + first = attack.affine_combo(source, rotation_degrees=0.2, zoom=0.004) + second = attack.affine_combo(source, rotation_degrees=0.2, zoom=0.004) + + assert first.shape == source.shape + assert first.dtype == np.uint8 + assert np.array_equal(first, second) + + +def test_bounded_smooth_warp_preserves_geometry_and_is_deterministic() -> None: + source = _fixture() + + first = attack.bounded_smooth_warp(source, max_displacement=1.8, sigma=8.0, seed=17) + second = attack.bounded_smooth_warp(source, max_displacement=1.8, sigma=8.0, seed=17) + + assert first.shape == source.shape + assert first.dtype == np.uint8 + assert np.array_equal(first, second) + assert not np.array_equal(first, source) + + +def test_zero_bounded_warp_is_pixel_identical() -> None: + source = _fixture() + + result = attack.bounded_smooth_warp(source, max_displacement=0.0, sigma=8.0, seed=17) + + assert np.array_equal(result, source) + + +def test_color_nudge_is_bounded_and_changes_pixels() -> None: + source = _fixture() + + result = attack.color_nudge( + source, + brightness=0.004, + contrast=0.006, + saturation=-0.005, + hue_degrees=0.15, + ) + + assert result.shape == source.shape + assert result.dtype == np.uint8 + assert not np.array_equal(result, source) + + +def test_jpeg_chain_rejects_bad_quality() -> None: + with pytest.raises(ValueError, match="quality"): + attack.jpeg_chain(_fixture(), (94, 101)) + + +def test_candidate_batch_has_control_target_and_sham() -> None: + candidates = attack.build_candidates(_fixture()) + + assert np.array_equal(candidates["control"], _fixture()) + assert "fragment-balanced" in candidates + assert "fragment-strong" in candidates + assert "sham-strong-rms" in candidates + assert "bounded-fragment-balanced" in candidates + assert "bounded-fragment-strong" in candidates + assert "sham-bounded-strong-rms" in candidates + assert all(pixels.shape == _fixture().shape for pixels in candidates.values()) diff --git a/tests/test_synthid_hybrid_attack.py b/tests/test_synthid_hybrid_attack.py new file mode 100644 index 0000000..d2db1e9 --- /dev/null +++ b/tests/test_synthid_hybrid_attack.py @@ -0,0 +1,50 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_color_space_probe as probe +import synthid_hybrid_attack as attack + + +def _write_image(path: Path, *, seed: int) -> None: + height = width = 64 + rng = np.random.default_rng(seed) + pixels = 100.0 + rng.normal(0.0, 2.0, size=(height, width, 3)) + yy, xx = np.mgrid[:height, :width] + wave = np.cos(2.0 * np.pi * (7.0 * yy / height + 5.0 * xx / width) + 0.4) + pixels[:, :, 0] += 14.0 * wave + pixels[:, :, 1] -= 10.0 * wave + Image.fromarray(np.clip(np.rint(pixels), 0, 255).astype(np.uint8), mode="RGB").save(path) + + +def test_hybrid_matrix_preserves_geometry_and_has_controls(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, seed=index) + positives.append(path) + source_path = tmp_path / "source.png" + _write_image(source_path, seed=10) + with Image.open(source_path) as image: + source = np.asarray(image.convert("RGB"), dtype=np.uint8) + bins = np.asarray([(7, 5, channel) for channel in range(3)], dtype=np.int32) + rgb_model = probe.discover_model(positives, color_space="rgb", candidate_bins=bins, peak_count=3) + hsv_model = probe.discover_model(positives, color_space="hsv", candidate_bins=bins, peak_count=3) + + candidates = attack.build_candidates(source, rgb_model, hsv_model) + + assert set(candidates) == { + "projection-075", + "bounded-100", + "projection-075-bounded-100", + "projection-075-bounded-polish", + "projection-100-elastic-075", + } + assert all(candidate.shape == source.shape for candidate in candidates.values()) + assert np.array_equal(candidates["projection-075"], source) is False diff --git a/tests/test_synthid_oracle_batch.py b/tests/test_synthid_oracle_batch.py new file mode 100644 index 0000000..b777ba7 --- /dev/null +++ b/tests/test_synthid_oracle_batch.py @@ -0,0 +1,212 @@ +from __future__ import annotations + +import json +import sys +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_oracle_batch as batch +from synthid_periodic_tile import fold_residual_template, unit_tile +from synthid_periodic_tile_probe import PeriodicTileModel +from synthid_periodic_tile_probe import save_model as save_tile_model +from synthid_phase_carrier import PhaseCarrierModel +from synthid_phase_carrier import save_model as save_phase_model + + +def _fixture(tmp_path: Path) -> tuple[Path, Path, Path]: + rng = np.random.default_rng(23) + raw_tile = rng.normal(size=(8, 8, 3)) + raw_tile -= np.mean(raw_tile, axis=(0, 1), keepdims=True) + raw_tile, _ = unit_tile(raw_tile) + source = np.clip(np.rint(128.0 + 24.0 * np.tile(raw_tile, (8, 8, 1))), 0, 255).astype(np.uint8) + source_path = tmp_path / "source.png" + Image.fromarray(source, mode="RGB").save(source_path) + + folded = fold_residual_template( + source, + tile_height=8, + tile_width=8, + denoise_sigma=1.0, + ) + template, expected_norm = unit_tile(folded) + tile_model = PeriodicTileModel(64, 64, 8, 8, 1.0, template, expected_norm) + tile_model_path = tmp_path / "tile-model.npz" + save_tile_model(tile_model_path, tile_model) + + spectra = np.stack([np.fft.rfft2(source[:, :, channel]) for channel in range(3)], axis=2) + magnitude = np.abs(spectra) + magnitude[0, 0, :] = 0.0 + row, column, channel = np.unravel_index(int(np.argmax(magnitude)), magnitude.shape) + phase_model = PhaseCarrierModel( + 64, + 64, + np.asarray([row], dtype=np.int32), + np.asarray([column], dtype=np.int32), + np.asarray([channel], dtype=np.int8), + np.asarray([np.angle(spectra[row, column, channel])]), + np.asarray([1.0]), + np.asarray([magnitude[row, column, channel]]), + ) + phase_model_path = tmp_path / "phase-model.npz" + save_phase_model(phase_model_path, phase_model) + return source_path, tile_model_path, phase_model_path + + +def _build(tmp_path: Path) -> tuple[Path, Path]: + source, tile_model, phase_model = _fixture(tmp_path) + output_dir = tmp_path / "oracle-batch" + manifest_path = batch.build_batch( + [source], + output_dir=output_dir, + tile_model_path=tile_model, + phase_model_path=phase_model, + tile_threshold=0.5, + phase_threshold=0.5, + active_threshold=0.0, + strength=2.0, + seed=20260810, + provider="google", + repository_root=Path(__file__).resolve().parent.parent, + ) + return output_dir, manifest_path + + +def test_build_preregisters_fixed_request_order_without_copying_source(tmp_path: Path) -> None: + output_dir, manifest_path = _build(tmp_path) + + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + + assert manifest["status"] == "preregistered_unsubmitted" + assert manifest["request_count"] == 5 + assert [row["role"] for row in manifest["rows"]] == list(batch.ROLE_ORDER) + assert manifest["rows"][0]["in_batch"] is False + assert not (output_dir / "source.png").exists() + assert manifest["provider"] == "google" + assert "shifted, and orthogonal_random are detected" in manifest["decision_rule"] + assert all(row["synthid_result"] is None for row in manifest["rows"]) + assert all(row["c2pa_result"] is None for row in manifest["rows"]) + template = json.loads((output_dir / "results-template.json").read_text(encoding="utf-8")) + assert template["manifest_sha256"] == batch.artifact_sha256(manifest_path) + assert [row["artifact_sha256"] for row in template["rows"]] == [row["artifact_sha256"] for row in manifest["rows"]] + assert ( + batch.verify_batch( + manifest_path, + repository_root=Path(__file__).resolve().parent.parent, + )["source_count"] + == 1 + ) + + +def test_verify_rejects_derivative_mutation(tmp_path: Path) -> None: + output_dir, manifest_path = _build(tmp_path) + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + aligned_row = next(row for row in manifest["rows"] if row["role"] == "aligned") + aligned_path = output_dir / aligned_row["path"] + with Image.open(aligned_path) as image: + pixels = np.asarray(image.convert("RGB"), dtype=np.uint8).copy() + pixels[0, 0, 0] ^= 1 + Image.fromarray(pixels, mode="RGB").save(aligned_path) + assert batch.artifact_sha256(aligned_path) != aligned_row["artifact_sha256"] + + with pytest.raises(ValueError, match="artifact hash mismatch"): + batch.verify_batch( + manifest_path, + repository_root=Path(__file__).resolve().parent.parent, + ) + + +def test_build_rejects_output_inside_repository(tmp_path: Path) -> None: + source, tile_model, phase_model = _fixture(tmp_path) + repository_root = Path(__file__).resolve().parent.parent + + with pytest.raises(ValueError, match="outside the repository"): + batch.build_batch( + [source], + output_dir=repository_root / ".local-eval/oracle-batch-test", + tile_model_path=tile_model, + phase_model_path=phase_model, + tile_threshold=0.5, + phase_threshold=0.5, + active_threshold=0.0, + strength=2.0, + seed=20260810, + provider="google", + repository_root=repository_root, + ) + + +def test_build_rejects_nonempty_output_directory(tmp_path: Path) -> None: + source, tile_model, phase_model = _fixture(tmp_path) + output_dir = tmp_path / "existing" + output_dir.mkdir() + (output_dir / "marker.txt").write_text("occupied", encoding="utf-8") + + with pytest.raises(ValueError, match="must not already contain"): + batch.build_batch( + [source], + output_dir=output_dir, + tile_model_path=tile_model, + phase_model_path=phase_model, + tile_threshold=0.5, + phase_threshold=0.5, + active_threshold=0.0, + strength=2.0, + seed=20260810, + provider="google", + repository_root=Path(__file__).resolve().parent.parent, + ) + + +def test_evaluate_requires_controls_and_aligned_outcome(tmp_path: Path) -> None: + output_dir, manifest_path = _build(tmp_path) + results = json.loads((output_dir / "results-template.json").read_text(encoding="utf-8")) + for row in results["rows"]: + row["synthid_result"] = "not_detected" if row["role"] == "aligned" else "detected" + row["c2pa_result"] = "unavailable" + row["raw_response"] = f"verbatim {row['role']} result" + row["submitted_at"] = "2026-08-10T20:00:00Z" + results_path = output_dir / "results.json" + results_path.write_text(json.dumps(results, indent=2) + "\n", encoding="utf-8") + + report = batch.evaluate_results( + manifest_path, + results_path, + repository_root=Path(__file__).resolve().parent.parent, + ) + + assert report["counts"] == { + "causal_success": 1, + "aligned_still_detected": 0, + "control_failed": 0, + "indeterminate": 0, + } + + shifted = next(row for row in results["rows"] if row["role"] == "shifted") + shifted["synthid_result"] = "not_detected" + results_path.write_text(json.dumps(results, indent=2) + "\n", encoding="utf-8") + control_report = batch.evaluate_results( + manifest_path, + results_path, + repository_root=Path(__file__).resolve().parent.parent, + ) + assert control_report["counts"]["control_failed"] == 1 + + +def test_evaluate_rejects_incomplete_result(tmp_path: Path) -> None: + output_dir, manifest_path = _build(tmp_path) + results = json.loads((output_dir / "results-template.json").read_text(encoding="utf-8")) + results["rows"].pop() + results_path = output_dir / "incomplete-results.json" + results_path.write_text(json.dumps(results, indent=2) + "\n", encoding="utf-8") + + with pytest.raises(ValueError, match="cover every"): + batch.evaluate_results( + manifest_path, + results_path, + repository_root=Path(__file__).resolve().parent.parent, + ) diff --git a/tests/test_synthid_periodic_tile_ablation.py b/tests/test_synthid_periodic_tile_ablation.py new file mode 100644 index 0000000..38fe18e --- /dev/null +++ b/tests/test_synthid_periodic_tile_ablation.py @@ -0,0 +1,122 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_periodic_tile_ablation as ablation +from synthid_periodic_tile import fold_residual_template, unit_tile +from synthid_periodic_tile_probe import PeriodicTileModel +from synthid_phase_carrier import PhaseCarrierModel + + +def test_exact_sign_test_detects_one_sided_direction() -> None: + assert ablation.exact_sign_test(30, 0) == pytest.approx(1.862645149230957e-9) + assert ablation.exact_sign_test(0, 0) == 1.0 + + +def test_control_templates_are_norm_matched_and_random_control_is_orthogonal() -> None: + rng = np.random.default_rng(7) + template, _ = unit_tile(rng.normal(size=(8, 8, 3))) + + controls = ablation.control_templates(template, seed=11) + + assert set(controls) == {"aligned", "shifted", "orthogonal_random"} + assert all(np.linalg.norm(control) == pytest.approx(1.0) for control in controls.values()) + assert np.sum(controls["orthogonal_random"] * template) == pytest.approx(0.0, abs=1e-12) + assert not np.array_equal(controls["shifted"], template) + + +def test_aligned_subtraction_controls_both_synthetic_representations(tmp_path: Path) -> None: + rng = np.random.default_rng(17) + raw_tile = rng.normal(size=(8, 8, 3)) + raw_tile -= np.mean(raw_tile, axis=(0, 1), keepdims=True) + raw_tile, _ = unit_tile(raw_tile) + source = np.clip(np.rint(128.0 + 24.0 * np.tile(raw_tile, (8, 8, 1))), 0, 255).astype(np.uint8) + source_path = tmp_path / "source.png" + Image.fromarray(source, mode="RGB").save(source_path) + + folded = fold_residual_template( + source, + tile_height=8, + tile_width=8, + denoise_sigma=1.0, + ) + template, expected_norm = unit_tile(folded) + tile_model = PeriodicTileModel( + height=64, + width=64, + tile_height=8, + tile_width=8, + denoise_sigma=1.0, + template=template, + expected_norm=expected_norm, + ) + + spectra = np.stack([np.fft.rfft2(source[:, :, channel]) for channel in range(3)], axis=2) + magnitude = np.abs(spectra) + magnitude[0, 0, :] = 0.0 + row, column, channel = np.unravel_index(int(np.argmax(magnitude)), magnitude.shape) + phase_model = PhaseCarrierModel( + height=64, + width=64, + rows=np.asarray([row], dtype=np.int32), + columns=np.asarray([column], dtype=np.int32), + channels=np.asarray([channel], dtype=np.int8), + phases=np.asarray([np.angle(spectra[row, column, channel])]), + weights=np.asarray([1.0]), + expected_magnitudes=np.asarray([magnitude[row, column, channel]]), + ) + + report = ablation.run_ablation( + [source_path], + tile_model=tile_model, + phase_model=phase_model, + tile_threshold=0.5, + phase_threshold=0.5, + active_threshold=0.0, + strengths=(1.0, 2.0), + phase_strength=2.0, + seed=20260810, + ) + + assert report["original"] == {"tile_accepted": 1, "phase_accepted": 1} + aligned = next(row for row in report["phase_summaries"] if row["control"] == "aligned") + assert aligned["accepted"] == 0 + comparisons = {(row["aligned_minus"], row["metric"]): row for row in report["paired_comparisons"]} + assert comparisons[("shifted", "tile_delta")]["difference"]["median"] < 0.0 + assert comparisons[("orthogonal_random", "tile_delta")]["difference"]["median"] < 0.0 + + +def test_phase_strength_must_be_part_of_sweep() -> None: + template = np.zeros((8, 8, 3), dtype=np.float64) + template[0, 0, 0] = 1.0 + tile_model = PeriodicTileModel(64, 64, 8, 8, 1.0, template, 1.0) + phase_model = PhaseCarrierModel( + 64, + 64, + np.asarray([1]), + np.asarray([1]), + np.asarray([0]), + np.asarray([0.0]), + np.asarray([1.0]), + np.asarray([1.0]), + ) + + with pytest.raises(ValueError, match="phase strength"): + ablation.run_ablation( + [Path("unused.png")], + tile_model=tile_model, + phase_model=phase_model, + tile_threshold=0.0, + phase_threshold=0.0, + active_threshold=0.0, + strengths=(1.0,), + phase_strength=2.0, + seed=1, + ) diff --git a/tests/test_synthid_periodic_tile_probe.py b/tests/test_synthid_periodic_tile_probe.py new file mode 100644 index 0000000..9fee720 --- /dev/null +++ b/tests/test_synthid_periodic_tile_probe.py @@ -0,0 +1,135 @@ +"""Tests for the periodic spatial-carrier research probe.""" + +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts")) + +import synthid_periodic_tile_probe as probe +from synthid_periodic_tile import cyclic_tile_correlations + + +def _carrier(seed: int, *, tile_height: int = 8, tile_width: int = 8) -> np.ndarray: + rng = np.random.default_rng(seed) + tile = rng.normal(size=(tile_height, tile_width, 3)) + return tile - np.mean(tile, axis=(0, 1), keepdims=True) + + +def _write_image(path: Path, carrier: np.ndarray, *, seed: int) -> None: + rng = np.random.default_rng(seed) + repeated = np.tile(carrier, (8, 8, 1)) + yy, xx = np.mgrid[:64, :64] + background = 110.0 + 0.2 * xx + 0.1 * yy + pixels = background[:, :, None] + 18.0 * repeated + rng.normal(scale=1.5, size=repeated.shape) + Image.fromarray(np.clip(np.rint(pixels), 0, 255).astype(np.uint8), mode="RGB").save(path) + + +def test_periodic_model_scores_matching_carrier_and_round_trips(tmp_path: Path) -> None: + carrier = _carrier(1) + positives = [] + for index in range(4): + path = tmp_path / f"positive-{index}.png" + _write_image(path, carrier, seed=index) + positives.append(path) + heldout = tmp_path / "heldout.png" + negative = tmp_path / "negative.png" + _write_image(heldout, carrier, seed=10) + _write_image(negative, _carrier(2), seed=11) + + model = probe.discover_model(positives, tile_height=8, tile_width=8) + matching = probe.score_image(heldout, model) + mismatching = probe.score_image(negative, model) + model_path = tmp_path / "model.npz" + probe.save_model(model_path, model) + restored = probe.load_model(model_path) + + assert matching.score > 0.9 + assert mismatching.score < 0.5 + assert matching.active_support > 0.0 + assert matching.repeat_count == 64 + assert np.array_equal(restored.template, model.template) + assert probe.score_image(heldout, restored).score == matching.score + + +def test_registration_recovers_cyclic_tile_shift(tmp_path: Path) -> None: + carrier = _carrier(3) + positives = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, carrier, seed=index) + positives.append(path) + source = tmp_path / "source.png" + shifted = tmp_path / "shifted.png" + _write_image(source, carrier, seed=20) + with Image.open(source) as image: + pixels = np.asarray(image).copy() + Image.fromarray(np.roll(pixels, shift=(1, 2), axis=(0, 1)), mode="RGB").save(shifted) + model = probe.discover_model(positives, tile_height=8, tile_width=8) + + fixed = probe.score_image(shifted, model) + registered = probe.score_image(shifted, model, register=True) + + assert registered.score > fixed.score + assert registered.score > 0.9 + assert (registered.row_shift, registered.column_shift) == (7, 6) + + +def test_array_scoring_matches_file_scoring(tmp_path: Path) -> None: + carrier = _carrier(12) + positives = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, carrier, seed=index) + positives.append(path) + model = probe.discover_model(positives, tile_height=8, tile_width=8) + with Image.open(positives[0]) as image: + pixels = np.asarray(image.convert("RGB"), dtype=np.uint8) + + file_score = probe.score_image(positives[0], model) + array_score = probe.score_pixels(pixels, model) + + assert array_score.score == pytest.approx(file_score.score) + assert array_score.active_support == pytest.approx(file_score.active_support) + assert array_score.path == "" + + +def test_fft_correlations_match_explicit_cyclic_shifts() -> None: + rng = np.random.default_rng(30) + template = rng.normal(size=(4, 5, 3)) + tile = rng.normal(size=(4, 5, 3)) + explicit = np.asarray( + [ + [np.sum(template * np.roll(tile, shift=(row, column), axis=(0, 1))) for column in range(5)] + for row in range(4) + ] + ) + + correlations = cyclic_tile_correlations(template, tile) + + assert correlations == pytest.approx(explicit) + + +def test_calibration_is_strictly_above_every_negative(tmp_path: Path) -> None: + carrier = _carrier(4) + positives = [] + negatives = [] + for index in range(3): + positive = tmp_path / f"positive-{index}.png" + negative = tmp_path / f"negative-{index}.png" + _write_image(positive, carrier, seed=index) + _write_image(negative, _carrier(10 + index), seed=20 + index) + positives.append(positive) + negatives.append(negative) + model = probe.discover_model(positives, tile_height=8, tile_width=8) + + threshold = probe.calibrate_threshold(negatives, model) + + assert all(probe.score_image(path, model).score < threshold for path in negatives) + with pytest.raises(ValueError, match="at least one calibration negative"): + probe.calibrate_threshold([], model) diff --git a/tests/test_synthid_phase_carrier.py b/tests/test_synthid_phase_carrier.py new file mode 100644 index 0000000..b394c57 --- /dev/null +++ b/tests/test_synthid_phase_carrier.py @@ -0,0 +1,165 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_phase_carrier as carrier + + +def _write_image(path: Path, *, phase: float | None, seed: int) -> None: + height = width = 64 + rng = np.random.default_rng(seed) + pixels = 100.0 + rng.normal(0.0, 3.0, size=(height, width, 3)) + if phase is not None: + yy, xx = np.mgrid[:height, :width] + wave = 12.0 * np.cos(2.0 * np.pi * (7.0 * yy / height + 5.0 * xx / width) + phase) + pixels += wave[:, :, None] + Image.fromarray(np.clip(np.rint(pixels), 0, 255).astype(np.uint8), mode="RGB").save(path) + + +def test_discovered_model_separates_shared_phase_from_noise(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(4): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + heldout = tmp_path / "heldout.png" + negative = tmp_path / "negative.png" + _write_image(heldout, phase=0.4, seed=10) + _write_image(negative, phase=None, seed=11) + + model = carrier.discover_model(positives, peak_count=8, min_radius=1.0) + + assert carrier.score_image(heldout, model).score > carrier.score_image(negative, model).score + + +def test_leave_one_out_coherence_exposes_phase_outlier() -> None: + units = np.asarray([1.0 + 0.0j, 1.0 + 0.0j, 1.0 + 0.0j, -1.0 + 0.0j]) + unit_sum = np.sum(units) + + coherences = [carrier._leave_one_out_coherence(unit_sum, unit, 4.0) for unit in units] + + assert min(coherences) == pytest.approx(1.0 / 3.0) + assert max(coherences) == pytest.approx(1.0) + + +def test_model_round_trip_is_pickle_free(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + model = carrier.discover_model(positives, peak_count=4, min_radius=1.0) + artifact = tmp_path / "model.npz" + + carrier.save_model(artifact, model) + loaded = carrier.load_model(artifact) + + assert loaded.height == model.height + assert loaded.width == model.width + assert np.array_equal(loaded.rows, model.rows) + assert np.isclose(np.sum(loaded.weights), 1.0) + + +def test_candidate_bins_restrict_discovery(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + bins = np.asarray([[7, 5, 0], [7, 5, 1], [7, 5, 2]], dtype=np.int32) + + model = carrier.discover_model(positives, peak_count=3, min_radius=1.0, candidate_bins=bins) + + actual = set(zip(model.rows.tolist(), model.columns.tolist(), model.channels.tolist(), strict=True)) + expected = set(map(tuple, bins.tolist())) + assert actual == expected + + +def test_discovery_requires_three_images(tmp_path: Path) -> None: + path = tmp_path / "positive.png" + _write_image(path, phase=0.4, seed=1) + + with pytest.raises(ValueError, match="at least three"): + carrier.discover_model([path, path], peak_count=4) + + +def test_scoring_rejects_geometry_mismatch(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + model = carrier.discover_model(positives, peak_count=4, min_radius=1.0) + mismatch = tmp_path / "mismatch.png" + Image.new("RGB", (80, 64)).save(mismatch) + + with pytest.raises(ValueError, match="does not match"): + carrier.score_image(mismatch, model) + + +def test_array_scoring_matches_file_scoring(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + model = carrier.discover_model(positives, peak_count=4, min_radius=1.0) + with Image.open(positives[0]) as image: + pixels = np.asarray(image.convert("RGB"), dtype=np.uint8) + + file_score = carrier.score_image(positives[0], model) + array_score = carrier.score_pixels(pixels, model) + + assert array_score.score == pytest.approx(file_score.score) + assert array_score.active_weight_fraction == pytest.approx(file_score.active_weight_fraction) + assert array_score.path == "" + + +def test_scoring_can_canonicalize_geometry(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(3): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + model = carrier.discover_model(positives, peak_count=4, min_radius=1.0) + mismatch = tmp_path / "mismatch.png" + Image.new("RGB", (80, 64), color=(100, 100, 100)).save(mismatch) + + score = carrier.score_image(mismatch, model, canonicalize_geometry=True) + + assert score.path == str(mismatch) + assert score.peak_count == 4 + + +def test_translation_search_recovers_shifted_carrier(tmp_path: Path) -> None: + positives: list[Path] = [] + for index in range(4): + path = tmp_path / f"positive-{index}.png" + _write_image(path, phase=0.4, seed=index) + positives.append(path) + heldout = tmp_path / "heldout.png" + shifted = tmp_path / "shifted.png" + _write_image(heldout, phase=0.4, seed=10) + with Image.open(heldout) as source: + pixels = np.asarray(source).copy() + Image.fromarray(np.roll(pixels, shift=(1, 1), axis=(0, 1)), mode="RGB").save(shifted) + model = carrier.discover_model(positives, peak_count=8, min_radius=1.0) + + fixed = carrier.score_image(shifted, model) + unregistered = carrier.score_translations(shifted, model, max_shift=0) + registered = carrier.score_translations(shifted, model, max_shift=2) + + assert unregistered.score == pytest.approx(fixed.score) + assert unregistered.active_weight_fraction == pytest.approx(fixed.active_weight_fraction) + assert registered.score > fixed.score + assert abs(registered.row_shift) <= 2 + assert abs(registered.column_shift) <= 2 + with pytest.raises(ValueError, match="between 0 and 32"): + carrier.score_translations(shifted, model, max_shift=33) diff --git a/tests/test_synthid_pixel_attack.py b/tests/test_synthid_pixel_attack.py new file mode 100644 index 0000000..82c7676 --- /dev/null +++ b/tests/test_synthid_pixel_attack.py @@ -0,0 +1,62 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +import pytest + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_pixel_attack as attack + + +def _fixture() -> np.ndarray: + yy, xx = np.mgrid[:128, :160] + channels = [ + 80 + xx * 0.4 + yy * 0.2, + 60 + xx * 0.3 + yy * 0.5, + 40 + xx * 0.6 + yy * 0.1, + ] + return np.clip(np.stack(channels, axis=2), 0, 255).astype(np.uint8) + + +def test_quantization_is_bounded_and_deterministic() -> None: + pixels = _fixture() + + first = attack.quantize(pixels, 4) + second = attack.quantize(pixels, 4) + + assert np.array_equal(first, second) + assert np.max(np.abs(first.astype(int) - pixels.astype(int))) <= 2 + + +def test_smooth_warp_preserves_geometry_and_is_deterministic() -> None: + pixels = _fixture() + + first = attack.smooth_warp(pixels, amplitude=0.35, sigma=8.0, seed=17) + second = attack.smooth_warp(pixels, amplitude=0.35, sigma=8.0, seed=17) + + assert first.shape == pixels.shape + assert first.dtype == np.uint8 + assert np.array_equal(first, second) + assert not np.array_equal(first, pixels) + + +def test_norm_matched_control_has_similar_rms(tmp_path: Path) -> None: + pixels = _fixture() + target = attack.quantize(pixels, 8) + + sham = attack.norm_matched_noise(pixels, target, seed=23) + target_metrics = attack.measure(pixels, target, name="target", path=tmp_path / "target.png") + sham_metrics = attack.measure(pixels, sham, name="sham", path=tmp_path / "sham.png") + + assert sham_metrics.residual_rms == pytest.approx(target_metrics.residual_rms, rel=0.1) + + +def test_crop_visible_badge() -> None: + pixels = _fixture() + + cropped = attack.crop_visible_badge(pixels, 16) + + assert cropped.shape == (112, 144, 3) diff --git a/tests/test_synthid_research_inventory.py b/tests/test_synthid_research_inventory.py new file mode 100644 index 0000000..86b1231 --- /dev/null +++ b/tests/test_synthid_research_inventory.py @@ -0,0 +1,107 @@ +"""Tests for label-free local SynthID research inventories.""" + +from __future__ import annotations + +import csv +import sys +from pathlib import Path + +import pytest +from PIL import Image +from PIL.PngImagePlugin import PngInfo + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_research_inventory as inventory + + +def _write_png(path: Path, color: tuple[int, int, int], *, note: str | None = None) -> None: + image = Image.new("RGB", (9, 7), color) + pnginfo = None + if note is not None: + pnginfo = PngInfo() + pnginfo.add_text("note", note) + image.save(path, format="PNG", pnginfo=pnginfo) + + +def test_inventory_is_stable_and_contains_no_evidence_labels(tmp_path: Path): + media = tmp_path / "media" + media.mkdir() + _write_png(media / "b.png", (20, 30, 40)) + _write_png(media / "a.png", (10, 20, 30)) + + rows = inventory.build_inventory(tmp_path, (Path("media"),)) + + assert [row.artifact_path for row in rows] == ["media/a.png", "media/b.png"] + assert set(inventory.FIELDNAMES).isdisjoint({"target_provider", "synthid_outcome", "verified_via", "split"}) + assert all(row.format == "png" and row.width == 9 and row.height == 7 for row in rows) + + +def test_inventory_marks_byte_and_pixel_duplicates(tmp_path: Path): + media = tmp_path / "media" + media.mkdir() + first = media / "first.png" + exact = media / "exact.png" + metadata_variant = media / "metadata.png" + _write_png(first, (10, 20, 30), note="one") + exact.write_bytes(first.read_bytes()) + _write_png(metadata_variant, (10, 20, 30), note="different metadata") + + rows = {row.artifact_path: row for row in inventory.build_inventory(tmp_path, (Path("media"),))} + + assert rows["media/exact.png"].artifact_duplicate_of == "" + assert rows["media/first.png"].artifact_duplicate_of == "media/exact.png" + assert rows["media/first.png"].pixel_duplicate_of == "media/exact.png" + assert rows["media/metadata.png"].artifact_duplicate_of == "" + assert rows["media/metadata.png"].pixel_duplicate_of == "media/exact.png" + assert len({row.exact_pixel_group for row in rows.values()}) == 1 + + +def test_discovery_rejects_source_outside_root(tmp_path: Path): + root = tmp_path / "root" + root.mkdir() + outside = tmp_path / "outside.png" + _write_png(outside, (10, 20, 30)) + + with pytest.raises(ValueError, match="outside inventory root"): + inventory.discover_images(root, (outside,)) + + +def test_inventory_uses_content_format_not_suffix(tmp_path: Path): + disguised = tmp_path / "disguised.jpg" + _write_png(disguised, (10, 20, 30)) + + row = inventory.build_inventory(tmp_path, (Path("disguised.jpg"),))[0] + + assert row.format == "png" + + +def test_write_refuses_to_replace_without_explicit_flag(tmp_path: Path): + media = tmp_path / "image.png" + _write_png(media, (10, 20, 30)) + rows = inventory.build_inventory(tmp_path, (Path("image.png"),)) + output = tmp_path / "inventory.csv" + inventory.write_inventory(output, rows) + + with pytest.raises(FileExistsError, match="--replace"): + inventory.write_inventory(output, rows) + + inventory.write_inventory(output, rows, replace=True) + with output.open(newline="", encoding="utf-8") as stream: + written = list(csv.DictReader(stream)) + assert written[0]["artifact_path"] == "image.png" + + +def test_summary_reports_only_aggregates(tmp_path: Path): + _write_png(tmp_path / "image.png", (10, 20, 30)) + rows = inventory.build_inventory(tmp_path, (Path("image.png"),)) + + summary = inventory.inventory_summary(rows) + + assert summary == { + "images": 1, + "unique_artifacts": 1, + "unique_pixels": 1, + "formats": {"png": 1}, + "geometries": {"9x7": 1}, + } diff --git a/tests/test_synthid_research_manifest.py b/tests/test_synthid_research_manifest.py new file mode 100644 index 0000000..706609d --- /dev/null +++ b/tests/test_synthid_research_manifest.py @@ -0,0 +1,252 @@ +"""Tests for the private SynthID research-manifest auditor.""" + +from __future__ import annotations + +import csv +import hashlib +import sys +from pathlib import Path + +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_research_manifest as manifest + + +def _write_image(path: Path, color: tuple[int, int, int]) -> tuple[str, str]: + image = Image.new("RGB", (8, 6), color) + image.save(path) + artifact_sha = hashlib.sha256(path.read_bytes()).hexdigest() + pixel_sha = hashlib.sha256(image.tobytes()).hexdigest() + return artifact_sha, pixel_sha + + +def _row(artifact_sha: str, pixel_sha: str, *, path: str = "image.png", **updates: str) -> dict[str, str]: + row = { + "artifact_sha256": artifact_sha, + "pixel_sha256": pixel_sha, + "artifact_path": path, + "parent_sha256": "", + "group_id": "group-1", + "target_provider": "openai", + "source_provider": "openai", + "surface": "api", + "model_epoch": "gpt-image-2026-08", + "generation_session": "session-1", + "content_stratum": "flat-graphic", + "width": "8", + "height": "6", + "format": "png", + "transform": "original", + "split": "train", + "c2pa_outcome": "detected", + "synthid_outcome": "detected", + "verified_via": "openai-api", + "evidence_reference": "", + "oracle_session": "oracle-1", + "oracle_role": "ordinary", + "captured_at": "2026-08-08T12:00:00Z", + "oracle_checked_at": "2026-08-08T12:05:00Z", + "notes": "synthetic test row", + } + row.update(updates) + return row + + +def _write_manifest(path: Path, rows: list[dict[str, str]]) -> None: + with path.open("w", newline="", encoding="utf-8") as stream: + writer = csv.DictWriter(stream, fieldnames=manifest.FIELDNAMES) + writer.writeheader() + writer.writerows(rows) + + +def test_valid_manifest_and_files_pass(tmp_path: Path): + artifact = tmp_path / "image.png" + artifact_sha, pixel_sha = _write_image(artifact, (10, 20, 30)) + path = tmp_path / "manifest.csv" + _write_manifest(path, [_row(artifact_sha, pixel_sha)]) + + assert manifest.audit_manifest(path, verify_files=True) == [] + + +def test_rejects_same_provider_negative_without_matching_oracle(tmp_path: Path): + artifact_sha = "a" * 64 + pixel_sha = "b" * 64 + path = tmp_path / "manifest.csv" + row = _row( + artifact_sha, + pixel_sha, + synthid_outcome="not_detected", + verified_via="source-evidence", + oracle_session="", + ) + _write_manifest(path, [row]) + + errors = manifest.audit_manifest(path) + + assert any("same-provider negative" in error for error in errors) + + +def test_rejects_external_source_evidence_without_reference(tmp_path: Path): + path = tmp_path / "manifest.csv" + row = _row( + "a" * 64, + "b" * 64, + source_provider="camera", + synthid_outcome="not_detected", + verified_via="source-evidence", + oracle_session="", + c2pa_outcome="not_present", + ) + _write_manifest(path, [row]) + + errors = manifest.audit_manifest(path) + + assert any("source-evidence requires evidence_reference" in error for error in errors) + + +def test_accepts_external_source_evidence_with_reference(tmp_path: Path): + path = tmp_path / "manifest.csv" + row = _row( + "a" * 64, + "b" * 64, + source_provider="camera", + synthid_outcome="not_detected", + verified_via="source-evidence", + evidence_reference="https://example.test/original-record", + oracle_session="", + c2pa_outcome="not_present", + ) + _write_manifest(path, [row]) + + assert manifest.audit_manifest(path) == [] + + +def test_rejects_indeterminate_training_label(tmp_path: Path): + path = tmp_path / "manifest.csv" + row = _row( + "a" * 64, + "b" * 64, + synthid_outcome="indeterminate", + verified_via="openai-api", + oracle_checked_at="", + ) + _write_manifest(path, [row]) + + errors = manifest.audit_manifest(path) + + assert any("requires a detected or not_detected" in error for error in errors) + + +def test_rejects_group_leakage_across_splits(tmp_path: Path): + first = _row("a" * 64, "b" * 64) + second = _row( + "c" * 64, + "d" * 64, + artifact_path="other.png", + split="test", + generation_session="session-2", + ) + path = tmp_path / "manifest.csv" + _write_manifest(path, [first, second]) + + errors = manifest.audit_manifest(path) + + assert any("leaks across splits" in error for error in errors) + + +def test_rejects_derivative_with_missing_parent(tmp_path: Path): + path = tmp_path / "manifest.csv" + row = _row("a" * 64, "b" * 64, transform="jpeg-q90", parent_sha256="c" * 64) + _write_manifest(path, [row]) + + errors = manifest.audit_manifest(path) + + assert any("parent_sha256 is not present" in error for error in errors) + + +def test_rejects_identical_pixels_in_different_groups(tmp_path: Path): + first = _row("a" * 64, "b" * 64) + second = _row( + "c" * 64, + "b" * 64, + artifact_path="other.png", + group_id="group-2", + generation_session="session-2", + ) + path = tmp_path / "manifest.csv" + _write_manifest(path, [first, second]) + + errors = manifest.audit_manifest(path) + + assert any("appears in multiple groups" in error for error in errors) + + +def test_verify_files_detects_changed_artifact(tmp_path: Path): + artifact = tmp_path / "image.png" + artifact_sha, pixel_sha = _write_image(artifact, (10, 20, 30)) + path = tmp_path / "manifest.csv" + _write_manifest(path, [_row(artifact_sha, pixel_sha)]) + _write_image(artifact, (11, 21, 31)) + + errors = manifest.audit_manifest(path, verify_files=True) + + assert any("artifact_sha256 does not match" in error for error in errors) + assert any("pixel_sha256 does not match" in error for error in errors) + + +def test_rejects_path_traversal_during_file_verification(tmp_path: Path): + path = tmp_path / "manifest.csv" + _write_manifest(path, [_row("a" * 64, "b" * 64, path="../outside.png")]) + + errors = manifest.audit_manifest(path, verify_files=True) + + assert any("safe manifest-relative path" in error for error in errors) + + +def test_accepts_candidate_negative_with_detected_session_control(tmp_path: Path): + control = _row("a" * 64, "b" * 64, oracle_role="source_control") + candidate = _row( + "c" * 64, + "d" * 64, + artifact_path="candidate.png", + parent_sha256="a" * 64, + transform="carrier-subtract", + synthid_outcome="not_detected", + oracle_role="candidate", + ) + path = tmp_path / "manifest.csv" + _write_manifest(path, [control, candidate]) + + assert manifest.audit_manifest(path) == [] + + +def test_rejects_candidate_negative_without_detected_session_control(tmp_path: Path): + candidate = _row( + "a" * 64, + "b" * 64, + synthid_outcome="not_detected", + oracle_role="candidate", + ) + path = tmp_path / "manifest.csv" + _write_manifest(path, [candidate]) + + errors = manifest.audit_manifest(path) + + assert any("requires a detected source_control" in error for error in errors) + + +def test_rejects_failed_source_control(tmp_path: Path): + control = _row( + "a" * 64, + "b" * 64, + synthid_outcome="not_detected", + oracle_role="source_control", + ) + path = tmp_path / "manifest.csv" + _write_manifest(path, [control]) + + errors = manifest.audit_manifest(path) + + assert any("source_control must have a detected" in error for error in errors) diff --git a/tests/test_synthid_routed_detector.py b/tests/test_synthid_routed_detector.py new file mode 100644 index 0000000..d9d72cc --- /dev/null +++ b/tests/test_synthid_routed_detector.py @@ -0,0 +1,78 @@ +from __future__ import annotations + +import json +import sys +from pathlib import Path + +from click.testing import CliRunner + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_routed_detector as detector +import synthid_routed_expert_bank as bank + + +def test_detect_path_routes_one_scored_record(monkeypatch, tmp_path: Path) -> None: + image_path = tmp_path / "image.png" + image_path.write_bytes(b"fixture") + + def score_path(path: Path) -> dict[str, object]: + assert path == image_path + return { + "id": "a" * 64, + "path": str(path), + "width": 1024, + "height": 1024, + "observations": [ + { + "name": bank.synthid_detector.DETECTOR_ID, + "supported": True, + "score": 0.5, + }, + { + "name": bank.synthid_detector.REGISTERED_DETECTOR_ID, + "supported": True, + "score": 0.0, + }, + { + "name": bank.synthid_detector.LARGE_DETECTOR_ID, + "supported": False, + "score": None, + }, + ], + } + + monkeypatch.setattr(detector, "score_path", score_path) + + result = detector.detect_path(image_path) + + assert result["result"]["verdict"] == "abstain" + assert result["result"]["reason"] == "fixed_only_ambiguous" + + +def test_cli_writes_combined_hash_pinned_report(monkeypatch, tmp_path: Path) -> None: + image_path = tmp_path / "image.png" + report_path = tmp_path / "report.json" + image_path.write_bytes(b"fixture") + monkeypatch.setattr( + detector, + "detect_path", + lambda path: { + "id": "b" * 64, + "path": str(path), + "width": 1024, + "height": 1024, + "observations": [], + "result": {"verdict": "detected", "reason": "registered_threshold_crossed"}, + }, + ) + + result = CliRunner().invoke( + detector.main, + [str(image_path), "--report-out", str(report_path)], + ) + + assert result.exit_code == 0, result.output + report = json.loads(report_path.read_text(encoding="utf-8")) + assert report["counts"] == {"detected": 1, "abstain": 0} + assert report["records"][0]["id"] == "b" * 64 diff --git a/tests/test_synthid_routed_expert_bank.py b/tests/test_synthid_routed_expert_bank.py new file mode 100644 index 0000000..6589db1 --- /dev/null +++ b/tests/test_synthid_routed_expert_bank.py @@ -0,0 +1,138 @@ +from __future__ import annotations + +import json +import sys +from pathlib import Path + +import pytest +from click.testing import CliRunner + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_routed_expert_bank as bank +from synthid_conformal_cascade import ExpertObservation + + +def _observations( + fixed_score: float | None, + registered_score: float | None, + *, + fixed_supported: bool = True, + registered_supported: bool = True, + large_score: float | None = None, + large_supported: bool = False, +) -> tuple[ExpertObservation, ...]: + return ( + ExpertObservation(bank.synthid_detector.DETECTOR_ID, fixed_supported, fixed_score), + ExpertObservation( + bank.synthid_detector.REGISTERED_DETECTOR_ID, + registered_supported, + registered_score, + ), + ExpertObservation( + bank.synthid_detector.LARGE_DETECTOR_ID, + large_supported, + large_score, + ), + ) + + +def test_registered_crossing_is_the_only_positive_route() -> None: + result = bank.classify_routed(_observations(-1.0, 1.1)) + + assert result.verdict == "detected" + assert result.reason == "registered_threshold_crossed" + assert result.selected_expert == bank.synthid_detector.REGISTERED_DETECTOR_ID + + +def test_large_crossing_is_a_separate_positive_route() -> None: + result = bank.classify_routed( + _observations( + None, + None, + fixed_supported=False, + registered_supported=False, + large_score=1.1, + large_supported=True, + ) + ) + + assert result.verdict == "detected" + assert result.reason == "large_threshold_crossed" + assert result.selected_expert == bank.synthid_detector.LARGE_DETECTOR_ID + + +def test_fixed_crossing_in_overlapping_geometry_abstains() -> None: + result = bank.classify_routed(_observations(0.5, 0.0)) + + assert result.verdict == "abstain" + assert result.reason == "fixed_only_ambiguous" + + +def test_fixed_crossing_outside_registered_geometry_abstains() -> None: + result = bank.classify_routed( + _observations(0.5, None, registered_supported=False), + ) + + assert result.verdict == "abstain" + assert result.reason == "fixed_only_geometry_uncalibrated" + + +def test_unsupported_bank_abstains() -> None: + result = bank.classify_routed( + _observations(None, None, fixed_supported=False, registered_supported=False), + ) + + assert result.verdict == "abstain" + assert result.reason == "unsupported" + + +def test_observations_must_cover_the_exact_routed_bank() -> None: + with pytest.raises(ValueError, match=r"missing=.*synthid-periodic-tile-large-v1"): + bank.classify_routed((ExpertObservation(bank.synthid_detector.DETECTOR_ID, True, 0.5),)) + + +def test_cli_writes_hash_pinned_report(tmp_path: Path) -> None: + observations_path = tmp_path / "observations.json" + report_path = tmp_path / "report.json" + observations_path.write_text( + json.dumps( + { + "schema_version": 1, + "records": [ + { + "id": "candidate-1", + "observations": [ + { + "name": bank.synthid_detector.DETECTOR_ID, + "supported": True, + "score": 0.5, + }, + { + "name": bank.synthid_detector.REGISTERED_DETECTOR_ID, + "supported": True, + "score": 0.0, + }, + { + "name": bank.synthid_detector.LARGE_DETECTOR_ID, + "supported": False, + "score": None, + }, + ], + } + ], + } + ), + encoding="utf-8", + ) + + result = CliRunner().invoke( + bank.main, + [str(observations_path), "--report-out", str(report_path)], + ) + + assert result.exit_code == 0, result.output + report = json.loads(report_path.read_text(encoding="utf-8")) + assert len(report["observation_sha256"]) == 64 + assert report["counts"] == {"detected": 0, "abstain": 1} + assert report["records"][0]["result"]["reason"] == "fixed_only_ambiguous" diff --git a/tests/test_synthid_runtime_expert_scores.py b/tests/test_synthid_runtime_expert_scores.py new file mode 100644 index 0000000..71ba852 --- /dev/null +++ b/tests/test_synthid_runtime_expert_scores.py @@ -0,0 +1,97 @@ +from __future__ import annotations + +import json +import sys +from pathlib import Path + +import numpy as np +from click.testing import CliRunner +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_runtime_expert_scores as scorer + + +def test_unsupported_geometry_emits_no_synthetic_scores() -> None: + observations = scorer.score_pixels(np.zeros((64, 64, 3), dtype=np.uint8)) + + assert observations == [ + {"name": scorer.FIXED_EXPERT_NAME, "supported": False, "score": None}, + {"name": scorer.REGISTERED_EXPERT_NAME, "supported": False, "score": None}, + {"name": scorer.LARGE_EXPERT_NAME, "supported": False, "score": None}, + ] + + +def test_supported_image_scores_each_expert_once(monkeypatch) -> None: + calls = {"fixed": 0, "registered": 0} + + def detect(path, *, image, register_scale=False): + branch = "registered" if register_scale else "fixed" + calls[branch] += 1 + return scorer.synthid_detector.SynthIDDetection( + status="detected", + width=1024, + height=1024, + score=1.5 if register_scale else 0.25, + threshold=1.0 if register_scale else 0.17, + ) + + monkeypatch.setattr(scorer.synthid_detector, "detect_synthid", detect) + + observations = scorer.score_pixels(np.zeros((1024, 1024, 3), dtype=np.uint8)) + + assert observations == [ + {"name": scorer.FIXED_EXPERT_NAME, "supported": True, "score": 0.25}, + {"name": scorer.REGISTERED_EXPERT_NAME, "supported": True, "score": 1.5}, + {"name": scorer.LARGE_EXPERT_NAME, "supported": False, "score": None}, + ] + assert calls == {"fixed": 1, "registered": 1} + + +def test_pixels_must_be_rgb_uint8() -> None: + with np.testing.assert_raises_regex(ValueError, "RGB uint8"): + scorer.score_pixels(np.zeros((64, 64, 3), dtype=np.float32)) + + +def test_cli_writes_hash_pinned_observation_manifest(tmp_path: Path) -> None: + image_path = tmp_path / "small.png" + report_path = tmp_path / "scores.json" + Image.new("RGB", (64, 64), (1, 2, 3)).save(image_path) + + result = CliRunner().invoke(scorer.main, [str(image_path), "--report-out", str(report_path)]) + + assert result.exit_code == 0, result.output + report = json.loads(report_path.read_text(encoding="utf-8")) + assert report["schema_version"] == 1 + assert report["experts"] == [ + scorer.FIXED_EXPERT_NAME, + scorer.REGISTERED_EXPERT_NAME, + scorer.LARGE_EXPERT_NAME, + ] + assert len(report["records"][0]["id"]) == 64 + assert report["records"][0]["width"] == 64 + assert all(not observation["supported"] for observation in report["records"][0]["observations"]) + + +def test_large_default_is_not_mislabeled_as_fixed(monkeypatch) -> None: + def detect(path, *, image, register_scale=False): + detector_id = scorer.REGISTERED_EXPERT_NAME if register_scale else scorer.LARGE_EXPERT_NAME + return scorer.synthid_detector.SynthIDDetection( + status="unsupported" if register_scale else "detected", + width=4096, + height=4096, + score=None if register_scale else 1.2, + threshold=1.0, + detector=detector_id, + ) + + monkeypatch.setattr(scorer.synthid_detector, "detect_synthid", detect) + + observations = scorer.score_pixels(np.zeros((4096, 4096, 3), dtype=np.uint8)) + + assert observations == [ + {"name": scorer.FIXED_EXPERT_NAME, "supported": False, "score": None}, + {"name": scorer.REGISTERED_EXPERT_NAME, "supported": False, "score": None}, + {"name": scorer.LARGE_EXPERT_NAME, "supported": True, "score": 1.2}, + ] diff --git a/tests/test_synthid_spectral_probe.py b/tests/test_synthid_spectral_probe.py new file mode 100644 index 0000000..d3286fa --- /dev/null +++ b/tests/test_synthid_spectral_probe.py @@ -0,0 +1,182 @@ +"""Synthetic tests for the paired SynthID spectral research harness.""" + +from __future__ import annotations + +import sys +from itertools import combinations +from pathlib import Path + +import numpy as np +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_spectral_probe as probe + + +def _carrier(height: int, width: int) -> np.ndarray: + yy, xx = np.mgrid[:height, :width] + channels = [ + np.cos(2 * np.pi * (14 * xx / width + 14 * yy / height)), + 1.3 * np.cos(2 * np.pi * (14 * xx / width + 14 * yy / height) + 0.2), + 0.8 * np.cos(2 * np.pi * (14 * xx / width + 14 * yy / height) - 0.3), + ] + return np.stack(channels, axis=2) + + +def _write_pair(tmp_path: Path, name: str, shape: tuple[int, int], base: int) -> tuple[Path, Path]: + height, width = shape + yy, xx = np.mgrid[:height, :width] + scene = base + 8 * np.sin(2 * np.pi * xx / width) + 5 * np.cos(2 * np.pi * yy / height) + clean = np.stack([scene, scene + 3, scene - 3], axis=2) + marked = clean + _carrier(height, width) + clean_path = tmp_path / f"{name}-clean.png" + marked_path = tmp_path / f"{name}-marked.png" + Image.fromarray(np.clip(np.rint(clean), 0, 255).astype(np.uint8)).save(clean_path) + Image.fromarray(np.clip(np.rint(marked), 0, 255).astype(np.uint8)).save(marked_path) + return clean_path, marked_path + + +def test_pair_residual_preserves_float_signal_across_shapes(tmp_path: Path): + first = _write_pair(tmp_path, "first", (96, 96), 100) + second = _write_pair(tmp_path, "second", (128, 80), 140) + + first_residual, first_measurement = probe.pair_residual(*first, size=64) + second_residual, second_measurement = probe.pair_residual(*second, size=64) + + assert first_residual.shape == (64, 64, 3) + assert first_measurement.changed_pixel_fraction > 0.2 + assert second_measurement.width == 80 + assert min(probe.channel_ncc(first_residual, second_residual)) > 0.75 + + +def test_selected_peaks_find_injected_frequency(tmp_path: Path): + pair = _write_pair(tmp_path, "pair", (96, 96), 100) + residual, _ = probe.pair_residual(*pair, size=64) + template = probe.build_template([residual]) + + peaks = probe.select_peaks(template, count=4) + + assert any(abs(int(dy)) == 14 and abs(int(dx)) == 14 for dy, dx in peaks) + + +def test_marked_image_scores_above_clean_image(tmp_path: Path): + pair = _write_pair(tmp_path, "pair", (128, 128), 100) + residual, _ = probe.pair_residual(*pair, size=128) + template = probe.build_template([residual]) + peaks = probe.select_peaks(template, count=8) + + clean_score = probe.score_image(pair[0], template, peaks) + marked_score = probe.score_image(pair[1], template, peaks) + + assert marked_score.phase_weighted > clean_score.phase_weighted + assert marked_score.top_two_channel_phase_12 > clean_score.top_two_channel_phase_12 + + +def test_template_round_trip_does_not_use_pickle(tmp_path: Path): + template = np.zeros((64, 64, 3), dtype=np.float64) + template[1, 1] = (1.0, 2.0, 3.0) + peaks = np.asarray([[1, 1], [2, 2]], dtype=np.int32) + path = tmp_path / "template.npz" + + probe.save_template(path, template, peaks) + loaded_template, loaded_peaks = probe.load_template(path) + + assert np.array_equal(loaded_template, template) + assert np.array_equal(loaded_peaks, peaks) + + +def test_discovery_report_contains_cross_pair_ncc(tmp_path: Path): + first = _write_pair(tmp_path, "first", (96, 96), 100) + second = _write_pair(tmp_path, "second", (128, 80), 140) + first_residual, first_measurement = probe.pair_residual(*first, size=64) + second_residual, second_measurement = probe.pair_residual(*second, size=64) + template = probe.build_template([first_residual, second_residual]) + peaks = probe.select_peaks(template, count=4) + + report = probe.discovery_report( + [first_residual, second_residual], + [first_measurement, second_measurement], + peaks, + ) + + assert report["pair_count"] == 2 + assert len(report["pairwise"]) == 1 + assert report["wavelet"]["wavelet"] == "db2" + assert len(report["wavelet"]["bands"]) == 9 + assert report["spectral"]["peaks"] + assert report["spectral"]["cepstral_peaks"] + assert report["permutation_control"] is not None + + +def test_wavelet_report_finds_repeatable_multiscale_carrier(tmp_path: Path): + first = _write_pair(tmp_path, "first", (96, 96), 100) + second = _write_pair(tmp_path, "second", (128, 80), 140) + first_residual, _ = probe.pair_residual(*first, size=64) + second_residual, _ = probe.pair_residual(*second, size=64) + + report = probe.wavelet_report([first_residual, second_residual]) + + assert len(report["bands"]) == 9 + assert max(max(band["coefficient_ncc"]) for band in report["bands"]) > 0.7 + assert all(len(band["median_rms"]) == 3 for band in report["bands"]) + + +def test_streaming_pairwise_ncc_matches_explicit_pairs(): + rng = np.random.default_rng(7) + arrays = [rng.normal(size=(16, 16, 3)) for _ in range(5)] + normalized = [probe.normalized_channels(values) for values in arrays] + + explicit = np.mean( + [probe.channel_ncc(arrays[first], arrays[second]) for first, second in combinations(range(len(arrays)), 2)], + axis=0, + ) + squared_norm_sum = np.sum([np.sum(np.square(values), axis=(0, 1)) for values in normalized], axis=0) + streamed = probe.pairwise_ncc_from_normalized_sum( + np.sum(normalized, axis=0), + squared_norm_sum, + len(normalized), + ) + + assert np.allclose(streamed, explicit) + + +def test_streaming_pairwise_ncc_handles_zero_norm_channels(): + arrays = [np.zeros((8, 8, 3)), np.zeros((8, 8, 3))] + normalized = [probe.normalized_channels(values) for values in arrays] + + streamed = probe.pairwise_ncc_from_normalized_sum( + np.sum(normalized, axis=0), + np.zeros(3), + len(normalized), + ) + + assert streamed == (0.0, 0.0, 0.0) + + +def test_spectral_report_finds_injected_frequency(tmp_path: Path): + first = _write_pair(tmp_path, "first", (96, 96), 100) + second = _write_pair(tmp_path, "second", (128, 80), 140) + first_residual, _ = probe.pair_residual(*first, size=64) + second_residual, _ = probe.pair_residual(*second, size=64) + + report = probe.spectral_report([first_residual, second_residual]) + + assert any(abs(peak["dy"]) == 14 and abs(peak["dx"]) == 14 for peak in report["peaks"]) + assert report["phase_coherence_p95"] >= report["phase_coherence_median"] + + +def test_permutation_control_breaks_exact_pair_repeatability(tmp_path: Path): + first = _write_pair(tmp_path, "first", (96, 96), 100) + second = _write_pair(tmp_path, "second", (128, 80), 140) + first_residual, first_measurement = probe.pair_residual(*first, size=64) + second_residual, second_measurement = probe.pair_residual(*second, size=64) + + report = probe.permutation_control( + [first_residual, second_residual], + [first_measurement, second_measurement], + ) + + assert report is not None + assert np.mean(report["true_pair_ncc"]) > np.mean(report["mismatched_pair_ncc"]) + assert report["mismatched_median_rms"] > report["true_median_rms"] diff --git a/tests/test_synthid_tile_attack.py b/tests/test_synthid_tile_attack.py new file mode 100644 index 0000000..12aa09e --- /dev/null +++ b/tests/test_synthid_tile_attack.py @@ -0,0 +1,68 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_tile_attack as attack +from synthid_periodic_tile import fold_residual_template + + +def test_modulo_folding_recovers_repeated_high_frequency_tile() -> None: + tile = np.fromfunction(lambda y, x, channel: ((x + y + channel) % 2) * 2.0 - 1.0, (8, 16, 3)) + pixels = 100.0 + np.tile(tile, (8, 4, 1)) + + estimated = fold_residual_template( + pixels, + tile_height=8, + tile_width=16, + denoise_sigma=1.0, + ) + + correlation = np.corrcoef(tile.ravel(), estimated.ravel())[0, 1] + assert correlation > 0.99 + + +def test_subtraction_reduces_repeated_tile_energy() -> None: + tile = np.fromfunction(lambda y, x, channel: ((x + y + channel) % 2) * 2.0 - 1.0, (8, 16, 3)) + pixels = np.clip(np.rint(100.0 + 4.0 * np.tile(tile, (8, 4, 1))), 0, 255).astype(np.uint8) + template = fold_residual_template( + pixels, + tile_height=8, + tile_width=16, + denoise_sigma=1.0, + ) + + result = attack.subtract_tiled_template(pixels, template, strength=1.0) + before = np.std(pixels.astype(np.float64) - np.mean(pixels, axis=(0, 1), keepdims=True)) + after = np.std(result.astype(np.float64) - np.mean(result, axis=(0, 1), keepdims=True)) + + assert after < before + + +def test_folding_accepts_nondivisible_geometry() -> None: + pixels = np.zeros((63, 64, 3), dtype=np.uint8) + + folded = fold_residual_template( + pixels, + tile_height=8, + tile_width=16, + denoise_sigma=1.0, + ) + + assert folded.shape == (8, 16, 3) + assert np.count_nonzero(folded) == 0 + + +def test_subtraction_accepts_nondivisible_geometry() -> None: + pixels = np.full((5, 7, 3), 100, dtype=np.uint8) + template = np.arange(2 * 3 * 3, dtype=np.float64).reshape(2, 3, 3) + + result = attack.subtract_tiled_template(pixels, template, strength=1.0) + + expected_template = np.tile(template, (3, 3, 1))[:5, :7] + expected = np.rint(100.0 - expected_template).astype(np.uint8) + np.testing.assert_array_equal(result, expected) diff --git a/tests/test_synthid_v3_carrier_subtract.py b/tests/test_synthid_v3_carrier_subtract.py new file mode 100644 index 0000000..f289191 --- /dev/null +++ b/tests/test_synthid_v3_carrier_subtract.py @@ -0,0 +1,66 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import click +import numpy as np +import pytest + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_v3_carrier_subtract as subtract +from synthid_v3_codebook_probe import V3CarrierModel + + +def _model() -> V3CarrierModel: + height = width = 64 + return V3CarrierModel( + height=height, + width=width, + rows=np.asarray([7, 7, 7], dtype=np.int32), + columns=np.asarray([5, 5, 5], dtype=np.int32), + channels=np.asarray([0, 1, 2], dtype=np.int8), + phases=np.asarray([0.4, 0.4, 0.4]), + weights=np.full(3, 1.0 / 3.0), + expected_magnitudes=np.full(3, 20.0 * height * width / 2.0), + ) + + +def _carrier(model: V3CarrierModel) -> np.ndarray: + yy, xx = np.mgrid[: model.height, : model.width] + values = 100.0 + 20.0 * np.cos(2.0 * np.pi * (7.0 * yy / model.height + 5.0 * xx / model.width) + 0.4) + return np.repeat(values[:, :, None], 3, axis=2).round().astype(np.uint8) + + +def test_subtract_carrier_preserves_shape_and_removes_known_component() -> None: + model = _model() + source = _carrier(model) + + result = subtract.subtract_carrier(source, model, strength=1.0) + + assert result.shape == source.shape + assert result.dtype == np.uint8 + assert np.std(result.astype(np.float64)) < 1.0 + + +def test_zero_strength_is_pixel_identical() -> None: + model = _model() + source = _carrier(model) + + result = subtract.subtract_carrier(source, model, strength=0.0) + + assert np.array_equal(result, source) + + +def test_rejects_wrong_geometry() -> None: + model = _model() + + with pytest.raises(ValueError, match="does not match"): + subtract.subtract_carrier(np.zeros((32, 32, 3), dtype=np.uint8), model, strength=1.0) + + +@pytest.mark.parametrize("value", ["1,0.5", "0.5,0.5", "-1,0.5", "x"]) +def test_rejects_invalid_strength_sweep(value: str) -> None: + with pytest.raises(click.BadParameter): + subtract.parse_strengths(value) diff --git a/tests/test_synthid_v3_codebook_probe.py b/tests/test_synthid_v3_codebook_probe.py new file mode 100644 index 0000000..08177a1 --- /dev/null +++ b/tests/test_synthid_v3_codebook_probe.py @@ -0,0 +1,116 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts")) + +import synthid_v3_codebook_probe as probe + + +def _write_codebook(path: Path, *, height: int, width: int, phase: float) -> None: + half_width = width // 2 + 1 + rows = np.asarray([7, 11, 13, 17], dtype=np.uint32) + columns = np.asarray([5, 9, 12, 15], dtype=np.uint32) + indices = np.ravel_multi_index((rows, columns), (height, half_width)).astype(np.uint32) + payload: dict[str, np.ndarray] = { + "format_version": np.asarray(2), + f"{height}x{width}/sparse": np.asarray(1), + } + for channel in range(3): + payload[f"{height}x{width}/idx_{channel}"] = indices + magnitudes = np.asarray([1000.0, 10.0, 10.0, 10.0]) + payload[f"{height}x{width}/mag_{channel}"] = np.log2(1.0 + magnitudes).astype(np.float16) + payload[f"{height}x{width}/phase_{channel}"] = np.full(4, phase, dtype=np.float16) + payload[f"{height}x{width}/cons_{channel}"] = np.full(4, 255, dtype=np.uint8) + np.savez(path, **payload) + + +def _write_dense_codebook(path: Path, *, height: int, width: int, phase: float) -> None: + half_width = width // 2 + 1 + magnitudes = np.zeros((height, half_width, 3), dtype=np.float16) + phases = np.zeros_like(magnitudes) + coherence = np.zeros_like(magnitudes, dtype=np.uint8) + rows = np.asarray([7, 11, 13, 17]) + columns = np.asarray([5, 9, 12, 15]) + for channel in range(3): + magnitudes[rows, columns, channel] = np.log2(1.0 + np.asarray([1000.0, 10.0, 10.0, 10.0])) + phases[rows, columns, channel] = phase + coherence[rows, columns, channel] = 255 + np.savez( + path, + format_version=np.asarray(2), + **{ + f"{height}x{width}/sparse": np.asarray(0), + f"{height}x{width}/mag": magnitudes, + f"{height}x{width}/phase": phases, + f"{height}x{width}/cons": coherence, + }, + ) + + +def _write_carrier(path: Path, *, height: int, width: int, phase: float) -> None: + yy, xx = np.mgrid[:height, :width] + carrier = 80.0 + 20.0 * np.cos(2.0 * np.pi * (7.0 * yy / height + 5.0 * xx / width) + phase) + pixels = np.repeat(carrier[:, :, None], 3, axis=2) + Image.fromarray(np.clip(np.rint(pixels), 0, 255).astype(np.uint8), mode="RGB").save(path) + + +def test_numeric_codebook_scores_matching_phase(tmp_path: Path) -> None: + height = width = 64 + codebook = tmp_path / "codebook.npz" + image = tmp_path / "image.png" + _write_codebook(codebook, height=height, width=width, phase=0.4) + _write_carrier(image, height=height, width=width, phase=0.4) + + model = probe.load_v3_model(codebook, height=height, width=width, peak_count=4, min_radius=1.0) + score = probe.score_image(image, model) + + assert score.peak_count == 4 + assert score.phase_score > 0.0 + + +def test_dense_numeric_codebook_scores_matching_phase(tmp_path: Path) -> None: + height = width = 64 + codebook = tmp_path / "dense-codebook.npz" + image = tmp_path / "image.png" + _write_dense_codebook(codebook, height=height, width=width, phase=0.4) + _write_carrier(image, height=height, width=width, phase=0.4) + + model = probe.load_v3_model(codebook, height=height, width=width, peak_count=4, min_radius=1.0) + score = probe.score_image(image, model) + + assert score.peak_count == 4 + assert score.phase_score > 0.0 + + +def test_translation_search_recovers_shifted_carrier(tmp_path: Path) -> None: + height = width = 64 + codebook = tmp_path / "codebook.npz" + image = tmp_path / "image.png" + shifted = tmp_path / "shifted.png" + _write_codebook(codebook, height=height, width=width, phase=0.4) + _write_carrier(image, height=height, width=width, phase=0.4) + with Image.open(image) as source: + pixels = np.asarray(source).copy() + Image.fromarray(np.roll(pixels, shift=(1, 1), axis=(0, 1)), mode="RGB").save(shifted) + model = probe.load_v3_model(codebook, height=height, width=width, peak_count=4, min_radius=1.0) + + fixed = probe.score_image(shifted, model) + registered = probe.score_translations(shifted, model, max_shift=2) + + assert registered.phase_score > fixed.phase_score + assert abs(registered.row_shift) <= 2 + assert abs(registered.column_shift) <= 2 + + +def test_rejects_wrong_format(tmp_path: Path) -> None: + artifact = tmp_path / "bad.npz" + np.savez(artifact, format_version=np.asarray(1)) + + with pytest.raises(ValueError, match="version 2"): + probe.load_v3_model(artifact, height=64, width=64) diff --git a/uv.lock b/uv.lock index 8373d5a..3a40942 100644 --- a/uv.lock +++ b/uv.lock @@ -1165,6 +1165,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" }, ] +[[package]] +name = "httpcore2" +version = "2.12.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "h11", marker = "sys_platform != 'emscripten'" }, + { name = "truststore", marker = "sys_platform != 'emscripten'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/be/ad/f4f0e57345f1870f3e8cb624e058d7eca6e5a27d33bcc3311d9b618734cd/httpcore2-2.12.0.tar.gz", hash = "sha256:9293522bba0aa7c4c8e9e3f040c16575bd8868e155a77fa30c7a9085a5eae648", size = 67548, upload-time = "2026-08-18T13:22:08.211Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d2/74/d370e55600d9bcfa0d9794b0166126d49291a3d2b20c268fc98c453a4948/httpcore2-2.12.0-py3-none-any.whl", hash = "sha256:7e04258ce01013d7d615e5b910a3b27fac937d7a95038227e79652b4ba3b4ceb", size = 83074, upload-time = "2026-08-18T13:22:05.854Z" }, +] + [[package]] name = "httpx" version = "0.28.1" @@ -1180,6 +1193,32 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" }, ] +[[package]] +name = "httpx2" +version = "2.12.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio", marker = "sys_platform != 'emscripten'" }, + { name = "httpcore2", marker = "sys_platform != 'emscripten'" }, + { name = "httpx2-jsfetch", marker = "python_full_version >= '3.12' and sys_platform == 'emscripten'" }, + { name = "idna" }, + { name = "truststore", marker = "sys_platform != 'emscripten'" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/7f/f8/579a8b51e42e38ee32647df9f08aa25643ae788e275cc625b199829c4671/httpx2-2.12.0.tar.gz", hash = "sha256:7631fe9887a8a2275f4a2540e053aa670fcc50742864a9ae7c66e609fdcf12cf", size = 100040, upload-time = "2026-08-18T13:22:09.086Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c8/95/411ba65569158e862368917aaf56597f3e5fa3b91b0502919638465a08f3/httpx2-2.12.0-py3-none-any.whl", hash = "sha256:cc8b6eecb8661c146b8f89a60e97456ee086e91a784ed31ac450c3a9e613dd36", size = 95427, upload-time = "2026-08-18T13:22:06.834Z" }, +] + +[[package]] +name = "httpx2-jsfetch" +version = "1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cd/c4/0e5636363151a2a1795e0a77617168b9ca438e1748ec05fc9b5687f93d64/httpx2_jsfetch-1.0.tar.gz", hash = "sha256:70a0e3eabfef7cce5ad9c629f7d01ca05e418f586646f4ddf14782e4c1454c60", size = 6872, upload-time = "2026-08-07T00:13:07.492Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9b/43/832f631d32e4f1211caa2ba368317739fe71f0b8530e4c9d15dc454bac2a/httpx2_jsfetch-1.0-py3-none-any.whl", hash = "sha256:cb916b707601e69a07721aabc8f3f6659be3a6893bc1ff5c6f9e02241df2da32", size = 6382, upload-time = "2026-08-07T00:13:06.567Z" }, +] + [[package]] name = "huggingface-hub" version = "1.28.0" @@ -1301,6 +1340,92 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" }, ] +[[package]] +name = "jiter" +version = "0.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/1d/1f/10936e16d8860c70698a1aa939a46aa0224813b782bce4e000e637da0b2d/jiter-0.16.0.tar.gz", hash = "sha256:7b24c3492c5f4f84a37946ad9cf504910cf6a782d6a4e0689b6673c5894b4a1c", size = 176431, upload-time = "2026-06-29T13:05:13.657Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4e/3f/fae6cc967d120ec89e31c5418a51176d8278b3087fbb384a9176754f353c/jiter-0.16.0-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:67fddeda1688f0cce2d2ae83ccf8a80f79936f2d2997d6cc2261f82fdb54a4d3", size = 309289, upload-time = "2026-06-29T13:02:52.301Z" }, + { url = "https://files.pythonhosted.org/packages/c8/e3/97c6c3562c077f6247d6e6ce5c82562500b6316c0d928e97e106b7a1321a/jiter-0.16.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:c90c0f63df322be920eda6ce622e3083d8906ba267f8220fe7873213b8b4430e", size = 315181, upload-time = "2026-06-29T13:02:53.964Z" }, + { url = "https://files.pythonhosted.org/packages/7b/89/d8d073f8aa2667e46c6c0873f86fe4a512bba4293cc730f626a076211a62/jiter-0.16.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:64c0203212098470032aabcde9356fc168f377aade3e43def61dfe17e92f2037", size = 340939, upload-time = "2026-06-29T13:02:55.412Z" }, + { url = "https://files.pythonhosted.org/packages/87/c9/db4fda3ed73fb864139305e935e5b8b38a5a24692a5a9dd356c22f1b9c8d/jiter-0.16.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:12288303c9844e61e1651d02a9a6f6633e47d39f897d6991d1427161ce6b746e", size = 364932, upload-time = "2026-06-29T13:02:57.28Z" }, + { url = "https://files.pythonhosted.org/packages/a2/74/52b5e86241057f52ddd7c9a580f90effb51f9d06239f6fc612279b91a838/jiter-0.16.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5cf109d010b4b05a105afb3d43be36a21322d345ad3111e13d15f680afef0e5b", size = 461132, upload-time = "2026-06-29T13:02:58.994Z" }, + { url = "https://files.pythonhosted.org/packages/a9/87/544a700f7447c1f31c5d7833821a4daa5683165c2d5a094fbf5b5800c3dc/jiter-0.16.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:62c1b7fe1f77925acf5af68b6140b8810fa87dfd4dc0a9c8568ec2fa2a10429c", size = 374857, upload-time = "2026-06-29T13:03:00.455Z" }, + { url = "https://files.pythonhosted.org/packages/40/cd/0fcc3f7d39183674d5bfa9ec640faaeb506c60be7c8f94625dfba366e37c/jiter-0.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8597d23c87f59294f83bcb6229b9ed1fccee13dbba967b46930d2f1759466fee", size = 347053, upload-time = "2026-06-29T13:03:02.045Z" }, + { url = "https://files.pythonhosted.org/packages/5c/ae/c7e64e7932ad597fa395b61440b249ada6366716e25c6e08dd2afbd021e6/jiter-0.16.0-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:3126a5dbad56401989ac769aca0cb56005bfb3e2366eea0ca99d1a91c3c1ee03", size = 356153, upload-time = "2026-06-29T13:03:03.706Z" }, + { url = "https://files.pythonhosted.org/packages/d4/1c/1c719044f14da814e1a060191ab19b96f3e99207bc5b4bfc6d6be34b3f80/jiter-0.16.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c4b4717bdb35ae456f831a6b08d01880fff399887a6bbc526a583a406e484eea", size = 393956, upload-time = "2026-06-29T13:03:05.165Z" }, + { url = "https://files.pythonhosted.org/packages/3b/dc/7b2f303a2847207e265503853a2d964a55354cffd62a5f2936c155486798/jiter-0.16.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:adff21bc78edfe086c15eb495b900306076de378dc2337c132401fc39bd79c91", size = 521081, upload-time = "2026-06-29T13:03:06.886Z" }, + { url = "https://files.pythonhosted.org/packages/c2/5f/501cf6e1e09caeb420195179ffc6f62aca603f1220ec53fd80d0d70b3e56/jiter-0.16.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:dab907db06fc593645e73109acf4581ba5b548897d28b9348dc41ddc8343b2d3", size = 552085, upload-time = "2026-06-29T13:03:08.339Z" }, + { url = "https://files.pythonhosted.org/packages/79/54/aa5be86520113b79455c3877f3d1f07a348098df4083ba3688e9537e52dd/jiter-0.16.0-cp311-cp311-win32.whl", hash = "sha256:560b2cf3fb03240cd34f27409a238547488708f05b7c3924f571a60422251ec7", size = 206755, upload-time = "2026-06-29T13:03:09.653Z" }, + { url = "https://files.pythonhosted.org/packages/64/ec/2feb893eb330bd69b413866f4d5daada33c3962f1c6f270c91ca2d87fdf9/jiter-0.16.0-cp311-cp311-win_amd64.whl", hash = "sha256:e431cfc9caf44c1d5459ff77d4e64cbf85fddb6a35dad836a15c6a9ec23087c1", size = 199155, upload-time = "2026-06-29T13:03:10.979Z" }, + { url = "https://files.pythonhosted.org/packages/b9/9c/ca040d94415048a3666fc237774df8151c96f8d2b661cbe3b184acc95876/jiter-0.16.0-cp311-cp311-win_arm64.whl", hash = "sha256:2a8e9e39cf083016137aa5cadafe3188adc2ba6ba1fbf1e5d18889ad3e9ad056", size = 194403, upload-time = "2026-06-29T13:03:12.341Z" }, + { url = "https://files.pythonhosted.org/packages/83/2b/52ace16ed031354f0539749a49e4bf33797d82bea5137910835fa4b09793/jiter-0.16.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:67c3bc1760f8c99d805dcab4e644027142a53b1d5d861f18780ebdbd5d40b72a", size = 306943, upload-time = "2026-06-29T13:03:14.035Z" }, + { url = "https://files.pythonhosted.org/packages/94/2e/34957c2c1b661c252ba9bcc60ae0bddc27e0f7202c6073326a13c5390eec/jiter-0.16.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:5af7780e4a26bd7d0d989592bf9ef12ebf806b74ab709223ecca37c749872ea9", size = 307779, upload-time = "2026-06-29T13:03:15.418Z" }, + { url = "https://files.pythonhosted.org/packages/88/6c/59bd309cab4460c54cf1079f3eb7fe7af6a4c895c5c957a53378693bad2b/jiter-0.16.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d5bf78d0e05e45cfdd66558893938d59afe3d1b1a824a202039b20e607d25a72", size = 335826, upload-time = "2026-06-29T13:03:17.11Z" }, + { url = "https://files.pythonhosted.org/packages/3b/8c/f5ef7b65f0df47afa16596969defb281ebb86e96df346d62be6fd853d620/jiter-0.16.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f4444a83f946605990c98f625cdd3d2725bfb818158760c5748c653170a20e0e", size = 362573, upload-time = "2026-06-29T13:03:18.781Z" }, + { url = "https://files.pythonhosted.org/packages/2b/0b/ace4354da061ee38844a0c27dc2c21eecd27aea119e8da324bea987522d0/jiter-0.16.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3a23f0e4f957e1be65752d2dfac9a5a06b1917af8dc85deb639c3b9d02e31290", size = 457979, upload-time = "2026-06-29T13:03:20.293Z" }, + { url = "https://files.pythonhosted.org/packages/55/40/c0253d3772eb9dcd8e6606ee9b2d53ec8e5b814589c47f140aa585f21eaa/jiter-0.16.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c22a488f7b9218e245a0025a9ba6b100e2e54700831cf4cf16833a27fba3ad01", size = 372302, upload-time = "2026-06-29T13:03:21.739Z" }, + { url = "https://files.pythonhosted.org/packages/a8/d2/4839422241aa12860ce597b20068727094ba0bc480723c74924ca5bad483/jiter-0.16.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:46add52f4ad47a08bfb1219f3e673da972191489a33016edefdb5ea55bfa8c48", size = 343805, upload-time = "2026-06-29T13:03:23.384Z" }, + { url = "https://files.pythonhosted.org/packages/e2/59/e196888a05befdda7dbe299b722d56f2f6eec65402bc34c0a3306d595feb/jiter-0.16.0-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:9c8a956fd72c2cf1e730d01ea080341f13aa0a97a4a33b51abebe725b7ae9ca9", size = 351107, upload-time = "2026-06-29T13:03:24.815Z" }, + { url = "https://files.pythonhosted.org/packages/ec/74/4cd9e0fca65232136400354b630fbfcd2de634e22ccbb96567725981b548/jiter-0.16.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:561926e0573ffe4a32498420a76d64b16c513e1ab413b9d28158a8764ac701e5", size = 388441, upload-time = "2026-06-29T13:03:26.266Z" }, + { url = "https://files.pythonhosted.org/packages/d9/8c/554691e48bc711299c0a293dd8a6179e24b2d66a54dc295421fcf64569c0/jiter-0.16.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:44d019fa8cdaf89bf29c71b39e3712143fdd0ac76725c6ef954f9957a5ea8730", size = 516354, upload-time = "2026-06-29T13:03:28.02Z" }, + { url = "https://files.pythonhosted.org/packages/a4/cb/01e9d69dc2cc6759d4f91e230b34489c4fdb2518992650633f9e20bece89/jiter-0.16.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:0df91907609837f33341b8e6fe73b95991fdaa57caf1a0fbd343dffe826f386f", size = 547880, upload-time = "2026-06-29T13:03:29.534Z" }, + { url = "https://files.pythonhosted.org/packages/79/70/2953195f1c6ad00f49fa67e13df7e60acb3dd4f387101bc15abccddd905e/jiter-0.16.0-cp312-cp312-win32.whl", hash = "sha256:51d7b836acb0108d7c77df1742332cac2a1fa04a74d6dacec46e7091f0e91274", size = 203473, upload-time = "2026-06-29T13:03:31.025Z" }, + { url = "https://files.pythonhosted.org/packages/2d/05/2909a8b10699a4d560f8c502b6b2c5f3991b682b1922c1eedda242b225bd/jiter-0.16.0-cp312-cp312-win_amd64.whl", hash = "sha256:1878349266f8ee36ecb1375cc5ba2f115f35fd9f0a1a4119e725e379126647f7", size = 196905, upload-time = "2026-06-29T13:03:32.472Z" }, + { url = "https://files.pythonhosted.org/packages/e9/a9/6b82bb1c8d7790d602489b967b982a909e5d092875a6c2ade96444c8dfc5/jiter-0.16.0-cp312-cp312-win_arm64.whl", hash = "sha256:2ed5738ae4af18271a51a528b8811b0cbfa4a1858de9d83359e4169855d6a331", size = 190618, upload-time = "2026-06-29T13:03:34.672Z" }, + { url = "https://files.pythonhosted.org/packages/91/c0/555fc60473d30d66894ba825e63615e3be7524fac23858356afa7a38906c/jiter-0.16.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:41977aa5654023948c2dae2a81cbf9c43343954bef1cd59a154dd15a4d84c195", size = 306203, upload-time = "2026-06-29T13:03:36.243Z" }, + { url = "https://files.pythonhosted.org/packages/d0/2b/c3eaf16f5d7c9bad66ea32f40a95bd169b29a91217fcc7f081375157e99c/jiter-0.16.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d28bb3c26762358dadf3e5bf0bccd29ae987d65e6988d2e6f49829c76b003c09", size = 306489, upload-time = "2026-06-29T13:03:37.846Z" }, + { url = "https://files.pythonhosted.org/packages/96/3f/02fdfc6705cad96127d883af5c34e4867f554f29ec7705ec1a46156400a9/jiter-0.16.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0542a7189c26920778658fc8fcf2af8bae05bae9924577f71804acef37996536", size = 335453, upload-time = "2026-06-29T13:03:39.221Z" }, + { url = "https://files.pythonhosted.org/packages/b2/a6/e4bda5920d4b0d7c5dfb7174ce4a6b2e4d3e11c9162c452ef0eab4cdbdbd/jiter-0.16.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:8fb8de1e23a0cb2a7f53c335049c7b72b6db41aa6227cdcc0972a1de5cb39450", size = 361625, upload-time = "2026-06-29T13:03:40.597Z" }, + { url = "https://files.pythonhosted.org/packages/b7/97/4e6b59b2c6e55cbb3e183595f81ad65dcfb21c915fee5e19e335df21bc55/jiter-0.16.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b72d0b2990ca754a9102779ac98d8597b7cb31678958562214a007f909eab78e", size = 456958, upload-time = "2026-06-29T13:03:42.074Z" }, + { url = "https://files.pythonhosted.org/packages/15/e0/97e9557686d2f94f4b93786eccb7eed28e9228ad132ea8237f44727314a7/jiter-0.16.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d5f91b1c27fc22a57993d5a5cb8a627cb8ed4b10502716fac1ffbfe1d19d84e8", size = 372017, upload-time = "2026-06-29T13:03:43.658Z" }, + { url = "https://files.pythonhosted.org/packages/0f/94/db768b6938e0df35c86beeba3dfbbb025c9ee5c19e1aa271f2396e50864d/jiter-0.16.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c682bea068a90b764577bdb78a60a4c1d1606daf9cd4c893832a37c7cc9d9026", size = 343320, upload-time = "2026-06-29T13:03:45.226Z" }, + { url = "https://files.pythonhosted.org/packages/c1/d6/5a59d938244a30735fe62d9433fd325f9021ea29d89780ea4596ea93bc89/jiter-0.16.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:8d031aabecc4f1b6276adfb42e3aabb77c89d468bf616600e8d3a11328929053", size = 350520, upload-time = "2026-06-29T13:03:46.671Z" }, + { url = "https://files.pythonhosted.org/packages/67/f8/c4a857f49c9af125f6bbcac7e3eee7f7978ed89682833062e2dbf62576b1/jiter-0.16.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:eab2cd170150e70153de16896a1774e3a1dca80154c56b54d7a812c479a7165e", size = 387550, upload-time = "2026-06-29T13:03:48.361Z" }, + { url = "https://files.pythonhosted.org/packages/8b/d6/5fbc2f7d6b67b754caa61a993a2e626e815dec47ffc2f9e35f01adfebec7/jiter-0.16.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:6edb63a46e65a82c26800a868e49b2cac30dd5a4218b88d74bc2c848c8ad60bb", size = 515424, upload-time = "2026-06-29T13:03:49.881Z" }, + { url = "https://files.pythonhosted.org/packages/ed/54/284f0164b64a5fed915fea6ba7e9ba9b3d8d37c67d59cf2e3bb99d45cdfe/jiter-0.16.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:659039cc50b5addcc35fcc87ae2c1833b7c0a8e5326ef631a75e4478447bcf84", size = 546981, upload-time = "2026-06-29T13:03:51.363Z" }, + { url = "https://files.pythonhosted.org/packages/13/c5/2a467585a576594384e1d2c43e1224deaafc085f24e243529cf98beef8e1/jiter-0.16.0-cp313-cp313-win32.whl", hash = "sha256:c9c53be232c2e206ef9cdbad81a48bfa74c3d3f08bcf8124630a8a748aad993e", size = 202853, upload-time = "2026-06-29T13:03:53.015Z" }, + { url = "https://files.pythonhosted.org/packages/88/6a/de61d04b9eec69c71719968d2f716532a3bc121170c44a39e14979c6be81/jiter-0.16.0-cp313-cp313-win_amd64.whl", hash = "sha256:baad945ed47f163ad833314f8e3288c396118934f94e7bbb9e243ce4b341a4fd", size = 196160, upload-time = "2026-06-29T13:03:54.447Z" }, + { url = "https://files.pythonhosted.org/packages/19/4b/b390ed59bafb3f31d008d1218578f10327714484b334439947f7e5b11e7f/jiter-0.16.0-cp313-cp313-win_arm64.whl", hash = "sha256:3c1fd2dbe1b0af19e987f03fe66c5f5bd105a2229c1aff4ab14890b24f41d21a", size = 189862, upload-time = "2026-06-29T13:03:55.754Z" }, + { url = "https://files.pythonhosted.org/packages/a7/89/bc4f1b57d5da938fd344a466396541e586d161320d70bffd929aaafcd8f4/jiter-0.16.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:b2c61484666ad42726029af0c00ef4541f0f3b5cdc550221f56c2343208018ee", size = 308239, upload-time = "2026-06-29T13:03:57.205Z" }, + { url = "https://files.pythonhosted.org/packages/65/7a/c415453e5213001bf3b411ff65dec3d303b0e76a4a2cfea9768cd4960994/jiter-0.16.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:63efadc657488f45db1c676d81e704cac2abf3fdb892def1faea61db053127e2", size = 308928, upload-time = "2026-06-29T13:03:58.643Z" }, + { url = "https://files.pythonhosted.org/packages/11/fc/1f4fb7ebf9a724c7741994f4aae18fba1e2f3133df14521a79194952c34a/jiter-0.16.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cf0d73f50e7b6935677854f6e8e31d499ca7064dd24734f703e060f5b237d883", size = 336998, upload-time = "2026-06-29T13:04:00.071Z" }, + { url = "https://files.pythonhosted.org/packages/a0/8d/72cadaac05ccfa7cc3a0a2232862e6c72443ca40cf300ba8b57f9f18b69b/jiter-0.16.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bf3ea07d9bc8e7d03a9fbc051295462e6dbc295b894fd72457c3136e3e43d898", size = 362112, upload-time = "2026-06-29T13:04:01.52Z" }, + { url = "https://files.pythonhosted.org/packages/58/4a/c4b0d5f651fda90a24ffce9f8d56cde462a2e09d31ae3de3c68cef34c04e/jiter-0.16.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:26798522707abb47d767db536e4148ceac1b14446bf028ee85e579a2e043cfe5", size = 459807, upload-time = "2026-06-29T13:04:03.214Z" }, + { url = "https://files.pythonhosted.org/packages/80/58/ef77879ea9aa56b50824edc5a445e226422c7a8d211f3fd2a56bcb9493cf/jiter-0.16.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bc837c1b9631be10abfe0191537fe8009838204cec7e44827401ace390ddb567", size = 373181, upload-time = "2026-06-29T13:04:04.629Z" }, + { url = "https://files.pythonhosted.org/packages/49/2e/ffbc3f254e4d8a66da3062c624a7df4b7c2b2cf9e1fe43cf394b3e104041/jiter-0.16.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:49060fd70737fad59d33ba9dcc0d83247dc9e77187de26053a19c16c9f32bd69", size = 344927, upload-time = "2026-06-29T13:04:06.067Z" }, + { url = "https://files.pythonhosted.org/packages/9a/f6/0be5dc6d64a89f80aa8fec984f94dedb2973e251edcae55841d60786d578/jiter-0.16.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:adbb8edeadd431bc4477879d5d371ece7cb1334486584e0f252656dd7ffada29", size = 352754, upload-time = "2026-06-29T13:04:07.477Z" }, + { url = "https://files.pythonhosted.org/packages/da/6e/7d31243b3b91cd261dd19e9d3557fc3251a80883d3d8049c86174e7ab7af/jiter-0.16.0-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:31aaee5b80f672c1dc21272bcfb9cbdcfc1ea04ff50f00ed5af500b80c44fa93", size = 390553, upload-time = "2026-06-29T13:04:08.92Z" }, + { url = "https://files.pythonhosted.org/packages/25/33/51ae371fde3c88897520f62b4d5f8b27ad7103e2bb10812ff52195609853/jiter-0.16.0-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:6722bcef4ffc86c835574b1b2fac6b33b9fb4a889c781e67950e891591f3c55a", size = 516900, upload-time = "2026-06-29T13:04:10.407Z" }, + { url = "https://files.pythonhosted.org/packages/a0/45/6449b3d123ea439ba79507c657288f461d55049e7bcbdc2cf8eb8210f491/jiter-0.16.0-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:5ab4f50ff971b611d656554ea10b75f80097392c827bc32923c6eeb6386c8b00", size = 548754, upload-time = "2026-06-29T13:04:12.046Z" }, + { url = "https://files.pythonhosted.org/packages/9b/e7/fd2fb11ae3e2649333da3aa170d04d7b3000bbdc3b270f6513382fdf4e04/jiter-0.16.0-cp314-cp314-pyemscripten_2026_0_wasm32.whl", hash = "sha256:710cc51d4ebdcd3c1f70b232c1db1ea1344a075770422bbd4bede5708335acbe", size = 122381, upload-time = "2026-06-29T13:04:13.413Z" }, + { url = "https://files.pythonhosted.org/packages/26/80/f0b147a62c315a164ed2168908286ca302310824c218d3aae52b06c0c9a9/jiter-0.16.0-cp314-cp314-win32.whl", hash = "sha256:57b37fc887a32d44798e4d8ebfa7c9683ff3da1d5bf38f08d1bb3573ccb39106", size = 204578, upload-time = "2026-06-29T13:04:14.813Z" }, + { url = "https://files.pythonhosted.org/packages/5e/e6/4758a14304b4523a6f5adb2419340086aa3593bd4327c2b25b5948a90548/jiter-0.16.0-cp314-cp314-win_amd64.whl", hash = "sha256:cbd18dd5e2df96b580487b5745adf57ef64ad89ba2d9662fc3c19386acce7db8", size = 198154, upload-time = "2026-06-29T13:04:16.272Z" }, + { url = "https://files.pythonhosted.org/packages/26/be/41fa54a2e7ea41d6c99f1dc5b1f0fd4cb474680304b5d268dd518e81da3a/jiter-0.16.0-cp314-cp314-win_arm64.whl", hash = "sha256:a32d2027a9fa67f109ff245a3252ece3ccc32cc56703e1deab6cc846a59e0585", size = 191458, upload-time = "2026-06-29T13:04:17.707Z" }, + { url = "https://files.pythonhosted.org/packages/81/6b/59127338b86d9fe4d99418f5a15118bea778103ee0fe9d9dd7e0af174e95/jiter-0.16.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2577196f4474ef3fc4779a088a23b0897bbf86f9ea3679c372d45b8383b43207", size = 316739, upload-time = "2026-06-29T13:04:19.663Z" }, + { url = "https://files.pythonhosted.org/packages/2d/95/49461034d5388196d3dabf98748935f017b7785d8f3f5349f834bcc4ed0d/jiter-0.16.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:616e89e008a93c01104161c75b4988e58716b01d62307ebfe161e52a56d2a818", size = 340911, upload-time = "2026-06-29T13:04:21.257Z" }, + { url = "https://files.pythonhosted.org/packages/cd/97/a4369f2fb82cb3dda13b98622f31249b2e014b223fe64ee534413ad72294/jiter-0.16.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:0e2e9efbe042210df657bade597f66d6d75723e3d8f45a12ea6d8167ff8bbce3", size = 361747, upload-time = "2026-06-29T13:04:22.677Z" }, + { url = "https://files.pythonhosted.org/packages/28/51/49b6ed456261646e1906016a6760367a28aacd3c24805e4e5fe64116c1db/jiter-0.16.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3f4d9e473a5ce7d27fef8b848df4dc16e283893d3f53b4a585e72c9595f3c284", size = 460225, upload-time = "2026-06-29T13:04:24.441Z" }, + { url = "https://files.pythonhosted.org/packages/33/b5/5689aff4f66c5b60be63106e591dbfcba2190df97d2c9c7cf052361ddb98/jiter-0.16.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8d30a4a1c87713060c8d1cc59a7b6c8fb6b8ef0a6900368014c76c87922a2929", size = 373169, upload-time = "2026-06-29T13:04:25.884Z" }, + { url = "https://files.pythonhosted.org/packages/a2/96/3ae1b85ee0d6d6cab254fb7f8da018272b932bbf2d69b07e98aa2a96c746/jiter-0.16.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bae96332410f866e5900d809298b1ed82735932986c672495f9701daacd80620", size = 350332, upload-time = "2026-06-29T13:04:27.302Z" }, + { url = "https://files.pythonhosted.org/packages/15/32/c99d7bafd78986556c95bf60ce84c6cc98786eac56066c12d7f828bb6747/jiter-0.16.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:da3d7ec75dc83bb18bca888b5edfae0656a26849056c59e05a7728badd17e7af", size = 353377, upload-time = "2026-06-29T13:04:28.731Z" }, + { url = "https://files.pythonhosted.org/packages/0e/4b/f99a8e571287c3dec766bcc18528bbe8e8fb5365522ab5e6d64c93e87066/jiter-0.16.0-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:ee6162b77d49a9939229df666dfa8af3e656b6701b54c4c84966d740e189264e", size = 387746, upload-time = "2026-06-29T13:04:30.319Z" }, + { url = "https://files.pythonhosted.org/packages/75/69/c78a5b3f71040e34eb5917df26fb7ae9a2174cad1ccbf277512507c53a6e/jiter-0.16.0-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:63ffdbdae7d4499f4cda14eadc12ddcabef0fc0c081191bdc2247489cb698077", size = 517292, upload-time = "2026-06-29T13:04:31.709Z" }, + { url = "https://files.pythonhosted.org/packages/c2/f7/095b38eda4c70d03651c403f29a5590f16d12ddc5d544aac9f9cddf72277/jiter-0.16.0-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:a111256a7193bea0759267b10385e5870949c239ed7b6ddbaaf57573edb38734", size = 549259, upload-time = "2026-06-29T13:04:33.721Z" }, + { url = "https://files.pythonhosted.org/packages/2e/c5/6a0207d90e5f656d95af98ebd0934f382d37674416f215aeda2ff8063e51/jiter-0.16.0-cp314-cp314t-win32.whl", hash = "sha256:de5ba8763e56b793561f43bed197c9ea55776daa5e9a6b91eed68a909bc9cdbf", size = 206523, upload-time = "2026-06-29T13:04:35.068Z" }, + { url = "https://files.pythonhosted.org/packages/a5/31/c757d5f30a8980fd945ce7b98be10be9e4ff59c7c42f5fd86804c2e87db8/jiter-0.16.0-cp314-cp314t-win_amd64.whl", hash = "sha256:b8a3f9a6008048fe9def7bf465180564a6e458047d2ce499149cfbe73c3ae9db", size = 200366, upload-time = "2026-06-29T13:04:36.61Z" }, + { url = "https://files.pythonhosted.org/packages/7c/a2/d88de6d313d734a544a7901353ad5db67cb38dcfcd91713b7979dafc345d/jiter-0.16.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0fa25b09b13075c46f5bc174f2690525a925a4fc2f7c82969a2bbabff22386ce", size = 190516, upload-time = "2026-06-29T13:04:38.004Z" }, + { url = "https://files.pythonhosted.org/packages/06/d3/8e278946d43eeca2585b4dd0834a887cd71136329b837f3a16ed86a8b4b0/jiter-0.16.0-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:850ccb1d7eedb4200f4014b1c0e8a577de114fc3cd88faad646dcc9bc4bb12ad", size = 304518, upload-time = "2026-06-29T13:05:00.172Z" }, + { url = "https://files.pythonhosted.org/packages/72/43/28d4ef495028bf0506a413d4db3f4eb3e7288a382e0f065f306a17bbeb5e/jiter-0.16.0-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:e34e97bda77eb63242a410243c071e28ac7e0d8c0948c5ee658498690a4b2f2f", size = 310207, upload-time = "2026-06-29T13:05:02.123Z" }, + { url = "https://files.pythonhosted.org/packages/e0/ca/c366b1012da1d640de975d9683acd44e4d150d9068845d0ca2610435253f/jiter-0.16.0-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b7dc85ea77d4abbae8bad0d3538678aedee75bceec4e2f6c8dfb1c74772e5aa5", size = 342771, upload-time = "2026-06-29T13:05:03.55Z" }, + { url = "https://files.pythonhosted.org/packages/16/52/50cc4056fc1ae02e7154704e7ecc89df0afb8300222cfe8a52d3f67e4730/jiter-0.16.0-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:17ca7fae79f6d99cd9a042b75f917eaada7b895cfc7dd2ee3a16089dcaec7a85", size = 346468, upload-time = "2026-06-29T13:05:05.452Z" }, + { url = "https://files.pythonhosted.org/packages/98/ab/664fd8c4be028b2bedd3d2ff08769c4ede23d0dbc87a77c62384a0515b5d/jiter-0.16.0-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:f17d61a28b4b3e0e3e2ba98490c70501403b4d196f78732439160e7fd3678127", size = 303106, upload-time = "2026-06-29T13:05:07.118Z" }, + { url = "https://files.pythonhosted.org/packages/1a/07/421f1d5b65493a76e16027b848aba6a7d28073ae75944fa4289cc914d39f/jiter-0.16.0-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:96e38eea538c8ddf853a35727c7be0741c76c13f04148ac5c116222f50ece3b3", size = 304658, upload-time = "2026-06-29T13:05:08.708Z" }, + { url = "https://files.pythonhosted.org/packages/0a/db/bba1155f01a01c3c37a89425d571da751bbedf5c54247b831a04cb971798/jiter-0.16.0-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d284fb8d94d5855d60c44fefcab4bf966f1da6fada73992b01f6f0c9bc0c6702", size = 339719, upload-time = "2026-06-29T13:05:10.41Z" }, + { url = "https://files.pythonhosted.org/packages/78/f7/18a1afcd64f35314b68c1f23afcd9994d0bc13e65cc77517afff4e83986d/jiter-0.16.0-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:64d613743df53199b1aa256a7d328340da6d7078aac7705a7db9d7a791e9cfd2", size = 343885, upload-time = "2026-06-29T13:05:12.087Z" }, +] + [[package]] name = "lightning" version = "2.6.5" @@ -1958,6 +2083,23 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/60/21/d0c04b561b46e9bff89b5f500fb7415b8ca0669f7902204f76ab06bb0c7e/onnxruntime-1.29.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:1ea91cef3b971506e51ae9c37c16d027774ec64994a524ec1bdfb027d68a9832", size = 23138547, upload-time = "2026-08-17T22:54:37.491Z" }, ] +[[package]] +name = "openai" +version = "3.6.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio" }, + { name = "httpx2" }, + { name = "jiter" }, + { name = "pydantic" }, + { name = "sniffio" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/76/913b755a1a6b54e2d9140eb8d488aa0d47c7359b1d7eac5e864cb7913bbf/openai-3.6.0.tar.gz", hash = "sha256:18fe3f6e96390ef41ee27b152fc9effefca321c33673bd9b956a572493d3ab9b", size = 1455376, upload-time = "2026-08-28T22:29:18.268Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a1/94/805b87ecc951c49ec8f247f5e8eb324ab064bd2ad73b6a0e704dd49aa073/openai-3.6.0-py3-none-any.whl", hash = "sha256:508e2158bf971687f953b62e44b02f207792c815aac306816386d7ba34d37f5f", size = 1699841, upload-time = "2026-08-28T22:29:16.436Z" }, +] + [[package]] name = "opencv-contrib-python" version = "4.10.0.84" @@ -3189,6 +3331,7 @@ all = [ { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.13'" }, { name = "numpy", version = "2.3.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.13'" }, { name = "onnxruntime" }, + { name = "openai" }, { name = "opencv-python-headless" }, { name = "pillow-heif" }, { name = "pywavelets" }, @@ -3297,6 +3440,9 @@ text-restoration = [ trustmark = [ { name = "trustmark", marker = "python_full_version < '3.13'" }, ] +verify = [ + { name = "openai" }, +] video = [ { name = "av" }, { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.13'" }, @@ -3326,6 +3472,7 @@ requires-dist = [ { name = "numpy", marker = "python_full_version < '3.13' and extra == 'pixels'", specifier = ">=1.24.0" }, { name = "onnxruntime", marker = "extra == 'lama'", specifier = ">=1.24.0" }, { name = "onnxruntime", marker = "extra == 'migan'", specifier = ">=1.24.0" }, + { name = "openai", marker = "extra == 'verify'", specifier = ">=2.52.0" }, { name = "opencv-python-headless", marker = "extra == 'pixels'", specifier = ">=4.8.0" }, { name = "packaging", marker = "extra == 'dev'", specifier = ">=24.0" }, { name = "paddleocr", marker = "extra == 'text-draft'", specifier = ">=3.3.3" }, @@ -3347,7 +3494,7 @@ requires-dist = [ { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'visible'" }, { name = "remove-ai-watermarks", extras = ["qwen-zimage", "lama"], marker = "extra == 'text-restoration'" }, { name = "remove-ai-watermarks", extras = ["video"], marker = "extra == 'dev'" }, - { name = "remove-ai-watermarks", extras = ["video", "heif", "detect", "trustmark", "text-restoration", "migan", "classify"], marker = "extra == 'all'" }, + { name = "remove-ai-watermarks", extras = ["video", "heif", "detect", "trustmark", "text-restoration", "migan", "classify", "verify"], marker = "extra == 'all'" }, { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'lama'" }, { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'migan'" }, { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'video'" }, @@ -3362,7 +3509,7 @@ requires-dist = [ { name = "transformers", marker = "extra == 'diffusion'", specifier = ">=5,<6" }, { name = "trustmark", marker = "python_full_version < '3.13' and extra == 'trustmark'", specifier = ">=0.8.0" }, ] -provides-extras = ["pixels", "heif", "visible", "video", "detect", "diffusion", "qwen-zimage", "text-restoration", "text-draft", "trustmark", "classify", "lama", "migan", "dev", "all"] +provides-extras = ["pixels", "heif", "visible", "video", "detect", "diffusion", "qwen-zimage", "text-restoration", "text-draft", "trustmark", "classify", "verify", "lama", "migan", "dev", "all"] [[package]] name = "requests" @@ -3587,6 +3734,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" }, ] +[[package]] +name = "sniffio" +version = "1.3.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a2/87/a6771e1546d97e7e041b6ae58d80074f81b7d5121207425c964ddf5cfdbd/sniffio-1.3.1.tar.gz", hash = "sha256:f4324edc670a0f49750a81b895f35c3adb843cca46f0530f79fc1babb23789dc", size = 20372, upload-time = "2024-02-25T23:20:04.057Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", size = 10235, upload-time = "2024-02-25T23:20:01.196Z" }, +] + [[package]] name = "sympy" version = "1.14.0" @@ -3844,6 +4000,15 @@ dependencies = [ ] sdist = { url = "https://files.pythonhosted.org/packages/87/0a/0a4232030c6a62d12b6a02ae73bdce6e99c8532bc8f05a5a2e6ce103da82/trustmark-0.9.1.tar.gz", hash = "sha256:dc79e3fb070f5d94765acf8868a51f50a612cc05b53223cf1e6b605d4ff1e0ae", size = 63949, upload-time = "2026-04-09T08:59:52.472Z" } +[[package]] +name = "truststore" +version = "0.10.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/53/a3/1585216310e344e8102c22482f6060c7a6ea0322b63e026372e6dcefcfd6/truststore-0.10.4.tar.gz", hash = "sha256:9d91bd436463ad5e4ee4aba766628dd6cd7010cf3e2461756b3303710eebc301", size = 26169, upload-time = "2025-08-12T18:49:02.73Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/19/97/56608b2249fe206a67cd573bc93cd9896e1efb9e98bce9c163bcdc704b88/truststore-0.10.4-py3-none-any.whl", hash = "sha256:adaeaecf1cbb5f4de3b1959b42d41f6fab57b2b1666adb59e89cb0b53361d981", size = 18660, upload-time = "2025-08-12T18:49:01.46Z" }, +] + [[package]] name = "typer" version = "0.27.1"