Files
remove-ai-watermarks/scripts/synthid_adaptive_carrier_suppress.py
T
Victor Kuznetsov 07b07634cf Merge the docs/arxiv-paper-review research line into main (#95)
* Add rigorous SynthID research and evaluation harness

* Record SynthID cross-epoch validation results

* Reset SynthID detector hypotheses to paired training

* Add paired wavelet and spectral SynthID probes

* Record closed-corpus SynthID detector ablations

* Add registered SynthID phase probing

* Add periodic SynthID tile probing

* Add calibrated SynthID pixel detector

* Harden scale-registered SynthID detection

* Record SynthID phase and channel rescue findings

* Add production SynthID routing and OpenAI verification

* Reframe periodic pixel route as pipeline lattice; confirm and harden detection

The frozen periodic experts read an origin-anchored generation-pipeline lattice destroyed by a crop off the tile grid, not the crop-robust SynthID mark. Route the pixel result as an experimental pipeline_lattice signal kept out of the watermark inventory, and carry the crop sensitivity in every verdict envelope.

Add split-patch phase/amplitude/codeword confirmation for registered-v3, affine-lattice and cyclostationary research probes, and timeout/retry/error-taxonomy hardening for the official OpenAI verification path.

* Record pipeline-lattice reframing and registered-v3 phase lock

Document that the pixel route detects an origin-anchored lattice rather than the watermark, and record the registered-v3 two-pixel phase-lock measurement: a 2px diagonal crop killed all 36 tested detections (28 foreign-generator, 8 Google) with recovery only at offsets that are multiples of four. Update README, CLI, Python API, supported-signals, known-limitations, module-internals, the SynthID reference, and the detector research plan.

* Split SynthID campaign docs by detector, classifiers, and removal

Keep Model 1 as a research AI-versus-camera result outside the library.
Retire the mixed oracle-ladders page to a routing hub and mark the plan
file as a chronological archive.

* Keep 2026-08-22 SynthID campaign results in research docs only

Drop Bayer, Model 1, and lattice re-check numbers from user guides and
module internals. The library surface is unchanged.

* Unplug lattice detector from package, expand research docs

* Remove lattice detector sources from the public package

* Document SynthID GitHub sweep findings

* Document general AI classifier sweep

* Document modern-negative taxonomy audit and classifier doc split

Split general AI-classifier research into ai-generated-image-classifiers.md
and record the modern-negative campaign: Unsplash/Openverse date-clean cells,
contamination controls, quarter-hard paired result, closed linear veto and
taxonomy-continuation arms.

pre-commit: 1) maintain.sh - exit 1, known uv-secure lightning advisory with no upstream fix (same triage as main); core checks separately green (ruff, format, pyright, 1564 tests); 2) /simplify - docs-only single pass, no findings; 3) docs sync - split references updated everywhere, none stale; 4) CLAUDE.md - routing line already updated, compliant

* Record wild-AI audit, Meta provider class, and stock-negative expansion

The wild vendor-flagged AI cell (300 stratified rows) puts Model 1 recall at
69.7% on unknown-renderer stock AI; the stock-negative harvest triples the
modern fashion/product cells and confirms the combined-pool veto control;
the Meta muse-image corpus doubles to 132 rows with its margin sweep; a
per-channel cv2 reference fixes the latent fold test under cv2 4.10.0.

pre-commit: 1) maintain.sh - exit 1, known uv-secure lightning advisory with no upstream fix; core checks separately green (ruff, format, pyright, 1665 tests); 2) /simplify - docs-only single pass, no findings; 3) docs sync - new run references point at the gitignored research store, none stale; 4) CLAUDE.md - compact, no changes needed

* Close representation-change campaign: both candidates falsified

Pre-registered gates (AI-test >=1772, fresh OI <=50, Kodak 0, FLUX >=288,
EvalGEN >=95) rejected DINOv2-L ridge head (1427/1847, 162/3000, 1/24, 29/300,
5/100) and the paired-reconstruction FFT tower (14/1847, 12/3000, 0/24, 0/300,
0/100; contract learned at 0.949 pairwise separation but no transfer to
production generators). Campaign verdict: general classifier beyond local
scale as pursued; product stays on documented remainder.

pre-commit: 1) maintain.sh - exit 1, known uv-secure lightning advisory with no upstream fix; core checks green minutes ago (ruff, format, pyright, 1665 tests); 2) /simplify - docs-only single pass, no findings; 3) docs sync - verdict mirrored in data/research/REPRESENTATION-CAMPAIGN.md, no stale refs; 4) CLAUDE.md - no changes

* Record full failure map, H2 router falsification, and H3 process detector closure

20 measured negative domains with batch-2 (memes, whiteboards, tickets, book
pages). H2 domain router falsified: 97.9% of AI-test looks structured, every
family 94-100%. H3 process detector on 369 contrastive pairs from three
generators (OpenAI gpt-image-1-mini, Meta muse-image-1.0, Gemini flash-image)
achieves 0.909 pairwise AUC on training pairs but transfers at AUC 0.55-0.60
to production AI; human receipts score higher than any AI family. The
generator-specific spectral trace does not generalize across three generators,
closing the process-trace family in its current form.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1665 tests); 2) /simplify - docs + research scripts, single pass, no findings; 3) docs sync - all artifacts in data/research/, no stale refs; 4) CLAUDE.md - no changes

* Record H4-H6 research hypotheses: MLP probe, spectral invariant, TTA voting

H4 (MLP on frozen CLIP): 98.6% AI-test recall but 15.8% photo FPR — same
trade-off curve as quarter-hard. H5 (spectral invariant from CVPR 2025):
2.4% recall, the 1/f^alpha profile does not discriminate current generators.
H6 (TTA voting): fixes museum FP but not documents/UI/fashion, loses 9% recall.
All three falsified against frozen gates; ten total closed hypotheses now
document the CLIP-L representation limit from every angle.

Also records the provider cascade across all 20 domains (69.8% stray on
documents mirrors the detector failure map) and the full Y-phase campaign
PSNR data awaiting oracle rate-limit reset.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1665 tests); 2) /simplify - docs-only single pass, no findings; 3) docs sync - all artifacts in data/research/, no stale refs; 4) CLAUDE.md - no changes

* Fix Y-phase frequency convention, record H4-H6 and campaign results

The initial Y-phase campaign targeted FFT bins 16-32 instead of spatial
periods 16-32 px (1/32 to 1/16 cycles/px). After the fix, the full-octave
Y-phase scramble reproduces the known kill (not_detected at 20.5 dB on s1
at native 1122x1402). Partial sub-bands are visually invisible (42.5 dB)
but stay detected: the mark is distributed across the full octave, not
concentrated. Point intervention is not viable within this frequency band.

H4 MLP probe: 98.6% recall, 15.8% FPR (same trade-off wall).
H5 spectral invariant: 2.4% recall (1/f^alpha does not discriminate).
H6 TTA voting: fixes museum FP but not documents/UI/fashion.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (1665 tests, verified minutes ago); 2) /simplify - docs-only; 3) docs sync - all artifacts synced; 4) CLAUDE.md - no changes

* Add TC260 fifth class and per-class margin calibration to provider cascade

Five-head cascade (openai/google/tc260/meta/no_ai): openai 68.8%, google
76.6%, tc260 63.8%, meta 71.8% mean recall at uniform margin 0.50.
Per-class margins calibrated against 1.67% photo budget: openai 0.31,
google 0.29, tc260 0.39, meta 0.55. TC260 test recall 71.4% at calibrated
margins on 196 held-out images. Chinese-AIGC is a viable pixel class.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (verified within the hour); 2) /simplify - research script, single pass; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes

* Record universal detector v3-v8: diverse training shifts the curve

Fresh MLP on frozen CLIP embeddings with 49 negative domains (12,258 images)
and 5,043 AI positives achieves 93.0% AI-test recall with 39/49 domains at
zero false positives. Digital art FP dropped from 99.3% (Model 1) to 24.8%,
fashion from 62-71% to 12%, UI from 99.3% to 12%. Photo FPR 0.7%. The
improvement comes from training data diversity on frozen embeddings, not
representation change (v5 fine-tune proved this by undoing the gains).
Also adds .local-eval to ruff exclude so the gate covers only tracked files.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - docs + research; 3) docs sync - all artifacts in data/research/; 4) CLAUDE.md - no changes

* Add three-tier ensemble detector: DEFINITELY/POSSIBLY/LIKELY_HUMAN

Combines Model 1 and v8 into calibrated confidence tiers. DEFINITELY_AI
(both models agree) achieves 79.6% recall at 0.50% photo FPR with 46/49
domains at zero FP. The union reaches 93.3% recall at 1.80% photo FPR
with Kodak 0/24. First detector working across ALL content types with
per-tier calibrated error rates.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - docs + ensemble; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes

* Add provider v2: diversity training eliminates document leakage

Same diversity-training insight applied to the provider cascade. Adding
3,769 structured negative features to training drops document/receipt
leakage from 69.8% to 5.1%, UI from 10% to 4.3%, fashion from 5-15% to
0-3%. AI recall improves: OpenAI 84% (was 75%), Google 88.8% (was 79%).
Photo leakage 1.4%. Worst regression: memes 47.9% (was 27%).

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - provider v2; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes

* Add universal v11: full taxonomy, 136 domains, 124 at zero FP

Master data plan mapped 148 visual domains. Massive collection (3,343
images, 73 domains from Openverse) plus gap-filler (724 images from
Safebooru anime/manga) expanded training to 19,768 images across 136
negative domains. Universal v11 (MLP-512 + Model 1 ensemble):
DEFINITELY 90.3% recall at 0.50% photo FPR, ANY 95.6% recall,
124/134 domains at exactly 0% FP, worst FP 13.8% (digital art).

New domains opened: anime, manga, chibi, mecha, vintage photos,
banknotes, newspapers, magazines, movie posters, antique maps,
watercolor/oil originals, patterns, cosmetics, real estate, 3D
renders, and 60+ more.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - docs + research; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes

* Confirm representation limit: v12 fine-tune destroys structured FP despite 6x data

Fine-tuning CLIP tower on 19,518 images achieves 95.1% standalone recall
but returns to 100% FP on art/UI and 14/59 zero-FP domains (vs 124/134
for frozen v11). This is a representation property, not a data issue.
Frozen CLIP + MLP (v11) is the only working architecture. Also records
provider v3 trained on all negative domains.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - docs + research; 3) docs sync - artifacts in data/research/; 4) CLAUDE.md - no changes

* Document v13 plateau: 128/136 zero FP, frozen-CLIP ceiling confirmed

v13 with top-plan expansion (1,169 new images) improves domain coverage
to 128/136 at zero FP and reduces fashion FP to 5.3%, but DEFINITELY
recall plateaus at 89.6% and art FP at 15.9%. The frozen CLIP + MLP
architecture has reached its ceiling.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (verified minutes ago, 1731 tests); 2) /simplify - docs-only; 3) docs sync - v13 recorded; 4) CLAUDE.md - no changes

* Document the 2026-08-31 photo-classifier freeze.

Record the AI-versus-camera contract, Meta v3 hold-out, Flickr UI
replacement, and that receipts stay out of the supported domain.

* Scrub Spaces catalog details from classifier research docs

* Record receipt layout-gate probe for the universal detector
2026-09-01 11:37:32 -07:00

198 lines
7.4 KiB
Python

"""Suppress the recovered periodic carrier without image regeneration.
This research tool controls the project's local fixed-template score. A local
score reversal is not evidence that a provider SynthID verifier will stop
detecting the image.
"""
from __future__ import annotations
import json
import logging
import math
import sys
import time
from pathlib import Path
from typing import TYPE_CHECKING, Any
sys.path.insert(0, str(Path(__file__).resolve().parent))
import click
from PIL import Image
from synthid_pixel_attack import load_rgb, measure # pyright: ignore[reportUnknownVariableType]
from synthid_research_manifest import artifact_sha256
from synthid_runtime.synthid_detector import (
TILE_THRESHOLD,
_geometry_supported, # pyright: ignore[reportPrivateUsage]
_load_template, # pyright: ignore[reportPrivateUsage]
folded_template_score,
)
from synthid_tile_attack import subtract_tiled_template
if TYPE_CHECKING:
from numpy.typing import NDArray
log = logging.getLogger(__name__)
def apply_template(pixels: NDArray[Any], template: NDArray[Any], *, amplitude: float) -> NDArray[Any]:
"""Subtract AMPLITUDE times periodic TEMPLATE from arbitrary RGB PIXELS."""
return subtract_tiled_template(pixels, template, strength=amplitude)
def carrier_score(pixels: NDArray[Any], template: NDArray[Any], sigma: float) -> float:
"""Return the local fixed-template carrier score for PIXELS."""
score, _folded = folded_template_score(pixels, template, sigma)
return score
def find_minimum_amplitude(
pixels: NDArray[Any],
template: NDArray[Any],
sigma: float,
*,
target_score: float,
maximum_amplitude: float,
iterations: int,
) -> tuple[float, NDArray[Any], float]:
"""Return the smallest searched amplitude whose score reaches TARGET_SCORE."""
if not math.isfinite(target_score):
raise ValueError("target score must be finite")
if not math.isfinite(maximum_amplitude) or maximum_amplitude <= 0.0:
raise ValueError("maximum amplitude must be finite and positive")
if iterations < 1:
raise ValueError("iterations must be positive")
maximum_pixels = apply_template(pixels, template, amplitude=maximum_amplitude)
maximum_score = carrier_score(maximum_pixels, template, sigma)
if maximum_score > target_score:
raise ValueError(
f"maximum amplitude {maximum_amplitude:g} reached score {maximum_score:.6f}, "
f"above target {target_score:.6f}"
)
low = 0.0
high = maximum_amplitude
best_pixels = maximum_pixels
best_score = maximum_score
for _iteration in range(iterations):
middle = (low + high) / 2.0
candidate = apply_template(pixels, template, amplitude=middle)
candidate_score = carrier_score(candidate, template, sigma)
if candidate_score <= target_score:
high = middle
best_pixels = candidate
best_score = candidate_score
else:
low = middle
return high, best_pixels, best_score
def suppress_carrier(
pixels: NDArray[Any],
*,
target_score: float = -0.25,
maximum_amplitude: float = 40.0,
iterations: int = 8,
) -> tuple[NDArray[Any], dict[str, float | int | str]]:
"""Suppress a locally detected carrier and return pixels plus measurements."""
height, width = pixels.shape[:2]
if not _geometry_supported(width, height):
raise ValueError(f"unsupported decoded geometry: {width}x{height}")
if target_score >= TILE_THRESHOLD:
raise ValueError(f"target score must be below the detector threshold {TILE_THRESHOLD:.6f}")
template, sigma, _model_height, _model_width, tile_height, tile_width = _load_template()
original_score = carrier_score(pixels, template, sigma)
if original_score < TILE_THRESHOLD:
raise ValueError(
f"local carrier is not detected: score {original_score:.6f} is below threshold {TILE_THRESHOLD:.6f}"
)
started = time.perf_counter()
amplitude, candidate, candidate_score = find_minimum_amplitude(
pixels,
template,
sigma,
target_score=target_score,
maximum_amplitude=maximum_amplitude,
iterations=iterations,
)
quality = measure(pixels, candidate, name="adaptive-carrier", path=Path("<memory>"))
return candidate, {
"status": "local_carrier_suppressed",
"detector_scope": "local fixed-template carrier, not provider-verified SynthID removal",
"width": width,
"height": height,
"tile_height": tile_height,
"tile_width": tile_width,
"threshold": TILE_THRESHOLD,
"target_score": target_score,
"original_score": original_score,
"candidate_score": candidate_score,
"amplitude": amplitude,
"maximum_amplitude": maximum_amplitude,
"iterations": iterations,
"residual_rms": quality.residual_rms,
"psnr_db": quality.psnr_db,
"ssim": quality.ssim,
"changed_pixel_fraction": quality.changed_pixel_fraction,
"elapsed_seconds": time.perf_counter() - started,
}
@click.command()
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
@click.argument("output", type=click.Path(dir_okay=False, path_type=Path))
@click.option("--target-score", type=float, default=-0.25, show_default=True)
@click.option("--maximum-amplitude", type=click.FloatRange(min=0.0, min_open=True), default=40.0, show_default=True)
@click.option("--iterations", type=click.IntRange(min=1), default=8, show_default=True)
@click.option("--report-out", type=click.Path(dir_okay=False, path_type=Path))
def main(
source: Path,
output: Path,
target_score: float,
maximum_amplitude: float,
iterations: int,
report_out: Path | None,
) -> None:
"""Write a lossless PNG with the recovered local carrier suppressed."""
logging.basicConfig(level=logging.INFO, format="%(message)s")
if output.suffix.lower() != ".png":
raise click.BadParameter("output must use the .png extension", param_hint="output")
report_path = report_out or output.with_suffix(".json")
for path in (output, report_path):
if path.exists():
raise click.ClickException(f"refusing to overwrite existing file: {path}")
try:
candidate, report = suppress_carrier(
load_rgb(source), # pyright: ignore[reportUnknownArgumentType]
target_score=target_score,
maximum_amplitude=maximum_amplitude,
iterations=iterations,
)
except ValueError as error:
raise click.ClickException(str(error)) from error
output.parent.mkdir(parents=True, exist_ok=True)
report_path.parent.mkdir(parents=True, exist_ok=True)
Image.fromarray(candidate, mode="RGB").save(output, format="PNG", compress_level=9)
report.update(
{
"source": str(source.resolve()),
"source_sha256": artifact_sha256(source),
"output": str(output.resolve()),
"output_sha256": artifact_sha256(output),
}
)
report_path.write_text(json.dumps(report, indent=2, allow_nan=False) + "\n", encoding="utf-8")
log.info(
"Suppressed local carrier %.6f -> %.6f at %.2f dB PSNR; wrote %s",
report["original_score"],
report["candidate_score"],
report["psnr_db"],
output,
)
log.info("Research caveat: this is not provider-verified SynthID removal")
if __name__ == "__main__":
main()