diff --git a/README.md b/README.md index 4918ea1..49ff821 100644 --- a/README.md +++ b/README.md @@ -81,13 +81,18 @@ Detect the supported SynthID pixel carrier after installing the pixel runtime: ```bash uv tool install --force "remove-ai-watermarks[pixels]" remove-ai-watermarks detect-synthid image.png +remove-ai-watermarks detect-synthid resized.png --register-scale ``` This detector is positive-only and limited to one measured carrier family in the [calibrated image-size range](docs/synthid.md#32-how-our-tool-detects-the-supported-carrier). -It expects the recovered carrier at its measured 16-pixel sampling scale; -arbitrary spatial resampling is not registered. `not_detected` or `unsupported` -is not a clean-image guarantee. +The fast default expects the recovered carrier at its measured 16-pixel +sampling scale. `--register-scale` opts into a much slower bounded scale search +for resized images from 250,000 through 10,000,000 decoded pixels, with both +sides at least 64 pixels. Its measured positive scale range is approximately +0.65 through 1.5; 0.5x resizes remain outside reliable detection. `identify` +keeps the fast default. `not_detected` or `unsupported` is not a clean-image +guarantee. For visible watermark removal, install the pixel dependencies: diff --git a/docs/cli.md b/docs/cli.md index 072cdca..bafee3b 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -64,13 +64,19 @@ TrustMark. Metadata inspection still runs. ```bash remove-ai-watermarks detect-synthid image.png remove-ai-watermarks detect-synthid image.png --json +remove-ai-watermarks detect-synthid resized.png --register-scale ``` The command returns one of `detected`, `not_detected`, or `unsupported`. The runtime detector covers one frozen periodic carrier family in the [calibrated image-size range](synthid.md#32-how-our-tool-detects-the-supported-carrier) -and needs the `pixels` extra. It never resizes the input and does not register -a carrier whose sampling period changed through arbitrary spatial resampling. +and needs the `pixels` extra. The default never resizes the input and does not +register a carrier whose sampling period changed through spatial resampling. +`--register-scale` enables a substantially slower bounded search over measured +carrier periods for images from 250,000 through 10,000,000 decoded pixels, with +both sides at least 64 pixels. It is opt-in and is not used by `identify`. +The measured positive scale range is approximately 0.65 through 1.5; 0.5x +resizes are not reliably detected. It is positive-only: `not_detected` means the score stayed below this detector's threshold, while `unsupported` means the image geometry is outside its scope. Neither result proves that another SynthID epoch or payload is absent. diff --git a/docs/known-limitations.md b/docs/known-limitations.md index 71e1cef..44a4e9d 100644 --- a/docs/known-limitations.md +++ b/docs/known-limitations.md @@ -70,13 +70,17 @@ Google does not publish the proprietary SynthID payload decoder. The project ships a positive-only detector for one measured periodic image carrier in a calibrated image-size range, plus provenance-based recognition from Google AI C2PA or current OpenAI C2PA with an explicit watermark action. -It does not cover images outside that size range, crop, strong JPEG compression, -video, or future carrier epochs. Arbitrary dimensions inside the range are -supported only while the recovered carrier retains its measured 16-pixel -sampling lattice. A spatial resize changes that period; the runtime does not -yet search fractional periods or infer the pre-resize geometry. A `not_detected` -or `unsupported` result is not a negative universal verdict, and removal still -requires the matching provider oracle for confirmation. +It does not cover images outside the selected mode's size range, crop, strong +JPEG compression, video, or future carrier epochs. Arbitrary dimensions inside +the default range are supported only while the recovered carrier retains its +measured 16-pixel sampling lattice. A spatial resize changes that period. The +opt-in `detect-synthid --register-scale` mode searches a bounded +fractional-period range for images from 250,000 through 10,000,000 pixels, but +requires both sides to be at least 64 pixels. Its measured positive range is +approximately scale 0.65 through 1.5; a 0.5 resize aliases the carrier with +ordinary resize lattices and is not reliably detected. It is also slower. A +`not_detected` or `unsupported` result is not a negative universal verdict, and +removal still requires the matching provider oracle for confirmation. For important outputs: diff --git a/docs/module-internals.md b/docs/module-internals.md index 4044b87..8fca511 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -500,7 +500,32 @@ the native threshold, compared with an upper bound of 48 when the true source geometry was supplied. The complete search still failed its small frozen control split: a threshold above 250 development controls accepted two of 150 final controls. Multiplying the canonical score by spectral-period confidence -also accepted two. This branch is not a calibrated runtime fallback. +also accepted two. That baseline was rejected rather than shipped at its +discovery threshold. + +The opt-in `register_scale=True` path is the later calibrated successor. It +keeps the harmonic search, adds three-level scale-space consistency and quadrant +agreement. Its calibrated decision requires the canonical winner to be the +strongest spectral-period candidate, its three-way amplitude to cross the +threshold for that period bin, and both independent high-frequency template +bands to correlate by at least `0.075`. The public registered score is the +minimum normalized margin across those gates, so its decision threshold is +`1.0`. + +The earlier single-threshold version produced 68 false positives among 11,273 +symmetrically downscaled Spaces controls. A top-candidate plus period-bin version +removed those but still produced 6 false positives among 1,000 source-independent +Open Images controls. The high-band gate removed them and then accepted none of +499 previously untouched Open Images reserve controls. The resulting rule +retained 229 of 355 source-disjoint transformed positives: 0/65 at scale 0.5 and +229/290 from scale 0.65 through 1.5. The explicit period-8 rescue is rejected +because resize lattices fully overlap its positive distribution. + +The separately measured geometry range remains 250,000 through 10,000,000 +decoded pixels with both sides at least 64 pixels. The default path and +`identify` remain the native fold. A 20-image real-corpus drift check was +byte-identical after integration. The calibration history and caveats are in the +linked detector research plan. ### Portable metadata record diff --git a/docs/python-api.md b/docs/python-api.md index a30dbb2..51c7b25 100644 --- a/docs/python-api.md +++ b/docs/python-api.md @@ -23,15 +23,23 @@ result = raiw.detect_synthid("input.png") print(result.status) # "detected" | "not_detected" | "unsupported" print(result.score) # float for a supported image size, otherwise None print(result.threshold) # frozen operating point + +# Opt in when the image may have been spatially resized. +registered = raiw.detect_synthid("resized.png", register_scale=True) ``` The detector is positive-only and covers one measured periodic carrier family in the [calibrated image-size range](synthid.md#32-how-our-tool-detects-the-supported-carrier). -Arbitrary dimensions are accepted inside that range, but arbitrary spatial -resampling can change the carrier period and is not registered. `not_detected` -means only that this model did not find its carrier; `unsupported` is kept -separate from a negative result. Neither is proof that the image contains no -SynthID watermark. +Arbitrary dimensions are accepted inside the default range, but spatial +resampling can change the carrier period. `register_scale=True` enables the +slower scale-registered operating point over 250,000 through 10,000,000 decoded +pixels, with both sides at least 64 pixels; the default remains the native-period +detector used by `identify`. Its score is a normalized multi-gate statistic with +a threshold of `1.0`, not the native detector's raw template correlation. Scale +0.5 is outside its reliable positive range. +`not_detected` means only that the selected model did not find its carrier; +`unsupported` is kept separate from a negative result. Neither is proof that +the image contains no SynthID watermark. ## Remove visible marks diff --git a/docs/supported-signals.md b/docs/supported-signals.md index c84214a..7437a19 100644 --- a/docs/supported-signals.md +++ b/docs/supported-signals.md @@ -132,9 +132,14 @@ calibrated image-size range, available through `detect-synthid` and the default pixel pass in `identify` when the `pixels` extra is installed. The unchanged fixed threshold accepted none of the public COCO views in both an observed-geometry challenge and a generated-geometry challenge covering all -modulo-16 edge cases. Arbitrary dimensions in the calibrated range are accepted, -but the input must retain the measured 16-pixel carrier scale: arbitrary spatial -resampling is not registered. The detector does not attribute a provider locally. +modulo-16 edge cases. Arbitrary dimensions in the default calibrated range are +accepted, but the input must retain the measured 16-pixel carrier scale. The +opt-in `detect-synthid --register-scale` mode performs a slower bounded scale +search over its separately measured 250,000-through-10,000,000-pixel range and +requires both sides to be at least 64 pixels. Its measured positive scale range +is approximately 0.65 through 1.5; 0.5x resizes are not reliably detected. +`identify` does not enable that search. Neither detector attributes a provider +locally. The tool also recognizes presence from supported provenance: Google AI C2PA under Google's all-media watermark policy, and current OpenAI C2PA carrying an diff --git a/docs/synthid-detector-removal-plan.md b/docs/synthid-detector-removal-plan.md index 1e9f377..ca22e6b 100644 --- a/docs/synthid-detector-removal-plan.md +++ b/docs/synthid-detector-removal-plan.md @@ -1331,6 +1331,99 @@ acceptance from 49 to 52 of 60 at its development threshold but accepted the same two final controls. This fractional FFT baseline is rejected for runtime use; better period estimation alone does not solve multiple-search specificity. +### 2026-08-11: multiscale calibration and opt-in registered detector + +The fractional search was revisited as a multiple-search calibration problem, +not as period estimation alone. A source-disjoint challenge used 200 confirmed +fixed-carrier positives, 40 independent sources at each scale 0.8, 0.9, 1.1, +1.2, and 1.333. Controls were 4,900 unique COCO images resized into the positive +pixel-count domain, split into 2,900 development and 2,000 final items. The ten +frozen candidates combined exact fractional canonical correlation, quadrant +agreement, and a three-level scale-space score through mean or minimum rules. +Pure box-counting dimensions, multifractal width, lacunarity, roughness, +wavelet-energy slope, and a generic log-Mellin response had already failed the +smaller pilot and were not promoted to the large challenge. + +At the development-only threshold, the three-way mean accepted 177 of 200 +positives versus 163 for canonical correlation alone, with 16 positives rescued +and two lost in the paired decision. Each produced one final false positive, on +different source images, so the development threshold was rejected. Recalibrating +above all 4,900 COCO controls left 176 of 200 three-way positives and 160 of 200 +canonical-only positives. This recalibration is exploratory rather than a new +validation because the final controls had then been inspected. + +A second source-disjoint corpus froze those thresholds before scoring 105 new +positives across scales 0.5, 0.65, 0.8, 1.0, 1.2, 1.333, and 1.5, plus all 879 +content-deduplicated Kodak and Picsum controls available locally. Natural +2816x1536 controls had a heavier null tail than upscaled COCO: the three-way +mean accepted 82 of 105 positives and six controls at the COCO threshold, +whereas canonical-only accepted 79 positives and ten controls. A threshold +above all 5,779 controls retained 253 of the first 305 positives for the +three-way mean. Four coarse pixel-count bins recovered only six more positives, +so size normalization did not explain the main 0.5x miss. + +At 0.5x, a fixed period-8 branch raised canonical correlation for seven of the +15 discovery positives but also won the unconstrained search on 629 of 879 +controls. Searching it unconditionally merely raised the null threshold. The +frozen remedy requires period-8 native correlation of at least 0.35 and takes +the maximum of the old and period-8 three-way scores. It added four positives +without an accepted control in the discovery corpus. On the earlier 4,900 +controls, the frozen gate never opened and none crossed the final three-way +threshold `0.2460603834083705`. + +The improvement itself was then tested on 50 additional source-disjoint 0.5x +positives that influenced neither gate nor threshold. The ungated three-way +score accepted 6; the frozen gate accepted 17, rescuing 11 and losing none. The +exact paired sign test was `p = 0.0009765625`. These results support an opt-in +scale-registered detector over 250,000 through 10,000,000 decoded pixels. The +runtime also requires both sides to be at least 64 pixels so each canonical +quadrant can retain a full 16-by-16 tile. These results do not establish a +universal SynthID decoder: the control maximum helped set the operating point, +33 of 50 new 0.5x positives still missed, and crop, strong codec changes, other +carrier epochs, and provider attribution remain outside the claim. The fast +native fold remains the default and the `identify` path. + +That period-8 conclusion was overturned by a later symmetric negative test. +Every one of the 11,506 known non-Google Spaces controls was downscaled by 0.5, +matching the positive transformation; 11,273 outputs remained in the registered +geometry range. The direct analytic 8-by-8 template was dominated by shared +resize structure: 80.8% of its energy projected onto a 2-by-2 lattice and 94.0% +onto a 4-by-4 lattice. The production period-8 gate opened on 73 controls, and +all 73 crossed the registered threshold. They spanned hosted jobs, Firefly, +China AIGC-labeled generators, Meta-tagged images, OpenAI, and local tools. No +tested secondary condition separated the 11 rescued positives from those 73 +controls. The explicit period-8 rescue is therefore rejected and not part of +the runtime detector. The 0.5x rescue result is retained here as an incident +record against asymmetric transformed-positive evaluation. + +The same symmetric corpus also invalidated the main single-threshold registered +rule: 68 of 11,273 supported controls crossed `0.2460603834083705`. Fixed +thresholds for nine recovered-period bins eliminated those errors while +retaining 238 of 355 transformed positives, but one of 4,900 COCO controls then +crossed through the third spectral candidate. Requiring the canonical winner to +match the strongest spectral candidate removed that case and all 68 Spaces +errors, with 239 of 355 positives retained. This version was not shipped: a +fresh, content-deduplicated 1,000-image Open Images validation corpus produced +six false positives. + +All six Open Images errors concentrated their apparent match in one +middle-frequency ring. Their correlations disappeared or changed sign in the +two higher-frequency rings of the 16-by-16 carrier, unlike the accepted Gemini +positives. Adding a frozen minimum correlation of `0.075` across those two rings +removed the six development errors and retained 229 of 355 positives. A +previously untouched reserve of 499 supported Open Images controls then produced +five amplitude-gate candidates and zero final positives; their maximum +high-band score was `0.004682175209264222`. + +The retained positives by scale were 0/65 at 0.5, 8/15 at 0.65, 53/55 at 0.8, +33/40 at 0.9, 13/15 at 1.0, 30/40 at 1.1, 38/55 at 1.2, 40/55 at 1.333, and +14/15 at 1.5. A physically transformed template recovered strong correlations +for some period-8 positives, but 975 top-candidate Spaces controls fully +overlapped every tested amplitude, spectral-band, and joint feature. No +zero-control operating point retained a period-8 positive. The calibrated +runtime therefore uses the top-candidate, period-bin, and high-band gates and +makes no 0.5x detection claim. + ### 2026-08-10: OpenAI periodic-carrier challenge The OpenAI track repeated the Google carrier method without runtime provenance diff --git a/docs/synthid.md b/docs/synthid.md index 90af5e1..4974e1d 100644 --- a/docs/synthid.md +++ b/docs/synthid.md @@ -483,6 +483,30 @@ less conservative per-period thresholds accepted five final controls. The runtime therefore detects arbitrary decoded dimensions only when the carrier retains its measured 16-pixel scale. +`remove-ai-watermarks detect-synthid image.png --register-scale` opts into the +separately calibrated scale-registered detector. It samples the 30 strongest +template harmonics over fractional periods, reconstructs the three strongest +canonical candidates, and combines full-frame correlation with quadrant and +three-level scale-space consistency. A positive result additionally requires +the canonical winner to be the strongest spectral candidate, a calibrated +threshold for its recovered period, and agreement in two independent +high-frequency template bands. The reported score is the minimum normalized +margin across these requirements; its threshold is `1.0`. + +The final rule retained 229 of 355 source-disjoint transformed positives and +accepted none of 499 previously untouched Open Images controls. It retained +229 of 290 positives from scale 0.65 through 1.5, but none of 65 at scale 0.5. +A separate period-8 rescue was rejected because symmetric 0.5x control resizing +reproduced the same lattice and fully overlapped the positive feature range. +The measured input range is 250,000 through 10,000,000 decoded pixels. +Both dimensions must also be at least 64 pixels so every canonical quadrant can +retain one 16-by-16 carrier tile. + +This mode is not the default because the bounded search is materially slower; +scale registration is explicit and `identify` keeps the native detector. The +registered mode does not reliably detect 0.5x carriers and does not make the +detector universal across crop, codecs, carrier epochs, or providers. + A positive result identifies the carrier but does not attribute a provider. Provider identity still comes from provenance. diff --git a/src/remove_ai_watermarks/_synthid_registered.py b/src/remove_ai_watermarks/_synthid_registered.py new file mode 100644 index 0000000..58ecbe3 --- /dev/null +++ b/src/remove_ai_watermarks/_synthid_registered.py @@ -0,0 +1,304 @@ +"""Opt-in scale registration for the measured periodic SynthID carrier.""" + +# The optional numeric libraries do not provide complete types for this path. +# pyright: reportMissingTypeStubs=false, reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false + +from __future__ import annotations + +import itertools +import math +from dataclasses import dataclass +from typing import TYPE_CHECKING, Any + +import cv2 +import numpy as np + +from remove_ai_watermarks.synthid_detector import folded_template_score + +if TYPE_CHECKING: + from numpy.typing import NDArray + +_PYRAMID_SCALES = (0.75, 1.0, 1.25) +_SEARCH_PERIODS = np.linspace(5.0, 32.0, 541, dtype=np.float64) +_CANONICAL_PERIODS = np.linspace(7.5, 24.5, 1701, dtype=np.float64) +_PERIOD_THRESHOLDS = ( + (7.5, 8.5, 0.3770629524888979), + (8.5, 10.0, 0.25174716660523494), + (10.0, 12.0, 0.284692023502354), + (12.0, 14.0, 0.19794247706938645), + (14.0, 16.0, 0.33930082812296375), + (16.0, 18.0, 0.28915284982686323), + (18.0, 20.0, 0.22885510746595789), + (20.0, 22.0, 0.24570317032768269), + (22.0, 24.5, 0.3142958338390489), +) +REGISTERED_HIGH_BAND_THRESHOLD = 0.075 + + +@dataclass(frozen=True) +class RegisteredComponents: + """Calibrated components of one scale-registered decision.""" + + raw_score: float + amplitude_threshold: float + selected_period: float + spectral_period: float + high_band_score: float + + @property + def decision_score(self) -> float: + """Return a statistic that reaches one only when every gate passes.""" + if self.selected_period != self.spectral_period: + return 0.0 + return min( + self.raw_score / self.amplitude_threshold, + self.high_band_score / REGISTERED_HIGH_BAND_THRESHOLD, + ) + + +def _resize(pixels: NDArray[Any], width: int, height: int) -> NDArray[Any]: + interpolation = cv2.INTER_AREA if width < pixels.shape[1] else cv2.INTER_CUBIC + return np.asarray(cv2.resize(pixels, (width, height), interpolation=interpolation)) + + +def _template_frequency_features( + template: NDArray[Any], +) -> tuple[NDArray[Any], NDArray[Any], NDArray[Any]]: + spectrum = np.fft.fft2(template, axes=(0, 1)) + power = np.sum(np.abs(spectrum) ** 2, axis=2) + power[0, 0] = 0.0 + indices = np.argsort(power.ravel())[::-1][:30] + rows, columns = np.unravel_index(indices, power.shape) + height, width = template.shape[:2] + signed_rows = np.where(rows <= height // 2, rows, rows - height) + signed_columns = np.where(columns <= width // 2, columns, columns - width) + harmonics = np.column_stack((signed_rows, signed_columns)).astype(np.float64) + return harmonics, spectrum[rows, columns], spectrum + + +def _bilinear_sample( + spectrum: NDArray[Any], + y: NDArray[Any], + x: NDArray[Any], +) -> NDArray[Any]: + height, width = spectrum.shape + y_floor = np.floor(y) + x_floor = np.floor(x) + y0 = y_floor.astype(np.int64) % height + x0 = x_floor.astype(np.int64) % width + y1 = (y0 + 1) % height + x1 = (x0 + 1) % width + dy = y - y_floor + dx = x - x_floor + return ( + spectrum[y0, x0] * (1.0 - dy) * (1.0 - dx) + + spectrum[y1, x0] * dy * (1.0 - dx) + + spectrum[y0, x1] * (1.0 - dy) * dx + + spectrum[y1, x1] * dy * dx + ) + + +def _spectral_curve( + pixels: NDArray[Any], + periods: NDArray[Any], + harmonics: NDArray[Any], + coefficients: NDArray[Any], +) -> NDArray[Any]: + height, width = pixels.shape[:2] + y = (periods[:, None] ** -1) * harmonics[None, :, 0] * height + x = (periods[:, None] ** -1) * harmonics[None, :, 1] * width + sampled = np.empty((len(periods), len(harmonics), 3), dtype=np.complex128) + for channel in range(3): + residual = pixels[:, :, channel].astype(np.float32) + residual -= cv2.GaussianBlur( + residual, + (0, 0), + sigmaX=1.0, + sigmaY=1.0, + borderType=cv2.BORDER_REFLECT_101, + ) + spectrum = np.fft.fft2(residual) + sampled[:, :, channel] = _bilinear_sample(spectrum, y % height, x % width) + numerator = np.real(np.sum(np.conj(coefficients)[None, :, :] * sampled, axis=(1, 2))) + denominator = np.linalg.norm(coefficients) * np.linalg.norm(sampled, axis=(1, 2)) + return np.divide( + numerator, + denominator, + out=np.zeros_like(numerator), + where=denominator > 0.0, + ) + + +def _period_candidates( + periods: NDArray[Any], + scores: NDArray[Any], + count: int = 3, +) -> list[float]: + candidates: list[float] = [] + for index in np.argsort(scores)[::-1]: + period = float(periods[index]) + if any(abs(period - existing_period) < 0.25 for existing_period in candidates): + continue + candidates.append(period) + if len(candidates) == count: + break + return candidates + + +def _period_threshold(period: float) -> float: + for index, (lower, upper, threshold) in enumerate(_PERIOD_THRESHOLDS): + if lower <= period < upper or (index == len(_PERIOD_THRESHOLDS) - 1 and period == upper): + return threshold + raise ValueError(f"registered period {period} is outside the calibrated range") + + +def _high_band_score( + folded: NDArray[Any], + template_spectrum: NDArray[Any], +) -> float: + folded_spectrum = np.fft.fft2(folded, axes=(0, 1)) + tile_height, tile_width = template_spectrum.shape[:2] + y_coordinates = np.minimum(np.arange(tile_height), tile_height - np.arange(tile_height)) + x_coordinates = np.minimum(np.arange(tile_width), tile_width - np.arange(tile_width)) + radius = np.sqrt(y_coordinates[:, None] ** 2 + x_coordinates[None, :] ** 2) + correlations = [] + for lower, upper in ((4.5, 6.5), (6.5, 12.0)): + mask = (radius >= lower) & (radius < upper) + selected_folded = folded_spectrum[mask] + selected_template = template_spectrum[mask] + denominator = np.linalg.norm(selected_folded) * np.linalg.norm(selected_template) + correlations.append( + float(np.real(np.vdot(selected_template, selected_folded)) / denominator) if denominator > 0.0 else 0.0 + ) + return min(correlations) + + +def _best_canonical( + pixels: NDArray[Any], + periods: list[float], + template: NDArray[Any], + sigma: float, +) -> tuple[float, NDArray[Any], NDArray[Any], float]: + best_score = -math.inf + best_canonical: NDArray[Any] | None = None + best_folded: NDArray[Any] | None = None + best_period: float | None = None + for period in periods: + predicted_width = round(pixels.shape[1] * template.shape[1] / period) + for delta in range(-4, 5): + width = predicted_width + delta + height = round(pixels.shape[0] * width / pixels.shape[1]) + canonical = _resize(pixels, width, height) + score, folded = folded_template_score(canonical, template, sigma) + if score > best_score: + best_score = score + best_canonical = canonical + best_folded = folded + best_period = period + if best_canonical is None or best_folded is None or best_period is None: + raise RuntimeError("scale registration produced no canonical view") + return float(best_score), best_canonical, best_folded, best_period + + +def _quadrant_median( + canonical: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> float: + tile_height, tile_width = template.shape[:2] + split_y = max(tile_height, (canonical.shape[0] // (2 * tile_height)) * tile_height) + split_x = max(tile_width, (canonical.shape[1] // (2 * tile_width)) * tile_width) + scores = [] + for region in ( + canonical[:split_y, :split_x], + canonical[:split_y, split_x:], + canonical[split_y:, :split_x], + canonical[split_y:, split_x:], + ): + score, _folded = folded_template_score(region, template, sigma) + scores.append(score) + return float(np.median(scores)) + + +def _pyramid_locked_mean( + pixels: NDArray[Any], + harmonics: NDArray[Any], + coefficients: NDArray[Any], + base_curve: NDArray[Any], +) -> float: + curves = [] + candidates = [] + for scale in _PYRAMID_SCALES: + if scale == 1.0: + curve = base_curve + else: + level = _resize( + pixels, + max(16, round(pixels.shape[1] * scale)), + max(16, round(pixels.shape[0] * scale)), + ) + curve = _spectral_curve(level, _SEARCH_PERIODS, harmonics, coefficients) + curves.append(curve) + candidates.append(_period_candidates(_SEARCH_PERIODS, curve)) + combinations = itertools.product(*candidates) + + def spread(combination: tuple[float, ...]) -> float: + normalized_periods = [ + candidate / scale + for candidate, scale in zip( + combination, + _PYRAMID_SCALES, + strict=True, + ) + ] + return float(np.std(np.log(normalized_periods))) + + best = min( + combinations, + key=spread, + ) + base_period = float(np.median([candidate / scale for candidate, scale in zip(best, _PYRAMID_SCALES, strict=True)])) + locked = [ + float(np.interp(base_period * scale, _SEARCH_PERIODS, curve)) + for curve, scale in zip(curves, _PYRAMID_SCALES, strict=True) + ] + return float(np.mean(locked)) + + +def registered_components( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> RegisteredComponents: + """Measure a carrier after bounded scale registration.""" + harmonics, coefficients, template_spectrum = _template_frequency_features(template) + combined_periods = np.concatenate((_SEARCH_PERIODS, _CANONICAL_PERIODS)) + combined_curve = _spectral_curve(pixels, combined_periods, harmonics, coefficients) + base_curve = combined_curve[: len(_SEARCH_PERIODS)] + canonical_curve = combined_curve[len(_SEARCH_PERIODS) :] + candidates = _period_candidates(_CANONICAL_PERIODS, canonical_curve) + baseline, canonical, folded, selected_period = _best_canonical(pixels, candidates, template, sigma) + quadrant = _quadrant_median(canonical, template, sigma) + pyramid = _pyramid_locked_mean( + pixels, + harmonics, + coefficients, + base_curve, + ) + raw_score = float((baseline + quadrant + pyramid) / 3.0) + return RegisteredComponents( + raw_score=raw_score, + amplitude_threshold=_period_threshold(selected_period), + selected_period=selected_period, + spectral_period=candidates[0], + high_band_score=_high_band_score(folded, template_spectrum), + ) + + +def registered_score( + pixels: NDArray[Any], + template: NDArray[Any], + sigma: float, +) -> float: + """Return the calibrated registered decision statistic.""" + return registered_components(pixels, template, sigma).decision_score diff --git a/src/remove_ai_watermarks/cli.py b/src/remove_ai_watermarks/cli.py index 16b8087..e2219a3 100644 --- a/src/remove_ai_watermarks/cli.py +++ b/src/remove_ai_watermarks/cli.py @@ -1322,7 +1322,12 @@ def cmd_video_batch( @main.command("detect-synthid") @click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) @click.option("--json", "as_json", is_flag=True, help="Emit the detector result as JSON.") -def cmd_detect_synthid(source: Path, as_json: bool) -> None: +@click.option( + "--register-scale", + is_flag=True, + help="Search the slower calibrated range of spatial carrier scales.", +) +def cmd_detect_synthid(source: Path, as_json: bool, register_scale: bool) -> None: """Detect the SynthID periodic pixel carrier at calibrated image sizes. A negative result means this detector did not find its supported carrier; it @@ -1332,7 +1337,7 @@ def cmd_detect_synthid(source: Path, as_json: bool) -> None: source = _validate_image(source) try: - result = detect_synthid(source) + result = detect_synthid(source, register_scale=register_scale) except RuntimeError as exc: raise click.ClickException(str(exc)) from exc @@ -1346,10 +1351,15 @@ def cmd_detect_synthid(source: Path, as_json: bool) -> None: if result.score is not None: console.print(f" Score: {result.score:.6f} (threshold: {result.threshold:.6f})") console.print(f" Detector: {result.detector}") + scale_scope = ( + " Bounded spatial-scale registration was enabled. A negative or\n" + if register_scale + else " Arbitrary spatial resampling was not registered. A negative or\n" + ) console.print( " Scope: one confirmed periodic carrier family in a calibrated image-size range.\n" - " Arbitrary spatial resampling is not registered. A negative or\n" - " unsupported result is not proof that SynthID is absent." + + scale_scope + + " unsupported result is not proof that SynthID is absent." ) diff --git a/src/remove_ai_watermarks/synthid_detector.py b/src/remove_ai_watermarks/synthid_detector.py index c6980a5..ee79500 100644 --- a/src/remove_ai_watermarks/synthid_detector.py +++ b/src/remove_ai_watermarks/synthid_detector.py @@ -2,8 +2,8 @@ This is a positive-only detector for one measured carrier epoch, not Google's private payload decoder. A positive result is strong local evidence for the -carrier. A negative result means only that this exact detector did not find it; -image sizes outside the calibrated pixel-count range are reported separately. +carrier. A negative result means only that the selected detector did not find +it; image sizes outside that mode's calibrated range are reported separately. The numeric runtime requires the ``pixels`` extra. Imports remain lazy so the package's metadata-only paths stay dependency-light. @@ -25,6 +25,7 @@ if TYPE_CHECKING: SynthIDDetectionStatus = Literal["detected", "not_detected", "unsupported"] DETECTOR_ID = "synthid-periodic-tile-v2" +REGISTERED_DETECTOR_ID = "synthid-periodic-tile-registered-v2" MODEL_FILENAME = "synthid_periodic_tile_2048_v1.npz" # The template remains frozen at this model geometry. Runtime images are never # resized. The supported pixel-count interval is the separately challenged domain: @@ -35,6 +36,12 @@ MODEL_HEIGHT = 2048 MIN_SUPPORTED_PIXELS = 1_000_000 MAX_SUPPORTED_PIXELS = 18_000_000 TILE_THRESHOLD = 0.17357069773071196 +REGISTERED_MIN_SUPPORTED_PIXELS = 250_000 +REGISTERED_MAX_SUPPORTED_PIXELS = 10_000_000 +REGISTERED_MIN_SIDE = 64 +# The registered score is the minimum normalized margin across its amplitude, +# spectral-candidate, and high-frequency agreement gates. +REGISTERED_THRESHOLD = 1.0 INSTALL_HINT = "install the pixel extra: uv add 'remove-ai-watermarks[pixels]'" @@ -206,11 +213,43 @@ def _geometry_supported(width: int, height: int) -> bool: return MIN_SUPPORTED_PIXELS <= pixels <= MAX_SUPPORTED_PIXELS -def detect_synthid(image_path: str | Path, *, image: NDArray[Any] | None = None) -> SynthIDDetection: +def _registered_geometry_supported(width: int, height: int) -> bool: + """Whether scale registration was challenged at this decoded size.""" + pixels = width * height + return ( + min(width, height) >= REGISTERED_MIN_SIDE + and REGISTERED_MIN_SUPPORTED_PIXELS <= pixels <= REGISTERED_MAX_SUPPORTED_PIXELS + ) + + +def folded_template_score( + pixels: NDArray[Any], + template: NDArray[Any], + denoise_sigma: float, +) -> tuple[float, NDArray[Any]]: + """Fold PIXELS at the model geometry and score the normalized tile.""" + tile_height, tile_width = template.shape[:2] + folded = fold_residual_template( + pixels, + tile_height=tile_height, + tile_width=tile_width, + denoise_sigma=denoise_sigma, + ) + normalized, _norm = unit_tile(folded) + return float((template * normalized).sum()), folded + + +def detect_synthid( + image_path: str | Path, + *, + image: NDArray[Any] | None = None, + register_scale: bool = False, +) -> SynthIDDetection: """Detect the supported periodic carrier in IMAGE_PATH. ``not_detected`` is not a clean-image guarantee. It means only that the - frozen periodic carrier did not cross its calibrated threshold. + frozen periodic carrier did not cross its calibrated threshold. Set + ``register_scale`` for the slower, separately calibrated resize search. """ path = Path(image_path) if image is None: @@ -219,13 +258,19 @@ def detect_synthid(image_path: str | Path, *, image: NDArray[Any] | None = None) if image.ndim != 3 or image.shape[2] != 3: raise ValueError("image must be a three-channel BGR array") height, width = image.shape[:2] - if not _geometry_supported(width, height): + geometry_supported = ( + _registered_geometry_supported(width, height) if register_scale else _geometry_supported(width, height) + ) + threshold = REGISTERED_THRESHOLD if register_scale else TILE_THRESHOLD + detector_id = REGISTERED_DETECTOR_ID if register_scale else DETECTOR_ID + if not geometry_supported: return SynthIDDetection( status="unsupported", width=width, height=height, score=None, - threshold=TILE_THRESHOLD, + threshold=threshold, + detector=detector_id, ) if not is_available(): raise RuntimeError(f"SynthID pixel detection needs numpy and OpenCV; {INSTALL_HINT}") @@ -233,7 +278,7 @@ def detect_synthid(image_path: str | Path, *, image: NDArray[Any] | None = None) import numpy as np from PIL import Image - template, sigma, _model_height, _model_width, tile_height, tile_width = _load_template() + template, sigma, *_model = _load_template() if image is None: with Image.open(path) as source: pixels = np.asarray(source.convert("RGB"), dtype=np.uint8) @@ -241,18 +286,17 @@ def detect_synthid(image_path: str | Path, *, image: NDArray[Any] | None = None) pixels = np.asarray(image[:, :, ::-1], dtype=np.uint8) if pixels.shape != (height, width, 3): raise RuntimeError("decoded image geometry does not match its header") - folded = fold_residual_template( - pixels, - tile_height=tile_height, - tile_width=tile_width, - denoise_sigma=sigma, - ) - normalized, _norm = unit_tile(folded) - score = float(np.sum(template * normalized)) + if register_scale: + from remove_ai_watermarks._synthid_registered import registered_score + + score = registered_score(pixels, template, sigma) + else: + score, _folded = folded_template_score(pixels, template, sigma) return SynthIDDetection( - status="detected" if score >= TILE_THRESHOLD else "not_detected", + status="detected" if score >= threshold else "not_detected", width=width, height=height, score=score, - threshold=TILE_THRESHOLD, + threshold=threshold, + detector=detector_id, ) diff --git a/tests/test_cli.py b/tests/test_cli.py index a1722a0..149b8ce 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -741,6 +741,7 @@ class TestDetectSynthIDCommand: result = runner.invoke(main, ["detect-synthid", "--help"]) assert result.exit_code == 0 assert "calibrated image sizes" in result.output + assert "--register-scale" in result.output def test_unsupported_geometry_is_machine_readable(self, runner, tmp_clean_png): result = runner.invoke(main, ["detect-synthid", str(tmp_clean_png), "--json"]) @@ -750,6 +751,23 @@ class TestDetectSynthIDCommand: assert payload["status"] == "unsupported" assert payload["score"] is None + def test_registered_scale_mode_is_machine_readable(self, runner, tmp_clean_png): + from remove_ai_watermarks.synthid_detector import ( + REGISTERED_DETECTOR_ID, + REGISTERED_THRESHOLD, + ) + + result = runner.invoke( + main, + ["detect-synthid", str(tmp_clean_png), "--register-scale", "--json"], + ) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload["status"] == "unsupported" + assert payload["threshold"] == REGISTERED_THRESHOLD + assert payload["detector"] == REGISTERED_DETECTOR_ID + def test_non_json_output_preserves_negative_scope(self, runner, tmp_clean_png): result = runner.invoke(main, ["detect-synthid", str(tmp_clean_png)]) @@ -757,6 +775,15 @@ class TestDetectSynthIDCommand: assert "unsupported" in result.output assert "not proof that SynthID is absent" in result.output + def test_registered_non_json_output_names_the_bounded_search(self, runner, tmp_clean_png): + result = runner.invoke( + main, + ["detect-synthid", str(tmp_clean_png), "--register-scale"], + ) + + assert result.exit_code == 0, result.output + assert "Bounded spatial-scale registration was enabled" in result.output + class TestBatchCommand: """Tests for the 'batch' subcommand.""" diff --git a/tests/test_synthid_detector.py b/tests/test_synthid_detector.py index 2f23429..885a264 100644 --- a/tests/test_synthid_detector.py +++ b/tests/test_synthid_detector.py @@ -28,6 +28,25 @@ def supported_images(tmp_path_factory: pytest.TempPathFactory) -> tuple[Path, Pa return positive, negative +@pytest.fixture(scope="module") +def registered_scale_positive(tmp_path_factory: pytest.TempPathFactory) -> Path: + """Create a strong period-12.8 carrier by shrinking a period-16 source.""" + import cv2 + + directory = tmp_path_factory.mktemp("synthid-registered") + template, *_model = detector._load_template() + scaled_tile = template / np.max(np.abs(template)) * 40.0 + source = np.tile(scaled_tile, (64, 64, 1)) + 128.0 + pixels = cv2.resize( + np.clip(np.rint(source), 0, 255).astype(np.uint8), + (819, 819), + interpolation=cv2.INTER_AREA, + ) + path = directory / "period-12.8-positive.png" + Image.fromarray(pixels, "RGB").save(path) + return path + + def test_bundled_model_is_the_frozen_calibrated_artifact() -> None: model = Path(detector.__file__).parent / "assets" / detector.MODEL_FILENAME @@ -55,6 +74,36 @@ def test_geometry_outside_the_challenged_pixel_count_range_is_unsupported( assert not detector._geometry_supported(width, height) +@pytest.mark.parametrize( + ("width", "height", "supported"), + [ + (500, 500, True), + (4000, 2500, True), + (64, 3907, True), + (499, 500, False), + (4001, 2500, False), + (32, 7813, False), + ], +) +def test_registered_geometry_uses_its_measured_pixel_count_range( + width: int, + height: int, + supported: bool, +) -> None: + assert detector._registered_geometry_supported(width, height) is supported + + +def test_registered_mode_rejects_a_side_too_short_for_quadrants(tmp_path: Path) -> None: + path = tmp_path / "too-narrow.png" + Image.new("RGB", (32, 7813), "white").save(path) + + result = detector.detect_synthid(path, register_scale=True) + + assert result.status == "unsupported" + assert result.score is None + assert result.detector == detector.REGISTERED_DETECTOR_ID + + def test_detects_supported_periodic_carrier(supported_images: tuple[Path, Path]) -> None: positive, _negative = supported_images @@ -86,6 +135,95 @@ def test_detects_unregistered_non_divisible_geometry_in_size_range(tmp_path: Pat assert result.score > result.threshold +def test_registered_mode_detects_a_rescaled_carrier(registered_scale_positive: Path) -> None: + default = detector.detect_synthid(registered_scale_positive) + registered = detector.detect_synthid(registered_scale_positive, register_scale=True) + + assert default.status == "unsupported" + assert registered.status == "detected" + assert registered.score is not None + assert registered.score > registered.threshold + assert registered.threshold == detector.REGISTERED_THRESHOLD + assert registered.detector == detector.REGISTERED_DETECTOR_ID + + +def test_registered_threshold_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + registered_scale_positive: Path, +) -> None: + baseline = detector.detect_synthid(registered_scale_positive, register_scale=True) + assert baseline.score is not None + mutated_threshold = float(np.nextafter(baseline.score, np.inf)) + monkeypatch.setattr(detector, "REGISTERED_THRESHOLD", mutated_threshold) + + mutated = detector.detect_synthid(registered_scale_positive, register_scale=True) + + assert mutated.status == "not_detected" + assert mutated.threshold == mutated_threshold + + +def test_registered_period_thresholds_cover_the_bounded_search() -> None: + from remove_ai_watermarks._synthid_registered import _period_threshold + + assert _period_threshold(7.5) == pytest.approx(0.3770629524888979) + assert _period_threshold(12.0) == pytest.approx(0.19794247706938645) + assert _period_threshold(24.5) == pytest.approx(0.3142958338390489) + with pytest.raises(ValueError, match="outside"): + _period_threshold(7.49) + + +def test_registered_amplitude_threshold_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + registered_scale_positive: Path, +) -> None: + import remove_ai_watermarks._synthid_registered as registered_detector + + baseline = detector.detect_synthid(registered_scale_positive, register_scale=True) + assert baseline.status == "detected" + monkeypatch.setattr( + registered_detector, + "_PERIOD_THRESHOLDS", + ((7.5, 24.5, float("inf")),), + ) + + mutated = detector.detect_synthid(registered_scale_positive, register_scale=True) + + assert mutated.status == "not_detected" + + +def test_registered_spectral_candidate_disagreement_blocks_decision() -> None: + from remove_ai_watermarks._synthid_registered import RegisteredComponents + + matching = RegisteredComponents(0.5, 0.25, 12.8, 12.8, 0.15) + mismatching = RegisteredComponents(0.5, 0.25, 12.8, 12.9, 0.15) + + assert matching.decision_score == pytest.approx(2.0) + assert mismatching.decision_score == pytest.approx(0.0) + + +def test_registered_high_band_mutation_changes_the_real_verdict( + monkeypatch: pytest.MonkeyPatch, + registered_scale_positive: Path, +) -> None: + import remove_ai_watermarks._synthid_registered as registered_detector + + components = registered_detector.registered_components( + np.asarray(Image.open(registered_scale_positive).convert("RGB"), dtype=np.uint8), + detector._load_template()[0], + detector._load_template()[1], + ) + assert components.decision_score >= detector.REGISTERED_THRESHOLD + monkeypatch.setattr( + registered_detector, + "REGISTERED_HIGH_BAND_THRESHOLD", + float(np.nextafter(components.high_band_score, np.inf)), + ) + + mutated = detector.detect_synthid(registered_scale_positive, register_scale=True) + + assert mutated.status == "not_detected" + + def test_supported_negative_does_not_claim_clean(supported_images: tuple[Path, Path]) -> None: _positive, negative = supported_images