Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
09a4b20ac6 | ||
|
|
9bb4e9cd54 | ||
|
|
5052dbe63c | ||
|
|
2fe4d3c3a0 | ||
|
|
adbc6a3046 | ||
|
|
326840bab4 | ||
|
|
29de21256b | ||
|
|
07b07634cf | ||
|
|
54a19059d5 | ||
|
|
5e85f3ac6f | ||
|
|
36c36212f0 | ||
|
|
84ea8ede2e | ||
|
|
8f30f8bdb3 | ||
|
|
925401c905 | ||
|
|
55e7c53699 | ||
|
|
2ff06d4c5a | ||
|
|
1410a8e064 | ||
|
|
11ca7d6054 | ||
|
|
5991894ca5 | ||
|
|
5fa14f9e26 | ||
|
|
f7e0a18e6c | ||
|
|
bc5922451f | ||
|
|
798c94e6ae | ||
|
|
67d948f57d | ||
|
|
1d0e6b54dd | ||
|
|
fa4748a6a8 | ||
|
|
1e9572e2d4 | ||
|
|
1a6f50db75 | ||
|
|
b684423fc4 | ||
|
|
16bc9879e9 | ||
|
|
343da7f87d | ||
|
|
6f2d50cde8 | ||
|
|
95f467b4ad | ||
|
|
34505e5cc2 | ||
|
|
01539f5f58 | ||
|
|
23696f42a8 | ||
|
|
08a4a8d299 |
No files matched your search
@@ -25,6 +25,21 @@ Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxat
|
||||
|
||||
From a worktree, `uv run` imports the package from the MAIN checkout -- that is where the editable install points. A script measuring a worktree's edit must insert that worktree's `src` at `sys.path[0]` and assert `module.__file__` resolves inside it, or it silently compares unmodified code against itself.
|
||||
|
||||
## Visible-mark example gallery
|
||||
|
||||
Every registered mark carries a committed example: `data/fixtures/visible/<key>/example.png`
|
||||
(image marks) and `example.mp4` (video marks), regenerated by
|
||||
`scripts/render_visible_examples.py`. `tests/test_visible_examples.py` is the canary: it
|
||||
fails when a mark is registered without an example and when an engine stops detecting its
|
||||
own example. The examples are deterministic synthetic composites of the committed
|
||||
silhouettes. When the generator fails after a geometry or gate change, fix the generator
|
||||
(and the engine) together; do not hand-edit the binaries.
|
||||
|
||||
Visible-mark content must distinguish the registry key, the provider's canonical
|
||||
product name, and the literal text rendered in the mark whenever those differ. Verify
|
||||
external provider names against the provider's official source rather than inferring
|
||||
them from an asset filename or OCR text.
|
||||
|
||||
## Model-adjacent tests
|
||||
|
||||
Do not classify an entire module as untestable because its main path downloads a model. Keep pure behavior covered without downloads, including:
|
||||
@@ -39,6 +54,9 @@ Do not classify an entire module as untestable because its main path downloads a
|
||||
stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a
|
||||
face. When one profile inherits another's stage, guard the invariants that stage
|
||||
relies on, not just the code path.
|
||||
- the photo-classify gate in `test_classify.py`: DEFINITELY is ridge AND MLP,
|
||||
POSSIBLY does not emit a provider, and `identify` does not import
|
||||
`remove_ai_watermarks.classify`.
|
||||
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
|
||||
mirror another, compare them field by field rather than pinning the values you happen
|
||||
to know about, so the next field added on one side and not the other fails at the
|
||||
@@ -74,6 +92,23 @@ status set; `TestIdentifyRealSamples::test_no_committed_fixture_reports_a_truste
|
||||
is that guard. Read a capability the stack does not have as a missing input, never as a
|
||||
negative finding.
|
||||
|
||||
## The docs are a seam, and they drift like one
|
||||
|
||||
`tests/test_docs_cover_the_public_surface.py` reads the live Click tree and the
|
||||
live `InvisibleOptions` dataclass and requires every knob to be NAMED in the
|
||||
user-facing pages. The code-to-code seams were already guarded and the
|
||||
code-to-docs seam was not, so it drifted exactly the way an unguarded seam does:
|
||||
`docs/cli.md` named none of `--adaptive-polish`, `--controlnet-scale`,
|
||||
`--detect/--no-detect`, `--humanize`, `--strength`, `--tile-size`,
|
||||
`--tile-overlap` or `--unsharp`, and `docs/python-api.md` named 9 of the 16
|
||||
option fields.
|
||||
|
||||
Group option spellings by the Click PARAMETER, never by string surgery on the
|
||||
names. A boolean pair can be spelled anything -- `--strip-metadata/--keep-metadata`,
|
||||
`--keep-standard/--remove-all` -- so deriving the partner from a `--no-` prefix
|
||||
silently splits those pairs and demands both halves. Either spelling documents
|
||||
the pair.
|
||||
|
||||
## One measurement, one gate seam
|
||||
|
||||
A detector is split into a trust-level-blind scan and a verdict that applies the
|
||||
@@ -88,6 +123,11 @@ rules follow, and both were broken in practice before they were written down:
|
||||
- Detection and the removal mask must read ONE sweep. The winning box travels on
|
||||
`TextMarkDetection.match_box` and the registry threads the detection into the mask
|
||||
builder; a mask path that re-runs its own sweep is how the two drift apart.
|
||||
- When the research lattice runtime in `scripts/synthid_runtime/` routes by image
|
||||
geometry, preserve the returned `SynthIDDetection.detector` in score manifests
|
||||
and downstream routers. Record an inactive expert as explicitly unsupported;
|
||||
never attribute a routed large-image score to the fixed expert. This expert is
|
||||
not part of the public package.
|
||||
|
||||
The C2PA manifest-store JSON is NOT stable across reads: the reader regenerates manifest
|
||||
URNs and instance ids. Compare the derived `c2pa_info`, never the raw store.
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
# HuggingFace token (optional; only needed for gated/private models)
|
||||
# Get yours at: https://huggingface.co/settings/tokens
|
||||
# HF_TOKEN=
|
||||
|
||||
# OpenAI API key (optional; only for explicit verify-openai-synthid uploads)
|
||||
# Create and manage keys at: https://platform.openai.com/api-keys
|
||||
# OPENAI_API_KEY=
|
||||
@@ -0,0 +1,72 @@
|
||||
name: Publish photo classifier to Hugging Face
|
||||
|
||||
# Manual only. Not part of a library release. The Space demo
|
||||
# (wiltodelta/remove-ai-watermarks) stays on distribute.yml / raiw-hf-space.
|
||||
#
|
||||
# Hub model: wiltodelta/raiw-photo-classify
|
||||
# Secret: HF_TOKEN, write-role Hugging Face token (same secret Space sync uses).
|
||||
#
|
||||
# Weights are not in git. Full mode downloads them from a GitHub Release
|
||||
# of this repository, then uploads with the card from docs/photo-classify-hf/.
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
confirm:
|
||||
description: "Type publish"
|
||||
required: true
|
||||
mode:
|
||||
description: "full = card plus weights from the release tag; card = README and operating-point only"
|
||||
type: choice
|
||||
options:
|
||||
- full
|
||||
- card
|
||||
default: full
|
||||
weights_tag:
|
||||
description: "GitHub Release tag that holds clip-l-ft.pt, probe, detector.pt, provider.pt"
|
||||
required: false
|
||||
default: photo-classify-freeze-2026-08-31
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
if: ${{ github.event.inputs.confirm == 'publish' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 180
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Download freeze weights from the GitHub Release
|
||||
if: ${{ github.event.inputs.mode == 'full' }}
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
TAG: ${{ github.event.inputs.weights_tag }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
mkdir -p weights
|
||||
gh release download "$TAG" --repo "${{ github.repository }}" --dir weights
|
||||
ls -l weights
|
||||
|
||||
- name: Publish to Hugging Face
|
||||
env:
|
||||
HF_TOKEN: ${{ secrets.HF_TOKEN }}
|
||||
MODE: ${{ github.event.inputs.mode }}
|
||||
SHA: ${{ github.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${HF_TOKEN:-}" ]; then
|
||||
echo "HF_TOKEN secret is not set" >&2
|
||||
exit 1
|
||||
fi
|
||||
extra=(--message "Photo-classify freeze (GitHub ${SHA:0:7})")
|
||||
if [ "$MODE" = "card" ]; then
|
||||
extra+=(--card-only)
|
||||
else
|
||||
extra+=(--src weights)
|
||||
fi
|
||||
uv run --no-project --with huggingface-hub python scripts/publish_photo_classify_hf.py "${extra[@]}"
|
||||
@@ -5,6 +5,9 @@ on:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: tests-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
@@ -0,0 +1,247 @@
|
||||
name: Verify release surfaces
|
||||
|
||||
# Post-release verification: after distribute.yml fans a release out to
|
||||
# Homebrew, the HF Space, and the ComfyUI Registry, confirm every published
|
||||
# surface actually landed and works. This workflow is the automated twin of
|
||||
# the manual checklist in docs/release-and-distribution.md ("Release
|
||||
# verification"), not a replacement for the library's own test suite (which
|
||||
# already ran in test.yml and publish.yml).
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: "Version to verify (no leading v). Blank = latest on PyPI."
|
||||
required: false
|
||||
workflow_run:
|
||||
workflows: ["Distribute on release"]
|
||||
types: [completed]
|
||||
|
||||
# No step reads the GitHub token any more: the ComfyUI check moved to the
|
||||
# registry's public API. Keep the job read-only so it cannot write on a repo
|
||||
# whose default token permission is read/write.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
verify:
|
||||
# Only run on a successful distribute (not on manual dispatch, where the
|
||||
# version input or PyPI-latest provides the target).
|
||||
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 25
|
||||
env:
|
||||
# The Space id appears in the Hub API, the raw-file URL and the app host,
|
||||
# spelled three different ways. Name it once so a move cannot half-land.
|
||||
SPACE_REPO: wiltodelta/remove-ai-watermarks
|
||||
SPACE_URL: https://wiltodelta-remove-ai-watermarks.hf.space
|
||||
|
||||
steps:
|
||||
- name: Resolve target version
|
||||
id: v
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -n "${{ github.event.inputs.version }}" ]; then
|
||||
v="${{ github.event.inputs.version }}"
|
||||
else
|
||||
v=$(curl -fsSL https://pypi.org/pypi/remove-ai-watermarks/json \
|
||||
| python3 -c "import sys,json;print(json.load(sys.stdin)['info']['version'])")
|
||||
fi
|
||||
echo "version=$v" >> "$GITHUB_OUTPUT"
|
||||
echo "Verifying release $v"
|
||||
|
||||
# -- PyPI ---------------------------------------------------------------
|
||||
- name: Verify PyPI
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking PyPI for $VERSION..."
|
||||
rel=$(curl -fsSL "https://pypi.org/pypi/remove-ai-watermarks/$VERSION/json")
|
||||
echo "$rel" | python3 -c "
|
||||
import json, sys
|
||||
d = json.load(sys.stdin)
|
||||
assert d['info']['version'] == '$VERSION', f\"PyPI version mismatch: {d['info']['version']}\"
|
||||
assert d['urls'], 'No files on PyPI'
|
||||
has_wheel = any(u['packagetype'] == 'bdist_wheel' for u in d['urls'])
|
||||
has_sdist = any(u['packagetype'] == 'sdist' for u in d['urls'])
|
||||
assert has_wheel, 'No wheel on PyPI'
|
||||
assert has_sdist, 'No sdist on PyPI'
|
||||
print(f' PyPI: {len(d[\"urls\"])} files (wheel + sdist) for $VERSION')
|
||||
"
|
||||
|
||||
# -- Clean install from PyPI ---------------------------------------------
|
||||
- name: Install uv
|
||||
run: curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
|
||||
- name: Verify clean install
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Installing remove-ai-watermarks==$VERSION in a clean environment..."
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
result=$(uvx --from "remove-ai-watermarks==$VERSION" remove-ai-watermarks --version 2>&1) \
|
||||
|| { echo "::error::Clean install failed: $result"; exit 1; }
|
||||
echo " Clean install: $result"
|
||||
|
||||
# -- Homebrew --------------------------------------------------------------
|
||||
- name: Verify Homebrew formula
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking Homebrew tap formula..."
|
||||
formula=$(curl -fsSL \
|
||||
"https://raw.githubusercontent.com/wiltodelta/homebrew-tap/main/Formula/remove-ai-watermarks.rb")
|
||||
echo "$formula" | grep -q "remove_ai_watermarks-$VERSION.tar.gz" \
|
||||
|| { echo "::error::Homebrew formula does not reference $VERSION"; \
|
||||
echo "Formula URL line:"; echo "$formula" | grep '^url'; exit 1; }
|
||||
echo " Homebrew: formula points to $VERSION"
|
||||
|
||||
# -- ComfyUI ------------------------------------------------------------------
|
||||
- name: Verify ComfyUI registry node requires this release
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking the ComfyUI Registry node..."
|
||||
# Read the published versions directly. Do NOT proxy through the sync
|
||||
# workflow's run status: the node repo runs that workflow on a daily
|
||||
# schedule too, so a green nightly says nothing about THIS release.
|
||||
#
|
||||
# And read the versions list, not `latest_version` on the node object.
|
||||
# The registry reviews a new upload before activating it, and
|
||||
# `latest_version` reports only the active one -- so for the whole
|
||||
# review window a correctly published version looks identical to one
|
||||
# that was never uploaded. Those are opposite situations: one is a
|
||||
# queue nobody here controls, the other is a broken release.
|
||||
versions=$(curl -fsSL "https://api.comfy.org/nodes/remove-ai-watermarks/versions")
|
||||
status=$(printf '%s' "$versions" | VERSION="$VERSION" python3 -c "
|
||||
import json, os, sys
|
||||
version = os.environ['VERSION']
|
||||
floor = f'>={version}'
|
||||
matches = [
|
||||
v for v in json.load(sys.stdin)
|
||||
if any(floor in d for d in (v.get('dependencies') or []))
|
||||
]
|
||||
if not matches:
|
||||
sys.exit(f'::error::No ComfyUI node version requires {version}; the sync never published one')
|
||||
# Sort rather than trusting the response order: the endpoint happens to
|
||||
# return newest-first today and nothing documents that it must.
|
||||
newest = max(matches, key=lambda v: v.get('createdAt') or '')
|
||||
print(f\"{newest.get('version')} {newest.get('status')}\")
|
||||
")
|
||||
node_version=${status%% *}
|
||||
node_status=${status##* }
|
||||
echo " ComfyUI: node $node_version requires $VERSION (status $node_status)"
|
||||
case "$node_status" in
|
||||
NodeVersionStatusActive)
|
||||
echo "- ComfyUI: registry node $node_version requires $VERSION and is live" >> "$GITHUB_STEP_SUMMARY"
|
||||
;;
|
||||
NodeVersionStatusPending)
|
||||
# Published and waiting on the registry's own review. Say exactly
|
||||
# that in the summary rather than reporting it as live, and do not
|
||||
# fail: an external moderation queue going red on every release
|
||||
# trains everyone to ignore this workflow.
|
||||
echo "::notice::ComfyUI node $node_version is published and awaiting registry review."
|
||||
echo "- ComfyUI: registry node $node_version requires $VERSION, awaiting registry activation" >> "$GITHUB_STEP_SUMMARY"
|
||||
;;
|
||||
*)
|
||||
echo "::error::ComfyUI node $node_version is $node_status"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
# -- HF Space -------------------------------------------------------------------
|
||||
- name: Verify the HF Space serves this release
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking the HF Space pin..."
|
||||
# The Space pin bump is a MANUAL step (see docs/release-and-distribution.md):
|
||||
# distribute.yml only factory-rebuilds whatever is already pinned. Failing
|
||||
# here is the point -- a demo left on an older release is an incomplete
|
||||
# release, and only this check can say so.
|
||||
reqs=$(curl -fsSL "https://huggingface.co/spaces/$SPACE_REPO/raw/main/requirements.txt")
|
||||
pinned=$(printf '%s\n' "$reqs" | sed -n 's/^remove-ai-watermarks==\([0-9][0-9a-zA-Z.]*\).*/\1/p' | head -1)
|
||||
if [ -z "$pinned" ]; then
|
||||
echo "::error::Space requirements.txt has no remove-ai-watermarks== pin"
|
||||
exit 1
|
||||
fi
|
||||
if [ "$pinned" != "$VERSION" ]; then
|
||||
echo "::error::HF Space is pinned to $pinned, not $VERSION."
|
||||
echo " Bump remove-ai-watermarks in wiltodelta/raiw-hf-space (pyproject.toml,"
|
||||
echo " uv lock, re-exported requirements.txt) and push; sync-to-hf.yml mirrors it."
|
||||
exit 1
|
||||
fi
|
||||
echo " HF Space: requirements pin is $pinned"
|
||||
|
||||
- name: Verify the HF Space is running and answering
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# A factory rebuild leaves the Space BUILDING for a while; wait for the
|
||||
# runtime stage the API reports rather than for a bare HTTP 200, which a
|
||||
# build page also returns.
|
||||
for attempt in $(seq 1 20); do
|
||||
stage=$(curl -fsSL "https://huggingface.co/api/spaces/$SPACE_REPO" \
|
||||
| python3 -c "import json,sys;print((json.load(sys.stdin).get('runtime') or {}).get('stage',''))")
|
||||
echo " attempt $attempt: runtime stage=$stage"
|
||||
[ "$stage" = "RUNNING" ] && break
|
||||
case "$stage" in
|
||||
BUILD_ERROR|RUNTIME_ERROR|CONFIG_ERROR)
|
||||
echo "::error::HF Space runtime stage is $stage"; exit 1 ;;
|
||||
esac
|
||||
sleep 15
|
||||
done
|
||||
if [ "$stage" != "RUNNING" ]; then
|
||||
echo "::error::HF Space never reached RUNNING (last stage: $stage)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Alive is not the same as working. Drive the two CPU surfaces the demo
|
||||
# exists for, against the Space's own committed example, and require a
|
||||
# real verdict rather than any HTTP 200.
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
curl -fsSL -o /tmp/example.png \
|
||||
"https://huggingface.co/spaces/$SPACE_REPO/resolve/main/examples/doubao-aigc.png"
|
||||
uv run --no-project --with 'gradio_client>=1,<2' python - <<'PYEOF'
|
||||
import os
|
||||
import sys
|
||||
|
||||
from gradio_client import Client, handle_file
|
||||
|
||||
# The committed example is a 5 MB PNG, and httpx's default 5 s write
|
||||
# timeout aborts that upload on a slow link before the Space ever sees
|
||||
# it -- which would read as a broken demo rather than a slow one.
|
||||
client = Client(os.environ["SPACE_URL"], verbose=False, httpx_kwargs={"timeout": 300})
|
||||
|
||||
report = client.predict(path=handle_file("/tmp/example.png"), api_name="/run_identify")
|
||||
if "Verdict:" not in report:
|
||||
sys.exit(f"::error::/run_identify returned no verdict: {report[:200]!r}")
|
||||
print(f" HF Space identify: {report.splitlines()[0]}")
|
||||
|
||||
_image, _file, info = client.predict(
|
||||
path=handle_file("/tmp/example.png"),
|
||||
sensitivity_label="auto (recommended)",
|
||||
api_name="/run_visible",
|
||||
)
|
||||
if "Removed:" not in info:
|
||||
sys.exit(f"::error::/run_visible removed nothing on the Doubao example: {info!r}")
|
||||
print(f" HF Space visible: {info.splitlines()[0]}")
|
||||
PYEOF
|
||||
echo "- HF Space: pinned to $VERSION, RUNNING, identify + visible answered" >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Summary
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
# Every check above appends its own line to the step summary as it passes,
|
||||
# and every one of them fails the job instead of warning. So this heading
|
||||
# can only be reached when all of them actually held -- an unconditional
|
||||
# "verified" block printed here once claimed surfaces nobody had checked.
|
||||
sed -i "1i ## Release $VERSION verified" "$GITHUB_STEP_SUMMARY"
|
||||
cat "$GITHUB_STEP_SUMMARY"
|
||||
@@ -30,6 +30,10 @@ _refs/
|
||||
yolov8n.pt
|
||||
.coverage
|
||||
|
||||
# Local prototype and evaluation outputs (scripts/chroma_scrub_prototype.py and
|
||||
# friends); the durable record of an oracle verdict lives under data/evaluations/
|
||||
out/
|
||||
|
||||
# Claude Code local settings
|
||||
.claude/settings.local.json
|
||||
.claude/scheduled_tasks.lock
|
||||
|
||||
@@ -31,6 +31,8 @@ Optional features and installation groups are documented in [`docs/installation.
|
||||
|
||||
`maintain.sh` runs dependency freshness and security checks, Ruff, Pyright scoped to `src/`, and the parallel test suite. Full-project Pyright is not the project gate because the ML dependency graph can exhaust Node memory.
|
||||
|
||||
This gate does NOT pass `--ignore-unfixed` to uv-secure, so a transitive CVE with no released fix stops it before Ruff, Pyright and the tests ever run. That is deliberate and it is not a blocker: run those three separately and report them, per the triage in [`docs/development.md`](docs/development.md). A red gate here is not the same signal as a red gate in a sibling repository that does ignore unfixed findings.
|
||||
|
||||
Command, gate, typing, and model-test invariants auto-load from [`.claude/rules/development.md`](.claude/rules/development.md). Environment recovery, CI behavior, and fixture policy live in [`docs/development.md`](docs/development.md).
|
||||
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Treat the release as complete only after PyPI, Homebrew, the Hugging Face Space, and the ComfyUI Registry are verified; conda is not published. Keep the source-distribution public allowlist.
|
||||
@@ -39,7 +41,7 @@ Before a release, read [`docs/release-and-distribution.md`](docs/release-and-dis
|
||||
|
||||
[`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem.
|
||||
|
||||
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md).
|
||||
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Pixel photo classification is [`docs/photo-classify.md`](docs/photo-classify.md); it is not `identify`. Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md). The pre-split chronological archive is [`docs/synthid-detector-removal-plan.md`](docs/synthid-detector-removal-plan.md).
|
||||
|
||||
## Data safety
|
||||
|
||||
@@ -51,4 +53,4 @@ Topic-specific rules live in `.claude/rules/*.md` and are auto-loaded when match
|
||||
|
||||
| File | Covers |
|
||||
|---|---|
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, and the detection-path measurement rule |
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, the docs-coverage seam, and the detection-path measurement rule |
|
||||
@@ -7,8 +7,8 @@ Remove AI provenance marks from images and video you generated yourself:
|
||||
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
|
||||
|
||||
Video support covers provenance identification, complete visible-plus-metadata
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo AI, and
|
||||
Kling AI mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Doubao, Dola,
|
||||
Hailuo AI, and Kling AI mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
removal.
|
||||
|
||||
> [raiw.cc](https://raiw.cc) runs this library as a hosted service, with the GPU
|
||||
@@ -32,6 +32,8 @@ removal.
|
||||
| Goal | Command | GPU |
|
||||
| --- | --- | --- |
|
||||
| Find provenance signals and watermarks | `identify` | No |
|
||||
| Classify a photograph from pixels (opt-in, not provenance) | `classify` | No |
|
||||
| Verify supported OpenAI SynthID from pixels with the official remote API | `verify-openai-synthid` | No |
|
||||
| Remove known visible AI marks | `visible` | No |
|
||||
| Erase a region you select | `erase` | No |
|
||||
| Strip AI metadata | `metadata` | No |
|
||||
@@ -54,6 +56,8 @@ its linked C2PA manifest; metadata stripping alone removes only the manifest.
|
||||
| Need | Install |
|
||||
| --- | --- |
|
||||
| Metadata inspection and stripping | `remove-ai-watermarks` |
|
||||
| Photograph AI-versus-camera classification | `remove-ai-watermarks[classify]` |
|
||||
| Official remote OpenAI SynthID verification | `remove-ai-watermarks[verify]` |
|
||||
| Visible detection and removal | `remove-ai-watermarks[visible]` |
|
||||
| Visible video processing | `remove-ai-watermarks[video]` |
|
||||
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
|
||||
@@ -80,6 +84,34 @@ Inspect an image:
|
||||
remove-ai-watermarks identify image.png
|
||||
```
|
||||
|
||||
To classify a photograph from pixels (AI versus camera, optional provider),
|
||||
install the extra and call `classify`. `identify` never starts it:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[classify]"
|
||||
remove-ai-watermarks classify image.png
|
||||
```
|
||||
|
||||
Guide: [photo pixel classification](docs/photo-classify.md).
|
||||
|
||||
Signed provenance is the supported route for SynthID and `identify` reads it.
|
||||
There is no local SynthID pixel detector in the package. Research on a
|
||||
periodic lattice expert is in `scripts/synthid_runtime/` and
|
||||
[synthid-detector-research.md](docs/synthid-detector-research.md).
|
||||
|
||||
For supported OpenAI images, the optional official verifier provides a pixel
|
||||
watermark verdict:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[verify]"
|
||||
remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload
|
||||
```
|
||||
|
||||
The command removes AI provenance metadata from a temporary copy, verifies that
|
||||
the decoded pixels are unchanged, uploads only that copy to OpenAI, and consumes
|
||||
only the independent SynthID response. It never runs implicitly from `identify`.
|
||||
An API key, endpoint access, and explicit upload acknowledgement are required.
|
||||
|
||||
For visible watermark removal, install the pixel dependencies:
|
||||
|
||||
```bash
|
||||
@@ -192,7 +224,7 @@ remains useful for unusually important files or after provider changes, but it
|
||||
is not a product result state.
|
||||
|
||||
For invisible watermark removal, install the `qwen-zimage` extra. **An NVIDIA GPU
|
||||
is required**: both profiles are CUDA-only, and there is no CPU or MPS fallback.
|
||||
is required**: all profiles are CUDA-only, and there is no CPU or MPS fallback.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
@@ -243,7 +275,10 @@ uv run remove-ai-watermarks visible demo_banana_before.png \
|
||||
`qwen-zimage` is the default profile: a Qwen-Image-2512 Lightning pass under Canny
|
||||
ControlNet, followed by SAM-masked Z-Image repair of any detected face. The
|
||||
alternative, `sdxl-zimage`, swaps the global stage for SDXL and keeps the same face
|
||||
stage. Both are CUDA only.
|
||||
stage. A third profile, `chroma-zimage`, uses the Apache-2.0 Chroma1 global pass
|
||||
with its own flat vendor floors; see `docs/chroma1-engine-research.md` for the
|
||||
calibration. `--pipeline auto` picks chroma-zimage for OpenAI and Microsoft
|
||||
provenance and qwen-zimage otherwise. All are CUDA only.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
@@ -320,7 +355,8 @@ remove-ai-watermarks batch ./images --mode all
|
||||
Visible mark support includes:
|
||||
|
||||
- Google Gemini and Nano Banana visible sparkle watermark;
|
||||
- Doubao, Jimeng, Qwen, Kling AI, Yuanbao, Baidu, LiblibAI, and RunningHub labels;
|
||||
- Doubao, the Jimeng wordmark and top-left `AI生成` pill, Qwen, Kling AI,
|
||||
Yuanbao, Baidu, LiblibAI, and RunningHub labels;
|
||||
- one calibrated Microsoft top-right white AI-badge variant;
|
||||
- one calibrated Samsung Galaxy AI label variant.
|
||||
|
||||
@@ -330,7 +366,9 @@ signals. Optional decoders add support for open DWT-DCT watermarks and Adobe
|
||||
TrustMark.
|
||||
|
||||
The exact support matrix, including important locale and detector limits, lives
|
||||
in [supported signals](docs/supported-signals.md).
|
||||
in [supported signals](docs/supported-signals.md). The
|
||||
[synthetic example gallery](data/fixtures/visible/README.md) carries one canonical
|
||||
sample for every registered image and video mark.
|
||||
|
||||
## How it works
|
||||
|
||||
@@ -363,6 +401,9 @@ import remove_ai_watermarks as raiw
|
||||
result, removed = raiw.remove_visible("watermarked.png", "clean.png")
|
||||
print(removed)
|
||||
|
||||
openai_synthid = raiw.verify_openai_synthid("image.png", acknowledge_upload=True)
|
||||
print(openai_synthid.status)
|
||||
|
||||
provenance = raiw.identify_video("input.mp4")
|
||||
report = raiw.inspect_video_metadata("input.mp4")
|
||||
complete = raiw.remove_video_all("input.mp4", "clean.mp4")
|
||||
@@ -405,7 +446,7 @@ invisible removal.
|
||||
detail.
|
||||
- Visible video removal recognizes the moving Sora 2 wordmark, the current Veo
|
||||
diamond plus legacy `Veo` text, the Seedance boxed `AI` label, and the fixed
|
||||
Dola, Hailuo AI, and Kling AI labels. It does not recognize the older Sora Turbo
|
||||
Doubao, Dola, Hailuo AI, and Kling AI labels. It does not recognize the older Sora Turbo
|
||||
corner swirl or unregistered layouts from those providers.
|
||||
The classical OpenCV backend can smear structured backgrounds; use MI-GAN or
|
||||
LaMa when recovery quality matters.
|
||||
@@ -413,7 +454,7 @@ invisible removal.
|
||||
The shipped profile is oracle-certified, but no public local decoder can
|
||||
certify an arbitrary output at runtime. Recheck unusually important outputs
|
||||
after provider changes.
|
||||
- Invisible-watermark removal requires CUDA. Both profiles refuse any other
|
||||
- Invisible-watermark removal requires CUDA. All profiles refuse any other
|
||||
device at construction rather than falling back to one that cannot run them.
|
||||
Visible removal, metadata stripping and `identify` still run anywhere.
|
||||
- Provider watermark systems can change. Validate important outputs with the
|
||||
@@ -445,8 +486,11 @@ reinterpret the pixel result returned `UNAVAILABLE`; that follow-up was not a
|
||||
verifier rerun and does not invalidate the built-in verdicts. A 2026-07-31
|
||||
full-clip check on a public eight-second Veo sample found `0.10` still detected
|
||||
and `0.15` not detected, so `0.15` is now the certified default. The
|
||||
reproducible hashes and verdicts live in
|
||||
`data/evaluations/video-synthid-oracle.csv`.
|
||||
reproducible hashes and verdicts for that full-clip check -- one carrier, two
|
||||
rows -- live in `data/evaluations/video-synthid-oracle.csv`. The earlier
|
||||
two-clip calibration is narrative only: its verdicts were not recorded in a
|
||||
tracked manifest, so treat the certified default as resting on the 2026-07-31
|
||||
rows.
|
||||
|
||||
## Documentation
|
||||
|
||||
|
||||
@@ -7,6 +7,8 @@ data/
|
||||
fixtures/
|
||||
provenance/ Real format and provenance fixtures used by tests
|
||||
(source records live in fixtures/README.md)
|
||||
visible/ Synthetic per-mark example gallery (one committed example per
|
||||
registered visible mark; see fixtures/visible/README.md)
|
||||
calibration/
|
||||
<vendor>/ Minimal controlled inputs needed to rebuild detector assets
|
||||
synthid/
|
||||
@@ -15,6 +17,8 @@ data/
|
||||
full-pipeline-quality.csv
|
||||
Reusable full-pipeline evaluation selection
|
||||
evaluations/
|
||||
engine-selection/
|
||||
Prompt-matched AI content matrix for deterministic auto-engine research
|
||||
fidelity/ Evaluation instructions and hand-verified ground truth
|
||||
video-synthid-oracle.csv
|
||||
Reproducible full-clip Gemini SynthID verdicts
|
||||
@@ -22,6 +26,12 @@ data/
|
||||
|
||||
## Storage rules
|
||||
|
||||
Large local-only research data (the 211 GB Spaces corpus and the frozen
|
||||
checkpoints, research corpora, and campaign reports) lives under gitignored
|
||||
paths documented in `data/research/README.md` (ignored via `.git/info/exclude`,
|
||||
never committed). Worktree `.local-eval/` directories are scratch copies; the
|
||||
durable store is `data/research/` in the main checkout.
|
||||
|
||||
1. Store each binary image once. Evaluation manifests and documentation point
|
||||
to its canonical location.
|
||||
2. Put executable test fixtures in `fixtures/`.
|
||||
@@ -32,6 +42,9 @@ data/
|
||||
commands, hashes, and oracle verdicts instead of committing another corpus
|
||||
copy. A small curated before-and-after example may live in `docs/images/`
|
||||
when it is part of the public documentation.
|
||||
`evaluations/engine-selection/` is the bounded exception for ORIGINAL study
|
||||
inputs: its manifest tracks a prompt-matched OpenAI/Meta content matrix, not
|
||||
generated model outputs or watermark-oracle evidence.
|
||||
6. Keep third-party fixture license notices in `licenses/`, outside directories
|
||||
that tests enumerate as media inputs.
|
||||
7. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# Engine-selection fixture set
|
||||
|
||||
This directory is the tracked, content-balanced input set for researching a
|
||||
deterministic `auto` engine choice. It is not a training corpus and it is not a
|
||||
watermark-oracle corpus.
|
||||
|
||||
## Content matrix
|
||||
|
||||
`content-manifest.csv` contains 19 prompt-matched pairs. Each pair uses the
|
||||
same prompt and generation index across OpenAI `gpt-image-1-mini` and Meta
|
||||
`muse-image-1.0`, covering the content classes that materially stress image
|
||||
regeneration:
|
||||
|
||||
- natural and urban scenes;
|
||||
- single-person portraits and people in context;
|
||||
- animals, food, products, interiors, architecture, macro detail, and action;
|
||||
- flat, isometric, watercolor, and surreal illustration styles.
|
||||
|
||||
The files are exact copies of generation outputs collected for the
|
||||
`docs/arxiv-paper-review` research branch. The manifest records the SHA-256 of
|
||||
the committed bytes. The old OpenAI collection manifest stored the API payload
|
||||
hash under `sha256`; `source_payload_sha256` preserves that value while
|
||||
`sha256` always means the actual file bytes here. Meta's two hashes are the
|
||||
same.
|
||||
|
||||
One generation per prompt and provider is enough for the first paired
|
||||
discovery pass. Any policy inferred from it must be challenged against unused
|
||||
generation indices from the local research archive before shipping. Generated
|
||||
outputs belong outside this directory.
|
||||
|
||||
`carrier-manifest.csv` adds the canonical, already tracked signal carriers to
|
||||
the study without copying their bytes. Together the two manifests cover
|
||||
ordinary content, text, single faces, face grids, and mixed face/text scenes.
|
||||
|
||||
## Signal carriers
|
||||
|
||||
The content matrix was re-encoded by its collection pipeline and has no local
|
||||
provenance signal. It must not be used to set or certify watermark-removal
|
||||
strengths. The tracked oracle/carrier inputs remain in their canonical homes:
|
||||
|
||||
- OpenAI and Google SynthID: `data/synthid/originals/` with
|
||||
`data/synthid/manifest.csv`;
|
||||
- Meta Content Seal: `data/contentseal/originals/` with
|
||||
`data/contentseal/manifest.csv`.
|
||||
|
||||
There is no tracked, publication-cleared Microsoft InvisMark carrier yet. The
|
||||
three Paint sources used by the earlier calibration are user uploads and remain
|
||||
outside git. A Microsoft row may be added only from a newly generated or
|
||||
otherwise publication-cleared source, with a pixel-identical metadata-stripped
|
||||
control that remains positive in Microsoft's detector.
|
||||
|
||||
## Integrity check
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run python scripts/verify_engine_selection_fixtures.py
|
||||
```
|
||||
|
||||
The check validates paths, byte hashes, dimensions, prompt pairing, and the
|
||||
declared provider/content matrix without loading any ML model.
|
||||
@@ -0,0 +1,13 @@
|
||||
provider,file,content_stratum,signal,verified_via,sha256,verified_text_geometry,verified_text_ground_truth
|
||||
openai,"../../synthid/originals/ChatGPT Image May 30, 2026, 10_31_08 AM.png",face_grid,OpenAI SynthID,openai-verify,28ff8732b037f98a4ef5bc277bbcdaa32e5eb9ccbd00b6c8c616e46ef68ae8a0,no,no
|
||||
openai,"../../synthid/originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png",multilingual_typography,OpenAI SynthID,c2pa-metadata,228e54413a3ef62b313176dcdba774a56a58c36b54227d1440bc26cbe0aefd82,yes,yes
|
||||
openai,"../../synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png",multilingual_typography,OpenAI SynthID,openai-verify,05b836ecfe40fd689177fda74384ae4fdcc446505bbc4281cd3cbb6523eb669e,yes,yes
|
||||
google,../../synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png,cjk_typography_scene,Google SynthID,gemini-app,4affd7f27767a445db6abf741355743ba8d95108ad922c9fff045feed8492236,yes,yes
|
||||
google,../../synthid/originals/Gemini_Generated_Image_akdbeiakdbeiakdb.png,cyrillic_typography_scene,Google SynthID,gemini-app,8c1a6fb03ef3d45a1f958fb3401e4264e409ff88c2a793061db7f29023454d0e,no,no
|
||||
google,../../synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png,crowd_faces,Google SynthID,gemini-app,45d79a683134fcba1b147b2aedb669783d474e1fb8a4df329729a0904fd1b46b,no,no
|
||||
google,../../synthid/originals/Gemini_Generated_Image_3mc4t93mc4t93mc4.png,faces_and_typography,Google SynthID,gemini-app,2c33e75a2db614ce74c83cc0a6ac6c3ac735aca83ab88c9c9345843b124f7856,no,no
|
||||
meta,../../contentseal/originals/gen_fox_forest.webp,photoreal_texture,Meta Content Seal,meta-oracle,6d7cecf322a5ac4b2547e2f32c767b3cb3e8a0e49f274cfd00aa31be6697eed4,no,no
|
||||
meta,../../contentseal/originals/gen_text_poster.webp,typography,Meta Content Seal,meta-oracle,873927662d8198e9c38b4ae71a2919ba5d417dfcd714649db1b0ae231e8cbf9b,no,no
|
||||
meta,../../contentseal/originals/gen_lighthouse_watercolor.webp,watercolor,Meta Content Seal,meta-oracle,69bee7c376608568361085602c5c7406900885849bd94c5279c3924cf793b27c,no,no
|
||||
meta,../../contentseal/originals/gen_night_city.webp,urban_night,Meta Content Seal,meta-oracle,345adf6b9f777b15ef7ce7bbdfb3b355a1d9e882740809e966e9f6caafd94d48,no,no
|
||||
meta,../../contentseal/originals/gen_studio_mug.webp,product_still_life,Meta Content Seal,meta-oracle,2b7bc7fd56fa2250c66ef74f8b3e49d9666fe147d894d5baf9461b5e36077a07,no,no
|
||||
|
@@ -0,0 +1,39 @@
|
||||
pair_id,provider,model,file,width,height,content_stratum,sha256,source_payload_sha256,prompt
|
||||
00,openai,gpt-image-1-mini,originals/openai/00.png,1024,1024,natural_landscape,698896455e0f1de57a227339227bf068143e690873b9c8c9672b31578de93ba0,1345f59fc7b51a1edf837745245bb90811feefd9ba7f9ba6f12dc89846aeba18,"a stunning photograph of a mountain lake at sunrise, highly detailed, professional photography"
|
||||
00,meta,muse-image-1.0,originals/meta/00.png,1600,1600,natural_landscape,83362c189b79cd0e700296b975e9ef8e7f2b2fe89813e407bc795299400ca5e0,83362c189b79cd0e700296b975e9ef8e7f2b2fe89813e407bc795299400ca5e0,"a stunning photograph of a mountain lake at sunrise, highly detailed, professional photography"
|
||||
01,openai,gpt-image-1-mini,originals/openai/01.png,1024,1024,street_scene,b885d8cb1490e4fbbe580b87750008780671390097d28e71ff18b50cf320ec85,772518e4686940d2c68df114298fb007a97a5c42548245092e989b6ca9bd5b90,"a vibrant street market in Bangkok with colorful fruits and spices, candid photo"
|
||||
01,meta,muse-image-1.0,originals/meta/01.png,1600,1600,street_scene,5ba08cf9ab07bf00f64f39fe93f1766a64f737817b00086783476566c5f3317f,5ba08cf9ab07bf00f64f39fe93f1766a64f737817b00086783476566c5f3317f,"a vibrant street market in Bangkok with colorful fruits and spices, candid photo"
|
||||
02,openai,gpt-image-1-mini,originals/openai/02.png,1024,1024,single_portrait,97618d1a5c39ddeba708df3a9448b27d1d26583ee68b490ba79f7f3e1eb2248d,6bace9d2adae4d630e1830449f0945a30d7ddb3635e3005bc7aad80206a57176,"a close-up portrait of a woman with freckles and green eyes, studio lighting, 85mm lens"
|
||||
02,meta,muse-image-1.0,originals/meta/02.png,1600,1600,single_portrait,4daf07fc8bdfe007b96a70b00ec0cfde87f180eb8c2d5501723e218ed29e7589,4daf07fc8bdfe007b96a70b00ec0cfde87f180eb8c2d5501723e218ed29e7589,"a close-up portrait of a woman with freckles and green eyes, studio lighting, 85mm lens"
|
||||
03,openai,gpt-image-1-mini,originals/openai/03.png,1024,1024,animal_action,eb6920aab03bbe6935a01756c8f2d524d383e307369cdfaad3fe44926fbd9e48,2012abb15be639ebb3c596d2210f6d4c50cce0e11dd6cfc88aeb29de1c82de18,"a golden retriever running through a field of sunflowers, action photography"
|
||||
03,meta,muse-image-1.0,originals/meta/03.png,1600,1600,animal_action,93dce3a5049f773adb882ef22243a84e4822b9c43ad30774741c40f3844ef405,93dce3a5049f773adb882ef22243a84e4822b9c43ad30774741c40f3844ef405,"a golden retriever running through a field of sunflowers, action photography"
|
||||
04,openai,gpt-image-1-mini,originals/openai/04.png,1024,1024,reflective_architecture,275d68de881041de989261ced7570dd460028e0406346642d880d0d1664bb9ae,1808d95c69080aafced4234e4c3ad7714121bad00f3f5dc84a744288d9757245,"a modern architecture building with glass facade reflecting sunset clouds"
|
||||
04,meta,muse-image-1.0,originals/meta/04.png,1600,1600,reflective_architecture,209c71f0b737a9ef60b4d620c66f518b34dbc42af4532b2b8e0b3086183c90ec,209c71f0b737a9ef60b4d620c66f518b34dbc42af4532b2b8e0b3086183c90ec,"a modern architecture building with glass facade reflecting sunset clouds"
|
||||
05,openai,gpt-image-1-mini,originals/openai/05.png,1024,1024,food_closeup,316a743ccf29f025904d1bcd047a6996f987715e4f93e8a0d9cf85b624349b07,a8bac088c350ef849bbf78a85a1e44aa1665d1054a0cdfae29461c1cdca56e4a,"a bowl of ramen with steam rising, moody food photography, shallow depth of field"
|
||||
05,meta,muse-image-1.0,originals/meta/05.png,1600,1600,food_closeup,4a50ca3e33ddacd2897b926cdc54615a389416fcd2edf4d8803ad882b85b33af,4a50ca3e33ddacd2897b926cdc54615a389416fcd2edf4d8803ad882b85b33af,"a bowl of ramen with steam rising, moody food photography, shallow depth of field"
|
||||
06,openai,gpt-image-1-mini,originals/openai/06.png,1024,1024,sports_action,390d87d1f59eb4fa38590c0c751b2848383c77ffcf549571a0bf00db0f783763,6cc24eeb0fce95a86741b9ac0c055002793a586ba37f0cf4d64201a5c5e73513,"a surfer riding a barrel wave at golden hour, backlit spray"
|
||||
06,meta,muse-image-1.0,originals/meta/06.png,1600,1600,sports_action,1d9fcbf0f471f733ae96ca94dff892afd9f5852767000e379d3486dc94e30b78,1d9fcbf0f471f733ae96ca94dff892afd9f5852767000e379d3486dc94e30b78,"a surfer riding a barrel wave at golden hour, backlit spray"
|
||||
07,openai,gpt-image-1-mini,originals/openai/07.png,1024,1024,interior_low_light,4c77c2814216bdcde3417d2d228c5b6f275e33777f13cbea7911241bc9bc376b,9e858b95a4f842c917cbf47afeb4b7e0e24e04c4df0b19da4afe00b7316a13a9,"a cozy reading nook with warm lamp light and rain on the window"
|
||||
07,meta,muse-image-1.0,originals/meta/07.png,1600,1600,interior_low_light,ecec76c086a7b33beec1b3cb787f62cad68c09cb362a8888a8af734e9d4f3425,ecec76c086a7b33beec1b3cb787f62cad68c09cb362a8888a8af734e9d4f3425,"a cozy reading nook with warm lamp light and rain on the window"
|
||||
08,openai,gpt-image-1-mini,originals/openai/08.png,1024,1024,isometric_illustration,d688f6da74d4a7b2bd29ef59ba64864d0915134433febd8c90eb5ce1dc80781f,37a801f66b7c1bf63e613a7a5717dbc00397120a7a32ec45414661e03137a180,"an isometric illustration of a cozy coffee shop interior, warm colors, detailed"
|
||||
08,meta,muse-image-1.0,originals/meta/08.png,1600,1600,isometric_illustration,d4683d16a55c5f358f5b3b876b89fbc810ae1e3332e74399395ae20c9f646ef5,d4683d16a55c5f358f5b3b876b89fbc810ae1e3332e74399395ae20c9f646ef5,"an isometric illustration of a cozy coffee shop interior, warm colors, detailed"
|
||||
09,openai,gpt-image-1-mini,originals/openai/09.png,1024,1024,watercolor,d0beefb80ee00a0937a851932a2ed6dab2c670f10104ffe94c5ff22869a9595d,b52401235efc16641d3cb8f9c7c6d20a75c36751ada9409b65ceefa0e57901da,"a watercolor painting of Venice canals at sunset, loose brushstrokes"
|
||||
09,meta,muse-image-1.0,originals/meta/09.png,1600,1600,watercolor,c76ffbca59fbcb2dba9b66c94598f0c0062b0ac4679e348cdf1e18a06d3eb64d,c76ffbca59fbcb2dba9b66c94598f0c0062b0ac4679e348cdf1e18a06d3eb64d,"a watercolor painting of Venice canals at sunset, loose brushstrokes"
|
||||
10,openai,gpt-image-1-mini,originals/openai/10.png,1024,1024,macro_detail,59d9c6dce5ce4c2b77427144ccbafc8a4b7ab87564673677119859c31598b0d4,3740f9de304f050fc7306447c6668dcaa8156a888709f759f05715e7987a321c,"a macro photograph of a dewdrop on a spider web at dawn, bokeh background"
|
||||
10,meta,muse-image-1.0,originals/meta/10.png,1600,1600,macro_detail,c86fd9c3a59aa0bec709c5bce7c2ef1be4ecb1d780f3a3f35460b73cd97d1a12,c86fd9c3a59aa0bec709c5bce7c2ef1be4ecb1d780f3a3f35460b73cd97d1a12,"a macro photograph of a dewdrop on a spider web at dawn, bokeh background"
|
||||
11,openai,gpt-image-1-mini,originals/openai/11.png,1024,1024,urban_night,e4f6f806ed8559973374eae1e4d379807e45046749b7d491edbbc8ad9647728e,06bfc68289ece1b85b169df4cedd5e1cf12d0efa30631bdecd0e3edce4d8c82e,"a cinematic wide shot of Tokyo at night with neon reflections on wet streets"
|
||||
11,meta,muse-image-1.0,originals/meta/11.png,1600,1600,urban_night,7877cf0c13b07fbc6651178d838db4ab253369896292af1cf3f85a957a559db8,7877cf0c13b07fbc6651178d838db4ab253369896292af1cf3f85a957a559db8,"a cinematic wide shot of Tokyo at night with neon reflections on wet streets"
|
||||
12,openai,gpt-image-1-mini,originals/openai/12.png,1024,1024,flat_illustration,6bf230aa0b6f40d7ff0d23b22b8aac2e9b66288eb85c02eebdf43b71a1597f5b,65567d5cecb34241ac755ac4db5eb5a7b271d9a00fa27d826c961d2a2ef6ac3d,"a flat-design poster of Santorini with bold shapes, screen print style"
|
||||
12,meta,muse-image-1.0,originals/meta/12.png,1600,1600,flat_illustration,1b71bd94aa18d737c8a1c319a33dd0f0e9e15c2e93210c9c3aa98242b9a67955,1b71bd94aa18d737c8a1c319a33dd0f0e9e15c2e93210c9c3aa98242b9a67955,"a flat-design poster of Santorini with bold shapes, screen print style"
|
||||
13,openai,gpt-image-1-mini,originals/openai/13.png,1024,1024,storm_landscape,d07b05eef633fcb5b6b54268cc2cdd787dc19f8050198872e462f69e383a054d,a692a8a0daebad375cd57d7c671be6e16ad3ef700f0b9214daacbe80b6148c90,"a dramatic storm cloud over a lonely lighthouse, moody, high contrast photography"
|
||||
13,meta,muse-image-1.0,originals/meta/13.png,1600,1600,storm_landscape,e669128fe2d3b59dc83ae0b216c5378fe5930e77fa3a692219da238dbf55d5f6,e669128fe2d3b59dc83ae0b216c5378fe5930e77fa3a692219da238dbf55d5f6,"a dramatic storm cloud over a lonely lighthouse, moody, high contrast photography"
|
||||
14,openai,gpt-image-1-mini,originals/openai/14.png,1024,1024,monochrome_portrait,9fb1153f4d0041932b35bcd1408d774b642c68f81d7bb403d22f50f8c891ede7,826acf13df78378499e57c7fe1a8309947ab26810f46482be3a64d3328de1ff6,"a black and white portrait of an elderly craftsman in his workshop"
|
||||
14,meta,muse-image-1.0,originals/meta/14.png,1600,1600,monochrome_portrait,8339475f0585491ea08992d29890f3c759f24db2184212c6bf4f7efc0932e177,8339475f0585491ea08992d29890f3c759f24db2184212c6bf4f7efc0932e177,"a black and white portrait of an elderly craftsman in his workshop"
|
||||
15,openai,gpt-image-1-mini,originals/openai/15.png,1024,1024,surreal_composite,8bf02f9028fd049ebfafd3410d2eb9d618819be1c14b4e05d53a490e22ba109c,e71de5a9a2f6579857d2e24b8e7a1516031f19ee824906cece158101d13fe001,"a surreal double exposure of a forest and a human profile, conceptual art"
|
||||
15,meta,muse-image-1.0,originals/meta/15.png,1600,1600,surreal_composite,20628541cc924e623984604337807c7afd453aa8a7652f087add742a13828f6a,20628541cc924e623984604337807c7afd453aa8a7652f087add742a13828f6a,"a surreal double exposure of a forest and a human profile, conceptual art"
|
||||
16,openai,gpt-image-1-mini,originals/openai/16.png,1024,1024,brutalist_architecture,e91ca5128ce8644510c5c4def6bba085e45e1716105b3b6a2761cc18d42018b3,0aa8a1819f160ce26dfc9e59b996f969698fdf4eba3402fe999af598fcad80d4,"a low-angle shot of a brutalist building against dramatic clouds"
|
||||
16,meta,muse-image-1.0,originals/meta/16.png,1600,1600,brutalist_architecture,d22702d26343b6575e89751c95d865696c6085bed053f6008f989434a96bac02,d22702d26343b6575e89751c95d865696c6085bed053f6008f989434a96bac02,"a low-angle shot of a brutalist building against dramatic clouds"
|
||||
17,openai,gpt-image-1-mini,originals/openai/17.png,1024,1024,product_still_life,4f41f4a46b80c0ded61a9a3ce1273412b1728476a0734519d1af34a1797013b0,1f60c25c59138c14bacb0dd485475037e7d6222bb45daa66ed85f9dd91c83e4e,"a minimalist product photo of a ceramic mug on a marble surface, clean commercial look"
|
||||
17,meta,muse-image-1.0,originals/meta/17.png,1600,1600,product_still_life,67bde86edfd3db08b88c0dbc5572958408875e188dac6fa8ed64f821b7e7e994,67bde86edfd3db08b88c0dbc5572958408875e188dac6fa8ed64f821b7e7e994,"a minimalist product photo of a ceramic mug on a marble surface, clean commercial look"
|
||||
18,openai,gpt-image-1-mini,originals/openai/18.png,1024,1024,person_in_context,9b049d3112a654a87022d591b93a01087693c0664be5af560d3c0ec490291963,ba873bc7989720245821f5298f12e36bd7b6051909694b72569645c29cdd6bd0,"a lifestyle photo of a person working on a laptop in a bright cafe"
|
||||
18,meta,muse-image-1.0,originals/meta/18.png,1600,1600,person_in_context,cc4b2bcfc4e80d7dfe6646beb12529168137aef04d2b6765c6382b830ce19ed7,cc4b2bcfc4e80d7dfe6646beb12529168137aef04d2b6765c6382b830ce19ed7,"a lifestyle photo of a person working on a laptop in a bright cafe"
|
||||
|
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 4.0 MiB |
|
After Width: | Height: | Size: 2.5 MiB |
|
After Width: | Height: | Size: 3.2 MiB |
|
After Width: | Height: | Size: 2.9 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 4.0 MiB |
|
After Width: | Height: | Size: 3.0 MiB |
|
After Width: | Height: | Size: 3.2 MiB |
|
After Width: | Height: | Size: 3.6 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.1 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 3.4 MiB |
|
After Width: | Height: | Size: 3.0 MiB |
|
After Width: | Height: | Size: 854 KiB |
|
After Width: | Height: | Size: 2.8 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
|
After Width: | Height: | Size: 1.5 MiB |
|
After Width: | Height: | Size: 1.5 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 1.6 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 1.3 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 1.6 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 1.1 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
@@ -13,3 +13,23 @@ example from [`adobe/trustmark`](https://github.com/adobe/trustmark) at commit
|
||||
- SHA-256: `e58c5825ed7e5d9fb04710ea541b61bd55879cad65554c0d46260aa24b3d0755`
|
||||
- Expected signal: TrustMark Variant P, schema 1
|
||||
- License: MIT, reproduced in `../licenses/adobe-trustmark-MIT.txt`
|
||||
|
||||
|
||||
## Ideogram C2PA attribution
|
||||
|
||||
`ideogram-c2pa.png` is a SYNTHETIC signed manifest, not a vendor download: a
|
||||
64x64 flat PNG carrying a C2PA claim (`trainedAlgorithmicMedia`,
|
||||
`claim_generator "Ideogram/3.0"`) signed by a throwaway test CA whose leaf
|
||||
certificate subject is `O=Ideogram, Inc, CN=Ideogram, Inc`. It exercises the
|
||||
issuer-to-platform ATTRIBUTION path end to end through the real reader; the
|
||||
signer is deliberately untrusted (no trust anchors ship), which is also the
|
||||
fixture's second assertion: attribution and signer trust are separate
|
||||
dimensions.
|
||||
|
||||
- Expected signal: C2PA, platform `Ideogram`, `ai_from_metadata` True
|
||||
- Expected signer: `untrusted` (test CA; attribution must not imply trust)
|
||||
- Recipe (no keys are committed): create a CA and an ES256 leaf with
|
||||
`digitalSignature` key usage and subject `CN=Ideogram, Inc`, then
|
||||
`c2pa.Builder.from_json(...).sign_file(png, out, Signer.from_info(
|
||||
C2paSignerInfo(b"es256", leaf_pem + ca_pem, pkcs8_key, None)))`.
|
||||
- No user upload and no Ideogram artifact contributes pixels or bytes.
|
||||
|
After Width: | Height: | Size: 43 KiB |
@@ -0,0 +1,41 @@
|
||||
# Visible-mark example gallery
|
||||
|
||||
One committed example per registered visible mark, so the repository carries a
|
||||
working sample of everything it supports. `tests/test_visible_examples.py` holds
|
||||
both sides to it: a mark registered without an example fails the suite, and so
|
||||
does an engine that stops detecting its own example.
|
||||
|
||||
## What these files are
|
||||
|
||||
Every example is SYNTHETIC: `scripts/render_visible_examples.py` composites the
|
||||
mark's committed silhouette (the same font-rendered asset the detector matches)
|
||||
onto a deterministic generated base photo at the engine's measured geometry.
|
||||
The silhouettes are repository-owned font renders generated by
|
||||
`scripts/render_vendor_silhouettes.py`; the gallery does not copy vendor raster
|
||||
assets.
|
||||
|
||||
The examples demonstrate DETECTION geometry and house style, not vendor raster
|
||||
fidelity; real-world variants (fonts, opacities, sizes) are covered by the
|
||||
engines' recorded calibration evidence.
|
||||
|
||||
## Regeneration
|
||||
|
||||
uv run python scripts/render_visible_examples.py
|
||||
|
||||
The generator self-verifies: it fails (exit 1) if any registered mark does not
|
||||
detect on its own example, so regeneration is the fix point for drift.
|
||||
|
||||
## Layout
|
||||
|
||||
<mark-key>/example.png 1536x2048..2048x2048 PNG, one per image mark
|
||||
<mark-key>/example.mp4 960x540 90-frame clip, one per video mark
|
||||
(kling carries both: it is registered in both registries)
|
||||
|
||||
Special cases: `gemini` composites the sparkle alpha map at the provider's
|
||||
configured position; `jimeng_pill` is the capture-less pill at the measured
|
||||
3:4 portrait geometry; `microsoft` is the opaque white pill with dark text
|
||||
holes (the discriminator its detector keys on). Video examples composite the
|
||||
detector's own synthetic template on every frame; where two marks share a
|
||||
shape family the example carries the discriminative variant (`veo` the legacy
|
||||
text form, `kling` the logo-plus-wordmark pair flush to the edge), because the
|
||||
temporal selection resolves cross-template ties by table order.
|
||||
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 2.6 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
@@ -54,7 +54,10 @@ versions, or runtime settings.
|
||||
Ground-truth quality, strongest first:
|
||||
|
||||
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
|
||||
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
|
||||
- `openai-verify` — checked through the OpenAI web verifier. This is the
|
||||
historical manifest label for existing rows. OpenAI now also documents a
|
||||
Content Provenance API with a separate SynthID result; add a distinct manifest
|
||||
value before recording API-derived labels rather than silently reusing this one.
|
||||
- `synthid-portal` — checked via Google's SynthID Detector portal.
|
||||
- `c2pa-metadata` — supported provenance evidence (Google AI C2PA, or OpenAI
|
||||
C2PA with an explicit `c2pa.watermarked.*` action). Weaker than a provider
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
# Private SynthID research manifest
|
||||
|
||||
The research manifest is a local CSV for detector and pixel-only removal
|
||||
experiments. It is intentionally separate from `manifest.csv`, which describes
|
||||
the small public regression corpus.
|
||||
|
||||
Store the CSV and its media below `.local-eval/synthid/`. Keep one manifest per
|
||||
detector target, for example `google-manifest.csv` and `openai-manifest.csv`.
|
||||
Separate manifests may reference the same external negative bytes without
|
||||
duplicating an artifact inside either provider's split graph. Do not commit
|
||||
private media, corpus sizes, oracle sessions, or provider-access details.
|
||||
|
||||
Before assigning labels, build a label-free inventory. It records byte and
|
||||
decoded-pixel hashes, exact duplicates, geometry, and format, but deliberately
|
||||
has no provider, oracle, outcome, or split columns:
|
||||
|
||||
```bash
|
||||
uv run python scripts/synthid_research_inventory.py \
|
||||
--root .local-eval/synthid negatives google openai \
|
||||
--inventory-out .local-eval/synthid/inventory.csv
|
||||
```
|
||||
|
||||
An inventory row becomes a manifest row only after its source, detector target,
|
||||
label evidence, group lineage, and split are established independently. A
|
||||
directory name is not evidence and must not be promoted mechanically.
|
||||
|
||||
Audit each local manifest with:
|
||||
|
||||
```bash
|
||||
uv run python scripts/synthid_research_manifest.py \
|
||||
.local-eval/synthid/google-manifest.csv --verify-files
|
||||
```
|
||||
|
||||
Once a provider manifest has ordinary positives and negatives in train,
|
||||
validation, and locked-test splits, run the D1 confound challenge:
|
||||
|
||||
```bash
|
||||
uv run --extra pixels python scripts/synthid_confound_probe.py \
|
||||
.local-eval/synthid/google-manifest.csv \
|
||||
--target-provider google \
|
||||
--report-out .local-eval/synthid/google-d1-confounds.json
|
||||
```
|
||||
|
||||
D1 excludes candidate, sham, and source-control rows. Its `container`,
|
||||
`thumbnail`, and `canonical` baselines intentionally measure how well export
|
||||
fields, geometry, and coarse generator/content style can imitate detection. A
|
||||
report is evidence-ready only when the locked test includes a same-provider hard
|
||||
negative and the temporal split contains both labels. Evidence readiness does
|
||||
not mean the D1 gate passed; a candidate signal still has to beat the frozen
|
||||
canonical baseline on those controls.
|
||||
|
||||
## Columns
|
||||
|
||||
| Column | Meaning |
|
||||
| --- | --- |
|
||||
| `artifact_sha256` | SHA-256 of the exact file submitted or measured. |
|
||||
| `pixel_sha256` | SHA-256 of decoded RGB bytes, used to catch lossless duplicates. |
|
||||
| `artifact_path` | Safe path relative to the manifest. |
|
||||
| `parent_sha256` | Exact parent artifact for a derivative, empty for an original. |
|
||||
| `group_id` | Leakage boundary shared by an original and all semantic or transformed siblings. |
|
||||
| `target_provider` | Detector target, `openai` or `google`. |
|
||||
| `source_provider` | Actual source family: `openai`, `google`, `camera`, `other_ai`, `synthetic`, or `editor`. |
|
||||
| `surface` | Product or export surface. |
|
||||
| `model_epoch` | Model family plus a dated epoch when the exact version is unavailable. |
|
||||
| `generation_session` | Groups outputs created in one provider session. |
|
||||
| `content_stratum` | Predeclared content class. |
|
||||
| `width`, `height`, `format` | Decoded geometry and file format. |
|
||||
| `transform` | `original` or the reproducible transform applied to `parent_sha256`. |
|
||||
| `split` | `discovery`, `train`, `validation`, `test`, or `temporal`. |
|
||||
| `c2pa_outcome` | C2PA result, recorded independently from the pixel signal. |
|
||||
| `synthid_outcome` | `detected`, `not_detected`, `indeterminate`, `refused`, or `not_checked`. |
|
||||
| `verified_via` | Matching provider oracle, external source evidence, or `none`. |
|
||||
| `evidence_reference` | Stable URL or local evidence-record reference required for `source-evidence`; never infer it from a directory name. |
|
||||
| `oracle_session` | Groups allowed checks made in one session. |
|
||||
| `oracle_role` | `ordinary`, `source_control`, `candidate`, or `sham`. |
|
||||
| `captured_at`, `oracle_checked_at` | Timezone-aware ISO-8601 timestamps. |
|
||||
| `notes` | Verbatim context that does not fit a structured column. |
|
||||
|
||||
`source-evidence` may establish an ordinary external negative, but never a
|
||||
positive or a same-provider negative. Provider positives and same-provider hard
|
||||
negatives require the matching provider verifier. An `indeterminate`, `refused`,
|
||||
or unchecked row may remain in `discovery`; it cannot enter a train, validation,
|
||||
test, or temporal split. Every `source-evidence` row must retain the source URL
|
||||
or evidence-record reference that establishes the claim.
|
||||
|
||||
The auditor also rejects duplicate artifact hashes, identical decoded pixels in
|
||||
different groups, derivatives without parents, cross-provider parentage,
|
||||
lineage cycles, and any group crossing split boundaries. A `not_detected`
|
||||
candidate or sham is valid only when the same provider, group, and oracle
|
||||
session contains a detected `source_control`. A session that misses its control
|
||||
cannot establish removal, even when its candidate response is negative.
|
||||
@@ -0,0 +1,550 @@
|
||||
# Chroma1 engine research (2026-08-29/30)
|
||||
|
||||
> Research archive. This page records experiments and decisions from the dates
|
||||
> above. Current package behavior is defined by the
|
||||
> [supported signals](supported-signals.md), [known limitations](known-limitations.md),
|
||||
> and [module internals](module-internals.md). Nothing here is shipped.
|
||||
|
||||
Cited experiment behind issue #88 ("Please integrate Flux 2"): FLUX.2 has no
|
||||
native strength img2img anywhere (BFL reference code or diffusers; its image
|
||||
inputs are reference conditioning and outputs start from pure noise), so the
|
||||
requestable scrub lane in the FLUX family is **Chroma1** -- the FLUX.1
|
||||
architecture re-trained on open data, Apache-2.0 on weights, with a native
|
||||
`ChromaImg2ImgPipeline` strength parameter. This page records what was measured
|
||||
before any integration decision.
|
||||
|
||||
## Setup
|
||||
|
||||
All Chroma1 runs: `lodestones/Chroma1-HD` bf16 on Modal H100, seed 0, the
|
||||
neutral scrub prompt (`high quality, sharp, detailed, faithful to the original`
|
||||
/ `blurry, lowres, distorted text, garbled text, artifacts`), guidance 5.0, and
|
||||
**step compensation**: Diffusers truncates the step COUNT (`int(steps *
|
||||
strength)`), so the requested count is scaled to always spend 4 effective
|
||||
denoising steps, the same semantics as `sdxl_zimage_pipeline.requested_steps`.
|
||||
Without it, strength 0.05 runs a single step and the floor reads artificially
|
||||
high (the first, uncompensated sweep put the OpenAI boundary at (0.05, 0.10];
|
||||
compensated it is (0.05, 0.06]).
|
||||
|
||||
Two Flux-family traps found while wiring the prototype:
|
||||
|
||||
- without explicit `width`/`height`, the img2img pipeline silently generates
|
||||
1024x1024 regardless of input aspect;
|
||||
- dimensions floor to the /16 latent-patch grid, so the result must be resized
|
||||
back to the input size for pixel-exact comparison.
|
||||
|
||||
Scripts: `scripts/chroma_scrub_prototype.py` (first sweep),
|
||||
`scripts/engine_quality_price_probe.py` (head-to-head with the production
|
||||
qwen-zimage path), `scripts/engine_cohort_calibration.py` (the cohort ladders
|
||||
below), `scripts/chroma_meta_high_rungs.py` (the Meta rungs above 0.08),
|
||||
`scripts/chroma_muse_expansion.py` (harvest-2 Meta API fixtures),
|
||||
`scripts/chroma_openai_expansion.py` (extra OpenAI SynthID carriers),
|
||||
`scripts/chroma_seed_probes.py` (seed-1/2 at measured first-cleans).
|
||||
Generated outputs stay outside the repository (`out/`, gitignored); the durable
|
||||
verdicts are the tables on this page.
|
||||
|
||||
## Measured floors
|
||||
|
||||
Oracle per cohort: OpenAI via openai.com/verify (SynthID read separately from
|
||||
C2PA); Meta via the anonymous meta.ai/identification oracle. Each fixture's
|
||||
rungs were checked ascending and stopped at the first clean verdict.
|
||||
|
||||
| cohort | fixture | detected at | clean from |
|
||||
|---|---|---|---|
|
||||
| OpenAI | typography card 1122x1402 | 0.05 | 0.06 |
|
||||
| OpenAI | 9-face grid 1448x1086 | 0.06 | 0.075 |
|
||||
| OpenAI | full-pipeline quality fixture 1122x1402 | 0.05 | 0.06 |
|
||||
| Meta | gen_lighthouse_watercolor 1600x1600 | 0.08 | 0.10 |
|
||||
| Meta | gen_fox_forest 1600x1600 | 0.06 | 0.08 |
|
||||
| Meta | gen_night_city 1920x1280 | 0.045 | 0.06 |
|
||||
| Meta | gen_studio_mug 1600x1600 | 0.03 | 0.045 |
|
||||
| Meta | gen_text_poster 1600x1600 | -- | <=0.03 |
|
||||
|
||||
Derived operating points by the shipped rule (worst first-clean boundary plus
|
||||
one observed cross-source spread):
|
||||
|
||||
- **OpenAI: 0.09** (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and
|
||||
sdxl-zimage 0.15. The typography boundary (0.05, 0.06] is seed-stable across
|
||||
seeds 0, 1, 2.
|
||||
- **Meta: 0.17** (0.10 + (0.10 - 0.03)), oracle-verified clean on the worst
|
||||
fixture at 0.17. Against qwen-zimage 0.1. Chroma needs ~1.7x qwen's strength
|
||||
on Content Seal.
|
||||
- **Google: 0.40, measured 2026-08-30 across all four fixtures.** A
|
||||
metadata-stripped original verified as pixel-DETECTED through the @synthid
|
||||
Verify AI agent (the C2PA-carrying original answers about C2PA instead,
|
||||
which is not a pixel verdict).
|
||||
|
||||
| fixture | detected at | clean from |
|
||||
|---|---|---|
|
||||
| Gemini_Generated_Image_633uuy | 0.20 | 0.25 |
|
||||
| Gemini_Generated_Image_akdbei | 0.20 | 0.25 |
|
||||
| Gemini_Generated_Image_y48j3c | 0.08 | 0.12 |
|
||||
| Gemini_Generated_Image_3mc4t9 | 0.08 | 0.12 |
|
||||
|
||||
Derived by the shipped rule: worst first-clean 0.25 + spread (0.25 - 0.12
|
||||
= 0.13) = 0.38, rounded up to the measured rung **0.40**, then
|
||||
oracle-verified clean on the worst fixture. Against qwen-zimage 0.27:
|
||||
Chroma needs ~1.5x the strength on Google SynthID. At 0.15-0.20 the tool
|
||||
still reads positive while its wording degrades from "most or all" to
|
||||
"part of", so the boundary band is real signal decay, not noise.
|
||||
|
||||
Oracle mechanics learned while bracketing: the quota is per ACCOUNT and
|
||||
additionally dedupes SIMILAR images ("quota for checking images similar to
|
||||
this one"), roughly three checks per account window; switching accounts
|
||||
inside one browser works. A fresh Playwright context with the same Google
|
||||
session cookies (exported browser-wide via CDP `Storage.getCookies`, never
|
||||
through the transcript) carries working sign-in and its own check budget.
|
||||
The agent's own instructions leak into the answer on some renders and
|
||||
include an abstention rule for text-dominated images - always read the TOOL
|
||||
outcome, never the prose.
|
||||
- **Microsoft: 0.125**, measured 2026-08-30 on the SAME three valid Paint
|
||||
carriers behind the qwen floor (staged under
|
||||
`out/cohort-calibration/microsoft/` with pixel-identical metadata-stripped
|
||||
controls; they are real raiw-corpus user uploads and are never committed).
|
||||
Controls re-verified `Microsoft AI detected` on the day of the run. Against
|
||||
qwen-zimage 0.15: Chroma needs LESS strength on InvisMark.
|
||||
|
||||
| source (Paint) | detected at | clean from | qwen first-clean |
|
||||
|---|---|---|---|
|
||||
| paint-1 (47d87e99, 1536x1024) | 0.06 | 0.08 | 0.055 |
|
||||
| paint-2 (a579fb70, 1536x1024) | -- | <=0.04 | 0.04125 |
|
||||
| paint-3 (883191b5, worst qwen source) | 0.06 | 0.08 | 0.095 |
|
||||
|
||||
Derived by the shipped rule: 0.08 + (0.08 - 0.04) = 0.12, rounded to the
|
||||
measured rung **0.125**, which was then oracle-verified clean on both worst
|
||||
sources. Oracle: the public `ai.azure.com/nextgen/validate` page; its
|
||||
`Inconclusive` verdict is the site's watermark-negative, weaker than an
|
||||
API-level `Watermark: false`, the same caveat the qwen cohort carries.
|
||||
|
||||
## Fidelity at the floors (scripts/fidelity_metrics.py)
|
||||
|
||||
qwen-zimage rows ran through the package's own `QwenZImagePipeline` on the same
|
||||
H100 (production path, both stacks resident, prompt cache warm).
|
||||
|
||||
OpenAI typography (chroma 0.06 vs qwen 0.07675):
|
||||
|
||||
| | CER | img LPIPS | SSIM | PSNR |
|
||||
|---|---|---|---|---|
|
||||
| qwen-zimage | 0.241 | 0.083 | 0.834 | 27.7 |
|
||||
| chroma1 | **0.138** | **0.049** | **0.881** | **31.2** |
|
||||
|
||||
OpenAI full-pipeline fixture (chroma 0.09 vs qwen 0.07675): chroma wins every
|
||||
metric (CER 0.155 vs 0.259, LPIPS 0.056 vs 0.094, PSNR 29.6 vs 26.1).
|
||||
|
||||
OpenAI 9-face grid (chroma 0.075 global-only vs qwen 0.07675 with its Z-Image
|
||||
face stage):
|
||||
|
||||
| | ID cos | face LPIPS | img LPIPS | SSIM | PSNR |
|
||||
|---|---|---|---|---|---|
|
||||
| qwen-zimage | **0.857** | **0.051** | 0.065 | 0.865 | 31.1 |
|
||||
| chroma1 (no face stage) | 0.788 | 0.067 | **0.060** | **0.890** | **32.4** |
|
||||
|
||||
The face stage regenerates from the ORIGINAL crops and is engine-independent,
|
||||
so an integrated `chroma-zimage` would composite the same faces over the
|
||||
chroma global; whether that closes the identity gap is an integration-time
|
||||
measurement, not a given.
|
||||
|
||||
Meta at the respective floors (chroma 0.17 vs qwen 0.1): qwen wins perceptual
|
||||
fidelity on textured content (LPIPS: fox 0.163 vs 0.211, lighthouse 0.173 vs
|
||||
0.248, night city 0.187 vs 0.277), chroma wins the text poster decisively
|
||||
(LPIPS 0.008 vs 0.012, CER 0.000 vs 0.021) and the studio mug is a split. The
|
||||
extra 0.07 of strength costs real fidelity on photoreal content.
|
||||
|
||||
Google at the respective floors (chroma 0.40 vs qwen 0.27, qwen row includes
|
||||
its Z-Image face stage):
|
||||
|
||||
| fixture | qwen LPIPS/SSIM/ID cos | chroma LPIPS/SSIM/ID cos |
|
||||
|---|---|---|
|
||||
| 633uuy | **0.405 / 0.577** / - | 0.499 / 0.576 / - (CER 1.000 vs 0.333) |
|
||||
| akdbei | **0.440 / 0.552** / - | 0.540 / 0.557 / - (CER 0.796 vs 0.367) |
|
||||
| y48j3c (18 faces) | **0.436 / 0.608 / 0.801** | 0.560 / 0.548 / 0.279 |
|
||||
| 3mc4t9 (5 faces) | **0.422 / 0.529 / 0.921** | 0.562 / 0.461 / 0.143 |
|
||||
|
||||
qwen wins every axis that matters on every fixture; at 0.40 the Chroma global
|
||||
regeneration itself collapses face identity (0.14-0.28 cosine) and destroys
|
||||
dense text (CER up to 1.0). The extra 0.13 of strength is simply too much
|
||||
regeneration. This is the Meta pattern amplified.
|
||||
|
||||
Microsoft at the respective floors (chroma 0.125 vs qwen 0.15): chroma wins
|
||||
EVERY metric on EVERY source while scrubbing at lower strength --
|
||||
|
||||
| source | CER q/ch | img LPIPS q/ch | SSIM q/ch | PSNR q/ch |
|
||||
|---|---|---|---|---|
|
||||
| paint-1 | 0.741 / 0.241 | 0.164 / 0.106 | 0.742 / 0.847 | 24.8 / 29.3 |
|
||||
| paint-2 | 0.826 / 0.391 | 0.253 / 0.108 | 0.536 / 0.633 | 22.5 / 25.5 |
|
||||
| paint-3 | 0.000 / 0.000 | 0.038 / 0.020 | 0.762 / 0.953 | 36.3 / 42.2 |
|
||||
|
||||
The Microsoft cohort therefore behaves like the OpenAI cohort, not like Meta:
|
||||
for InvisMark carriers Chroma1 is the better engine at a lower operating point.
|
||||
|
||||
## Price (warm H100 container)
|
||||
|
||||
| | qwen-zimage | chroma1 |
|
||||
|---|---|---|
|
||||
| seconds per image | 2.44 | 3.62 |
|
||||
| VRAM peak | 31.9 GiB | 29.2 GiB |
|
||||
| USD per 1000 images ($3.95/h) | $2.68 | $3.97 |
|
||||
| weights in cache | ~70 GB | ~28 GB |
|
||||
|
||||
The chroma number includes a T5 encode per call; the shipped qwen path caches
|
||||
prompt embeddings, and the same treatment would put chroma near 1.5-2 s/image.
|
||||
Both stacks fit an 80 GB card; chroma's smaller footprint likely fits smaller
|
||||
cards untested.
|
||||
|
||||
## Matched-strength comparison: the floors hide the engine (2026-08-30 addendum)
|
||||
|
||||
The floor-based tables above compare chroma at its DERIVED floor (worst case
|
||||
+ spread) against qwen at its shipped floor - different perturbation levels.
|
||||
At the strength where each fixture ACTUALLY turns clean (chroma first-clean
|
||||
rungs, all oracle-verified above), the picture reverses:
|
||||
|
||||
Google (chroma@first-clean vs qwen@0.27):
|
||||
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | CER q/ch | ID q/ch |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 633uuy | 0.25 | 0.405/**0.375** | 0.577/**0.673** | 18.8/**22.3** | **0.333**/0.667 | - |
|
||||
| akdbei | 0.25 | 0.440/**0.409** | 0.552/**0.650** | 18.6/**22.5** | 0.367/**0.163** | - |
|
||||
| y48j3c | 0.12 | 0.436/**0.329** | 0.608/**0.700** | 21.0/**25.8** | 1.0/1.0 | **0.801**/0.611 |
|
||||
| 3mc4t9 | 0.12 | 0.422/**0.318** | 0.529/**0.644** | 17.7/**21.3** | 0.443/**0.229** | **0.921**/0.410 |
|
||||
|
||||
Meta (chroma@first-clean vs qwen@0.1):
|
||||
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | CER q/ch |
|
||||
|---|---|---|---|---|---|
|
||||
| lighthouse | 0.10 (equal) | 0.173/**0.097** | 0.783/**0.886** | 25.6/**30.8** | - |
|
||||
| fox | 0.08 | 0.163/**0.083** | 0.778/**0.907** | 25.6/**31.4** | - |
|
||||
| night_city | 0.06 | 0.187/**0.092** | 0.748/**0.893** | 23.9/**29.6** | 0.850/**0.250** |
|
||||
| studio_mug | 0.045 | 0.024/**0.009** | 0.984/**0.992** | **39.0**/37.8 | - |
|
||||
| text_poster | 0.03 | 0.012/**0.002** | 0.984/**0.994** | 34.0/**41.2** | 0.021/**0.007** |
|
||||
|
||||
At effective-clean strength Chroma1 wins whole-image fidelity on essentially
|
||||
every fixture (often by 4-6 dB) and text on five of six; qwen's remaining
|
||||
edge is face identity, which its Z-Image face stage buys (0.80-0.92 vs
|
||||
0.41-0.61 cosine) and a chroma-zimage profile would inherit on top of its own
|
||||
global pass.
|
||||
|
||||
**Why the floors still matter:** a shipped strength must clear the worst
|
||||
unseen content, and Chroma1's per-fixture boundaries scatter much wider
|
||||
(Google 0.12-0.25, Meta 0.03-0.10) than the spread the flat floors were built
|
||||
to absorb - so its derived floors (0.40 / 0.17) pay a large margin penalty
|
||||
and at those floors the extra strength genuinely destroys quality. The
|
||||
engines differ less in "quality per strength" than in how predictable their
|
||||
boundary is. The integration lever this exposes: a content-adaptive strength
|
||||
policy for chroma-zimage (the project already ships resolution-adaptive
|
||||
denoise for qwen) could capture chroma's low per-content boundaries; the two
|
||||
hard Google fixtures are both dense-text cards and the easy ones are
|
||||
photoreal, so a content-class signal may predict the rung.
|
||||
|
||||
## Pre-ship validation (2026-08-30)
|
||||
|
||||
With the `chroma-zimage` profile implemented
|
||||
(`_internal/chroma_zimage_pipeline.py` on `TwoStageZImagePipeline`, floors in
|
||||
`watermark_profiles`), two ship-gate checks were run:
|
||||
|
||||
1. **Seed stability at the worst Meta boundary.** lighthouse @ 0.10 with
|
||||
seeds 1 and 2: both CLEAN on the meta.ai oracle. The Meta boundary is not a
|
||||
seed-0 fluke. The Google seed probes (633uuy @ 0.25) delivered a critical
|
||||
finding: **seed 0 was CLEAN but seed 1 was DETECTED** -- the 633uuy
|
||||
boundary is seed-dependent, which validates the flat 0.40 floor's 0.15
|
||||
margin exactly the way the qwen SDXL precedent did (the same image at the
|
||||
same strength passing or failing run-to-run near the threshold). Seed 2
|
||||
of `seedprobe_google_633uuy_s0.25_seed2.png` was checked twice on
|
||||
2026-08-31 after a human drag-drop. Work Gemini (u/2, Pro,
|
||||
`https://gemini.google.com/u/2/app/ebc7c19d22f85bb9`) with the
|
||||
ordinary question "Is there a SynthID watermark in this image?"
|
||||
auto-titled "SynthID Watermark Detection Results" and returned "No
|
||||
reliable signals were detected indicating how the content was created,
|
||||
and you should look for other supporting evidence from trusted
|
||||
sources." Plus `@synthid` (u/0,
|
||||
`https://gemini.google.com/u/0/app/658b1470307811b7`) with the built-in
|
||||
verification question auto-titled "SynthID Image Verification Results"
|
||||
and returned "No digital watermark from Google AI was detected in this
|
||||
image, indicating that it was not created or edited using Google's AI
|
||||
models." Both are CLEAN. An earlier Plus `@synthid` attempt on the
|
||||
same file hung on "Connecting to Verify AI" with auto-title "SynthID
|
||||
Image Verification Failed"; Flash prose abstained because "the tool
|
||||
response does not confirm" Google AI, which is not a tool verdict.
|
||||
Workspace Gemini's More-uploads menu does not list Photos, but a
|
||||
human drag-drop of a local PNG does attach; one Workspace account in
|
||||
the same Chrome has no Gemini app. Seed 0 CLEAN / seed 1 DETECTED /
|
||||
seed 2 CLEAN at 0.25
|
||||
leaves the 633uuy boundary seed-dependent on the Plus `@synthid`
|
||||
path; the 0.40 floor is already established by the seed-1 flip.
|
||||
2. **Full-path face-stage interaction.** The complete profile run (Chroma1
|
||||
global at the shipped Google floor 0.40, then the inherited
|
||||
YuNet/SAM/Z-Image face stage) on the 5-face fixture 3mc4t9: CLEAN on the
|
||||
@synthid oracle. The composited face regions do not re-introduce a
|
||||
detectable signal on top of the floor.
|
||||
|
||||
Generation script: `scripts/chroma_preship_validation.py`; outputs under
|
||||
`out/preship-chroma/` (gitignored).
|
||||
|
||||
## Verdict
|
||||
|
||||
Engine quality is cohort-dependent, and after measuring all four cohorts the
|
||||
split is even and sharp:
|
||||
|
||||
- On OpenAI-provenance content Chroma1 is better at every measured fidelity
|
||||
axis at its floor (0.09 vs 0.07675), for comparable money.
|
||||
- On Microsoft InvisMark content Chroma1 is better at every measured fidelity
|
||||
axis at a LOWER floor than qwen (0.125 vs 0.15).
|
||||
- On Meta Content Seal Chroma1 needs 1.7x the strength (0.17 vs 0.1) and loses
|
||||
LPIPS on textured content at its floor.
|
||||
- On Google SynthID Chroma1 needs ~1.5x the strength (0.40 vs 0.27) and loses
|
||||
every fidelity axis on every fixture, with face identity collapsing at 0.40
|
||||
(0.14-0.28 cosine vs qwen's 0.80-0.92).
|
||||
|
||||
A single-engine `chroma-zimage` default at a flat per-vendor floor is NOT
|
||||
supported: Google and Meta floors land too high and the quality at those
|
||||
floors is worse than qwen's. But the matched-strength addendum changes what
|
||||
IS supported: at the strength each image actually needs, Chroma1 regenerates
|
||||
better than qwen almost everywhere except face identity (which the shared
|
||||
face stage would supply). The two honest integration options:
|
||||
|
||||
1. Per-cohort engine router at flat floors: chroma for OpenAI/Microsoft,
|
||||
qwen for Google/Meta - conservative, ships on the measurements as they
|
||||
stand.
|
||||
2. `chroma-zimage` with a content-adaptive strength policy that predicts the
|
||||
per-image rung - potentially better everywhere, but the predictor is new
|
||||
research (the measured signal so far: dense-text content needs the high
|
||||
rungs, photoreal the low ones) and needs its own oracle validation before
|
||||
any ship.
|
||||
|
||||
Both need the seed-sensitivity and face-stage composition checks called out
|
||||
above before shipping. Both checks have now been run (Pre-ship validation
|
||||
above): the Meta boundary is seed-stable across seeds 0-2 and the full
|
||||
profile path stays clean through the face stage on the shipped Google floor.
|
||||
The profile ships with option 1's floors plus a content-adaptive Google arm
|
||||
(face content resolves to 0.125 instead of 0.40, keyed on YuNet detection).
|
||||
Option 2's full adaptive policy for the remaining cohorts remains the
|
||||
documented follow-up. The Meta arm was measured on 2026-08-30/31 and
|
||||
does not have a shippable split; see the expansion section below.
|
||||
|
||||
## Meta content-adaptive expansion (2026-08-30/31)
|
||||
|
||||
Goal: test whether `flat_ratio` (fraction of 16x16 blocks with luma std < 8)
|
||||
predicts Chroma1's first-clean Content Seal boundary well enough to ship a
|
||||
Meta arm analogous to Google's face-count split.
|
||||
|
||||
**Harvest 1, from a 495-file spaces dump, was not a diverse expansion.**
|
||||
Seven "unique" files were staged under `out/cohort-calibration/meta/muse-*`
|
||||
(gitignored). After control checks on `meta.ai/identification`:
|
||||
|
||||
| fixture | control | first-clean | note |
|
||||
|---|---|---|---|
|
||||
| muse-1 | CLEAN | -- | seal did not survive the metadata-stripped control |
|
||||
| muse-2 | DETECTED | (0.08, 0.10] | photoreal portrait; 0.06 and 0.08 DETECTED, 0.10 CLEAN |
|
||||
| muse-3 | DETECTED | (0.015, 0.03] | photoreal portrait, different subject |
|
||||
| muse-4 | DETECTED | (0.08, 0.10] | same subject as muse-2/6/7; 0.06 and 0.08 DETECTED, 0.10 CLEAN |
|
||||
| muse-5 | -- | -- | byte-identical to committed `gen_lighthouse_watercolor.webp` |
|
||||
| muse-6 | DETECTED | (0.06, 0.08] | same subject as muse-4; 0.06 DETECTED, 0.08 CLEAN |
|
||||
| muse-7 | DETECTED | (0.045, 0.06] | same subject as muse-2/4/6; 0.045 DETECTED, 0.06 CLEAN |
|
||||
|
||||
Five valid new points, four of them near-duplicate portraits of one
|
||||
subject, with first-clean already spanning 0.06-0.10 **inside that
|
||||
subject**. Google's shipped split had zero overlap between classes and
|
||||
identical boundaries inside a class. This harvest cannot support a
|
||||
predictor.
|
||||
|
||||
`flat_ratio` on the original five plus the new portraits is not monotonic
|
||||
with first-clean either (studio_mug 0.973 / 0.045 easy; lighthouse 0.602 /
|
||||
0.10 hard; muse-4 0.789 / 0.10 hard). OpenAI stays unshippable for the
|
||||
same reason as before: one text fixture vs one face fixture, and the face
|
||||
fixture was the harder one, the opposite of Google's split.
|
||||
|
||||
**Harvest 2** takes six files from the 2026-08-26 Muse Image API corpus
|
||||
(61 independent `muse-image-1.0` generations, no hash overlap with the
|
||||
committed five), chosen to span class and `flat_ratio`:
|
||||
|
||||
| fixture | class | flat_ratio | edge_density |
|
||||
|---|---|---|---|
|
||||
| architecture_mosque | dense architecture | 0.037 | 0.367 |
|
||||
| product_sneaker | product / high-flat | 0.874 | 0.018 |
|
||||
| text_poster_bakery | text | 0.368 | 0.191 |
|
||||
| portrait_weathered_fisherman | photoreal portrait | 0.470 | 0.108 |
|
||||
| illustration_watercolor_botanical | illustration | 0.442 | 0.089 |
|
||||
| scene_tokyo_alley | busy scene | 0.258 | 0.175 |
|
||||
|
||||
Script: `scripts/chroma_muse_expansion.py`. Outputs under
|
||||
`out/cohort-calibration/meta-api/` (gitignored). Chroma1 ladders (seven
|
||||
rungs, seed 0, four effective steps) ran for all six. Oracle
|
||||
2026-08-31, anonymous `playwright-isolated` against
|
||||
`meta.ai/identification`. All six controls DETECTED (valid carriers).
|
||||
|
||||
| fixture | flat_ratio | detected at | clean from |
|
||||
|---|---|---|---|
|
||||
| architecture_mosque | 0.037 | 0.08 | 0.10 |
|
||||
| illustration_watercolor_botanical | 0.442 | 0.08 | 0.10 |
|
||||
| scene_tokyo_alley | 0.258 | 0.06 | 0.08 |
|
||||
| text_poster_bakery | 0.368 | 0.045 | 0.06 |
|
||||
| portrait_weathered_fisherman | 0.470 | 0.045 | 0.06 |
|
||||
| product_sneaker | 0.874 | 0.03 | 0.045 |
|
||||
|
||||
The worst first-clean is still 0.10, the same as the committed
|
||||
lighthouse, so the shipped Meta floor 0.17 does not move.
|
||||
|
||||
**No Meta adaptive arm.** Google's shipped split had zero overlap
|
||||
between classes and identical first-cleans inside a class. Combined
|
||||
with the original five, Meta first-cleans are a continuum from 0.03 to
|
||||
0.10, and `flat_ratio` does not separate them:
|
||||
|
||||
- the 0.10 cluster is mosque 0.037, botanical 0.442, and lighthouse
|
||||
0.602;
|
||||
- the 0.045 cluster is sneaker 0.874 and studio_mug 0.973;
|
||||
- harvest-1 photoreal portraits were high-flat (0.75-0.85) AND hard
|
||||
(0.08-0.10), so a high-flat easy-arm would misroute them.
|
||||
|
||||
A content-class rule fails the same way: two text posters first-clean
|
||||
at 0.03 and 0.06, and two watercolor-like images (lighthouse, botanical)
|
||||
share 0.10 with dense architecture.
|
||||
|
||||
The leftover rungs were finished in a 2026-08-31 evening UTC
|
||||
anonymous isolated session (fresh-navigation, wait for "Upload
|
||||
another file"; all ten `POST /api/ai-detector` calls returned 200):
|
||||
|
||||
| check | verdict |
|
||||
|---|---|
|
||||
| muse-2 @ 0.08 | DETECTED |
|
||||
| muse-3 @ 0.015 / 0.03 / 0.045 | DETECTED / CLEAN / CLEAN |
|
||||
| muse-7 @ 0.045 / 0.06 | DETECTED / CLEAN |
|
||||
| mosque @ 0.10 seeds 1, 2 | CLEAN / CLEAN |
|
||||
| botanical @ 0.10 seeds 1, 2 | DETECTED / DETECTED |
|
||||
|
||||
botanical's seed-0 first-clean of 0.10 is not seed-stable;
|
||||
lighthouse @ 0.10 seeds 1 and 2 were CLEAN. That is the same class
|
||||
of run-to-run flip the 0.17 floor's margin exists for. No
|
||||
Google-style binary split: harvest-1 first-cleans now span 0.03
|
||||
(muse-3) through 0.10 (muse-2/4), and 0.06-0.10 inside one subject
|
||||
(muse-7 / muse-6 / muse-2).
|
||||
|
||||
## OpenAI content-adaptive expansion (2026-08-31)
|
||||
|
||||
Goal: test whether face_count or `flat_ratio` predicts Chroma1's OpenAI
|
||||
SynthID first-clean the way YuNet does for Google.
|
||||
|
||||
Committed three (YuNet + `flat_ratio` on the files, first-cleans from the
|
||||
2026-08-29/30 web oracle):
|
||||
|
||||
| fixture | faces | flat_ratio | first-clean |
|
||||
|---|---|---|---|
|
||||
| typography 02_03_55 | 0 | 0.642 | 0.06 |
|
||||
| quality 02_02_23 | 0 | 0.695 | 0.06 |
|
||||
| 9-face grid 05_30 | 9 | 0.473 | 0.075 |
|
||||
|
||||
Face content is the harder of the two classes, the opposite of Google.
|
||||
Two zero-face cards share a boundary, which is a hint, not a split.
|
||||
|
||||
**Images API is not a carrier source.** `gpt-image-1` and `gpt-image-1.5`
|
||||
generations (medium, 1024) carry C2PA (`detected`) but SynthID
|
||||
`not_detected` on `POST /v1/content_provenance_checks`. Contrastive-pair
|
||||
"openai" recreations are also SynthID-negative. New ChatGPT-UI downloads
|
||||
are not in the tree beyond the three committed files.
|
||||
|
||||
**Spaces harvest (one day of `_full_scan`, 2026-07-24):** 41 files with
|
||||
OpenAI C2PA, of which 7 checked by the provenance API were SynthID
|
||||
`DETECTED` (one `NOT_DETECTED`). Four diverse DETECTED files were staged
|
||||
under `out/cohort-calibration/openai-expand/` (gitignored; user uploads,
|
||||
not committed) and given a Chroma1 ladder (0.04-0.12, seed 0) by
|
||||
`scripts/chroma_openai_expansion.py`:
|
||||
|
||||
| id | faces | flat_ratio |
|
||||
|---|---|---|
|
||||
| dense_ed68 | 0 | 0.118 |
|
||||
| midflat_038c | 0 | 0.323 |
|
||||
| textlike_3b17 | 0 | 0.649 |
|
||||
| faces2_4dab | 2 | 0.279 |
|
||||
|
||||
The provenance API then 429'd. The same checks were finished on
|
||||
`https://openai.com/research/verify/` (the page reports SynthID and C2PA
|
||||
separately under View details). Chroma PNG outputs have no C2PA, so a
|
||||
"No OpenAI signals detected" heading is a pixel-SynthID negative.
|
||||
|
||||
| id | faces | @0.04 | @0.05 | @0.06 |
|
||||
|---|---|---|---|---|
|
||||
| dense_ed68 | 0 | CLEAN | CLEAN | CLEAN |
|
||||
| midflat_038c | 0 | -- | -- | CLEAN |
|
||||
| textlike_3b17 | 0 | -- | -- | CLEAN |
|
||||
| faces2_4dab | 2 | -- | -- | CLEAN |
|
||||
|
||||
9-face grid @ 0.075, seeds 1 and 2: both CLEAN (SynthID not detected).
|
||||
The committed seed-0 first-clean is not a fluke.
|
||||
|
||||
**No OpenAI adaptive arm.** Zero-face extras clear at or below 0.06, the
|
||||
2-face extra also clears at 0.06, and only the 9-face grid needs 0.075.
|
||||
That is not a YuNet split: `face_count > 0` would over-strengthen the
|
||||
2-face file, and a high face-count cutoff would be one fixture. The
|
||||
spread (0.075 - 0.06) is already absorbed by the shipped 0.09 floor.
|
||||
Images API generations remain non-carriers (C2PA only).
|
||||
|
||||
## Content-balanced engine selection check (2026-08-31)
|
||||
|
||||
The earlier tables were intentionally carrier-focused and therefore small. A
|
||||
separate discovery pass tested whether ordinary image content contradicts the
|
||||
cohort choice at the shipped floors. The tracked matrix contains 19 prompts,
|
||||
matched across OpenAI `gpt-image-1-mini` and Meta `muse-image-1.0`, spanning
|
||||
photoreal scenes, faces, action, text-like flat art, products, architecture,
|
||||
watercolor, and surreal illustration. These files were re-encoded by collection
|
||||
and are not signal carriers, so this pass measures fidelity only; removal remains
|
||||
established by the provider-oracle calibration above.
|
||||
|
||||
Qwen and Chroma ran in separate sequential H100 invocations, once per source,
|
||||
with the production provider floor and fixed seed. The comparison isolated the
|
||||
global stage because both profiles inherit the same source-based Z-Image face
|
||||
repair. Per-image differences, rather than differences between aggregate means,
|
||||
were evaluated with exact two-sided sign tests:
|
||||
|
||||
| provider | metric | Chroma wins | Qwen wins | two-sided p |
|
||||
|---|---|---:|---:|---:|
|
||||
| OpenAI | SSIM | 19 | 0 | 0.0000038 |
|
||||
| OpenAI | PSNR | 19 | 0 | 0.0000038 |
|
||||
| OpenAI | MAE | 18 | 1 | 0.000076 |
|
||||
| OpenAI | edge F1 | 15 | 4 | 0.019 |
|
||||
| OpenAI | LPIPS | 12 | 7 | 0.359 |
|
||||
| Meta | LPIPS | 1 | 18 | 0.000076 |
|
||||
| Meta | edge F1 | 1 | 18 | 0.000076 |
|
||||
| Meta | SSIM | 5 | 14 | 0.064 |
|
||||
|
||||
Meta's remaining pixel metrics disagree: Chroma wins PSNR on 16/19 and MAE on
|
||||
14/19, while losing both perceptual distance and edge preservation on 18/19 at
|
||||
its higher removal floor. This is not evidence for a genre classifier. The
|
||||
direction instead follows the already measured provenance cohort: Chroma for
|
||||
OpenAI, Qwen for Meta. On OpenAI, the seven LPIPS exceptions include unrelated
|
||||
classes (landscape, sport, low-light interior, watercolor, monochrome portrait,
|
||||
surreal art, and brutalism), while Chroma still wins SSIM and PSNR on every one;
|
||||
there is no coherent input feature that justifies overriding the cohort choice.
|
||||
|
||||
Inputs and integrity manifests are under
|
||||
`data/evaluations/engine-selection/`. Raw generated outputs remain gitignored
|
||||
under `out/engine-selection-study/`; `scripts/engine_selection_study.py`
|
||||
reproduces the run and `scripts/analyze_engine_selection_study.py` reproduces
|
||||
the paired statistics. One generation per prompt is a discovery set, not a
|
||||
license to fit thresholds. Any future content override must first predict a
|
||||
rule here and then survive unused generation indices without changing it.
|
||||
|
||||
## Oracle session notes
|
||||
|
||||
- openai.com/verify throttles anonymous sessions through Cloudflare Turnstile
|
||||
after ~15 rapid checks; a fresh browser context clears it. The programmatic
|
||||
`POST /v1/content_provenance_checks` API is the better oracle (separate
|
||||
SynthID/C2PA outcomes). The `OPENAI_API_KEY` in the main-checkout `.env`
|
||||
returned 401 org-wide, including `/v1/models`, through the 2026-08-30
|
||||
calibration; it was replaced on 2026-08-31 and both `/v1/models` and
|
||||
a provenance check on a committed ChatGPT original then succeeded
|
||||
(DETECTED).
|
||||
- meta.ai/identification enforces a sliding per-IP window: a burst of ~15
|
||||
checks exhausted it twice; it reopens minutes later, and the page sometimes
|
||||
overstates this as a daily limit. Pacing of roughly 2-3 checks per window
|
||||
worked. On 2026-08-31 an anonymous isolated session got a real 429 from
|
||||
`POST /api/ai-detector` with page text "You’ve reached the daily limit
|
||||
for identifications. Try again tomorrow" after about six checks in the
|
||||
UTC morning and again after about fifteen in the UTC afternoon, so the
|
||||
daily-limit wording is not always overstated; the afternoon window was
|
||||
large enough to finish harvest 2. An evening isolated session the same
|
||||
day got ten 200s and finished the leftover harvest-1 rungs plus the
|
||||
mosque/botanical seed probes.
|
||||
- The Gemini app oracle requires the user's logged-in session. A human
|
||||
drag-drop of a local PNG works on Work Gemini (u/2, 2026-08-31) and on
|
||||
Plus (u/0 the same day); Playwright cannot drive the OS picker, and
|
||||
the first `input[type=file].accept` is documents/code only. Plus
|
||||
`@synthid` hung once after attach, then returned a tool CLEAN on a
|
||||
second human-drag chat. Do not drive the live Chrome while it is in
|
||||
active use (wrong-tab risk).
|
||||
@@ -14,6 +14,8 @@ defaults. This page focuses on choosing the right command.
|
||||
| Command or signal | Required installation |
|
||||
| --- | --- |
|
||||
| `metadata` and metadata-only `identify` | Default package |
|
||||
| `classify` | `remove-ai-watermarks[classify]` |
|
||||
| `verify-openai-synthid` | `remove-ai-watermarks[verify]`, API access, and `OPENAI_API_KEY` |
|
||||
| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` |
|
||||
| Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` |
|
||||
@@ -62,8 +64,76 @@ Metadata only inspection:
|
||||
remove-ai-watermarks identify image.png --no-visible
|
||||
```
|
||||
|
||||
Despite the historical option name, `--no-visible` skips both visible and open
|
||||
invisible pixel detectors. Metadata inspection still runs.
|
||||
Despite the historical option name, `--no-visible` skips all pixel detectors,
|
||||
including visible marks, open DWT-DCT, and TrustMark. Metadata inspection still
|
||||
runs.
|
||||
|
||||
## Verify OpenAI SynthID from pixels
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[verify]"
|
||||
remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload
|
||||
remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload --json
|
||||
```
|
||||
|
||||
This is an explicit remote check against OpenAI's official Content Provenance
|
||||
API, not the incomplete local OpenAI carrier research model. Before upload, the
|
||||
command writes a temporary copy with AI provenance metadata removed and aborts
|
||||
unless the decoded RGBA pixels are identical to the source. It then reads only
|
||||
the API's independent `synthid` entry; a C2PA-only response cannot become a
|
||||
SynthID detection. The source is never modified.
|
||||
|
||||
The API supports PNG, JPEG, and WebP files up to 50 MiB. The command requires
|
||||
`OPENAI_API_KEY` and an organization with endpoint access. Because the sanitized
|
||||
raster is uploaded to OpenAI and the endpoint is not eligible for Zero Data
|
||||
Retention, `--acknowledge-upload` is mandatory. This command is never called by
|
||||
`identify`. `not_detected` means only that OpenAI's verifier did not recognize a
|
||||
supported watermark in this file; it is not proof of human authorship.
|
||||
|
||||
The built-in client bounds the request at 120 seconds and disables automatic
|
||||
SDK retries, so one acknowledgement cannot silently upload the media multiple
|
||||
times. A timeout, disconnect, malformed response, access failure, or rate limit
|
||||
is an error, never a negative watermark verdict. API failures expose status,
|
||||
error code, request id, `Retry-After`, and whether an explicit caller-controlled
|
||||
retry is appropriate through `OpenAIProvenanceError`; the verifier itself never
|
||||
retries an upload.
|
||||
The JSON result uses the same provider-scope, backend, pixel-preservation, and
|
||||
metadata-use audit fields as `verify_openai_synthid`.
|
||||
|
||||
The Python API enforces the same boundary with the required explicit intent
|
||||
flag `verify_openai_synthid(path, acknowledge_upload=True)`.
|
||||
|
||||
## Classify a photograph from pixels
|
||||
|
||||
This is not provenance. `identify` never starts it, including after a
|
||||
no-signal metadata scan. Install the extra first:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[classify]"
|
||||
```
|
||||
|
||||
Then:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks classify image.png
|
||||
remove-ai-watermarks classify image.png --json
|
||||
```
|
||||
|
||||
The command runs the frozen photo detector (CLIP-L-ft ridge AND freeze MLP).
|
||||
Only a DEFINITELY result is reported as `ai`. On that result only, the 124-d
|
||||
provider head may return `openai`, `google`, `muse-image`, or `tc260`. POSSIBLY is
|
||||
`unknown`. Camera-like photographs are `human`. The call does not run cleanup
|
||||
and does not set `is_ai_generated`.
|
||||
|
||||
The contract is AI versus camera. Receipts, UI, and digital art are out of
|
||||
scope. Microsoft is not a pixel class: a DALL-E Bing image scores as `openai`,
|
||||
an Imagen Designer image as `google`. `tc260` is the China AIGC label
|
||||
standard, not one producer. Weights download on first use from
|
||||
[`wiltodelta/raiw-photo-classify`](https://huggingface.co/wiltodelta/raiw-photo-classify),
|
||||
or from `RAIW_CLASSIFY_WEIGHTS` if that directory already holds the freeze
|
||||
files.
|
||||
Guide: [photo pixel classification](photo-classify.md). Hub card:
|
||||
[photo-classify-hf/README.md](photo-classify-hf/README.md).
|
||||
|
||||
## Remove known visible marks
|
||||
|
||||
@@ -105,6 +175,18 @@ Use the strict visual gate without metadata or sibling corroboration:
|
||||
remove-ai-watermarks visible image.png --sensitivity strict -o clean.png
|
||||
```
|
||||
|
||||
Act on a mark you can see but the detector will not confirm, by naming it and
|
||||
skipping the gate:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks visible image.png --mark gemini --no-detect -o clean.png
|
||||
```
|
||||
|
||||
`--detect` is the default. `--no-detect` removes the named mark's footprint
|
||||
unconditionally, so it only makes sense together with an explicit `--mark`; on
|
||||
an image that never carried that mark it inpaints a region for nothing. Prefer
|
||||
`erase --region` when you know the geometry.
|
||||
|
||||
When no known mark is detected, the command does not write a new output. Use
|
||||
`erase` if you can identify the affected region yourself.
|
||||
|
||||
@@ -327,8 +409,8 @@ provider provenance may relax the visual score only for registered
|
||||
provenance-aware marks; metadata alone never creates a detection.
|
||||
|
||||
`--mark auto` is the default. It evaluates all providers in one decode pass and
|
||||
selects the first stable match in specificity order: Sora, Veo, Seedance, Dola,
|
||||
Hailuo AI, then Kling AI. Their confidence scores are independently calibrated and
|
||||
selects the first stable match in specificity order: Sora, Veo, Seedance,
|
||||
Doubao, Dola, Hailuo AI, then Kling AI. Their confidence scores are independently calibrated and
|
||||
are not compared across providers. Pass an explicit `--mark` to scan only that
|
||||
provider.
|
||||
|
||||
@@ -367,7 +449,7 @@ failed encode does not overwrite an existing result.
|
||||
|
||||
## Remove invisible watermarks
|
||||
|
||||
Install the removal dependencies first. Both profiles are CUDA-only and both
|
||||
Install the removal dependencies first. All profiles are CUDA-only and all
|
||||
run the DiffSynth Z-Image face stage, so this is the extra either one needs:
|
||||
|
||||
```bash
|
||||
@@ -410,8 +492,10 @@ The same option exists on `all` and `batch`, and as
|
||||
| --- | --- |
|
||||
| `qwen-zimage` | Default. Qwen-Image-2512 global pass plus a SAM-masked Z-Image face stage |
|
||||
| `sdxl-zimage` | The same recipe and face stage on an SDXL global pass, at a higher denoise |
|
||||
| `chroma-zimage` | The same face stage on an Apache-2.0 Chroma1 global pass, with lower OpenAI and Microsoft floors and higher Google and Meta floors |
|
||||
| `auto` | Pick the engine from provenance: chroma-zimage for OpenAI and Microsoft, qwen-zimage for Google, Meta, and unknown |
|
||||
|
||||
**Both are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
**All four are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
removal. The former `controlnet`, `sdxl`, `qwen` and `default` profiles were removed
|
||||
rather than kept as a CPU path: none of them matched this recipe's face preservation,
|
||||
so offering them implied a quality the library no longer delivers. Passing a retired
|
||||
@@ -434,17 +518,18 @@ user can act on rather than after a model load.
|
||||
### Restore operator-verified text
|
||||
|
||||
`--text-manifest` enables the experimental `vae-glyphs` post-pass. It reconstructs
|
||||
the source with the Qwen VAE, erases the annotated candidate glyphs with LaMa, and
|
||||
composites only the reconstructed glyph cores through source-derived silhouettes. It
|
||||
does not run OCR or choose which strings are correct. The optional `--fidelity-anchor`
|
||||
described below additionally blends 15% of the reconstruction across the full frame.
|
||||
the source with the selected profile's VAE, erases the annotated candidate glyphs
|
||||
with LaMa, and composites only the reconstructed glyph cores through source-derived
|
||||
silhouettes. It does not run OCR or choose which strings are correct. The optional
|
||||
Qwen-only `--fidelity-anchor` described below additionally blends 15% of the
|
||||
reconstruction across the full frame.
|
||||
|
||||
Install the combined extra and run only with an operator-verified manifest:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --text-manifest verified-lines.json --force
|
||||
--pipeline auto --text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
``verified: true`` may also be set by an automated operator that attests
|
||||
@@ -452,7 +537,8 @@ machine-verified geometry: stability-gated detector boxes inside sane caps. Such
|
||||
operators should use the geometry-only schema 2, which carries no transcription
|
||||
or script metadata.
|
||||
|
||||
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default: it
|
||||
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default and
|
||||
remains available only with `qwen-zimage`: it
|
||||
was measured to return detector-visible OpenAI SynthID on poster-scale manifests
|
||||
(official Content Provenance API, 2026-08-19). `--fidelity-anchor` restores the
|
||||
0.27.0 research behavior; text-box fidelity lost by the default is well under one
|
||||
@@ -469,8 +555,9 @@ container changes remain valid while a resized or edited source fails closed. Th
|
||||
experimental helper
|
||||
`remove_ai_watermarks._internal.text_restoration.source_pixel_sha256` computes it.
|
||||
|
||||
This mode is supported only by `qwen-zimage` at native geometry with
|
||||
`humanize=0`, `unsharp=0`, and adaptive polish disabled. `all` also accepts the flag,
|
||||
This mode is supported by `qwen-zimage`, `chroma-zimage`, and `auto` at native
|
||||
geometry with `humanize=0`, `unsharp=0`, and adaptive polish disabled. The legacy
|
||||
`sdxl-zimage` profile does not expose a verified-text VAE donor. `all` also accepts the flag,
|
||||
but its manifest must match the pixels entering the invisible stage; if visible-mark
|
||||
removal changes those pixels, the hash check rejects the run. One oracle verdict does
|
||||
not certify another manifest, seed, model/runtime version, or output hash.
|
||||
@@ -500,7 +587,33 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
```
|
||||
|
||||
Tiling avoids the explicit downscale but each tile is regenerated separately.
|
||||
It is a memory strategy, not a guarantee of better quality.
|
||||
It is a memory strategy, not a guarantee of better quality. `--tile-size` and
|
||||
`--tile-overlap` set the tile geometry (defaults 1024 and 128 px); a larger
|
||||
overlap costs time and hides seams.
|
||||
|
||||
### Every tuning option
|
||||
|
||||
These apply to `invisible`, `all`, and `batch`. Each has a measured default, so
|
||||
reach for one only when you know why the default is wrong for your file.
|
||||
|
||||
| Option | Effect |
|
||||
| --- | --- |
|
||||
| `--strength` | Denoising strength, 0.0-1.0. Default is profile- and vendor-adaptive; see [choose a strength cohort](#choose-a-strength-cohort). Raising it removes more and drifts further from the original. |
|
||||
| `--vendor` | Strength cohort, and an assertion that the watermark is present. |
|
||||
| `--pipeline` | Profile: `qwen-zimage` (default), `sdxl-zimage`, `chroma-zimage`, `auto`. |
|
||||
| `--seed` | Fixed at 0 by default: every profile is certified at that seed, and removal near the floor is seed-dependent. |
|
||||
| `--controlnet-scale` | Canny ControlNet conditioning on the global stage. Higher keeps the original structure and text closer. |
|
||||
| `--humanize` | Analog film grain, 0 = off, typical 2.0-6.0. |
|
||||
| `--unsharp` | Unsharp-mask sharpening, 0 = off. |
|
||||
| `--adaptive-polish` / `--no-adaptive-polish` | Overrides the profile's own choice. Unset lets the profile decide: off for `qwen-zimage`, whose output already matches the input's detail level, on for `sdxl-zimage`. |
|
||||
| `--max-resolution` | Cap the long side before diffusion; 0 = native. |
|
||||
| `--tile`, `--tile-size`, `--tile-overlap` | Tiled regeneration for large inputs, as above. |
|
||||
| `--cpu-offload` | Stream model components between CPU and GPU to lower CUDA memory pressure, at the cost of speed. |
|
||||
| `--hf-token` | Hugging Face token for the model download; also read from the environment and a local `.env`. |
|
||||
|
||||
There is no `--model`, `--steps`, `--guidance-scale` or `--device` option; the
|
||||
profile pins all four. See [exit behavior](#exit-behavior) for what each command
|
||||
returns.
|
||||
|
||||
## Run the full pipeline
|
||||
|
||||
|
||||
@@ -79,8 +79,9 @@ enough yet barely touches flat fills. So the survivors FLIP by content type —
|
||||
choice alone does not guarantee removal.
|
||||
|
||||
**2. Seed non-determinism near threshold.** img2img uses a random seed unless `--seed`
|
||||
is passed, and there is no local SynthID detector to self-verify. The bracelet survived
|
||||
controlnet @0.15 in one run and CLEARED @0.15 in another (same pipeline+strength+res).
|
||||
is passed, and these geometries are outside the current local detector's scope.
|
||||
The bracelet survived controlnet @0.15 in one run and CLEARED @0.15 in another
|
||||
(same pipeline+strength+res).
|
||||
So a single clean run does NOT establish a strength as safe — characterizing a reliable
|
||||
floor needs a seed-repeatability sweep (N runs, varied seed), not one pass.
|
||||
|
||||
@@ -144,8 +145,8 @@ Gemini app; the two payloads are vendor-specific and never cross-checked):
|
||||
but never recovered original identity precisely — every setting traded one problem
|
||||
for another. See `docs/synthid-robust-identity-research-2026-06-08.md`
|
||||
"Empirical follow-up" for the full sweep.
|
||||
- **No local SynthID detector exists** → the service can't self-verify; bake in strength
|
||||
margin and periodic oracle spot-checks.
|
||||
- **No applicable local detector exists for these geometries** → the service
|
||||
can't self-verify; bake in strength margin and periodic oracle spot-checks.
|
||||
- **Lesson:** visual-quality / face-identity recovery does NOT prove removal — only the
|
||||
oracle does, across MULTIPLE content types; never conclude from a partial result (the
|
||||
photoreal-only data first read as "controlnet shields, default removes"; the flat-graphic
|
||||
@@ -298,8 +299,9 @@ attention-slicing; ~1-2 min/image, so a coarse sweep is a sub-hour background ru
|
||||
is needed ONLY for the separate
|
||||
native-large-Gemini (2816 px) case, which OOMs even without a ControlNet (that requires a
|
||||
GPU task). The genuine external dependency is NOT compute but the **manual SynthID oracle**:
|
||||
there is no local SynthID detector, so removal is verified by hand in the Gemini app
|
||||
("Verify with SynthID") per image, regardless of where the diffusion runs.
|
||||
these geometries are outside the current local detector's scope, so removal is
|
||||
verified by hand in the Gemini app ("Verify with SynthID") per image, regardless
|
||||
of where the diffusion runs.
|
||||
|
||||
Runner: **`scripts/controlnet_sweep.py`** (built 2026-06-02) implements exactly this sweep —
|
||||
SDXL base 1.0 + an SDXL-native ControlNet img2img, one output per (control x strength x scale)
|
||||
@@ -441,8 +443,9 @@ shielding risk; defer to a v2 after the single-canny path is dialed in.
|
||||
|
||||
**Hard caveat:** every change that increases preservation (higher scale, denser canny, fuller window,
|
||||
softer edges) marginally REDUCES effective regeneration and so raises the chance the watermark
|
||||
survives -- exactly the shielding failure mode. There is no local SynthID detector, so each tuning
|
||||
change must be re-confirmed on the oracle. These are img2img-context recommendations derived from
|
||||
survives -- exactly the shielding failure mode. These geometries are outside the
|
||||
current local detector's scope, so each tuning change must be re-confirmed on
|
||||
the oracle. These are img2img-context recommendations derived from
|
||||
generation-context sources plus our own measurements; treat the playbook as hypotheses to verify, not
|
||||
settled defaults.
|
||||
|
||||
|
||||
@@ -12,6 +12,8 @@ to run the tool. Use the maintainer references only when changing the code.
|
||||
| [Python API](python-api.md) | You want to call the package from Python. |
|
||||
| [Supported signals](supported-signals.md) | You need to know which visible marks, metadata formats, and invisible signals are covered. |
|
||||
| [Known limitations](known-limitations.md) | You need the quality, device, format, or verification boundaries. |
|
||||
| [Photo pixel classification](photo-classify.md) | You want `classify` / `classify_pixels`: AI versus camera, optional provider. |
|
||||
| [Photo classifier training](photo-classify-training.md) | You need the retrain pack, public sources, and what is not published. |
|
||||
| [Scope, safety, and legal notes](legal-and-safety.md) | You need the intended use and legal context. |
|
||||
|
||||
## Maintainer references
|
||||
@@ -22,9 +24,9 @@ to run the tool. Use the maintainer references only when changing the code.
|
||||
| [Development](development.md) | Environment setup, dependency recovery, CI behavior, and fixture policy. |
|
||||
| [Code provenance](code-provenance.md) | Required notices for licensed derivative work. |
|
||||
| [Verification plan](verification-plan.md) | Verification methods, completed measurements, and remaining validation gaps. |
|
||||
| [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, ComfyUI Registry, and the release workflow. |
|
||||
| [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, photo-classify Hub model, ComfyUI Registry, and the release workflow. |
|
||||
| [Watermarking landscape](watermarking-landscape.md) | Vendor signals and detection approaches. |
|
||||
| [SynthID technical reference](synthid.md) | Mechanism, detector access, robustness, and implications for this project. |
|
||||
| [SynthID technical reference](synthid.md) | Mechanism, provenance, robustness, regeneration. |
|
||||
|
||||
## Research archive
|
||||
|
||||
@@ -35,7 +37,15 @@ The current behavior is defined by the code, tests, README, and user guides.
|
||||
- [ControlNet removal research](controlnet-removal-pipeline-research.md)
|
||||
- [Qwen improvement research](qwen-improvement-research.md)
|
||||
- [Doubao reverse-alpha research](research-doubao-distillation.md)
|
||||
- [AI-generated image classifiers](ai-generated-image-classifiers.md) (photo AI-versus-camera freeze consumed by `classify_pixels`, not `identify`)
|
||||
- [Photo classifier Hugging Face card](photo-classify-hf/README.md) (Hub publication text of the 2026-08-31 freeze)
|
||||
- [SynthID source classifiers](synthid-classifiers.md) (metadata-free OpenAI/Gemini source finding and provider-lineage experiments)
|
||||
- [SynthID local detector research](synthid-detector-research.md)
|
||||
- [SynthID mark removal research](synthid-removal-research.md)
|
||||
- [SynthID identity research](synthid-robust-identity-research.md)
|
||||
- [SynthID identity follow-up](synthid-robust-identity-research-2026-06-08.md)
|
||||
- [Video SynthID quality research](video-synthid-quality-research.md)
|
||||
- [OpenAI SynthID oracle ladders](synthid-oracle-ladders.md) (routing hub)
|
||||
- [SynthID detector and removal plan](synthid-detector-removal-plan.md) (chronological mixed archive)
|
||||
- [Text protection research](text-protection-research.md)
|
||||
- [Chroma1 engine research](chroma1-engine-research.md)
|
||||
@@ -75,11 +75,16 @@ uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
```
|
||||
|
||||
Both remaining profiles run a Z-Image face stage on the DiffSynth runtime, so
|
||||
both need this extra. It includes the `diffusion` dependencies; `diffusion` on its
|
||||
own covers the torch and diffusers imports but not the face stage, so it is not
|
||||
enough to run a removal.
|
||||
both need this extra. The third profile, `chroma-zimage`, uses diffusers'
|
||||
ChromaImg2ImgPipeline instead of DiffSynth for its global stage but inherits
|
||||
the same face stage, so it needs this extra too. The extra also installs
|
||||
SentencePiece for Chroma1-HD's binary `spiece.model`; without that backend,
|
||||
Transformers cannot construct the Chroma tokenizer. It includes the `diffusion`
|
||||
dependencies; `diffusion` on its own covers the torch and diffusers imports but
|
||||
not the face stage or Chroma tokenizer, so it is not enough to run a removal.
|
||||
|
||||
**An NVIDIA GPU is required.** `qwen-zimage` and `sdxl-zimage` are CUDA-only, and
|
||||
**An NVIDIA GPU is required.** `qwen-zimage`, `sdxl-zimage` and `chroma-zimage`
|
||||
are CUDA-only, and
|
||||
construction refuses any other device rather than falling back to a slow or broken
|
||||
one. There is no CPU, MPS or XPU path for invisible-watermark removal. Visible-mark
|
||||
removal, metadata stripping and every `identify` command still run anywhere.
|
||||
@@ -103,17 +108,19 @@ application actually uses:
|
||||
|
||||
| Extra | Capability | Automatically includes | Torch or model download |
|
||||
| --- | --- | --- | --- |
|
||||
| `pixels` | Shared BGR array and image-processing runtime | NumPy, headless OpenCV | No |
|
||||
| `pixels` | Shared BGR runtime and calibrated-size SynthID carrier detection | NumPy, headless OpenCV | No |
|
||||
| `heif` | HEIC, HEIF, and AVIF pixel decoding | pillow-heif | No |
|
||||
| `visible` | Visible mark detection, OpenCV inpainting, and manual erasing | `pixels` | No |
|
||||
| `video` | Visible video identification/removal and timestamp preservation | `visible`, PyAV | No |
|
||||
| `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No |
|
||||
| `trustmark` | Adobe TrustMark detection on Python 3.11-3.12 | trustmark | Yes |
|
||||
| `classify` | Metadata-free photo AI-versus-camera classifier plus gated provider | `pixels`, Torch, Transformers | Yes |
|
||||
| `verify` | Official remote OpenAI SynthID verification | OpenAI SDK | No |
|
||||
| `diffusion` | Torch and Diffusers runtime; video SynthID regeneration | `pixels`, Torch, Diffusers | Yes |
|
||||
| `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
|
||||
| `qwen-zimage` | Invisible image-watermark removal, both CUDA-only profiles | `diffusion`, DiffSynth | Yes |
|
||||
| `text-restoration` | Opt-in verified Qwen-VAE glyph restoration | `qwen-zimage`, `lama` | Yes |
|
||||
| `qwen-zimage` | Invisible image-watermark removal, all CUDA-only profiles | `diffusion`, DiffSynth, SentencePiece | Yes |
|
||||
| `text-restoration` | Opt-in verified profile-VAE glyph restoration | `qwen-zimage`, `lama` | Yes |
|
||||
| `text-draft` | Draft OCR proposals for operator verification | PaddleOCR, PaddlePaddle | Model download, no Torch |
|
||||
| `all` | Every production feature available on the active Python | All compatible rows above | Yes |
|
||||
| `dev` | Tests, linting, typing, and upstream parity checks | `video`, `detect`, upstream invisible-watermark | Yes, for parity tests |
|
||||
@@ -134,9 +141,11 @@ flowchart LR
|
||||
draft["text-draft"]
|
||||
heif
|
||||
trustmark
|
||||
classify --> pixels
|
||||
verify
|
||||
```
|
||||
|
||||
`heif`, `trustmark`, and `text-draft` are independent branches. Combine them
|
||||
`heif`, `trustmark`, `verify`, and `text-draft` are independent branches. Combine them
|
||||
explicitly with another feature when required. `text-draft` is excluded from
|
||||
`all` because it proposes unverified OCR annotations and is not a production
|
||||
removal path. TrustMark requires NumPy 1.x, which has no
|
||||
@@ -159,6 +168,9 @@ uv tool install --force "remove-ai-watermarks[video]"
|
||||
# DWT-DCT and TrustMark detection without diffusion removal
|
||||
uv tool install --force "remove-ai-watermarks[detect,trustmark]"
|
||||
|
||||
# Official OpenAI SynthID verification
|
||||
uv tool install --force "remove-ai-watermarks[verify]"
|
||||
|
||||
# Every production capability compatible with this Python
|
||||
uv tool install --force "remove-ai-watermarks[all]"
|
||||
|
||||
@@ -171,6 +183,13 @@ do not install libheif. `detect` uses the in-tree torch-free decoder and does
|
||||
not install the upstream `invisible-watermark` package. Optional models download
|
||||
their weights on first use.
|
||||
|
||||
The `verify` extra makes an explicit remote request. The
|
||||
`verify-openai-synthid` command first removes AI provenance metadata from a
|
||||
temporary copy, checks that its decoded pixels are unchanged, and then uploads
|
||||
that copy to OpenAI. It needs `OPENAI_API_KEY`; the command never runs from
|
||||
`identify` and refuses to upload without `--acknowledge-upload`. The Python API
|
||||
requires the equivalent explicit `acknowledge_upload=True` argument.
|
||||
|
||||
The old `gpu` and `remove` aliases are intentionally not provided. Use
|
||||
`diffusion` and `visible` respectively.
|
||||
|
||||
@@ -241,5 +260,5 @@ image came from a relevant generator, use `--force`.
|
||||
|
||||
If the CLI reports that the removal dependencies are unavailable, install the
|
||||
`qwen-zimage` extra. `diffusion` alone covers Torch and Diffusers but not the
|
||||
DiffSynth face stage that both profiles run. Video SynthID removal is a separate
|
||||
DiffSynth face stage that all profiles run. Video SynthID removal is a separate
|
||||
path and needs `video` and `diffusion`.
|
||||
@@ -4,6 +4,18 @@ This page describes current product limits. Historical measurements and
|
||||
superseded experiments live in the research archive listed in
|
||||
[the documentation index](index.md).
|
||||
|
||||
## Pixel classification
|
||||
|
||||
`classify` / `classify_pixels` is AI versus camera, not a universal AI detector.
|
||||
Receipts, UI screenshots, and digital art are out of contract: a thermal receipt
|
||||
can score as AI, a photorealistic AI receipt as human. POSSIBLY is reported as
|
||||
`unknown`. Provider attribution runs only after DEFINITELY and can abstain when
|
||||
124-d extraction refuses the file. Microsoft is not a pixel class.
|
||||
|
||||
The command is never started by `identify`. A no-signal provenance result stays
|
||||
unknown. This is not a clean verdict and it does not run cleanup. Full guide:
|
||||
[photo pixel classification](photo-classify.md).
|
||||
|
||||
## Visible removal
|
||||
|
||||
### Fill quality depends on the background
|
||||
@@ -73,10 +85,12 @@ as SynthID lives in the pixels everywhere, including inside the glyphs, so
|
||||
frozen text regions would keep the watermark
|
||||
([text protection research](text-protection-research.md)).
|
||||
|
||||
`qwen-zimage` is the default profile and `sdxl-zimage` the only alternative.
|
||||
Both are CUDA only and differ only in the global regeneration model: each
|
||||
conditions that stage on a canny edge map, which preserves structure but not
|
||||
identity or exact texture, and each then runs the same face stage.
|
||||
`qwen-zimage` is the default profile. `sdxl-zimage` and `chroma-zimage` keep
|
||||
the same face stage and swap the global model. `auto` picks chroma-zimage for
|
||||
OpenAI and Microsoft provenance and qwen-zimage otherwise. All are CUDA only.
|
||||
Qwen and SDXL condition the global stage on a canny edge map, which preserves
|
||||
structure but not identity or exact texture. Chroma1 is a plain strength pass
|
||||
without Canny.
|
||||
Existing face evaluations favor `qwen-zimage`, but there is no blanket fidelity
|
||||
ordering across content types. A fixed-seed, three-scene text comparison at the
|
||||
profile defaults found no stable winner: SDXL won one poster, Qwen won one, and
|
||||
@@ -97,12 +111,14 @@ rates are kept in
|
||||
[`text-protection-research.md`](text-protection-research.md) and the
|
||||
[`fidelity` evaluation record](../data/evaluations/fidelity/README.md).
|
||||
|
||||
### Removal cannot be verified locally for proprietary SynthID
|
||||
### There is no local SynthID pixel detector in the package
|
||||
|
||||
The project has no public local SynthID pixel decoder. It recognizes presence
|
||||
from supported provenance metadata: Google AI C2PA under Google's all-media
|
||||
policy, or current OpenAI C2PA with an explicit watermark action. A missing
|
||||
provenance signal is not a negative pixel verdict.
|
||||
Google does not publish the proprietary SynthID payload decoder, and the
|
||||
package does not ship one. Signed provenance is the supported route:
|
||||
Google AI C2PA or current OpenAI C2PA with an explicit watermark action.
|
||||
`verify-openai-synthid` is the official remote pixel check for OpenAI.
|
||||
Research on a periodic lattice expert is in
|
||||
[synthid-detector-research.md](synthid-detector-research.md).
|
||||
|
||||
For important outputs:
|
||||
|
||||
@@ -185,7 +201,7 @@ path has not yet answered.
|
||||
|
||||
### Strength is content and seed dependent
|
||||
|
||||
The two profiles resolve an unset strength differently, because different things
|
||||
The profiles resolve an unset strength differently, because different things
|
||||
were measured for each.
|
||||
|
||||
`qwen-zimage` reads unknown content from the resolution-adaptive denoise curve.
|
||||
@@ -208,10 +224,25 @@ values are flat rather than a curve because flat values are what was measured: e
|
||||
verdict came from a fixed strength at one size, and no size dependence has been
|
||||
established for that stage.
|
||||
|
||||
An explicit `--strength` overrides both. The defaults are operating points, not
|
||||
universal guarantees. Near a removal threshold, different content or a different
|
||||
random seed may change the verifier result, which is why both profiles are
|
||||
certified at a fixed seed. The live resolver is
|
||||
`chroma-zimage` also reads a flat ladder from the provenance cohort, measured
|
||||
2026-08-29/30 across the four vendor oracles (full record:
|
||||
[`chroma1-engine-research.md`](chroma1-engine-research.md)):
|
||||
|
||||
- OpenAI: `0.09` (below qwen's `0.07675` is false economy neither way -- the
|
||||
matched-strength fidelity favors Chroma1 on this cohort);
|
||||
- Microsoft InvisMark: `0.125` (below qwen's `0.15`);
|
||||
- Google: `0.40` for zero-face content (above qwen's `0.27`; at this floor the
|
||||
regeneration destroys dense text and collapses face identity -- the tradeoff
|
||||
that motivates the adaptive arm below), or `0.125` when YuNet detects at
|
||||
least one face (both measured face fixtures clear at 0.12 with zero
|
||||
cross-source spread; oracle-verified 2026-08-30);
|
||||
- Meta Content Seal: `0.17` (above qwen's `0.1`);
|
||||
- unknown: `0.40`, following the strictest measured cohort.
|
||||
|
||||
An explicit `--strength` overrides all three. The defaults are operating points,
|
||||
not universal guarantees. Near a removal threshold, different content or a
|
||||
different random seed may change the verifier result, which is why the profiles
|
||||
are certified at a fixed seed. The live resolver is
|
||||
[`watermark_profiles.py`](../src/remove_ai_watermarks/_internal/watermark_profiles.py).
|
||||
|
||||
### Pipelines have different quality tradeoffs
|
||||
@@ -220,6 +251,8 @@ certified at a fixed seed. The live resolver is
|
||||
| --- | --- |
|
||||
| `qwen-zimage` | CUDA only, large model stack, and limited broad certification across seeds and content. |
|
||||
| `sdxl-zimage` | CUDA only. Its strength ladder is flat per vendor, not a resolution curve, because flat values are what was measured. |
|
||||
| `chroma-zimage` | CUDA only. Higher Google and Meta floors than qwen; Google face content uses 0.125 instead of 0.40. |
|
||||
| `auto` | CUDA only. Routes OpenAI and Microsoft to chroma-zimage and everything else to qwen-zimage. |
|
||||
|
||||
Only manually verified `vae-glyphs` is an optional production stage, and it is
|
||||
experimental rather than a default.
|
||||
@@ -241,8 +274,9 @@ per-stage schedule, CFG 1.0 and CUDA.
|
||||
### Small images are processed at their native size
|
||||
|
||||
There is no minimum-resolution floor. It existed to enlarge small inputs toward
|
||||
SDXL's ~1024 training resolution and was removed with the SDXL profiles, which
|
||||
never applied it anyway. Both surviving profiles run at native geometry, so a
|
||||
SDXL's ~1024 training resolution and was removed with the standalone SDXL and
|
||||
ControlNet profiles, which never applied it anyway. The surviving `sdxl-zimage`
|
||||
is a different profile and is unaffected. All three surviving profiles run at native geometry, so a
|
||||
small input is neither enlarged before diffusion nor restored afterward.
|
||||
|
||||
`--max-resolution` still caps very large inputs, and only ever scales down.
|
||||
@@ -272,7 +306,7 @@ device and re-reads every parameter from disk on each stage transition.
|
||||
|
||||
Invisible-watermark removal refuses any device but CUDA at construction. There is
|
||||
no MPS out-of-memory fallback that continues on CPU, and no lighter profile to
|
||||
drop to: both remaining profiles need an NVIDIA GPU.
|
||||
drop to: all three profiles need an NVIDIA GPU.
|
||||
|
||||
When a run does not fit, the levers are `--tile` (native geometry, tiled
|
||||
diffusion), `--max-resolution` (an explicit downscale), and `--cpu-offload`.
|
||||
@@ -426,9 +460,46 @@ with plain `torch.load`. Bump lightning and cut a patch release when a fix
|
||||
ships. The calibration history is in
|
||||
[module internals](module-internals.md#metadata-and-provenance).
|
||||
|
||||
External AI versus real image classifiers are out of scope. The project
|
||||
identifies concrete local provenance signals instead of shipping a generic
|
||||
statistical classifier.
|
||||
A generic metadata-free AI-generated-image classifier is not shipped. It is a
|
||||
separate open research task from provenance detection: the current Model 1
|
||||
result is AI-versus-camera and still confuses some conventional graphics, CGI,
|
||||
product cutouts, and scans with generation. The OpenAI/Gemini source finder is
|
||||
narrower again and cannot substitute for the general classifier. The shipped
|
||||
product identifies concrete local provenance signals instead of presenting
|
||||
either research classifier as a supported verdict.
|
||||
|
||||
Frozen transfers of Community Forensics, SPAI, SAFE, RINE, Nonescape Mini,
|
||||
Dual Data Alignment, PGC, and DGS-Net did not close this gap at the required low
|
||||
false-positive rate. DDA supplies a complementary representation but its
|
||||
independent errors make simple hybrids worse. PGC SD1.4 is strong on OpenAI,
|
||||
but its published output and the useful global/residual ablations misclassify
|
||||
most or all of the Kodak scan set; removing that branch removes the OpenAI
|
||||
gain. DGS-Net is weak at the same operating point and also adds independent
|
||||
photo errors. A source-disjoint public-domain museum audit confirmed a separate
|
||||
human-art failure: Model 1 called 355/1,050 drawings, prints, manuscripts,
|
||||
paintings, textiles, and studio-object or historical-photo rows AI-generated.
|
||||
Its error was 0/150 on the historical-photo control, so ordinary camera accuracy
|
||||
does not bound accuracy on other human-created visual media. A domain veto
|
||||
removed the museum errors in development but failed the existing AI-test and
|
||||
fresh-photo core. A second source-disjoint modern-negative audit measured the
|
||||
commercial cells: across 421 date-clean Unsplash and CC-licensed Flickr images,
|
||||
Model 1 accepted 16.4% as AI, concentrated in logo/graphic design (38.3%),
|
||||
retouched fashion photography (34.1% on the Unsplash cell; 2/40 on the Flickr
|
||||
replication), and product cutouts (16.0%), while composites
|
||||
stayed clean at 0/50 and long exposure near clean at 11/146 after a
|
||||
two-part contamination control (a 0/450 provenance-metadata scan and a
|
||||
pre-2022-08 date bound that dropped 29 rows, correcting one contaminated
|
||||
Flickr fashion cell). The failure pattern therefore spans both historical art
|
||||
and modern stylized graphics, not photographic exposure technique. A
|
||||
source-disjoint linear veto fit on the date-clean modern cells was then
|
||||
rejected one gate earlier: no searched operating point repaired even one
|
||||
modern dev error without dropping below the frozen EvalGEN and FLUX floors,
|
||||
because modern human graphics and AI generations overlap almost one-to-one in
|
||||
the frozen CLIP embedding, at roughly 1.3 to 1.5 AI positives lost per negative
|
||||
repaired. Model 1 also lacks a time/device-disjoint negative contract
|
||||
for modern computational photography. The measured public protocol, GitHub
|
||||
survey, and rejected fusions are recorded in
|
||||
[AI-generated image classifier research](ai-generated-image-classifiers.md#general-ai-classifier-github-sweep-2026-08-25).
|
||||
|
||||
## Output and traceability
|
||||
|
||||
|
||||
@@ -11,7 +11,8 @@ updated in the same change.
|
||||
|
||||
## Architecture
|
||||
|
||||
The package has four main paths:
|
||||
The package has four main paths, plus an opt-in pixel classifier that is not
|
||||
on the provenance or removal graph:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -19,13 +20,18 @@ flowchart LR
|
||||
Input --> Visible[Visible mark removal]
|
||||
Input --> Invisible[Diffusion regeneration]
|
||||
Input --> Metadata[Metadata stripping]
|
||||
Input --> Classify[classify_pixels]
|
||||
|
||||
Identify --> Report[ProvenanceReport]
|
||||
Visible --> VisibleOutput[Localized and filled image]
|
||||
Invisible --> InvisibleOutput[Regenerated image]
|
||||
Metadata --> MetadataOutput[Container with AI metadata removed]
|
||||
Classify --> PixelClassification
|
||||
```
|
||||
|
||||
`identify` does not call `classify_pixels`. `has_invisible_target` and `all`
|
||||
do not read it.
|
||||
|
||||
The `all` command runs visible removal, optional invisible regeneration, and
|
||||
metadata stripping in that order.
|
||||
|
||||
@@ -296,7 +302,7 @@ from the gate. The full-clip oracle floor is
|
||||
detected while `0.15` did not.
|
||||
|
||||
[`video_visible.py`](../src/remove_ai_watermarks/video_visible.py) implements
|
||||
the first pixel stages for Sora, Veo, Seedance, Dola, Hailuo AI, and Kling AI. The
|
||||
the first pixel stages for Sora, Veo, Seedance, Doubao, Dola, Hailuo AI, and Kling AI. The
|
||||
Sora detector searches a normalized frame with a fully synthetic
|
||||
mascot-and-text silhouette at several scales. The Veo detector uses separate
|
||||
synthetic silhouettes for the current four-point diamond and legacy `Veo`
|
||||
@@ -327,8 +333,8 @@ carries `accepts_provenance`, which forces `provenance=False` for Hailuo AI and
|
||||
— they have no metadata that could confirm them, and the guarantee used to be
|
||||
structural (their wrappers took no `provenance` parameter at all). Provenance can relax a low-contrast run only
|
||||
after recurring visual evidence exists. Sora transition frames follow the
|
||||
nearest confirmed moving position only with Sora provenance. Seedance, Dola,
|
||||
Hailuo AI, and Kling AI additionally require candidates to remain anchored to the
|
||||
nearest confirmed moving position only with Sora provenance. Seedance, Doubao,
|
||||
Dola, Hailuo AI, and Kling AI additionally require candidates to remain anchored to the
|
||||
start of a run. This rejects slowly drifting scene details that still have
|
||||
high frame-to-frame overlap. Hailuo AI and Kling AI do not infer provenance from
|
||||
technical encoder tags; their confirmed public samples carried no provider
|
||||
@@ -533,7 +539,382 @@ metadata extraction from verdict logic:
|
||||
both extractors reach the same answer. It did not, and the record path silently
|
||||
reported no SynthID for images the file path flagged.
|
||||
- `identify` preserves the path-based API and adds the optional registered
|
||||
visible-mark and open invisible-watermark decoders after extraction.
|
||||
visible-mark and open invisible-watermark detectors after extraction.
|
||||
|
||||
### SynthID periodic carrier detector (research only)
|
||||
|
||||
The local lattice expert is not part of the public package. Runtime code lives
|
||||
in [`scripts/synthid_runtime/`](../scripts/synthid_runtime/) and the campaign
|
||||
log is [`synthid-detector-research.md`](synthid-detector-research.md). The
|
||||
notes below are the calibration history of that research expert.
|
||||
|
||||
[`synthid_detector.py`](../scripts/synthid_runtime/synthid_detector.py) is the
|
||||
runtime form of the frozen 2048x2048 periodic-tile experiment. It folds a
|
||||
Gaussian high-pass residual modulo 16x16 within a calibrated pixel-count range
|
||||
and compares the normalized RGB tile with the bundled float64 template
|
||||
`scripts/synthid_runtime/synthid_periodic_tile_2048_v1.npz`. Exact multiples use the original
|
||||
reshape-and-mean path; other sizes use count-correct modulo folding,
|
||||
without resize. Channels are filtered and folded sequentially, and partial edge
|
||||
blocks are accumulated without a full-frame padding buffer so the 18-megapixel
|
||||
ceiling does not require multiple three-channel float workspaces. The model hash
|
||||
is pinned by a test, and the unchanged operating threshold is
|
||||
`0.17357069773071196` through 10 megapixels.
|
||||
|
||||
That fixed threshold is not production-qualified. A later source-fresh
|
||||
Open Images test-split challenge produced 5 crossings among the 211 images in
|
||||
its supported geometry. A precision-first `0.28` replacement is frozen as a
|
||||
research candidate after retaining all 12 available source-diverse native
|
||||
positives and rejecting those five crossings. It then failed the second
|
||||
untouched holdout at 1/213, with score `0.322542963`. The replacement is
|
||||
rejected and fixed-v2 remains an explicit diagnostic only.
|
||||
|
||||
The direct API returns `detected`, `indeterminate`, or `unsupported`. Passing
|
||||
`register_scale=False` selects fixed-v2 from 1,000,000 through 10,000,000
|
||||
decoded pixels as an explicit diagnostic. Its frozen threshold accepted
|
||||
none of 5,000 public COCO views balanced across every observed target geometry,
|
||||
and none of a separate 5,000-view challenge over 256 generated geometries
|
||||
covering every pair of modulo-16 edge remainders. The original 2048x2048
|
||||
verdicts and exact scores remain unchanged. Runtime matches do not attribute a
|
||||
provider. `identify` adds only positive matches as high-confidence
|
||||
evidence and never turns a local negative into a clean verdict.
|
||||
The result envelope also names the signal family, provider scope, backend,
|
||||
whether metadata contributed to the verdict, whether pixels were preserved,
|
||||
and an explicit reason for unsupported or indeterminate results. These fields
|
||||
are shared with the official OpenAI verifier's JSON boundary.
|
||||
|
||||
The production router selects `synthid-periodic-tile-large-v1` above 10 through 18
|
||||
megapixels when both dimensions are at least 2,048 pixels. It evaluates all
|
||||
phase-aligned 2,048-square windows and combines the minimum fixed-template,
|
||||
Red-minus-Green, and Blue-minus-Yellow spatial correlations with the most
|
||||
negative Blue-minus-Yellow mid-band correlation. The 3072x5504 portrait
|
||||
geometry also applies a Green mid-band alias veto. Each component is normalized
|
||||
to its frozen gate and the public threshold is `1.0`.
|
||||
|
||||
All 37 inferred large candidates cross the rule, and all seven metadata-free,
|
||||
pixel-identical candidates checked by the official Gemini verifier were
|
||||
detected. The constants rejected all 17,417 exposed external controls. A
|
||||
post-freeze production-path challenge then rejected all 2,637 decoded-pixel-
|
||||
unique controls drawn from 2,000 COCO images excluded from the earlier large
|
||||
color-phase challenge and 637 deduplicated Picsum controls. Four large
|
||||
geometries and four resampling kernels were balanced; the maximum score was
|
||||
`0.0592777965`. The source collections were not freshly acquired, so this is a
|
||||
feature-unseen holdout rather than a fresh-source estimate.
|
||||
|
||||
A separate post-freeze Open Images download yielded 41 completed,
|
||||
decoded-pixel-unique controls after excluding incomplete `.aria2` files and all
|
||||
prior Open Images hashes. The frozen production path accepted 0/41 and reached
|
||||
a maximum score of `0.4083013324`. This source-fresh audit is too small to
|
||||
replace the main holdout interval but checks the acquisition boundary.
|
||||
|
||||
The same seven official positives were then re-encoded at unchanged dimensions.
|
||||
JPEG-95 and JPEG-90 each reduced detection from 7/7 native files to 0/7. The
|
||||
large operating point is therefore native-pixel and lossless-copy support, not
|
||||
a codec-robust claim.
|
||||
|
||||
Arbitrary geometry is not the same as arbitrary spatial resampling. On a
|
||||
stratified 80-image fixed-positive sample, one-step resizes at seven nonidentity
|
||||
scales from 0.5 through 1.5 reduced the unchanged 16x16 detector from 80 accepted
|
||||
sources to zero at every scale. Restoring the original geometry recovered 58-80
|
||||
sources, showing that the carrier period scaled with the pixels. A discovery
|
||||
bank that scaled the template to integer periods 8, 10, 12, 14, 18, 20, and 24
|
||||
was promising: a threshold frozen above 3,000 resized COCO controls accepted no
|
||||
view in a 2,000-control final partition and accepted 672 of 800 source-disjoint
|
||||
provider positives. It also stayed below threshold on the tracked OpenAI and
|
||||
Adobe controls. The branch remains research-only because noninteger periods at
|
||||
scales 0.8, 0.9, 1.1, 1.2, and 1.333 collapsed, while separate per-period
|
||||
thresholds accepted five final controls. The runtime therefore keeps only the
|
||||
fixed 16-pixel lattice.
|
||||
|
||||
A follow-up fractional-period probe sampled the 30 strongest template harmonics
|
||||
over a continuous 7.5-24.5 period range. The correct period appeared within
|
||||
0.05 pixels among the top three candidates for 58 of 60 transformed positives.
|
||||
Testing nine neighboring reconstructed geometries then recovered 44 of 60 at
|
||||
the native threshold, compared with an upper bound of 48 when the true source
|
||||
geometry was supplied. The complete search still failed its small frozen
|
||||
control split: a threshold above 250 development controls accepted two of 150
|
||||
final controls. Multiplying the canonical score by spectral-period confidence
|
||||
also accepted two. That baseline was rejected rather than shipped at its
|
||||
discovery threshold.
|
||||
|
||||
The research-only
|
||||
[`synthid_affine_lattice_probe.py`](../scripts/synthid_affine_lattice_probe.py)
|
||||
adds split-confirm synchronization. It estimates complex harmonic coherence on
|
||||
one checkerboard of patches, confirms the selected period on the other, and
|
||||
reports amplitude-aware confirmation, a locally content-whitened multichannel
|
||||
code match, and phase-preserving and cyclically registered template scores. A
|
||||
0.1-pixel grid recovered the expected
|
||||
12.8, 14.4, 17.6, and 19.2 periods in 20/20 transformed views from five official
|
||||
positive parents. A provisional conjunction accepted none of 800 corresponding
|
||||
views from 200 oracle-negative parents, none of 469 OpenAI-labeled rows, and
|
||||
three of 276 broad non-Google rows whose TC260 or Samsung provenance prevents
|
||||
treating them as clean oracle negatives. This probe is not runtime routing:
|
||||
positive diversity is still inadequate, period 8 remains rejected, and the
|
||||
amplitude stage currently handles isotropic scale at zero rotation only. Full
|
||||
protocol and caveats are in the detector research plan.
|
||||
|
||||
The whitened match uses neighboring noncarrier bins to estimate complex
|
||||
Green/opponent-color covariance around every selected harmonic. It corrected a
|
||||
period-24 alias on one native official positive, giving the broad native search
|
||||
the correct period 16 on all five ordinary-size positives. It was retained only
|
||||
as a candidate reranker: its positive-to-negative margin was smaller than the
|
||||
existing spatial-template margin in both the native pilot and a locked 0.8
|
||||
resize challenge. Two of the three TC260/Samsung-provenance challenge crossings
|
||||
also matched the full whitened code more strongly than the weakest official
|
||||
large positive, so the pixel result identifies a compatible signal family, not
|
||||
a provider.
|
||||
|
||||
The same probe also exposes a payload-agnostic H5 confirmation. It estimates a
|
||||
complex harmonic vector independently on the two checkerboards and measures
|
||||
their fixed relative-phase inner product; the codeword need not match the known
|
||||
template. Free cyclic-shift selection was rejected because natural phase aliases
|
||||
overlapped the controls. The fixed all-harmonic statistic separated five native
|
||||
official positives from 130 controls with margin `0.1619`, and five 0.8-resized
|
||||
views from 200 fresh-parent controls with margin `0.1541`; all seven large
|
||||
official positives also passed the observed gap. It remains research-only
|
||||
because these tests supplied the period, reuse one negative source family, and
|
||||
contain only 12 independent positive parents.
|
||||
|
||||
The separate research-only
|
||||
[`synthid_cyclostationary_probe.py`](../scripts/synthid_cyclostationary_probe.py)
|
||||
measures full complex cross-channel spectral correlation at carrier shifts
|
||||
against neighboring-shift same-image nulls. It recognizes synthetic amplitude
|
||||
modulation and rejects both independent noise and a purely additive carrier.
|
||||
The real native pilot overlapped completely: five official-positive joint
|
||||
contrasts ranged from `-0.0027` to `0.0337`, while 130 controls reached `0.0593`.
|
||||
The cyclostationary branch is rejected rather than tuned on new frequency bands.
|
||||
|
||||
The affine-lattice research CLI also supports symmetric in-memory JPEG and
|
||||
top-left crop challenges. At fixed native period 16, codec-specific
|
||||
phase-preserving template thresholds retained all five repeated positive views
|
||||
and accepted none of 80 locked controls at JPEG qualities 95, 90, and 80. The
|
||||
locked margins declined from `0.1054` to `0.0257`; whitened and unknown-codeword
|
||||
scores were rejected for lossy JPEG. A separate registered crop conjunction
|
||||
retained all five views and accepted none of 80 controls after both 5% and 10%
|
||||
top-left crops, with registered-template margins above `0.22`. These are
|
||||
research pilots, not runtime experts: the period was supplied, positive parents
|
||||
were reused, and only one codec implementation and one negative family were
|
||||
tested.
|
||||
|
||||
The `register_scale=True` path is the calibrated ordinary-size production
|
||||
expert. It
|
||||
keeps the harmonic search, adds three-level scale-space consistency and quadrant
|
||||
agreement. Its calibrated decision requires the canonical winner to be the
|
||||
strongest spectral-period candidate, its three-way amplitude to cross the
|
||||
threshold for that period bin, and both independent high-frequency template
|
||||
bands to correlate by at least `0.075`. The public registered score is the
|
||||
minimum normalized margin across those gates, so its decision threshold is
|
||||
`1.0`.
|
||||
|
||||
The earlier single-threshold version produced 68 false positives among 11,273
|
||||
symmetrically downscaled Spaces controls. A top-candidate plus period-bin version
|
||||
removed those but still produced 6 false positives among 1,000 source-independent
|
||||
Open Images controls. The high-band gate removed them and then accepted none of
|
||||
499 previously untouched Open Images reserve controls. The resulting rule
|
||||
retained 229 of 355 source-disjoint transformed positives: 0/65 at scale 0.5 and
|
||||
229/290 from scale 0.65 through 1.5. The explicit period-8 rescue is rejected
|
||||
because resize lattices fully overlap its positive distribution.
|
||||
|
||||
A subsequently frozen 1,000-image Open Images reserve accepted zero in
|
||||
registered mode. The fixed expert supported only 81 of those geometries and
|
||||
accepted seven, so fixed and registered results cannot safely be unioned. In
|
||||
overlapping geometry the registered decision remains the validated path;
|
||||
fixed-only evidence is a diagnostic rather than a universal-cascade positive.
|
||||
|
||||
A later source-fresh 3,000-image Open Images test-split challenge superseded the
|
||||
registered-v2 precision claim: it accepted 4 of 2,997 supported controls. The
|
||||
registered-v3 confirmation uses disjoint patch groups for phase, cyclic
|
||||
amplitude, and held-out-codeword evidence. It rejected all four crossings,
|
||||
retained 147 of 148 base-accepted source-diverse positives, and retained all 359
|
||||
base-accepted views in a dense 0.65-1.50 transform matrix over 12 independent
|
||||
parents. Frozen unchanged, it then accepted 0/2,996 controls from a second
|
||||
nonoverlapping Open Images cohort and 0/2,366 supported controls from a
|
||||
3,000-image COCO second-family challenge. Registered-v3 is now the default
|
||||
ordinary-size positive route. The exact gates, acquisition hashes, and rejected
|
||||
weak-signal rescue are recorded in the detector research plan.
|
||||
|
||||
Every one of those control rates is photographic. Against 223 corpus images
|
||||
whose C2PA names a non-Google generator, the unchanged entry point accepted 29
|
||||
(`0.130`, Adobe Firefly `0.241`, highest foreign score `3.01`), all from
|
||||
registered-v3. The branch reads a lattice shared across generation pipelines,
|
||||
which is why it must not be reported as a watermark. The public `identify`
|
||||
path no longer calls this expert.
|
||||
|
||||
The branch is also phase-locked to the image origin, exactly like the large
|
||||
expert. A two-pixel diagonal crop killed all 28 in-geometry foreign detections
|
||||
and all 8 detected Google provenance positives (maximum remaining scores
|
||||
`0.779` and `0.311` against the `1.0` threshold); the signal recovers only at
|
||||
offsets that are multiples of four. Registered-v3 therefore detects the same
|
||||
crop-destroyed generation-pipeline lattice as `large-v1`, on ordinary sizes.
|
||||
|
||||
When registered-v3 abstains, opponent-registered-v1 searches the same frozen
|
||||
template in Red-minus-Green and Blue-minus-Yellow space. It reranks three
|
||||
separated scale candidates with fixed RGB and two spatial opponent-color gates,
|
||||
then accepts only periods 7.9-12.0 on 1-10 megapixel rasters whose sides are at
|
||||
least 768 pixels. Period-8 candidates additionally require Red-Green and
|
||||
Blue-Yellow 8-pixel edge ratios no greater than `1.05`; this vetoes the
|
||||
deterministic JPEG block lattice without using container metadata. The final
|
||||
rule recovered 49/49 lossless 0.5x-0.75x views from seven official-positive
|
||||
parents. It rejected all 1,790 measured period-8 codec crossings, while 350
|
||||
identically resized controls had no base crossing and the earlier period-band
|
||||
rule accepted 0/1,000 post-freeze Picsum controls. Period 12.8 remains excluded,
|
||||
and lossy JPEG/WebP views remain inconclusive.
|
||||
|
||||
The runtime precedence is registered-v3, the bounded opponent fallback, then
|
||||
large-v1 above 10 megapixels. Passing `register_scale=False` selects the legacy
|
||||
fixed diagnostic explicitly. The research bank in
|
||||
`scripts/synthid_routed_expert_bank.py` keeps all three observations for audits.
|
||||
Neither runtime nor research routing returns a clean-image verdict.
|
||||
|
||||
An unchanged registered challenge from 10 to 18 megapixels retained only 1 of
|
||||
37 Google candidates and zero of 89 non-Google controls. Twenty-eight positives
|
||||
cleared amplitude, 21 had matching spatial and spectral periods, but only three
|
||||
cleared high-band agreement. The 10-megapixel ceiling therefore remains.
|
||||
Phase-aligned 2,048-square fixed windows did not provide a fallback: median
|
||||
consensus retained 36 positives and accepted 10 controls, while all-window
|
||||
consensus retained 36 and accepted eight.
|
||||
One frozen full-frame pre-resize to eight megapixels also retained only the same
|
||||
1 of 37 positives and zero controls; just three positives cleared high-band
|
||||
agreement. Large images therefore cannot be routed through a canonical-size
|
||||
registered fallback; the later native opponent-color branch is separate.
|
||||
The remaining phase-aligned window variants closed this branch: a single center
|
||||
2,048-square registered window retained 1 of 37 positives and zero of 89
|
||||
controls, while accepting any phase-aligned 2,048-square window retained 2 of
|
||||
37 and zero controls. The latter control maximum was already 0.968 against the
|
||||
1.0 decision threshold. Neither the coverage nor the exposed specificity
|
||||
margin supports a registered-window expert; these results do not apply to the
|
||||
later native opponent-color branch.
|
||||
|
||||
A separate half-scale patch-consensus branch initially looked promising. Its
|
||||
64-pixel, 90th-percentile patch statistic retained 33 of 49 validation positives
|
||||
and zero of 166 controls, then 27 of 52 locked-test positives and zero of 140
|
||||
controls. The frozen broad challenge rejected it: 61 of 10,906 symmetrically
|
||||
half-scaled Spaces controls and 22 of 1,000 independent Open Images controls
|
||||
crossed the unchanged threshold. A post-challenge Green negative-phase spread
|
||||
gate removed those errors but retained only 2 of 234 later Google rows and zero
|
||||
of 689 later controls. Half-scale support therefore remains explicitly absent.
|
||||
|
||||
An independent reproduction of the public `aloshdenny/reverse-SynthID` V4
|
||||
cross-color codebook did not provide another expert. Its best-of-two-model phase
|
||||
rule retained 141 of 355 positives but accepted 191 of 499 controls and 386 of
|
||||
a fresh 1,000-control reserve; AUC was 0.517. The repository's stricter
|
||||
aspect-ratio routing reduced coverage but preserved the same approximately 38%
|
||||
positive and control acceptance within supported rows. Its older V3 phase score
|
||||
had AUC 0.473, and two prespecified amplitude-aware V4 rerankings reached only
|
||||
0.521 and 0.524 AUC. These external rules are research baselines only. Their useful
|
||||
contribution is the solid-reference phase-consensus construction, not either
|
||||
published single-image threshold. Full methodology and the DALL-E reference-set
|
||||
confound are recorded in the detector research plan.
|
||||
A later exact repeat on the hash-disjoint v7 challenge confirmed the same
|
||||
failure: the published cut accepted 77/200 Google and 76/200 OpenAI, so it did
|
||||
not separate the two target providers.
|
||||
A direct exact-1024 reproduction of the same V4 artifact confirmed rather than
|
||||
rescued that verdict: the union of its two published profiles accepted 177 of
|
||||
443 Google rows and 60 of 162 controls. The artifact's floor left only about two
|
||||
active bins in the inspected profile/channel, explaining why its score acts
|
||||
as a weak generic phase-coherence test rather than a specific codebook match.
|
||||
|
||||
An exact-1086x1448 OpenAI follow-up also rejected the solid-reference transfer.
|
||||
A low-texture 256-bin phase codebook achieved native and JPEG-95 test AUCs of
|
||||
0.512, and independent train halves shared only two exact bins. The paper's
|
||||
content-dependent alternative was tested with 60 OKLab masking-alignment
|
||||
features. Its native/JPEG minimum reached test AUC 0.593 and 15/562 positives
|
||||
versus 1/228 controls at the frozen boundary, but deterministic half-fits shared
|
||||
no positive decision and had coefficient cosine 0.148. Neither branch is a
|
||||
runtime OpenAI expert.
|
||||
|
||||
An exact-1254 assertion-contrast search selected an sRGB period-8 Blue-channel
|
||||
direction at 45 of 259 validation asserted rows and zero of 16 same- or
|
||||
other-provider validation rows. It failed its locked specificity check at 39 of
|
||||
233 asserted rows and one of 18 controls. Across the complete exact-geometry
|
||||
audit it accepted 185 of 992 asserted rows, one of 55 same-provider rows without
|
||||
an assertion, and one of 13 other-provider rows. Raising the threshold above
|
||||
the wider controls reduced recall to a small minority, so this branch remains
|
||||
research-only.
|
||||
|
||||
Three attempted veto or missing-codeword rescues also failed. The official
|
||||
InvisMark decoder checkpoint passed its own embedded-watermark self-test at
|
||||
0.997 confidence and 0.97 bit accuracy, but all OpenAI and provider-control
|
||||
cohorts clustered around 0.18 confidence with no useful separation. DTCWT
|
||||
modulus before periodic folding retained 5 of 158 locked-test OpenAI rows and
|
||||
zero of 135 controls, then zero of 40 fresh OpenAI rows. A 16-codeword whitened
|
||||
period-8 model retained 5 of 158 and zero controls, then 1 of 40 fresh OpenAI
|
||||
rows and 1 of 50 fresh controls. They neither explain the known oracle-positive
|
||||
miss nor justify runtime routing.
|
||||
|
||||
An image-level patch-bag model over nine normalized high-pass patches was the
|
||||
strongest later local OpenAI classifier. It accepted 324/1,364 development-test
|
||||
OpenAI rows with 4/3,373 heterogeneous controls, then transferred to 120/486
|
||||
OpenAI rows, 8/504 general controls, and 5/33 same-provider non-assertion rows
|
||||
in the later temporal challenge. Assertion enrichment over the same-provider
|
||||
stratum was not independently significant (`p = 0.151`, one-sided exact test).
|
||||
JPEG-95 reduced the unchanged conjunction to
|
||||
1/486 positives and zero controls. The native temporal hits were complementary
|
||||
to the signed period-8 hits, but the same-provider rate and codec collapse
|
||||
identify another export noiseprint rather than a runtime SynthID expert.
|
||||
|
||||
The separate OpenAI period-8 DTCWT component is persistent rather than tied to
|
||||
one short rollout: exact-generator asserted hits were 3/16, 46/365, and 26/200
|
||||
from May through July, with 75/581 overall versus 1/52 same-generator rows
|
||||
without an assertion. Its native/JPEG minimum score reached 0.721 AUC between
|
||||
those indeterminate strata. Sorting all 64 cyclic correlation scores removed
|
||||
absolute phase but also removed locked-test discrimination at 0/158 positives
|
||||
and 0/135 controls. This remains research evidence for a weak signed carrier,
|
||||
not a runtime OpenAI detector.
|
||||
|
||||
A four-family open-proxy challenge also failed to justify a generic neural
|
||||
watermark expert. A fixed residual frontend and cross-family residual mixing
|
||||
were trained on three of TrustMark P, VideoSeal, DWT-DCT, and WAM while the
|
||||
fourth encoder and its test sources remained unseen. Held-out AUCs ranged from
|
||||
0.437 to 0.562. Equal-power phase-scrambled hard negatives prevented simple
|
||||
spectral-energy shortcuts, but did not produce architecture transfer. A
|
||||
separate translation-invariant Gemini bicoherence search selected none of 20
|
||||
development positives and finished at 0/50 positives, 0/199 controls, and AUC
|
||||
0.374. Neither branch is part of runtime routing; full split and oracle details
|
||||
are in the detector research plan.
|
||||
|
||||
The separately measured registered geometry range remains 250,000 through
|
||||
10,000,000 decoded pixels with both sides at least 256 pixels. The default path
|
||||
and `identify` use registered-v3, then the narrower opponent-registered-v1
|
||||
fallback in its 1-10 megapixel domain, and large-v1 above 10 megapixels. A
|
||||
20-image real-corpus drift check was byte-identical after the earlier v2 integration.
|
||||
The calibration history and caveats are in the linked detector research plan.
|
||||
|
||||
### Official OpenAI SynthID verifier
|
||||
|
||||
[`openai_provenance.py`](../src/remove_ai_watermarks/openai_provenance.py)
|
||||
provides the explicit remote production backend exposed as
|
||||
`verify-openai-synthid`. It is intentionally separate from `identify`, because
|
||||
one invocation uploads a sanitized raster to OpenAI. The CLI requires
|
||||
`--acknowledge-upload`, and the optional OpenAI SDK lives in the independent
|
||||
`verify` extra.
|
||||
|
||||
The backend accepts only PNG, JPEG, and WebP. It computes a decoded RGBA pixel
|
||||
fingerprint, removes AI provenance metadata into a temporary file through
|
||||
`metadata.strip_and_verify`, recomputes the fingerprint, and aborts before any
|
||||
request if metadata survived, the format changed, the pixels changed, or the
|
||||
sanitized file exceeds the endpoint's 50 MiB limit. It then sends exactly one
|
||||
multipart file to `content_provenance_checks.create` and parses exactly one
|
||||
`type == "synthid"` result. The independent C2PA entry is never returned or
|
||||
used as fallback evidence. Missing, duplicate, or unknown SynthID outcomes are
|
||||
errors rather than negative detections.
|
||||
|
||||
The default SDK client has a 120-second request timeout and zero automatic
|
||||
retries. One upload acknowledgement therefore authorizes at most one media
|
||||
transmission rather than inheriting the SDK's retry default. Request logs keep
|
||||
the endpoint, temporary basename, media type, byte count, timeout, retry policy,
|
||||
duration, HTTP status, error code, and request id when available, but omit the
|
||||
source path, image bytes, credentials, and decoded-pixel fingerprint.
|
||||
|
||||
`OpenAIProvenanceError` preserves the status, API error code, request id,
|
||||
`Retry-After` value, and a transient-only `retryable` flag. The library does not
|
||||
automatically act on that flag: an explicit caller invocation is required for
|
||||
every additional upload. Transport and schema failures remain errors rather
|
||||
than becoming `not_detected` or a local detector result.
|
||||
|
||||
The result remains provider-scoped and positive-evidence-only. `not_detected`
|
||||
does not mean human-created, and the official endpoint's published prohibition
|
||||
on repeated reverse-engineering or evasion queries prevents using this backend
|
||||
as an adaptive training or removal oracle.
|
||||
|
||||
### Portable metadata record
|
||||
|
||||
@@ -576,6 +957,22 @@ provenance normalizer. Integration code publishes the strict
|
||||
`ProvenanceReport.to_dict()` alongside it rather than letting operational fields or
|
||||
derived results influence detection.
|
||||
|
||||
### Photo pixel classifier
|
||||
|
||||
[`classify.py`](../src/remove_ai_watermarks/classify.py) is the 2026-08-31
|
||||
photo freeze: CLIP-L-ft ridge AND freeze MLP, then 124-d focal heads only on
|
||||
DEFINITELY. The public label is `ai` / `human` / `unknown`. POSSIBLY is
|
||||
`unknown`. The named class is `openai` / `google` / `muse-image` / `tc260` /
|
||||
`None`. `tc260` is the China AIGC label standard, not one producer. The freeze
|
||||
file keys the Muse Image head `meta_muse_image`. Tests in
|
||||
`tests/test_classify.py` pin the gate without downloads and pin that
|
||||
`identify` does not import this module.
|
||||
|
||||
Weights stay out of git. The Hub snapshot is `wiltodelta/raiw-photo-classify`.
|
||||
`RAIW_CLASSIFY_WEIGHTS` overrides it. The extra is `classify`. User guide:
|
||||
[photo-classify.md](photo-classify.md). Hub card:
|
||||
[photo-classify-hf/README.md](photo-classify-hf/README.md).
|
||||
|
||||
### Pixel forensics
|
||||
|
||||
[`pixel_evidence.py`](../src/remove_ai_watermarks/pixel_evidence.py) measures six
|
||||
@@ -823,8 +1220,12 @@ and 1200 non-overlapping no-signal controls. At the strict 0.38 gate, 15/17
|
||||
confirmed carriers fired (min 0.249, p50 0.519, p90 0.578, max 0.579), while
|
||||
0/1200 controls fired (p99 0.200, max 0.293). The provenance cohort produced
|
||||
78/343 fires, but that is not a recall measurement because provenance identifies
|
||||
the provider, not the presence of this visible layout. No provenance relaxation
|
||||
ships until that cohort is visually labeled.
|
||||
the provider, not the presence of this visible layout. That cohort was later
|
||||
labeled by an OCR badge census (2026-08-28: 86 badge carriers, 257 badge-less,
|
||||
badge-less max 0.251 / p99 0.213), which is what enabled the shipped provenance
|
||||
relaxation 0.7: the relaxed band [0.251, 0.38) holds three faint badges and
|
||||
zero false fills, re-verified as 3 band detections with 0/257 badge-less fires
|
||||
on the provenance path.
|
||||
|
||||
The detector and removal mask must use compatible geometry. A detector that
|
||||
fires while producing an empty or misplaced mask is a removal failure even if
|
||||
@@ -846,11 +1247,38 @@ box grows), not the whole `footprint_mask`. Baidu extends right to the corner ta
|
||||
and LiblibAI extends left to the triangle logo; both inherit every guard around
|
||||
that arithmetic.
|
||||
|
||||
Doubao is the deliberate exception because its captured alpha supplies more
|
||||
information than a rectangle. Its continuous top-hat response locates the mark,
|
||||
then `DoubaoEngine.footprint_mask` resizes the alpha to that same winning
|
||||
`match_box` and masks only the glyphs. The canonical 2048-pixel fixture has bright
|
||||
branch texture behind the bottom-edge wordmark; bounding the thresholded response,
|
||||
padding it, and dilating it produced a solid 404-by-134 mask clamped to the bottom
|
||||
and right edges. OpenCV then had no context beyond either edge and filled the hole
|
||||
with large triangular wedges. The aligned sparse mask keeps both frame edges
|
||||
untouched and still clears the detector. `force` cannot align a missing detection,
|
||||
so it retains the shared geometry-box fallback.
|
||||
|
||||
Yuanbao uses the polarity-independent `contrast` front end because its standard
|
||||
two-line mark can be light on dark scenes or dark on light scenes. Its detector
|
||||
and footprint both use the same best-match box. The separate one-line overlay
|
||||
variant is not covered.
|
||||
|
||||
#### Bare-upload pill arm: measured impossible with the current detector (2026-08-28)
|
||||
|
||||
`_keep_pill` never removes a metadata-bare pill, and the question "can a bare arm
|
||||
(score + footprint-flatness) be opened" was measured to a closed NO over the local
|
||||
spaces corpus: 68 OCR-confirmed bare pills (an independent pixel-level label: the
|
||||
band sweep read a flush-left `AI生成`) score p50 0.181 / max 0.317, while 799 clean
|
||||
no-signal negatives reach 0.353 and 18 of them already pass the shipped 0.22 raw
|
||||
gate. The true-pill and clean distributions overlap completely -- there is no
|
||||
threshold -- and the engine's own high scores (0.34-0.48 on the flagged cohort)
|
||||
correlate with textured corners, not with pills: the edge-NCC keys on "text-like
|
||||
structure top-left". As with the Jimeng wordmark's silhouette, no threshold repairs
|
||||
this; a bare arm needs a detector that keys on the pill's rounded-rectangle
|
||||
geometry, and until one exists the bare-arm question is settled by this
|
||||
measurement, not by gate tuning. Harness: `data/spaces/_pill_bare_measure.py`
|
||||
(cohorts A/B/C over the no-signal pool).
|
||||
|
||||
The capture-less Jimeng pill lives in
|
||||
[`pill_engine.py`](../src/remove_ai_watermarks/pill_engine.py). It uses a
|
||||
synthetic silhouette for detection and a fixed top-left footprint.
|
||||
@@ -862,6 +1290,32 @@ Shared behavior is covered by:
|
||||
- [`test_text_mark_faint_mask.py`](../tests/test_text_mark_faint_mask.py)
|
||||
- [`test_text_mark_memory.py`](../tests/test_text_mark_memory.py)
|
||||
|
||||
#### Doubao video mark (2026-08-28 registration)
|
||||
|
||||
`detect_doubao_frame` reuses the image engine's synthetic `doubao_alpha.png` as
|
||||
its video template (same `豆包AI生成` run, bottom-right) through
|
||||
`_asset_template`, so image and video cannot drift apart on the glyph. Search
|
||||
profile: heights 3.2-6.0 percent of the short side from origin (0.60, 0.76),
|
||||
measured on the corpus. Policy is seedance-style single-floor: weak 0.35 /
|
||||
strong 0.55 / anchor IoU 0.80, calibrated by sequential decode over 39
|
||||
TC260-confirmed Doubao videos and 25 negatives -- a 12-frame stable run at 0.35
|
||||
accepts 33/39 positives and 0/25 negatives, while 0.30 already accepted one
|
||||
negative, so the bar stays at 0.35. Through the shipped arbiter
|
||||
(`identify_video`): 32/39 selected as doubao, 0/25 false; the remainder split
|
||||
between no stable run (5) and cross-template ties handed to earlier table
|
||||
entries (sora 1, kling 1) -- the registry's known order-decides behavior, not a
|
||||
doubao-specific defect. Harness: `data/spaces/_doubao_seq_measure.py`.
|
||||
|
||||
Two further video candidates were measured and PARKED the same day (2026-08-28,
|
||||
`data/spaces/_rh_jm_video_calibrate.py`): the Jimeng CN wordmark does not
|
||||
separate on video (1/5 positives vs 2/25 negatives hold a 12-frame 0.35 run --
|
||||
the same silhouette-generality the image jimeng engine documents), and
|
||||
RunningHub video separates (1/2 positives, 0/25 negatives) but a two-positive
|
||||
cohort is below the registration bar; both await corpus growth. The top-left
|
||||
"AI生成" pill on video is parked by analysis, not measurement: its image-engine
|
||||
false-fire mode is a STATIC textured corner, which is temporally stable, so the
|
||||
recurrence arbiter cannot screen it.
|
||||
|
||||
### Fill backends and region erasing
|
||||
|
||||
[`region_eraser.py`](../src/remove_ai_watermarks/region_eraser.py) implements the
|
||||
@@ -894,12 +1348,15 @@ is the source of truth for:
|
||||
- profile names and their underscore spellings;
|
||||
- the fixed seed;
|
||||
- the SDXL global-stage checkpoint id (`SDXL_MODEL_ID`) and the Canny ControlNet id;
|
||||
- strength resolution for both profiles.
|
||||
- strength resolution for every profile.
|
||||
|
||||
The current profiles are `qwen-zimage` (the default) and `sdxl-zimage`, and both
|
||||
are CUDA-only. `controlnet`, `sdxl`, `qwen` and `default` were removed rather than
|
||||
kept as a CPU path, and are rejected rather than aliased onward. There is no
|
||||
content-dependent automatic router.
|
||||
The current profiles are `qwen-zimage` (the default), `sdxl-zimage`,
|
||||
`chroma-zimage`, and `auto`, and all four are CUDA-only. `controlnet`, `sdxl`,
|
||||
`qwen` and `default` were removed rather than kept as a CPU path, and are
|
||||
rejected rather than aliased onward. `auto` is a deterministic per-cohort
|
||||
selection policy:
|
||||
chroma-zimage for OpenAI and Microsoft, qwen-zimage for Google, Meta, and
|
||||
unknown. It does not run a learned router or classify the image's genre.
|
||||
|
||||
`qwen-zimage` normally resolves global denoise from image area for unknown content.
|
||||
Measured provider cohorts bypass that curve with flat operating points. The values,
|
||||
@@ -917,8 +1374,10 @@ by the profile, so none of them appears in `WatermarkRemover.__init__`,
|
||||
then rejected several frames down; a signature that refuses the argument outright
|
||||
fails where the caller can act on it, and stops a wrapper from threading a value
|
||||
that would silently do nothing. The step count and CFG live with the stage that
|
||||
runs them (`GLOBAL_STEPS`, `FACE_STEPS`, `GLOBAL_CFG`, `FACE_CFG` in
|
||||
`qwen_zimage_pipeline.py`). The dtype is likewise profile-owned: see "Face-stage
|
||||
runs them: `GLOBAL_STEPS`/`GLOBAL_CFG` with the Qwen global stage
|
||||
(`qwen_zimage_pipeline.py`), `SDXL_STEPS` with the SDXL one
|
||||
(`sdxl_zimage_pipeline.py`), and `FACE_STEPS`/`FACE_CFG` with the shared face
|
||||
stage (`two_stage_pipeline.py`). The dtype is likewise profile-owned: see "Face-stage
|
||||
dtype" for what an override cost the last time one existed.
|
||||
|
||||
`device` is the exception and remains a library parameter: `None` or `"auto"`
|
||||
@@ -945,7 +1404,7 @@ same pixels. They diverged before, in opposite directions, for exactly this knob
|
||||
|
||||
The global and face prompts are calibrated model inputs, and the Canny edge map
|
||||
uses fixed thresholds of `_CANNY_LOW = 13` / `_CANNY_HIGH = 64`
|
||||
(`qwen_zimage_pipeline.py`). Treat those values as behavioral compatibility
|
||||
(`two_stage_pipeline.py`). Treat those values as behavioral compatibility
|
||||
contracts: a refactor must preserve them, and any deliberate change requires
|
||||
image-quality evaluation rather than only a unit-test pass. The prompt and
|
||||
edge-map regression guards are
|
||||
@@ -993,8 +1452,19 @@ Regression coverage:
|
||||
|
||||
### Qwen plus Z-Image
|
||||
|
||||
The profiles are siblings of one shared recipe rather than a chain:
|
||||
[`_internal/two_stage_pipeline.py`](../src/remove_ai_watermarks/_internal/two_stage_pipeline.py)
|
||||
holds `TwoStageZImagePipeline`, the base that owns everything profile-independent
|
||||
(the Z-Image face stage with YuNet detection and SAM masks, sizing, compositing,
|
||||
the prompt cache, and the `run`/`preload` orchestration). A profile subclass
|
||||
implements `_load_global` and `_run_global` for its regeneration model and may
|
||||
implement `_vae_roundtrip` to expose that stack's VAE as a verified-text donor;
|
||||
nothing else differs, and a test asserts the shared methods are the same objects
|
||||
on every subclass. Adding a third global stage means implementing those hooks,
|
||||
not inheriting from another profile's pipeline.
|
||||
|
||||
[`_internal/qwen_zimage_pipeline.py`](../src/remove_ai_watermarks/_internal/qwen_zimage_pipeline.py)
|
||||
implements the fixed CUDA-only two-stage profile:
|
||||
implements the fixed CUDA-only two-stage profile on that base:
|
||||
|
||||
1. Qwen Image with Canny conditioning regenerates the frame.
|
||||
2. YuNet locates faces, SAM builds masks, and Z-Image regenerates the selected
|
||||
@@ -1044,13 +1514,14 @@ geometry alone in schema 2, plus a SHA-256 over decoded RGB width, height, and p
|
||||
Validation happens before model loading. The library never treats OCR confidence as
|
||||
verification, and geometry-only operators do not need to invent text or script fields.
|
||||
|
||||
When enabled, `QwenZImagePipeline` reconstructs the source once through its already
|
||||
loaded Qwen VAE, runs the ordinary global and face stages, and calls the shared
|
||||
restoration compositor. The optional fidelity anchor first blends 15% of the VAE
|
||||
When enabled, the one profile selected before model loading reconstructs the source
|
||||
once through its already loaded VAE, runs the ordinary global and face stages, and
|
||||
calls the shared restoration compositor. Qwen and Chroma implement this donor hook;
|
||||
SDXL does not. The optional Qwen-only fidelity anchor first blends 15% of the VAE
|
||||
reconstruction into the clean result; it is off by default because the blend returned
|
||||
detector-visible OpenAI SynthID in the measured poster fixtures. The compositor derives
|
||||
binary source and candidate silhouettes, groups nearby lines, uses LaMa for the initial
|
||||
and residual-glyph erase passes, paints fresh silhouette edges, then copies the Qwen-VAE
|
||||
and residual-glyph erase passes, paints fresh silhouette edges, then copies the profile-VAE
|
||||
core with a 0.5-pixel feather. The evaluation script
|
||||
imports these same mask and compositing helpers so the two implementations cannot
|
||||
silently drift. Silhouette crops start 12% of line height beyond each horizontal
|
||||
@@ -1066,6 +1537,21 @@ geometry or final pixels after the verified layer and have no measured oracle re
|
||||
It remains opt-in because annotations are manual and provider verdicts apply only to
|
||||
the exact tested output hashes, not to the mechanism in general.
|
||||
|
||||
The first full-path H100 check on 2026-08-31 ran each generative engine exactly
|
||||
once per source and used that SAME engine's VAE donor; it did not compose Qwen and
|
||||
Chroma in one production result. On the two verified multilingual OpenAI posters,
|
||||
restoration reduced PaddleOCR CER from 0.303 to 0.160 and 0.279 to 0.136 for Qwen,
|
||||
and from 0.213 to 0.130 and 0.204 to 0.108 for Chroma. PSNR improved on all four
|
||||
arms (Qwen 26.1 to 29.9 dB and 27.7 to 31.6 dB; Chroma 29.6 to 33.1 dB and 29.7
|
||||
to 32.9 dB). The verified CJK Google fixture is the counterexample that prevents
|
||||
an unconditional quality claim: Qwen CER stayed 0.074, while Chroma moved from
|
||||
0.037 to 0.074 despite better LPIPS, SSIM, PSNR, and text-box pixel error. The
|
||||
stage is therefore implemented and fidelity-positive on both OpenAI sources, but
|
||||
not assumed to improve every OCR reading. This is a fidelity check, not a provider
|
||||
oracle certification; exact output hashes and per-box metrics are produced by
|
||||
`scripts/chroma_text_restoration_study.py` under the gitignored
|
||||
`out/text-restoration-engine-study/` directory.
|
||||
|
||||
A matched stage-isolation check on the 18-face Gemini portrait grid confirms the
|
||||
division of responsibility. The visible-cleaned, metadata-stripped control and the
|
||||
Z-Image face-only output were both SynthID-positive; Qwen global-only and the full
|
||||
@@ -1094,13 +1580,85 @@ oracle outcomes, and the DiffSynth/GGUF, scheduler, detector, and seed caveats a
|
||||
recorded in
|
||||
[`data/evaluations/fidelity/upstream-v2-reproduction-2026-08-13.csv`](../data/evaluations/fidelity/upstream-v2-reproduction-2026-08-13.csv).
|
||||
|
||||
### Chroma1 plus Z-Image
|
||||
|
||||
[`_internal/chroma_zimage_pipeline.py`](../src/remove_ai_watermarks/_internal/chroma_zimage_pipeline.py)
|
||||
runs the same two-stage recipe on a Chroma1-HD (`lodestones/Chroma1-HD`,
|
||||
Apache-2.0) global pass through diffusers' `ChromaImg2ImgPipeline`. It is the
|
||||
answer to issue #88's FLUX.2 request through the model that actually exposes
|
||||
strength-controlled img2img in that family; the full research record,
|
||||
including why FLUX.2 itself is not integrable this way, is
|
||||
[`chroma1-engine-research.md`](chroma1-engine-research.md).
|
||||
|
||||
The profile also implements the shared verified-text donor hook through its already
|
||||
loaded Chroma VAE. It edge-pads native geometry to the /16 latent grid, takes the
|
||||
deterministic mode of the encoder distribution rather than sampling it, decodes with
|
||||
Diffusers' own image processor, and crops back to the exact source dimensions. Thus
|
||||
`auto` still loads and runs only the profile selected by the measured vendor table;
|
||||
the presence of a manifest does not introduce a second generative model or override
|
||||
that decision. A CPU smoke run against the official Chroma1-HD `AutoencoderKL`
|
||||
weights on 2026-08-31 verified the real encode/decode contract and exact restoration
|
||||
of a non-grid-aligned source size; model-free tests pin mode-not-sample and padding.
|
||||
Chroma-specific full-pipeline provider-oracle certification remains an output-hash
|
||||
measurement, not something either check can establish.
|
||||
|
||||
The profile swaps only the global stage (same inheritance invariant as
|
||||
sdxl-zimage, asserted by the same test shape). Three things are bound to the
|
||||
calibration and must not drift: the NEUTRAL prompt (`high quality, sharp,
|
||||
detailed, faithful to the original` -- deliberately NOT the shared canny-stage
|
||||
prompt, which was never calibrated against Chroma1), guidance 5.0, and the
|
||||
four-EFFECTIVE-step schedule with `requested_steps` compensation (the same
|
||||
Diffusers truncation trap sdxl-zimage documents). There is no Canny
|
||||
conditioning: the floors were measured on a plain strength pass.
|
||||
|
||||
The flat vendor floors (`CHROMA_ZIMAGE_*_STRENGTH` in watermark_profiles):
|
||||
OpenAI 0.09 and Microsoft 0.125 (both BELOW qwen's, with better
|
||||
matched-strength fidelity), Google 0.40 for zero-face content / 0.125 when
|
||||
faces are detected (a content-adaptive arm from the clean face-count split
|
||||
in the calibration: both text cards need 0.25, both face fixtures clear at
|
||||
0.12, so the YuNet detector -- already loaded for the face stage -- routes
|
||||
the operating point), and Meta 0.17 (ABOVE qwen's, because Chroma1's
|
||||
per-fixture boundaries scatter wider). The matched-strength addendum in the
|
||||
research doc is the honest read: at the strength each image actually needs,
|
||||
Chroma1 regenerates better almost everywhere except face identity (which the
|
||||
inherited Z-Image face stage supplies); at the flat worst-case floors it
|
||||
destroys dense text and face identity. The Google face-content arm is the
|
||||
first shipped piece of the content-adaptive policy. A Meta arm was measured
|
||||
on 2026-08-30/31 (docs/chroma1-engine-research.md, expansion section) and
|
||||
does not ship: first-cleans from 0.03 to 0.10 form a continuum,
|
||||
`flat_ratio` overlaps the hard and easy clusters, and a high-flat
|
||||
easy-arm would misroute the harvest-1 portraits. The worst seed-0
|
||||
first-clean is still 0.10; botanical seeds 1 and 2 stay DETECTED at
|
||||
that rung, which is the margin the 0.17 floor already holds. OpenAI stays on the flat 0.09 floor: extra SynthID-positive carriers
|
||||
clear at or below 0.06, including a 2-face file, and only the 9-face
|
||||
grid needs 0.075 (seed-stable on 0, 1, 2 via openai.com/research/verify).
|
||||
|
||||
A separate content-balanced check on 2026-08-31 tested the exact production
|
||||
OpenAI and Meta floors over 19 prompt-matched image strata per provider. Each
|
||||
engine ran in its own sequential H100 invocation, and the comparison isolated
|
||||
the global stage because the shared face repair does not change with this choice.
|
||||
On OpenAI content, Chroma won SSIM and PSNR on 19/19 pairs, MAE on 18/19, edge
|
||||
F1 on 15/19, and LPIPS on 12/19. On Meta content, Qwen won LPIPS and edge F1 on
|
||||
18/19 pairs; Chroma's better PSNR on 16/19 did not overcome its systematically
|
||||
worse perceptual and edge preservation at the higher required floor. Exact
|
||||
two-sided sign-test p-values were 3.8e-6 for the OpenAI SSIM and PSNR directions
|
||||
and 7.6e-5 for the Meta LPIPS and edge-F1 directions. The tracked discovery
|
||||
inputs and manifests are in
|
||||
[`data/evaluations/engine-selection/`](../data/evaluations/engine-selection/),
|
||||
the run is reproduced by `scripts/engine_selection_study.py`, and paired analysis
|
||||
by `scripts/analyze_engine_selection_study.py`. These content files were locally
|
||||
re-encoded and are not watermark oracles; they validate fidelity at already
|
||||
oracle-calibrated floors, not removal by themselves. No content-stratum override
|
||||
is supported by this pass, so `auto` keeps the measured provenance-cohort table.
|
||||
|
||||
### SDXL plus Z-Image
|
||||
|
||||
[`_internal/sdxl_zimage_pipeline.py`](../src/remove_ai_watermarks/_internal/sdxl_zimage_pipeline.py)
|
||||
runs the same two-stage recipe on an SDXL global pass. `SdxlZImagePipeline` subclasses
|
||||
`QwenZImagePipeline` and overrides only `_run_global` and `preload`, so the face stage
|
||||
is inherited rather than copied and cannot drift between the profiles; a test asserts
|
||||
the shared methods are the same objects.
|
||||
runs the same two-stage recipe on an SDXL global pass. `SdxlZImagePipeline`
|
||||
subclasses `TwoStageZImagePipeline` and implements only the global stage
|
||||
(`_load_global`, `_run_global`), so the face stage is inherited rather than copied
|
||||
and cannot drift between the profiles; a test asserts the shared methods are the
|
||||
same objects.
|
||||
|
||||
Four things are architecture-bound and swap with the model: the ControlNet
|
||||
(`xinsir/controlnet-canny-sdxl-1.0`), the four-step distillation LoRA
|
||||
@@ -1138,7 +1696,7 @@ avoid model downloads, so nothing exercised the loader. The lesson is narrower t
|
||||
was a dtype the subclass silently changed out from under it.
|
||||
|
||||
Note what the seam is, because it decides where the fix belongs.
|
||||
`SdxlZImagePipeline._load_sdxl` hardcodes fp16 for its own ControlNet, VAE and
|
||||
`SdxlZImagePipeline._load_global` hardcodes fp16 for its own ControlNet, VAE and
|
||||
pipeline, so `self.torch_dtype` was never actually the global stage's dtype on this
|
||||
profile -- its only remaining readers were face-stage code. SAM was the second one:
|
||||
it never crashed, because it casts its own inputs and leaves through `.float()`, but it
|
||||
@@ -1225,7 +1783,7 @@ within one pipeline.
|
||||
`_cache_static_prompt_embeddings` therefore persists what the text encoder produced
|
||||
under `_model_cache_dir()/prompt-embeddings`, keyed by cache version, model id,
|
||||
pipeline output params, and the exact prompt string. Once that file exists,
|
||||
`_load_qwen` and `_load_zimage` drop the text-encoder `ModelConfig` from the model
|
||||
`_load_global` and `_load_zimage` drop the text-encoder `ModelConfig` from the model
|
||||
stack entirely and serve the stored tensors instead. Measured on an H100 volume in
|
||||
August 2026, that removes **15.45 GiB** (Qwen2.5-VL) and **7.49 GiB** (Z-Image) of a
|
||||
**87.6 GiB** per-request read, worth a median **11.76 s** and **4.10 s** of load time
|
||||
@@ -1277,7 +1835,8 @@ grain, unsharp masking, and adaptive polish helpers.
|
||||
|
||||
`upscaler.py` held an optional Real-ESRGAN path, reachable only when enlarging a
|
||||
small image to the minimum-resolution floor. That floor existed to lift small
|
||||
inputs toward SDXL's ~1024 training size; when the SDXL profiles were removed it
|
||||
inputs toward SDXL's ~1024 training size; when the standalone SDXL and ControlNet
|
||||
profiles were removed (`sdxl-zimage` is a later, surviving profile) it
|
||||
was forced to 0 on every path, so the module, the `--min-resolution` and
|
||||
`--upscaler` options and the `esrgan` extra were all unreachable and went with
|
||||
it. Only the `max_resolution` cap can move geometry now, and it only scales down.
|
||||
|
||||
@@ -0,0 +1,243 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
license: apache-2.0
|
||||
library_name: transformers
|
||||
pipeline_tag: image-classification
|
||||
base_model: openai/clip-vit-large-patch14
|
||||
inference: false
|
||||
tags:
|
||||
- clip
|
||||
- image-classification
|
||||
- ai-generated-content
|
||||
- photo
|
||||
- provider-attribution
|
||||
datasets:
|
||||
- detection-datasets/coco
|
||||
---
|
||||
|
||||
# Photo AI-versus-camera classifier
|
||||
|
||||
Two heads, one call. Model 1 decides whether a **photograph** looks generated
|
||||
or camera-like. Model 2 names a provider only after Model 1 is DEFINITELY AI.
|
||||
|
||||
This is not a SynthID decoder. It is not provenance. It is not a universal
|
||||
"AI or not" detector for receipts, UI, or digital art. The Python package
|
||||
[`remove-ai-watermarks`](https://github.com/wiltodelta/remove-ai-watermarks)
|
||||
loads these files through an explicit `classify` extra. `identify` never
|
||||
starts this model, including after a no-signal metadata scan.
|
||||
|
||||
Library guide:
|
||||
[photo pixel classification](https://github.com/wiltodelta/remove-ai-watermarks/blob/main/docs/photo-classify.md).
|
||||
Research archive:
|
||||
[AI-generated image classifiers](https://github.com/wiltodelta/remove-ai-watermarks/blob/main/docs/ai-generated-image-classifiers.md).
|
||||
The demo Space
|
||||
[`wiltodelta/remove-ai-watermarks`](https://huggingface.co/spaces/wiltodelta/remove-ai-watermarks)
|
||||
is a watermark-removal UI, not this classifier.
|
||||
|
||||
The training catalog is not in this Hub repository and is not on GitHub.
|
||||
|
||||
## Intended use
|
||||
|
||||
Use it when you have a photographic still and you want a pixel opinion after
|
||||
metadata is gone:
|
||||
|
||||
- `label=ai` plus `provider=openai|google|muse-image|tc260` on a stripped ChatGPT,
|
||||
Gemini, Muse Image, or TC260 photograph
|
||||
- `label=human` on camera photographs at a low false-positive rate
|
||||
- `label=unknown` when the detector is only POSSIBLY AI, or when 124-d
|
||||
features cannot be extracted
|
||||
|
||||
Do not use it to assert that a file is clean. Do not use it as a legal
|
||||
authorship test. Do not run Model 2 on every image.
|
||||
|
||||
## Approach
|
||||
|
||||
Content embeddings and forensic residuals answer different questions.
|
||||
|
||||
CLIP-L, after a light vision-block fine-tune, separates **generated
|
||||
photographs** from **camera photographs**. It does not reliably separate
|
||||
generated photographs from receipts, UI, charts, or illustration. A ridge
|
||||
trained on that embedding at a 1% Open Images false-positive cut is the
|
||||
strongest Model 1 we measured. A small MLP on the same vectors, ANDed with
|
||||
the ridge, is the freeze DEFINITELY gate.
|
||||
|
||||
The 124-d bank is a different feature: patch-local residual ratios (FFT band
|
||||
energy, comb contrast, autocovariance) on 256 px tiles. On that bank, OpenAI
|
||||
versus Gemini is easy (AUC 0.989 in the research sweep) while "AI or not" is
|
||||
the wrong job. Firefly and PixelBin sit near Google on those residuals, which
|
||||
is why a provider finder must keep foreign generators in the test and must
|
||||
abstain.
|
||||
|
||||
The two heads therefore stay specialized:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
image[Photograph] --> m1[Model 1: CLIP-L-ft ridge AND MLP]
|
||||
m1 -->|likely_human| human[label human, provider none]
|
||||
m1 -->|possibly| unk[label unknown, provider none]
|
||||
m1 -->|definitely| m2[Model 2: 124-d one-vs-rest focal]
|
||||
m2 -->|openai google muse-image tc260| named[label ai plus provider]
|
||||
m2 -->|no_ai or extract fail| aiOnly[label ai, provider none]
|
||||
```
|
||||
|
||||
CLIP content space is the wrong place to train "receipt versus rest" or
|
||||
"UI versus rest" if the product contract is photographic. Those domains
|
||||
remain open. The shipped heads do not include a receipt specialist.
|
||||
|
||||
Provider attribution tracks the **renderer**, not the front-end. Bing Image
|
||||
Creator rows signed Microsoft, OpenAI score as `openai`. Designer rows signed
|
||||
Microsoft, Google LLC score as `google`. Native Designer is mixed. There is
|
||||
no Microsoft pixel class. `openai` and `google` are provider classes.
|
||||
`muse-image` is Muse Image output, not a general Meta class. Instagram
|
||||
`made_with_ai` is not that class. `tc260` is the China AIGC label standard,
|
||||
not one producer: Doubao, Jimeng, Qwen, Kling, and others share that class.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Model 1
|
||||
|
||||
- Backbone: `openai/clip-vit-large-patch14`
|
||||
- Fine-tune: last two vision blocks, visual projection, post-layernorm
|
||||
- Preprocess: 224 letterbox, pad RGB `(123, 117, 104)`, BICUBIC, L2-normalized
|
||||
`get_image_features` (`pooler_output` when present)
|
||||
- Ridge: linear probe on 768-d vectors, threshold `thr_oi_1pct` = 0.3056
|
||||
from the locked `photo_dev_oi` 1% cut
|
||||
- MLP: `768-512-128-1` with dropout 0.3 / 0.1, threshold 5.9586, 1.67%
|
||||
photo-dev quantile
|
||||
- DEFINITELY = ridge AND MLP. POSSIBLY = either. The public `label` is `ai`
|
||||
only on DEFINITELY.
|
||||
|
||||
### Model 2
|
||||
|
||||
- Input: 124-d residual vector, or abstain if the image is smaller than
|
||||
256 px or the extractor refuses
|
||||
- Heads: one-vs-rest focal MLP `124-64-1` per class
|
||||
(`openai`, `google`, `tc260`, `meta_muse_image`, `no_ai`)
|
||||
- Decision: a class wins only if it beats `no_ai` by margin 0.30, then argmax
|
||||
among those that passed
|
||||
- Public name `muse-image` is the Muse Image class. The freeze file still
|
||||
keys that head `meta_muse_image`. `tc260` is the China AIGC label
|
||||
standard (mixed producers), not a company. `no_ai` is not reported as
|
||||
`human`; it becomes `provider=None` on an `ai` label (for example FLUX)
|
||||
|
||||
## Training
|
||||
|
||||
CPU. Two consecutive freeze runs from the same catalog and caches were
|
||||
byte-identical.
|
||||
|
||||
| Item | Value |
|
||||
| --- | --- |
|
||||
| Catalog identity | sha256 of file bytes, 32,690 listed files |
|
||||
| CLIP-L embeddings keyed | 26,573 |
|
||||
| Detector seed | 20260940 |
|
||||
| Provider seed | 20260956 |
|
||||
| Sample seed for provider OpenAI/Google draws | 20260821 |
|
||||
| MLP epochs / width | 40 / 512 |
|
||||
| Provider focal gamma / epochs | 2.0 / 40 |
|
||||
| Private UI in train | 0. Replaced with 150 CC Flickr screenshots via Openverse, host `live.staticflickr.com` only |
|
||||
|
||||
Flickr licenses in that replacement cell: CC BY 2.0 42, CC BY-NC 2.0 42,
|
||||
CC BY-SA 2.0 29, CC BY-NC-SA 2.0 28, CC BY-NC-ND 2.0 6, CC BY-ND 2.0 2,
|
||||
CC0 1.
|
||||
|
||||
The pixel catalog is **not published on the Hub**. Head retrain uses
|
||||
sha256-keyed CLIP-L and 124-d caches plus `catalog.json` roles. The original
|
||||
files remain stored privately so embeddings and 124-d features can be rebuilt.
|
||||
Public named sources in those roles include COCO, Open Images, Kodak, Picsum,
|
||||
Hugging Face FLUX, Openverse CC Flickr, and Meta Muse API hold-out v3.
|
||||
In-house labeled OpenAI, Gemini, and TC260 photographs stay private. See
|
||||
[photo-classify-training.md](https://github.com/wiltodelta/remove-ai-watermarks/blob/main/docs/photo-classify-training.md).
|
||||
|
||||
## Evaluation (run 1 = run 2)
|
||||
|
||||
DEFINITELY is the shipped operating point.
|
||||
|
||||
| Head | Cell | Result |
|
||||
| --- | --- | ---: |
|
||||
| Detector DEFINITELY | AI-test | 92.4% (n=1,847) |
|
||||
| Detector DEFINITELY | Open Images fresh | 1.3% (n=3,000) |
|
||||
| Detector DEFINITELY | Kodak | 0/24 |
|
||||
| Detector DEFINITELY | FLUX hold | 83.0% (n=300) |
|
||||
| Class | OpenAI test | 90.8% (345/380 of 381) |
|
||||
| Class | Google test | 90.9% (339/373 of 377) |
|
||||
| Class | TC260 test | 78.6% (298/379 of 384) |
|
||||
| Class | Muse Image hold-out v2 | 85.7% (66/77 listed 79) |
|
||||
| Class | Muse Image hold-out v3 | 89.4% (177/198) |
|
||||
| Class | Muse Image hold-out pooled | 88.4% (243/275 listed 277) |
|
||||
| Class | meme templates, ungated | 29.1% leak (86 of 97) |
|
||||
|
||||
The ungated meme leak is why Model 2 must not run on every file. Gated on
|
||||
DEFINITELY, that leak is not a provider attribution.
|
||||
|
||||
Ridge-only Model 1 at the earlier 1% Open Images cut (before the AND with
|
||||
the freeze MLP) is documented in the research page: Kodak 0/24, fresh 1.7%,
|
||||
AI-test 93.0%, FLUX 92.7%, Firefly 94.0%. Nobody in that sweep hit both
|
||||
≤1% fresh FPR and ≥90% TPR. The freeze AND is stricter on photos (1.3%
|
||||
DEF) at a small recall cost.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Photographs only.** Receipts, UI, scans of forms, maps, and community
|
||||
digital art sit in the AI region of CLIP-L. A real thermal receipt can
|
||||
score as AI. A photorealistic AI receipt can score as human.
|
||||
- **Not SynthID.** A `provider=openai` result means the pixels look like an
|
||||
OpenAI-rendered photograph, not that a payload was decoded.
|
||||
- **Not provenance.** C2PA, IPTC, and visible marks stay on `identify`.
|
||||
- **124-d can refuse.** Some sizes and some images yield no vector. The
|
||||
detector can still return DEFINITELY; provider is then `None`.
|
||||
- **Microsoft has no head.** Route by renderer, or use C2PA for the
|
||||
front-end name.
|
||||
- **Open-world generators.** FLUX is a hold-out at 83% DEF, not 100%.
|
||||
Unknown generators may land on `ai` with `provider=None`, or miss
|
||||
DEFINITELY.
|
||||
- **`tc260` is mixed producers.** Doubao, Jimeng, Qwen, Kling, and others
|
||||
share one label-standard head. A later retrain should split by
|
||||
`ContentProducer`. Do not read `tc260` as a named manufacturer.
|
||||
|
||||
## How to use
|
||||
|
||||
Install the extra. First call downloads this repository into the Hugging
|
||||
Face cache, or reads `RAIW_CLASSIFY_WEIGHTS` if you already have the files.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[classify]"
|
||||
remove-ai-watermarks classify image.png
|
||||
```
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from remove_ai_watermarks.classify import classify_pixels
|
||||
|
||||
result = classify_pixels(Path("image.png"))
|
||||
# result.label, result.detector, result.provider
|
||||
```
|
||||
|
||||
`device` is a library parameter (`None` / `"auto"` / `"cpu"` / `"cuda"`),
|
||||
not a CLI option. Pin a Hub revision in production so a later upload cannot
|
||||
silently change the heads.
|
||||
|
||||
## Files in this repository
|
||||
|
||||
| File | Role |
|
||||
| --- | --- |
|
||||
| `clip-l-ft.pt` | HeadedCLIP state dict (CLIP-L plus unused linear head) |
|
||||
| `probe-weights-clip-l-ft.npz` | Ridge mean, scale, weights, `thr_oi_1pct` |
|
||||
| `detector.pt` | Freeze MLP |
|
||||
| `provider.pt` | Focal heads keyed `openai`, `google`, `tc260`, `meta_muse_image`, `no_ai` |
|
||||
| `operating-point.json` | Seeds, thresholds, margin |
|
||||
|
||||
The image catalog is not in this repository.
|
||||
|
||||
## License
|
||||
|
||||
Apache-2.0 for the trained heads and this card. CLIP-L inherits its upstream
|
||||
license from [`openai/clip-vit-large-patch14`](https://huggingface.co/openai/clip-vit-large-patch14).
|
||||
Flickr replacement images are CC-licensed per file; they are not redistributed
|
||||
here.
|
||||
|
||||
## Citation
|
||||
|
||||
Package [`remove-ai-watermarks`](https://github.com/wiltodelta/remove-ai-watermarks).
|
||||
Protocol and rejected variants: [docs/ai-generated-image-classifiers.md](https://github.com/wiltodelta/remove-ai-watermarks/blob/main/docs/ai-generated-image-classifiers.md).
|
||||
Provider renderer study: [docs/synthid-classifiers.md](https://github.com/wiltodelta/remove-ai-watermarks/blob/main/docs/synthid-classifiers.md).
|
||||
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"freeze": "2026-08-31",
|
||||
"task": "photo_ai_versus_camera",
|
||||
"not": ["synthid_decoder", "provenance", "universal_ai_detector"],
|
||||
"seeds": {
|
||||
"sample": 20260821,
|
||||
"detector": 20260940,
|
||||
"provider": 20260956
|
||||
},
|
||||
"model1": {
|
||||
"backbone": "openai/clip-vit-large-patch14",
|
||||
"letterbox": 224,
|
||||
"pad_rgb": [123, 117, 104],
|
||||
"ridge_threshold_key": "thr_oi_1pct",
|
||||
"ridge_threshold": 0.3056212276800537,
|
||||
"mlp_threshold": 5.9586493237495395,
|
||||
"mlp_width": 512,
|
||||
"definitely": "ridge_and_mlp",
|
||||
"public_ai_label": "definitely_only"
|
||||
},
|
||||
"model2": {
|
||||
"feature_width": 124,
|
||||
"margin": 0.3,
|
||||
"classes": ["openai", "google", "tc260", "muse-image", "no_ai"],
|
||||
"class_kind": {
|
||||
"openai": "provider",
|
||||
"google": "provider",
|
||||
"tc260": "label-standard",
|
||||
"muse-image": "model",
|
||||
"no_ai": "negative"
|
||||
},
|
||||
"checkpoint_keys": {
|
||||
"muse-image": "meta_muse_image"
|
||||
},
|
||||
"runs_only_after": "definitely"
|
||||
},
|
||||
"files": [
|
||||
"clip-l-ft.pt",
|
||||
"probe-weights-clip-l-ft.npz",
|
||||
"detector.pt",
|
||||
"provider.pt"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
# Photo classifier training and data
|
||||
|
||||
How the 2026-08-31 freeze was fit, what a retrain needs, and where those
|
||||
files come from. Inference is [photo-classify.md](photo-classify.md). The
|
||||
campaign log is [ai-generated-image-classifiers.md](ai-generated-image-classifiers.md).
|
||||
|
||||
The shipped package does not include the catalog or the embeddings. Fitting
|
||||
the heads again uses the cache pack. The original pixels still have to be
|
||||
kept: a missing embedding, a new backbone, or an audit all need the files.
|
||||
|
||||
## Retrain pack (caches)
|
||||
|
||||
CPU retrain of both heads needs sha256-keyed matrices and the catalog of
|
||||
roles. That step does not open JPEG or PNG files. It is not permission to
|
||||
drop the pixels.
|
||||
|
||||
| File | Role |
|
||||
| --- | --- |
|
||||
| `catalog.json` | 32,690 rows: `sha256`, `roles`, optional `family` / public `source` |
|
||||
| `embeddings-by-sha256.npz` | CLIP-L-ft 768-d vectors for Model 1 |
|
||||
| `extra-clip-l.npz` | CLIP-L extras for the 150 Flickr UI hashes |
|
||||
| `features-124d.npz` | 124-d residual vectors for Model 2 |
|
||||
| `probe-weights-clip-l-ft.npz` | Ridge mean, scale, weights, `thr_oi_1pct` |
|
||||
| `meta-split.json` | Meta train 217, hold-out v2 79, hold-out v3 198 |
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run python scripts/retrain_photo_classify.py --pack DIR --out DIR
|
||||
```
|
||||
|
||||
Seeds are fixed: detector `20260940`, provider `20260956`. Two consecutive
|
||||
runs must be byte-identical.
|
||||
|
||||
Not in the **published** retrain pack: raw photographs, COCO/Open Images/Kodak
|
||||
binaries, CORD, Danbooru. Those files stay in their public corpora or in the
|
||||
private freeze archive. Pixel rebuild of a missing embedding still needs
|
||||
`clip-l-ft.pt` and the original file.
|
||||
|
||||
## Where the rows come from
|
||||
|
||||
Identity is always sha256 of the file bytes.
|
||||
|
||||
### Public, named corpora
|
||||
|
||||
| Role | Source |
|
||||
| --- | --- |
|
||||
| `detector_photo_train`, `detector_photo_coco_hold` | COCO 2017 validation stills |
|
||||
| `detector_open_images_*`, `detector_photo_dev_oi` | Open Images photographs. Locked fresh never enters train |
|
||||
| `detector_photo_kodak` | Kodak lossless true color, 24 files |
|
||||
| `detector_photo_picsum_eval` | Picsum |
|
||||
| `detector_flux_hf_hold` | Hugging Face FLUX stills, hold-out |
|
||||
| Flickr UI cell (150) | Openverse `license_type=all-cc`, host `live.staticflickr.com` only |
|
||||
| Meta hold-out v3 (198) | Meta Muse Image API, eval-only, not in train |
|
||||
|
||||
### In-house labeled photographs, not redistributed
|
||||
|
||||
Provider OpenAI, Google, and TC260 train and test rows, and most detector AI
|
||||
train and test rows, are labeled photographic generations held privately.
|
||||
The published pack carries only their sha256, roles, and precomputed vectors.
|
||||
It does not carry the pixels.
|
||||
|
||||
### Out of the freeze train
|
||||
|
||||
CORD receipts, FUNSD forms, Danbooru, charts, and map tiles were measured as
|
||||
failure domains. They are not Model 1 train. A future universal router treats
|
||||
them as other domains, not as extra CLIP-L negatives.
|
||||
|
||||
## What the freeze script does
|
||||
|
||||
Historically `freeze_both_models.py` built the catalog from local paths, then
|
||||
trained. That campaign code is private and is not in this repository. The
|
||||
public entry is `scripts/retrain_photo_classify.py`, which skips catalog
|
||||
construction and fits the same heads from the pack above.
|
||||
|
||||
1. Load CLIP-L-ft vectors by sha256. Fit the 768-512-128-1 MLP on
|
||||
`detector_ai_train` plus extras versus `detector_photo_train` plus struct.
|
||||
Threshold is the 1.67% quantile on `detector_photo_dev_oi`.
|
||||
2. AND that MLP with the frozen ridge in the probe file. DEFINITELY is both.
|
||||
3. Load 124-d vectors. Fit one-vs-rest focal heads for openai, google, tc260,
|
||||
meta_muse_image, no_ai. Margin 0.30. The public class for the Muse Image
|
||||
head is `muse-image`. `tc260` is the China AIGC label standard, mixed
|
||||
producers, not one company.
|
||||
4. Repeat with the same seeds. Compare bytes.
|
||||
|
||||
CLIP-L fine-tune of the last two vision blocks is earlier, seed 20260822, and
|
||||
is not repeated by the retrain script. The backbone file `clip-l-ft.pt` is an
|
||||
inference and embedding dependency, not a retrain-pack input.
|
||||
|
||||
## Next retrain
|
||||
|
||||
Keep the current `tc260` head until then. The next provider-head fit should
|
||||
split that catalog by `ContentProducer` (Doubao, Jimeng, Qwen, Kling, and
|
||||
the rest) instead of one label-standard class. Do not treat a `tc260` score
|
||||
as a named manufacturer.
|
||||
@@ -0,0 +1,126 @@
|
||||
# Photo pixel classification
|
||||
|
||||
`classify` gives a pixel opinion on a **photograph**: camera-like, generated,
|
||||
or unknown, and optionally which renderer it resembles. It is a separate
|
||||
command from `identify`. Metadata inspection never starts it.
|
||||
|
||||
This page is the library guide. The Hugging Face model card is the same freeze
|
||||
written for Hub publication: [photo-classify-hf/README.md](photo-classify-hf/README.md).
|
||||
Campaign notes and rejected variants live in
|
||||
[AI-generated image classifiers](ai-generated-image-classifiers.md).
|
||||
|
||||
## When to use it
|
||||
|
||||
Use `identify` for C2PA, IPTC, visible marks, and other provenance. Use
|
||||
`classify` when those signals are gone and the file is still a photograph.
|
||||
|
||||
Do not use `classify` to prove a file is clean. Do not use it on receipts, UI,
|
||||
or digital art. Do not treat `provider=openai` as a decoded SynthID payload.
|
||||
|
||||
## Install and run
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[classify]"
|
||||
remove-ai-watermarks classify image.png
|
||||
remove-ai-watermarks classify image.png --json
|
||||
```
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from remove_ai_watermarks.classify import classify_pixels
|
||||
|
||||
result = classify_pixels(Path("image.png"))
|
||||
print(result.label, result.detector, result.provider)
|
||||
```
|
||||
|
||||
Missing extra raises `RuntimeError` with
|
||||
`pip install 'remove-ai-watermarks[classify]'`. `device` is a library
|
||||
parameter (`None` / `"auto"` / `"cpu"` / `"cuda"`), not a CLI option.
|
||||
|
||||
Weights are not in git. First call downloads
|
||||
[`wiltodelta/raiw-photo-classify`](https://huggingface.co/wiltodelta/raiw-photo-classify)
|
||||
at the freeze revision pinned in `classify.py`, or reads
|
||||
`RAIW_CLASSIFY_WEIGHTS` if that directory already has `clip-l-ft.pt`,
|
||||
`probe-weights-clip-l-ft.npz`, `detector.pt`, and `provider.pt`. The training
|
||||
catalog is not published with the package.
|
||||
|
||||
## What one call returns
|
||||
|
||||
One request runs both heads. Model 2 runs only after Model 1 is DEFINITELY.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
image[Photograph] --> m1[Model 1 CLIP-L-ft ridge AND MLP]
|
||||
m1 -->|likely_human| human["label human, provider none"]
|
||||
m1 -->|possibly| unk["label unknown, provider none"]
|
||||
m1 -->|definitely| m2[Model 2 124-d focal heads]
|
||||
m2 -->|openai google muse-image tc260| named["label ai plus provider"]
|
||||
m2 -->|no_ai or extract fail| aiOnly["label ai, provider none"]
|
||||
```
|
||||
|
||||
| Field | Values | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `label` | `ai`, `human`, `unknown` | Public verdict. `ai` only on DEFINITELY |
|
||||
| `domain` | `photo` | This freeze is photographic only |
|
||||
| `detector` | `definitely`, `possibly`, `likely_human` | Raw Model 1 gate |
|
||||
| `provider` | `openai`, `google`, `muse-image`, `tc260`, or `None` | Model 2, only if `label` is `ai` |
|
||||
|
||||
`human` is camera-like under this contract, not a proof of authorship.
|
||||
`unknown` is POSSIBLY, or a domain this freeze does not support. `provider=None`
|
||||
on an `ai` file is often FLUX or another generator that is not one of the four
|
||||
heads, or a 124-d extract refusal.
|
||||
|
||||
## Approach
|
||||
|
||||
CLIP-L content embeddings separate generated photographs from camera
|
||||
photographs. They do not separate generated photographs from receipts or UI.
|
||||
A ridge on fine-tuned CLIP-L (last two vision blocks, 224 letterbox) is Model
|
||||
1. A small MLP on the same vectors, ANDed with the ridge, is the freeze
|
||||
DEFINITELY gate.
|
||||
|
||||
Provider attribution uses a different feature: 124-d residual ratios on 256 px
|
||||
patches. OpenAI versus Gemini is a strong split there. "AI or not" is the
|
||||
wrong job on that bank, which is why Model 2 is gated.
|
||||
|
||||
Provider names the renderer, not the product UI. Bing Image Creator signed
|
||||
Microsoft, OpenAI scores `openai`. Designer signed Microsoft, Google LLC
|
||||
scores `google`. There is no Microsoft pixel class. `openai` and `google` are
|
||||
provider classes. `muse-image` is Muse Image output, not a general Meta
|
||||
class. `tc260` is the China AIGC label standard, not one producer: Doubao,
|
||||
Jimeng, Qwen, Kling, and others share that residual class.
|
||||
|
||||
## Evaluation
|
||||
|
||||
Two CPU retrains were byte-identical. DEFINITELY is the shipped cut.
|
||||
|
||||
| Head | Cell | Result |
|
||||
| --- | --- | ---: |
|
||||
| Detector DEFINITELY | AI-test | 92.4% (n=1,847) |
|
||||
| Detector DEFINITELY | Open Images fresh | 1.3% (n=3,000) |
|
||||
| Detector DEFINITELY | Kodak | 0/24 |
|
||||
| Detector DEFINITELY | FLUX hold | 83.0% (n=300) |
|
||||
| Class | OpenAI | 90.8% (345/380 of 381) |
|
||||
| Class | Google | 90.9% (339/373 of 377) |
|
||||
| Class | TC260 | 78.6% (298/379 of 384) |
|
||||
| Class | Muse Image hold-out v3 | 89.4% (177/198) |
|
||||
| Class | Muse Image hold-out pooled | 88.4% (243/275 listed 277) |
|
||||
| Class | meme templates, ungated | 29.1% leak |
|
||||
|
||||
The ungated meme row is why Model 2 never runs on `human` or `unknown`.
|
||||
|
||||
## Limits
|
||||
|
||||
- Receipts, UI, scans, maps, and community art are out of contract.
|
||||
- Not SynthID, not C2PA, not `is_ai_generated`.
|
||||
- Images under 256 px cannot yield 124-d features, so provider abstains.
|
||||
- FLUX is a hold-out at 83% DEF, not a named provider class.
|
||||
- `tc260` is mixed producers under one label standard. A later retrain
|
||||
should split it by manufacturer.
|
||||
|
||||
`identify`, `has_invisible_target`, `all`, and `invisible` do not import this
|
||||
module. A no-signal provenance result stays unknown until you call `classify`
|
||||
yourself.
|
||||
|
||||
Training data, the retrain pack (caches only, no images), and
|
||||
`scripts/retrain_photo_classify.py` are in
|
||||
[photo-classify-training.md](photo-classify-training.md).
|
||||
@@ -6,12 +6,29 @@ and pipeline modules are intended for maintainers and specialized workflows.
|
||||
Dependency groups are identical for the CLI and Python API. The default install
|
||||
covers metadata extraction, normalization, verdict logic, and stripping.
|
||||
Array/pixel APIs use `pixels`; visible removal uses `visible`; DWT-DCT detection
|
||||
uses `detect`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and
|
||||
uses `detect`; pixel photo classification uses `classify`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and
|
||||
visible video processing uses `video`. Video SynthID removal is a separate VAE
|
||||
path that still runs on CPU and combines `video` and `diffusion`. Add `heif`
|
||||
independently when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See
|
||||
the complete [feature-extra matrix](installation.md#feature-extras).
|
||||
|
||||
## Verify OpenAI SynthID
|
||||
|
||||
```python
|
||||
import remove_ai_watermarks as raiw
|
||||
|
||||
result = raiw.verify_openai_synthid("input.png", acknowledge_upload=True)
|
||||
print(result.status)
|
||||
```
|
||||
|
||||
Official OpenAI JSON results expose `signal_family`,
|
||||
`provider_scope`, `backend`, `metadata_used_for_verdict`, and
|
||||
`pixels_preserved`. Remote transport and response failures raise `OpenAIProvenanceError`. Its
|
||||
`status_code`, `error_code`, `request_id`, `retry_after`, and `retryable`
|
||||
attributes let a caller implement bounded backoff or a circuit breaker without
|
||||
turning an API outage into a false `not_detected` result. One function call still
|
||||
performs at most one upload.
|
||||
|
||||
## Remove visible marks
|
||||
|
||||
Install `remove-ai-watermarks[visible]` before using the visible-removal API.
|
||||
@@ -129,6 +146,31 @@ engine's own parameter names and defaults. `force`, which decides whether the
|
||||
engine runs at all, is a parameter of `remove_all` and `remove_batch` alongside
|
||||
`backend` and `sensitivity`.
|
||||
|
||||
The complete field set, with the shipped defaults:
|
||||
|
||||
| Field | Default | Effect |
|
||||
| --- | --- | --- |
|
||||
| `strength` | `None` | Denoising strength. `None` resolves per profile and vendor cohort. |
|
||||
| `pipeline` | `"qwen-zimage"` | Profile: `qwen-zimage`, `sdxl-zimage`, `chroma-zimage`, or `auto`. |
|
||||
| `vendor` | `None` | Strength cohort, and an assertion that the watermark is present. |
|
||||
| `seed` | `None` | `None` uses the certified seed 0. Removal near the floor is seed-dependent. |
|
||||
| `hf_token` | `None` | Hugging Face token for the model download; the environment and a local `.env` are read when unset. |
|
||||
| `humanize` | `0.0` | Analog film grain. 0 is off; 2.0-6.0 is the useful band. |
|
||||
| `unsharp` | `0.0` | Unsharp-mask sharpening. 0 is off. |
|
||||
| `adaptive_polish` | `None` | `None` lets the profile decide (off for `qwen-zimage`, on for `sdxl-zimage`); `True`/`False` override it. |
|
||||
| `max_resolution` | `0` | Cap the long side before diffusion. 0 keeps native geometry and the most detail. |
|
||||
| `controlnet_conditioning_scale` | `1.0` | Canny ControlNet conditioning on the global stage. Higher stays closer to the original structure and text. |
|
||||
| `cpu_offload` | `False` | Stream model components between CPU and GPU. Lower CUDA memory, slower. |
|
||||
| `tile` | `False` | Regenerate large inputs in overlapping tiles instead of downscaling. |
|
||||
| `tile_size` | `1024` | Tile side in pixels when `tile` is set. |
|
||||
| `tile_overlap` | `128` | Tile overlap in pixels. More overlap hides seams and costs time. |
|
||||
| `text_manifest` | `None` | Path to an operator-verified text manifest; see the text-restoration section. |
|
||||
| `fidelity_anchor` | `False` | Opt into the fidelity-anchor pass. |
|
||||
|
||||
`tests/test_docs_cover_the_public_surface.py` fails when a field is added here
|
||||
and not to this table, because this seam had already drifted by eight fields
|
||||
before anything checked it.
|
||||
|
||||
If AI metadata survives the strip, `remove_all` raises `MetadataStripIncomplete`
|
||||
**before** writing anything: an AI-readable output is worse than no output.
|
||||
|
||||
@@ -144,6 +186,34 @@ print(summary.invisible_unavailable) # outputs that still carry the watermark
|
||||
`mode` is `all`, `visible`, `invisible`, or `metadata`. Pass a constructed
|
||||
`InvisibleEngine` as `engine` to load the model once for the whole directory.
|
||||
|
||||
## Classify a photograph from pixels
|
||||
|
||||
This is not provenance. `identify` does not call it. Install
|
||||
`remove-ai-watermarks[classify]` first.
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks.classify import classify_pixels
|
||||
|
||||
result = classify_pixels(Path("input.png"))
|
||||
print(result.label, result.detector, result.provider)
|
||||
```
|
||||
|
||||
`label` is `ai` only on a DEFINITELY detector result (ridge AND freeze MLP).
|
||||
POSSIBLY is `unknown`. Camera-like photographs are `human`. `provider` is
|
||||
`openai`, `google`, `muse-image`, or `tc260` only when `label` is `ai` and the 124-d
|
||||
head beats `no_ai` by the freeze margin. `tc260` is the China AIGC label
|
||||
standard (mixed producers), not a company. Otherwise it is `None`, including
|
||||
when 124-d extraction refuses the file.
|
||||
|
||||
`device` is a library parameter: `None` / `"auto"` detect, `"cpu"` or `"cuda"`
|
||||
pin. It is not a CLI option.
|
||||
|
||||
Missing extra raises `RuntimeError` with the quoted install command
|
||||
`'remove-ai-watermarks[classify]'`. Guide:
|
||||
[photo pixel classification](photo-classify.md).
|
||||
|
||||
## Inspect provenance
|
||||
|
||||
The default installation evaluates file metadata. Add `visible`, `detect`, or
|
||||
@@ -344,8 +414,8 @@ Timings and spatial artifacts are opt-in. Artifacts include image-identifying da
|
||||
such as a thumbnail and perceptual hash; aggregate feature families do not.
|
||||
|
||||
`identify_from_evidence` does not reopen the source file by default: it evaluates
|
||||
metadata only, and registered visible marks and pixel-backed invisible watermarks
|
||||
remain in the path-based `identify` call.
|
||||
metadata only, and the pixel-backed checks remain in the path-based `identify`
|
||||
call: registered visible marks and open invisible-watermark decoders.
|
||||
|
||||
Pass `image_path` together with `check_visible` or `check_invisible` to add those
|
||||
pixel detectors on top of the SAME evidence. That is how a caller asking one file
|
||||
@@ -587,7 +657,7 @@ to 8-bit SDR.
|
||||
|
||||
## Remove invisible watermarks
|
||||
|
||||
Install `remove-ai-watermarks[qwen-zimage]`. Both profiles need it, and both
|
||||
Install `remove-ai-watermarks[qwen-zimage]`. All three profiles need it, and all
|
||||
need an NVIDIA GPU.
|
||||
|
||||
```python
|
||||
@@ -596,7 +666,7 @@ from pathlib import Path
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine
|
||||
|
||||
engine = InvisibleEngine(
|
||||
pipeline="qwen-zimage", # the default; the only other value is "sdxl-zimage"
|
||||
pipeline="qwen-zimage", # the default; also "sdxl-zimage", "chroma-zimage", or "auto"
|
||||
device=None,
|
||||
cpu_offload=False,
|
||||
)
|
||||
@@ -620,7 +690,7 @@ engine = InvisibleEngine(
|
||||
)
|
||||
```
|
||||
|
||||
Both profiles are CUDA-only, so on a machine without an NVIDIA GPU `device=None`
|
||||
All profiles are CUDA-only, so on a machine without an NVIDIA GPU `device=None`
|
||||
resolves to `cpu` and construction raises. For the SDXL global stage instead of
|
||||
Qwen:
|
||||
|
||||
@@ -628,8 +698,10 @@ Qwen:
|
||||
engine = InvisibleEngine(pipeline="sdxl-zimage")
|
||||
```
|
||||
|
||||
The `qwen-zimage` extra is required for both profiles: each runs the same
|
||||
DiffSynth Z-Image face stage.
|
||||
The `qwen-zimage` extra is required for every profile, including `auto`: each
|
||||
concrete engine runs the same DiffSynth Z-Image face stage. `pipeline="auto"`
|
||||
selects chroma-zimage for OpenAI and Microsoft provenance and qwen-zimage
|
||||
otherwise, after the vendor is known and before strength resolution.
|
||||
|
||||
The opt-in verified-text stage uses the same `text_manifest` argument as the CLI:
|
||||
|
||||
@@ -643,11 +715,13 @@ engine.remove_watermark(
|
||||
|
||||
Install `remove-ai-watermarks[text-restoration]`. The manifest schema and safety
|
||||
constraints are documented in the CLI guide. The engine verifies its decoded RGB
|
||||
hash before loading the diffusion models and rejects SDXL, downscaling, and
|
||||
postprocessing combinations that were not evaluated. Tiling is also rejected because
|
||||
the combined tiled-restoration path has no provider-oracle calibration. `InvisibleOptions`
|
||||
exposes the same field for `remove_all`; after a visible-stage edit, the manifest must
|
||||
be built against the staged pixels rather than the pristine source.
|
||||
hash before loading the diffusion models. Qwen and Chroma reconstruct the donor with
|
||||
the VAE already loaded for the one profile selected by `auto`; no second generative
|
||||
profile runs. The engine rejects SDXL, downscaling, and postprocessing combinations
|
||||
that were not evaluated. Tiling is also rejected because the combined
|
||||
tiled-restoration path has no provider-oracle calibration. `InvisibleOptions` exposes
|
||||
the same field for `remove_all`; after a visible-stage edit, the manifest must be built
|
||||
against the staged pixels rather than the pristine source.
|
||||
|
||||
Use manifest schema 1 for manually reviewed text plus script metadata. Automated
|
||||
operators that verify only text-region geometry should emit schema 2 lines with a
|
||||
@@ -658,7 +732,8 @@ default** (`fidelity_anchor=False`): that whole-frame blend was measured to
|
||||
return detector-visible OpenAI SynthID on poster-scale manifests (official
|
||||
Content Provenance API, 2026-08-19 - detected x6 with the anchor, clean x6
|
||||
without it, controls and base outputs validated in the same sessions). Pass
|
||||
`fidelity_anchor=True` to reproduce the 0.27.0 research behavior.
|
||||
`fidelity_anchor=True` with `qwen-zimage` to reproduce the 0.27.0 research
|
||||
behavior. Chroma rejects that Qwen-specific reproduction flag.
|
||||
|
||||
### Drafting manifest lines
|
||||
|
||||
|
||||
@@ -73,6 +73,18 @@ Upgrading the Hugging Face Space is a separate, manual step: bump
|
||||
pin. Its callback smoke tests run in that repository's CI on every push; they
|
||||
are the scenario check for the demo surface.
|
||||
|
||||
The Hugging Face **model** `wiltodelta/raiw-photo-classify` is not part of that
|
||||
release fan-out. It holds the photo-classify freeze weights. The library extra
|
||||
`classify` downloads that snapshot on first use, or reads
|
||||
`RAIW_CLASSIFY_WEIGHTS`. Update the Hub repo only with
|
||||
`.github/workflows/publish-photo-classify-hf.yml` (`workflow_dispatch`,
|
||||
type `publish`). Full mode downloads the four weight files from the GitHub
|
||||
Release tag `photo-classify-freeze-2026-08-31` (or the tag you pass) and
|
||||
uploads them with the card in `docs/photo-classify-hf/`. Card mode updates
|
||||
the README and `operating-point.json` only. A library version bump does not
|
||||
upload new heads. Pin a Hub revision in production so a later freeze cannot
|
||||
silently change installed clients.
|
||||
|
||||
The workflow can also be started manually with an optional version input.
|
||||
|
||||
If a distribution job fails because a repository or Hugging Face credential is
|
||||
@@ -121,7 +133,24 @@ without removing schema 1 so long-lived consumers can update separately. A packa
|
||||
release must never silently substitute its latest schema when a caller explicitly
|
||||
requests an older supported one.
|
||||
|
||||
After publication, verify:
|
||||
After publication, the `verify-release.yml` workflow runs automatically once
|
||||
`distribute.yml` completes and checks every surface below. It can also be
|
||||
dispatched manually with a specific version. Every check fails the job; none of
|
||||
them warns and continues, because a surface nobody can confirm is not a verified
|
||||
surface, with one deliberate exception noted below. The ComfyUI check reads the
|
||||
registry's JSON API and asserts that some published node version declares this
|
||||
library release. It deliberately does NOT proxy through the sync workflow's run
|
||||
status: the node repository also runs that workflow on a daily schedule, so a
|
||||
green nightly run says nothing about whether this release was synced.
|
||||
|
||||
Read `/nodes/remove-ai-watermarks/versions`, not `latest_version` on the node
|
||||
object. The registry reviews an upload before activating it, and
|
||||
`latest_version` reports only the active version, so during that review window a
|
||||
correctly published version is indistinguishable from one that was never
|
||||
uploaded. A new version therefore sits in `NodeVersionStatusPending` for a while
|
||||
after a release; the check reports that state in the run summary and passes,
|
||||
because it is the registry's queue rather than an incomplete release. A missing
|
||||
version, or any other status, fails. The full checklist for completeness:
|
||||
|
||||
- both wheel and source distribution exist on PyPI;
|
||||
- the package version matches the tag;
|
||||
|
||||
@@ -20,7 +20,7 @@ The `visible` command registers these mark keys:
|
||||
| `runninghub` | `RunningHub AI生成` | Top left | Strict visual and position gates. |
|
||||
| `baidu` | `百度 AI生成` | Bottom right | Detector and extended removal footprint. |
|
||||
| `liblib` | `LiblibAI` | Bottom center | Includes a minimum image size gate. |
|
||||
| `microsoft` | One Microsoft white AI-badge variant | Top right | Strict gate; other documented icon, text, and position variants are not covered. |
|
||||
| `microsoft` | One Microsoft white AI-badge variant | Top right | Strict uses the visual gate; auto can use Microsoft provenance for the measured [relaxed gate](module-internals.md#visible-mark-removal). Other documented icon, text, and position variants are not covered. |
|
||||
| `jimeng_pill` | `AI生成` pill | Top left | Weak detector with additional product and background gates. |
|
||||
|
||||
`--mark auto` evaluates all registered marks and removes every selected match.
|
||||
@@ -30,12 +30,17 @@ the masked area.
|
||||
Marks from other vendors are not detected automatically. Use `erase --region`
|
||||
when you can select the affected area yourself.
|
||||
|
||||
Synthetic examples for every registered image and video mark live in the
|
||||
[visible-mark gallery](../data/fixtures/visible/README.md). They demonstrate the
|
||||
detectors' canonical geometry and house style, not vendor raster fidelity.
|
||||
|
||||
### Visible video marks
|
||||
|
||||
| Key | Mark | Motion | Important limit |
|
||||
| --- | --- | --- | --- |
|
||||
| `sora` | Sora 2 mascot and wordmark | Moves among frame positions | Requires a temporally recurring visual match; the older Sora Turbo corner swirl is a different unsupported mark. |
|
||||
| `veo` | Current four-point diamond and legacy `Veo` text | Fixed bottom-right corner | Uses separate silhouettes and requires a recurring match; learned fill is preferable on structured backgrounds. |
|
||||
| `doubao` | `豆包AI生成` text run | Fixed bottom-right corner | Reuses the image engine's synthetic alpha as the template; a stable 12-frame run above 0.35 is required. |
|
||||
| `seedance` | Boxed `AI` label | Fixed bottom-right corner | Requires an anchored recurring match; the full localized box is filled because a thinner synthetic shape mask leaves the real translucent rim behind. |
|
||||
| `dola` | `Dola AI` text | Fixed bottom-right corner | Requires an anchored recurring match; ByteDance or BytePlus provenance can relax only an existing visual run. |
|
||||
| `hailuo` | `MINIMAX \| hailuo AI` composite label | Fixed lower edge | Uses a synthetic waveform, text, separator, and ring silhouette; the complete recurring label box is filled. A TC260 label naming MiniMax as producer can relax only an existing stable run. |
|
||||
@@ -126,20 +131,33 @@ The `invisible` command uses diffusion regeneration. It targets watermark
|
||||
patterns by changing the image rather than decoding and deleting a known
|
||||
payload.
|
||||
|
||||
Current pipeline values, both CUDA-only:
|
||||
Current pipeline values, all CUDA-only:
|
||||
|
||||
- `qwen-zimage`, the default;
|
||||
- `sdxl-zimage`, the same recipe and the same face stage on an SDXL global pass.
|
||||
- `sdxl-zimage`, the same recipe and the same face stage on an SDXL global pass;
|
||||
- `chroma-zimage`, the same face stage on a Chroma1 global pass;
|
||||
- `auto`, chroma-zimage for OpenAI and Microsoft provenance, otherwise qwen-zimage.
|
||||
|
||||
The `controlnet`, `sdxl`, `qwen` and `default` values were removed. A retired name
|
||||
is rejected at parse time rather than remapped onto a surviving profile.
|
||||
|
||||
SynthID does not have a public local pixel decoder in this project. The tool
|
||||
recognizes presence from supported provenance: Google AI C2PA under Google's
|
||||
all-media watermark policy, and current OpenAI C2PA carrying an explicit
|
||||
`c2pa.watermarked.*` action. Legacy OpenAI C2PA without that action does not
|
||||
assert SynthID. After provenance metadata is removed, a local negative result
|
||||
is still inconclusive.
|
||||
Google does not publish the SynthID payload decoder. This package does not
|
||||
ship a local pixel detector for that watermark. Research on a periodic
|
||||
lattice expert is in [synthid-detector-research.md](synthid-detector-research.md)
|
||||
and `scripts/synthid_runtime/`.
|
||||
|
||||
The tool recognizes presence from supported provenance: Google AI C2PA
|
||||
under Google's all-media watermark policy, and current OpenAI C2PA carrying an
|
||||
explicit `c2pa.watermarked.*` action. Legacy OpenAI C2PA without that action
|
||||
does not assert SynthID.
|
||||
|
||||
The optional `verify-openai-synthid` command is a separate official remote
|
||||
verifier for supported OpenAI watermarks. It strips AI provenance metadata from
|
||||
a temporary PNG, JPEG, or WebP copy, proves that decoded RGBA pixels are
|
||||
unchanged, and uses only the API's SynthID result. It is therefore independent
|
||||
of C2PA for its decision, but it is not local: the sanitized raster is uploaded
|
||||
to OpenAI after explicit acknowledgement. It is intentionally excluded from
|
||||
`identify` and its negative result remains inconclusive.
|
||||
|
||||
Microsoft Paint can name `com.microsoft.invismark.1` in a C2PA soft-binding
|
||||
assertion. Inspection reports both that exact algorithm and its signed `value`,
|
||||
@@ -212,7 +230,7 @@ not a universal clean verdict.
|
||||
|
||||
| Provider or family | Visible | Invisible path | Metadata or provenance |
|
||||
| --- | --- | --- | --- |
|
||||
| Google Gemini | Sparkle | Diffusion regeneration for SynthID | C2PA and related source signals |
|
||||
| Google Gemini | Sparkle | Diffusion regeneration | C2PA and related source signals |
|
||||
| Google Veo video | Veo diamond and legacy text | Oracle-certified VAE removal for SynthID | C2PA and related source signals |
|
||||
| OpenAI image generators | None registered | Diffusion regeneration for supported invisible signals | C2PA and generator provenance |
|
||||
| Meta Muse Image | None on Muse output (legacy `Imagined with AI` unregistered) | Diffusion regeneration for Content Seal, oracle-verified on the default profile | XMP IPTC `trainedAlgorithmicMedia` companion tag; no local Content Seal decoder |
|
||||
|
||||
@@ -0,0 +1,518 @@
|
||||
# SynthID source classifiers (research)
|
||||
|
||||
> Research archive for metadata-free OpenAI/Gemini source finding and
|
||||
> provider-lineage classifiers. These are not SynthID payload decoders and are
|
||||
> not shipped product verdicts. Current behavior:
|
||||
> [supported signals](supported-signals.md) and
|
||||
> [known limitations](known-limitations.md).
|
||||
>
|
||||
> Sister pages: [general AI-generated image classifiers](ai-generated-image-classifiers.md),
|
||||
> [SynthID local detector](synthid-detector-research.md),
|
||||
> [SynthID mark removal](synthid-removal-research.md), and
|
||||
> [mechanism reference](synthid.md).
|
||||
|
||||
A source classifier is reliable only when photographs and non-target generators
|
||||
are explicit negatives, a strict rule can abstain, and any watermark claim uses
|
||||
an independent oracle. CLIP content embeddings and the 124-d origin-locked
|
||||
residual bank are different features for different jobs.
|
||||
|
||||
Krawetz's Gemini-chat TPR critique is a verifier-quality claim, not a
|
||||
feature we can ship. [Lead Stories, 2026-07](https://leadstories.com/analysis/2026/07/google-gemini-synthid-detector-confuses-results-within-same-chat.html)
|
||||
documented Gemini repeating the first file's SynthID verdict inside a
|
||||
chat; Google said that was fixed 2026-07-16. The OpenAI provenance API is
|
||||
a different endpoint.
|
||||
|
||||
## Provider names from pixels
|
||||
|
||||
The task is: given a file with no metadata, is this OpenAI, Gemini, or
|
||||
unknown, with almost no errors on camera photographs or other generators. That
|
||||
is this section. `unknown` does not mean `not AI`; it includes AI images from
|
||||
other providers and target-provider images the strict rule misses. This is not
|
||||
a general AI-generation detector or a SynthID detector. Firefly, PixelBin, and
|
||||
other generators have to sit in the test, because a head that only sees OpenAI
|
||||
versus Gemini versus COCO will call Firefly a provider.
|
||||
|
||||
Three-way `openai` / `google` / `other` on Model 1 embeddings fails the
|
||||
Firefly gate. CLIP-L-ft test accuracy 0.53; Firefly 35/31/18. CLIP-H 0.57;
|
||||
Firefly 36/33/15. OpenAI versus Gemini AUC on CLIP-L-ft is 0.845; on the
|
||||
124-d lattice bank it is 0.989. They are two pipelines, not one class.
|
||||
|
||||
### Renderer, not front-end (cross-carrier validation, 2026-08-26)
|
||||
|
||||
A local C2PA-labeled collection's issuer field allows a provider-label audit the
|
||||
family labels cannot express: Bing Image Creator rows signed `Microsoft,
|
||||
OpenAI` (renderer DALL-E, n=152), Microsoft-native rows (MAI-Image / Designer,
|
||||
n=85), Designer rows signed `Microsoft, Google LLC` (renderer Imagen, n=36),
|
||||
and the unattributed Instagram `made_with_ai` label set (n=275). Scored with
|
||||
the frozen 124-feature openai/google/no_ai cascade, no training, metadata
|
||||
used only to build cells: argmax called 124/152 (81.6%) of the Bing+DALL-E
|
||||
cell `openai` and 34/36 (94.4%) of the Microsoft+Imagen cell `google`. Pixel
|
||||
attribution therefore tracks the renderer behind the front-end, which is the
|
||||
correct semantic for a stripped file: a Bing export is an OpenAI-rendered
|
||||
image. The Microsoft-native cell split 48 openai / 35 google, consistent with
|
||||
Designer historically routing between DALL-E and Imagen rather than one
|
||||
pipeline. The Instagram `made_with_ai` set is not a Meta pixel class: argmax
|
||||
leaned `google` at 196/275 with 68 `openai`, and the photo-first 0.50 margin
|
||||
sent 182/275 to `no_ai`, so the label marks detected AI content of mixed
|
||||
origin, not Meta-rendered pixels. A real Meta class needs fresh
|
||||
`Imagined with AI` generations with known provenance; the catalog label is
|
||||
not one. Artifacts:
|
||||
`provider-renderer-cells-2026-08-26/report.json`.
|
||||
|
||||
### A Meta class from muse-image-1.0, 2026-08-26
|
||||
|
||||
The Meta Model API (`api.meta.ai/v1`, OpenAI-compatible images endpoint) made
|
||||
a known-provenance Meta corpus possible without any account browser session:
|
||||
61 images generated from a 61-prompt grid spanning portraits, product shots,
|
||||
scenes, food, animals, architecture, illustration styles, text posters, and
|
||||
abstract work across all three aspect ratios, plus the five oracle-verified
|
||||
oracle-verified content-seal samples already on disk. All 66 are content-hash
|
||||
unique, delivered at 1600x1600 / 1920x1280 / 1280x1920, and every API row
|
||||
carries IPTC `trainedAlgorithmicMedia` plus a Content Seal generation id
|
||||
recorded in the manifest. A four-head control with a `microsoft_native` class
|
||||
(83 issuer-verified rows) failed first: repeated-split argmax recall 0.38 mean
|
||||
with 0.16-0.56 spread, because Designer routes between renderers and the class
|
||||
is mixed. The same recipe with `meta_muse_image` in its place holds: argmax
|
||||
recall 0.86 mean (0.75-1.00) on just 66 images, openai 0.952 and google 0.956
|
||||
unharmed, and the frozen cross-carrier cells keep their renderer semantics
|
||||
(Bing+DALL-E 119/146 openai, Microsoft+Imagen 32/36 google) with only small
|
||||
meta leakage (14 and 3 argmax rows). The class label names what the corpus is:
|
||||
muse-image-1.0 API output, not the unverified assumption that the consumer
|
||||
Imagine feature renders identically. The full-data model calls 59/66 meta rows
|
||||
correctly, with seven leaking to google and none to openai. The photo-first
|
||||
0.50 margin does not transfer to a 66-image class (0.398 mean recall); argmax
|
||||
is the honest operating rule until the corpus grows. Muse Image is therefore
|
||||
a separable fourth provider class on pixels, unlike the Microsoft front-end.
|
||||
Artifacts: `meta-muse-corpus-2026-08-26/`, `four-head-provider-2026-08-26/`,
|
||||
`meta-provider-cells-2026-08-26/`.
|
||||
|
||||
A paired chat-vs-API check then closed the label question. Two prompts from
|
||||
the API grid (weathered fisherman portrait, white-sneaker product shot) were
|
||||
submitted to the consumer Meta AI chat imagine flow in the user's own logged-in
|
||||
browser session; the chat delivered 1280x1920 and 1920x1280 WebP files from the
|
||||
`t39.105495-1` CDN family, each carrying IPTC `trainedAlgorithmicMedia` in the
|
||||
downloaded bytes. Scored by the frozen API-trained four-head model, which had
|
||||
never seen a chat image: both chat rows landed on `meta_muse_image` under
|
||||
argmax (0.251 and 0.233) with score profiles matching their API twins almost
|
||||
exactly, and the margin rule agreed pairwise (fisherman meta at margin, sneaker
|
||||
no_ai at margin 0.50, same as its API counterpart). The chat imagine pipeline
|
||||
and `muse-image-1.0` are therefore pixel-indistinguishable to the provider
|
||||
classifier on this pair, so `meta_muse_image` honestly names both
|
||||
distributions. Caveat: n=2 paired prompts from one session; this is a
|
||||
consistency result, not a deployment-scale equivalence claim. Artifact:
|
||||
`meta-chat-check-2026-08-26/`.
|
||||
|
||||
Collapsing OpenAI and Gemini into one pixel class versus other generators
|
||||
does not fix that. Binary ridge AUC 0.686, TPR 75% at FPR 45%. Canva 98%,
|
||||
Microsoft 75%, Firefly 68% leak into the union; FLUX HF hold stays out at
|
||||
3%. Training the same union only against photographs recreates Model 1 with
|
||||
a narrower train set (fresh FPR 2.1%, Firefly still 95%).
|
||||
|
||||
`provider-report-clip-l-ft.json`, `provider-union-report.json`.
|
||||
|
||||
### Strict abstention does not recover provider identity
|
||||
|
||||
An `unknown` class makes the claim narrower, but neither existing
|
||||
representation has a useful high-precision operating point. A
|
||||
leave-one-foreign-family-out ridge on the 124-d residual bank set each cut
|
||||
above every calibration negative. Mean OpenAI recall fell to 1.3% and mean
|
||||
Google recall to 4.4%, while a held-out foreign family was still attributed to
|
||||
a provider up to 5.0% of the time and photographs were called OpenAI up to
|
||||
0.52% of the time.
|
||||
|
||||
The CLIP-L-ft head has the same failure at a 0.1% calibration-negative cut.
|
||||
On 400 disjoint queries per provider, OpenAI recall was 1.0% and Google recall
|
||||
was 0.5%. It still called 1/11 held-out FLUX images OpenAI and 1/750 COCO
|
||||
photographs OpenAI. Zero calibration false positives reduced both provider
|
||||
recalls to zero. These are abstaining content classifiers, not precise source
|
||||
finders.
|
||||
|
||||
A nonlinear head and local-neighbor decision do not expose a hidden provider
|
||||
partition in the same embeddings. Across five balanced 768-128-3 MLP seeds,
|
||||
strict test recall ranged from 0.5-7.0% for OpenAI and 0.75-1.0% for Google,
|
||||
with 6-19 false attributions among 6,550 locked negatives. The best
|
||||
calibration-selected `k=3` local-neighbor rule reached 0.5% OpenAI and 3.25%
|
||||
Google recall and still called one TC260 image Google.
|
||||
|
||||
### Pixel finetuning learns the train split, not an exact provider boundary
|
||||
|
||||
The last two CLIP-L vision blocks were then finetuned directly for
|
||||
`openai` / `google` / `unknown`: 9,063 fit images, 3,537 disjoint calibration
|
||||
images, 400 balanced steps, and random JPEG 40-95, 85-100% crop, and mild blur.
|
||||
Each provider cut was placed above every calibration negative. Calibration
|
||||
recall was 4.3% OpenAI and 5.2% Google.
|
||||
|
||||
The time-disjoint locked result was 6/400 OpenAI and 10/400 Google. One Google
|
||||
image and one TC260 image were called OpenAI. All 500 unseen-AI controls and
|
||||
all 4,945 locked photographs stayed `unknown`, including 3,000 fresh Open
|
||||
Images, but that photo specificity does not repair an AI-source error. An
|
||||
oracle cut above both locked OpenAI errors leaves only 1/400 OpenAI; it is an
|
||||
upper bound, not a valid post-test threshold. The model is not shippable.
|
||||
|
||||
The independent high-frequency route is already closed at the tested
|
||||
capacity. A four-layer opponent-residual patch CNN reached AUC 0.44-0.56
|
||||
against foreign generators, reversed to 0.15 on a fresh era, and accepted
|
||||
95-100% of several held-out Firefly, Microsoft, fal.ai, and PixelBin families
|
||||
at its photo-median threshold. It learned AI rendering versus photography,
|
||||
not vendor identity.
|
||||
|
||||
### External surrogate and forensic-descriptor audit
|
||||
|
||||
The public
|
||||
[`newideas99/gpt-image-synthid-detector`](https://github.com/newideas99/gpt-image-synthid-detector/tree/5495e09)
|
||||
does not supply a causal SynthID contrast. Its negatives are lightly
|
||||
regenerated positive images, so the trained ResNet/EfficientNet ensemble can
|
||||
read the regeneration pipeline. On a blind 517-file local pilot, OpenAI versus
|
||||
all AUC was 0.630. At the repository's 0.5 cut it retained 92/100 OpenAI and
|
||||
accepted 307/417 negatives, including 104/120 Open Images, 26/30 COCO, 34/50
|
||||
Google, and 8/10 Firefly. A later exact repeat on the hash-disjoint v7 challenge
|
||||
retained 172/200 OpenAI but accepted 110/200 Google. It therefore fails source
|
||||
specificity before any photograph gate is considered. It is a visual-domain
|
||||
classifier, not an independent confirmation signal.
|
||||
|
||||
The current [`aloshdenny/reverse-SynthID`](https://github.com/aloshdenny/reverse-SynthID/tree/b110836)
|
||||
V4 codebook also adds no useful hybrid evidence. A pickle-free exact inference
|
||||
repeat on v7 accepted 77/200 Google and 76/200 OpenAI at its published 0.52
|
||||
threshold. Applied only to v11 `unknown` rows, that threshold would rescue 26
|
||||
Google files while misrouting two OpenAI files. The older V3 published cut
|
||||
would add two v11 Google misses, but it previously accepted 5/499 controls and
|
||||
6/1,000 fresh Open Images. A 1%-recall OR rule with that measured false-positive
|
||||
history is also rejected.
|
||||
|
||||
The public [`Ristellise/REGRET`](https://github.com/Ristellise/REGRET/tree/7d449034bf323987e7e608e7886e029ed20fd847)
|
||||
SPAM model is another forensic descriptor, not a decoder. The audited pickle
|
||||
contained only an sklearn pipeline, scaler, logistic regression, and numeric
|
||||
numpy globals; inference used an exact restricted allowlist. At the published
|
||||
0.5 cut it accepted 139/200 Google and 141/200 OpenAI. On the disjoint public
|
||||
extension, the same published cut also accepted 214/600 ImageNet photographs,
|
||||
14/75 BigGAN, 17/75 Midjourney, 52/75 SDXL, and 10/75 VQDM. It adds no safe v11
|
||||
rescue.
|
||||
|
||||
[`vordme2010/synthid-dataset`](https://github.com/vordme2010/synthid-dataset/tree/133a27088f6f4d695c79db9a1a70fa8e7fa3adad)
|
||||
publishes a useful flat-field corpus but an invalid open-world classifier
|
||||
contrast. Its Tier-1 matrix has 500 Gemini-flat positives and 1,500 synthetic,
|
||||
spectrum-matched, or phase-scrambled negatives, with no real negative. The
|
||||
33 features include noise scale and radial power as well as six hand-selected
|
||||
carrier bins. Rebuilding the repository's seed-42 RBF SVM from the safe numeric
|
||||
matrix, without loading joblib, accepted 1/200 current Google and 0/200 OpenAI
|
||||
on v7. The reported AUC above 0.999 measures the synthetic negative recipe and
|
||||
flat renderer epoch; it cannot confirm the current source finder or a SynthID
|
||||
payload.
|
||||
|
||||
[Forensic Self-Descriptions](https://github.com/ductai199x/Forensic-Self-Descriptions-CVPR25/tree/50f2eae)
|
||||
(CVPR 2025) is a genuinely different representation: constrained prediction
|
||||
residuals are summarized as a 960-d forensic descriptor. Its ready attribution
|
||||
head is not usable here. Three current OpenAI files were all called `Real`,
|
||||
while one Microsoft file was called `GPT-Image 1/1.5` at confidence 0.863.
|
||||
|
||||
A custom source head on a native 256-pixel center crop was more informative.
|
||||
PCA-64 plus logistic regression, with each provider cut above every calibration
|
||||
negative, gave OpenAI AUC 0.872 and 3/50 strict OpenAI recall with 0/388 test
|
||||
false attributions. Its Google head failed on Bytedance, Canva, FLUX,
|
||||
Microsoft, and an Open Images photograph. On a later unused-hash challenge the
|
||||
OpenAI FSD head alone accepted 14/300 OpenAI, 3/300 TC260, and 3/90 PixelBin.
|
||||
FSD is therefore an independent source cue, not a sufficient classifier and
|
||||
not a watermark statistic.
|
||||
|
||||
### Rejected narrow two-signal cascades, 2026-08-23
|
||||
|
||||
The visible Gemini sparkle is absent from the target generation and is not a
|
||||
feature or a gate. Two metadata-free intersections did survive a new challenge:
|
||||
|
||||
| Cascade | Development | Unused-hash challenge |
|
||||
| --- | --- | --- |
|
||||
| Google provider probability >= 0.8876249 **and** registered-v3 lattice >= 1.0 | 3/45 Google, 0/30 lattice-positive foreign files | 3/300 Google, 0/690 OpenAI/TC260/PixelBin |
|
||||
| OpenAI provider probability >= 0.8822824 **and** FSD probability >= 0.9912320 | The two cuts were frozen independently | 3/300 OpenAI, 0/690 Google/TC260/PixelBin, 0/500 photographs |
|
||||
|
||||
The Google result is a low-recall intersection of a content head and an
|
||||
origin-locked renderer lattice. A two-pixel crop destroys the lattice, so an
|
||||
indeterminate result says nothing about SynthID. It is evidence for a narrow
|
||||
Gemini/Imagen pipeline, not a public decoder.
|
||||
|
||||
All three OpenAI challenge positives were 1086x1448 RGB PNGs, the measured
|
||||
ChatGPT export geometry. Thirty known non-OpenAI files at exactly 1086x1448
|
||||
(TC260, Apple, Microsoft, Canva, Firefly, Google, and others) produced zero
|
||||
first-gate positives. A scan of 1,194 unique files at that geometry with no
|
||||
provider or platform label produced 29 pixel-gate candidates and four final
|
||||
hybrid candidates. Three had only PNG image-data chunks; the fourth carried
|
||||
generic DPI/orientation fields but no provider or provenance label. Visual
|
||||
inspection found four plausible AI generations. They remain candidates, not
|
||||
ground truth, until a held-out oracle or original source confirms them.
|
||||
|
||||
The full luma 16-32-pixel phase-ring head is not a third confirmation. It
|
||||
rejected all three known and all four unlabeled OpenAI hybrid candidates while
|
||||
accepting 3/30 known non-OpenAI same-geometry controls. Its global test AUC was
|
||||
0.600 for OpenAI and it stays closed.
|
||||
|
||||
These cascades established that independent intersections could suppress the
|
||||
observed false positives, but about 1% recall is not useful for the source-hunt
|
||||
task. They are rejected as operating detectors. The FSD implementation and
|
||||
weights are also research-only under CC BY-NC-SA 4.0.
|
||||
|
||||
Local artifacts: `lattice-pixel-hybrid-challenge.json`,
|
||||
`fsd-openai-challenge-report.json`,
|
||||
`unlabeled-1086x1448-openai-hybrid-candidates.json`. The research entry point
|
||||
is `.local-eval/synthid/ai-photo-2026-08-22/hybrid_source_finder.py`.
|
||||
|
||||
### Canonical forensic source finder, 2026-08-23
|
||||
|
||||
An overlooked pixel representation materially improves the strict result. It
|
||||
normalizes each file to 512x512, then extracts 1,794 metadata-free features
|
||||
from opponent-color residuals: patch power and variation, phase coherence,
|
||||
autocorrelation, low-bit correlations, and residual-error histograms. The
|
||||
OpenAI branch intersects its canonical forensic score with a weak provider
|
||||
pixel vote. The Gemini branch intersects a forensic/provider fusion with the
|
||||
registered-v3 lattice. The visible Gemini sparkle remains excluded.
|
||||
|
||||
The single-scale cuts were frozen before a third hash-disjoint challenge. The
|
||||
result was:
|
||||
|
||||
| Challenge cell | OpenAI candidate | Gemini candidate | Any candidate |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| OpenAI | 51/300 | 1/300 | 52/300 (17.3%) |
|
||||
| Google | 0/300 | 34/300 | 34/300 (11.3%) |
|
||||
| Open Images photographs | 1/1,000 | 0/1,000 | 1/1,000 (0.1%) |
|
||||
| TC260 | 0/7 | 0/7 | 0/7 |
|
||||
|
||||
For the requested union, 86/600 OpenAI-or-Google targets were found, 14.3%
|
||||
recall instead of the rejected cascades' 1%. One OpenAI file crossed through
|
||||
the Gemini branch, and one known photograph crossed through the OpenAI branch.
|
||||
The seven remaining unused TC260 files are too few to establish specificity,
|
||||
and no fresh PixelBin, Firefly, Microsoft, or other generator hashes remained
|
||||
outside the earlier model and challenge manifests.
|
||||
|
||||
This v3 result established a strict source-candidate finder, not an exact
|
||||
identifier or a SynthID decoder. It does not recover a payload, and an
|
||||
`unknown` result says nothing about whether SynthID is present.
|
||||
|
||||
Local artifacts: `source-finder-v3-selection.json`,
|
||||
`source-finder-v3-challenge.json`, `google-canonical-forensic.json`, and
|
||||
`google-forensic-pixel-fusion.json`. The same local research entry point now
|
||||
implements this rule without the FSD dependency.
|
||||
|
||||
### Multiscale strict finder, 2026-08-24
|
||||
|
||||
The 512x512 normalization was discarding usable scale-specific evidence. A
|
||||
frozen union now takes the strongest OpenAI forensic response over 512, 640,
|
||||
768, and native views, gated by the provider pixel head. The Google branch
|
||||
takes the strongest forensic/provider fusion over 384, 512, 640, 768, and
|
||||
native views, then requires the same registered-v3 lattice. Development on
|
||||
the second and third challenges found 539/1,200 targets and 0/1,657
|
||||
photograph/TC260 controls.
|
||||
|
||||
The views and cuts were written before opening a fourth challenge whose hashes
|
||||
were disjoint from every model manifest and the first three challenges:
|
||||
|
||||
| Challenge cell | OpenAI candidate | Gemini candidate | Any candidate |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| OpenAI | 92/300 | 11/300 | 103/300 (34.3%) |
|
||||
| Google | 0/300 | 173/300 | 173/300 (57.7%) |
|
||||
| Open Images photographs | 0/1,000 | 0/1,000 | 0/1,000 |
|
||||
| TC260 | 1/25 | 0/25 | 1/25 |
|
||||
|
||||
For the requested union, the blind result is 276/600, 46.0% recall, with
|
||||
1/1,025 non-target candidates. This is 3.2 times the single-scale v3 recall
|
||||
and 46 times the rejected 1% cascades. The one false candidate is TC260, not a
|
||||
camera photograph. Eleven OpenAI files crossed through the Gemini branch;
|
||||
that is a provider-attribution error but still a correct hit for the declared
|
||||
OpenAI-or-Google union.
|
||||
|
||||
This remains a source-candidate finder, not an exact identifier or a SynthID
|
||||
decoder. Fresh unused paths from the other generator families were not
|
||||
available for v4, so the 0.1% observed non-target rate is not an open-world
|
||||
precision claim. Robustness to crop, resize, re-encoding, and screenshot
|
||||
capture is also not established. Keep the models and paths in `.local-eval`;
|
||||
do not add a runtime or public CLI until a new temporal challenge with fresh
|
||||
foreign-generator families establishes positive precision.
|
||||
|
||||
Local artifacts: `source-finder-v4-selection.json`,
|
||||
`source-finder-v4-rule.json`, `source-finder-v4-challenge.json`, and
|
||||
`multiscale-forensic-development.json`. The local research entry point
|
||||
implements the frozen multiscale rule and still uses no metadata or visible
|
||||
sparkle.
|
||||
|
||||
A post-hoc OR over every per-view zero-development-error OpenAI cut is
|
||||
rejected. It raised v4 OpenAI recall to 153/300 but also accepted 6/1,000
|
||||
photographs and 3/25 TC260 controls. The apparent union of many individually
|
||||
strict cuts was multiple-testing overfit, not additional independent evidence.
|
||||
|
||||
### Original-export hybrids, 2026-08-24
|
||||
|
||||
Three more hash-disjoint challenges tested whether multiscale fusion could be
|
||||
made useful without metadata. The v5 ExtraTrees union improved exact provider
|
||||
recall to 363/600 (60.5%) and provider-union recall to 373/600 (62.2%), but it
|
||||
also accepted 8/1,000 photograph and foreign-generator controls. A revised
|
||||
Google confirmation removed those eight development errors. Adding an
|
||||
AI-versus-camera gate in v6 did not transfer: exact recall fell to 327/600
|
||||
(54.5%) and union recall was 341/600 (56.8%).
|
||||
|
||||
The specificity failures exposed a stronger but narrower signal. Current
|
||||
OpenAI exports in these sets are PNGs produced with adaptive scanline filters,
|
||||
while the earlier TC260 error was a PNG encoded with filter zero on every row.
|
||||
A strict PNG parser now requires a non-interlaced PNG with at least one adaptive
|
||||
filter before the OpenAI branch can emit a result. This reads the image container
|
||||
and pixels, not EXIF, C2PA, a filename, or a visible label. It also changes the
|
||||
claim: a re-encoded OpenAI JPEG must abstain.
|
||||
|
||||
The complete frozen v7 rule reached 215/400 exact provider matches (53.8%) and
|
||||
221/400 provider-union matches (55.3%) on a new challenge. Its cells were
|
||||
114/200 exact OpenAI and 101/200 exact Google. The PNG gate repaired the
|
||||
observed specificity problem, but the old OpenAI forensic head remained the
|
||||
recall bottleneck.
|
||||
|
||||
A subsequent v8 development hybrid trains an ExtraTrees OpenAI head on v4-v5
|
||||
multiscale forensic scores, pixel probabilities, and PNG encoding structure.
|
||||
Model selection used v6. The final 0.47 precision cut was chosen after v7 had
|
||||
been opened, so the following is a post-hoc development measurement, not
|
||||
another blind result:
|
||||
|
||||
| v7 cell under v8 development rule | OpenAI | Gemini | Unknown |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| OpenAI | 190/200 | 3/200 | 7/200 |
|
||||
| Google | 0/200 | 102/200 | 98/200 |
|
||||
|
||||
That is 292/400 exact provider matches (73.0%) and 295/400 provider-union
|
||||
matches (73.8%).
|
||||
|
||||
The remaining Google miss set contained two different export pipelines: PNG
|
||||
and JPEG. A second development branch parses only JPEG codestream parameters,
|
||||
including quantization tables, chroma sampling, and progressive encoding; it
|
||||
explicitly skips APP0-APP15 and COM segments. Training one Google model per
|
||||
encoding class on v4 and selecting zero-validation-error cuts on v5-v6 raised
|
||||
the v11 transfer result to:
|
||||
|
||||
| v7 cell under v11 development rule | OpenAI | Gemini | Unknown |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| OpenAI | 190/200 | 4/200 | 6/200 |
|
||||
| Google | 0/200 | 126/200 | 74/200 |
|
||||
|
||||
This is 316/400 exact provider matches (79.0%) and 320/400 provider-union
|
||||
matches (80.0%).
|
||||
|
||||
This is the best local source finder in the campaign, but it is still not a
|
||||
SynthID detector, payload decoder, or open-world precision proof. The v8 rule
|
||||
is post-hoc, and neither v8 nor v11 has an independent open-world negative
|
||||
proof. The OpenAI branch is intentionally scoped to original-style PNG exports.
|
||||
A new temporal blind challenge with new foreign generators and PNG
|
||||
camera/editor controls is required before a runtime or public CLI is justified.
|
||||
|
||||
Local artifacts: `source-finder-v7-selection.json`,
|
||||
`source-finder-v7-challenge.json`, `source-finder-v8-rule.json`,
|
||||
`source-finder-v8-openai-extra-trees.joblib`, and
|
||||
`source-finder-v11-google-per-codec.joblib`.
|
||||
|
||||
### Published few-shot attribution also fails the open-world gate
|
||||
|
||||
[OmniDFA](https://arxiv.org/abs/2509.25682) is a purpose-built few-shot source
|
||||
attributor rather than a generic content embedding. Its published `part1`
|
||||
checkpoint is the correct unseen-generator fold for DALL-E 2 and DALL-E 3:
|
||||
those generators are in `part1` validation and absent from its training list.
|
||||
The same checkpoint has seen Imagen, so its Google result is not a clean
|
||||
unseen-Imagen benchmark; the OpenAI result is sufficient to reject the shared
|
||||
runtime.
|
||||
|
||||
With 20 support images per provider and provider-specific similarity plus
|
||||
margin cuts calibrated to zero false attributions over 160 negatives, a
|
||||
content-hash-disjoint 745-image evaluation produced:
|
||||
|
||||
| Cell | Result |
|
||||
| --- | ---: |
|
||||
| OpenAI recall | 9/50 (18%) |
|
||||
| Google recall | 4/50 (8%) |
|
||||
| Microsoft called OpenAI | 3/15 (20%) |
|
||||
| Kodak called OpenAI | 3/24 (12.5%) |
|
||||
| Canva called Google | 1/15 (6.7%) |
|
||||
| fal.ai called OpenAI | 1/15 (6.7%) |
|
||||
| xAI called OpenAI | 1/15 (6.7%) |
|
||||
| unseen Higgsfield called OpenAI | 1/11 (9.1%) |
|
||||
| fresh Open Images / COCO false attributions | 0/100 / 0/50 |
|
||||
|
||||
Provider multimodality is not the missing fix. Choosing 1-10 spherical
|
||||
prototypes only by calibration recall selected five: test recall fell to 16%
|
||||
OpenAI and 4% Google, while false attributions remained on Firefly (2/15),
|
||||
Kodak (2/24), Microsoft, ByteDance, TC260, and Made-with-AI samples.
|
||||
|
||||
Native files already fail, so JPEG, resize, crop, and screenshot variants were
|
||||
not run for OmniDFA. Do not add a provider-attribution runtime or CLI from that
|
||||
model. General exact OpenAI/Gemini identification remains unsupported. The
|
||||
strict source finder above emits candidates; it does not read the SynthID
|
||||
payload.
|
||||
|
||||
### 124-d lattice as pipeline ID, not a vendor CLIP head
|
||||
|
||||
Provider-class ridge on 124 native residual features (70/30 once, not a
|
||||
watermark gate). OpenAI L1 n=285, Google corpus n=533, foreign n=218, COCO
|
||||
n=289: OpenAI vs COCO 0.965; Google vs COCO 0.999; OpenAI vs Google 0.989;
|
||||
OpenAI vs foreign 0.725; Google vs foreign 0.922. OpenAI vs Firefly-class
|
||||
is the weak cell.
|
||||
|
||||
One-vs-rest: Google head at TPR 90% has FPR 0% vs COCO, 14% vs foreign, 2%
|
||||
vs OpenAI. A Gemini-like pixel class is close to what `pipeline_lattice`
|
||||
already is. An OpenAI-like pixel class on this bank would label Firefly as
|
||||
OpenAI about half the time and is not shippable.
|
||||
|
||||
Three-class `openai` / `google` / `no_ai` on 2,000 catalog OpenAI, 2,000
|
||||
catalog Google, and 1,936 COCO photos. Photo-first margin 0.50: openai
|
||||
74.7%, google 78.9%, no_ai 99.8%; Kodak 24/24 `no_ai`. Other generators
|
||||
are leakage, not classes:
|
||||
|
||||
| Platform | n | openai | google | no_ai |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Firefly | 106 | 37 | 27 | 42 |
|
||||
| Microsoft | 117 | 39 | 22 | 56 |
|
||||
| PixelBin | 90 | 14 | 46 | 30 |
|
||||
| HuggingFace job | 82 | 3 | 62 | 17 |
|
||||
| ByteDance C2PA | 86 | 5 | 35 | 46 |
|
||||
| SD / Comfy | 120 | 30 | 11 | 79 |
|
||||
| fal.ai | 98 | 11 | 18 | 69 |
|
||||
| Made-with-AI tag | 115 | 4 | 32 | 79 |
|
||||
| TC260 | 118 | 1 | 13 | 104 |
|
||||
| xAI | 114 | 1 | 20 | 93 |
|
||||
| Canva | 76 | 11 | 3 | 62 |
|
||||
| Apple Clean Up | 114 | 1 | 23 | 90 |
|
||||
| Aweme | 37 | 2 | 7 | 28 |
|
||||
| FLUX | 11 | 0 | 0 | 11 |
|
||||
| Reve | 10 | 0 | 0 | 10 |
|
||||
| NovelAI | 9 | 0 | 0 | 9 |
|
||||
| Higgsfield | 11 | 1 | 5 | 5 |
|
||||
|
||||
PixelBin and HuggingFace jobs lean `google` (shared renderer lineage).
|
||||
FLUX, NovelAI, and Reve stay `no_ai`. Local probe:
|
||||
`uv run python .local-eval/synthid/prc-oklab-attack-2026-08-15/classify_openai_gemini.py image.png`.
|
||||
|
||||
## Research lattice expert (google-lineage renderer)
|
||||
|
||||
Not a watermark and not in `identify`. `scripts/synthid_runtime/`
|
||||
`detect_synthid` re-check on 628 frozen holdouts, seed 20260822, threshold
|
||||
1.0.
|
||||
|
||||
| Family | n | detected | rate | max score |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Google / Gemini | 80 | 45 | 0.56 | 3.03 |
|
||||
| Firefly | 84 | 15 | 0.18 | 3.00 |
|
||||
| PixelBin | 80 | 11 | 0.14 | 2.48 |
|
||||
| Microsoft | 60 | 2 | 0.03 | 2.40 |
|
||||
| OpenAI | 80 | 1 | 0.01 | 1.38 |
|
||||
| xAI | 40 | 1 | 0.03 | 1.26 |
|
||||
| FLUX HF | 40 | 0 | 0 | 0.85 |
|
||||
| TC260 | 40 | 0 | 0 | 0.97 |
|
||||
| Kodak | 24 | 0 | 0 | 0.49 |
|
||||
| Open Images fresh | 60 | 0 | 0 | 0.64 |
|
||||
| COCO hold | 40 | 0 | 0 | 0.74 |
|
||||
|
||||
Firefly 18% and PixelBin 14% match the 2026-08-16 signed-foreign rates
|
||||
(24% and 14%) in order of magnitude. Both Microsoft hits have issuer
|
||||
`Microsoft, Google LLC`. A 2 px crop killed every sampled positive,
|
||||
including Firefly and PixelBin. Google TPR 56% is mixed collection eras, not
|
||||
the oracle-positive 147/148 cell. Honest name:
|
||||
`google_lineage_renderer` = Gemini/Imagen + Firefly + PixelBin.
|
||||
|
||||
Registered-v3 photographic controls remain 0/5,993 Open Images and 0/2,366
|
||||
COCO. Against 223 C2PA-named non-Google generators on 2026-08-16: 29
|
||||
accepted (0.130), Firefly 0.241. `.local-eval/synthid/lattice-check-2026-08-22/`.
|
||||
|
||||
## Local collection note (2026-08-21)
|
||||
|
||||
Unlabeled rows in the local C2PA-labeled collection are not photographs.
|
||||
Microsoft and Firefly rows also carry `synthid_from_provenance=true`, so
|
||||
that flag is not an OpenAI-plus-Gemini class.
|
||||
@@ -0,0 +1,591 @@
|
||||
# SynthID local detector research
|
||||
|
||||
> Research archive for the hunt for a local, keyless SynthID pixel detector.
|
||||
> Not a statement of current product capability. Shipped behavior:
|
||||
> [supported signals](supported-signals.md) and
|
||||
> [known limitations](known-limitations.md).
|
||||
>
|
||||
> Sister pages: [SynthID source classifiers](synthid-classifiers.md),
|
||||
> [SynthID mark removal](synthid-removal-research.md),
|
||||
> [mechanism reference](synthid.md). Dated measurements live in the
|
||||
> [chronological plan](synthid-detector-removal-plan.md).
|
||||
|
||||
## Result
|
||||
|
||||
The hunt for a local, keyless OpenAI SynthID pixel detector is open.
|
||||
Google has no public payload decoder, and nothing in this project
|
||||
reproduces one. Closed routes below failed to read the official oracle
|
||||
contrast on photographs. The remaining work is a new waveform, a new
|
||||
pair mint, or a new statistic, not a rerun of those routes.
|
||||
|
||||
The mark behaves like a keyed spread-spectrum residual: a content-adaptive
|
||||
`x' = x + g(x)` with a paired decoder (DeepMind patent family, optional
|
||||
secret in intermediate layers, encoder/decoder ensembles that do not
|
||||
recognize each other). Without that pair, the residual sits under the
|
||||
scene. Keyless energy, TrustMark transfer, L1 distillation, a flat-field
|
||||
stamp, and a 16-32 band student all failed to read the official oracle
|
||||
contrast on photographs.
|
||||
|
||||
S4, 2026-08-15: the two providers are not doing the same thing. Cross-image
|
||||
correlation of the folded residual is high for Google (tile16 pos-pos
|
||||
`+0.326`, chance `0.036`) and at chance for OpenAI (`+0.032`). Google
|
||||
shares one fixed phase-coherent pattern, also present in its controls at
|
||||
about half the amplitude. OpenAI shares nothing, which is what a
|
||||
content-dependent post-hoc encoder produces. Comb experts that work on
|
||||
Google therefore cannot be reused as an OpenAI watermark detector.
|
||||
|
||||
M2, 2026-08-15, re-verified 2026-08-16: Google unwatermarked pairs cannot
|
||||
be minted. `imagen-*` ids 404 from the model garden. `gemini-*-image`
|
||||
rejects `addWatermark` (`Cannot find field`). Current Gemini API docs
|
||||
state that all generated images include a SynthID watermark. There is no
|
||||
encoder-off Google path.
|
||||
|
||||
What the product uses for the *watermark* is signed provenance and
|
||||
`verify-openai-synthid`. The periodic-lattice expert is research-only under
|
||||
`scripts/synthid_runtime/` and is not called from `identify` or the CLI.
|
||||
Lineage measurements of that expert are in
|
||||
[SynthID source classifiers](synthid-classifiers.md).
|
||||
|
||||
## Public GitHub sweep, 2026-08-24
|
||||
|
||||
GitHub's primary-repository search returned 78 repositories with `synthid` in
|
||||
the name and 81 with it in the description, 133 unique repositories in total.
|
||||
The fork-inclusive searches were much larger and capped or dominated by copies
|
||||
(860 name hits and 1,311 description hits), so this is not a claim that every
|
||||
fork or unindexed private repository was read. The sweep also used targeted
|
||||
code searches for detector, verifier, correlation, codebook, bypass, and
|
||||
removal implementations. SynthID-Text, visible-corner-mark removers, metadata
|
||||
cleaners, wrappers, and literal ports were classified before inspecting the
|
||||
remaining image-pixel candidates at pinned commits.
|
||||
|
||||
The audits are not null results. They separate three claims that must not be
|
||||
collapsed: a repeatable image statistic exists, that statistic is a calibrated
|
||||
single-image SynthID detector, and changing its score transfers to the
|
||||
provider's production decoder. The repositories support parts of the first
|
||||
claim. None supports the second or third on the current challenge data.
|
||||
|
||||
| Repository | Retained contribution | Rejected use | Decisive result |
|
||||
| --- | --- | --- | --- |
|
||||
| [`Rinne414/SynthID-detector`](https://github.com/Rinne414/SynthID-detector/tree/b08abff06b67c37db9dfa7c6ed63760a446d560f) | A reproducible fixed-residual correlator and a useful spoofability negative control | Current-image detector or fixed-template remover | All three templates detected 0/400 current OpenAI and Google images. Adding the expected template spoofed the public detector, while subtracting 0.25-2x from an official positive left SynthID `detected` down to 25.72 dB |
|
||||
| [`newideas99/gpt-image-synthid-detector`](https://github.com/newideas99/gpt-image-synthid-detector/tree/5495e09c62dc26112f9a0e6cfab469f9a10f37e3) | Evidence that a CNN can learn a renderer/regeneration-domain signature, plus a white-box shortcut baseline | Payload decoder, independent hybrid confirmation, or removal oracle | The published cut retained 172/200 OpenAI but accepted 110/200 Google. A gradient drove the surrogate effectively to zero, but official SynthID stayed `detected` through +/-8/255 at 34.30 dB |
|
||||
| [`aloshdenny/reverse-SynthID`](https://github.com/aloshdenny/reverse-SynthID/tree/b11083676fd3ee3ff97ce9d03c0e409e46905902) | Cross-color phase consensus and further evidence for a repeating Google renderer-lineage pattern | The published V3/V4 thresholds as presence detectors, or Round-06 as a verified quiet remover | V4 accepted 77/200 Google and 76/200 OpenAI at the same cut. Round-06 has no published per-image oracle evidence and uses regeneration/distortion with 14/11 dB PSNR floors |
|
||||
| [`Ristellise/REGRET`](https://github.com/Ristellise/REGRET/tree/7d449034bf323987e7e608e7886e029ed20fd847) | A compact 147-d second-order SPAM descriptor and a safely auditable sklearn model | Google or OpenAI detector, or a high-precision hybrid rescue | The published cut accepted 139/200 Google, 141/200 OpenAI, and 214/600 disjoint ImageNet photographs |
|
||||
| [`vordme2010/synthid-dataset`](https://github.com/vordme2010/synthid-dataset/tree/133a27088f6f4d695c79db9a1a70fa8e7fa3adad) | A public 2026 Gemini flat-field corpus, safe numeric features, and explicit candidate carrier bins | Its reported near-perfect Tier-1 AUC as open-world SynthID evidence | Training used 500 Gemini flats against 1,500 synthetic or phase-scrambled negatives and zero real photographs. Rebuilding the SVM from the safe matrix accepted 1/200 Google and 0/200 OpenAI on v7 |
|
||||
|
||||
Useful forensic signal is therefore not synonymous with a production
|
||||
watermark decoder. The Google lattice idea is retained only as a
|
||||
research-lineage component; it cannot be labelled SynthID. The fixed template
|
||||
and CNN gradient are retained as negative controls. None of these public
|
||||
implementations is shipped as a detector or remover. Detailed classifier and
|
||||
attack measurements live in [SynthID source classifiers](synthid-classifiers.md) and
|
||||
[mark removal](synthid-removal-research.md).
|
||||
|
||||
### Google verifier surfaces found in the sweep
|
||||
|
||||
Two real but non-local verifier paths surfaced. The historical Vertex SDK
|
||||
exposed `WatermarkVerificationModel.from_pretrained("imageverification@001")`.
|
||||
The class still exists in `google-cloud-aiplatform` 1.165.1, but warns that it
|
||||
was deprecated 2025-06-24 with removal scheduled for 2026-06-24. Calls in two
|
||||
projects with Vertex enabled now reject the publisher model id as invalid. A
|
||||
repository using `response.watermark_detected` is also stale: the actual SDK
|
||||
property was `watermark_verification_result`, backed by the prediction's
|
||||
`decision` field.
|
||||
|
||||
A newer unlisted upload route,
|
||||
`POST https://synthid.googleapis.com/upload/v1:verifyContent`, is live enough
|
||||
to identify itself as the SynthID API and return `SERVICE_DISABLED` for a real
|
||||
public image. It has no public discovery document, did not appear in the
|
||||
project's available-services listing, and no authorized successful response
|
||||
was obtained. Enabling a cloud service was deliberately not attempted. This
|
||||
may be an access-gated research oracle, but it is not a local decoder and does
|
||||
not satisfy the metadata-free offline product requirement. Gemini chat remains
|
||||
the only Google verifier actually evidenced by the public bypass corpora.
|
||||
|
||||
## Closed detector routes
|
||||
|
||||
| Route | Close | Why |
|
||||
| --- | --- | --- |
|
||||
| Wavelets / FFT / cepstrum as a single-image detector | 2026-08-09 | TrustMark proxy: 318-d wavelet/spectral summary AUC 0.653 and 0 TPR at a clean calibration cut. Complex FFT maps AUC 0.516. Spatial RGB still won |
|
||||
| `aloshdenny/reverse-SynthID` V3 phase codebook | 2026-08-09/24 | Pickle-free numeric audit. The original pilot was 5/5 Google positives and 0/194 then-available negatives. The broader audit was 5/355 Google, 5/499 controls, and 6/1,000 fresh Open Images. On v7 it accepted 4/200 Google and 1/200 OpenAI. As a v11 `unknown` rescue it adds 2/200 Google on this test, but the historical control errors reject a 1%-recall OR rule |
|
||||
| `aloshdenny/reverse-SynthID` V4 | 2026-08-13/24 | Commit `b110836`. Better-of-two profiles: 141/355 Google positives and 191/499 controls. Frozen 1,000 Open Images: 386 accepted. Paired AUC 0.517. An exact repeat on the hash-disjoint v7 challenge accepted 77/200 Google and 76/200 OpenAI. Strict GUI aspect routing reduced support but not false acceptance. The threshold sits 0.02 above chance phase similarity |
|
||||
| `cebeuq/Synthid-Bypass` as a local decoder | 2026-08-13 | 12 public before/after diffusion pairs. Fixed carrier 10/12 before, 0/12 after. Gemini Verify still saw SynthID on two `before` files the carrier missed. Reconstruction removal, not a payload read |
|
||||
| S4: one content-independent pattern for both providers | 2026-08-15 | Google has a shared phase-coherent tile. OpenAI is at chance. See Result |
|
||||
| M2: mint Google encoder-off pairs | 2026-08-15 | No current Google path emits an unwatermarked image. See Result |
|
||||
| U1: one feature space for both providers | 2026-08-15 | Transfer 0.535 and 0.469 |
|
||||
| Hand statistics vs a perceptually masked keyed mark (M1b) | 2026-08-15 | At equal RMS, fixed 0.670, keyed 0.669, keyed+masked 0.546. Masking, not keying, defeats hand features |
|
||||
| Decoder-substitution / foreign VAE as a verified negative | 2026-08-17 | OpenAI oracle stayed `detected` at 22.3 dB. Attacking a positive does not mint a labelled negative |
|
||||
| `aoxo/reverse-synthid` as extra Google templates | 2026-08-19 | Hugging Face dump scored against the frozen era bank. Extra family means raised foreign scores instead of isolating a mark |
|
||||
| Origin-locked lattice as SynthID | 2026-08-16 | Two-pixel crop kills it; published SynthID keeps 99.97% TPR under aggressive crop. See classifiers for `google_lineage_renderer` |
|
||||
| Keyless energy in 16-32 px | 2026-08-20 | L1 AUC 0.53; official decoder reads phase structure, not energy |
|
||||
| TrustMark / open-encoder transfer | 2026-08-20 | E3 leave-one-encoder-out at chance on TrustMark (0.505 ridge, 0.510 conv) |
|
||||
| L1 distillation of oracle bits | 2026-08-20 | Geometry-only AUC 0.78-0.83 beats pixel 124-d (0.64). ChatGPT export and `opened` C2PA predict `not_detected`. Inside 1254x1254, permutation p=0.45 |
|
||||
| Flat 16-32 matched filter | 2026-08-21 | Leave-one-out residual correlation 0.645 on gray flats; 0.007-0.025 on COCO photo residuals. L1 AUC 0.70 was a size confound (size-matched 0.59 / 0.44) |
|
||||
| `gpt-image-1` as encoder-off pair | 2026-08-21 | Does not stamp. `gpt-image-2` does |
|
||||
| CNN on raw L1 bits | 2026-08-21 | Do not train. Labels are export geometry and presentation, not the mark |
|
||||
| 128-photo causal wavelet student on the flat template | 2026-08-21 | Photo 16-32 jam flips only at a=24 / 18.6 dB (`a*_photo < 0.53` failed). A student cannot beat a matched filter if the photo-domain mark is that quiet |
|
||||
| Photo 16-32 amplitude without the flat-arm template | 2026-08-23 | 11 COCO `gpt-image-2` edits plus the red-dot pair. Residual is the camera JPEG resized onto the edit raster, top-left 32 px dropped. Median 16-32 residual RMS 12.2, same order as 8-16 (9.0) and 32-64 (16.3). On the five tightest pairs (full residual RMS < 16): 16-32 median 2.69, still ~24x the 0.11 flat stamp and not louder than 8-16 (3.81). The photo residual does not isolate a 16-32 codeword |
|
||||
| Sort-by-luma 1D chroma stego | 2026-08-23 | Flat LOO 0.914 but a luma-order shuffle is already 0.792. COCO plus tiny noise: 0.642 vs shuffle 0.640. The sort invents the correlation |
|
||||
| Retinex / local-gain then 16-32 LOO | 2026-08-23 | 0.742 vs plain 16-32 LOO 0.752 on the same eight gray flats. No new carrier |
|
||||
| Distant 16-32 patch coupling | 2026-08-23 | Near lag 0.13 of a tile: -0.013. Far lag ~500 px: +0.054. No nonlocal lock |
|
||||
| Bispectrum of the 16-32 ring | 2026-08-23 | Degenerate: 1.0 on flats and 1.0 on COCO. Not a statistic |
|
||||
| Length-255 m-sequence vs 16x16 fold | 2026-08-23 | Max abs corr 0.059. Chance for n=255 is `1/sqrt(255) ≈ 0.063` |
|
||||
| Row-as-audio pitch peak | 2026-08-23 | Peak-to-mean 51 on flats, 46 on COCO noise residuals. Shared FFT peakiness, not a pitch |
|
||||
| 16x16 origin search of the flat 16-32 template on photos | 2026-08-23 | Zero-shift NCC 0.006-0.024, best of 256 shifts 0.013-0.026. Origin is not hiding a lock |
|
||||
| Affine search of the 16-32 residual vs the flat template | 2026-08-23 | 11 `gpt-image-2` photos: zero-deg median 0.009, best rotation in ±4° 0.020, lift 0.011. Six COCO noise residuals: lift 0.008. Flats already peak at 0°. No hidden rotation or 3% scale lock |
|
||||
| Fourier-angle scramble of 16-32 (true `(r, θ)` of the annulus) | 2026-08-23 | s1 24.1 dB official `not_detected`, cartesian replicate 24.8 dB `not_detected`. Decoder reads phase in that annulus, including angular organization. Not quieter than scramble |
|
||||
| Baker-map of the 16-32 band | 2026-08-23 | s1 27.8 dB still `detected` |
|
||||
| Poisson / photon-shot embedding | 2026-08-23 | Flat Anscombe LOO 0.7518 vs linear 0.7519. s1 Poisson 30.7 dB still `detected` |
|
||||
| Nested LSB plus SynthID | 2026-08-23 | s1 55.9 dB still `detected`. Extra open stego does not hide or replace the mark |
|
||||
| ICC sRGB rewrite | 2026-08-23 | No-op, PSNR inf. Mark is not in a profile that a pixel-preserving assign changes |
|
||||
| Seam carve 8 columns of 16-32 energy | 2026-08-23 | s1 27.7 dB still `detected` |
|
||||
| Extra OpenAI image ids as a new ensemble | 2026-08-23 | `gpt-image-1.5` and `gpt-image-1-mini` are encoder-off: C2PA `created, converted`, official `not_detected`, 16-32 corr vs the `gpt-image-2` gray stamp 0.003. Dated `gpt-image-2-2026-04-21` is the same family: C2PA `watermarked.unbound`, 16-32 corr 0.189. `chatgpt-image-latest` generate stays 403 until org verification |
|
||||
| DIRE reconstruction error as a payload | 2026-08-23 | SDXL float32, 512 px. VAE RMS: gray 1.24, s1 9.15, photo edit 9.89, camera 11.84. DDIM DIRE RMS: gray 2.40, photo 30.9, s1 31.5, camera 33.0. Rank follows texture. Float16 DDIM NaN'd on MPS. Not a payload |
|
||||
| HEIF / AV1 as a hidden carrier | 2026-08-23 | s1 HEIF q80 46.3 dB, q50 39.3 dB, AV1 CRF 32 37.2 dB, all official `detected`. Codec round-trip does not mint a new statistic |
|
||||
| Print-scan simulation as the kill | 2026-08-23 | s1 24.95 dB still `detected`. Same PSNR class as 16-32 scramble, which kills. Generic 25 dB is not the mark |
|
||||
| Mellin / log-polar \|F\| of the 16-32 residual | 2026-08-23 | Flat LOO 0.996-1.000, photo 0.996, COCO 0.991-0.995. The annulus envelope, not a payload. Scale chirps `r^{iα}`: flat 0.51, photo -0.27, COCO -0.58. No photo lock |
|
||||
| Zernike n≤6 and PCET of the residual | 2026-08-23 | Flat LOO 0.77 / 0.75, same order as cartesian 16-32. Photo -0.007 / 0.044. COCO 0.23 / 0.35, above photos. Moments of the flat stamp, not a photograph decoder |
|
||||
| Fresnel quadratic-phase of 16-32 (z=8,16,32,64) | 2026-08-23 | Flat LOO 0.750-0.753, photo 0.012, COCO -0.006. Unitary copy of the cartesian residual. No new shell |
|
||||
| 8-bin Fourier orientation energy | 2026-08-23 | 0.999 flat, 0.966 photo, 0.995 COCO. Saturates |
|
||||
| Chroma 16-32 as the payload | 2026-08-23 | Gray-flat Y LOO 0.682, Cb 0.064, Cr 0.025. s1 Y scramble 24.6 dB `not_detected`; Cb 45.0 dB and Cr 43.9 dB stay `detected`. Mark is in luma |
|
||||
| One 90° Fourier sector of 16-32 | 2026-08-23 | s1 27.5 dB still `detected`. Decoder needs the whole annulus, not one wedge |
|
||||
| Radial-phase-only scramble of 16-32 | 2026-08-23 | s1 25.0 dB `not_detected`. Phase as a function of radius, over the full ring, is enough to kill. Combined with Fourier-angle scramble, either polar coordinate of the annulus phase is a kill if the whole ring is hit |
|
||||
| [`Rinne414/SynthID-detector`](https://github.com/Rinne414/SynthID-detector/tree/b08abff06b67c37db9dfa7c6ed63760a446d560f) fixed residual template | 2026-08-23/24 | The repository reports 0/33 held-out AI images and identifies its original result as self-correlation. Its pinned pre-May GPT-Image2 template first detected 0/11 current source-matched edits and 0/11 COCO sources. An exact repeat on 200 current OpenAI and 200 current Google images produced zero detections for all three published templates at the 0.08 cut; maximum GPT-Image2 score on OpenAI was 0.0167. Adding the expected template can spoof the public correlator, but that template is not present in held-out content. Independent repeat of the rejected fixed-template route, not a payload decoder |
|
||||
| [`newideas99/gpt-image-synthid-detector`](https://github.com/newideas99/gpt-image-synthid-detector/tree/5495e09) CNN ensemble | 2026-08-24 | Exact frozen inference retained 172/200 OpenAI but accepted 110/200 Google at the published 0.5 cut. The training negatives are regenerated positives, while its confound probe transforms only positives and has no foreign-image control. A full-frame white-box attack drove its probability from 0.99995 to 0.25956 at ±2/255 and effectively zero at ±4/255, but the official OpenAI decoder stayed `detected` through ±8/255. Renderer/confound classifier, not a mark decoder |
|
||||
| [`Ristellise/REGRET`](https://github.com/Ristellise/REGRET/tree/7d449034bf323987e7e608e7886e029ed20fd847) SPAM model | 2026-08-24 | At the published cut, exact model inference accepted 139/200 Google, 141/200 OpenAI, 214/600 disjoint ImageNet photographs, and substantial fractions of foreign generators. The apparent high-precision tail did not transfer |
|
||||
| [`vordme2010/synthid-dataset`](https://github.com/vordme2010/synthid-dataset/tree/133a27088f6f4d695c79db9a1a70fa8e7fa3adad) Tier-1 SVM | 2026-08-24 | Rebuilt without loading joblib, using the repository's 2,000x33 numeric matrix, reference phases, noise template, seed-42 split, scaler, and RBF SVM. It accepted 1/200 current Google and no current OpenAI rows. Its published AUC above 0.999 distinguishes Gemini flats from generated derivatives, not current photographs |
|
||||
| Generic GitHub heuristics and literal ports | 2026-08-24 | `hackerfactor/reverse-SynthID-C`, `BIRSAx2/ripmark`, and `xiaoyao9184/reverse-from-synthid` reproduce or wrap the already-rejected reverse-SynthID codebook. `AI-SCERN` uses uncalibrated annular energy, decoder-grid, and FFT-symmetry heuristics with no weights or held-out corpus. LSB flips, ±1 dither, and 100.5% resize tools publish no matching-oracle evidence. None adds an independent signal |
|
||||
|
||||
## Information budget on photographs (2026-08-21)
|
||||
|
||||
On 11 `gpt-image-2` photo edits, the flat-derived 16-32 template at full
|
||||
amplitude against per-image whitening has median `d' = 0.93` (range
|
||||
0.68-2.46). That is a best-case single-image AUC of about 0.75 even with a
|
||||
perfect, perfectly aligned template. A P5 gate (FPR 0.1%, TPR 90%) needs
|
||||
`d' >= 4.37`. The deficit is 4.7x in amplitude, 13.4 dB, *unless* the
|
||||
photo-domain mark is louder than the flat measurement. Titration said it
|
||||
is not: fish 16-32 additive jam stays `detected` through a=16 / 21.9 dB.
|
||||
|
||||
2026-08-23, without using that flat stamp as `G`: the aligned photo
|
||||
residual in 16-32 is redraw, not a codeword. Median RMS 12.2 across 12
|
||||
pairs; 2.69 on the five tightest. Neighboring octaves are as loud.
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/photo-band-amplitude-2026-08-23.json`.
|
||||
|
||||
Do not report an AUC from those 11 pairs as a detector result. With n=11
|
||||
the standard error on AUC is about 0.12. Notes:
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/agent-detector-claude.md`.
|
||||
|
||||
## External literature (surveyed 2026-08-23)
|
||||
|
||||
Primary sources, not abstracts. Each row is mapped onto a closed or open
|
||||
route in this campaign. Mechanism detail stays in
|
||||
[synthid.md](synthid.md). Removal papers are on
|
||||
[mark removal](synthid-removal-research.md). Classifier papers are on
|
||||
[SynthID source classifiers](synthid-classifiers.md).
|
||||
|
||||
### Official mark, not a public decoder
|
||||
|
||||
| Source | What it is | Map to this campaign |
|
||||
| --- | --- | --- |
|
||||
| Gowal et al., [arXiv:2510.09263](https://arxiv.org/abs/2510.09263) | Post-hoc encoder `f` / decoder `g`. Detection logit is not payload recovery. SynthID-O (partner variant) 136 bits at 512x512. TPR at 0.1% FPR 99.98% aggregated, 99.97% on the hardest spatial crop+resize. Trains against sampled semantics-preserving transforms, including weak VAE regeneration. Production decoder unpublished | Matches the architecture we treat as keyed `x' = x + g(x)`. Explains why a two-pixel crop kills `pipeline_lattice` but not the official oracle, and why a 22.3 dB foreign VAE still reads `detected` |
|
||||
| DeepMind [US12094474B1](https://patents.google.com/patent/US12094474B1/en) and continuation [US20250149048A1](https://patents.google.com/patent/US20250149048A1/en) | Residual U-Net encoder, separate decoder, optional key, encoder/decoder ensembles that need not recognize each other | Constraint, not a recipe. Ensemble non-recognition is why one recovered Google tile cannot be reused as an OpenAI detector (S4) |
|
||||
| Dathathri et al., [Nature 634:818-823 (2024)](https://www.nature.com/articles/s41586-024-08025-4) | SynthID-Text: tournament sampling of LLM tokens, open-source | Different system. Image/audio/video remain proprietary |
|
||||
| OpenAI, [advancing content provenance](https://openai.com/index/advancing-content-provenance/) (2026-05-19, audio 2026-07-31) and [content provenance API](https://developers.openai.com/api/docs/guides/content-provenance) | ChatGPT / API / Codex images carry C2PA plus SynthID. Audio from 2026-07-31. `POST /v1/content_provenance_checks`. `not_detected` does not rule out another vendor | This is the oracle. C2PA and SynthID are independent entries. Do not abuse the endpoint as an adaptive reverse-engineering loop |
|
||||
|
||||
### Keyless presence detectors in the literature
|
||||
|
||||
| Source | Claim | Caveat against our gates |
|
||||
| --- | --- | --- |
|
||||
| Ao et al., [arXiv:2603.06723](https://arxiv.org/abs/2603.06723) (AWPD / FSNet, SAFE@CVPR 2026) | Leave-one-algorithm-out presence detection. SynthID held out: FSNet Acc 0.894 / F1 0.886, ResNet-50 Acc 0.845 / F1 0.812, ConvNeXt V2 Acc 0.866. LSB and Patchwork both fail below 60%. Hypothesis: modern invisible marks share dense high-frequency spectral anomalies | UniFreq Table 3: SynthID is 2,000 images, all AIGC, zero photographs, from `imagen-4.0-fast-generate-001` only, resized to 256x256. No OpenAI. No 0.1% FPR. Closest published analog to E3/L1 distillation. Our TrustMark leave-one-encoder-out was chance (0.505 ridge). Their high-frequency commonality is the same energy that L1 failed to read as an OpenAI payload |
|
||||
| `aloshdenny/reverse-SynthID` V3/V4; Google to [The Verge, 2026-04-14](https://www.theverge.com/ai-artificial-intelligence/911579/google-synthid-ai-watermarking-system-reverse-engineered) | Spectral codebook from averaged Gemini flats. Spokesperson Myriam Khan: "It is incorrect to say this tool can systematically remove SynthID watermarks." The author also said the bypass confuses the decoder rather than deleting a payload | Our pickle-free V4 audit: paired AUC 0.517, frozen Open Images 386/1000 accepted. Not 0.1% FPR |
|
||||
| Krawetz, [Hacker Factor "Reversing SynthID"](https://www.hackerfactor.com/blog/index.php?/archives/1092-Reversing-SynthID.html) and ["Meta's Un-Stable Signature"](https://hackerfactor.com/blog/index.php?/archives/1098-Metas-Un-Stable-Signature.html) | Gemini chat TPR closer to 1/20 than the paper's 99.97%. Detector weak on flats. reverse-SynthID field accuracy ~70% vs the author's 90% | Gemini-app chat is not the OpenAI provenance API. Our gray `gpt-image-2` flats are `detected` on the official OpenAI oracle. Do not mix the two verifiers |
|
||||
| vitotitto LAB-a logistic (community, tiny holdout) | Reported 97.7% AUC on 20/20 | Not a P5 gate. Ignore as a detector claim |
|
||||
|
||||
### Different embedding loci (not post-hoc SynthID)
|
||||
|
||||
These papers are often cited as if they were SynthID. They are not.
|
||||
|
||||
| Source | Locus | Why it is not this mark |
|
||||
| --- | --- | --- |
|
||||
| Gunn, Zhao, Song, [arXiv:2410.07369](https://arxiv.org/abs/2410.07369) (PRC, ICLR 2025) | Pseudorandom error-correcting code in the *initial diffusion latent* | Cryptographic undetectability is a latent-code property. SynthID-Image is applied after the VAE has already decoded pixels. Our OKLab "PRC-style" noise was a category error |
|
||||
| Francati et al., [arXiv:2509.10577](https://arxiv.org/abs/2509.10577) (EuroS&P 2026) | Crop-and-resize flipped about half of PRC latent signs and blocked belief-propagation decode | Confirms PRC is origin-locked in latent space. Matches why a 2 px shift kills `pipeline_lattice` and does not kill OpenAI SynthID |
|
||||
| Fernandez et al. Stable Signature; Wen et al. Tree-Ring | Fine-tuned VAE decoder, or ring constraints on initial noise | In-generation. Google's paper is explicit that SynthID-Image does not modify the generator |
|
||||
| TrustMark ([arXiv:2311.18297](https://arxiv.org/abs/2311.18297)), HiDDeN, StegaStamp | Open post-hoc encoder/decoder pairs | Transfer from these is E3: chance on TrustMark. WAVES ([arXiv:2401.08573](https://arxiv.org/abs/2401.08573)) stress-tests this family, not production SynthID |
|
||||
|
||||
Do not train a student on AWPD's Imagen-only 256 crop and call it an
|
||||
OpenAI SynthID detector. That would repeat L1's contamination: a
|
||||
generator/size cue, not a payload.
|
||||
|
||||
### Adjacent literature, not SynthID
|
||||
|
||||
These papers are the published versions of the wild hypotheses and of
|
||||
Model 1. They detect *something else*: a camera pipeline, a generator
|
||||
fingerprint, an open watermark, or steganography. None of them is a
|
||||
keyless OpenAI payload decoder.
|
||||
|
||||
**Classical spread-spectrum and open neural marks**
|
||||
|
||||
| Source | What it is | Map |
|
||||
| --- | --- | --- |
|
||||
| Cox, Kilian, Leighton, Shamoon, IEEE TIP 1997 | i.i.d. Gaussian vector in perceptually significant spectral components. Informed detector. Collusion-resistant by construction | Ancestor of keyed CDMA. Our length-255 m-sequence vs 16x16 fold was at chance because we did not have the key, and because OpenAI is content-adaptive, not a fixed Gold code |
|
||||
| Zhu et al., HiDDeN, ECCV 2018 | Joint encoder / noise layer / decoder | Open pair. AWPD leave-one-out Acc 0.985. Transfer to SynthID is E3 |
|
||||
| Tancik, Mildenhall, Ng, StegaStamp, 2020 | 100 bits at 400x400, trained through print-scan | Open pair. Survives recapture. WAVES: TPR at 1% FPR collapses from 1.00 to 0.01 under regeneration |
|
||||
| Wen et al., [arXiv:2305.20030](https://arxiv.org/abs/2305.20030) (Tree-Ring, NeurIPS 2023) | Pattern in the *initial diffusion noise*, recovered by DDIM inversion | In-generation, origin-locked in latent Fourier space. Crop-invariant by design. Not post-hoc SynthID |
|
||||
| Fernandez et al., Stable Signature, ICCV 2023 | Fine-tune the LDM VAE decoder so every decode carries a bit string | In-generation. Gowal is explicit that SynthID-Image does not modify the generator |
|
||||
| Lin and Juarez, [arXiv:2506.10502](https://arxiv.org/abs/2506.10502) (USENIX 2025) | Public-knowledge attack that removes Tree-Ring | Confirms Tree-Ring is a different object with a different kill |
|
||||
|
||||
**Steganalysis as a presence detector**
|
||||
|
||||
Fridrich and Kodovsky Spatial Rich Models (TIFS 2012) and Boroumand,
|
||||
Chen, Fridrich SRNet (TIFS 2018) detect sub-bit-per-pixel spatial
|
||||
stego by high-pass residuals, with pooling disabled in the front of
|
||||
SRNet so the weak signal is not averaged away. AWPD cites both and
|
||||
says they drift on modern deep / generative marks. That matches our
|
||||
wavelet/FFT single-image detector (AUC 0.653, 0 TPR at a clean cut)
|
||||
and the 16-32 energy miss on photographs: a residual energy detector
|
||||
without the matching key is steganalysis of a mark that was trained
|
||||
not to look like LSB.
|
||||
|
||||
**Generator fingerprints in the Fourier domain**
|
||||
|
||||
Corvi, Cozzolino, Poggi, Nagano, Verdoliva,
|
||||
[arXiv:2304.06408](https://arxiv.org/abs/2304.06408) (CVPRW 2023):
|
||||
GAN, diffusion, and VQ-GAN images show spectral peaks and anomalous
|
||||
autocorrelation; real vs synthetic differ in mid-high radial and
|
||||
angular power. reverse-SynthID averaged Gemini flats and called the
|
||||
peak a watermark codebook. Corvi's result says many generators leave
|
||||
*some* peak. Our V4 Open Images 386/1000 is what a generator-fingerprint
|
||||
detector looks like when you calibrate it as if it were a payload.
|
||||
|
||||
Yao and Juarez, [arXiv:2512.11771](https://arxiv.org/abs/2512.11771)
|
||||
("Smudged Fingerprints"): 14 fingerprinting methods across RGB,
|
||||
frequency, and learned features; removal attacks >80% white-box, >50%
|
||||
black-box. A fingerprint you can see without a key is a fingerprint
|
||||
you can wipe without a key.
|
||||
|
||||
### Image investigation and data hiding (any method)
|
||||
|
||||
These are not SynthID papers. They are the rest of the toolkit: how
|
||||
people hide bits in pictures, and how people tell a picture was
|
||||
touched. Several of our wild hypotheses already had a published form
|
||||
here.
|
||||
|
||||
Hiding is not one problem. Cover modification (change an existing
|
||||
image), coverless / generative (sample an image that already carries
|
||||
the bits), and signed metadata (C2PA) fail under different attacks.
|
||||
|
||||
**Cover modification, classical**
|
||||
|
||||
| Source | Hide how | Detect / limit |
|
||||
| --- | --- | --- |
|
||||
| LSB, Patchwork (Bender et al., IBM SJ 1996) | Flip low bits, or luminance of random pixel pairs | AWPD Acc < 0.60. Sparse or ±1 amplitude. SRNet / FSNet average it away |
|
||||
| Westfeld F5 (2001), Fridrich nsF5 | JPEG DCT coefficients, matrix embedding | Histogram attacks on F5; nsF5 was the shrinkage fix. Domain is the codec, not a mid-band residual |
|
||||
| Pevny, Filler, Bas HUGO (2010); Holub and Fridrich WOW (2012); Holub, Fridrich, Denemark UNIWARD (IH&MMSec 2013 / EURASIP 2014) | Content-adaptive costs, bits placed in texture via syndrome-trellis codes (Filler, IH 2011) | The modern spatial/JPEG floor. Distortion is *designed* to look like the cover. A 16-32 energy detector is the wrong statistic |
|
||||
| Cox et al. 1997 | Spread-spectrum in significant DCT | Keyed. Already mapped above |
|
||||
|
||||
**Cover modification, neural (hide a whole image, not 32 bits)**
|
||||
|
||||
| Source | What it does | Map |
|
||||
| --- | --- | --- |
|
||||
| Baluja, NeurIPS 2017, "Hiding Images in Plain Sight" | Full-resolution secret image distributed across all bits of a same-size cover | Capacity is the point. Not robust, not keyed provenance. HiDDeN later added a noise layer so the secret survives JPEG |
|
||||
| Jing et al., HiNet, ICCV 2021 | Invertible neural net: hide and recover as one bijection | High-capacity lossless-ish hiding. Recovery needs the exact inverse, not an official oracle |
|
||||
| Yang et al., PRIS, [arXiv:2309.13620](https://arxiv.org/abs/2309.13620) | Invertible net plus robustness modules | Same family, trained through distortion |
|
||||
|
||||
A HiNet-style student on OpenAI pairs would learn the *edit*, not a
|
||||
SynthID codeword. That is the L1 contamination again.
|
||||
|
||||
**Coverless / generative hiding**
|
||||
|
||||
The cover is never modified because there is no cover. The sampler
|
||||
emits an image whose latents already encode the message.
|
||||
|
||||
| Source | Locus | Map |
|
||||
| --- | --- | --- |
|
||||
| Yang et al., Gaussian Shading, [arXiv:2404.04956](https://arxiv.org/abs/2404.04956) (CVPR 2024) | Map bits onto Gaussian latents indistinguishable from ordinary noise. Recover by DDIM inversion | Sibling of Tree-Ring / PRC. Training-free, performance-lossless *for the generator*. Not a post-hoc pixel stamp |
|
||||
| Peng et al., StegaDDPM (ACM MM 2023) and later LDStega | Bits in the diffusion sampling distribution | Spatial SRNet is the wrong detector (NS-DSer, [arXiv:2602.10219](https://arxiv.org/abs/2602.10219): move steganalysis into noise space) |
|
||||
| CRoSS, Pulsar, MDDM | Message-to-noise projections | Same locus. A pixel scramble does not invert the sampler |
|
||||
|
||||
**Passive forensics (the picture was touched, no secret assumed)**
|
||||
|
||||
| Source | Cue | Map |
|
||||
| --- | --- | --- |
|
||||
| Krawetz, "A Picture's Worth", 2007 (ELA) | Re-JPEG at lower quality, subtract | Already measured: COCO 3.13, s1 1.97, gray stamp 0.49. Codec history, not a payload. Farid publicly called ELA as likely to mislabel originals as it is to catch edits |
|
||||
| Farid, IEEE TIFS 2009, JPEG ghosts | Difference energy vs a sweep of JPEG qualities; spliced regions ghost at their original Q | Local: s1, a `gpt-image-2` photo, and a camera JPEG all minimize at Q90. Codec, not a payload |
|
||||
| Popescu and Farid, TR2004-515 | Copy-move via duplicated regions | Not generation, not a watermark |
|
||||
| Popescu and Farid, IEEE TSP 2005 | Resampling periodic correlations | Affine search cousin. A rotated SynthID residual is a different question |
|
||||
| Wang et al., DIRE, [arXiv:2303.09295](https://arxiv.org/abs/2303.09295) (ICCV 2023) | Diffusion reconstruction error: generated images reconstruct, cameras do not | Model 1 sibling. SDXL float32 at 512: DIRE RMS camera 33.0, s1 31.5, photo edit 30.9, gray stamp 2.40. Texture rank, not a payload. Inverse of the foreign-VAE remover: there the mark survived 22.3 dB |
|
||||
| Wang, Wang, Zhang, Owens, Efros, [arXiv:1912.11035](https://arxiv.org/abs/1912.11035) (CVPR 2020, CNNDetect) | One ProGAN classifier, heavy JPEG/crop aug, transfers to many CNNs | Ancestor of "train on one generator". Ojha showed the sink-class failure once diffusion arrived. We required Firefly for that reason |
|
||||
|
||||
C2PA is the non-pixel stack: a signed manifest, stripped by
|
||||
`metadata --remove`. Durable Content Credentials (spec 2.4) add a
|
||||
soft binding that can re-link a stripped file to a repository. That
|
||||
is provenance, not hiding.
|
||||
|
||||
Do not train on ELA, JPEG ghosts, DIRE, or a HiNet reconstruction and
|
||||
name the score SynthID.
|
||||
|
||||
### Waveforms that can live in a picture
|
||||
|
||||
A mark is a function on the pixel lattice. The literature does not
|
||||
use one wave. It picks a basis whose symmetries match the attack it
|
||||
fears, then hides a keyed coefficient vector in that basis. Cartesian
|
||||
16-32 is one shell. Polar, scale, and diffraction are different
|
||||
shells.
|
||||
|
||||
**Standing waves on a rectangle.** DFT / DCT / DST. A 2-D sinusoid
|
||||
`cos(2π(ux + vy)/N)`. JPEG lives here. Our octave scramble destroys
|
||||
one annular *radius* of these frequencies, not one orientation. A
|
||||
Gabor packet is the same sinusoid windowed in space.
|
||||
|
||||
**Circular and log-radial waves.** Functions of `(r, θ)`, not
|
||||
`(x, y)`.
|
||||
|
||||
| Basis | Wave | Invariance it buys |
|
||||
| --- | --- | --- |
|
||||
| Fourier-Mellin / log-polar (Ó Ruanaidh and Pun, Signal Processing 1998) | `r^{iα} exp(ikθ)` after a DFT magnitude | Rotation and scale become translations |
|
||||
| Logarithmic radial harmonics (IH 2002) | Same family, added in pixels, detected by complex correlation | RST without going through the unstable log-polar resample |
|
||||
| Polar harmonic transforms: PCET, PCT, PST | `exp(±i2π n r²)` and polar cos/sin | Rotation. Moments, not a dense codebook |
|
||||
| Zernike / pseudo-Zernike | Orthogonal polynomials on the disk, radial part related to Bessel | Rotation. Classical moment watermark |
|
||||
| Circular chirp (SPIE 6072, 2006) | Polar map of a 1-D chirp onto a ring | JPEG (tune chirp rate) plus rotation (the ring) |
|
||||
| Tree-Ring (Wen 2023) | Concentric rings in the *latent* Fourier plane | Crop/flip by construction. Not a pixel wave |
|
||||
|
||||
The 2026-08-23 file named polar-1632 is cartesian annular phase
|
||||
shuffle, a scramble replicate (`not_detected` at 25.6 dB). True
|
||||
Fourier-angle scramble of the same annulus is `not_detected` at 24.1 dB.
|
||||
Radial-phase-only scramble is `not_detected` at 25.0 dB. One 90°
|
||||
sector plus its conjugate stays `detected` at 27.5 dB. Affine/rotation
|
||||
search of the 16-32 residual against the flat template does not lock
|
||||
(lift 0.011, COCO 0.008).
|
||||
|
||||
**Scale chirps.** Hyperbolic / Mellin monomials (arXiv:1208.5842):
|
||||
real 1-D Mellin patterns tiled in 2-D. Run 2026-08-23:
|
||||
`r^{iα}` coefficients flat LOO 0.51, photo -0.27, COCO -0.58.
|
||||
Log-polar `|F|` and a 64-bin radial Mellin profile saturate on COCO
|
||||
(0.99). Not a payload, and not quieter than cartesian 16-32.
|
||||
|
||||
**Directional packets.** Dual-tree complex wavelets (approximate
|
||||
analytic wave), Gabor/Morlet, ridgelets, curvelets, shearlets,
|
||||
contourlets, bandelets. Multiplicative spread-spectrum on curvelet
|
||||
coefficients is a published detector-design paper, not a SynthID
|
||||
decoder. Our wavelet summary AUC 0.653 already said a *generic*
|
||||
packet energy is not the OpenAI payload.
|
||||
|
||||
**Optical diffraction, actual wave physics.**
|
||||
|
||||
| Transform | What the wave is | Map |
|
||||
| --- | --- | --- |
|
||||
| Fresnel | Quadratic phase `exp(iπ r² / λz)`, a radial chirp. Energy concentrates with distance | Holographic watermarks embed the diffracted field, not the picture |
|
||||
| Fractional Fourier | Rotation of the time-frequency plane. Order is an extra key | Extra key space on top of DFT |
|
||||
| Double random phase encoding (Réfrégier and Javidi, Optics Letters 1995) | Random phase at input *and* Fourier plane. Ciphertext is stationary white noise | Closest optical analog of a keyed `x' = x + g(x)` that looks like noise without both screens |
|
||||
| Cascaded Fresnel / CGH | Two holograms at different depths | Hardware key. Not a pixel residual we can match-filter |
|
||||
|
||||
A Fresnel/FrFT student without the distance or the fractional order
|
||||
is another unmatched filter. Fresnel at z=8, 16, 32, 64 on the 16-32
|
||||
residual is a unitary copy of the cartesian band (flat LOO 0.75, photo
|
||||
0.012). FrFT is the same unmatched-filter claim with an extra order
|
||||
key; it does not isolate a photograph lock the four distances missed.
|
||||
|
||||
**Cyclostationary and clock-warped waves.** Periodic clock change
|
||||
(Martin, Chabert, Lacaze, EUSIPCO 2005) turns a stationary host into
|
||||
a cyclostationary one by a periodic time warp. Our H2 multiplicative
|
||||
cyclostationarity test already put official positives *below*
|
||||
controls. Do not rerun the same statistic in a new basis and call it
|
||||
new evidence.
|
||||
|
||||
**How to read a candidate wave.** Name the coordinates `(x,y)` vs
|
||||
`(r,θ)` vs `log r`, the generating function (sinusoid, chirp,
|
||||
Bessel, Zernike), additive vs multiplicative, and the symmetry it
|
||||
claims. Then pick one attack that kills that symmetry and one that
|
||||
should not. Cartesian 16-32 scramble and a 2 px shift are done
|
||||
(scramble kills, shift does not). True Fourier-angle scramble kills s1
|
||||
at 24.1 dB. Radial-phase scramble kills at 25.0 dB. A 90° sector does
|
||||
not. Y scramble kills; Cb/Cr do not. Affine, Mellin, Zernike, PCET,
|
||||
and Fresnel of the 16-32 residual do not lock on photographs.
|
||||
|
||||
## Still open or incomplete
|
||||
|
||||
- Google: no public payload decoder and no usable programmatic oracle.
|
||||
The Gemini app remains quota-limited. Vertex was 403.
|
||||
- `chatgpt-image-latest` is an optional third pair mint, not a blocker.
|
||||
`gpt-image-1`, `gpt-image-1.5`, and `gpt-image-1-mini` are encoder-off.
|
||||
`gpt-image-2` and dated `gpt-image-2-2026-04-21` are encoder-on, same
|
||||
16-32 stamp (corr 0.189 on gray). Latest would only matter if ChatGPT's
|
||||
current consumer id used a different ensemble. `images.generate` is 403
|
||||
until the organization is verified.
|
||||
- Physical print-scan: Brother DCP-L2520DW is idle and accepting. No
|
||||
`scanimage`, no ImageCapture pyobjc. Recapture needs a human at the
|
||||
flatbed. Simulation at 24.95 dB stayed `detected`. Do not print
|
||||
unattended.
|
||||
- Adding unmarked context (frame, pad, hstack, collage) to hunt a
|
||||
residual is a presentation gate, not a mark statistic. Those rungs
|
||||
belong with identity and layout below, and with
|
||||
[mark removal](synthid-removal-research.md).
|
||||
- Camera-to-ChatGPT "unchanged crop" pairs are dirty counterfactuals.
|
||||
The instructed edit redraws; aligned PSNR on the frame pair was
|
||||
16.6 dB. Compare pixels only after registration and a frozen residual
|
||||
bound, as in the `gpt-image-2` red-dot row.
|
||||
|
||||
A Gemini app caption that an image "looks like a photograph" is not a
|
||||
SynthID verdict. Use the provider pixel check or signed provenance.
|
||||
|
||||
## Wild hypotheses
|
||||
|
||||
Battery 2026-08-23, no official oracle. Local numbers:
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/wild-hypotheses-2026-08-23.json`.
|
||||
Prepared attack rasters wait in `wild-attacks-2026-08-23/` for a later
|
||||
verifier window. Jacobian / adaptive queries against
|
||||
`verify-openai-synthid` stay out: the endpoint forbids reverse-engineering.
|
||||
|
||||
### Tested locally, not a mark
|
||||
|
||||
| Hypothesis | Result |
|
||||
| --- | --- |
|
||||
| Chroma ordered by luma is a 1D payload | Correlation is an artifact of the sort |
|
||||
| Mark is `g(x)` of a Retinex envelope | LOO unchanged vs the raw 16-32 residual |
|
||||
| Nonlocal patch agreement in 16-32 | Far patches are not coupled |
|
||||
| Quadratic phase coupling (bispectrum) | Statistic saturates on camera images too |
|
||||
| CDMA Gold / m-sequence in the 16x16 fold | At chance for length 255 |
|
||||
| Rows as a pitched waveform | COCO noise has the same peak-to-mean |
|
||||
| The photo mark is the flat stamp at an unknown 16-phase | Exhaustive shift does not lift NCC |
|
||||
| The photo mark is the flat stamp at a small rotation or 3% scale | Affine search lift 0.011, COCO noise 0.008 |
|
||||
| Self-keyed predictor from a 16 px luma thumb | Already 0.48 vs L1, 0.50 vs COCO (`self-keyed.json`) |
|
||||
| Global multiplicative cyclostationarity (H2) | Official positives below controls, 2026-08-14 |
|
||||
| Mellin radial profile / log-polar \|F\| | Saturates on COCO (0.99). Annulus envelope |
|
||||
| Scale chirps `r^{iα}` | Flat 0.51, photo -0.27, COCO -0.58 |
|
||||
| Zernike n≤6 | Flat 0.77, photo -0.007, COCO 0.23 |
|
||||
| PCET polar harmonics | Flat 0.75, photo 0.044, COCO 0.35 |
|
||||
| Fresnel z=8..64 of 16-32 | Copy of cartesian LOO |
|
||||
| 8-bin orientation energy | Saturates on COCO |
|
||||
| Chroma 16-32 on gray flats | Cb 0.064, Cr 0.025 vs Y 0.682. Stamp is luma |
|
||||
|
||||
### Not a watermark, might still be an AI-vs-camera extra
|
||||
|
||||
Battery 2026-08-23, `remaining-hypotheses-2026-08-23.json`. Kodak corpus was empty here, camera arm is COCO plus the photo-pair sources.
|
||||
|
||||
| Hypothesis | Result |
|
||||
| --- | --- |
|
||||
| Absence of camera PRNU | Gray highpass RMS 0.25 vs COCO 14.6 was texture. Wiener residual on *photographs*: `gpt-image-2` edits 4.61 vs camera 8.05. Direction is camera-like leftover of the donor JPEG, not a missing sensor, and not a payload |
|
||||
| JPEG ELA | Flat 0.49, s1 1.97, COCO 3.13. Compression history |
|
||||
| CFA interpolation error (Bayer as detector) | Ratio photo-edit 0.117 vs camera 0.184 vs gray stamp 0.588. Weak camera vote, overlaps. Inverse of the Bayer remover, not SynthID |
|
||||
| JPEG ghosts (Farid) | s1, a `gpt-image-2` photo, and a camera JPEG all minimize at Q90 with similar spreads. Codec, not a payload |
|
||||
| PNG Paeth / filter-byte residual | `gpt-image-2` PNG is 99.9% Paeth. Camera re-saved PNG 73% Paeth, s1 74%. Export encoder fingerprint |
|
||||
| Perfect-circle / text-edge rate | Hough circles per MP 385 (edits) vs 536 (camera). Edge fraction 0.052 vs 0.072. Too noisy for a payload or a provider |
|
||||
| Face-gated 16-32 | s1: 3 Haar faces, inside RMS 12.6 vs outside 10.9. One photo with faces: outside/in 1.49. No systematic "spend the mark off faces" |
|
||||
| CLIP-space watermark | That is Model 1. Not a payload decoder |
|
||||
|
||||
### Non-local, 2026-08-23
|
||||
|
||||
Battery `.local-eval/synthid/prc-oklab-attack-2026-08-15/nonlocal-2026-08-23/`.
|
||||
Oracle after AI-metadata strip, 25 s gap, then one dated-id retry after a 429.
|
||||
|
||||
| Hypothesis | Result |
|
||||
| --- | --- |
|
||||
| HEIF q80 / q50 | 46.3 / 39.3 dB, official `detected` |
|
||||
| AV1 CRF 32 still | 37.2 dB, `detected` |
|
||||
| Print-scan simulation (0.35 deg, blur σ 0.9, 300/360 dpi, JPEG q42) | 24.95 dB, still `detected`. Same PSNR class as 16-32 scramble, which kills |
|
||||
| Physical print-scan | Brother DCP-L2520DW idle and accepting. No `scanimage`, no ImageCapture pyobjc. Not printed unattended |
|
||||
| `gpt-image-1.5` gray reproduce | C2PA `created, converted`. Oracle `not_detected`. 16-32 corr vs `gpt-image-2` stamp 0.003. Encoder-off. Dirty redraw 24.0 dB |
|
||||
| `gpt-image-1-mini` gray reproduce | Same C2PA. Oracle `not_detected`. Corr 0.003. PSNR 31.1 dB |
|
||||
| `gpt-image-2-2026-04-21` gray reproduce | C2PA `created, converted, watermarked.unbound`. Official `detected`. 16-32 corr vs unversioned `gpt-image-2` stamp 0.189. Same encoder family. PSNR 47.5 dB |
|
||||
| `chatgpt-image-latest` generate | 403, org verification. Listed among image models, not callable |
|
||||
| DIRE SDXL VAE + DDIM float32 | VAE RMS 512: gray 1.24, s1 9.15, photo edit 9.89, camera 11.84. DIRE RMS: gray 2.40, photo 30.9, s1 31.5, camera 33.0. Texture rank, not a payload |
|
||||
|
||||
### Still blocked after that battery
|
||||
|
||||
| Hypothesis | Why it stays blocked |
|
||||
| --- | --- |
|
||||
| Physical print-scan recapture | Human at the flatbed |
|
||||
| Encoder ensemble that does not recognize itself | Needs another actual SynthID encoder/decoder pair. Extra OpenAI ids did not mint one. Open-proxy transfer is already closed: source-matched TrustMark, VideoSeal, DWT-DCT, and WAM pairs produced held-out-family AUC 0.437-0.562, so another generic public pair does not answer this hypothesis |
|
||||
| `chatgpt-image-latest` as a third pair mint | `images.generate` 403 until org verification. Optional: only if ChatGPT latest used a different ensemble than `gpt-image-2` |
|
||||
|
||||
Do not train on official decoder bits beyond the frozen L1 study. Do not
|
||||
mint a 128-photo student on the flat stamp. The information budget and
|
||||
the 2026-08-23 residual measurement both say the photo-domain mark is
|
||||
under the scene.
|
||||
|
||||
## Oracle and seeds
|
||||
|
||||
Official `POST /v1/content_provenance_checks`, metadata stripped, pixels
|
||||
preserved. Raw files stay gitignored under
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/`.
|
||||
|
||||
| Seed | Size |
|
||||
| --- | --- |
|
||||
| s1 | 1024x1536 |
|
||||
| s2 | 1024x1536 |
|
||||
| s3 | 1086x1448 |
|
||||
|
||||
All three `detected` before attacks.
|
||||
|
||||
## Source-matched pairs (2026-08-21)
|
||||
|
||||
`gpt-image-2` (ChatGPT Images 2.0) can mint encoder-on/off pairs.
|
||||
`chatgpt-image-latest` is optional and 403 until org verification.
|
||||
|
||||
| Input | C2PA | Oracle | Aligned PSNR |
|
||||
| --- | --- | --- | ---: |
|
||||
| Camera JPEG 960x1280 | none | not_detected | - |
|
||||
| ChatGPT UI + frame 1086x1448 | watermarked.unbound | detected | 16.6 dB (redraw) |
|
||||
| `gpt-image-1` edit, high fidelity, red dot | created, converted only | not_detected | 30.9 dB |
|
||||
| `gpt-image-1.5` gray reproduce 1024 | created, converted only | not_detected | 24.0 dB (redraw) |
|
||||
| `gpt-image-1-mini` gray reproduce 1024 | created, converted only | not_detected | 31.1 dB |
|
||||
| `gpt-image-2` edit, red dot 1024x1536 | watermarked.unbound | detected | 28.2 dB |
|
||||
| `gpt-image-2-2026-04-21` gray reproduce 1024 | watermarked.unbound | detected | 47.5 dB |
|
||||
|
||||
The `gpt-image-2` photo residual is broadband (std 9.1), not a 16-32 stamp.
|
||||
COCO `gpt-image-2` edits: 11/12 `watermarked.unbound`, one safety block.
|
||||
|
||||
Solid gray/white through `gpt-image-2` "reproduce exactly" is
|
||||
`watermarked.unbound`. Gray 128 at 1024x1536: PSNR 41.9 dB vs the flat,
|
||||
source `not_detected`, output `detected`. Other gray levels 32-220: PSNR
|
||||
38-49 dB. Eight gray 1024x1536 residuals, leave-one-out correlation after
|
||||
highpass: 0.645 (shuffle 0). That shared stamp does not transfer to COCO
|
||||
photo residuals (0.007-0.025).
|
||||
|
||||
## Presentation gate versus residual
|
||||
|
||||
An official `not_detected` is not one event. Collage, photo-pad, two-panel
|
||||
self-concat, and 8x8 scramble can be a *presentation gate* (this upload is
|
||||
not a single photograph). Blur sigma 7 and in-band 16-32 destruction sit on
|
||||
the single-photo manifold and are residual damage; those belong in
|
||||
[mark removal](synthid-removal-research.md).
|
||||
|
||||
Do not train a surrogate on mixed collage misses if the goal is a mark
|
||||
student.
|
||||
|
||||
Identity (s1, 2026-08-19): JPEG q80/q20, crop 2 px, crop 8 px, keep 0.75
|
||||
and 0.50, rot90, rotate 5, flip, grayscale all stay `detected`.
|
||||
|
||||
Token / layout (s1): each quadrant at 512x768 `detected`; `hstack` and
|
||||
`vstack` of marked\|marked `not_detected`; 4x4 tile scramble `detected`;
|
||||
8x8 `not_detected`. Two copies of a detecting image still miss. Tomography
|
||||
3x3: all nine cells `not_detected` (a 1/9 window is below support).
|
||||
|
||||
Preprocess E1 (s1): stretch 2x `detected`; centre-crop of marked\|marked
|
||||
hstack back to native `detected`; 0.20x pixels on a native-size white
|
||||
canvas `not_detected` (the same 0.20x file uploaded alone was `detected`).
|
||||
|
||||
## L1 is not a mark task
|
||||
|
||||
A missing `watermarked.unbound` assertion is not a clean negative. On
|
||||
2026-08-16 the official verifier called 65 of 94 OpenAI rows without that
|
||||
assertion `detected` (69%), with 9 of 9 interleaved health positives
|
||||
detected so the endpoint was answering. Google's own API emits watermarked
|
||||
images with no assertion, per its documentation. Corpus AUCs that treat
|
||||
"no assertion" as unmarked rest on a negative class that is roughly
|
||||
two-thirds positive.
|
||||
|
||||
Control-only 283 rows: 203 detected, 80 not_detected. Forward-temporal
|
||||
nested ridge on 124 pixel features: AUC 0.649 / 0.641. Geometry-only:
|
||||
0.781 / 0.826. All 21 `claim_generator=ChatGPT` controls are
|
||||
`not_detected`. C2PA `opened`: 24/24 `not_detected`. Inside 1254x1254,
|
||||
mean-feature permutation p=0.45.
|
||||
|
||||
OpenAI-supervised ridge on 16-32 band-passed `image_features`:
|
||||
forward-temporal AUC 0.53 vs L1 `not_detected`, 0.95 vs COCO, 0.97 vs COCO
|
||||
after a 2 px crop. The student learns OpenAI-versus-photo and still does
|
||||
not see the oracle mark contrast.
|
||||
|
||||
## Product remainder for the watermark
|
||||
|
||||
Signed provenance (`identify`) and `verify-openai-synthid` (remote, explicit
|
||||
upload). A research lattice miss is not a clean SynthID negative.
|
||||
@@ -0,0 +1,54 @@
|
||||
# OpenAI SynthID oracle ladders
|
||||
|
||||
> This page is a routing hub. The mixed 2026-08 campaign log was split by
|
||||
> purpose on 2026-08-22. Numeric tables now live on the page for that
|
||||
> purpose. Raw images stay gitignored under
|
||||
> `.local-eval/synthid/prc-oklab-attack-2026-08-15/`.
|
||||
>
|
||||
> Seeds s1, s2, s3 are listed in
|
||||
> [SynthID local detector research](synthid-detector-research.md).
|
||||
|
||||
| Page | Use it for |
|
||||
| --- | --- |
|
||||
| [SynthID local detector research](synthid-detector-research.md) | Hunt for a keyless local mark detector. Open. Closed routes on that page. |
|
||||
| [AI-generated image classifiers](ai-generated-image-classifiers.md) | Primary metadata-free AI-generation task, Model 1 partial result, and frozen-checkpoint sweep. |
|
||||
| [SynthID source classifiers](synthid-classifiers.md) | OpenAI/Gemini source finding and `pipeline_lattice` as google-lineage. |
|
||||
| [SynthID mark removal research](synthid-removal-research.md) | Quiet-remover hunt. Closed except ~19-24 dB 16-32 scramble and blur sigma 7. |
|
||||
| [Mechanism reference](synthid.md) | How SynthID works, provenance, robustness, regeneration. |
|
||||
| [Chronological plan archive](synthid-detector-removal-plan.md) | Dated H-gates, corpora, and session notes in original order. |
|
||||
|
||||
## Where former sections went
|
||||
|
||||
| Former heading | Now |
|
||||
| --- | --- |
|
||||
| 2026-08-21 pairs, flats, L1 labels | [detector](synthid-detector-research.md) |
|
||||
| Identity, token/layout, tomography, preprocess E1 | [detector](synthid-detector-research.md) (presentation gate) |
|
||||
| L1 repair, L1 geometry, camera vs edit pair, E3 | [detector](synthid-detector-research.md) |
|
||||
| CLIP-L photo vs AI, provider CLIP, union | [AI classifiers](ai-generated-image-classifiers.md), [source classifiers](synthid-classifiers.md) |
|
||||
| 124-d three-class and binary AI | [source classifiers](synthid-classifiers.md) |
|
||||
| `pipeline_lattice` re-check, Spaces census | [source classifiers](synthid-classifiers.md) |
|
||||
| Attack / kill / flip / add / surround ladders | [removal](synthid-removal-research.md) |
|
||||
| 16-32 titration, E2 scramble, Bayer and geometry | [removal](synthid-removal-research.md) |
|
||||
| S4 provider split, M2 Imagen `addWatermark`, reverse-SynthID, Bypass | [detector](synthid-detector-research.md) |
|
||||
| Photo `d'` budget 13.4 dB, 128-photo student, 16-32 residual without flat `G` | [detector](synthid-detector-research.md) |
|
||||
| OKLab codeword replacement, add-context as presentation | [removal](synthid-removal-research.md) |
|
||||
| Three-class OpenAI / Gemini / photo ask | [source classifiers](synthid-classifiers.md) |
|
||||
| Wild hypotheses 2026-08-23 (sort, CDMA, bispectrum, PRNU, affine 16-32 NCC no lock) | [detector](synthid-detector-research.md) |
|
||||
| Wild oracle 2026-08-23: misnamed polar-1632 is cartesian scramble replicate `not_detected`; band-transplant `not_detected`; palette64 `detected` | [removal](synthid-removal-research.md) |
|
||||
| External literature 2026-08-23 (Gowal, AWPD, PRC, Zhao, UnMarker, CtrlRegen, MarkNull, reverse-SynthID) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md), [AI classifiers](ai-generated-image-classifiers.md) |
|
||||
| Adjacent literature, not SynthID (Cox, HiDDeN, StegaStamp, Tree-Ring, Ojha CLIP, Corvi Fourier, DCCT CFA, PRNU PCE) | [detector](synthid-detector-research.md), [AI classifiers](ai-generated-image-classifiers.md), [removal](synthid-removal-research.md) |
|
||||
| Image investigation and data hiding (LSB, UNIWARD, Baluja, HiNet, Gaussian Shading, ELA, JPEG ghosts, DIRE, CNNDetect) | [detector](synthid-detector-research.md), [AI classifiers](ai-generated-image-classifiers.md) |
|
||||
| Waveforms in a picture (DFT, Fourier-Mellin, Zernike, chirps, Fresnel, DRPE, cyclostationary) | [detector](synthid-detector-research.md) |
|
||||
| Remaining hypotheses 2026-08-23 (CFA, JPEG ghost, Paeth, face-gate, Baker, Poisson, nested LSB, angular scramble) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md), [AI classifiers](ai-generated-image-classifiers.md) |
|
||||
| Non-local 2026-08-23 (HEIF/AV1 survive, print-scan sim 24.95 dB still detected, gpt-image-1.5/mini encoder-off, dated gpt-image-2 same stamp, DIRE DDIM texture) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md), [AI classifiers](ai-generated-image-classifiers.md) |
|
||||
| Waveform shells 2026-08-23 (Mellin/Zernike/PCET/Fresnel no photo lock; Y scramble kills, chroma and 90° sector do not; radial-phase kills) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md) |
|
||||
| Public decoder sweep 2026-08-23/24 (`Rinne414` fixed templates 0/400 on current OpenAI/Google images; injected template spoofs the public correlator, but 0.25-2x subtraction stays official `detected` at 43.13-25.72 dB) | [detector](synthid-detector-research.md), [removal](synthid-removal-research.md) |
|
||||
| Public CNN sweep 2026-08-24 (`newideas99` retains 172/200 OpenAI but accepts 110/200 Google; a whole-frame gradient makes the ensemble effectively zero but stays official `detected` through ±8/255 / 34.30 dB) | [detector](synthid-detector-research.md), [source classifiers](synthid-classifiers.md), [removal](synthid-removal-research.md) |
|
||||
| `reverse-SynthID` V4 repeat 2026-08-24 (77/200 Google and 76/200 OpenAI at the published cut; Round-06 manual Gemini verdicts are not published as per-image evidence) | [detector](synthid-detector-research.md), [source classifiers](synthid-classifiers.md), [removal](synthid-removal-research.md) |
|
||||
| Broad GitHub sweep 2026-08-24 (133 unique primary repositories; REGRET and the vordme flat-field SVM fail the strict v7 transfer; literal ports and generic heuristics add no signal) | [detector](synthid-detector-research.md), [source classifiers](synthid-classifiers.md) |
|
||||
| Additional public removal sweep 2026-08-24 (DeSynth stays official `detected`; froggeric and other ControlNet/diffusion projects corroborate lossy regeneration; PortOS closes high-fidelity phase/noise jamming) | [removal](synthid-removal-research.md) |
|
||||
| Google verifier surfaces 2026-08-24 (retired Vertex `imageverification@001`; live but unlisted and inaccessible `synthid.googleapis.com/upload/v1:verifyContent`) | [detector](synthid-detector-research.md) |
|
||||
| Metadata-free source hunt 2026-08-24 (1% cascades rejected; frozen multiscale fusion found 276/600 OpenAI-or-Google targets, 0/1,000 photographs and 1/25 TC260 on blind v4) | [source classifiers](synthid-classifiers.md) |
|
||||
| Original-export source hybrids 2026-08-24 (frozen v7 215/400 exact; post-hoc v8 292/400 and per-codec v11 316/400 exact) | [source classifiers](synthid-classifiers.md) |
|
||||
| Frozen general-detector transfer 2026-08-24 (Community Forensics and SPAI do not replace or safely fuse with Model 1 at the 1% calibration FPR; B-Free weights unavailable and license noncommercial) | [AI classifiers](ai-generated-image-classifiers.md#frozen-public-checkpoint-transfer-2026-08-24) |
|
||||
| General-classifier GitHub sweep 2026-08-25 (2,006 unique repositories found; DDA and PGC expose complementary frozen signals, but DDA raises independent FPR and PGC's OpenAI gain confounds 24/24 Kodak scans; DGS-Net, SAFE, RINE, and Nonescape Mini rejected) | [AI classifiers](ai-generated-image-classifiers.md#general-ai-classifier-github-sweep-2026-08-25) |
|
||||
@@ -0,0 +1,243 @@
|
||||
# SynthID mark removal research
|
||||
|
||||
> Research archive for pixel-only SynthID removal, not diffusion
|
||||
> regeneration. Not a statement of current product capability. Shipped
|
||||
> invisible removal is lossy regeneration:
|
||||
> [known limitations](known-limitations.md).
|
||||
>
|
||||
> Sister pages: [SynthID local detector](synthid-detector-research.md),
|
||||
> [SynthID source classifiers](synthid-classifiers.md),
|
||||
> [mechanism reference](synthid.md).
|
||||
|
||||
## Result
|
||||
|
||||
The quality-preserving OpenAI SynthID remover hunt closed 2026-08-20.
|
||||
Bayer, VNG demosaic, upscale-then-Bayer, barrel distortion, scanline
|
||||
jitter, and a 2 px shift closed 2026-08-22 on s1/s2 and 2026-08-23 on s3
|
||||
and fish: they leave the official oracle `detected`.
|
||||
|
||||
Working residual kills on photographs cost about 19-26 dB:
|
||||
|
||||
- 16-32 px cartesian phase scramble (s1 24.6-24.8 dB, fish 23.2 dB, s2 19.0 dB)
|
||||
- Fourier-angle scramble of the same annulus (s1 24.1 dB)
|
||||
- Radial-phase scramble of the same annulus (s1 25.0 dB)
|
||||
- Y-only 16-32 scramble (s1 24.6 dB); Cb/Cr-only do not kill
|
||||
- File named polar-1632, actually cartesian (s1 25.6 dB)
|
||||
- Replace 16-32 with a COCO photo's 16-32 (s1 25.2 dB)
|
||||
- Gaussian blur sigma 7 (23.8 dB), holds 3/3 versus sigma 6
|
||||
- Additive 16-32 jam only at a=24 / 18.6 dB, worse than scramble
|
||||
|
||||
Baker-map, 8-seam carve, Poisson, nested LSB, palette64, and ICC rewrite
|
||||
do not kill at a better PSNR.
|
||||
|
||||
JPEG q5, noise sigma 16, grayscale, rot90, flip, 5°, downscale 0.20x,
|
||||
median 7, posterize 4, VAE round-trip, and white pad to 40% linear stay
|
||||
`detected`. Elastic warp is not a stable kill (s3 still `detected` at
|
||||
22.0 dB).
|
||||
|
||||
The product remainder is diffusion regeneration (`qwen-zimage` /
|
||||
`sdxl-zimage`), which does not decode and delete a payload. Defeating the
|
||||
verifier does not restore forensic deniability; see
|
||||
[synthid.md](synthid.md#23-removal-attacks-and-forensic-detectability).
|
||||
|
||||
Official `not_detected` on collage, photo-pad, and two-panel layouts is a
|
||||
presentation gate, not residual damage. Those rungs are in
|
||||
[detector research](synthid-detector-research.md).
|
||||
|
||||
## Closed quiet removers
|
||||
|
||||
| Attack | Close | Notes |
|
||||
| --- | --- | --- |
|
||||
| Quality-preserving photo remover | 2026-08-20 | Every residual `not_detected` that is not a collage is below usable quality |
|
||||
| Additive in-band jam as a quiet remover | 2026-08-21 | Fish flips only at 18.6 dB; 4-8 px at the same PSNR stays `detected` |
|
||||
| Bayer bilinear / VNG / upscale-Bayer | 2026-08-23 | s1/s2/s3 and fish still `detected`. s3 bilinear 37.1 dB, fish 34.7 dB. VNG on s2 is dirtier than scramble (18.5 vs 19.0 dB) and the mark remains |
|
||||
| Barrel k1=0.06, scanline ±0.8 px, and shift 2 px | 2026-08-23 | s1/s2/s3/fish `detected` even at 14-20 dB barrel. Fish scanline 32.5 dB and shift 22.7 dB still `detected` |
|
||||
| TrustMark-style micro-warp | 2026-08-21 | 0.25 px / 32 dB still 100% detect on TrustMark P; OpenAI elastic ~21 dB unreplicated |
|
||||
| OKLab random-codeword replacement as a quiet wipe | 2026-08-15 | On four public Google-oracle positives, replacing the period-16 tile at 0.95 dropped the *local* lattice score below 0.173 at ~54 dB. Gemini pixel verify never accepted the candidates (`Connecting to Verify AI`). Local-score intervention, not an official wipe |
|
||||
| PRC-style OKLab a/b noise plus spectral peak cleaning | 2026-08-15 | Independent high-pass OKLab noise at 32.6 dB moved the median local score by only `-0.0219`. Comb suppression moved one of four below threshold at 58 dB. Not a quiet official kill |
|
||||
| Rinne414 fixed-template subtraction | 2026-08-24 | The pinned pre-May GPT-Image2 residual was resized onto a current verified OpenAI image and subtracted at 0.25x, 0.5x, 1x, and 2x. Official SynthID stayed `detected` at 43.13, 37.49, 31.60, and 25.72 dB after metadata stripping. Even the dirtiest rung is no better than the existing 24-25 dB annulus-phase scramble. The template can spoof its own public correlator but does not cancel the current mark |
|
||||
| `newideas99` CNN gradient | 2026-08-24 | A 3x3 crop-grid objective spread the white-box perturbation over the full marked frame. The frozen ensemble fell from 0.99995 to 0.25956 at ±2/255 (45.92 dB) and to effectively zero at ±4/255 (40.30 dB). Official OpenAI SynthID stayed `detected` at ±1, ±2, ±4, and ±8/255, through 34.30 dB. The gradient attacks the surrogate's shortcut, not the production mark |
|
||||
| `reverse-SynthID` V4 Round-06 | 2026-08-24 | The advertised `final`/`nuke` path is SD-VAE regeneration plus elastic and affine warps, resize squeeze, color change, residual FFT subtraction, and a JPEG/noise chain, with PSNR floors of only 14/11 dB. The repository claims 20 manual Gemini-app successes but contains no manifest, tally, or per-image verdicts. Its four bundled older cleaned pairs are 45.7-50.2 dB, yet the reproduced V4 score rises on three and is nearly unchanged on the fourth. No callable Google pixel oracle was available for an independent Round-06 verdict. This is an unverified lossy regeneration/distortion stack, not a quiet pattern cancellation |
|
||||
| [`0xROOTPLS/DeSynth`](https://github.com/0xROOTPLS/DeSynth/tree/96db920731c2a3d04bf13163a5077b9a67706d1b) | 2026-08-24 | Qwen Image img2img at strength 0.25, followed by a Gaussian frequency split that restores the source high frequencies above sigma 1.95. The repository's public OpenAI original, default output, and edge-guided output all returned official `detected` in a current repeat after metadata stripping. The published `not found` claim does not reproduce against the current OpenAI oracle |
|
||||
| [`froggeric/gemini-watermark-and-synthid-remover`](https://github.com/froggeric/gemini-watermark-and-synthid-remover/tree/5918384ce403968de0560cefd889e50eba0163bc) | 2026-08-24 | SDXL img2img with a documented manual Google-verifier ladder. The author reports 7/8 clears at strength 0.08 and 9/9, including a double mark, at strength 0.10 with five effective denoise steps and PSNR 29-41 dB. The exact nine before/after verdict artifacts are not tracked, so this is useful external regeneration corroboration, not an independently reproduced oracle result |
|
||||
| [`atomantic/PortOS`](https://github.com/atomantic/PortOS/tree/b11a93e110262925c64a1b145a154ca87b340055) adversarial-jamming experiment | 2026-08-24 | Its own one-image manual OpenAI run found that quality-preserving phase noise, band noise, blur, and 0.70 resize squeeze stayed detected. Only visibly destructive phase perturbation cleared. A 0.85-0.90 resize caused repeated detector timeouts, which the repository correctly keeps separate from `not_detected`. This independently closes high-fidelity additive/phase jamming, but the source artifacts are not published |
|
||||
| Generic regeneration claims | 2026-08-24 | `mertizci/noai-watermark`, `BovineOverlord/Loyal-Bear`, `obaskly/NeuralBleach`, and `tymongumienik/unwatermark` are SD/CtrlRegen/ControlNet redraws. Their algorithms are already covered by the product's regeneration remainder and the CtrlRegen literature. README verdicts without a frozen manifest do not establish a quieter or more reliable operating point |
|
||||
| Add unmarked context (frame, pad, hstack, collage) | 2026-08-19 | Official `not_detected` is a presentation gate. White pad 40% linear stays `detected`; photo fill at the same geometry is `not_detected`. See detector identity/layout |
|
||||
| `cebeuq/Synthid-Bypass` diffusion reconstruction | 2026-08-13 | 12 public pairs. Lattice and Gemini Verify go quiet on the *after* images because the image was redrawn, not because a payload was deleted. Faces and text are a quality claim for regeneration, not a local wipe |
|
||||
|
||||
Oracle: `verify_openai_synthid` after AI-metadata strip. Seeds s1, s2, s3
|
||||
as in [detector research](synthid-detector-research.md). Raw files:
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/`.
|
||||
|
||||
## Band that actually carries the mark
|
||||
|
||||
Phase-randomize one octave at a time, preserve energy (E2, s1, replicated
|
||||
p16_32 on s2 and s3):
|
||||
|
||||
| Octave (period px) | Verdict |
|
||||
| --- | --- |
|
||||
| 2-4 | detected |
|
||||
| 4-8 | detected |
|
||||
| 8-16 | detected |
|
||||
| 16-32 | not_detected |
|
||||
| 32-64 | detected |
|
||||
| 64-128 | detected |
|
||||
|
||||
Destroying only 16-32 px periods silences the official decoder. The
|
||||
lattice band 8-16 px does not. On a marked gray flat the same scramble is
|
||||
51.1 dB `not_detected` because the band is almost empty. On a photograph
|
||||
the band holds the scene (24.3 dB). Replacing native 16-32 with a
|
||||
2x-pyramid prediction is 48.4 dB and still `detected`.
|
||||
|
||||
Fish additive jam, luma RMS `a`:
|
||||
|
||||
| Band | a | PSNR | Verdict |
|
||||
| --- | ---: | ---: | --- |
|
||||
| 16-32 | 0.06-4.0 | 51.2-33.6 | detected |
|
||||
| 16-32 | 8 | 27.7 | detected |
|
||||
| 16-32 | 16 | 21.9 | detected |
|
||||
| 16-32 | 24 | 18.6 | not_detected |
|
||||
| 4-8 | 16 | 21.9 | detected |
|
||||
| 4-8 | 24 | 18.6 | detected |
|
||||
|
||||
Preregistered close `a*_photo < 0.53` failed (`a* > 16`).
|
||||
|
||||
## Bayer and geometry (2026-08-22)
|
||||
|
||||
Frozen one-pass batch. Lattice scores on these OpenAI sources are already
|
||||
`indeterminate` and are not the label.
|
||||
|
||||
| Attack | s1 | s2 | s3 | fish |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| source | detected | detected | detected | detected |
|
||||
| Bayer bilinear | 32.7 detected | 28.4 detected | 37.1 detected | 34.7 detected |
|
||||
| Bayer VNG | 25.0 detected | 18.5 detected | 26.4 detected | 25.4 detected |
|
||||
| upscale 1.15 then Bayer | 33.9 detected | 29.8 detected | 38.4 detected | 36.0 detected |
|
||||
| barrel k1=0.06 | 19.9 detected | 14.2 detected | 20.2 detected | 18.5 detected |
|
||||
| scanline ±0.8 px | 31.5 detected | 26.7 detected | 35.7 detected | 32.5 detected |
|
||||
| shift 2 px | 23.1 detected | 15.9 detected | 24.7 detected | 22.7 detected |
|
||||
| 16-32 phase scramble | 24.6 not_detected | 19.0 not_detected | 25.2 not_detected | 23.2 not_detected |
|
||||
|
||||
Camera pipeline and mild geometry do not hit the decoder basis. A 2 px
|
||||
shift, which kills `pipeline_lattice`, left s1/s2/s3/fish `detected`.
|
||||
Scramble remains the residual kill on 4/4 sources.
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/bayer-geometry-2026-08-22/`.
|
||||
Manifest `status` is `oracle_complete` (2026-08-22 22:07).
|
||||
|
||||
## Attack ladders (OpenAI oracle)
|
||||
|
||||
Pilot 2026-08-17 (n=3): JPEG q40, 0.5x, noise8+jpeg85, 0.35x+jpeg50 all
|
||||
`detected` (s2 0.35x+jpeg50 at 20.1 dB still detected).
|
||||
|
||||
Kill and flip (s1), then replication 3/3:
|
||||
|
||||
| Rung | s1 | s2 | s3 |
|
||||
| --- | --- | --- | --- |
|
||||
| blur sigma 6 | detected | detected | detected |
|
||||
| blur sigma 7 | not_detected | not_detected | not_detected |
|
||||
| keep 0.50 linear | detected | detected | detected |
|
||||
| keep 0.40 linear | not_detected | not_detected | not_detected |
|
||||
| elastic ~24 dB | detected 24.6 | detected 17.9 | detected 26.1 |
|
||||
| elastic ~21 dB | not_detected 20.8 | not_detected 14.8 | detected 22.0 |
|
||||
| pad white 40% linear | detected | detected | detected |
|
||||
| hstack with photo | not_detected | not_detected | not_detected |
|
||||
| blend 25% marked | detected | detected | detected |
|
||||
| inset native 1:1 on 2x photo | not_detected | not_detected | not_detected |
|
||||
| JPEG q5 | detected | detected | detected |
|
||||
|
||||
Holds on 3/3: blur 6 vs 7, crop keep 0.50 vs 0.40, white pad 40%
|
||||
`detected`, hstack `not_detected`, blend 25% `detected`, JPEG q5
|
||||
`detected`. Elastic at ~21 dB does not hold. Crop keep-0.40 and hstack
|
||||
are presentation or support, not a quiet residual wipe.
|
||||
|
||||
Surround: photo fill at the white-pad geometries is `not_detected` on
|
||||
3/3; white pad at those sizes was `detected`. Unmarked photo strip flips
|
||||
between 25% width (`detected`) and 35% (`not_detected`). Photo frame 15%
|
||||
each side is `not_detected` on 3/3; 12% does not hold.
|
||||
|
||||
## Wild removers (oracle 2026-08-23)
|
||||
|
||||
Preregistered on s1, one pass, 25 s gap. Manifest:
|
||||
`.local-eval/synthid/prc-oklab-attack-2026-08-15/wild-attacks-2026-08-23/oracle-manifest.json`.
|
||||
|
||||
| Attack | PSNR | Verdict | Note |
|
||||
| --- | ---: | --- | --- |
|
||||
| Replace 16-32 with a COCO photo's 16-32 | 25.2 | not_detected | Same annulus as cartesian scramble (24.6 dB). Substituting a camera band kills the decoder, 0.6 dB quieter than scramble on s1, not a new quality class |
|
||||
| File named polar-1632 | 25.6 | not_detected | Misnamed. The raster randomizes cartesian FFT phase in the 16-32 annulus, same family as scramble (24.6 dB). It is a scramble replicate, not a `(r, θ)` polar test |
|
||||
| 64-color median-cut palette | 35.6 | detected | Quieter than scramble and still marked. Posterize-4 already stayed `detected`; a smarter quantizer is not enough |
|
||||
| PIL RGB to CMYK to RGB | inf | not submitted | No-op on this PNG |
|
||||
|
||||
The 16-32 kill is luma phase in that annulus. Cartesian, Fourier-angle,
|
||||
and radial-phase all silence the decoder near 25 dB. A 90° sector of the
|
||||
same ring does not (27.5 dB `detected`). Cb-only and Cr-only 16-32
|
||||
scrambles stay `detected` at 44-45 dB. A foreign-scene transplant of the
|
||||
same band also silences the decoder near 25 dB. Palette, Baker-map, seam
|
||||
carve, Poisson, nested LSB, and ICC do not.
|
||||
|
||||
Follow-up 2026-08-23, s1, 25 s gap, `remaining-2026-08-23/oracle-manifest.json`:
|
||||
|
||||
| Attack | PSNR | Verdict |
|
||||
| --- | ---: | --- |
|
||||
| Fourier-angle 16-32 scramble | 24.1 | not_detected |
|
||||
| Cartesian 16-32 scramble (replicate) | 24.8 | not_detected |
|
||||
| Baker-map of the 16-32 band | 27.8 | detected |
|
||||
| Poisson noise | 30.7 | detected |
|
||||
| Nested LSB in blue | 55.9 | detected |
|
||||
| Seam carve 8 | 27.7 | detected |
|
||||
| ICC sRGB rewrite | inf | not submitted, no-op |
|
||||
|
||||
Non-local codecs, 2026-08-23, s1, 25 s gap:
|
||||
|
||||
| Attack | PSNR | Verdict |
|
||||
| --- | ---: | --- |
|
||||
| HEIF q80 | 46.3 | detected |
|
||||
| HEIF q50 | 39.3 | detected |
|
||||
| AV1 CRF 32 | 37.2 | detected |
|
||||
| Print-scan simulation | 24.95 | detected |
|
||||
|
||||
Physical print-scan is still blocked unattended (Brother DCP-L2520DW idle and
|
||||
accepting, no `scanimage`, no ImageCapture pyobjc). Face-gated scramble is
|
||||
unnecessary: Haar on s1 put *more* 16-32 energy on faces. Generic 25 dB is
|
||||
not the kill: this simulation stays `detected` at the PSNR where 16-32
|
||||
phase scramble does not.
|
||||
|
||||
Waveform-shell splits, 2026-08-23, s1, 25 s gap,
|
||||
`waveforms-shells-2026-08-23/oracle-manifest.json`:
|
||||
|
||||
| Attack | PSNR | Verdict |
|
||||
| --- | ---: | --- |
|
||||
| Y-only 16-32 scramble | 24.6 | not_detected |
|
||||
| Cb-only 16-32 scramble | 45.0 | detected |
|
||||
| Cr-only 16-32 scramble | 43.9 | detected |
|
||||
| 90° Fourier sector of 16-32 | 27.5 | detected |
|
||||
| Radial-phase-only 16-32 | 25.0 | not_detected |
|
||||
|
||||
## External literature (surveyed 2026-08-23)
|
||||
|
||||
Primary sources. Detector papers live in
|
||||
[detector research](synthid-detector-research.md). Forensic stealth of
|
||||
regeneration is already in [synthid.md](synthid.md#23-removal-attacks-and-forensic-detectability).
|
||||
|
||||
| Source | Attack | Against SynthID? | Map to this campaign |
|
||||
| --- | --- | --- | --- |
|
||||
| Zhao et al., [arXiv:2306.01953](https://arxiv.org/abs/2306.01953) (NeurIPS 2024) | Add noise, then denoise or regenerate (VAE / diffusion). Pixel-level invisible marks are provably removable. Semantic watermarks proposed as the alternative | Open post-hoc schemes, not production SynthID | This is the family our product uses (`qwen-zimage` / `sdxl-zimage`). Gowal trains SynthID-O against *weak* VAE regeneration. Our foreign-VAE round-trip at 22.3 dB stayed `detected`. Regeneration works when it redraws, not when it is a light codec |
|
||||
| Liu et al., [arXiv:2410.05470](https://arxiv.org/abs/2410.05470) (CtrlRegen, ICLR 2025) | Controllable diffusion from clean noise, with a knob on how many noise steps to add | SOTA open watermarks | Same family. Goonatilake later finds CtrlRegen+ the *most* forensically detectable remover (AUROC 0.9999) |
|
||||
| Kassis and Hengartner, [arXiv:2405.08363](https://arxiv.org/abs/2405.08363) (UnMarker, IEEE S&P 2025) | No decoder feedback. Two adversarial spectral optimizations. Breaks even some semantic watermarks (best remaining detection 43%) | Not production SynthID | Spectral disruption without an oracle is the honest analog of our 16-32 scramble, except UnMarker is optimized and we used a one-octave phase shuffle. Goonatilake: UnMarker TPR 98.28% at 0.1% FPR as a *forensic* leftover |
|
||||
| Tallam et al., [arXiv:2505.08234](https://arxiv.org/abs/2505.08234) (SemanticRegen) | Partial, label-free regeneration of main objects | Tree-Ring, StegaStamp, StableSig, DWT/DCT. Not SynthID | Partial redraw. Our collage / photo-pad `not_detected` is a presentation gate, not this attack |
|
||||
| Cao et al., [arXiv:2608.10166](https://arxiv.org/abs/2608.10166) (MarkNull, USENIX Security 2026) | On-manifold latent decorrelation via a public diffusion proxy. Claims 100% on 20 Imagen-3 Gemini-verify images. PSNR 25.36 dB, SSIM 0.80 | Small Gemini-verify set | Independent evidence that a no-box latent reconstruction can confuse Gemini. Does not meet this project's 40 dB / 0.99 SSIM release gate. Still generation, not a pixel-only wipe |
|
||||
| Goonatilake and Ateniese, [arXiv:2605.09203](https://arxiv.org/abs/2605.09203) | Six removers all leave a forensic residue a ResNet-50 sees at >98% TPR @ 1% FPR | Applies to UnMarker, Zhao's WatermarkAttacker, CtrlRegen+ | Defeating `verify-openai-synthid` is not deniability. This is the product remainder |
|
||||
| An et al., [arXiv:2401.08573](https://arxiv.org/abs/2401.08573) (WAVES, ICML 2024) | 26 attacks on StegaStamp, Stable Signature, Tree-Ring. Regeneration, not JPEG, is the attack that matters. StegaStamp TPR@1%FPR 1.00 to 0.01; Tree-Ring 0.99 to 0.12 | Open watermarks | Protocol. Our blur-sigma-7 and 16-32 scramble are closer to WAVES "distortion" than to regeneration |
|
||||
| Wen et al., [arXiv:2305.20030](https://arxiv.org/abs/2305.20030) (Tree-Ring) | In-generation Fourier pattern in initial noise | Not SynthID | Kill is DDIM inversion plus latent wipe, not a pixel scramble. Lin and Juarez [arXiv:2506.10502](https://arxiv.org/abs/2506.10502) (USENIX 2025) remove it from public knowledge |
|
||||
| Fernandez et al., Stable Signature, ICCV 2023 | Fine-tuned VAE decoder | Not SynthID | Regeneration that replaces the decoder is in-family. Our foreign VAE at 22.3 dB did not kill OpenAI SynthID, which is the expected mismatch |
|
||||
| Google to [The Verge, 2026-04-14](https://www.theverge.com/ai-artificial-intelligence/911579/google-synthid-ai-watermarking-system-reverse-engineered) | reverse-SynthID does not systematically remove SynthID | Google spokesperson on the spectral bypass | Agrees with our V4 audit and with Bypass: reconstruction or decoder confusion, not a payload delete |
|
||||
|
||||
Quiet pixel-only removal is not a published result on production
|
||||
OpenAI or Google SynthID. Every paper that "beats" a watermark either
|
||||
redraws the image, attacks an open encoder, or reports PSNR in the same
|
||||
19-25 dB band as our 16-32 scramble.
|
||||
|
||||
## Product remainder for removal
|
||||
|
||||
Invisible removal in this package regenerates through `qwen-zimage` or
|
||||
`sdxl-zimage`. It is lossy. There is no shipped pixel-only OpenAI or
|
||||
Google SynthID wipe. Do not add Bayer or geometry as remover arms.
|
||||
@@ -248,7 +248,7 @@ from the test set + this doc).
|
||||
watermark. Stop and reconsider.
|
||||
2. **PhotoMaker-V1 prototype** in the existing `controlnet` pipeline:
|
||||
- Mirror the `_load_controlnet_pipeline` path (removed in 0.24.0 with the
|
||||
controlnet profile; the equivalent seam is now `SdxlZImagePipeline._load_sdxl`):
|
||||
controlnet profile; the equivalent seam is now `SdxlZImagePipeline._load_global`):
|
||||
add a PhotoMaker variant that
|
||||
loads SDXL + canny ControlNet + PhotoMaker adapter on the same engine.
|
||||
- Extract the OpenCLIP face embedding from the watermarked face crops (use
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
# SynthID: technical reference
|
||||
|
||||
> Technical research reference. Current package behavior is defined by the
|
||||
> Technical research reference for the mark itself and for what this package
|
||||
> ships. Current package behavior is defined by the
|
||||
> [supported signals](supported-signals.md), [known limitations](known-limitations.md),
|
||||
> and [module internals](module-internals.md). Dated measurements below are
|
||||
> and [module internals](module-internals.md). Campaign results are split:
|
||||
> [local detector](synthid-detector-research.md),
|
||||
> [SynthID source classifiers](synthid-classifiers.md),
|
||||
> [mark removal](synthid-removal-research.md). Dated measurements below are
|
||||
> historical evidence and should not be read as current CLI defaults.
|
||||
|
||||
This document covers how Google SynthID for images works mechanically, what it
|
||||
@@ -42,11 +46,58 @@ after generation, by a separate neural network. This means:
|
||||
- Replacing the generative model does not remove the watermarking capability.
|
||||
- The encoder/decoder pair can be updated independently of the generative model.
|
||||
|
||||
The detector's training target is narrower than provider or AI-image
|
||||
classification. Equation 1 trains on watermarked `f(x)` as positive and the
|
||||
corresponding unwatermarked distribution `x` as negative, with both drawn from
|
||||
the same target image distribution. Equation 2 applies the same sampled
|
||||
semantics-preserving transformation to both sides. Unrelated generators and
|
||||
camera images are useful later for false-positive evaluation, but they do not
|
||||
replace source-matched clean examples during signal-identifiability training.
|
||||
|
||||
The decoder produces a dedicated detection logit whose threshold is calibrated
|
||||
after training for a target false-positive rate. Payload recovery is a separate
|
||||
problem, not the definition of the presence score. The paper additionally
|
||||
describes two-sided conformal calibration for the `not watermarked` and
|
||||
`watermarked` hypotheses, which permits a detector to abstain instead of forcing
|
||||
an unsupported binary verdict.
|
||||
|
||||
The paper does not disclose the internal architecture of the encoder/decoder
|
||||
networks (layer types, capacity). The external variant SynthID-O is available
|
||||
to partners; the production internal variant is not published.
|
||||
|
||||
### 1.2 How it differs from classical DWT-DCT watermarks
|
||||
### 1.2 Patent-backed architecture clues
|
||||
|
||||
A related [DeepMind patent family](https://patents.google.com/patent/US12094474B1/en),
|
||||
filed in 2023 and naming several authors of the SynthID-Image paper, describes
|
||||
the likely architectural design space in more detail. Patent alternatives are
|
||||
not proof that every option is deployed in production SynthID, so the following
|
||||
items are constraints and research clues rather than implementation claims.
|
||||
|
||||
The patent's image example forms a content-dependent residual, `x' = x + g(x)`.
|
||||
It describes a U-Net-like watermark generator with convolution, attention, and
|
||||
skip connections; a separately trained convolutional decoder; optional message
|
||||
or secret-key injection into intermediate layers; and paired training on clean
|
||||
and watermarked images under sampled differentiable transformations. It also
|
||||
allows an image to be resized to a trained target size for watermarking and then
|
||||
resized back to its original dimensions.
|
||||
|
||||
Most importantly for blind detection, a
|
||||
[continuation patent](https://patents.google.com/patent/US20250149048A1/en)
|
||||
describes groups or ensembles of paired encoder/decoder networks. A decoder can
|
||||
be trained not to recognize marks from another pair, while deployment can
|
||||
select one pair or combine several outputs. This supplies a concrete mechanism
|
||||
for coexisting watermark versions, provider-specific states, or distinct
|
||||
codewords without requiring one universal pixel template.
|
||||
|
||||
The fixed periodic carriers measured in this project should therefore be
|
||||
treated as linear experts for particular observable states, not as the
|
||||
definition of SynthID. A broader detector needs independently validated experts
|
||||
for additional encoder states and transformations, followed by joint
|
||||
false-positive calibration. The paper's separate detection logit and two-sided
|
||||
conformal decision provide the correct target behavior: present, absent, or
|
||||
abstain when neither hypothesis is supported.
|
||||
|
||||
### 1.3 How it differs from classical DWT-DCT watermarks
|
||||
|
||||
The open watermarks used by Stable Diffusion / SDXL / FLUX (via the
|
||||
`imwatermark` library) use classical **DWT-DCT** frequency-domain embedding: a
|
||||
@@ -70,7 +121,7 @@ embeds in fixed, predictable frequency bins, making it brittle to any
|
||||
operation that hits those bins (e.g., JPEG re-quantization wipes it cleanly at
|
||||
quality <= 90).
|
||||
|
||||
### 1.3 Payload capacity
|
||||
### 1.4 Payload capacity
|
||||
|
||||
SynthID-O (the external/partnership variant) encodes:
|
||||
|
||||
@@ -90,7 +141,7 @@ paper separates watermark **detection** (is this watermarked?) from payload
|
||||
**recovery** (what does the payload say?): the detection path is what oracles
|
||||
like the Gemini app's "Verify with SynthID" exercise.
|
||||
|
||||
### 1.4 Where in the pipeline it lives
|
||||
### 1.5 Where in the pipeline it lives
|
||||
|
||||
```
|
||||
[Diffusion model]
|
||||
@@ -176,6 +227,232 @@ conversion (the body text of Section 10 is absent from the HTML).
|
||||
|
||||
**What is known empirically from our own oracle-verified testing.**
|
||||
|
||||
A 2026-08-09 non-generative pilot found a promising Google phase-correlate,
|
||||
but did not establish a releasable local detector or pixel-only remover.
|
||||
The 2026-08-20/22 OpenAI campaign is recorded in
|
||||
[synthid-detector-research.md](synthid-detector-research.md),
|
||||
[synthid-classifiers.md](synthid-classifiers.md), and
|
||||
[synthid-removal-research.md](synthid-removal-research.md).
|
||||
JPEG q5 and 16-32 px phase structure survive as the official mark; a
|
||||
quality-preserving local remover and a local SynthID detector for
|
||||
photographs were not found. `gpt-image-2` source-matched flat pairs exist;
|
||||
their residual does not transfer to photographs. The
|
||||
best independently fitted spectral model relearned phase and magnitude from four of our
|
||||
positives while using third-party candidate coordinates; its second frozen
|
||||
epoch had zero false positives on 279 new exact-size external images and
|
||||
detected the one new confirmed positive used for validation. An additional
|
||||
3,000 upscaled photographs all fell outside the measured active-carrier support
|
||||
and therefore count as abstentions, not negatives. The corpus is still far too
|
||||
small and lacks same-provider hard negatives needed for a 0.1% FPR claim.
|
||||
Repeating the spectral fit in six color spaces produced zero false positives
|
||||
on the same 279-image comparison set for every branch. HSV had the best
|
||||
observed worst-negative margin, but paired normalized negative scores did not
|
||||
show a general improvement over RGB. Its apparent benefit came from saturation
|
||||
and value; hue failed its channel-level separation check. Luminance-like
|
||||
channels dominated YCbCr, YCoCg, opponent, and Lab. This narrows the next
|
||||
hypothesis to intensity/contrast and HSV S+V projections rather than a
|
||||
hue-specific carrier, but it does not add independent positives or certify an
|
||||
operating point.
|
||||
The resulting positive-only RGB plus S/V ensemble passed four
|
||||
leave-one-positive-out checks, detected all five available positive controls,
|
||||
and emitted no positive verdict on 330 newly collected exact-size images.
|
||||
Almost all external images lacked sufficient measured carrier support and
|
||||
therefore remained abstentions rather than proven negatives.
|
||||
A later post-freeze challenge used three additional exact-size Google originals
|
||||
with signed Google LLC C2PA and explicit SynthID-present assertions. The
|
||||
ensemble abstained on all three. They do not replace matching-oracle pixel
|
||||
labels, but this zero-of-three source-provenance result rejects the frozen
|
||||
ensemble as a general Google detector and narrows it to an epoch- or
|
||||
surface-specific correlate.
|
||||
|
||||
A multi-epoch leave-one-positive-out refit recovered six of the eight total
|
||||
Google positives under a strict RGB-plus-HSV decision; RGB alone ranked every
|
||||
excluded positive above the 50-image calibration maximum but produced up to
|
||||
eight false positives on 279 held-out external negatives. The signal therefore
|
||||
transfers across the two source groups, but remains content-sensitive and
|
||||
cannot be separated from provider or generation-pipeline attribution without
|
||||
same-provider oracle-negative controls.
|
||||
|
||||
Directly projecting out the ensemble phases cleared the local detector above
|
||||
51 dB PSNR, but three frozen candidates remained detected by Gemini in a
|
||||
healthy control session. A subsequent spatial analysis found that 81.25% of
|
||||
the top 256 carrier bins lie on a lattice corresponding to a repeating 16x32
|
||||
pixel cell. Modulo-folding and subtracting the complete high-pass tile gave
|
||||
phase-specific matched-control separation across all five Google positives at
|
||||
at least 55.67 dB PSNR: aligned outputs cleared the local ensemble, while
|
||||
one-pixel-shifted controls remained positive. The mild locally clearing tile
|
||||
candidate nevertheless remained detected in a healthy Google Verify AI
|
||||
session, so the tile correlate is not a sufficient removal loss.
|
||||
Sparse subtraction reduced the local score sharply at more than 50 dB PSNR,
|
||||
yet healthy Gemini sessions still detected SynthID. A one-off negative was
|
||||
discarded because the same session also missed the source-positive control.
|
||||
An OpenAI-specific 8-pixel phase candidate likewise changed its local score at
|
||||
46.70 dB PSNR but remained `SynthID detected` in one frozen OpenAI Verify
|
||||
check. No Google threshold or carrier was used in that experiment, and no
|
||||
further public OpenAI checks were made because the documented verifier guidance
|
||||
rules out repeated watermark-removal queries.
|
||||
|
||||
A later local EOT pilot combined periodic full-frame residuals with JPEG-aware
|
||||
optimization against provider-specific CNN surrogates. It produced candidates
|
||||
that stayed below three local models after actual JPEG round trips, but the
|
||||
models remain provider classifiers without same-provider oracle-negative
|
||||
controls. The selected Google candidate retained 36.80 dB PSNR and 0.9241
|
||||
SSIM; the selected OpenAI candidate retained 34.97 dB and 0.9356. Both miss the
|
||||
research fidelity gate. The OpenAI periodic residual also reduced all three
|
||||
local model scores on 11 of 11 additional images before and after JPEG-90, but
|
||||
neither provider candidate has a negative matching-provider oracle verdict, so
|
||||
neither is an established remover.
|
||||
|
||||
A later paired open-method control tested stationary wavelet and complex
|
||||
spectral representations without treating another watermark as SynthID. On a
|
||||
source-disjoint TrustMark P proxy, the SWT branch moved 59 of 60 transformed
|
||||
clean/watermarked pairs in the correct direction but did not improve the
|
||||
spatial model's aggregate AUC. Complex FFT maps were indistinguishable from
|
||||
chance, and fixed phase scoring collapsed under crop. Wavelets therefore remain
|
||||
a gated late-fusion ablation, while complex spectral analysis remains a
|
||||
residual-discovery tool rather than a presence score.
|
||||
|
||||
Expanding that proxy to four causally paired watermark families did not produce
|
||||
a universal residual detector. TrustMark P, VideoSeal Y-256, DWT-DCT, and WAM
|
||||
each used 90 source-matched images and random payloads; official decoders
|
||||
validated which transformed positives still carried readable marks. A fixed
|
||||
steganalysis residual bank and a neural model trained with cross-family
|
||||
residual mixing, sign and phase augmentation, and equal-power phase-scrambled
|
||||
hard negatives were evaluated leave-one-family-out. The four unseen-family
|
||||
AUCs were 0.562, 0.474, 0.497, and 0.437. A network can therefore learn each
|
||||
available encoder without learning a transferable class called "neural
|
||||
watermark." This open-proxy route is not evidence for Google or OpenAI SynthID.
|
||||
|
||||
The same conclusion held on a larger temporally split closed corpus. A
|
||||
three-seed normalized-patch ensemble reached 0.884 AUC, but its threshold above
|
||||
all validation negatives produced a test false positive and detected only
|
||||
2.45% and 8.97% of the two positive-source classes. Naive full-field pooling,
|
||||
late-fused SWT, robust patch aggregation, and a separate binary detection head
|
||||
did not improve that low-FPR operating point. Vendor classification was highly
|
||||
accurate among accepted positives, but it cannot rescue an unreliable presence
|
||||
decision. Source labels therefore remain suitable for vendor triage and hard
|
||||
negative evaluation, not for establishing a SynthID detector without
|
||||
counterfactual or oracle watermark labels.
|
||||
|
||||
A later low-content matrix also rejected wavelet energy and normalized
|
||||
wavelet-shape classifiers: they separated real target outputs from artificial
|
||||
flat, Gaussian, and phase-random controls, then accepted 61.6-100% of real
|
||||
external negatives. The surviving branch is narrower. A provider-specific
|
||||
1536x2816 phase carrier detected all four temporal-test positives, bounded
|
||||
translation registration recovered all four one-pixel shifts, and the joint
|
||||
phase/support rule produced zero positives on 194 frozen negatives. A
|
||||
scale-and-translation discovery rule also produced zero positives on a
|
||||
preregistered 3,000-image COCO challenge, but every COCO image was outside
|
||||
carrier support and the scale threshold was selected post hoc. The result is
|
||||
therefore a positive-only, geometry- and epoch-specific expert with abstention,
|
||||
not a universal SynthID detector. Exact measurements and remaining calibration
|
||||
gates are in the
|
||||
[`detector and removal research plan`](synthid-detector-removal-plan.md#2026-08-10-low-content-controls-and-registered-phase-carrier).
|
||||
|
||||
The next native-geometry experiment isolated a stronger mechanism. At
|
||||
2048x2048, 108 selected spatial frequencies formed a 128-bin lattice, implying
|
||||
a 16x16 periodic residual tile. Folding and averaging 16,384 tile repetitions
|
||||
produced a spatial detector that accepted 29 of 30 test positives, none of 49
|
||||
calibration negatives, and none of 38 held-out test negatives. It also accepted
|
||||
the same two of 182 earlier external-source images as the independent RGB and
|
||||
HSV phase branches. Those cases count against operational source-label FPR,
|
||||
but may contain the same carrier through an upstream encoder; only an oracle
|
||||
can distinguish the two explanations. A normalized tile challenge accepted
|
||||
none of 3,000 general images, while symmetric attacks showed strong resize but
|
||||
limited JPEG and crop robustness. The pickle-free research implementation is
|
||||
`scripts/synthid_periodic_tile_probe.py`; exact evidence and caveats are in the
|
||||
[`2048 periodic-tile experiment`](synthid-detector-removal-plan.md#2026-08-10-2048-periodic-tile-detector).
|
||||
|
||||
The historical 2048 reports retain aggregate counts and model hashes, but not
|
||||
the 111 fitting paths. A later exact-1024 audit demonstrated that perceptual
|
||||
siblings can cross an image-level split despite different file and decoded-pixel
|
||||
hashes, so the old 2048 train/validation/test split cannot be retrospectively
|
||||
cleared of that leakage mode. Its external control challenges remain valid;
|
||||
its positive rates are conditional on the historical split. The next model
|
||||
calibration must preserve content-group membership across every partition.
|
||||
|
||||
An aligned-subtraction ablation strengthened the mechanism finding without
|
||||
clearing the oracle gate. At a discovery-selected amplitude, it reversed both
|
||||
the fixed-tile and independently fitted phase decisions on all 30 test images
|
||||
at a median 53.74 dB PSNR and 0.99681 SSIM. The aligned edit reduced both scores
|
||||
more than cyclic-shifted and orthogonal random tile controls on every paired
|
||||
image. A shifted tile nevertheless suppressed the phase score enough to leave
|
||||
only one accepted image, so these local reversals remain surrogate evidence,
|
||||
not verified SynthID removal. The exact controls and caveats are recorded in
|
||||
the linked experiment section.
|
||||
|
||||
A later adaptive projection reduced that local cost substantially. It repeats
|
||||
the frozen unit-norm 16x16 carrier over the decoded image and uses a bounded
|
||||
scalar search for the smallest subtraction amplitude that reaches a requested
|
||||
fixed-tile score. There is no generative model, spatial optimizer, or change of
|
||||
image dimensions. Among the 30 historical 2048x2048 ablation sources, 27 were
|
||||
positive under both the fixed-tile and independent phase rules before editing.
|
||||
Searching only the scalar amplitude cleared both rules on all 27 at a median
|
||||
59.83 dB PSNR and 0.99890 SSIM; the minima were 55.19 dB and 0.99705. Applying
|
||||
the same selected amplitudes to a one-pixel cyclic shift of the carrier cleared
|
||||
none of the 27. Of the 16 sources whose carrier survived an otherwise identical
|
||||
JPEG-95 round trip, aligned projection cleared all 16 after JPEG-95 while the
|
||||
shifted control cleared one.
|
||||
|
||||
The same native-period edit works without resizing when dimensions are not
|
||||
multiples of 16: repeat the carrier past both image boundaries and crop it to
|
||||
the decoded geometry. A deterministic development challenge selected three
|
||||
first-carrier-positive Spaces images from each of the 20 most frequent native
|
||||
geometries, for 60 images total. Searching to a zero fixed-tile target cleared
|
||||
60 of 60, compared with 5 of 60 norm-matched one-pixel-shifted controls. Median
|
||||
fidelity was 60.37 dB PSNR and 0.99919 SSIM; the minima were 53.56 dB and
|
||||
0.99581. A more conservative target score of -0.25 retained 60 of 60 native
|
||||
local clearances at median 57.70 dB PSNR and 0.99863 SSIM, with minima of
|
||||
52.38 dB and 0.99443.
|
||||
|
||||
The frozen -0.25 rule was then evaluated once on one deterministic first-carrier
|
||||
positive from every native geometry represented by that Spaces subset: 647
|
||||
images at 647 distinct decoded sizes. It reached the local target on all 647,
|
||||
with no maximum-amplitude failure. Median fidelity was 57.67 dB PSNR and
|
||||
0.99867 SSIM; the worst case was 51.78 dB and 0.99400. Among the 323 images
|
||||
whose source carrier survived JPEG-95, the aligned candidate cleared 323 of
|
||||
323 after the same codec round trip, compared with 58 of 323 one-pixel-shifted
|
||||
controls. At the deeper transform margin, shifted controls also clear more
|
||||
often natively, 153 of 647, so exact carrier phase remains causal but the edit
|
||||
is no longer phase-exclusive on every content. The all-geometry corpus was
|
||||
already inspected during detector development and contributes mechanism
|
||||
coverage, not an independent generalization split.
|
||||
|
||||
The conservative target also survived every tested transform for which the
|
||||
same transformed source remained locally positive: 32 of 32 JPEG-95, 13 of 13
|
||||
JPEG-90, 8 of 8 JPEG-85, 15 of 15 WebP-95, and 54 of 54 0.75x resize round
|
||||
trips. The shifted controls cleared 7, 1, 1, 5, and 17 of those respective
|
||||
source-positive subsets. These results establish fast, geometry-independent
|
||||
control over the recovered carrier and quantify the quality-versus-transport
|
||||
margin. They do not establish SynthID removal. The previous provider check
|
||||
showed that Google can remain positive after a locally favorable analytical
|
||||
edit, and this adaptive recipe has not received a matching-provider negative
|
||||
oracle verdict. It therefore remains a research candidate rather than a public
|
||||
removal path.
|
||||
|
||||
The reproducible local tool is
|
||||
`scripts/synthid_adaptive_carrier_suppress.py`. It accepts one locally positive
|
||||
image, writes a lossless PNG without overwriting existing files, and records
|
||||
the input/output hashes, target, selected amplitude, scores, fidelity, and
|
||||
runtime in JSON. Its default -0.25 target is the transform-margin setting above.
|
||||
The tool intentionally remains outside the public package CLI and refuses
|
||||
locally negative inputs; its output status names carrier suppression rather
|
||||
than provider-verified removal.
|
||||
|
||||
The immutable oracle-batch and result evaluator are implemented. On 2026-08-10,
|
||||
four new 2048x2048 Gemini images were generated after the rule was frozen and
|
||||
registered as a 20-request confirmatory batch. The first source group exhausted
|
||||
the account's verification quota after five requests. The untouched source
|
||||
returned Google C2PA Content Credentials without a separate SynthID verdict;
|
||||
the lossless re-encode, aligned subtraction, and cyclic-shifted control all
|
||||
still returned a Google AI signal. The orthogonal control was refused because
|
||||
the quota had been exceeded, and the remaining 15 requests were not submitted.
|
||||
This incomplete run is already negative evidence for the frozen pixel-only
|
||||
recipe: the aligned candidate cleared the phase detector but remained positive
|
||||
under the tile detector and the provider oracle. The local carrier expert
|
||||
therefore ships only as a positive-only, exact-geometry detector; it is not a
|
||||
universal SynthID detector or a remover.
|
||||
|
||||
A controlled study (June 2026, clean v0.8.6 with text/face protection OFF,
|
||||
native resolution on this repo's default SDXL pipeline) measured the minimum
|
||||
img2img strength that removes the SynthID pixel watermark, verified per image on
|
||||
@@ -186,14 +463,15 @@ Generated cleaned outputs are not committed; the table below is the durable
|
||||
record of the historical oracle verdicts. One third-party image from issue #14
|
||||
was oracle-verified but is not committed.
|
||||
|
||||
**Oracle validation order: start with OpenAI.** When validating removal across
|
||||
vendors, run the OpenAI arm first. `openai.com/verify` is more accessible than the
|
||||
Gemini app -- fewer per-check restrictions, so it gives the fastest signal and is
|
||||
the strongest candidate for automation (Playwright / Chrome MCP driving
|
||||
`openai.com/verify`); the Gemini "Verify with SynthID" flow is more manual. This is
|
||||
an ordering/throughput choice, not a substitution: each oracle only reads its own
|
||||
vendor's SynthID (`openai.com/verify` is OpenAI-scoped), so Google content still
|
||||
needs the Gemini app.
|
||||
**Historical oracle validation order: start with OpenAI.** The June study used
|
||||
the OpenAI web verifier first because it was more accessible than the Gemini app.
|
||||
OpenAI now documents a Content Provenance API, but its usage guidance explicitly
|
||||
rules out repeated queries for reverse engineering or watermark removal. New
|
||||
adaptive experiments require separate authorization and must follow the oracle
|
||||
boundary in the
|
||||
[`detector and removal research plan`](synthid-detector-removal-plan.md). This is
|
||||
not a cross-provider substitution: each oracle reads only its own vendor's
|
||||
SynthID, so Google content still needs the Gemini flow.
|
||||
|
||||
| Vendor | Images | Resolution(s) | Pipeline | Removed at |
|
||||
|--------|--------|---------------|----------|------------|
|
||||
@@ -265,11 +543,29 @@ notable because it is also the most powerful removal attack. The paper notes
|
||||
that diffusion regeneration "leaves a strong reconstruction signature from the
|
||||
diffusion prior."
|
||||
|
||||
A newer [MarkNull study](https://arxiv.org/abs/2608.10166), accepted at USENIX
|
||||
Security 2026, evaluates a no-box latent-space attack on 20 Imagen-3 images and
|
||||
checks the outputs with Gemini's SynthID verification flow. MarkNull, its
|
||||
amortized variant, and most advanced regeneration or adversarial baselines
|
||||
reached 100% attack success in that small evaluation; a simple VAE round trip
|
||||
reached 0%. The per-image method inverts the input with a clean public diffusion
|
||||
proxy, optimizes its latent to decorrelate the reconstructed initial noise, and
|
||||
constrains LPIPS, MSE, and SSIM before decoding it again.
|
||||
|
||||
This is important independent evidence that removal need not know SynthID's
|
||||
carrier or decoder, but it is still a generative latent reconstruction, not the
|
||||
pixel-only analytical path pursued here. The paper's milder SynthID setting
|
||||
reports PSNR 25.36 dB and SSIM 0.80 at 100% attack success. Those values do not
|
||||
meet this project's 40 dB median PSNR and 0.99 median SSIM release gate, despite
|
||||
the paper's favorable composite quality score. MarkNull-A's reported 0.50-second
|
||||
runtime and 6.3 GB VRAM are attractive as a future optional fallback, but they
|
||||
do not establish a fast, visually lossless pixel-only remover.
|
||||
|
||||
---
|
||||
|
||||
## 3. Detectability and verifier access
|
||||
|
||||
### 3.1 No public local detector
|
||||
### 3.1 No public payload decoder
|
||||
|
||||
The SynthID decoder is proprietary and not released:
|
||||
|
||||
@@ -278,18 +574,78 @@ The SynthID decoder is proprietary and not released:
|
||||
> available to trusted testers."
|
||||
> -- Gowal et al., arXiv:2510.09263
|
||||
|
||||
There is no public API, no released decoder weights, and no reproducible
|
||||
algorithm for local detection. The verification service (SynthID Detector) is:
|
||||
There are no released payload-decoder weights or public algorithm. Google
|
||||
provides verification in Gemini and a limited SynthID Detector
|
||||
portal. OpenAI now documents a synchronous Content Provenance API whose image
|
||||
response contains separate C2PA and SynthID outcomes. That API is a remote,
|
||||
OpenAI-scoped verifier, not a local decoder. Its documentation also says not to
|
||||
use repeated queries to reverse-engineer, remove, or evade a watermark, so an
|
||||
adaptive research loop requires separate authorization.
|
||||
|
||||
Google's SynthID Detector service is:
|
||||
|
||||
> "a verification portal" in early testing with "journalists and media
|
||||
> professionals" on a waitlist
|
||||
> -- deepmind.google/models/synthid/
|
||||
|
||||
The external variant SynthID-O is available "through partnerships" only. Our
|
||||
tool cannot locally detect SynthID presence or absence -- this is by design,
|
||||
not a gap we can fill.
|
||||
The external variant SynthID-O is available "through partnerships" only. This
|
||||
package does not ship a local payload decoder. Research on a periodic lattice
|
||||
expert is in [synthid-detector-research.md](synthid-detector-research.md).
|
||||
A 2026-08-23 literature map (Gowal, AWPD/FSNet, PRC, reverse-SynthID) is
|
||||
in that page's external-literature section. The product routes are signed
|
||||
provenance and `verify-openai-synthid`.
|
||||
|
||||
### 3.2 How our tool recognizes SynthID from provenance
|
||||
### 3.2 How our tool detects the supported carrier
|
||||
|
||||
This heading is kept because older README and CLI links pointed here. The
|
||||
package does not ship a local SynthID pixel detector. The periodic-lattice
|
||||
expert is research-only under `scripts/synthid_runtime/` and
|
||||
[synthid-detector-research.md](synthid-detector-research.md). It is not a
|
||||
payload decoder and is not called from `identify` or the CLI.
|
||||
|
||||
The product routes for SynthID are signed provenance, documented in the next
|
||||
sections, and `verify-openai-synthid`.
|
||||
|
||||
### 3.3 Official OpenAI pixel verification
|
||||
|
||||
`remove-ai-watermarks verify-openai-synthid image.png
|
||||
--acknowledge-upload` uses OpenAI's official Content Provenance API for the
|
||||
OpenAI half of the production detector. This is not the weak local period-8
|
||||
research expert and it does not use C2PA as a proxy.
|
||||
|
||||
Before making one request, the command strips AI provenance metadata into a
|
||||
temporary PNG, JPEG, or WebP file and compares hashes of the decoded RGBA raster
|
||||
before and after. It refuses the upload if any AI marker survived, if the format
|
||||
or pixels changed, or if the sanitized file exceeds 50 MiB. It then reads
|
||||
exactly one independent `type == "synthid"` response entry and ignores the C2PA
|
||||
entry. The source is not modified. Tests deliberately cover C2PA-only positive
|
||||
responses, pixel mutation, surviving metadata, malformed response shapes, and
|
||||
documented access and rate-limit failures.
|
||||
|
||||
The default SDK client fixes a 120-second request timeout and disables
|
||||
automatic retries. One explicit acknowledgement therefore cannot silently
|
||||
transmit the sanitized raster more than once. Timeout and connection failures
|
||||
are errors, and request logs omit source paths and decoded-pixel hashes.
|
||||
|
||||
A live 2026-08-14 web-verifier smoke used the same sanitization invariant. Two
|
||||
metadata-stripped, pixel-identical OpenAI images at 1536 by 1024 and 1024 by
|
||||
1536 both returned `SynthID detected` with `Content Credentials not detected`.
|
||||
An oracle-positive Google SynthID image and a COCO photograph, sanitized through
|
||||
the same path, both returned OpenAI `SynthID not detected` and no Content
|
||||
Credentials. This directly validates pixel-only and provider-specific behavior
|
||||
for the production design. Four files are only a functional smoke test, not a
|
||||
false-positive calibration, and the credentialed SDK endpoint itself was not
|
||||
called because no API key was available.
|
||||
|
||||
This backend is deliberately excluded from `identify`: verification uploads
|
||||
the sanitized raster to OpenAI, the endpoint is not eligible for Zero Data
|
||||
Retention, and explicit acknowledgement is required. It is suitable for
|
||||
individual supported OpenAI provenance checks, not adaptive detector fitting or
|
||||
removal search. The API documentation prohibits repeated queries for watermark
|
||||
reverse engineering or evasion. As with every detector in this project,
|
||||
`not_detected` is absence of recognized evidence, not proof of human authorship.
|
||||
|
||||
### 3.4 How our tool recognizes SynthID from provenance
|
||||
|
||||
We recognize SynthID indirectly from supported C2PA evidence; this is not a
|
||||
pixel watermark decode. Google states that all media generated by its tools is
|
||||
@@ -310,28 +666,26 @@ This is why:
|
||||
- A quiet `identify` output is not proof that SynthID was removed -- it only
|
||||
means the metadata signal is gone.
|
||||
|
||||
### 3.3 Oracle scope: each vendor detects only their own
|
||||
### 3.5 Oracle scope: each vendor detects only their own
|
||||
|
||||
From openai.com/research/verify (verbatim, verified 2026-05-31):
|
||||
|
||||
> "OpenAI generation signals will only be detected if the image was generated
|
||||
> with our tools."
|
||||
> "Content could also still be AI-generated by another company's model, which
|
||||
> the tool currently does not detect."
|
||||
OpenAI's current Content Provenance API documentation says it checks supported
|
||||
OpenAI signals and is not a general-purpose AI detector. Google's current Gemini
|
||||
documentation likewise says Gemini recognizes SynthID from Google AI tools,
|
||||
not other companies' payloads.
|
||||
|
||||
SynthID technology is used by multiple vendors, but each verifier is keyed to
|
||||
its own payload:
|
||||
|
||||
| Oracle | Detects | Does NOT detect |
|
||||
|-------------------------------|------------------|-------------------------|
|
||||
| Gemini app "Verify with SynthID" | Google SynthID | OpenAI SynthID |
|
||||
| openai.com/research/verify | OpenAI SynthID | Google SynthID |
|
||||
| Oracle | Detects | Does not detect |
|
||||
| --- | --- | --- |
|
||||
| Gemini app verification | Google SynthID | OpenAI SynthID |
|
||||
| OpenAI Content Provenance API or web verifier | Supported OpenAI SynthID | Google SynthID |
|
||||
|
||||
A Google-SynthID image reads clean on openai.com/verify. An OpenAI image reads
|
||||
clean in the Gemini oracle. They are different payloads within the same
|
||||
framework.
|
||||
|
||||
### 3.4 Video verification and attack harness
|
||||
### 3.6 Video verification and attack harness
|
||||
|
||||
Gemini's built-in verification flow reports whether and where it detects Google
|
||||
SynthID in a video. This remains a proprietary oracle: invoke `@synthid`, use
|
||||
@@ -461,9 +815,9 @@ conditioning, never by copying original pixels.**
|
||||
|
||||
> **Superseded in 0.24.0.** The `controlnet`, `sdxl`, `qwen` and `default` profiles
|
||||
> were removed, and a retired name is now rejected at parse time rather than routed
|
||||
> onward. `--pipeline` accepts only `qwen-zimage` (the default) and `sdxl-zimage`;
|
||||
> both are CUDA-only, both condition their global stage on a canny edge map, and both
|
||||
> run the same Z-Image face stage. The bullets below are a record of what was measured
|
||||
> onward. Current `--pipeline` values are `qwen-zimage` (the default),
|
||||
> `sdxl-zimage`, `chroma-zimage`, and `auto`; all are CUDA-only and share the
|
||||
> Z-Image face stage. The bullets below are a record of what was measured
|
||||
> on the removed profiles, kept because the oracle verdicts are still the evidence the
|
||||
> current design rests on.
|
||||
|
||||
@@ -560,7 +914,8 @@ photoreal, `sdxl` on flat graphics, the §5.1 content-x-pipeline table), BUT on
|
||||
hard case (flat fills) `sdxl` is the WEAKER remover (plain img2img barely perturbs a
|
||||
flat region at low strength), so it needs AT LEAST controlnet's strength -- the
|
||||
certified floor is therefore the right floor for `sdxl` too. This is a MARGIN argument
|
||||
for `sdxl`, not a separate certification (no local SynthID detector to self-verify).
|
||||
for `sdxl`, not a separate certification (the tested geometries are outside the
|
||||
current local detector's scope).
|
||||
The higher strength costs little quality where it matters, because `controlnet` is now
|
||||
the default pipeline, so `sdxl` is reached only via an explicit `--pipeline sdxl` (a
|
||||
deliberate opt-down), where over-regeneration has no faces/text to damage.
|
||||
@@ -620,8 +975,10 @@ Two constraints on reading this:
|
||||
subjects.** JPEG re-encoding removes C2PA metadata but does NOT remove the
|
||||
SynthID pixel watermark (verified June 2026 on issue #14 pic3). Do not
|
||||
dismiss these as "not faithful originals" for SynthID-removal tests.
|
||||
- **The correct oracle for OpenAI images is openai.com/verify**, not the Gemini
|
||||
app. The two oracles detect different payloads.
|
||||
- **The correct oracle for OpenAI images is an authorized OpenAI provenance
|
||||
verifier**, not the Gemini app. OpenAI now documents both a web tool and a
|
||||
Content Provenance API; the API's published use restrictions still apply.
|
||||
The OpenAI and Google oracles detect different payloads.
|
||||
- **A quiet `identify` output after processing is not proof of removal.** It
|
||||
means the provenance evidence is gone. The pixel watermark state is unknown without
|
||||
an oracle check.
|
||||
@@ -669,7 +1026,7 @@ random (unset) seed differed between runs. So **0.15 is the borderline floor for
|
||||
controlnet photoreal, not a robust guarantee**: at the threshold the same
|
||||
image+settings can pass or fail run-to-run. img2img runs with `seed=None` (random)
|
||||
unless `--seed` is passed, so a removal SERVICE gets a coin-flip near threshold and
|
||||
has no local SynthID detector to self-verify.
|
||||
has no applicable local SynthID detector at these geometries to self-verify.
|
||||
|
||||
**Controlnet strength ladder on the two photoreal images (oracle, `--auto`,
|
||||
`--max-resolution 1536`):**
|
||||
@@ -684,7 +1041,7 @@ has no local SynthID detector to self-verify.
|
||||
non-deterministic borderline); both photoreal survivors cleared at 0.20. Honest
|
||||
caveat: 0.20 is one confirming run WITH margin, not an N-run repeatability proof --
|
||||
for a removal service, add a little more margin or validate repeatability, since
|
||||
there is no local SynthID detector to self-check. **Implications:** (1) the
|
||||
these geometries are outside the current local detector's scope. **Implications:** (1) the
|
||||
content×pipeline table above conflates a borderline/non-deterministic 0.15 result
|
||||
with deterministic content behavior -- the photoreal-survives-controlnet effect is
|
||||
solid at 0.10 but at 0.15 it is near-threshold noise; (2) for reliable removal pick
|
||||
@@ -735,7 +1092,7 @@ a resolution trend here; measure it.
|
||||
**Current implication:** the old floor table remains evidence about the dated
|
||||
test set, not the current resolver. The SDXL and ControlNet profiles it measured
|
||||
no longer exist; the shipped defaults are defined in `watermark_profiles.py`, and
|
||||
both surviving profiles run face repair as a built-in second stage rather than as
|
||||
all three surviving profiles run face repair as a built-in second stage rather than as
|
||||
an optional restore. Removal near a threshold remains seed dependent, so
|
||||
reproducible verification requires a fixed seed.
|
||||
|
||||
@@ -756,8 +1113,8 @@ reproducible verification requires a fixed seed.
|
||||
Forensic Stealth in Generative-AI Watermark Removal.** arXiv:2605.09203.
|
||||
https://arxiv.org/abs/2605.09203
|
||||
|
||||
5. OpenAI. **Verify tool for AI-generated images.** openai.com/research/verify.
|
||||
Accessed 2026-05-31.
|
||||
5. OpenAI. **Content provenance.**
|
||||
https://developers.openai.com/api/docs/guides/content-provenance
|
||||
|
||||
6. Google. **Verify AI-generated images, videos, and audio.**
|
||||
https://support.google.com/gemini/answer/16722517
|
||||
@@ -770,6 +1127,99 @@ reproducible verification requires a fixed seed.
|
||||
Watermarking.** arXiv:2505.21620.
|
||||
https://arxiv.org/abs/2505.21620
|
||||
|
||||
9. OpenAI. **ChatGPT Images 2.0 system card.**
|
||||
https://deploymentsafety.openai.com/chatgpt-images-2-0/automated-evaluations-and-adversarial-testing
|
||||
|
||||
10. Cao et al. (2026). **MarkNull: Model-Agnostic Watermark Removal in
|
||||
AI-Generated Images via On-Manifold Latent Manipulation.** USENIX Security
|
||||
2026, arXiv:2608.10166. https://arxiv.org/abs/2608.10166
|
||||
|
||||
11. Ao, Du, Wang, Chen, Lu (2026). **AWPD: Frequency Shield Network for
|
||||
Agnostic Watermark Presence Detection.** arXiv:2603.06723.
|
||||
https://arxiv.org/abs/2603.06723
|
||||
|
||||
12. Gunn, Zhao, Song (2025). **An Undetectable Watermark for Generative
|
||||
Image Models.** ICLR 2025, arXiv:2410.07369.
|
||||
https://arxiv.org/abs/2410.07369
|
||||
|
||||
13. Francati, Goonatilake, Pawar, Venturi, Ateniese (2026). **The Coding
|
||||
Limits of Robust Watermarking for Generative Models.** IEEE EuroS&P 2026,
|
||||
arXiv:2509.10577. https://arxiv.org/abs/2509.10577
|
||||
|
||||
14. Kassis and Hengartner (2025). **UnMarker: A Universal Attack on
|
||||
Defensive Image Watermarking.** IEEE S&P 2025, arXiv:2405.08363.
|
||||
https://arxiv.org/abs/2405.08363
|
||||
|
||||
15. Liu et al. (2025). **Image Watermarks are Removable Using Controllable
|
||||
Regeneration from Clean Noise.** ICLR 2025, arXiv:2410.05470.
|
||||
https://arxiv.org/abs/2410.05470
|
||||
|
||||
16. An et al. (2024). **WAVES: Benchmarking the Robustness of Image
|
||||
Watermarks.** ICML 2024, arXiv:2401.08573.
|
||||
https://arxiv.org/abs/2401.08573
|
||||
|
||||
17. Bui, Agarwal, Collomosse (2023). **TrustMark: Universal Watermarking
|
||||
for Arbitrary Resolution Images.** arXiv:2311.18297.
|
||||
https://arxiv.org/abs/2311.18297
|
||||
|
||||
18. Dathathri et al. (2024). **Scalable watermarking for identifying large
|
||||
language model outputs.** Nature 634, 818-823.
|
||||
https://www.nature.com/articles/s41586-024-08025-4
|
||||
|
||||
19. DeepMind. **US12094474B1** and continuation **US20250149048A1**.
|
||||
https://patents.google.com/patent/US12094474B1/en
|
||||
|
||||
20. Weatherbed (2026-04-14). **Has Google's AI watermarking system been
|
||||
reverse-engineered?** The Verge.
|
||||
https://www.theverge.com/ai-artificial-intelligence/911579/google-synthid-ai-watermarking-system-reverse-engineered
|
||||
|
||||
21. Cox, Kilian, Leighton, Shamoon (1997). **Secure spread spectrum
|
||||
watermarking for multimedia.** IEEE Trans. Image Process. 6(12).
|
||||
|
||||
22. Wen, Kirchenbauer, Geiping, Goldstein (2023). **Tree-Ring Watermarks.**
|
||||
NeurIPS 2023, arXiv:2305.20030. https://arxiv.org/abs/2305.20030
|
||||
|
||||
23. Ojha, Li, Lee (2023). **Towards Universal Fake Image Detectors that
|
||||
Generalize Across Generative Models.** CVPR 2023, arXiv:2302.10174.
|
||||
https://arxiv.org/abs/2302.10174
|
||||
|
||||
24. Corvi, Cozzolino, Poggi, Nagano, Verdoliva (2023). **Intriguing
|
||||
properties of synthetic images.** CVPRW 2023, arXiv:2304.06408.
|
||||
https://arxiv.org/abs/2304.06408
|
||||
|
||||
25. Zhong, Xu, Zou (2026). **Color Matters: Demosaicing-Guided Color
|
||||
Correlation Training.** arXiv:2601.22778.
|
||||
https://arxiv.org/abs/2601.22778
|
||||
|
||||
26. Baluja (2017). **Hiding Images in Plain Sight: Deep Steganography.**
|
||||
NeurIPS 2017.
|
||||
|
||||
27. Holub, Fridrich, Denemark (2014). **Universal distortion function for
|
||||
steganography in an arbitrary domain.** EURASIP J. Information Security.
|
||||
|
||||
28. Jing et al. (2021). **HiNet: Deep Image Hiding by Invertible Network.**
|
||||
ICCV 2021.
|
||||
|
||||
29. Yang et al. (2024). **Gaussian Shading: Provable Performance-Lossless
|
||||
Image Watermarking for Diffusion Models.** CVPR 2024, arXiv:2404.04956.
|
||||
https://arxiv.org/abs/2404.04956
|
||||
|
||||
30. Farid (2009). **Exposing digital forgeries from JPEG ghosts.** IEEE
|
||||
Trans. Information Forensics and Security 4(1).
|
||||
|
||||
31. Wang et al. (2023). **DIRE for Diffusion-Generated Image Detection.**
|
||||
ICCV 2023, arXiv:2303.09295. https://arxiv.org/abs/2303.09295
|
||||
|
||||
32. Wang, Wang, Zhang, Owens, Efros (2020). **CNN-generated images are
|
||||
surprisingly easy to spot... for now.** CVPR 2020, arXiv:1912.11035.
|
||||
https://arxiv.org/abs/1912.11035
|
||||
|
||||
33. Ó Ruanaidh and Pun (1998). **Rotation, scale and translation invariant
|
||||
spread spectrum digital image watermarking.** Signal Processing 66(3).
|
||||
|
||||
34. Réfrégier and Javidi (1995). **Optical image encryption based on input
|
||||
plane and Fourier plane random encoding.** Optics Letters 20(7).
|
||||
|
||||
**Google floor on qwen-zimage (0.27.2, remeasured 2026-08-25).** The bracket above did not
|
||||
survive the full production path: on the 4.33 MP CJK-sign fixture (visible-stage
|
||||
sparkle removal -> qwen-zimage seed 0 at the curve's 0.154 top -> resize-back ->
|
||||
@@ -779,5 +1229,6 @@ initially took a flat `0.30` floor instead of the area curve. The later provider
|
||||
remeasurement established a common pass at `0.25` across three valid sources and
|
||||
separately repeated `QWEN_ZIMAGE_GOOGLE_STRENGTH = 0.27` across those sources and
|
||||
three accounts. OpenAI now has its own measured flat operating point; unknown
|
||||
content keeps the resolution curve. An explicit strength still wins. Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-18/` (the failing re-baseline).
|
||||
content keeps the resolution curve. An explicit strength still wins.
|
||||
Certification artifacts: private archive `text-restoration-2026-08-18` (the
|
||||
failing re-baseline).
|
||||
@@ -367,5 +367,5 @@ against base MAEs of 24.7/16.8/10.2. Since 0.27.1 the anchor is off by default
|
||||
(`fidelity_anchor` parameter on `remove_watermark` / `InvisibleOptions` /
|
||||
`--fidelity-anchor`); Google rows were not re-queried because the default
|
||||
strength leaves that fixture's base detected (the vendor-strength lane tracks
|
||||
that separately). Certification artifacts: raiw-app
|
||||
`data/certification/text-restoration-2026-08-19-A1/` (sha256-stamped rows).
|
||||
that separately). Certification artifacts: private archive
|
||||
`text-restoration-2026-08-19-A1` (sha256-stamped rows).
|
||||
@@ -184,7 +184,7 @@ Three harness rules are load-bearing and must not be relaxed: score a mark only
|
||||
crop's adjudication scope; take provenance from metadata, never from labels; and never
|
||||
report recall from the detector-sampled set.
|
||||
|
||||
## Tier D -- external oracles (manual, not automatable here)
|
||||
## Tier D -- external oracles
|
||||
|
||||
Proprietary watermark removal cannot be verified locally by design -- no public decoder
|
||||
exists. Each vendor has its own oracle and it covers only that vendor's content:
|
||||
@@ -195,6 +195,13 @@ needs a pixel-identical metadata-stripped control: the control must lose `C2PA`
|
||||
remaining `Watermark`-positive before a candidate's negative result can be attributed to
|
||||
pixel regeneration. A quiet metadata proxy is **not** proof the pixel watermark is gone.
|
||||
|
||||
OpenAI's API documentation says not to use repeated queries to reverse-engineer, remove,
|
||||
or evade a watermark. Using it as an adaptive research oracle therefore requires explicit
|
||||
authorization. Without that authorization it must not become a training loss, search loop,
|
||||
or automated removal gate. The provider-specific detector and pixel-only removal research
|
||||
protocol is in [`synthid-detector-removal-plan.md`](synthid-detector-removal-plan.md).
|
||||
|
||||
|
||||
Scope honestly: this tier certifies strength floors on a handful of images per vendor, and
|
||||
that is all it can do. See `docs/synthid.md`.
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
# on it, including the ComfyUI node package. The console script below carries
|
||||
# the same weight, since users have it on PATH.
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.34.0"
|
||||
version = "0.37.1"
|
||||
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11,<3.15"
|
||||
@@ -118,6 +118,9 @@ diffusion = [
|
||||
qwen-zimage = [
|
||||
"remove-ai-watermarks[diffusion]",
|
||||
"diffsynth>=2.0.17,<3",
|
||||
# Chroma1-HD ships a binary spiece.model. Without the matching backend,
|
||||
# Transformers falls through to tiktoken, which cannot parse that file.
|
||||
"sentencepiece>=0.2.2,<0.3",
|
||||
"torchvision>=0.20.0",
|
||||
]
|
||||
# Opt-in verified-text reconstruction over qwen-zimage. LaMa removes the changed
|
||||
@@ -142,6 +145,20 @@ trustmark = [
|
||||
# wheels. Keep this feature installable only on its real runtime range.
|
||||
"trustmark>=0.8.0; python_version < '3.13'",
|
||||
]
|
||||
# Metadata-free photo classifier (Model 1 then gated Model 2). CPU torch is
|
||||
# enough. identify never imports this extra. Weights download on first use.
|
||||
classify = [
|
||||
"remove-ai-watermarks[pixels]",
|
||||
"torch>=2.0.0",
|
||||
"transformers>=5,<6",
|
||||
"tokenizers>=0.22,<0.23",
|
||||
"huggingface-hub>=0.20.0",
|
||||
]
|
||||
# Official remote OpenAI SynthID verification. The command strips AI provenance
|
||||
# metadata and proves pixel identity before upload; it is never called implicitly.
|
||||
verify = [
|
||||
"openai>=2.52.0",
|
||||
]
|
||||
# Universal region eraser backend -- big-LaMa via onnxruntime (Carve/LaMa-ONNX,
|
||||
# Apache-2.0). CPU, no torch. Model (~200 MB) is downloaded on first use and
|
||||
# cached by huggingface_hub; it is never bundled in this repo. The default cv2
|
||||
@@ -174,7 +191,7 @@ dev = [
|
||||
]
|
||||
# ``qwen-zimage`` already pulls ``diffusion``; naming both would suggest diffusion is
|
||||
# independently sufficient for a removal, which it is not.
|
||||
all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan]"]
|
||||
all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan,classify,verify]"]
|
||||
|
||||
[project.scripts]
|
||||
remove-ai-watermarks = "remove_ai_watermarks.cli:main"
|
||||
@@ -207,13 +224,13 @@ exclude = [
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
pythonpath = ["src"]
|
||||
pythonpath = ["src", "scripts"]
|
||||
addopts = "-v --tb=short"
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py311"
|
||||
line-length = 120
|
||||
exclude = ["_refs"]
|
||||
exclude = ["_refs", ".local-eval"]
|
||||
extend-exclude = ["*.md"]
|
||||
|
||||
[tool.ruff.lint]
|
||||
@@ -224,7 +241,7 @@ ignore = [
|
||||
]
|
||||
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
"scripts/*.py" = ["G004", "S108", "S310", "T20"]
|
||||
"scripts/**/*.py" = ["G004", "S108", "S310", "T20"]
|
||||
"tests/*.py" = ["ANN", "S101", "S105", "S106", "S108"]
|
||||
"src/remove_ai_watermarks/_internal/watermark_remover.py" = ["S603", "S606", "S607"] # nvidia-smi capability probe
|
||||
"src/remove_ai_watermarks/_internal/c2pa.py" = ["S110"] # try-except-pass for corrupt file handling
|
||||
@@ -236,7 +253,7 @@ indent-style = "space"
|
||||
[tool.pyright]
|
||||
pythonVersion = "3.11"
|
||||
typeCheckingMode = "strict"
|
||||
exclude = ["_refs"]
|
||||
exclude = ["_refs", ".local-eval"]
|
||||
|
||||
[[tool.pyright.executionEnvironments]]
|
||||
root = "tests"
|
||||
|
||||
@@ -43,13 +43,19 @@ explicitly names a tracked canonical result.
|
||||
| `visible_positives.py` | List corpus images carrying a registered visible mark. |
|
||||
| `visible_recall_sample.py` | Build an unbiased blinded sample for recall measurement. |
|
||||
| `visible_sheets.py` | Build blinded contact sheets for relaxation candidates. |
|
||||
| `retrain_photo_classify.py` | CPU-retrain the 2026-08-31 photo heads from a sha256-keyed cache pack, no images. |
|
||||
| `publish_photo_classify_hf.py` | Upload the photo-classify card and freeze weights to Hugging Face `wiltodelta/raiw-photo-classify`. Manual; the Action `publish-photo-classify-hf.yml` is the write-token path. |
|
||||
| `verify_engine_selection_fixtures.py` | Verify hashes, dimensions, and prompt pairing in the tracked auto-engine content matrix. |
|
||||
|
||||
## Research and diagnostic prototypes
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `chroma_text_restoration_study.py` | Compare verified-text restoration for Qwen and Chroma on H100 without running both engines in production. |
|
||||
| `cjk_tail_probe.py` | Test a generic template for otherwise uncovered CJK labels. |
|
||||
| `controlnet_sweep.py` | Sweep the historical ControlNet removal prototype. |
|
||||
| `engine_selection_study.py` | Run one global engine at a time over the tracked prompt-matched content matrix. |
|
||||
| `analyze_engine_selection_study.py` | Reduce engine-study output to paired deltas and exact sign tests. |
|
||||
| `infer_text_lines.py` | Draft stable source-text lines without modifying pixels. |
|
||||
| `qwen_scrub_prototype.py` | Probe low-strength Qwen regeneration on a GPU. |
|
||||
| `selective_text_restoration.py` | Evaluate text restoration over a scrubbed image. |
|
||||
|
||||