Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
09a4b20ac6 | ||
|
|
9bb4e9cd54 | ||
|
|
5052dbe63c | ||
|
|
2fe4d3c3a0 | ||
|
|
adbc6a3046 | ||
|
|
326840bab4 | ||
|
|
29de21256b | ||
|
|
07b07634cf | ||
|
|
54a19059d5 | ||
|
|
5e85f3ac6f | ||
|
|
36c36212f0 | ||
|
|
84ea8ede2e | ||
|
|
8f30f8bdb3 | ||
|
|
925401c905 | ||
|
|
55e7c53699 | ||
|
|
2ff06d4c5a | ||
|
|
1410a8e064 | ||
|
|
11ca7d6054 | ||
|
|
5991894ca5 | ||
|
|
5fa14f9e26 | ||
|
|
f7e0a18e6c | ||
|
|
bc5922451f | ||
|
|
798c94e6ae | ||
|
|
67d948f57d | ||
|
|
1d0e6b54dd | ||
|
|
fa4748a6a8 | ||
|
|
1e9572e2d4 | ||
|
|
1a6f50db75 | ||
|
|
b684423fc4 | ||
|
|
16bc9879e9 | ||
|
|
343da7f87d | ||
|
|
6f2d50cde8 | ||
|
|
95f467b4ad | ||
|
|
34505e5cc2 | ||
|
|
01539f5f58 | ||
|
|
23696f42a8 | ||
|
|
08a4a8d299 | ||
|
|
86ce55ced6 | ||
|
|
ee561308ef | ||
|
|
a8d4bce14a | ||
|
|
17408b958e | ||
|
|
834342db07 | ||
|
|
dcc1bf5e07 | ||
|
|
d8fcd0f79b | ||
|
|
0a3b227f2c | ||
|
|
f91f9fd9fc | ||
|
|
c4aecfbbdf | ||
|
|
ab528ec0e8 | ||
|
|
a1811b6221 | ||
|
|
e09104e39f | ||
|
|
29f17105de | ||
|
|
2810b53585 | ||
|
|
ba5710836b | ||
|
|
c927560614 | ||
|
|
f83da5aee4 | ||
|
|
251ba9ac79 | ||
|
|
c5764661b4 | ||
|
|
97cba11efe | ||
|
|
809d980fb7 | ||
|
|
10da166f3e | ||
|
|
8871eb3f67 | ||
|
|
44ec203c22 | ||
|
|
4beb61463a | ||
|
|
0d11e4099c | ||
|
|
fa83c1444f | ||
|
|
f52e40ae24 | ||
|
|
0aa03c4e89 | ||
|
|
91aa49b03a | ||
|
|
ee865e95e4 | ||
|
|
d84fd1a125 | ||
|
|
e315eacb14 | ||
|
|
3ba4ace7e0 | ||
|
|
13888704f8 | ||
|
|
e938b57f8c | ||
|
|
3cd1e47935 | ||
|
|
2298282b8b | ||
|
|
42d427731f | ||
|
|
50d9f44829 | ||
|
|
2e1428406e | ||
|
|
21ce1dfadd | ||
|
|
2369d351a9 | ||
|
|
432b63b6d7 | ||
|
|
8c00525946 | ||
|
|
f8003f505a | ||
|
|
2eab24a2e1 | ||
|
|
8201ada070 | ||
|
|
b7d44ff1a3 | ||
|
|
e8f65022cc | ||
|
|
4855586834 | ||
|
|
37789e02f6 | ||
|
|
2d03a00a39 | ||
|
|
acbf6afac3 | ||
|
|
25eaa7458c | ||
|
|
58146dbc6d | ||
|
|
b08899dc00 | ||
|
|
b1319b1d5a | ||
|
|
353bc5f12c | ||
|
|
370556ffe6 | ||
|
|
14a27d224d | ||
|
|
015df40f49 | ||
|
|
f9beef365f | ||
|
|
9b656513f1 | ||
|
|
ab03ecc507 |
No files matched your search
@@ -25,6 +25,21 @@ Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxat
|
||||
|
||||
From a worktree, `uv run` imports the package from the MAIN checkout -- that is where the editable install points. A script measuring a worktree's edit must insert that worktree's `src` at `sys.path[0]` and assert `module.__file__` resolves inside it, or it silently compares unmodified code against itself.
|
||||
|
||||
## Visible-mark example gallery
|
||||
|
||||
Every registered mark carries a committed example: `data/fixtures/visible/<key>/example.png`
|
||||
(image marks) and `example.mp4` (video marks), regenerated by
|
||||
`scripts/render_visible_examples.py`. `tests/test_visible_examples.py` is the canary: it
|
||||
fails when a mark is registered without an example and when an engine stops detecting its
|
||||
own example. The examples are deterministic synthetic composites of the committed
|
||||
silhouettes. When the generator fails after a geometry or gate change, fix the generator
|
||||
(and the engine) together; do not hand-edit the binaries.
|
||||
|
||||
Visible-mark content must distinguish the registry key, the provider's canonical
|
||||
product name, and the literal text rendered in the mark whenever those differ. Verify
|
||||
external provider names against the provider's official source rather than inferring
|
||||
them from an asset filename or OCR text.
|
||||
|
||||
## Model-adjacent tests
|
||||
|
||||
Do not classify an entire module as untestable because its main path downloads a model. Keep pure behavior covered without downloads, including:
|
||||
@@ -39,6 +54,9 @@ Do not classify an entire module as untestable because its main path downloads a
|
||||
stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a
|
||||
face. When one profile inherits another's stage, guard the invariants that stage
|
||||
relies on, not just the code path.
|
||||
- the photo-classify gate in `test_classify.py`: DEFINITELY is ridge AND MLP,
|
||||
POSSIBLY does not emit a provider, and `identify` does not import
|
||||
`remove_ai_watermarks.classify`.
|
||||
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
|
||||
mirror another, compare them field by field rather than pinning the values you happen
|
||||
to know about, so the next field added on one side and not the other fails at the
|
||||
@@ -65,6 +83,32 @@ that is what actually happened.
|
||||
|
||||
Use availability checks only for paths that actually load large models.
|
||||
|
||||
A verdict branch must be reachable with the dependency stack as shipped. High-confidence
|
||||
C2PA attribution required `signingCredential.trusted`, a code the reader emits only when a
|
||||
trust anchor list is loaded, and none ships -- so the branch was dead in production for
|
||||
every vendor while a hand-built info dict stamping that code kept it green from 0.27.0
|
||||
through 0.30.0. Assert reachability against the committed fixtures rather than a synthesized
|
||||
status set; `TestIdentifyRealSamples::test_no_committed_fixture_reports_a_trusted_signer`
|
||||
is that guard. Read a capability the stack does not have as a missing input, never as a
|
||||
negative finding.
|
||||
|
||||
## The docs are a seam, and they drift like one
|
||||
|
||||
`tests/test_docs_cover_the_public_surface.py` reads the live Click tree and the
|
||||
live `InvisibleOptions` dataclass and requires every knob to be NAMED in the
|
||||
user-facing pages. The code-to-code seams were already guarded and the
|
||||
code-to-docs seam was not, so it drifted exactly the way an unguarded seam does:
|
||||
`docs/cli.md` named none of `--adaptive-polish`, `--controlnet-scale`,
|
||||
`--detect/--no-detect`, `--humanize`, `--strength`, `--tile-size`,
|
||||
`--tile-overlap` or `--unsharp`, and `docs/python-api.md` named 9 of the 16
|
||||
option fields.
|
||||
|
||||
Group option spellings by the Click PARAMETER, never by string surgery on the
|
||||
names. A boolean pair can be spelled anything -- `--strip-metadata/--keep-metadata`,
|
||||
`--keep-standard/--remove-all` -- so deriving the partner from a `--no-` prefix
|
||||
silently splits those pairs and demands both halves. Either spelling documents
|
||||
the pair.
|
||||
|
||||
## One measurement, one gate seam
|
||||
|
||||
A detector is split into a trust-level-blind scan and a verdict that applies the
|
||||
@@ -79,6 +123,11 @@ rules follow, and both were broken in practice before they were written down:
|
||||
- Detection and the removal mask must read ONE sweep. The winning box travels on
|
||||
`TextMarkDetection.match_box` and the registry threads the detection into the mask
|
||||
builder; a mask path that re-runs its own sweep is how the two drift apart.
|
||||
- When the research lattice runtime in `scripts/synthid_runtime/` routes by image
|
||||
geometry, preserve the returned `SynthIDDetection.detector` in score manifests
|
||||
and downstream routers. Record an inactive expert as explicitly unsupported;
|
||||
never attribute a routed large-image score to the fixed expert. This expert is
|
||||
not part of the public package.
|
||||
|
||||
The C2PA manifest-store JSON is NOT stable across reads: the reader regenerates manifest
|
||||
URNs and instance ids. Compare the derived `c2pa_info`, never the raw store.
|
||||
@@ -98,6 +147,20 @@ record is byte-identical, and a green test suite does not establish that on its
|
||||
change that is meant to FIX detection is the exception that proves the rule: the diff
|
||||
must then be exactly the files you intended to change, named in advance.
|
||||
|
||||
When the baseline is a published release rather than the previous commit, get it with
|
||||
`uv run --isolated --no-project --with 'remove-ai-watermarks[heif]==<version>' python <script>`:
|
||||
it resolves that release from PyPI without touching the working tree or the editable
|
||||
install. Quote the extra -- bare `pkg[extra]` is a glob in zsh.
|
||||
|
||||
Two corollaries in `dwt_dct.py`, where "close enough" has an exact meaning. The bit test
|
||||
is `peak % 36 > 18.0` and for uint8 input the exact Haar value is a multiple of 0.5, so it
|
||||
lands ON the threshold once per 72 blocks and a 1-ulp difference flips real bits: leave the
|
||||
transform to `pywt` however slow it looks, because its C convolution contracts into an FMA
|
||||
that numpy has no ufunc for (`docs/module-internals.md` carries what that costs). And
|
||||
`tests/test_invisible_watermark.py` is `skipif(not is_available())` -- without the `detect`
|
||||
extra the upstream-parity test never runs, so a green suite is not evidence here and the
|
||||
verdict record is not optional.
|
||||
|
||||
## A certified operating point is data, not a constant
|
||||
|
||||
The video SynthID default is only meaningful as a row in
|
||||
|
||||
@@ -1,14 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"WebSearch",
|
||||
"WebFetch"
|
||||
]
|
||||
},
|
||||
"enabledPlugins": {
|
||||
"pyright-lsp@claude-plugins-official": true,
|
||||
"context7@claude-plugins-official": true,
|
||||
"code-simplifier@claude-plugins-official": true,
|
||||
"claude-md-management@claude-plugins-official": true
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,7 @@
|
||||
# HuggingFace token (optional; only needed for gated/private models)
|
||||
# Get yours at: https://huggingface.co/settings/tokens
|
||||
# HF_TOKEN=
|
||||
|
||||
# OpenAI API key (optional; only for explicit verify-openai-synthid uploads)
|
||||
# Create and manage keys at: https://platform.openai.com/api-keys
|
||||
# OPENAI_API_KEY=
|
||||
@@ -1,11 +1,10 @@
|
||||
name: Distribute on release
|
||||
|
||||
# Fans a published GitHub Release out to the channels that need a nudge.
|
||||
# PyPI is handled by publish.yml; the conda-forge channel is handled by its
|
||||
# autotick bot. This workflow updates the repository recipe and event-drives
|
||||
# the three channels that would otherwise be manual:
|
||||
# PyPI is handled by publish.yml. This workflow event-drives the three channels
|
||||
# that would otherwise be manual:
|
||||
# - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
|
||||
# - HF Space: factory-rebuild so it reinstalls the latest sdist from PyPI.
|
||||
# - HF Space: factory-rebuild the version already pinned in the Space repo.
|
||||
# - ComfyUI: sync, test, and publish the node against the exact library release.
|
||||
# All wait for the freshly published sdist to appear on PyPI first, since the
|
||||
# Release event fires in parallel with publish.yml's upload.
|
||||
@@ -89,34 +88,6 @@ jobs:
|
||||
git commit -m "remove-ai-watermarks $VERSION" || { echo "Formula already current"; exit 0; }
|
||||
git push
|
||||
|
||||
conda-recipe:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- name: Checkout main
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
ref: main
|
||||
- name: Bump the repository conda recipe
|
||||
env:
|
||||
SHA: ${{ needs.resolve.outputs.sha }}
|
||||
VERSION: ${{ needs.resolve.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
python scripts/sync_conda_recipe.py --version "$VERSION" --sha256 "$SHA"
|
||||
git diff --check
|
||||
if git diff --quiet -- packaging/conda/recipe.yaml; then
|
||||
echo "Conda recipe already current"
|
||||
exit 0
|
||||
fi
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git add packaging/conda/recipe.yaml
|
||||
git commit -m "Sync conda recipe with v$VERSION"
|
||||
git push
|
||||
|
||||
hf-space:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
name: Publish photo classifier to Hugging Face
|
||||
|
||||
# Manual only. Not part of a library release. The Space demo
|
||||
# (wiltodelta/remove-ai-watermarks) stays on distribute.yml / raiw-hf-space.
|
||||
#
|
||||
# Hub model: wiltodelta/raiw-photo-classify
|
||||
# Secret: HF_TOKEN, write-role Hugging Face token (same secret Space sync uses).
|
||||
#
|
||||
# Weights are not in git. Full mode downloads them from a GitHub Release
|
||||
# of this repository, then uploads with the card from docs/photo-classify-hf/.
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
confirm:
|
||||
description: "Type publish"
|
||||
required: true
|
||||
mode:
|
||||
description: "full = card plus weights from the release tag; card = README and operating-point only"
|
||||
type: choice
|
||||
options:
|
||||
- full
|
||||
- card
|
||||
default: full
|
||||
weights_tag:
|
||||
description: "GitHub Release tag that holds clip-l-ft.pt, probe, detector.pt, provider.pt"
|
||||
required: false
|
||||
default: photo-classify-freeze-2026-08-31
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
if: ${{ github.event.inputs.confirm == 'publish' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 180
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Download freeze weights from the GitHub Release
|
||||
if: ${{ github.event.inputs.mode == 'full' }}
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
TAG: ${{ github.event.inputs.weights_tag }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
mkdir -p weights
|
||||
gh release download "$TAG" --repo "${{ github.repository }}" --dir weights
|
||||
ls -l weights
|
||||
|
||||
- name: Publish to Hugging Face
|
||||
env:
|
||||
HF_TOKEN: ${{ secrets.HF_TOKEN }}
|
||||
MODE: ${{ github.event.inputs.mode }}
|
||||
SHA: ${{ github.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${HF_TOKEN:-}" ]; then
|
||||
echo "HF_TOKEN secret is not set" >&2
|
||||
exit 1
|
||||
fi
|
||||
extra=(--message "Photo-classify freeze (GitHub ${SHA:0:7})")
|
||||
if [ "$MODE" = "card" ]; then
|
||||
extra+=(--card-only)
|
||||
else
|
||||
extra+=(--src weights)
|
||||
fi
|
||||
uv run --no-project --with huggingface-hub python scripts/publish_photo_classify_hf.py "${extra[@]}"
|
||||
@@ -5,6 +5,9 @@ on:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: tests-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
@@ -28,13 +31,26 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Cross-OS x Python-floor/recent. The default-install surface only
|
||||
# needs the core deps plus dev tooling (no GPU extra): the model-running
|
||||
# paths are availability-gated and skip when torch/diffusers are absent,
|
||||
# so this matrix exercises metadata, identify, visible, and the cv2
|
||||
# eraser on every OS without pulling the heavy ML stack.
|
||||
os: [ubuntu-latest, macos-latest, windows-latest]
|
||||
python-version: ["3.10", "3.12"]
|
||||
# Exercise every supported minor. The floor and latest minor run on all
|
||||
# operating systems; intermediate minors run on Linux to bound the
|
||||
# matrix without leaving a declared Python version untested.
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.11"
|
||||
- os: macos-latest
|
||||
python-version: "3.11"
|
||||
- os: windows-latest
|
||||
python-version: "3.11"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.12"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.13"
|
||||
- os: ubuntu-latest
|
||||
python-version: "3.14"
|
||||
- os: macos-latest
|
||||
python-version: "3.14"
|
||||
- os: windows-latest
|
||||
python-version: "3.14"
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
|
||||
@@ -0,0 +1,247 @@
|
||||
name: Verify release surfaces
|
||||
|
||||
# Post-release verification: after distribute.yml fans a release out to
|
||||
# Homebrew, the HF Space, and the ComfyUI Registry, confirm every published
|
||||
# surface actually landed and works. This workflow is the automated twin of
|
||||
# the manual checklist in docs/release-and-distribution.md ("Release
|
||||
# verification"), not a replacement for the library's own test suite (which
|
||||
# already ran in test.yml and publish.yml).
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: "Version to verify (no leading v). Blank = latest on PyPI."
|
||||
required: false
|
||||
workflow_run:
|
||||
workflows: ["Distribute on release"]
|
||||
types: [completed]
|
||||
|
||||
# No step reads the GitHub token any more: the ComfyUI check moved to the
|
||||
# registry's public API. Keep the job read-only so it cannot write on a repo
|
||||
# whose default token permission is read/write.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
verify:
|
||||
# Only run on a successful distribute (not on manual dispatch, where the
|
||||
# version input or PyPI-latest provides the target).
|
||||
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 25
|
||||
env:
|
||||
# The Space id appears in the Hub API, the raw-file URL and the app host,
|
||||
# spelled three different ways. Name it once so a move cannot half-land.
|
||||
SPACE_REPO: wiltodelta/remove-ai-watermarks
|
||||
SPACE_URL: https://wiltodelta-remove-ai-watermarks.hf.space
|
||||
|
||||
steps:
|
||||
- name: Resolve target version
|
||||
id: v
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -n "${{ github.event.inputs.version }}" ]; then
|
||||
v="${{ github.event.inputs.version }}"
|
||||
else
|
||||
v=$(curl -fsSL https://pypi.org/pypi/remove-ai-watermarks/json \
|
||||
| python3 -c "import sys,json;print(json.load(sys.stdin)['info']['version'])")
|
||||
fi
|
||||
echo "version=$v" >> "$GITHUB_OUTPUT"
|
||||
echo "Verifying release $v"
|
||||
|
||||
# -- PyPI ---------------------------------------------------------------
|
||||
- name: Verify PyPI
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking PyPI for $VERSION..."
|
||||
rel=$(curl -fsSL "https://pypi.org/pypi/remove-ai-watermarks/$VERSION/json")
|
||||
echo "$rel" | python3 -c "
|
||||
import json, sys
|
||||
d = json.load(sys.stdin)
|
||||
assert d['info']['version'] == '$VERSION', f\"PyPI version mismatch: {d['info']['version']}\"
|
||||
assert d['urls'], 'No files on PyPI'
|
||||
has_wheel = any(u['packagetype'] == 'bdist_wheel' for u in d['urls'])
|
||||
has_sdist = any(u['packagetype'] == 'sdist' for u in d['urls'])
|
||||
assert has_wheel, 'No wheel on PyPI'
|
||||
assert has_sdist, 'No sdist on PyPI'
|
||||
print(f' PyPI: {len(d[\"urls\"])} files (wheel + sdist) for $VERSION')
|
||||
"
|
||||
|
||||
# -- Clean install from PyPI ---------------------------------------------
|
||||
- name: Install uv
|
||||
run: curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
|
||||
- name: Verify clean install
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Installing remove-ai-watermarks==$VERSION in a clean environment..."
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
result=$(uvx --from "remove-ai-watermarks==$VERSION" remove-ai-watermarks --version 2>&1) \
|
||||
|| { echo "::error::Clean install failed: $result"; exit 1; }
|
||||
echo " Clean install: $result"
|
||||
|
||||
# -- Homebrew --------------------------------------------------------------
|
||||
- name: Verify Homebrew formula
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking Homebrew tap formula..."
|
||||
formula=$(curl -fsSL \
|
||||
"https://raw.githubusercontent.com/wiltodelta/homebrew-tap/main/Formula/remove-ai-watermarks.rb")
|
||||
echo "$formula" | grep -q "remove_ai_watermarks-$VERSION.tar.gz" \
|
||||
|| { echo "::error::Homebrew formula does not reference $VERSION"; \
|
||||
echo "Formula URL line:"; echo "$formula" | grep '^url'; exit 1; }
|
||||
echo " Homebrew: formula points to $VERSION"
|
||||
|
||||
# -- ComfyUI ------------------------------------------------------------------
|
||||
- name: Verify ComfyUI registry node requires this release
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking the ComfyUI Registry node..."
|
||||
# Read the published versions directly. Do NOT proxy through the sync
|
||||
# workflow's run status: the node repo runs that workflow on a daily
|
||||
# schedule too, so a green nightly says nothing about THIS release.
|
||||
#
|
||||
# And read the versions list, not `latest_version` on the node object.
|
||||
# The registry reviews a new upload before activating it, and
|
||||
# `latest_version` reports only the active one -- so for the whole
|
||||
# review window a correctly published version looks identical to one
|
||||
# that was never uploaded. Those are opposite situations: one is a
|
||||
# queue nobody here controls, the other is a broken release.
|
||||
versions=$(curl -fsSL "https://api.comfy.org/nodes/remove-ai-watermarks/versions")
|
||||
status=$(printf '%s' "$versions" | VERSION="$VERSION" python3 -c "
|
||||
import json, os, sys
|
||||
version = os.environ['VERSION']
|
||||
floor = f'>={version}'
|
||||
matches = [
|
||||
v for v in json.load(sys.stdin)
|
||||
if any(floor in d for d in (v.get('dependencies') or []))
|
||||
]
|
||||
if not matches:
|
||||
sys.exit(f'::error::No ComfyUI node version requires {version}; the sync never published one')
|
||||
# Sort rather than trusting the response order: the endpoint happens to
|
||||
# return newest-first today and nothing documents that it must.
|
||||
newest = max(matches, key=lambda v: v.get('createdAt') or '')
|
||||
print(f\"{newest.get('version')} {newest.get('status')}\")
|
||||
")
|
||||
node_version=${status%% *}
|
||||
node_status=${status##* }
|
||||
echo " ComfyUI: node $node_version requires $VERSION (status $node_status)"
|
||||
case "$node_status" in
|
||||
NodeVersionStatusActive)
|
||||
echo "- ComfyUI: registry node $node_version requires $VERSION and is live" >> "$GITHUB_STEP_SUMMARY"
|
||||
;;
|
||||
NodeVersionStatusPending)
|
||||
# Published and waiting on the registry's own review. Say exactly
|
||||
# that in the summary rather than reporting it as live, and do not
|
||||
# fail: an external moderation queue going red on every release
|
||||
# trains everyone to ignore this workflow.
|
||||
echo "::notice::ComfyUI node $node_version is published and awaiting registry review."
|
||||
echo "- ComfyUI: registry node $node_version requires $VERSION, awaiting registry activation" >> "$GITHUB_STEP_SUMMARY"
|
||||
;;
|
||||
*)
|
||||
echo "::error::ComfyUI node $node_version is $node_status"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
# -- HF Space -------------------------------------------------------------------
|
||||
- name: Verify the HF Space serves this release
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking the HF Space pin..."
|
||||
# The Space pin bump is a MANUAL step (see docs/release-and-distribution.md):
|
||||
# distribute.yml only factory-rebuilds whatever is already pinned. Failing
|
||||
# here is the point -- a demo left on an older release is an incomplete
|
||||
# release, and only this check can say so.
|
||||
reqs=$(curl -fsSL "https://huggingface.co/spaces/$SPACE_REPO/raw/main/requirements.txt")
|
||||
pinned=$(printf '%s\n' "$reqs" | sed -n 's/^remove-ai-watermarks==\([0-9][0-9a-zA-Z.]*\).*/\1/p' | head -1)
|
||||
if [ -z "$pinned" ]; then
|
||||
echo "::error::Space requirements.txt has no remove-ai-watermarks== pin"
|
||||
exit 1
|
||||
fi
|
||||
if [ "$pinned" != "$VERSION" ]; then
|
||||
echo "::error::HF Space is pinned to $pinned, not $VERSION."
|
||||
echo " Bump remove-ai-watermarks in wiltodelta/raiw-hf-space (pyproject.toml,"
|
||||
echo " uv lock, re-exported requirements.txt) and push; sync-to-hf.yml mirrors it."
|
||||
exit 1
|
||||
fi
|
||||
echo " HF Space: requirements pin is $pinned"
|
||||
|
||||
- name: Verify the HF Space is running and answering
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# A factory rebuild leaves the Space BUILDING for a while; wait for the
|
||||
# runtime stage the API reports rather than for a bare HTTP 200, which a
|
||||
# build page also returns.
|
||||
for attempt in $(seq 1 20); do
|
||||
stage=$(curl -fsSL "https://huggingface.co/api/spaces/$SPACE_REPO" \
|
||||
| python3 -c "import json,sys;print((json.load(sys.stdin).get('runtime') or {}).get('stage',''))")
|
||||
echo " attempt $attempt: runtime stage=$stage"
|
||||
[ "$stage" = "RUNNING" ] && break
|
||||
case "$stage" in
|
||||
BUILD_ERROR|RUNTIME_ERROR|CONFIG_ERROR)
|
||||
echo "::error::HF Space runtime stage is $stage"; exit 1 ;;
|
||||
esac
|
||||
sleep 15
|
||||
done
|
||||
if [ "$stage" != "RUNNING" ]; then
|
||||
echo "::error::HF Space never reached RUNNING (last stage: $stage)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Alive is not the same as working. Drive the two CPU surfaces the demo
|
||||
# exists for, against the Space's own committed example, and require a
|
||||
# real verdict rather than any HTTP 200.
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
curl -fsSL -o /tmp/example.png \
|
||||
"https://huggingface.co/spaces/$SPACE_REPO/resolve/main/examples/doubao-aigc.png"
|
||||
uv run --no-project --with 'gradio_client>=1,<2' python - <<'PYEOF'
|
||||
import os
|
||||
import sys
|
||||
|
||||
from gradio_client import Client, handle_file
|
||||
|
||||
# The committed example is a 5 MB PNG, and httpx's default 5 s write
|
||||
# timeout aborts that upload on a slow link before the Space ever sees
|
||||
# it -- which would read as a broken demo rather than a slow one.
|
||||
client = Client(os.environ["SPACE_URL"], verbose=False, httpx_kwargs={"timeout": 300})
|
||||
|
||||
report = client.predict(path=handle_file("/tmp/example.png"), api_name="/run_identify")
|
||||
if "Verdict:" not in report:
|
||||
sys.exit(f"::error::/run_identify returned no verdict: {report[:200]!r}")
|
||||
print(f" HF Space identify: {report.splitlines()[0]}")
|
||||
|
||||
_image, _file, info = client.predict(
|
||||
path=handle_file("/tmp/example.png"),
|
||||
sensitivity_label="auto (recommended)",
|
||||
api_name="/run_visible",
|
||||
)
|
||||
if "Removed:" not in info:
|
||||
sys.exit(f"::error::/run_visible removed nothing on the Doubao example: {info!r}")
|
||||
print(f" HF Space visible: {info.splitlines()[0]}")
|
||||
PYEOF
|
||||
echo "- HF Space: pinned to $VERSION, RUNNING, identify + visible answered" >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Summary
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
# Every check above appends its own line to the step summary as it passes,
|
||||
# and every one of them fails the job instead of warning. So this heading
|
||||
# can only be reached when all of them actually held -- an unconditional
|
||||
# "verified" block printed here once claimed surfaces nobody had checked.
|
||||
sed -i "1i ## Release $VERSION verified" "$GITHUB_STEP_SUMMARY"
|
||||
cat "$GITHUB_STEP_SUMMARY"
|
||||
@@ -30,6 +30,10 @@ _refs/
|
||||
yolov8n.pt
|
||||
.coverage
|
||||
|
||||
# Local prototype and evaluation outputs (scripts/chroma_scrub_prototype.py and
|
||||
# friends); the durable record of an oracle verdict lives under data/evaluations/
|
||||
out/
|
||||
|
||||
# Claude Code local settings
|
||||
.claude/settings.local.json
|
||||
.claude/scheduled_tasks.lock
|
||||
@@ -48,3 +52,13 @@ gfpgan/
|
||||
|
||||
# Local evaluation data
|
||||
.local-eval/
|
||||
|
||||
# Other harnesses' generated config. Claude's sync owns these; a committed copy
|
||||
# is a second source nobody reconciles. Anchored so a same-named source file
|
||||
# deeper in the tree is not caught.
|
||||
/.agents/
|
||||
/.codex/
|
||||
/sch_*.yaml
|
||||
/.opencode/
|
||||
/opencode.json
|
||||
/opencode.jsonc
|
||||
@@ -31,15 +31,17 @@ Optional features and installation groups are documented in [`docs/installation.
|
||||
|
||||
`maintain.sh` runs dependency freshness and security checks, Ruff, Pyright scoped to `src/`, and the parallel test suite. Full-project Pyright is not the project gate because the ML dependency graph can exhaust Node memory.
|
||||
|
||||
This gate does NOT pass `--ignore-unfixed` to uv-secure, so a transitive CVE with no released fix stops it before Ruff, Pyright and the tests ever run. That is deliberate and it is not a blocker: run those three separately and report them, per the triage in [`docs/development.md`](docs/development.md). A red gate here is not the same signal as a red gate in a sibling repository that does ignore unfixed findings.
|
||||
|
||||
Command, gate, typing, and model-test invariants auto-load from [`.claude/rules/development.md`](.claude/rules/development.md). Environment recovery, CI behavior, and fixture policy live in [`docs/development.md`](docs/development.md).
|
||||
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Keep the source-distribution public allowlist.
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Treat the release as complete only after PyPI, Homebrew, the Hugging Face Space, and the ComfyUI Registry are verified; conda is not published. Keep the source-distribution public allowlist.
|
||||
|
||||
## Module architecture
|
||||
|
||||
[`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem.
|
||||
|
||||
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md).
|
||||
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Pixel photo classification is [`docs/photo-classify.md`](docs/photo-classify.md); it is not `identify`. Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md). The pre-split chronological archive is [`docs/synthid-detector-removal-plan.md`](docs/synthid-detector-removal-plan.md).
|
||||
|
||||
## Data safety
|
||||
|
||||
@@ -51,4 +53,4 @@ Topic-specific rules live in `.claude/rules/*.md` and are auto-loaded when match
|
||||
|
||||
| File | Covers |
|
||||
|---|---|
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, and the detection-path measurement rule |
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, the docs-coverage seam, and the detection-path measurement rule |
|
||||
@@ -2,17 +2,19 @@
|
||||
|
||||
Remove AI provenance marks from images and video you generated yourself:
|
||||
|
||||
- known visible labels such as the Gemini sparkle and vendor text marks;
|
||||
- known visible labels such as the Google Gemini sparkle watermark and vendor text marks;
|
||||
- invisible pixel watermarks through diffusion regeneration;
|
||||
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
|
||||
|
||||
Video support covers provenance identification, complete visible-plus-metadata
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo, and
|
||||
Kling mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Doubao, Dola,
|
||||
Hailuo AI, and Kling AI mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
removal.
|
||||
|
||||
> Try it online at [raiw.cc](https://raiw.cc) if you do not want to install Python
|
||||
> or run diffusion models locally.
|
||||
> [raiw.cc](https://raiw.cc) runs this library as a hosted service, with the GPU
|
||||
> included and nothing to install. Visible mark and metadata removal at Standard
|
||||
> output up to 12 MP are free there; original resolution above 12 MP and invisible
|
||||
> watermark removal are paid.
|
||||
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
[](https://pypi.org/project/remove-ai-watermarks/)
|
||||
@@ -30,6 +32,8 @@ removal.
|
||||
| Goal | Command | GPU |
|
||||
| --- | --- | --- |
|
||||
| Find provenance signals and watermarks | `identify` | No |
|
||||
| Classify a photograph from pixels (opt-in, not provenance) | `classify` | No |
|
||||
| Verify supported OpenAI SynthID from pixels with the official remote API | `verify-openai-synthid` | No |
|
||||
| Remove known visible AI marks | `visible` | No |
|
||||
| Erase a region you select | `erase` | No |
|
||||
| Strip AI metadata | `metadata` | No |
|
||||
@@ -43,22 +47,28 @@ removal.
|
||||
| Run visible, invisible, and metadata removal | `all` | Recommended |
|
||||
| Process a directory | `batch` | Depends on mode |
|
||||
|
||||
Microsoft Paint and Photos InvisMark declarations are routed automatically to
|
||||
pixel regeneration. The `all` command removes both the hidden pixel watermark and
|
||||
its linked C2PA manifest; metadata stripping alone removes only the manifest.
|
||||
|
||||
## Installation modes
|
||||
|
||||
| Need | Install |
|
||||
| --- | --- |
|
||||
| Metadata inspection and stripping | `remove-ai-watermarks` |
|
||||
| Photograph AI-versus-camera classification | `remove-ai-watermarks[classify]` |
|
||||
| Official remote OpenAI SynthID verification | `remove-ai-watermarks[verify]` |
|
||||
| Visible detection and removal | `remove-ai-watermarks[visible]` |
|
||||
| Visible video processing | `remove-ai-watermarks[video]` |
|
||||
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
|
||||
| Torch-free DWT-DCT detection | `remove-ai-watermarks[detect]` |
|
||||
| Invisible image removal (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
|
||||
| Every production feature | `remove-ai-watermarks[all]` |
|
||||
| Every production feature available on the active Python | `remove-ai-watermarks[all]` |
|
||||
|
||||
Lower-level and specialized extras include `pixels`, `heif`, `trustmark`,
|
||||
`migan`, `lama`, and `diffusion`. The
|
||||
[installation guide](docs/installation.md#feature-extras) documents their exact
|
||||
dependency composition and model requirements.
|
||||
dependency composition, Python compatibility, and model requirements.
|
||||
|
||||
## Quick start
|
||||
|
||||
@@ -74,6 +84,34 @@ Inspect an image:
|
||||
remove-ai-watermarks identify image.png
|
||||
```
|
||||
|
||||
To classify a photograph from pixels (AI versus camera, optional provider),
|
||||
install the extra and call `classify`. `identify` never starts it:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[classify]"
|
||||
remove-ai-watermarks classify image.png
|
||||
```
|
||||
|
||||
Guide: [photo pixel classification](docs/photo-classify.md).
|
||||
|
||||
Signed provenance is the supported route for SynthID and `identify` reads it.
|
||||
There is no local SynthID pixel detector in the package. Research on a
|
||||
periodic lattice expert is in `scripts/synthid_runtime/` and
|
||||
[synthid-detector-research.md](docs/synthid-detector-research.md).
|
||||
|
||||
For supported OpenAI images, the optional official verifier provides a pixel
|
||||
watermark verdict:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[verify]"
|
||||
remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload
|
||||
```
|
||||
|
||||
The command removes AI provenance metadata from a temporary copy, verifies that
|
||||
the decoded pixels are unchanged, uploads only that copy to OpenAI, and consumes
|
||||
only the independent SynthID response. It never runs implicitly from `identify`.
|
||||
An API key, endpoint access, and explicit upload acknowledgement are required.
|
||||
|
||||
For visible watermark removal, install the pixel dependencies:
|
||||
|
||||
```bash
|
||||
@@ -106,7 +144,9 @@ The `video metadata` command does not transcode video or audio streams. Unlike
|
||||
the image command above, when `-o` is omitted it writes `<source>_clean` and
|
||||
preserves the original. MP4 and MOV
|
||||
inspection includes the native TC260 `AIGC` tag in
|
||||
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload.
|
||||
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload,
|
||||
plus the QuickTime-form `meta` variants Doubao's iOS export writes (a bare
|
||||
`meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list).
|
||||
MKV and WebM inspection reads the normative
|
||||
`Segment.Tags.Tag.SimpleTag` placement. AVI uses `LIST/INFO/AIGC`, while FLV
|
||||
uses `script.onMetaData.AIGC`. The non-ISOBMFF formats are remuxed with stream
|
||||
@@ -161,8 +201,8 @@ the specificity order shown below. Pass an explicit mark to restrict detection
|
||||
to one provider.
|
||||
Sora covers the moving Sora 2 mascot and wordmark. Veo covers both the current
|
||||
four-point diamond and the legacy `Veo` text. Seedance covers the fixed boxed
|
||||
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo covers the composite
|
||||
`MINIMAX | hailuo AI` label, and Kling covers the bottom-right `KLING AI`
|
||||
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo AI covers the composite
|
||||
`MINIMAX | hailuo AI` label, and Kling AI covers the bottom-right `KLING AI`
|
||||
label with its version suffix. A completed encode is published atomically. No
|
||||
output is written when no stable mark is found.
|
||||
HDR, PQ/HLG, and greater-than-8-bit inputs are rejected before encoding rather
|
||||
@@ -184,7 +224,7 @@ remains useful for unusually important files or after provider changes, but it
|
||||
is not a product result state.
|
||||
|
||||
For invisible watermark removal, install the `qwen-zimage` extra. **An NVIDIA GPU
|
||||
is required**: both profiles are CUDA-only, and there is no CPU or MPS fallback.
|
||||
is required**: all profiles are CUDA-only, and there is no CPU or MPS fallback.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
@@ -198,6 +238,20 @@ image came from an AI generator, add `--force`:
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
Typography-heavy images can opt into the experimental verified-text post-pass.
|
||||
It accepts manually reviewed strings and line boxes or an operator-verified
|
||||
geometry-only manifest; it never treats raw OCR output as ground truth or runs
|
||||
automatically:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
See the [CLI guide](docs/cli.md#restore-operator-verified-text) for the manifest
|
||||
schema, compatibility restrictions, and oracle caveats.
|
||||
|
||||
See the [installation guide](docs/installation.md) for Homebrew, uv, optional
|
||||
features, and development setup.
|
||||
|
||||
@@ -209,12 +263,22 @@ features, and development setup.
|
||||
| --- | --- |
|
||||
|  |  |
|
||||
|
||||
The `after` raster is generated from the tracked `before` raster by the public path:
|
||||
|
||||
```bash
|
||||
uv run remove-ai-watermarks visible demo_banana_before.png \
|
||||
--backend cv2 -o demo_banana_after.png
|
||||
```
|
||||
|
||||
### High quality invisible removal
|
||||
|
||||
`qwen-zimage` is the default profile: a Qwen-Image-2512 Lightning pass under Canny
|
||||
ControlNet, followed by SAM-masked Z-Image repair of any detected face. The
|
||||
alternative, `sdxl-zimage`, swaps the global stage for SDXL and keeps the same face
|
||||
stage. Both are CUDA only.
|
||||
stage. A third profile, `chroma-zimage`, uses the Apache-2.0 Chroma1 global pass
|
||||
with its own flat vendor floors; see `docs/chroma1-engine-research.md` for the
|
||||
calibration. `--pipeline auto` picks chroma-zimage for OpenAI and Microsoft
|
||||
provenance and qwen-zimage otherwise. All are CUDA only.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
@@ -290,8 +354,10 @@ remove-ai-watermarks batch ./images --mode all
|
||||
|
||||
Visible mark support includes:
|
||||
|
||||
- Google Gemini and Nano Banana sparkle;
|
||||
- Doubao, Jimeng, Qwen, Kling, Yuanbao, Baidu, LibLibAI, and RunningHub labels;
|
||||
- Google Gemini and Nano Banana visible sparkle watermark;
|
||||
- Doubao, the Jimeng wordmark and top-left `AI生成` pill, Qwen, Kling AI,
|
||||
Yuanbao, Baidu, LiblibAI, and RunningHub labels;
|
||||
- one calibrated Microsoft top-right white AI-badge variant;
|
||||
- one calibrated Samsung Galaxy AI label variant.
|
||||
|
||||
Metadata and provenance inspection covers C2PA, EXIF, XMP, IPTC, common
|
||||
@@ -300,7 +366,9 @@ signals. Optional decoders add support for open DWT-DCT watermarks and Adobe
|
||||
TrustMark.
|
||||
|
||||
The exact support matrix, including important locale and detector limits, lives
|
||||
in [supported signals](docs/supported-signals.md).
|
||||
in [supported signals](docs/supported-signals.md). The
|
||||
[synthetic example gallery](data/fixtures/visible/README.md) carries one canonical
|
||||
sample for every registered image and video mark.
|
||||
|
||||
## How it works
|
||||
|
||||
@@ -333,6 +401,9 @@ import remove_ai_watermarks as raiw
|
||||
result, removed = raiw.remove_visible("watermarked.png", "clean.png")
|
||||
print(removed)
|
||||
|
||||
openai_synthid = raiw.verify_openai_synthid("image.png", acknowledge_upload=True)
|
||||
print(openai_synthid.status)
|
||||
|
||||
provenance = raiw.identify_video("input.mp4")
|
||||
report = raiw.inspect_video_metadata("input.mp4")
|
||||
complete = raiw.remove_video_all("input.mp4", "clean.mp4")
|
||||
@@ -375,7 +446,7 @@ invisible removal.
|
||||
detail.
|
||||
- Visible video removal recognizes the moving Sora 2 wordmark, the current Veo
|
||||
diamond plus legacy `Veo` text, the Seedance boxed `AI` label, and the fixed
|
||||
Dola, Hailuo, and Kling labels. It does not recognize the older Sora Turbo
|
||||
Doubao, Dola, Hailuo AI, and Kling AI labels. It does not recognize the older Sora Turbo
|
||||
corner swirl or unregistered layouts from those providers.
|
||||
The classical OpenCV backend can smear structured backgrounds; use MI-GAN or
|
||||
LaMa when recovery quality matters.
|
||||
@@ -383,7 +454,7 @@ invisible removal.
|
||||
The shipped profile is oracle-certified, but no public local decoder can
|
||||
certify an arbitrary output at runtime. Recheck unusually important outputs
|
||||
after provider changes.
|
||||
- Invisible-watermark removal requires CUDA. Both profiles refuse any other
|
||||
- Invisible-watermark removal requires CUDA. All profiles refuse any other
|
||||
device at construction rather than falling back to one that cannot run them.
|
||||
Visible removal, metadata stripping and `identify` still run anywhere.
|
||||
- Provider watermark systems can change. Validate important outputs with the
|
||||
@@ -415,8 +486,11 @@ reinterpret the pixel result returned `UNAVAILABLE`; that follow-up was not a
|
||||
verifier rerun and does not invalidate the built-in verdicts. A 2026-07-31
|
||||
full-clip check on a public eight-second Veo sample found `0.10` still detected
|
||||
and `0.15` not detected, so `0.15` is now the certified default. The
|
||||
reproducible hashes and verdicts live in
|
||||
`data/evaluations/video-synthid-oracle.csv`.
|
||||
reproducible hashes and verdicts for that full-clip check -- one carrier, two
|
||||
rows -- live in `data/evaluations/video-synthid-oracle.csv`. The earlier
|
||||
two-clip calibration is narrative only: its verdicts were not recorded in a
|
||||
tracked manifest, so treat the certified default as resting on the 2026-07-31
|
||||
rows.
|
||||
|
||||
## Documentation
|
||||
|
||||
|
||||
@@ -6,6 +6,9 @@ Tracked data is organized by purpose:
|
||||
data/
|
||||
fixtures/
|
||||
provenance/ Real format and provenance fixtures used by tests
|
||||
(source records live in fixtures/README.md)
|
||||
visible/ Synthetic per-mark example gallery (one committed example per
|
||||
registered visible mark; see fixtures/visible/README.md)
|
||||
calibration/
|
||||
<vendor>/ Minimal controlled inputs needed to rebuild detector assets
|
||||
synthid/
|
||||
@@ -14,6 +17,8 @@ data/
|
||||
full-pipeline-quality.csv
|
||||
Reusable full-pipeline evaluation selection
|
||||
evaluations/
|
||||
engine-selection/
|
||||
Prompt-matched AI content matrix for deterministic auto-engine research
|
||||
fidelity/ Evaluation instructions and hand-verified ground truth
|
||||
video-synthid-oracle.csv
|
||||
Reproducible full-clip Gemini SynthID verdicts
|
||||
@@ -21,6 +26,12 @@ data/
|
||||
|
||||
## Storage rules
|
||||
|
||||
Large local-only research data (the 211 GB Spaces corpus and the frozen
|
||||
checkpoints, research corpora, and campaign reports) lives under gitignored
|
||||
paths documented in `data/research/README.md` (ignored via `.git/info/exclude`,
|
||||
never committed). Worktree `.local-eval/` directories are scratch copies; the
|
||||
durable store is `data/research/` in the main checkout.
|
||||
|
||||
1. Store each binary image once. Evaluation manifests and documentation point
|
||||
to its canonical location.
|
||||
2. Put executable test fixtures in `fixtures/`.
|
||||
@@ -31,7 +42,12 @@ data/
|
||||
commands, hashes, and oracle verdicts instead of committing another corpus
|
||||
copy. A small curated before-and-after example may live in `docs/images/`
|
||||
when it is part of the public documentation.
|
||||
6. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
`evaluations/engine-selection/` is the bounded exception for ORIGINAL study
|
||||
inputs: its manifest tracks a prompt-matched OpenAI/Meta content matrix, not
|
||||
generated model outputs or watermark-oracle evidence.
|
||||
6. Keep third-party fixture license notices in `licenses/`, outside directories
|
||||
that tests enumerate as media inputs.
|
||||
7. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
Unregistered research candidates belong in
|
||||
`scripts/assets/visible-mark-candidates/` so they are not shipped in the
|
||||
wheel.
|
||||
@@ -45,16 +61,17 @@ runtime assets.
|
||||
video removal profile works, so it is also the source of truth for three shipped
|
||||
defaults: `tests/test_video_invisible.py` asserts that `noise_std`, `long_side`,
|
||||
and `fps` together match a row this manifest records as certified. Changing one
|
||||
of those three without adding the row that certifies it fails the suite. `vae` is
|
||||
deliberately outside that check because neither tracked row records one; add it
|
||||
to the assertion in the same commit as the first row that does.
|
||||
of those three without adding the row that certifies it fails the suite. Both
|
||||
historical rows mark `vae` as `unrecorded`: the repository must not infer a model
|
||||
identity from the current default. Add it to the assertion in the same commit as
|
||||
the first oracle row that records and verifies one.
|
||||
|
||||
| Column | Meaning |
|
||||
| --- | --- |
|
||||
| `date`, `source_url`, `source_sha256` | Identify the carrier. |
|
||||
| `source_width`, `source_height`, `source_fps` | Carrier geometry. Without it the actual downscale factor of a row cannot be recovered later. |
|
||||
| `duration_seconds`, `source_verdict` | Clip length submitted and the verifier's reading of the untouched carrier. |
|
||||
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The full run configuration. |
|
||||
| `vae`, `noise_std`, `long_side`, `fps`, `seed` | The run configuration. `unrecorded` means the historical run did not preserve the VAE identity. |
|
||||
| `output_sha256` | Identifies the exact submitted file. |
|
||||
| `output_verdict` | One of `detected`, `not_detected`, `indeterminate`, `refused`. |
|
||||
| `output_verdict_text` | The verifier's wording, verbatim. |
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# Content Seal oracle corpus
|
||||
|
||||
Muse Image (`muse-image-1.0`) generations with externally verified Content Seal
|
||||
verdicts, produced through the Meta Model API on 2026-08-26 and checked against
|
||||
the public detector at `https://meta.ai/identification` (anonymous session, no
|
||||
login). Follows the `data/synthid/originals` pattern: binaries live in
|
||||
`originals/`, every derived variant is recorded in `manifest.csv` as a recipe
|
||||
plus hash and is not stored.
|
||||
|
||||
## What this corpus establishes
|
||||
|
||||
- The Meta Model API image endpoint (`POST /v1/images/generations`) stamps the
|
||||
same Content Seal pixel watermark as the consumer Meta AI app: all five
|
||||
generations verified positive with attribution "Muse Image 1 - Meta".
|
||||
- The detector response carries a per-generation ID and creation timestamp
|
||||
embedded in the watermark payload. Both survived a 512 px LANCZOS resize and
|
||||
a full-size JPEG q85 re-encode (same ID returned), so the payload is more
|
||||
robust than the detection threshold.
|
||||
- Three checked center crops lost the seal: 50% and 33% linear crops of the fox
|
||||
and the 50% crop of the text poster returned "No AI signatures from Meta were
|
||||
found". The text poster's 33% crop was not checked because the daily oracle
|
||||
limit was reached, so its empty verdict is not evidence either way. The checked
|
||||
results are consistent with the Reuters 2026-07-11 analysis (55% missed after
|
||||
cropping).
|
||||
- API outputs carry XMP `iptcExt:DigitalSourceType =
|
||||
trainedAlgorithmicMedia`, so local `identify` flags them via the existing
|
||||
Made-with-AI path. Metadata-stripping transforms fall back to unknown, and
|
||||
Content Seal has no local decoder in this project: the oracle is the only
|
||||
reader.
|
||||
- Drift finding: the 512 px resize of `gen_fox_forest` triggers a
|
||||
medium-confidence false positive "Tencent Yuanbao (visible 元宝 / AI生成
|
||||
mark)" in this project's `identify`. Recorded here as a reproducible case.
|
||||
|
||||
## Oracle limits and wire format
|
||||
|
||||
There is no public or documented checking API. Verified against the official
|
||||
developer documentation on 2026-08-26 (`https://dev.meta.ai/docs/`): the full
|
||||
Meta Model API reference lists only Responses, Chat Completions, Messages,
|
||||
Files, Images (`/v1/images/generations`, `/v1/images/edits`), and Models, with
|
||||
no identification, detection, or watermark endpoint, and the image-generation,
|
||||
Muse Image cookbook, and pricing pages never mention watermark, Content Seal,
|
||||
or provenance at all. The API applies the seal (every generation in this corpus
|
||||
carries it) while documenting nothing about it. The web tool drives an internal
|
||||
REST pair, captured from the browser network log on 2026-08-26:
|
||||
|
||||
1. `POST https://rupload.meta.ai/gen_ai_document_gen_ai_tenant/<uuid>` with the
|
||||
raw file bytes, `x-entity-type`, `x-entity-length`, `ai_detector_upload: true`,
|
||||
and an anonymous `authorization: OAuth ecto1:<token>` session token minted by
|
||||
the page.
|
||||
2. `POST https://meta.ai/api/ai-detector` with `Bearer ecto1:<token>` and body
|
||||
`{"media_id": "...", "fileName": "...", "mimeType": "..."}`.
|
||||
|
||||
The rate limit is enforced at that endpoint, server-side, and keyed beyond the
|
||||
browser session: the API itself returns `429 {"errorType": "rate_limited"}`,
|
||||
and clearing cookies and storage changed nothing, so driving the internal pair
|
||||
directly does not bypass it. The Meta Model API (`api.meta.ai/v1`, where the
|
||||
generation key works) has no identification endpoint; plausible paths all
|
||||
return 404. Rows with an empty `oracle_verdict` were transformed but not yet
|
||||
checkable. Read a verdict only from the settled page text after the
|
||||
result-complete state ("Upload another file"): a wait for a verdict string can
|
||||
match the previous upload's text, and the fresh-navigation protocol used for
|
||||
the calibration rows below is the race-free variant.
|
||||
|
||||
## Strength floor calibration (qwen-zimage, seed 0)
|
||||
|
||||
The library resolves strength per vendor with measured floors (OpenAI
|
||||
0.07675 / Google 0.27 / Microsoft InvisMark 0.15 in
|
||||
`_internal/watermark_profiles.py`). Meta Content Seal had no floor before this
|
||||
calibration; these rows measure one by the same methodology: independent
|
||||
generations, each one's first-clean boundary, floor = worst boundary plus the
|
||||
observed cross-source spread.
|
||||
|
||||
Measured (2026-08-26/27, oracle `meta.ai/identification`):
|
||||
|
||||
- Default pipeline clears Content Seal: tested samples came back clean at the
|
||||
default resolution-adaptive strength (~0.1305 at 2.56 MP), including the
|
||||
worst source.
|
||||
- Five independent generations bracketed. First-clean boundaries:
|
||||
lighthouse (0.0525, 0.06], fox (0.03, 0.0375], night_city (0.03, 0.0375],
|
||||
mug <= 0.03, text <= 0.015. Cross-source spread is wide (a factor of four
|
||||
between easiest and hardest).
|
||||
- Derived Meta floor by the existing worst-boundary-plus-cross-source-spread
|
||||
method: 0.06 + (0.0525 - 0.015) = 0.0975, rounded to **0.1**.
|
||||
- Shipped as `QWEN_ZIMAGE_META_STRENGTH`: auto mode routes standalone-AI-IPTC
|
||||
files onto the cohort, and `--vendor meta` / `InvisibleOptions.vendor`
|
||||
names it explicitly on stripped files (implying the scrub runs).
|
||||
## Regeneration
|
||||
|
||||
The eight deterministic crop, resize, and JPEG variants can be reproduced and
|
||||
hash-checked from the tracked originals:
|
||||
|
||||
```bash
|
||||
uv run python scripts/contentseal_transforms.py /tmp/contentseal-derived
|
||||
```
|
||||
|
||||
The Meta API generations and remote GPU outputs are not reproducible from this
|
||||
repository alone. Their prompts, exact output hashes, model/profile settings,
|
||||
and oracle results are recorded in `manifest.csv`, but the generation API is
|
||||
stochastic and the private worker environment is not tracked. `MUSE_API_KEY` and
|
||||
the anonymous detector session are deliberately absent. A new calibration must
|
||||
therefore create new manifest rows rather than claiming to recreate these bytes.
|
||||
@@ -0,0 +1,44 @@
|
||||
name,file,origin,source_url,prompt,requested_size,delivered_size,sha256,xmp_iptc_trainedAlgorithmicMedia,oracle_verdict,oracle_attribution,oracle_generation_id,oracle_created_utc,checked_at_utc,notes
|
||||
gen_fox_forest,originals/gen_fox_forest.webp,meta-model-api,,A red fox resting on a mossy log in a misty forest at dawn photorealistic shallow depth of field,1024x1024,1600x1600,6d7cecf322a5ac4b2547e2f32c767b3cb3e8a0e49f274cfd00aa31be6697eed4,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:11Z,checked twice; ID stable across uploads
|
||||
gen_text_poster,originals/gen_text_poster.webp,meta-model-api,,A cheerful flat-design poster that says 'WATERMARK FIELD DAY 2026' in bold rounded letters over a coral and cream striped background,1024x1024,1600x1600,873927662d8198e9c38b4ae71a2919ba5d417dfcd714649db1b0ae231e8cbf9b,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbJuIKXPAYxsd,2026-08-26T19:08Z,2026-08-26T19:08Z,
|
||||
gen_lighthouse_watercolor,originals/gen_lighthouse_watercolor.webp,meta-model-api,,A watercolor illustration of a lighthouse on a cliff at golden hour seagulls in the distance soft loose washes white paper texture,1024x1024,1600x1600,69bee7c376608568361085602c5c7406900885849bd94c5279c3924cf793b27c,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbB5uWbAmWmwp,2026-08-26T19:08Z,2026-08-26T19:08Z,
|
||||
gen_night_city,originals/gen_night_city.webp,meta-model-api,,A night city skyline in the rain neon reflections on wet asphalt cinematic wide shot moody,1536x1024,1920x1280,345adf6b9f777b15ef7ce7bbdfb3b355a1d9e882740809e966e9f6caafd94d48,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbH5uWbAmWmwp,2026-08-26T19:08Z,2026-08-26T19:08Z,
|
||||
gen_studio_mug,originals/gen_studio_mug.webp,meta-model-api,,A minimalist studio photo of a single matte ceramic mug on a pastel blue background soft shadow clean commercial look,1024x1024,1600x1600,2b7bc7fd56fa2250c66ef74f8b3e49d9666fe147d894d5baf9461b5e36077a07,yes,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbAsSozV2XCzw,2026-08-26T19:08Z,2026-08-26T19:08Z,
|
||||
fox_crop50,,derived:gen_fox_forest,,center crop 0.5 linear each side then WEBP q95,,800x800,80b91995560a7c4b08596a26b3401a29afb82691fad193502697a49785eab7be,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost; PIL re-encode strips XMP
|
||||
fox_crop33,,derived:gen_fox_forest,,center crop 0.33 linear each side then WEBP q95,,528x528,3b3aded8f4fcb98d0599431abccb73b9581854ae34c6bc869724cd5c86a1e35e,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost
|
||||
fox_res512,,derived:gen_fox_forest,,resize long side 512 LANCZOS then WEBP q95,,512x512,19b85f615d56f5e9f55cd97bae0ee343f4f432e6bfa0a6fc7acb07dec2cc1d81,no_stripped_by_reencode,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:13Z,generation ID and creation time identical to the original generation: watermark payload survives the resize
|
||||
fox_jpeg85,,derived:gen_fox_forest,,full-size JPEG q85 re-encode,,1600x1600,3bc7dc11bf3c16c83814df1e6485cdb6e49e21141511dda56d210cc920a8c5e3,no_stripped_by_reencode,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T19:13Z,payload survives JPEG recompression; local identify returns unknown (metadata stripped)
|
||||
text_crop50,,derived:gen_text_poster,,center crop 0.5 linear each side then WEBP q95,,800x800,3f0d4075012ee4fec820209c5c1c32bb739998813325f49b1124f379162f9837,no_stripped_by_reencode,not_detected,,,,2026-08-26T19:13Z,seal lost
|
||||
text_crop33,,derived:gen_text_poster,,center crop 0.33 linear each side then WEBP q95,,528x528,95cef87002961831b21f6f47cd9a0824a67e2ee6f8503356fd5db6a3fcfcc998,no_stripped_by_reencode,,,,,,daily oracle limit reached before this row could be checked
|
||||
text_res512,,derived:gen_text_poster,,resize long side 512 LANCZOS then WEBP q95,,512x512,b319727d02e908f22e59f148050e6b4686cbd1a622bd69db51fe4072a6f66b34,no_stripped_by_reencode,,,,,,daily oracle limit reached; also the fox twin of this transform triggered the Yuanbao false positive below
|
||||
text_jpeg85,,derived:gen_text_poster,,full-size JPEG q85 re-encode,,1600x1600,76f00fcde1e974549afa5d90e1879c705e3c3631ef1f35cab901fc912cd15fa4,no_stripped_by_reencode,,,,,,daily oracle limit reached before this row could be checked
|
||||
blog_qr_manhwa,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1807972953501959&version=1783318590&transcode_extension=webp,,,1920x1280,8ee7700dff59f21d7de393a619664a0b9360f148d18df289364ecea939b3dea9,yes,detected,Muse Image 1 - Meta,,,2026-08-26T18:59Z,not stored; re-downloadable from source_url
|
||||
blog_fractal_poster,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1499727434782014&version=1783727678&transcode_extension=webp,,,2096x992,2a96168db026444b47ac37807fd99a88c8b7fa6866e1c93a54ab427fbe581f76,no,detected,Muse Image 1 - Meta,,,2026-08-26T18:59Z,seal survived CDN transcode without any metadata
|
||||
blog_flipbook_punch,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=2521328538303639&version=1783396590&transcode_extension=webp,,,1920x1280,35f28dc2cbdf44aed190bc8efaa0fe7bfa288a2196638712dd83bc941cc294d5,yes,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,
|
||||
blog_teen_cat,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=1325391919706734&version=1783319609&transcode_extension=webp,,,1600x1600,c2e7df000ffa6520589d8efc5f0bcbeb759175cdaa0c827189c28a64f75d3dd7,yes,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,
|
||||
blog_hero_1,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=2140990336800598&version=1783727678,,,1344x1792,7655639acb9f826cdb864d869530153efc1e1467ff414b5a63e80aa7b39b2fb7,no,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,seal survived CDN transcode without any metadata
|
||||
control_synth_gradient,,synthetic-local,,PIL gradient with rectangle JPEG q90; never touched any AI pipeline,,1024x768,25f29c0b993ed1a8e41060ed50de778d875a6d94847ced180aafc5a6ad429f82,no,not_detected,,,,2026-08-26T19:01Z,negative control: No AI signatures from Meta were found
|
||||
fox_nometa,,derived:gen_fox_forest,,metadata --remove only (no pixel change),,1600x1600,685d3a5c5d74e653ec7615fc7a347a0584c277fc7fb59594d388ac619ffe3985,no_stripped,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T22:19Z,metadata stripping does NOT remove Content Seal; seal lives in pixels; same generation ID as original
|
||||
fox_modal_invisible,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,6daa3e9b5dde5cb443d8071c5294262742f001c02b694243764a50345769d8b8,no_stripped,not_detected,,,,2026-08-26T22:21Z,current default pipeline clears Content Seal
|
||||
text_modal_invisible,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,8530a82fb856b0a00651076ac1ad8a7544b5d8873cac9bd5780704fbe10f53f8,no_stripped,not_detected,,,,2026-08-26T22:22Z,current default pipeline clears Content Seal
|
||||
fox_str0.06000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,7b5e5ae75ca20f2a3cdfcfa7666104752d9557b08ac236db6f1548434940a1f3,no_stripped,not_detected,,,,2026-08-26T22:28Z,strength calibration ladder
|
||||
fox_str0.04500,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,26da15694b3096e2f1717154db213e814cdd27f2fc40e5f924db2324213cf960,no_stripped,not_detected,,,,2026-08-27T02:28Z,strength calibration ladder; re-confirmed clean at 0.045
|
||||
fox_str0.03750,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,b7eb0616d10042933edf22cfc30aff4df915384828289b409f6cfa4cdf2807d4,no_stripped,not_detected,,,,2026-08-26T22:30Z,fresh-page protocol; fox first-clean boundary is in (0.03 0.0375]
|
||||
fox_str0.03000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,cff80d18d9d00000f47e0f97c419a37c13a13831795768aee8727aa9516d1401,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:31Z,fresh-page protocol; seal survives 0.03
|
||||
fox_str0.02000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,b19183ec60b63a5b73d3e1c723e7943584912857d51a4b5f85259c5c77da0e53,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:28Z,settled-read protocol; monotonic below the 0.03 boundary
|
||||
text_str0.03750,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,bacfd4eca7b13616502f74497cbe4f02ef6893918870f13f9e76fdb93b43107f,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol
|
||||
text_str0.03000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,d5c6cffbf471ef8bb74ee8c0d22cc17d9da2c446b83e46c4be67e6ea4cdd09de,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol; text first-clean boundary is at or below 0.03 - cross-source spread vs fox confirmed
|
||||
lh_str0.03750,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,f5169debc5fd2792399c0ea3fa88cb1d05f523d95004bec240c73db0e9aaee51,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.0375 where fox is already clean
|
||||
lh_str0.03000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,110d2112d91dd53df5ccd1839e0e2b41c44aa1832adb0090032660e071fb1fd4,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,fresh-page protocol; lighthouse holds its seal below the fox boundary: cross-source spread is wide
|
||||
text_str0.02500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,f46a6c53b4f3d39fccb1d62cca82ced2e163e4424d13c98e6468a8e083c0fa54,no_stripped,not_detected,,,,2026-08-27T02:28Z,mid rung inside the clean zone
|
||||
text_str0.02000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,a19cbe22313e0f19768c98567ea89f2d9dc30827691fc7ec2971c52f3e7c6cc6,no_stripped,not_detected,,,,2026-08-27T02:28Z,text clean at 0.02: boundary <= 0.02 - the easiest source measured
|
||||
text_str0.01500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.015,,1600x1600,aa3816c75e38f3ab5afc378d349caafad56f2bb6da859a5ee68fdea57ad00d73,no_stripped,not_detected,,,,2026-08-27T02:41Z,text clean at 0.015: boundary <= 0.015 - the easiest source measured
|
||||
lh_str0.02500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,989c4a7a804bc8420ec4d86b9445bc3a37382c1834782a7f8842f781d69b05e2,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
|
||||
lh_str0.02000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,6a4a4ec94292a145e7d73e6900b8ec6d0eede17137f2b91fba7757ec0bdba5ec,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
|
||||
nc_str0.03750,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1920x1280,46f322335f801ee54427eb39275216465f9a7cfa3d918c698e4d90a2439a0815,no_stripped,not_detected,,,,2026-08-27T02:28Z,boundary (0.03; 0.0375]
|
||||
nc_str0.03000,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1920x1280,3f42632319248ad1a91ef2457c7e0b13ed17e1d2569decbbaab67c2072e9a305,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,night_city holds seal at 0.03
|
||||
mug_str0.03750,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,3b6109ae7e86b8ea74ce33315e3f2a7eda1cadb724d429f0951f13c3ead34b0f,no_stripped,not_detected,,,,2026-08-27T02:28Z,upper rung for the mug bracket
|
||||
mug_str0.03000,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,1c24c984677e1528da83003ac40594f6e5ef1fa9351f1368b6d31a0101308583,no_stripped,not_detected,,,,2026-08-27T02:28Z,mug clean at 0.03: boundary <= 0.03
|
||||
lh_str0.05250,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0525,,1600x1600,8912ba1ff4a26790f69224c49dd649e46835e277c375276ac9604f5339775b2a,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:41Z,bisection: lighthouse first-clean boundary bracketed to (0.0525; 0.06]
|
||||
lh_str0.04500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,cfdcf7bbb27c740846a1de4edffee3b25a624c2a8e1aba2362feaa25b682e3ae,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.045: lighthouse is the worst source measured
|
||||
lh_str0.06000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,4e214be00a89dc9f1f32ec39483c5e9668a1ec7f38b15e5794942479c839fcca,no_stripped,not_detected,,,,2026-08-27T02:28Z,lighthouse first-clean boundary bracketed to (0.045 0.06]
|
||||
|
|
After Width: | Height: | Size: 585 KiB |
|
After Width: | Height: | Size: 510 KiB |
|
After Width: | Height: | Size: 652 KiB |
|
After Width: | Height: | Size: 83 KiB |
|
After Width: | Height: | Size: 192 KiB |
@@ -0,0 +1,61 @@
|
||||
# Engine-selection fixture set
|
||||
|
||||
This directory is the tracked, content-balanced input set for researching a
|
||||
deterministic `auto` engine choice. It is not a training corpus and it is not a
|
||||
watermark-oracle corpus.
|
||||
|
||||
## Content matrix
|
||||
|
||||
`content-manifest.csv` contains 19 prompt-matched pairs. Each pair uses the
|
||||
same prompt and generation index across OpenAI `gpt-image-1-mini` and Meta
|
||||
`muse-image-1.0`, covering the content classes that materially stress image
|
||||
regeneration:
|
||||
|
||||
- natural and urban scenes;
|
||||
- single-person portraits and people in context;
|
||||
- animals, food, products, interiors, architecture, macro detail, and action;
|
||||
- flat, isometric, watercolor, and surreal illustration styles.
|
||||
|
||||
The files are exact copies of generation outputs collected for the
|
||||
`docs/arxiv-paper-review` research branch. The manifest records the SHA-256 of
|
||||
the committed bytes. The old OpenAI collection manifest stored the API payload
|
||||
hash under `sha256`; `source_payload_sha256` preserves that value while
|
||||
`sha256` always means the actual file bytes here. Meta's two hashes are the
|
||||
same.
|
||||
|
||||
One generation per prompt and provider is enough for the first paired
|
||||
discovery pass. Any policy inferred from it must be challenged against unused
|
||||
generation indices from the local research archive before shipping. Generated
|
||||
outputs belong outside this directory.
|
||||
|
||||
`carrier-manifest.csv` adds the canonical, already tracked signal carriers to
|
||||
the study without copying their bytes. Together the two manifests cover
|
||||
ordinary content, text, single faces, face grids, and mixed face/text scenes.
|
||||
|
||||
## Signal carriers
|
||||
|
||||
The content matrix was re-encoded by its collection pipeline and has no local
|
||||
provenance signal. It must not be used to set or certify watermark-removal
|
||||
strengths. The tracked oracle/carrier inputs remain in their canonical homes:
|
||||
|
||||
- OpenAI and Google SynthID: `data/synthid/originals/` with
|
||||
`data/synthid/manifest.csv`;
|
||||
- Meta Content Seal: `data/contentseal/originals/` with
|
||||
`data/contentseal/manifest.csv`.
|
||||
|
||||
There is no tracked, publication-cleared Microsoft InvisMark carrier yet. The
|
||||
three Paint sources used by the earlier calibration are user uploads and remain
|
||||
outside git. A Microsoft row may be added only from a newly generated or
|
||||
otherwise publication-cleared source, with a pixel-identical metadata-stripped
|
||||
control that remains positive in Microsoft's detector.
|
||||
|
||||
## Integrity check
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run python scripts/verify_engine_selection_fixtures.py
|
||||
```
|
||||
|
||||
The check validates paths, byte hashes, dimensions, prompt pairing, and the
|
||||
declared provider/content matrix without loading any ML model.
|
||||
@@ -0,0 +1,13 @@
|
||||
provider,file,content_stratum,signal,verified_via,sha256,verified_text_geometry,verified_text_ground_truth
|
||||
openai,"../../synthid/originals/ChatGPT Image May 30, 2026, 10_31_08 AM.png",face_grid,OpenAI SynthID,openai-verify,28ff8732b037f98a4ef5bc277bbcdaa32e5eb9ccbd00b6c8c616e46ef68ae8a0,no,no
|
||||
openai,"../../synthid/originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png",multilingual_typography,OpenAI SynthID,c2pa-metadata,228e54413a3ef62b313176dcdba774a56a58c36b54227d1440bc26cbe0aefd82,yes,yes
|
||||
openai,"../../synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png",multilingual_typography,OpenAI SynthID,openai-verify,05b836ecfe40fd689177fda74384ae4fdcc446505bbc4281cd3cbb6523eb669e,yes,yes
|
||||
google,../../synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png,cjk_typography_scene,Google SynthID,gemini-app,4affd7f27767a445db6abf741355743ba8d95108ad922c9fff045feed8492236,yes,yes
|
||||
google,../../synthid/originals/Gemini_Generated_Image_akdbeiakdbeiakdb.png,cyrillic_typography_scene,Google SynthID,gemini-app,8c1a6fb03ef3d45a1f958fb3401e4264e409ff88c2a793061db7f29023454d0e,no,no
|
||||
google,../../synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png,crowd_faces,Google SynthID,gemini-app,45d79a683134fcba1b147b2aedb669783d474e1fb8a4df329729a0904fd1b46b,no,no
|
||||
google,../../synthid/originals/Gemini_Generated_Image_3mc4t93mc4t93mc4.png,faces_and_typography,Google SynthID,gemini-app,2c33e75a2db614ce74c83cc0a6ac6c3ac735aca83ab88c9c9345843b124f7856,no,no
|
||||
meta,../../contentseal/originals/gen_fox_forest.webp,photoreal_texture,Meta Content Seal,meta-oracle,6d7cecf322a5ac4b2547e2f32c767b3cb3e8a0e49f274cfd00aa31be6697eed4,no,no
|
||||
meta,../../contentseal/originals/gen_text_poster.webp,typography,Meta Content Seal,meta-oracle,873927662d8198e9c38b4ae71a2919ba5d417dfcd714649db1b0ae231e8cbf9b,no,no
|
||||
meta,../../contentseal/originals/gen_lighthouse_watercolor.webp,watercolor,Meta Content Seal,meta-oracle,69bee7c376608568361085602c5c7406900885849bd94c5279c3924cf793b27c,no,no
|
||||
meta,../../contentseal/originals/gen_night_city.webp,urban_night,Meta Content Seal,meta-oracle,345adf6b9f777b15ef7ce7bbdfb3b355a1d9e882740809e966e9f6caafd94d48,no,no
|
||||
meta,../../contentseal/originals/gen_studio_mug.webp,product_still_life,Meta Content Seal,meta-oracle,2b7bc7fd56fa2250c66ef74f8b3e49d9666fe147d894d5baf9461b5e36077a07,no,no
|
||||
|
@@ -0,0 +1,39 @@
|
||||
pair_id,provider,model,file,width,height,content_stratum,sha256,source_payload_sha256,prompt
|
||||
00,openai,gpt-image-1-mini,originals/openai/00.png,1024,1024,natural_landscape,698896455e0f1de57a227339227bf068143e690873b9c8c9672b31578de93ba0,1345f59fc7b51a1edf837745245bb90811feefd9ba7f9ba6f12dc89846aeba18,"a stunning photograph of a mountain lake at sunrise, highly detailed, professional photography"
|
||||
00,meta,muse-image-1.0,originals/meta/00.png,1600,1600,natural_landscape,83362c189b79cd0e700296b975e9ef8e7f2b2fe89813e407bc795299400ca5e0,83362c189b79cd0e700296b975e9ef8e7f2b2fe89813e407bc795299400ca5e0,"a stunning photograph of a mountain lake at sunrise, highly detailed, professional photography"
|
||||
01,openai,gpt-image-1-mini,originals/openai/01.png,1024,1024,street_scene,b885d8cb1490e4fbbe580b87750008780671390097d28e71ff18b50cf320ec85,772518e4686940d2c68df114298fb007a97a5c42548245092e989b6ca9bd5b90,"a vibrant street market in Bangkok with colorful fruits and spices, candid photo"
|
||||
01,meta,muse-image-1.0,originals/meta/01.png,1600,1600,street_scene,5ba08cf9ab07bf00f64f39fe93f1766a64f737817b00086783476566c5f3317f,5ba08cf9ab07bf00f64f39fe93f1766a64f737817b00086783476566c5f3317f,"a vibrant street market in Bangkok with colorful fruits and spices, candid photo"
|
||||
02,openai,gpt-image-1-mini,originals/openai/02.png,1024,1024,single_portrait,97618d1a5c39ddeba708df3a9448b27d1d26583ee68b490ba79f7f3e1eb2248d,6bace9d2adae4d630e1830449f0945a30d7ddb3635e3005bc7aad80206a57176,"a close-up portrait of a woman with freckles and green eyes, studio lighting, 85mm lens"
|
||||
02,meta,muse-image-1.0,originals/meta/02.png,1600,1600,single_portrait,4daf07fc8bdfe007b96a70b00ec0cfde87f180eb8c2d5501723e218ed29e7589,4daf07fc8bdfe007b96a70b00ec0cfde87f180eb8c2d5501723e218ed29e7589,"a close-up portrait of a woman with freckles and green eyes, studio lighting, 85mm lens"
|
||||
03,openai,gpt-image-1-mini,originals/openai/03.png,1024,1024,animal_action,eb6920aab03bbe6935a01756c8f2d524d383e307369cdfaad3fe44926fbd9e48,2012abb15be639ebb3c596d2210f6d4c50cce0e11dd6cfc88aeb29de1c82de18,"a golden retriever running through a field of sunflowers, action photography"
|
||||
03,meta,muse-image-1.0,originals/meta/03.png,1600,1600,animal_action,93dce3a5049f773adb882ef22243a84e4822b9c43ad30774741c40f3844ef405,93dce3a5049f773adb882ef22243a84e4822b9c43ad30774741c40f3844ef405,"a golden retriever running through a field of sunflowers, action photography"
|
||||
04,openai,gpt-image-1-mini,originals/openai/04.png,1024,1024,reflective_architecture,275d68de881041de989261ced7570dd460028e0406346642d880d0d1664bb9ae,1808d95c69080aafced4234e4c3ad7714121bad00f3f5dc84a744288d9757245,"a modern architecture building with glass facade reflecting sunset clouds"
|
||||
04,meta,muse-image-1.0,originals/meta/04.png,1600,1600,reflective_architecture,209c71f0b737a9ef60b4d620c66f518b34dbc42af4532b2b8e0b3086183c90ec,209c71f0b737a9ef60b4d620c66f518b34dbc42af4532b2b8e0b3086183c90ec,"a modern architecture building with glass facade reflecting sunset clouds"
|
||||
05,openai,gpt-image-1-mini,originals/openai/05.png,1024,1024,food_closeup,316a743ccf29f025904d1bcd047a6996f987715e4f93e8a0d9cf85b624349b07,a8bac088c350ef849bbf78a85a1e44aa1665d1054a0cdfae29461c1cdca56e4a,"a bowl of ramen with steam rising, moody food photography, shallow depth of field"
|
||||
05,meta,muse-image-1.0,originals/meta/05.png,1600,1600,food_closeup,4a50ca3e33ddacd2897b926cdc54615a389416fcd2edf4d8803ad882b85b33af,4a50ca3e33ddacd2897b926cdc54615a389416fcd2edf4d8803ad882b85b33af,"a bowl of ramen with steam rising, moody food photography, shallow depth of field"
|
||||
06,openai,gpt-image-1-mini,originals/openai/06.png,1024,1024,sports_action,390d87d1f59eb4fa38590c0c751b2848383c77ffcf549571a0bf00db0f783763,6cc24eeb0fce95a86741b9ac0c055002793a586ba37f0cf4d64201a5c5e73513,"a surfer riding a barrel wave at golden hour, backlit spray"
|
||||
06,meta,muse-image-1.0,originals/meta/06.png,1600,1600,sports_action,1d9fcbf0f471f733ae96ca94dff892afd9f5852767000e379d3486dc94e30b78,1d9fcbf0f471f733ae96ca94dff892afd9f5852767000e379d3486dc94e30b78,"a surfer riding a barrel wave at golden hour, backlit spray"
|
||||
07,openai,gpt-image-1-mini,originals/openai/07.png,1024,1024,interior_low_light,4c77c2814216bdcde3417d2d228c5b6f275e33777f13cbea7911241bc9bc376b,9e858b95a4f842c917cbf47afeb4b7e0e24e04c4df0b19da4afe00b7316a13a9,"a cozy reading nook with warm lamp light and rain on the window"
|
||||
07,meta,muse-image-1.0,originals/meta/07.png,1600,1600,interior_low_light,ecec76c086a7b33beec1b3cb787f62cad68c09cb362a8888a8af734e9d4f3425,ecec76c086a7b33beec1b3cb787f62cad68c09cb362a8888a8af734e9d4f3425,"a cozy reading nook with warm lamp light and rain on the window"
|
||||
08,openai,gpt-image-1-mini,originals/openai/08.png,1024,1024,isometric_illustration,d688f6da74d4a7b2bd29ef59ba64864d0915134433febd8c90eb5ce1dc80781f,37a801f66b7c1bf63e613a7a5717dbc00397120a7a32ec45414661e03137a180,"an isometric illustration of a cozy coffee shop interior, warm colors, detailed"
|
||||
08,meta,muse-image-1.0,originals/meta/08.png,1600,1600,isometric_illustration,d4683d16a55c5f358f5b3b876b89fbc810ae1e3332e74399395ae20c9f646ef5,d4683d16a55c5f358f5b3b876b89fbc810ae1e3332e74399395ae20c9f646ef5,"an isometric illustration of a cozy coffee shop interior, warm colors, detailed"
|
||||
09,openai,gpt-image-1-mini,originals/openai/09.png,1024,1024,watercolor,d0beefb80ee00a0937a851932a2ed6dab2c670f10104ffe94c5ff22869a9595d,b52401235efc16641d3cb8f9c7c6d20a75c36751ada9409b65ceefa0e57901da,"a watercolor painting of Venice canals at sunset, loose brushstrokes"
|
||||
09,meta,muse-image-1.0,originals/meta/09.png,1600,1600,watercolor,c76ffbca59fbcb2dba9b66c94598f0c0062b0ac4679e348cdf1e18a06d3eb64d,c76ffbca59fbcb2dba9b66c94598f0c0062b0ac4679e348cdf1e18a06d3eb64d,"a watercolor painting of Venice canals at sunset, loose brushstrokes"
|
||||
10,openai,gpt-image-1-mini,originals/openai/10.png,1024,1024,macro_detail,59d9c6dce5ce4c2b77427144ccbafc8a4b7ab87564673677119859c31598b0d4,3740f9de304f050fc7306447c6668dcaa8156a888709f759f05715e7987a321c,"a macro photograph of a dewdrop on a spider web at dawn, bokeh background"
|
||||
10,meta,muse-image-1.0,originals/meta/10.png,1600,1600,macro_detail,c86fd9c3a59aa0bec709c5bce7c2ef1be4ecb1d780f3a3f35460b73cd97d1a12,c86fd9c3a59aa0bec709c5bce7c2ef1be4ecb1d780f3a3f35460b73cd97d1a12,"a macro photograph of a dewdrop on a spider web at dawn, bokeh background"
|
||||
11,openai,gpt-image-1-mini,originals/openai/11.png,1024,1024,urban_night,e4f6f806ed8559973374eae1e4d379807e45046749b7d491edbbc8ad9647728e,06bfc68289ece1b85b169df4cedd5e1cf12d0efa30631bdecd0e3edce4d8c82e,"a cinematic wide shot of Tokyo at night with neon reflections on wet streets"
|
||||
11,meta,muse-image-1.0,originals/meta/11.png,1600,1600,urban_night,7877cf0c13b07fbc6651178d838db4ab253369896292af1cf3f85a957a559db8,7877cf0c13b07fbc6651178d838db4ab253369896292af1cf3f85a957a559db8,"a cinematic wide shot of Tokyo at night with neon reflections on wet streets"
|
||||
12,openai,gpt-image-1-mini,originals/openai/12.png,1024,1024,flat_illustration,6bf230aa0b6f40d7ff0d23b22b8aac2e9b66288eb85c02eebdf43b71a1597f5b,65567d5cecb34241ac755ac4db5eb5a7b271d9a00fa27d826c961d2a2ef6ac3d,"a flat-design poster of Santorini with bold shapes, screen print style"
|
||||
12,meta,muse-image-1.0,originals/meta/12.png,1600,1600,flat_illustration,1b71bd94aa18d737c8a1c319a33dd0f0e9e15c2e93210c9c3aa98242b9a67955,1b71bd94aa18d737c8a1c319a33dd0f0e9e15c2e93210c9c3aa98242b9a67955,"a flat-design poster of Santorini with bold shapes, screen print style"
|
||||
13,openai,gpt-image-1-mini,originals/openai/13.png,1024,1024,storm_landscape,d07b05eef633fcb5b6b54268cc2cdd787dc19f8050198872e462f69e383a054d,a692a8a0daebad375cd57d7c671be6e16ad3ef700f0b9214daacbe80b6148c90,"a dramatic storm cloud over a lonely lighthouse, moody, high contrast photography"
|
||||
13,meta,muse-image-1.0,originals/meta/13.png,1600,1600,storm_landscape,e669128fe2d3b59dc83ae0b216c5378fe5930e77fa3a692219da238dbf55d5f6,e669128fe2d3b59dc83ae0b216c5378fe5930e77fa3a692219da238dbf55d5f6,"a dramatic storm cloud over a lonely lighthouse, moody, high contrast photography"
|
||||
14,openai,gpt-image-1-mini,originals/openai/14.png,1024,1024,monochrome_portrait,9fb1153f4d0041932b35bcd1408d774b642c68f81d7bb403d22f50f8c891ede7,826acf13df78378499e57c7fe1a8309947ab26810f46482be3a64d3328de1ff6,"a black and white portrait of an elderly craftsman in his workshop"
|
||||
14,meta,muse-image-1.0,originals/meta/14.png,1600,1600,monochrome_portrait,8339475f0585491ea08992d29890f3c759f24db2184212c6bf4f7efc0932e177,8339475f0585491ea08992d29890f3c759f24db2184212c6bf4f7efc0932e177,"a black and white portrait of an elderly craftsman in his workshop"
|
||||
15,openai,gpt-image-1-mini,originals/openai/15.png,1024,1024,surreal_composite,8bf02f9028fd049ebfafd3410d2eb9d618819be1c14b4e05d53a490e22ba109c,e71de5a9a2f6579857d2e24b8e7a1516031f19ee824906cece158101d13fe001,"a surreal double exposure of a forest and a human profile, conceptual art"
|
||||
15,meta,muse-image-1.0,originals/meta/15.png,1600,1600,surreal_composite,20628541cc924e623984604337807c7afd453aa8a7652f087add742a13828f6a,20628541cc924e623984604337807c7afd453aa8a7652f087add742a13828f6a,"a surreal double exposure of a forest and a human profile, conceptual art"
|
||||
16,openai,gpt-image-1-mini,originals/openai/16.png,1024,1024,brutalist_architecture,e91ca5128ce8644510c5c4def6bba085e45e1716105b3b6a2761cc18d42018b3,0aa8a1819f160ce26dfc9e59b996f969698fdf4eba3402fe999af598fcad80d4,"a low-angle shot of a brutalist building against dramatic clouds"
|
||||
16,meta,muse-image-1.0,originals/meta/16.png,1600,1600,brutalist_architecture,d22702d26343b6575e89751c95d865696c6085bed053f6008f989434a96bac02,d22702d26343b6575e89751c95d865696c6085bed053f6008f989434a96bac02,"a low-angle shot of a brutalist building against dramatic clouds"
|
||||
17,openai,gpt-image-1-mini,originals/openai/17.png,1024,1024,product_still_life,4f41f4a46b80c0ded61a9a3ce1273412b1728476a0734519d1af34a1797013b0,1f60c25c59138c14bacb0dd485475037e7d6222bb45daa66ed85f9dd91c83e4e,"a minimalist product photo of a ceramic mug on a marble surface, clean commercial look"
|
||||
17,meta,muse-image-1.0,originals/meta/17.png,1600,1600,product_still_life,67bde86edfd3db08b88c0dbc5572958408875e188dac6fa8ed64f821b7e7e994,67bde86edfd3db08b88c0dbc5572958408875e188dac6fa8ed64f821b7e7e994,"a minimalist product photo of a ceramic mug on a marble surface, clean commercial look"
|
||||
18,openai,gpt-image-1-mini,originals/openai/18.png,1024,1024,person_in_context,9b049d3112a654a87022d591b93a01087693c0664be5af560d3c0ec490291963,ba873bc7989720245821f5298f12e36bd7b6051909694b72569645c29cdd6bd0,"a lifestyle photo of a person working on a laptop in a bright cafe"
|
||||
18,meta,muse-image-1.0,originals/meta/18.png,1600,1600,person_in_context,cc4b2bcfc4e80d7dfe6646beb12529168137aef04d2b6765c6382b830ce19ed7,cc4b2bcfc4e80d7dfe6646beb12529168137aef04d2b6765c6382b830ce19ed7,"a lifestyle photo of a person working on a laptop in a bright cafe"
|
||||
|
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 4.0 MiB |
|
After Width: | Height: | Size: 2.5 MiB |
|
After Width: | Height: | Size: 3.2 MiB |
|
After Width: | Height: | Size: 2.9 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 4.0 MiB |
|
After Width: | Height: | Size: 3.0 MiB |
|
After Width: | Height: | Size: 3.2 MiB |
|
After Width: | Height: | Size: 3.6 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.1 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 3.4 MiB |
|
After Width: | Height: | Size: 3.0 MiB |
|
After Width: | Height: | Size: 854 KiB |
|
After Width: | Height: | Size: 2.8 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
|
After Width: | Height: | Size: 1.5 MiB |
|
After Width: | Height: | Size: 1.5 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 1.6 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 1.3 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 1.6 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 1.1 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
@@ -7,31 +7,394 @@ ground truth and instructions.
|
||||
|
||||
| Original | Provider | Content | Exercises |
|
||||
| --- | --- | --- | --- |
|
||||
| `ChatGPT Image May 31, 2026, 02_02_23 PM.png` | OpenAI | Light multilingual typography | Text preservation |
|
||||
| `ChatGPT Image May 31, 2026, 02_03_55 PM.png` | OpenAI | Multilingual typography | Text preservation |
|
||||
| `Gemini_Generated_Image_633uuy633uuy633u.png` | Google | Landscape with a Chinese sign | CJK text preservation |
|
||||
| `Gemini_Generated_Image_y48j3cy48j3cy48j.png` | Google | Portrait grid | Face identity and skin texture |
|
||||
|
||||
## Text ground truth
|
||||
|
||||
`ground-truth.json` contains hand-verified OCR for the two text-bearing
|
||||
`ground-truth.json` contains hand-verified OCR for the three text-bearing
|
||||
originals. To regenerate an OCR seed:
|
||||
|
||||
```bash
|
||||
uv run scripts/fidelity_metrics.py ocr \
|
||||
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png" \
|
||||
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png" \
|
||||
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
|
||||
--langs en,ru,ch \
|
||||
--out data/evaluations/fidelity/ground-truth.json
|
||||
```
|
||||
|
||||
Verify and correct the generated text by hand before using it as ground truth.
|
||||
`text-lines.json` contains the verified per-line strings and source-space boxes
|
||||
used by the evaluation-only selective-restoration experiment. It is not an
|
||||
automatic scene-text annotation set.
|
||||
|
||||
`scripts/infer_text_lines.py` can generate a draft from source pixels without
|
||||
existing annotations. On the two posters it proposed 20 and 18 lines at the
|
||||
default threshold, but exact-text precision was only 90.0% and 94.4% because
|
||||
high-confidence OCR still dropped an English comma and replaced a Chinese comma
|
||||
with ASCII. Its output therefore requires manual verification of every line;
|
||||
`accepted` means crop-stable, not ground-truth-correct.
|
||||
|
||||
## Text-preservation benchmark
|
||||
|
||||
`text-preservation-2026-08-13.csv` records a fixed-seed comparison of the two
|
||||
current profiles and a global Z-Image Turbo prototype on all three text
|
||||
fixtures. Each candidate ran through the
|
||||
complete `visible -> invisible -> metadata` route with its profile default
|
||||
strength and adaptive-polish setting. The Z-Image prototype instead sweeps
|
||||
0.08, 0.10, 0.15, 0.20, and 0.25 without polish because its provider-specific
|
||||
removal floor is not known. The output hash identifies the exact bytes
|
||||
measured; generated outputs remain outside the repository.
|
||||
|
||||
The character-weighted added CER is 0.262 for `qwen-zimage` and 0.256 for
|
||||
`sdxl-zimage`. That 0.006 absolute difference is not a
|
||||
stable ordering: SDXL wins the light poster, Qwen wins the dark poster, and the
|
||||
Chinese sign is tied. A paired sign test on the two non-ties is 1-1 (`p=1.0`).
|
||||
The measured sample therefore does not support a general text-preservation
|
||||
winner. Both profiles substantially degrade the smallest multilingual poster
|
||||
text and preserve the larger Chinese sign.
|
||||
|
||||
Z-Image Turbo is the clear fidelity lead. At strength 0.10 its
|
||||
character-weighted CER is 0.093, against the unchanged sources' 0.124 OCR
|
||||
floor, while whole-image LPIPS stays between 0.047 and 0.086. Visual inspection
|
||||
still finds substitutions in the smallest Cyrillic and Chinese poster lines,
|
||||
so a favorable OCR score does not mean pixel-exact text. Quality drops quickly
|
||||
above 0.15; at 0.25 the weighted CER is 0.428.
|
||||
|
||||
Provider-oracle checks on 2026-08-13 bracket the OpenAI requirement at no more
|
||||
than 0.10 for this sample. Both original OpenAI controls were detected by
|
||||
`openai.com/verify`; the light poster was clean at 0.08, while the dark poster
|
||||
was detected at 0.08 and clean at 0.10. The light 0.10 output was not separately
|
||||
checked. Google fails the entire tested sweep. The original Gemini image was
|
||||
detected through C2PA, and a pixel-identical copy with AI metadata stripped was
|
||||
separately detected by Gemini's built-in SynthID verifier. Z-Image outputs at
|
||||
0.08, 0.10, 0.15, 0.20, and 0.25 were all still detected. At 0.25 the weighted
|
||||
CER has already risen to 0.428, worse than the raw weighted CER of both current
|
||||
profiles (Qwen 0.387, SDXL 0.381). Increasing strength beyond the measured grid
|
||||
would therefore no longer serve the text-preservation objective without a new
|
||||
mechanism or hypothesis. Z-Image is not a viable global replacement on this
|
||||
evidence: no clean Google operating point was found before it lost its fidelity
|
||||
advantage.
|
||||
|
||||
Qwen-Image-2.0 was not added to the numeric comparison. Its weights are not
|
||||
published, and its hosted edit API exposes an editing instruction and seed but
|
||||
no low-strength denoise control. It can be evaluated as a separate hosted edit
|
||||
strategy when credentials are available, but it is not a drop-in replacement
|
||||
for the partial-regeneration mechanism measured here.
|
||||
|
||||
The OCR floor is the source image scored against the hand-verified text. Use
|
||||
`added_cer = text_cer - ocr_floor` when interpreting pipeline damage, because the
|
||||
unchanged poster sources already score 0.127 CER. `oracle_rechecked=true` marks
|
||||
the exact Z-Image bytes checked above; the remaining rows were not rechecked.
|
||||
The table does not certify other seeds, content classes, or strengths beyond
|
||||
the recorded provider verdicts.
|
||||
|
||||
## Text-restoration prototype
|
||||
|
||||
`text-restoration-2026-08-13.csv` evaluates an OCR-driven post-pass on the exact
|
||||
Qwen outputs above. The prototype recognizes English and Russian with macOS
|
||||
Vision and CJK with PaddleOCR, derives glyph masks independently from the
|
||||
source and Qwen output, removes both sets of glyphs with block-wise LaMa, and
|
||||
draws the recognized strings with new system-font pixels. It never composites
|
||||
source pixels back into the result.
|
||||
|
||||
On the two multilingual posters, character-weighted CER fell from 0.338 and
|
||||
0.305 to 0.007 on both. OpenAI Verify reported no OpenAI signals for both Qwen
|
||||
controls and both restored outputs in the same run. The improvement comes with
|
||||
a substantial whole-image fidelity cost: LPIPS rose from 0.107 to 0.174 and
|
||||
from 0.095 to 0.162, while PSNR fell by about 10.5-11.4 dB. Visual inspection
|
||||
found one residual shadow in the smallest English line of the light poster;
|
||||
the dark poster was clean but the substitute fonts visibly changed typography.
|
||||
|
||||
The Chinese sign did not improve: CER rose from 0.074 to 0.111 because the OCR
|
||||
and renderer changed punctuation. The Gemini verifier returned detected for
|
||||
both the restored output and its byte-identical Qwen control on a third work
|
||||
account, although that Qwen hash had previously returned clean on another work
|
||||
account. This run therefore does not isolate a restoration-stage regression;
|
||||
its Google verdict is inconclusive until a source-positive, Qwen-negative
|
||||
control can be reproduced in the same available account.
|
||||
|
||||
This prototype is not ready to ship. Its strongest result establishes that
|
||||
fresh-glyph reconstruction can recover literal text without reintroducing an
|
||||
OpenAI signal, but portable OCR, font/style reconstruction, a tighter mask, and
|
||||
a reproducible Google oracle control remain prerequisites.
|
||||
|
||||
### Selective restoration follow-up
|
||||
|
||||
`selective-text-restoration-2026-08-13.csv` compares that full compositor with
|
||||
a selective prototype on the same two posters. The prototype leaves a Qwen line
|
||||
unchanged when padded source and output recognition agree, and applies the same
|
||||
LaMa plus fresh-system-font reconstruction only to lines whose recognized text
|
||||
changed. This reduced the edited area from 15.4% and 17.0% to 5.5% on both
|
||||
posters.
|
||||
|
||||
Under one consistent Paddle `en+ru+ch` measurement route, selective restoration
|
||||
reduced Qwen CER from 0.378 to 0.101 on the light poster and from 0.413 to 0.112
|
||||
on the dark poster. Its image LPIPS was 0.120 and 0.103, substantially closer to
|
||||
Qwen than the full compositor's 0.174 and 0.162. These CER values must not be
|
||||
mixed with the preceding table's Vision/Paddle hybrid values: the comparison
|
||||
file remeasures all three variants through Paddle so their relative result is
|
||||
valid on one OCR route.
|
||||
|
||||
OpenAI Verify returned `No OpenAI signals detected` for both selective outputs,
|
||||
then detected the original light poster as `Generated with OpenAI tools` in the
|
||||
same Chrome sequence. Visual inspection found the dark output clean, but the
|
||||
light output still retained a local shadow around one replaced fine-text line.
|
||||
Selective restoration is therefore the strongest current direction, not a
|
||||
production-ready default. The next implementation needs automatic line matching
|
||||
and a tighter source-plus-candidate glyph mask before it can be proposed for the
|
||||
pipeline.
|
||||
|
||||
A mask-only follow-up added two pixels of dilation around every selected glyph.
|
||||
It visually removed the light poster's shadow and improved its CER from 0.101 to
|
||||
0.081 with effectively unchanged LPIPS, but increased the edited area from 5.5%
|
||||
to 7.0%. Applying the same rule to the dark poster worsened CER from 0.112 to
|
||||
0.147. These expanded-mask rows were not resubmitted to the provider oracle.
|
||||
The opposing result rules out one global dilation constant and makes per-line
|
||||
residual detection the next fidelity gate.
|
||||
|
||||
An automatic follow-up removed the hand-selected line list. For each known
|
||||
source line, it runs a recognition-only model on padded, upscaled source and
|
||||
Qwen crops. It preserves a line only when both confidence scores are at least
|
||||
0.75, the source recognition is within 0.25 normalized edit distance of the
|
||||
verified line, and normalized source and candidate recognition match exactly.
|
||||
This reproduced the manual dark-poster line selection and additionally kept one
|
||||
correct large Russian line on the light poster.
|
||||
|
||||
The automatic prototype also replaced uniform dilation with a second measured
|
||||
LaMa pass. After the first erase, it extracts contrast remaining at the original
|
||||
glyph positions, dilates only that residual, and inpaints it before drawing fresh
|
||||
text. Visual inspection found no remaining double glyphs on either output. The
|
||||
effective edited fractions were 4.1% and 5.7%. Image LPIPS was 0.113 and 0.104,
|
||||
against 0.120 and 0.103 for the manual selective version. Detector-based page CER
|
||||
was 0.123 and 0.119; the light value is inflated by the detector splitting the
|
||||
preserved Cyrillic line into Latin-like fragments even though the padded Russian
|
||||
recognizer reads it exactly. This is another reason that line decisions use
|
||||
recognition-only crops rather than the page detector's aggregate CER.
|
||||
|
||||
Both exact automatic output hashes returned `No OpenAI signals detected`, and
|
||||
the original light poster returned `Generated with OpenAI tools` immediately
|
||||
afterward in the same Chrome sequence. The tracked script reproduced the dark
|
||||
hash exactly. Its light output differed from the initial prototype in only 386
|
||||
pixels by at most one channel value because separate ONNX environments rounded
|
||||
differently; the script's exact `900def5a...` hash was therefore submitted and
|
||||
also returned clean before the positive source control.
|
||||
|
||||
The script and its required verified line annotations are tracked as
|
||||
`scripts/selective_text_restoration.py` and `text-lines.json`. For example:
|
||||
|
||||
```bash
|
||||
PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK=True \
|
||||
uv run scripts/selective_text_restoration.py \
|
||||
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
|
||||
path/to/qwen-candidate.png \
|
||||
--restoration vae-glyphs \
|
||||
--glyph-donor path/to/qwen-vae-reconstruction.png \
|
||||
--glyph-feather 0.5 \
|
||||
--output path/to/selective-output.png \
|
||||
--manifest path/to/manifest.json
|
||||
```
|
||||
|
||||
The restoration mode is deliberately explicit. `vae-glyphs` requires a
|
||||
separately generated VAE reconstruction in `--glyph-donor`; it takes exact
|
||||
thresholded glyph cores from that donor, uses a fresh silhouette beneath them,
|
||||
and defaults to a narrow 0.5-pixel donor edge. The donor and the resulting exact bytes still require
|
||||
full-pipeline oracle evaluation. `rerender` reproduces the tracked
|
||||
fresh-system-font experiment. `source-glyphs` is only for compositing a
|
||||
separately regenerated, oracle-evaluated layer; feeding it the watermarked
|
||||
original would paste provenance-bearing pixels back into the result. The
|
||||
`--keep-background --composite-mask boxes` combination is an aligned-layer
|
||||
experiment and is not a production text restorer.
|
||||
|
||||
The `source-silhouette` follow-up keeps the source glyph geometry but discards
|
||||
its pixel amplitudes: it thresholds each source line to a binary shape, samples
|
||||
one foreground color, and synthesizes new antialiasing over the scrubbed image.
|
||||
On a 24-image matrix spanning serif, Latin-diacritic, Cyrillic, CJK, tiny-UI,
|
||||
and rotated text, the original outer-feather compositor preserved all 34
|
||||
source-readable lines exactly under crop OCR, with median text-box SSIM 0.859.
|
||||
An inner-antialias variant raised median text-box SSIM to 0.902 and left
|
||||
whole-image SSIM unchanged at 0.672, but preserved 33/34 lines: OCR read one
|
||||
middle dot as a colon. The higher-fidelity antialiasing is retained in the
|
||||
evaluation script, with that punctuation miss recorded as a caveat. Its exact
|
||||
outputs were not submitted to the provider oracle because the
|
||||
public verifier reached its request limit; a clean verdict from the earlier
|
||||
outer-feather bytes does not transfer to the new hashes.
|
||||
|
||||
Follow-up visual review on a typography-rich control rejected both silhouette
|
||||
compositors despite their OCR and SSIM scores. They preserved literal content
|
||||
and approximate glyph geometry, but changed stroke weight, color variation,
|
||||
edge antialiasing, and small decorative details enough to be plainly different
|
||||
from the source. OCR exactness and text-box SSIM are therefore screening metrics,
|
||||
not acceptance gates for source-typography preservation. A candidate must also
|
||||
retain source-like edge pixels and pass direct visual comparison at native size.
|
||||
|
||||
A narrower Qwen-VAE donor follow-up keeps the scrubbed background, then copies
|
||||
only VAE-reconstructed pixels through the source-silhouette mask with a
|
||||
0.5-pixel feather. Across the 48-case typography matrix (548 annotated lines),
|
||||
text-box SSIM improved in 47/48 cases and glyph-edge error improved in 48/48.
|
||||
Median text-box SSIM rose from 0.854 to 0.914, while median glyph-edge MAE fell
|
||||
from 37.59 to 32.42. The median nonzero alpha area was 3.62%; on dense or large
|
||||
text this is still substantially wider than the target control. These are
|
||||
fidelity results only; the 48 matrix outputs were not submitted to the provider
|
||||
oracle. On a separate untracked dense-typography control, the exact core-only
|
||||
Qwen-VAE donor with a 1.43% source-silhouette area returned `No OpenAI signals
|
||||
detected`. Its 0.5-pixel feathered sibling, which raises the nonzero donor area
|
||||
to 2.75%, returned the same verdict and improved mean text-box SSIM from 0.957
|
||||
to 0.960. Crop OCR recovered from 7/15 exact lines on the raw pass to 14/15,
|
||||
matching the source OCR floor. Whole-image LPIPS was 0.082, but only 0.108% of
|
||||
pixels were exactly unchanged and the detected face retained 0.670 of source
|
||||
Laplacian variance. The tracked script reproduced the feathered file byte for
|
||||
byte. These two exact-byte verdicts do not certify other images or the larger
|
||||
matrix masks, and the global smoothing fails a strict unchanged-image criterion.
|
||||
|
||||
The opt-in production port was rechecked separately on 2026-08-15. Its current
|
||||
LaMa runtime did not reproduce the earlier evaluation PNG byte for byte, but all
|
||||
changed pixels were confined to the erased background outside the donor glyph
|
||||
core. The exact production artifact returned `No OpenAI signals detected` in
|
||||
3/3 OpenAI Verify runs, while the matched source control returned `Generated
|
||||
with OpenAI tools` in 2/2 runs in the same Chrome session; expanded details
|
||||
identified SynthID and no C2PA manifest on the control. The private control and
|
||||
artifact hashes remain outside the public repository. This certifies only that
|
||||
runtime, verified manifest, and output, not arbitrary text masks or images.
|
||||
|
||||
The Google result is negative. On the synthetic CJK sign case, two separate
|
||||
work-account runs both detected SynthID in the resaved source control and in the
|
||||
exact Qwen-VAE donor output. The candidate improved mean text-box SSIM from
|
||||
0.791 to 0.835 and glyph-edge MAE from 35.48 to 22.49 with a 3.67% donor area,
|
||||
but Gemini still detected it. The intermediate Qwen silhouette base and the
|
||||
earlier original-sign Qwen baseline were also detected in the same account.
|
||||
This isolates the current blocker upstream of text restoration: the global
|
||||
Google pass did not reach the SynthID removal floor, so the OpenAI-clean
|
||||
`vae-glyphs` configuration cannot be used as a Google operating point.
|
||||
|
||||
On the large Chinese-sign control, source and candidate recognition agreed on
|
||||
all three lines. The script selected no changed lines, emitted a zero mask, and
|
||||
copied the candidate byte for byte instead of re-encoding it. This corrects the
|
||||
earlier unnecessary CJK rerender and proves the no-edit branch. It does not add
|
||||
a Google-negative oracle result: the available Google account still cannot
|
||||
reproduce a source-positive, Qwen-negative control sequence.
|
||||
|
||||
The earlier automatic rerender was the first variant in the experiment to pass
|
||||
the recorded visual, fidelity, and OpenAI-oracle gates on its two poster
|
||||
fixtures. The later typography-rich control shows that result does not
|
||||
generalize to source-typography preservation. It remains evaluation-only: it
|
||||
depends on verified source text and source line boxes, uses macOS system fonts,
|
||||
and has not been validated on natural scene text, rotated text, or automatic
|
||||
line-box discovery.
|
||||
|
||||
An opt-in `--detect-boxes` follow-up tested automatic geometry. Grouping Paddle
|
||||
word detections by vertical overlap found exactly 20/20, 20/20, and 3/3 lines;
|
||||
mean IoU with verified boxes was 0.857, 0.847, and 1.000. Reusing the annotation
|
||||
crop padding was unstable and preserved only 4/20 dark-poster lines. Reducing
|
||||
vertical recognition padding to 10% restored the exact 8/20 and 7/20 selection
|
||||
decisions, but detector CER was 0.127 and 0.154 rather than 0.123 and 0.119. The
|
||||
dark regression failed the fidelity gate, so these hashes were not submitted to
|
||||
the provider oracle. The flag is retained only to reproduce that negative
|
||||
evaluation and still requires verified strings and an exact line-count match.
|
||||
|
||||
### AnyText2 glyph-conditioned follow-up
|
||||
|
||||
`anytext2-restoration-2026-08-13.csv` tests the official Apache-2.0
|
||||
AnyText2 checkpoint as a local text-editing pass over the exact Qwen Chinese-sign
|
||||
output. The checkpoint hash was verified against ModelScope. Its own edit example
|
||||
successfully replaced a masked blackboard line with the requested `DADDY`, so the
|
||||
runtime and checkpoint were functional before the tracked fixture was measured.
|
||||
|
||||
AnyText2 failed the fidelity gate. The standard full-image detector scored the
|
||||
default edit at CER 0.185 and the source-font mimic at 0.222, against 0.074 for
|
||||
Qwen; image LPIPS moved from 0.289 to 0.338 and 0.345. A padded crop-recognition
|
||||
cross-check removed the detector's punctuation misses: Qwen scored 0.000, the
|
||||
default edit still scored 0.074 with two substituted characters, and the mimic
|
||||
edit stayed at 0.222. Visual inspection agreed, with additional line-shape and
|
||||
punctuation errors in the mimic output. No provider-oracle submission was made
|
||||
because neither candidate preserved the requested content better than its
|
||||
already-clean Qwen input.
|
||||
|
||||
This result covers the published SD1.5 checkpoint, three Chinese lines of no more
|
||||
than 20 characters, one seed, and one fixture. The repository still describes
|
||||
AnyText2XL as unreleased, and the inference wrapper truncates each requested line
|
||||
after 20 characters, so the published model cannot directly cover the longer
|
||||
English and Russian poster lines.
|
||||
|
||||
The padded cross-check also changes the interpretation of this fixture's earlier
|
||||
restoration result. Its Qwen image already contains all three correct Chinese
|
||||
lines; the reported 0.074 came from the detector cropping the final punctuation
|
||||
from two lines. A deterministic rerender with horizontally padded recognition
|
||||
and a mask-matched Hiragino Sans GB W6 font reached detector CER 0.000, but image
|
||||
LPIPS worsened from 0.289 to 0.327 and the font was visibly heavier. The correct
|
||||
policy on this large-text fixture is therefore to leave a Qwen line unchanged
|
||||
when padded recognition confirms the source and candidate strings agree.
|
||||
|
||||
## Face-stage isolation
|
||||
|
||||
`face-stage-isolation-2026-08-13.csv` separates the two stages on the 18-face
|
||||
Gemini portrait-grid fixture. All variants use the same visible-cleaned,
|
||||
metadata-stripped input and seed 0. The prepared input remained positive in the
|
||||
Gemini SynthID verifier, so the surrounding visible and metadata stages did not
|
||||
create the clean verdicts.
|
||||
|
||||
| Route | Gemini oracle | ID cosine | Face LPIPS | Laplacian ratio | Image LPIPS |
|
||||
| --- | --- | ---: | ---: | ---: | ---: |
|
||||
| Prepared control | detected | - | - | - | - |
|
||||
| Z-Image face stage only | detected | 0.927 | 0.028 | 0.612 | 0.013 |
|
||||
| Qwen global stage only | clean | 0.589 | 0.217 | 0.331 | 0.288 |
|
||||
| Full Qwen then Z-Image | clean | 0.852 | 0.050 | 0.492 | 0.252 |
|
||||
|
||||
This isolates the roles: the Qwen global pass removes the watermark, while the
|
||||
masked Z-Image pass restores face identity and detail without reintroducing a
|
||||
detectable whole-image signal. Z-Image face-only does not remove SynthID. The
|
||||
result explains why the face stage succeeded while global Z-Image failed: the
|
||||
earlier clean verdict belonged to the two-stage pipeline, not to Z-Image as a
|
||||
standalone remover. This is one fixture, one seed, and one oracle pass per
|
||||
variant; it does not establish a general reintroduction threshold for mask size.
|
||||
|
||||
## Upstream Synthid-Bypass v2 reproduction
|
||||
|
||||
`upstream-v2-reproduction-2026-08-13.csv` records a source-level audit and a
|
||||
close reproduction of the public
|
||||
[`Synthid-Bypass-v2.0.json`](https://github.com/cebeuq/Synthid-Bypass/blob/3007d0351596ae0a78b7074dae7ad179710b1e48/Synthid-Bypass-v2.0.json).
|
||||
The audited workflow is SHA-256
|
||||
`41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6` at
|
||||
upstream commit `3007d0351596ae0a78b7074dae7ad179710b1e48`.
|
||||
|
||||
Tracing links backward from `SaveImage` confirms that the active global stage is
|
||||
Qwen-Image-2512 Q4 with the Lightning four-step LoRA at 0.8, DiffSynth Canny at
|
||||
1.0, AuraFlow shift 3, `dpmpp_2m` plus `sgm_uniform`, CFG 1, and four steps. Its
|
||||
resolution helper uses adaptive level 6 over 0.08..0.15, which resolves to the
|
||||
0.154 ceiling for the 2816x1536 portrait fixture. Z-Image Turbo is used only by
|
||||
the face detailer: eight steps, CFG 1, `res_2s` plus `bong_tangent`, a 768 px
|
||||
guide, 1024 px cap, and direct adaptive denoise
|
||||
`clamp(0.10 * largest_face_ratio / 0.03, 0.05, 0.28)`. The active face regions
|
||||
come from YOLOv8-face plus SAM. Although the README describes MediaPipe as part
|
||||
of the strict path, the MediaPipe nodes do not reach the saved output. The
|
||||
1.2-megapixel scaler is also bypassed (`mode=4`).
|
||||
|
||||
The published upstream pair 12 changed from SynthID-positive to clean in the
|
||||
Gemini verifier and scored 0.975 face identity. On the project portrait fixture,
|
||||
the prepared control was positive, while both the close global reproduction and
|
||||
the close full reproduction were clean. The global stage scored 0.589 identity
|
||||
and 0.217 face LPIPS; the upstream-strength face stage improved those to 0.783
|
||||
and 0.083. The maintained profile's weaker face pass scored 0.852 and 0.050 on
|
||||
the same control, so copying upstream's roughly doubled face strength would be a
|
||||
quality regression on this fixture.
|
||||
|
||||
The close reproduction is intentionally labeled rather than presented as an
|
||||
exact ComfyUI run. It uses full-precision DiffSynth bf16 weights instead of the
|
||||
Q4 GGUF files, the maintained DiffSynth Lightning scheduler approximation
|
||||
instead of the ComfyUI sampler pair, YuNet plus SAM instead of YOLOv8-face plus
|
||||
SAM, and fixed seed 0 instead of randomized seeds. The two independent oracle
|
||||
controls and the published upstream pair establish the direction of the result;
|
||||
they do not make the local output byte-equivalent to upstream.
|
||||
|
||||
## Compare
|
||||
|
||||
```bash
|
||||
uv run scripts/fidelity_metrics.py compare \
|
||||
--original data/synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png \
|
||||
--variant controlnet=<out>.png \
|
||||
--variant qwen=<out>.png \
|
||||
--variant qwen-zimage=<qwen-out>.png \
|
||||
--variant sdxl-zimage=<sdxl-out>.png \
|
||||
--ocr-langs ""
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
date,source_commit,fixture,variant,model,model_revision,model_sha256,ocr_route,text_cer,img_lpips,ssim,psnr_db,output_sha256,provider_oracle,notes
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,,,,"Paddle ch",0.074,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_run,Exact Qwen baseline reused from text-restoration benchmark
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-default,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.185,0.338,0.642,18.2,87729fe7e0482d2cead6d3b7c7dd386ecc75ab3d8658131c54e67528b8f9feeb,not_run,Fidelity gate failed; padded crop recognition CER 0.074 still shows two substituted characters; locally composited three-line edit; 20 DDIM steps; seed 0
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-anytext2-mimic,AnyText2-SD1.5,b06c583a583818f3679665ef67b51363f107853c,7d5d593928d842500e0da101ab6a1ae28a8d21403441ca62c813c4f92529ace8,"Paddle ch",0.222,0.345,0.644,17.9,f5198d8f2365b79880aba8ea6b0c3885142b9de21c9f7f4f6712ac4a0db8dd85,not_run,Fidelity gate failed; source glyph masks supplied as font hints; 20 DDIM steps; seed 0
|
||||
|
@@ -0,0 +1,5 @@
|
||||
date,source_commit,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,zimage-face-only,0,,0.06604512532552083,18,0.927,0.028,0.612,0.013,0.959,34.7,c388e9bbf737bb9b22a7cb2216ffb8026321ca632f9db29e1e456b7c14a8553f,synthid_detected
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-global-only,0,0.153999996,,18,0.589,0.217,0.331,0.288,0.646,24.2,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_y48j3cy48j3cy48j.png,qwen-zimage-full,0,0.153999996,0.06604512532552083,18,0.852,0.050,0.492,0.252,0.671,24.7,69de191bddbfbf53348c7638bdba661b6d45d257c00a8a2646b2558670c5a4c2,clean
|
||||
|
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
|
||||
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png": "每天都是一个新的机会。\n用微笑开始它。\n世界也会向你微笑。"
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Paddle en+ru+ch",0.378,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.048,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.101,0.120,0.796,19.5,0.054799,15b40b8cdf9e97558c66d4acdcb97d46769e64f04f143f8061746bc731fbbc21,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.081,0.119,0.796,19.5,0.070267,5a8a0bcbe5763d1b1af0d64149cec7e35b16c5751e995d3bacf17581d64177ec,not_checked,fidelity-only follow-up
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.123,0.113,0.808,21.2,0.041279,900def5a433c039aa474789eddc4dc4c63b8126ba1aff6829d2e8d600b9f9c71,no_openai_signals_detected,source detected immediately afterward in the same Chrome sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Paddle en+ru+ch",0.413,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Paddle en+ru+ch",0.086,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective,"Paddle en+ru+ch",0.112,0.103,0.780,20.8,0.054912,846f39db0ad4a48a734a5019f0b0ff5d110eecb3210223407e72529dfa4fcf54,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-expanded-mask,"Paddle en+ru+ch",0.147,0.104,0.779,20.8,0.072556,8ae76e1dedeec26bebb8d35e71fd92d77f142dbb0f3440b330cdf33b79558afe,not_checked,fidelity-only follow-up
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-automatic,"Paddle detector en+ru+ch; padded recognizer selection",0.119,0.104,0.779,20.8,0.057110,ac4201bd2e342b1a37737319438cedb2203f6bb58cd9113f825cf9b80d913755,no_openai_signals_detected,source detected in same browser sequence
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-selective-automatic-passthrough,"Paddle detector ch; horizontal-only padded recognizer selection",0.074,0.289,0.696,23.3,0.000000,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,not_checked,all three source and candidate recognitions agreed exactly; output is byte-identical to candidate
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.127,0.113,0.805,20.9,0.041280,70c52fed8b8a1d98f4cda8558ae08825313e510dfb372e448370d8dcf1da73b0,not_checked,fidelity-only automatic-box follow-up; same 8/20 preserve decisions
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-selective-detected-boxes,"Paddle ch detector boxes; 0.10-height recognizer padding",0.154,0.104,0.778,20.8,0.057146,1fa01ce706042c003a61429d5d97fc6c928ec32278071a91926d588f1e760b00,not_checked,fidelity-only automatic-box follow-up; same 7/20 preserve decisions but worse CER
|
||||
|
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"ChatGPT Image May 31, 2026, 02_02_23 PM.png": [
|
||||
{"box": [156, 206, 709, 296], "text": "This is a longer", "script": "alphabetic"},
|
||||
{"box": [157, 293, 939, 379], "text": "sample text in English.", "script": "alphabetic"},
|
||||
{"box": [165, 393, 774, 434], "text": "Typography can flow smoothly from", "script": "alphabetic"},
|
||||
{"box": [161, 434, 521, 477], "text": "large to medium size.", "script": "alphabetic"},
|
||||
{"box": [165, 495, 566, 522], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
|
||||
{"box": [165, 524, 354, 550], "text": "rhythm, and clarity.", "script": "alphabetic"},
|
||||
{"box": [165, 564, 448, 585], "text": "Even the finest text should remain", "script": "alphabetic"},
|
||||
{"box": [165, 593, 330, 613], "text": "clean and readable.", "script": "alphabetic"},
|
||||
{"box": [158, 673, 903, 740], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
|
||||
{"box": [162, 753, 697, 793], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
|
||||
{"box": [164, 807, 524, 829], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
|
||||
{"box": [163, 845, 483, 868], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
|
||||
{"box": [162, 951, 780, 1009], "text": "Это более длинный пример", "script": "alphabetic"},
|
||||
{"box": [161, 1019, 723, 1062], "text": "текста на русском языке.", "script": "alphabetic"},
|
||||
{"box": [165, 1076, 639, 1104], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
|
||||
{"box": [165, 1111, 505, 1133], "text": "крупного размера к среднему.", "script": "alphabetic"},
|
||||
{"box": [165, 1151, 513, 1172], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
|
||||
{"box": [167, 1178, 287, 1192], "text": "ритм и ясность.", "script": "alphabetic"},
|
||||
{"box": [165, 1210, 417, 1227], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
|
||||
{"box": [165, 1235, 381, 1247], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
|
||||
],
|
||||
"ChatGPT Image May 31, 2026, 02_03_55 PM.png": [
|
||||
{"box": [165, 204, 754, 299], "text": "This is a longer", "script": "alphabetic"},
|
||||
{"box": [167, 295, 1006, 385], "text": "sample text in English.", "script": "alphabetic"},
|
||||
{"box": [171, 401, 831, 446], "text": "Typography can flow smoothly from", "script": "alphabetic"},
|
||||
{"box": [169, 446, 560, 491], "text": "large to medium size.", "script": "alphabetic"},
|
||||
{"box": [173, 507, 605, 536], "text": "Smaller lines help demonstrate hierarchy,", "script": "alphabetic"},
|
||||
{"box": [173, 538, 379, 567], "text": "rhythm, and clarity.", "script": "alphabetic"},
|
||||
{"box": [175, 583, 481, 603], "text": "Even the finest text should remain", "script": "alphabetic"},
|
||||
{"box": [173, 611, 352, 632], "text": "clean and readable.", "script": "alphabetic"},
|
||||
{"box": [170, 693, 956, 757], "text": "这是一段较长的中文示例文本。", "script": "cjk"},
|
||||
{"box": [174, 775, 740, 812], "text": "排版可以从大字号逐渐过渡到中字号。", "script": "cjk"},
|
||||
{"box": [175, 829, 561, 852], "text": "更小的文字能够展示层次、节奏与清晰度。", "script": "cjk"},
|
||||
{"box": [174, 869, 534, 892], "text": "即使是最小的一行,也应该保持清楚易读。", "script": "cjk"},
|
||||
{"box": [169, 958, 824, 1022], "text": "Это более длинный пример", "script": "alphabetic"},
|
||||
{"box": [171, 1029, 764, 1074], "text": "текста на русском языке.", "script": "alphabetic"},
|
||||
{"box": [175, 1090, 678, 1119], "text": "Типографика может плавно переходить от", "script": "alphabetic"},
|
||||
{"box": [175, 1125, 546, 1151], "text": "крупного размера к среднему.", "script": "alphabetic"},
|
||||
{"box": [175, 1169, 552, 1192], "text": "Более мелкие строки показывают иерархию,", "script": "alphabetic"},
|
||||
{"box": [175, 1196, 312, 1217], "text": "ритм и ясность.", "script": "alphabetic"},
|
||||
{"box": [175, 1233, 479, 1253], "text": "Даже самый маленький текст должен", "script": "alphabetic"},
|
||||
{"box": [177, 1259, 440, 1272], "text": "оставаться чистым и читаемым.", "script": "alphabetic"}
|
||||
],
|
||||
"Gemini_Generated_Image_633uuy633uuy633u.png": [
|
||||
{"box": [1281, 650, 2357, 818], "text": "每天都是一个新的机会。", "script": "cjk"},
|
||||
{"box": [1466, 788, 2167, 957], "text": "用微笑开始它。", "script": "cjk"},
|
||||
{"box": [1367, 910, 2264, 1112], "text": "世界也会向你微笑。", "script": "cjk"}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
date,source_commit,fixture,profile,route,seed,strength,adaptive_polish,text_cer,ocr_floor,added_cer,img_lpips,ssim,psnr_db,output_sha256,oracle_rechecked
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.378,0.127,0.251,0.107,0.832,24.4,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.356,0.127,0.229,0.125,0.707,25.3,9bb932d6e66b52f34ff07c3fcdbdb9c33eb601ae2cd84bec6b8c7b5e30df9926,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",original,reference,,,,0.127,0.127,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,visible-invisible-metadata,0,0.11020972541176471,false,0.413,0.127,0.286,0.095,0.818,26.0,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",sdxl-zimage,visible-invisible-metadata,0,0.15,true,0.422,0.127,0.295,0.099,0.754,25.9,7605b1175b0c99f481ae6929acc488f34d98bc0f2571d42a1d2972b671d9b8cb,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,original,reference,,,,0.037,0.037,0.000,0.000,1.000,inf,,not_applicable
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,visible-invisible-metadata,0,0.153999996,false,0.074,0.037,0.037,0.289,0.696,23.3,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,sdxl-zimage,visible-invisible-metadata,0,0.25,true,0.074,0.037,0.037,0.176,0.769,26.1,d9f89465c31bd2d38ffb111775b9bbea4577f40713fcbebf421e05d344ab27ad,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.068,0.127,-0.059,0.042,0.874,31.4,534de6422a3a03cfde44377aa9bf824a8267db3b73511240f8f07fb8dcb95eff,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.086,0.127,-0.041,0.047,0.862,30.8,4dba8211e8d84ebbfff72de00f54d4365d0d202394acd412d1419ae93d1e9f71,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.177,0.127,0.050,0.065,0.838,29.1,fc45c5fff6d33f15d30e05dd02580fe0addd97098b735b2894015f9abdaa43a7,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.248,0.127,0.121,0.093,0.813,27.3,1e55e11f0e289a752098370e9f1d53fed6cd4fab7ceff46fe86dd20c927f9ffb,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.343,0.127,0.216,0.120,0.787,25.7,f5a2d2829e88a790cff5bedc50fca1f878a92cdb392d530afb49d0d95c23b884,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.08,false,0.143,0.127,0.016,0.042,0.877,31.5,5cfa466e276ab37cf532f7458d6dd86ea4f0807e2cbdb6788b7c01ca6ee35a1a,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.10,false,0.101,0.127,-0.026,0.047,0.864,30.9,ade869ec7c5b08098495d9b9d67ee900773f189f94bbb411b043f242f5c2fa37,true
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.15,false,0.208,0.127,0.081,0.060,0.839,29.3,a11976963e3f2ca856ce66278e22064d51345d449fa6d672029039a79517d60a,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.20,false,0.341,0.127,0.214,0.070,0.819,27.6,1ab1ae075565e9d39960a80cf5d61e5b0c50d7e23283493ea0277056b3e964ab,false
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",zimage-global,visible-invisible-metadata,0,0.25,false,0.538,0.127,0.411,0.084,0.799,25.6,5ff5a364d84f738de4c56ead7d4950a2f9a1aaf4bec16e97bda6a65915744a57,false
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.08,false,0.074,0.037,0.037,0.068,0.865,30.8,81419eddc3e5150956859dd10c26dd841bb93d9c8b34c01f344e6d69bca6edbf,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.10,false,0.074,0.037,0.037,0.086,0.844,29.8,4d1f32bd0dac0e05885ff943e7c10230b06de7d8547e6215f8e2941f732a1df5,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.15,false,0.000,0.037,-0.037,0.135,0.793,27.6,526c7987b34896f9513f7439b63b8a87573b59d15930fa307f3bb924435f3cec,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.20,false,0.000,0.037,-0.037,0.179,0.746,25.9,325364cf7a3b5d39ac0c44af82181753ca4d6fde0f1b522c896414191725e8fc,true
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,zimage-global,visible-invisible-metadata,0,0.25,false,0.000,0.037,-0.037,0.218,0.703,24.5,f01d64dc0cabd45d66a93c1be39b819b988f29cda1f0a7bc0d5a1954571d807e,true
|
||||
|
@@ -0,0 +1,7 @@
|
||||
date,source_commit,fixture,variant,ocr_route,text_cer,img_lpips,ssim,psnr_db,mask_fraction,output_sha256,provider_oracle,oracle_control
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.338,0.107,0.832,24.4,,bed68cf79a023cad582a11b41c4c2962bd90a218037ce931a2b3a8697e3bf4bd,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_02_23 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.174,0.703,13.9,0.154,3221b2694f39420d509dd37cdaf69581432f9a189e87397346de0c78762402c6,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage,"Vision en+ru; Paddle ch",0.305,0.095,0.818,26.0,,83e777ad28bf80a8e5aa7d2e6ef2348337c667212e2c71075460352ad7423713,no_openai_signals_detected,source-positive control established earlier
|
||||
2026-08-13,2d03a00,"ChatGPT Image May 31, 2026, 02_03_55 PM.png",qwen-zimage-text-restored,"Vision en+ru; Paddle ch",0.007,0.162,0.682,14.6,0.170,95812399424feb70fc56d3e518d1ae0a3a50a3c2049d5902b00686ede9d3022d,no_openai_signals_detected,qwen-zimage clean in same run
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage,"Paddle ch",0.074,0.289,0.696,23.3,,4b05489c7a79f993d35d32528de792baad5a3e33ee950afa147caf5d3e0621a6,synthid_detected,inconclusive account: clean baseline not reproduced
|
||||
2026-08-13,2d03a00,Gemini_Generated_Image_633uuy633uuy633u.png,qwen-zimage-text-restored,"Paddle ch",0.111,0.328,0.653,18.1,0.051,2ab7666a4161ff0e12c53d9412b23d013970c7613a8a55a1b1ae8a2744e3290d,synthid_detected,inconclusive account: qwen-zimage control also detected
|
||||
|
@@ -0,0 +1,6 @@
|
||||
date,upstream_commit,workflow_sha256,fixture,route,seed,global_strength,face_strength,face_count,id_cos,face_lpips,lapvar_ratio,img_lpips,ssim,psnr_db,output_sha256,gemini_oracle,implementation
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/before/12.png,official-before,,,,10,,,,,,,056276464387c883f5fd89bd2f1415faeb425abb3153cd10efb25d1b9053bdc2,synthid_detected,upstream-published-bytes
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,comparison/after/12.png,official-after,,,,10,0.975,0.093,0.354,0.172,0.820,23.8,a822d805666224adb3e360b3ee89e14f20a341f1e7c4d3a5d3d5251e9286aefe,clean,upstream-published-bytes
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,visible-metadata-control,0,,,18,,,,,,,9f67264beb1da66a710885464a11f2ae85989094149f6145521d302cd2db940b,synthid_detected,project-runtime-control
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-global-close-reproduction,0,0.153999996,,18,0.589,0.217,0.331,0.286,0.647,24.3,281a1d28d786d157cbbd13d00a32b215de4dfb3174a3c5e4e83c843e5402138f,clean,diffsynth-bf16-yunet-sam
|
||||
2026-08-13,3007d0351596ae0a78b7074dae7ad179710b1e48,41911b3b8e01bf51450361dc8beccd84c9513f78ddb160430cdd9bebc294adf6,Gemini_Generated_Image_y48j3cy48j3cy48j.png,upstream-full-close-reproduction,0,0.153999996,0.13209025065104166,18,0.783,0.083,0.470,0.260,0.659,24.7,2d7a8f76f98b905f14d69e5f9e11979610c72c6be7b6ac298dec4659f17bd819,clean,diffsynth-bf16-yunet-sam
|
||||
|
@@ -1,3 +1,3 @@
|
||||
date,source_url,source_sha256,source_width,source_height,source_fps,duration_seconds,source_verdict,vae,noise_std,long_side,fps,seed,output_sha256,output_verdict,output_verdict_text,output_detected_range,track,session_id,stratum,psnr_db,temporal_residual_ratio
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,,,,8,detected_all_frames,,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,,,,8,detected_all_frames,,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,unrecorded,0.10,512,12,0,079165105d4c56e1612091987c08c2627049423025f74c0d4e245fb47c2ff0e3,detected,,,,,,26.2932,1.0072
|
||||
2026-07-31,https://storage.googleapis.com/gdm-deepmind-com-prod-public/media/media/veo__veo-3__off-road.mp4,79a552b9406a079682440c31f14d33a10ba8e1b8b2e96425f5de70f63350299d,1920,1080,24,8,detected_all_frames,unrecorded,0.15,512,12,0,1c4046bcfdead138353b4e2a73339ba227bb5e544878d80c5bc6cd8427c7b00e,not_detected,,,,,,25.3911,1.0578
|
||||
|
@@ -0,0 +1,35 @@
|
||||
# Provenance fixtures
|
||||
|
||||
These files exercise real container, metadata, and pixel-provenance formats.
|
||||
They are test inputs, not evaluation corpora.
|
||||
|
||||
## Adobe TrustMark
|
||||
|
||||
`adobe-trustmark-p.png` is Adobe's official `images/ghost_P.png` Variant P
|
||||
example from [`adobe/trustmark`](https://github.com/adobe/trustmark) at commit
|
||||
`0ed40cbe8188f664fd9cbbeacd969807de27440a`.
|
||||
|
||||
- Source: `https://raw.githubusercontent.com/adobe/trustmark/0ed40cbe8188f664fd9cbbeacd969807de27440a/images/ghost_P.png`
|
||||
- SHA-256: `e58c5825ed7e5d9fb04710ea541b61bd55879cad65554c0d46260aa24b3d0755`
|
||||
- Expected signal: TrustMark Variant P, schema 1
|
||||
- License: MIT, reproduced in `../licenses/adobe-trustmark-MIT.txt`
|
||||
|
||||
|
||||
## Ideogram C2PA attribution
|
||||
|
||||
`ideogram-c2pa.png` is a SYNTHETIC signed manifest, not a vendor download: a
|
||||
64x64 flat PNG carrying a C2PA claim (`trainedAlgorithmicMedia`,
|
||||
`claim_generator "Ideogram/3.0"`) signed by a throwaway test CA whose leaf
|
||||
certificate subject is `O=Ideogram, Inc, CN=Ideogram, Inc`. It exercises the
|
||||
issuer-to-platform ATTRIBUTION path end to end through the real reader; the
|
||||
signer is deliberately untrusted (no trust anchors ship), which is also the
|
||||
fixture's second assertion: attribution and signer trust are separate
|
||||
dimensions.
|
||||
|
||||
- Expected signal: C2PA, platform `Ideogram`, `ai_from_metadata` True
|
||||
- Expected signer: `untrusted` (test CA; attribution must not imply trust)
|
||||
- Recipe (no keys are committed): create a CA and an ES256 leaf with
|
||||
`digitalSignature` key usage and subject `CN=Ideogram, Inc`, then
|
||||
`c2pa.Builder.from_json(...).sign_file(png, out, Signer.from_info(
|
||||
C2paSignerInfo(b"es256", leaf_pem + ca_pem, pkcs8_key, None)))`.
|
||||
- No user upload and no Ideogram artifact contributes pixels or bytes.
|
||||
|
After Width: | Height: | Size: 360 KiB |
|
After Width: | Height: | Size: 43 KiB |
@@ -0,0 +1,41 @@
|
||||
# Visible-mark example gallery
|
||||
|
||||
One committed example per registered visible mark, so the repository carries a
|
||||
working sample of everything it supports. `tests/test_visible_examples.py` holds
|
||||
both sides to it: a mark registered without an example fails the suite, and so
|
||||
does an engine that stops detecting its own example.
|
||||
|
||||
## What these files are
|
||||
|
||||
Every example is SYNTHETIC: `scripts/render_visible_examples.py` composites the
|
||||
mark's committed silhouette (the same font-rendered asset the detector matches)
|
||||
onto a deterministic generated base photo at the engine's measured geometry.
|
||||
The silhouettes are repository-owned font renders generated by
|
||||
`scripts/render_vendor_silhouettes.py`; the gallery does not copy vendor raster
|
||||
assets.
|
||||
|
||||
The examples demonstrate DETECTION geometry and house style, not vendor raster
|
||||
fidelity; real-world variants (fonts, opacities, sizes) are covered by the
|
||||
engines' recorded calibration evidence.
|
||||
|
||||
## Regeneration
|
||||
|
||||
uv run python scripts/render_visible_examples.py
|
||||
|
||||
The generator self-verifies: it fails (exit 1) if any registered mark does not
|
||||
detect on its own example, so regeneration is the fix point for drift.
|
||||
|
||||
## Layout
|
||||
|
||||
<mark-key>/example.png 1536x2048..2048x2048 PNG, one per image mark
|
||||
<mark-key>/example.mp4 960x540 90-frame clip, one per video mark
|
||||
(kling carries both: it is registered in both registries)
|
||||
|
||||
Special cases: `gemini` composites the sparkle alpha map at the provider's
|
||||
configured position; `jimeng_pill` is the capture-less pill at the measured
|
||||
3:4 portrait geometry; `microsoft` is the opaque white pill with dark text
|
||||
holes (the discriminator its detector keys on). Video examples composite the
|
||||
detector's own synthetic template on every frame; where two marks share a
|
||||
shape family the example carries the discriminative variant (`veo` the legacy
|
||||
text form, `kling` the logo-plus-wordmark pair flush to the edge), because the
|
||||
temporal selection resolves cross-template ties by table order.
|
||||
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 2.6 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
@@ -0,0 +1,25 @@
|
||||
Copyright 2023 Adobe
|
||||
All Rights Reserved.
|
||||
|
||||
NOTICE: Adobe permits you to use, modify, and distribute this file in
|
||||
accordance with the terms of the license agreement accompanying it.
|
||||
|
||||
MIT License
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -54,7 +54,10 @@ versions, or runtime settings.
|
||||
Ground-truth quality, strongest first:
|
||||
|
||||
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
|
||||
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
|
||||
- `openai-verify` — checked through the OpenAI web verifier. This is the
|
||||
historical manifest label for existing rows. OpenAI now also documents a
|
||||
Content Provenance API with a separate SynthID result; add a distinct manifest
|
||||
value before recording API-derived labels rather than silently reusing this one.
|
||||
- `synthid-portal` — checked via Google's SynthID Detector portal.
|
||||
- `c2pa-metadata` — supported provenance evidence (Google AI C2PA, or OpenAI
|
||||
C2PA with an explicit `c2pa.watermarked.*` action). Weaker than a provider
|
||||
@@ -160,7 +163,7 @@ sparkle; the SynthID *pixel* itself has no local detector (oracle only).
|
||||
|
||||
Key takeaways:
|
||||
- The same model differs by *surface*: Gemini app wraps C2PA, AI Studio (API/playground) emits none -- only the pixel + sparkle survive.
|
||||
- Microsoft Designer's DALL-E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL-E).
|
||||
- Microsoft Designer's DALL·E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL·E).
|
||||
- Meta uses the IPTC `digitalSourceType` marker, not C2PA or SynthID.
|
||||
- The open imwatermark fires only on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL) -- not from re-hosts (Krea, Stability hosted SDXL) or re-encoded design exports (Recraft, Canva). Ideogram's only signal is the EXIF `Make` tag.
|
||||
- Bing and Grok web UIs are uncooperative for autonomous capture (no document_idle for screenshots; blob downloads intermittently no-op; low-res in-chat previews). Use their native download button manually if a full-res sample is needed.
|
||||
@@ -0,0 +1,91 @@
|
||||
# Private SynthID research manifest
|
||||
|
||||
The research manifest is a local CSV for detector and pixel-only removal
|
||||
experiments. It is intentionally separate from `manifest.csv`, which describes
|
||||
the small public regression corpus.
|
||||
|
||||
Store the CSV and its media below `.local-eval/synthid/`. Keep one manifest per
|
||||
detector target, for example `google-manifest.csv` and `openai-manifest.csv`.
|
||||
Separate manifests may reference the same external negative bytes without
|
||||
duplicating an artifact inside either provider's split graph. Do not commit
|
||||
private media, corpus sizes, oracle sessions, or provider-access details.
|
||||
|
||||
Before assigning labels, build a label-free inventory. It records byte and
|
||||
decoded-pixel hashes, exact duplicates, geometry, and format, but deliberately
|
||||
has no provider, oracle, outcome, or split columns:
|
||||
|
||||
```bash
|
||||
uv run python scripts/synthid_research_inventory.py \
|
||||
--root .local-eval/synthid negatives google openai \
|
||||
--inventory-out .local-eval/synthid/inventory.csv
|
||||
```
|
||||
|
||||
An inventory row becomes a manifest row only after its source, detector target,
|
||||
label evidence, group lineage, and split are established independently. A
|
||||
directory name is not evidence and must not be promoted mechanically.
|
||||
|
||||
Audit each local manifest with:
|
||||
|
||||
```bash
|
||||
uv run python scripts/synthid_research_manifest.py \
|
||||
.local-eval/synthid/google-manifest.csv --verify-files
|
||||
```
|
||||
|
||||
Once a provider manifest has ordinary positives and negatives in train,
|
||||
validation, and locked-test splits, run the D1 confound challenge:
|
||||
|
||||
```bash
|
||||
uv run --extra pixels python scripts/synthid_confound_probe.py \
|
||||
.local-eval/synthid/google-manifest.csv \
|
||||
--target-provider google \
|
||||
--report-out .local-eval/synthid/google-d1-confounds.json
|
||||
```
|
||||
|
||||
D1 excludes candidate, sham, and source-control rows. Its `container`,
|
||||
`thumbnail`, and `canonical` baselines intentionally measure how well export
|
||||
fields, geometry, and coarse generator/content style can imitate detection. A
|
||||
report is evidence-ready only when the locked test includes a same-provider hard
|
||||
negative and the temporal split contains both labels. Evidence readiness does
|
||||
not mean the D1 gate passed; a candidate signal still has to beat the frozen
|
||||
canonical baseline on those controls.
|
||||
|
||||
## Columns
|
||||
|
||||
| Column | Meaning |
|
||||
| --- | --- |
|
||||
| `artifact_sha256` | SHA-256 of the exact file submitted or measured. |
|
||||
| `pixel_sha256` | SHA-256 of decoded RGB bytes, used to catch lossless duplicates. |
|
||||
| `artifact_path` | Safe path relative to the manifest. |
|
||||
| `parent_sha256` | Exact parent artifact for a derivative, empty for an original. |
|
||||
| `group_id` | Leakage boundary shared by an original and all semantic or transformed siblings. |
|
||||
| `target_provider` | Detector target, `openai` or `google`. |
|
||||
| `source_provider` | Actual source family: `openai`, `google`, `camera`, `other_ai`, `synthetic`, or `editor`. |
|
||||
| `surface` | Product or export surface. |
|
||||
| `model_epoch` | Model family plus a dated epoch when the exact version is unavailable. |
|
||||
| `generation_session` | Groups outputs created in one provider session. |
|
||||
| `content_stratum` | Predeclared content class. |
|
||||
| `width`, `height`, `format` | Decoded geometry and file format. |
|
||||
| `transform` | `original` or the reproducible transform applied to `parent_sha256`. |
|
||||
| `split` | `discovery`, `train`, `validation`, `test`, or `temporal`. |
|
||||
| `c2pa_outcome` | C2PA result, recorded independently from the pixel signal. |
|
||||
| `synthid_outcome` | `detected`, `not_detected`, `indeterminate`, `refused`, or `not_checked`. |
|
||||
| `verified_via` | Matching provider oracle, external source evidence, or `none`. |
|
||||
| `evidence_reference` | Stable URL or local evidence-record reference required for `source-evidence`; never infer it from a directory name. |
|
||||
| `oracle_session` | Groups allowed checks made in one session. |
|
||||
| `oracle_role` | `ordinary`, `source_control`, `candidate`, or `sham`. |
|
||||
| `captured_at`, `oracle_checked_at` | Timezone-aware ISO-8601 timestamps. |
|
||||
| `notes` | Verbatim context that does not fit a structured column. |
|
||||
|
||||
`source-evidence` may establish an ordinary external negative, but never a
|
||||
positive or a same-provider negative. Provider positives and same-provider hard
|
||||
negatives require the matching provider verifier. An `indeterminate`, `refused`,
|
||||
or unchecked row may remain in `discovery`; it cannot enter a train, validation,
|
||||
test, or temporal split. Every `source-evidence` row must retain the source URL
|
||||
or evidence-record reference that establishes the claim.
|
||||
|
||||
The auditor also rejects duplicate artifact hashes, identical decoded pixels in
|
||||
different groups, derivatives without parents, cross-provider parentage,
|
||||
lineage cycles, and any group crossing split boundaries. A `not_detected`
|
||||
candidate or sham is valid only when the same provider, group, and oracle
|
||||
session contains a detected `source_control`. A session that misses its control
|
||||
cannot establish removal, even when its candidate response is negative.
|
||||
|
Before Width: | Height: | Size: 3.6 MiB After Width: | Height: | Size: 5.8 MiB |
@@ -0,0 +1,550 @@
|
||||
# Chroma1 engine research (2026-08-29/30)
|
||||
|
||||
> Research archive. This page records experiments and decisions from the dates
|
||||
> above. Current package behavior is defined by the
|
||||
> [supported signals](supported-signals.md), [known limitations](known-limitations.md),
|
||||
> and [module internals](module-internals.md). Nothing here is shipped.
|
||||
|
||||
Cited experiment behind issue #88 ("Please integrate Flux 2"): FLUX.2 has no
|
||||
native strength img2img anywhere (BFL reference code or diffusers; its image
|
||||
inputs are reference conditioning and outputs start from pure noise), so the
|
||||
requestable scrub lane in the FLUX family is **Chroma1** -- the FLUX.1
|
||||
architecture re-trained on open data, Apache-2.0 on weights, with a native
|
||||
`ChromaImg2ImgPipeline` strength parameter. This page records what was measured
|
||||
before any integration decision.
|
||||
|
||||
## Setup
|
||||
|
||||
All Chroma1 runs: `lodestones/Chroma1-HD` bf16 on Modal H100, seed 0, the
|
||||
neutral scrub prompt (`high quality, sharp, detailed, faithful to the original`
|
||||
/ `blurry, lowres, distorted text, garbled text, artifacts`), guidance 5.0, and
|
||||
**step compensation**: Diffusers truncates the step COUNT (`int(steps *
|
||||
strength)`), so the requested count is scaled to always spend 4 effective
|
||||
denoising steps, the same semantics as `sdxl_zimage_pipeline.requested_steps`.
|
||||
Without it, strength 0.05 runs a single step and the floor reads artificially
|
||||
high (the first, uncompensated sweep put the OpenAI boundary at (0.05, 0.10];
|
||||
compensated it is (0.05, 0.06]).
|
||||
|
||||
Two Flux-family traps found while wiring the prototype:
|
||||
|
||||
- without explicit `width`/`height`, the img2img pipeline silently generates
|
||||
1024x1024 regardless of input aspect;
|
||||
- dimensions floor to the /16 latent-patch grid, so the result must be resized
|
||||
back to the input size for pixel-exact comparison.
|
||||
|
||||
Scripts: `scripts/chroma_scrub_prototype.py` (first sweep),
|
||||
`scripts/engine_quality_price_probe.py` (head-to-head with the production
|
||||
qwen-zimage path), `scripts/engine_cohort_calibration.py` (the cohort ladders
|
||||
below), `scripts/chroma_meta_high_rungs.py` (the Meta rungs above 0.08),
|
||||
`scripts/chroma_muse_expansion.py` (harvest-2 Meta API fixtures),
|
||||
`scripts/chroma_openai_expansion.py` (extra OpenAI SynthID carriers),
|
||||
`scripts/chroma_seed_probes.py` (seed-1/2 at measured first-cleans).
|
||||
Generated outputs stay outside the repository (`out/`, gitignored); the durable
|
||||
verdicts are the tables on this page.
|
||||
|
||||
## Measured floors
|
||||
|
||||
Oracle per cohort: OpenAI via openai.com/verify (SynthID read separately from
|
||||
C2PA); Meta via the anonymous meta.ai/identification oracle. Each fixture's
|
||||
rungs were checked ascending and stopped at the first clean verdict.
|
||||
|
||||
| cohort | fixture | detected at | clean from |
|
||||
|---|---|---|---|
|
||||
| OpenAI | typography card 1122x1402 | 0.05 | 0.06 |
|
||||
| OpenAI | 9-face grid 1448x1086 | 0.06 | 0.075 |
|
||||
| OpenAI | full-pipeline quality fixture 1122x1402 | 0.05 | 0.06 |
|
||||
| Meta | gen_lighthouse_watercolor 1600x1600 | 0.08 | 0.10 |
|
||||
| Meta | gen_fox_forest 1600x1600 | 0.06 | 0.08 |
|
||||
| Meta | gen_night_city 1920x1280 | 0.045 | 0.06 |
|
||||
| Meta | gen_studio_mug 1600x1600 | 0.03 | 0.045 |
|
||||
| Meta | gen_text_poster 1600x1600 | -- | <=0.03 |
|
||||
|
||||
Derived operating points by the shipped rule (worst first-clean boundary plus
|
||||
one observed cross-source spread):
|
||||
|
||||
- **OpenAI: 0.09** (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and
|
||||
sdxl-zimage 0.15. The typography boundary (0.05, 0.06] is seed-stable across
|
||||
seeds 0, 1, 2.
|
||||
- **Meta: 0.17** (0.10 + (0.10 - 0.03)), oracle-verified clean on the worst
|
||||
fixture at 0.17. Against qwen-zimage 0.1. Chroma needs ~1.7x qwen's strength
|
||||
on Content Seal.
|
||||
- **Google: 0.40, measured 2026-08-30 across all four fixtures.** A
|
||||
metadata-stripped original verified as pixel-DETECTED through the @synthid
|
||||
Verify AI agent (the C2PA-carrying original answers about C2PA instead,
|
||||
which is not a pixel verdict).
|
||||
|
||||
| fixture | detected at | clean from |
|
||||
|---|---|---|
|
||||
| Gemini_Generated_Image_633uuy | 0.20 | 0.25 |
|
||||
| Gemini_Generated_Image_akdbei | 0.20 | 0.25 |
|
||||
| Gemini_Generated_Image_y48j3c | 0.08 | 0.12 |
|
||||
| Gemini_Generated_Image_3mc4t9 | 0.08 | 0.12 |
|
||||
|
||||
Derived by the shipped rule: worst first-clean 0.25 + spread (0.25 - 0.12
|
||||
= 0.13) = 0.38, rounded up to the measured rung **0.40**, then
|
||||
oracle-verified clean on the worst fixture. Against qwen-zimage 0.27:
|
||||
Chroma needs ~1.5x the strength on Google SynthID. At 0.15-0.20 the tool
|
||||
still reads positive while its wording degrades from "most or all" to
|
||||
"part of", so the boundary band is real signal decay, not noise.
|
||||
|
||||
Oracle mechanics learned while bracketing: the quota is per ACCOUNT and
|
||||
additionally dedupes SIMILAR images ("quota for checking images similar to
|
||||
this one"), roughly three checks per account window; switching accounts
|
||||
inside one browser works. A fresh Playwright context with the same Google
|
||||
session cookies (exported browser-wide via CDP `Storage.getCookies`, never
|
||||
through the transcript) carries working sign-in and its own check budget.
|
||||
The agent's own instructions leak into the answer on some renders and
|
||||
include an abstention rule for text-dominated images - always read the TOOL
|
||||
outcome, never the prose.
|
||||
- **Microsoft: 0.125**, measured 2026-08-30 on the SAME three valid Paint
|
||||
carriers behind the qwen floor (staged under
|
||||
`out/cohort-calibration/microsoft/` with pixel-identical metadata-stripped
|
||||
controls; they are real raiw-corpus user uploads and are never committed).
|
||||
Controls re-verified `Microsoft AI detected` on the day of the run. Against
|
||||
qwen-zimage 0.15: Chroma needs LESS strength on InvisMark.
|
||||
|
||||
| source (Paint) | detected at | clean from | qwen first-clean |
|
||||
|---|---|---|---|
|
||||
| paint-1 (47d87e99, 1536x1024) | 0.06 | 0.08 | 0.055 |
|
||||
| paint-2 (a579fb70, 1536x1024) | -- | <=0.04 | 0.04125 |
|
||||
| paint-3 (883191b5, worst qwen source) | 0.06 | 0.08 | 0.095 |
|
||||
|
||||
Derived by the shipped rule: 0.08 + (0.08 - 0.04) = 0.12, rounded to the
|
||||
measured rung **0.125**, which was then oracle-verified clean on both worst
|
||||
sources. Oracle: the public `ai.azure.com/nextgen/validate` page; its
|
||||
`Inconclusive` verdict is the site's watermark-negative, weaker than an
|
||||
API-level `Watermark: false`, the same caveat the qwen cohort carries.
|
||||
|
||||
## Fidelity at the floors (scripts/fidelity_metrics.py)
|
||||
|
||||
qwen-zimage rows ran through the package's own `QwenZImagePipeline` on the same
|
||||
H100 (production path, both stacks resident, prompt cache warm).
|
||||
|
||||
OpenAI typography (chroma 0.06 vs qwen 0.07675):
|
||||
|
||||
| | CER | img LPIPS | SSIM | PSNR |
|
||||
|---|---|---|---|---|
|
||||
| qwen-zimage | 0.241 | 0.083 | 0.834 | 27.7 |
|
||||
| chroma1 | **0.138** | **0.049** | **0.881** | **31.2** |
|
||||
|
||||
OpenAI full-pipeline fixture (chroma 0.09 vs qwen 0.07675): chroma wins every
|
||||
metric (CER 0.155 vs 0.259, LPIPS 0.056 vs 0.094, PSNR 29.6 vs 26.1).
|
||||
|
||||
OpenAI 9-face grid (chroma 0.075 global-only vs qwen 0.07675 with its Z-Image
|
||||
face stage):
|
||||
|
||||
| | ID cos | face LPIPS | img LPIPS | SSIM | PSNR |
|
||||
|---|---|---|---|---|---|
|
||||
| qwen-zimage | **0.857** | **0.051** | 0.065 | 0.865 | 31.1 |
|
||||
| chroma1 (no face stage) | 0.788 | 0.067 | **0.060** | **0.890** | **32.4** |
|
||||
|
||||
The face stage regenerates from the ORIGINAL crops and is engine-independent,
|
||||
so an integrated `chroma-zimage` would composite the same faces over the
|
||||
chroma global; whether that closes the identity gap is an integration-time
|
||||
measurement, not a given.
|
||||
|
||||
Meta at the respective floors (chroma 0.17 vs qwen 0.1): qwen wins perceptual
|
||||
fidelity on textured content (LPIPS: fox 0.163 vs 0.211, lighthouse 0.173 vs
|
||||
0.248, night city 0.187 vs 0.277), chroma wins the text poster decisively
|
||||
(LPIPS 0.008 vs 0.012, CER 0.000 vs 0.021) and the studio mug is a split. The
|
||||
extra 0.07 of strength costs real fidelity on photoreal content.
|
||||
|
||||
Google at the respective floors (chroma 0.40 vs qwen 0.27, qwen row includes
|
||||
its Z-Image face stage):
|
||||
|
||||
| fixture | qwen LPIPS/SSIM/ID cos | chroma LPIPS/SSIM/ID cos |
|
||||
|---|---|---|
|
||||
| 633uuy | **0.405 / 0.577** / - | 0.499 / 0.576 / - (CER 1.000 vs 0.333) |
|
||||
| akdbei | **0.440 / 0.552** / - | 0.540 / 0.557 / - (CER 0.796 vs 0.367) |
|
||||
| y48j3c (18 faces) | **0.436 / 0.608 / 0.801** | 0.560 / 0.548 / 0.279 |
|
||||
| 3mc4t9 (5 faces) | **0.422 / 0.529 / 0.921** | 0.562 / 0.461 / 0.143 |
|
||||
|
||||
qwen wins every axis that matters on every fixture; at 0.40 the Chroma global
|
||||
regeneration itself collapses face identity (0.14-0.28 cosine) and destroys
|
||||
dense text (CER up to 1.0). The extra 0.13 of strength is simply too much
|
||||
regeneration. This is the Meta pattern amplified.
|
||||
|
||||
Microsoft at the respective floors (chroma 0.125 vs qwen 0.15): chroma wins
|
||||
EVERY metric on EVERY source while scrubbing at lower strength --
|
||||
|
||||
| source | CER q/ch | img LPIPS q/ch | SSIM q/ch | PSNR q/ch |
|
||||
|---|---|---|---|---|
|
||||
| paint-1 | 0.741 / 0.241 | 0.164 / 0.106 | 0.742 / 0.847 | 24.8 / 29.3 |
|
||||
| paint-2 | 0.826 / 0.391 | 0.253 / 0.108 | 0.536 / 0.633 | 22.5 / 25.5 |
|
||||
| paint-3 | 0.000 / 0.000 | 0.038 / 0.020 | 0.762 / 0.953 | 36.3 / 42.2 |
|
||||
|
||||
The Microsoft cohort therefore behaves like the OpenAI cohort, not like Meta:
|
||||
for InvisMark carriers Chroma1 is the better engine at a lower operating point.
|
||||
|
||||
## Price (warm H100 container)
|
||||
|
||||
| | qwen-zimage | chroma1 |
|
||||
|---|---|---|
|
||||
| seconds per image | 2.44 | 3.62 |
|
||||
| VRAM peak | 31.9 GiB | 29.2 GiB |
|
||||
| USD per 1000 images ($3.95/h) | $2.68 | $3.97 |
|
||||
| weights in cache | ~70 GB | ~28 GB |
|
||||
|
||||
The chroma number includes a T5 encode per call; the shipped qwen path caches
|
||||
prompt embeddings, and the same treatment would put chroma near 1.5-2 s/image.
|
||||
Both stacks fit an 80 GB card; chroma's smaller footprint likely fits smaller
|
||||
cards untested.
|
||||
|
||||
## Matched-strength comparison: the floors hide the engine (2026-08-30 addendum)
|
||||
|
||||
The floor-based tables above compare chroma at its DERIVED floor (worst case
|
||||
+ spread) against qwen at its shipped floor - different perturbation levels.
|
||||
At the strength where each fixture ACTUALLY turns clean (chroma first-clean
|
||||
rungs, all oracle-verified above), the picture reverses:
|
||||
|
||||
Google (chroma@first-clean vs qwen@0.27):
|
||||
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | CER q/ch | ID q/ch |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 633uuy | 0.25 | 0.405/**0.375** | 0.577/**0.673** | 18.8/**22.3** | **0.333**/0.667 | - |
|
||||
| akdbei | 0.25 | 0.440/**0.409** | 0.552/**0.650** | 18.6/**22.5** | 0.367/**0.163** | - |
|
||||
| y48j3c | 0.12 | 0.436/**0.329** | 0.608/**0.700** | 21.0/**25.8** | 1.0/1.0 | **0.801**/0.611 |
|
||||
| 3mc4t9 | 0.12 | 0.422/**0.318** | 0.529/**0.644** | 17.7/**21.3** | 0.443/**0.229** | **0.921**/0.410 |
|
||||
|
||||
Meta (chroma@first-clean vs qwen@0.1):
|
||||
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | CER q/ch |
|
||||
|---|---|---|---|---|---|
|
||||
| lighthouse | 0.10 (equal) | 0.173/**0.097** | 0.783/**0.886** | 25.6/**30.8** | - |
|
||||
| fox | 0.08 | 0.163/**0.083** | 0.778/**0.907** | 25.6/**31.4** | - |
|
||||
| night_city | 0.06 | 0.187/**0.092** | 0.748/**0.893** | 23.9/**29.6** | 0.850/**0.250** |
|
||||
| studio_mug | 0.045 | 0.024/**0.009** | 0.984/**0.992** | **39.0**/37.8 | - |
|
||||
| text_poster | 0.03 | 0.012/**0.002** | 0.984/**0.994** | 34.0/**41.2** | 0.021/**0.007** |
|
||||
|
||||
At effective-clean strength Chroma1 wins whole-image fidelity on essentially
|
||||
every fixture (often by 4-6 dB) and text on five of six; qwen's remaining
|
||||
edge is face identity, which its Z-Image face stage buys (0.80-0.92 vs
|
||||
0.41-0.61 cosine) and a chroma-zimage profile would inherit on top of its own
|
||||
global pass.
|
||||
|
||||
**Why the floors still matter:** a shipped strength must clear the worst
|
||||
unseen content, and Chroma1's per-fixture boundaries scatter much wider
|
||||
(Google 0.12-0.25, Meta 0.03-0.10) than the spread the flat floors were built
|
||||
to absorb - so its derived floors (0.40 / 0.17) pay a large margin penalty
|
||||
and at those floors the extra strength genuinely destroys quality. The
|
||||
engines differ less in "quality per strength" than in how predictable their
|
||||
boundary is. The integration lever this exposes: a content-adaptive strength
|
||||
policy for chroma-zimage (the project already ships resolution-adaptive
|
||||
denoise for qwen) could capture chroma's low per-content boundaries; the two
|
||||
hard Google fixtures are both dense-text cards and the easy ones are
|
||||
photoreal, so a content-class signal may predict the rung.
|
||||
|
||||
## Pre-ship validation (2026-08-30)
|
||||
|
||||
With the `chroma-zimage` profile implemented
|
||||
(`_internal/chroma_zimage_pipeline.py` on `TwoStageZImagePipeline`, floors in
|
||||
`watermark_profiles`), two ship-gate checks were run:
|
||||
|
||||
1. **Seed stability at the worst Meta boundary.** lighthouse @ 0.10 with
|
||||
seeds 1 and 2: both CLEAN on the meta.ai oracle. The Meta boundary is not a
|
||||
seed-0 fluke. The Google seed probes (633uuy @ 0.25) delivered a critical
|
||||
finding: **seed 0 was CLEAN but seed 1 was DETECTED** -- the 633uuy
|
||||
boundary is seed-dependent, which validates the flat 0.40 floor's 0.15
|
||||
margin exactly the way the qwen SDXL precedent did (the same image at the
|
||||
same strength passing or failing run-to-run near the threshold). Seed 2
|
||||
of `seedprobe_google_633uuy_s0.25_seed2.png` was checked twice on
|
||||
2026-08-31 after a human drag-drop. Work Gemini (u/2, Pro,
|
||||
`https://gemini.google.com/u/2/app/ebc7c19d22f85bb9`) with the
|
||||
ordinary question "Is there a SynthID watermark in this image?"
|
||||
auto-titled "SynthID Watermark Detection Results" and returned "No
|
||||
reliable signals were detected indicating how the content was created,
|
||||
and you should look for other supporting evidence from trusted
|
||||
sources." Plus `@synthid` (u/0,
|
||||
`https://gemini.google.com/u/0/app/658b1470307811b7`) with the built-in
|
||||
verification question auto-titled "SynthID Image Verification Results"
|
||||
and returned "No digital watermark from Google AI was detected in this
|
||||
image, indicating that it was not created or edited using Google's AI
|
||||
models." Both are CLEAN. An earlier Plus `@synthid` attempt on the
|
||||
same file hung on "Connecting to Verify AI" with auto-title "SynthID
|
||||
Image Verification Failed"; Flash prose abstained because "the tool
|
||||
response does not confirm" Google AI, which is not a tool verdict.
|
||||
Workspace Gemini's More-uploads menu does not list Photos, but a
|
||||
human drag-drop of a local PNG does attach; one Workspace account in
|
||||
the same Chrome has no Gemini app. Seed 0 CLEAN / seed 1 DETECTED /
|
||||
seed 2 CLEAN at 0.25
|
||||
leaves the 633uuy boundary seed-dependent on the Plus `@synthid`
|
||||
path; the 0.40 floor is already established by the seed-1 flip.
|
||||
2. **Full-path face-stage interaction.** The complete profile run (Chroma1
|
||||
global at the shipped Google floor 0.40, then the inherited
|
||||
YuNet/SAM/Z-Image face stage) on the 5-face fixture 3mc4t9: CLEAN on the
|
||||
@synthid oracle. The composited face regions do not re-introduce a
|
||||
detectable signal on top of the floor.
|
||||
|
||||
Generation script: `scripts/chroma_preship_validation.py`; outputs under
|
||||
`out/preship-chroma/` (gitignored).
|
||||
|
||||
## Verdict
|
||||
|
||||
Engine quality is cohort-dependent, and after measuring all four cohorts the
|
||||
split is even and sharp:
|
||||
|
||||
- On OpenAI-provenance content Chroma1 is better at every measured fidelity
|
||||
axis at its floor (0.09 vs 0.07675), for comparable money.
|
||||
- On Microsoft InvisMark content Chroma1 is better at every measured fidelity
|
||||
axis at a LOWER floor than qwen (0.125 vs 0.15).
|
||||
- On Meta Content Seal Chroma1 needs 1.7x the strength (0.17 vs 0.1) and loses
|
||||
LPIPS on textured content at its floor.
|
||||
- On Google SynthID Chroma1 needs ~1.5x the strength (0.40 vs 0.27) and loses
|
||||
every fidelity axis on every fixture, with face identity collapsing at 0.40
|
||||
(0.14-0.28 cosine vs qwen's 0.80-0.92).
|
||||
|
||||
A single-engine `chroma-zimage` default at a flat per-vendor floor is NOT
|
||||
supported: Google and Meta floors land too high and the quality at those
|
||||
floors is worse than qwen's. But the matched-strength addendum changes what
|
||||
IS supported: at the strength each image actually needs, Chroma1 regenerates
|
||||
better than qwen almost everywhere except face identity (which the shared
|
||||
face stage would supply). The two honest integration options:
|
||||
|
||||
1. Per-cohort engine router at flat floors: chroma for OpenAI/Microsoft,
|
||||
qwen for Google/Meta - conservative, ships on the measurements as they
|
||||
stand.
|
||||
2. `chroma-zimage` with a content-adaptive strength policy that predicts the
|
||||
per-image rung - potentially better everywhere, but the predictor is new
|
||||
research (the measured signal so far: dense-text content needs the high
|
||||
rungs, photoreal the low ones) and needs its own oracle validation before
|
||||
any ship.
|
||||
|
||||
Both need the seed-sensitivity and face-stage composition checks called out
|
||||
above before shipping. Both checks have now been run (Pre-ship validation
|
||||
above): the Meta boundary is seed-stable across seeds 0-2 and the full
|
||||
profile path stays clean through the face stage on the shipped Google floor.
|
||||
The profile ships with option 1's floors plus a content-adaptive Google arm
|
||||
(face content resolves to 0.125 instead of 0.40, keyed on YuNet detection).
|
||||
Option 2's full adaptive policy for the remaining cohorts remains the
|
||||
documented follow-up. The Meta arm was measured on 2026-08-30/31 and
|
||||
does not have a shippable split; see the expansion section below.
|
||||
|
||||
## Meta content-adaptive expansion (2026-08-30/31)
|
||||
|
||||
Goal: test whether `flat_ratio` (fraction of 16x16 blocks with luma std < 8)
|
||||
predicts Chroma1's first-clean Content Seal boundary well enough to ship a
|
||||
Meta arm analogous to Google's face-count split.
|
||||
|
||||
**Harvest 1, from a 495-file spaces dump, was not a diverse expansion.**
|
||||
Seven "unique" files were staged under `out/cohort-calibration/meta/muse-*`
|
||||
(gitignored). After control checks on `meta.ai/identification`:
|
||||
|
||||
| fixture | control | first-clean | note |
|
||||
|---|---|---|---|
|
||||
| muse-1 | CLEAN | -- | seal did not survive the metadata-stripped control |
|
||||
| muse-2 | DETECTED | (0.08, 0.10] | photoreal portrait; 0.06 and 0.08 DETECTED, 0.10 CLEAN |
|
||||
| muse-3 | DETECTED | (0.015, 0.03] | photoreal portrait, different subject |
|
||||
| muse-4 | DETECTED | (0.08, 0.10] | same subject as muse-2/6/7; 0.06 and 0.08 DETECTED, 0.10 CLEAN |
|
||||
| muse-5 | -- | -- | byte-identical to committed `gen_lighthouse_watercolor.webp` |
|
||||
| muse-6 | DETECTED | (0.06, 0.08] | same subject as muse-4; 0.06 DETECTED, 0.08 CLEAN |
|
||||
| muse-7 | DETECTED | (0.045, 0.06] | same subject as muse-2/4/6; 0.045 DETECTED, 0.06 CLEAN |
|
||||
|
||||
Five valid new points, four of them near-duplicate portraits of one
|
||||
subject, with first-clean already spanning 0.06-0.10 **inside that
|
||||
subject**. Google's shipped split had zero overlap between classes and
|
||||
identical boundaries inside a class. This harvest cannot support a
|
||||
predictor.
|
||||
|
||||
`flat_ratio` on the original five plus the new portraits is not monotonic
|
||||
with first-clean either (studio_mug 0.973 / 0.045 easy; lighthouse 0.602 /
|
||||
0.10 hard; muse-4 0.789 / 0.10 hard). OpenAI stays unshippable for the
|
||||
same reason as before: one text fixture vs one face fixture, and the face
|
||||
fixture was the harder one, the opposite of Google's split.
|
||||
|
||||
**Harvest 2** takes six files from the 2026-08-26 Muse Image API corpus
|
||||
(61 independent `muse-image-1.0` generations, no hash overlap with the
|
||||
committed five), chosen to span class and `flat_ratio`:
|
||||
|
||||
| fixture | class | flat_ratio | edge_density |
|
||||
|---|---|---|---|
|
||||
| architecture_mosque | dense architecture | 0.037 | 0.367 |
|
||||
| product_sneaker | product / high-flat | 0.874 | 0.018 |
|
||||
| text_poster_bakery | text | 0.368 | 0.191 |
|
||||
| portrait_weathered_fisherman | photoreal portrait | 0.470 | 0.108 |
|
||||
| illustration_watercolor_botanical | illustration | 0.442 | 0.089 |
|
||||
| scene_tokyo_alley | busy scene | 0.258 | 0.175 |
|
||||
|
||||
Script: `scripts/chroma_muse_expansion.py`. Outputs under
|
||||
`out/cohort-calibration/meta-api/` (gitignored). Chroma1 ladders (seven
|
||||
rungs, seed 0, four effective steps) ran for all six. Oracle
|
||||
2026-08-31, anonymous `playwright-isolated` against
|
||||
`meta.ai/identification`. All six controls DETECTED (valid carriers).
|
||||
|
||||
| fixture | flat_ratio | detected at | clean from |
|
||||
|---|---|---|---|
|
||||
| architecture_mosque | 0.037 | 0.08 | 0.10 |
|
||||
| illustration_watercolor_botanical | 0.442 | 0.08 | 0.10 |
|
||||
| scene_tokyo_alley | 0.258 | 0.06 | 0.08 |
|
||||
| text_poster_bakery | 0.368 | 0.045 | 0.06 |
|
||||
| portrait_weathered_fisherman | 0.470 | 0.045 | 0.06 |
|
||||
| product_sneaker | 0.874 | 0.03 | 0.045 |
|
||||
|
||||
The worst first-clean is still 0.10, the same as the committed
|
||||
lighthouse, so the shipped Meta floor 0.17 does not move.
|
||||
|
||||
**No Meta adaptive arm.** Google's shipped split had zero overlap
|
||||
between classes and identical first-cleans inside a class. Combined
|
||||
with the original five, Meta first-cleans are a continuum from 0.03 to
|
||||
0.10, and `flat_ratio` does not separate them:
|
||||
|
||||
- the 0.10 cluster is mosque 0.037, botanical 0.442, and lighthouse
|
||||
0.602;
|
||||
- the 0.045 cluster is sneaker 0.874 and studio_mug 0.973;
|
||||
- harvest-1 photoreal portraits were high-flat (0.75-0.85) AND hard
|
||||
(0.08-0.10), so a high-flat easy-arm would misroute them.
|
||||
|
||||
A content-class rule fails the same way: two text posters first-clean
|
||||
at 0.03 and 0.06, and two watercolor-like images (lighthouse, botanical)
|
||||
share 0.10 with dense architecture.
|
||||
|
||||
The leftover rungs were finished in a 2026-08-31 evening UTC
|
||||
anonymous isolated session (fresh-navigation, wait for "Upload
|
||||
another file"; all ten `POST /api/ai-detector` calls returned 200):
|
||||
|
||||
| check | verdict |
|
||||
|---|---|
|
||||
| muse-2 @ 0.08 | DETECTED |
|
||||
| muse-3 @ 0.015 / 0.03 / 0.045 | DETECTED / CLEAN / CLEAN |
|
||||
| muse-7 @ 0.045 / 0.06 | DETECTED / CLEAN |
|
||||
| mosque @ 0.10 seeds 1, 2 | CLEAN / CLEAN |
|
||||
| botanical @ 0.10 seeds 1, 2 | DETECTED / DETECTED |
|
||||
|
||||
botanical's seed-0 first-clean of 0.10 is not seed-stable;
|
||||
lighthouse @ 0.10 seeds 1 and 2 were CLEAN. That is the same class
|
||||
of run-to-run flip the 0.17 floor's margin exists for. No
|
||||
Google-style binary split: harvest-1 first-cleans now span 0.03
|
||||
(muse-3) through 0.10 (muse-2/4), and 0.06-0.10 inside one subject
|
||||
(muse-7 / muse-6 / muse-2).
|
||||
|
||||
## OpenAI content-adaptive expansion (2026-08-31)
|
||||
|
||||
Goal: test whether face_count or `flat_ratio` predicts Chroma1's OpenAI
|
||||
SynthID first-clean the way YuNet does for Google.
|
||||
|
||||
Committed three (YuNet + `flat_ratio` on the files, first-cleans from the
|
||||
2026-08-29/30 web oracle):
|
||||
|
||||
| fixture | faces | flat_ratio | first-clean |
|
||||
|---|---|---|---|
|
||||
| typography 02_03_55 | 0 | 0.642 | 0.06 |
|
||||
| quality 02_02_23 | 0 | 0.695 | 0.06 |
|
||||
| 9-face grid 05_30 | 9 | 0.473 | 0.075 |
|
||||
|
||||
Face content is the harder of the two classes, the opposite of Google.
|
||||
Two zero-face cards share a boundary, which is a hint, not a split.
|
||||
|
||||
**Images API is not a carrier source.** `gpt-image-1` and `gpt-image-1.5`
|
||||
generations (medium, 1024) carry C2PA (`detected`) but SynthID
|
||||
`not_detected` on `POST /v1/content_provenance_checks`. Contrastive-pair
|
||||
"openai" recreations are also SynthID-negative. New ChatGPT-UI downloads
|
||||
are not in the tree beyond the three committed files.
|
||||
|
||||
**Spaces harvest (one day of `_full_scan`, 2026-07-24):** 41 files with
|
||||
OpenAI C2PA, of which 7 checked by the provenance API were SynthID
|
||||
`DETECTED` (one `NOT_DETECTED`). Four diverse DETECTED files were staged
|
||||
under `out/cohort-calibration/openai-expand/` (gitignored; user uploads,
|
||||
not committed) and given a Chroma1 ladder (0.04-0.12, seed 0) by
|
||||
`scripts/chroma_openai_expansion.py`:
|
||||
|
||||
| id | faces | flat_ratio |
|
||||
|---|---|---|
|
||||
| dense_ed68 | 0 | 0.118 |
|
||||
| midflat_038c | 0 | 0.323 |
|
||||
| textlike_3b17 | 0 | 0.649 |
|
||||
| faces2_4dab | 2 | 0.279 |
|
||||
|
||||
The provenance API then 429'd. The same checks were finished on
|
||||
`https://openai.com/research/verify/` (the page reports SynthID and C2PA
|
||||
separately under View details). Chroma PNG outputs have no C2PA, so a
|
||||
"No OpenAI signals detected" heading is a pixel-SynthID negative.
|
||||
|
||||
| id | faces | @0.04 | @0.05 | @0.06 |
|
||||
|---|---|---|---|---|
|
||||
| dense_ed68 | 0 | CLEAN | CLEAN | CLEAN |
|
||||
| midflat_038c | 0 | -- | -- | CLEAN |
|
||||
| textlike_3b17 | 0 | -- | -- | CLEAN |
|
||||
| faces2_4dab | 2 | -- | -- | CLEAN |
|
||||
|
||||
9-face grid @ 0.075, seeds 1 and 2: both CLEAN (SynthID not detected).
|
||||
The committed seed-0 first-clean is not a fluke.
|
||||
|
||||
**No OpenAI adaptive arm.** Zero-face extras clear at or below 0.06, the
|
||||
2-face extra also clears at 0.06, and only the 9-face grid needs 0.075.
|
||||
That is not a YuNet split: `face_count > 0` would over-strengthen the
|
||||
2-face file, and a high face-count cutoff would be one fixture. The
|
||||
spread (0.075 - 0.06) is already absorbed by the shipped 0.09 floor.
|
||||
Images API generations remain non-carriers (C2PA only).
|
||||
|
||||
## Content-balanced engine selection check (2026-08-31)
|
||||
|
||||
The earlier tables were intentionally carrier-focused and therefore small. A
|
||||
separate discovery pass tested whether ordinary image content contradicts the
|
||||
cohort choice at the shipped floors. The tracked matrix contains 19 prompts,
|
||||
matched across OpenAI `gpt-image-1-mini` and Meta `muse-image-1.0`, spanning
|
||||
photoreal scenes, faces, action, text-like flat art, products, architecture,
|
||||
watercolor, and surreal illustration. These files were re-encoded by collection
|
||||
and are not signal carriers, so this pass measures fidelity only; removal remains
|
||||
established by the provider-oracle calibration above.
|
||||
|
||||
Qwen and Chroma ran in separate sequential H100 invocations, once per source,
|
||||
with the production provider floor and fixed seed. The comparison isolated the
|
||||
global stage because both profiles inherit the same source-based Z-Image face
|
||||
repair. Per-image differences, rather than differences between aggregate means,
|
||||
were evaluated with exact two-sided sign tests:
|
||||
|
||||
| provider | metric | Chroma wins | Qwen wins | two-sided p |
|
||||
|---|---|---:|---:|---:|
|
||||
| OpenAI | SSIM | 19 | 0 | 0.0000038 |
|
||||
| OpenAI | PSNR | 19 | 0 | 0.0000038 |
|
||||
| OpenAI | MAE | 18 | 1 | 0.000076 |
|
||||
| OpenAI | edge F1 | 15 | 4 | 0.019 |
|
||||
| OpenAI | LPIPS | 12 | 7 | 0.359 |
|
||||
| Meta | LPIPS | 1 | 18 | 0.000076 |
|
||||
| Meta | edge F1 | 1 | 18 | 0.000076 |
|
||||
| Meta | SSIM | 5 | 14 | 0.064 |
|
||||
|
||||
Meta's remaining pixel metrics disagree: Chroma wins PSNR on 16/19 and MAE on
|
||||
14/19, while losing both perceptual distance and edge preservation on 18/19 at
|
||||
its higher removal floor. This is not evidence for a genre classifier. The
|
||||
direction instead follows the already measured provenance cohort: Chroma for
|
||||
OpenAI, Qwen for Meta. On OpenAI, the seven LPIPS exceptions include unrelated
|
||||
classes (landscape, sport, low-light interior, watercolor, monochrome portrait,
|
||||
surreal art, and brutalism), while Chroma still wins SSIM and PSNR on every one;
|
||||
there is no coherent input feature that justifies overriding the cohort choice.
|
||||
|
||||
Inputs and integrity manifests are under
|
||||
`data/evaluations/engine-selection/`. Raw generated outputs remain gitignored
|
||||
under `out/engine-selection-study/`; `scripts/engine_selection_study.py`
|
||||
reproduces the run and `scripts/analyze_engine_selection_study.py` reproduces
|
||||
the paired statistics. One generation per prompt is a discovery set, not a
|
||||
license to fit thresholds. Any future content override must first predict a
|
||||
rule here and then survive unused generation indices without changing it.
|
||||
|
||||
## Oracle session notes
|
||||
|
||||
- openai.com/verify throttles anonymous sessions through Cloudflare Turnstile
|
||||
after ~15 rapid checks; a fresh browser context clears it. The programmatic
|
||||
`POST /v1/content_provenance_checks` API is the better oracle (separate
|
||||
SynthID/C2PA outcomes). The `OPENAI_API_KEY` in the main-checkout `.env`
|
||||
returned 401 org-wide, including `/v1/models`, through the 2026-08-30
|
||||
calibration; it was replaced on 2026-08-31 and both `/v1/models` and
|
||||
a provenance check on a committed ChatGPT original then succeeded
|
||||
(DETECTED).
|
||||
- meta.ai/identification enforces a sliding per-IP window: a burst of ~15
|
||||
checks exhausted it twice; it reopens minutes later, and the page sometimes
|
||||
overstates this as a daily limit. Pacing of roughly 2-3 checks per window
|
||||
worked. On 2026-08-31 an anonymous isolated session got a real 429 from
|
||||
`POST /api/ai-detector` with page text "You’ve reached the daily limit
|
||||
for identifications. Try again tomorrow" after about six checks in the
|
||||
UTC morning and again after about fifteen in the UTC afternoon, so the
|
||||
daily-limit wording is not always overstated; the afternoon window was
|
||||
large enough to finish harvest 2. An evening isolated session the same
|
||||
day got ten 200s and finished the leftover harvest-1 rungs plus the
|
||||
mosque/botanical seed probes.
|
||||
- The Gemini app oracle requires the user's logged-in session. A human
|
||||
drag-drop of a local PNG works on Work Gemini (u/2, 2026-08-31) and on
|
||||
Plus (u/0 the same day); Playwright cannot drive the OS picker, and
|
||||
the first `input[type=file].accept` is documents/code only. Plus
|
||||
`@synthid` hung once after attach, then returned a tool CLEAN on a
|
||||
second human-drag chat. Do not drive the live Chrome while it is in
|
||||
active use (wrong-tab risk).
|
||||
@@ -14,9 +14,11 @@ defaults. This page focuses on choosing the right command.
|
||||
| Command or signal | Required installation |
|
||||
| --- | --- |
|
||||
| `metadata` and metadata-only `identify` | Default package |
|
||||
| `classify` | `remove-ai-watermarks[classify]` |
|
||||
| `verify-openai-synthid` | `remove-ai-watermarks[verify]`, API access, and `OPENAI_API_KEY` |
|
||||
| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` |
|
||||
| Adobe TrustMark signals in `identify` | `remove-ai-watermarks[trustmark]` |
|
||||
| Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` |
|
||||
| `visible` and `erase` with OpenCV | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| `visible` or `erase` with MI-GAN | `remove-ai-watermarks[migan]` |
|
||||
| `visible` or `erase` with big-LaMa | `remove-ai-watermarks[lama]` |
|
||||
@@ -26,7 +28,7 @@ defaults. This page focuses on choosing the right command.
|
||||
| `video visible`, `video all`, and visible/all batch modes | `remove-ai-watermarks[video]` plus ffmpeg on PATH |
|
||||
| `video invisible` and `video all --invisible` | `remove-ai-watermarks[video,diffusion]` plus ffmpeg on PATH |
|
||||
| HEIC/HEIF/AVIF pixel input | Add `remove-ai-watermarks[heif]` |
|
||||
| Every production command and backend | `remove-ai-watermarks[all]` |
|
||||
| Every production command and Python-compatible backend | `remove-ai-watermarks[all]` |
|
||||
|
||||
`batch` requires the same extra as its selected mode. Extras can be combined in
|
||||
one installation, for example `remove-ai-watermarks[visible,detect,heif]`.
|
||||
@@ -40,7 +42,15 @@ remove-ai-watermarks identify image.png
|
||||
`identify` always inspects supported metadata. When pixel extras are installed,
|
||||
it also evaluates supported visible and invisible pixel signals. When no signal
|
||||
is found, it reports the origin as unknown. It does not claim the image is
|
||||
clean.
|
||||
clean. For C2PA files, the text report shows asset integrity, claim-signature,
|
||||
signer-trust, and signer-validity results separately. Confidence follows the
|
||||
binding: an intact asset binding and claim signature is high confidence, and an
|
||||
unanchored or expired signer is reported as a caveat rather than a lower score,
|
||||
because no trust anchor list ships with the reader. A failed asset binding or
|
||||
signature, or a revoked signing credential, does not confirm the claimed origin. When a structured
|
||||
C2PA soft binding is present, the report also names its exact algorithm and
|
||||
signed value; removing the manifest does not remove the referenced pixel
|
||||
watermark or content fingerprint.
|
||||
|
||||
Machine readable output:
|
||||
|
||||
@@ -54,8 +64,76 @@ Metadata only inspection:
|
||||
remove-ai-watermarks identify image.png --no-visible
|
||||
```
|
||||
|
||||
Despite the historical option name, `--no-visible` skips both visible and open
|
||||
invisible pixel detectors. Metadata inspection still runs.
|
||||
Despite the historical option name, `--no-visible` skips all pixel detectors,
|
||||
including visible marks, open DWT-DCT, and TrustMark. Metadata inspection still
|
||||
runs.
|
||||
|
||||
## Verify OpenAI SynthID from pixels
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[verify]"
|
||||
remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload
|
||||
remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload --json
|
||||
```
|
||||
|
||||
This is an explicit remote check against OpenAI's official Content Provenance
|
||||
API, not the incomplete local OpenAI carrier research model. Before upload, the
|
||||
command writes a temporary copy with AI provenance metadata removed and aborts
|
||||
unless the decoded RGBA pixels are identical to the source. It then reads only
|
||||
the API's independent `synthid` entry; a C2PA-only response cannot become a
|
||||
SynthID detection. The source is never modified.
|
||||
|
||||
The API supports PNG, JPEG, and WebP files up to 50 MiB. The command requires
|
||||
`OPENAI_API_KEY` and an organization with endpoint access. Because the sanitized
|
||||
raster is uploaded to OpenAI and the endpoint is not eligible for Zero Data
|
||||
Retention, `--acknowledge-upload` is mandatory. This command is never called by
|
||||
`identify`. `not_detected` means only that OpenAI's verifier did not recognize a
|
||||
supported watermark in this file; it is not proof of human authorship.
|
||||
|
||||
The built-in client bounds the request at 120 seconds and disables automatic
|
||||
SDK retries, so one acknowledgement cannot silently upload the media multiple
|
||||
times. A timeout, disconnect, malformed response, access failure, or rate limit
|
||||
is an error, never a negative watermark verdict. API failures expose status,
|
||||
error code, request id, `Retry-After`, and whether an explicit caller-controlled
|
||||
retry is appropriate through `OpenAIProvenanceError`; the verifier itself never
|
||||
retries an upload.
|
||||
The JSON result uses the same provider-scope, backend, pixel-preservation, and
|
||||
metadata-use audit fields as `verify_openai_synthid`.
|
||||
|
||||
The Python API enforces the same boundary with the required explicit intent
|
||||
flag `verify_openai_synthid(path, acknowledge_upload=True)`.
|
||||
|
||||
## Classify a photograph from pixels
|
||||
|
||||
This is not provenance. `identify` never starts it, including after a
|
||||
no-signal metadata scan. Install the extra first:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[classify]"
|
||||
```
|
||||
|
||||
Then:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks classify image.png
|
||||
remove-ai-watermarks classify image.png --json
|
||||
```
|
||||
|
||||
The command runs the frozen photo detector (CLIP-L-ft ridge AND freeze MLP).
|
||||
Only a DEFINITELY result is reported as `ai`. On that result only, the 124-d
|
||||
provider head may return `openai`, `google`, `muse-image`, or `tc260`. POSSIBLY is
|
||||
`unknown`. Camera-like photographs are `human`. The call does not run cleanup
|
||||
and does not set `is_ai_generated`.
|
||||
|
||||
The contract is AI versus camera. Receipts, UI, and digital art are out of
|
||||
scope. Microsoft is not a pixel class: a DALL-E Bing image scores as `openai`,
|
||||
an Imagen Designer image as `google`. `tc260` is the China AIGC label
|
||||
standard, not one producer. Weights download on first use from
|
||||
[`wiltodelta/raiw-photo-classify`](https://huggingface.co/wiltodelta/raiw-photo-classify),
|
||||
or from `RAIW_CLASSIFY_WEIGHTS` if that directory already holds the freeze
|
||||
files.
|
||||
Guide: [photo pixel classification](photo-classify.md). Hub card:
|
||||
[photo-classify-hf/README.md](photo-classify-hf/README.md).
|
||||
|
||||
## Remove known visible marks
|
||||
|
||||
@@ -97,6 +175,18 @@ Use the strict visual gate without metadata or sibling corroboration:
|
||||
remove-ai-watermarks visible image.png --sensitivity strict -o clean.png
|
||||
```
|
||||
|
||||
Act on a mark you can see but the detector will not confirm, by naming it and
|
||||
skipping the gate:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks visible image.png --mark gemini --no-detect -o clean.png
|
||||
```
|
||||
|
||||
`--detect` is the default. `--no-detect` removes the named mark's footprint
|
||||
unconditionally, so it only makes sense together with an explicit `--mark`; on
|
||||
an image that never carried that mark it inpaints a region for nothing. Prefer
|
||||
`erase --region` when you know the geometry.
|
||||
|
||||
When no known mark is detected, the command does not write a new output. Use
|
||||
`erase` if you can identify the affected region yourself.
|
||||
|
||||
@@ -154,6 +244,11 @@ remove-ai-watermarks metadata image.png --remove -o clean.png
|
||||
When `-o` is omitted, removal overwrites the source. Standard metadata is kept
|
||||
unless you pass `--remove-all`.
|
||||
|
||||
A quiet `--check` or a successful `--remove` is not a clean verdict. The command
|
||||
only inspects and strips embedded AI metadata; a pixel watermark such as SynthID
|
||||
has no local decoder once that metadata proxy is gone. `identify` reports the
|
||||
same limit.
|
||||
|
||||
The command also supports the audio and video containers listed in
|
||||
[supported signals](supported-signals.md). ffmpeg must be available for the
|
||||
non-ISOBMFF audio and video path.
|
||||
@@ -222,11 +317,17 @@ remove-ai-watermarks video metadata input.mp4 --check
|
||||
remove-ai-watermarks video metadata input.mp4 --remove -o clean.mp4
|
||||
```
|
||||
|
||||
As with the generic `metadata` command, a quiet check or a successful strip is
|
||||
not a clean verdict: video SynthID is not decoded locally after the metadata
|
||||
proxy is gone.
|
||||
|
||||
Supported containers are MP4, MOV, M4V, WebM, MKV, AVI, and FLV. The operation
|
||||
delegates to the same verified metadata scanner and stripper as the generic
|
||||
`metadata` command, so detection and removal stay in parity. Video and audio
|
||||
streams are not transcoded. For MP4 and MOV, this includes the native TC260
|
||||
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`. The inspector
|
||||
`AIGC` key and JSON value stored in `moov.udta.meta.keys/ilst`, plus the
|
||||
QuickTime-form `meta` variants Doubao's iOS export writes (a bare `meta` box
|
||||
as a direct `moov` child, and a keyless `hdlr=mdir` metadata list). The inspector
|
||||
seeks past a large `mdat` to find a tail `moov`. Removal stream-copies the
|
||||
container in bounded chunks, converts supported top-level provenance boxes to
|
||||
same-size `free` boxes, and blanks the TC260 key/value in place. Box sizes,
|
||||
@@ -298,7 +399,7 @@ remove-ai-watermarks video visible kling.mp4 --mark kling -o kling_clean.mp4
|
||||
|
||||
The command supports the moving Sora mascot and wordmark, two Veo
|
||||
corner variants, the Seedance boxed `AI` label, the `Dola AI` text label, the
|
||||
composite `MINIMAX | hailuo AI` label, and the bottom-right Kling label. Sora
|
||||
composite `MINIMAX | hailuo AI` label, and the bottom-right Kling AI label. Sora
|
||||
searches the whole frame at multiple scales. The other detectors search bounded
|
||||
lower-frame regions with separate synthetic silhouettes. Kling additionally
|
||||
requires its bright low-saturation label near the frame edge. Every mark
|
||||
@@ -308,8 +409,8 @@ provider provenance may relax the visual score only for registered
|
||||
provenance-aware marks; metadata alone never creates a detection.
|
||||
|
||||
`--mark auto` is the default. It evaluates all providers in one decode pass and
|
||||
selects the first stable match in specificity order: Sora, Veo, Seedance, Dola,
|
||||
Hailuo, then Kling. Their confidence scores are independently calibrated and
|
||||
selects the first stable match in specificity order: Sora, Veo, Seedance,
|
||||
Doubao, Dola, Hailuo AI, then Kling AI. Their confidence scores are independently calibrated and
|
||||
are not compared across providers. Pass an explicit `--mark` to scan only that
|
||||
provider.
|
||||
|
||||
@@ -348,7 +449,7 @@ failed encode does not overwrite an existing result.
|
||||
|
||||
## Remove invisible watermarks
|
||||
|
||||
Install the removal dependencies first. Both profiles are CUDA-only and both
|
||||
Install the removal dependencies first. All profiles are CUDA-only and all
|
||||
run the DiffSynth Z-Image face stage, so this is the extra either one needs:
|
||||
|
||||
```bash
|
||||
@@ -368,14 +469,33 @@ detected. Use `--force` when you know the image should be processed:
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
### Choose a strength cohort
|
||||
|
||||
`--vendor` selects the cohort the default strength resolves from. `auto`
|
||||
(the default) derives it from provenance: the C2PA issuer for
|
||||
OpenAI/Google/Microsoft, and the standalone AI IPTC tag for Meta Content Seal
|
||||
(Muse output carries no C2PA; the tag is a standard code, so C2PA evidence
|
||||
always wins first). An explicit value both names the cohort for stripped files
|
||||
and implies the scrub runs -- naming the cohort asserts the pixel watermark is
|
||||
present -- exactly like `--force` plus a measured floor:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks invisible muse_output.webp -o clean.png --vendor meta
|
||||
```
|
||||
|
||||
The same option exists on `all` and `batch`, and as
|
||||
`InvisibleOptions(vendor="meta")` in the Python API.
|
||||
|
||||
### Choose a pipeline
|
||||
|
||||
| Pipeline | When to use it |
|
||||
| --- | --- |
|
||||
| `qwen-zimage` | Default. Qwen-Image-2512 global pass plus a SAM-masked Z-Image face stage |
|
||||
| `sdxl-zimage` | The same recipe and face stage on an SDXL global pass, at a higher denoise |
|
||||
| `chroma-zimage` | The same face stage on an Apache-2.0 Chroma1 global pass, with lower OpenAI and Microsoft floors and higher Google and Meta floors |
|
||||
| `auto` | Pick the engine from provenance: chroma-zimage for OpenAI and Microsoft, qwen-zimage for Google, Meta, and unknown |
|
||||
|
||||
**Both are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
**All four are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
removal. The former `controlnet`, `sdxl`, `qwen` and `default` profiles were removed
|
||||
rather than kept as a CPU path: none of them matched this recipe's face preservation,
|
||||
so offering them implied a quality the library no longer delivers. Passing a retired
|
||||
@@ -395,6 +515,53 @@ schedule, CFG 1.0 and CUDA, so every one of those flags existed only to be refus
|
||||
several layers down. They are not parsed at all now, which fails at the point the
|
||||
user can act on rather than after a model load.
|
||||
|
||||
### Restore operator-verified text
|
||||
|
||||
`--text-manifest` enables the experimental `vae-glyphs` post-pass. It reconstructs
|
||||
the source with the selected profile's VAE, erases the annotated candidate glyphs
|
||||
with LaMa, and composites only the reconstructed glyph cores through source-derived
|
||||
silhouettes. It does not run OCR or choose which strings are correct. The optional
|
||||
Qwen-only `--fidelity-anchor` described below additionally blends 15% of the
|
||||
reconstruction across the full frame.
|
||||
|
||||
Install the combined extra and run only with an operator-verified manifest:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline auto --text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
``verified: true`` may also be set by an automated operator that attests
|
||||
machine-verified geometry: stability-gated detector boxes inside sane caps. Such
|
||||
operators should use the geometry-only schema 2, which carries no transcription
|
||||
or script metadata.
|
||||
|
||||
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default and
|
||||
remains available only with `qwen-zimage`: it
|
||||
was measured to return detector-visible OpenAI SynthID on poster-scale manifests
|
||||
(official Content Provenance API, 2026-08-19). `--fidelity-anchor` restores the
|
||||
0.27.0 research behavior; text-box fidelity lost by the default is well under one
|
||||
MAE point on the measured fixtures.
|
||||
|
||||
The manifest is a JSON object with `verified: true`, decoded RGB dimensions,
|
||||
`source_pixel_sha256`, and a non-empty `lines` array. Schema 1 is retained for
|
||||
manually reviewed annotations: each line has an integer `[x1, y1, x2, y2]` box,
|
||||
exact `text`, a non-empty `script`, and an optional angle from -30 to 30 degrees.
|
||||
Schema 2 is geometry-only: each line has the box and optional angle, with no
|
||||
required `text` or `script`. Lines must be in top-to-bottom, left-to-right order.
|
||||
The hash binds the annotations to decoded RGB geometry and pixels, so metadata-only
|
||||
container changes remain valid while a resized or edited source fails closed. The
|
||||
experimental helper
|
||||
`remove_ai_watermarks._internal.text_restoration.source_pixel_sha256` computes it.
|
||||
|
||||
This mode is supported by `qwen-zimage`, `chroma-zimage`, and `auto` at native
|
||||
geometry with `humanize=0`, `unsharp=0`, and adaptive polish disabled. The legacy
|
||||
`sdxl-zimage` profile does not expose a verified-text VAE donor. `all` also accepts the flag,
|
||||
but its manifest must match the pixels entering the invisible stage; if visible-mark
|
||||
removal changes those pixels, the hash check rejects the run. One oracle verdict does
|
||||
not certify another manifest, seed, model/runtime version, or output hash.
|
||||
|
||||
### Work with limited memory
|
||||
|
||||
Lower CUDA memory pressure:
|
||||
@@ -420,13 +587,39 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
```
|
||||
|
||||
Tiling avoids the explicit downscale but each tile is regenerated separately.
|
||||
It is a memory strategy, not a guarantee of better quality.
|
||||
It is a memory strategy, not a guarantee of better quality. `--tile-size` and
|
||||
`--tile-overlap` set the tile geometry (defaults 1024 and 128 px); a larger
|
||||
overlap costs time and hides seams.
|
||||
|
||||
### Every tuning option
|
||||
|
||||
These apply to `invisible`, `all`, and `batch`. Each has a measured default, so
|
||||
reach for one only when you know why the default is wrong for your file.
|
||||
|
||||
| Option | Effect |
|
||||
| --- | --- |
|
||||
| `--strength` | Denoising strength, 0.0-1.0. Default is profile- and vendor-adaptive; see [choose a strength cohort](#choose-a-strength-cohort). Raising it removes more and drifts further from the original. |
|
||||
| `--vendor` | Strength cohort, and an assertion that the watermark is present. |
|
||||
| `--pipeline` | Profile: `qwen-zimage` (default), `sdxl-zimage`, `chroma-zimage`, `auto`. |
|
||||
| `--seed` | Fixed at 0 by default: every profile is certified at that seed, and removal near the floor is seed-dependent. |
|
||||
| `--controlnet-scale` | Canny ControlNet conditioning on the global stage. Higher keeps the original structure and text closer. |
|
||||
| `--humanize` | Analog film grain, 0 = off, typical 2.0-6.0. |
|
||||
| `--unsharp` | Unsharp-mask sharpening, 0 = off. |
|
||||
| `--adaptive-polish` / `--no-adaptive-polish` | Overrides the profile's own choice. Unset lets the profile decide: off for `qwen-zimage`, whose output already matches the input's detail level, on for `sdxl-zimage`. |
|
||||
| `--max-resolution` | Cap the long side before diffusion; 0 = native. |
|
||||
| `--tile`, `--tile-size`, `--tile-overlap` | Tiled regeneration for large inputs, as above. |
|
||||
| `--cpu-offload` | Stream model components between CPU and GPU to lower CUDA memory pressure, at the cost of speed. |
|
||||
| `--hf-token` | Hugging Face token for the model download; also read from the environment and a local `.env`. |
|
||||
|
||||
There is no `--model`, `--steps`, `--guidance-scale` or `--device` option; the
|
||||
profile pins all four. See [exit behavior](#exit-behavior) for what each command
|
||||
returns.
|
||||
|
||||
## Run the full pipeline
|
||||
|
||||
The `all` command and the `all` installation extra are separate concepts. The
|
||||
command runs every applicable stage. Installing `remove-ai-watermarks[all]`
|
||||
makes every production backend available; a smaller installation such as
|
||||
makes every backend compatible with the active Python available; a smaller installation such as
|
||||
`remove-ai-watermarks[visible,qwen-zimage]` can also run the command with fewer
|
||||
optional backends.
|
||||
|
||||
|
||||
@@ -7,3 +7,15 @@ This page records notices required by source dependencies and licensed derivativ
|
||||
- The DWT-DCT implementation derives from ShieldMnt's
|
||||
[`invisible-watermark`](https://github.com/ShieldMnt/invisible-watermark), licensed
|
||||
under MIT. Its notice ships in `src/remove_ai_watermarks/licenses/invisible-watermark-MIT.txt`.
|
||||
The decode path is a vectorized reformulation, not a transcription: it produces
|
||||
the same bits and is checked against upstream's own decoder, but it no longer
|
||||
corresponds line by line to `imwatermark/maxDct.py`. Diffing the two files will
|
||||
show structurally different code, which is expected and does not mean the
|
||||
derivation notice is stale.
|
||||
|
||||
## Licensed test fixtures
|
||||
|
||||
- `data/fixtures/provenance/adobe-trustmark-p.png` is Adobe's official
|
||||
TrustMark Variant P example, licensed under MIT. Its source commit, digest,
|
||||
and reproduced license are recorded beside the fixture in
|
||||
`data/fixtures/README.md` and `data/licenses/adobe-trustmark-MIT.txt`.
|
||||