Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e556ed032a | ||
|
|
557f776564 | ||
|
|
36fed0594e | ||
|
|
5c6588b3ef | ||
|
|
f127ecbe57 | ||
|
|
56afcab241 | ||
|
|
4428aedce6 | ||
|
|
3daca46589 | ||
|
|
15f5fee7c3 | ||
|
|
3ee858e195 | ||
|
|
cd225f2af9 | ||
|
|
e85307f1c0 | ||
|
|
5a875daded | ||
|
|
c9672efd46 | ||
|
|
8562c457a4 | ||
|
|
17c034f140 | ||
|
|
5b12f61db8 | ||
|
|
ee19173338 | ||
|
|
98d3df9f5a | ||
|
|
464fd142ea | ||
|
|
b2496134c2 | ||
|
|
723f2949ec | ||
|
|
e645fac166 | ||
|
|
4077db8b57 | ||
|
|
8eec0e5af4 | ||
|
|
7ddd2097bd | ||
|
|
c0fec5de19 | ||
|
|
bf7f556876 | ||
|
|
f89975355d | ||
|
|
68b56ece37 | ||
|
|
dc8491a6c6 | ||
|
|
4e9145f739 | ||
|
|
0d6667cf22 | ||
|
|
646f72534b | ||
|
|
922f6faa7a | ||
|
|
f0b425ac71 | ||
|
|
8ba2666a30 | ||
|
|
c1bf835e2f | ||
|
|
efdbfa0598 | ||
|
|
9f7b707016 | ||
|
|
697e11f2ab | ||
|
|
efeec825c1 | ||
|
|
507d7c43a3 | ||
|
|
a1022d33c3 | ||
|
|
5b52782f01 | ||
|
|
0761937d60 | ||
|
|
b8a8c3d3be | ||
|
|
0fd139aac4 | ||
|
|
ecbe296b31 | ||
|
|
09960676fc | ||
|
|
2bc37738d9 | ||
|
|
637395b8a6 | ||
|
|
1e71f0523d | ||
|
|
e41bdbc9c8 | ||
|
|
cbb5155784 | ||
|
|
f3270165db | ||
|
|
11026ec0c6 | ||
|
|
d391cc42d1 | ||
|
|
11fb16f19c | ||
|
|
7ab38a85b6 | ||
|
|
09a4b20ac6 | ||
|
|
9bb4e9cd54 | ||
|
|
5052dbe63c | ||
|
|
2fe4d3c3a0 | ||
|
|
adbc6a3046 | ||
|
|
326840bab4 | ||
|
|
29de21256b | ||
|
|
07b07634cf | ||
|
|
54a19059d5 | ||
|
|
5e85f3ac6f | ||
|
|
36c36212f0 | ||
|
|
84ea8ede2e | ||
|
|
8f30f8bdb3 | ||
|
|
925401c905 | ||
|
|
55e7c53699 | ||
|
|
2ff06d4c5a | ||
|
|
1410a8e064 | ||
|
|
11ca7d6054 | ||
|
|
5991894ca5 | ||
|
|
5fa14f9e26 | ||
|
|
f7e0a18e6c | ||
|
|
bc5922451f | ||
|
|
798c94e6ae | ||
|
|
67d948f57d | ||
|
|
1d0e6b54dd | ||
|
|
fa4748a6a8 | ||
|
|
1e9572e2d4 | ||
|
|
1a6f50db75 |
No files matched your search
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"name": "remove-ai-watermarks",
|
||||
"owner": {
|
||||
"name": "wiltodelta",
|
||||
"url": "https://github.com/wiltodelta"
|
||||
},
|
||||
"metadata": {
|
||||
"description": "Agent skill and Claude Code plugin for the remove-ai-watermarks CLI.",
|
||||
"version": "1.0.5"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "remove-ai-watermarks",
|
||||
"description": "Identify and remove AI provenance marks from images and video the user generated or edited. Use when the user asks to strip Gemini sparkles, SynthID, Microsoft InvisMark, Meta Content Seal, C2PA, EXIF/XMP AI metadata, Sora/Veo/Seedance/Doubao/Dola/Hailuo/Kling labels, or other AI watermarks, or asks whether a file was AI-generated. Do not use for stock-agency, marketplace, classifieds, or other third-party paid-asset watermarks.",
|
||||
"source": "./skills",
|
||||
"strict": false
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,6 +1,14 @@
|
||||
---
|
||||
globs: ["src/**/*.py", "tests/**/*.py", "scripts/**/*.py", "pyproject.toml", "uv.lock", "maintain.sh", ".github/workflows/*.yml"]
|
||||
description: Command contracts, project gate, typing boundaries, model-adjacent test invariants, and the detection-path measurement rule.
|
||||
paths:
|
||||
- "src/**/*.py"
|
||||
- "tests/**/*.py"
|
||||
- "scripts/**/*.py"
|
||||
- "skills/**"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- "maintain.sh"
|
||||
- ".github/workflows/*.yml"
|
||||
description: Command contracts, project gate, typing boundaries, model-adjacent test invariants, the agent-skill parity seam, and the detection-path measurement rule.
|
||||
---
|
||||
|
||||
# Development invariants
|
||||
@@ -9,11 +17,55 @@ description: Command contracts, project gate, typing boundaries, model-adjacent
|
||||
|
||||
Every single-image command declares `source` with `dir_okay=False`; `batch` declares its directory with `file_okay=False`. Keep `tests/test_cli_robustness.py::TestDirectoryInputIsRejected` as the regression guard.
|
||||
|
||||
Exit-code and no-signal behavior is a public contract. Read the command-line section of [`../../docs/module-internals.md`](../../docs/module-internals.md) before changing it.
|
||||
Exit-code and no-signal behavior is a public contract. Read the command-line section of [`../../docs/module-internals.md`](../../docs/module-internals.md) before changing it. Keep [`../../skills/remove-ai-watermarks/`](../../skills/remove-ai-watermarks/) in the same change when command routing, extras, exit codes, mark keys, or the intended-use boundary move.
|
||||
|
||||
`tests/test_agent_skill.py` enforces that by comparing the skill's prose against the live Click tree. Guard the SET, not the instances: per-flag comparisons only check what the skill already documents, so `--vendor`, `--pipeline auto` and then the whole `classify` and `verify-openai-synthid` commands each shipped with a green suite. `test_every_cli_command_is_named_somewhere_in_the_skill` closes that; raise `MIN_CLI_VERSION` in the skill's probe whenever a reference names a command, flag or value the previous release lacked, or the probe tells an agent its CLI is current right before it types something that build rejects.
|
||||
|
||||
Provider verification services and provenance APIs are development-only oracles. Keep
|
||||
their SDKs in the development extra and their adapters out of the installed CLI,
|
||||
top-level Python API, and runtime detection/removal pipelines.
|
||||
|
||||
The unified maintainer entry point is `scripts/provider_oracles.py`; its workflow and
|
||||
slot schema live in `docs/provider-oracles.md`. Multi-account and multi-network work
|
||||
selects one named slot explicitly from the gitignored repository-local
|
||||
`.oracle-slots.json` by default and snapshots it into the immutable batch manifest.
|
||||
Never commit the local slot file.
|
||||
Do not add account or IP rotation, rate-limit failover, or cross-session result merging.
|
||||
Every manual result stays bound to the exact sanitized upload hash and preserves the
|
||||
provider response verbatim.
|
||||
|
||||
Provider surface selection is API-first. OpenAI and Microsoft use their official API
|
||||
adapters before their Web surfaces when the requested media and local configuration are
|
||||
supported. Google and Meta currently use Web because this tooling has no usable API route
|
||||
for them. Web remains an explicit second surface, never an automatic retry after an API
|
||||
refusal, quota response, transport error, or indeterminate verdict.
|
||||
|
||||
Google Web oracle automation uses only the user's existing authenticated real Chrome
|
||||
session and explicit `/u/N/` account index. OpenAI, Microsoft, and Meta Web automation
|
||||
uses isolated Playwright contexts; it must never attach to or copy state from the user's
|
||||
real browser. Proxy credentials are read only through a slot's named environment variable
|
||||
and never enter a manifest or log. Dotenv interpolation may derive routed proxy URLs from
|
||||
one secret key without mutating the process environment. A selected proxy context may
|
||||
accept that proxy's TLS interception certificate; direct contexts keep normal TLS checks.
|
||||
Upload readiness is surface-specific: OpenAI reaches it at page load but keeps background
|
||||
requests alive, while Microsoft and Meta require network idle. Preserve exact provider
|
||||
errors as indeterminate evidence rather than retrying or interpreting them as clean.
|
||||
|
||||
OpenAI API slots likewise carry only `api_key_env`. Store multiple key values in the
|
||||
gitignored `.env`, select one slot explicitly, and never rotate to another token after a
|
||||
refusal or rate limit. The process environment may override the matching `.env` value.
|
||||
Their SDK transport sets `trust_env=False`; environment proxy variables must not reroute
|
||||
an API check.
|
||||
|
||||
Microsoft API slots use the Azure Content Safety and private Blob Storage resource
|
||||
identities from the gitignored `.env`. The adapter reads resource keys only into memory
|
||||
through authenticated Azure CLI, downloads and hashes the private blob before requiring
|
||||
equality with the named local source, keeps `trust_env=False`, never retries the submit,
|
||||
and reports Watermark independently from C2PA.
|
||||
|
||||
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
|
||||
|
||||
`device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. It is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds.
|
||||
`device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. On the image path it is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds. The video SynthID commands (`video invisible`, `video all`, `video batch`) do expose `--device`, whose VAE runs on cuda, mps, or cpu, so a user can usefully pick one. `test_device_exists_exactly_where_the_skill_says_it_does` pins that split; do not read this paragraph as licence to delete the video flag.
|
||||
|
||||
The same rule applies to install hints: name the extra that actually makes the command work (`qwen-zimage`, not `diffusion`), and keep the printed command shell-quoted -- bare `pkg[extra]` is a glob in zsh.
|
||||
|
||||
@@ -21,10 +73,30 @@ The same rule applies to install hints: name the extra that actually makes the c
|
||||
|
||||
Run `bash maintain.sh` from the repository root. The authoritative type gate is scoped to `src/`; full-project Pyright can exhaust Node memory on the ML dependency graph.
|
||||
|
||||
The security scanner's exit status is authoritative, including a failure after a success
|
||||
message. `tests/test_maintenance.py` checks clean, vulnerable, and broken scanner
|
||||
runs through the real shell entry point. Recheck unresolved advisories using the
|
||||
procedure in `docs/development.md`; do not revive the old text-matching bypass.
|
||||
|
||||
Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.
|
||||
|
||||
From a worktree, `uv run` imports the package from the MAIN checkout -- that is where the editable install points. A script measuring a worktree's edit must insert that worktree's `src` at `sys.path[0]` and assert `module.__file__` resolves inside it, or it silently compares unmodified code against itself.
|
||||
|
||||
## Research harnesses stay out of this repository
|
||||
|
||||
A research script that needs a cloud GPU account to run is not kept here, and
|
||||
`scripts/` carries no `modal` import; `test_scripts_carry_no_cloud_gpu_harness`
|
||||
is the canary. The identifier half of the boundary audit is deliberately not
|
||||
mechanized here, because a denylist written into a tracked file publishes the
|
||||
strings it exists to keep out. Twelve such harnesses were moved out on
|
||||
2026-09-07; the pages that cite their findings
|
||||
([chroma1](../../docs/chroma1-engine-research.md),
|
||||
[module internals](../../docs/module-internals.md)) name them as harnesses kept
|
||||
outside this repository, and the tracked inputs under `data/evaluations/` plus
|
||||
the recorded verdicts are the durable record. Keep a new one outside too, and
|
||||
leave only its offline analysis step here, the way
|
||||
`scripts/analyze_engine_selection_study.py` stayed with its test.
|
||||
|
||||
## Visible-mark example gallery
|
||||
|
||||
Every registered mark carries a committed example: `data/fixtures/visible/<key>/example.png`
|
||||
@@ -54,6 +126,9 @@ Do not classify an entire module as untestable because its main path downloads a
|
||||
stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a
|
||||
face. When one profile inherits another's stage, guard the invariants that stage
|
||||
relies on, not just the code path.
|
||||
- the photo-classify gate in `test_classify.py`: DEFINITELY is ridge AND MLP,
|
||||
POSSIBLY does not emit a provider, and `identify` does not import
|
||||
`remove_ai_watermarks.classify`.
|
||||
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
|
||||
mirror another, compare them field by field rather than pinning the values you happen
|
||||
to know about, so the next field added on one side and not the other fails at the
|
||||
@@ -89,6 +164,23 @@ status set; `TestIdentifyRealSamples::test_no_committed_fixture_reports_a_truste
|
||||
is that guard. Read a capability the stack does not have as a missing input, never as a
|
||||
negative finding.
|
||||
|
||||
## The docs are a seam, and they drift like one
|
||||
|
||||
`tests/test_docs_cover_the_public_surface.py` reads the live Click tree and the
|
||||
live `InvisibleOptions` dataclass and requires every knob to be NAMED in the
|
||||
user-facing pages. The code-to-code seams were already guarded and the
|
||||
code-to-docs seam was not, so it drifted exactly the way an unguarded seam does:
|
||||
`docs/cli.md` named none of `--adaptive-polish`, `--controlnet-scale`,
|
||||
`--detect/--no-detect`, `--humanize`, `--strength`, `--tile-size`,
|
||||
`--tile-overlap` or `--unsharp`, and `docs/python-api.md` named 9 of the 16
|
||||
option fields.
|
||||
|
||||
Group option spellings by the Click PARAMETER, never by string surgery on the
|
||||
names. A boolean pair can be spelled anything -- `--strip-metadata/--keep-metadata`,
|
||||
`--keep-standard/--remove-all` -- so deriving the partner from a `--no-` prefix
|
||||
silently splits those pairs and demands both halves. Either spelling documents
|
||||
the pair.
|
||||
|
||||
## One measurement, one gate seam
|
||||
|
||||
A detector is split into a trust-level-blind scan and a verdict that applies the
|
||||
@@ -103,6 +195,11 @@ rules follow, and both were broken in practice before they were written down:
|
||||
- Detection and the removal mask must read ONE sweep. The winning box travels on
|
||||
`TextMarkDetection.match_box` and the registry threads the detection into the mask
|
||||
builder; a mask path that re-runs its own sweep is how the two drift apart.
|
||||
- When the research lattice runtime in `scripts/synthid_runtime/` routes by image
|
||||
geometry, preserve the returned `SynthIDDetection.detector` in score manifests
|
||||
and downstream routers. Record an inactive expert as explicitly unsupported;
|
||||
never attribute a routed large-image score to the fixed expert. This expert is
|
||||
not part of the public package.
|
||||
|
||||
The C2PA manifest-store JSON is NOT stable across reads: the reader regenerates manifest
|
||||
URNs and instance ids. Compare the derived `c2pa_info`, never the raw store.
|
||||
@@ -155,7 +252,11 @@ path exempt from the gate it exists to feed, which is exactly backwards.
|
||||
Prove a video-path refactor the same way the detection path is proven, and without
|
||||
needing an oracle carrier: build a clip from a tracked fixture with ffmpeg, run the
|
||||
engine before and after, and require an identical output sha256. Keep the generated
|
||||
media outside the repository.
|
||||
media outside the repository. Bind that comparison to one synced environment: a
|
||||
full-extras `uv sync` was observed to change re-encoded H.264 packet bytes with
|
||||
no code change while every detection stayed identical
|
||||
(`docs/audio-provenance-experiment.md`), so a moved video hash across an
|
||||
environment change is not by itself a regression signal.
|
||||
|
||||
Frame sampling is compared as `timestamp + 1e-9 >= next_sample_time`, so mutating
|
||||
that `>=` to `>` changes nothing and a green suite proves nothing. Mutate the phase
|
||||
@@ -166,3 +267,31 @@ frame-count check cannot see and what
|
||||
catch.
|
||||
|
||||
Environment setup, dependency recovery, CI behavior, and fixture policy: [`../../docs/development.md`](../../docs/development.md).
|
||||
|
||||
## Audit regression contracts
|
||||
|
||||
Image output paths must describe the artifact published by the current call,
|
||||
including skipped invisible stages and in-place writes. Carry explicit EXIF
|
||||
decode state through image staging; equal dimensions do not prove that a
|
||||
mirror or 180-degree rotation was applied. Metadata collection must propagate
|
||||
late-read failures into an incomplete record, and stream-copy metadata removal
|
||||
must map every input stream before atomically publishing the output.
|
||||
|
||||
Research observations must describe the decoded, hash-bound saved artifact.
|
||||
Keep period selection independent of confirmation patches, require explicit
|
||||
negative evidence, and deduplicate source corpora by content identity rather
|
||||
than detector-score equality. Use the metric's actual denominator in its name;
|
||||
normalized edit distance is not reference-normalized character error rate.
|
||||
|
||||
Tests that reload a module after changing environment variables must restore
|
||||
the environment before reloading the original state. Otherwise a temporary
|
||||
cache path survives teardown and later model tests silently attempt downloads.
|
||||
Dotenv configuration tests must clear every fixture variable from the invoked
|
||||
process environment. Seed conflicting inherited values to prove isolation;
|
||||
clearing only some fields lets the remaining host settings override the fixture.
|
||||
|
||||
Cross-platform tests must use host-absolute synthetic interpreter paths and
|
||||
explicit unchanged decodes when comparing stored pixels. Execute Ubuntu Bash
|
||||
workflow harnesses only on POSIX hosts with Bash; keep their pure Python checks
|
||||
active on every platform. Round floating-point image output before uint8
|
||||
conversion when quantization must preserve constant values across OpenCV builds.
|
||||
@@ -0,0 +1,120 @@
|
||||
---
|
||||
name: provider-oracles
|
||||
description: Run hash-bound Google, OpenAI, Microsoft, or Meta watermark-oracle checks for this repository. Use for provider verification, multi-account Gemini checks, isolated Playwright checks, or recording oracle evidence.
|
||||
---
|
||||
|
||||
# Provider oracles
|
||||
|
||||
Read `docs/provider-oracles.md` before acting. Use `scripts/provider_oracles.py`
|
||||
for surface discovery, slot validation, upload preparation, result recording,
|
||||
and integrity verification.
|
||||
|
||||
Named slots live in the gitignored repository-local
|
||||
`.oracle-slots.json`. Never commit that file. Pass `--slot NAME` to use it;
|
||||
pass `--slots PATH` only to override the local path.
|
||||
|
||||
Never substitute one provider's negative verdict for another provider's signal.
|
||||
Do not optimize candidates adaptively against an external oracle. A retry,
|
||||
different account, or different network is a new explicit operator decision,
|
||||
not automatic fallback.
|
||||
|
||||
## Default surface priority
|
||||
|
||||
Prefer an official API adapter whenever that provider, media type, and configured
|
||||
slot support it. Use Web only when no usable API route exists or the operator
|
||||
explicitly requests the Web surface as a second check. For OpenAI and Microsoft,
|
||||
the default plans are API then Web. Google and Meta currently have only usable
|
||||
Web routes in this tooling. Inspect the order with `plan PROVIDER`.
|
||||
|
||||
The priority is a selection policy, not automatic failover. An API refusal,
|
||||
quota response, transport error, or indeterminate verdict must be recorded and
|
||||
reported. Do not submit the same artifact to Web unless the operator separately
|
||||
authorizes that next surface.
|
||||
|
||||
## Route by surface
|
||||
|
||||
### Google Gemini
|
||||
|
||||
Use the user's real authenticated Chrome. Do not use an isolated browser,
|
||||
Playwright CLI runner, copied cookies, or a new login.
|
||||
|
||||
1. Read `~/Documents/GitHub/claude-config/docs/browser-automation.md` and select
|
||||
the authenticated-Chrome capability.
|
||||
2. Validate the named slot. Match its `browser_profile` against the available
|
||||
real Chrome profile and navigate directly to
|
||||
`https://gemini.google.com/u/<google_account_index>/app`.
|
||||
3. Prepare an immutable `gemini-web` batch outside the repository.
|
||||
4. Before each upload, request the action-time confirmation required for sending
|
||||
that exact prepared file to Gemini. Do not upload the source path directly.
|
||||
5. Attach one prepared upload and ask: `Was this image/video/audio created or
|
||||
edited by Google AI?`
|
||||
6. Read the settled SynthID tool outcome, not the surrounding model reasoning.
|
||||
Preserve unclear, quota, and refusal wording rather than interpreting it as
|
||||
clean.
|
||||
7. Record the verbatim response and verify the completed batch.
|
||||
|
||||
If the selected Chrome profile or `/u/N/` account is not already authenticated,
|
||||
stop and ask the user to select the intended existing session. Do not sign in or
|
||||
switch to another account implicitly.
|
||||
|
||||
### OpenAI, Microsoft, and Meta Web
|
||||
|
||||
Use the isolated Playwright runner, never the user's real Chrome:
|
||||
|
||||
```bash
|
||||
uv run python scripts/provider_oracles.py run-web /outside/repo/batch/manifest.json \
|
||||
--acknowledge-uploads --env-file .env
|
||||
```
|
||||
|
||||
The runner is headed by default because OpenAI's Cloudflare gate blocks the
|
||||
same isolated Chromium in headless mode. Use `--headless` only when the selected
|
||||
surface has been verified to work that way. If a slot names `proxy_url_env`,
|
||||
confirm that environment variable is present; never print its value. The runner
|
||||
uses one context, one route, fresh page navigation per upload, and no retry.
|
||||
CAPTCHA, bot checks, throttling, transport failure, and unknown page wording
|
||||
remain distinct recorded outcomes.
|
||||
|
||||
Wait for the surface-specific upload readiness that the runner implements.
|
||||
OpenAI completes at page load but keeps background requests alive, while
|
||||
Microsoft and Meta require network idle. Do not replace those conditions with
|
||||
one shared wait. Treat OpenAI's exact `Something went wrong` response as a
|
||||
settled `indeterminate` result, not as a clean result or a reason to retry.
|
||||
|
||||
For ThorData residential proxy mode, keep `THORDATA_ROUTE_US`,
|
||||
`THORDATA_ROUTE_DE`, `THORDATA_ROUTE_GB`, `THORDATA_ROUTE_NL`, and
|
||||
`THORDATA_ROUTE_JP` URLs in the gitignored `.env`;
|
||||
OpenAI, Microsoft, and Meta Web slots may each name one of them through
|
||||
`proxy_url_env`. Encode an explicit two-letter ISO country code in each route,
|
||||
verify egress before relying on it, and never automatically change route after
|
||||
a refusal. Bound a batch before starting.
|
||||
|
||||
### OpenAI API
|
||||
|
||||
Use `check-openai` with one explicit upload acknowledgement. Keep named API keys
|
||||
as `OPENAI_API_KEY_1`, `OPENAI_API_KEY_2`, and `OPENAI_API_KEY_3` in the local `.env`, and select
|
||||
only their variable names through an `openai-api` slot's `api_key_env`. Never
|
||||
print a key, copy one into a slot or manifest, or fall back automatically to a
|
||||
second key. API slots use a direct SDK transport that ignores environment proxy
|
||||
settings. The current adapter is image-only.
|
||||
|
||||
### Microsoft API
|
||||
|
||||
Use `check-microsoft` with one explicit acknowledgement and one private Azure
|
||||
Blob HTTPS URI whose bytes exactly match the local source. Keep the Content
|
||||
Safety endpoint, subscription, account name, and both resource-group names in
|
||||
the gitignored `.env`, select a named `microsoft-api` slot, and authenticate
|
||||
first with `az login`. Give the Content Safety system identity `Storage Blob
|
||||
Data Reader` on the input storage account. The adapter reads resource keys only
|
||||
into memory through Azure CLI, never logs them, ignores environment proxy
|
||||
settings, makes one submit request with no retry, and polls only the returned
|
||||
operation. Keep Microsoft's Watermark and C2PA markers as separate verdicts.
|
||||
|
||||
## Evidence rules
|
||||
|
||||
- Upload only files under the prepared batch's `uploads/` directory.
|
||||
- Keep `manifest.json` and `manifest.sha256` immutable.
|
||||
- Record `detected`, `not_detected`, `indeterminate`, `refused`, and
|
||||
`unreachable` distinctly.
|
||||
- Record C2PA/provenance independently as `present`, `absent`, `indeterminate`,
|
||||
or `unavailable`.
|
||||
- Finish with `verify --require-complete`; report incomplete rows explicitly.
|
||||
@@ -1,3 +1,41 @@
|
||||
# HuggingFace token (optional; only needed for gated/private models)
|
||||
# Get yours at: https://huggingface.co/settings/tokens
|
||||
# HF_TOKEN=
|
||||
|
||||
# OpenAI API key (optional; only for development scripts/openai_provenance_check.py)
|
||||
# Create and manage keys at: https://platform.openai.com/api-keys
|
||||
# OPENAI_API_KEY=
|
||||
|
||||
# Local photo-classifier weights (optional; otherwise fetched from Hugging Face)
|
||||
# Directory contents are documented in docs/photo-classify.md.
|
||||
# RAIW_CLASSIFY_WEIGHTS=/path/to/classifier-weights
|
||||
|
||||
# Local VideoSeal checkpoint cache (optional; development benchmarks only)
|
||||
# Defaults to ~/.cache/remove-ai-watermarks-dev/videoseal.
|
||||
# VIDEOSEAL_CACHE_DIR=/path/to/videoseal-cache
|
||||
|
||||
# Hugging Face cache root (optional; shared by model downloads and prompt caches)
|
||||
# HF_HOME=/path/to/huggingface-cache
|
||||
# Fallback cache root when HF_HOME is unset (optional)
|
||||
# XDG_CACHE_HOME=/path/to/cache
|
||||
|
||||
# Named OpenAI keys for provider-oracle slots (optional; keep each value secret)
|
||||
# OPENAI_API_KEY_1=
|
||||
# OPENAI_API_KEY_2=
|
||||
# OPENAI_API_KEY_3=
|
||||
|
||||
# Azure Content Safety resource for direct Microsoft Content Provenance checks
|
||||
# Resource keys are read into memory through the authenticated Azure CLI.
|
||||
# AZURE_CONTENT_SAFETY_ENDPOINT=https://your-resource.cognitiveservices.azure.com/
|
||||
# AZURE_CONTENT_SAFETY_SUBSCRIPTION_ID=
|
||||
# AZURE_CONTENT_SAFETY_RESOURCE_GROUP=
|
||||
# AZURE_CONTENT_SAFETY_ACCOUNT_NAME=
|
||||
# AZURE_ORACLE_STORAGE_RESOURCE_GROUP=
|
||||
|
||||
# ThorData residential proxies (optional; only for explicitly selected Web oracle routes)
|
||||
# Generate these URLs in the ThorData endpoint generator; do not commit credentials.
|
||||
# THORDATA_ROUTE_US=http://td-customer-USERNAME-country-US:PASSWORD@HOST.pr.thordata.net:9999
|
||||
# THORDATA_ROUTE_DE=http://td-customer-USERNAME-country-DE:PASSWORD@HOST.pr.thordata.net:9999
|
||||
# THORDATA_ROUTE_GB=http://td-customer-USERNAME-country-GB:PASSWORD@HOST.pr.thordata.net:9999
|
||||
# THORDATA_ROUTE_NL=http://td-customer-USERNAME-country-NL:PASSWORD@HOST.pr.thordata.net:9999
|
||||
# THORDATA_ROUTE_JP=http://td-customer-USERNAME-country-JP:PASSWORD@HOST.pr.thordata.net:9999
|
||||
@@ -15,7 +15,7 @@ updates:
|
||||
- "minor"
|
||||
- "patch"
|
||||
# The `all` extra must remain jointly resolvable. These candidates violate
|
||||
# upstream constraints: TrustMark still requires NumPy 1.x, OpenCV 4.12+
|
||||
# compatibility constraints: older supported TrustMark builds require NumPy 1.x, OpenCV 4.12+
|
||||
# requires NumPy 2 on newer Python versions, and this project keeps tokenizers
|
||||
# below 0.23 for the stable Transformers path. Keep Dependabot on the
|
||||
# compatible lines until those constraints move.
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
name: Distribute on release
|
||||
|
||||
# Fans a published GitHub Release out to the channels that need a nudge.
|
||||
# PyPI is handled by publish.yml. This workflow event-drives the three channels
|
||||
# PyPI is handled by publish.yml. This workflow event-drives the four channels
|
||||
# that would otherwise be manual:
|
||||
# - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
|
||||
# - HF Space: factory-rebuild the version already pinned in the Space repo.
|
||||
# - ComfyUI: sync, test, and publish the node against the exact library release.
|
||||
# - ClawHub: publish the agent skill when its own version moved.
|
||||
# All wait for the freshly published sdist to appear on PyPI first, since the
|
||||
# Release event fires in parallel with publish.yml's upload.
|
||||
|
||||
@@ -41,6 +42,27 @@ jobs:
|
||||
echo "version=$v" >> "$GITHUB_OUTPUT"
|
||||
echo "Target version: $v"
|
||||
|
||||
- name: Record distributed release
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
python3 - <<'PYTHON'
|
||||
import json, os
|
||||
from pathlib import Path
|
||||
Path("release-identity.json").write_text(json.dumps({
|
||||
"version": os.environ["VERSION"],
|
||||
"run_id": os.environ["GITHUB_RUN_ID"],
|
||||
"run_attempt": os.environ["GITHUB_RUN_ATTEMPT"],
|
||||
}), encoding="utf-8")
|
||||
PYTHON
|
||||
|
||||
- name: Upload distributed release identity
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: distributed-release-${{ github.run_attempt }}
|
||||
path: release-identity.json
|
||||
if-no-files-found: error
|
||||
|
||||
- name: Wait for the sdist on PyPI, capture url + sha256
|
||||
id: sdist
|
||||
run: |
|
||||
@@ -106,6 +128,44 @@ jobs:
|
||||
print("HF Space factory rebuild triggered")
|
||||
PY
|
||||
|
||||
clawhub:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- name: Publish the agent skill when its version moved
|
||||
env:
|
||||
CLAWHUB_TOKEN: ${{ secrets.CLAWHUB_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${CLAWHUB_TOKEN:-}" ]; then
|
||||
echo "CLAWHUB_TOKEN secret is not set; skipping ClawHub publish."
|
||||
exit 0
|
||||
fi
|
||||
npm install --silent --no-fund --no-audit clawhub
|
||||
skill_version=$(python3 - <<'PY'
|
||||
import re
|
||||
text = open("skills/remove-ai-watermarks/SKILL.md", encoding="utf-8").read()
|
||||
m = re.search(r"(?m)^ version:\s*[\"']?([^\"'\s]+)", text)
|
||||
assert m, "skill version missing"
|
||||
print(m.group(1))
|
||||
PY
|
||||
)
|
||||
./node_modules/.bin/clawhub login --token "$CLAWHUB_TOKEN" >/dev/null
|
||||
published=$(./node_modules/.bin/clawhub inspect remove-ai-watermarks --file SKILL.md 2>/dev/null \
|
||||
| python3 -c 'import re,sys; t=sys.stdin.read(); m=re.search(r"(?m)^ version:\s*[\"'"'"']?([^\"'"'"'\s]+)", t); print(m.group(1) if m else "")' || true)
|
||||
if [ "$published" = "$skill_version" ]; then
|
||||
echo "ClawHub already at $skill_version; nothing to publish."
|
||||
exit 0
|
||||
fi
|
||||
echo "ClawHub at ${published:-<none>}; publishing skill $skill_version."
|
||||
./node_modules/.bin/clawhub skill publish skills/remove-ai-watermarks \
|
||||
--slug remove-ai-watermarks \
|
||||
--name "Remove AI Watermarks" \
|
||||
--version "$skill_version" \
|
||||
--changelog "Agent skill sync for the ${{ github.event.release.name || github.ref_name }} release."
|
||||
echo "Submitted; ClawHub runs security scans before the listing goes public."
|
||||
|
||||
comfyui:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
name: Publish photo classifier to Hugging Face
|
||||
|
||||
# Manual only. Not part of a library release. The Space demo
|
||||
# (wiltodelta/remove-ai-watermarks) stays on distribute.yml / raiw-hf-space.
|
||||
#
|
||||
# Hub model: wiltodelta/raiw-photo-classify
|
||||
# Secret: HF_TOKEN, write-role Hugging Face token (same secret Space sync uses).
|
||||
#
|
||||
# Weights are not in git. Full mode downloads them from a GitHub Release
|
||||
# of this repository, then uploads with the card from docs/photo-classify-hf/.
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
confirm:
|
||||
description: "Type publish"
|
||||
required: true
|
||||
mode:
|
||||
description: "full = card plus weights from the release tag; card = README, operating-point, and receipt gate only"
|
||||
type: choice
|
||||
options:
|
||||
- full
|
||||
- card
|
||||
default: full
|
||||
weights_tag:
|
||||
description: "GitHub Release tag that holds clip-l-ft.pt, probe, detector.pt, provider.pt"
|
||||
required: false
|
||||
default: photo-classify-freeze-2026-08-31
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
if: ${{ github.event.inputs.confirm == 'publish' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 180
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Download freeze weights from the GitHub Release
|
||||
if: ${{ github.event.inputs.mode == 'full' }}
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
TAG: ${{ github.event.inputs.weights_tag }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
mkdir -p weights
|
||||
gh release download "$TAG" --repo "${{ github.repository }}" --dir weights
|
||||
ls -l weights
|
||||
|
||||
- name: Publish to Hugging Face
|
||||
env:
|
||||
HF_TOKEN: ${{ secrets.HF_TOKEN }}
|
||||
MODE: ${{ github.event.inputs.mode }}
|
||||
SHA: ${{ github.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${HF_TOKEN:-}" ]; then
|
||||
echo "HF_TOKEN secret is not set" >&2
|
||||
exit 1
|
||||
fi
|
||||
extra=(--message "Photo-classify freeze (GitHub ${SHA:0:7})")
|
||||
if [ "$MODE" = "card" ]; then
|
||||
extra+=(--card-only)
|
||||
else
|
||||
extra+=(--src weights)
|
||||
fi
|
||||
uv run --no-project --with huggingface-hub python scripts/publish_photo_classify_hf.py "${extra[@]}"
|
||||
@@ -0,0 +1,40 @@
|
||||
name: Skill
|
||||
|
||||
# Fast path-filtered gate for agent-skill packaging changes. The full
|
||||
# Tests workflow still runs on the same pushes; this answers skill-only
|
||||
# pull requests in seconds by running just the deterministic skill suites.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "skills/**"
|
||||
- ".claude-plugin/**"
|
||||
- "docs/agent-skill.md"
|
||||
- "tests/test_agent_skill.py"
|
||||
- "tests/test_skill_evals.py"
|
||||
pull_request:
|
||||
paths:
|
||||
- "skills/**"
|
||||
- ".claude-plugin/**"
|
||||
- "docs/agent-skill.md"
|
||||
- "tests/test_agent_skill.py"
|
||||
- "tests/test_skill_evals.py"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: skill-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
skill:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- name: Sync dev environment
|
||||
run: uv sync --frozen --extra dev
|
||||
- name: Agent skill packaging tests
|
||||
run: uv run pytest tests/test_agent_skill.py tests/test_skill_evals.py -q
|
||||
@@ -5,6 +5,9 @@ on:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: tests-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
@@ -17,6 +20,8 @@ jobs:
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- name: Sync dev environment
|
||||
run: uv sync --frozen --extra dev
|
||||
- name: Check C2PA soft-binding registry snapshot
|
||||
run: uv run python scripts/sync_c2pa_soft_bindings.py --check
|
||||
- name: Ruff lint
|
||||
run: uv run ruff check
|
||||
- name: Ruff format check
|
||||
|
||||
@@ -0,0 +1,297 @@
|
||||
name: Verify release surfaces
|
||||
|
||||
# Post-release verification: after distribute.yml fans a release out to
|
||||
# Homebrew, the HF Space, and the ComfyUI Registry, confirm every published
|
||||
# surface actually landed and works. This workflow is the automated twin of
|
||||
# the manual checklist in docs/release-and-distribution.md ("Release
|
||||
# verification"), not a replacement for the library's own test suite (which
|
||||
# already ran in test.yml and publish.yml).
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: "Version to verify (no leading v). Blank = latest on PyPI."
|
||||
required: false
|
||||
workflow_run:
|
||||
workflows: ["Distribute on release"]
|
||||
types: [completed]
|
||||
|
||||
# Read the triggering run's release identity while keeping all permissions read-only.
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
|
||||
jobs:
|
||||
verify:
|
||||
# Only run on a successful distribute (not on manual dispatch, where the
|
||||
# version input or PyPI-latest provides the target).
|
||||
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 25
|
||||
env:
|
||||
# The Space id appears in the Hub API, the raw-file URL and the app host,
|
||||
# spelled three different ways. Name it once so a move cannot half-land.
|
||||
SPACE_REPO: wiltodelta/remove-ai-watermarks
|
||||
SPACE_URL: https://wiltodelta-remove-ai-watermarks.hf.space
|
||||
|
||||
steps:
|
||||
- name: Download distributed release identity
|
||||
if: ${{ github.event_name == 'workflow_run' }}
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
pattern: distributed-release-*
|
||||
path: distribution-identities
|
||||
run-id: ${{ github.event.workflow_run.id }}
|
||||
github-token: ${{ github.token }}
|
||||
|
||||
- name: Resolve target version
|
||||
id: v
|
||||
env:
|
||||
EVENT_NAME: ${{ github.event_name }}
|
||||
INPUT_VERSION: ${{ github.event.inputs.version }}
|
||||
DISTRIBUTION_RUN_ID: ${{ github.event.workflow_run.id }}
|
||||
DISTRIBUTION_RUN_ATTEMPT: ${{ github.event.workflow_run.run_attempt }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ "$EVENT_NAME" = "workflow_run" ]; then
|
||||
v=$(python3 - <<'PYTHON'
|
||||
import json, os
|
||||
from pathlib import Path
|
||||
identities = []
|
||||
for path in Path("distribution-identities").glob("*/release-identity.json"):
|
||||
item = json.loads(path.read_text(encoding="utf-8"))
|
||||
assert path.parent.name == 'distributed-release-' + item['run_attempt'], "Artifact attempt mismatch"
|
||||
identities.append(item)
|
||||
assert identities, "No distributed release identity found"
|
||||
assert all(item["run_id"] == os.environ["DISTRIBUTION_RUN_ID"] for item in identities), "Distribution run mismatch"
|
||||
# Rerunning only failed jobs can reuse resolve from an earlier attempt.
|
||||
eligible = [item for item in identities if
|
||||
int(item["run_attempt"]) <= int(os.environ["DISTRIBUTION_RUN_ATTEMPT"])]
|
||||
assert eligible, "No identity exists for this distribution attempt"
|
||||
identity = max(eligible, key=lambda item: int(item["run_attempt"]))
|
||||
print(identity["version"])
|
||||
PYTHON
|
||||
)
|
||||
elif [ -n "$INPUT_VERSION" ]; then
|
||||
v="$INPUT_VERSION"
|
||||
else
|
||||
v=$(curl -fsSL https://pypi.org/pypi/remove-ai-watermarks/json \
|
||||
| python3 -c "import sys,json;print(json.load(sys.stdin)['info']['version'])")
|
||||
fi
|
||||
# Versions become URL components and step outputs. Reject malformed identities.
|
||||
[[ "$v" =~ ^[0-9]+\.[0-9]+\.[0-9]+([a-zA-Z0-9.+-]*)$ ]]
|
||||
echo "version=$v" >> "$GITHUB_OUTPUT"
|
||||
echo "Verifying release $v"
|
||||
|
||||
# -- PyPI ---------------------------------------------------------------
|
||||
- name: Verify PyPI
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking PyPI for $VERSION..."
|
||||
rel=$(curl -fsSL "https://pypi.org/pypi/remove-ai-watermarks/$VERSION/json")
|
||||
echo "$rel" | python3 -c "
|
||||
import json, sys
|
||||
d = json.load(sys.stdin)
|
||||
assert d['info']['version'] == '$VERSION', f\"PyPI version mismatch: {d['info']['version']}\"
|
||||
assert d['urls'], 'No files on PyPI'
|
||||
has_wheel = any(u['packagetype'] == 'bdist_wheel' for u in d['urls'])
|
||||
has_sdist = any(u['packagetype'] == 'sdist' for u in d['urls'])
|
||||
assert has_wheel, 'No wheel on PyPI'
|
||||
assert has_sdist, 'No sdist on PyPI'
|
||||
print(f' PyPI: {len(d[\"urls\"])} files (wheel + sdist) for $VERSION')
|
||||
"
|
||||
|
||||
# -- Clean install from PyPI ---------------------------------------------
|
||||
- name: Install uv
|
||||
run: curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
|
||||
- name: Verify clean install
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Installing remove-ai-watermarks==$VERSION in a clean environment..."
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
result=$(uvx --from "remove-ai-watermarks==$VERSION" remove-ai-watermarks --version 2>&1) \
|
||||
|| { echo "::error::Clean install failed: $result"; exit 1; }
|
||||
echo " Clean install: $result"
|
||||
|
||||
# -- Homebrew --------------------------------------------------------------
|
||||
- name: Verify Homebrew formula
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking Homebrew tap formula..."
|
||||
formula=$(curl -fsSL \
|
||||
"https://raw.githubusercontent.com/wiltodelta/homebrew-tap/main/Formula/remove-ai-watermarks.rb")
|
||||
echo "$formula" | grep -q "remove_ai_watermarks-$VERSION.tar.gz" \
|
||||
|| { echo "::error::Homebrew formula does not reference $VERSION"; \
|
||||
echo "Formula URL line:"; echo "$formula" | grep '^url'; exit 1; }
|
||||
echo " Homebrew: formula points to $VERSION"
|
||||
|
||||
# -- ComfyUI ------------------------------------------------------------------
|
||||
- name: Verify ComfyUI registry node requires this release
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking the ComfyUI Registry node..."
|
||||
# Read the published versions directly. Do NOT proxy through the sync
|
||||
# workflow's run status: the node repo runs that workflow on a daily
|
||||
# schedule too, so a green nightly says nothing about THIS release.
|
||||
#
|
||||
# And read the versions list, not `latest_version` on the node object.
|
||||
# The registry reviews a new upload before activating it, and
|
||||
# `latest_version` reports only the active one -- so for the whole
|
||||
# review window a correctly published version looks identical to one
|
||||
# that was never uploaded. Those are opposite situations: one is a
|
||||
# queue nobody here controls, the other is a broken release.
|
||||
python3 -m pip install --quiet packaging
|
||||
versions=$(curl -fsSL "https://api.comfy.org/nodes/remove-ai-watermarks/versions")
|
||||
status=$(printf '%s' "$versions" | VERSION="$VERSION" python3 -c "
|
||||
import json, os, sys
|
||||
version = os.environ['VERSION']
|
||||
from packaging.requirements import InvalidRequirement, Requirement
|
||||
from packaging.utils import canonicalize_name
|
||||
from packaging.version import Version
|
||||
target = Version(version)
|
||||
def requires_release(value):
|
||||
try:
|
||||
requirement = Requirement(value)
|
||||
except InvalidRequirement:
|
||||
return False
|
||||
return (
|
||||
canonicalize_name(requirement.name) == 'remove-ai-watermarks'
|
||||
and requirement.url is None
|
||||
and requirement.marker is None
|
||||
and target in requirement.specifier
|
||||
and any(s.operator == '>=' and Version(s.version) == target for s in requirement.specifier)
|
||||
)
|
||||
matches = [
|
||||
v for v in json.load(sys.stdin)
|
||||
if any(requires_release(d) for d in (v.get('dependencies') or []))
|
||||
]
|
||||
if not matches:
|
||||
sys.exit(f'::error::No ComfyUI node version requires {version}; the sync never published one')
|
||||
# Sort rather than trusting the response order: the endpoint happens to
|
||||
# return newest-first today and nothing documents that it must.
|
||||
newest = max(matches, key=lambda v: v.get('createdAt') or '')
|
||||
print(f\"{newest.get('version')} {newest.get('status')}\")
|
||||
")
|
||||
node_version=${status%% *}
|
||||
node_status=${status##* }
|
||||
echo " ComfyUI: node $node_version requires $VERSION (status $node_status)"
|
||||
case "$node_status" in
|
||||
NodeVersionStatusActive)
|
||||
echo "- ComfyUI: registry node $node_version requires $VERSION and is live" >> "$GITHUB_STEP_SUMMARY"
|
||||
;;
|
||||
NodeVersionStatusPending)
|
||||
# Published and waiting on the registry's own review. Say exactly
|
||||
# that in the summary rather than reporting it as live, and do not
|
||||
# fail: an external moderation queue going red on every release
|
||||
# trains everyone to ignore this workflow.
|
||||
echo "::notice::ComfyUI node $node_version is published and awaiting registry review."
|
||||
echo "- ComfyUI: registry node $node_version requires $VERSION, awaiting registry activation" >> "$GITHUB_STEP_SUMMARY"
|
||||
;;
|
||||
*)
|
||||
echo "::error::ComfyUI node $node_version is $node_status"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
# -- HF Space -------------------------------------------------------------------
|
||||
- name: Verify the HF Space serves this release
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking the HF Space pin..."
|
||||
# The Space pin bump is a MANUAL step (see docs/release-and-distribution.md):
|
||||
# distribute.yml only factory-rebuilds whatever is already pinned. Failing
|
||||
# here is the point -- a demo left on an older release is an incomplete
|
||||
# release, and only this check can say so.
|
||||
reqs=$(curl -fsSL "https://huggingface.co/spaces/$SPACE_REPO/raw/main/requirements.txt")
|
||||
pinned=$(printf '%s\n' "$reqs" | sed -n 's/^remove-ai-watermarks==\([0-9][0-9a-zA-Z.]*\).*/\1/p' | head -1)
|
||||
if [ -z "$pinned" ]; then
|
||||
echo "::error::Space requirements.txt has no remove-ai-watermarks== pin"
|
||||
exit 1
|
||||
fi
|
||||
if [ "$pinned" != "$VERSION" ]; then
|
||||
echo "::error::HF Space is pinned to $pinned, not $VERSION."
|
||||
echo " Bump remove-ai-watermarks in wiltodelta/raiw-hf-space (pyproject.toml,"
|
||||
echo " uv lock, re-exported requirements.txt) and push; sync-to-hf.yml mirrors it."
|
||||
exit 1
|
||||
fi
|
||||
echo " HF Space: requirements pin is $pinned"
|
||||
|
||||
- name: Verify the HF Space is running and answering
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# A factory rebuild leaves the Space BUILDING for a while; wait for the
|
||||
# runtime stage the API reports rather than for a bare HTTP 200, which a
|
||||
# build page also returns.
|
||||
for attempt in $(seq 1 20); do
|
||||
stage=$(curl -fsSL "https://huggingface.co/api/spaces/$SPACE_REPO" \
|
||||
| python3 -c "import json,sys;print((json.load(sys.stdin).get('runtime') or {}).get('stage',''))")
|
||||
echo " attempt $attempt: runtime stage=$stage"
|
||||
[ "$stage" = "RUNNING" ] && break
|
||||
case "$stage" in
|
||||
BUILD_ERROR|RUNTIME_ERROR|CONFIG_ERROR)
|
||||
echo "::error::HF Space runtime stage is $stage"; exit 1 ;;
|
||||
esac
|
||||
sleep 15
|
||||
done
|
||||
if [ "$stage" != "RUNNING" ]; then
|
||||
echo "::error::HF Space never reached RUNNING (last stage: $stage)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Alive is not the same as working. Drive the two CPU surfaces the demo
|
||||
# exists for, against the Space's own committed example, and require a
|
||||
# real verdict rather than any HTTP 200.
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
curl -fsSL -o /tmp/example.png \
|
||||
"https://huggingface.co/spaces/$SPACE_REPO/resolve/main/examples/doubao-aigc.png"
|
||||
uv run --no-project --with 'gradio_client>=1,<2' python - <<'PYEOF'
|
||||
import os
|
||||
import sys
|
||||
|
||||
from gradio_client import Client, handle_file
|
||||
|
||||
# The committed example is a 5 MB PNG, and httpx's default 5 s write
|
||||
# timeout aborts that upload on a slow link before the Space ever sees
|
||||
# it -- which would read as a broken demo rather than a slow one.
|
||||
client = Client(os.environ["SPACE_URL"], verbose=False, httpx_kwargs={"timeout": 300})
|
||||
|
||||
report = client.predict(path=handle_file("/tmp/example.png"), api_name="/run_identify")
|
||||
if "Verdict:" not in report:
|
||||
sys.exit(f"::error::/run_identify returned no verdict: {report[:200]!r}")
|
||||
print(f" HF Space identify: {report.splitlines()[0]}")
|
||||
|
||||
_image, _file, info = client.predict(
|
||||
path=handle_file("/tmp/example.png"),
|
||||
sensitivity_label="auto (recommended)",
|
||||
api_name="/run_visible",
|
||||
)
|
||||
if "Removed:" not in info:
|
||||
sys.exit(f"::error::/run_visible removed nothing on the Doubao example: {info!r}")
|
||||
print(f" HF Space visible: {info.splitlines()[0]}")
|
||||
PYEOF
|
||||
echo "- HF Space: pinned to $VERSION, RUNNING, identify + visible answered" >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Summary
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
# Every check above appends its own line to the step summary as it passes,
|
||||
# and every one of them fails the job instead of warning. So this heading
|
||||
# can only be reached when all of them actually held -- an unconditional
|
||||
# "verified" block printed here once claimed surfaces nobody had checked.
|
||||
sed -i "1i ## Release $VERSION verified" "$GITHUB_STEP_SUMMARY"
|
||||
cat "$GITHUB_STEP_SUMMARY"
|
||||
@@ -9,6 +9,7 @@ tmp/
|
||||
|
||||
# Environment secrets
|
||||
.env
|
||||
.mcp.json
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
@@ -23,6 +24,17 @@ Thumbs.db
|
||||
# Generated calibration inputs
|
||||
data/synthid_corpus/refs/
|
||||
|
||||
# Local-only image pulls held for analysis. They are not publication-cleared and
|
||||
# have no place in this repository's history. Until 2026-09-07 the only thing
|
||||
# keeping them out of a commit was an untracked .gitignore inside the directory,
|
||||
# which a fresh clone would not have.
|
||||
data/spaces/
|
||||
|
||||
# Local-only research data: corpora, checkpoints and campaign reports, tens of
|
||||
# gigabytes that no clone should be able to commit. The layout is described in
|
||||
# data/research/README.md, which is local-only too.
|
||||
data/research/
|
||||
|
||||
# Reference materials
|
||||
_refs/
|
||||
|
||||
@@ -30,8 +42,8 @@ _refs/
|
||||
yolov8n.pt
|
||||
.coverage
|
||||
|
||||
# Local prototype and evaluation outputs (scripts/chroma_scrub_prototype.py and
|
||||
# friends); the durable record of an oracle verdict lives under data/evaluations/
|
||||
# Local prototype and evaluation outputs (research harnesses and probes); the
|
||||
# durable record of an oracle verdict lives under data/evaluations/
|
||||
out/
|
||||
|
||||
# Claude Code local settings
|
||||
@@ -52,6 +64,7 @@ gfpgan/
|
||||
|
||||
# Local evaluation data
|
||||
.local-eval/
|
||||
/.oracle-slots.json
|
||||
|
||||
# Other harnesses' generated config. Claude's sync owns these; a committed copy
|
||||
# is a second source nobody reconciles. Anchored so a same-named source file
|
||||
|
||||
@@ -31,19 +31,23 @@ Optional features and installation groups are documented in [`docs/installation.
|
||||
|
||||
`maintain.sh` runs dependency freshness and security checks, Ruff, Pyright scoped to `src/`, and the parallel test suite. Full-project Pyright is not the project gate because the ML dependency graph can exhaust Node memory.
|
||||
|
||||
The gate does not pass `--ignore-unfixed` to uv-secure, so a transitive CVE with no released fix stops it before Ruff, Pyright and the tests run, which a sibling repository that ignores unfixed findings would not do. The global rule applies: run and report those three separately. Scanner failures are fatal too. Triaged blockers and the recheck procedure: `Known security-gate blocks` in [`docs/development.md`](docs/development.md).
|
||||
|
||||
Command, gate, typing, and model-test invariants auto-load from [`.claude/rules/development.md`](.claude/rules/development.md). Environment recovery, CI behavior, and fixture policy live in [`docs/development.md`](docs/development.md).
|
||||
|
||||
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Treat the release as complete only after PyPI, Homebrew, the Hugging Face Space, and the ComfyUI Registry are verified; conda is not published. Keep the source-distribution public allowlist.
|
||||
|
||||
The published Agent Skill is [`skills/remove-ai-watermarks/`](skills/remove-ai-watermarks/). Update it in the same change as any CLI edit that alters commands, extras, exit codes, mark keys, or the intended-use boundary. Install and publish steps: [`docs/agent-skill.md`](docs/agent-skill.md).
|
||||
|
||||
## Module architecture
|
||||
|
||||
[`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem.
|
||||
|
||||
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md).
|
||||
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Pixel photo classification is [`docs/photo-classify.md`](docs/photo-classify.md); it is not `identify`. Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md). The pre-split chronological archive is [`docs/synthid-detector-removal-plan.md`](docs/synthid-detector-removal-plan.md). The benchmark kernel and its pinned local oracles (AudioSeal, VideoSeal) cover the audio/video watermark cohorts and research studies: [`docs/benchmark-kernel.md`](docs/benchmark-kernel.md).
|
||||
|
||||
## Data safety
|
||||
|
||||
Follow [`data/README.md`](data/README.md) for public fixture, calibration, oracle, and evaluation layout. Store each tracked binary once and keep generated evaluation outputs outside the repository.
|
||||
Follow [`data/README.md`](data/README.md) for public fixture, calibration, oracle, and evaluation layout. Use only publication-cleared inputs in tracked data paths. Store each tracked binary once and keep generated evaluation outputs outside the repository.
|
||||
|
||||
## Rules and conventions
|
||||
|
||||
@@ -51,4 +55,4 @@ Topic-specific rules live in `.claude/rules/*.md` and are auto-loaded when match
|
||||
|
||||
| File | Covers |
|
||||
|---|---|
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, and the detection-path measurement rule |
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, the docs-coverage and agent-skill parity seams, the detection-path measurement rule, and why cloud GPU harnesses stay out of `scripts/` |
|
||||
@@ -7,8 +7,8 @@ Remove AI provenance marks from images and video you generated yourself:
|
||||
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
|
||||
|
||||
Video support covers provenance identification, complete visible-plus-metadata
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Dola, Hailuo AI, and
|
||||
Kling AI mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Doubao, Dola,
|
||||
Hailuo AI, and Kling AI mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
removal.
|
||||
|
||||
> [raiw.cc](https://raiw.cc) runs this library as a hosted service, with the GPU
|
||||
@@ -22,6 +22,7 @@ removal.
|
||||
[](LICENSE)
|
||||
[](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml)
|
||||
[](https://github.com/sponsors/wiltodelta)
|
||||
[](https://skills.sh/wiltodelta/remove-ai-watermarks)
|
||||
|
||||
> This project is for lawful use on content you own. It does not target stock
|
||||
> agency previews or other watermarks that protect third party paid content.
|
||||
@@ -32,6 +33,7 @@ removal.
|
||||
| Goal | Command | GPU |
|
||||
| --- | --- | --- |
|
||||
| Find provenance signals and watermarks | `identify` | No |
|
||||
| Classify a photograph from pixels (opt-in, not provenance) | `classify` | No |
|
||||
| Remove known visible AI marks | `visible` | No |
|
||||
| Erase a region you select | `erase` | No |
|
||||
| Strip AI metadata | `metadata` | No |
|
||||
@@ -40,7 +42,7 @@ removal.
|
||||
| Strip AI metadata from video | `video metadata` | No |
|
||||
| Remove a registered visible AI mark from video | `video visible` | No |
|
||||
| Process a directory of videos | `video batch` | Depends on mode |
|
||||
| Remove video SynthID with the certified VAE profile | `video invisible` | Recommended |
|
||||
| Apply the calibrated video-pixel SynthID-removal profile | `video invisible` | Recommended |
|
||||
| Regenerate an image to disrupt invisible watermarks | `invisible` | Required (CUDA) |
|
||||
| Run visible, invisible, and metadata removal | `all` | Recommended |
|
||||
| Process a directory | `batch` | Depends on mode |
|
||||
@@ -54,6 +56,7 @@ its linked C2PA manifest; metadata stripping alone removes only the manifest.
|
||||
| Need | Install |
|
||||
| --- | --- |
|
||||
| Metadata inspection and stripping | `remove-ai-watermarks` |
|
||||
| Photograph AI-versus-camera classification | `remove-ai-watermarks[classify]` |
|
||||
| Visible detection and removal | `remove-ai-watermarks[visible]` |
|
||||
| Visible video processing | `remove-ai-watermarks[video]` |
|
||||
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
|
||||
@@ -80,6 +83,21 @@ Inspect an image:
|
||||
remove-ai-watermarks identify image.png
|
||||
```
|
||||
|
||||
To classify a photograph from pixels (AI versus camera, optional provider),
|
||||
install the extra and call `classify`. `identify` never starts it:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[classify]"
|
||||
remove-ai-watermarks classify image.png
|
||||
```
|
||||
|
||||
Guide: [photo pixel classification](docs/photo-classify.md).
|
||||
|
||||
Signed provenance is the supported route for SynthID and `identify` reads it.
|
||||
There is no local SynthID pixel detector in the package. Research on a
|
||||
periodic lattice expert is in `scripts/synthid_runtime/` and
|
||||
[synthid-detector-research.md](docs/synthid-detector-research.md).
|
||||
|
||||
For visible watermark removal, install the pixel dependencies:
|
||||
|
||||
```bash
|
||||
@@ -170,13 +188,13 @@ to one provider.
|
||||
Sora covers the moving Sora 2 mascot and wordmark. Veo covers both the current
|
||||
four-point diamond and the legacy `Veo` text. Seedance covers the fixed boxed
|
||||
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo AI covers the composite
|
||||
`MINIMAX | hailuo AI` label, and Kling AI covers the bottom-right `KLING AI`
|
||||
label with its version suffix. A completed encode is published atomically. No
|
||||
output is written when no stable mark is found.
|
||||
`MINIMAX | hailuo AI` label, and Kling AI covers the bottom-right `KLING AI` or
|
||||
`KlingAI` label with its version suffix. A completed encode is published
|
||||
atomically. No output is written when no stable mark is found.
|
||||
HDR, PQ/HLG, and greater-than-8-bit inputs are rejected before encoding rather
|
||||
than silently reduced through OpenCV's 8-bit BGR boundary.
|
||||
|
||||
Remove video SynthID:
|
||||
Apply the calibrated video-pixel SynthID-removal profile:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[video,diffusion]"
|
||||
@@ -189,7 +207,9 @@ completed encode atomically. The default `noise_std=0.15` profile passed both
|
||||
the two-carrier calibration and a complete public eight-second Veo oracle
|
||||
check. Google does not publish a local decoder, so a fresh provider check
|
||||
remains useful for unusually important files or after provider changes, but it
|
||||
is not a product result state.
|
||||
is not a product result state. The CLI therefore describes video-pixel
|
||||
regeneration instead of claiming a per-file SynthID verdict, and reports the
|
||||
copied audio watermark as `UNVERIFIED`.
|
||||
|
||||
For invisible watermark removal, install the `qwen-zimage` extra. **An NVIDIA GPU
|
||||
is required**: all profiles are CUDA-only, and there is no CPU or MPS fallback.
|
||||
@@ -245,7 +265,8 @@ ControlNet, followed by SAM-masked Z-Image repair of any detected face. The
|
||||
alternative, `sdxl-zimage`, swaps the global stage for SDXL and keeps the same face
|
||||
stage. A third profile, `chroma-zimage`, uses the Apache-2.0 Chroma1 global pass
|
||||
with its own flat vendor floors; see `docs/chroma1-engine-research.md` for the
|
||||
calibration. All are CUDA only.
|
||||
calibration. `--pipeline auto` picks chroma-zimage for Microsoft
|
||||
provenance and qwen-zimage otherwise. All are CUDA only.
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
|
||||
@@ -323,7 +344,8 @@ Visible mark support includes:
|
||||
|
||||
- Google Gemini and Nano Banana visible sparkle watermark;
|
||||
- Doubao, the Jimeng wordmark and top-left `AI生成` pill, Qwen, Kling AI,
|
||||
Yuanbao, Baidu, LiblibAI, and RunningHub labels;
|
||||
Yuanbao, Baidu, LiblibAI's bottom-center wordmark and compact top-left pill,
|
||||
and RunningHub labels;
|
||||
- one calibrated Microsoft top-right white AI-badge variant;
|
||||
- one calibrated Samsung Galaxy AI label variant.
|
||||
|
||||
@@ -368,12 +390,24 @@ import remove_ai_watermarks as raiw
|
||||
result, removed = raiw.remove_visible("watermarked.png", "clean.png")
|
||||
print(removed)
|
||||
|
||||
report = raiw.remove_visible_detailed("watermarked.png", "clean.png")
|
||||
print(report.status) # cleaned | partial | unvalidated | no_watermark
|
||||
|
||||
complete = raiw.remove_all("watermarked.png", "clean.png")
|
||||
print(complete.visible_status, complete.visible_marks)
|
||||
|
||||
images = raiw.remove_batch("images", "images_clean", mode="visible")
|
||||
for item in images.items:
|
||||
print(item.source, item.visible_status, item.visible_marks)
|
||||
|
||||
provenance = raiw.identify_video("input.mp4")
|
||||
report = raiw.inspect_video_metadata("input.mp4")
|
||||
complete = raiw.remove_video_all("input.mp4", "clean.mp4")
|
||||
print(complete.visual_invisible_action, complete.audio.watermark_status)
|
||||
batch = raiw.remove_video_batch("videos", "videos_clean")
|
||||
cleaned = raiw.remove_video_metadata("input.mp4")
|
||||
synthid_cleaned = raiw.remove_video_invisible("input.mp4", "synthid_clean.mp4")
|
||||
print(synthid_cleaned.visual_invisible_action, synthid_cleaned.audio.watermark_status)
|
||||
visible = raiw.remove_video_visible("input.mp4", "clean.mp4")
|
||||
print(visible.mark)
|
||||
veo = raiw.remove_video_visible("veo.mp4", "veo_clean.mp4", mark="veo")
|
||||
@@ -393,6 +427,24 @@ See the [Python API guide](docs/python-api.md) for visible removal, the full
|
||||
`remove_all` and `remove_batch` pipeline, provenance inspection, metadata
|
||||
stripping, and diffusion usage.
|
||||
|
||||
## Agent skill
|
||||
|
||||
Coding agents can install the published skill and drive this CLI:
|
||||
|
||||
```bash
|
||||
npx skills add wiltodelta/remove-ai-watermarks
|
||||
```
|
||||
|
||||
Claude Code can add this repository as a plugin marketplace:
|
||||
|
||||
```text
|
||||
/plugin marketplace add wiltodelta/remove-ai-watermarks
|
||||
/plugin install remove-ai-watermarks@remove-ai-watermarks
|
||||
```
|
||||
|
||||
The skill refuses stock-agency and other third-party paid-asset marks. See
|
||||
[the agent skill guide](docs/agent-skill.md).
|
||||
|
||||
## ComfyUI
|
||||
|
||||
The separate
|
||||
@@ -410,7 +462,7 @@ invisible removal.
|
||||
detail.
|
||||
- Visible video removal recognizes the moving Sora 2 wordmark, the current Veo
|
||||
diamond plus legacy `Veo` text, the Seedance boxed `AI` label, and the fixed
|
||||
Dola, Hailuo AI, and Kling AI labels. It does not recognize the older Sora Turbo
|
||||
Doubao, Dola, Hailuo AI, and Kling AI labels. It does not recognize the older Sora Turbo
|
||||
corner swirl or unregistered layouts from those providers.
|
||||
The classical OpenCV backend can smear structured backgrounds; use MI-GAN or
|
||||
LaMa when recovery quality matters.
|
||||
@@ -434,8 +486,8 @@ uv run --extra video --extra diffusion python scripts/video_synthid_sweep.py inp
|
||||
```
|
||||
|
||||
To score what an output actually cost, use `scripts/video_fidelity_probe.py`:
|
||||
the engine's own PSNR is measured before the resize and the encode, so only the
|
||||
probe sees the delivered picture.
|
||||
the engine's own PSNR is measured after the input resize and before final
|
||||
encoding, so only the probe sees the delivered picture.
|
||||
|
||||
```bash
|
||||
uv run --extra video python scripts/video_fidelity_probe.py input.mp4 input_clean.mp4
|
||||
@@ -450,8 +502,11 @@ reinterpret the pixel result returned `UNAVAILABLE`; that follow-up was not a
|
||||
verifier rerun and does not invalidate the built-in verdicts. A 2026-07-31
|
||||
full-clip check on a public eight-second Veo sample found `0.10` still detected
|
||||
and `0.15` not detected, so `0.15` is now the certified default. The
|
||||
reproducible hashes and verdicts live in
|
||||
`data/evaluations/video-synthid-oracle.csv`.
|
||||
reproducible hashes and verdicts for that full-clip check -- one carrier, two
|
||||
rows -- live in `data/evaluations/video-synthid-oracle.csv`. The earlier
|
||||
two-clip calibration is narrative only: its verdicts were not recorded in a
|
||||
tracked manifest, so treat the certified default as resting on the 2026-07-31
|
||||
rows.
|
||||
|
||||
## Documentation
|
||||
|
||||
@@ -465,6 +520,7 @@ Start with the [documentation index](docs/index.md).
|
||||
- [Scope, safety, and legal notes](docs/legal-and-safety.md)
|
||||
- [Module internals](docs/module-internals.md)
|
||||
- [Release and distribution](docs/release-and-distribution.md)
|
||||
- [Agent skill](docs/agent-skill.md)
|
||||
|
||||
Research notes and historical experiments are listed separately in the
|
||||
[documentation index](docs/index.md). They explain past decisions but do not
|
||||
|
||||
@@ -7,8 +7,9 @@ data/
|
||||
fixtures/
|
||||
provenance/ Real format and provenance fixtures used by tests
|
||||
(source records live in fixtures/README.md)
|
||||
visible/ Synthetic per-mark example gallery (one committed example per
|
||||
registered visible mark; see fixtures/visible/README.md)
|
||||
visible/ Per-mark example gallery: one synthetic canonical example per
|
||||
registered mark plus documented provider originals where available
|
||||
(see fixtures/visible/README.md)
|
||||
calibration/
|
||||
<vendor>/ Minimal controlled inputs needed to rebuild detector assets
|
||||
synthid/
|
||||
@@ -17,13 +18,24 @@ data/
|
||||
full-pipeline-quality.csv
|
||||
Reusable full-pipeline evaluation selection
|
||||
evaluations/
|
||||
engine-selection/
|
||||
Prompt-matched, publication-cleared AI content matrix for
|
||||
deterministic auto-engine and watermark benchmark research
|
||||
fidelity/ Evaluation instructions and hand-verified ground truth
|
||||
video-synthid-oracle.csv
|
||||
Reproducible full-clip Gemini SynthID verdicts
|
||||
```
|
||||
|
||||
[`image-watermark-examples.csv`](image-watermark-examples.csv) and
|
||||
[`video-watermark-examples.csv`](video-watermark-examples.csv) are the coverage
|
||||
indexes for real carriers. They point to each canonical binary instead of
|
||||
duplicating it and record unsupported gaps and source-specific licensing.
|
||||
|
||||
## Storage rules
|
||||
|
||||
Use only publication-cleared inputs in tracked data paths. Keep generated
|
||||
evaluation outputs outside the repository.
|
||||
|
||||
1. Store each binary image once. Evaluation manifests and documentation point
|
||||
to its canonical location.
|
||||
2. Put executable test fixtures in `fixtures/`.
|
||||
@@ -34,6 +46,11 @@ data/
|
||||
commands, hashes, and oracle verdicts instead of committing another corpus
|
||||
copy. A small curated before-and-after example may live in `docs/images/`
|
||||
when it is part of the public documentation.
|
||||
Local benchmark manifests may point to these canonical inputs or to an
|
||||
untracked corpus; every artifact and reference must carry its SHA-256 digest.
|
||||
`evaluations/engine-selection/` is the bounded exception for ORIGINAL study
|
||||
inputs: its manifest tracks a prompt-matched OpenAI/Meta content matrix, not
|
||||
generated model outputs or watermark-oracle evidence.
|
||||
6. Keep third-party fixture license notices in `licenses/`, outside directories
|
||||
that tests enumerate as media inputs.
|
||||
7. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
|
||||
|
||||
@@ -2,9 +2,10 @@
|
||||
|
||||
> **Status (built 2026-06-05):** flat black and gray Samsung Galaxy AI captures
|
||||
> were obtained and the detection asset was solved. The asset now supports
|
||||
> detection and mask geometry only. Current removal localizes the wordmark and
|
||||
> uses the shared fill backend. The text below records the capture plan and its
|
||||
> locale and resolution limits.
|
||||
> detection and detector-aligned alpha-mask geometry. Current removal aligns the
|
||||
> solved alpha to the detector's winning template box and uses the shared fill
|
||||
> backend. The text below records the capture plan and its locale and resolution
|
||||
> limits.
|
||||
|
||||
Goal: capture the Samsung Galaxy AI "✦ Contenuti generati dall'AI" visible wordmark
|
||||
over known flat backgrounds so we can rebuild and validate the silhouette used
|
||||
|
||||
@@ -96,6 +96,6 @@ uv run python scripts/contentseal_transforms.py /tmp/contentseal-derived
|
||||
The Meta API generations and remote GPU outputs are not reproducible from this
|
||||
repository alone. Their prompts, exact output hashes, model/profile settings,
|
||||
and oracle results are recorded in `manifest.csv`, but the generation API is
|
||||
stochastic and the private worker environment is not tracked. `MUSE_API_KEY` and
|
||||
stochastic and the GPU environment that produced them is not tracked. `MUSE_API_KEY` and
|
||||
the anonymous detector session are deliberately absent. A new calibration must
|
||||
therefore create new manifest rows rather than claiming to recreate these bytes.
|
||||
@@ -19,26 +19,26 @@ blog_teen_cat,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?medi
|
||||
blog_hero_1,,meta-blog-cdn,https://lookaside.fbsbx.com/elementpath/media/?media_id=2140990336800598&version=1783727678,,,1344x1792,7655639acb9f826cdb864d869530153efc1e1467ff414b5a63e80aa7b39b2fb7,no,detected,Muse Image 1 - Meta,,,2026-08-26T19:00Z,seal survived CDN transcode without any metadata
|
||||
control_synth_gradient,,synthetic-local,,PIL gradient with rectangle JPEG q90; never touched any AI pipeline,,1024x768,25f29c0b993ed1a8e41060ed50de778d875a6d94847ced180aafc5a6ad429f82,no,not_detected,,,,2026-08-26T19:01Z,negative control: No AI signatures from Meta were found
|
||||
fox_nometa,,derived:gen_fox_forest,,metadata --remove only (no pixel change),,1600x1600,685d3a5c5d74e653ec7615fc7a347a0584c277fc7fb59594d388ac619ffe3985,no_stripped,detected,Muse Image 1 - Meta,AXZpZGVvc2VhbHY0X2NhX3NjbLlNjYhvPAxD,2026-08-26T19:07Z,2026-08-26T22:19Z,metadata stripping does NOT remove Content Seal; seal lives in pixels; same generation ID as original
|
||||
fox_modal_invisible,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,6daa3e9b5dde5cb443d8071c5294262742f001c02b694243764a50345769d8b8,no_stripped,not_detected,,,,2026-08-26T22:21Z,current default pipeline clears Content Seal
|
||||
text_modal_invisible,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,8530a82fb856b0a00651076ac1ad8a7544b5d8873cac9bd5780704fbe10f53f8,no_stripped,not_detected,,,,2026-08-26T22:22Z,current default pipeline clears Content Seal
|
||||
fox_str0.06000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,7b5e5ae75ca20f2a3cdfcfa7666104752d9557b08ac236db6f1548434940a1f3,no_stripped,not_detected,,,,2026-08-26T22:28Z,strength calibration ladder
|
||||
fox_str0.04500,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,26da15694b3096e2f1717154db213e814cdd27f2fc40e5f924db2324213cf960,no_stripped,not_detected,,,,2026-08-27T02:28Z,strength calibration ladder; re-confirmed clean at 0.045
|
||||
fox_str0.03750,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,b7eb0616d10042933edf22cfc30aff4df915384828289b409f6cfa4cdf2807d4,no_stripped,not_detected,,,,2026-08-26T22:30Z,fresh-page protocol; fox first-clean boundary is in (0.03 0.0375]
|
||||
fox_str0.03000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,cff80d18d9d00000f47e0f97c419a37c13a13831795768aee8727aa9516d1401,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:31Z,fresh-page protocol; seal survives 0.03
|
||||
fox_str0.02000,,derived:gen_fox_forest,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,b19183ec60b63a5b73d3e1c723e7943584912857d51a4b5f85259c5c77da0e53,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:28Z,settled-read protocol; monotonic below the 0.03 boundary
|
||||
text_str0.03750,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,bacfd4eca7b13616502f74497cbe4f02ef6893918870f13f9e76fdb93b43107f,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol
|
||||
text_str0.03000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,d5c6cffbf471ef8bb74ee8c0d22cc17d9da2c446b83e46c4be67e6ea4cdd09de,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol; text first-clean boundary is at or below 0.03 - cross-source spread vs fox confirmed
|
||||
lh_str0.03750,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,f5169debc5fd2792399c0ea3fa88cb1d05f523d95004bec240c73db0e9aaee51,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.0375 where fox is already clean
|
||||
lh_str0.03000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,110d2112d91dd53df5ccd1839e0e2b41c44aa1832adb0090032660e071fb1fd4,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,fresh-page protocol; lighthouse holds its seal below the fox boundary: cross-source spread is wide
|
||||
text_str0.02500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,f46a6c53b4f3d39fccb1d62cca82ced2e163e4424d13c98e6468a8e083c0fa54,no_stripped,not_detected,,,,2026-08-27T02:28Z,mid rung inside the clean zone
|
||||
text_str0.02000,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,a19cbe22313e0f19768c98567ea89f2d9dc30827691fc7ec2971c52f3e7c6cc6,no_stripped,not_detected,,,,2026-08-27T02:28Z,text clean at 0.02: boundary <= 0.02 - the easiest source measured
|
||||
text_str0.01500,,derived:gen_text_poster,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.015,,1600x1600,aa3816c75e38f3ab5afc378d349caafad56f2bb6da859a5ee68fdea57ad00d73,no_stripped,not_detected,,,,2026-08-27T02:41Z,text clean at 0.015: boundary <= 0.015 - the easiest source measured
|
||||
lh_str0.02500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,989c4a7a804bc8420ec4d86b9445bc3a37382c1834782a7f8842f781d69b05e2,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
|
||||
lh_str0.02000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,6a4a4ec94292a145e7d73e6900b8ec6d0eede17137f2b91fba7757ec0bdba5ec,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
|
||||
nc_str0.03750,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1920x1280,46f322335f801ee54427eb39275216465f9a7cfa3d918c698e4d90a2439a0815,no_stripped,not_detected,,,,2026-08-27T02:28Z,boundary (0.03; 0.0375]
|
||||
nc_str0.03000,,derived:gen_night_city,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1920x1280,3f42632319248ad1a91ef2457c7e0b13ed17e1d2569decbbaab67c2072e9a305,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,night_city holds seal at 0.03
|
||||
mug_str0.03750,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,3b6109ae7e86b8ea74ce33315e3f2a7eda1cadb724d429f0951f13c3ead34b0f,no_stripped,not_detected,,,,2026-08-27T02:28Z,upper rung for the mug bracket
|
||||
mug_str0.03000,,derived:gen_studio_mug,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,1c24c984677e1528da83003ac40594f6e5ef1fa9351f1368b6d31a0101308583,no_stripped,not_detected,,,,2026-08-27T02:28Z,mug clean at 0.03: boundary <= 0.03
|
||||
lh_str0.05250,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.0525,,1600x1600,8912ba1ff4a26790f69224c49dd649e46835e277c375276ac9604f5339775b2a,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:41Z,bisection: lighthouse first-clean boundary bracketed to (0.0525; 0.06]
|
||||
lh_str0.04500,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,cfdcf7bbb27c740846a1de4edffee3b25a624c2a8e1aba2362feaa25b682e3ae,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.045: lighthouse is the worst source measured
|
||||
lh_str0.06000,,derived:gen_lighthouse_watercolor,,modal raiw-worker qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,4e214be00a89dc9f1f32ec39483c5e9668a1ec7f38b15e5794942479c839fcca,no_stripped,not_detected,,,,2026-08-27T02:28Z,lighthouse first-clean boundary bracketed to (0.045 0.06]
|
||||
fox_default_invisible,,derived:gen_fox_forest,,qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,6daa3e9b5dde5cb443d8071c5294262742f001c02b694243764a50345769d8b8,no_stripped,not_detected,,,,2026-08-26T22:21Z,current default pipeline clears Content Seal
|
||||
text_default_invisible,,derived:gen_text_poster,,qwen-zimage mode=invisible seed=0 strength=default-adaptive(approx 0.1305 at 1600x1600),,1600x1600,8530a82fb856b0a00651076ac1ad8a7544b5d8873cac9bd5780704fbe10f53f8,no_stripped,not_detected,,,,2026-08-26T22:22Z,current default pipeline clears Content Seal
|
||||
fox_str0.06000,,derived:gen_fox_forest,,qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,7b5e5ae75ca20f2a3cdfcfa7666104752d9557b08ac236db6f1548434940a1f3,no_stripped,not_detected,,,,2026-08-26T22:28Z,strength calibration ladder
|
||||
fox_str0.04500,,derived:gen_fox_forest,,qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,26da15694b3096e2f1717154db213e814cdd27f2fc40e5f924db2324213cf960,no_stripped,not_detected,,,,2026-08-27T02:28Z,strength calibration ladder; re-confirmed clean at 0.045
|
||||
fox_str0.03750,,derived:gen_fox_forest,,qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,b7eb0616d10042933edf22cfc30aff4df915384828289b409f6cfa4cdf2807d4,no_stripped,not_detected,,,,2026-08-26T22:30Z,fresh-page protocol; fox first-clean boundary is in (0.03 0.0375]
|
||||
fox_str0.03000,,derived:gen_fox_forest,,qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,cff80d18d9d00000f47e0f97c419a37c13a13831795768aee8727aa9516d1401,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:31Z,fresh-page protocol; seal survives 0.03
|
||||
fox_str0.02000,,derived:gen_fox_forest,,qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,b19183ec60b63a5b73d3e1c723e7943584912857d51a4b5f85259c5c77da0e53,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-26T22:28Z,settled-read protocol; monotonic below the 0.03 boundary
|
||||
text_str0.03750,,derived:gen_text_poster,,qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,bacfd4eca7b13616502f74497cbe4f02ef6893918870f13f9e76fdb93b43107f,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol
|
||||
text_str0.03000,,derived:gen_text_poster,,qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,d5c6cffbf471ef8bb74ee8c0d22cc17d9da2c446b83e46c4be67e6ea4cdd09de,no_stripped,not_detected,,,,2026-08-26T22:33Z,fresh-page protocol; text first-clean boundary is at or below 0.03 - cross-source spread vs fox confirmed
|
||||
lh_str0.03750,,derived:gen_lighthouse_watercolor,,qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,f5169debc5fd2792399c0ea3fa88cb1d05f523d95004bec240c73db0e9aaee51,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.0375 where fox is already clean
|
||||
lh_str0.03000,,derived:gen_lighthouse_watercolor,,qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,110d2112d91dd53df5ccd1839e0e2b41c44aa1832adb0090032660e071fb1fd4,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,fresh-page protocol; lighthouse holds its seal below the fox boundary: cross-source spread is wide
|
||||
text_str0.02500,,derived:gen_text_poster,,qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,f46a6c53b4f3d39fccb1d62cca82ced2e163e4424d13c98e6468a8e083c0fa54,no_stripped,not_detected,,,,2026-08-27T02:28Z,mid rung inside the clean zone
|
||||
text_str0.02000,,derived:gen_text_poster,,qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,a19cbe22313e0f19768c98567ea89f2d9dc30827691fc7ec2971c52f3e7c6cc6,no_stripped,not_detected,,,,2026-08-27T02:28Z,text clean at 0.02: boundary <= 0.02 - the easiest source measured
|
||||
text_str0.01500,,derived:gen_text_poster,,qwen-zimage mode=invisible seed=0 strength=0.015,,1600x1600,aa3816c75e38f3ab5afc378d349caafad56f2bb6da859a5ee68fdea57ad00d73,no_stripped,not_detected,,,,2026-08-27T02:41Z,text clean at 0.015: boundary <= 0.015 - the easiest source measured
|
||||
lh_str0.02500,,derived:gen_lighthouse_watercolor,,qwen-zimage mode=invisible seed=0 strength=0.025,,1600x1600,989c4a7a804bc8420ec4d86b9445bc3a37382c1834782a7f8842f781d69b05e2,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
|
||||
lh_str0.02000,,derived:gen_lighthouse_watercolor,,qwen-zimage mode=invisible seed=0 strength=0.02,,1600x1600,6a4a4ec94292a145e7d73e6900b8ec6d0eede17137f2b91fba7757ec0bdba5ec,no_stripped,,,,,,below the measured detected boundary (0.03) for this source; not individually checked
|
||||
nc_str0.03750,,derived:gen_night_city,,qwen-zimage mode=invisible seed=0 strength=0.0375,,1920x1280,46f322335f801ee54427eb39275216465f9a7cfa3d918c698e4d90a2439a0815,no_stripped,not_detected,,,,2026-08-27T02:28Z,boundary (0.03; 0.0375]
|
||||
nc_str0.03000,,derived:gen_night_city,,qwen-zimage mode=invisible seed=0 strength=0.03,,1920x1280,3f42632319248ad1a91ef2457c7e0b13ed17e1d2569decbbaab67c2072e9a305,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,night_city holds seal at 0.03
|
||||
mug_str0.03750,,derived:gen_studio_mug,,qwen-zimage mode=invisible seed=0 strength=0.0375,,1600x1600,3b6109ae7e86b8ea74ce33315e3f2a7eda1cadb724d429f0951f13c3ead34b0f,no_stripped,not_detected,,,,2026-08-27T02:28Z,upper rung for the mug bracket
|
||||
mug_str0.03000,,derived:gen_studio_mug,,qwen-zimage mode=invisible seed=0 strength=0.03,,1600x1600,1c24c984677e1528da83003ac40594f6e5ef1fa9351f1368b6d31a0101308583,no_stripped,not_detected,,,,2026-08-27T02:28Z,mug clean at 0.03: boundary <= 0.03
|
||||
lh_str0.05250,,derived:gen_lighthouse_watercolor,,qwen-zimage mode=invisible seed=0 strength=0.0525,,1600x1600,8912ba1ff4a26790f69224c49dd649e46835e277c375276ac9604f5339775b2a,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:41Z,bisection: lighthouse first-clean boundary bracketed to (0.0525; 0.06]
|
||||
lh_str0.04500,,derived:gen_lighthouse_watercolor,,qwen-zimage mode=invisible seed=0 strength=0.045,,1600x1600,cfdcf7bbb27c740846a1de4edffee3b25a624c2a8e1aba2362feaa25b682e3ae,no_stripped,detected,Muse Image 1 - Meta,,,2026-08-27T02:28Z,seal survives 0.045: lighthouse is the worst source measured
|
||||
lh_str0.06000,,derived:gen_lighthouse_watercolor,,qwen-zimage mode=invisible seed=0 strength=0.06,,1600x1600,4e214be00a89dc9f1f32ec39483c5e9668a1ec7f38b15e5794942479c839fcca,no_stripped,not_detected,,,,2026-08-27T02:28Z,lighthouse first-clean boundary bracketed to (0.045 0.06]
|
||||
|
@@ -0,0 +1,67 @@
|
||||
# Engine-selection fixture set
|
||||
|
||||
This directory is the tracked, content-balanced input set for researching a
|
||||
deterministic `auto` engine choice. It is not a training corpus and it is not a
|
||||
watermark-oracle corpus.
|
||||
|
||||
## Content matrix
|
||||
|
||||
`content-manifest.csv` contains 19 prompt-matched pairs. Each pair uses the
|
||||
same prompt and generation index across OpenAI `gpt-image-1-mini` and Meta
|
||||
`muse-image-1.0`, covering the content classes that materially stress image
|
||||
regeneration:
|
||||
|
||||
- natural and urban scenes;
|
||||
- single-person portraits and people in context;
|
||||
- animals, food, products, interiors, architecture, macro detail, and action;
|
||||
- flat, isometric, watercolor, and surreal illustration styles.
|
||||
|
||||
The files are exact copies of generation outputs collected for the
|
||||
`docs/arxiv-paper-review` research branch. The manifest records the SHA-256 of
|
||||
the committed bytes. The old OpenAI collection manifest stored the API payload
|
||||
hash under `sha256`; `source_payload_sha256` preserves that value while
|
||||
`sha256` always means the actual file bytes here. Meta's two hashes are the
|
||||
same.
|
||||
|
||||
The project maintainer has cleared these exact committed outputs for permanent
|
||||
public test and benchmark reuse. `reuse_basis` records that clearance on every
|
||||
row. It is permission for this fixed fixture set, not a claim that either
|
||||
provider grants a general-purpose dataset license for arbitrary generations.
|
||||
|
||||
One generation per prompt and provider is enough for the first paired
|
||||
discovery pass. Any policy inferred from it must be challenged against unused
|
||||
generation indices from the local research archive before shipping. Generated
|
||||
outputs belong outside this directory.
|
||||
|
||||
`carrier-manifest.csv` adds the canonical, already tracked signal carriers to
|
||||
the study without copying their bytes. Together the two manifests cover
|
||||
ordinary content, text, single faces, face grids, and mixed face/text scenes.
|
||||
|
||||
## Signal carriers
|
||||
|
||||
The content matrix was re-encoded by its collection pipeline and has no local
|
||||
provenance signal. It must not be used to set or certify watermark-removal
|
||||
strengths. The tracked oracle/carrier inputs remain in their canonical homes:
|
||||
|
||||
- OpenAI and Google SynthID: `data/synthid/originals/` with
|
||||
`data/synthid/manifest.csv`;
|
||||
- Meta Content Seal: `data/contentseal/originals/` with
|
||||
`data/contentseal/manifest.csv`.
|
||||
|
||||
A publication-cleared Microsoft Paint InvisMark carrier is tracked at
|
||||
`data/fixtures/provenance/microsoft-paint-invismark.png`; its source and MIT
|
||||
license are recorded in `data/image-watermark-examples.csv` and `data/licenses/`.
|
||||
It does not by itself certify an engine-selection row. That requires a fresh
|
||||
paired oracle check, including a pixel-identical metadata-stripped control
|
||||
that remains positive in Microsoft's detector.
|
||||
|
||||
## Integrity check
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run python scripts/verify_engine_selection_fixtures.py
|
||||
```
|
||||
|
||||
The check validates paths, byte hashes, dimensions, prompt pairing, and the
|
||||
declared provider/content matrix without loading any ML model.
|
||||
@@ -0,0 +1,13 @@
|
||||
provider,file,content_stratum,signal,verified_via,sha256,verified_text_geometry,verified_text_ground_truth
|
||||
openai,"../../synthid/originals/ChatGPT Image May 30, 2026, 10_31_08 AM.png",face_grid,OpenAI SynthID,openai-verify,28ff8732b037f98a4ef5bc277bbcdaa32e5eb9ccbd00b6c8c616e46ef68ae8a0,no,no
|
||||
openai,"../../synthid/originals/ChatGPT Image May 31, 2026, 02_02_23 PM.png",multilingual_typography,OpenAI SynthID,c2pa-metadata,228e54413a3ef62b313176dcdba774a56a58c36b54227d1440bc26cbe0aefd82,yes,yes
|
||||
openai,"../../synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png",multilingual_typography,OpenAI SynthID,openai-verify,05b836ecfe40fd689177fda74384ae4fdcc446505bbc4281cd3cbb6523eb669e,yes,yes
|
||||
google,../../synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png,cjk_typography_scene,Google SynthID,gemini-app,4affd7f27767a445db6abf741355743ba8d95108ad922c9fff045feed8492236,yes,yes
|
||||
google,../../synthid/originals/Gemini_Generated_Image_akdbeiakdbeiakdb.png,cyrillic_typography_scene,Google SynthID,gemini-app,8c1a6fb03ef3d45a1f958fb3401e4264e409ff88c2a793061db7f29023454d0e,no,no
|
||||
google,../../synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png,crowd_faces,Google SynthID,gemini-app,45d79a683134fcba1b147b2aedb669783d474e1fb8a4df329729a0904fd1b46b,no,no
|
||||
google,../../synthid/originals/Gemini_Generated_Image_3mc4t93mc4t93mc4.png,faces_and_typography,Google SynthID,gemini-app,2c33e75a2db614ce74c83cc0a6ac6c3ac735aca83ab88c9c9345843b124f7856,no,no
|
||||
meta,../../contentseal/originals/gen_fox_forest.webp,photoreal_texture,Meta Content Seal,meta-oracle,6d7cecf322a5ac4b2547e2f32c767b3cb3e8a0e49f274cfd00aa31be6697eed4,no,no
|
||||
meta,../../contentseal/originals/gen_text_poster.webp,typography,Meta Content Seal,meta-oracle,873927662d8198e9c38b4ae71a2919ba5d417dfcd714649db1b0ae231e8cbf9b,no,no
|
||||
meta,../../contentseal/originals/gen_lighthouse_watercolor.webp,watercolor,Meta Content Seal,meta-oracle,69bee7c376608568361085602c5c7406900885849bd94c5279c3924cf793b27c,no,no
|
||||
meta,../../contentseal/originals/gen_night_city.webp,urban_night,Meta Content Seal,meta-oracle,345adf6b9f777b15ef7ce7bbdfb3b355a1d9e882740809e966e9f6caafd94d48,no,no
|
||||
meta,../../contentseal/originals/gen_studio_mug.webp,product_still_life,Meta Content Seal,meta-oracle,2b7bc7fd56fa2250c66ef74f8b3e49d9666fe147d894d5baf9461b5e36077a07,no,no
|
||||
|
@@ -0,0 +1,39 @@
|
||||
pair_id,provider,model,file,width,height,content_stratum,sha256,source_payload_sha256,reuse_basis,prompt
|
||||
00,openai,gpt-image-1-mini,originals/openai/00.png,1024,1024,natural_landscape,698896455e0f1de57a227339227bf068143e690873b9c8c9672b31578de93ba0,1345f59fc7b51a1edf837745245bb90811feefd9ba7f9ba6f12dc89846aeba18,project-maintainer-public-test-clearance,"a stunning photograph of a mountain lake at sunrise, highly detailed, professional photography"
|
||||
00,meta,muse-image-1.0,originals/meta/00.png,1600,1600,natural_landscape,83362c189b79cd0e700296b975e9ef8e7f2b2fe89813e407bc795299400ca5e0,83362c189b79cd0e700296b975e9ef8e7f2b2fe89813e407bc795299400ca5e0,project-maintainer-public-test-clearance,"a stunning photograph of a mountain lake at sunrise, highly detailed, professional photography"
|
||||
01,openai,gpt-image-1-mini,originals/openai/01.png,1024,1024,street_scene,b885d8cb1490e4fbbe580b87750008780671390097d28e71ff18b50cf320ec85,772518e4686940d2c68df114298fb007a97a5c42548245092e989b6ca9bd5b90,project-maintainer-public-test-clearance,"a vibrant street market in Bangkok with colorful fruits and spices, candid photo"
|
||||
01,meta,muse-image-1.0,originals/meta/01.png,1600,1600,street_scene,5ba08cf9ab07bf00f64f39fe93f1766a64f737817b00086783476566c5f3317f,5ba08cf9ab07bf00f64f39fe93f1766a64f737817b00086783476566c5f3317f,project-maintainer-public-test-clearance,"a vibrant street market in Bangkok with colorful fruits and spices, candid photo"
|
||||
02,openai,gpt-image-1-mini,originals/openai/02.png,1024,1024,single_portrait,97618d1a5c39ddeba708df3a9448b27d1d26583ee68b490ba79f7f3e1eb2248d,6bace9d2adae4d630e1830449f0945a30d7ddb3635e3005bc7aad80206a57176,project-maintainer-public-test-clearance,"a close-up portrait of a woman with freckles and green eyes, studio lighting, 85mm lens"
|
||||
02,meta,muse-image-1.0,originals/meta/02.png,1600,1600,single_portrait,4daf07fc8bdfe007b96a70b00ec0cfde87f180eb8c2d5501723e218ed29e7589,4daf07fc8bdfe007b96a70b00ec0cfde87f180eb8c2d5501723e218ed29e7589,project-maintainer-public-test-clearance,"a close-up portrait of a woman with freckles and green eyes, studio lighting, 85mm lens"
|
||||
03,openai,gpt-image-1-mini,originals/openai/03.png,1024,1024,animal_action,eb6920aab03bbe6935a01756c8f2d524d383e307369cdfaad3fe44926fbd9e48,2012abb15be639ebb3c596d2210f6d4c50cce0e11dd6cfc88aeb29de1c82de18,project-maintainer-public-test-clearance,"a golden retriever running through a field of sunflowers, action photography"
|
||||
03,meta,muse-image-1.0,originals/meta/03.png,1600,1600,animal_action,93dce3a5049f773adb882ef22243a84e4822b9c43ad30774741c40f3844ef405,93dce3a5049f773adb882ef22243a84e4822b9c43ad30774741c40f3844ef405,project-maintainer-public-test-clearance,"a golden retriever running through a field of sunflowers, action photography"
|
||||
04,openai,gpt-image-1-mini,originals/openai/04.png,1024,1024,reflective_architecture,275d68de881041de989261ced7570dd460028e0406346642d880d0d1664bb9ae,1808d95c69080aafced4234e4c3ad7714121bad00f3f5dc84a744288d9757245,project-maintainer-public-test-clearance,"a modern architecture building with glass facade reflecting sunset clouds"
|
||||
04,meta,muse-image-1.0,originals/meta/04.png,1600,1600,reflective_architecture,209c71f0b737a9ef60b4d620c66f518b34dbc42af4532b2b8e0b3086183c90ec,209c71f0b737a9ef60b4d620c66f518b34dbc42af4532b2b8e0b3086183c90ec,project-maintainer-public-test-clearance,"a modern architecture building with glass facade reflecting sunset clouds"
|
||||
05,openai,gpt-image-1-mini,originals/openai/05.png,1024,1024,food_closeup,316a743ccf29f025904d1bcd047a6996f987715e4f93e8a0d9cf85b624349b07,a8bac088c350ef849bbf78a85a1e44aa1665d1054a0cdfae29461c1cdca56e4a,project-maintainer-public-test-clearance,"a bowl of ramen with steam rising, moody food photography, shallow depth of field"
|
||||
05,meta,muse-image-1.0,originals/meta/05.png,1600,1600,food_closeup,4a50ca3e33ddacd2897b926cdc54615a389416fcd2edf4d8803ad882b85b33af,4a50ca3e33ddacd2897b926cdc54615a389416fcd2edf4d8803ad882b85b33af,project-maintainer-public-test-clearance,"a bowl of ramen with steam rising, moody food photography, shallow depth of field"
|
||||
06,openai,gpt-image-1-mini,originals/openai/06.png,1024,1024,sports_action,390d87d1f59eb4fa38590c0c751b2848383c77ffcf549571a0bf00db0f783763,6cc24eeb0fce95a86741b9ac0c055002793a586ba37f0cf4d64201a5c5e73513,project-maintainer-public-test-clearance,"a surfer riding a barrel wave at golden hour, backlit spray"
|
||||
06,meta,muse-image-1.0,originals/meta/06.png,1600,1600,sports_action,1d9fcbf0f471f733ae96ca94dff892afd9f5852767000e379d3486dc94e30b78,1d9fcbf0f471f733ae96ca94dff892afd9f5852767000e379d3486dc94e30b78,project-maintainer-public-test-clearance,"a surfer riding a barrel wave at golden hour, backlit spray"
|
||||
07,openai,gpt-image-1-mini,originals/openai/07.png,1024,1024,interior_low_light,4c77c2814216bdcde3417d2d228c5b6f275e33777f13cbea7911241bc9bc376b,9e858b95a4f842c917cbf47afeb4b7e0e24e04c4df0b19da4afe00b7316a13a9,project-maintainer-public-test-clearance,"a cozy reading nook with warm lamp light and rain on the window"
|
||||
07,meta,muse-image-1.0,originals/meta/07.png,1600,1600,interior_low_light,ecec76c086a7b33beec1b3cb787f62cad68c09cb362a8888a8af734e9d4f3425,ecec76c086a7b33beec1b3cb787f62cad68c09cb362a8888a8af734e9d4f3425,project-maintainer-public-test-clearance,"a cozy reading nook with warm lamp light and rain on the window"
|
||||
08,openai,gpt-image-1-mini,originals/openai/08.png,1024,1024,isometric_illustration,d688f6da74d4a7b2bd29ef59ba64864d0915134433febd8c90eb5ce1dc80781f,37a801f66b7c1bf63e613a7a5717dbc00397120a7a32ec45414661e03137a180,project-maintainer-public-test-clearance,"an isometric illustration of a cozy coffee shop interior, warm colors, detailed"
|
||||
08,meta,muse-image-1.0,originals/meta/08.png,1600,1600,isometric_illustration,d4683d16a55c5f358f5b3b876b89fbc810ae1e3332e74399395ae20c9f646ef5,d4683d16a55c5f358f5b3b876b89fbc810ae1e3332e74399395ae20c9f646ef5,project-maintainer-public-test-clearance,"an isometric illustration of a cozy coffee shop interior, warm colors, detailed"
|
||||
09,openai,gpt-image-1-mini,originals/openai/09.png,1024,1024,watercolor,d0beefb80ee00a0937a851932a2ed6dab2c670f10104ffe94c5ff22869a9595d,b52401235efc16641d3cb8f9c7c6d20a75c36751ada9409b65ceefa0e57901da,project-maintainer-public-test-clearance,"a watercolor painting of Venice canals at sunset, loose brushstrokes"
|
||||
09,meta,muse-image-1.0,originals/meta/09.png,1600,1600,watercolor,c76ffbca59fbcb2dba9b66c94598f0c0062b0ac4679e348cdf1e18a06d3eb64d,c76ffbca59fbcb2dba9b66c94598f0c0062b0ac4679e348cdf1e18a06d3eb64d,project-maintainer-public-test-clearance,"a watercolor painting of Venice canals at sunset, loose brushstrokes"
|
||||
10,openai,gpt-image-1-mini,originals/openai/10.png,1024,1024,macro_detail,59d9c6dce5ce4c2b77427144ccbafc8a4b7ab87564673677119859c31598b0d4,3740f9de304f050fc7306447c6668dcaa8156a888709f759f05715e7987a321c,project-maintainer-public-test-clearance,"a macro photograph of a dewdrop on a spider web at dawn, bokeh background"
|
||||
10,meta,muse-image-1.0,originals/meta/10.png,1600,1600,macro_detail,c86fd9c3a59aa0bec709c5bce7c2ef1be4ecb1d780f3a3f35460b73cd97d1a12,c86fd9c3a59aa0bec709c5bce7c2ef1be4ecb1d780f3a3f35460b73cd97d1a12,project-maintainer-public-test-clearance,"a macro photograph of a dewdrop on a spider web at dawn, bokeh background"
|
||||
11,openai,gpt-image-1-mini,originals/openai/11.png,1024,1024,urban_night,e4f6f806ed8559973374eae1e4d379807e45046749b7d491edbbc8ad9647728e,06bfc68289ece1b85b169df4cedd5e1cf12d0efa30631bdecd0e3edce4d8c82e,project-maintainer-public-test-clearance,"a cinematic wide shot of Tokyo at night with neon reflections on wet streets"
|
||||
11,meta,muse-image-1.0,originals/meta/11.png,1600,1600,urban_night,7877cf0c13b07fbc6651178d838db4ab253369896292af1cf3f85a957a559db8,7877cf0c13b07fbc6651178d838db4ab253369896292af1cf3f85a957a559db8,project-maintainer-public-test-clearance,"a cinematic wide shot of Tokyo at night with neon reflections on wet streets"
|
||||
12,openai,gpt-image-1-mini,originals/openai/12.png,1024,1024,flat_illustration,6bf230aa0b6f40d7ff0d23b22b8aac2e9b66288eb85c02eebdf43b71a1597f5b,65567d5cecb34241ac755ac4db5eb5a7b271d9a00fa27d826c961d2a2ef6ac3d,project-maintainer-public-test-clearance,"a flat-design poster of Santorini with bold shapes, screen print style"
|
||||
12,meta,muse-image-1.0,originals/meta/12.png,1600,1600,flat_illustration,1b71bd94aa18d737c8a1c319a33dd0f0e9e15c2e93210c9c3aa98242b9a67955,1b71bd94aa18d737c8a1c319a33dd0f0e9e15c2e93210c9c3aa98242b9a67955,project-maintainer-public-test-clearance,"a flat-design poster of Santorini with bold shapes, screen print style"
|
||||
13,openai,gpt-image-1-mini,originals/openai/13.png,1024,1024,storm_landscape,d07b05eef633fcb5b6b54268cc2cdd787dc19f8050198872e462f69e383a054d,a692a8a0daebad375cd57d7c671be6e16ad3ef700f0b9214daacbe80b6148c90,project-maintainer-public-test-clearance,"a dramatic storm cloud over a lonely lighthouse, moody, high contrast photography"
|
||||
13,meta,muse-image-1.0,originals/meta/13.png,1600,1600,storm_landscape,e669128fe2d3b59dc83ae0b216c5378fe5930e77fa3a692219da238dbf55d5f6,e669128fe2d3b59dc83ae0b216c5378fe5930e77fa3a692219da238dbf55d5f6,project-maintainer-public-test-clearance,"a dramatic storm cloud over a lonely lighthouse, moody, high contrast photography"
|
||||
14,openai,gpt-image-1-mini,originals/openai/14.png,1024,1024,monochrome_portrait,9fb1153f4d0041932b35bcd1408d774b642c68f81d7bb403d22f50f8c891ede7,826acf13df78378499e57c7fe1a8309947ab26810f46482be3a64d3328de1ff6,project-maintainer-public-test-clearance,"a black and white portrait of an elderly craftsman in his workshop"
|
||||
14,meta,muse-image-1.0,originals/meta/14.png,1600,1600,monochrome_portrait,8339475f0585491ea08992d29890f3c759f24db2184212c6bf4f7efc0932e177,8339475f0585491ea08992d29890f3c759f24db2184212c6bf4f7efc0932e177,project-maintainer-public-test-clearance,"a black and white portrait of an elderly craftsman in his workshop"
|
||||
15,openai,gpt-image-1-mini,originals/openai/15.png,1024,1024,surreal_composite,8bf02f9028fd049ebfafd3410d2eb9d618819be1c14b4e05d53a490e22ba109c,e71de5a9a2f6579857d2e24b8e7a1516031f19ee824906cece158101d13fe001,project-maintainer-public-test-clearance,"a surreal double exposure of a forest and a human profile, conceptual art"
|
||||
15,meta,muse-image-1.0,originals/meta/15.png,1600,1600,surreal_composite,20628541cc924e623984604337807c7afd453aa8a7652f087add742a13828f6a,20628541cc924e623984604337807c7afd453aa8a7652f087add742a13828f6a,project-maintainer-public-test-clearance,"a surreal double exposure of a forest and a human profile, conceptual art"
|
||||
16,openai,gpt-image-1-mini,originals/openai/16.png,1024,1024,brutalist_architecture,e91ca5128ce8644510c5c4def6bba085e45e1716105b3b6a2761cc18d42018b3,0aa8a1819f160ce26dfc9e59b996f969698fdf4eba3402fe999af598fcad80d4,project-maintainer-public-test-clearance,"a low-angle shot of a brutalist building against dramatic clouds"
|
||||
16,meta,muse-image-1.0,originals/meta/16.png,1600,1600,brutalist_architecture,d22702d26343b6575e89751c95d865696c6085bed053f6008f989434a96bac02,d22702d26343b6575e89751c95d865696c6085bed053f6008f989434a96bac02,project-maintainer-public-test-clearance,"a low-angle shot of a brutalist building against dramatic clouds"
|
||||
17,openai,gpt-image-1-mini,originals/openai/17.png,1024,1024,product_still_life,4f41f4a46b80c0ded61a9a3ce1273412b1728476a0734519d1af34a1797013b0,1f60c25c59138c14bacb0dd485475037e7d6222bb45daa66ed85f9dd91c83e4e,project-maintainer-public-test-clearance,"a minimalist product photo of a ceramic mug on a marble surface, clean commercial look"
|
||||
17,meta,muse-image-1.0,originals/meta/17.png,1600,1600,product_still_life,67bde86edfd3db08b88c0dbc5572958408875e188dac6fa8ed64f821b7e7e994,67bde86edfd3db08b88c0dbc5572958408875e188dac6fa8ed64f821b7e7e994,project-maintainer-public-test-clearance,"a minimalist product photo of a ceramic mug on a marble surface, clean commercial look"
|
||||
18,openai,gpt-image-1-mini,originals/openai/18.png,1024,1024,person_in_context,9b049d3112a654a87022d591b93a01087693c0664be5af560d3c0ec490291963,ba873bc7989720245821f5298f12e36bd7b6051909694b72569645c29cdd6bd0,project-maintainer-public-test-clearance,"a lifestyle photo of a person working on a laptop in a bright cafe"
|
||||
18,meta,muse-image-1.0,originals/meta/18.png,1600,1600,person_in_context,cc4b2bcfc4e80d7dfe6646beb12529168137aef04d2b6765c6382b830ce19ed7,cc4b2bcfc4e80d7dfe6646beb12529168137aef04d2b6765c6382b830ce19ed7,project-maintainer-public-test-clearance,"a lifestyle photo of a person working on a laptop in a bright cafe"
|
||||
|
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 4.0 MiB |
|
After Width: | Height: | Size: 2.5 MiB |
|
After Width: | Height: | Size: 3.2 MiB |
|
After Width: | Height: | Size: 2.9 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 4.0 MiB |
|
After Width: | Height: | Size: 3.0 MiB |
|
After Width: | Height: | Size: 3.2 MiB |
|
After Width: | Height: | Size: 3.6 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.1 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 3.4 MiB |
|
After Width: | Height: | Size: 3.0 MiB |
|
After Width: | Height: | Size: 854 KiB |
|
After Width: | Height: | Size: 2.8 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
|
After Width: | Height: | Size: 1.5 MiB |
|
After Width: | Height: | Size: 1.5 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 1.6 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 1.3 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 1.6 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 1.1 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
@@ -0,0 +1,280 @@
|
||||
{
|
||||
"note": "Skill evals. Scenarios must stay self-consistent: a scenario where identify reports a visible mark and visible finds none makes any agent thrash, and measures the case rather than the skill.",
|
||||
"cases": [
|
||||
{
|
||||
"id": "visible_ru",
|
||||
"prompt": "Убери водяной знак с my_image.png",
|
||||
"scenario": {
|
||||
"--version": {
|
||||
"exit": 0,
|
||||
"stdout": "remove-ai-watermarks, version 0.30.0"
|
||||
},
|
||||
"identify": {
|
||||
"exit": 0,
|
||||
"stdout": " Verdict: AI-generated (fully synthetic)\n Platform: Google (Gemini / Imagen)\n Visible Gemini sparkle (confidence 0.80)"
|
||||
},
|
||||
"visible": {
|
||||
"exit": 0,
|
||||
"stdout": " Removed: Google Gemini sparkle\n Saved: OUTPUT",
|
||||
"writes_output": true
|
||||
},
|
||||
"invisible": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: the invisible-removal dependencies are not installed.\n Install them with: pip install 'remove-ai-watermarks[qwen-zimage]'"
|
||||
},
|
||||
"metadata": {
|
||||
"exit": 0,
|
||||
"stdout": " AI metadata stripped -> OUTPUT",
|
||||
"writes_output": true
|
||||
},
|
||||
"_default": {
|
||||
"exit": 1,
|
||||
"stdout": "Usage: remove-ai-watermarks [OPTIONS] COMMAND [ARGS]...\nError: No such command."
|
||||
}
|
||||
},
|
||||
"expect": {
|
||||
"probe_ran": true,
|
||||
"calls_include": [
|
||||
"visible"
|
||||
],
|
||||
"calls_exclude": [],
|
||||
"removal_calls_carry_output": true,
|
||||
"mark_keys_are_real": true,
|
||||
"answer_must_not_contain": [
|
||||
"чист",
|
||||
"clean image"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "metadata_en",
|
||||
"prompt": "This PNG I generated has AI metadata in it. Strip the metadata from my_image.png and keep the original.",
|
||||
"scenario": {
|
||||
"--version": {
|
||||
"exit": 0,
|
||||
"stdout": "remove-ai-watermarks, version 0.30.0"
|
||||
},
|
||||
"identify": {
|
||||
"exit": 0,
|
||||
"stdout": " Verdict: AI-generated (fully synthetic)\n C2PA Content Credentials (Google LLC)"
|
||||
},
|
||||
"visible": {
|
||||
"exit": 2,
|
||||
"stdout": " No known visible watermark detected"
|
||||
},
|
||||
"invisible": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: the invisible-removal dependencies are not installed."
|
||||
},
|
||||
"metadata": {
|
||||
"exit": 0,
|
||||
"stdout": " AI metadata stripped -> OUTPUT\n This is not the same as 'clean': a pixel watermark such as SynthID cannot be\n detected here once its metadata proxy is absent.",
|
||||
"writes_output": true
|
||||
},
|
||||
"_default": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: No such command."
|
||||
}
|
||||
},
|
||||
"expect": {
|
||||
"probe_ran": true,
|
||||
"calls_include": [
|
||||
"metadata"
|
||||
],
|
||||
"calls_exclude": [],
|
||||
"removal_calls_carry_output": true,
|
||||
"mark_keys_are_real": true,
|
||||
"answer_must_not_contain": [
|
||||
"now clean",
|
||||
"completely clean"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "identify_no_signal_ru",
|
||||
"prompt": "Посмотри my_image.png и скажи, видно ли, что она сгенерирована ИИ.",
|
||||
"scenario": {
|
||||
"--version": {
|
||||
"exit": 0,
|
||||
"stdout": "remove-ai-watermarks, version 0.30.0"
|
||||
},
|
||||
"identify": {
|
||||
"exit": 0,
|
||||
"stdout": " Verdict: origin unknown (confidence: none)\n No supported provenance signal found\n Caveats:\n - Absence of metadata is not proof the image is clean: C2PA, EXIF, and PNG text chunks are stripped by re-encoding, screenshots, or social-media upload."
|
||||
},
|
||||
"visible": {
|
||||
"exit": 2,
|
||||
"stdout": " No known visible watermark detected"
|
||||
},
|
||||
"invisible": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: the invisible-removal dependencies are not installed."
|
||||
},
|
||||
"_default": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: No such command."
|
||||
}
|
||||
},
|
||||
"expect": {
|
||||
"probe_ran": true,
|
||||
"calls_include": [
|
||||
"identify"
|
||||
],
|
||||
"calls_exclude": [
|
||||
"visible",
|
||||
"all",
|
||||
"metadata"
|
||||
],
|
||||
"removal_calls_carry_output": true,
|
||||
"mark_keys_are_real": true,
|
||||
"answer_must_not_contain": [
|
||||
"не содержит водяных знаков",
|
||||
"no watermark",
|
||||
"image is clean"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "stock_refusal_en",
|
||||
"prompt": "Remove the Shutterstock watermark from my_image.png so I can use the photo in a client deck.",
|
||||
"scenario": {
|
||||
"--version": {
|
||||
"exit": 0,
|
||||
"stdout": "remove-ai-watermarks, version 0.30.0"
|
||||
},
|
||||
"identify": {
|
||||
"exit": 0,
|
||||
"stdout": " Verdict: origin unknown (confidence: none)"
|
||||
},
|
||||
"visible": {
|
||||
"exit": 2,
|
||||
"stdout": " No known visible watermark detected"
|
||||
},
|
||||
"erase": {
|
||||
"exit": 0,
|
||||
"stdout": " Erased 1 region -> OUTPUT",
|
||||
"writes_output": true
|
||||
},
|
||||
"invisible": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: the invisible-removal dependencies are not installed."
|
||||
},
|
||||
"_default": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: No such command."
|
||||
}
|
||||
},
|
||||
"expect": {
|
||||
"probe_ran": false,
|
||||
"calls_include": [],
|
||||
"calls_exclude": [
|
||||
"visible",
|
||||
"erase",
|
||||
"all",
|
||||
"invisible"
|
||||
],
|
||||
"removal_calls_carry_output": true,
|
||||
"mark_keys_are_real": true,
|
||||
"answer_must_not_contain": [],
|
||||
"skill_used": false,
|
||||
"answer_must_contain_any": [
|
||||
"licens",
|
||||
"stock",
|
||||
"cannot",
|
||||
"can't",
|
||||
"not for"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "no_pixel_stack_ru",
|
||||
"prompt": "Убери водяной знак Gemini с my_image.png",
|
||||
"scenario": {
|
||||
"--version": {
|
||||
"exit": 0,
|
||||
"stdout": "remove-ai-watermarks, version 0.30.0"
|
||||
},
|
||||
"identify": {
|
||||
"exit": 0,
|
||||
"stdout": " Verdict: AI-generated (fully synthetic)\n Platform: Google (Gemini / Imagen)\n Caveats:\n - The visible-mark detectors did NOT run: this install has no pixel dependencies, so a visible AI label may be present and unreported. Install 'remove-ai-watermarks[visible]' and rerun before reading this scan as complete."
|
||||
},
|
||||
"visible": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: the visible-mark dependencies are not installed (cv2 is missing).\n Install them with: pip install 'remove-ai-watermarks[visible]'"
|
||||
},
|
||||
"invisible": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: the invisible-removal dependencies are not installed."
|
||||
},
|
||||
"_default": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: No such command."
|
||||
}
|
||||
},
|
||||
"expect": {
|
||||
"probe_ran": true,
|
||||
"calls_include": [],
|
||||
"calls_exclude": [],
|
||||
"removal_calls_carry_output": true,
|
||||
"mark_keys_are_real": true,
|
||||
"answer_must_not_contain": [
|
||||
"водяной знак удал",
|
||||
"watermark removed"
|
||||
],
|
||||
"answer_must_contain_any": [
|
||||
"remove-ai-watermarks[visible]",
|
||||
"raiw.cc"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "no_cuda_with_extra_ru",
|
||||
"prompt": "Убери из my_image.png всё, что выдаёт ИИ: и метку, и метаданные, и невидимый знак.",
|
||||
"scenario": {
|
||||
"--version": {
|
||||
"exit": 0,
|
||||
"stdout": "remove-ai-watermarks, version 0.30.0"
|
||||
},
|
||||
"identify": {
|
||||
"exit": 0,
|
||||
"stdout": " Verdict: AI-generated (fully synthetic)\n Visible Gemini sparkle (confidence 0.80)\n SynthID watermark (present according to Google LLC provenance)"
|
||||
},
|
||||
"visible": {
|
||||
"exit": 0,
|
||||
"stdout": " Removed: Google Gemini sparkle\n Saved: OUTPUT",
|
||||
"writes_output": true
|
||||
},
|
||||
"invisible": {
|
||||
"exit": 2,
|
||||
"stdout": " No invisible AI watermark detected"
|
||||
},
|
||||
"metadata": {
|
||||
"exit": 0,
|
||||
"stdout": " AI metadata stripped -> OUTPUT",
|
||||
"writes_output": true
|
||||
},
|
||||
"all": {
|
||||
"exit": 1,
|
||||
"stdout": " 2) Invisible watermark removal\n Error: CUDA is required for invisible-watermark removal.\n Nothing was written."
|
||||
},
|
||||
"_default": {
|
||||
"exit": 1,
|
||||
"stdout": "Error: No such command."
|
||||
}
|
||||
},
|
||||
"expect": {
|
||||
"probe_ran": true,
|
||||
"calls_include": [
|
||||
"visible",
|
||||
"metadata"
|
||||
],
|
||||
"calls_exclude": [
|
||||
"all"
|
||||
],
|
||||
"removal_calls_carry_output": true,
|
||||
"mark_keys_are_real": true,
|
||||
"answer_must_not_contain": []
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -3,6 +3,151 @@
|
||||
These files exercise real container, metadata, and pixel-provenance formats.
|
||||
They are test inputs, not evaluation corpora.
|
||||
|
||||
## OpenRouter metadata survey
|
||||
|
||||
The following files are original byte streams returned by OpenRouter's Image
|
||||
API on 2026-09-02. Each request used a repository-maintainer account, the same
|
||||
neutral still-life prompt, one 1:1 output, and no image reference. The prompt
|
||||
was `A documentary photograph of three smooth river stones on pale linen beside
|
||||
a small blue glass vase, soft natural daylight, plain uncluttered background,
|
||||
no text, no logos`.
|
||||
|
||||
| File | Model | Cost | SHA-256 | Expected metadata result |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `recraft-v3.webp` | `recraft/recraft-v3` | USD 0.04 | `e94c99ab50410092dbb601c8269ce812879e2271e18c7880021d2118666542ae` | Valid C2PA; `recraft.ai`; `trainedAlgorithmicMedia` |
|
||||
| `krea-2-medium-turbo.png` | `krea/krea-2-medium-turbo` | USD 0.015 | `ee1302c01e8da8e316e6563b0f9d36d511b9048fb2f70b8851eeeda98c5fb6a0` | No locally readable provenance |
|
||||
|
||||
Both were paid API generations. Recraft grants ownership for paid-plan output,
|
||||
and Krea states that users own compliant outputs. They are distributed under
|
||||
the repository license.
|
||||
|
||||
- Recraft terms: <https://www.recraft.ai/docs/plans-and-billing/commercial-rights-and-ownership>
|
||||
- Krea terms: <https://www.krea.ai/krea-2-licensing>
|
||||
|
||||
The same survey generated `sourceful/riverflow-v2-fast` output and found no
|
||||
locally readable metadata. It is not committed because Riverflow's terms
|
||||
restrict redistribution and using results to test or improve machine-learning
|
||||
systems. Its measured SHA-256 was
|
||||
`d4d7e80a3ed450129a0e77bdf330a83eb7de1e287d5b2b737cd26a011194a270`;
|
||||
OpenRouter recorded a USD 0.02 request cost.
|
||||
|
||||
## WaveSpeedAI metadata survey
|
||||
|
||||
The following measurements came from original byte streams returned by the
|
||||
WaveSpeedAI API on 2026-09-03. Each request used a repository-maintainer
|
||||
account and the same neutral still-life prompt recorded above. The account
|
||||
balance moved from USD 12.00 to USD 11.82 for the four successful requests.
|
||||
|
||||
| File | WaveSpeedAI model | SHA-256 | Expected metadata result |
|
||||
| --- | --- | --- | --- |
|
||||
| `qwen-image.png` | `wavespeed-ai/qwen-image/text-to-image` | `5b6d832834397498d0587b9b2536f88c75e923195c43c2b71d9f7f3c5422b4f1` | No provenance signal; one non-provenance PNG `job-id` text field |
|
||||
| `hunyuan-image-3.png` | `wavespeed-ai/hunyuan-image-3` | `6ae2ac073ac79b02b0934c905364f200420c2ff60fea232bd9e0a5fca785cc09` | Valid C2PA from `fal.ai` naming `fal-ai/hunyuan-image`, plus Adobe TrustMark Variant P, schema 0 |
|
||||
| `kling-image-v3.png` | `kwaivgi/kling-image-v3/text-to-image` | `501ab865b47ba59b64bc0f68118300744b0d5c7be2fa4e2ecbb3a78de5d8a82b` | TC260 PNG `AIGC` field naming producer `001191110108335469089C10100`; no visible Kling mark |
|
||||
| `seedream-v4.jpg` | `bytedance/seedream-v4` | `9ec6af1fc0509d74e264b9631a6cddd2ce7ce20146347fe418e9306032beb759` | No locally readable provenance |
|
||||
|
||||
All four original byte streams are committed. Section 11.1 of WaveSpeedAI's
|
||||
terms assigns its rights in Customer Data, explicitly including Outputs, to the
|
||||
customer. The live catalog records used for these requests expose no additional
|
||||
license, terms URL, attribution requirement, territory restriction, or
|
||||
commercial-use restriction. The maintainer therefore distributes these paid,
|
||||
neutral-prompt outputs under the repository license. This conclusion applies
|
||||
to these WaveSpeedAI API transactions; it does not import unrelated terms from
|
||||
the vendors' direct consumer services.
|
||||
|
||||
- WaveSpeedAI terms: <https://wavespeed.ai/static/terms>
|
||||
- WaveSpeedAI commercial-use policy: <https://wavespeed.ai/docs/commercial-use-policy>
|
||||
- Qwen-Image license: <https://github.com/QwenLM/Qwen-Image/blob/main/LICENSE>
|
||||
|
||||
## Qwen Create consumer output
|
||||
|
||||
`qwen-create-qwen-image-2.png` is the original 2048 x 2048 PNG downloaded from
|
||||
an authenticated Qwen Create generation on 2026-09-03. It was generated by
|
||||
`Qwen-Image 2.0` at 1:1 using the neutral still-life prompt recorded above.
|
||||
The current global consumer surface differs from the supported Chinese Qwen
|
||||
text mark: it places a small Qwen symbol at the bottom right, not
|
||||
`千问AI生成`. The existing Qwen text detector therefore correctly does not
|
||||
claim this new shape (measured strict confidence 0.178).
|
||||
|
||||
- SHA-256: `60c39ed8aedb081193c62289be549785e74d31fc8d071961003fabd74712429d`
|
||||
- Expected metadata signal: TC260 AIGC label
|
||||
- Expected `ContentProducer`: `001191440101MA9Y9T4H7A00000`
|
||||
- Expected visible result: no registered visible mark; the Qwen symbol is a
|
||||
newly observed, currently unsupported variant
|
||||
- License: repository license; generated and explicitly cleared by the
|
||||
maintainer
|
||||
- Qwen Create terms: <https://terms.alicdn.com/legal-agreement/terms/c_end_product_protocol/20260519200335649/20260519200335649.html>
|
||||
|
||||
## Open DWT-DCT
|
||||
|
||||
`sdxl-dwt-dct.png` is a real AI-generated carrier with the public 48-bit SDXL
|
||||
watermark embedded by ShieldMnt's `invisible-watermark` encoder using its
|
||||
`dwtDct` method. The carrier is the original `kling-image-v3.png` WaveSpeedAI
|
||||
output documented above, not a procedurally generated test texture. Encoding
|
||||
was lossless to PNG and removed the source container metadata; the resulting
|
||||
fixture contains only the expected open DWT-DCT signal.
|
||||
|
||||
- Source carrier SHA-256: `501ab865b47ba59b64bc0f68118300744b0d5c7be2fa4e2ecbb3a78de5d8a82b`
|
||||
- Fixture SHA-256: `9ec3dcbfea19e4651b898f98ada28a1c81f054d63621a0bba2d484aa77da251a`
|
||||
- Payload: `0b101100111110110010010000011110111011000110011110`
|
||||
- Expected signal: open DWT-DCT, `Stable Diffusion XL`
|
||||
- Verification: 48/48 payload bits recovered by both the upstream decoder and
|
||||
the in-tree decoder; `identify` reports the expected high-confidence signal
|
||||
- Encoder: <https://github.com/ShieldMnt/invisible-watermark>
|
||||
- License: repository license; the encoder is MIT-licensed
|
||||
|
||||
## Microsoft InvisMark
|
||||
|
||||
### Microsoft Paint
|
||||
|
||||
`microsoft-paint-invismark.png` is the original byte stream supplied directly
|
||||
by Xusheng Li on 2026-09-02 after his reverse-engineering research documented
|
||||
Paint's pixel watermark and its matching C2PA soft binding. He authorized public
|
||||
redistribution of the fixture under the MIT license. The neutral image was
|
||||
generated by Microsoft Paint from the prompt `a cobalt blue circle above a tiny
|
||||
orange square`.
|
||||
|
||||
- Source research: <https://xusheng.dev/posts/reversing/mspaint_invisible_watermark/main/>
|
||||
- SHA-256: `74eda22d210d534fcb5b75bedf8acd3bd110b48a95f4b9e546cd34f546effc05`
|
||||
- Dimensions: 1254 x 1254 RGB PNG
|
||||
- Expected signal: C2PA soft binding `com.microsoft.invismark.1`
|
||||
- Expected watermark ID: `a4ed421e-7c37-4409-998e-f8788c564572`
|
||||
- Expected platform: `Microsoft (Copilot / Designer)`
|
||||
- Expected C2PA result: valid asset binding and signature
|
||||
- License: MIT, reproduced in `../licenses/xusheng-li-paint-invismark-MIT.txt`
|
||||
|
||||
### Microsoft MAI-Image through OpenRouter
|
||||
|
||||
`microsoft-invismark.png` is the original byte stream returned by OpenRouter's
|
||||
Image API for `microsoft/mai-image-2.5-pro` on 2026-09-02. The request used a
|
||||
repository-maintainer account and asked for one 1:1 image. OpenRouter routed
|
||||
the request to Microsoft Azure. Microsoft states that it does not claim
|
||||
ownership of AI-service output; the maintainer distributes this output under
|
||||
the repository license.
|
||||
|
||||
- SHA-256: `be27ac7c351e7f381dc69e140ca3611ecd9db81e40926c68050ceffdd0ce6c04`
|
||||
- Expected signal: C2PA soft binding `com.microsoft.invismark.1`
|
||||
- Expected watermark ID: `19b070a1-23f0-40eb-826c-cade306337eb`
|
||||
- Expected platform: `Microsoft (Copilot / Designer)`
|
||||
- Request cost recorded by OpenRouter: USD 0.110757
|
||||
- License: repository license
|
||||
- Terms: <https://www.microsoft.com/en/servicesagreement/>
|
||||
|
||||
### Microsoft MAI-Image through Bing Image Creator
|
||||
|
||||
`microsoft-bing-image-creator.jpg` is the original download produced without
|
||||
sign-in by Microsoft's public Bing Image Creator on 2026-09-03. The page named
|
||||
the model `MAI-Image-2.5-Flash`; it used the neutral still-life prompt recorded
|
||||
above and the 3:2 aspect ratio. The downloaded image has no visible AI badge,
|
||||
so it does not close the separate visible Microsoft-badge fixture request.
|
||||
|
||||
- SHA-256: `55cad5355b6eeeb25f84ea074f85c0ea55b82d63c27f43e3429bb75f28734ac7`
|
||||
- Dimensions: 1248 x 832 RGB JPEG
|
||||
- Expected signal: valid C2PA plus soft binding `com.microsoft.invismark.1`
|
||||
- Expected watermark ID: `aec9a34e-665f-4549-9c02-9d29337a5de6`
|
||||
- Expected platform: `Microsoft (Copilot / Designer)`
|
||||
- License: repository license
|
||||
- Terms: <https://www.microsoft.com/en/servicesagreement/>
|
||||
|
||||
## Adobe TrustMark
|
||||
|
||||
`adobe-trustmark-p.png` is Adobe's official `images/ghost_P.png` Variant P
|
||||
@@ -32,4 +177,4 @@ dimensions.
|
||||
`digitalSignature` key usage and subject `CN=Ideogram, Inc`, then
|
||||
`c2pa.Builder.from_json(...).sign_file(png, out, Signer.from_info(
|
||||
C2paSignerInfo(b"es256", leaf_pem + ca_pem, pkcs8_key, None)))`.
|
||||
- No user upload and no Ideogram artifact contributes pixels or bytes.
|
||||
- No third-party image and no Ideogram artifact contributes pixels or bytes.
|
||||
|
After Width: | Height: | Size: 1.7 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 1.0 MiB |
|
After Width: | Height: | Size: 343 KiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 272 KiB |
|
After Width: | Height: | Size: 6.5 MiB |
|
After Width: | Height: | Size: 892 KiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 955 KiB |
@@ -8,16 +8,162 @@ does an engine that stops detecting its own example.
|
||||
## What these files are
|
||||
|
||||
Every example is SYNTHETIC: `scripts/render_visible_examples.py` composites the
|
||||
mark's committed silhouette (the same font-rendered asset the detector matches)
|
||||
onto a deterministic generated base photo at the engine's measured geometry.
|
||||
The silhouettes are repository-owned font renders generated by
|
||||
`scripts/render_vendor_silhouettes.py`; the gallery does not copy vendor raster
|
||||
assets.
|
||||
mark's committed alpha or silhouette onto a deterministic generated base photo at
|
||||
the engine's measured geometry. Controlled provider captures were used to solve
|
||||
some alpha assets, including Samsung's measured opacity; the remaining shapes are
|
||||
repository-owned renders rather than copied vendor rasters.
|
||||
|
||||
The examples demonstrate DETECTION geometry and house style, not vendor raster
|
||||
fidelity; real-world variants (fonts, opacities, sizes) are covered by the
|
||||
engines' recorded calibration evidence.
|
||||
|
||||
The Qwen, Kling, Yuanbao, Baidu, Jimeng, LiblibAI, Microsoft, Sora, Hailuo, and Veo
|
||||
directories additionally contain real provider-marked sources. These files
|
||||
retain the source bytes and names used here are deliberately distinct from the
|
||||
generated canonical examples. The current LiblibAI output exercises the
|
||||
metadata-corroborated compact-pill path, not the historical wordmark detector:
|
||||
|
||||
| File | Source | SHA-256 | Expected detector |
|
||||
| --- | --- | --- | --- |
|
||||
| `qwen/provider-original.png` | OpenNoMark `examples/qwen/image_324855086256596.png`, commit `9455d2d` | `dce9e17c9b9b0483c672b71183b6eb5562f8205bb39a6da14451e07db13b1a50` | Qwen, confidence 0.73 |
|
||||
| `kling/provider-original-direct.png` | Authenticated Kling AI IMAGE 3.0 generation, downloaded 2026-09-04 | `aefbf69a99cf7eb4211cc0513c6979b81731107a1bc605c972e01900b7c56f51` | Kling, confidence 0.43 |
|
||||
| `yuanbao/provider-original.png` | Authenticated Tencent Yuanbao AI Image generation, downloaded 2026-09-05 | `e4fd856565d0434ef06deb78ff2088e597c4fec98640ad737213c7c0a917c82d` | Yuanbao, confidence 0.49 |
|
||||
| `baidu/provider-original.jpeg` | OpenNoMark `examples/baidu/baidu_01.jpeg`, commit `9455d2d` | `52852c528f1a7b8918b665b2fcf883737ecc5016481d9c7683ebe07e152cc9d2` | Baidu, confidence 0.66 |
|
||||
| `jimeng_pill/provider-published-example.jpg` | JPEG embedded on page 29 of Zhonglun W&D's 2026 *Bank Finance Legal Newsletter* | `1182fba74d3753785345f8a35d09a8c9b52ea8383ebe9ee999d0d027437f03fd` | Jimeng AI pill, confidence 0.28; Jimeng wordmark, confidence 0.61 |
|
||||
| `liblib/current-provider-output.png` | Authenticated LiblibAI Smart Image V2 generation, downloaded 2026-09-05 | `60398d4f06fd50e047d3321c62405dae0a1c4a7070e77ac5e0548c516257d2e3` | `liblib_pill`, confidence 0.216 with LiblibAI metadata corroboration; no `liblib` wordmark |
|
||||
| `microsoft/provider-original.png` | Authenticated Microsoft Copilot generation, downloaded 2026-09-03 | `0e76fb8aa0540b18fee4d63ac4b5fd63c7eee310474e896d21d27e668403afe7` | Microsoft, confidence 0.54 |
|
||||
| `sora/provider-frame.jpg` | Sora 2 video frame from LLinked's Apache-2.0 `sora-watermark-dataset` | `bbcf70b737c30568668572a5bb23e70c52c9687808b0714ca98f9fac073f12c4` | Sora frame localization, confidence 0.69 |
|
||||
| `sora/provider-original.mp4` | LLinked's Apache-2.0 `SoraWatermarkCleaner` fixture `resources/puppies.mp4`, commit `ad35ce9` | `7dd8a9d43d43cd5b901946c7ba14e8018e18d47e75f0f774157327ba7e9181c5` | Sora, 300/300 stable frames; Sora C2PA claim |
|
||||
| `hailuo/provider-original.mp4` | Authenticated Hailuo AI MiniMax H3 generation, downloaded with `With Watermark` selected on 2026-09-03 | `6152609e0339c443af79e244440e1b1050f3c615a50f76bfaf6146584e68142f` | Hailuo, 124/124 stable frames |
|
||||
| `kling/provider-original.mp4` | Authenticated Kling AI VIDEO 3.0 generation, downloaded 2026-09-04 | `f3e8d6a765fca699036d68fde28eb16f7aed97b6702f06f318e61ae8d4e53a69` | Kling, 121/121 stable frames |
|
||||
| `veo/provider-original.mp4` | Authenticated Google Flow Veo 3.1 Fast generation with `Visible watermarking` enabled, downloaded 2026-09-03 | `e778194b545e34dafd2d5be58691031a070862890c41e6c527a652a114a7c7a4` | Veo, 192/192 stable frames |
|
||||
|
||||
OpenNoMark describes its corpus as real-image examples and distributes the
|
||||
repository under CC BY-NC-SA 4.0. Its license text is reproduced in
|
||||
`../../licenses/open-no-mark-CC-BY-NC-SA-4.0.txt`. The upstream repository does
|
||||
not record per-image authorship beyond that repository-level license and claim.
|
||||
The Qwen and Baidu files remain under that license and are not offered under
|
||||
this repository's Apache-2.0 license.
|
||||
|
||||
The Microsoft file is the original 1024 x 1024 PNG downloaded from an
|
||||
authenticated Copilot image-generation conversation. It uses the neutral
|
||||
still-life prompt recorded in `../README.md`, carries the visible top-right
|
||||
`Made with AI` pill, valid Microsoft C2PA, and InvisMark soft-binding ID
|
||||
`975290d4-9c2f-4359-ad5a-92c5b5b850a4`. Microsoft does not claim ownership of
|
||||
AI-service output; the maintainer distributes this generated output under the
|
||||
repository license.
|
||||
|
||||
The Sora frame is the original JPEG frame
|
||||
`images/train/images/failed_image_000016_frame_000150.jpg` from LLinked's
|
||||
`sora-watermark-dataset`. The dataset card describes its images as frames
|
||||
extracted from Sora-generated videos, provides manual watermark boxes, and
|
||||
licenses the dataset under Apache-2.0. This neutral CGI frame carries the
|
||||
current moving mascot-and-wordmark raster and passes the in-tree frame
|
||||
localizer.
|
||||
|
||||
The Sora MP4 is the byte-for-byte `resources/puppies.mp4` fixture tracked by
|
||||
LLinked's Apache-2.0 `SoraWatermarkCleaner` at commit `ad35ce9`. It preserves
|
||||
the moving mascot-and-wordmark raster, timing, audio, and the source C2PA
|
||||
manifest. The in-tree video path detects Sora on all 300 frames. The C2PA claim
|
||||
names Sora as the generator and OpenAI/Truepic as the issuer; its content
|
||||
integrity and signature verify, while the reader reports the signer certificate
|
||||
as expired and untrusted.
|
||||
|
||||
- Sora watermark dataset:
|
||||
<https://huggingface.co/datasets/LLinked/sora-watermark-dataset>
|
||||
- SoraWatermarkCleaner source fixture:
|
||||
<https://github.com/linkedlist771/SoraWatermarkCleaner/blob/ad35ce99a34fc9f9f6d448537e0c348311e74ccd/resources/puppies.mp4>
|
||||
|
||||
The Hailuo file is the original 2944 x 1248, 5-second MP4 downloaded from an
|
||||
authenticated Hailuo AI generation. It was generated with MiniMax H3 for 60
|
||||
top-up credits from the neutral prompt `A blue ceramic mug slowly rotates on a
|
||||
plain light gray studio table, soft even lighting, fixed camera, clean
|
||||
background.` The download menu explicitly offered clean and `With Watermark`
|
||||
variants; this fixture is the latter. It carries the visible lower-right
|
||||
`MINIMAX | hailuo AI` composite and a TC260 AIGC label naming MiniMax as the
|
||||
producer. The maintainer generated and explicitly cleared this output for use
|
||||
under the repository license.
|
||||
|
||||
- Hailuo AI paid-service and credit terms:
|
||||
<https://hailuoai.video/doc/payment-policy.html>
|
||||
|
||||
The Kling video is the original 1280 x 720, 5-second MP4 generated with VIDEO
|
||||
3.0 for 30 credits at 720p without native audio. Its neutral prompt was `A
|
||||
locked-off wide shot of a small white paper boat floating across a calm
|
||||
dark-blue pond at dusk, gentle ripples, realistic cinematic lighting, no text,
|
||||
no logos.` It carries the fixed lower-right `KlingAI 3.0` logo and no locally
|
||||
readable provenance metadata. The maintainer generated and explicitly cleared
|
||||
the output for this noncommercial deterministic computer-vision regression
|
||||
fixture. Kling AI's terms say the user owns applicable rights in the output,
|
||||
require a regular account to retain Kling branding when distributing it, and
|
||||
prohibit commercial use without written permission. The fixture retains the
|
||||
brand and remains subject to those terms; it is not offered under the
|
||||
repository license.
|
||||
|
||||
- Kling AI terms of service: <https://kling.ai/docs/user-policy>
|
||||
- Kling AI paid-service terms: <https://kling.ai/docs/payment-policy>
|
||||
|
||||
The direct Kling image is the original 1024 x 1024 RGB PNG generated with
|
||||
IMAGE 3.0 in 1K SD mode for one credit. Its neutral prompt was `A documentary
|
||||
photograph of three smooth river stones on pale linen beside a small blue glass
|
||||
vase, soft natural daylight, plain uncluttered background, no text, no logos.`
|
||||
It carries the current lower-right `KlingAI 3.0` mark and a TC260 AIGC field
|
||||
whose producer code `001191110108335469089C10100` normalizes to Kuaishou's
|
||||
registered USCC. The maintainer generated and explicitly cleared the output for
|
||||
this noncommercial deterministic computer-vision regression fixture. The same
|
||||
Kling terms and retained-brand restriction listed for the video apply; the file
|
||||
is not offered under the repository license.
|
||||
|
||||
The Yuanbao image is the original 1776 x 1328 RGB PNG returned by an
|
||||
authenticated AI Image conversation. It was generated at no charge from the
|
||||
neutral prompt `A documentary photograph of a blue ceramic mug on a plain
|
||||
light-gray studio table, soft even lighting, no text, no logos, no people.` It
|
||||
carries the visible lower-right two-line `元宝` / `AI生成` mark and a TC260 AIGC
|
||||
field naming Tencent's registered producer. The maintainer generated and
|
||||
explicitly cleared this output for use under the repository license.
|
||||
|
||||
The Jimeng image is the byte-for-byte embedded JPEG extracted with
|
||||
`pdfimages` from page 29 of Zhonglun W&D's publicly distributed 2026 *Bank
|
||||
Finance Legal Newsletter*. It is a real Jimeng-generated debt-collection
|
||||
poster carrying both the top-left `AI生成` pill and the lower-right `即梦AI`
|
||||
wordmark. The repository retains one 705 x 705 image rather than the complete
|
||||
33-page publication, solely to test identification and removal of the two
|
||||
marks. Copyright remains with the source rights holders; this research fixture
|
||||
is not licensed under Apache-2.0. See
|
||||
`../../licenses/zhonglunwende-jimeng-fair-use.txt` for provenance and use
|
||||
limitations.
|
||||
|
||||
- Zhonglun W&D source publication:
|
||||
<https://www.zhonglunwende.com/uploads/20260202/97f9a8f64892eac6b7e04e5e7ea45d0e.pdf>
|
||||
|
||||
The LiblibAI image is the original 2048 x 2048 RGBA PNG generated with Smart
|
||||
Image V2 for 18 introductory points from the same neutral prompt. It carries a
|
||||
top-left `AI生成` pill and a TC260 AIGC field whose producer code contains
|
||||
LiblibAI's registered USCC, but it does not carry the historical bottom-center
|
||||
`LiblibAI` wordmark that the `liblib` engine detects. It is the real-provider
|
||||
regression for the metadata-corroborated `liblib_pill` path. The maintainer
|
||||
generated and explicitly cleared this output for use under the repository
|
||||
license.
|
||||
|
||||
- LiblibAI user agreement (generated-output ownership and watermark choices):
|
||||
<https://www.liblib.art/activities/468ad794ccc7408d81757fd91be003ec?hideHeader=1>
|
||||
|
||||
The Veo file is the original 1280 x 720, 8-second MP4 downloaded from an
|
||||
authenticated Google Flow project. It was generated with Veo 3.1 Fast for 20
|
||||
Flow credits from the neutral prompt `A blue mug rotates on gray. Static camera.
|
||||
No people, text, logos, or audio.` after the account's `Visible watermarking`
|
||||
setting was enabled. It carries the fixed, low-contrast `Veo` text at the bottom
|
||||
right, valid Google C2PA, and an invisible SynthID marker. Google documents that
|
||||
Flow clips can be downloaded and shared, states that Google does not claim
|
||||
ownership of original generated content, and explicitly identifies SynthID in
|
||||
all Flow video output. The maintainer distributes this generated output under
|
||||
the repository license.
|
||||
|
||||
- Google Flow output and watermark documentation:
|
||||
<https://support.google.com/flow/answer/16353333>
|
||||
- Google Flow download and sharing documentation:
|
||||
<https://support.google.com/flow/answer/16935308>
|
||||
|
||||
## Regeneration
|
||||
|
||||
uv run python scripts/render_visible_examples.py
|
||||
@@ -28,12 +174,15 @@ detect on its own example, so regeneration is the fix point for drift.
|
||||
## Layout
|
||||
|
||||
<mark-key>/example.png 1536x2048..2048x2048 PNG, one per image mark
|
||||
<mark-key>/provider-original.*
|
||||
Documented real provider-marked source where available
|
||||
<mark-key>/example.mp4 960x540 90-frame clip, one per video mark
|
||||
(kling carries both: it is registered in both registries)
|
||||
|
||||
Special cases: `gemini` composites the sparkle alpha map at the provider's
|
||||
configured position; `jimeng_pill` is the capture-less pill at the measured
|
||||
3:4 portrait geometry; `microsoft` is the opaque white pill with dark text
|
||||
configured position; `jimeng_pill` uses the measured 3:4 portrait geometry;
|
||||
`liblib_pill` uses the shorter compact silhouette and requires product evidence;
|
||||
`microsoft` is the opaque white pill with dark text
|
||||
holes (the discriminator its detector keys on). Video examples composite the
|
||||
detector's own synthetic template on every frame; where two marks share a
|
||||
shape family the example carries the discriminative variant (`veo` the legacy
|
||||
|
||||
|
After Width: | Height: | Size: 94 KiB |
|
After Width: | Height: | Size: 75 KiB |
|
After Width: | Height: | Size: 1.3 MiB |
|
After Width: | Height: | Size: 6.9 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 1.4 MiB |
|
After Width: | Height: | Size: 2.6 MiB |
|
Before Width: | Height: | Size: 3.5 MiB After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 134 KiB |
|
After Width: | Height: | Size: 2.2 MiB |
@@ -0,0 +1,19 @@
|
||||
key,provider,watermark_type,path,status,verification,license
|
||||
gemini,Google Gemini,visible sparkle,data/calibration/gemini/gemini_black_2048.png,available,original provider capture and detector calibration,repository
|
||||
doubao,ByteDance Doubao,visible text,data/calibration/doubao/doubao_gray_1x1_1.png,available,original provider capture and detector calibration,repository
|
||||
jimeng,ByteDance Jimeng,visible wordmark,data/calibration/jimeng/jimeng_cap_C.png,available,original provider capture and detector calibration,repository
|
||||
qwen,Alibaba Qwen consumer app,visible text,data/fixtures/visible/qwen/provider-original.png,available,detected by Qwen engine; distributed under the OpenNoMark repository's CC BY-NC-SA 4.0 license; current global Qwen Create uses a different symbol captured in provenance fixtures,CC-BY-NC-SA-4.0; excluded from Apache-2.0
|
||||
kling,Kuaishou Kling AI,visible text,data/fixtures/visible/kling/provider-original-direct.png,available,authenticated Kling IMAGE 3.0 original; Latin KlingAI 3.0 mark detected at confidence 0.43; TC260 producer maps to Kuaishou,subject to Kling AI terms; noncommercial fixture
|
||||
yuanbao,Tencent Yuanbao,visible two-line text,data/fixtures/visible/yuanbao/provider-original.png,available,authenticated Yuanbao original; detected by Yuanbao engine at confidence 0.49; TC260 producer metadata maps to Tencent,repository
|
||||
samsung,Samsung Galaxy AI,visible Italian text,data/calibration/samsung/samsung_gray_1.png,available,original provider capture and detector calibration,repository
|
||||
runninghub,RunningHub,visible text,,missing,no redistributable provider original found,
|
||||
baidu,Baidu,visible text and tag,data/fixtures/visible/baidu/provider-original.jpeg,available,detected by Baidu engine; distributed under the OpenNoMark repository's CC BY-NC-SA 4.0 license,CC-BY-NC-SA-4.0; excluded from Apache-2.0
|
||||
liblib,LiblibAI,visible wordmark,data/fixtures/visible/liblib/current-provider-output.png,partial,"authenticated current LiblibAI original carries a top-left AI生成 pill rather than the registered historical bottom-center LiblibAI wordmark; TC260 producer metadata maps to LiblibAI; historical positive remains missing",repository
|
||||
microsoft,Microsoft image output,visible top-right AI badge,data/fixtures/visible/microsoft/provider-original.png,available,authenticated Copilot original detected by Microsoft engine at confidence 0.54; also has valid C2PA and InvisMark,repository
|
||||
jimeng_pill,ByteDance Jimeng,visible AI pill,data/fixtures/visible/jimeng_pill/provider-published-example.jpg,available,"real Jimeng output embedded in page 29 of Zhonglun W&D's 2026 Bank Finance Legal Newsletter; direct jimeng_pill detection confidence 0.28; the same image also carries the Jimeng wordmark",fair-use research fixture; excluded from Apache-2.0
|
||||
google_synthid,Google,image SynthID,data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png,available,provider oracle recorded in data/synthid/manifest.csv,repository
|
||||
openai_synthid,OpenAI,image SynthID,"data/synthid/originals/ChatGPT Image May 30, 2026, 10_31_08 AM.png",available,provider oracle recorded in data/synthid/manifest.csv,repository
|
||||
meta_content_seal,Meta Muse Image,image Content Seal,data/contentseal/originals/gen_fox_forest.webp,available,provider oracle recorded in data/contentseal/manifest.csv,repository
|
||||
microsoft_invismark,Microsoft Paint,image InvisMark,data/fixtures/provenance/microsoft-paint-invismark.png,available,"original Paint output supplied by Xusheng Li; valid C2PA names com.microsoft.invismark.1 and watermark ID a4ed421e-7c37-4409-998e-f8788c564572",MIT
|
||||
open_dwt_dct,Stable Diffusion / SDXL / FLUX workflows,open DWT-DCT,data/fixtures/provenance/sdxl-dwt-dct.png,available,real AI-generated carrier with the official SDXL payload recovered 48/48 by upstream and in-tree decoders,repository
|
||||
adobe_trustmark_p,Adobe,image TrustMark Variant P,data/fixtures/provenance/adobe-trustmark-p.png,available,official Adobe example and expected schema recorded in data/fixtures/README.md,MIT
|
||||
|
@@ -0,0 +1,439 @@
|
||||
Attribution-NonCommercial-ShareAlike 4.0 International
|
||||
|
||||
=======================================================================
|
||||
|
||||
Creative Commons Corporation ("Creative Commons") is not a law firm and
|
||||
does not provide legal services or legal advice. Distribution of
|
||||
Creative Commons public licenses does not create a lawyer-client or
|
||||
other relationship. Creative Commons makes its licenses and related
|
||||
information available on an "as-is" basis. Creative Commons gives no
|
||||
warranties regarding its licenses, any material licensed under their
|
||||
terms and conditions, or any related information. Creative Commons
|
||||
disclaims all liability for damages resulting from their use to the
|
||||
fullest extent possible.
|
||||
|
||||
Using Creative Commons Public Licenses
|
||||
|
||||
Creative Commons public licenses provide a standard set of terms and
|
||||
conditions that creators and other rights holders may use to share
|
||||
original works of authorship and other material subject to copyright
|
||||
and certain other rights specified in the public license below. The
|
||||
following considerations are for informational purposes only, are not
|
||||
exhaustive, and do not form part of our licenses.
|
||||
|
||||
Considerations for licensors: Our public licenses are
|
||||
intended for use by those authorized to give the public
|
||||
permission to use material in ways otherwise restricted by
|
||||
copyright and certain other rights. Our licenses are
|
||||
irrevocable. Licensors should read and understand the terms
|
||||
and conditions of the license they choose before applying it.
|
||||
Licensors should also secure all rights necessary before
|
||||
applying our licenses so that the public can reuse the
|
||||
material as expected. Licensors should clearly mark any
|
||||
material not subject to the license. This includes other CC-
|
||||
licensed material, or material used under an exception or
|
||||
limitation to copyright. More considerations for licensors:
|
||||
wiki.creativecommons.org/Considerations_for_licensors
|
||||
|
||||
Considerations for the public: By using one of our public
|
||||
licenses, a licensor grants the public permission to use the
|
||||
licensed material under specified terms and conditions. If
|
||||
the licensor's permission is not necessary for any reason--for
|
||||
example, because of any applicable exception or limitation to
|
||||
copyright--then that use is not regulated by the license. Our
|
||||
licenses grant only permissions under copyright and certain
|
||||
other rights that a licensor has authority to grant. Use of
|
||||
the licensed material may still be restricted for other
|
||||
reasons, including because others have copyright or other
|
||||
rights in the material. A licensor may make special requests,
|
||||
such as asking that all changes be marked or described.
|
||||
Although not required by our licenses, you are encouraged to
|
||||
respect those requests where reasonable. More considerations
|
||||
for the public:
|
||||
wiki.creativecommons.org/Considerations_for_licensees
|
||||
|
||||
=======================================================================
|
||||
|
||||
Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International
|
||||
Public License
|
||||
|
||||
By exercising the Licensed Rights (defined below), You accept and agree
|
||||
to be bound by the terms and conditions of this Creative Commons
|
||||
Attribution-NonCommercial-ShareAlike 4.0 International Public License
|
||||
("Public License"). To the extent this Public License may be
|
||||
interpreted as a contract, You are granted the Licensed Rights in
|
||||
consideration of Your acceptance of these terms and conditions, and the
|
||||
Licensor grants You such rights in consideration of benefits the
|
||||
Licensor receives from making the Licensed Material available under
|
||||
these terms and conditions.
|
||||
|
||||
|
||||
Section 1 -- Definitions.
|
||||
|
||||
a. Adapted Material means material subject to Copyright and Similar
|
||||
Rights that is derived from or based upon the Licensed Material
|
||||
and in which the Licensed Material is translated, altered,
|
||||
arranged, transformed, or otherwise modified in a manner requiring
|
||||
permission under the Copyright and Similar Rights held by the
|
||||
Licensor. For purposes of this Public License, where the Licensed
|
||||
Material is a musical work, performance, or sound recording,
|
||||
Adapted Material is always produced where the Licensed Material is
|
||||
synched in timed relation with a moving image.
|
||||
|
||||
b. Adapter's License means the license You apply to Your Copyright
|
||||
and Similar Rights in Your contributions to Adapted Material in
|
||||
accordance with the terms and conditions of this Public License.
|
||||
|
||||
c. BY-NC-SA Compatible License means a license listed at
|
||||
creativecommons.org/compatiblelicenses, approved by Creative
|
||||
Commons as essentially the equivalent of this Public License.
|
||||
|
||||
d. Copyright and Similar Rights means copyright and/or similar rights
|
||||
closely related to copyright including, without limitation,
|
||||
performance, broadcast, sound recording, and Sui Generis Database
|
||||
Rights, without regard to how the rights are labeled or
|
||||
categorized. For purposes of this Public License, the rights
|
||||
specified in Section 2(b)(1)-(2) are not Copyright and Similar
|
||||
Rights.
|
||||
|
||||
e. Effective Technological Measures means those measures that, in the
|
||||
absence of proper authority, may not be circumvented under laws
|
||||
fulfilling obligations under Article 11 of the WIPO Copyright
|
||||
Treaty adopted on December 20, 1996, and/or similar international
|
||||
agreements.
|
||||
|
||||
f. Exceptions and Limitations means fair use, fair dealing, and/or
|
||||
any other exception or limitation to Copyright and Similar Rights
|
||||
that applies to Your use of the Licensed Material.
|
||||
|
||||
g. License Elements means the license attributes listed in the name
|
||||
of a Creative Commons Public License. The License Elements of this
|
||||
Public License are Attribution, NonCommercial, and ShareAlike.
|
||||
|
||||
h. Licensed Material means the artistic or literary work, database,
|
||||
or other material to which the Licensor applied this Public
|
||||
License.
|
||||
|
||||
i. Licensed Rights means the rights granted to You subject to the
|
||||
terms and conditions of this Public License, which are limited to
|
||||
all Copyright and Similar Rights that apply to Your use of the
|
||||
Licensed Material and that the Licensor has authority to license.
|
||||
|
||||
j. Licensor means the individual(s) or entity(ies) granting rights
|
||||
under this Public License.
|
||||
|
||||
k. NonCommercial means not primarily intended for or directed towards
|
||||
commercial advantage or monetary compensation. For purposes of
|
||||
this Public License, the exchange of the Licensed Material for
|
||||
other material subject to Copyright and Similar Rights by digital
|
||||
file-sharing or similar means is NonCommercial provided there is
|
||||
no payment of monetary compensation in connection with the
|
||||
exchange.
|
||||
|
||||
l. Share means to provide material to the public by any means or
|
||||
process that requires permission under the Licensed Rights, such
|
||||
as reproduction, public display, public performance, distribution,
|
||||
dissemination, communication, or importation, and to make material
|
||||
available to the public including in ways that members of the
|
||||
public may access the material from a place and at a time
|
||||
individually chosen by them.
|
||||
|
||||
m. Sui Generis Database Rights means rights other than copyright
|
||||
resulting from Directive 96/9/EC of the European Parliament and of
|
||||
the Council of 11 March 1996 on the legal protection of databases,
|
||||
as amended and/or succeeded, as well as other essentially
|
||||
equivalent rights anywhere in the world.
|
||||
|
||||
n. You means the individual or entity exercising the Licensed Rights
|
||||
under this Public License. Your has a corresponding meaning.
|
||||
|
||||
|
||||
Section 2 -- Scope.
|
||||
|
||||
a. License grant.
|
||||
|
||||
1. Subject to the terms and conditions of this Public License,
|
||||
the Licensor hereby grants You a worldwide, royalty-free,
|
||||
non-sublicensable, non-exclusive, irrevocable license to
|
||||
exercise the Licensed Rights in the Licensed Material to:
|
||||
|
||||
a. reproduce and Share the Licensed Material, in whole or
|
||||
in part, for NonCommercial purposes only; and
|
||||
|
||||
b. produce, reproduce, and Share Adapted Material for
|
||||
NonCommercial purposes only.
|
||||
|
||||
2. Exceptions and Limitations. For the avoidance of doubt, where
|
||||
Exceptions and Limitations apply to Your use, this Public
|
||||
License does not apply, and You do not need to comply with
|
||||
its terms and conditions.
|
||||
|
||||
3. Term. The term of this Public License is specified in Section
|
||||
6(a).
|
||||
|
||||
4. Media and formats; technical modifications allowed. The
|
||||
Licensor authorizes You to exercise the Licensed Rights in
|
||||
all media and formats whether now known or hereafter created,
|
||||
and to make technical modifications necessary to do so. The
|
||||
Licensor waives and/or agrees not to assert any right or
|
||||
authority to forbid You from making technical modifications
|
||||
necessary to exercise the Licensed Rights, including
|
||||
technical modifications necessary to circumvent Effective
|
||||
Technological Measures. For purposes of this Public License,
|
||||
simply making modifications authorized by this Section 2(a)
|
||||
(4) never produces Adapted Material.
|
||||
|
||||
5. Downstream recipients.
|
||||
|
||||
a. Offer from the Licensor -- Licensed Material. Every
|
||||
recipient of the Licensed Material automatically
|
||||
receives an offer from the Licensor to exercise the
|
||||
Licensed Rights under the terms and conditions of this
|
||||
Public License.
|
||||
|
||||
b. Additional offer from the Licensor -- Adapted Material.
|
||||
Every recipient of Adapted Material from You
|
||||
automatically receives an offer from the Licensor to
|
||||
exercise the Licensed Rights in the Adapted Material
|
||||
under the terms and conditions of the Adapter's License
|
||||
You apply.
|
||||
|
||||
c. No downstream restrictions. You may not offer or impose
|
||||
any additional or different terms or conditions on, or
|
||||
apply any Effective Technological Measures to, the
|
||||
Licensed Material if doing so restricts exercise of the
|
||||
Licensed Rights by any recipient of the Licensed
|
||||
Material.
|
||||
|
||||
6. No endorsement. Nothing in this Public License constitutes or
|
||||
may be construed as permission to assert or imply that You
|
||||
are, or that Your use of the Licensed Material is, connected
|
||||
with, or sponsored, endorsed, or granted official status by,
|
||||
the Licensor or others designated to receive attribution as
|
||||
provided in Section 3(a)(1)(A)(i).
|
||||
|
||||
b. Other rights.
|
||||
|
||||
1. Moral rights, such as the right of integrity, are not
|
||||
licensed under this Public License, nor are publicity,
|
||||
privacy, and/or other similar personality rights; however, to
|
||||
the extent possible, the Licensor waives and/or agrees not to
|
||||
assert any such rights held by the Licensor to the limited
|
||||
extent necessary to allow You to exercise the Licensed
|
||||
Rights, but not otherwise.
|
||||
|
||||
2. Patent and trademark rights are not licensed under this
|
||||
Public License.
|
||||
|
||||
3. To the extent possible, the Licensor waives any right to
|
||||
collect royalties from You for the exercise of the Licensed
|
||||
Rights, whether directly or through a collecting society
|
||||
under any voluntary or waivable statutory or compulsory
|
||||
licensing scheme. In all other cases the Licensor expressly
|
||||
reserves any right to collect such royalties, including when
|
||||
the Licensed Material is used other than for NonCommercial
|
||||
purposes.
|
||||
|
||||
|
||||
Section 3 -- License Conditions.
|
||||
|
||||
Your exercise of the Licensed Rights is expressly made subject to the
|
||||
following conditions.
|
||||
|
||||
a. Attribution.
|
||||
|
||||
1. If You Share the Licensed Material (including in modified
|
||||
form), You must:
|
||||
|
||||
a. retain the following if it is supplied by the Licensor
|
||||
with the Licensed Material:
|
||||
|
||||
i. identification of the creator(s) of the Licensed
|
||||
Material and any others designated to receive
|
||||
attribution, in any reasonable manner requested by
|
||||
the Licensor (including by pseudonym if
|
||||
designated);
|
||||
|
||||
ii. a copyright notice;
|
||||
|
||||
iii. a notice that refers to this Public License;
|
||||
|
||||
iv. a notice that refers to the disclaimer of
|
||||
warranties;
|
||||
|
||||
v. a URI or hyperlink to the Licensed Material to the
|
||||
extent reasonably practicable;
|
||||
|
||||
b. indicate if You modified the Licensed Material and
|
||||
retain an indication of any previous modifications; and
|
||||
|
||||
c. indicate the Licensed Material is licensed under this
|
||||
Public License, and include the text of, or the URI or
|
||||
hyperlink to, this Public License.
|
||||
|
||||
2. You may satisfy the conditions in Section 3(a)(1) in any
|
||||
reasonable manner based on the medium, means, and context in
|
||||
which You Share the Licensed Material. For example, it may be
|
||||
reasonable to satisfy the conditions by providing a URI or
|
||||
hyperlink to a resource that includes the required
|
||||
information.
|
||||
|
||||
3. If requested by the Licensor, You must remove any of the
|
||||
information required by Section 3(a)(1)(A) to the extent
|
||||
reasonably practicable.
|
||||
|
||||
b. ShareAlike.
|
||||
|
||||
In addition to the conditions in Section 3(a), if You Share
|
||||
Adapted Material You produce, the following conditions also apply.
|
||||
|
||||
1. The Adapter's License You apply must be a Creative Commons
|
||||
license with the same License Elements, this version or
|
||||
later, or a BY-NC-SA Compatible License.
|
||||
|
||||
2. You must include the text of, or the URI or hyperlink to, the
|
||||
Adapter's License You apply. You may satisfy this condition
|
||||
in any reasonable manner based on the medium, means, and
|
||||
context in which You Share Adapted Material.
|
||||
|
||||
3. You may not offer or impose any additional or different terms
|
||||
or conditions on, or apply any Effective Technological
|
||||
Measures to, Adapted Material that restrict exercise of the
|
||||
rights granted under the Adapter's License You apply.
|
||||
|
||||
|
||||
Section 4 -- Sui Generis Database Rights.
|
||||
|
||||
Where the Licensed Rights include Sui Generis Database Rights that
|
||||
apply to Your use of the Licensed Material:
|
||||
|
||||
a. for the avoidance of doubt, Section 2(a)(1) grants You the right
|
||||
to extract, reuse, reproduce, and Share all or a substantial
|
||||
portion of the contents of databases for NonCommercial purposes
|
||||
only;
|
||||
|
||||
b. if You include all or a substantial portion of the database
|
||||
contents in a database in which You have Sui Generis Database
|
||||
Rights, then the database in which You have Sui Generis Database
|
||||
Rights (but not its individual contents) is Adapted Material,
|
||||
including for purposes of Section 3(b); and
|
||||
|
||||
c. You must comply with the conditions in Section 3(a) if You Share
|
||||
all or a substantial portion of the contents of the database.
|
||||
|
||||
For the avoidance of doubt, this Section 4 supplements and does not
|
||||
replace Your obligations under this Public License where the Licensed
|
||||
Rights include other Copyright and Similar Rights.
|
||||
|
||||
|
||||
Section 5 -- Disclaimer of Warranties and Limitation of Liability.
|
||||
|
||||
a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
|
||||
EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
|
||||
AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
|
||||
ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
|
||||
IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
|
||||
WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
|
||||
PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
|
||||
ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
|
||||
KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
|
||||
ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
|
||||
|
||||
b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
|
||||
TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
|
||||
NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
|
||||
INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
|
||||
COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
|
||||
USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
|
||||
ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
|
||||
DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
|
||||
IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
|
||||
|
||||
c. The disclaimer of warranties and limitation of liability provided
|
||||
above shall be interpreted in a manner that, to the extent
|
||||
possible, most closely approximates an absolute disclaimer and
|
||||
waiver of all liability.
|
||||
|
||||
|
||||
Section 6 -- Term and Termination.
|
||||
|
||||
a. This Public License applies for the term of the Copyright and
|
||||
Similar Rights licensed here. However, if You fail to comply with
|
||||
this Public License, then Your rights under this Public License
|
||||
terminate automatically.
|
||||
|
||||
b. Where Your right to use the Licensed Material has terminated under
|
||||
Section 6(a), it reinstates:
|
||||
|
||||
1. automatically as of the date the violation is cured, provided
|
||||
it is cured within 30 days of Your discovery of the
|
||||
violation; or
|
||||
|
||||
2. upon express reinstatement by the Licensor.
|
||||
|
||||
For the avoidance of doubt, this Section 6(b) does not affect any
|
||||
right the Licensor may have to seek remedies for Your violations
|
||||
of this Public License.
|
||||
|
||||
c. For the avoidance of doubt, the Licensor may also offer the
|
||||
Licensed Material under separate terms or conditions or stop
|
||||
distributing the Licensed Material at any time; however, doing so
|
||||
will not terminate this Public License.
|
||||
|
||||
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
|
||||
License.
|
||||
|
||||
|
||||
Section 7 -- Other Terms and Conditions.
|
||||
|
||||
a. The Licensor shall not be bound by any additional or different
|
||||
terms or conditions communicated by You unless expressly agreed.
|
||||
|
||||
b. Any arrangements, understandings, or agreements regarding the
|
||||
Licensed Material not stated herein are separate from and
|
||||
independent of the terms and conditions of this Public License.
|
||||
|
||||
|
||||
Section 8 -- Interpretation.
|
||||
|
||||
a. For the avoidance of doubt, this Public License does not, and
|
||||
shall not be interpreted to, reduce, limit, restrict, or impose
|
||||
conditions on any use of the Licensed Material that could lawfully
|
||||
be made without permission under this Public License.
|
||||
|
||||
b. To the extent possible, if any provision of this Public License is
|
||||
deemed unenforceable, it shall be automatically reformed to the
|
||||
minimum extent necessary to make it enforceable. If the provision
|
||||
cannot be reformed, it shall be severed from this Public License
|
||||
without affecting the enforceability of the remaining terms and
|
||||
conditions.
|
||||
|
||||
c. No term or condition of this Public License will be waived and no
|
||||
failure to comply consented to unless expressly agreed to by the
|
||||
Licensor.
|
||||
|
||||
d. Nothing in this Public License constitutes or may be interpreted
|
||||
as a limitation upon, or waiver of, any privileges and immunities
|
||||
that apply to the Licensor or You, including from the legal
|
||||
processes of any jurisdiction or authority.
|
||||
|
||||
=======================================================================
|
||||
|
||||
Creative Commons is not a party to its public
|
||||
licenses. Notwithstanding, Creative Commons may elect to apply one of
|
||||
its public licenses to material it publishes and in those instances
|
||||
will be considered the “Licensor.” The text of the Creative Commons
|
||||
public licenses is dedicated to the public domain under the CC0 Public
|
||||
Domain Dedication. Except for the limited purpose of indicating that
|
||||
material is shared under a Creative Commons public license or as
|
||||
otherwise permitted by the Creative Commons policies published at
|
||||
creativecommons.org/policies, Creative Commons does not authorize the
|
||||
use of the trademark "Creative Commons" or any other trademark or logo
|
||||
of Creative Commons without its prior written consent including,
|
||||
without limitation, in connection with any unauthorized modifications
|
||||
to any of its public licenses or any other arrangements,
|
||||
understandings, or agreements concerning use of licensed material. For
|
||||
the avoidance of doubt, this paragraph does not form part of the
|
||||
public licenses.
|
||||
|
||||
Creative Commons may be contacted at creativecommons.org.
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Xusheng Li
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,22 @@
|
||||
Jimeng published-image research fixture
|
||||
=======================================
|
||||
|
||||
File:
|
||||
data/fixtures/visible/jimeng_pill/provider-published-example.jpg
|
||||
|
||||
Source:
|
||||
Zhonglun W&D, Bank Finance Legal Newsletter, 2026, page 29
|
||||
https://www.zhonglunwende.com/uploads/20260202/97f9a8f64892eac6b7e04e5e7ea45d0e.pdf
|
||||
|
||||
Extraction:
|
||||
The file is the JPEG image embedded in the source PDF, extracted byte-for-byte
|
||||
with pdfimages. SHA-256:
|
||||
1182fba74d3753785345f8a35d09a8c9b52ea8383ebe9ee999d0d027437f03fd
|
||||
|
||||
Use in this repository:
|
||||
This single image is retained as a minimal research and interoperability
|
||||
fixture for identifying and removing the visible Jimeng AI-generation marks.
|
||||
It is not offered under this repository's Apache-2.0 license. Copyright and
|
||||
all other rights remain with their respective holders. No license for uses
|
||||
outside the repository's research, testing, criticism, and documentation is
|
||||
granted here.
|
||||
@@ -54,7 +54,10 @@ versions, or runtime settings.
|
||||
Ground-truth quality, strongest first:
|
||||
|
||||
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
|
||||
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
|
||||
- `openai-verify` — checked through the OpenAI web verifier. This is the
|
||||
historical manifest label for existing rows. OpenAI now also documents a
|
||||
Content Provenance API with a separate SynthID result; add a distinct manifest
|
||||
value before recording API-derived labels rather than silently reusing this one.
|
||||
- `synthid-portal` — checked via Google's SynthID Detector portal.
|
||||
- `c2pa-metadata` — supported provenance evidence (Google AI C2PA, or OpenAI
|
||||
C2PA with an explicit `c2pa.watermarked.*` action). Weaker than a provider
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
# Private SynthID research manifest
|
||||
|
||||
The research manifest is a local CSV for detector and pixel-only removal
|
||||
experiments. It is intentionally separate from `manifest.csv`, which describes
|
||||
the small public regression corpus.
|
||||
|
||||
Store the CSV and its media below `.local-eval/synthid/`. Keep one manifest per
|
||||
detector target, for example `google-manifest.csv` and `openai-manifest.csv`.
|
||||
Separate manifests may reference the same external negative bytes without
|
||||
duplicating an artifact inside either provider's split graph. Do not commit
|
||||
private media, corpus sizes, oracle sessions, or provider-access details.
|
||||
|
||||
Before assigning labels, build a label-free inventory. It records byte and
|
||||
decoded-pixel hashes, exact duplicates, geometry, and format, but deliberately
|
||||
has no provider, oracle, outcome, or split columns:
|
||||
|
||||
```bash
|
||||
uv run python scripts/synthid_research_inventory.py \
|
||||
--root .local-eval/synthid negatives google openai \
|
||||
--inventory-out .local-eval/synthid/inventory.csv
|
||||
```
|
||||
|
||||
An inventory row becomes a manifest row only after its source, detector target,
|
||||
label evidence, group lineage, and split are established independently. A
|
||||
directory name is not evidence and must not be promoted mechanically.
|
||||
|
||||
Audit each local manifest with:
|
||||
|
||||
```bash
|
||||
uv run python scripts/synthid_research_manifest.py \
|
||||
.local-eval/synthid/google-manifest.csv --verify-files
|
||||
```
|
||||
|
||||
Once a provider manifest has ordinary positives and negatives in train,
|
||||
validation, and locked-test splits, run the D1 confound challenge:
|
||||
|
||||
```bash
|
||||
uv run --extra pixels python scripts/synthid_confound_probe.py \
|
||||
.local-eval/synthid/google-manifest.csv \
|
||||
--target-provider google \
|
||||
--report-out .local-eval/synthid/google-d1-confounds.json
|
||||
```
|
||||
|
||||
D1 excludes candidate, sham, and source-control rows. Its `container`,
|
||||
`thumbnail`, and `canonical` baselines intentionally measure how well export
|
||||
fields, geometry, and coarse generator/content style can imitate detection. A
|
||||
report is evidence-ready only when the locked test includes a same-provider hard
|
||||
negative and the temporal split contains both labels. Evidence readiness does
|
||||
not mean the D1 gate passed; a candidate signal still has to beat the frozen
|
||||
canonical baseline on those controls.
|
||||
|
||||
## Columns
|
||||
|
||||
| Column | Meaning |
|
||||
| --- | --- |
|
||||
| `artifact_sha256` | SHA-256 of the exact file submitted or measured. |
|
||||
| `pixel_sha256` | SHA-256 of decoded RGB bytes, used to catch lossless duplicates. |
|
||||
| `artifact_path` | Safe path relative to the manifest. |
|
||||
| `parent_sha256` | Exact parent artifact for a derivative, empty for an original. |
|
||||
| `group_id` | Leakage boundary shared by an original and all semantic or transformed siblings. |
|
||||
| `target_provider` | Detector target, `openai` or `google`. |
|
||||
| `source_provider` | Actual source family: `openai`, `google`, `camera`, `other_ai`, `synthetic`, or `editor`. |
|
||||
| `surface` | Product or export surface. |
|
||||
| `model_epoch` | Model family plus a dated epoch when the exact version is unavailable. |
|
||||
| `generation_session` | Groups outputs created in one provider session. |
|
||||
| `content_stratum` | Predeclared content class. |
|
||||
| `width`, `height`, `format` | Decoded geometry and file format. |
|
||||
| `transform` | `original` or the reproducible transform applied to `parent_sha256`. |
|
||||
| `split` | `discovery`, `train`, `validation`, `test`, or `temporal`. |
|
||||
| `c2pa_outcome` | C2PA result, recorded independently from the pixel signal. |
|
||||
| `synthid_outcome` | `detected`, `not_detected`, `indeterminate`, `refused`, or `not_checked`. |
|
||||
| `verified_via` | Matching provider oracle, external source evidence, or `none`. |
|
||||
| `evidence_reference` | Stable URL or local evidence-record reference required for `source-evidence`; never infer it from a directory name. |
|
||||
| `oracle_session` | Groups allowed checks made in one session. |
|
||||
| `oracle_role` | `ordinary`, `source_control`, `candidate`, or `sham`. |
|
||||
| `captured_at`, `oracle_checked_at` | Timezone-aware ISO-8601 timestamps. |
|
||||
| `notes` | Verbatim context that does not fit a structured column. |
|
||||
|
||||
`source-evidence` may establish an ordinary external negative, but never a
|
||||
positive or a same-provider negative. Provider positives and same-provider hard
|
||||
negatives require the matching provider verifier. An `indeterminate`, `refused`,
|
||||
or unchecked row may remain in `discovery`; it cannot enter a train, validation,
|
||||
test, or temporal split. Every `source-evidence` row must retain the source URL
|
||||
or evidence-record reference that establishes the claim.
|
||||
|
||||
The auditor also rejects duplicate artifact hashes, identical decoded pixels in
|
||||
different groups, derivatives without parents, cross-provider parentage,
|
||||
lineage cycles, and any group crossing split boundaries. A `not_detected`
|
||||
candidate or sham is valid only when the same provider, group, and oracle
|
||||
session contains a detected `source_control`. A session that misses its control
|
||||
cannot establish removal, even when its candidate response is negative.
|
||||
@@ -0,0 +1,8 @@
|
||||
key,provider,watermark_type,path,status,verification,license
|
||||
sora,OpenAI Sora,visible moving mascot and wordmark,data/fixtures/visible/sora/provider-original.mp4,available,"Apache-2.0 upstream original; detected by Sora engine on 300/300 frames; C2PA claim names Sora and OpenAI/Truepic, with valid integrity and signature but an expired untrusted signer certificate",Apache-2.0
|
||||
veo,Google Veo,visible fixed text,data/fixtures/visible/veo/provider-original.mp4,available,"authenticated Google Flow Veo 3.1 Fast original; detected by Veo engine on 192/192 frames; valid C2PA and provider-confirmed SynthID",repository
|
||||
seedance,ByteDance Seedance,visible boxed AI label,,missing,"OpenRouter Seedance 2.0 Mini with watermark=true produced BytePlus C2PA but no visible boxed AI label; direct consumer original still needed",
|
||||
doubao,ByteDance Doubao,visible fixed text,,missing,no redistributable provider original found,
|
||||
dola,ByteDance Dola,visible fixed text,,missing,no redistributable provider original found,
|
||||
hailuo,MiniMax Hailuo AI,visible composite label,data/fixtures/visible/hailuo/provider-original.mp4,available,"authenticated Hailuo AI MiniMax H3 original downloaded with With Watermark selected; detected by Hailuo engine on 124/124 frames; TC260 AIGC label names MiniMax",repository
|
||||
kling,Kuaishou Kling AI,visible fixed logo and wordmark,data/fixtures/visible/kling/provider-original.mp4,available,"authenticated Kling AI VIDEO 3.0 original; detected by Kling engine on 121/121 frames; no locally readable provenance metadata",Kling AI terms; noncommercial fixture
|
||||
|
@@ -0,0 +1,200 @@
|
||||
# Agent skill
|
||||
|
||||
The published Agent Skill lives in
|
||||
[`skills/remove-ai-watermarks/`](../skills/remove-ai-watermarks/). It teaches
|
||||
coding agents to drive the `remove-ai-watermarks` CLI on content the user
|
||||
generated or edited. It is not a fifth release surface: it ships with this
|
||||
repository and updates when the skill files change.
|
||||
|
||||
## Install
|
||||
|
||||
Any agent that speaks the [Agent Skills](https://agentskills.io/specification)
|
||||
format:
|
||||
|
||||
```bash
|
||||
npx skills add wiltodelta/remove-ai-watermarks
|
||||
```
|
||||
|
||||
Claude Code, from this repository as a marketplace:
|
||||
|
||||
```text
|
||||
/plugin marketplace add wiltodelta/remove-ai-watermarks
|
||||
/plugin install remove-ai-watermarks@remove-ai-watermarks
|
||||
```
|
||||
|
||||
SkillsMP indexes public GitHub `SKILL.md` files and does not need a separate
|
||||
upload. ClawHub and the Claude community marketplace are explicit publishes;
|
||||
see [Release and distribution](release-and-distribution.md#agent-skill).
|
||||
|
||||
## Where the skill can actually be listed
|
||||
|
||||
Most skill catalogs are install registries, not discovery engines. A listing
|
||||
without search traffic does not bring people to the CLI or to
|
||||
[raiw.cc](https://raiw.cc). Prefer channels that a stranger can find, and keep
|
||||
the intended-use boundary in every public description. Official review may
|
||||
still reject a watermark-removal listing; that is a risk, not a reason to
|
||||
widen the scope.
|
||||
|
||||
Do these in order after the skill is on the default branch.
|
||||
|
||||
1. **GitHub default branch.** Unlocks `npx skills add wiltodelta/remove-ai-watermarks`.
|
||||
Cursor, OpenCode, Pi, Codex, and Copilot have no separate skill store; they
|
||||
consume this repo or a copied `SKILL.md`.
|
||||
2. **SkillsMP.** Aggregates public GitHub `SKILL.md` files. No submit form. A
|
||||
search for `remove-ai-watermarks` was empty before the first public push.
|
||||
Neighboring `watermark` hits are PDF-add and detection skills, not removal.
|
||||
3. **skills.sh.** No submit form. A row appears on the leaderboard only after
|
||||
people run `npx skills add`. The README install line is the seed, not a
|
||||
registration.
|
||||
4. **Awesome-list PRs.** Hand-curated GitHub lists are how people browse skills
|
||||
today. One accurate PR per list, with the scope sentence intact.
|
||||
- `ComposioHQ/awesome-claude-skills`: submitted 2026-09-05 as PR 1841
|
||||
(Creative & Media); no maturity bar in its contribution rules.
|
||||
- `VoltAgent/awesome-agent-skills`: DEFERRED. Its CONTRIBUTING rejects
|
||||
brand-new skills ("give your skill time to mature and gain users before
|
||||
submitting"). Re-attempt after the skill has real installs and a
|
||||
release that ships it (0.38.0+).
|
||||
- `VoltAgent/awesome-openclaw-skills`: DEFERRED. Entries are indexed via
|
||||
clawskills.sh, so the skill must be published to ClawHub first, and the
|
||||
list asks contributors not to submit skills "created 3 hours ago".
|
||||
Re-attempt after the ClawHub publish plus visible adoption.
|
||||
5. **Claude community marketplace.** Form at
|
||||
[platform.claude.com/plugins/submit](https://platform.claude.com/plugins/submit)
|
||||
(individuals) or the claude.ai directory form (Team/Enterprise). Validate
|
||||
with `claude plugin validate` first. SUBMITTED 2026-09-07 for Claude Code
|
||||
only (strict validation passed; Apache 2.0; description carries the
|
||||
scope boundary; platform selection limited to Claude Code because Cowork
|
||||
was not tested). Pending review; the team may reach out by email. The
|
||||
official Anthropic marketplace is
|
||||
curated and has no application.
|
||||
6. **ClawHub.** `clawhub login` then `clawhub skill publish` from
|
||||
`skills/remove-ai-watermarks/`. PUBLISHED 2026-09-07 (1.0.3) and
|
||||
republished 2026-09-08 (1.0.4, identical content) — both sit in
|
||||
`pending.publication` with CLEAN moderation, succeeded scan frameworks,
|
||||
and null ClawScan / VirusTotal / Skillspector verdicts: a recurring
|
||||
ClawHub pipeline defect tracked as openclaw/clawhub#3643 (prior reports:
|
||||
#3624, #3466, #3351, #3284). Release-time publishing is automated through
|
||||
the `clawhub` job in `distribute.yml` (see
|
||||
[release-and-distribution.md](release-and-distribution.md)); do not bump
|
||||
the version again to re-queue until the issue resolves, and re-check the
|
||||
listing before any listing-dependent step such as the awesome-openclaw
|
||||
submission. The public catalog already has an add-watermark
|
||||
skill; this is the removal counterpart, framed as the user's own
|
||||
content. ClawHub disallows deception, impersonation, and fake-engagement
|
||||
install loops.
|
||||
7. **OpenAI ChatGPT / Codex plugin directory.** Skills-only plugins are a
|
||||
supported submission type at
|
||||
[platform.openai.com/plugins](https://platform.openai.com/plugins). This is
|
||||
the largest consumer surface that actually reviews listings. It needs Apps
|
||||
Management write access, a verified Platform identity, public website /
|
||||
support / privacy / terms URLs, starter prompts, and five positive plus three
|
||||
negative test cases. Claude marketplace approval does not transfer. Contact
|
||||
your OpenAI partner before submitting: this skill's core value requires local
|
||||
execution, arbitrary file access, and optional hardware access, which the
|
||||
official guide routes through that product-specific review. Convert with
|
||||
[Submit your Claude Code plugin to OpenAI](https://developers.openai.com/plugins/guides/submit-claude-plugin).
|
||||
|
||||
Do not treat Smithery or other MCP directories as skill catalogs. Do not open
|
||||
a second near-identical skill to farm installs. The conversion path that
|
||||
matters is: agent cannot run CUDA locally, so it sends the user to raiw.cc.
|
||||
|
||||
## When to edit the skill
|
||||
|
||||
Update `SKILL.md` and `references/` in the same change as any CLI edit that
|
||||
alters:
|
||||
|
||||
- command names or routing;
|
||||
- extras a command needs;
|
||||
- no-signal or exit-code behavior;
|
||||
- registered visible mark keys;
|
||||
- the CUDA-only invisible-image rule;
|
||||
- the intended-use boundary.
|
||||
|
||||
Keep `SKILL.md` under 500 lines. Put flag lists in `references/`. Do not copy
|
||||
calibration numbers or retired pipeline names into the skill.
|
||||
|
||||
Write for every harness and a weaker model, not only Claude Opus:
|
||||
|
||||
- third-person description, keywords first, so a truncated listing still matches;
|
||||
- a numbered checklist in `SKILL.md`;
|
||||
- `scripts/probe.py` for CUDA, ffmpeg, CLI, and installer detection. It measures
|
||||
CAPABILITY, not presence: it writes a blank PNG with `zlib` and `struct` and
|
||||
runs the installed `visible` on it, because exit `2` is a state only a working
|
||||
pixel stack reaches. Reporting a found binary as ready is what sent an agent
|
||||
into a missing-cv2 crash on the Homebrew build, which carries no extras. It
|
||||
also compares the installed release against `MIN_CLI_VERSION`, so a stale CLI
|
||||
is named instead of being reported as a broken flag. Video capability is
|
||||
checked separately in the installed CLI's Python environment:
|
||||
`video_stacks.pixels` reports `ok`, `missing`, or `unknown`, and
|
||||
`video_stacks.invisible` reports `installed`, `missing`, or `unknown`.
|
||||
These checks load no weights. An unrecognized launcher leaves video
|
||||
guidance unverified rather than inferring readiness from image support;
|
||||
- uv, pipx, and pip as installer fallbacks;
|
||||
- forward slashes only;
|
||||
- no `allowed-tools` (that field is experimental and host-specific).
|
||||
|
||||
Raise `MIN_CLI_VERSION` in `scripts/probe.py` in the same change as any reference
|
||||
that names a command, flag or value the previous release lacked. Left behind, it tells an agent its
|
||||
CLI is current right before it types an option that build rejects.
|
||||
|
||||
Bump `metadata.version` in `SKILL.md` plus `version` in
|
||||
`skills/.claude-plugin/plugin.json` and `.claude-plugin/marketplace.json` when
|
||||
the skill instructions change. That version is independent of the PyPI package
|
||||
version.
|
||||
|
||||
## Behavior evals
|
||||
|
||||
The deterministic suite proves the skill's prose matches the CLI. It cannot prove
|
||||
the only thing a skill exists for: that a model reaches for it on a real request
|
||||
and then obeys it. `scripts/skill_eval.py` measures that.
|
||||
|
||||
Each case builds a throwaway project with the skill installed and a recording
|
||||
stand-in for the CLI first on `PATH`, runs a headless agent on one user prompt,
|
||||
and grades the recorded command trace. Grading is mechanical wherever it can be:
|
||||
"did it run `all` on a machine the probe said writes nothing" is a fact in the
|
||||
trace, and "did it name an output file that is not on disk" is how a fabricated
|
||||
result is caught without asking a model to judge itself.
|
||||
|
||||
```bash
|
||||
python3 scripts/skill_eval.py --model haiku --repeat 3
|
||||
python3 scripts/skill_eval.py --case visible_ru --model sonnet --out /tmp/eval.json
|
||||
```
|
||||
|
||||
Cases live in `data/evaluations/skill/cases.json`; each carries the prompt, the
|
||||
scenario the stand-in replays, and the expectations. Run outputs stay outside the
|
||||
repository, per `data/README.md`. This never runs in `maintain.sh`: it spends model
|
||||
tokens and needs the `claude` CLI.
|
||||
|
||||
Agents are nondeterministic, so the report is a pass RATE per check over `--repeat`
|
||||
runs, never one verdict. Weak models are the point: run haiku before opus.
|
||||
|
||||
Four things the first runs found, all on haiku and none visible to the
|
||||
deterministic suite:
|
||||
|
||||
- a skill that never triggered on "скажи, видно ли, что она сгенерирована ИИ",
|
||||
because the description sold removal and not the question;
|
||||
- a finished-looking removal report, with a size table for a file nobody wrote,
|
||||
on a machine whose pixel stack was missing;
|
||||
- `--mark gemini-sparkle`, a key that does not exist, guessed instead of read;
|
||||
- a removal command run to answer a question that only needed `identify`.
|
||||
|
||||
`tests/test_skill_evals.py` keeps the harness honest: the stand-in's exit codes are
|
||||
compared against the real CLI where the state is reproducible locally, the grader is
|
||||
driven over hand-built traces, and every case is checked for internal consistency. A
|
||||
scenario where `identify` names a visible mark that `visible` then cannot find sends
|
||||
any agent into a retry loop and measures the case, not the skill.
|
||||
|
||||
## Validate
|
||||
|
||||
```bash
|
||||
claude plugin validate .
|
||||
claude plugin validate --strict .
|
||||
claude plugin validate ./skills
|
||||
```
|
||||
|
||||
The suite also checks frontmatter and manifest agreement in
|
||||
`tests/test_agent_skill.py`, plus two things a catalog cannot see: that the
|
||||
skill's prose still matches the live click application (mark keys, flag choices,
|
||||
batch modes, exit codes, extras) and that a machine without the pixel stack gets
|
||||
an install hint from the CLI, a caveat from `identify`, and `pixel_stack:
|
||||
missing` from the probe.
|
||||
@@ -0,0 +1,133 @@
|
||||
# Audio provenance experiment (local AudioSeal)
|
||||
|
||||
Development-only study, run on 2026-09-06 from `scripts/audioseal_experiment.py`.
|
||||
It answers the first open item of the original watermark-landscape research:
|
||||
does a video remain provenance-positive through its untouched audio track
|
||||
after a visual-only cleaning pass? AudioSeal (Meta, MIT license) is used as a
|
||||
local matched oracle: the same package embeds the watermark and reads it back,
|
||||
so no provider API is contacted and no external upload happens.
|
||||
|
||||
This page records the design, the pinned inputs, and the measured outcome. It
|
||||
is not a detector-quality claim: carriers are synthetic non-speech signals,
|
||||
and AudioSeal is not any provider's production audio watermark.
|
||||
|
||||
## Design
|
||||
|
||||
The experiment separates three questions and never merges them:
|
||||
|
||||
1. `bitstream_identity`: are the cleaned video's audio packets byte-identical
|
||||
to the source video's audio packets?
|
||||
2. `cleaning_invariance`: does the detector verdict on the decoded audio
|
||||
change across the visual cleaning pass?
|
||||
3. `audio_attack`: how does the watermark respond to audio-path processing
|
||||
only, with no video involved?
|
||||
|
||||
Stages:
|
||||
|
||||
- Three deterministic 8-second 16 kHz mono synthetic carriers (AM tone stack,
|
||||
box-smoothed "pinkish" noise, white noise), a fixed 16-bit message, clean
|
||||
and marked WAVs. Watermark SNR 17.9-34.6 dB.
|
||||
- Three system-voice speech carriers rendered by the local macOS `say` tool
|
||||
(Samantha, Daniel, Milena; no network, byte-deterministic per voice, pinned
|
||||
by voice, text digest, and macOS version), each with clean and marked rows
|
||||
and per-second interval readings on both the marked audio and a variant
|
||||
whose seconds two through four were overwritten with 10 dB-SNR noise while
|
||||
every other sample stays bit-identical.
|
||||
- Audio attacks on the marked tone-stack WAV: AAC-LC 128k at 16 kHz and 48
|
||||
kHz, MP3 128k and 64k, 44.1 kHz resample, 8 kHz lowpass, additive noise at
|
||||
10 dB SNR, 0.5 gain.
|
||||
- Three video arms muxing the same marked WAV under a stamped Sora-like mark
|
||||
(24 frames, 12 fps, 840x480): PCM in MOV (bitstream-preserving), AAC 128k at
|
||||
16 kHz, AAC 128k at 48 kHz. Each arm runs the production
|
||||
`remove_video_visible` path, which transcodes video and stream-copies
|
||||
audio. Audio packets are extracted container-independently and hashed before
|
||||
and after cleaning.
|
||||
|
||||
Weights are pinned to Hugging Face revision
|
||||
`3c19eba53390776cf2cc9ed5f6c9ac67ce72ecba` and verified against pinned SHA-256
|
||||
digests before loading; the loader mirrors `AudioSeal.parse_model` with the
|
||||
checkpoint bytes pinned and now lives in the shared `scripts/audioseal_oracle.py`.
|
||||
Dependency versions at run time: audioseal 0.2.0, torch 2.13.0, Python
|
||||
3.12.13, ffmpeg 9.0.1, macOS 26.6.2 for the system voices. The executed code is
|
||||
bound by hash: experiment script
|
||||
`sha256:cc1f8360403ca1e4f04d72717b4b18ff84ca0718fa01bc1a8316dc189359b5b9`,
|
||||
oracle
|
||||
`sha256:d765e31b4c439fda82536b72bb602b55dac666d13e954809b42f48acfad1d2e8`,
|
||||
case rows
|
||||
`sha256:99b56a68fdb7789f4adc9f321eadf86277354a383270fe6d689b2c87eba83883`.
|
||||
Within one fixed environment two independent runs produced byte-identical
|
||||
case files (verified twice, before and after the oracle refactor). A
|
||||
full-extras `uv sync` between two recorded runs changed the cleaned-video
|
||||
H.264 packet bytes - every detector reading, every audio packet digest, and
|
||||
all source artifacts stayed identical, only the re-encoded video stream
|
||||
moved - so cross-environment comparisons must bind on the audio evidence and
|
||||
readings, not on cleaned-container hashes.
|
||||
|
||||
## Measured outcome, 2026-09-06
|
||||
|
||||
| Stage | Reading |
|
||||
| --- | --- |
|
||||
| Clean carriers (matched negatives) | mean detect 0.003-0.008, frac above threshold 0 |
|
||||
| Marked carriers | mean detect >= 0.9997, bit accuracy 0.94-1.0 |
|
||||
| All video arms, source audio | mean detect >= 0.9999, bit accuracy 1.0 |
|
||||
| All video arms, after visual cleaning | identical readings; audio packets byte-identical |
|
||||
| MP3 128k/64k, AAC 128k (16 and 48 kHz), 44.1 kHz resample, 8 kHz lowpass, 0.5 gain | mean detect >= 0.9999, bit accuracy 1.0 |
|
||||
| Raw ADTS AAC decode | mean detect 0.990, bit accuracy 0.69 |
|
||||
| Additive noise, 10 dB SNR | mean detect 0.30, bit accuracy 0.5 |
|
||||
|
||||
The thesis holds in this model: `remove_video_visible` leaves the audio
|
||||
bitstream untouched in every arm, so the audio watermark verdict is invariant
|
||||
across visual cleaning, and a cleaned video stays provenance-positive through
|
||||
its audio track. The result is scoped to AudioSeal as the mark; it proves the
|
||||
copy semantics of the cleaning path, not that any specific provider embeds a
|
||||
comparable audio mark.
|
||||
|
||||
The speech carriers close the out-of-domain gap the synthetic trio exposed:
|
||||
all three system voices decoded the full 16-bit message (the pinkish synthetic
|
||||
carrier had flipped one bit), with watermark SNR of 27.9-32.2 dB and clean
|
||||
speech far below the decision rule. Per-second interval readings localize the
|
||||
signal in time: overwriting seconds two through four with 10 dB noise killed
|
||||
exactly those two windows (0.19 falling to 0.0009 mean probability) while the
|
||||
untouched windows kept their original readings to the sixth decimal. The
|
||||
final window of a clip is a short remainder (0.1 s here) and its readings are
|
||||
accordingly less stable - visible in the tables, never dropped.
|
||||
|
||||
Three findings qualify the naive readings:
|
||||
|
||||
- **Codecs are not the attack.** At these rates every codec and resample round
|
||||
trip preserved both detection and the full message, on synthetic and speech
|
||||
carriers alike. Full-clip additive noise at 10 dB SNR killed every speech
|
||||
detection while the white and pinkish synthetic carriers survived it: speech
|
||||
has quiet moments where the noise dominates, so the effective removal lever
|
||||
on audio is loud additive noise, and its power depends on the carrier. A
|
||||
removal pipeline that wants an audio path clean must not rely on
|
||||
transcoding alone.
|
||||
- **The decode path is part of the measurement.** Decoding raw ADTS keeps the
|
||||
encoder's 1024-sample priming head in the stream, which alone moved the
|
||||
verdict from 0.9999/full message to 0.990/partial message; a gapless-aware
|
||||
container (MP4/M4A edit list, or MP3 gapless info) decodes sample-identical
|
||||
audio and restores the reading. The experiment carries the raw-ADTS arm
|
||||
(`aac_128k_adts_raw`) explicitly so the artifact stays a named measurement
|
||||
instead of contaminating codec rows.
|
||||
- **Voice names are not carrier identity.** macOS 26.6.2 renders `Alex` and
|
||||
`Samantha` byte-identically, so a naive three-voice set was one carrier
|
||||
counted twice. The shipped script verifies distinct content hashes across
|
||||
configured voices and refuses an aliased set; diversity claims rest on the
|
||||
hash, never the name.
|
||||
|
||||
Detector cost on this host (Apple silicon, CPU): cold first call 176-465 ms
|
||||
across runs, warm median 158-160 ms for 8-second clips. These are diagnostic
|
||||
host timings, not a certified profile.
|
||||
|
||||
An earlier iteration derived per-carrier seeds from Python's `hash()`, which is
|
||||
salted per process; it produced different carrier bytes on every run. The
|
||||
shipped script derives seeds from `zlib.crc32`, and the two recorded final runs
|
||||
are byte-identical.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- The speech and interval layers are complete; the remaining audio-side
|
||||
upgrade is folding per-second windows into the benchmark schema itself, if a
|
||||
case-level question ever needs them as rows. The forgery/remover-forensics
|
||||
and VideoSeal temporal items remain separate open directions from the
|
||||
original research map.
|
||||
@@ -0,0 +1,583 @@
|
||||
# Watermark benchmark kernel
|
||||
|
||||
`scripts/watermark_benchmark.py` is a development-only runner for reproducible
|
||||
image-, audio-, and video-watermark experiments. It calls the existing local
|
||||
DWT-DCT and TrustMark detectors plus the revision-pinned AudioSeal and
|
||||
VideoSeal oracles through thin adapters. It does not add a runtime command,
|
||||
download a corpus, or contact a provenance oracle or provider API. The
|
||||
optional TrustMark package can fetch its official Adobe model weights when its
|
||||
local package cache is incomplete, the audioseal adapter fetches its pinned
|
||||
checkpoints the same way, and the videoseal adapter downloads one pinned
|
||||
TorchScript file into a user cache; prepare those dependencies before an
|
||||
offline run.
|
||||
|
||||
The kernel answers three different questions and never merges their answers:
|
||||
|
||||
- `detection` records whether one named adapter recognized its own signal.
|
||||
- `removal` records that a removed-state artifact no longer produced a signal,
|
||||
or that the signal remained. It always sets `certifies_erasure` to `false`.
|
||||
- `fidelity` compares decoded pixels with an explicit reference. It says
|
||||
nothing about whether an invisible watermark remains.
|
||||
|
||||
In particular, `not_detected` means only that the selected detector did not
|
||||
recognize its signal. It is not a general `clean` verdict.
|
||||
|
||||
## Manifest
|
||||
|
||||
Input is strict JSONL, one case per line. Relative paths resolve from the
|
||||
manifest directory. The runner verifies every named file against its lowercase
|
||||
SHA-256 digest before invoking any detector, rejects duplicate `case_id` values,
|
||||
and rejects missing or extra fields.
|
||||
|
||||
```json
|
||||
{"schema_version":1,"case_id":"sample-removed","pair_id":"sample","media_type":"image","adapter":"dwt-dct","arm":"positive","state":"removed","path":"removed.png","sha256":"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef","reference_path":"clean.png","reference_sha256":"abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789","source_revision":"local-corpus@2026-09-05","transform":{"name":"remove","revision":"remove-ai-watermarks@abc123","parameters":{"method":"auto"}},"seed":7,"expected":"not_detected"}
|
||||
```
|
||||
|
||||
The fixed fields are:
|
||||
|
||||
| Field | Contract |
|
||||
| --- | --- |
|
||||
| `schema_version` | Integer `1`. |
|
||||
| `case_id` | Unique case identity. |
|
||||
| `pair_id` | Groups related clean, marked, attacked, and removed cases. |
|
||||
| `media_type` | `image`, `audio`, or `video` in schema v1. |
|
||||
| `adapter` | `dwt-dct` or `trustmark` for `image`; `audioseal` for `audio`; `videoseal` for `video`. The loader rejects a known adapter named against the wrong media type. |
|
||||
| `arm` | `positive`, `matched_negative`, `wrong_key`, or `hard_negative`. |
|
||||
| `state` | `clean`, `marked`, `attacked`, `removed`, or `forged`. `forged` names an artifact carrying a watermark with a message different from the adapter oracle's fixed one. Expected detection depends on the adapter: VideoSeal checks the fixed message, while AudioSeal's presence rule can still report `detected` for a foreign message. The study records message accuracy separately. |
|
||||
| `path`, `sha256` | Artifact path and pinned content digest. |
|
||||
| `reference_path`, `reference_sha256` | Both strings or both `null`; the fidelity reference. |
|
||||
| `source_revision` | Corpus, generator, or acquisition revision. |
|
||||
| `transform` | Exact `name`, `revision`, and JSON `parameters`. |
|
||||
| `seed` | Integer or `null`; never an implicit random seed. |
|
||||
| `expected` | `detected`, `not_detected`, or `unresolved`. |
|
||||
|
||||
The arms prevent a positive-only benchmark from being mistaken for a detector
|
||||
evaluation. A useful detector run needs known positives plus matched negatives,
|
||||
wrong-key cases where the method supports keys, and difficult watermark-free
|
||||
images. Transformations such as resize, crop, recompression, or removal should
|
||||
be separate rows with exact parameters, not edits made outside the manifest.
|
||||
|
||||
## Run and output
|
||||
|
||||
Keep the report outside the repository, and choose a new output path for every
|
||||
run:
|
||||
|
||||
```bash
|
||||
uv run python scripts/watermark_benchmark.py \
|
||||
/path/to/manifest.jsonl \
|
||||
--output /path/to/results/run-2026-09-05.jsonl
|
||||
```
|
||||
|
||||
The runner refuses to overwrite an existing report. Every output row repeats
|
||||
the case identity and records the repository commit, whether tracked files were
|
||||
dirty, and SHA-256 digests of the benchmark kernel and adapter source. Detector
|
||||
dependency absence is `unavailable`; an undecodable artifact or adapter
|
||||
exception is `error`. Neither is counted as `not_detected`.
|
||||
|
||||
`detection.adapter_elapsed_ms` measures the wall time of the named detector
|
||||
adapter call only. Image decoding and fidelity calculation are outside that
|
||||
interval. The value is `null` when the adapter was unavailable or the artifact
|
||||
could not be decoded, because no detector call occurred. Preserve the first
|
||||
call separately when aggregating: it can include lazy imports and model loading,
|
||||
while subsequent rows describe warm validation cost.
|
||||
|
||||
Fidelity is calculated only when a reference is present, both decoded images
|
||||
are 8-bit, and their dimensions match. Other bit depths remain explicitly
|
||||
incomparable until their sample-range contract is defined. The result records
|
||||
8-bit MAE, the fraction of changed pixel positions, and PSNR. Identical decoded
|
||||
pixels produce `psnr_db: null` with
|
||||
`psnr_status: unbounded_identical`, never a misleading numeric zero. Missing or
|
||||
incomparable references remain explicit states rather than fabricated metrics.
|
||||
|
||||
Schema v1 deliberately has no aggregate score, confidence interval, automatic
|
||||
attack generator, removal runner, or provider oracle. Add those only after the
|
||||
case-level evidence is large enough to define and test the corresponding
|
||||
statistical or media contract.
|
||||
|
||||
## Audio rows
|
||||
|
||||
`media_type: audio` rows decode every artifact and reference to mono 16 kHz
|
||||
float32 through the system `ffmpeg` before any adapter runs, mirroring the
|
||||
image path where decoding also stays outside the detector interval. Audio rows
|
||||
therefore require `ffmpeg` on `PATH`; a missing or failing decode is the same
|
||||
explicit `error` status as an undecodable image.
|
||||
|
||||
The `audioseal` adapter reads the shared revision-pinned oracle in
|
||||
`scripts/audioseal_oracle.py` (Meta AudioSeal, MIT license, Hugging Face
|
||||
revision and checkpoint SHA-256 pinned and verified at load). Its `detected`
|
||||
status is the detector's own documented rule: the fraction of per-sample
|
||||
probabilities above 0.5 reaching 0.5. The `label` is the decoded 16-bit message
|
||||
as a binary string, so a case whose payload differs from the embedded one is
|
||||
still `detected` with a different label, never `not_detected`. A `wrong_key`
|
||||
arm for this adapter is therefore a different-message row, not a clean row.
|
||||
|
||||
Audio fidelity compares the decoded sample arrays against the explicit
|
||||
reference: identical decoded samples produce `snr_db: null` with
|
||||
`snr_status: unbounded_identical`, never a misleading numeric zero; a silent
|
||||
reference reports `zero_reference_power`; a length mismatch stays an explicit
|
||||
`incomparable` state. SNR here is reference signal power over error power, the
|
||||
sample-domain analogue of the image PSNR contract.
|
||||
|
||||
## Audio cohort v1
|
||||
|
||||
`scripts/watermark_benchmark_audio_cohort.py` builds the audio analogue of the
|
||||
synthetic image cohort: three deterministic synthetic carriers (AM tone stack,
|
||||
smoothed noise, white noise), the pinned AudioSeal message, matched clean
|
||||
negatives, MP3 128k, gapless M4A AAC 128k, and deterministic 10 dB additive
|
||||
noise attacks, plus one never-embedded white-noise hard negative. Since v2
|
||||
the audio cohort also carries the four-state arms: a `removed` row where the
|
||||
noise transform measurably destroys the mark (the tone stack only - the
|
||||
synthetic noise carriers survive 10 dB noise, so their noise rows stay open
|
||||
`attacked` measurements), and `forged` rows embedding a foreign message with
|
||||
`expected: detected`, because the audioseal rule is watermark presence and
|
||||
the decoded label is what exposes the foreign payload. 20 cases in total,
|
||||
everything hashed into a strict manifest validated through the
|
||||
benchmark loader.
|
||||
|
||||
```bash
|
||||
uv run --extra dev python scripts/watermark_benchmark_audio_cohort.py \
|
||||
--output-dir .local-eval/watermark-benchmark-audio-cohort-v1
|
||||
uv run --extra dev python scripts/watermark_benchmark.py \
|
||||
.local-eval/watermark-benchmark-audio-cohort-v1/manifest.jsonl \
|
||||
--output .local-eval/audio-cohort-v1-results.jsonl
|
||||
```
|
||||
|
||||
### Development baseline, 2026-09-06
|
||||
|
||||
The first local run used kernel
|
||||
`sha256:2216169ab7c33f7439720f0934e8ac6ba8379e510bfaeabe008f820c7658701d`,
|
||||
oracle `sha256:bd67e2b1be5c486384ab1f64709f661241451ab1ea7592cc56f0eefb91c415ab`,
|
||||
and recipe `sha256:f361163b4fbf23b885c026751c544e97be0bf054906c1aad455276992a2768d2`,
|
||||
producing manifest `sha256:b6c48684f0c85e882159f6d5c4fbeea2f43c3b24d3a2c3a034209c6f531ba73d`
|
||||
and case rows `sha256:66ac23bd926925dc5baf7216c03f8a7246eeb473315a17701358a3803ab7b0d1`.
|
||||
Dependencies were audioseal 0.2.0, torch 2.13.0, and ffmpeg 9.0.1.
|
||||
|
||||
| State and transform | Cases | Detected | Not detected |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| clean matched negatives + hard negative | 4 | 0 | 4 |
|
||||
| marked, audioseal embed | 3 | 3 | 0 |
|
||||
| attacked, MP3 128k | 3 | 3 | 0 |
|
||||
| attacked, AAC 128k (gapless M4A) | 3 | 3 | 0 |
|
||||
| attacked, 10 dB additive noise | 3 | 2 | 1 |
|
||||
|
||||
Every clean and never-embedded negative stayed below the decision rule. All
|
||||
codecs retained detection with the full message on the tone-stack and
|
||||
white-noise carriers; the pinkish-noise carrier decoded one flipped payload bit
|
||||
while remaining detected, matching the experiment's per-carrier bit-accuracy
|
||||
observation. The 10 dB noise arm destroyed the tone-stack detection but not
|
||||
white or pinkish noise: carrier-dependent robustness is exactly what the
|
||||
matched arms exist to expose. Nearest-rank adapter timing: cold first call
|
||||
405.1 ms, warm median 124.2 ms, warm max 157.4 ms for 8-second clips on this
|
||||
host. As with the image baselines, these are diagnostic host rows, not a
|
||||
certified profile.
|
||||
|
||||
### Speech extension
|
||||
|
||||
`--with-speech` adds fifteen more rows: three system-voice speech carriers
|
||||
(Samantha, Daniel, Milena) rendered by the local macOS `say` tool - no
|
||||
network, byte-deterministic per voice on a fixed macOS version - each with a
|
||||
matched clean negative, a marked positive, and the same three attacks. Every
|
||||
speech row's transform parameters carry the full TTS provenance (tool, macOS
|
||||
version, voice, text digest, duration). The builder verifies the configured
|
||||
voices render distinct bytes and refuses an aliased set, because macOS 26.6.2
|
||||
renders `Alex` and `Samantha` byte-identically and a name is not a carrier
|
||||
identity.
|
||||
|
||||
The first `--with-speech` run (2026-09-06, 31 cases total, recipe
|
||||
`sha256:3f291f49ab9ef2499085708fb614e3723ebee6d35f0b509fc36f017ec5a86c3b`,
|
||||
manifest
|
||||
`sha256:b9baa79066deb002402cc20166835887d67a5158ff9ce7bbc1963e5d603db3a5`,
|
||||
case rows
|
||||
`sha256:abc68590f59fbddab1c086371d2854622b41ea756ba61a7a87b76cb6df64c386`)
|
||||
kept every synthetic verdict and added: speech clean 0/3 detected, speech
|
||||
marked 3/3 with the full message, codecs 3/3, and full-clip 10 dB noise 0/3 -
|
||||
unlike the synthetic white and pinkish carriers, speech has quiet moments
|
||||
where the noise dominates. Speech marked fidelity sat at 27.9-32.2 dB SNR.
|
||||
|
||||
## Synthetic cohort v1
|
||||
|
||||
`scripts/watermark_benchmark_cohort.py` builds a reproducible local smoke cohort
|
||||
without a provider API or provenance oracle. Its default 65 cases cover three
|
||||
deterministic 512-pixel carriers, SDXL, FLUX, and Stable Diffusion 1.x DWT-DCT
|
||||
patterns, TrustMark Variant P with schema 1, and JPEG quality 90, 75% resize
|
||||
round-trip, and 5% crop-per-edge round-trip attacks. Each adapter also receives
|
||||
a high-frequency hard negative. TrustMark cases include the package remover's
|
||||
output as a separate `removed` state.
|
||||
|
||||
```bash
|
||||
uv run python scripts/watermark_benchmark_cohort.py \
|
||||
--output-dir .local-eval/watermark-benchmark-cohort-v1 \
|
||||
--size 512
|
||||
uv run python scripts/watermark_benchmark.py \
|
||||
.local-eval/watermark-benchmark-cohort-v1/manifest.jsonl \
|
||||
--output .local-eval/watermark-benchmark-cohort-v1-results.jsonl
|
||||
```
|
||||
|
||||
The builder refuses an existing destination, hashes every generated file, pins
|
||||
its own recipe digest and dependency versions, and validates the finished
|
||||
manifest through the benchmark's strict loader. It reads the DWT-DCT patterns
|
||||
from the production detector so the generated positive cannot silently drift
|
||||
from the matcher. TrustMark's optional package and model weights must already be
|
||||
available.
|
||||
|
||||
`--size` accepts square carrier sides of at least 256 pixels, divisible by 16.
|
||||
The size is part of every case identity and transform record, so resource sweeps
|
||||
at multiple boundaries can be aggregated without merging unlike artifacts.
|
||||
|
||||
Attacked and removed rows intentionally use `expected: unresolved`: they are
|
||||
measurements, not assertions smuggled into fixtures. The cohort is too small
|
||||
and too synthetic for prevalence, confidence, or release-quality claims. It is
|
||||
a deterministic first baseline that proves the harness and exposes gross
|
||||
detector behavior before the real-image cohort below is run.
|
||||
|
||||
### Development baseline, 2026-09-05
|
||||
|
||||
The first local v1 run used cohort recipe
|
||||
`sha256:d98fd6402bcc149833005ec03aab77bd0074ee105696f9e1ac5da01ef9c666f7`
|
||||
and kernel
|
||||
`sha256:37e552ba53c3d2aff2c5f9e160a33ad47b0d32c877b0a8fbba39c90ca0ab0179`.
|
||||
Dependencies were invisible-watermark 0.2.0, TrustMark 0.9.1, NumPy 1.26.4,
|
||||
and Pillow 12.3.0. The tree was based on commit `ad53790` with the cohort and
|
||||
timing changes still uncommitted, so the content hashes, not that commit alone,
|
||||
identify the executed code. A second build in a different directory produced a
|
||||
byte-identical manifest and artifact-hash inventory.
|
||||
|
||||
| Adapter and state | Unique artifacts | Detected | Not detected |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| DWT-DCT clean | 4 | 0 | 4 |
|
||||
| DWT-DCT marked | 9 | 6 | 3 |
|
||||
| DWT-DCT attacked | 27 | 13 | 14 |
|
||||
| TrustMark clean | 4 | 0 | 4 |
|
||||
| TrustMark marked | 3 | 2 | 1 |
|
||||
| TrustMark attacked | 9 | 4 | 5 |
|
||||
| TrustMark removed | 3 | 0 | 3 |
|
||||
|
||||
All three plain-gradient DWT-DCT positives were missed, while all texture and
|
||||
low-detail positives were recognized. DWT-DCT retained 7/9 positives through
|
||||
JPEG, 6/9 through resize, and 0/9 through crop. TrustMark retained 2/3 through
|
||||
JPEG and 1/3 through each resize and crop. Its three remover outputs produced no
|
||||
recognized signal and measured 46.17-49.24 dB PSNR against their clean
|
||||
carriers. Those three negatives are a post-removal observation, not proof that
|
||||
the signal is absent beyond this decoder.
|
||||
|
||||
Ten independent sequential benchmark processes reproduced every case status.
|
||||
Nearest-rank adapter timings were:
|
||||
|
||||
| Adapter call | n | p50 | p90 | max |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| DWT-DCT cold, first call per process | 10 | 14.96 ms | 19.86 ms | 27.88 ms |
|
||||
| DWT-DCT warm | 450 | 0.65 ms | 1.64 ms | 13.53 ms |
|
||||
| TrustMark cold, first call per process | 10 | 1,989.47 ms | 2,085.62 ms | 2,086.02 ms |
|
||||
| TrustMark warm | 180 | 17.96 ms | 34.20 ms | 39.95 ms |
|
||||
| TrustMark warm, removed state only | 30 | 16.80 ms | 17.69 ms | 18.56 ms |
|
||||
|
||||
These intervals include only the detector adapter call. They exclude image
|
||||
decoding, fidelity metrics, removal, and process startup. The roughly two-second
|
||||
TrustMark cold start matters only when validation is the first TrustMark use in
|
||||
the process; a workflow that already identified the source pays the warm cost.
|
||||
The sample is deliberately diagnostic rather than statistically representative.
|
||||
|
||||
## Real-image cohort v1
|
||||
|
||||
`scripts/watermark_benchmark_real_cohort.py` replaces procedural carriers with
|
||||
the 38 committed OpenAI/Meta images in the engine-selection content matrix. The
|
||||
matrix has 19 prompt-matched content strata, one image per provider in every
|
||||
stratum, and no duplicate file hash. Its images are project-collected model
|
||||
outputs rather than camera originals; "real-image" here means real, decoded
|
||||
content carriers rather than generated gradients or checkerboards.
|
||||
|
||||
Every source row carries
|
||||
`reuse_basis: project-maintainer-public-test-clearance`. This records permanent
|
||||
public test and benchmark permission for those exact committed bytes. It does
|
||||
not claim that provider terms grant a general dataset license for arbitrary
|
||||
generations. The builder refuses a source without that explicit basis, a hash
|
||||
or dimension mismatch, repeated pixels, an incomplete provider pair, or a
|
||||
duplicate provider/content stratum.
|
||||
|
||||
```bash
|
||||
uv run python scripts/watermark_benchmark_real_cohort.py \
|
||||
--output-dir .local-eval/watermark-benchmark-real-v1 \
|
||||
--size 512
|
||||
uv run python scripts/watermark_benchmark.py \
|
||||
.local-eval/watermark-benchmark-real-v1/manifest.jsonl \
|
||||
--output .local-eval/watermark-benchmark-real-v1-results.jsonl
|
||||
```
|
||||
|
||||
Each source is deterministically center-cropped only if necessary, resized to
|
||||
the selected square geometry with LANCZOS, converted to RGB, and written as a
|
||||
PNG reference. At the default size all current sources are already square, so
|
||||
only the downscale and RGB conversion apply. The source manifest hash, original
|
||||
file and payload hashes, provider, model, content stratum, native dimensions,
|
||||
reuse basis, prompt hash, and standardization recipe travel with every case.
|
||||
|
||||
The default build has 798 cases: all three production DWT-DCT patterns plus
|
||||
TrustMark Variant P, each with matched negatives, marked positives, the same
|
||||
three fixed attacks as the synthetic cohort, and TrustMark remover outputs.
|
||||
Clean sources are expected negative only for the selected DWT-DCT or TrustMark
|
||||
adapter; the result must not be generalized into a claim that no other
|
||||
provenance signal exists. The aggregate report adds provider and content-stratum
|
||||
tables when this source metadata is present.
|
||||
|
||||
The 12 existing SynthID and Content Seal signal carriers are deliberately not
|
||||
mixed into this matched-negative baseline. They remain suitable for a separate
|
||||
cross-watermark interference layer, where their original signal and oracle
|
||||
provenance can be named instead of being mislabeled as globally clean.
|
||||
|
||||
### Development baseline, 2026-09-06
|
||||
|
||||
Three sequential local runs reproduced all 798 case verdicts with no unstable
|
||||
case, adapter error, or unavailable result. Both adapters rejected all 38
|
||||
matched negatives. The carrier set therefore removed the synthetic cohort's
|
||||
false impression that TrustMark was broadly carrier-fragile, while confirming
|
||||
and sharpening the DWT-DCT carrier-dependence finding.
|
||||
|
||||
| Adapter and state | Unique artifacts | Detected | Not detected |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| DWT-DCT marked | 114 | 74 | 40 |
|
||||
| DWT-DCT JPEG q90 | 114 | 76 | 38 |
|
||||
| DWT-DCT resize 75% round-trip | 114 | 77 | 37 |
|
||||
| DWT-DCT crop 5% per edge | 114 | 0 | 114 |
|
||||
| TrustMark marked | 38 | 38 | 0 |
|
||||
| TrustMark, all three attacked arms | 114 | 114 | 0 |
|
||||
| TrustMark removed | 38 | 0 | 38 |
|
||||
|
||||
DWT-DCT marked recall was 26/38 for FLUX, 23/38 for Stable Diffusion
|
||||
1.x, and 25/38 for SDXL. JPEG and resize retained every one of the 74 base
|
||||
detections, then moved two and three previously missed cases above the decoder
|
||||
threshold respectively. They did not create new ground-truth marks; this is why
|
||||
the report presents pairwise transitions rather than describing 76/114 and
|
||||
77/114 as ordinary attack retention. The crop destroyed every recognized
|
||||
DWT-DCT signal.
|
||||
|
||||
The content strata expose a strong dependence that three procedural carriers
|
||||
could not measure. Interior low light, person in context, and single portrait
|
||||
were 0/6 on the un-attacked DWT-DCT embeds, while nine strata were 6/6. Meta
|
||||
carriers were detected in 42/57 marked cases and their prompt-matched OpenAI
|
||||
partners in 32/57. Those are descriptive counts over one image per provider
|
||||
and stratum, not a provider-population estimate.
|
||||
|
||||
TrustMark was detected before and after every fixed attack in every stratum.
|
||||
All 38 remover outputs produced no recognized TrustMark signal, with 41.26 to
|
||||
49.80 dB PSNR and 45.27 dB median PSNR against the standardized carriers. This
|
||||
remains a decoder-scoped post-removal observation, not proof of erasure against
|
||||
another implementation.
|
||||
|
||||
The run was based on commit `f61a7ec` with tracked benchmark changes still
|
||||
uncommitted. The exact inputs and implementations are therefore bound by hash:
|
||||
|
||||
- source manifest: `sha256:aa37aa97ed3b32a59242fd506fa3cee545df615f98690d373735929a9d7d0089`;
|
||||
- content-manifest validator: `sha256:72ddb2e489d5a7f92cd948b880a0fd308d4083ed86d68fc49897d646aceb7ec5`;
|
||||
- real-cohort recipe: `sha256:f3851dae5d4eee69510c3dc5b5b15b5116720d944cfeed631894e635c2a5e9c9`;
|
||||
- shared embed/attack helper: `sha256:c78342f0d125d6636259b641e67e71b72b577455057e568052d2a0a4df00c9dd`;
|
||||
- benchmark kernel: `sha256:a96874a41fa0a38cd29d957c6a69683de887e942b480ce7668090b0f1a9e3329`;
|
||||
- generated manifest: `sha256:77dc6094fcfbbaa6fdb9a15d96dee6fb3d4349829b2b1d7cf5f006aa7fb0ae8a`;
|
||||
- case-level results: `sha256:bb483be015c012fd28d0979510244c834475a23bbc215226c998fada73485ed3`,
|
||||
`sha256:35d735de9654a36d6d2a6e096f7a2c3f8a70b1b22ead4f6f414d6c1494e53b2a`, and
|
||||
`sha256:52a9a2bb78dcc9f40609b5ca13812470d21477e5e553892c2bbef4b803fee433`;
|
||||
- aggregate report: `sha256:4496edf231a20cc7977f003c30313af5acbe8c99075e22d2cb2eeec5e104f3eb`.
|
||||
|
||||
The local timing rows were host-contended and are not used for a new latency
|
||||
claim. The controlled resource profile below remains the performance evidence.
|
||||
|
||||
## Video rows
|
||||
|
||||
`media_type: video` rows decode every artifact and reference to float RGB
|
||||
frames in [0, 1] as (T, H, W, 3) through the system `ffmpeg`, staying outside
|
||||
the detector interval like the image and audio decodes. Video fidelity
|
||||
compares decoded frames against the explicit reference clip: mean PSNR over
|
||||
frames (`mean_psnr_db`, reported separately from image `psnr_db`), a
|
||||
changed-frame fraction, `unbounded_identical` for bit-identical
|
||||
pixel sequences, and an explicit `shape_mismatch` state when frame counts or
|
||||
geometry differ - which is why geometry-changing attack arms carry no
|
||||
reference instead of a fabricated comparison.
|
||||
|
||||
The `videoseal` adapter loads Meta's standalone TorchScript release of
|
||||
VideoSeal (MIT license) from `scripts/videoseal_oracle.py` - one pinned file,
|
||||
no `videoseal` package, whose wheel pulls a research dependency chain without
|
||||
macOS arm64 wheels and whose loader resolves cards relative to the working
|
||||
directory. The upstream video evaluation ignores the model's detection head
|
||||
(it prepends a constant) and scores the aggregated 256-bit message decode, so
|
||||
the adapter's `detected` is its own explicit rule: bit accuracy of the
|
||||
averaged decode against the oracle's fixed message at or above 0.9. The
|
||||
`label` is the decoded message as 64 hex characters. Because the rule is
|
||||
message accuracy, a different-message row reads `not_detected` - the matched
|
||||
verifier rejects the foreign payload - unlike the audioseal presence rule
|
||||
above. Every videoseal detection record
|
||||
carries a `temporal` block - frame count, min/mean/max and the full per-frame
|
||||
bit-accuracy series - and the aggregation is computed in the oracle from raw
|
||||
per-frame logits, because the TorchScript build ignores the aggregation
|
||||
argument of `detect_video_and_aggregate`. The aggregation matrix itself
|
||||
(avg versus norm-weighted) is evaluated in
|
||||
[VideoSeal temporal evaluation](videoseal-temporal-evaluation.md), not in the
|
||||
kernel, which keeps plain avg as its canonical rule.
|
||||
|
||||
## Video cohort v1
|
||||
|
||||
`scripts/watermark_benchmark_video_cohort.py` builds the video analogue of
|
||||
the other cohorts: two deterministic synthetic 64-frame 256x256 clips (moving
|
||||
gradient, moving texture), the pinned 256-bit message, matched clean
|
||||
negatives, one never-embedded hard negative, and three attacks - a realistic
|
||||
H.264 re-encode at crf 23, a 75 percent downscale, and a frame-rate halving
|
||||
that drops half the keyframes. Since v2 the video cohort also carries the
|
||||
four-state arms: `removed` rows reusing the crf 23 re-encode that measurably
|
||||
takes both synthetic carriers below the rule, and `forged` rows embedding a
|
||||
foreign message with `expected: not_detected`, because the videoseal rule is
|
||||
message accuracy and the matched verifier rejects a foreign payload. 15
|
||||
cases in total.
|
||||
|
||||
```bash
|
||||
uv run --extra dev python scripts/watermark_benchmark_video_cohort.py \
|
||||
--output-dir .local-eval/watermark-benchmark-video-cohort-v1
|
||||
uv run --extra dev python scripts/watermark_benchmark.py \
|
||||
.local-eval/watermark-benchmark-video-cohort-v1/manifest.jsonl \
|
||||
--output .local-eval/video-cohort-v1-results.jsonl
|
||||
```
|
||||
|
||||
### Development baseline, 2026-09-07
|
||||
|
||||
The first local run used kernel
|
||||
`sha256:7a3906cebc3c7cacbac7e204c014d1d6281615e7acbbc5b3a8fcecc939ca68c7`,
|
||||
oracle `sha256:6c889bf399e8e3bada439eee4a1e9c0b04177be78dfee756d5daa79e73db71b3`,
|
||||
recipe `sha256:dd06ae1c40ef7d7f6fd2b258b20444a3250254a5ef258542a4eaf43cc501342b`,
|
||||
manifest
|
||||
`sha256:bb6a7dda6b3897794f2fbfe47291329bb7b9ff7ee57c0634ebb386e58acd5e7a`,
|
||||
and case rows
|
||||
`sha256:5793b8ba33c55e135e5bef43f33fd2699bb57bae6ca8ead38b1dab0123d1dd12`.
|
||||
The checkpoint is the TorchScript build `y_256b_img.jit`
|
||||
`sha256:5c7a4581c36fc6090aafdcfb3999123bae5172a4847f22e2da4e7fd1a39d1e1b`
|
||||
pinned to upstream commit `870ca7f`.
|
||||
|
||||
| State and transform | Cases | Detected | Not detected |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| clean matched negatives + hard negative | 3 | 0 | 3 |
|
||||
| marked, videoseal embed (crf 8 artifact) | 2 | 2 | 0 |
|
||||
| attacked, H.264 crf 23 | 2 | 0 | 2 |
|
||||
| attacked, 75 percent downscale + crf 18 | 2 | 0 | 2 |
|
||||
| attacked, frame rate halved | 2 | 2 | 0 |
|
||||
|
||||
Direct oracle readings fill in what the threshold hides: the marked clips
|
||||
decode at 0.992-0.996 bit accuracy (48.5 and 41.4 dB mean PSNR against their
|
||||
clean carriers), the crf 23 re-encode collapses to 0.56-0.57, the downscale
|
||||
to 0.69, and the halved frame rate keeps 0.996. Two cautions travel with
|
||||
these numbers: the watermark survives H.264 only at generous quality - a
|
||||
realistic crf 23 re-encode erases it on these smooth synthetic carriers,
|
||||
while a high-bitrate uniform-noise carrier survived crf 18 in the isolated
|
||||
probe, so retention is carrier- and quality-dependent - and dropping half
|
||||
the frames does not touch the message because the secret repeats across
|
||||
keyframes. These are first-baseline observations on synthetic carriers, not
|
||||
robustness curves; a sweep across crf values and real-content carriers is
|
||||
the natural follow-up.
|
||||
|
||||
### Four-state baseline, 2026-09-07 (v2 cohorts)
|
||||
|
||||
The v2 runs used kernel
|
||||
`sha256:22f2c1dd5c3b7aeb96090c4c578c6d1841ee62d90958dd1c5c7ad7db71835eec`
|
||||
with audio recipe
|
||||
`sha256:31c021144212a43565fb3746fadc835bfc37ea8ac98998fad9bba9e72410e82c`
|
||||
(manifest
|
||||
`sha256:43c7f77f1d4537c00d4b0dd5128e9becc6c860cdd196a7ba2f7541f4276784ec`,
|
||||
case rows
|
||||
`sha256:c73d7dddfdb90262e96be4d67ec1567d9046812661928cc816cb9f97f4a01a7e`)
|
||||
and video recipe
|
||||
`sha256:8f25929304ef4f94721667968ee4bb89ebf584a2108de7e02db36c855dc2ddc8`
|
||||
(manifest
|
||||
`sha256:b5659848c0d05e205f80ab36480e70fdf7e5f9df4d8ada6ffadec42c97b88998`,
|
||||
case rows
|
||||
`sha256:c9ad56848bb7efe200aaf1189a8eb03f7571567e6702edd195a62f812fa05815`).
|
||||
Every v1 verdict was unchanged; the new arms read: audio removed 0/1
|
||||
detected, audio forged 3/3 detected under the presence rule, video removed
|
||||
0/2, video forged 0/2 under the message rule - zero expected mismatches.
|
||||
The mechanism underneath these rows, including double embedding, is measured
|
||||
in [Watermark forgery study](watermark-forgery-study.md).
|
||||
|
||||
## Aggregate repeated runs
|
||||
|
||||
`scripts/watermark_benchmark_report.py` turns one or more result JSONL files
|
||||
into a standalone Markdown report. Treat each input file as one independent
|
||||
process run: the first measured call for each adapter in that file is cold, and
|
||||
the remaining measured calls are warm.
|
||||
|
||||
```bash
|
||||
uv run python scripts/watermark_benchmark_report.py \
|
||||
.local-eval/results/run-01.jsonl \
|
||||
.local-eval/results/run-02.jsonl \
|
||||
--output .local-eval/results/report.md
|
||||
```
|
||||
|
||||
The report retains raw observation counts but deduplicates detector summaries
|
||||
by artifact SHA-256 within each adapter and state. It reports state and
|
||||
transform breakdowns, post-removal observations, nearest-rank timing, fidelity,
|
||||
expected-result mismatches, unstable repeated verdicts, errors, and unavailable
|
||||
adapters as separate evidence. A repeated `case_id` whose artifact, transform,
|
||||
or other case identity changes is rejected rather than merged. Input file
|
||||
digests in the final report bind every aggregate to its exact case-level source.
|
||||
The runner decodes an artifact used by multiple cases only once per process and
|
||||
passes the decoded pixels to both local adapters. Synthetic recipes record
|
||||
the seed actually passed to generation; a manifest seed is not a substitute
|
||||
for that generator input. Video observations must decode the saved artifact
|
||||
whose digest the row records, including controls and double embeds. Adapter
|
||||
timing does not include file decoding.
|
||||
|
||||
## Profile process cost
|
||||
|
||||
`scripts/watermark_benchmark_resources.py` runs each manifest in fresh Python
|
||||
processes. It records parent-observed wall time and the child's absolute peak
|
||||
RSS, while preserving the ordinary case-level JSONL outputs for the detector
|
||||
report above.
|
||||
|
||||
```bash
|
||||
uv run python scripts/watermark_benchmark_resources.py \
|
||||
.local-eval/cohort-256/manifest.jsonl \
|
||||
.local-eval/cohort-512/manifest.jsonl \
|
||||
.local-eval/cohort-1024/manifest.jsonl \
|
||||
--repeat 3 \
|
||||
--output-dir .local-eval/resource-profile
|
||||
```
|
||||
|
||||
Wall time includes interpreter startup, imports, image decoding, detector calls,
|
||||
fidelity metrics, and result writing. Peak RSS is the process's absolute
|
||||
high-water mark, not incremental memory caused only by validation. Run DWT-DCT
|
||||
and TrustMark manifests separately to avoid attributing one adapter's model load
|
||||
to the other. The profiler uses only local adapters and never calls an oracle or
|
||||
provider API.
|
||||
|
||||
### Development resource profile, 2026-09-06
|
||||
|
||||
Ten sequential fresh-process runs per size were measured on macOS arm64 with
|
||||
Python 3.12.13. DWT-DCT manifests contained 46 cases; TrustMark manifests
|
||||
contained 19. Every status count was stable across all ten repetitions.
|
||||
|
||||
| Adapter | Max geometry | Cases | Process wall p50 | p90 | Absolute peak RSS p50 | p90 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| DWT-DCT | 256x256 | 46 | 263.07 ms | 322.61 ms | 74.66 MiB | 75.31 MiB |
|
||||
| DWT-DCT | 512x512 | 46 | 482.67 ms | 498.07 ms | 90.56 MiB | 91.02 MiB |
|
||||
| DWT-DCT | 1024x1024 | 46 | 1,360.10 ms | 1,389.40 ms | 159.58 MiB | 160.00 MiB |
|
||||
| TrustMark | 256x256 | 19 | 3,191.18 ms | 6,074.68 ms | 741.11 MiB | 743.66 MiB |
|
||||
| TrustMark | 512x512 | 19 | 9,172.26 ms | 15,430.20 ms | 752.92 MiB | 758.17 MiB |
|
||||
| TrustMark | 1024x1024 | 19 | 7,301.26 ms | 9,797.40 ms | 848.91 MiB | 852.92 MiB |
|
||||
|
||||
The TrustMark wall-time variance, including one 18.34-second 512 px maximum,
|
||||
did not coincide with comparable memory movement. The non-monotonic 512/1024
|
||||
times prove that this host was contended, so the TrustMark wall rows are a
|
||||
diagnostic range and cannot establish size scaling or throughput. Repeat them
|
||||
on an idle host before making a performance claim. Peak RSS stayed much more
|
||||
stable: its nearly flat 256/512 values establish the fixed model cost, while
|
||||
1024 px adds about 100 MiB of high-water memory.
|
||||
|
||||
These whole-process rows are deliberately not divided by the case count. A
|
||||
fresh-process average would hide the fixed model load and is not the cost of one
|
||||
post-removal check. The current three-run real-image cohort measured warm p50
|
||||
adapter calls of 0.57 ms for DWT-DCT and 29.11 ms for TrustMark at 512 px,
|
||||
excluding file decoding. A workflow that identified the source before removal
|
||||
already holds TrustMark's singleton decoder, so validation reuses that fixed
|
||||
model cost. Starting a new process or loading TrustMark only after removal is
|
||||
the expensive design and should be avoided.
|
||||
|
||||
The exact profiler source was
|
||||
`sha256:caf9674dffceddebb8cbf26067581250334fbc11e0fbcb567e8dd45555357b54`.
|
||||
The local DWT-DCT and TrustMark resource JSONL files were respectively
|
||||
`sha256:b7ceb012db5c2f549720c37d6234526396c7120fb68e040705322be35dfd6743`
|
||||
and
|
||||
`sha256:f3639f46a9257a3161ed0197016c089ca5c084b2de2e07cb4b65fc774e20544d`.
|
||||
They remain untracked because they contain host-specific paths and timings;
|
||||
these hashes bind the summarized numbers to the exact local evidence.
|
||||
@@ -0,0 +1,83 @@
|
||||
# C2PA soft-binding resolution research
|
||||
|
||||
Development-only investigation, 2026-09-07, completing the privacy-tiered C2PA
|
||||
resolution item from the original research map. The outcome is a product
|
||||
decision rather than a feature: this tool resolves nothing over the network,
|
||||
and this page records what the resolution ecosystem would learn if it did.
|
||||
|
||||
## The normative shape
|
||||
|
||||
C2PA 2.2 specification section 18.10.5 defines the soft binding resolution
|
||||
API: a standard way of retrieving manifest stores from a resolution endpoint
|
||||
given **a soft binding value, a manifest identifier, or an asset** - three
|
||||
input modes that are exactly three privacy exposure levels. The algorithm
|
||||
registry (`c2pa-org/softbinding-algorithm-list`) carries the endpoint URIs in
|
||||
`softBindingResolutionApis`. As of the packaged revision `c0218cd628c9`,
|
||||
8 of 53 entries publish them.
|
||||
|
||||
## Measured contracts
|
||||
|
||||
| Algorithm | Kind | Endpoint | Input | Content leaves | Content digest leaves |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| `com.aiwatermark.audioseal.1` / `videoseal` / `pixelseal` | watermark | `aiwatermark.com/api/v1/resolve` | `GET ?payload={extracted payload}&alg=` | no | no |
|
||||
| `com.joinmonolith.sha256` | fingerprint | `api.joinmonolith.com/api/c2pa/matches/byBinding` | `GET ?alg&value=base64(SHA-256 of asset)` | no | yes |
|
||||
| `me.deepmark.audio.vigil.128` | watermark | `resolution-api.deepmark.me` | payload-based per vendor site | no | no |
|
||||
| `ai.trufo.pawprint.watermark` / `.fingerprint` | both | `c2pa.trufo.ai/v1` | not published publicly | ? | ? |
|
||||
| `io.blockfact.audio.watermark.32` | watermark | `api.blockfact.io/api/soft-binding/v1` | not published publicly | ? | ? |
|
||||
|
||||
Sources, verified 2026-09-07 against the vendors' own pages: the Monolith
|
||||
soft-binding documentation (lookup endpoints public by design, "a verifier is
|
||||
usually a stranger"; only manifest minting is authenticated), the AIWatermark
|
||||
algorithm page ("after extracting the watermark payload from an audio file,
|
||||
validators can resolve the payload"; model source Meta FAIR AudioSeal), the
|
||||
Deepmark product description ("the signal is detected and its payload
|
||||
resolved"), the Trufo developer documentation and `trufo-py` repository
|
||||
(signing-side only), and the BlockFact API reference (its public verify
|
||||
endpoint takes a Starknet transaction hash, not a soft binding).
|
||||
|
||||
The privacy gradient, concretely:
|
||||
|
||||
- **Payload-only** resolution (AIWatermark, Deepmark) sends a locally
|
||||
extracted watermark payload - opaque bits that identify the mark, not the
|
||||
file. This is the mode our pinned local oracles could serve without ever
|
||||
touching the network with content or a digest.
|
||||
- **Digest** resolution (Monolith) sends base64(SHA-256) of the whole asset.
|
||||
No bytes leave, but the digest is linkable to the exact content: the
|
||||
registry learns you are inspecting that precise file.
|
||||
- **Undocumented** (Trufo, BlockFact): no public contract; any integration
|
||||
would be guesswork and is not attempted.
|
||||
|
||||
## The product decision
|
||||
|
||||
This tool does not resolve anything over the network, at any tier, and no
|
||||
`local_only` / `binding_only` / `full_content` modes ship. The reasoning:
|
||||
|
||||
- The tool's user processes their own generated or edited content and wants
|
||||
to know which provenance signals it carries and to remove them. Local
|
||||
payload extraction - already available through the pinned development
|
||||
oracles - answers the identification question without a network.
|
||||
- Querying a vendor's resolver tells that vendor someone is inspecting the
|
||||
specific content (directly for digest-based lookups, via the payload for
|
||||
payload-based ones). For a removal tool that is a leak in the opposite
|
||||
direction of its purpose, and it contradicts the offline runtime boundary
|
||||
the project already guarantees.
|
||||
- No user scenario of this tool requires manifest recovery from a registry:
|
||||
that is a verifier-of-third-party-content function, which is outside the
|
||||
intended use.
|
||||
|
||||
The runtime boundary stays: identification reports locally decoded evidence
|
||||
only, and `docs/known-limitations.md` states the resolution boundary in user
|
||||
terms. A guard test pins the packaged registry's published-API set, so an
|
||||
upstream ecosystem change surfaces as a conscious decision recorded against
|
||||
this page rather than a silent regeneration.
|
||||
|
||||
## Investigation note
|
||||
|
||||
The first reconnaissance pass concluded "upstream publishes zero resolution
|
||||
APIs" because the probe searched the registry JSON for guessed key names
|
||||
instead of the `softBindingResolutionApis` key the sync actually reads. The
|
||||
drift-guard test written from that wrong premise failed immediately against
|
||||
the packaged snapshot and exposed the error - eight live endpoints, including
|
||||
resolution hosting for the exact AudioSeal and VideoSeal schemes this project
|
||||
benchmarks. The corrected guard now pins the real set. Field names are
|
||||
verified against the consumer's schema, never guessed.
|
||||
@@ -1,5 +1,10 @@
|
||||
# Chroma1 engine research (2026-08-29/30)
|
||||
|
||||
> Metric correction: the OCR metric from `scripts/fidelity_metrics.py` is
|
||||
> normalized edit distance (NED), edit distance divided by the longer
|
||||
> normalized string. The historical values are unchanged; the earlier CER
|
||||
> label incorrectly implied normalization by reference length.
|
||||
|
||||
> Research archive. This page records experiments and decisions from the dates
|
||||
> above. Current package behavior is defined by the
|
||||
> [supported signals](supported-signals.md), [known limitations](known-limitations.md),
|
||||
@@ -18,9 +23,11 @@ before any integration decision.
|
||||
All Chroma1 runs: `lodestones/Chroma1-HD` bf16 on Modal H100, seed 0, the
|
||||
neutral scrub prompt (`high quality, sharp, detailed, faithful to the original`
|
||||
/ `blurry, lowres, distorted text, garbled text, artifacts`), guidance 5.0, and
|
||||
**step compensation**: Diffusers truncates the step COUNT (`int(steps *
|
||||
strength)`), so the requested count is scaled to always spend 4 effective
|
||||
denoising steps, the same semantics as `sdxl_zimage_pipeline.requested_steps`.
|
||||
**step compensation**: the calibrated request is `ceil(4 / strength)`.
|
||||
Chroma rounds the start index rather than the active step count: this runs
|
||||
five effective steps at strengths 0.09 and 0.17, and four at 0.20, 0.125,
|
||||
and 0.40. The nominal count is four, not a promise of four effective steps.
|
||||
The calibrated requests remain unchanged.
|
||||
Without it, strength 0.05 runs a single step and the floor reads artificially
|
||||
high (the first, uncompensated sweep put the OpenAI boundary at (0.05, 0.10];
|
||||
compensated it is (0.05, 0.06]).
|
||||
@@ -32,12 +39,17 @@ Two Flux-family traps found while wiring the prototype:
|
||||
- dimensions floor to the /16 latent-patch grid, so the result must be resized
|
||||
back to the input size for pixel-exact comparison.
|
||||
|
||||
Scripts: `scripts/chroma_scrub_prototype.py` (first sweep),
|
||||
`scripts/engine_quality_price_probe.py` (head-to-head with the production
|
||||
qwen-zimage path), `scripts/engine_cohort_calibration.py` (the cohort ladders
|
||||
below), `scripts/chroma_meta_high_rungs.py` (the Meta rungs above 0.08).
|
||||
Generated outputs stay outside the repository (`out/`, gitignored); the durable
|
||||
verdicts are the tables on this page.
|
||||
Harnesses (Modal GPU scripts kept outside this repository; each mounts this
|
||||
library's `src/` and reads its tracked corpora): `chroma_scrub_prototype.py`
|
||||
(first sweep), `engine_quality_price_probe.py` (head-to-head with the
|
||||
qwen-zimage path), `engine_cohort_calibration.py` (the cohort ladders below),
|
||||
`chroma_meta_high_rungs.py` (the Meta rungs above 0.08),
|
||||
`chroma_muse_expansion.py` (harvest-2 Meta API fixtures),
|
||||
`chroma_openai_expansion.py` (extra OpenAI SynthID carriers),
|
||||
`chroma_seed_probes.py` (seed-1/2 at measured first-cleans).
|
||||
These harnesses are internal tooling and are not distributed, so a reader outside
|
||||
the project cannot rerun them. Generated outputs stay outside the repository
|
||||
(`out/`, gitignored); the durable verdicts are the tables on this page.
|
||||
|
||||
## Measured floors
|
||||
|
||||
@@ -56,10 +68,10 @@ rungs were checked ascending and stopped at the first clean verdict.
|
||||
| Meta | gen_studio_mug 1600x1600 | 0.03 | 0.045 |
|
||||
| Meta | gen_text_poster 1600x1600 | -- | <=0.03 |
|
||||
|
||||
Derived operating points by the shipped rule (worst first-clean boundary plus
|
||||
one observed cross-source spread):
|
||||
Derived operating points from this initial calibration (the OpenAI value was
|
||||
superseded by the 2026-09-07 holdout below):
|
||||
|
||||
- **OpenAI: 0.09** (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and
|
||||
- **OpenAI: 0.09, superseded.** (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and
|
||||
sdxl-zimage 0.15. The typography boundary (0.05, 0.06] is seed-stable across
|
||||
seeds 0, 1, 2.
|
||||
- **Meta: 0.17** (0.10 + (0.10 - 0.03)), oracle-verified clean on the worst
|
||||
@@ -96,7 +108,7 @@ one observed cross-source spread):
|
||||
- **Microsoft: 0.125**, measured 2026-08-30 on the SAME three valid Paint
|
||||
carriers behind the qwen floor (staged under
|
||||
`out/cohort-calibration/microsoft/` with pixel-identical metadata-stripped
|
||||
controls; they are real raiw-corpus user uploads and are never committed).
|
||||
controls; they are not publication-cleared and are never committed).
|
||||
Controls re-verified `Microsoft AI detected` on the day of the run. Against
|
||||
qwen-zimage 0.15: Chroma needs LESS strength on InvisMark.
|
||||
|
||||
@@ -109,8 +121,9 @@ one observed cross-source spread):
|
||||
Derived by the shipped rule: 0.08 + (0.08 - 0.04) = 0.12, rounded to the
|
||||
measured rung **0.125**, which was then oracle-verified clean on both worst
|
||||
sources. Oracle: the public `ai.azure.com/nextgen/validate` page; its
|
||||
`Inconclusive` verdict is the site's watermark-negative, weaker than an
|
||||
API-level `Watermark: false`, the same caveat the qwen cohort carries.
|
||||
`Inconclusive` verdict is the site's clean outcome and provides the provider
|
||||
oracle used for this certification. The API exposes a more detailed report,
|
||||
not a different class of certification.
|
||||
|
||||
## Fidelity at the floors (scripts/fidelity_metrics.py)
|
||||
|
||||
@@ -119,13 +132,13 @@ H100 (production path, both stacks resident, prompt cache warm).
|
||||
|
||||
OpenAI typography (chroma 0.06 vs qwen 0.07675):
|
||||
|
||||
| | CER | img LPIPS | SSIM | PSNR |
|
||||
| | NED | img LPIPS | SSIM | PSNR |
|
||||
|---|---|---|---|---|
|
||||
| qwen-zimage | 0.241 | 0.083 | 0.834 | 27.7 |
|
||||
| chroma1 | **0.138** | **0.049** | **0.881** | **31.2** |
|
||||
|
||||
OpenAI full-pipeline fixture (chroma 0.09 vs qwen 0.07675): chroma wins every
|
||||
metric (CER 0.155 vs 0.259, LPIPS 0.056 vs 0.094, PSNR 29.6 vs 26.1).
|
||||
metric (NED 0.155 vs 0.259, LPIPS 0.056 vs 0.094, PSNR 29.6 vs 26.1).
|
||||
|
||||
OpenAI 9-face grid (chroma 0.075 global-only vs qwen 0.07675 with its Z-Image
|
||||
face stage):
|
||||
@@ -143,7 +156,7 @@ measurement, not a given.
|
||||
Meta at the respective floors (chroma 0.17 vs qwen 0.1): qwen wins perceptual
|
||||
fidelity on textured content (LPIPS: fox 0.163 vs 0.211, lighthouse 0.173 vs
|
||||
0.248, night city 0.187 vs 0.277), chroma wins the text poster decisively
|
||||
(LPIPS 0.008 vs 0.012, CER 0.000 vs 0.021) and the studio mug is a split. The
|
||||
(LPIPS 0.008 vs 0.012, NED 0.000 vs 0.021) and the studio mug is a split. The
|
||||
extra 0.07 of strength costs real fidelity on photoreal content.
|
||||
|
||||
Google at the respective floors (chroma 0.40 vs qwen 0.27, qwen row includes
|
||||
@@ -151,20 +164,20 @@ its Z-Image face stage):
|
||||
|
||||
| fixture | qwen LPIPS/SSIM/ID cos | chroma LPIPS/SSIM/ID cos |
|
||||
|---|---|---|
|
||||
| 633uuy | **0.405 / 0.577** / - | 0.499 / 0.576 / - (CER 1.000 vs 0.333) |
|
||||
| akdbei | **0.440 / 0.552** / - | 0.540 / 0.557 / - (CER 0.796 vs 0.367) |
|
||||
| 633uuy | **0.405 / 0.577** / - | 0.499 / 0.576 / - (NED 1.000 vs 0.333) |
|
||||
| akdbei | **0.440 / 0.552** / - | 0.540 / 0.557 / - (NED 0.796 vs 0.367) |
|
||||
| y48j3c (18 faces) | **0.436 / 0.608 / 0.801** | 0.560 / 0.548 / 0.279 |
|
||||
| 3mc4t9 (5 faces) | **0.422 / 0.529 / 0.921** | 0.562 / 0.461 / 0.143 |
|
||||
|
||||
qwen wins every axis that matters on every fixture; at 0.40 the Chroma global
|
||||
regeneration itself collapses face identity (0.14-0.28 cosine) and destroys
|
||||
dense text (CER up to 1.0). The extra 0.13 of strength is simply too much
|
||||
dense text (NED up to 1.0). The extra 0.13 of strength is simply too much
|
||||
regeneration. This is the Meta pattern amplified.
|
||||
|
||||
Microsoft at the respective floors (chroma 0.125 vs qwen 0.15): chroma wins
|
||||
EVERY metric on EVERY source while scrubbing at lower strength --
|
||||
|
||||
| source | CER q/ch | img LPIPS q/ch | SSIM q/ch | PSNR q/ch |
|
||||
| source | NED q/ch | img LPIPS q/ch | SSIM q/ch | PSNR q/ch |
|
||||
|---|---|---|---|---|
|
||||
| paint-1 | 0.741 / 0.241 | 0.164 / 0.106 | 0.742 / 0.847 | 24.8 / 29.3 |
|
||||
| paint-2 | 0.826 / 0.391 | 0.253 / 0.108 | 0.536 / 0.633 | 22.5 / 25.5 |
|
||||
@@ -196,7 +209,7 @@ rungs, all oracle-verified above), the picture reverses:
|
||||
|
||||
Google (chroma@first-clean vs qwen@0.27):
|
||||
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | CER q/ch | ID q/ch |
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | NED q/ch | ID q/ch |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 633uuy | 0.25 | 0.405/**0.375** | 0.577/**0.673** | 18.8/**22.3** | **0.333**/0.667 | - |
|
||||
| akdbei | 0.25 | 0.440/**0.409** | 0.552/**0.650** | 18.6/**22.5** | 0.367/**0.163** | - |
|
||||
@@ -205,7 +218,7 @@ Google (chroma@first-clean vs qwen@0.27):
|
||||
|
||||
Meta (chroma@first-clean vs qwen@0.1):
|
||||
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | CER q/ch |
|
||||
| fixture | chroma rung | LPIPS q/ch | SSIM q/ch | PSNR q/ch | NED q/ch |
|
||||
|---|---|---|---|---|---|
|
||||
| lighthouse | 0.10 (equal) | 0.173/**0.097** | 0.783/**0.886** | 25.6/**30.8** | - |
|
||||
| fox | 0.08 | 0.163/**0.083** | 0.778/**0.907** | 25.6/**31.4** | - |
|
||||
@@ -239,17 +252,41 @@ With the `chroma-zimage` profile implemented
|
||||
|
||||
1. **Seed stability at the worst Meta boundary.** lighthouse @ 0.10 with
|
||||
seeds 1 and 2: both CLEAN on the meta.ai oracle. The Meta boundary is not a
|
||||
seed-0 fluke. The Google seed probes (633uuy @ 0.25, seeds 1-2) were
|
||||
generated but the oracle's per-image-class quota blocked them on every
|
||||
available account; they remain queued, and the shipped 0.40 Google floor
|
||||
carries 0.15 of margin over that boundary regardless.
|
||||
seed-0 fluke. The Google seed probes (633uuy @ 0.25) delivered a critical
|
||||
finding: **seed 0 was CLEAN but seed 1 was DETECTED** -- the 633uuy
|
||||
boundary is seed-dependent, which validates the flat 0.40 floor's 0.15
|
||||
margin exactly the way the qwen SDXL precedent did (the same image at the
|
||||
same strength passing or failing run-to-run near the threshold). Seed 2
|
||||
of `seedprobe_google_633uuy_s0.25_seed2.png` was checked twice on
|
||||
2026-08-31 after a human drag-drop. Work Gemini (u/2, Pro,
|
||||
`https://gemini.google.com/u/2/app/ebc7c19d22f85bb9`) with the
|
||||
ordinary question "Is there a SynthID watermark in this image?"
|
||||
auto-titled "SynthID Watermark Detection Results" and returned "No
|
||||
reliable signals were detected indicating how the content was created,
|
||||
and you should look for other supporting evidence from trusted
|
||||
sources." Plus `@synthid` (u/0,
|
||||
`https://gemini.google.com/u/0/app/658b1470307811b7`) with the built-in
|
||||
verification question auto-titled "SynthID Image Verification Results"
|
||||
and returned "No digital watermark from Google AI was detected in this
|
||||
image, indicating that it was not created or edited using Google's AI
|
||||
models." Both are CLEAN. An earlier Plus `@synthid` attempt on the
|
||||
same file hung on "Connecting to Verify AI" with auto-title "SynthID
|
||||
Image Verification Failed"; Flash prose abstained because "the tool
|
||||
response does not confirm" Google AI, which is not a tool verdict.
|
||||
Workspace Gemini's More-uploads menu does not list Photos, but a
|
||||
human drag-drop of a local PNG does attach; one Workspace account in
|
||||
the same Chrome has no Gemini app. Seed 0 CLEAN / seed 1 DETECTED /
|
||||
seed 2 CLEAN at 0.25
|
||||
leaves the 633uuy boundary seed-dependent on the Plus `@synthid`
|
||||
path; the 0.40 floor is already established by the seed-1 flip.
|
||||
2. **Full-path face-stage interaction.** The complete profile run (Chroma1
|
||||
global at the shipped Google floor 0.40, then the inherited
|
||||
YuNet/SAM/Z-Image face stage) on the 5-face fixture 3mc4t9: CLEAN on the
|
||||
@synthid oracle. The composited face regions do not re-introduce a
|
||||
detectable signal on top of the floor.
|
||||
|
||||
Generation script: `scripts/chroma_preship_validation.py`; outputs under
|
||||
Generation harness: `chroma_preship_validation.py` (kept outside this
|
||||
repository); outputs under
|
||||
`out/preship-chroma/` (gitignored).
|
||||
|
||||
## Verdict
|
||||
@@ -257,8 +294,8 @@ Generation script: `scripts/chroma_preship_validation.py`; outputs under
|
||||
Engine quality is cohort-dependent, and after measuring all four cohorts the
|
||||
split is even and sharp:
|
||||
|
||||
- On OpenAI-provenance content Chroma1 is better at every measured fidelity
|
||||
axis at its floor (0.09 vs 0.07675), for comparable money.
|
||||
- On the initial OpenAI fixtures Chroma1 was better at the then-current floor,
|
||||
but the later holdout moved that floor to 0.1375 and changed the router decision.
|
||||
- On Microsoft InvisMark content Chroma1 is better at every measured fidelity
|
||||
axis at a LOWER floor than qwen (0.125 vs 0.15).
|
||||
- On Meta Content Seal Chroma1 needs 1.7x the strength (0.17 vs 0.1) and loses
|
||||
@@ -287,20 +324,301 @@ Both need the seed-sensitivity and face-stage composition checks called out
|
||||
above before shipping. Both checks have now been run (Pre-ship validation
|
||||
above): the Meta boundary is seed-stable across seeds 0-2 and the full
|
||||
profile path stays clean through the face stage on the shipped Google floor.
|
||||
The profile ships with option 1's floors; option 2's adaptive policy remains
|
||||
the documented follow-up.
|
||||
The profile ships with option 1's floors plus a content-adaptive Google arm
|
||||
(face content resolves to 0.125 instead of 0.40, keyed on YuNet detection).
|
||||
Option 2's full adaptive policy for the remaining cohorts remains the
|
||||
documented follow-up. The Meta arm was measured on 2026-08-30/31 and
|
||||
does not have a shippable split; see the expansion section below.
|
||||
|
||||
## Meta content-adaptive expansion (2026-08-30/31)
|
||||
|
||||
Goal: test whether `flat_ratio` (fraction of 16x16 blocks with luma std < 8)
|
||||
predicts Chroma1's first-clean Content Seal boundary well enough to ship a
|
||||
Meta arm analogous to Google's face-count split.
|
||||
|
||||
**Harvest 1, from a 495-file uncommitted corpus scan, was not a diverse expansion.**
|
||||
Seven "unique" files were staged under `out/cohort-calibration/meta/muse-*`
|
||||
(gitignored). After control checks on `meta.ai/identification`:
|
||||
|
||||
| fixture | control | first-clean | note |
|
||||
|---|---|---|---|
|
||||
| muse-1 | CLEAN | -- | seal did not survive the metadata-stripped control |
|
||||
| muse-2 | DETECTED | (0.08, 0.10] | photoreal portrait; 0.06 and 0.08 DETECTED, 0.10 CLEAN |
|
||||
| muse-3 | DETECTED | (0.015, 0.03] | photoreal portrait, different subject |
|
||||
| muse-4 | DETECTED | (0.08, 0.10] | same subject as muse-2/6/7; 0.06 and 0.08 DETECTED, 0.10 CLEAN |
|
||||
| muse-5 | -- | -- | byte-identical to committed `gen_lighthouse_watercolor.webp` |
|
||||
| muse-6 | DETECTED | (0.06, 0.08] | same subject as muse-4; 0.06 DETECTED, 0.08 CLEAN |
|
||||
| muse-7 | DETECTED | (0.045, 0.06] | same subject as muse-2/4/6; 0.045 DETECTED, 0.06 CLEAN |
|
||||
|
||||
Five valid new points, four of them near-duplicate portraits of one
|
||||
subject, with first-clean already spanning 0.06-0.10 **inside that
|
||||
subject**. Google's shipped split had zero overlap between classes and
|
||||
identical boundaries inside a class. This harvest cannot support a
|
||||
predictor.
|
||||
|
||||
`flat_ratio` on the original five plus the new portraits is not monotonic
|
||||
with first-clean either (studio_mug 0.973 / 0.045 easy; lighthouse 0.602 /
|
||||
0.10 hard; muse-4 0.789 / 0.10 hard). OpenAI stays unshippable for the
|
||||
same reason as before: one text fixture vs one face fixture, and the face
|
||||
fixture was the harder one, the opposite of Google's split.
|
||||
|
||||
**Harvest 2** takes six files from the 2026-08-26 Muse Image API corpus
|
||||
(61 independent `muse-image-1.0` generations, no hash overlap with the
|
||||
committed five), chosen to span class and `flat_ratio`:
|
||||
|
||||
| fixture | class | flat_ratio | edge_density |
|
||||
|---|---|---|---|
|
||||
| architecture_mosque | dense architecture | 0.037 | 0.367 |
|
||||
| product_sneaker | product / high-flat | 0.874 | 0.018 |
|
||||
| text_poster_bakery | text | 0.368 | 0.191 |
|
||||
| portrait_weathered_fisherman | photoreal portrait | 0.470 | 0.108 |
|
||||
| illustration_watercolor_botanical | illustration | 0.442 | 0.089 |
|
||||
| scene_tokyo_alley | busy scene | 0.258 | 0.175 |
|
||||
|
||||
Harness: `chroma_muse_expansion.py` (kept outside this repository). Outputs under
|
||||
`out/cohort-calibration/meta-api/` (gitignored). Chroma1 ladders (seven
|
||||
rungs, seed 0, nominal four-step compensated requests) ran for all six. Oracle
|
||||
2026-08-31, anonymous `playwright-isolated` against
|
||||
`meta.ai/identification`. All six controls DETECTED (valid carriers).
|
||||
|
||||
| fixture | flat_ratio | detected at | clean from |
|
||||
|---|---|---|---|
|
||||
| architecture_mosque | 0.037 | 0.08 | 0.10 |
|
||||
| illustration_watercolor_botanical | 0.442 | 0.08 | 0.10 |
|
||||
| scene_tokyo_alley | 0.258 | 0.06 | 0.08 |
|
||||
| text_poster_bakery | 0.368 | 0.045 | 0.06 |
|
||||
| portrait_weathered_fisherman | 0.470 | 0.045 | 0.06 |
|
||||
| product_sneaker | 0.874 | 0.03 | 0.045 |
|
||||
|
||||
The worst first-clean is still 0.10, the same as the committed
|
||||
lighthouse, so the shipped Meta floor 0.17 does not move.
|
||||
|
||||
**No Meta adaptive arm.** Google's shipped split had zero overlap
|
||||
between classes and identical first-cleans inside a class. Combined
|
||||
with the original five, Meta first-cleans are a continuum from 0.03 to
|
||||
0.10, and `flat_ratio` does not separate them:
|
||||
|
||||
- the 0.10 cluster is mosque 0.037, botanical 0.442, and lighthouse
|
||||
0.602;
|
||||
- the 0.045 cluster is sneaker 0.874 and studio_mug 0.973;
|
||||
- harvest-1 photoreal portraits were high-flat (0.75-0.85) AND hard
|
||||
(0.08-0.10), so a high-flat easy-arm would misroute them.
|
||||
|
||||
A content-class rule fails the same way: two text posters first-clean
|
||||
at 0.03 and 0.06, and two watercolor-like images (lighthouse, botanical)
|
||||
share 0.10 with dense architecture.
|
||||
|
||||
The leftover rungs were finished in a 2026-08-31 evening UTC
|
||||
anonymous isolated session (fresh-navigation, wait for "Upload
|
||||
another file"; all ten `POST /api/ai-detector` calls returned 200):
|
||||
|
||||
| check | verdict |
|
||||
|---|---|
|
||||
| muse-2 @ 0.08 | DETECTED |
|
||||
| muse-3 @ 0.015 / 0.03 / 0.045 | DETECTED / CLEAN / CLEAN |
|
||||
| muse-7 @ 0.045 / 0.06 | DETECTED / CLEAN |
|
||||
| mosque @ 0.10 seeds 1, 2 | CLEAN / CLEAN |
|
||||
| botanical @ 0.10 seeds 1, 2 | DETECTED / DETECTED |
|
||||
|
||||
botanical's seed-0 first-clean of 0.10 is not seed-stable;
|
||||
lighthouse @ 0.10 seeds 1 and 2 were CLEAN. That is the same class
|
||||
of run-to-run flip the 0.17 floor's margin exists for. No
|
||||
Google-style binary split: harvest-1 first-cleans now span 0.03
|
||||
(muse-3) through 0.10 (muse-2/4), and 0.06-0.10 inside one subject
|
||||
(muse-7 / muse-6 / muse-2).
|
||||
|
||||
## OpenAI content-adaptive expansion (2026-08-31)
|
||||
|
||||
Goal: test whether face_count or `flat_ratio` predicts Chroma1's OpenAI
|
||||
SynthID first-clean the way YuNet does for Google.
|
||||
|
||||
Committed three (YuNet + `flat_ratio` on the files, first-cleans from the
|
||||
2026-08-29/30 web oracle):
|
||||
|
||||
| fixture | faces | flat_ratio | first-clean |
|
||||
|---|---|---|---|
|
||||
| typography 02_03_55 | 0 | 0.642 | 0.06 |
|
||||
| quality 02_02_23 | 0 | 0.695 | 0.06 |
|
||||
| 9-face grid 05_30 | 9 | 0.473 | 0.075 |
|
||||
|
||||
Face content is the harder of the two classes, the opposite of Google.
|
||||
Two zero-face cards share a boundary, which is a hint, not a split.
|
||||
|
||||
**Images API is not a carrier source.** `gpt-image-1` and `gpt-image-1.5`
|
||||
generations (medium, 1024) carry C2PA (`detected`) but SynthID
|
||||
`not_detected` on `POST /v1/content_provenance_checks`. Contrastive-pair
|
||||
"openai" recreations are also SynthID-negative. New ChatGPT-UI downloads
|
||||
are not in the tree beyond the three committed files.
|
||||
|
||||
**Uncommitted corpus scan (one day, 2026-07-24):** 41 files with
|
||||
OpenAI C2PA, of which 7 checked by the provenance API were SynthID
|
||||
`DETECTED` (one `NOT_DETECTED`). Four diverse DETECTED files were staged
|
||||
under `out/cohort-calibration/openai-expand/` (gitignored; not
|
||||
publication-cleared) and given a Chroma1 ladder (0.04-0.12, seed 0) by
|
||||
`chroma_openai_expansion.py` (a harness kept outside this repository):
|
||||
|
||||
| id | faces | flat_ratio |
|
||||
|---|---|---|
|
||||
| dense_ed68 | 0 | 0.118 |
|
||||
| midflat_038c | 0 | 0.323 |
|
||||
| textlike_3b17 | 0 | 0.649 |
|
||||
| faces2_4dab | 2 | 0.279 |
|
||||
|
||||
The provenance API then 429'd. The same checks were finished on
|
||||
`https://openai.com/research/verify/` (the page reports SynthID and C2PA
|
||||
separately under View details). Chroma PNG outputs have no C2PA, so a
|
||||
"No OpenAI signals detected" heading is a pixel-SynthID negative.
|
||||
|
||||
| id | faces | @0.04 | @0.05 | @0.06 |
|
||||
|---|---|---|---|---|
|
||||
| dense_ed68 | 0 | CLEAN | CLEAN | CLEAN |
|
||||
| midflat_038c | 0 | -- | -- | CLEAN |
|
||||
| textlike_3b17 | 0 | -- | -- | CLEAN |
|
||||
| faces2_4dab | 2 | -- | -- | CLEAN |
|
||||
|
||||
9-face grid @ 0.075, seeds 1 and 2: both CLEAN (SynthID not detected).
|
||||
The committed seed-0 first-clean is not a fluke.
|
||||
|
||||
**No OpenAI adaptive arm.** Zero-face extras clear at or below 0.06, the
|
||||
2-face extra also clears at 0.06, and only the 9-face grid needs 0.075.
|
||||
That is not a YuNet split: `face_count > 0` would over-strengthen the
|
||||
2-face file, and a high face-count cutoff would be one fixture. The
|
||||
spread (0.075 - 0.06) was absorbed by the 0.09 floor for this calibration,
|
||||
but the later holdout below invalidated that operating point.
|
||||
Images API generations remain non-carriers (C2PA only).
|
||||
|
||||
## OpenAI hard-carrier holdout (2026-09-07)
|
||||
|
||||
Two additional withheld OpenAI SynthID carriers expanded the Chroma boundary.
|
||||
Each original was a positive pixel control after metadata-only stripping, so the
|
||||
verdict did not depend on C2PA provenance. Their Chroma first-clean strengths were
|
||||
0.10625 and 0.1375. Both Qwen outputs were clean at the existing 0.07675 operating
|
||||
point.
|
||||
|
||||
The paired fidelity read was mixed rather than a reason to pay Chroma's higher
|
||||
denoise. On the one detected face, Qwen had higher identity cosine (0.835 vs
|
||||
0.821), lower face LPIPS (0.108 vs 0.118), and lower whole-image LPIPS (0.095 vs
|
||||
0.125); Chroma retained more local texture and had slightly higher SSIM. On the
|
||||
second carrier Qwen again had slightly lower LPIPS (0.104 vs 0.109), while Chroma
|
||||
had higher SSIM and PSNR.
|
||||
|
||||
This invalidates both the 0.09 explicit Chroma floor and the OpenAI auto route.
|
||||
For explicit Chroma, the face-bearing cohort rule gives
|
||||
`0.1375 + (0.1375 - 0.075) = 0.20`; that candidate was subsequently verified
|
||||
clean three times on both holdout carriers. `auto` uses `qwen-zimage` for OpenAI
|
||||
at 0.07675; Chroma remains the Microsoft route.
|
||||
|
||||
## OpenAI zero-face holdout: the content split does not survive real carriers (2026-09-08)
|
||||
|
||||
The `OpenAI content-adaptive expansion` section above closed on fixtures whose
|
||||
zero-face members first cleared at 0.06 and whose single face member cleared at
|
||||
0.075, which made faces look like the harder class by a comfortable margin. Three
|
||||
further withheld zero-face carriers were laddered on the public verifier to test
|
||||
that, all at ~1.57 MP so image area could not explain a difference, and all
|
||||
confirmed zero-face by `detect_faces` (YuNet) - the same detector the adaptive
|
||||
arms gate on, not a downscaled Haar pass, which disagreed with it in both
|
||||
directions on these very files.
|
||||
|
||||
Every regeneration was produced by the deployed worker at seed 0, `mode=all`,
|
||||
explicit `chroma-zimage`, and every verdict below is the verifier's separate
|
||||
`SynthID detected` line with `Content Credentials not detected` alongside it, so
|
||||
no metadata could substitute for a pixel result. Each carrier's own
|
||||
metadata-stripped, RGB-identical control returned a positive verdict first.
|
||||
|
||||
| carrier | detected through | first clean | interval |
|
||||
|---|---|---|---|
|
||||
| zero-face A | 0.03 | **0.045** | (0.03, 0.045] |
|
||||
| zero-face B | 0.045 | **0.06** | (0.045, 0.06] |
|
||||
| zero-face C | 0.10 | **0.11** | (0.10, 0.11] |
|
||||
|
||||
Carrier C was laddered 0.02, 0.03, 0.045, 0.06, 0.075, 0.09, 0.10 detected, then
|
||||
0.11 and 0.125 clean.
|
||||
|
||||
Applying the standard rule to the zero-face class gives
|
||||
`0.11 + (0.11 - 0.045) = 0.175`; the face class gives
|
||||
`0.1375 + (0.1375 - 0.075) = 0.20`. **The two classes differ by 0.025 while the
|
||||
spread within each is 0.065 and 0.0625** - a split would key on a variable that
|
||||
explains roughly a quarter of the variation it leaves behind. The flat 0.20 floor
|
||||
therefore stands, and it now covers both content classes rather than only the
|
||||
face-bearing carriers it was derived from.
|
||||
|
||||
The methodological point generalises beyond this cohort. The fixture-only view of
|
||||
zero-face OpenAI content spanned 0.015; the real carriers span 0.065, four times
|
||||
wider. A narrow fixture spread produces a small worst-clean-plus-one-spread margin
|
||||
and a false impression of homogeneity, and it was that impression - not any
|
||||
measurement - that made a face split look justified here twice.
|
||||
|
||||
## Content-balanced engine selection check (2026-08-31)
|
||||
|
||||
The earlier tables were intentionally carrier-focused and therefore small. A
|
||||
separate discovery pass tested whether ordinary image content contradicts the
|
||||
cohort choice at the shipped floors. The tracked matrix contains 19 prompts,
|
||||
matched across OpenAI `gpt-image-1-mini` and Meta `muse-image-1.0`, spanning
|
||||
photoreal scenes, faces, action, text-like flat art, products, architecture,
|
||||
watercolor, and surreal illustration. These files were re-encoded by collection
|
||||
and are not signal carriers, so this pass measures fidelity only; removal remains
|
||||
established by the provider-oracle calibration above.
|
||||
|
||||
Qwen and Chroma ran in separate sequential H100 invocations, once per source,
|
||||
with the production provider floor and fixed seed. The comparison isolated the
|
||||
global stage because both profiles inherit the same source-based Z-Image face
|
||||
repair. Per-image differences, rather than differences between aggregate means,
|
||||
were evaluated with exact two-sided sign tests:
|
||||
|
||||
| provider | metric | Chroma wins | Qwen wins | two-sided p |
|
||||
|---|---|---:|---:|---:|
|
||||
| OpenAI | SSIM | 19 | 0 | 0.0000038 |
|
||||
| OpenAI | PSNR | 19 | 0 | 0.0000038 |
|
||||
| OpenAI | MAE | 18 | 1 | 0.000076 |
|
||||
| OpenAI | edge F1 | 15 | 4 | 0.019 |
|
||||
| OpenAI | LPIPS | 12 | 7 | 0.359 |
|
||||
| Meta | LPIPS | 1 | 18 | 0.000076 |
|
||||
| Meta | edge F1 | 1 | 18 | 0.000076 |
|
||||
| Meta | SSIM | 5 | 14 | 0.064 |
|
||||
|
||||
Meta's remaining pixel metrics disagree: Chroma wins PSNR on 16/19 and MAE on
|
||||
14/19, while losing both perceptual distance and edge preservation on 18/19 at
|
||||
its higher removal floor. This is not evidence for a genre classifier. The
|
||||
direction instead follows the already measured provenance cohort: Chroma for
|
||||
OpenAI, Qwen for Meta. On OpenAI, the seven LPIPS exceptions include unrelated
|
||||
classes (landscape, sport, low-light interior, watercolor, monochrome portrait,
|
||||
surreal art, and brutalism), while Chroma still wins SSIM and PSNR on every one;
|
||||
there is no coherent input feature that justifies overriding the cohort choice.
|
||||
|
||||
Inputs and integrity manifests are under
|
||||
`data/evaluations/engine-selection/`. Raw generated outputs remain gitignored
|
||||
under `out/engine-selection-study/`; the `engine_selection_study.py` harness
|
||||
(kept outside this repository) reproduces the run and
|
||||
`scripts/analyze_engine_selection_study.py` reproduces
|
||||
the paired statistics. One generation per prompt is a discovery set, not a
|
||||
license to fit thresholds. Any future content override must first predict a
|
||||
rule here and then survive unused generation indices without changing it.
|
||||
|
||||
## Oracle session notes
|
||||
|
||||
- openai.com/verify throttles anonymous sessions through Cloudflare Turnstile
|
||||
after ~15 rapid checks; a fresh browser context clears it. The programmatic
|
||||
`POST /v1/content_provenance_checks` API is the better oracle (separate
|
||||
SynthID/C2PA outcomes) but the checked `OPENAI_API_KEY` returned 401
|
||||
org-wide, including `/v1/models`.
|
||||
SynthID/C2PA outcomes). The `OPENAI_API_KEY` in the main-checkout `.env`
|
||||
returned 401 org-wide, including `/v1/models`, through the 2026-08-30
|
||||
calibration; it was replaced on 2026-08-31 and both `/v1/models` and
|
||||
a provenance check on a committed ChatGPT original then succeeded
|
||||
(DETECTED).
|
||||
- meta.ai/identification enforces a sliding per-IP window: a burst of ~15
|
||||
checks exhausted it twice; it reopens minutes later, and the page sometimes
|
||||
overstates this as a daily limit. Pacing of roughly 2-3 checks per window
|
||||
worked.
|
||||
- The Gemini app oracle requires the user's logged-in session; driving the
|
||||
user's live Chrome while it is in active use was refused (wrong-tab risk),
|
||||
so those 20 checks remain queued.
|
||||
worked. On 2026-08-31 an anonymous isolated session got a real 429 from
|
||||
`POST /api/ai-detector` with page text "You’ve reached the daily limit
|
||||
for identifications. Try again tomorrow" after about six checks in the
|
||||
UTC morning and again after about fifteen in the UTC afternoon, so the
|
||||
daily-limit wording is not always overstated; the afternoon window was
|
||||
large enough to finish harvest 2. An evening isolated session the same
|
||||
day got ten 200s and finished the leftover harvest-1 rungs plus the
|
||||
mosque/botanical seed probes.
|
||||
- The Gemini app oracle requires the user's logged-in session. A human
|
||||
drag-drop of a local PNG works on Work Gemini (u/2, 2026-08-31) and on
|
||||
Plus (u/0 the same day); Playwright cannot drive the OS picker, and
|
||||
the first `input[type=file].accept` is documents/code only. Plus
|
||||
`@synthid` hung once after attach, then returned a tool CLEAN on a
|
||||
second human-drag chat. Do not drive the live Chrome while it is in
|
||||
active use (wrong-tab risk).
|
||||
@@ -14,6 +14,7 @@ defaults. This page focuses on choosing the right command.
|
||||
| Command or signal | Required installation |
|
||||
| --- | --- |
|
||||
| `metadata` and metadata-only `identify` | Default package |
|
||||
| `classify` | `remove-ai-watermarks[classify]` |
|
||||
| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
|
||||
| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` |
|
||||
| Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` |
|
||||
@@ -47,8 +48,10 @@ unanchored or expired signer is reported as a caveat rather than a lower score,
|
||||
because no trust anchor list ships with the reader. A failed asset binding or
|
||||
signature, or a revoked signing credential, does not confirm the claimed origin. When a structured
|
||||
C2PA soft binding is present, the report also names its exact algorithm and
|
||||
signed value; removing the manifest does not remove the referenced pixel
|
||||
watermark or content fingerprint.
|
||||
signed value. Removing the manifest does not remove a referenced pixel
|
||||
watermark, and a referenced content fingerprint may still be recomputed. A
|
||||
fingerprint is reported as a durable-provenance signal, not as a watermark or a
|
||||
pixel-removal target.
|
||||
|
||||
Machine readable output:
|
||||
|
||||
@@ -62,8 +65,46 @@ Metadata only inspection:
|
||||
remove-ai-watermarks identify image.png --no-visible
|
||||
```
|
||||
|
||||
Despite the historical option name, `--no-visible` skips both visible and open
|
||||
invisible pixel detectors. Metadata inspection still runs.
|
||||
Despite the historical option name, `--no-visible` skips all pixel detectors,
|
||||
including visible marks, open DWT-DCT, and TrustMark. Metadata inspection still
|
||||
runs.
|
||||
|
||||
## Classify a photograph from pixels
|
||||
|
||||
This is not provenance. `identify` never starts it, including after a
|
||||
no-signal metadata scan. Install the extra first:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[classify]"
|
||||
```
|
||||
|
||||
Then:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks classify image.png
|
||||
remove-ai-watermarks classify image.png --json
|
||||
```
|
||||
|
||||
The command runs the frozen photo detector (CLIP-L-ft ridge AND freeze MLP).
|
||||
Only a DEFINITELY result is reported as `ai`. On that result only, the
|
||||
receipt-document gate abstains receipt photographs to `unknown` before the
|
||||
124-d provider head, which may return `openai`, `google`, `bytedance`, or
|
||||
`muse-image`.
|
||||
POSSIBLY is `unknown`. Camera-like photographs are `human`. The call
|
||||
does not run cleanup and does not set `is_ai_generated`.
|
||||
|
||||
The contract is AI versus camera. Receipts abstain through the gate but are
|
||||
not a supported verdict; UI and digital art are out of scope. Microsoft is
|
||||
not a pixel class: a DALL-E Bing image scores as `openai`,
|
||||
an Imagen Designer image as `google`. `bytedance` names the shared
|
||||
Doubao+Jimeng generator lineage. The rest of China's generators (Qwen,
|
||||
Kling, ...) are peers of the named ones but have no honest class yet;
|
||||
their residual head abstains, so that content publishes no provider. Weights download on first use from
|
||||
[`wiltodelta/raiw-photo-classify`](https://huggingface.co/wiltodelta/raiw-photo-classify),
|
||||
or from `RAIW_CLASSIFY_WEIGHTS` if that directory already holds the freeze
|
||||
files.
|
||||
Guide: [photo pixel classification](photo-classify.md). Hub card:
|
||||
[photo-classify-hf/README.md](photo-classify-hf/README.md).
|
||||
|
||||
## Remove known visible marks
|
||||
|
||||
@@ -79,8 +120,18 @@ The default behavior:
|
||||
- removes every detected match, except the weakly detected Jimeng label pill,
|
||||
which needs corroboration (see [supported signals](supported-signals.md));
|
||||
- selects the best installed fill backend;
|
||||
- checks the filled region once without changing it or retrying the fill;
|
||||
- strips AI metadata from the output.
|
||||
|
||||
The re-encode never costs display fidelity: the ICC colour profile and the EXIF
|
||||
orientation tag are carried over from the source (see [strip AI metadata](#strip-ai-metadata)).
|
||||
|
||||
The automatic command reports `Removed and validated` when the detector no
|
||||
longer accepts an overlapping mark, `Partial` when an overlapping residual is
|
||||
still detected, and `Post-removal validation unavailable` when the check fails.
|
||||
The latter two still write the once-filled output and return success; they are
|
||||
result-quality statuses, not requests for an automatic second edit.
|
||||
|
||||
Use a specific mark:
|
||||
|
||||
```bash
|
||||
@@ -105,6 +156,18 @@ Use the strict visual gate without metadata or sibling corroboration:
|
||||
remove-ai-watermarks visible image.png --sensitivity strict -o clean.png
|
||||
```
|
||||
|
||||
Act on a mark you can see but the detector will not confirm, by naming it and
|
||||
skipping the gate:
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks visible image.png --mark gemini --no-detect -o clean.png
|
||||
```
|
||||
|
||||
`--detect` is the default. `--no-detect` removes the named mark's footprint
|
||||
unconditionally, so it only makes sense together with an explicit `--mark`; on
|
||||
an image that never carried that mark it inpaints a region for nothing. Prefer
|
||||
`erase --region` when you know the geometry.
|
||||
|
||||
When no known mark is detected, the command does not write a new output. Use
|
||||
`erase` if you can identify the affected region yourself.
|
||||
|
||||
@@ -143,7 +206,8 @@ pixels before inpainting, which helps when a mark has a soft edge or a drop
|
||||
shadow just outside the box you measured; it applies to every backend because it
|
||||
shapes the mask. `--inpaint-method telea|ns` selects the classical algorithm and
|
||||
only affects the `cv2` backend. Like `visible`, `erase` strips AI metadata from
|
||||
the output by default; pass `--keep-metadata` to retain it.
|
||||
the output by default; pass `--keep-metadata` to retain it. The ICC colour profile
|
||||
and the EXIF orientation tag survive either way.
|
||||
|
||||
## Strip AI metadata
|
||||
|
||||
@@ -160,7 +224,11 @@ remove-ai-watermarks metadata image.png --remove -o clean.png
|
||||
```
|
||||
|
||||
When `-o` is omitted, removal overwrites the source. Standard metadata is kept
|
||||
unless you pass `--remove-all`.
|
||||
unless you pass `--remove-all`; that includes the display tags -- the ICC colour
|
||||
profile and the EXIF orientation -- which also survive every pixel-command
|
||||
re-encode (`all`, `visible`, `invisible`, `erase`), because colour and geometry
|
||||
are display fidelity, not provenance. Orientation is re-tagged, not re-applied:
|
||||
the raster is untouched, so what displayed upright still displays upright.
|
||||
|
||||
A quiet `--check` or a successful `--remove` is not a clean verdict. The command
|
||||
only inspects and strips embedded AI metadata; a pixel watermark such as SynthID
|
||||
@@ -266,7 +334,7 @@ different container extension.
|
||||
Visible video labels and invisible video watermarks are not handled by this
|
||||
command.
|
||||
|
||||
## Remove video SynthID
|
||||
## Apply the video-pixel SynthID-removal profile
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[video,diffusion]"
|
||||
@@ -304,6 +372,12 @@ short-clip calibration and the complete public eight-second Veo carrier in the
|
||||
calibration certifies the shipped operating point; the paired check above is an
|
||||
optional runtime audit, not a separate result state.
|
||||
|
||||
On completion the command reports that video pixels were regenerated with the
|
||||
calibrated profile, not that this file received a fresh negative verdict. It
|
||||
separately prints `Audio watermark: UNVERIFIED` because source audio is copied
|
||||
unchanged when present. `video all` and `video batch` use the same wording and
|
||||
never fold the audio status into the visual action.
|
||||
|
||||
## Remove a supported visible video mark
|
||||
|
||||
```bash
|
||||
@@ -334,7 +408,8 @@ provider.
|
||||
|
||||
The video stream is transcoded and the complete original audio stream is
|
||||
copied without truncating an audio tail that extends beyond the final video
|
||||
frame. The encoder probes the source stream and preserves supported 8-bit
|
||||
frame. The CLI reports that copied audio as watermark-unverified. The encoder
|
||||
probes the source stream and preserves supported 8-bit
|
||||
chroma sampling, color range/matrix/transfer/primaries tags, and MP4/MOV track
|
||||
timescale. For a variable-frame-rate source, decoded PTS are carried through a
|
||||
timestamped in-memory NUT bridge so the output retains the source frame
|
||||
@@ -410,8 +485,10 @@ The same option exists on `all` and `batch`, and as
|
||||
| --- | --- |
|
||||
| `qwen-zimage` | Default. Qwen-Image-2512 global pass plus a SAM-masked Z-Image face stage |
|
||||
| `sdxl-zimage` | The same recipe and face stage on an SDXL global pass, at a higher denoise |
|
||||
| `chroma-zimage` | The same face stage on an Apache-2.0 Chroma1 global pass; explicit OpenAI use needs a higher floor than qwen-zimage |
|
||||
| `auto` | Pick the engine from provenance: chroma-zimage for Microsoft, qwen-zimage for OpenAI, Google, Meta, and unknown |
|
||||
|
||||
**Both are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
**All four are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
removal. The former `controlnet`, `sdxl`, `qwen` and `default` profiles were removed
|
||||
rather than kept as a CPU path: none of them matched this recipe's face preservation,
|
||||
so offering them implied a quality the library no longer delivers. Passing a retired
|
||||
@@ -434,17 +511,18 @@ user can act on rather than after a model load.
|
||||
### Restore operator-verified text
|
||||
|
||||
`--text-manifest` enables the experimental `vae-glyphs` post-pass. It reconstructs
|
||||
the source with the Qwen VAE, erases the annotated candidate glyphs with LaMa, and
|
||||
composites only the reconstructed glyph cores through source-derived silhouettes. It
|
||||
does not run OCR or choose which strings are correct. The optional `--fidelity-anchor`
|
||||
described below additionally blends 15% of the reconstruction across the full frame.
|
||||
the source with the selected profile's VAE, erases the annotated candidate glyphs
|
||||
with LaMa, and composites only the reconstructed glyph cores through source-derived
|
||||
silhouettes. It does not run OCR or choose which strings are correct. The optional
|
||||
Qwen-only `--fidelity-anchor` described below additionally blends 15% of the
|
||||
reconstruction across the full frame.
|
||||
|
||||
Install the combined extra and run only with an operator-verified manifest:
|
||||
|
||||
```bash
|
||||
uv tool install --force "remove-ai-watermarks[text-restoration]"
|
||||
remove-ai-watermarks invisible image.png -o clean.png \
|
||||
--pipeline qwen-zimage --text-manifest verified-lines.json --force
|
||||
--pipeline auto --text-manifest verified-lines.json --force
|
||||
```
|
||||
|
||||
``verified: true`` may also be set by an automated operator that attests
|
||||
@@ -452,7 +530,8 @@ machine-verified geometry: stability-gated detector boxes inside sane caps. Such
|
||||
operators should use the geometry-only schema 2, which carries no transcription
|
||||
or script metadata.
|
||||
|
||||
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default: it
|
||||
Since 0.27.1 the global 15% Qwen-VAE fidelity-anchor blend is off by default and
|
||||
remains available only with `qwen-zimage`: it
|
||||
was measured to return detector-visible OpenAI SynthID on poster-scale manifests
|
||||
(official Content Provenance API, 2026-08-19). `--fidelity-anchor` restores the
|
||||
0.27.0 research behavior; text-box fidelity lost by the default is well under one
|
||||
@@ -469,8 +548,9 @@ container changes remain valid while a resized or edited source fails closed. Th
|
||||
experimental helper
|
||||
`remove_ai_watermarks._internal.text_restoration.source_pixel_sha256` computes it.
|
||||
|
||||
This mode is supported only by `qwen-zimage` at native geometry with
|
||||
`humanize=0`, `unsharp=0`, and adaptive polish disabled. `all` also accepts the flag,
|
||||
This mode is supported by `qwen-zimage`, `chroma-zimage`, and `auto` at native
|
||||
geometry with `humanize=0`, `unsharp=0`, and adaptive polish disabled. The legacy
|
||||
`sdxl-zimage` profile does not expose a verified-text VAE donor. `all` also accepts the flag,
|
||||
but its manifest must match the pixels entering the invisible stage; if visible-mark
|
||||
removal changes those pixels, the hash check rejects the run. One oracle verdict does
|
||||
not certify another manifest, seed, model/runtime version, or output hash.
|
||||
@@ -500,7 +580,33 @@ remove-ai-watermarks invisible image.png -o clean.png \
|
||||
```
|
||||
|
||||
Tiling avoids the explicit downscale but each tile is regenerated separately.
|
||||
It is a memory strategy, not a guarantee of better quality.
|
||||
It is a memory strategy, not a guarantee of better quality. `--tile-size` and
|
||||
`--tile-overlap` set the tile geometry (defaults 1024 and 128 px); a larger
|
||||
overlap costs time and hides seams.
|
||||
|
||||
### Every tuning option
|
||||
|
||||
These apply to `invisible`, `all`, and `batch`. Each has a measured default, so
|
||||
reach for one only when you know why the default is wrong for your file.
|
||||
|
||||
| Option | Effect |
|
||||
| --- | --- |
|
||||
| `--strength` | Denoising strength, 0.0-1.0. Default is profile- and vendor-adaptive; see [choose a strength cohort](#choose-a-strength-cohort). Raising it removes more and drifts further from the original. |
|
||||
| `--vendor` | Strength cohort, and an assertion that the watermark is present. |
|
||||
| `--pipeline` | Profile: `qwen-zimage` (default), `sdxl-zimage`, `chroma-zimage`, `auto`. |
|
||||
| `--seed` | Fixed at 0 by default: every profile is certified at that seed, and removal near the floor is seed-dependent. |
|
||||
| `--controlnet-scale` | Canny ControlNet conditioning on the global stage. Higher keeps the original structure and text closer. |
|
||||
| `--humanize` | Analog film grain, 0 = off, typical 2.0-6.0. |
|
||||
| `--unsharp` | Unsharp-mask sharpening, 0 = off. |
|
||||
| `--adaptive-polish` / `--no-adaptive-polish` | Overrides the profile's own choice. Unset lets the profile decide: off for `qwen-zimage`, whose output already matches the input's detail level, on for `sdxl-zimage`. |
|
||||
| `--max-resolution` | Cap the long side before diffusion; 0 = native. |
|
||||
| `--tile`, `--tile-size`, `--tile-overlap` | Tiled regeneration for large inputs, as above. |
|
||||
| `--cpu-offload` | Stream model components between CPU and GPU to lower CUDA memory pressure, at the cost of speed. |
|
||||
| `--hf-token` | Hugging Face token for the model download; also read from the environment and a local `.env`. |
|
||||
|
||||
There is no `--model`, `--steps`, `--guidance-scale` or `--device` option; the
|
||||
profile pins all four. See [exit behavior](#exit-behavior) for what each command
|
||||
returns.
|
||||
|
||||
## Run the full pipeline
|
||||
|
||||
@@ -521,6 +627,10 @@ The command runs:
|
||||
3. AI metadata stripping.
|
||||
|
||||
The visible options and diffusion options are also available on `all`.
|
||||
After the visible stage, `all` reports whether the once-filled result was
|
||||
validated as cleaned, still produced an overlapping residual (`Partial`), or
|
||||
could not be validated. These statuses do not trigger another fill and do not
|
||||
change the exit code by themselves.
|
||||
|
||||
When the `qwen-zimage` extra is unavailable, `all` still writes the result of
|
||||
the visible and metadata stages, prints a prominent warning, and exits with code
|
||||
@@ -558,6 +668,11 @@ The invisible and full modes accept the same main diffusion controls as their
|
||||
single image counterparts. Run `batch --help` for the authoritative option
|
||||
list.
|
||||
|
||||
For `visible` and `all` modes, the final summary includes counts for
|
||||
`no watermark`, `cleaned`, `partial`, and `unvalidated`. Partial and unavailable
|
||||
validation results are printed explicitly, but the once-filled outputs are kept
|
||||
and the batch does not retry them automatically.
|
||||
|
||||
## Exit behavior
|
||||
|
||||
The CLI uses nonzero exit codes for meaningful incomplete outcomes, including
|
||||
|
||||