Compare commits
No files matched your search
@@ -6,7 +6,7 @@
|
||||
},
|
||||
"metadata": {
|
||||
"description": "Agent skill and Claude Code plugin for the remove-ai-watermarks CLI.",
|
||||
"version": "1.0.10"
|
||||
"version": "1.0.12"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
|
||||
@@ -29,9 +29,9 @@ The unified maintainer entry point is `scripts/provider_oracles.py`; its workflo
|
||||
slot schema live in `docs/provider-oracles.md`. Multi-account and multi-network work
|
||||
selects each named slot explicitly from the gitignored repository-local
|
||||
`.oracle-slots.json` by default and snapshots it into a separate immutable batch
|
||||
manifest. A bounded provider check may continue through its finite set of authorized
|
||||
account slots after a refusal; do not add new logins, IP rotation, network-route
|
||||
failover, unbounded retries, or cross-session result merging. Never commit the local
|
||||
manifest. A provider check may use the finite set of configured account slots and surfaces
|
||||
after a refusal. Do not add new logins, IP rotation, network-route failover,
|
||||
unbounded retries, or cross-session result merging. Never commit the local
|
||||
slot file.
|
||||
Every manual result stays bound to the exact sanitized upload hash and preserves the
|
||||
provider response verbatim.
|
||||
@@ -39,11 +39,14 @@ provider response verbatim.
|
||||
Provider surface selection is API-first. OpenAI and Microsoft use their official API
|
||||
adapters before their Web surfaces when the requested media and local configuration are
|
||||
supported. Google and Meta currently use Web because this tooling has no usable API route
|
||||
for them. Web remains an explicit second surface, never an automatic retry after an API
|
||||
refusal, quota response, transport error, or indeterminate verdict.
|
||||
for them. A request to use a provider covers every configured surface, including Web,
|
||||
without a second confirmation. An API refusal, quota response, transport error, or
|
||||
indeterminate verdict stays on that surface and never silently submits the file to Web.
|
||||
|
||||
Google Web oracle automation uses only the user's existing authenticated real Chrome
|
||||
session and explicit `/u/N/` account index. OpenAI, Microsoft, and Meta Web automation
|
||||
session. Gemini uses an explicit `/u/N/` account index. The SynthID Detector at
|
||||
`https://synthid.com/` (`synthid-web`) uses the same Chrome profile and does not take a
|
||||
`/u/N/` index; its terms forbid driving the page as an automated API. OpenAI, Microsoft, and Meta Web automation
|
||||
uses isolated Playwright contexts; it must never attach to or copy state from the user's
|
||||
real browser. Proxy credentials are read only through a slot's named environment variable
|
||||
and never enter a manifest or log. Dotenv interpolation may derive routed proxy URLs from
|
||||
@@ -79,9 +82,12 @@ The same rule applies to install hints: name the extra that actually makes the c
|
||||
|
||||
Run `bash maintain.sh` from the repository root. The authoritative type gate is scoped to `src/`; full-project Pyright can exhaust Node memory on the ML dependency graph.
|
||||
|
||||
The full gate outlasts the agent's 10-minute foreground shell cap, which kills it mid-suite. Run it detached under `nohup caffeinate -dimu` with its exit code written to a file and poll that file. To stop a stray run, kill its own PID, never `pkill -f pytest`, which also takes down other sessions' suites on the machine.
|
||||
|
||||
The security scanner's exit status is authoritative, including a failure after a success
|
||||
message. `tests/test_maintenance.py` checks clean, vulnerable, and broken scanner
|
||||
runs through the real shell entry point. Recheck unresolved advisories using the
|
||||
runs through the real shell entry point, and pins the CI `security` job to the
|
||||
same scanner line with no `continue-on-error`. Recheck unresolved advisories using the
|
||||
procedure in `docs/development.md`; do not revive the old text-matching bypass.
|
||||
|
||||
Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: provider-oracles
|
||||
description: Run hash-bound Google, OpenAI, Microsoft, or Meta watermark-oracle checks for this repository. Use for provider verification, multi-account Gemini checks, isolated Playwright checks, or recording oracle evidence.
|
||||
metadata:
|
||||
internal: true
|
||||
---
|
||||
|
||||
# Provider oracles
|
||||
@@ -29,22 +31,24 @@ request permission for each account or file already covered by the task.
|
||||
Submit each frozen artifact at most once per slot in that pass. Preserve
|
||||
inconclusive results and disagreements between accounts. Stop when the bounded
|
||||
plan is complete or no authorized usable slot remains; report remaining rows.
|
||||
New logins, another provider, a new network route, or another pass after the plan
|
||||
is exhausted require authorization unless already requested. Account
|
||||
New logins, another provider, or a new network route require authorization.
|
||||
The oracles and slots already configured for the requested provider may be used
|
||||
again, including their Web surfaces, without a second confirmation. Account
|
||||
continuation does not authorize changing candidate pixels or erasing failures.
|
||||
|
||||
## Default surface priority
|
||||
|
||||
Prefer an official API adapter whenever that provider, media type, and configured
|
||||
slot support it. Use Web only when no usable API route exists or the operator
|
||||
explicitly requests the Web surface as a second check. For OpenAI and Microsoft,
|
||||
the default plans are API then Web. Google and Meta currently have only usable
|
||||
Web routes in this tooling. Inspect the order with `plan PROVIDER`.
|
||||
slot support it. For OpenAI and Microsoft, the default plans are API then Web.
|
||||
Google and Meta currently have only usable Web routes in this tooling. Google's
|
||||
routes are `gemini-web` and `synthid-web`. Inspect the order with `plan PROVIDER`.
|
||||
The same request may use every configured surface.
|
||||
|
||||
The priority is a selection policy, not automatic failover. An API refusal,
|
||||
quota response, transport error, or indeterminate verdict must be recorded and
|
||||
reported. Do not submit the same artifact to Web unless the operator separately
|
||||
authorizes that next surface.
|
||||
reported. The same request already covers every configured surface of that
|
||||
provider, including Web. Run Web as its own acknowledged check and keep its
|
||||
verdict separate. Do not replace the API result with the Web result.
|
||||
|
||||
## Route by surface
|
||||
|
||||
@@ -84,6 +88,27 @@ record that it is unavailable and continue with another authorized existing
|
||||
session. Ask the user only when no usable authorized session remains. Do not
|
||||
start a new login.
|
||||
|
||||
### SynthID Detector
|
||||
|
||||
Use the user's real authenticated Chrome. Do not use isolated Playwright: the
|
||||
portal's additional terms forbid using the page as an automated API, bulk
|
||||
verification, and quota evasion.
|
||||
|
||||
1. Validate a `synthid-web` slot. It names `browser_profile` and `account_label`.
|
||||
It does not use `google_account_index` or a proxy variable.
|
||||
2. Prepare an immutable batch outside the repository. Inspect images the same
|
||||
way as for Gemini, and do not submit a file that still shows a provider logo.
|
||||
3. Open `https://synthid.com/` in the named Chrome profile. Accept the terms
|
||||
dialog when it is shown. Do not start a new login. If the page requires
|
||||
sign-in and that profile has no session, leave the row unsubmitted.
|
||||
4. Attach one prepared upload through the page's own file control.
|
||||
5. Read the settled label. `Watermark detected` is `detected`. `Watermark not
|
||||
detected` is `not_detected`. Quota, sign-in, and unclear wording stay
|
||||
`refused` or `indeterminate`.
|
||||
6. Record the verbatim response. The result covers partner SynthID from Google
|
||||
DeepMind, Nvidia, OpenAI, and Kakao, and it does not name which partner. Do
|
||||
not record it as a Gemini-only or OpenAI-only verdict.
|
||||
|
||||
### OpenAI, Microsoft, and Meta Web
|
||||
|
||||
Use the isolated Playwright runner, never the user's real Chrome:
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: signal-discovery
|
||||
description: Discover missing AI provenance and watermark coverage in the remove-ai-watermarks library from fresh or historical corpora, then validate detection and removal parity. Use for corpus gap mining, periodic signal audits, new-provider discovery, or deciding whether old retained data must be replayed. Do not use for ordinary per-file identification or removal.
|
||||
metadata:
|
||||
internal: true
|
||||
---
|
||||
|
||||
# Signal discovery
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"name": "remove-ai-watermarks",
|
||||
"version": "1.0.12",
|
||||
"description": "Identify and remove AI provenance marks from images and video the user generated or edited, using the remove-ai-watermarks CLI.",
|
||||
"author": {
|
||||
"name": "wiltodelta",
|
||||
"url": "https://github.com/wiltodelta"
|
||||
},
|
||||
"homepage": "https://raiw.cc",
|
||||
"repository": "https://github.com/wiltodelta/remove-ai-watermarks",
|
||||
"license": "Apache-2.0",
|
||||
"keywords": [
|
||||
"ai-watermark",
|
||||
"watermark-removal",
|
||||
"c2pa",
|
||||
"synthid",
|
||||
"provenance",
|
||||
"gemini",
|
||||
"sora"
|
||||
],
|
||||
"skills": "./skills/",
|
||||
"interface": {
|
||||
"displayName": "Remove AI Watermarks",
|
||||
"shortDescription": "Identify and remove AI watermarks and provenance metadata",
|
||||
"longDescription": "Runs the open-source remove-ai-watermarks CLI locally to identify C2PA, IPTC, EXIF and TC260 provenance, visible AI-generator labels and supported invisible watermarks, then removes them from images and video the user generated or edited. It does not remove stock, agency or other third-party marks.",
|
||||
"developerName": "wiltodelta",
|
||||
"category": "Productivity",
|
||||
"capabilities": ["Read", "Write"],
|
||||
"websiteURL": "https://github.com/wiltodelta/remove-ai-watermarks",
|
||||
"defaultPrompt": [
|
||||
"Check this image for AI watermarks and provenance metadata",
|
||||
"Remove the AI metadata from my generated video without re-encoding it",
|
||||
"Remove the visible AI label from the corner of my image"
|
||||
],
|
||||
"brandColor": "#A78BFA",
|
||||
"composerIcon": "./assets/icon.svg",
|
||||
"logo": "./assets/icon.svg"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
# Local state and secrets never belong in a plugin bundle.
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
.oracle-slots.json
|
||||
.local-eval/
|
||||
.venv/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
# Generated media and model artifacts.
|
||||
data/
|
||||
*.mp4
|
||||
*.onnx
|
||||
*.safetensors
|
||||
*.pt
|
||||
@@ -62,7 +62,7 @@ jobs:
|
||||
PYTHON
|
||||
|
||||
- name: Upload distributed release identity
|
||||
uses: actions/upload-artifact@v7
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
with:
|
||||
name: distributed-release-${{ github.run_attempt }}
|
||||
path: release-identity.json
|
||||
@@ -104,7 +104,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout the tap
|
||||
uses: actions/checkout@v7
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
repository: wiltodelta/homebrew-tap
|
||||
token: ${{ secrets.HOMEBREW_TAP_TOKEN }}
|
||||
@@ -227,7 +227,7 @@ jobs:
|
||||
needs: resolve
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- name: Publish the agent skill when its version moved
|
||||
env:
|
||||
CLAWHUB_TOKEN: ${{ secrets.CLAWHUB_TOKEN }}
|
||||
|
||||
@@ -21,8 +21,8 @@ jobs:
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Download derivative weights
|
||||
|
||||
@@ -36,9 +36,9 @@ jobs:
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
|
||||
@@ -22,9 +22,9 @@ jobs:
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
|
||||
@@ -15,9 +15,9 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
environment: pypi
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
|
||||
- name: Verify release tag matches package version
|
||||
run: |
|
||||
|
||||
@@ -32,8 +32,8 @@ jobs:
|
||||
skill:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
- name: Sync dev environment
|
||||
run: uv sync --frozen --extra dev
|
||||
- name: Agent skill packaging tests
|
||||
|
||||
@@ -16,8 +16,8 @@ jobs:
|
||||
lint:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
- name: Sync dev environment
|
||||
run: uv sync --frozen --extra dev
|
||||
- name: Check C2PA soft-binding registry snapshot
|
||||
@@ -29,6 +29,19 @@ jobs:
|
||||
- name: Ruff format check
|
||||
run: uv run ruff format --check
|
||||
|
||||
security:
|
||||
name: lockfile vulnerability scan
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
with:
|
||||
enable-cache: false
|
||||
# Policy and triaged blocks: docs/development.md "Known security-gate blocks".
|
||||
- name: Scan uv.lock with uv-secure
|
||||
run: uvx uv-secure uv.lock
|
||||
|
||||
test:
|
||||
name: test ${{ matrix.os }} py${{ matrix.python-version }}
|
||||
runs-on: ${{ matrix.os }}
|
||||
@@ -56,8 +69,8 @@ jobs:
|
||||
- os: windows-latest
|
||||
python-version: "3.14"
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
- name: Sync dev environment
|
||||
@@ -69,8 +82,8 @@ jobs:
|
||||
name: versioned transport contracts py3.14
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
with:
|
||||
python-version: "3.14"
|
||||
- name: Create clean runtime contract environment
|
||||
@@ -91,8 +104,8 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Install ffmpeg
|
||||
|
||||
@@ -38,7 +38,7 @@ jobs:
|
||||
steps:
|
||||
- name: Download distributed release identity
|
||||
if: ${{ github.event_name == 'workflow_run' }}
|
||||
uses: actions/download-artifact@v8
|
||||
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
|
||||
with:
|
||||
pattern: distributed-release-*
|
||||
path: distribution-identities
|
||||
|
||||
@@ -40,6 +40,8 @@ Before a release, read [`docs/release-and-distribution.md`](docs/release-and-dis
|
||||
|
||||
The published Agent Skill is [`skills/remove-ai-watermarks/`](skills/remove-ai-watermarks/). Update it in the same change as any CLI edit that alters commands, extras, exit codes, mark keys, or the intended-use boundary. Install and publish steps: [`docs/agent-skill.md`](docs/agent-skill.md).
|
||||
|
||||
The same skill tree is the Codex plugin (`.codex-plugin/plugin.json`). `remove-ai-watermarks mcp` is a standard-library MCP server that runs one CLI command per tool call; keep it dependency-free, because the MCP registry launches it as a bare `uvx remove-ai-watermarks mcp` that cannot add extras ([`docs/mcp-server.md`](docs/mcp-server.md)).
|
||||
|
||||
## Module architecture
|
||||
|
||||
[`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem.
|
||||
|
||||
@@ -6,6 +6,15 @@ Remove AI provenance marks from images and video you generated yourself:
|
||||
- invisible pixel watermarks through direct local-format disruption or diffusion regeneration;
|
||||
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
|
||||
|
||||
| Before | After |
|
||||
| --- | --- |
|
||||
|  |  |
|
||||
|
||||
```bash
|
||||
uv tool install "remove-ai-watermarks[visible]" # CLI for visible marks and metadata
|
||||
npx skills add wiltodelta/remove-ai-watermarks # agent skill for Claude Code, Codex, Cursor, and other agents
|
||||
```
|
||||
|
||||
Video support covers provenance identification, complete visible-plus-metadata
|
||||
cleaning, directory batches, visible Sora, Veo, Seedance, Doubao, Dola,
|
||||
Hailuo AI, and Kling AI mark removal, and oracle-certified VAE regeneration for video SynthID
|
||||
@@ -274,11 +283,8 @@ features, and development setup.
|
||||
|
||||
### Visible Gemini mark
|
||||
|
||||
| Before | After |
|
||||
| --- | --- |
|
||||
|  |  |
|
||||
|
||||
The `after` raster is generated from the tracked `before` raster by the public path:
|
||||
The `after` raster at the top of this page is generated from the tracked `before`
|
||||
raster by the public path:
|
||||
|
||||
```bash
|
||||
uv run remove-ai-watermarks visible demo_banana_before.png \
|
||||
@@ -370,11 +376,11 @@ remove-ai-watermarks batch ./images --mode all
|
||||
Visible mark support includes:
|
||||
|
||||
- Google Gemini and Nano Banana visible sparkle watermark;
|
||||
- Doubao, the Jimeng wordmark and top-left `AI生成` pill, Qwen, Kling AI,
|
||||
Yuanbao, Baidu, LiblibAI's bottom-center wordmark and compact top-left pill,
|
||||
and RunningHub labels;
|
||||
- one calibrated Microsoft top-right white AI-badge variant;
|
||||
- one calibrated Samsung Galaxy AI label variant;
|
||||
- Doubao, the Jimeng wordmark and top-left `AI生成` layouts, Qwen, Kling AI,
|
||||
Yuanbao, Dola, WorkBuddy, Baidu, LiblibAI's bottom-center wordmark and compact
|
||||
top-left pill, and RunningHub labels;
|
||||
- calibrated Microsoft top-right white and beige AI-badge layouts;
|
||||
- calibrated Italian and Korean Samsung Galaxy AI disclosures;
|
||||
- a generic brand-less bottom-right `AI生成` fallback for OS/gallery-level
|
||||
AI-edit stamps that carry no vendor wordmark of their own (vivo and Xiaomi
|
||||
Gallery confirmed); synthetic-only calibration, weaker evidence than the
|
||||
@@ -476,9 +482,23 @@ Claude Code can add this repository as a plugin marketplace:
|
||||
/plugin install remove-ai-watermarks@remove-ai-watermarks
|
||||
```
|
||||
|
||||
Codex reads the same skill through [`.codex-plugin/plugin.json`](.codex-plugin/plugin.json).
|
||||
The skill refuses stock-agency and other third-party paid-asset marks. See
|
||||
[the agent skill guide](docs/agent-skill.md).
|
||||
|
||||
## MCP server
|
||||
|
||||
`remove-ai-watermarks mcp` serves `identify`, `remove` and `batch` as Model
|
||||
Context Protocol tools over stdio, each a thin wrapper around one CLI command:
|
||||
|
||||
```json
|
||||
{"mcpServers": {"remove-ai-watermarks": {"command": "uvx", "args": ["remove-ai-watermarks", "mcp"]}}}
|
||||
```
|
||||
|
||||
See [the MCP server guide](docs/mcp-server.md).
|
||||
|
||||
<!-- mcp-name: io.github.wiltodelta/remove-ai-watermarks -->
|
||||
|
||||
## ComfyUI
|
||||
|
||||
The separate
|
||||
@@ -490,6 +510,8 @@ invisible removal.
|
||||
|
||||
- A missing local signal means unknown, not clean. Proprietary pixel
|
||||
watermarks may remain after metadata has been stripped.
|
||||
[What each path leaves behind](docs/known-limitations.md#what-each-path-leaves-behind)
|
||||
lists, per command, what may remain and how to check it.
|
||||
- Visible removal reconstructs a small region. Results depend on the background
|
||||
and selected fill backend.
|
||||
- Invisible removal changes the whole image and may alter faces, text, or fine
|
||||
@@ -571,7 +593,8 @@ bash maintain.sh
|
||||
```
|
||||
|
||||
See [module internals](docs/module-internals.md) before changing a subsystem
|
||||
with documented invariants.
|
||||
with documented invariants. Report vulnerabilities as described in
|
||||
[SECURITY.md](SECURITY.md).
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
# Security policy
|
||||
|
||||
## Supported versions
|
||||
|
||||
Security fixes go into the latest release. Older releases are not patched;
|
||||
upgrade through the channel you installed from (PyPI or Homebrew).
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Report vulnerabilities privately through
|
||||
[GitHub private vulnerability reporting](https://github.com/wiltodelta/remove-ai-watermarks/security/advisories/new).
|
||||
Do not open a public issue for an unpatched vulnerability. Include the affected
|
||||
version, the command or API call, and a minimal input that reproduces it.
|
||||
|
||||
Vulnerabilities in a dependency, such as a CVE in a package from `uv.lock`, can
|
||||
be reported as a regular issue when the advisory is already public.
|
||||
|
||||
## Dependency vulnerabilities
|
||||
|
||||
Dependabot raises alerts and update pull requests for `uv.lock` and GitHub
|
||||
Actions, and CI scans `uv.lock` with
|
||||
[`uv-secure`](https://github.com/owenlamont/uv-secure) on every push and pull
|
||||
request. A vulnerable dependency is upgraded to a fixed release; advisories
|
||||
without one are tracked under
|
||||
[Known security-gate blocks](docs/development.md#known-security-gate-blocks).
|
||||
@@ -0,0 +1,20 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="512" height="512" fill="none" stroke="url(#lg)" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
||||
<!--
|
||||
The raiw.cc brand mark: the Lucide "sparkle" and "x" glyphs.
|
||||
Lucide, https://lucide.dev, ISC License, Copyright (c) Lucide Contributors.
|
||||
Permission to use, copy, modify, and/or distribute this software for any purpose
|
||||
with or without fee is hereby granted, provided that the above copyright notice
|
||||
and this permission notice appear in all copies.
|
||||
-->
|
||||
<path d="M11.017 2.814a1 1 0 0 1 1.966 0l1.051 5.558a2 2 0 0 0 1.594 1.594l5.558 1.051a1 1 0 0 1 0 1.966l-5.558 1.051a2 2 0 0 0-1.594 1.594l-1.051 5.558a1 1 0 0 1-1.966 0l-1.051-5.558a2 2 0 0 0-1.594-1.594l-5.558-1.051a1 1 0 0 1 0-1.966l5.558-1.051a2 2 0 0 0 1.594-1.594z" fill="url(#lg)" fill-opacity="0.15"/>
|
||||
<g transform="translate(1 17) scale(0.3)" stroke-width="5">
|
||||
<path d="M18 6 6 18"/>
|
||||
<path d="m6 6 12 12"/>
|
||||
</g>
|
||||
<defs>
|
||||
<linearGradient id="lg" x1="0" y1="0" x2="24" y2="24">
|
||||
<stop stop-color="#a78bfa"/>
|
||||
<stop offset="1" stop-color="#06b6d4"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.2 KiB |
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 2.0 MiB |
@@ -6,9 +6,9 @@ qwen,Alibaba Qwen consumer app,visible text,data/fixtures/visible/qwen/provider-
|
||||
kling,Kuaishou Kling AI,visible text,data/fixtures/visible/kling/provider-original-direct.png,available,authenticated Kling IMAGE 3.0 original; Latin KlingAI 3.0 mark detected at confidence 0.43; TC260 producer maps to Kuaishou,subject to Kling AI terms; noncommercial fixture
|
||||
yuanbao,Tencent Yuanbao,visible two-line text,data/fixtures/visible/yuanbao/provider-original.png,available,authenticated Yuanbao original; detected by Yuanbao engine at confidence 0.49; TC260 producer metadata maps to Tencent,repository
|
||||
samsung,Samsung Galaxy AI,visible Italian text,data/calibration/samsung/samsung_gray_1.png,available,original provider capture and detector calibration,repository
|
||||
runninghub,RunningHub,visible text,,missing,no redistributable provider original found,
|
||||
runninghub,RunningHub,visible text,data/fixtures/visible/runninghub/example.png,partial,synthetic detector fixture available; no redistributable provider original found,repository
|
||||
baidu,Baidu,visible text and tag,data/fixtures/visible/baidu/provider-original.jpeg,available,detected by Baidu engine; distributed under the OpenNoMark repository's CC BY-NC-SA 4.0 license,CC-BY-NC-SA-4.0; excluded from Apache-2.0
|
||||
liblib,LiblibAI,visible wordmark,data/fixtures/visible/liblib/current-provider-output.png,partial,"authenticated current LiblibAI original carries a top-left AI生成 pill rather than the registered historical bottom-center LiblibAI wordmark; TC260 producer metadata maps to LiblibAI; historical positive remains missing",repository
|
||||
liblib,LiblibAI,visible wordmark,data/fixtures/visible/liblib/current-provider-output.png,partial,"authenticated current LiblibAI original carries a top-left AI生成 pill rather than the registered historical bottom-center LiblibAI wordmark; TC260 producer metadata maps to LiblibAI; a synthetic historical-wordmark fixture is available at data/fixtures/visible/liblib/example.png, but a historical provider positive remains missing",repository
|
||||
microsoft,Microsoft image output,visible top-right AI badge,data/fixtures/visible/microsoft/provider-original.png,available,authenticated Copilot original detected by Microsoft engine at confidence 0.54; also has valid C2PA and InvisMark,repository
|
||||
jimeng_pill,ByteDance Jimeng,visible AI pill,data/fixtures/visible/jimeng_pill/provider-published-example.jpg,available,"real Jimeng output embedded in page 29 of Zhonglun W&D's 2026 Bank Finance Legal Newsletter; direct jimeng_pill detection confidence 0.28; the same image also carries the Jimeng wordmark",fair-use research fixture; excluded from Apache-2.0
|
||||
google_synthid,Google,image SynthID,data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png,available,provider oracle recorded in data/synthid/manifest.csv,repository
|
||||
|
||||
|
@@ -1,8 +1,8 @@
|
||||
key,provider,watermark_type,path,status,verification,license
|
||||
sora,OpenAI Sora,visible moving mascot and wordmark,data/fixtures/visible/sora/provider-original.mp4,available,"Apache-2.0 upstream original; detected by Sora engine on 300/300 frames; C2PA claim names Sora and OpenAI/Truepic, with valid integrity and signature but an expired untrusted signer certificate",Apache-2.0
|
||||
veo,Google Veo,visible fixed text,data/fixtures/visible/veo/provider-original.mp4,available,"authenticated Google Flow Veo 3.1 Fast original; detected by Veo engine on 192/192 frames; valid C2PA and provider-confirmed SynthID",repository
|
||||
seedance,ByteDance Seedance,visible boxed AI label,,missing,"OpenRouter Seedance 2.0 Mini with watermark=true produced BytePlus C2PA but no visible boxed AI label; direct consumer original still needed",
|
||||
doubao,ByteDance Doubao,visible fixed text,,missing,no redistributable provider original found,
|
||||
dola,ByteDance Dola,visible fixed text,,missing,no redistributable provider original found,
|
||||
seedance,ByteDance Seedance,visible boxed AI label,data/fixtures/visible/seedance/example.mp4,partial,"synthetic detector fixture available; OpenRouter Seedance 2.0 Mini with watermark=true produced BytePlus C2PA but no visible boxed AI label; direct consumer original still needed",repository
|
||||
doubao,ByteDance Doubao,visible fixed text,data/fixtures/visible/doubao/example.mp4,partial,"synthetic detector fixture available; a real public carrier is retained locally but is not publication-cleared",repository
|
||||
dola,ByteDance Dola,visible fixed text,data/fixtures/visible/dola/example.mp4,partial,synthetic detector fixture available; no redistributable provider original found,repository
|
||||
hailuo,MiniMax Hailuo AI,visible composite label,data/fixtures/visible/hailuo/provider-original.mp4,available,"authenticated Hailuo AI MiniMax H3 original downloaded with With Watermark selected; detected by Hailuo engine on 124/124 frames; TC260 AIGC label names MiniMax",repository
|
||||
kling,Kuaishou Kling AI,visible fixed logo and wordmark,data/fixtures/visible/kling/provider-original.mp4,available,"authenticated Kling AI VIDEO 3.0 original; detected by Kling engine on 121/121 frames; no locally readable provenance metadata",Kling AI terms; noncommercial fixture
|
||||
kling,Kuaishou Kling AI,visible fixed logo and wordmark,data/fixtures/visible/kling/provider-original.mp4,available,"authenticated Kling AI VIDEO 3.0 original; detected by Kling engine on 121/121 frames; no container provenance metadata, but the Kling H.264 SEI tag (kling_sei) is readable",Kling AI terms; noncommercial fixture
|
||||
|
@@ -26,6 +26,11 @@ SkillsMP indexes public GitHub `SKILL.md` files and does not need a separate
|
||||
upload. ClawHub and the Claude community marketplace are explicit publishes;
|
||||
see [Release and distribution](release-and-distribution.md#agent-skill).
|
||||
|
||||
Every maintainer skill under `.claude/skills/` carries `metadata.internal: true`.
|
||||
The skills CLI discovers every `SKILL.md` in that directory, so without the flag
|
||||
`npx skills add` offers it and skills.sh lists it beside the published skill.
|
||||
`tests/test_agent_skill.py` fails on a project skill that lacks it.
|
||||
|
||||
## Where the skill can actually be listed
|
||||
|
||||
Most skill catalogs are install registries, not discovery engines. A listing
|
||||
@@ -65,6 +70,17 @@ Do these in order after the skill is on the default branch.
|
||||
are indexed via clawskills.sh, and the ClawHub prerequisite in step 6 is
|
||||
now met. The list still requires proven real-world usage. Re-attempt once
|
||||
the ClawHub listing shows organic installs.
|
||||
- `hashgraph-online/awesome-codex-plugins`: READY, not submitted. It lists
|
||||
only Codex plugins, so it needs `.codex-plugin/plugin.json` and the icon in
|
||||
`assets/icon.svg`. Its contribution CI runs `scripts/validate-plugin-pr.py`
|
||||
against the repository's `HEAD.zip`, then the HOL plugin scanner with a
|
||||
minimum score of 80 and no high finding; that scan job is
|
||||
`continue-on-error`, so it reports rather than blocks. The scanner's `\beval\s*\(`
|
||||
rule also matches PyTorch's `.eval()`, which is why the code calls
|
||||
`.train(False)`. On 2026-10-17 `plugin-scanner` 3.49.0 scored the tree 86
|
||||
with no high finding. The remaining loss is a tracked video over the
|
||||
scanner's 16 MiB file budget. Recheck with
|
||||
`uvx --from plugin-scanner==3.49.0 plugin-scanner scan . --min-score 80 --fail-on-severity high`.
|
||||
5. **Claude community marketplace.** Form at
|
||||
[platform.claude.com/plugins/submit](https://platform.claude.com/plugins/submit)
|
||||
(individuals) or the claude.ai directory form (Team/Enterprise). Validate
|
||||
@@ -186,8 +202,8 @@ that names a command, flag or value the previous release lacked. Left behind, it
|
||||
CLI is current right before it types an option that build rejects.
|
||||
|
||||
Bump `metadata.version` in `SKILL.md` plus `version` in
|
||||
`skills/.claude-plugin/plugin.json` and `.claude-plugin/marketplace.json` when
|
||||
the skill instructions change. That version is independent of the PyPI package
|
||||
`skills/.claude-plugin/plugin.json`, `.claude-plugin/marketplace.json` and
|
||||
`.codex-plugin/plugin.json` when the skill instructions change. That version is independent of the PyPI package
|
||||
version.
|
||||
|
||||
## Behavior evals
|
||||
|
||||
@@ -1597,6 +1597,38 @@ Local reproducibility artifacts are under
|
||||
`contamination-scan-2026-08-26`, and `taxonomy-continuation-clean-2026-08-26`
|
||||
run directories.
|
||||
|
||||
## Modern frozen-backbone campaign closed, 2026-09-27
|
||||
|
||||
A second pre-registered representation campaign tested exact-revision,
|
||||
Apache-2.0 frozen vision backbones with universal ridge heads against the same
|
||||
product contract. The candidates were `google/siglip2-base-patch16-224` at
|
||||
revision `75de2d55ec2d0b4efc50b3e9ad70dba96a7b2fa2` and
|
||||
`facebook/PE-Core-B16-224` at revision
|
||||
`a16450b46fef32363459920c2685a1b4ef13dcd9`. The protocol was frozen before
|
||||
inference and used 28,743 unique source images. Its comparison cells contained
|
||||
1,873 AI-test images, 3,000 fresh Open Images photographs, 24 Kodak images, and
|
||||
300 FLUX images.
|
||||
|
||||
| Model at its frozen operating point | AI test | Fresh photo false positives | Kodak false positives | FLUX |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Production Model 1 baseline | 1,676/1,873 (89.5%) | 35/3,000 (1.17%) | 0/24 | 256/300 (85.3%) |
|
||||
| SigLIP2 universal ridge | 1,602/1,873 (85.5%) | 74/3,000 (2.47%) | 0/24 | 125/300 (41.7%) |
|
||||
| PE-Core B universal ridge | 1,715/1,873 (91.6%) | 86/3,000 (2.87%) | 0/24 | 170/300 (56.7%) |
|
||||
|
||||
SigLIP2 failed the AI-test, fresh-photo, and FLUX gates. PE-Core improved the
|
||||
AI-test cell but failed the fresh-photo and FLUX gates. Matching the baseline's
|
||||
exact 35-false-positive photo budget after the fact did not recover the lost
|
||||
coverage: SigLIP2 reached 1,505/1,873 AI and 98/300 FLUX, PE-Core universal
|
||||
reached 1,625/1,873 and 133/300, and PE-Core photo-only reached 1,709/1,873 and
|
||||
97/300. No threshold matched the baseline photo budget, AI coverage, and FLUX
|
||||
coverage simultaneously.
|
||||
|
||||
The bounded newer-backbone hypothesis is therefore closed as a Model 1
|
||||
replacement. Keep `clip-l-ft.pt`, its ONNX graph, hashes, and Hub revision
|
||||
unchanged. Reproducibility artifacts are local under
|
||||
`.local-eval/synthid/ai-photo-2026-08-22/modern-representation-campaign-2026-09-27/`;
|
||||
the campaign did not enter the publication stage.
|
||||
|
||||
### Wild extras, not SynthID
|
||||
|
||||
| Hypothesis | 2026-08-23 | Use |
|
||||
|
||||
@@ -28,6 +28,7 @@ defaults. This page focuses on choosing the right command.
|
||||
| `video invisible` and `video all --invisible` | `remove-ai-watermarks[video,diffusion]` plus ffmpeg on PATH |
|
||||
| HEIC/HEIF/AVIF pixel input | Add `remove-ai-watermarks[heif]` |
|
||||
| Every production command and Python-compatible backend | `remove-ai-watermarks[all]` |
|
||||
| `mcp` | Default package; each tool needs the extra of the command it runs |
|
||||
|
||||
`batch` requires the same extra as its selected mode. Extras can be combined in
|
||||
one installation, for example `remove-ai-watermarks[visible,detect,heif]`.
|
||||
@@ -469,6 +470,11 @@ detected. Use `--force` when you know the image should be processed:
|
||||
remove-ai-watermarks invisible image.png -o clean.png --force
|
||||
```
|
||||
|
||||
For transparent inputs, use PNG or WebP output to retain transparency. The engine
|
||||
preserves the original alpha channel, including partially transparent edges,
|
||||
while regenerating RGB pixels. JPEG output cannot preserve transparency. This
|
||||
also applies to `batch --mode invisible`.
|
||||
|
||||
### Choose a strength cohort
|
||||
|
||||
`--vendor` selects the cohort the default strength resolves from. `auto`
|
||||
@@ -685,6 +691,16 @@ For `visible` and `all` modes, the final summary includes counts for
|
||||
validation results are printed explicitly, but the once-filled outputs are kept
|
||||
and the batch does not retry them automatically.
|
||||
|
||||
## Serve the CLI to agent clients
|
||||
|
||||
```bash
|
||||
remove-ai-watermarks mcp
|
||||
```
|
||||
|
||||
Runs a Model Context Protocol server on stdin and stdout whose `identify`,
|
||||
`remove` and `batch` tools each run one command of this CLI. Client setup and
|
||||
the tool contract are in the [MCP server guide](mcp-server.md).
|
||||
|
||||
## Exit behavior
|
||||
|
||||
The CLI uses nonzero exit codes for meaningful incomplete outcomes, including
|
||||
|
||||
@@ -29,7 +29,7 @@ Do not add advisory ignores or suppress the scanner's exit status. Run and
|
||||
report Ruff, Pyright scoped to `src/`, and tests separately when security blocks
|
||||
the gate.
|
||||
|
||||
Rechecked on 2026-09-10 against the locked versions and the advisory sources:
|
||||
Rechecked on 2026-10-02 against the locked versions and the advisory sources:
|
||||
|
||||
- `lightning` and `pytorch-lightning`, pulled by the optional `trustmark`
|
||||
extra, are pinned to 2.6.6. That release clears PYSEC-2026-3624 and
|
||||
@@ -37,18 +37,19 @@ Rechecked on 2026-09-10 against the locked versions and the advisory sources:
|
||||
- `accelerate`, required by the diffusion stack, is affected by
|
||||
[GHSA-4j2p-28q2-5m79](https://github.com/advisories/GHSA-4j2p-28q2-5m79)
|
||||
and PYSEC-2026-3804. The sharded-checkpoint loader accepts unsafe
|
||||
`weight_map` paths; neither advisory lists a patched release. 1.15.0 is
|
||||
on PyPI and is outside the advisory's `<= 1.14.0` range, but its
|
||||
`weight_map` paths; neither advisory lists a patched release. The lock
|
||||
carries 1.15.0, which is outside the advisory's `<= 1.14.0` range, so the
|
||||
scanner no longer blocks the gate. Its
|
||||
`load_checkpoint_in_model` still joins `weight_map` entries with no
|
||||
containment check, and the sanitizer PRs
|
||||
([#4138](https://github.com/huggingface/accelerate/pull/4138),
|
||||
[#4214](https://github.com/huggingface/accelerate/pull/4214)) did not
|
||||
land. Do not bump solely to silence the scanner. The project has no
|
||||
[#4214](https://github.com/huggingface/accelerate/pull/4214)) were closed
|
||||
without merging. Do not treat the passing scan as a fix. The project has no
|
||||
direct call to `load_checkpoint_in_model` or
|
||||
`load_checkpoint_and_dispatch`, but this does not prove indirect
|
||||
model-loading paths are unaffected.
|
||||
|
||||
Before carrying the remaining block forward, rerun `uvx uv-secure uv.lock` and check
|
||||
When a new block appears, or before carrying one forward, rerun `uvx uv-secure uv.lock` and check
|
||||
PyPI for a fixed release. Upgrade a fixed package with
|
||||
`uv lock --upgrade-package <package>` and rerun the gate. Removing the current
|
||||
dependency would remove a supported optional feature, so that is not a
|
||||
@@ -98,6 +99,11 @@ run the full-clip video test. Diffusion and model-running tests skip in that
|
||||
matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the
|
||||
OpenCV eraser remain covered across operating systems.
|
||||
|
||||
A separate `security` job runs the `maintain.sh` lockfile scan, so an advisory
|
||||
without a released fix turns `main` red; see
|
||||
[Known security-gate blocks](#known-security-gate-blocks). Vulnerability
|
||||
reporting is described in [`SECURITY.md`](../SECURITY.md).
|
||||
|
||||
Keep `uv.lock` compatible with `uv sync --frozen`. Dependency pull-request checks use GitHub's merge result against current `main`; if `main` moves, merge it locally and rerun the full gate because a newer linter can expose stale directives in later code.
|
||||
|
||||
`test.yml` runs one concurrency group per ref with `cancel-in-progress: true`, so a newer push cancels the previous run mid-flight. A cancelled run is not a failure verdict: the superseding run's head includes the cancelled commits as ancestors, so watch that run to completion instead of retrying the push.
|
||||
|
||||
@@ -27,7 +27,8 @@ to run the tool. Use the maintainer references only when changing the code.
|
||||
| [Code provenance](code-provenance.md) | Required notices for licensed derivative work. |
|
||||
| [Verification plan](verification-plan.md) | Verification methods, completed measurements, and remaining validation gaps. |
|
||||
| [Release and distribution](release-and-distribution.md) | PyPI, Homebrew, Hugging Face Space, photo-classify Hub model, ComfyUI Registry, and the release workflow. |
|
||||
| [Agent skill](agent-skill.md) | Install and maintain the published Agent Skill and Claude plugin. |
|
||||
| [Agent skill](agent-skill.md) | Install and maintain the published Agent Skill, the Claude plugin and the Codex plugin. |
|
||||
| [MCP server](mcp-server.md) | Connect an MCP client to `remove-ai-watermarks mcp`, or publish it to the MCP registry. |
|
||||
| [Watermarking landscape](watermarking-landscape.md) | Vendor signals and detection approaches. |
|
||||
| [SynthID technical reference](synthid.md) | Mechanism, provenance, robustness, regeneration. |
|
||||
|
||||
|
||||
@@ -4,6 +4,21 @@ This page describes current product limits. Historical measurements and
|
||||
superseded experiments live in the research archive listed in
|
||||
[the documentation index](index.md).
|
||||
|
||||
## What each path leaves behind
|
||||
|
||||
No command produces a "clean" verdict. Each one removes a specific layer, and
|
||||
the remaining layers need their own check.
|
||||
|
||||
| Path | Removes | May remain | How to check what remains |
|
||||
| --- | --- | --- | --- |
|
||||
| `metadata --remove`, `video metadata --remove` | Supported AI metadata: C2PA manifests, TC260 labels, IPTC AI fields, app-export JSON, generation parameters | Every pixel and audio watermark (SynthID, Meta Content Seal, InvisMark, TrustMark, DWT-DCT). A C2PA soft binding can still lead a resolver to a manifest stored remotely. Strings inside encoded video streams, such as H.264 SEI messages, stay too, except Kling's SEI tag, which `video metadata --remove` blanks | `identify` re-reads the metadata. Only the provider's verifier can confirm a pixel watermark. The pixels are not regenerated, so an AI-image classifier scores the output the same as the input |
|
||||
| `visible`, `erase`, `video visible` | The selected label region, refilled. AI metadata too, unless `--keep-metadata` is given | Any invisible watermark across the rest of the frame. The fill can leave a visible patch on structured backgrounds | Look at the output. Use `identify` for metadata and the provider's verifier for invisible marks |
|
||||
| `invisible`, `all`, `video invisible` | Regenerates pixels to disrupt invisible watermarks. `all` also removes visible marks and metadata | No local decoder confirms that SynthID is gone from a given file, and the outcome depends on content and seed. `video invisible` copies the audio track unchanged, so an audio watermark survives | The provider's verifier, such as Google's SynthID Detector or OpenAI's provenance check. A pass there says nothing about AI-image classifiers |
|
||||
| `identify`, `video identify` | Nothing | Not applicable | `unknown` means no supported local signal was found. It is not a clean verdict |
|
||||
|
||||
AI-image classifiers are a separate question from provenance. They score how
|
||||
the pixels look rather than read a mark, and none of these paths targets them.
|
||||
|
||||
## Pixel classification
|
||||
|
||||
`classify` / `classify_pixels` is AI versus camera, not a universal AI detector.
|
||||
@@ -179,9 +194,12 @@ fresh-font and silhouette variants visibly changed typography. The higher-fideli
|
||||
and line geometry. It builds its donor internally but still requires an independently
|
||||
clean global anchor. Automatic OCR and line-box proposals are not reliable enough to
|
||||
remove those requirements, and exact oracle results do not establish a general mask,
|
||||
seed, runtime, or provider operating range. Qwen-Image-2.0 is hosted-only and exposes
|
||||
no equivalent low-strength denoise control. Exact experiments, controls, and pass
|
||||
rates are kept in
|
||||
seed, runtime, or provider operating range. Qwen-Image-2.0 is hosted-only. Qwen-Image-2.1
|
||||
(issue #111) has open weights, but its Diffusers pipeline uses `image=` only as
|
||||
reference conditioning and starts the target latents from noise, with no `strength`
|
||||
or latent noise-mask input. Its Qwen Research License also limits the weights to
|
||||
non-commercial research and evaluation, so it is not a supported-profile candidate.
|
||||
Exact experiments, controls, and pass rates are kept in
|
||||
[`text-protection-research.md`](text-protection-research.md) and the
|
||||
[`fidelity` evaluation record](../data/evaluations/fidelity/README.md).
|
||||
|
||||
@@ -260,7 +278,9 @@ guarantee, and check the audio separately when it matters. The question is no
|
||||
longer Google-only: since 2026 OpenAI (ChatGPT and API audio) and ElevenLabs
|
||||
also embed SynthID in generated audio, and Microsoft lists provenance for its
|
||||
Azure speech and GPT audio models, so voice-over or music from those services
|
||||
muxed into any video survives this path unchanged.
|
||||
muxed into any video survives this path unchanged. Suno announced audio
|
||||
watermarking and fingerprinting on 2026-08-06 without naming the technology, so
|
||||
Suno music belongs on the same list.
|
||||
|
||||
The public video results make that boundary explicit without decoding audio:
|
||||
`visual_invisible_action` records whether video pixels were regenerated, while
|
||||
@@ -397,12 +417,20 @@ global Qwen stage is tiled; the face stage runs after tile blending.
|
||||
every profile, streaming weights instead of pinning them. It reduces CUDA memory
|
||||
pressure at the cost of speed.
|
||||
|
||||
It reaches the global stack of `qwen-zimage` only. That stack is each
|
||||
profile's own, and `chroma-zimage` and `sdxl-zimage` load theirs with a plain
|
||||
`.to(device)`. On a card below the face-stage residency floor, where the face
|
||||
stack was already offloading, the flag changes nothing at all for those two: the
|
||||
run behaves as if it were absent. The library says so on the way past, as a
|
||||
warning ahead of the model load.
|
||||
It reaches the global stack of `qwen-zimage` and `sdxl-zimage`. That stack is
|
||||
each profile's own, and `chroma-zimage` loads its stack with a plain `.to(device)`.
|
||||
On a card below the face-stage residency floor, where the face stack was already
|
||||
offloading, the flag changes nothing at all for that profile: the run behaves as
|
||||
if it were absent. The library says so on the way past, as a warning ahead of the
|
||||
model load.
|
||||
|
||||
`sdxl-zimage` streams its global stack without being asked on a card below
|
||||
`RESIDENT_SDXL_GLOBAL_MIN_VRAM_GIB` (12 GiB). Its UNet and Canny ControlNet run
|
||||
together on every step, so the stack does not fit an 8 GiB card even with
|
||||
model-level offload. A resident load there does not always fail loudly: under
|
||||
the Windows display driver, including WSL2, CUDA can spill the overflow into
|
||||
shared system memory, and the run goes on at a fraction of the speed instead of
|
||||
raising an out-of-memory error.
|
||||
|
||||
Residency is otherwise chosen from the card's total VRAM. On a card large enough
|
||||
to hold a stack, offloading is pure waste: DiffSynth drops weights to the meta
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
# MCP server
|
||||
|
||||
`remove-ai-watermarks mcp` runs a [Model Context Protocol](https://modelcontextprotocol.io/)
|
||||
server on stdin and stdout, so an agent client can identify and clean the
|
||||
user's own images and video without a shell. It is a thin wrapper: each tool
|
||||
runs one command of this CLI in a child process and returns that command's exit
|
||||
code, output path and text.
|
||||
|
||||
The subcommand ships since 0.46.0. The server uses only the Python standard
|
||||
library, so the default package runs it. The commands behind a tool need the
|
||||
same extras they need on the command line
|
||||
([CLI guide](cli.md#command-dependency-map)). A missing extra comes back as a
|
||||
tool error carrying the CLI's install hint.
|
||||
|
||||
## Client configuration
|
||||
|
||||
Most clients take a command and arguments. With [uv](https://docs.astral.sh/uv/):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"remove-ai-watermarks": {
|
||||
"command": "uvx",
|
||||
"args": ["remove-ai-watermarks", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
To make visible-mark removal and pixel detectors available, name the extras:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"remove-ai-watermarks": {
|
||||
"command": "uvx",
|
||||
"args": ["--from", "remove-ai-watermarks[visible,video]", "remove-ai-watermarks", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
An existing installation can be started directly as `remove-ai-watermarks mcp`.
|
||||
Paths in tool calls are read and written on the machine that runs the server.
|
||||
|
||||
## Tools
|
||||
|
||||
| Tool | Runs | Writes |
|
||||
| --- | --- | --- |
|
||||
| `identify` | `identify --json`, or `video identify --json` for a video suffix; `check_visible: false` adds `--no-visible` | Nothing |
|
||||
| `remove` | `metadata --remove`, `visible` or `all` for an image; `video metadata --remove`, `video visible` or `video all` for a video | One output file, `<name>_clean` beside the source unless `output` is given |
|
||||
| `batch` | `batch --mode` for images or `video batch --mode` for video | A separate output directory, `<dir>_clean` unless `output_dir` is given |
|
||||
|
||||
`mode` is `metadata` (the default), `visible` or `all`. For an image, `all`
|
||||
adds the invisible-watermark scrub where the installed extras and a CUDA GPU
|
||||
allow it; `video all` removes visible marks and metadata only. The server never
|
||||
writes over an input: an `output` equal to the source, or an `output_dir` equal
|
||||
to the input directory, is refused before anything runs.
|
||||
|
||||
Exit codes keep their CLI meaning. `0` and `2` are results, not tool errors, and
|
||||
`2` means no supported signal was found, which is unknown rather than clean.
|
||||
Click also exits `2` on a usage error such as a missing file; its `Usage:` line on
|
||||
stderr marks that case as a tool error. Every other nonzero exit code is returned
|
||||
with `isError: true` and the command's stderr.
|
||||
|
||||
## Scope
|
||||
|
||||
The server carries the project's intended-use boundary in its `instructions`:
|
||||
it is for content the user generated or edited themselves, and not for stock,
|
||||
agency, marketplace or other marks that protect a third party's asset. See
|
||||
[legal and safety](legal-and-safety.md).
|
||||
|
||||
## Protocol
|
||||
|
||||
The server answers the initialize handshake for protocol versions 2025-11-25,
|
||||
2025-06-18, 2025-03-26 and 2024-11-05, and lists its tools with input schemas
|
||||
and annotations: `identify` is read-only, and `remove` and `batch` write new
|
||||
files without destroying inputs. It does not implement `server/discover`. A
|
||||
client that probes it first, such as the official Python SDK's `Client` in its
|
||||
default mode, receives method-not-found and falls back to the handshake.
|
||||
`tests/test_mcp_server.py` runs that client against the real subcommand.
|
||||
|
||||
## Registry listing
|
||||
|
||||
[`server.json`](../server.json) describes the PyPI package for the
|
||||
[official MCP registry](https://registry.modelcontextprotocol.io/) under
|
||||
`io.github.wiltodelta/remove-ai-watermarks`. The registry verifies ownership by
|
||||
the `mcp-name:` marker in `README.md`, which becomes the PyPI description, so it
|
||||
can only verify a release whose README carries that marker.
|
||||
[`glama.json`](../glama.json) names the maintainer for Glama. Publishing steps
|
||||
are in [release and distribution](release-and-distribution.md#mcp-registry).
|
||||
@@ -47,6 +47,9 @@ Important contracts:
|
||||
`EXIT_NO_VISIBLE_MARK`.
|
||||
- `invisible` writes no output when no supported local signal is found, unless
|
||||
`--force` is supplied.
|
||||
It decodes the source before the scrub gate and exits `1` when it cannot,
|
||||
because the gate reads undecodable bytes as "no target" and would otherwise
|
||||
report the no-signal code for a file it never read (fixed 2026-10-17).
|
||||
- The two no-signal conditions currently share exit code `2`.
|
||||
- A quiet `metadata --check` and a successful `metadata --remove` (same for
|
||||
`video metadata`) end by repeating the `identify` limit: the pixel channel is
|
||||
@@ -236,10 +239,46 @@ payloads; TC260 removal changes the four-byte key to `free` and blanks only the
|
||||
validated JSON value with same-length spaces. Keyed generation metadata such
|
||||
as `workflow` and `prompt` uses the same structural walk; removal blanks the
|
||||
complete key and value in place. This preserves every box size,
|
||||
`stco`/`co64` offset, encoded stream byte, and source-sized memory bound.
|
||||
`stco`/`co64` offset, encoded stream byte apart from the Kling SEI tag below, and
|
||||
source-sized memory bound.
|
||||
Publication is atomic, and a malformed top-level walk is copied unchanged. A
|
||||
generic `AIGC` key whose value has no TC260 field is ignored.
|
||||
|
||||
App-export JSON in a keyed or keyless metadata item is blanked the same way.
|
||||
Dreamina's MP4 export writes it twice, under `LvMetaInfo` and
|
||||
`com.apple.quicktime.artwork`. The file-level scan read both from the raw head
|
||||
bytes while the stripper walked only TC260 and generation keys, so
|
||||
`video metadata --remove` exited 1 with both markers still present on the
|
||||
tracked Dreamina fixture (found 2026-10-17). Its `UserComment`
|
||||
(`{"aigc_label_type":0,"source_info":"dreamina"}`) and `comment` (`vid:...`)
|
||||
items carry no marker the scan counts, so they stay.
|
||||
|
||||
`test_no_flagged_metadata_value_survives_in_the_output_bytes` strips every
|
||||
tracked MP4 fixture and searches the output byte for byte for each flagged
|
||||
source value, after the pattern of byte-level leftover checks in public video
|
||||
scrubbers. The walk reaches container metadata only. Strings inside the encoded
|
||||
streams are out of its reach with one exception: x264 writes its version and
|
||||
options as an H.264 SEI message, which is not AI provenance and survives the
|
||||
stream copy; only a path that re-encodes the video stream (`video visible`,
|
||||
`video invisible`) replaces it.
|
||||
|
||||
The exception is Kling's encoder tag. Every Kling video in the tracked fixtures
|
||||
and local captures (Kling 2.6, 3.0, 3.0 Omni and O3, through Higgsfield, Runway
|
||||
and Kling itself) carries one H.264 `user_data_unregistered` SEI message, payload
|
||||
type 5, whose 25-byte payload is UUID `91ca6061-4aee-3854-8614-2d5f73f4ae2e`,
|
||||
`kling-ai` and a NUL. `get_ai_metadata` reports it as `kling_sei` from the head
|
||||
window, and `identify_video` names Kling from it when no TC260 label does, as on
|
||||
`data/fixtures/visible/kling/provider-original.mp4`, which carries no other
|
||||
signal. The streaming stripper finds every copy of the UUID plus `kling-ai` in
|
||||
each top-level `mdat` during the copy pass it already makes, so the file is still
|
||||
read once, and overwrites those 24 bytes with spaces. The NAL length,
|
||||
SEI header and payload size stay, and a run of 0x20 cannot form an
|
||||
emulation-prevention sequence, so the message stays well-formed and no decoder
|
||||
reads it: `ffmpeg -f framemd5` hashes of every frame match before and after on
|
||||
both tracked Kling fixtures (found and fixed 2026-10-17). The QuickTime `©cmt` item in
|
||||
the Grok fixture (`Signature: <base64>`) also survives, because the xAI
|
||||
signature rule requires the UUID half of the pair and that half is absent here.
|
||||
|
||||
[`_internal/ebml.py`](../src/remove_ai_watermarks/_internal/ebml.py) provides the
|
||||
corresponding bounded Matroska/WebM reader. It seeks over clusters and accepts
|
||||
only a `Segment.Tags.Tag.SimpleTag` pairing `TagName=AIGC` with a JSON
|
||||
@@ -637,6 +676,10 @@ the official C2PA registry. Refresh it with
|
||||
upstream schema invariants, records the exact source revision and writes a
|
||||
deterministic module. Runtime inspection remains offline, and registry membership
|
||||
is name-only evidence rather than proof that a compatible decoder ships.
|
||||
`--check`, run by `maintain.sh` and the CI lint job ahead of Ruff, compares the
|
||||
snapshot with the live upstream revision, so an upstream registry commit fails
|
||||
lint on `main` and on every pull request until the snapshot is refreshed, and the
|
||||
Ruff step behind it does not run until then.
|
||||
The GitHub API revision lookup sends `GITHUB_TOKEN` (or `GH_TOKEN`) when set,
|
||||
because the unauthenticated 60 requests/hour per IP limit is exhausted on
|
||||
shared CI runners; `.env.example` documents the variable.
|
||||
@@ -743,8 +786,11 @@ Key contracts:
|
||||
A.1 string (`"Label":"1","ContentProducer":"...","ProduceID":"..."`) under the
|
||||
default SM2 user ID, and MiniMax's `LabelMataData` spelling is accepted beside
|
||||
the guide's `Md`. Only the label signature is checked: MiniMax's binding
|
||||
signature also verifies, but no verdict reads it, and its content hash is not
|
||||
recomputed because MiniMax names no `CntSel`. The key travels in the file, so
|
||||
signature also verifies, but no verdict reads it. The guide's appendix A.2
|
||||
defines `DFT` for non-JPEG/PNG files as all bytes except the metadata label;
|
||||
MiniMax names no `CntSel`, and measured whole-file, direct DFT byte-range,
|
||||
metadata-box, and media-payload interpretations do not reproduce its content
|
||||
hash. The key travels in the file, so
|
||||
only a key pinned in `TC260_PRODUCERS` (the source of `TC260_SIGNING_KEYS`)
|
||||
names a signer, and a pinned signer
|
||||
is checked with the pinned point whatever parity the file claims; an unpinned
|
||||
@@ -1498,6 +1544,15 @@ schemas 0-2 count as positives. Schema 3 is below the precision threshold: all
|
||||
same false payload after re-encoding. The official Adobe Variant P schema-1
|
||||
fixture in `data/fixtures/provenance/` is the positive regression control.
|
||||
|
||||
An independent benchmark supports gating beyond a valid codeword. Kazempourradi
|
||||
et al. ([arXiv:2610.04151](https://arxiv.org/abs/2610.04151), section 4.4)
|
||||
found that the blind detectors of TrustMark B, C, P and Q in BCH_5 mode reported
|
||||
a valid codeword on at least one of four decodes for 5.9% (P) to 15.3% (C) of 85
|
||||
unmarked images, and on 3 of 8 images in a 16-bit HDR subset. Testing the same
|
||||
decodes against the expected payload produced no false match. This detector is
|
||||
blind and has no expected payload, so the re-encode stability and schema
|
||||
restriction above stand in for that test.
|
||||
|
||||
#### Why the DWT-DCT decoder looks the way it does
|
||||
|
||||
Two things in `dwt_dct.py` are load-bearing and neither is obvious from the code.
|
||||
@@ -1593,8 +1648,8 @@ their own: the Gemini sparkle has its own higher-confidence path, and the pill
|
||||
alone is too weak to attribute.
|
||||
|
||||
The set of marks that veto the Jimeng pill is derived from the registry rows:
|
||||
every other product from the same manufacturer. Doubao therefore vetoes the pill
|
||||
because both products are ByteDance, while Qwen, Kling, Yuanbao, RunningHub,
|
||||
every other product from the same manufacturer. Doubao and Dola therefore veto the pill
|
||||
because all three products are ByteDance, while Qwen, Kling, Yuanbao, RunningHub,
|
||||
Baidu, and LiblibAI do not become siblings merely because they use the same TC260
|
||||
standard. Marks from other manufacturers cannot enable or veto the
|
||||
ByteDance-specific arm.
|
||||
@@ -1653,8 +1708,7 @@ assets and template ladder are immutable.
|
||||
|
||||
A decorative sparkle/glint/star glyph in stylized typography -- a bright,
|
||||
near-white four-point accent, the same silhouette family the engine's own
|
||||
template captures -- is a known, unresolved false-positive class. Incident
|
||||
2026-08-27 (raiw-app, Spaces uid 7e01ef8cb99949bba14ab4607a66ae06): a book-cover
|
||||
template captures -- is a known, unresolved false-positive class. A book-cover
|
||||
title's decorative diamond accent inside a neon-glow letterform scored 0.64, just
|
||||
under `_SPARKLE_FP_CONF`, and the removal step painted a permanent smudge over
|
||||
the artwork. Investigation (margin, gradient, saturation, and a prototyped local
|
||||
@@ -1694,6 +1748,11 @@ Regression coverage:
|
||||
[`_text_mark_engine.py`](../src/remove_ai_watermarks/_text_mark_engine.py)
|
||||
provides common localization, detection front ends, template caching, rival
|
||||
comparison, and footprint construction.
|
||||
[`_text_mark_variants.py`](../src/remove_ai_watermarks/_text_mark_variants.py)
|
||||
combines independently gated layouts under one registry key while retaining the
|
||||
winning layout's template and mask geometry. Its bottom-right variant searches
|
||||
only anchor-valid placements, so an unrelated stronger match elsewhere in the
|
||||
corner ROI cannot hide the real edge-hugging mark.
|
||||
|
||||
Each vendor module supplies a `TextMarkConfig` and only the behavior that cannot
|
||||
be represented by the shared base:
|
||||
@@ -1703,6 +1762,8 @@ be represented by the shared base:
|
||||
- [`qwen_engine.py`](../src/remove_ai_watermarks/qwen_engine.py)
|
||||
- [`kling_engine.py`](../src/remove_ai_watermarks/kling_engine.py)
|
||||
- [`yuanbao_engine.py`](../src/remove_ai_watermarks/yuanbao_engine.py)
|
||||
- [`dola_engine.py`](../src/remove_ai_watermarks/dola_engine.py)
|
||||
- [`workbuddy_engine.py`](../src/remove_ai_watermarks/workbuddy_engine.py)
|
||||
- [`samsung_engine.py`](../src/remove_ai_watermarks/samsung_engine.py)
|
||||
- [`runninghub_engine.py`](../src/remove_ai_watermarks/runninghub_engine.py)
|
||||
- [`baidu_engine.py`](../src/remove_ai_watermarks/baidu_engine.py)
|
||||
@@ -1747,7 +1808,10 @@ on 2026-09-04 uses `KlingAI 3.0`. The Latin template scored 0.429 on that 1024 x
|
||||
across the 94 available neighboring real and synthetic image controls, while an
|
||||
adversarial solid corner blob reached 0.379, so its strict gate is 0.40. This is
|
||||
one-positive provisional calibration, intentionally separate from the older
|
||||
CJK detector rather than presented as broad recall evidence.
|
||||
CJK detector rather than presented as broad recall evidence. Both layouts now
|
||||
project row-wide and column-wide structure out of the winning patch before a
|
||||
second correlation. This preserves glyph detail while rejecting ordinary panel
|
||||
and image boundaries that previously matched the Latin template.
|
||||
|
||||
The measured Microsoft badge variant (2026-08-27 registration) is the first
|
||||
`tr`-corner mark and the first `long`-side scale basis: the pill tracks the
|
||||
@@ -1758,7 +1822,8 @@ undersized the template by the aspect ratio (portrait carriers fell to
|
||||
its synthetic internal shapes knocked out - the holes are what separate it from
|
||||
any other bright rounded corner element (a plain white pill scores below the gate
|
||||
in the tests). It does not claim coverage of Microsoft's other documented icon,
|
||||
wording, or position variants.
|
||||
wording, or position variants beyond the separately gated beige `Made with AI`
|
||||
text pill.
|
||||
|
||||
The 2026-08-27 rerun used the registered engine through
|
||||
`scripts/registered_mark_calibrate.py`, rather than a copied detector
|
||||
@@ -1795,8 +1860,9 @@ box grows), not the whole `footprint_mask`. Baidu extends right to the corner ta
|
||||
and LiblibAI extends left to the triangle logo; both inherit every guard around
|
||||
that arithmetic.
|
||||
|
||||
Doubao, Kling, and Samsung are deliberate sparse-mask exceptions because their alpha assets
|
||||
supply more information than a rectangle. Doubao's continuous top-hat response
|
||||
Doubao, Kling, and Samsung retain deliberate sparse masks for their original
|
||||
layouts because those alpha assets supply more information than a rectangle.
|
||||
Doubao's continuous top-hat response
|
||||
locates the mark, then `DoubaoEngine.footprint_mask` resizes the alpha to that same winning
|
||||
`match_box` and masks only the glyphs. The canonical 2048-pixel fixture has bright
|
||||
branch texture behind the bottom-edge wordmark; bounding the thresholded response,
|
||||
@@ -1806,6 +1872,13 @@ with large triangular wedges. The aligned sparse mask keeps both frame edges
|
||||
untouched and still clears the detector. `force` cannot align a missing detection,
|
||||
so it retains the shared geometry-box fallback.
|
||||
|
||||
The added outlined Doubao, stacked Yuanbao, Dola, WorkBuddy, Korean Samsung, and
|
||||
beige Microsoft layouts use solid detector-aligned footprints where a sparse
|
||||
synthetic glyph mask left visible outlines, sparkles, capsules, or shadows in
|
||||
rendered removal checks. Each layout keeps its own detection threshold and mask
|
||||
geometry; selecting one does not replace the calibrated behavior of the older
|
||||
layout under the same registry key.
|
||||
|
||||
Doubao keeps the original low-saturation scan for pale corners. When enough of
|
||||
its corner is strongly colored and that scan falls below the strict gate, a
|
||||
wider color filter searches for white glyphs tinted by the background. The
|
||||
@@ -1933,14 +2006,12 @@ exists.
|
||||
|
||||
#### OpenArt wordmark
|
||||
|
||||
Registered 2026-09-21 for a confirmed production miss reported through raiw-app:
|
||||
a free `visible`-mode run (2026-09-11, 1528x2712 portrait, Spaces uid
|
||||
`aefb1e07306f4a7280615a8daea01a39`) left a large, prominent OpenArt wordmark
|
||||
Registered 2026-09-21 after a `visible`-mode run left a large, prominent OpenArt wordmark
|
||||
(a bowtie/infinity icon followed by "OpenArt" in bold sans-serif, semi-
|
||||
transparent white) completely unremoved -- pixel-identical before and after.
|
||||
`identify` found nothing at all for that case: no C2PA, EXIF AI tag, or China
|
||||
AIGC/TC260 label, so the visible mark is the only known attribution route for
|
||||
a bare OpenArt export. The user rated the result 0 ("Didn't work").
|
||||
a bare OpenArt export.
|
||||
|
||||
**This is the only registered mark placed at the FRAME CENTER rather than a
|
||||
corner.** Every other engine in this family anchors to one of the four corners
|
||||
@@ -2004,10 +2075,14 @@ geometry, and until one exists the bare-arm question is settled by this
|
||||
measurement, not by gate tuning. The local calibration harness is not tracked.
|
||||
(cohorts A/B/C over the no-signal pool).
|
||||
|
||||
The weak Jimeng pill detector lives in
|
||||
[`pill_engine.py`](../src/remove_ai_watermarks/pill_engine.py). It uses a
|
||||
synthetic silhouette for detection and a fixed top-left footprint. Its public,
|
||||
published real-world regression carrier is
|
||||
The Jimeng pill detector lives in
|
||||
[`pill_engine.py`](../src/remove_ai_watermarks/pill_engine.py). It combines the
|
||||
existing weak synthetic-outline path with a high-confidence contrast template
|
||||
for the measured solid `AI生成` label. A Jimeng provenance match can corroborate
|
||||
that verified label without the flat-background guard; the weak outline keeps
|
||||
both the provenance and background gates described above. Removal follows the
|
||||
matched label when verified and otherwise uses the fixed top-left footprint. Its
|
||||
public, published real-world regression carrier is
|
||||
`data/fixtures/visible/jimeng_pill/provider-published-example.jpg`; the direct
|
||||
detector scores the pill at 0.28 and the same image's Jimeng wordmark at 0.61.
|
||||
|
||||
@@ -2133,6 +2208,13 @@ image sizing, postprocessing, and the public engine
|
||||
interface. It delegates model execution to
|
||||
[`_internal/watermark_remover.py`](../src/remove_ai_watermarks/_internal/watermark_remover.py).
|
||||
|
||||
The engine preserves source transparency for alpha-capable outputs. It captures
|
||||
alpha after EXIF orientation is applied and before resizing, then rejoins the
|
||||
unchanged alpha plane after restoring the RGB result to the original dimensions
|
||||
and applying postprocessing. This also handles palette and colour-key PNG
|
||||
transparency; diffusion itself still receives RGB. Regression coverage lives in
|
||||
`test_invisible_engine.py::TestTransparency`.
|
||||
|
||||
`get_device` in that module answers only `cuda` or `cpu`. An `mps` or `xpu` answer
|
||||
would travel one frame to the same CUDA-only refusal while costing a device probe,
|
||||
and reporting it implied an Apple-silicon or Intel-GPU path that does not exist.
|
||||
@@ -2162,27 +2244,27 @@ Regression coverage:
|
||||
|
||||
### CPU offload
|
||||
|
||||
CPU offload is enabled only when requested. Nothing calls Diffusers'
|
||||
`enable_model_cpu_offload` any more -- that belonged to the deleted single-stage
|
||||
profiles. `--cpu-offload` sets both residency fields, and they do not travel the
|
||||
same distance: the face field is read by the shared base and so applies to every
|
||||
profile, while the global field is read by `QwenZImagePipeline` alone.
|
||||
`ChromaZImagePipeline` and `SdxlZImagePipeline` load their global stack with a
|
||||
plain `.to(device)` and never consult it.
|
||||
Nothing calls Diffusers' `enable_model_cpu_offload` any more -- that belonged to
|
||||
the deleted single-stage profiles. `--cpu-offload` sets both residency fields, and
|
||||
they do not travel the same distance: the face field is read by the shared base and
|
||||
so applies to every profile, while the global field is read by `QwenZImagePipeline`
|
||||
and `SdxlZImagePipeline`. `ChromaZImagePipeline` loads its global stack with a
|
||||
plain `.to(device)` and never consults it.
|
||||
|
||||
`GLOBAL_OFFLOAD_PROFILES` in `watermark_profiles.py` is the list, and
|
||||
`global_offload_supported` the question; `WatermarkRemover._warn_if_global_offload_unsupported`
|
||||
warns ahead of the model load rather than at construction, because `auto` picks
|
||||
its engine per-image. An explicit `auto` preload uses the qwen-zimage fallback and
|
||||
therefore keeps global offload active before that resolution. Below
|
||||
`RESIDENT_FACE_MODEL_MIN_VRAM_GIB` the flag is a no-op for the other two profiles --
|
||||
the face stack was offloading anyway and the global one does not ask -- which is
|
||||
what the warning exists to say before a download starts.
|
||||
`RESIDENT_FACE_MODEL_MIN_VRAM_GIB` the flag is a no-op for `chroma-zimage` -- the
|
||||
face stack was offloading anyway and the global one does not ask -- which is what
|
||||
the warning exists to say before a download starts.
|
||||
|
||||
Residency is otherwise chosen from the card's total VRAM, once per stack:
|
||||
`resolve_global_model_residency` gates the mandatory Qwen stack at
|
||||
`RESIDENT_GLOBAL_MODEL_MIN_VRAM_GIB` and `resolve_face_model_residency` gates the
|
||||
optional Z-Image stack at `RESIDENT_FACE_MODEL_MIN_VRAM_GIB`.
|
||||
`RESIDENT_GLOBAL_MODEL_MIN_VRAM_GIB`, `resolve_sdxl_global_residency` gates the
|
||||
SDXL stack at `RESIDENT_SDXL_GLOBAL_MIN_VRAM_GIB`, and `resolve_face_model_residency`
|
||||
gates the optional Z-Image stack at `RESIDENT_FACE_MODEL_MIN_VRAM_GIB`.
|
||||
|
||||
Below the global floor, `_qwen_vram_config` streams the stack from disk, which is
|
||||
what makes a 20B model runnable on a consumer card. At or above it, streaming is
|
||||
@@ -2199,6 +2281,43 @@ The resident config deliberately passes no `"disk"` value anywhere. DiffSynth la
|
||||
`disk_offload` once, from `offload_dtype`, so leaving the sentinel in place while
|
||||
pointing every device at CUDA would keep the meta-drop and re-read.
|
||||
|
||||
Sequential SDXL offload receives the current CUDA device index explicitly, so
|
||||
its hooks use the same GPU as the residency probes, input tensors, and seeded
|
||||
generators. Omitting the index makes Diffusers select GPU 0 even after
|
||||
`torch.cuda.set_device(1)`. `test_streaming_uses_the_current_cuda_device` in
|
||||
`test_cpu_offload.py` covers automatic and forced streaming on nonzero devices.
|
||||
|
||||
The SDXL stack (fp16 UNet, Canny ControlNet, both text encoders, the fp16-fix VAE)
|
||||
is ~8.8 GiB of weights, and the UNet and the ControlNet run together on every
|
||||
denoising step. Below the 12 GiB floor `_load_global` fuses the Lightning LoRA on
|
||||
the CPU and then calls `enable_sequential_cpu_offload`; the order matters, because
|
||||
the offload hooks stream their own copy of each weight and a later fuse would never
|
||||
reach it. Measured on an RTX 5060 Laptop (8 GiB, WSL2, 16 GiB system RAM) in
|
||||
October 2026, one 1672x941 global pass at strength 0.15:
|
||||
|
||||
| Placement | Warm pass | Peak VRAM | Peak system RAM |
|
||||
|---|---|---|---|
|
||||
| resident `.to(device)` (previous behaviour) | did not finish in 10 min | 7.9 GiB used, spilling | -- |
|
||||
| `enable_model_cpu_offload` | 237 s | 9.2 GiB reserved on an 8 GiB card | 12.4 GiB |
|
||||
| group offload, block level, CUDA stream | 92 s, or hung | 12.4 GiB reserved | 15.6 GiB |
|
||||
| group offload, leaf level, CUDA stream | 7.8 s, or hung | 5.6 GiB | 15.7 GiB |
|
||||
| group offload, leaf level, `low_cpu_mem_usage` | 21.4 s | 5.6 GiB | 9.4 GiB |
|
||||
| `enable_sequential_cpu_offload` | 12.8 s | 3-4 GiB | 8.5 GiB |
|
||||
|
||||
The resident and model-offload rows did not fail: under the Windows display driver
|
||||
CUDA falls back to shared system memory, so PyTorch reserved more than the card
|
||||
holds and the run crawled instead of raising. The streamed group-offload variants
|
||||
pin every weight in host memory up front; on a 16 GiB machine that left the run
|
||||
finishing or hanging depending on what else held RAM, which rules them out as a
|
||||
default. All completed placements produced bit-identical images.
|
||||
|
||||
Fusing on the CPU instead of on CUDA rounds 788 of the UNet's 3256 fused tensors
|
||||
differently, by at most one fp16 ulp (1.2e-4). On the pass above that moved the
|
||||
output by a mean of 0.16 levels (PSNR 54.3 dB against a CUDA-fused, equally
|
||||
streamed run). Cards at or above the floor keep the CUDA fuse and therefore the
|
||||
previous pixels; only a streamed run, automatic or forced with `--cpu-offload`,
|
||||
sees the difference.
|
||||
|
||||
Regression coverage:
|
||||
|
||||
- [`test_cpu_offload.py`](../tests/test_cpu_offload.py)
|
||||
|
||||
@@ -14,6 +14,7 @@ through the installed CLI or public Python API.
|
||||
| `openai-api` | OpenAI SynthID | Official Content Provenance API | Images |
|
||||
| `microsoft-api` | Microsoft InvisMark | Azure Content Provenance Detection API | Images |
|
||||
| `gemini-web` | Google SynthID | Signed-in Gemini web account | Images and videos |
|
||||
| `synthid-web` | Partner SynthID | SynthID Detector at synthid.com, real Chrome | None recorded |
|
||||
| `meta-web` | Meta Content Seal | Anonymous Meta identification page | Images |
|
||||
| `microsoft-web` | Microsoft InvisMark | Public Azure AI validation page | Images |
|
||||
| `openai-web` | OpenAI SynthID | Public web verifier | Images |
|
||||
@@ -23,26 +24,46 @@ through the installed CLI or public Python API.
|
||||
The default selection order is API first, then Web. OpenAI therefore plans
|
||||
`openai-api` before `openai-web`, and Microsoft plans `microsoft-api` before
|
||||
`microsoft-web`. Google and Meta currently have no usable API adapter in this
|
||||
tooling, so their Web surfaces are primary rather than fallback routes.
|
||||
tooling, so each of their Web surfaces is primary. Google has two: `gemini-web`
|
||||
and `synthid-web`.
|
||||
|
||||
Inspect the machine-readable order before preparing a batch:
|
||||
|
||||
```bash
|
||||
uv run python scripts/provider_oracles.py plan openai --json
|
||||
uv run python scripts/provider_oracles.py plan microsoft
|
||||
uv run python scripts/provider_oracles.py plan google
|
||||
```
|
||||
|
||||
API-first is a selection policy, not automatic failover. Use Web only when the
|
||||
provider, requested media, or local configuration has no usable API route, or
|
||||
when the operator explicitly requests a second surface. An API refusal, quota
|
||||
response, transport failure, or indeterminate verdict is recorded as-is and
|
||||
never silently triggers a Web upload.
|
||||
A request to use a provider covers every configured surface of that provider,
|
||||
including Web, without a second confirmation. API-first is still the order to
|
||||
try, not a lock on the other surfaces. An API refusal, quota response, transport
|
||||
failure, or indeterminate verdict is recorded as-is. It does not silently
|
||||
upload the same file to Web; run that surface as its own acknowledged check and
|
||||
keep the two verdicts separate.
|
||||
|
||||
The old Google SynthID Detector portal does not work. On 2026-09-07, the
|
||||
official `labs.google/synthid` entry point redirected a signed-in browser to a
|
||||
Google 404; on 2026-09-23 it redirected to
|
||||
`deepmind.google.com/frontiers/synthid`, which also returned 404. Historical corpus rows may still name `synthid-portal`; that value
|
||||
describes how an old result was obtained and is not an operational surface.
|
||||
The old Google SynthID Detector portal at `labs.google/synthid` does not work.
|
||||
On 2026-09-07 that entry point redirected a signed-in browser to a Google 404;
|
||||
on 2026-09-23 it redirected to `deepmind.google.com/frontiers/synthid`, which
|
||||
also returned 404. Historical corpus rows may still name `synthid-portal`; that
|
||||
value describes how an old result was obtained and is not an operational surface.
|
||||
|
||||
The public SynthID Detector is a different site, [synthid.com](https://synthid.com/),
|
||||
and it is the `synthid-web` surface. On 2026-10-17 the page title was
|
||||
`SynthID Detector`. After the terms dialog, the uploader was present without a
|
||||
completed sign-in in that session. The Account menu still offered Sign in, and
|
||||
no file was submitted, so this check does not show whether a result requires an
|
||||
account. The live FAQ named the partners as Google DeepMind, Nvidia, OpenAI,
|
||||
and Kakao, and it did not list Apple. It defined two settled labels:
|
||||
`Watermark detected` and `Watermark not detected`. The file input accepted
|
||||
`.jpg`, `.jpeg`, `.png`, `.bmp`, `.webp`, `.avif`, `.heic`, `.heif`, `.tiff`,
|
||||
`.tif`, `.gif`, `.wav`, `.mp3`, `.ogg`, `.flac`, `.aac`, `.m4a`, `.mp4`,
|
||||
`.mov`, and `.webm`. This tooling still prepares only the project's existing
|
||||
JPEG, PNG, WebP, MP4, MP3, and WAV suffixes. The portal's additional terms
|
||||
forbid using the page as an automated API, bulk verification, and quota evasion,
|
||||
so `run-web` does not drive it. Use the real Chrome workflow below. A
|
||||
`Watermark detected` result is partner SynthID, not a Gemini-only or OpenAI-only
|
||||
verdict.
|
||||
Google's newer Search verification entry point did not provide an independent
|
||||
oracle allowance in the session tested on 2026-09-13. Lens accepted the
|
||||
metadata-stripped, previously Gemini-positive image
|
||||
@@ -161,9 +182,12 @@ Adobe Inspect and local C2PA, DWT-DCT, TrustMark, and research detectors are
|
||||
useful evidence readers, not interchangeable provider pixel-watermark oracles.
|
||||
|
||||
The surface capability and this project's evidence coverage are separate. The
|
||||
current Google, Microsoft, and Meta pages accept image, video, and audio inputs;
|
||||
the OpenAI API accepts image and audio. A media type does not become certified
|
||||
merely because a surface accepts it. Run `list --json` to read both fields.
|
||||
current Google, Microsoft, and Meta pages, including `https://synthid.com/`,
|
||||
accept image, video, and audio inputs; the OpenAI API accepts image and audio.
|
||||
A media type does not become certified merely because a surface accepts it.
|
||||
`synthid-web` therefore lists image, video, and audio as accepted media and
|
||||
records no project evidence until a batch is verified. Run `list --json` to read
|
||||
both fields.
|
||||
|
||||
## Execution slots
|
||||
|
||||
@@ -193,6 +217,13 @@ default; `--slots PATH` explicitly overrides it.
|
||||
"google_account_index": 2,
|
||||
"network_label": "office"
|
||||
},
|
||||
{
|
||||
"name": "synthid-personal",
|
||||
"surface": "synthid-web",
|
||||
"account_label": "google-account-a",
|
||||
"browser_profile": "Your Chrome",
|
||||
"network_label": "home"
|
||||
},
|
||||
{
|
||||
"name": "openai-web-thordata-us",
|
||||
"surface": "openai-web",
|
||||
@@ -261,7 +292,9 @@ default; `--slots PATH` explicitly overrides it.
|
||||
```
|
||||
|
||||
A routed batch selects one slot explicitly. A Gemini slot must name the Google
|
||||
account, its real Chrome profile, and its nonnegative `/u/N/` account index; an
|
||||
account, its real Chrome profile, and its nonnegative `/u/N/` account index. A
|
||||
SynthID Detector slot names the account and the same real Chrome profile, and it
|
||||
does not use `google_account_index`. An
|
||||
API slot must name its account or project and cannot name a browser profile. The
|
||||
tool records the network route that the operator has already established.
|
||||
Non-Google Web slots may name `proxy_url_env`; Playwright reads the secret proxy
|
||||
@@ -279,9 +312,10 @@ again for files or accounts already covered by the task.
|
||||
Submit each frozen artifact at most once per slot in that pass. Keep every
|
||||
verbatim refusal, inconclusive reply, and disagreement between accounts. Stop
|
||||
when the planned checks finish or no authorized usable slot remains, and report
|
||||
unsubmitted rows. A new login, another provider, a new network route, or a new
|
||||
pass after the bounded plan is exhausted needs authorization unless already
|
||||
requested. Candidate pixels remain frozen throughout account continuation.
|
||||
unsubmitted rows. A new login, another provider, or a new network route needs
|
||||
authorization. The oracles and slots already configured for the requested
|
||||
provider may be used again, including their Web surfaces, without a second
|
||||
confirmation. Candidate pixels remain frozen throughout account continuation.
|
||||
|
||||
Each `openai-api` slot must name exactly one `api_key_env`. Put the corresponding
|
||||
values in the gitignored local `.env`, never in the slot JSON:
|
||||
@@ -385,18 +419,23 @@ uv sync --extra dev
|
||||
uv run playwright install chromium
|
||||
```
|
||||
|
||||
Google is intentionally excluded from the standalone runner because its oracle
|
||||
requires the user's existing authenticated Chrome session. Invoke the project
|
||||
`provider-oracles` skill to prepare the batch, select the named real Chrome
|
||||
profile and `/u/N/` Google account, upload each prepared artifact, capture the
|
||||
settled SynthID tool outcome, and record it.
|
||||
Google is intentionally excluded from the standalone runner. Both `gemini-web`
|
||||
and `synthid-web` use the user's existing real Chrome profile, and the SynthID
|
||||
Detector terms forbid driving that page as an automated API. Invoke the project
|
||||
`provider-oracles` skill to prepare the batch and record the settled outcome.
|
||||
For Gemini, select the named profile and `/u/N/` account. For SynthID Detector,
|
||||
open `https://synthid.com/` in that profile. The portal has no `/u/N/` index.
|
||||
Accept the terms dialog when it is shown, attach one prepared upload, and read
|
||||
the explicit `Watermark detected` or `Watermark not detected` label. If the page
|
||||
requires sign-in and that profile has no session, record the row unsubmitted
|
||||
and do not start a new login.
|
||||
|
||||
Ask the direct question stored in the batch manifest: `Does this
|
||||
For Gemini, ask the direct question stored in the batch manifest: `Does this
|
||||
image/video/audio contain a SynthID watermark? Use the SynthID verification
|
||||
tool only; do not infer from visible content, logos, text, or metadata.` A
|
||||
general question about whether Google AI created or edited the media is not
|
||||
equivalent: it may elicit origin or visual-content reasoning without a SynthID
|
||||
verdict.
|
||||
verdict. SynthID Detector has no prompt; its settled labels are the verdict.
|
||||
|
||||
OpenAI Web, Microsoft Web, and Meta run entirely through a fresh isolated
|
||||
Playwright browser. A direct-route example:
|
||||
|
||||
@@ -24,9 +24,10 @@ The package version appears in:
|
||||
|
||||
- `pyproject.toml`;
|
||||
- `src/remove_ai_watermarks/__init__.py`;
|
||||
- both `version` fields in `server.json`;
|
||||
- the root package entry generated in `uv.lock`.
|
||||
|
||||
Update the first two, then refresh the lock file with uv. Do not edit a
|
||||
Update the first three, then refresh the lock file with uv. Do not edit a
|
||||
line-number-specific location in `uv.lock`; its package order changes.
|
||||
|
||||
Before choosing a version, verify it is absent from git tags, GitHub Releases,
|
||||
@@ -159,7 +160,10 @@ The Agent Skill and Claude plugin live in this repository:
|
||||
is the Claude plugin manifest, with source `./skills` so an install copies
|
||||
only the skill tree;
|
||||
- [`.claude-plugin/marketplace.json`](../.claude-plugin/marketplace.json)
|
||||
makes the repository a Claude Code marketplace.
|
||||
makes the repository a Claude Code marketplace;
|
||||
- [`.codex-plugin/plugin.json`](../.codex-plugin/plugin.json) is the Codex
|
||||
plugin manifest, pointing at the same `./skills/` tree, with its icon in
|
||||
[`assets/icon.svg`](../assets/icon.svg).
|
||||
|
||||
They are not a fifth required release surface. A CLI change that affects
|
||||
command routing, extras, exit codes, mark keys, or the intended-use boundary
|
||||
@@ -215,6 +219,26 @@ the catalog has not accepted the initial submission. The catalog-specific
|
||||
submission state and links live in
|
||||
[`docs/agent-skill.md`](agent-skill.md#where-the-skill-can-actually-be-listed).
|
||||
|
||||
## MCP registry
|
||||
|
||||
[`server.json`](../server.json) lists the PyPI package in the
|
||||
[official MCP registry](https://registry.modelcontextprotocol.io/) as
|
||||
`io.github.wiltodelta/remove-ai-watermarks`, started as
|
||||
`uvx remove-ai-watermarks mcp`. The registry accepts it only after PyPI serves a
|
||||
release whose description, the README, carries the `mcp-name:` marker, and
|
||||
`tests/test_mcp_server.py` pins `server.json` to the package version. The listing
|
||||
is not a release surface that blocks a release; after PyPI publishes a version:
|
||||
|
||||
1. `mcp-publisher login github` with an account that owns `wiltodelta`;
|
||||
2. `mcp-publisher validate` and then `mcp-publisher publish` from the
|
||||
repository root;
|
||||
3. read the entry back from
|
||||
`https://registry.modelcontextprotocol.io/v0/servers?search=io.github.wiltodelta/remove-ai-watermarks`
|
||||
and check its version.
|
||||
|
||||
[`glama.json`](../glama.json) names the maintainer for Glama's claim flow.
|
||||
Client setup and the tool contract are in [the MCP server guide](mcp-server.md).
|
||||
|
||||
## Release verification
|
||||
|
||||
Forensic transports are versioned independently from the package. Before publishing
|
||||
|
||||
@@ -33,22 +33,28 @@ This page is an index. Each finding lives in its canonical page.
|
||||
| Dependencies | TrustMark extra on every supported Python (`trustmark>=0.9.2`) | [installation.md](installation.md) |
|
||||
| Live captures | Gemini Omni, Grok, Vidu, Wan, Apple, Google Photos, Nano Banana 2 Lite and Gemini Omni Flash through the API; Runway, Higgsfield and Dreamina | [watermarking-landscape.md](watermarking-landscape.md#live-captures-2026-09-23-and-2026-09-24) |
|
||||
|
||||
## Open items
|
||||
## Follow-up status
|
||||
|
||||
- **Google Photos Reimagine manifests.** Magic Editor, and so Reimagine, is not
|
||||
offered in Google Photos on iOS, so no Reimagine file is captured. The
|
||||
The original follow-up list now mixes completed experiments, deliberately
|
||||
excluded claims, and acquisition gaps. Each row below names its current status
|
||||
as of 2026-10-01 so this section can serve as the research backlog without
|
||||
discarding the measured results.
|
||||
|
||||
- **Closed - Google Photos Reimagine manifests.** Magic Editor, and so
|
||||
Reimagine, is not offered in Google Photos on iOS, so no Reimagine file is
|
||||
captured. The
|
||||
question it was meant to answer is settled without one: Google's checker found
|
||||
SynthID in all four Photos AI edits tested on 2026-09-25, none of which
|
||||
records SynthID in its manifest.
|
||||
- **Vidu and Wan marks on more samples.** Both are registered from one real
|
||||
export each (`vidu` video, `wan` image); their gates are provisional until a
|
||||
- **Open - Vidu and Wan marks on more samples.** Both are registered from one
|
||||
real export each (`vidu` video, `wan` image); their gates are provisional until a
|
||||
wider cohort is measured, and a Wan video label is not yet captured. Public
|
||||
Vidu feed videos (2026-09-26) carry the TC260 label naming ShengShu, and one
|
||||
names Tongyi Yunqi as producer with Vidu as propagator, but none shows the
|
||||
visible `Vidu` mark; Wan's public demo videos show no mark either. Both marks
|
||||
appear only on account downloads.
|
||||
- **TC260 practice guides.** The audio placements from TC260-PG-202510A are
|
||||
now read (the guide PDFs load in a browser from
|
||||
- **Partially complete - TC260 practice guides.** The audio placements from
|
||||
TC260-PG-202510A are now read (the guide PDFs load in a browser from
|
||||
`tc260.org.cn/tc260/sjzn/list.shtml`, not to scripted requests). The text-file
|
||||
guide (TC260-PG-20258A) covers documents this image and video tool does not
|
||||
process. The security-protection guide (TC260-PG-202511A) records a
|
||||
@@ -66,15 +72,25 @@ This page is an index. Each finding lives in its canonical page.
|
||||
point; changing one `ProduceID` digit fails it. MiniMax's own spellings depart
|
||||
from the guide: `TBSData.Type` is `LabelMataData` (the guide's `Md`) and
|
||||
`Binding` with `BType` `0` (its `Bnd`); the second signature verifies over the
|
||||
compact JSON of the `Bindings` entry, also on all four. That entry's SM3
|
||||
content hash matched neither the whole file nor the file without its label,
|
||||
and it names no `CntSel`, so the content binding is unverified. Other producers fill the same
|
||||
compact JSON of the `Bindings` entry, also on all four. TC260-PG-202511A
|
||||
appendix A.2 was re-read from the
|
||||
[official PDF](https://www.tc260.org.cn/tc260/sjzn/202508/63ab55a3265f45c7865e9b8be15d4a29/files/dc379ec776f94abc8f1806affa1cda35.pdf)
|
||||
on 2026-10-01. MiniMax omits its `CntSel`, and none of the guide-derived
|
||||
byte selections reproduced its SM3 content hash on four measured videos, so
|
||||
the content binding remains unverified
|
||||
([module internals](module-internals.md)). Other producers fill the same
|
||||
fields with opaque vendor strings (Qwen a base64 digest, Vidu 32 hex digits,
|
||||
Wan and HappyHorse `L-`/`K-` prefixed values), not `SecurityData`. The key sits
|
||||
in the file it signs, so a check proves authenticity only against a pinned
|
||||
MiniMax key. `identify` now verifies it with a built-in SM3/SM2 and a pinned
|
||||
MiniMax key ([module internals](module-internals.md)).
|
||||
- **Experiments.** Chroma restoration after regeneration (arXiv:2508.21072) was
|
||||
MiniMax key. The remaining work is
|
||||
MiniMax content-binding verification if its byte-selection rule becomes known,
|
||||
and Honor's unsupported RSA/content form if an original export and signing
|
||||
contract become available.
|
||||
- **Partially complete - experiments.** Five of the six follow-ups are closed:
|
||||
Z-Image-Turbo, Perth and PixelSeal, re-watermarking, VACE/ROSE, and newer
|
||||
frozen-backbone probes. Chroma
|
||||
restoration after regeneration (arXiv:2508.21072) was
|
||||
measured on 2026-09-27 and is not shipped: transplanted source chroma drew one
|
||||
explicit "inconclusive" and two unclear Gemini answers where the untouched
|
||||
outputs were all clean, and ghosts redrawn objects
|
||||
@@ -126,12 +142,20 @@ This page is an index. Each finding lives in its canonical page.
|
||||
profile; retain its harness and local evidence for future comparison only.
|
||||
The existing source-conditioned Z-Image face-repair stage remains part of
|
||||
`qwen-zimage`, `sdxl-zimage`, and `chroma-zimage`.
|
||||
Still open: linear probes on newer frozen backbones for `classify` and RAVEN
|
||||
view-synthesis removal
|
||||
(arXiv:2601.08832, no code released as of 2026-09-24).
|
||||
The pre-registered modern-backbone campaign found that neither frozen
|
||||
SigLIP2 Base nor PE-Core B could match the production model's combined photo,
|
||||
AI, and FLUX constraints, so the replacement hypothesis is closed
|
||||
([classifier research](ai-generated-image-classifiers.md#modern-frozen-backbone-campaign-closed-2026-09-27)).
|
||||
Only RAVEN view-synthesis removal remains blocked. RAVEN's
|
||||
[official repository](https://github.com/fahadshamshad/raven-watermark-removal)
|
||||
now exists, but its 2026-08-08 README is still a release placeholder: the
|
||||
authors say they will publish code after their SynthID testing and Google
|
||||
Security Bounty submission. There is no implementation to evaluate as of
|
||||
2026-10-17, when the repository still held only that README (arXiv:2601.08832).
|
||||
MiniMax-Remover (CC-BY-NC-4.0 weights) and ProPainter (S-Lab License,
|
||||
non-commercial) are excluded on license.
|
||||
- **New C2PA signers without a vendor row.** The
|
||||
non-commercial) are deliberately excluded on license and are not backlog
|
||||
items.
|
||||
- **Open - new C2PA signers without a vendor row.** The
|
||||
[C2PA conformance list](https://github.com/c2pa-org/conformance-public/blob/main/conforming-products/conforming-products-list.json)
|
||||
names generator certificates `Amazon Bedrock`, `Getty Images
|
||||
AI-Generated Image` and `AI-Modified Image` (and the iStock pair), `Jasper`,
|
||||
@@ -148,7 +172,7 @@ This page is an index. Each finding lives in its canonical page.
|
||||
carrying C2PA: their marketing and model-page images are unsigned. A vivo
|
||||
X300 camera capture carries a vivo device manifest (`digitalCapture`) that
|
||||
the bundled reader cannot open (`unknown algorithm`).
|
||||
- **Chinese phone galleries.** Huawei's help pages say the Xiaoyi photo-edit AI
|
||||
- **Open - Chinese phone galleries.** Huawei's help pages say the Xiaoyi photo-edit AI
|
||||
watermark can be switched off; vivo's Album setting is already covered.
|
||||
The public-web follow-up below found Honor YOYO metadata and additional
|
||||
primary-source labeling evidence, but not a controlled gallery export set. A live
|
||||
@@ -159,11 +183,12 @@ This page is an index. Each finding lives in its canonical page.
|
||||
of an unedited control, an ordinary crop, and a generative edit are still needed
|
||||
per app before selecting an AI attribution rule. Record the device, OS/app
|
||||
version, region, operation, and original download path with each sample.
|
||||
- **Claims not supported by a primary source.** Kling embedding SynthID
|
||||
- **Closed - claims not supported by a primary source.** Kling embedding SynthID
|
||||
(repeated by remover blogs; Kuaishou is not in Google's adopter list),
|
||||
Hailuo pixel watermarks, and platforms stripping C2PA on upload. None is used
|
||||
by the library.
|
||||
- **Google Photos small edits (further work deferred, 2026-10-01).** The
|
||||
- **Partially complete - Google Photos small edits (further work deferred,
|
||||
2026-10-01).** The
|
||||
maintainer deferred further photographic tiny Magic Eraser captures and
|
||||
verification retries for the inconclusive small edits. Resume only on an
|
||||
explicit request; retain the existing evidence and uncertainty. On
|
||||
@@ -193,8 +218,13 @@ This page is an index. Each finding lives in its canonical page.
|
||||
negative and one inconclusive reply. The quota refusal, synthetic capture,
|
||||
pixel comparison, mode caveat, and hash-bound outcomes are recorded in
|
||||
[synthid.md](synthid.md).
|
||||
- **Captures blocked by region or device.** Doubao, Jimeng, Samsung. Higgsfield,
|
||||
Runway and Dreamina were captured on paid plans on 2026-09-24.
|
||||
- **Partially complete - direct captures blocked by region or device.** Direct,
|
||||
controlled captures remain blocked for Doubao, Jimeng, and Samsung. The
|
||||
public-web follow-up below later found one real marked Doubao video and
|
||||
validated its visible mark and TC260 metadata, but the file remains local
|
||||
because publication has not been cleared; it is not a controlled account
|
||||
capture or a redistributable fixture. Higgsfield, Runway, and Dreamina were
|
||||
captured on paid plans on 2026-09-24.
|
||||
|
||||
### Public-web sample search, 2026-10-01 UTC
|
||||
|
||||
@@ -251,10 +281,16 @@ provider exports. No third-party photograph was added to tracked fixtures.
|
||||
[Getty's generator page](https://www.gettyimages.com/ai/generation),
|
||||
[iStock's generator page](https://www.istockphoto.com/ai/generation), and
|
||||
[Reface's product site](https://reface.ai/) did not carry locally recognized
|
||||
C2PA. Jasper explicitly distinguishes input/mask/output examples, but an
|
||||
unsigned web derivative cannot establish its signed export's manifest shape.
|
||||
C2PA. A 2026-10-01 recheck of Jasper's current official Image landing page
|
||||
reached its pre-rendered endpoint outputs directly and again found no C2PA.
|
||||
Jasper explicitly distinguishes input/mask/output examples, but an unsigned
|
||||
web derivative cannot establish its signed export's manifest shape.
|
||||
Conformance certificates alone still do not justify fabricating that shape.
|
||||
- **Visible-image/video gaps remain.** The downloaded RunningHub workflow
|
||||
- **Visible-image/video provider-original gaps remain.** Every registered mark
|
||||
already has a publication-safe synthetic detector fixture in the
|
||||
[visible-mark gallery](../data/fixtures/visible/README.md); the sample ledgers
|
||||
now point to those fixtures and distinguish them from provider evidence. The
|
||||
downloaded RunningHub workflow
|
||||
illustrations are explanatory/branding assets, not verified output examples;
|
||||
Liblib API documentation's cat illustration has no visible historical wordmark.
|
||||
A public [Inkpainting model page](https://www.liblib.art/modelinfo/2acd393c23924ff9b5363c9c1ec9129f)
|
||||
|
||||
@@ -11,19 +11,21 @@ The `visible` command registers these mark keys:
|
||||
| Key | Mark | Expected area | Important limit |
|
||||
| --- | --- | --- | --- |
|
||||
| `gemini` | Google Gemini visible watermark (sparkle) | Usually bottom right | Detection includes a false positive gate. |
|
||||
| `doubao` | `豆包AI生成` | Bottom right | Vendor specific text detector. |
|
||||
| `doubao` | `豆包AI生成` | Bottom right | Covers filled and outlined text layouts; the weak outlined layout requires matching Doubao provenance. |
|
||||
| `jimeng` | `★ 即梦AI` | Bottom right | Vendor specific text detector. |
|
||||
| `qwen` | `千问AI生成` or the three-lobe Qwen symbol | Bottom right | Separate strict templates cover the text and Qwen Create symbol variants. |
|
||||
| `wan` | Wan logo and `Wan` wordmark | Bottom right, hugging the corner | Provisional: one real Wan 2.7 Pro export. Requires NCC >= 0.55 and a match that ends within 0.015 of the short side from both edges. |
|
||||
| `kling` | `可灵AI 3.0` or `KlingAI 3.0` | Bottom right | Separate calibrated silhouettes cover the older CJK and current IMAGE 3.0 Latin variants. |
|
||||
| `yuanbao` | `元宝` over `AI生成` | Bottom right | Standard two-line variant only. |
|
||||
| `samsung` | `✦ Contenuti generati dall'AI` | Bottom left | Calibrated for the Italian text variant. |
|
||||
| `kling` | `可灵AI 3.0` or `KlingAI 3.0` | Bottom right | Separate calibrated silhouettes cover the older CJK and current IMAGE 3.0 Latin variants; a glyph-detail gate rejects plain UI boundaries. |
|
||||
| `yuanbao` | `元宝` over `AI生成` | Bottom right | Separate strict and provenance-corroborated templates cover the measured two-line rasterizations. |
|
||||
| `dola` | `Dola AI` | Bottom right | Text detector and detector-aligned wordmark footprint. |
|
||||
| `workbuddy` | `AI生成` over `WORKBUDDY>_` | Bottom right | Text detector and detector-aligned stacked-label footprint. |
|
||||
| `samsung` | Italian or Korean Galaxy AI disclosure | Bottom left | Independently gated text layouts cover `Contenuti generati dall'AI` and `AI로 생성한 콘텐츠`. |
|
||||
| `runninghub` | `RunningHub AI生成` | Top left | Strict visual and position gates. |
|
||||
| `baidu` | `百度 AI生成` | Bottom right | Detector and extended removal footprint. |
|
||||
| `liblib` | `LiblibAI` | Bottom center | Historical wordmark variant; includes a minimum image size gate. |
|
||||
| `liblib_pill` | Compact `AI生成` pill | Top left | Auto removal requires LiblibAI metadata or the bottom-center wordmark, plus a flat background under the pill. |
|
||||
| `microsoft` | One Microsoft white AI-badge variant | Top right | Strict uses the visual gate; auto can use Microsoft provenance for the measured [relaxed gate](module-internals.md#visible-mark-removal). Other documented icon, text, and position variants are not covered. |
|
||||
| `jimeng_pill` | `AI生成` pill | Top left | Weak detector with additional product and background gates. |
|
||||
| `microsoft` | Microsoft white AI badge or beige `Made with AI` pill | Top right | Each layout has its own strict visual gate; auto can use Microsoft provenance for the original badge's measured [relaxed gate](module-internals.md#visible-mark-removal). |
|
||||
| `jimeng_pill` | `AI生成` pill | Top left | A verified label layout can bypass the flat-background guard only with Jimeng provenance; the weak outline detector retains the product and background gates. |
|
||||
| `generic_ai_label` | Bare `AI生成` text, no vendor wordmark | Bottom right | Brand-less TC260 compliance-stamp fallback (vivo and Xiaomi Gallery AI-edit output confirmed; see [module-internals.md](module-internals.md#generic-bare-ai生成-label)). Font-rendered synthetic asset; **calibrated on a synthetic corpus only**, not a real captured generator screenshot -- treat as weaker evidence than the vendor-tuned detectors above. |
|
||||
| `openart` | `OpenArt` wordmark (bowtie/infinity icon + brand name) | Frame center, not a corner | Confirmed from one customer-reported production case; no known metadata signal, so the visible mark is the only attribution route. **UNCALIBRATED**: procedurally reconstructed asset, no captured corpus -- see [module-internals.md](module-internals.md#openart-wordmark). |
|
||||
|
||||
@@ -93,6 +95,8 @@ The inspection and stripping code handles signals in these groups:
|
||||
- exact app-export provenance and AIGC disclosures from supported
|
||||
ByteDance-family products, with product-only provenance excluded from the
|
||||
generated-image verdict;
|
||||
- Kling's H.264 `user_data_unregistered` SEI tag (`kling-ai`) in MP4/MOV
|
||||
video, blanked in place by `video metadata --remove` without re-encoding;
|
||||
- IPTC AI disclosure fields;
|
||||
- PNG text chunks and embedded generation parameters, including keyed
|
||||
`workflow` and `prompt` tags in MP4/MOV metadata lists;
|
||||
@@ -298,7 +302,7 @@ not a universal clean verdict.
|
||||
| Luma AI | None registered | No registered pixel decoder | PNG text generator tags (Uni-1) |
|
||||
| ByteDance generators | Doubao and Jimeng marks | No registered pixel decoder | TC260 AIGC, supported C2PA, and exact app-export AIGC disclosures |
|
||||
| Qwen | Qwen mark | No registered pixel decoder | TC260 AIGC |
|
||||
| Kling AI | Kling AI image and video marks | No registered pixel decoder | TC260 AIGC |
|
||||
| Kling AI | Kling AI image and video marks | No registered pixel decoder | TC260 AIGC and the video SEI tag |
|
||||
| Vidu (ShengShu) | Vidu video mark | No registered pixel decoder | TC260 AIGC in the MP4 `AIGC` key |
|
||||
| Runway | None measured on paid-plan output | No registered pixel decoder | C2PA signed `RUNWAY AI, INC.` on its own models; third-party models keep their vendor's manifest |
|
||||
| Amazon Bedrock (Nova Canvas, Nova Reel) | None | Invisible watermark readable only through the Bedrock API | C2PA signed `Amazon Web Services, Inc.`, claim generator `Amazon Bedrock` |
|
||||
|
||||
@@ -200,7 +200,7 @@ route in this campaign. Mechanism detail stays in
|
||||
| --- | --- | --- |
|
||||
| Gowal et al., [arXiv:2510.09263](https://arxiv.org/abs/2510.09263) | Post-hoc encoder `f` / decoder `g`. Detection logit is not payload recovery. SynthID-O (partner variant) 136 bits at 512x512. TPR at 0.1% FPR 99.98% aggregated, 99.97% on the hardest spatial crop+resize. Trains against sampled semantics-preserving transforms, including weak VAE regeneration. Production decoder unpublished | Matches the architecture we treat as keyed `x' = x + g(x)`. Explains why a two-pixel crop kills `pipeline_lattice` but not the official oracle, and why a 22.3 dB foreign VAE still reads `detected` |
|
||||
| DeepMind [US12094474B1](https://patents.google.com/patent/US12094474B1/en) and continuation [US20250149048A1](https://patents.google.com/patent/US20250149048A1/en) | Residual U-Net encoder, separate decoder, optional key, encoder/decoder ensembles that need not recognize each other | Constraint, not a recipe. Ensemble non-recognition is why one recovered Google tile cannot be reused as an OpenAI detector (S4) |
|
||||
| Dathathri et al., [Nature 634:818-823 (2024)](https://www.nature.com/articles/s41586-024-08025-4) | SynthID-Text: tournament sampling of LLM tokens, open-source | Different system. Image/audio/video remain proprietary |
|
||||
| Dathathri et al., [Nature 634:818-823 (2024)](https://www.nature.com/articles/s41586-024-08025-4) | SynthID-Text: tournament sampling of LLM tokens, open-source. Anthropic's [Claude text watermark](https://www.anthropic.com/news/claude-text-watermark) (announced 2026-08-14 for future Claude models, updated 2026-09-01) "is a version of the SynthID-Text approach", with its detection API in private preview | Different system. Image/audio/video remain proprietary |
|
||||
| OpenAI, [advancing content provenance](https://openai.com/index/advancing-content-provenance/) (2026-05-19, audio 2026-07-31) and [content provenance API](https://developers.openai.com/api/docs/guides/content-provenance) | ChatGPT / API / Codex images carry C2PA plus SynthID. Audio from 2026-07-31. `POST /v1/content_provenance_checks`. `not_detected` does not rule out another vendor | This is the oracle. C2PA and SynthID are independent entries. Do not abuse the endpoint as an adaptive reverse-engineering loop |
|
||||
|
||||
### Keyless presence detectors in the literature
|
||||
|
||||
@@ -229,12 +229,20 @@ regeneration is already in [synthid.md](synthid.md#23-removal-attacks-and-forens
|
||||
| Goonatilake and Ateniese, [arXiv:2605.09203](https://arxiv.org/abs/2605.09203) | Six removers all leave a forensic residue a ResNet-50 sees at >98% TPR @ 1% FPR | Applies to UnMarker, Zhao's WatermarkAttacker, CtrlRegen+ | Defeating a provider oracle is not deniability. This is the product remainder |
|
||||
| An et al., [arXiv:2401.08573](https://arxiv.org/abs/2401.08573) (WAVES, ICML 2024) | 26 attacks on StegaStamp, Stable Signature, Tree-Ring. Regeneration, not JPEG, is the attack that matters. StegaStamp TPR@1%FPR 1.00 to 0.01; Tree-Ring 0.99 to 0.12 | Open watermarks | Protocol. Our blur-sigma-7 and 16-32 scramble are closer to WAVES "distortion" than to regeneration |
|
||||
| Bulychev et al., [arXiv:2605.16796](https://arxiv.org/abs/2605.16796) (Watermarks Attack Watermarks) | Apply a second watermark, usually with the victim's own post-processing encoder and a different message. On MS-COCO, its end-to-end pipeline lowers Video Seal victim bit accuracy from 0.998 to 0.520 (TPR@1%FPR 0.046) and Pixel Seal from 0.999 to 0.514 (TPR 0.029); the same-method attacker message recovers at 0.99 bit accuracy for both schemes. Quality is reported as an eight-metric normalized composite, not a raw PSNR operating point | Eight open image schemes, not production SynthID; Video Seal is tested on images | Directly motivated the [local open-encoder replication](#re-watermarking-on-open-encoders-2026-09-27). Its MIT [code](https://github.com/MariaBulychev/Watermarks-Attack-Watermarks/tree/0444f52ad972da9e009555f2dc1028d9abe0610e) calls external encoder packages and weights; the top-level license does not license those dependencies |
|
||||
| Yang et al., [arXiv:2609.01249](https://arxiv.org/abs/2609.01249) (One Prompt Is Enough) | One reconstruction prompt to a closed image editor (GPT Image 1, 1.5 and 2; Nano Banana, Pro and 2) launders the payload over 1,800 outputs; OpenAI models disrupt it most | DwtDct, DwtDctSvd and RivaGAN only. "SynthID was not tested" | Regeneration through a provider's own model. It breaks the scheme our DWT-DCT detector reads, and says nothing about production SynthID, which the same editors add to their output |
|
||||
| Yao et al., [arXiv:2610.11290](https://arxiv.org/abs/2610.11290) (ProxyEraseAgent) | Ranks a pool of open decoders by calibrated response and attacks the strongest as proxy boundary estimators; 94.8% success over 11 systems against 79.5% for the best fixed Regen-VAE | Open schemes, TrustMark and VideoSeal among them. Not SynthID | Needs a decoder that responds to the mark. No public decoder responds to production SynthID, so it does not transfer to an oracle-only target |
|
||||
| Bao et al., [arXiv:2609.08213](https://arxiv.org/abs/2609.08213) (DRIFT) | Deflects the diffusion trajectory. The adaptive variant searches a strength ladder per image for "the first verifier-rejected rung" from binary verifier feedback; 98-100% success on nine marks | In-generation diffusion marks (Tree-Ring family, Gaussian Shading and others). Not SynthID | The ladder stop is our [attack ladders](#attack-ladders-openai-oracle) run per image. Against a provider verifier that is the adaptive query loop its terms forbid, so it applies to local oracles only |
|
||||
| Ye et al., [arXiv:2610.03166](https://arxiv.org/abs/2610.03166) (LiBRA) | White-box latent optimization with the key and decoder, steering decoding confidence toward chance from both sides so the mark is not left inverted but detectable; Stable Signature detection 1.00 to 0.03 | Released checkpoints of Stable Signature, the YU1 GAN fingerprint, HiDDeN and MBRS | Needs the decoder, so it is out of reach here. The inverted-mark point applies to any bit-flip attack |
|
||||
| Aistov et al., [arXiv:2610.02010](https://arxiv.org/abs/2610.02010) (Latent Frequency Masking, ICDM 2026) | Replaces selected Fourier coefficients of the latent; [code](https://github.com/alvitrioliks/lfm) | Stable Signature, Tree-Ring, RingID, Gaussian Shading (resists), Maxsive, METR. Not SynthID | In-generation latent marks. Not the post-hoc encoder family Gowal describes |
|
||||
| Li et al., [arXiv:2609.31693](https://arxiv.org/abs/2609.31693) (Disentangle and Drop) | Splits the image into a grayscale carrier and a residual branch, then suppresses the residual, with one shared operating setting | SSL, DwtDct, DwtDctSvd, RivaGAN, StegaStamp, Stable Signature, Tree-Ring | Open schemes only |
|
||||
| Yang et al., [arXiv:2609.31722](https://arxiv.org/abs/2609.31722) (Push-Pull) | Key-free and decoder-free at deployment; average BER 0.3958 at 31.07 dB PSNR on a four-method sweep | Seven open methods. Not SynthID | A BER near 0.40 leaves much of the payload intact, as with SPFM-Net below |
|
||||
| Ding et al., [arXiv:2609.34003](https://arxiv.org/abs/2609.34003) (DeMark, audio) | Query-free black box that suppresses the time-frequency artifacts the embedding leaves; average attack success 0.92 and 0.96 on two speech sets | AudioSeal, WavMark, Timbre, AudioMarkNet. Not audio SynthID | A candidate cohort for the pinned AudioSeal oracle in the [benchmark kernel](benchmark-kernel.md). No code located |
|
||||
| Wen et al., [arXiv:2305.20030](https://arxiv.org/abs/2305.20030) (Tree-Ring) | In-generation Fourier pattern in initial noise | Not SynthID | Kill is DDIM inversion plus latent wipe, not a pixel scramble. Lin and Juarez [arXiv:2506.10502](https://arxiv.org/abs/2506.10502) (USENIX 2025) remove it from public knowledge |
|
||||
| Fernandez et al., Stable Signature, ICCV 2023 | Fine-tuned VAE decoder | Not SynthID | Regeneration that replaces the decoder is in-family. Our foreign VAE at 22.3 dB did not kill OpenAI SynthID, which is the expected mismatch |
|
||||
| Google to [The Verge, 2026-04-14](https://www.theverge.com/ai-artificial-intelligence/911579/google-synthid-ai-watermarking-system-reverse-engineered) | reverse-SynthID does not systematically remove SynthID | Google spokesperson on the spectral bypass | Agrees with our V4 audit and with Bypass: reconstruction or decoder confusion, not a payload delete |
|
||||
|
||||
Quiet pixel-only removal is not a published result on production
|
||||
OpenAI or Google SynthID (still true in the 2026-09-27 literature sweep).
|
||||
OpenAI or Google SynthID (still true in the 2026-10-17 literature sweep).
|
||||
Against open encoders the fidelity band has moved: WmForger
|
||||
([arXiv:2510.20468](https://arxiv.org/abs/2510.20468)) and Hide&Seek
|
||||
([arXiv:2603.01067](https://arxiv.org/abs/2603.01067)) report roughly 30 dB,
|
||||
|
||||
@@ -575,7 +575,11 @@ The SynthID decoder is proprietary and not released:
|
||||
> -- Gowal et al., arXiv:2510.09263
|
||||
|
||||
There are no released payload-decoder weights or public algorithm. Google
|
||||
provides verification in Gemini. The old limited SynthID Detector portal does
|
||||
provides verification in Gemini and, since the public SynthID Detector opened,
|
||||
at [synthid.com](https://synthid.com/). That site is the `synthid-web` oracle
|
||||
surface in [`provider-oracles.md`](provider-oracles.md). On 2026-10-17 its FAQ
|
||||
named Google DeepMind, Nvidia, OpenAI, and Kakao, and its settled labels were
|
||||
`Watermark detected` and `Watermark not detected`. The old limited portal does
|
||||
not work: on 2026-09-07, its official `labs.google/synthid` entry point
|
||||
redirected a signed-in browser to a Google 404. On 2026-09-23 the same entry
|
||||
point redirected to `deepmind.google.com/frontiers/synthid`, which also returned
|
||||
|
||||
@@ -262,8 +262,8 @@ and implemented by `scripts/provider_oracles.py`.
|
||||
|
||||
Proprietary watermark removal cannot be verified locally by design -- no public decoder
|
||||
exists. Each vendor has its own oracle and it covers only that vendor's content:
|
||||
the Content Provenance API first and `openai.com/verify` as an explicit fallback for
|
||||
OpenAI, the Gemini app for Google, and Microsoft's
|
||||
the Content Provenance API first and `openai.com/verify` for
|
||||
OpenAI, the Gemini app and the SynthID Detector at `synthid.com` for Google, and Microsoft's
|
||||
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
|
||||
for InvisMark. The Microsoft API reports `Watermark` and `C2PA` separately. It therefore
|
||||
needs a pixel-identical metadata-stripped control: the control must lose `C2PA` while
|
||||
|
||||
@@ -37,6 +37,12 @@ claim generator still falls back to the certificate issuer.
|
||||
Exact claim-generator mappings also cover `Grok Imagine` as xAI Grok Imagine
|
||||
and `Suno` as Suno. Both can carry a valid AI source claim without a separately
|
||||
readable issuer, so issuer-only attribution leaves the platform blank.
|
||||
Suno also [announced on 2026-08-06](https://suno.com/blog/building-the-future-of-music-responsibly)
|
||||
that it would adopt "new audio watermarking and fingerprinting technology" in
|
||||
the coming weeks. It did not name the technology
|
||||
([TechCrunch](https://techcrunch.com/2026/08/06/amid-legal-battles-suno-says-it-will-start-watermarking-songs/)
|
||||
asked whether it would be SynthID and got no answer), so there is no local
|
||||
detector, and metadata stripping leaves any such mark in the audio.
|
||||
|
||||
Some signers expose the product in `signature_info` while retaining a generic
|
||||
claim generator. xAI Grok Imagine is recognized from that signed identity even
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"$schema": "https://glama.ai/mcp/schemas/server.json",
|
||||
"maintainers": ["wiltodelta"]
|
||||
}
|
||||
@@ -4,7 +4,7 @@
|
||||
# on it, including the ComfyUI node package. The console script below carries
|
||||
# the same weight, since users have it on PATH.
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.44.0"
|
||||
version = "0.46.0"
|
||||
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11,<3.15"
|
||||
@@ -205,6 +205,9 @@ dev = [
|
||||
"httpx>=0.28,<1",
|
||||
# Development-only isolated browser automation for public provider oracles.
|
||||
"playwright>=1.62,<2",
|
||||
# The official MCP client, used only to test `remove-ai-watermarks mcp`
|
||||
# against the reference implementation. The server itself is stdlib-only.
|
||||
"mcp>=2.3,<3",
|
||||
"pytest>=8.0.0",
|
||||
"pytest-cov>=4.1.0",
|
||||
"pytest-xdist>=3.5.0",
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
from datetime import date
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -10,15 +10,15 @@ if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def corpus_day(path: Path, corpus: Path) -> datetime.date | None:
|
||||
def corpus_day(path: Path, corpus: Path) -> date | None:
|
||||
"""Read the leading YYYY-MM-DD corpus segment, if present."""
|
||||
try:
|
||||
return datetime.date.fromisoformat(path.relative_to(corpus).parts[0])
|
||||
return date.fromisoformat(path.relative_to(corpus).parts[0])
|
||||
except (ValueError, IndexError):
|
||||
return None
|
||||
|
||||
|
||||
def corpus_files(corpus: Path, since: datetime.date | None, *, suffixes: Collection[str] | None = None) -> list[Path]:
|
||||
def corpus_files(corpus: Path, since: date | None, *, suffixes: Collection[str] | None = None) -> list[Path]:
|
||||
"""Return a deterministic recursive walk, optionally date- and suffix-bounded."""
|
||||
if since is None:
|
||||
candidates = corpus.rglob("*")
|
||||
|
||||
@@ -23,7 +23,6 @@ from the provider's canonical name, the provider is named separately.
|
||||
| `samsung_en_alpha.png` | Samsung Galaxy AI label, English locale ("AI-generated content", bottom-left) | 5 corpus files | POS 0.11-0.30 vs NEG max 0.40 (binary); tophat/gray tried, no separation. The registered Italian engine scores 0.18-0.31 on the same files -- same layout class, wrong glyph template. |
|
||||
| `gemini_text_alpha.png` | "Generated with Gemini" text label (bottom-right; the registered gemini mark is the sparkle icon) | 3 corpus files | POS 0.07-0.22 vs NEG max 0.32; coverage gate finds the blob (0.29-0.35) but the Arial silhouette misses Google's letterforms. |
|
||||
| `notebooklm_alpha.png` | NotebookLM wordmark (bottom-right) | 12 corpus files | locate geometry not yet fitted; POS max 0.12. |
|
||||
| `dola_alpha.png` | DolaAI on images (the video mark is registered) | 12 corpus files | POS 0.11-0.21 vs NEG max 0.30. |
|
||||
| `mindvideo_alpha.png` | MindVideo.AI (top-right) | 11 corpus files | POS 0.29-0.32 vs NEG max 0.30 -- borderline overlap, not shippable. |
|
||||
| `higgsfield_alpha.png` | `HIGGSFIELD AI` wordmark (Higgsfield AI, bottom-right; the boxed `AI` variant shares the cohort) | 5 wordmark files (16 in the boxed-AI OCR cluster) | POS max 0.26 vs NEG max 0.22 -- no separation; the mark may be two-part (wordmark + boxed AI) and needs a composed template. |
|
||||
| `jianying_alpha.png` | `剪映AI` wordmark (Jianying / 剪映, the China product in the CapCut family; bottom-right) | 2 corpus files | POS 0.29 vs NEG max 0.35. |
|
||||
|
||||
|
Before Width: | Height: | Size: 3.4 KiB |
@@ -121,7 +121,7 @@ def load_pinned_models() -> tuple[object, object]:
|
||||
raw = convert_state_dict_for_scriptable_model(raw)
|
||||
model = create(config, device="cpu") # type: ignore[call-arg]
|
||||
load_state_dict(model, state_dict=raw)
|
||||
model.eval()
|
||||
model.train(False)
|
||||
return model
|
||||
|
||||
generator = load(GENERATOR_CARD, "generator_base.pth", AudioSealWMConfig, create_generator)
|
||||
|
||||
@@ -51,7 +51,7 @@ def export_onnx(checkpoint: Path, output: Path) -> None:
|
||||
if output.exists():
|
||||
raise SystemExit(f"refusing to overwrite existing output: {output}")
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
model = VisionEmbedding(load_headed_clip(checkpoint, torch.device("cpu"))).eval()
|
||||
model = VisionEmbedding(load_headed_clip(checkpoint, torch.device("cpu"))).train(False)
|
||||
export_options: dict[str, Any] = {}
|
||||
if "dynamo" in inspect.signature(torch.onnx.export).parameters:
|
||||
export_options["dynamo"] = False
|
||||
|
||||
@@ -252,7 +252,7 @@ def _lpips_model() -> tuple[Any, Any]:
|
||||
import torch
|
||||
|
||||
model = lpips.LPIPS(net="alex", verbose=False)
|
||||
model.eval()
|
||||
model.train(False)
|
||||
return model, torch
|
||||
|
||||
|
||||
|
||||
@@ -173,7 +173,7 @@ def load_model() -> object:
|
||||
raise RuntimeError(
|
||||
f"PixelSeal checkpoint/source mismatch: missing={loaded.missing_keys}, unexpected={loaded.unexpected_keys}"
|
||||
)
|
||||
model.eval()
|
||||
model.train(False)
|
||||
return model
|
||||
|
||||
|
||||
|
||||
@@ -288,8 +288,8 @@ def run_web_batch(
|
||||
oracles.verify_batch(manifest_path)
|
||||
manifest, results = oracles.load_batch(manifest_path)
|
||||
surface = manifest["surface"]
|
||||
if surface == "gemini-web":
|
||||
raise ValueError("gemini-web requires the real Chrome workflow, not isolated Playwright")
|
||||
if oracles.SURFACES[surface].automation == "real_browser":
|
||||
raise ValueError(f"{surface} requires the real Chrome workflow, not isolated Playwright")
|
||||
if surface not in PLAYWRIGHT_SURFACES:
|
||||
raise ValueError(f"surface {surface} is not a Playwright Web oracle")
|
||||
slot = manifest["slot"]
|
||||
|
||||
@@ -169,6 +169,26 @@ SURFACES: dict[str, OracleSurface] = {
|
||||
account_slots=True,
|
||||
network_slots=True,
|
||||
),
|
||||
"synthid-web": OracleSurface(
|
||||
key="synthid-web",
|
||||
provider_key="google",
|
||||
provider="Google",
|
||||
signal_family="synthid",
|
||||
surface="SynthID Detector",
|
||||
automation="real_browser",
|
||||
url="https://synthid.com/",
|
||||
media_types=("image", "video", "audio"),
|
||||
project_evidence=(),
|
||||
prompt=None,
|
||||
clean_result_note=(
|
||||
"Record detected only for the explicit label Watermark detected, and not_detected only for "
|
||||
"Watermark not detected. The portal reports partner SynthID from Google DeepMind, Nvidia, "
|
||||
"OpenAI, and Kakao without naming which partner. A not_detected result is not proof of "
|
||||
"human authorship. Preserve quota, sign-in, and unclear wording."
|
||||
),
|
||||
account_slots=True,
|
||||
network_slots=True,
|
||||
),
|
||||
"meta-web": OracleSurface(
|
||||
key="meta-web",
|
||||
provider_key="meta",
|
||||
@@ -278,6 +298,11 @@ def ordered_surfaces() -> list[OracleSurface]:
|
||||
return sorted(SURFACES.values(), key=_surface_order)
|
||||
|
||||
|
||||
def _automation_keys(automation: str) -> frozenset[str]:
|
||||
"""Return the surface keys that use one automation kind."""
|
||||
return frozenset(key for key, surface in SURFACES.items() if surface.automation == automation)
|
||||
|
||||
|
||||
def _inside(path: Path, parent: Path) -> bool:
|
||||
"""Return whether resolved PATH is inside resolved PARENT."""
|
||||
try:
|
||||
@@ -364,6 +389,8 @@ def load_slots(path: Path) -> dict[str, ExecutionSlot]:
|
||||
"api_key_env",
|
||||
}
|
||||
slots: dict[str, ExecutionSlot] = {}
|
||||
real_browser = _automation_keys("real_browser")
|
||||
playwright = _automation_keys("playwright")
|
||||
for index, raw_slot in enumerate(slot_items):
|
||||
if not isinstance(raw_slot, dict):
|
||||
raise ValueError(f"slot {index} must be a JSON object")
|
||||
@@ -407,19 +434,17 @@ def load_slots(path: Path) -> dict[str, ExecutionSlot]:
|
||||
raise ValueError(f"surface {surface_key} requires an account_label")
|
||||
if account_label is not None and not SURFACES[surface_key].account_slots:
|
||||
raise ValueError(f"surface {surface_key} does not use account slots")
|
||||
if surface_key == "gemini-web" and browser_profile is None:
|
||||
raise ValueError("surface gemini-web requires a browser_profile")
|
||||
if surface_key in real_browser and browser_profile is None:
|
||||
raise ValueError(f"surface {surface_key} requires a browser_profile")
|
||||
if surface_key == "gemini-web" and google_account_index is None:
|
||||
raise ValueError("surface gemini-web requires a google_account_index")
|
||||
if surface_key != "gemini-web" and google_account_index is not None:
|
||||
raise ValueError("only gemini-web uses google_account_index")
|
||||
if SURFACES[surface_key].automation == "api" and browser_profile is not None:
|
||||
raise ValueError(f"API surface {surface_key} does not use a browser_profile")
|
||||
if surface_key != "gemini-web" and browser_profile is not None:
|
||||
raise ValueError("only gemini-web uses a real browser_profile")
|
||||
if surface_key == "gemini-web" and proxy_url_env is not None:
|
||||
raise ValueError("surface gemini-web does not use proxy_url_env")
|
||||
if proxy_url_env is not None and surface_key not in {"openai-web", "microsoft-web", "meta-web"}:
|
||||
if surface_key not in real_browser and browser_profile is not None:
|
||||
raise ValueError("only a real-browser surface uses a browser_profile")
|
||||
if proxy_url_env is not None and surface_key not in playwright:
|
||||
raise ValueError(f"surface {surface_key} does not use proxy_url_env")
|
||||
if surface_key == "openai-api" and api_key_env is None:
|
||||
raise ValueError("surface openai-api requires an api_key_env")
|
||||
@@ -858,18 +883,17 @@ def list_oracles(as_json: bool) -> None:
|
||||
def plan(provider: str, as_json: bool) -> None:
|
||||
"""Show the API-first surface order for one provider."""
|
||||
definitions = provider_plan(provider)
|
||||
has_api = any(definition.automation == "api" for definition in definitions)
|
||||
rows = []
|
||||
for index, definition in enumerate(definitions):
|
||||
for definition in definitions:
|
||||
row = definition.to_dict()
|
||||
row["preference"] = "primary" if index == 0 else "fallback"
|
||||
row["preference"] = "fallback" if has_api and definition.automation != "api" else "primary"
|
||||
rows.append(row)
|
||||
if as_json:
|
||||
click.echo(json.dumps(rows, indent=2, sort_keys=True))
|
||||
return
|
||||
for row in rows:
|
||||
click.echo(f"{row['preference']}\t{row['key']}\t{row['automation']}\t{row['surface']}")
|
||||
if len(rows) > 1:
|
||||
click.echo("Web fallback requires a separate operator decision; failures do not trigger it automatically.")
|
||||
|
||||
|
||||
@cli.command()
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Rebuild synthetic silhouettes for additional AI disclosure layouts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
from scripts.render_vendor_silhouettes import render # noqa: E402
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
ASSETS = ROOT / "src" / "remove_ai_watermarks" / "assets"
|
||||
CJK = {"font": "/System/Library/Fonts/Hiragino Sans GB.ttc", "font_index": 2}
|
||||
|
||||
|
||||
def centered_lines(
|
||||
top: str, bottom: str, *, top_scale: float, bottom_height: float = 1.0, shear: float = 0.0
|
||||
) -> np.ndarray:
|
||||
upper, lower = render(top, opts=CJK), render(bottom, opts=CJK)
|
||||
if bottom_height != 1.0:
|
||||
lower = cv2.resize(lower, (upper.shape[1], round(upper.shape[0] * bottom_height)))
|
||||
else:
|
||||
upper = cv2.resize(upper, (round(lower.shape[1] * top_scale), round(lower.shape[0] * 0.95)))
|
||||
lines = [upper, lower]
|
||||
result = np.zeros((sum(line.shape[0] for line in lines) + 2, max(line.shape[1] for line in lines)), np.uint8)
|
||||
y = 0
|
||||
for line in lines:
|
||||
x = (result.shape[1] - line.shape[1]) // 2
|
||||
result[y : y + line.shape[0], x : x + line.shape[1]] = line
|
||||
y += line.shape[0] + 2
|
||||
if shear:
|
||||
h, w = result.shape
|
||||
extra = round(abs(shear) * h)
|
||||
result = cv2.warpAffine(result, np.float32([[1, shear, extra], [0, 1, 0]]), (w + extra, h))
|
||||
return result
|
||||
|
||||
|
||||
def silhouettes() -> dict[str, np.ndarray]:
|
||||
return {
|
||||
"workbuddy_alpha.png": centered_lines("AI生成", "WORKBUDDY>_", top_scale=1.0, bottom_height=0.45),
|
||||
"yuanbao_compact_alpha.png": centered_lines("元宝", "AI生成", top_scale=0.65, shear=-0.15),
|
||||
"yuanbao_compact_gray_alpha.png": centered_lines("元宝", "AI生成", top_scale=0.65, shear=-0.15),
|
||||
"doubao_outline_alpha.png": render("豆包AI生成", opts=CJK),
|
||||
"samsung_ko_alpha.png": render(
|
||||
"AI로 생성한 콘텐츠", opts={"font": "/System/Library/Fonts/AppleSDGothicNeo.ttc"}
|
||||
),
|
||||
"microsoft_text_alpha.png": render(
|
||||
"Made with AI", opts={"font": "/System/Library/Fonts/Supplemental/Arial.ttf"}
|
||||
),
|
||||
"jimeng_label_alpha.png": render("AI生成", opts={"font": "/System/Library/Fonts/STHeiti Medium.ttc"}),
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
for name, alpha in silhouettes().items():
|
||||
Image.fromarray(alpha).save(ASSETS / name)
|
||||
log.info("Rendered %s, %s x %s", name, alpha.shape[1], alpha.shape[0])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -73,7 +73,7 @@ MARKS = {
|
||||
"gemini_text_alpha.png": "Generated with Gemini",
|
||||
# Candidate wordmarks measured on a local evaluation corpus (unregistered).
|
||||
"notebooklm_alpha.png": "NotebookLM",
|
||||
"dola_alpha.png": "DolaAI",
|
||||
"dola_alpha.png": "Dola AI",
|
||||
"mindvideo_alpha.png": "MindVideo.AI",
|
||||
"higgsfield_alpha.png": "HIGGSFIELD AI",
|
||||
"capcut_alpha.png": "CapCut AI",
|
||||
@@ -92,6 +92,7 @@ _REGISTERED = {f"{key}_alpha.png" for key in mark_keys()} & MARKS.keys()
|
||||
|
||||
# Per-mark post-processing for the multi-line / slanted stamps (see render()).
|
||||
MARK_OPTS: dict[str, dict[str, Any]] = {
|
||||
"dola_alpha.png": {"font": "/System/Library/Fonts/Supplemental/Arial Bold.ttf"},
|
||||
# Hiragino Sans GB W6, tight leading, dilation, and negative shear match the
|
||||
# standard Yuanbao stamp without clipping the lower line.
|
||||
"yuanbao_alpha.png": {
|
||||
|
||||
@@ -98,7 +98,7 @@ def _train_detector(emb: dict[str, np.ndarray], catalog: dict) -> tuple[nn.Modul
|
||||
optimizer.zero_grad()
|
||||
loss_fn(model(features).squeeze(-1), labels).backward()
|
||||
optimizer.step()
|
||||
model.eval()
|
||||
model.train(False)
|
||||
dev = np.stack(
|
||||
[
|
||||
emb[row["sha256"]]
|
||||
@@ -161,7 +161,7 @@ def _train_provider(feat: dict[str, np.ndarray], catalog: dict) -> dict[str, nn.
|
||||
optimizer.zero_grad()
|
||||
_focal_loss(model(features).squeeze(-1), labels).backward()
|
||||
optimizer.step()
|
||||
heads[name] = model.eval()
|
||||
heads[name] = model.train(False)
|
||||
log.info("provider %s pos=%d", name, len(positive))
|
||||
return heads
|
||||
|
||||
|
||||
@@ -127,7 +127,7 @@ def load_model() -> object:
|
||||
import torch
|
||||
|
||||
model = torch.jit.load(str(checkpoint_path()), map_location="cpu")
|
||||
model.eval()
|
||||
model.train(False)
|
||||
return model
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
|
||||
"name": "io.github.wiltodelta/remove-ai-watermarks",
|
||||
"title": "Remove AI Watermarks",
|
||||
"description": "Identify and remove AI watermarks and provenance metadata from your own images and video.",
|
||||
"version": "0.46.0",
|
||||
"repository": {
|
||||
"url": "https://github.com/wiltodelta/remove-ai-watermarks",
|
||||
"source": "github"
|
||||
},
|
||||
"websiteUrl": "https://github.com/wiltodelta/remove-ai-watermarks/blob/main/docs/mcp-server.md",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "pypi",
|
||||
"registryBaseUrl": "https://pypi.org",
|
||||
"identifier": "remove-ai-watermarks",
|
||||
"version": "0.46.0",
|
||||
"runtimeHint": "uvx",
|
||||
"transport": {
|
||||
"type": "stdio"
|
||||
},
|
||||
"packageArguments": [
|
||||
{
|
||||
"type": "positional",
|
||||
"value": "mcp"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "remove-ai-watermarks",
|
||||
"displayName": "Remove AI Watermarks",
|
||||
"version": "1.0.10",
|
||||
"version": "1.0.12",
|
||||
"description": "Identify and remove AI provenance marks from images and video the user generated or edited, using the remove-ai-watermarks CLI.",
|
||||
"author": {
|
||||
"name": "wiltodelta",
|
||||
|
||||
@@ -5,7 +5,7 @@ license: Apache-2.0
|
||||
compatibility: Python 3.11-3.14. Invisible image removal needs NVIDIA CUDA. Some video writes need ffmpeg. Installer may be uv, pipx, or pip.
|
||||
metadata:
|
||||
author: wiltodelta
|
||||
version: "1.0.10"
|
||||
version: "1.0.12"
|
||||
homepage: https://raiw.cc
|
||||
repository: https://github.com/wiltodelta/remove-ai-watermarks
|
||||
---
|
||||
|
||||
@@ -21,8 +21,8 @@ remove-ai-watermarks visible image.png --keep-metadata -o clean.png
|
||||
```
|
||||
|
||||
Image mark keys: `gemini`, `doubao`, `jimeng`, `qwen`, `wan`, `kling`, `yuanbao`,
|
||||
`samsung`, `runninghub`, `baidu`, `liblib`, `liblib_pill`, `microsoft`,
|
||||
`generic_ai_label`, `openart`, `jimeng_pill`. `wan` is new (since CLI 0.43.0). Default
|
||||
`dola`, `workbuddy`, `samsung`, `runninghub`, `baidu`, `liblib`, `liblib_pill`, `microsoft`,
|
||||
`generic_ai_label`, `openart`, `jimeng_pill`. `dola` and `workbuddy` are new (since CLI 0.45.0). Default
|
||||
`--mark auto` removes every selected match. `microsoft` is the top-right AI
|
||||
badge, separate from the invisible InvisMark watermark on the same vendor's
|
||||
images. `jimeng_pill` is a weak detector and needs corroboration. The compact
|
||||
@@ -138,6 +138,13 @@ user names a device, and never to claim a device the probe did not report.
|
||||
Without `-o`, video commands write `<source>_clean` and keep the original.
|
||||
Output extension must match the input container.
|
||||
|
||||
## MCP server
|
||||
|
||||
`remove-ai-watermarks mcp` (since CLI 0.46.0) serves `identify`, `remove`
|
||||
and `batch` as MCP tools over stdin and stdout for MCP clients. It blocks
|
||||
waiting for protocol messages, so do not run it from this skill; call the
|
||||
commands above directly.
|
||||
|
||||
## Exit codes
|
||||
|
||||
| Code | Meaning |
|
||||
|
||||
@@ -32,7 +32,7 @@ _GPU_MARKERS = ("GPU ", "gpu:")
|
||||
# told an agent its CLI was current right before it typed a value that build
|
||||
# rejects. tests/test_agent_skill.py pins it against the package version and
|
||||
# against every "(since CLI X.Y.Z)" annotation in the skill.
|
||||
MIN_CLI_VERSION = (0, 43, 0)
|
||||
MIN_CLI_VERSION = (0, 46, 0)
|
||||
# Markers a build without the pixel stack prints. The first is the current CLI's own
|
||||
# install hint; the rest are what older builds emit before the guard existed, and the
|
||||
# Homebrew formula ships exactly such a build.
|
||||
|
||||
@@ -35,7 +35,7 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
|
||||
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
|
||||
|
||||
|
||||
__version__ = "0.44.0"
|
||||
__version__ = "0.46.0"
|
||||
|
||||
__all__ = [
|
||||
"BatchItemResult",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Generated C2PA soft-binding registry; do not edit by hand.
|
||||
|
||||
Source: https://raw.githubusercontent.com/c2pa-org/softbinding-algorithm-list/main/softbinding-algorithm-list.json
|
||||
Revision: d36d1c1742b1753ca064bb3307abfd95db912a28
|
||||
Revision: d5960248ab69c17dc194906d2ef7d11ad164be70
|
||||
Source license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
|
||||
Changes: validated, ordered, and reduced to fields used at runtime; descriptions
|
||||
are shortened to their first sentence for display.
|
||||
@@ -12,7 +12,7 @@ from __future__ import annotations
|
||||
C2PA_SOFT_BINDING_SOURCE_URL = (
|
||||
"https://raw.githubusercontent.com/c2pa-org/softbinding-algorithm-list/main/softbinding-algorithm-list.json"
|
||||
)
|
||||
C2PA_SOFT_BINDING_SOURCE_REVISION = "d36d1c1742b1753ca064bb3307abfd95db912a28"
|
||||
C2PA_SOFT_BINDING_SOURCE_REVISION = "d5960248ab69c17dc194906d2ef7d11ad164be70"
|
||||
C2PA_SOFT_BINDING_SOURCE_LICENSE = "CC BY 4.0"
|
||||
|
||||
C2PA_SOFT_BINDING_ROWS: tuple[
|
||||
@@ -632,4 +632,15 @@ C2PA_SOFT_BINDING_ROWS: tuple[
|
||||
(),
|
||||
False,
|
||||
),
|
||||
(
|
||||
54,
|
||||
"be.immostory.videoseal.1",
|
||||
"watermark",
|
||||
("video",),
|
||||
("video/mp4",),
|
||||
"ImmoStory AI deployment of Meta FAIR's Video Seal 1.0 invisible video watermark",
|
||||
"2026-10-12T00:00:00.000Z",
|
||||
(),
|
||||
False,
|
||||
),
|
||||
)
|
||||
@@ -67,7 +67,7 @@ def load_headed_clip(checkpoint: Path, device: torch.device) -> HeadedCLIP:
|
||||
model = HeadedCLIP(clip)
|
||||
state = torch.load(checkpoint, map_location="cpu", weights_only=True)
|
||||
model.load_state_dict(state, strict=True)
|
||||
return model.to(device).eval()
|
||||
return model.to(device).train(False)
|
||||
|
||||
|
||||
def load_processor() -> Any:
|
||||
|
||||
@@ -7,7 +7,9 @@ without re-encoding, the image path drops matching boxes and emits the rest
|
||||
verbatim. The streaming MP4/MOV/M4A path instead preserves all offsets by
|
||||
retyping matching boxes as ``free`` and blanking their payloads in place. The
|
||||
codestream (``mdat`` for ISOBMFF, ``jxlc`` / ``jxlp`` for JPEG-XL) is untouched,
|
||||
so pixel, video, and audio data is preserved bit-for-bit.
|
||||
so pixel, video, and audio data is preserved bit-for-bit. The one exception is
|
||||
Kling's SEI tag inside ``mdat``: its bytes are blanked in place, which no decoder
|
||||
reads, so every decoded frame stays identical.
|
||||
|
||||
TC260-PG-20257A video metadata is nested instead:
|
||||
``moov.udta.meta.keys/ilst``. Its detector seeks through those boxes without
|
||||
@@ -43,8 +45,11 @@ from remove_ai_watermarks.metadata import (
|
||||
C2PA_UUID,
|
||||
IPTC_AI_FIELD_MARKERS,
|
||||
IPTC_AI_MARKERS,
|
||||
KLING_SEI_TAG,
|
||||
LOCAL_GENERATION_METADATA_KEYS,
|
||||
MAX_TC260_VALUE_BYTES,
|
||||
app_generator_from_metadata,
|
||||
app_provenance_from_metadata,
|
||||
parse_tc260_aigc_json,
|
||||
)
|
||||
|
||||
@@ -347,6 +352,55 @@ def _ai_metadata_regions(
|
||||
return regions
|
||||
|
||||
|
||||
def _app_metadata_regions(
|
||||
stream: BinaryIO,
|
||||
file_size: int,
|
||||
) -> list[tuple[tuple[int, int] | None, int, int]]:
|
||||
"""Locate keyed or keyless values carrying an app-export provenance object.
|
||||
|
||||
Dreamina writes its export JSON under ordinary keys (``LvMetaInfo`` and a
|
||||
second copy under ``com.apple.quicktime.artwork``). The file-level scan reads
|
||||
it from the raw head bytes, so removal must blank every item it would match.
|
||||
"""
|
||||
return [
|
||||
(key_span, value_start, value_end)
|
||||
for _name, key_span, value_start, value_end, value in _metadata_items(stream, file_size)
|
||||
if app_provenance_from_metadata(value) or app_generator_from_metadata(value)
|
||||
]
|
||||
|
||||
|
||||
def _copy_finding_kling_sei(
|
||||
source: BinaryIO,
|
||||
target: BinaryIO,
|
||||
mdat_ranges: list[tuple[int, int]],
|
||||
) -> list[int]:
|
||||
"""Copy ``source`` to ``target`` and return each Kling SEI tag offset in ``mdat``.
|
||||
|
||||
Detection reads the tag from the head window, but removal covers every copy in
|
||||
the media data. The search rides on the copy, so the file is read once. Each
|
||||
window carries one byte less than the tag from the previous chunk, so a copy
|
||||
straddling a read boundary is found exactly once.
|
||||
"""
|
||||
keep = len(KLING_SEI_TAG) - 1
|
||||
offsets: list[int] = []
|
||||
tail = b""
|
||||
pos = 0
|
||||
while chunk := source.read(_STREAM_COPY_BYTES):
|
||||
target.write(chunk)
|
||||
if mdat_ranges:
|
||||
window = tail + chunk
|
||||
base = pos - len(tail)
|
||||
found = window.find(KLING_SEI_TAG)
|
||||
while found != -1:
|
||||
start = base + found
|
||||
if any(low <= start and start + len(KLING_SEI_TAG) <= high for low, high in mdat_ranges):
|
||||
offsets.append(start)
|
||||
found = window.find(KLING_SEI_TAG, found + 1)
|
||||
tail = window[-keep:]
|
||||
pos += len(chunk)
|
||||
return offsets
|
||||
|
||||
|
||||
def ai_metadata_tags(path: str | Path, *, strict: bool = False) -> dict[str, str]:
|
||||
"""Read keyed AI-generation metadata from an ISOBMFF container."""
|
||||
try:
|
||||
@@ -601,15 +655,16 @@ def strip_isobmff_media_file(
|
||||
|
||||
The output retains every box size and byte offset. A top-level C2PA/JUMBF or
|
||||
AI-label box is converted to a ``free`` box and its payload is zeroed; native
|
||||
TC260 key/value spans are blanked in place. Keeping the original lengths is
|
||||
required because removing a pre-``mdat`` box would invalidate absolute media
|
||||
offsets in an existing sample table.
|
||||
TC260, generation-key and app-export key/value spans, and the Kling SEI tag in
|
||||
``mdat``, are blanked in place. Keeping the original lengths is required
|
||||
because removing a pre-``mdat`` box would invalidate absolute media offsets in
|
||||
an existing sample table.
|
||||
|
||||
The source is copied in bounded chunks to a sibling temporary file and
|
||||
atomically published only after all patches succeed. A malformed top-level
|
||||
walk is fail-safe: the input is copied unchanged.
|
||||
|
||||
Returns ``(provenance_boxes_blanked, native_tc260_keys_blanked)``.
|
||||
Returns ``(provenance_boxes_blanked, native_tags_blanked)``.
|
||||
"""
|
||||
from pathlib import Path as _Path
|
||||
|
||||
@@ -627,12 +682,23 @@ def strip_isobmff_media_file(
|
||||
)
|
||||
tc260_regions = _tc260_aigc_regions(stream, file_size) if targets is not None else []
|
||||
ai_metadata_regions = _ai_metadata_regions(stream, file_size) if targets is not None else []
|
||||
app_regions = _app_metadata_regions(stream, file_size) if targets is not None else []
|
||||
mdat_ranges = (
|
||||
[
|
||||
(payload, end)
|
||||
for _start, end, box_type, payload in iter_file_boxes(stream, 0, file_size)
|
||||
if box_type == b"mdat"
|
||||
]
|
||||
if targets is not None
|
||||
else []
|
||||
)
|
||||
tc260_key_spans = {region[0] for region in tc260_regions if region[0] is not None}
|
||||
ai_metadata_key_spans = {region[0] for region in ai_metadata_regions}
|
||||
app_key_spans = {region[0] for region in app_regions if region[0] is not None}
|
||||
|
||||
with atomic_video_output(output_path) as temporary_path:
|
||||
with source_path.open("rb") as source_stream, temporary_path.open("r+b") as temporary:
|
||||
shutil.copyfileobj(source_stream, temporary, length=_STREAM_COPY_BYTES)
|
||||
kling_sei_offsets = _copy_finding_kling_sei(source_stream, temporary, mdat_ranges)
|
||||
if targets is not None:
|
||||
for box_start, payload_start, box_end in targets:
|
||||
temporary.seek(box_start + 4)
|
||||
@@ -647,6 +713,15 @@ def strip_isobmff_media_file(
|
||||
temporary.seek(key_span[0])
|
||||
temporary.write(b"free" + b" " * (key_span[1] - key_span[0] - 4))
|
||||
_overwrite_range(temporary, value_start, value_end, byte=b" ")
|
||||
for key_span, value_start, value_end in app_regions:
|
||||
if key_span is not None:
|
||||
temporary.seek(key_span[0])
|
||||
temporary.write(b"free" + b" " * (key_span[1] - key_span[0] - 4))
|
||||
_overwrite_range(temporary, value_start, value_end, byte=b" ")
|
||||
# Spaces keep the SEI message well-formed: its length fields stand,
|
||||
# and a 0x20 run cannot form an emulation-prevention sequence.
|
||||
for offset in kling_sei_offsets:
|
||||
_overwrite_range(temporary, offset, offset + len(KLING_SEI_TAG), byte=b" ")
|
||||
temporary.flush()
|
||||
os.fsync(temporary.fileno())
|
||||
shutil.copymode(source_path, temporary_path)
|
||||
@@ -657,7 +732,8 @@ def strip_isobmff_media_file(
|
||||
source_path,
|
||||
)
|
||||
return 0, 0
|
||||
return len(targets), len(tc260_key_spans | ai_metadata_key_spans)
|
||||
# Every copy of the Kling tag is one tag, however many samples repeat it.
|
||||
return len(targets), len(tc260_key_spans | ai_metadata_key_spans | app_key_spans) + bool(kling_sei_offsets)
|
||||
|
||||
|
||||
def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
|
||||
|
||||
@@ -47,6 +47,12 @@ _LATENT_GRID = 8
|
||||
# The reference graph loads the Qwen LoRA at 0.8; carrying that number to a different
|
||||
# LoRA on a different architecture would be imitation, not parity.
|
||||
SDXL_STEPS = 4
|
||||
# Below this floor the global stack streams its weights from CPU layer by layer; at or
|
||||
# above it the stack stays resident. The fp16 UNet, Canny ControlNet, both text
|
||||
# encoders and the VAE are ~8.8 GiB of weights, and the UNet and the ControlNet run
|
||||
# together on every step, so model-level offload cannot get under an 8 GiB card.
|
||||
# Benchmark in docs/module-internals.md, "CPU offload".
|
||||
RESIDENT_SDXL_GLOBAL_MIN_VRAM_GIB = 12.0
|
||||
|
||||
|
||||
def sdxl_target_size(width: int, height: int) -> tuple[int, int]:
|
||||
@@ -54,6 +60,17 @@ def sdxl_target_size(width: int, height: int) -> tuple[int, int]:
|
||||
return _target_size(width, height, _LATENT_GRID)
|
||||
|
||||
|
||||
def resolve_sdxl_global_residency(
|
||||
requested: bool | None,
|
||||
*,
|
||||
total_memory_gib: float,
|
||||
) -> bool:
|
||||
"""Keep the SDXL global stack resident when explicitly requested or safely sized."""
|
||||
if requested is not None:
|
||||
return requested
|
||||
return total_memory_gib >= RESIDENT_SDXL_GLOBAL_MIN_VRAM_GIB
|
||||
|
||||
|
||||
@dataclass
|
||||
class SdxlZImagePipeline(TwoStageZImagePipeline):
|
||||
"""Lazy runtime for the SDXL global stage plus the inherited face stage."""
|
||||
@@ -64,6 +81,12 @@ class SdxlZImagePipeline(TwoStageZImagePipeline):
|
||||
super().__post_init__()
|
||||
self._sdxl_pipe: Any = None
|
||||
|
||||
def _keep_global_models_resident(self) -> bool:
|
||||
return resolve_sdxl_global_residency(
|
||||
self.keep_global_models_on_device,
|
||||
total_memory_gib=self._total_vram_gib(),
|
||||
)
|
||||
|
||||
def _load_global(self) -> Any:
|
||||
if self._sdxl_pipe is not None:
|
||||
return self._sdxl_pipe
|
||||
@@ -89,11 +112,19 @@ class SdxlZImagePipeline(TwoStageZImagePipeline):
|
||||
variant="fp16",
|
||||
add_watermarker=False,
|
||||
**token,
|
||||
).to(self.device)
|
||||
)
|
||||
resident = self._keep_global_models_resident()
|
||||
if resident:
|
||||
pipe = pipe.to(self.device)
|
||||
pipe.load_lora_weights(hf_hub_download(SDXL_LIGHTNING_MODEL_ID, SDXL_LIGHTNING_PATTERN, **token))
|
||||
pipe.fuse_lora()
|
||||
# SDXL-Lightning is distilled against trailing timestep spacing.
|
||||
pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
|
||||
if not resident:
|
||||
# After the fuse: the offload hooks keep their own CPU copy of each weight
|
||||
# and stream that copy, so fusing later would never reach what runs.
|
||||
self._progress("Streaming the SDXL stack from CPU (card below the residency floor)...")
|
||||
pipe.enable_sequential_cpu_offload(device=self.device, gpu_id=torch.cuda.current_device())
|
||||
self._sdxl_pipe = pipe
|
||||
return pipe
|
||||
|
||||
|
||||
@@ -823,7 +823,7 @@ class TwoStageZImagePipeline:
|
||||
torch_dtype=self._face_stage_dtype(),
|
||||
**kwargs,
|
||||
).to(self.device)
|
||||
model.eval()
|
||||
model.train(False)
|
||||
self._sam_model = model
|
||||
self._sam_processor = processor
|
||||
return model, processor
|
||||
|
||||
@@ -63,13 +63,13 @@ PROFILE_ADAPTIVE_POLISH = {
|
||||
|
||||
# Which profiles let ``--cpu-offload`` reach their GLOBAL stack. The face stage
|
||||
# honours the flag everywhere, because residency lives on the shared base, but the
|
||||
# global stack is each profile's own: only qwen-zimage streams it, through
|
||||
# ``_qwen_vram_config``. chroma-zimage and sdxl-zimage load theirs with a plain
|
||||
# ``.to(device)``, so on a card below the face-stage residency floor the flag has
|
||||
# nothing left to change and the run behaves as if it were absent. Data rather than
|
||||
# a subclass attribute, because the caller that must warn cannot import a profile
|
||||
# module without pulling in Diffusers.
|
||||
GLOBAL_OFFLOAD_PROFILES = frozenset({QWEN_ZIMAGE_PROFILE})
|
||||
# global stack is each profile's own: qwen-zimage streams it through
|
||||
# ``_qwen_vram_config`` and sdxl-zimage through Diffusers sequential offload.
|
||||
# chroma-zimage loads its stack with a plain ``.to(device)``, so on a card below the
|
||||
# face-stage residency floor the flag has nothing left to change there and the run
|
||||
# behaves as if it were absent. Data rather than a subclass attribute, because the
|
||||
# caller that must warn cannot import a profile module without pulling in Diffusers.
|
||||
GLOBAL_OFFLOAD_PROFILES = frozenset({QWEN_ZIMAGE_PROFILE, SDXL_ZIMAGE_PROFILE})
|
||||
|
||||
SDXL_LIGHTNING_MODEL_ID = "ByteDance/SDXL-Lightning"
|
||||
SDXL_LIGHTNING_PATTERN = "sdxl_lightning_4step_lora.safetensors"
|
||||
|
||||
@@ -175,9 +175,9 @@ class WatermarkRemover:
|
||||
return
|
||||
logger.warning(
|
||||
"--cpu-offload does not reach the global stack of the '%s' pipeline: only "
|
||||
"qwen-zimage streams its global model between CUDA calls. The face stage "
|
||||
"still honours the flag, so below the face-stage residency floor this run "
|
||||
"behaves as if --cpu-offload were absent.",
|
||||
"qwen-zimage and sdxl-zimage stream their global model between CUDA calls. "
|
||||
"The face stage still honours the flag, so below the face-stage residency "
|
||||
"floor this run behaves as if --cpu-offload were absent.",
|
||||
self.model_profile,
|
||||
)
|
||||
|
||||
|
||||
@@ -181,6 +181,7 @@ class TextMarkDetection:
|
||||
# ``footprint_mask`` reuses a threaded detection only when it matches the STRICT
|
||||
# level its own re-detect would have used -- see TextMarkEngine._strict_detection.
|
||||
provenance: bool = False
|
||||
template_asset: str | None = None
|
||||
|
||||
|
||||
def best_detection(*detections: TextMarkDetection) -> TextMarkDetection:
|
||||
@@ -665,7 +666,7 @@ class TextMarkEngine:
|
||||
def _verdict(self, scan: TextMarkScan, *, provenance: bool) -> TextMarkDetection:
|
||||
"""Apply the trust-level-dependent tail to a scan, as a fresh result object."""
|
||||
c = self.config
|
||||
det = TextMarkDetection(provenance=provenance)
|
||||
det = TextMarkDetection(provenance=provenance, template_asset=c.asset_name)
|
||||
if scan.loc is None or scan.box is None:
|
||||
return det
|
||||
det.region = scan.loc.bbox
|
||||
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Combine independently gated text layouts and retain their mask geometry."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import cv2
|
||||
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
BottomRightAnchoredEngine,
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkLocation,
|
||||
best_detection,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
|
||||
|
||||
class TextMarkVariantsMixin(TextMarkEngine):
|
||||
"""Add independently configured layouts to an existing text-mark engine."""
|
||||
|
||||
_variants: tuple[TextMarkEngine, ...] = ()
|
||||
config: TextMarkConfig
|
||||
|
||||
def detect_both(self, image: NDArray[Any] | None) -> tuple[TextMarkDetection, TextMarkDetection]:
|
||||
pairs = [super().detect_both(image), *(engine.detect_both(image) for engine in self._variants)]
|
||||
return best_detection(*(pair[0] for pair in pairs)), best_detection(*(pair[1] for pair in pairs))
|
||||
|
||||
def detect(self, image: NDArray[Any], *, provenance: bool = False) -> TextMarkDetection:
|
||||
return self.detect_both(image)[int(provenance)]
|
||||
|
||||
def variant_engine(self, detection: TextMarkDetection) -> TextMarkEngine:
|
||||
for engine in self._variants:
|
||||
if engine.config.asset_name == detection.template_asset:
|
||||
return engine
|
||||
return self
|
||||
|
||||
def footprint_mask(
|
||||
self,
|
||||
image: NDArray[Any] | None,
|
||||
*,
|
||||
force: bool = False,
|
||||
dilate: int | None = None,
|
||||
detection: TextMarkDetection | None = None,
|
||||
) -> NDArray[Any] | None:
|
||||
if image is None or image.size == 0 or force:
|
||||
return super().footprint_mask(image, force=force, dilate=dilate, detection=detection)
|
||||
det = detection if detection is not None else self.detect(image)
|
||||
engine = self.variant_engine(det)
|
||||
if engine is self:
|
||||
return super().footprint_mask(image, force=False, dilate=dilate, detection=det)
|
||||
return engine.footprint_mask(image, force=False, dilate=dilate, detection=det)
|
||||
|
||||
|
||||
class GlyphTextMarkEngine(TextMarkEngine):
|
||||
"""Use the accepted template's aligned strokes instead of a textured ROI."""
|
||||
|
||||
def footprint_mask(
|
||||
self,
|
||||
image: NDArray[Any] | None,
|
||||
*,
|
||||
force: bool = False,
|
||||
dilate: int | None = None,
|
||||
detection: TextMarkDetection | None = None,
|
||||
) -> NDArray[Any] | None:
|
||||
from remove_ai_watermarks._text_mark_engine import load_alpha_template
|
||||
|
||||
if force:
|
||||
return super().footprint_mask(image, force=True, dilate=dilate, detection=detection)
|
||||
if image is None or image.size == 0:
|
||||
return None
|
||||
det = detection if detection is not None else self.detect(image)
|
||||
alpha = load_alpha_template(self.config.asset_name)
|
||||
if alpha is None or not det.detected:
|
||||
return None
|
||||
return self._aligned_alpha_mask(
|
||||
image,
|
||||
det,
|
||||
alpha,
|
||||
alpha_floor=0.05,
|
||||
dilate=2 if dilate is None else max(0, dilate),
|
||||
)
|
||||
|
||||
|
||||
class BottomRightGlyphTextMarkEngine(GlyphTextMarkEngine, BottomRightAnchoredEngine):
|
||||
"""Glyph-mask variant whose accepted match must hug the bottom-right corner."""
|
||||
|
||||
_ANCHOR_MAX_RIGHT = 0.04
|
||||
_ANCHOR_MAX_BOTTOM = 0.04
|
||||
|
||||
def _ladder_best(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Find the strongest match among placements that satisfy the anchor."""
|
||||
response = self._detect_response(image, loc)
|
||||
silhouette = self._glyph_silhouette()
|
||||
if response is None or silhouette is None:
|
||||
return 0.0, None
|
||||
frame_h, frame_w = image.shape[:2]
|
||||
anchor_base = min(frame_h, frame_w)
|
||||
base = self.scale_base(image)
|
||||
best_score = 0.0
|
||||
best_box: tuple[int, int, int, int] | None = None
|
||||
for scale in self.config.ladder:
|
||||
glyph_w = max(self.config.min_gw, int(self.config.alpha_width_frac * base * scale))
|
||||
glyph_h = max(4, int(self.config.alpha_height_frac * base * scale))
|
||||
if glyph_w >= response.shape[1] or glyph_h >= response.shape[0]:
|
||||
continue
|
||||
template = cv2.resize(silhouette, (glyph_w, glyph_h), interpolation=cv2.INTER_AREA)
|
||||
scores = cv2.matchTemplate(response, template, cv2.TM_CCOEFF_NORMED)
|
||||
min_x = max(0, round(frame_w - loc.x - glyph_w - self._ANCHOR_MAX_RIGHT * anchor_base))
|
||||
max_x = min(scores.shape[1] - 1, frame_w - loc.x - glyph_w)
|
||||
min_y = max(0, round(frame_h - loc.y - glyph_h - self._ANCHOR_MAX_BOTTOM * anchor_base))
|
||||
max_y = min(scores.shape[0] - 1, frame_h - loc.y - glyph_h)
|
||||
if min_x > max_x or min_y > max_y:
|
||||
continue
|
||||
anchored = scores[min_y : max_y + 1, min_x : max_x + 1]
|
||||
_, score, _, point = cv2.minMaxLoc(anchored)
|
||||
if score > best_score:
|
||||
x, y = min_x + int(point[0]), min_y + int(point[1])
|
||||
best_score = float(score)
|
||||
best_box = (x, y, x + glyph_w - 1, y + glyph_h - 1)
|
||||
return best_score, best_box
|
||||
|
||||
|
||||
def _box_footprint(
|
||||
engine: GlyphTextMarkEngine,
|
||||
image: NDArray[Any] | None,
|
||||
*,
|
||||
force: bool,
|
||||
dilate: int | None,
|
||||
detection: TextMarkDetection | None,
|
||||
pad_left_height: float,
|
||||
pad_right_height: float,
|
||||
) -> NDArray[Any] | None:
|
||||
"""Mask the full winning text rectangle for outlined or stacked layouts."""
|
||||
if image is None or image.size == 0 or force:
|
||||
return GlyphTextMarkEngine.footprint_mask(
|
||||
engine,
|
||||
image,
|
||||
force=force,
|
||||
dilate=dilate,
|
||||
detection=detection,
|
||||
)
|
||||
det = detection if detection is not None else engine.detect(image)
|
||||
if not det.detected or det.match_box is None:
|
||||
return None
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
loc = engine.locate(image)
|
||||
x0, y0, x1, y1 = det.match_box
|
||||
extra_left = round((y1 - y0 + 1) * pad_left_height)
|
||||
extra_right = round((y1 - y0 + 1) * pad_right_height)
|
||||
pad = 2 if dilate is None else max(0, dilate)
|
||||
return region_eraser.boxes_to_mask(
|
||||
image.shape[:2],
|
||||
[
|
||||
(
|
||||
loc.x + x0 - extra_left,
|
||||
loc.y + y0,
|
||||
x1 - x0 + 1 + extra_left + extra_right,
|
||||
y1 - y0 + 1,
|
||||
)
|
||||
],
|
||||
dilate=pad,
|
||||
)
|
||||
|
||||
|
||||
class BoxTextMarkEngine(GlyphTextMarkEngine):
|
||||
"""Text detector with a solid detector-aligned removal footprint."""
|
||||
|
||||
_BOX_PAD_LEFT_HEIGHT = 0.0
|
||||
_BOX_PAD_RIGHT_HEIGHT = 0.0
|
||||
|
||||
def footprint_mask(
|
||||
self,
|
||||
image: NDArray[Any] | None,
|
||||
*,
|
||||
force: bool = False,
|
||||
dilate: int | None = None,
|
||||
detection: TextMarkDetection | None = None,
|
||||
) -> NDArray[Any] | None:
|
||||
return _box_footprint(
|
||||
self,
|
||||
image,
|
||||
force=force,
|
||||
dilate=dilate,
|
||||
detection=detection,
|
||||
pad_left_height=self._BOX_PAD_LEFT_HEIGHT,
|
||||
pad_right_height=self._BOX_PAD_RIGHT_HEIGHT,
|
||||
)
|
||||
|
||||
|
||||
class BottomRightBoxTextMarkEngine(BottomRightGlyphTextMarkEngine):
|
||||
"""Anchored text detector with a solid detector-aligned footprint."""
|
||||
|
||||
_BOX_PAD_LEFT_HEIGHT = 0.0
|
||||
_BOX_PAD_RIGHT_HEIGHT = 0.0
|
||||
|
||||
def footprint_mask(
|
||||
self,
|
||||
image: NDArray[Any] | None,
|
||||
*,
|
||||
force: bool = False,
|
||||
dilate: int | None = None,
|
||||
detection: TextMarkDetection | None = None,
|
||||
) -> NDArray[Any] | None:
|
||||
return _box_footprint(
|
||||
self,
|
||||
image,
|
||||
force=force,
|
||||
dilate=dilate,
|
||||
detection=detection,
|
||||
pad_left_height=self._BOX_PAD_LEFT_HEIGHT,
|
||||
pad_right_height=self._BOX_PAD_RIGHT_HEIGHT,
|
||||
)
|
||||
|
After Width: | Height: | Size: 2.8 KiB |
|
After Width: | Height: | Size: 3.7 KiB |
|
After Width: | Height: | Size: 4.1 KiB |
|
After Width: | Height: | Size: 2.9 KiB |
|
After Width: | Height: | Size: 2.8 KiB |
|
After Width: | Height: | Size: 9.5 KiB |
|
After Width: | Height: | Size: 8.3 KiB |
|
After Width: | Height: | Size: 8.3 KiB |
@@ -467,7 +467,7 @@ def _load_runtime(device: Any, backend: ClassifierBackend = "torch") -> _Runtime
|
||||
probe = np.load(_weight_path(folder, PROBE_FILE))
|
||||
detector = _build_detector()
|
||||
detector.load_state_dict(torch.load(_weight_path(folder, DETECTOR_FILE), map_location="cpu", weights_only=True))
|
||||
detector.to(device).eval()
|
||||
detector.to(device).train(False)
|
||||
packed = torch.load(_weight_path(folder, PROVIDER_FILE), map_location="cpu", weights_only=True)
|
||||
heads = {}
|
||||
for name in PROVIDER_LABELS:
|
||||
@@ -476,7 +476,7 @@ def _load_runtime(device: Any, backend: ClassifierBackend = "torch") -> _Runtime
|
||||
continue
|
||||
head = _build_provider_head()
|
||||
head.load_state_dict(packed[name])
|
||||
heads[name] = head.to(device).eval()
|
||||
heads[name] = head.to(device).train(False)
|
||||
ridge_threshold = float(probe["thr_oi_1pct"])
|
||||
mlp_threshold, provider_margin = _operating_point(folder)
|
||||
log.info("loaded photo-classify freeze from %s", folder)
|
||||
|
||||
@@ -308,8 +308,9 @@ _cpu_offload_option = click.option(
|
||||
"Offload model components to CPU between CUDA calls instead of keeping the "
|
||||
"whole pipeline in VRAM, at the cost of speed. Forces the face stack to "
|
||||
"offload instead of using automatic residency, on every profile. It reaches "
|
||||
"the global stack of qwen-zimage only: chroma-zimage and sdxl-zimage keep "
|
||||
"theirs resident either way."
|
||||
"the global stack of qwen-zimage and sdxl-zimage, which also stream theirs "
|
||||
"automatically on a card too small to hold it; chroma-zimage keeps its "
|
||||
"global stack resident either way."
|
||||
),
|
||||
)
|
||||
|
||||
@@ -914,6 +915,11 @@ def cmd_invisible(
|
||||
source = _validate_image(source)
|
||||
if output is None:
|
||||
output = source.with_stem(source.stem + "_clean")
|
||||
# The scrub gate reads an undecodable file as "no target", which would exit 2
|
||||
# (nothing to remove) for a file this command never looked at.
|
||||
if image_io.imread(source) is None:
|
||||
console.print(f"Error: Failed to read image: {source}")
|
||||
raise SystemExit(1)
|
||||
|
||||
# An explicit --vendor wins over detection (see the option help) and implies the
|
||||
# scrub runs: naming the cohort asserts the pixel watermark is present, so the
|
||||
@@ -1905,5 +1911,19 @@ def cmd_batch(
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
# ── MCP server ──
|
||||
@main.command("mcp")
|
||||
def cmd_mcp() -> None:
|
||||
"""Run a Model Context Protocol server on stdin/stdout.
|
||||
|
||||
Exposes identify, remove and batch as MCP tools for agent clients. Each tool
|
||||
runs one command of this CLI, so results and exit codes match it. Start it as
|
||||
`uvx remove-ai-watermarks mcp` from the client's server configuration.
|
||||
"""
|
||||
from remove_ai_watermarks.mcp_server import serve
|
||||
|
||||
serve()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,35 @@
|
||||
"""Detect the bottom-right Dola AI image disclosure, independently of video marks."""
|
||||
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig
|
||||
from remove_ai_watermarks._text_mark_variants import BoxTextMarkEngine
|
||||
|
||||
_CONFIG = TextMarkConfig(
|
||||
name="Dola AI image disclosure",
|
||||
asset_name="dola_alpha.png",
|
||||
corner="br",
|
||||
margin_floor=4,
|
||||
width_frac=0.28,
|
||||
height_frac=0.09,
|
||||
margin_x_frac=0.002,
|
||||
margin_bottom_frac=0.002,
|
||||
max_saturation=255,
|
||||
logo_min_luma=110,
|
||||
tophat_delta=8,
|
||||
morph_open_size=3,
|
||||
detect_min_coverage=0.01,
|
||||
detect_ncc_threshold=0.50,
|
||||
alpha_width_frac=0.10,
|
||||
alpha_height_frac=0.10 * 86 / 347,
|
||||
detect_frontend="tophat",
|
||||
scale_basis="width",
|
||||
ladder=(0.95, 1.0, 1.05, 1.20, 1.26),
|
||||
provenance_ncc_factor=1.0,
|
||||
min_gw=24,
|
||||
)
|
||||
|
||||
|
||||
class DolaEngine(BoxTextMarkEngine):
|
||||
"""Detect and localize the Dola AI image wordmark."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
@@ -25,10 +25,12 @@ from __future__ import annotations
|
||||
import dataclasses
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine, image_io
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine, TextMarkScan
|
||||
from remove_ai_watermarks._text_mark_variants import BottomRightBoxTextMarkEngine, TextMarkVariantsMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -117,6 +119,41 @@ _CONFIG = TextMarkConfig(
|
||||
_TINTED_ENGINE = TextMarkEngine(dataclasses.replace(_CONFIG, max_saturation=_TINTED_MAX_SATURATION))
|
||||
|
||||
|
||||
class _OutlinedDoubaoEngine(BottomRightBoxTextMarkEngine):
|
||||
# The outlined layout's provenance-relaxed NCC floor overlaps ordinary corner
|
||||
# texture. Require bright neutral strokes and edge detail in the winning box
|
||||
# to distinguish the label from unmarked corners. Metadata confirms
|
||||
# Doubao as the producer, not that the export retained a visible label.
|
||||
_MIN_BRIGHT_NEUTRAL_FRACTION = 0.29
|
||||
_MIN_MEAN_LAPLACIAN = 10.0
|
||||
_BOX_PAD_LEFT_HEIGHT = 1.2
|
||||
_BOX_PAD_RIGHT_HEIGHT = 0.7
|
||||
|
||||
@classmethod
|
||||
def _outline_pixels_ok(cls, patch: NDArray[Any]) -> bool:
|
||||
if patch.size == 0:
|
||||
return False
|
||||
patch = image_io.to_bgr(patch)
|
||||
gray = cv2.cvtColor(patch, cv2.COLOR_BGR2GRAY)
|
||||
saturation = cv2.cvtColor(patch, cv2.COLOR_BGR2HSV)[:, :, 1]
|
||||
bright_neutral = float(np.mean((gray > 160) & (saturation < 60)))
|
||||
edge_detail = float(np.mean(np.abs(cv2.Laplacian(gray, cv2.CV_32F))))
|
||||
return bright_neutral >= cls._MIN_BRIGHT_NEUTRAL_FRACTION and edge_detail >= cls._MIN_MEAN_LAPLACIAN
|
||||
|
||||
def _scan(self, image: NDArray[Any] | None) -> TextMarkScan:
|
||||
scan = super()._scan(image)
|
||||
if image is None or scan.loc is None or scan.match_box is None or scan.score is None:
|
||||
return scan
|
||||
x0, y0, x1, y1 = scan.match_box
|
||||
patch = image[
|
||||
scan.loc.y + y0 : scan.loc.y + y1 + 1,
|
||||
scan.loc.x + x0 : scan.loc.x + x1 + 1,
|
||||
]
|
||||
if not self._outline_pixels_ok(patch):
|
||||
return dataclasses.replace(scan, score=0.0)
|
||||
return scan
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Doubao alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
@@ -132,11 +169,29 @@ def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class DoubaoEngine(TextMarkEngine):
|
||||
class DoubaoEngine(TextMarkVariantsMixin):
|
||||
"""Detect/localize the visible Doubao "豆包AI生成" watermark (locate -> mask; mask feeds the fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
self._variants = (
|
||||
_OutlinedDoubaoEngine(
|
||||
dataclasses.replace(
|
||||
_CONFIG,
|
||||
asset_name="doubao_outline_alpha.png",
|
||||
width_frac=0.28,
|
||||
height_frac=0.13,
|
||||
scale_basis="width",
|
||||
alpha_width_frac=0.14,
|
||||
alpha_height_frac=0.14 * 0.75 * 77 / 346,
|
||||
max_saturation=255,
|
||||
detect_frontend="contrast",
|
||||
ladder=(0.95, 1.0, 1.05),
|
||||
detect_ncc_threshold=0.50,
|
||||
provenance_ncc_factor=0.25,
|
||||
)
|
||||
),
|
||||
)
|
||||
|
||||
def _scan(self, image: NDArray[Any] | None) -> TextMarkScan:
|
||||
base = super()._scan(image)
|
||||
@@ -179,12 +234,15 @@ class DoubaoEngine(TextMarkEngine):
|
||||
det = detection if detection is not None else self.detect(image)
|
||||
if not det.detected or det.match_box is None:
|
||||
return super().footprint_mask(image, force=False, dilate=dilate, detection=det)
|
||||
alpha = _alpha_template()
|
||||
engine = self.variant_engine(det)
|
||||
if engine is not self:
|
||||
return engine.footprint_mask(image, force=False, dilate=dilate, detection=det)
|
||||
alpha = _text_mark_engine.load_alpha_template(engine.config.asset_name)
|
||||
if alpha is None:
|
||||
return super().footprint_mask(image, force=False, dilate=dilate, detection=det)
|
||||
|
||||
radius = _FOOTPRINT_DILATE if dilate is None else max(0, dilate)
|
||||
return self._aligned_alpha_mask(
|
||||
return engine._aligned_alpha_mask(
|
||||
image,
|
||||
det,
|
||||
alpha,
|
||||
|
||||
@@ -957,7 +957,8 @@ def _integrity_clashes(
|
||||
clashes.append(
|
||||
"Conflicting AI-origin attributions from independent signals: "
|
||||
+ " vs ".join(parts)
|
||||
+ " -- one provenance set was likely spoofed, transplanted, or laundered."
|
||||
+ " -- these fields may describe different stages of an edit chain; "
|
||||
"the available evidence does not establish which attribution describes the final pixels."
|
||||
)
|
||||
|
||||
# Rule 2: a camera-capture C2PA device next to an AI-generation marker. Only
|
||||
@@ -967,13 +968,15 @@ def _integrity_clashes(
|
||||
# AI edit in ONE manifest, so the AI vendor is named only from that same
|
||||
# manifest (C2PA issuer + SynthID provenance) -- a legitimate edit chain, not a
|
||||
# spoof. An EXIF/XMP generator, IPTC field, TC260 AIGC label, or second
|
||||
# manifest naming AI on a camera capture is the real laundering tell.
|
||||
# manifest naming AI on a camera capture records a second stage whose relation
|
||||
# to the capture cannot be resolved from these fields alone.
|
||||
independent_ai_marker = any(grp != _C2PA_MANIFEST_SOURCE for grp in source.values())
|
||||
if camera_label and camera_has_ai_marker and independent_ai_marker:
|
||||
vendors = ", ".join(sorted(set(ai_vendors.values()))) or "present"
|
||||
clashes.append(
|
||||
f"Camera-capture C2PA credentials ({camera_label}) coexist with AI-generation markers "
|
||||
f"({vendors}) -- a genuine camera capture is not AI-generated, so the provenance is inconsistent."
|
||||
f"({vendors}) -- these fields may describe a captured image followed by an AI edit; "
|
||||
"the available evidence does not establish which stage describes the final pixels."
|
||||
)
|
||||
|
||||
return clashes
|
||||
|
||||
@@ -172,7 +172,8 @@ class InvisibleEngine:
|
||||
cpu_offload: Offload model components to CPU between CUDA calls instead
|
||||
of keeping the whole pipeline in VRAM, at the cost of speed. Forces
|
||||
the face stack to offload instead of using automatic residency, on
|
||||
every profile; reaches the global stack of qwen-zimage only.
|
||||
every profile; reaches the global stack of qwen-zimage and
|
||||
sdxl-zimage, not chroma-zimage.
|
||||
CUDA only.
|
||||
"""
|
||||
|
||||
@@ -310,6 +311,15 @@ class InvisibleEngine:
|
||||
with Image.open(image_path) as opened:
|
||||
image = ImageOps.exif_transpose(opened)
|
||||
orig_size = image.size # (width, height)
|
||||
# Diffusion only processes RGB. Keep alpha at the original display geometry
|
||||
# (after EXIF, before any resize) and rejoin it after all RGB postprocessing.
|
||||
# Palette and colour-key PNGs can carry transparency without an A band.
|
||||
alpha = (
|
||||
image.convert("RGBA").getchannel("A")
|
||||
if output_path.suffix.lower() in image_io.ALPHA_FORMATS
|
||||
and ("A" in image.getbands() or "transparency" in image.info)
|
||||
else None
|
||||
)
|
||||
# Full-res original, kept for the adaptive-polish detail target (image is
|
||||
# reassigned to the resized copy below; PIL resize returns a new object).
|
||||
reference_pil = image
|
||||
@@ -368,7 +378,7 @@ class InvisibleEngine:
|
||||
# Route those outputs through the same final resize so --no-adaptive-polish does
|
||||
# not silently change e.g. 1448x1086 into 1448x1080.
|
||||
needs_restore = target is not None or any(dimension % 8 for dimension in orig_size)
|
||||
if humanize > 0.0 or unsharp > 0.0 or adaptive_polish or needs_restore:
|
||||
if humanize > 0.0 or unsharp > 0.0 or adaptive_polish or needs_restore or alpha is not None:
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
@@ -389,7 +399,13 @@ class InvisibleEngine:
|
||||
progress=self._progress_callback,
|
||||
)
|
||||
|
||||
if not image_io.imwrite(out_path, out_cv, display_tags_from=image_path, orientation_applied=True):
|
||||
if not image_io.write_bgr_with_alpha(
|
||||
out_path,
|
||||
out_cv,
|
||||
np.asarray(alpha) if alpha is not None else None,
|
||||
display_tags_from=image_path,
|
||||
orientation_applied=True,
|
||||
):
|
||||
raise OSError(f"failed to write postprocessed output: {out_path}")
|
||||
|
||||
return out_path
|
||||
|
||||
@@ -56,6 +56,7 @@ from dataclasses import replace
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
@@ -136,8 +137,12 @@ _CONFIG = TextMarkConfig(
|
||||
_LATIN_CONFIG = replace(
|
||||
_CONFIG,
|
||||
asset_name="kling_latin_alpha.png",
|
||||
alpha_width_frac=0.095,
|
||||
alpha_height_frac=0.028,
|
||||
width_frac=0.22,
|
||||
height_frac=0.07,
|
||||
margin_x_frac=0.002,
|
||||
margin_bottom_frac=0.002,
|
||||
alpha_width_frac=0.14,
|
||||
alpha_height_frac=0.032,
|
||||
ladder=(0.9, 1.0, 1.1),
|
||||
detect_ncc_threshold=0.40,
|
||||
)
|
||||
@@ -153,12 +158,37 @@ def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
class KlingEngine(TextMarkEngine):
|
||||
class _KlingTextEngine(TextMarkEngine):
|
||||
"""Require glyph structure beyond a horizontal or vertical background edge."""
|
||||
|
||||
def _ladder_best(
|
||||
self, image: NDArray[Any], loc: _text_mark_engine.TextMarkLocation
|
||||
) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
score, box = super()._ladder_best(image, loc)
|
||||
response = self._detect_response(image, loc)
|
||||
silhouette = self._glyph_silhouette()
|
||||
if box is None or response is None or silhouette is None:
|
||||
return score, box
|
||||
x, y, x1, y1 = box
|
||||
patch = response[y : y1 + 1, x : x1 + 1].astype(np.float32)
|
||||
template = cv2.resize(silhouette, (patch.shape[1], patch.shape[0]), interpolation=cv2.INTER_AREA)
|
||||
|
||||
def glyph_detail(array: NDArray[Any]) -> NDArray[Any]:
|
||||
array = array.astype(np.float32)
|
||||
return array - array.mean(axis=0, keepdims=True) - array.mean(axis=1, keepdims=True) + array.mean()
|
||||
|
||||
patch, template = glyph_detail(patch), glyph_detail(template)
|
||||
denominator = float(np.linalg.norm(patch) * np.linalg.norm(template))
|
||||
detail_score = float((patch * template).sum()) / denominator if denominator > 1e-6 else 0.0
|
||||
return max(0.0, min(score, detail_score)), box
|
||||
|
||||
|
||||
class KlingEngine(_KlingTextEngine):
|
||||
"""Detect/localize Kling AI's CJK and Latin 3.0 marks (locate -> mask -> fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
self._latin = TextMarkEngine(_LATIN_CONFIG)
|
||||
self._latin = _KlingTextEngine(_LATIN_CONFIG)
|
||||
|
||||
def detect(self, image: NDArray[Any], *, provenance: bool = False) -> TextMarkDetection:
|
||||
"""Return the strongest CJK or Latin Kling wordmark verdict."""
|
||||
@@ -195,14 +225,15 @@ class KlingEngine(TextMarkEngine):
|
||||
return super().footprint_mask(image, force=force, dilate=dilate, detection=detection)
|
||||
|
||||
det = detection if detection is not None else self.detect(image)
|
||||
legacy = super().footprint_mask(image, force=False, dilate=dilate, detection=det)
|
||||
if not det.detected:
|
||||
return None
|
||||
engine = self._latin if det.template_asset == _LATIN_CONFIG.asset_name else self
|
||||
legacy = TextMarkEngine.footprint_mask(engine, image, force=False, dilate=dilate, detection=det)
|
||||
radius = _FOOTPRINT_DILATE if dilate is None else max(0, dilate)
|
||||
result = legacy
|
||||
for config in (_CONFIG, _LATIN_CONFIG):
|
||||
alpha = _text_mark_engine.load_alpha_template(config.asset_name)
|
||||
if alpha is None:
|
||||
continue
|
||||
core = self._aligned_alpha_mask(
|
||||
alpha = _text_mark_engine.load_alpha_template(engine.config.asset_name)
|
||||
if alpha is not None:
|
||||
core = engine._aligned_alpha_mask(
|
||||
image,
|
||||
det,
|
||||
alpha,
|
||||
|
||||
@@ -170,7 +170,7 @@ class LibLibPillEngine:
|
||||
"""Detect the compact top-left pill only when LiblibAI is corroborated."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._pill = PillEngine()
|
||||
self._pill = PillEngine(include_label=False)
|
||||
|
||||
def _raw(self, image: NDArray[Any]) -> PillDetection:
|
||||
return self._pill.detect(image)
|
||||
|
||||
@@ -0,0 +1,297 @@
|
||||
"""Model Context Protocol server over stdio, a thin wrapper around this package's CLI.
|
||||
|
||||
``remove-ai-watermarks mcp`` speaks newline-delimited JSON-RPC 2.0 on stdin and
|
||||
stdout with the initialize handshake. Each tool call runs one CLI command in a
|
||||
child process of the same interpreter, so a tool returns exactly what that command
|
||||
would: its exit code, its output file and its text. Nothing but protocol messages
|
||||
reaches stdout, and a crash inside a command cannot corrupt the stream.
|
||||
|
||||
The server needs no dependency beyond the standard library, so the bare package
|
||||
runs it through ``uvx remove-ai-watermarks mcp``. Commands that need an optional
|
||||
extra fail with the CLI's own install hint, returned as a tool error.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from typing import IO, Any, cast
|
||||
|
||||
from remove_ai_watermarks import __version__
|
||||
from remove_ai_watermarks.video import VIDEO_EXTENSIONS
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# Newest first. A client asking for an unlisted version gets the newest, and
|
||||
# decides for itself whether it can continue.
|
||||
PROTOCOL_VERSIONS = ("2025-11-25", "2025-06-18", "2025-03-26", "2024-11-05")
|
||||
|
||||
INSTRUCTIONS = (
|
||||
"Identify and remove AI provenance marks from images and video the user generated "
|
||||
"or edited themselves: C2PA, IPTC, EXIF and TC260 metadata, visible AI-generator "
|
||||
"labels, and supported invisible watermarks. Do not use it on stock, agency, "
|
||||
"marketplace or other third-party marks that protect someone else's asset. A "
|
||||
"result of 'unknown' or exit code 2 means no supported signal was found, not "
|
||||
"that the file is clean. Paths are read on the machine running the server."
|
||||
)
|
||||
|
||||
_MODES = ("metadata", "visible", "all")
|
||||
|
||||
TOOLS: list[dict[str, Any]] = [
|
||||
{
|
||||
"name": "identify",
|
||||
"title": "Identify AI provenance",
|
||||
"description": (
|
||||
"Report which AI provenance signals an image or video carries: metadata "
|
||||
"(C2PA, IPTC, EXIF, TC260), visible AI-generator labels, and supported local "
|
||||
"invisible-watermark decoders. Read-only. Video is detected by file suffix."
|
||||
),
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"path": {"type": "string", "description": "Image or video file to inspect."},
|
||||
"check_visible": {
|
||||
"type": "boolean",
|
||||
"default": True,
|
||||
"description": "Run the pixel and visible-mark detectors. False reads metadata only.",
|
||||
},
|
||||
},
|
||||
"required": ["path"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
"annotations": {"readOnlyHint": True, "openWorldHint": False},
|
||||
},
|
||||
{
|
||||
"name": "remove",
|
||||
"title": "Remove AI watermarks from one file",
|
||||
"description": (
|
||||
"Write a cleaned copy of an image or video the user generated or edited. "
|
||||
"mode=metadata strips AI metadata without touching pixels or re-encoding video; "
|
||||
"mode=visible also removes a known visible AI label; for an image, mode=all adds "
|
||||
"the invisible-watermark scrub where the installed extras and hardware allow it. "
|
||||
"The source is never overwritten; the default output is <name>_clean beside it."
|
||||
),
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"path": {"type": "string", "description": "Image or video file to clean."},
|
||||
"mode": {"type": "string", "enum": list(_MODES), "default": "metadata"},
|
||||
"output": {"type": "string", "description": "Output file. Must differ from the source."},
|
||||
},
|
||||
"required": ["path"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
"annotations": {"readOnlyHint": False, "destructiveHint": False, "openWorldHint": False},
|
||||
},
|
||||
{
|
||||
"name": "batch",
|
||||
"title": "Remove AI watermarks from a directory",
|
||||
"description": (
|
||||
"Clean every supported image (media=image) or video (media=video) in a directory "
|
||||
"the user owns, writing results to a separate directory (default <dir>_clean). "
|
||||
"Modes as for the remove tool."
|
||||
),
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"directory": {"type": "string", "description": "Directory of input files."},
|
||||
"mode": {"type": "string", "enum": list(_MODES), "default": "metadata"},
|
||||
"media": {"type": "string", "enum": ["image", "video"], "default": "image"},
|
||||
"output_dir": {"type": "string", "description": "Output directory. Must differ from the input."},
|
||||
},
|
||||
"required": ["directory"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
"annotations": {"readOnlyHint": False, "destructiveHint": False, "openWorldHint": False},
|
||||
},
|
||||
]
|
||||
|
||||
_TOOL_SCHEMAS = {tool["name"]: tool["inputSchema"] for tool in TOOLS}
|
||||
|
||||
Runner = Callable[[list[str]], subprocess.CompletedProcess[str]]
|
||||
|
||||
|
||||
class ToolInputError(ValueError):
|
||||
"""A tool call whose arguments the server refuses before running anything."""
|
||||
|
||||
|
||||
def run_cli(args: list[str]) -> subprocess.CompletedProcess[str]:
|
||||
"""Run one CLI command in a child process of this interpreter."""
|
||||
env = {**os.environ, "COLUMNS": "120", "NO_COLOR": "1", "TERM": "dumb"}
|
||||
command = [sys.executable, "-m", "remove_ai_watermarks.cli", *args]
|
||||
log.info("Running CLI command: %s", command)
|
||||
completed = subprocess.run( # noqa: S603 -- fixed interpreter, argument list, no shell
|
||||
command,
|
||||
stdin=subprocess.DEVNULL,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=env,
|
||||
check=False,
|
||||
)
|
||||
log.info("CLI command exited %d: %s", completed.returncode, command)
|
||||
return completed
|
||||
|
||||
|
||||
def _validated(tool: str, arguments: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Check arguments against the tool's own input schema and fill its defaults."""
|
||||
schema = _TOOL_SCHEMAS[tool]
|
||||
properties: dict[str, dict[str, Any]] = schema["properties"]
|
||||
unknown = sorted(set(arguments) - set(properties))
|
||||
if unknown:
|
||||
raise ToolInputError(f"unknown argument(s): {', '.join(unknown)}")
|
||||
values: dict[str, Any] = {}
|
||||
for key, spec in properties.items():
|
||||
if key not in arguments:
|
||||
if key in schema["required"]:
|
||||
raise ToolInputError(f"'{key}' is required")
|
||||
if "default" in spec:
|
||||
values[key] = spec["default"]
|
||||
continue
|
||||
value = arguments[key]
|
||||
if spec["type"] == "boolean" and not isinstance(value, bool):
|
||||
raise ToolInputError(f"'{key}' must be a boolean")
|
||||
if spec["type"] == "string" and (not isinstance(value, str) or not value.strip()):
|
||||
raise ToolInputError(f"'{key}' must be a non-empty string")
|
||||
if "enum" in spec and value not in spec["enum"]:
|
||||
raise ToolInputError(f"'{key}' must be one of {', '.join(spec['enum'])}")
|
||||
values[key] = value
|
||||
return values
|
||||
|
||||
|
||||
def _is_video(path: str) -> bool:
|
||||
return Path(path).suffix.lower() in VIDEO_EXTENSIONS
|
||||
|
||||
|
||||
def _identify_command(values: dict[str, Any]) -> tuple[list[str], str | None]:
|
||||
path = values["path"]
|
||||
prefix = ["video", "identify"] if _is_video(path) else ["identify"]
|
||||
return [*prefix, "--json", *([] if values["check_visible"] else ["--no-visible"]), "--", path], None
|
||||
|
||||
|
||||
def _remove_command(values: dict[str, Any]) -> tuple[list[str], str | None]:
|
||||
path, mode = values["path"], values["mode"]
|
||||
source = Path(path)
|
||||
output = values.get("output") or str(source.with_stem(source.stem + "_clean"))
|
||||
if Path(output).resolve() == source.resolve():
|
||||
raise ToolInputError("'output' must differ from the source; the server never overwrites an input")
|
||||
command = ["metadata", "--remove"] if mode == "metadata" else [mode]
|
||||
return [*(["video"] if _is_video(path) else []), *command, "-o", output, "--", path], output
|
||||
|
||||
|
||||
def _batch_command(values: dict[str, Any]) -> tuple[list[str], str | None]:
|
||||
directory = values["directory"]
|
||||
resolved = Path(directory).resolve()
|
||||
output_dir = values.get("output_dir") or str(resolved.with_name(resolved.name + "_clean"))
|
||||
if Path(output_dir).resolve() == resolved:
|
||||
raise ToolInputError("'output_dir' must differ from the input directory")
|
||||
command = ["video", "batch"] if values["media"] == "video" else ["batch"]
|
||||
return [*command, "--mode", values["mode"], "-o", output_dir, "--", directory], output_dir
|
||||
|
||||
|
||||
_COMMANDS: dict[str, Callable[[dict[str, Any]], tuple[list[str], str | None]]] = {
|
||||
"identify": _identify_command,
|
||||
"remove": _remove_command,
|
||||
"batch": _batch_command,
|
||||
}
|
||||
|
||||
|
||||
def call_tool(name: str, arguments: dict[str, Any], runner: Runner = run_cli) -> dict[str, Any]:
|
||||
"""Run one tool and return an MCP ``tools/call`` result."""
|
||||
try:
|
||||
args, output = _COMMANDS[name](_validated(name, arguments))
|
||||
except ToolInputError as exc:
|
||||
return {"content": [{"type": "text", "text": f"Invalid arguments: {exc}"}], "isError": True}
|
||||
completed = runner(args)
|
||||
code = completed.returncode
|
||||
text = completed.stdout.strip()
|
||||
stderr = completed.stderr.strip()[-4000:]
|
||||
structured: dict[str, Any] = {"command": ["remove-ai-watermarks", *args], "exit_code": code, "output": output}
|
||||
if "--json" in args and code == 0:
|
||||
with contextlib.suppress(json.JSONDecodeError):
|
||||
structured["report"] = json.loads(text)
|
||||
if "report" not in structured:
|
||||
structured["stdout"] = text
|
||||
if stderr:
|
||||
structured["stderr"] = stderr
|
||||
# The CLI's own exit 2 means no supported signal, a result. Click also exits 2 on
|
||||
# a usage error, such as a missing file, and prints its usage line on stderr.
|
||||
no_signal = code == 2 and "Usage:" not in stderr
|
||||
is_error = code != 0 and not no_signal
|
||||
summary = f"exit code {code}"
|
||||
if no_signal:
|
||||
summary += " (no supported signal found; this is unknown, not proof the file is clean)"
|
||||
body = "\n\n".join(part for part in (summary, text, stderr if is_error else "") if part)
|
||||
return {"content": [{"type": "text", "text": body}], "structuredContent": structured, "isError": is_error}
|
||||
|
||||
|
||||
def _error(request_id: object, code: int, message: str) -> dict[str, Any]:
|
||||
return {"jsonrpc": "2.0", "id": request_id, "error": {"code": code, "message": message}}
|
||||
|
||||
|
||||
def _object(value: object) -> dict[str, Any] | None:
|
||||
"""A decoded JSON object, or ``None`` for any other JSON value."""
|
||||
return cast("dict[str, Any]", value) if isinstance(value, dict) else None
|
||||
|
||||
|
||||
def handle(raw: object, runner: Runner = run_cli) -> dict[str, Any] | None:
|
||||
"""Answer one JSON-RPC message; ``None`` for notifications and client responses."""
|
||||
message = _object(raw)
|
||||
if message is None or message.get("jsonrpc") != "2.0":
|
||||
return _error(None, -32600, "Invalid Request")
|
||||
method = message.get("method")
|
||||
if "id" not in message:
|
||||
return None # a notification or a response to nothing this server sent
|
||||
request_id: object = message["id"]
|
||||
if not isinstance(method, str):
|
||||
return _error(request_id, -32600, "Invalid Request")
|
||||
params = _object(message.get("params") or {})
|
||||
if params is None:
|
||||
return _error(request_id, -32602, "params must be an object")
|
||||
|
||||
if method == "initialize":
|
||||
requested = params.get("protocolVersion")
|
||||
result: dict[str, Any] = {
|
||||
"protocolVersion": requested if requested in PROTOCOL_VERSIONS else PROTOCOL_VERSIONS[0],
|
||||
"capabilities": {"tools": {"listChanged": False}},
|
||||
"serverInfo": {"name": "remove-ai-watermarks", "title": "Remove AI Watermarks", "version": __version__},
|
||||
"instructions": INSTRUCTIONS,
|
||||
}
|
||||
elif method == "ping":
|
||||
result = {}
|
||||
elif method == "tools/list":
|
||||
result = {"tools": TOOLS}
|
||||
elif method == "tools/call":
|
||||
name = params.get("name")
|
||||
arguments = _object(params.get("arguments") or {})
|
||||
if not isinstance(name, str) or name not in _COMMANDS:
|
||||
return _error(request_id, -32602, f"Unknown tool: {name}")
|
||||
if arguments is None:
|
||||
return _error(request_id, -32602, "arguments must be an object")
|
||||
result = call_tool(name, arguments, runner)
|
||||
else:
|
||||
return _error(request_id, -32601, f"Method not found: {method}")
|
||||
return {"jsonrpc": "2.0", "id": request_id, "result": result}
|
||||
|
||||
|
||||
def serve(stdin: IO[str] | None = None, stdout: IO[str] | None = None, runner: Runner = run_cli) -> None:
|
||||
"""Serve newline-delimited JSON-RPC until stdin closes."""
|
||||
source: IO[str] = sys.stdin if stdin is None else stdin
|
||||
sink: IO[str] = sys.stdout if stdout is None else stdout
|
||||
for line in source:
|
||||
if not line.strip():
|
||||
continue
|
||||
try:
|
||||
message = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
response: dict[str, Any] | None = _error(None, -32700, "Parse error")
|
||||
else:
|
||||
response = handle(message, runner)
|
||||
if response is not None:
|
||||
sink.write(json.dumps(response, ensure_ascii=False) + "\n")
|
||||
sink.flush()
|
||||
@@ -85,6 +85,11 @@ AI_KEYWORDS: tuple[str, ...] = (
|
||||
# Reference: https://spec.c2pa.org/specifications/specifications/2.1/specs/C2PA_Specification.html
|
||||
C2PA_UUID: bytes = bytes.fromhex("d8fec3d61b0e483c92975828877ec481")
|
||||
|
||||
# Kling's encoder writes an H.264 ``user_data_unregistered`` SEI message (payload
|
||||
# type 5) whose payload is this UUID followed by ``kling-ai``. It lives in the coded
|
||||
# stream inside ``mdat``, where no metadata box walk reaches it.
|
||||
KLING_SEI_TAG: bytes = bytes.fromhex("91ca60614aee385486142d5f73f4ae2e") + b"kling-ai"
|
||||
|
||||
|
||||
def c2pa_marker_in(data: bytes) -> bool:
|
||||
"""True if ``data`` carries a real C2PA manifest marker, not just an
|
||||
@@ -1430,6 +1435,10 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
if app_generator:
|
||||
result.setdefault("app_aigc", f"App AIGC disclosure ({app_generator})")
|
||||
|
||||
# Kling's encoder tag, an H.264 SEI message in the first video sample.
|
||||
if KLING_SEI_TAG in app_scan:
|
||||
result.setdefault("kling_sei", "Kling AI encoder tag (H.264 SEI kling-ai)")
|
||||
|
||||
# xAI / Grok signature pair in EXIF or an equivalent container-native field.
|
||||
if xai_signature(image_path):
|
||||
result.setdefault("xai_signature", "xAI/Grok signature (UUID + Signature blob)")
|
||||
|
||||
@@ -38,10 +38,12 @@ status was not adjudicated, and 1200 non-overlapping no-signal controls:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection
|
||||
from remove_ai_watermarks._text_mark_variants import GlyphTextMarkEngine, TextMarkVariantsMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -101,8 +103,62 @@ def _alpha_template() -> NDArray[Any] | None:
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
class MicrosoftEngine(TextMarkEngine):
|
||||
class MicrosoftEngine(TextMarkVariantsMixin):
|
||||
"""Detect/localize the measured Microsoft top-right AI badge."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
self._variants = (
|
||||
GlyphTextMarkEngine(
|
||||
replace(
|
||||
_CONFIG,
|
||||
asset_name="microsoft_text_alpha.png",
|
||||
width_frac=0.20,
|
||||
height_frac=0.065,
|
||||
margin_x_frac=0.002,
|
||||
margin_bottom_frac=0.002,
|
||||
scale_basis="width",
|
||||
alpha_width_frac=0.14,
|
||||
alpha_height_frac=0.14 * 49 / 339,
|
||||
detect_frontend="contrast",
|
||||
ladder=(0.95, 1.0, 1.05),
|
||||
detect_ncc_threshold=0.60,
|
||||
provenance_ncc_factor=1.0,
|
||||
max_saturation=255,
|
||||
)
|
||||
),
|
||||
)
|
||||
|
||||
def footprint_mask(
|
||||
self,
|
||||
image: NDArray[Any] | None,
|
||||
*,
|
||||
force: bool = False,
|
||||
dilate: int | None = None,
|
||||
detection: TextMarkDetection | None = None,
|
||||
) -> NDArray[Any] | None:
|
||||
"""Cover the enclosing beige pill for the text-only layout."""
|
||||
if force or image is None or image.size == 0:
|
||||
return super().footprint_mask(image, force=force, dilate=dilate, detection=detection)
|
||||
det = detection if detection is not None else self.detect(image)
|
||||
engine = self.variant_engine(det)
|
||||
if engine is self or not det.detected or det.match_box is None:
|
||||
return super().footprint_mask(image, force=False, dilate=dilate, detection=det)
|
||||
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
loc = engine.locate(image)
|
||||
x0, y0, x1, y1 = det.match_box
|
||||
pad_x = int(image.shape[1] * 0.012)
|
||||
pad_y = int(image.shape[1] * 0.014)
|
||||
box = (
|
||||
max(0, loc.x + x0 - pad_x),
|
||||
max(0, loc.y + y0 - pad_y),
|
||||
x1 - x0 + 1 + 2 * pad_x,
|
||||
y1 - y0 + 1 + 2 * pad_y,
|
||||
)
|
||||
return region_eraser.boxes_to_mask(
|
||||
image.shape[:2],
|
||||
[box],
|
||||
dilate=0 if dilate is None else max(0, dilate),
|
||||
)
|
||||
@@ -3,8 +3,7 @@
|
||||
OpenArt (openart.ai) stamps free-plan generations with a semi-transparent white
|
||||
"OpenArt" wordmark (a bowtie/infinity icon followed by the brand name) placed
|
||||
**near the center of the frame**, not in a corner like every other text mark
|
||||
this repository detects. Confirmed from one customer-reported production case
|
||||
(raiw-app, 2026-09-11, 1528x2712 portrait output): the mark sat roughly centered
|
||||
this repository detects. In the observed portrait layout, the mark sat roughly centered
|
||||
over the subject's torso, and neither C2PA, EXIF AI tags, nor the China AIGC/TC260
|
||||
label were present -- OpenArt apparently ships no machine-readable provenance
|
||||
signal alongside this visible mark, so a bare export was previously reported as
|
||||
|
||||
@@ -1,18 +1,20 @@
|
||||
"""Jimeng-basic 'AI生成' pill: a CAPTURE-LESS visible mark (issue #54).
|
||||
"""Jimeng-basic ``AI生成`` pill visible mark (issue #54).
|
||||
|
||||
The Jimeng free-tier TC260 label is a rounded pill with 'AI生成' in the TOP-LEFT
|
||||
corner -- distinct from the ``jimeng`` "★ 即梦AI" wordmark (bottom-right). It has no
|
||||
captured alpha map, so unlike the other marks it is detected purely by a synthetic
|
||||
silhouette; like every mark it is then removed by the shared localize -> fill:
|
||||
captured alpha map for its outline. Detection therefore has two paths before the
|
||||
shared localize -> fill step:
|
||||
|
||||
* Detect: edge-NCC of a font-rendered SILHOUETTE (``assets/jimeng_pill.png``,
|
||||
* A high-confidence contrast template for the measured solid ``AI生成`` label
|
||||
(``assets/jimeng_label_alpha.png``). With Jimeng provenance this verified
|
||||
label can bypass the flat-background safety gate.
|
||||
* Edge-NCC of a font-rendered SILHOUETTE (``assets/jimeng_pill.png``,
|
||||
synthetic, data-safe -- see ``scripts/render_pill_silhouette.py``) against the
|
||||
top-left ROI, at the pill's known width fraction. The calibrated
|
||||
``_DETECT_THRESHOLD`` is 0.22.
|
||||
* Remove: place the pill footprint at the matched location and inpaint it
|
||||
(MI-GAN / cv2 via the registry). Quality comes from the inpaint backend, so the
|
||||
silhouette need not be pixel-accurate -- which is why a synthetic render is
|
||||
sufficient and no source-derived asset is committed.
|
||||
top-left ROI, at the pill's known width fraction. Its calibrated
|
||||
``_DETECT_THRESHOLD`` is 0.22 and it retains the provenance and flat-background
|
||||
product gates.
|
||||
* Remove: mask the verified label match or the stable outline footprint and
|
||||
inpaint it (MI-GAN / cv2 via the registry).
|
||||
|
||||
Geometry uses width ~0.161*W,
|
||||
height ~0.091*W, top-left, margins ~0.02-0.05.
|
||||
@@ -27,6 +29,7 @@ import cv2
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks import image_io
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -74,6 +77,32 @@ class PillDetection(NamedTuple):
|
||||
detected: bool
|
||||
confidence: float
|
||||
region: tuple[int, int, int, int] # x, y, w, h of the matched pill
|
||||
verified_label: bool = False
|
||||
|
||||
|
||||
_LABEL_CONFIG = TextMarkConfig(
|
||||
name="Jimeng AI生成 pill label",
|
||||
asset_name="jimeng_label_alpha.png",
|
||||
corner="tl",
|
||||
margin_floor=2,
|
||||
width_frac=0.34,
|
||||
height_frac=0.14,
|
||||
margin_x_frac=0.0,
|
||||
margin_bottom_frac=0.0,
|
||||
max_saturation=255,
|
||||
logo_min_luma=100,
|
||||
tophat_delta=8,
|
||||
morph_open_size=3,
|
||||
detect_min_coverage=0.01,
|
||||
detect_ncc_threshold=0.55,
|
||||
detect_frontend="contrast",
|
||||
scale_basis="width",
|
||||
ladder=(0.95, 1.0, 1.05),
|
||||
alpha_width_frac=0.14,
|
||||
alpha_height_frac=0.14 * 128 / 353,
|
||||
min_gw=24,
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
|
||||
def _load_silhouette() -> NDArray[Any] | None:
|
||||
@@ -92,7 +121,18 @@ def _grad(gray: NDArray[Any]) -> NDArray[Any]:
|
||||
|
||||
|
||||
class PillEngine:
|
||||
"""Detect + build the removal mask for the top-left 'AI生成' pill (edge-NCC of a synthetic silhouette)."""
|
||||
"""Detect the top-left ``AI生成`` label or its synthetic pill outline."""
|
||||
|
||||
def __init__(self, *, include_label: bool = True) -> None:
|
||||
self._label = TextMarkEngine(_LABEL_CONFIG) if include_label else None
|
||||
|
||||
@staticmethod
|
||||
def _label_box(detection: TextMarkDetection) -> tuple[int, int, int, int] | None:
|
||||
if not detection.detected or detection.match_box is None:
|
||||
return None
|
||||
x, y, _width, _height = detection.region
|
||||
x0, y0, x1, y1 = detection.match_box
|
||||
return x + x0, y + y0, x1 - x0 + 1, y1 - y0 + 1
|
||||
|
||||
def _match(self, image: NDArray[Any]) -> tuple[float, tuple[int, int, int, int]] | None:
|
||||
sil = _load_silhouette()
|
||||
@@ -113,12 +153,33 @@ class PillEngine:
|
||||
_, score, _, loc = cv2.minMaxLoc(res)
|
||||
return float(score), (int(loc[0]), int(loc[1]), tw, th)
|
||||
|
||||
def detect(self, image: NDArray[Any]) -> PillDetection:
|
||||
m = self._match(image)
|
||||
if m is None:
|
||||
return PillDetection(False, 0.0, (0, 0, 0, 0))
|
||||
score, box = m
|
||||
return PillDetection(score >= _DETECT_THRESHOLD, score, box)
|
||||
def _detection(
|
||||
self,
|
||||
match: tuple[float, tuple[int, int, int, int]] | None,
|
||||
label: TextMarkDetection,
|
||||
) -> PillDetection:
|
||||
label_box = self._label_box(label)
|
||||
if match is None:
|
||||
return PillDetection(label.detected, label.confidence, label_box or label.region, label.detected)
|
||||
score, box = match
|
||||
return PillDetection(
|
||||
score >= _DETECT_THRESHOLD or label.detected,
|
||||
max(score, label.confidence),
|
||||
label_box if label.detected and label_box is not None else box,
|
||||
label.detected,
|
||||
)
|
||||
|
||||
def detect(self, image: NDArray[Any], *, provenance: bool = False) -> PillDetection:
|
||||
"""Detect the outline, plus the solid label when Jimeng is corroborated."""
|
||||
label_scan = self._label.detect(image) if self._label is not None else TextMarkDetection()
|
||||
label = label_scan if provenance else TextMarkDetection()
|
||||
return self._detection(self._match(image), label)
|
||||
|
||||
def detect_both(self, image: NDArray[Any]) -> tuple[PillDetection, PillDetection]:
|
||||
"""Return strict outline and Jimeng-corroborated label verdicts from one scan."""
|
||||
match = self._match(image)
|
||||
label = self._label.detect(image) if self._label is not None else TextMarkDetection()
|
||||
return self._detection(match, TextMarkDetection()), self._detection(match, label)
|
||||
|
||||
def _footprint_box(self, image: NDArray[Any]) -> tuple[int, int, int, int] | None:
|
||||
h, w = image.shape[:2]
|
||||
@@ -152,19 +213,34 @@ class PillEngine:
|
||||
"""True when the top-left footprint is flat enough for an invisible inpaint."""
|
||||
return self.footprint_texture(image) <= thresh
|
||||
|
||||
def footprint_mask(self, image: NDArray[Any], *, force: bool = False) -> NDArray[Any] | None:
|
||||
def footprint_mask(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
*,
|
||||
force: bool = False,
|
||||
detection: PillDetection | None = None,
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame uint8 mask (255 = pill) over the pill's known top-left region.
|
||||
|
||||
Uses stable GEOMETRY (a generous fixed box), not the NCC match position: the
|
||||
synthetic silhouette localizes only approximately, so a match-positioned mask
|
||||
leaves outline residue, while the top-left corner is negative space, so a
|
||||
generous geometric box removes the pill cleanly and harmlessly. The caller
|
||||
A verified label uses its detector-aligned box with measured padding. The
|
||||
weaker synthetic outline uses stable geometry because its NCC position is
|
||||
only approximate and a match-positioned mask leaves rim residue. The caller
|
||||
gates on :meth:`detect`, so a clean corner is never masked. ``force`` is
|
||||
accepted for a uniform engine signature but ignored (the geometry box is
|
||||
fixed regardless)."""
|
||||
accepted for a uniform engine signature but ignored."""
|
||||
if image is None or image.size == 0:
|
||||
return None
|
||||
box = self._footprint_box(image)
|
||||
if detection is not None and detection.verified_label:
|
||||
x, y, width, height = detection.region
|
||||
pad_x = int(image.shape[1] * 0.025)
|
||||
pad_y = int(image.shape[1] * 0.014)
|
||||
box = (
|
||||
max(0, x - pad_x),
|
||||
max(0, y - pad_y),
|
||||
min(image.shape[1], x + width + pad_x),
|
||||
min(image.shape[0], y + height + pad_y),
|
||||
)
|
||||
else:
|
||||
box = self._footprint_box(image)
|
||||
if box is None:
|
||||
return None
|
||||
# Same primitive the shared fill uses, rather than a private zeros/fill copy.
|
||||
|
||||
@@ -25,10 +25,12 @@ signal via the registry.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine, image_io
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection
|
||||
from remove_ai_watermarks._text_mark_variants import GlyphTextMarkEngine, TextMarkVariantsMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -107,11 +109,27 @@ def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class SamsungEngine(TextMarkEngine):
|
||||
class SamsungEngine(TextMarkVariantsMixin):
|
||||
"""Detect/localize the visible Samsung Galaxy AI text mark (locate -> mask; mask feeds the fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
self._variants = (
|
||||
GlyphTextMarkEngine(
|
||||
replace(
|
||||
_CONFIG,
|
||||
asset_name="samsung_ko_alpha.png",
|
||||
width_frac=0.28,
|
||||
height_frac=0.065,
|
||||
alpha_width_frac=0.20,
|
||||
alpha_height_frac=0.20 * 49 / 344,
|
||||
ladder=(0.95, 1.0, 1.05),
|
||||
max_saturation=255,
|
||||
detect_frontend="contrast",
|
||||
detect_ncc_threshold=0.46,
|
||||
)
|
||||
),
|
||||
)
|
||||
|
||||
def footprint_mask(
|
||||
self,
|
||||
@@ -134,10 +152,29 @@ class SamsungEngine(TextMarkEngine):
|
||||
|
||||
image = image_io.to_bgr(image)
|
||||
det = detection if detection is not None else self.detect(image)
|
||||
alpha = _alpha_template()
|
||||
engine = self.variant_engine(det)
|
||||
radius = _FOOTPRINT_DILATE if dilate is None else max(0, dilate)
|
||||
if engine is not self and det.detected and det.match_box is not None:
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
x0, y0, x1, y1 = det.match_box
|
||||
loc = engine.locate(image)
|
||||
glyph_height = y1 - y0 + 1
|
||||
return region_eraser.boxes_to_mask(
|
||||
image.shape[:2],
|
||||
[
|
||||
(
|
||||
max(0, loc.x + x0 - round(1.3 * glyph_height)),
|
||||
loc.y + y0,
|
||||
x1 - x0 + 1 + round(1.3 * glyph_height),
|
||||
glyph_height,
|
||||
)
|
||||
],
|
||||
dilate=radius,
|
||||
)
|
||||
alpha = _text_mark_engine.load_alpha_template(engine.config.asset_name)
|
||||
if alpha is not None:
|
||||
radius = _FOOTPRINT_DILATE if dilate is None else max(0, dilate)
|
||||
aligned = self._aligned_alpha_mask(
|
||||
aligned = engine._aligned_alpha_mask(
|
||||
image,
|
||||
det,
|
||||
alpha,
|
||||
|
||||
@@ -377,6 +377,8 @@ def _platform_from_video_metadata(markers: dict[str, str]) -> str | None:
|
||||
return _tc260_video_platform(markers.get("aigc_producer", "")) or (
|
||||
"China AIGC-labeled content (TC260 standard)"
|
||||
)
|
||||
if "kling_sei" in markers:
|
||||
return _tc260_video_platform("kling")
|
||||
return None
|
||||
|
||||
|
||||
|
||||
@@ -135,7 +135,7 @@ def load_video_vae_runtime(
|
||||
dtype = torch.float16 if resolved_device == "cuda" else torch.float32
|
||||
log.info("Loading %s on %s", model, resolved_device)
|
||||
vae = AutoencoderKL.from_pretrained(model, torch_dtype=dtype).to(resolved_device)
|
||||
vae.eval()
|
||||
vae.train(False)
|
||||
vae.enable_slicing()
|
||||
scaling_factor = float(vae.config.scaling_factor)
|
||||
log.info("Latent scaling factor %.5f", scaling_factor)
|
||||
|
||||
@@ -24,13 +24,15 @@ Entries:
|
||||
- ``qwen`` -- Alibaba Cloud Qwen "千问AI生成" text or three-lobe symbol, bottom-right.
|
||||
- ``kling`` -- Kuaishou Kling AI "可灵AI 3.0" / "KlingAI 3.0" strip, bottom-right.
|
||||
- ``yuanbao`` -- Tencent Yuanbao "元宝 / AI生成" two-line mark, bottom-right.
|
||||
- ``samsung`` -- Samsung Galaxy AI "Contenuti generati dall'AI" strip, bottom-left.
|
||||
- ``jimeng_pill`` -- Jimeng-basic "AI生成" pill, top-left (capture-less).
|
||||
- ``dola`` -- ByteDance Dola "Dola AI" image disclosure, bottom-right.
|
||||
- ``workbuddy`` -- Tencent WorkBuddy "AI生成 / WORKBUDDY" disclosure, bottom-right.
|
||||
- ``samsung`` -- Samsung Galaxy AI Italian or Korean disclosure, bottom-left.
|
||||
- ``jimeng_pill`` -- Jimeng-basic "AI生成" pill or solid label, top-left.
|
||||
- ``runninghub`` -- RunningHub "RunningHub AI生成" text, top-left (gray front-end).
|
||||
- ``baidu`` -- Baidu "百度 AI生成" text + white tag, bottom-right.
|
||||
- ``liblib`` -- LiblibAI "LiblibAI" wordmark, bottom-center.
|
||||
- ``liblib_pill`` -- LiblibAI compact "AI生成" pill, top-left.
|
||||
- ``microsoft`` -- one measured Microsoft white AI-badge variant, top-right.
|
||||
- ``microsoft`` -- measured Microsoft white AI badge or beige text pill, top-right.
|
||||
- ``generic_ai_label`` -- brand-less bare "AI生成" TC260 label (no vendor wordmark),
|
||||
bottom-right; the fallback for a compliance stamp not tied to any tuned mark above.
|
||||
- ``openart`` -- OpenArt "OpenArt" wordmark (bowtie/infinity icon + brand name),
|
||||
@@ -448,6 +450,8 @@ _ENGINE_CLASS: dict[str, tuple[str, str]] = {
|
||||
"jimeng": ("jimeng_engine", "JimengEngine"),
|
||||
"qwen": ("qwen_engine", "QwenEngine"),
|
||||
"kling": ("kling_engine", "KlingEngine"),
|
||||
"dola": ("dola_engine", "DolaEngine"),
|
||||
"workbuddy": ("workbuddy_engine", "WorkBuddyEngine"),
|
||||
"yuanbao": ("yuanbao_engine", "YuanbaoEngine"),
|
||||
"samsung": ("samsung_engine", "SamsungEngine"),
|
||||
"jimeng_pill": ("pill_engine", "PillEngine"),
|
||||
@@ -641,29 +645,44 @@ def _text_mark(
|
||||
)
|
||||
|
||||
|
||||
# ── Capture-less mark: the Jimeng-basic "AI生成" pill (top-left) ──
|
||||
# Detection is edge-NCC of a synthetic silhouette; the mask is a fixed top-left
|
||||
# geometry box (see pill_engine). Removal is the same localize -> fill as the rest.
|
||||
# ── Jimeng-basic "AI生成" pill or solid label (top-left) ──
|
||||
# The weak outline uses synthetic edge-NCC and fixed geometry. The verified solid
|
||||
# label uses its own contrast template and detector-aligned mask (see pill_engine).
|
||||
def _pill_detect(image: NDArray[Any], *, provenance: bool = False) -> MarkDetection:
|
||||
del provenance # the pill detector is provenance-independent; its relaxation lives entirely in _keep_pill
|
||||
d = _engine("jimeng_pill").detect(image)
|
||||
return MarkDetection("jimeng_pill", "Jimeng AI生成 pill", "top-left", d.detected, d.confidence, d.region)
|
||||
d = _engine("jimeng_pill").detect(image, provenance=provenance)
|
||||
return MarkDetection(
|
||||
"jimeng_pill",
|
||||
"Jimeng AI生成 pill",
|
||||
"top-left",
|
||||
d.detected,
|
||||
d.confidence,
|
||||
d.region,
|
||||
engine_detection=d,
|
||||
)
|
||||
|
||||
|
||||
def _pill_detect_both(image: NDArray[Any]) -> tuple[MarkDetection, MarkDetection]:
|
||||
# The pill detector is provenance-independent (`_pill_detect` discards the flag), so
|
||||
# one call answers both levels. MarkDetection is frozen, so sharing it is safe.
|
||||
d = _pill_detect(image)
|
||||
return d, d
|
||||
strict, relaxed = _engine("jimeng_pill").detect_both(image)
|
||||
|
||||
def wrap(detection: Any) -> MarkDetection:
|
||||
return MarkDetection(
|
||||
"jimeng_pill",
|
||||
"Jimeng AI生成 pill",
|
||||
"top-left",
|
||||
detection.detected,
|
||||
detection.confidence,
|
||||
detection.region,
|
||||
engine_detection=detection,
|
||||
)
|
||||
|
||||
return wrap(strict), wrap(relaxed)
|
||||
|
||||
|
||||
def _pill_mask(
|
||||
image: NDArray[Any], *, force: bool = False, detection: MarkDetection | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
# The pill mask is a fixed top-left geometry box, independent of the detection;
|
||||
# accepted for the uniform _mask signature.
|
||||
del detection
|
||||
return _engine("jimeng_pill").footprint_mask(image, force=force)
|
||||
engine_detection = detection.engine_detection if detection is not None else None
|
||||
return _engine("jimeng_pill").footprint_mask(image, force=force, detection=engine_detection)
|
||||
|
||||
|
||||
def _pill_features(image: NDArray[Any]) -> dict[str, float]:
|
||||
@@ -732,6 +751,24 @@ _REGISTRY: tuple[KnownMark, ...] = (
|
||||
platform="Tencent Yuanbao (visible 元宝 / AI生成 mark detected)",
|
||||
manufacturer="tencent",
|
||||
),
|
||||
_text_mark(
|
||||
"dola",
|
||||
"Dola AI image wordmark",
|
||||
"bottom-right",
|
||||
platform="Dola AI (visible Dola AI disclosure)",
|
||||
manufacturer="bytedance",
|
||||
label_regime=None,
|
||||
provenance_signals=(),
|
||||
),
|
||||
_text_mark(
|
||||
"workbuddy",
|
||||
"AI生成 / WORKBUDDY disclosure",
|
||||
"bottom-right",
|
||||
platform="WorkBuddy (visible AI content disclosure)",
|
||||
manufacturer="tencent",
|
||||
label_regime=None,
|
||||
provenance_signals=(),
|
||||
),
|
||||
# Samsung Galaxy AI is a device editing marker (samsung_genai), not a TC260 label.
|
||||
_text_mark(
|
||||
"samsung",
|
||||
@@ -940,8 +977,14 @@ def _pill_suppressors() -> set[str]:
|
||||
return {m.key for m in _REGISTRY if m.manufacturer == pill.manufacturer and m.product != pill.product}
|
||||
|
||||
|
||||
def _keep_pill(keys: set[str], *, provenance: frozenset[str], footprint_flat: bool) -> bool:
|
||||
"""Whether to auto-remove the capture-less 'AI生成' pill given the fired marks.
|
||||
def _keep_pill(
|
||||
keys: set[str],
|
||||
*,
|
||||
provenance: frozenset[str],
|
||||
footprint_flat: bool,
|
||||
verified_label: bool = False,
|
||||
) -> bool:
|
||||
"""Whether to auto-remove the ``AI生成`` pill or verified label.
|
||||
|
||||
Pure decision (the flatness feature is precomputed at perception time and passed
|
||||
in). The pill detector is weak and metadata/intent confirms the platform, not pill
|
||||
@@ -950,23 +993,24 @@ def _keep_pill(keys: set[str], *, provenance: frozenset[str], footprint_flat: bo
|
||||
* bottom-right "★ 即梦AI" wordmark fired -> ~94% precise, and it survives
|
||||
metadata-STRIPPED images: remove the pill unrestricted;
|
||||
* TC260 metadata confirms Jimeng (``"jimeng" in provenance``, no wordmark) -> remove ONLY when the
|
||||
top-left footprint is flat enough for an invisible fill (``footprint_flat``),
|
||||
so real flat-scene pills (and harmless flat false fires) are cleaned while the
|
||||
damaging textured false fires are left untouched.
|
||||
solid label template verifies the text, or when the weaker outline's top-left
|
||||
footprint is flat enough for an invisible fill (``footprint_flat``). Real
|
||||
flat-scene pills are cleaned while damaging textured false fires are left.
|
||||
A Doubao image is TC260 too but is not Jimeng-basic, so its mark suppresses the
|
||||
pill. Marks from other TC260 manufacturers are unrelated and do not participate in
|
||||
this ByteDance-family decision.
|
||||
pill. The independently attributed LiblibAI pill also wins over the generic
|
||||
``AI生成`` label shape. Marks from other TC260 manufacturers are unrelated and do
|
||||
not participate in this ByteDance-family decision.
|
||||
No confirmation at all -> never remove (blocks false fires on non-Jimeng content).
|
||||
|
||||
The suppressor set is derived from the manufacturer and product fields rather than
|
||||
from the shared TC260 standard. Marks from Google, Samsung, or another TC260
|
||||
manufacturer cannot enable or veto this ByteDance-specific arm."""
|
||||
if _pill_suppressors() & keys:
|
||||
if "liblib_pill" in keys or _pill_suppressors() & keys:
|
||||
return False
|
||||
if "jimeng" in keys:
|
||||
return True
|
||||
if "jimeng" in provenance:
|
||||
return footprint_flat
|
||||
return footprint_flat or verified_label
|
||||
return False
|
||||
|
||||
|
||||
@@ -990,6 +1034,8 @@ def _build_candidates(image: NDArray[Any]) -> list[Candidate]:
|
||||
continue
|
||||
strict, relaxed = m.detect_both(image)
|
||||
feats = m.features(image) if (strict.detected or relaxed.detected) else {}
|
||||
if relaxed.engine_detection is not None and getattr(relaxed.engine_detection, "verified_label", False):
|
||||
feats["verified_label"] = 1.0
|
||||
cands.append(
|
||||
Candidate(
|
||||
m.key,
|
||||
@@ -1010,7 +1056,7 @@ def decide(candidates: list[Candidate], context: Context) -> list[Decision]:
|
||||
|
||||
All policy lives here, in one place: per-mark trust resolution (:func:`resolve_trust`,
|
||||
which needs the strict-detected siblings for ``auto`` cross-mark corroboration) and
|
||||
the capture-less pill gate (:func:`_keep_pill`). No image, no I/O -- so it is unit-testable in isolation and
|
||||
the Jimeng pill/label gate (:func:`_keep_pill`). No image, no I/O -- so it is unit-testable in isolation and
|
||||
the same decision drives every caller."""
|
||||
strict_keys = {c.key for c in candidates if c.detected_strict}
|
||||
fired: list[Decision] = []
|
||||
@@ -1026,7 +1072,13 @@ def decide(candidates: list[Candidate], context: Context) -> list[Decision]:
|
||||
if "jimeng_pill" in keys:
|
||||
pill = next(d for d in fired if d.candidate.key == "jimeng_pill")
|
||||
flat = bool(pill.candidate.features.get("footprint_flat", 0.0))
|
||||
if not _keep_pill(keys, provenance=context.provenance, footprint_flat=flat):
|
||||
verified_label = bool(pill.candidate.features.get("verified_label", 0.0))
|
||||
if not _keep_pill(
|
||||
keys,
|
||||
provenance=context.provenance,
|
||||
footprint_flat=flat,
|
||||
verified_label=verified_label,
|
||||
):
|
||||
fired = [d for d in fired if d.candidate.key != "jimeng_pill"]
|
||||
return fired
|
||||
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
"""Detect the corner AI生成 / WORKBUDDY disclosure without inferring a base model."""
|
||||
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig
|
||||
from remove_ai_watermarks._text_mark_variants import BottomRightBoxTextMarkEngine
|
||||
|
||||
_CONFIG = TextMarkConfig(
|
||||
name="WorkBuddy AI disclosure",
|
||||
asset_name="workbuddy_alpha.png",
|
||||
corner="br",
|
||||
margin_floor=4,
|
||||
width_frac=0.22,
|
||||
height_frac=0.13,
|
||||
margin_x_frac=0.002,
|
||||
margin_bottom_frac=0.002,
|
||||
max_saturation=255,
|
||||
logo_min_luma=110,
|
||||
tophat_delta=8,
|
||||
morph_open_size=3,
|
||||
detect_min_coverage=0.01,
|
||||
detect_ncc_threshold=0.60,
|
||||
alpha_width_frac=0.12,
|
||||
alpha_height_frac=0.12 * 177 / 351,
|
||||
detect_frontend="tophat",
|
||||
scale_basis="width",
|
||||
ladder=(0.95, 1.0, 1.05),
|
||||
provenance_ncc_factor=1.0,
|
||||
min_gw=24,
|
||||
)
|
||||
|
||||
|
||||
class WorkBuddyEngine(BottomRightBoxTextMarkEngine):
|
||||
"""Detect and localize the two-line WorkBuddy disclosure."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
@@ -22,6 +22,7 @@ covered by this silhouette.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
@@ -29,6 +30,7 @@ from remove_ai_watermarks._text_mark_engine import (
|
||||
BottomRightAnchoredEngine,
|
||||
TextMarkConfig,
|
||||
)
|
||||
from remove_ai_watermarks._text_mark_variants import BottomRightBoxTextMarkEngine, TextMarkVariantsMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -79,7 +81,7 @@ def _alpha_template() -> NDArray[Any] | None:
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
class YuanbaoEngine(BottomRightAnchoredEngine):
|
||||
class YuanbaoEngine(TextMarkVariantsMixin, BottomRightAnchoredEngine):
|
||||
"""Detect and localize the bottom-right Yuanbao mark."""
|
||||
|
||||
_ANCHOR_MAX_RIGHT = 0.04
|
||||
@@ -87,3 +89,36 @@ class YuanbaoEngine(BottomRightAnchoredEngine):
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
self._variants = (
|
||||
BottomRightBoxTextMarkEngine(
|
||||
replace(
|
||||
_CONFIG,
|
||||
asset_name="yuanbao_compact_alpha.png",
|
||||
width_frac=0.28,
|
||||
height_frac=0.13,
|
||||
scale_basis="width",
|
||||
alpha_width_frac=0.09,
|
||||
alpha_height_frac=0.09 * 238 / 387 * 0.75,
|
||||
ladder=(0.95, 1.0, 1.05),
|
||||
detect_ncc_threshold=0.50,
|
||||
provenance_ncc_factor=0.62,
|
||||
max_saturation=255,
|
||||
)
|
||||
),
|
||||
BottomRightBoxTextMarkEngine(
|
||||
replace(
|
||||
_CONFIG,
|
||||
asset_name="yuanbao_compact_gray_alpha.png",
|
||||
width_frac=0.28,
|
||||
height_frac=0.13,
|
||||
scale_basis="width",
|
||||
alpha_width_frac=0.06,
|
||||
alpha_height_frac=0.06 * 238 / 387 * 0.90,
|
||||
ladder=(0.95, 1.0, 1.05),
|
||||
detect_frontend="gray",
|
||||
detect_ncc_threshold=0.50,
|
||||
provenance_ncc_factor=0.50,
|
||||
max_saturation=255,
|
||||
)
|
||||
),
|
||||
)
|
||||
@@ -14,6 +14,7 @@ ROOT = Path(__file__).resolve().parents[1]
|
||||
SKILL = ROOT / "skills" / "remove-ai-watermarks" / "SKILL.md"
|
||||
PLUGIN = ROOT / "skills" / ".claude-plugin" / "plugin.json"
|
||||
MARKETPLACE = ROOT / ".claude-plugin" / "marketplace.json"
|
||||
CODEX_PLUGIN = ROOT / ".codex-plugin" / "plugin.json"
|
||||
AGENT_DOC = ROOT / "docs" / "agent-skill.md"
|
||||
INSTALL_DOC = ROOT / "docs" / "installation.md"
|
||||
INSTALL_REFERENCE = ROOT / "skills" / "remove-ai-watermarks" / "references" / "install.md"
|
||||
@@ -55,6 +56,16 @@ def test_skill_frontmatter_matches_agent_skills_spec() -> None:
|
||||
assert "remove-ai-watermarks" in description
|
||||
|
||||
|
||||
def test_project_skills_stay_out_of_public_skill_discovery() -> None:
|
||||
# `npx skills add` and skills.sh discover every SKILL.md under .claude/skills;
|
||||
# metadata.internal hides a maintainer-only skill from that listing.
|
||||
project_skills = sorted((ROOT / ".claude" / "skills").glob("*/SKILL.md"))
|
||||
assert project_skills
|
||||
for path in project_skills:
|
||||
frontmatter = _frontmatter(path.read_text(encoding="utf-8"))
|
||||
assert re.search(r"(?m)^metadata:\n internal: true$", frontmatter), path
|
||||
|
||||
|
||||
def test_plugin_and_marketplace_names_agree() -> None:
|
||||
plugin = json.loads(PLUGIN.read_text(encoding="utf-8"))
|
||||
marketplace = json.loads(MARKETPLACE.read_text(encoding="utf-8"))
|
||||
@@ -531,15 +542,16 @@ def test_probe_minimum_version_is_a_release_that_exists() -> None:
|
||||
assert current >= probe.MIN_CLI_VERSION, "the probe demands a version the project has not released"
|
||||
|
||||
|
||||
def test_skill_version_agrees_across_all_three_manifests() -> None:
|
||||
"""Three files carry this version, and a catalog reads a different one from each."""
|
||||
def test_skill_version_agrees_across_every_manifest() -> None:
|
||||
"""Four files carry this version, and a catalog reads a different one from each."""
|
||||
frontmatter = _frontmatter(SKILL.read_text(encoding="utf-8"))
|
||||
match = re.search(r'(?m)^\s+version:\s*"?([0-9][^"\n]*?)"?\s*$', frontmatter)
|
||||
assert match is not None, "SKILL.md frontmatter no longer carries metadata.version"
|
||||
plugin = json.loads(PLUGIN.read_text(encoding="utf-8"))
|
||||
marketplace = json.loads(MARKETPLACE.read_text(encoding="utf-8"))
|
||||
codex = json.loads(CODEX_PLUGIN.read_text(encoding="utf-8"))
|
||||
|
||||
assert match.group(1) == plugin["version"] == marketplace["metadata"]["version"]
|
||||
assert match.group(1) == plugin["version"] == marketplace["metadata"]["version"] == codex["version"]
|
||||
|
||||
|
||||
def test_a_bad_invocation_shares_the_no_signal_exit_code(tmp_path: Path) -> None:
|
||||
|
||||
@@ -1326,3 +1326,29 @@ def test_unreadable_input_exits_cleanly(runner, tmp_path, name, content, cmd):
|
||||
else:
|
||||
assert result.exit_code == 1, result.output # cannot remove a mark from unreadable input
|
||||
assert "Error" in result.output
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("name", "content"),
|
||||
[
|
||||
("empty.png", b""),
|
||||
("notimage.jpg", b"plain text, not an image at all " * 20),
|
||||
("truncated.png", b"\x89PNG\r\n\x1a\n" + b"\x00" * 40),
|
||||
],
|
||||
)
|
||||
@pytest.mark.parametrize("force", [[], ["--force"]])
|
||||
def test_invisible_rejects_unreadable_input_instead_of_reporting_no_signal(runner, tmp_path, name, content, force):
|
||||
"""The scrub gate reads undecodable bytes as "no target", so without a decode
|
||||
check `invisible` exited 2 -- the no-signal code -- on a file it never read,
|
||||
while `all` and `batch` exited 1."""
|
||||
bad = tmp_path / name
|
||||
bad.write_bytes(content)
|
||||
mock_cls, mock_engine = _mock_invisible_engine()
|
||||
with (
|
||||
patch("remove_ai_watermarks.invisible_engine.InvisibleEngine", mock_cls),
|
||||
patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True),
|
||||
):
|
||||
result = runner.invoke(main, ["invisible", str(bad), "-o", str(tmp_path / "out.png"), *force])
|
||||
assert result.exit_code == 1, result.output
|
||||
assert "Failed to read image" in result.output
|
||||
mock_engine.remove_watermark.assert_not_called()
|
||||
@@ -0,0 +1,75 @@
|
||||
"""The Codex plugin manifest, held to the checks a Codex plugin catalog runs on it."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
MANIFEST = ROOT / ".codex-plugin" / "plugin.json"
|
||||
CLAUDE_PLUGIN = ROOT / "skills" / ".claude-plugin" / "plugin.json"
|
||||
|
||||
|
||||
def _manifest() -> dict:
|
||||
return json.loads(MANIFEST.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def test_manifest_carries_every_field_the_catalog_validator_requires() -> None:
|
||||
manifest = _manifest()
|
||||
for field in ("name", "version", "description", "repository", "license"):
|
||||
assert str(manifest.get(field, "")).strip(), field
|
||||
interface = manifest["interface"]
|
||||
for field in ("displayName", "shortDescription", "composerIcon"):
|
||||
assert str(interface.get(field, "")).strip(), field
|
||||
assert re.fullmatch(r"[a-z0-9]+(?:-[a-z0-9]+)*", manifest["name"])
|
||||
assert re.fullmatch(r"\d+\.\d+\.\d+", manifest["version"])
|
||||
assert re.fullmatch(r"#[0-9A-Fa-f]{6}", interface["brandColor"])
|
||||
|
||||
|
||||
def test_manifest_describes_the_same_plugin_as_the_claude_manifest() -> None:
|
||||
manifest = _manifest()
|
||||
claude = json.loads(CLAUDE_PLUGIN.read_text(encoding="utf-8"))
|
||||
for field in ("name", "description", "repository", "license", "homepage"):
|
||||
assert manifest[field] == claude[field], field
|
||||
|
||||
|
||||
def test_every_declared_path_stays_inside_the_repository() -> None:
|
||||
manifest = _manifest()
|
||||
interface = manifest["interface"]
|
||||
paths = [manifest["skills"], interface["composerIcon"], interface["logo"]]
|
||||
for declared in paths:
|
||||
assert declared.startswith("./"), declared
|
||||
resolved = (ROOT / declared).resolve()
|
||||
assert resolved.is_relative_to(ROOT), declared
|
||||
assert resolved.exists(), declared
|
||||
assert (ROOT / manifest["skills"] / "remove-ai-watermarks" / "SKILL.md").is_file()
|
||||
|
||||
|
||||
def test_icon_is_a_small_self_contained_svg() -> None:
|
||||
icon = ROOT / _manifest()["interface"]["composerIcon"]
|
||||
content = icon.read_text(encoding="utf-8")
|
||||
assert icon.suffix == ".svg"
|
||||
assert icon.stat().st_size <= 50 * 1024
|
||||
for forbidden in ("data:image/", "base64", "TODO", "PLACEHOLDER"):
|
||||
assert forbidden not in content, forbidden
|
||||
|
||||
|
||||
def test_no_tracked_code_matches_the_scanner_dynamic_execution_rule() -> None:
|
||||
"""The catalog's scanner reports a high finding for any call spelled like
|
||||
Python's built-in evaluator in its code files, and that rule cannot tell it from
|
||||
the PyTorch module method of the same name. Its scan job is advisory, but the
|
||||
finding still lands on the listing PR, so the code switches modules to inference
|
||||
with the equivalent ``module.train(False)``."""
|
||||
tracked = subprocess.run(
|
||||
["git", "ls-files", "*.py", "*.js", "*.mjs", "*.cjs", "*.ts", "*.tsx", "*.jsx"], # noqa: S607
|
||||
cwd=ROOT,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout.split()
|
||||
assert tracked
|
||||
rule = re.compile(r"\beval\s*\(")
|
||||
offenders = [name for name in tracked if rule.search((ROOT / name).read_text(encoding="utf-8", errors="ignore"))]
|
||||
assert offenders == []
|
||||
@@ -1,20 +1,27 @@
|
||||
"""Unit tests for how far --cpu-offload travels, per profile.
|
||||
|
||||
The flag sets two residency fields. The face one is read by the shared base and
|
||||
reaches every profile; the global one is read by ``QwenZImagePipeline`` alone, so
|
||||
on ``chroma-zimage`` and ``sdxl-zimage`` it never touches the stack the user is
|
||||
trying to fit. ``WatermarkRemover`` says so before the model load.
|
||||
reaches every profile; the global one is read by ``QwenZImagePipeline`` and
|
||||
``SdxlZImagePipeline``, so on ``chroma-zimage`` it never touches the stack the user
|
||||
is trying to fit. ``WatermarkRemover`` says so before the model load.
|
||||
|
||||
Both halves are exercised with an uninitialized remover, so the core CI matrix
|
||||
needs no diffusion dependency, model download, or GPU.
|
||||
needs no diffusion dependency, model download, or GPU. The SDXL loader placement is
|
||||
driven through stubbed Diffusers constructors and skips without the diffusion extra.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from remove_ai_watermarks._internal.sdxl_zimage_pipeline import (
|
||||
RESIDENT_SDXL_GLOBAL_MIN_VRAM_GIB,
|
||||
resolve_sdxl_global_residency,
|
||||
)
|
||||
from remove_ai_watermarks._internal.watermark_profiles import (
|
||||
AUTO_PROFILE,
|
||||
CHROMA_ZIMAGE_PROFILE,
|
||||
@@ -29,7 +36,7 @@ from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
|
||||
OFFLOAD_SUPPORT_CASES = (
|
||||
(QWEN_ZIMAGE_PROFILE, True),
|
||||
(CHROMA_ZIMAGE_PROFILE, False),
|
||||
(SDXL_ZIMAGE_PROFILE, False),
|
||||
(SDXL_ZIMAGE_PROFILE, True),
|
||||
# Auto preloads its concrete Qwen fallback before an image vendor is known.
|
||||
(AUTO_PROFILE, True),
|
||||
)
|
||||
@@ -69,18 +76,20 @@ class TestGlobalOffloadSupport:
|
||||
|
||||
|
||||
class TestUnsupportedProfileWarns:
|
||||
@pytest.mark.parametrize("profile", [CHROMA_ZIMAGE_PROFILE, SDXL_ZIMAGE_PROFILE])
|
||||
def test_it_names_the_profile_and_the_flag(self, profile: str, caplog: pytest.LogCaptureFixture) -> None:
|
||||
def test_it_names_the_profile_and_the_flag(self, caplog: pytest.LogCaptureFixture) -> None:
|
||||
with caplog.at_level(logging.WARNING):
|
||||
_remover(profile, cpu_offload=True)._warn_if_global_offload_unsupported()
|
||||
_remover(CHROMA_ZIMAGE_PROFILE, cpu_offload=True)._warn_if_global_offload_unsupported()
|
||||
assert len(caplog.records) == 1
|
||||
message = caplog.records[0].getMessage()
|
||||
assert "--cpu-offload" in message
|
||||
assert profile in message
|
||||
assert CHROMA_ZIMAGE_PROFILE in message
|
||||
|
||||
def test_qwen_zimage_is_quiet(self, caplog: pytest.LogCaptureFixture) -> None:
|
||||
@pytest.mark.parametrize("profile", [QWEN_ZIMAGE_PROFILE, SDXL_ZIMAGE_PROFILE])
|
||||
def test_a_profile_that_streams_its_global_stack_is_quiet(
|
||||
self, profile: str, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
with caplog.at_level(logging.WARNING):
|
||||
_remover(QWEN_ZIMAGE_PROFILE, cpu_offload=True)._warn_if_global_offload_unsupported()
|
||||
_remover(profile, cpu_offload=True)._warn_if_global_offload_unsupported()
|
||||
assert caplog.records == []
|
||||
|
||||
def test_a_run_that_did_not_ask_for_offload_is_quiet(self, caplog: pytest.LogCaptureFixture) -> None:
|
||||
@@ -132,3 +141,118 @@ class TestUnsupportedProfileWarns:
|
||||
with pytest.raises(ImportError):
|
||||
remover._load_qwen_zimage_pipeline()
|
||||
assert order == ["warned", "loaded"]
|
||||
|
||||
|
||||
class TestSdxlGlobalResidency:
|
||||
@pytest.mark.parametrize(
|
||||
("requested", "total_memory_gib", "expected"),
|
||||
[
|
||||
# Unset: the card decides, and an 8 GiB card cannot hold the stack.
|
||||
(None, 7.6, False),
|
||||
(None, RESIDENT_SDXL_GLOBAL_MIN_VRAM_GIB, True),
|
||||
(None, 23.5, True),
|
||||
# An unreadable card reports 0.0 and must stream rather than spill.
|
||||
(None, 0.0, False),
|
||||
# Explicit requests win in both directions.
|
||||
(False, 79.2, False),
|
||||
(True, 7.6, True),
|
||||
],
|
||||
)
|
||||
def test_request_wins_and_the_card_decides_otherwise(
|
||||
self, requested: bool | None, total_memory_gib: float, expected: bool
|
||||
) -> None:
|
||||
assert resolve_sdxl_global_residency(requested, total_memory_gib=total_memory_gib) is expected
|
||||
|
||||
|
||||
class _RecordingSdxlPipe:
|
||||
"""Stands in for the Diffusers SDXL pipeline and records placement calls in order."""
|
||||
|
||||
def __init__(self, calls: list[Any]) -> None:
|
||||
self.calls = calls
|
||||
self.scheduler = MagicMock()
|
||||
|
||||
def to(self, device: str) -> _RecordingSdxlPipe:
|
||||
self.calls.append(("to", device))
|
||||
return self
|
||||
|
||||
def load_lora_weights(self, _path: str) -> None:
|
||||
self.calls.append(("load_lora_weights",))
|
||||
|
||||
def fuse_lora(self) -> None:
|
||||
self.calls.append(("fuse_lora",))
|
||||
|
||||
def enable_sequential_cpu_offload(self, *, device: str, gpu_id: int | None = None) -> None:
|
||||
self.calls.append(("enable_sequential_cpu_offload", device, gpu_id))
|
||||
|
||||
|
||||
class TestSdxlGlobalPlacement:
|
||||
@staticmethod
|
||||
def _load(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
*,
|
||||
requested: bool | None,
|
||||
total_memory_gib: float,
|
||||
current_device: int = 0,
|
||||
) -> list[Any]:
|
||||
diffusers = pytest.importorskip("diffusers")
|
||||
torch = pytest.importorskip("torch")
|
||||
monkeypatch.setattr(torch.cuda, "current_device", lambda: current_device)
|
||||
from remove_ai_watermarks._internal.sdxl_zimage_pipeline import SdxlZImagePipeline
|
||||
|
||||
calls: list[Any] = []
|
||||
monkeypatch.setattr(diffusers.ControlNetModel, "from_pretrained", lambda *a, **k: MagicMock())
|
||||
monkeypatch.setattr(diffusers.AutoencoderKL, "from_pretrained", lambda *a, **k: MagicMock())
|
||||
monkeypatch.setattr(
|
||||
diffusers.StableDiffusionXLControlNetImg2ImgPipeline,
|
||||
"from_pretrained",
|
||||
lambda *a, **k: _RecordingSdxlPipe(calls),
|
||||
)
|
||||
monkeypatch.setattr("huggingface_hub.hf_hub_download", lambda *a, **k: "lora.safetensors")
|
||||
monkeypatch.setattr(diffusers.EulerDiscreteScheduler, "from_config", lambda *a, **k: MagicMock())
|
||||
|
||||
pipeline = SdxlZImagePipeline(device="cuda", torch_dtype=None, keep_global_models_on_device=requested)
|
||||
monkeypatch.setattr(type(pipeline), "_require_cuda", lambda self: None)
|
||||
monkeypatch.setattr(type(pipeline), "_total_vram_gib", lambda self: total_memory_gib)
|
||||
pipeline._load_global()
|
||||
return calls
|
||||
|
||||
def test_a_small_card_streams_the_fused_stack_instead_of_moving_it(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
calls = self._load(monkeypatch, requested=None, total_memory_gib=7.6)
|
||||
# The offload hooks stream their own copy of each weight, so they must follow the fuse.
|
||||
assert calls == [("load_lora_weights",), ("fuse_lora",), ("enable_sequential_cpu_offload", "cuda", 0)]
|
||||
|
||||
def test_a_large_card_keeps_the_stack_resident_as_before(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
calls = self._load(monkeypatch, requested=None, total_memory_gib=23.5)
|
||||
assert calls == [("to", "cuda"), ("load_lora_weights",), ("fuse_lora",)]
|
||||
|
||||
def test_cpu_offload_streams_even_on_a_large_card(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
calls = self._load(monkeypatch, requested=False, total_memory_gib=79.2)
|
||||
assert ("to", "cuda") not in calls
|
||||
assert calls[-1] == ("enable_sequential_cpu_offload", "cuda", 0)
|
||||
|
||||
@pytest.mark.parametrize("current_device", [1, 2])
|
||||
@pytest.mark.parametrize(("requested", "total_memory_gib"), [(None, 7.6), (False, 79.2)])
|
||||
def test_streaming_uses_the_current_cuda_device(
|
||||
self, monkeypatch: pytest.MonkeyPatch, current_device: int, requested: bool | None, total_memory_gib: float
|
||||
) -> None:
|
||||
calls = self._load(
|
||||
monkeypatch,
|
||||
requested=requested,
|
||||
total_memory_gib=total_memory_gib,
|
||||
current_device=current_device,
|
||||
)
|
||||
assert ("to", "cuda") not in calls
|
||||
assert calls[-1] == ("enable_sequential_cpu_offload", "cuda", current_device)
|
||||
|
||||
def test_cpu_offload_reaches_the_sdxl_stack(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
from remove_ai_watermarks._internal import sdxl_zimage_pipeline
|
||||
|
||||
captured: dict[str, object] = {}
|
||||
|
||||
class Recorder:
|
||||
def __init__(self, **kwargs: object) -> None:
|
||||
captured.update(kwargs)
|
||||
|
||||
monkeypatch.setattr(sdxl_zimage_pipeline, "SdxlZImagePipeline", Recorder)
|
||||
_remover(SDXL_ZIMAGE_PROFILE, cpu_offload=True)._load_qwen_zimage_pipeline()
|
||||
assert captured["keep_global_models_on_device"] is False
|
||||