Compare commits

..
Author SHA1 Message Date
Victor KuznetsovandClaude Opus 5.5 540d5648cc Update awesome-list PR statuses in the agent skill guide (#128)
* Update awesome-list PR statuses in the agent skill guide

VoltAgent and hashgraph lines predated both submissions; record PR 1225
(open), PR 541 (merged), the punkpeye MCP list PR 16924, the heilcheng
on-hold label and a 2026-10-19 recheck of the Composio PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep only durable listing facts in the awesome-list section

Dated review states drifted within weeks, which is how the section came
to call submitted and merged PRs DEFERRED and READY. Keep each list's
entry rules, the submission date and a PR link, and read live state from
GitHub with one gh search command instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 15:35:30 -07:00
Victor Kuznetsov a752bcda33 Merge pull request #127 from wiltodelta/fix/verified-output-publish
Publish the invisible output only after a clean strip; release 0.47.0
2026-10-19 14:47:14 -07:00
Victor KuznetsovandClaude Opus 5.5 04364efde2 Release 0.47.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:29:43 -07:00
Victor KuznetsovandClaude Opus 5.5 303aba6b8a Publish the invisible output only after a clean strip, and skip re-probing own encodes
The diffusion writer stripped its output in place and then raised on a
surviving marker, which left the AI-readable file at the output path. It now
writes to a staged sibling and publishes only when nothing survives, as
`visible`, `erase` and `remove_all` already do.

The package's own writers strip through the new
`metadata.strip_written_output`, which skips the readability probe:
`strip_and_verify` decoded every pixel of a file this run had just encoded,
doubling the cost of the strip. `strip_and_verify` keeps the probe for
user-supplied sources.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:29:43 -07:00
Victor Kuznetsov f6b7c366e8 Merge pull request #126 from wiltodelta/feat/watermark-library-followups
Fail closed on unreadable inputs, empty batches and failed strips
2026-10-19 12:32:39 -07:00
Victor Kuznetsov 30423aca5e Merge pull request #125 from wiltodelta/fix/isobmff-strip-keeps-offsets
Blank ISOBMFF provenance boxes in place so HEIF iloc offsets stay valid
2026-10-19 12:16:14 -07:00
Victor KuznetsovandClaude Opus 5.5 de6fe3d549 Bump aiohttp to 3.14.5 for GHSA-5wr8-gcqf-gqf3, GHSA-2g87-pp6c-29x5 and GHSA-494j-54xm-rwp3
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 11:58:05 -07:00
Victor KuznetsovandClaude Opus 5.5 7e3563b879 Blank ISOBMFF provenance boxes in place so HEIF iloc offsets stay valid
strip_c2pa_boxes dropped a top-level C2PA uuid, AI-label XMP uuid or jumb
box. When that box precedes mdat, dropping it moves the coded data while
HEIF iloc and MP4 stco/co64 still hold the old absolute offsets, so a phone
HEIC laid out as ftyp, uuid (C2PA), mdat, meta came out undecodable.
Turn the box into a same-size free box with a zeroed payload instead, as
strip_isobmff_media_file already does for MP4/MOV.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 11:08:39 -07:00
Victor Kuznetsov 76462f9916 Drive login-free web oracles through Playwright with a fresh proxy session
Gemini stays on the existing Chrome account. SynthID Detector starts as an
anonymous Playwright page and continues in that Chrome only when the page
asks for sign-in. Each ThorData run picks a new Europe or US session.
2026-10-18 17:44:21 -07:00
Victor KuznetsovandClaude Opus 5.5 3818f40096 CLAUDE.md: document the C2PA soft-binding snapshot check in the gate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 17:44:21 -07:00
Victor Kuznetsov 577d1bd953 Merge pull request #124 from wiltodelta/docs/glama-badge
Add the Glama MCP server badge to the README
2026-10-18 17:31:21 -07:00
Victor KuznetsovandClaude Opus 5.5 7376d278fd Add the Glama MCP server badge to the README
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 17:16:11 -07:00
28 changed files with 725 additions and 335 deletions

No files matched your search

+41 -24
View File
@@ -29,9 +29,15 @@ The unified maintainer entry point is `scripts/provider_oracles.py`; its workflo
slot schema live in `docs/provider-oracles.md`. Multi-account and multi-network work
selects each named slot explicitly from the gitignored repository-local
`.oracle-slots.json` by default and snapshots it into a separate immutable batch
manifest. A provider check may use the finite set of configured account slots and surfaces
after a refusal. Do not add new logins, IP rotation, network-route failover,
unbounded retries, or cross-session result merging. Never commit the local
manifest. A provider check walks that provider's existing order: the official API
surface, then its Web oracle, then the remaining configured account slots.
After a refusal, quota response, transport error, or indeterminate verdict,
record the response and continue with the next item as its own command. Do not
ask for confirmation before the switch. Anyone doing the task may run the
configured checks. Do not add new logins, IP rotation, or unbounded retries.
Do not walk a prepared route list or rotate addresses inside one call; the next
run is a new command and picks a new session. Do not merge results from
different sessions into one response. Never commit the local
slot file.
Every manual result stays bound to the exact sanitized upload hash and preserves the
provider response verbatim.
@@ -39,38 +45,49 @@ provider response verbatim.
Provider surface selection is API-first. OpenAI and Microsoft use their official API
adapters before their Web surfaces when the requested media and local configuration are
supported. Google and Meta currently use Web because this tooling has no usable API route
for them. A request to use a provider covers every configured surface, including Web,
without a second confirmation. An API refusal, quota response, transport error, or
indeterminate verdict stays on that surface and never silently submits the file to Web.
for them. Google's Web surfaces are `gemini-web` and `synthid-web`
(`https://synthid.com/`). After the current surface is exhausted, record the
response and continue with the next surface as its own command. One call does
not upload the same file to the next surface, and the verdicts stay separate.
Google Web oracle automation uses only the user's existing authenticated real Chrome
session. Gemini uses an explicit `/u/N/` account index. The SynthID Detector at
`https://synthid.com/` (`synthid-web`) uses the same Chrome profile and does not take a
`/u/N/` index; its terms forbid driving the page as an automated API. OpenAI, Microsoft, and Meta Web automation
uses isolated Playwright contexts; it must never attach to or copy state from the user's
real browser. Proxy credentials are read only through a slot's named environment variable
A signed-in account is required only for `gemini-web`. That surface uses the user's
existing authenticated real Chrome session and an explicit `/u/N/` account index.
Every other Web surface, including `synthid-web`, uses an isolated Playwright
context and may name a proxy. Those contexts must never attach to or copy state
from the user's real browser. `synthid-web` has no `/u/N/` index. Its terms forbid
bulk verification and quota evasion, so one run submits the prepared rows once
and does not start a sign-in. If the page says `Please sign in before detection.`
and returns no watermark label, keep that row and continue the same prepared
upload in the user's existing real Chrome. That Chrome session stays on the
direct route: no proxy, no new login, and no cookie copy into Playwright. If
that Chrome also asks for sign-in, leave its row unsubmitted. API calls
stay on the direct SDK transport and do not take `proxy_url_env`. Proxy credentials are read only through a slot's named environment variable
and never enter a manifest or log. Dotenv interpolation may derive routed proxy URLs from
one secret key without mutating the process environment. A selected proxy context may
accept that proxy's TLS interception certificate; direct contexts keep normal TLS checks.
ThorData rotating endpoints are pinned to one manifest-derived sticky session for the
life of a Web batch so one browser context does not change exit identity between page
requests. This is route continuity, not account or IP failover; a refusal still ends on
the selected slot and never selects another route automatically.
A ThorData exit is not prepared ahead of time. Each unauthenticated Web run picks
a new session and a country in Europe or the United States, and keeps that one exit
for the browser context so page requests do not change identity. A country or
`sessid` already written into the proxy URL is discarded. The next run picks again.
Upload readiness is surface-specific: OpenAI reaches it at page load but keeps background
requests alive, while Microsoft and Meta require network idle. Preserve exact provider
errors as indeterminate evidence rather than retrying or interpreting them as clean.
requests alive, while Microsoft, Meta, and SynthID Detector require network idle. Preserve exact provider
errors as indeterminate evidence, then continue with the next configured route or surface.
OpenAI API slots likewise carry only `api_key_env`. Store multiple key values in the
gitignored `.env`, select one slot explicitly, and never rotate to another token after a
refusal or rate limit. The process environment may override the matching `.env` value.
Their SDK transport sets `trust_env=False`; environment proxy variables must not reroute
an API check.
gitignored `.env` and select one slot explicitly. The process environment may override
the matching `.env` value. Their SDK transport sets `trust_env=False`, so ambient proxy
variables do not reroute an API check, and the slot does not take `proxy_url_env`.
After a refusal or rate limit, continue with the next configured slot in a new call,
then with the Web surface.
Microsoft API slots use the Azure Content Safety and private Blob Storage resource
identities from the gitignored `.env`. The adapter reads resource keys only into memory
through authenticated Azure CLI, downloads and hashes the private blob before requiring
equality with the named local source, keeps `trust_env=False`, never retries the submit,
and reports Watermark independently from C2PA.
equality with the named local source, keeps `trust_env=False`, and makes one submit
inside that call. Ambient proxy settings do not apply, and the slot does not take
`proxy_url_env`. The next slot or the Web surface is a new call and runs after a
refusal, quota response, transport error, or indeterminate verdict. Report
Watermark independently from C2PA.
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
+100 -78
View File
@@ -1,54 +1,65 @@
---
name: provider-oracles
description: Run hash-bound Google, OpenAI, Microsoft, or Meta watermark-oracle checks for this repository. Use for provider verification, multi-account Gemini checks, isolated Playwright checks, or recording oracle evidence.
description: Run hash-bound Google, OpenAI, Microsoft, or Meta watermark-oracle checks for this repository. Use for provider verification, multi-account Gemini checks, SynthID Detector checks at synthid.com, isolated Playwright checks, or recording oracle evidence. Walk the existing plan automatically when a task needs a provider verdict.
metadata:
internal: true
---
# Provider oracles
Read `docs/provider-oracles.md` before acting. Use `scripts/provider_oracles.py`
for surface discovery, slot validation, upload preparation, result recording,
and integrity verification.
Read `docs/provider-oracles.md` for surface keys, slot schema, preparation, and
recording commands. Use `scripts/provider_oracles.py` for surface discovery, slot
validation, upload preparation, result recording, and integrity verification.
Named slots live in the gitignored repository-local
`.oracle-slots.json`. Never commit that file. Pass `--slot NAME` to use it;
pass `--slots PATH` only to override the local path.
Never substitute one provider's negative verdict for another provider's signal.
Do not optimize candidates adaptively against an external oracle.
Do not optimize candidates adaptively against an external oracle. Keep each
submitted artifact frozen inside its own batch and bind the result to that
artifact's hash. Do not erase failures.
Anyone doing the task may run the configured checks. Do not ask who is allowed
to use the oracle, and do not ask for confirmation before moving to the next
surface, slot, or route.
## Order
Walk `plan PROVIDER`. That command only prints the order. The order itself stays:
1. The official API surface, when this tooling has one for that provider, media
type, and configured slot. OpenAI is `openai-api`. Microsoft is
`microsoft-api`.
2. That provider's Web oracle. OpenAI is `openai-web` and Microsoft is
`microsoft-web`, both through isolated Playwright. Meta is `meta-web`
through isolated Playwright. Google is `gemini-web` on the user's existing
real Chrome, then `synthid-web` at `https://synthid.com/` through isolated
Playwright. Meta and Google have no API surface, so their first Web oracle is
the first row.
3. The remaining configured account slots of those surfaces. A signed-in
account is required only for Gemini, and that slot stays on the direct
route. Every other Web surface may use isolated Playwright and a proxy.
That proxy exit is chosen when the run starts.
When the current surface, slot, or route is exhausted by a refusal, quota
response, transport error, or indeterminate verdict, record the exact response
and run the next item as its own command. One adapter call still makes one
request.
## Account continuation
A request to verify a bounded set of files with a provider authorizes using that
provider's configured account slots unless the user limits the account scope.
Before starting, identify the finite set of available authenticated slots and
prepare one immutable batch per slot. After a quota refusal or service failure,
record the exact response, leave that slot's remaining rows unsubmitted, and
continue automatically through the remaining authorized slots. Do not stop to
request permission for each account or file already covered by the task.
Before starting, identify the finite set of configured slots and prepare one
immutable batch per slot. After a quota refusal or service failure, record the
exact response, leave that slot's remaining rows unsubmitted, and continue
through the remaining slots in the order above.
Submit each frozen artifact at most once per slot in that pass. Preserve
inconclusive results and disagreements between accounts. Stop when the bounded
plan is complete or no authorized usable slot remains; report remaining rows.
New logins, another provider, or a new network route require authorization.
The oracles and slots already configured for the requested provider may be used
again, including their Web surfaces, without a second confirmation. Account
continuation does not authorize changing candidate pixels or erasing failures.
## Default surface priority
Prefer an official API adapter whenever that provider, media type, and configured
slot support it. For OpenAI and Microsoft, the default plans are API then Web.
Google and Meta currently have only usable Web routes in this tooling. Google's
routes are `gemini-web` and `synthid-web`. Inspect the order with `plan PROVIDER`.
The same request may use every configured surface.
The priority is a selection policy, not automatic failover. An API refusal,
quota response, transport error, or indeterminate verdict must be recorded and
reported. The same request already covers every configured surface of that
provider, including Web. Run Web as its own acknowledged check and keep its
verdict separate. Do not replace the API result with the Web result.
inconclusive results and disagreements between accounts. Stop when the plan is
complete or no configured usable slot or route remains, and report remaining
rows. Do not start a new login. Do not change candidate pixels. An API result
is not replaced by the Web result. Run Web as its own acknowledged check and
keep its verdict separate.
## Route by surface
@@ -69,10 +80,8 @@ Playwright CLI runner, copied cookies, or a new login.
AI metadata. Inspect the prepared upload again and do not submit it if a visible
provider mark remains. A model answer based on a visible logo is not a SynthID
result.
4. Confirm that the prepared files and provider are covered by the user's
request or existing upload approval. Ask only if that scope is missing or
changed; do not request confirmation again for each file or account in the
authorized plan. Do not upload the source path directly.
4. Upload the prepared files for the task. Pass the acknowledgement flag where
the command requires one. Do not upload the source path directly.
5. Attach one prepared upload and ask: `Does this image/video/audio contain a
SynthID watermark? Use the SynthID verification tool only; do not infer from
visible content, logos, text, or metadata.` Do not substitute a general
@@ -84,34 +93,43 @@ Playwright CLI runner, copied cookies, or a new login.
7. Record the verbatim response and verify the completed batch.
If a planned Chrome profile or `/u/N/` account is not already authenticated,
record that it is unavailable and continue with another authorized existing
session. Ask the user only when no usable authorized session remains. Do not
start a new login.
record that it is unavailable and continue with another configured existing
session. When no usable Gemini session remains, record that and continue with
`synthid-web`. Do not start a new login.
### SynthID Detector
Use the user's real authenticated Chrome. Do not use isolated Playwright: the
portal's additional terms forbid using the page as an automated API, bulk
verification, and quota evasion.
Use the isolated Playwright runner at `https://synthid.com/`. The page does not
require an account login, so the slot may name a proxy. Do not start a sign-in.
The portal's terms forbid bulk verification and quota evasion, so one invocation
submits the prepared rows once and does not retry.
1. Validate a `synthid-web` slot. It names `browser_profile` and `account_label`.
It does not use `google_account_index` or a proxy variable.
1. Validate a `synthid-web` slot. It does not name `account_label`,
`browser_profile`, or `google_account_index`. It may name `proxy_url_env`.
2. Prepare an immutable batch outside the repository. Inspect images the same
way as for Gemini, and do not submit a file that still shows a provider logo.
3. Open `https://synthid.com/` in the named Chrome profile. Accept the terms
dialog when it is shown. Do not start a new login. If the page requires
sign-in and that profile has no session, leave the row unsubmitted.
4. Attach one prepared upload through the page's own file control.
5. Read the settled label. `Watermark detected` is `detected`. `Watermark not
detected` is `not_detected`. Quota, sign-in, and unclear wording stay
`refused` or `indeterminate`.
3. Run `run-web` with the upload acknowledgement. The runner attaches one
prepared upload through the page's file input.
4. Read the settled label. `Watermark detected` is `detected`. `Watermark not
detected` and `SynthID was not detected` are `not_detected`. Quota and
unclear wording stay `refused` or `indeterminate`. Do not click a sign-in
button.
5. If the anonymous page says `Please sign in before detection.` and returns no
watermark label, keep that row. Open the same prepared upload in the user's
existing real Chrome, where a session is already signed in. Read
`~/Documents/GitHub/claude-config/docs/browser-automation.md` and use that
authenticated Chrome. Do not start a new login, do not copy cookies into
Playwright, and do not send that Chrome session through a proxy. If that
Chrome also asks for sign-in, leave its row unsubmitted.
6. Record the verbatim response. The result covers partner SynthID from Google
DeepMind, Nvidia, OpenAI, and Kakao, and it does not name which partner. Do
not record it as a Gemini-only or OpenAI-only verdict.
### OpenAI, Microsoft, and Meta Web
### OpenAI, Microsoft, Meta, and SynthID Detector Web
Use the isolated Playwright runner, never the user's real Chrome:
Use the isolated Playwright runner. Do not attach that runner to the user's
real Chrome. The sign-in fallback in the SynthID Detector section is a separate
session in the existing Chrome, not a proxy through this runner:
```bash
uv run python scripts/provider_oracles.py run-web /outside/repo/batch/manifest.json \
@@ -122,41 +140,43 @@ The runner is headed by default because OpenAI's Cloudflare gate blocks the
same isolated Chromium in headless mode. Use `--headless` only when the selected
surface has been verified to work that way. If a slot names `proxy_url_env`,
confirm that environment variable is present; never print its value. The runner
uses one context, one route, fresh page navigation per upload, and no retry.
CAPTCHA, bot checks, throttling, transport failure, and unknown page wording
remain distinct recorded outcomes.
uses one context, one route, and fresh page navigation per upload. One
invocation does not retry internally. CAPTCHA, bot checks, throttling, transport
failure, and unknown page wording remain distinct recorded outcomes. After any
of those, or after a refusal, run the next configured route or surface.
Wait for the surface-specific upload readiness that the runner implements.
OpenAI completes at page load but keeps background requests alive, while
Microsoft and Meta require network idle. Do not replace those conditions with
Microsoft, Meta, and SynthID Detector require network idle. Do not replace those conditions with
one shared wait. Treat OpenAI's exact `Something went wrong` response as a
settled `indeterminate` result, not as a clean result or a reason to retry.
settled `indeterminate` result, not as a clean result. Do not resubmit that
same page in the same batch; continue with the next configured route or surface.
For ThorData residential proxy mode, keep `THORDATA_ROUTE_US`,
`THORDATA_ROUTE_DE`, `THORDATA_ROUTE_GB`, `THORDATA_ROUTE_NL`, and
`THORDATA_ROUTE_JP` URLs in the gitignored `.env`;
OpenAI, Microsoft, and Meta Web slots may each name one of them through
`proxy_url_env`. Encode an explicit two-letter ISO country code in each route,
verify egress before relying on it, and never automatically change route after
a refusal. Bound a batch before starting.
A signed-in account is required only for Gemini. Every other Web surface,
including SynthID Detector, starts in isolated Playwright and may use a proxy.
When SynthID Detector asks for a sign-in, continue that one file in the user's
existing real Chrome, on the direct route. An
API call stays on the direct SDK transport and does not take a proxy.
ThorData user-and-password endpoints without `sessid` rotate the exit on each
request, which breaks a browser page whose resources must share one network
identity. The runner therefore derives one sticky `sessid` from the immutable
manifest hash and keeps it for that batch. A new prepared batch gets a new
session; rerunning or resuming the same batch keeps its session while the proxy
retains it. An explicit `sessid` in the configured route is preserved. This is
session continuity, not permission to retry a refused batch on another route.
A ThorData credential may live in one gitignored environment variable. Do not
prepare a country or a session in that URL or in a slot. Each run of an
unauthenticated Web oracle discards a country or `sessid` already present and
picks a new session in a country in Europe or the United States. That one
session stays for the browser context, because a ThorData endpoint without
`sessid` rotates the exit on each request and breaks a page whose resources
must share one network identity. The next run picks again. After a refusal,
start that next run; do not walk a list of prepared country routes.
### OpenAI API
Use `check-openai` with one explicit upload acknowledgement. Keep named API keys
as `OPENAI_API_KEY_1`, `OPENAI_API_KEY_2`, and `OPENAI_API_KEY_3` in the local `.env`, and select
only their variable names through an `openai-api` slot's `api_key_env`. Never
print a key or copy one into a slot or manifest. Continue through authorized
named API slots according to the bounded account plan above; keep a separate
batch for each. API slots use a direct SDK transport that ignores environment proxy
settings. The current adapter is image-only.
print a key or copy one into a slot or manifest. Continue through the configured
named API slots, with a separate batch for each, and then continue to
`openai-web`. API slots use a direct SDK transport that ignores environment
proxy settings. The current adapter is image-only. One adapter call does not
switch tokens; the next slot is a new call.
### Microsoft API
@@ -167,8 +187,10 @@ the gitignored `.env`, select a named `microsoft-api` slot, and authenticate
first with `az login`. Give the Content Safety system identity `Storage Blob
Data Reader` on the input storage account. The adapter reads resource keys only
into memory through Azure CLI, never logs them, ignores environment proxy
settings, makes one submit request with no retry, and polls only the returned
operation. Keep Microsoft's Watermark and C2PA markers as separate verdicts.
settings, makes one submit request with no retry inside that call, and polls
only the returned operation. After that call is refused, hits quota, fails, or
returns indeterminate, continue with another `microsoft-api` slot or with
`microsoft-web`. Keep Microsoft's Watermark and C2PA markers as separate verdicts.
## Evidence rules
+10 -4
View File
@@ -84,7 +84,13 @@ finding.
9. Run `bash maintain.sh`. Report evidence limits, incomplete rows, crashes,
unsupported formats, and oracle-unavailable cohorts explicitly.
External oracle uploads are separate mutations. Use the `provider-oracles`
skill only for a bounded, already-selected candidate, and obtain the required
action-time upload approval. Do not adapt candidates against an oracle, retry
automatically, or substitute one provider's verdict for another's.
External oracle uploads use the `provider-oracles` skill. When a cohort needs a
provider verdict, walk that skill's plan automatically: API first, then the Web
oracle, including `synthid-web` at synthid.com for Google, then the remaining
configured account slots. A signed-in account is required only for Gemini.
Every other Web surface, including `synthid-web`, may use isolated Playwright
and a proxy, and each proxied run picks a new session in Europe or the United
States. If SynthID Detector asks for a sign-in, continue that file in the user's
existing real Chrome on the direct route, and do not start a new login. Do not ask for
confirmation before switching, and do not adapt candidates against an oracle.
Do not substitute one provider's verdict for another's.
+1
View File
@@ -32,6 +32,7 @@ removal.
[![Tests](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml/badge.svg)](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml)
[![Sponsor](https://img.shields.io/badge/Sponsor-GitHub-db61a2?logo=githubsponsors&logoColor=white)](https://github.com/sponsors/wiltodelta)
[![skills.sh](https://skills.sh/b/wiltodelta/remove-ai-watermarks)](https://skills.sh/wiltodelta/remove-ai-watermarks)
[![Glama MCP server](https://glama.ai/mcp/servers/wiltodelta/remove-ai-watermarks/badges/score.svg)](https://glama.ai/mcp/servers/wiltodelta/remove-ai-watermarks)
> This is a research project on AI provenance signals, for lawful use on content
> you own. It is provided as is, without warranty or liability, and each user is
+32 -23
View File
@@ -52,35 +52,44 @@ Do these in order after the skill is on the default branch.
people run `npx skills add`. The listing is live and has recorded real
installs; the README install line remains the seed, not a registration.
4. **Awesome-list PRs.** Hand-curated GitHub lists are how people browse skills
today. One accurate PR per list, with the scope sentence intact.
- `ComposioHQ/awesome-claude-skills`: submitted 2026-09-05 as PR 1841
(Creative & Media); no maturity bar in its contribution rules. Rechecked
2026-09-11: still open, CI green, no review comments, waiting on the
list maintainers.
- `heilcheng/awesome-agent-skills`: submitted 2026-09-09 as PR 483 under
today. One accurate PR per list, with the scope sentence intact. This list
keeps each list's entry rules and the submission; read live review state
from GitHub (`gh search prs --author @me --state open remove-ai-watermarks`
lists every open one), not from here.
- `ComposioHQ/awesome-claude-skills`: submitted 2026-09-05 as
[PR 1841](https://github.com/ComposioHQ/awesome-claude-skills/pull/1841)
(Creative & Media); no maturity bar in its contribution rules.
- `heilcheng/awesome-agent-skills`: submitted 2026-09-09 as
[PR 483](https://github.com/heilcheng/awesome-agent-skills/pull/483) under
Community Skills > Media. The skill's three concrete input/output examples
satisfy the list's working-example requirement. Rechecked 2026-09-11:
still open; the only check failure is their Vercel deploy authorization,
satisfy the list's working-example requirement. The maintainer put it
`on hold` on 2026-10-15: "Tool strips AI provenance (SynthID/C2PA); policy
call for maintainer". Its Vercel check fails on deploy authorization,
which a contributor cannot grant.
- `VoltAgent/awesome-agent-skills`: DEFERRED. Its CONTRIBUTING rejects
brand-new skills ("give your skill time to mature and gain users before
submitting"). The skill now has real installs and shipped releases, but
adoption remains too small to make the list's maturity claim comfortably.
- `VoltAgent/awesome-agent-skills`: submitted 2026-10-17 as
[PR 1225](https://github.com/VoltAgent/awesome-agent-skills/pull/1225), a
one-line entry. Its CONTRIBUTING requires "real community usage" ("Give
your skill time to mature and gain users before submitting").
- `VoltAgent/awesome-openclaw-skills`: DEFERRED on adoption only. Entries
are indexed via clawskills.sh, and the ClawHub prerequisite in step 6 is
now met. The list still requires proven real-world usage. Re-attempt once
the ClawHub listing shows organic installs.
- `hashgraph-online/awesome-codex-plugins`: READY, not submitted. It lists
only Codex plugins, so it needs `.codex-plugin/plugin.json` and the icon in
`assets/icon.svg`. Its contribution CI runs `scripts/validate-plugin-pr.py`
against the repository's `HEAD.zip`, then the HOL plugin scanner with a
minimum score of 80 and no high finding; that scan job is
`continue-on-error`, so it reports rather than blocks. The scanner's `\beval\s*\(`
rule also matches PyTorch's `.eval()`, which is why the code calls
`.train(False)`. On 2026-10-17 `plugin-scanner` 3.49.0 scored the tree 86
with no high finding. The remaining loss is a tracked video over the
scanner's 16 MiB file budget. Recheck with
`uvx --from plugin-scanner==3.49.0 plugin-scanner scan . --min-score 80 --fail-on-severity high`.
- `hashgraph-online/awesome-codex-plugins`: merged 2026-10-19 as
[PR 541](https://github.com/hashgraph-online/awesome-codex-plugins/pull/541).
It lists only Codex plugins, so the entry depends on
`.codex-plugin/plugin.json` and the icon in `assets/icon.svg`. Its
contribution CI runs `scripts/validate-plugin-pr.py` against the
repository's `HEAD.zip`, then the HOL plugin scanner with a minimum score
of 80 and no high finding, as `continue-on-error`. The scanner's
`\beval\s*\(` rule also matches PyTorch's `.eval()`, which is why the code
calls `.train(False)`. Before changing the plugin, rescan with
`uvx --from plugin-scanner==3.49.0 plugin-scanner scan . --min-score 80 --fail-on-severity high`
(86 with no high finding on 2026-10-17; the loss is a tracked video over
the scanner's 16 MiB file budget).
- `punkpeye/awesome-mcp-servers`: submitted 2026-10-18 as
[PR 16924](https://github.com/punkpeye/awesome-mcp-servers/pull/16924) for
the MCP server. The list requires a Glama listing and the Glama score
badge; its bot checks both and labels the PR `has-glama`.
5. **Claude community marketplace.** Form at
[platform.claude.com/plugins/submit](https://platform.claude.com/plugins/submit)
(individuals) or the claude.ai directory form (Team/Enterprise). Validate
+14 -3
View File
@@ -68,10 +68,14 @@ Important contracts:
- `visible` and `erase` strip and verify AI metadata on a staged sibling and exit
`1` with nothing published when a marker survives; they used to swallow a failed
strip unless `--verbose` was set. The library `remove_visible` does the same and
raises `MetadataStripIncomplete`, as `remove_all` already did, and the diffusion
writer behind `invisible` raises when its own strip leaves a marker.
raises `MetadataStripIncomplete`, as `remove_all` already did. The diffusion
writer behind `invisible` stages the same way and raises with nothing published
(fixed 2026-10-19; it used to leave the marked file at the output path).
`_internal.utils.staged_sibling` is the shared stage: it never pre-creates the
name, so the published file keeps the default permissions.
name, so the published file keeps the default permissions. These writers strip
through `metadata.strip_written_output`, which skips the readability probe:
the file is the package's own encode, and the probe's full decode doubled the
cost of the strip (decided 2026-10-19).
- A quiet `metadata --check` and a successful `metadata --remove` (same for
`video metadata`) end by repeating the `identify` limit: the pixel channel is
untouched and a watermark such as SynthID has no local decoder once its
@@ -266,6 +270,13 @@ Publication is atomic, and a malformed top-level walk is copied unchanged,
although `strip_and_verify` refuses such a source before the stripper runs. A
generic `AIGC` key whose value has no TC260 field is ignored.
The in-memory image path (AVIF, HEIF, JPEG-XL and unknown suffixes, through
`strip_c2pa_boxes`) blanks the same top-level boxes to same-size `free` boxes.
It used to drop them, which moved `mdat` under an unchanged `iloc`: a phone HEIC
whose C2PA `uuid` box preceded `mdat` came out undecodable (2026-10-13).
`tests/test_metadata.py::TestIsobmffStripKeepsOffsets` decodes a cleaned HEIC to
guard it.
App-export JSON in a keyed or keyless metadata item is blanked the same way.
Dreamina's MP4 export writes it twice, under `LvMetaInfo` and
`com.apple.quicktime.artwork`. The file-level scan read both from the raw head
+89 -102
View File
@@ -1,9 +1,9 @@
# Provider oracle tooling
`scripts/provider_oracles.py` is the development-only entry point for the four
remote watermark-oracle families used by this project. It drives Google through
the user's real authenticated Chrome and drives the other public Web surfaces
through isolated Playwright. It keeps provider scope, execution identity,
remote watermark-oracle families used by this project. It drives Gemini through
the user's real authenticated Chrome and drives every other public Web surface,
including SynthID Detector, through isolated Playwright. It keeps provider scope, execution identity,
immutable media, and verbatim results in one format without exposing any oracle
through the installed CLI or public Python API.
@@ -13,8 +13,8 @@ through the installed CLI or public Python API.
| --- | --- | --- | --- |
| `openai-api` | OpenAI SynthID | Official Content Provenance API | Images |
| `microsoft-api` | Microsoft InvisMark | Azure Content Provenance Detection API | Images |
| `gemini-web` | Google SynthID | Signed-in Gemini web account | Images and videos |
| `synthid-web` | Partner SynthID | SynthID Detector at synthid.com, real Chrome | None recorded |
| `gemini-web` | Google SynthID | User's authenticated Chrome | Images and videos |
| `synthid-web` | Partner SynthID | Anonymous Playwright page at synthid.com | None recorded |
| `meta-web` | Meta Content Seal | Anonymous Meta identification page | Images |
| `microsoft-web` | Microsoft InvisMark | Public Azure AI validation page | Images |
| `openai-web` | OpenAI SynthID | Public web verifier | Images |
@@ -25,7 +25,7 @@ The default selection order is API first, then Web. OpenAI therefore plans
`openai-api` before `openai-web`, and Microsoft plans `microsoft-api` before
`microsoft-web`. Google and Meta currently have no usable API adapter in this
tooling, so each of their Web surfaces is primary. Google has two: `gemini-web`
and `synthid-web`.
and `synthid-web` at `https://synthid.com/`.
Inspect the machine-readable order before preparing a batch:
@@ -35,12 +35,15 @@ uv run python scripts/provider_oracles.py plan microsoft
uv run python scripts/provider_oracles.py plan google
```
A request to use a provider covers every configured surface of that provider,
including Web, without a second confirmation. API-first is still the order to
try, not a lock on the other surfaces. An API refusal, quota response, transport
failure, or indeterminate verdict is recorded as-is. It does not silently
upload the same file to Web; run that surface as its own acknowledged check and
keep the two verdicts separate.
API-first is the default order. Run the API surface when this tooling has a
usable route for the provider, media, and local configuration. Google and Meta
have no API surface, so each of their Web oracles is primary. For Google that
is `gemini-web`, then `synthid-web`. After a refusal, quota response, transport
failure, or indeterminate verdict, record that result and continue with the next
surface as its own acknowledged command, then the remaining configured account
slots of those surfaces. One call does not upload the same file to the next
surface. Keep the verdicts separate. Do not ask for confirmation before the
switch. Anyone doing the task may run the configured checks.
The old Google SynthID Detector portal at `labs.google/synthid` does not work.
On 2026-09-07 that entry point redirected a signed-in browser to a Google 404;
@@ -60,10 +63,14 @@ and Kakao, and it did not list Apple. It defined two settled labels:
`.tif`, `.gif`, `.wav`, `.mp3`, `.ogg`, `.flac`, `.aac`, `.m4a`, `.mp4`,
`.mov`, and `.webm`. This tooling still prepares only the project's existing
JPEG, PNG, WebP, MP4, MP3, and WAV suffixes. The portal's additional terms
forbid using the page as an automated API, bulk verification, and quota evasion,
so `run-web` does not drive it. Use the real Chrome workflow below. A
`Watermark detected` result is partner SynthID, not a Gemini-only or OpenAI-only
verdict.
forbid bulk verification and quota evasion. `run-web` may submit one prepared
upload through isolated Playwright, with a proxy when the slot names one, and
it does not start a sign-in. If the page says `Please sign in before detection.`
and returns no watermark label, keep that row and open the same prepared upload
in the user's existing real Chrome. That session stays on the direct route, with
no proxy and no new login. If that Chrome also asks for sign-in, leave its row
unsubmitted. A `Watermark detected` result is partner SynthID, not a Gemini-only
or OpenAI-only verdict.
Google's newer Search verification entry point did not provide an independent
oracle allowance in the session tested on 2026-09-13. Lens accepted the
metadata-stripped, previously Gemini-positive image
@@ -218,43 +225,20 @@ default; `--slots PATH` explicitly overrides it.
"network_label": "office"
},
{
"name": "synthid-personal",
"name": "synthid-proxy",
"surface": "synthid-web",
"account_label": "google-account-a",
"browser_profile": "Your Chrome",
"network_label": "home"
"account_label": null,
"browser_profile": null,
"network_label": "thordata",
"proxy_url_env": "THORDATA_PROXY"
},
{
"name": "openai-web-thordata-us",
"name": "openai-web-proxy",
"surface": "openai-web",
"account_label": null,
"browser_profile": null,
"network_label": "thordata-us-residential",
"proxy_url_env": "THORDATA_ROUTE_US"
},
{
"name": "microsoft-web-thordata-de",
"surface": "microsoft-web",
"account_label": null,
"browser_profile": null,
"network_label": "thordata-de-residential",
"proxy_url_env": "THORDATA_ROUTE_DE"
},
{
"name": "meta-web-thordata-us",
"surface": "meta-web",
"account_label": null,
"browser_profile": null,
"network_label": "thordata-us-residential",
"proxy_url_env": "THORDATA_ROUTE_US"
},
{
"name": "openai-web-thordata-de",
"surface": "openai-web",
"account_label": null,
"browser_profile": null,
"network_label": "thordata-de-residential",
"proxy_url_env": "THORDATA_ROUTE_DE"
"network_label": "thordata",
"proxy_url_env": "THORDATA_PROXY"
},
{
"name": "openai-api-1",
@@ -292,30 +276,34 @@ default; `--slots PATH` explicitly overrides it.
```
A routed batch selects one slot explicitly. A Gemini slot must name the Google
account, its real Chrome profile, and its nonnegative `/u/N/` account index. A
SynthID Detector slot names the account and the same real Chrome profile, and it
does not use `google_account_index`. An
API slot must name its account or project and cannot name a browser profile. The
account, its real Chrome profile, and its nonnegative `/u/N/` account index. An
API slot must name its account or project and cannot name a browser profile. A
Playwright slot for `openai-web`, `microsoft-web`, `meta-web`, or `synthid-web`
does not name an account or a browser profile. The
tool records the network route that the operator has already established.
Non-Google Web slots may name `proxy_url_env`; Playwright reads the secret proxy
URL from that environment variable without writing or logging it. The tool does
not merge sessions or automatically change network routes. Account continuation
is orchestrated by the skill, not by an individual adapter call. A request to
verify a bounded set of files with a provider covers its configured account slots
unless the operator restricts the account scope. Identify the finite set of
available authenticated slots before starting and prepare one immutable batch
per slot. Record quota refusals as `refused` and service failures according to
their actual response, leave remaining rows in that slot unsubmitted, and
continue automatically with the next authorized slot. Do not request permission
again for files or accounts already covered by the task.
Those four pages may name `proxy_url_env`; Playwright reads the
secret proxy URL from that environment variable without writing or logging it.
The country and session are not part of that preparation. Each run replaces
them with a new session in a country in Europe or the United States.
A signed-in account is required only for `gemini-web`. API calls stay on the
direct SDK transport and do not take `proxy_url_env`. Every other Web surface
uses isolated Playwright and may use a proxy. The tool does
not merge sessions inside one adapter call. Account continuation is orchestrated
by the skill, not by an individual adapter call. A request to verify files with
a provider covers its configured account slots, surfaces, and network routes.
Anyone doing the task may run those checks. Identify the finite set of
configured slots before starting and prepare one immutable batch per slot.
Record quota refusals as `refused` and service failures according to their
actual response, leave remaining rows in that slot unsubmitted, and continue
with the next slot, surface, or configured route. Do not ask for confirmation
before that switch.
Submit each frozen artifact at most once per slot in that pass. Keep every
Submit each frozen artifact at most once per slot in one pass. Keep every
verbatim refusal, inconclusive reply, and disagreement between accounts. Stop
when the planned checks finish or no authorized usable slot remains, and report
unsubmitted rows. A new login, another provider, or a new network route needs
authorization. The oracles and slots already configured for the requested
provider may be used again, including their Web surfaces, without a second
confirmation. Candidate pixels remain frozen throughout account continuation.
when the plan is complete or no configured usable slot, surface, or route
remains, and report anything still open. Do not start a new login. Each
submitted artifact stays frozen inside its own batch. Do not change candidate
pixels during the continuation.
Each `openai-api` slot must name exactly one `api_key_env`. Put the corresponding
values in the gitignored local `.env`, never in the slot JSON:
@@ -324,29 +312,27 @@ values in the gitignored local `.env`, never in the slot JSON:
OPENAI_API_KEY_1=replace-with-first-key
OPENAI_API_KEY_2=replace-with-second-key
OPENAI_API_KEY_3=replace-with-third-key
THORDATA_ROUTE_US=http://td-customer-USERNAME-country-US:PASSWORD@HOST.pr.thordata.net:9999
THORDATA_ROUTE_DE=http://td-customer-USERNAME-country-DE:PASSWORD@HOST.pr.thordata.net:9999
THORDATA_ROUTE_GB=http://td-customer-USERNAME-country-GB:PASSWORD@HOST.pr.thordata.net:9999
THORDATA_ROUTE_NL=http://td-customer-USERNAME-country-NL:PASSWORD@HOST.pr.thordata.net:9999
THORDATA_ROUTE_JP=http://td-customer-USERNAME-country-JP:PASSWORD@HOST.pr.thordata.net:9999
THORDATA_PROXY=http://td-customer-USERNAME:PASSWORD@HOST.pr.thordata.net:9999
```
The tool reads only the variable selected by the requested slot. A nonempty
process environment value overrides the matching `.env` entry. It never prints
the value or falls through to another token after authentication, entitlement,
quota, or transport failure. The no-slot compatibility path reads
the value. One call reads only its selected slot. After authentication,
entitlement, quota, or transport failure, continue with the next configured
slot in a new call. The no-slot compatibility path reads
`OPENAI_API_KEY`.
Several Google accounts may share one real Chrome profile: give them distinct
slot names and `google_account_index` values while keeping `browser_profile` the
same.
The ThorData examples use its [residential proxy country parameter](https://doc.thordata.com/doc/proxies/residential-proxies/location-settings/country-region).
The proxy username encodes one explicit two-letter ISO country code. Route
selection remains explicit and never changes after a refusal. OpenAI,
Microsoft, and Meta Web slots may reuse any route variable; the surface remains
part of each distinct slot. OpenAI API slots never accept `proxy_url_env` and
use the current direct network.
ThorData encodes the exit in the username with its [country parameter](https://doc.thordata.com/doc/proxies/residential-proxies/location-settings/country-region)
and a [session id](https://doc.thordata.com/doc/proxies/residential-proxies/session-control).
A stored `country-XX` or `sessid` is not the exit this run will use. The runner
discards both and appends one new session and one country from Europe or the
United States. That pair stays for the browser context of the run. The next run
chooses again, including after a refusal. Gemini and API slots never accept
`proxy_url_env`. Anonymous Playwright slots may.
Validate and inspect the slots:
@@ -419,16 +405,18 @@ uv sync --extra dev
uv run playwright install chromium
```
Google is intentionally excluded from the standalone runner. Both `gemini-web`
and `synthid-web` use the user's existing real Chrome profile, and the SynthID
Detector terms forbid driving that page as an automated API. Invoke the project
`provider-oracles` skill to prepare the batch and record the settled outcome.
For Gemini, select the named profile and `/u/N/` account. For SynthID Detector,
open `https://synthid.com/` in that profile. The portal has no `/u/N/` index.
Accept the terms dialog when it is shown, attach one prepared upload, and read
the explicit `Watermark detected` or `Watermark not detected` label. If the page
requires sign-in and that profile has no session, record the row unsubmitted
and do not start a new login.
Gemini is the only surface excluded from the standalone runner, because it is
the only one that requires a signed-in account. Invoke the project
`provider-oracles` skill to drive that existing Chrome profile at the named
`/u/N/` account. `synthid-web` is an anonymous page: `run-web` opens
`https://synthid.com/` in isolated Playwright, attaches one prepared upload
through the page's file input, and reads the explicit `Watermark detected`,
`Watermark not detected`, or `SynthID was not detected` label. The portal has
no `/u/N/` index. If the page says `Please sign in before detection.` and
returns no watermark label, keep that row and continue the same prepared upload
in the user's existing real Chrome. That session is direct: no proxy, no new
login, and no cookie copy into the isolated browser. If that Chrome also asks
for sign-in, leave its row unsubmitted.
For Gemini, ask the direct question stored in the batch manifest: `Does this
image/video/audio contain a SynthID watermark? Use the SynthID verification
@@ -437,8 +425,8 @@ general question about whether Google AI created or edited the media is not
equivalent: it may elicit origin or visual-content reasoning without a SynthID
verdict. SynthID Detector has no prompt; its settled labels are the verdict.
OpenAI Web, Microsoft Web, and Meta run entirely through a fresh isolated
Playwright browser. A direct-route example:
OpenAI Web, Microsoft Web, Meta, and SynthID Detector run through a fresh
isolated Playwright browser. A direct-route example:
```bash
uv run python scripts/provider_oracles.py prepare meta-web input.png \
@@ -449,8 +437,9 @@ uv run python scripts/provider_oracles.py run-web \
/tmp/meta-check-001/manifest.json --acknowledge-uploads
```
For a ThorData-routed OpenAI, Microsoft, or Meta Web slot, keep the generated
proxy URLs in the gitignored `.env`, then select one route explicitly:
For an OpenAI, Microsoft, Meta, or SynthID Detector slot that names a ThorData
credential, keep that URL in the gitignored `.env`. The run chooses the country
and the session:
```bash
uv run python scripts/provider_oracles.py run-web \
@@ -467,12 +456,10 @@ ThorData [user-and-password endpoints without a `sessid` rotate the exit on each
request](https://doc.thordata.com/doc/proxies/residential-proxies/session-control).
That is unsuitable inside one browser context: page resources can
arrive through different exits and trigger the provider's bot checks. For a
ThorData hostname, the runner derives a non-secret sticky session id from the
immutable manifest hash and appends a 90-minute session window when the
configured username has no explicit session. The same batch therefore keeps
one route while the provider retains it, a new prepared batch gets a distinct
route, and an operator-supplied `sessid` remains unchanged. This pins a selected
route; it does not retry or fail over after a refusal.
ThorData hostname, each run therefore discards a prepared country and session,
then keeps one new session and one country in Europe or the United States for
that browser context. The next invocation, including a retry after a refusal,
chooses again.
For a selected proxy route it enables certificate handling in that isolated
Chromium launch and context; direct launches keep Playwright's normal TLS
verification. A 2026-09-07 preflight sent one read-only request through each
@@ -489,7 +476,7 @@ quota refusal is `refused`, a transport failure is `unreachable`, and unknown or
changed page wording is `indeterminate`.
Each provider mounts its file input before the upload handler is necessarily
ready. Microsoft and Meta reach a usable handler at network idle. OpenAI keeps
ready. Microsoft, Meta, and SynthID Detector reach a usable handler at network idle. OpenAI keeps
background requests alive and never reaches network idle, so its observable
readiness seam is the page `load` state instead. The adapter waits for the
surface-specific state before assigning the file. OpenAI's static FAQ contains
+3 -1
View File
@@ -579,7 +579,9 @@ provides verification in Gemini and, since the public SynthID Detector opened,
at [synthid.com](https://synthid.com/). That site is the `synthid-web` oracle
surface in [`provider-oracles.md`](provider-oracles.md). On 2026-10-17 its FAQ
named Google DeepMind, Nvidia, OpenAI, and Kakao, and its settled labels were
`Watermark detected` and `Watermark not detected`. The old limited portal does
`Watermark detected` and `Watermark not detected`. On 2026-10-19 UTC the
signed-in page also used the status line `SynthID was not detected`, which this
tooling records as `not_detected`. The old limited portal does
not work: on 2026-09-07, its official `labs.google/synthid` entry point
redirected a signed-in browser to a Google 404. On 2026-09-23 the same entry
point redirected to `deepmind.google.com/frontiers/synthid`, which also returned
+1 -1
View File
@@ -4,7 +4,7 @@
# on it, including the ComfyUI node package. The console script below carries
# the same weight, since users have it on PATH.
name = "remove-ai-watermarks"
version = "0.46.0"
version = "0.47.0"
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
readme = "README.md"
requires-python = ">=3.11,<3.15"
+83 -14
View File
@@ -5,7 +5,9 @@ from __future__ import annotations
import logging
import os
import random
import re
import secrets
from dataclasses import dataclass
from datetime import UTC, datetime
from time import monotonic
@@ -22,7 +24,35 @@ if TYPE_CHECKING:
log = logging.getLogger(__name__)
PLAYWRIGHT_SURFACES = frozenset({"openai-web", "microsoft-web", "meta-web"})
PLAYWRIGHT_SURFACES = frozenset({"openai-web", "microsoft-web", "meta-web", "synthid-web"})
# A proxy exit is chosen when the run starts. These are the countries that
# choice may use: the United States and European ISO codes.
EUROPE_OR_US_COUNTRIES = (
"AT",
"BE",
"CH",
"CZ",
"DE",
"DK",
"ES",
"FI",
"FR",
"GB",
"GR",
"HU",
"IE",
"IT",
"NL",
"NO",
"PL",
"PT",
"RO",
"SE",
"US",
)
_THORDATA_COUNTRY = re.compile(r"-country-[A-Za-z]{2}", re.IGNORECASE)
_THORDATA_SESSION = re.compile(r"-sessid-[A-Za-z0-9]+", re.IGNORECASE)
_THORDATA_SESSION_TIME = re.compile(r"-sesstime-\d+", re.IGNORECASE)
@dataclass(frozen=True)
@@ -51,6 +81,11 @@ _RESULT_MARKERS = {
"No AI signatures from Meta were found",
"No AI signatures were found",
),
"synthid-web": (
"Watermark detected",
"Watermark not detected",
"SynthID was not detected",
),
}
_REFUSAL_MARKERS = (
@@ -63,7 +98,10 @@ _REFUSAL_MARKERS = (
"access denied",
)
_INDETERMINATE_MARKERS = ("Something went wrong. Please try again. If the issue persists, contact support.",)
_INDETERMINATE_MARKERS = (
"Something went wrong. Please try again. If the issue persists, contact support.",
"Please sign in before detection.",
)
def _contains_any(text: str, markers: tuple[str, ...]) -> bool:
@@ -111,6 +149,16 @@ def parse_provider_result(surface: str, page_text: str) -> WebVerdict:
return WebVerdict("detected", provenance, response)
if "inconclusive" in lowered or "no provenance detected" in lowered:
return WebVerdict("not_detected", provenance, response)
elif surface == "synthid-web":
# The exact labels are the verdict. A page that shows both, or that
# asks for a sign-in, has no single watermark result. The sign-in
# sentence is kept; the skill then continues in the user's existing Chrome.
detected = "Watermark detected" in response
not_detected = "Watermark not detected" in response or "SynthID was not detected" in response
if not_detected and not detected:
return WebVerdict("not_detected", "unavailable", response)
if detected and not not_detected:
return WebVerdict("detected", "unavailable", response)
else:
if "no ai signatures from meta were found" in lowered or "no ai signatures were found" in lowered:
return WebVerdict("not_detected", "unavailable", response)
@@ -123,13 +171,37 @@ def parse_provider_result(surface: str, page_text: str) -> WebVerdict:
return WebVerdict("indeterminate", provenance, response or "Provider page returned no readable result")
def choose_proxy_country(rng: random.Random | None = None) -> str:
"""Pick one exit country from Europe or the United States."""
picker = random.SystemRandom() if rng is None else rng
return picker.choice(EUROPE_OR_US_COUNTRIES)
def new_proxy_session() -> str:
"""Return a new ThorData session id. A later call does not reuse it."""
return "s" + secrets.token_hex(8)
def fresh_thordata_username(username: str, *, country: str, session_id: str) -> str:
"""Drop a prepared country and session, then attach the ones for this run."""
if country not in EUROPE_OR_US_COUNTRIES:
raise ValueError("proxy country must be in Europe or the US")
if re.fullmatch(r"[A-Za-z0-9]{1,32}", session_id) is None:
raise ValueError("proxy session id must be 1-32 ASCII letters or digits")
base = _THORDATA_SESSION_TIME.sub("", username)
base = _THORDATA_SESSION.sub("", base)
base = _THORDATA_COUNTRY.sub("", base).strip("-")
return f"{base}-country-{country}-sessid-{session_id}-sesstime-90"
def proxy_settings(
slot: oracles.ExecutionSlot,
environ: Mapping[str, str] = os.environ,
*,
sticky_session_id: str | None = None,
country: str | None = None,
session_id: str | None = None,
) -> ProxySettings | None:
"""Resolve a secret proxy URL and pin rotating ThorData routes per batch."""
"""Resolve a secret proxy URL. A ThorData exit is chosen for this call."""
variable = slot.get("proxy_url_env")
if variable is None:
return None
@@ -147,10 +219,11 @@ def proxy_settings(
if parsed.username is not None:
username = unquote(parsed.username)
is_thordata = parsed.hostname.endswith((".thordata.net", ".thordata.online"))
if is_thordata and sticky_session_id is not None and "-sessid-" not in username.casefold():
if re.fullmatch(r"[A-Za-z0-9]{1,32}", sticky_session_id) is None:
raise ValueError("sticky_session_id must be 1-32 ASCII letters or digits")
username = f"{username}-sessid-{sticky_session_id}-sesstime-90"
if is_thordata:
chosen_country = choose_proxy_country() if country is None else country
chosen_session = new_proxy_session() if session_id is None else session_id
username = fresh_thordata_username(username, country=chosen_country, session_id=chosen_session)
log.info("ThorData proxy exit country=%s session=%s", chosen_country, chosen_session)
settings["username"] = username
if parsed.password is not None:
settings["password"] = unquote(parsed.password)
@@ -198,7 +271,7 @@ def _set_upload_file(page: Page, surface: str, upload: Path, timeout_seconds: fl
# OpenAI keeps background requests alive, so network-idle is
# unreachable. Its visible input becomes handler-ready at page load.
page.wait_for_load_state("load", timeout=timeout_ms)
elif surface in {"meta-web", "microsoft-web"}:
elif surface in {"meta-web", "microsoft-web", "synthid-web"}:
# These pages mount hidden inputs before their selection handlers are
# active. Network-idle is the observable handler-readiness seam.
page.wait_for_load_state("networkidle", timeout=timeout_ms)
@@ -296,11 +369,7 @@ def run_web_batch(
if slot is None:
raise ValueError("Playwright batches require an explicit execution slot")
secret_values = dict(environ) if env_file is None else oracles.secret_environment(env_file, environ=environ)
proxy = proxy_settings(
slot,
secret_values,
sticky_session_id=f"oracle{results['manifest_sha256'][:16]}",
)
proxy = proxy_settings(slot, secret_values)
pending = [
(manifest_row, result_row)
for manifest_row, result_row in zip(manifest["rows"], results["rows"], strict=True)
+10 -7
View File
@@ -175,18 +175,18 @@ SURFACES: dict[str, OracleSurface] = {
provider="Google",
signal_family="synthid",
surface="SynthID Detector",
automation="real_browser",
automation="playwright",
url="https://synthid.com/",
media_types=("image", "video", "audio"),
project_evidence=(),
prompt=None,
clean_result_note=(
"Record detected only for the explicit label Watermark detected, and not_detected only for "
"Watermark not detected. The portal reports partner SynthID from Google DeepMind, Nvidia, "
"OpenAI, and Kakao without naming which partner. A not_detected result is not proof of "
"human authorship. Preserve quota, sign-in, and unclear wording."
"Record detected only for the explicit label Watermark detected, and not_detected for "
"Watermark not detected or SynthID was not detected. The portal reports partner SynthID from "
"Google DeepMind, Nvidia, OpenAI, and Kakao without naming which partner. A not_detected "
"result is not proof of human authorship. Preserve quota, sign-in, and unclear wording."
),
account_slots=True,
account_slots=False,
network_slots=True,
),
"meta-web": OracleSurface(
@@ -444,6 +444,7 @@ def load_slots(path: Path) -> dict[str, ExecutionSlot]:
raise ValueError(f"API surface {surface_key} does not use a browser_profile")
if surface_key not in real_browser and browser_profile is not None:
raise ValueError("only a real-browser surface uses a browser_profile")
# A proxy belongs to Playwright pages. Gemini and API calls stay direct.
if proxy_url_env is not None and surface_key not in playwright:
raise ValueError(f"surface {surface_key} does not use proxy_url_env")
if surface_key == "openai-api" and api_key_env is None:
@@ -894,6 +895,8 @@ def plan(provider: str, as_json: bool) -> None:
return
for row in rows:
click.echo(f"{row['preference']}\t{row['key']}\t{row['automation']}\t{row['surface']}")
if len(rows) > 1:
click.echo("This command only prints the order. After a failure, run the next surface as its own command.")
@cli.command()
@@ -1035,7 +1038,7 @@ def run_web(
timeout_seconds: float,
env_file: Path,
) -> None:
"""Submit a prepared non-Google Web batch through isolated Playwright."""
"""Submit a prepared anonymous Web batch through isolated Playwright."""
from provider_oracle_web import run_web_batch
report = run_web_batch(
+2 -2
View File
@@ -3,7 +3,7 @@
"name": "io.github.wiltodelta/remove-ai-watermarks",
"title": "Remove AI Watermarks",
"description": "Identify and remove AI watermarks and provenance metadata from your own images and video.",
"version": "0.46.0",
"version": "0.47.0",
"repository": {
"url": "https://github.com/wiltodelta/remove-ai-watermarks",
"source": "github"
@@ -14,7 +14,7 @@
"registryType": "pypi",
"registryBaseUrl": "https://pypi.org",
"identifier": "remove-ai-watermarks",
"version": "0.46.0",
"version": "0.47.0",
"runtimeHint": "uvx",
"transport": {
"type": "stdio"
+1 -1
View File
@@ -32,7 +32,7 @@ _GPU_MARKERS = ("GPU ", "gpu:")
# told an agent its CLI was current right before it typed a value that build
# rejects. tests/test_agent_skill.py pins it against the package version and
# against every "(since CLI X.Y.Z)" annotation in the skill.
MIN_CLI_VERSION = (0, 46, 0)
MIN_CLI_VERSION = (0, 47, 0)
# Markers a build without the pixel stack prints. The first is the current CLI's own
# install hint; the rest are what older builds emit before the guard existed, and the
# Homebrew formula ships exactly such a build.
+1 -1
View File
@@ -35,7 +35,7 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
__version__ = "0.46.0"
__version__ = "0.47.0"
__all__ = [
"BatchItemResult",
+24 -13
View File
@@ -736,10 +736,17 @@ def strip_isobmff_media_file(
return len(targets), len(tc260_key_spans | ai_metadata_key_spans | app_key_spans) + bool(kling_sei_offsets)
def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
"""Return ``(cleaned_bytes, stripped_count)`` with AI-provenance boxes removed.
def _blank_box(data: bytes, start: int, end: int, payload_off: int) -> bytes:
"""Return ``data[start:end]`` as a ``free`` box of the same length and size
encoding, with its payload zeroed, so no later file offset moves."""
return data[start : start + 4] + b"free" + data[start + 8 : payload_off] + bytes(end - payload_off)
Walks top-level boxes and drops:
def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
"""Return ``(cleaned_bytes, stripped_count)`` with AI-provenance boxes blanked.
Walks top-level boxes and turns each of these into a same-length ``free`` box
with a zeroed payload:
- any ``uuid`` box whose UUID equals ``C2PA_UUID`` (a C2PA manifest);
- any ``uuid`` box whose payload carries an AI-label marker (an XMP packet
with a TC260 / IPTC / IPTC-2025.1 AI field -- caught by content, not by the
@@ -747,10 +754,15 @@ def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
non-AI XMP intact);
- any ``jumb`` box (JPEG-XL JUMBF container).
All other boxes (incl. ``mdat`` / codestream) are emitted verbatim, so pixel
and audio data is preserved bit-for-bit. Non-ISOBMFF input is returned
The box is blanked rather than dropped because HEIF ``iloc`` and MP4
``stco``/``co64`` locate the coded data by absolute file offset: dropping a
box that precedes ``mdat`` moved the image without rewriting them, and a
phone HEIC cleaned that way no longer decoded (2026-10-13). This matches
:func:`strip_isobmff_media_file`. All other boxes (incl. ``mdat`` /
codestream) are emitted verbatim, so pixel and audio data is preserved
bit-for-bit. Non-ISOBMFF input is returned
unchanged. Despite the name this also covers MP4/MOV/M4A video and audio
(all ISOBMFF). NOTE: this drops only top-level boxes. AI metadata stored as an
(all ISOBMFF). NOTE: this blanks only top-level boxes. AI metadata stored as an
*item inside the ``meta`` box* (typical for AVIF/HEIF) is handled separately and
in place (same length, no offset rewrite): AI-label XMP by
:func:`blank_ai_xmp_packets`, and AI-generator tokens in an ``Exif`` item by
@@ -764,17 +776,16 @@ def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
consumed = 0
for start, end, box_type, payload_off in _iter_top_level_boxes(data):
consumed = end
provenance = box_type == b"jumb"
if box_type == b"uuid":
# uuid boxes carry the 16-byte UUID immediately after the type.
is_c2pa = payload_off + 16 <= end and data[payload_off : payload_off + 16] == C2PA_UUID
has_ai_label = any(marker in data[payload_off:end] for marker in _AI_LABEL_MARKERS)
if is_c2pa or has_ai_label:
stripped += 1
continue
elif box_type == b"jumb":
provenance = is_c2pa or any(marker in data[payload_off:end] for marker in _AI_LABEL_MARKERS)
if provenance:
stripped += 1
continue
out.extend(data[start:end])
out.extend(_blank_box(data, start, end, payload_off))
else:
out.extend(data[start:end])
# Fail-safe: the walker returns early on a malformed box (bad size, or a box
# that runs past EOF), so anything after it was never visited. Emitting `out`
@@ -212,17 +212,18 @@ class WatermarkRemover:
import numpy as np
from remove_ai_watermarks import image_io
from remove_ai_watermarks._internal.utils import staged_sibling
from remove_ai_watermarks.metadata import strip_written_output
output_path.parent.mkdir(parents=True, exist_ok=True)
bgr = np.ascontiguousarray(np.asarray(image.convert("RGB"))[:, :, ::-1])
if not image_io.imwrite(str(output_path), bgr, display_tags_from=display_tags_from):
image.save(output_path)
from remove_ai_watermarks.metadata import strip_and_verify
# Verified, so a marker that survives fails the run instead of shipping with it.
_, leftover = strip_and_verify(output_path, output_path, keep_standard=True)
if leftover:
raise RuntimeError(f"AI metadata survived the strip: {', '.join(sorted(leftover))}")
# Staged and verified, so a marker that survives fails the run with nothing
# published at output_path.
with staged_sibling(output_path) as staged:
if not image_io.imwrite(str(staged), bgr, display_tags_from=display_tags_from):
image.save(staged)
if leftover := strip_written_output(staged):
raise RuntimeError(f"AI metadata survived the strip: {', '.join(sorted(leftover))}")
def remove_watermark(
self,
+4 -6
View File
@@ -157,10 +157,9 @@ def _write_visible_result(
raise OSError(f"failed to write output (is the destination writable?): {out_path}")
if strip_metadata:
from remove_ai_watermarks.metadata import strip_and_verify
from remove_ai_watermarks.metadata import strip_written_output
_, leftover = strip_and_verify(staged, staged)
if leftover:
if leftover := strip_written_output(staged):
raise MetadataStripIncomplete(set(leftover))
@@ -464,7 +463,7 @@ def remove_all(
import tempfile
from remove_ai_watermarks import image_io, watermark_registry
from remove_ai_watermarks.metadata import strip_and_verify
from remove_ai_watermarks.metadata import strip_written_output
def say(stage: str, detail: str) -> None:
if progress is not None:
@@ -519,8 +518,7 @@ def remove_all(
# ── 3. AI metadata ──
# Read the pristine ORIGINAL for provenance above and the STAGED file here:
# the visible pass has already dropped this file's C2PA.
_, leftover = strip_and_verify(staged, staged)
if leftover:
if leftover := strip_written_output(staged):
# Before the write, on purpose -- see MetadataStripIncomplete.
raise MetadataStripIncomplete(set(leftover))
say("metadata", "stripped")
+2 -2
View File
@@ -467,7 +467,7 @@ def _write_output_or_exit(
published, so a marker that survives exits 1 with nothing written. Both commands
using this used to swallow a failed strip unless ``--verbose`` was set."""
from remove_ai_watermarks._internal.utils import staged_sibling
from remove_ai_watermarks.metadata import strip_and_verify
from remove_ai_watermarks.metadata import strip_written_output
output.parent.mkdir(parents=True, exist_ok=True)
with staged_sibling(output) as staged:
@@ -477,7 +477,7 @@ def _write_output_or_exit(
if not strip_metadata:
return
try:
_, leftover = strip_and_verify(staged, staged)
leftover = strip_written_output(staged)
except (OSError, RuntimeError, ValueError) as e:
console.print(f" Error: failed to strip AI metadata from {output}: {e}")
raise SystemExit(1) from e
+19
View File
@@ -1968,6 +1968,25 @@ def strip_and_verify(
"""
if (reason := unreadable_reason(source_path)) is not None:
raise ValueError(f"cannot read {source_path.name}: {reason}")
return _strip_and_rescan(source_path, output_path, keep_standard=keep_standard)
def strip_written_output(path: Path, *, keep_standard: bool = True) -> dict[str, str]:
"""Strip ``path`` in place and return the markers that survived.
For a file whose pixels this run has already decoded or written itself, so the
readability probe that :func:`strip_and_verify` runs first would only decode
them a second time.
"""
return _strip_and_rescan(path, path, keep_standard=keep_standard)[1]
def _strip_and_rescan(
source_path: Path,
output_path: Path | None,
*,
keep_standard: bool,
) -> tuple[Path, dict[str, str]]:
out = remove_ai_metadata(source_path, output_path, keep_standard=keep_standard)
remaining = get_ai_metadata(out)
if not remaining:
+1 -1
View File
@@ -430,7 +430,7 @@ class TestRemoveAllLibrary:
from remove_ai_watermarks import api, invisible_engine, metadata
monkeypatch.setattr(invisible_engine, "is_available", lambda: False)
monkeypatch.setattr(metadata, "strip_and_verify", lambda src, dst: (dst, {"c2pa"}))
monkeypatch.setattr(metadata, "strip_written_output", lambda _path: {"c2pa": "survived"})
out = tmp_path / "clean.png"
with pytest.raises(api.MetadataStripIncomplete, match="c2pa"):
+1 -4
View File
@@ -675,10 +675,7 @@ class TestAllCommand:
# The no-signal gate lives in the library now; patch it there, not on the
# CLI module, or the diffusion stage runs for real.
patch("remove_ai_watermarks.api._SourceEvidence.has_invisible_target", return_value=False),
patch(
"remove_ai_watermarks.metadata.strip_and_verify",
return_value=(tmp_path / "intermediate.png", {"c2pa": True}),
),
patch("remove_ai_watermarks.metadata.strip_written_output", return_value={"c2pa": "survived"}),
):
result = runner.invoke(main, ["all", str(sample_png), "-o", str(output)])
+23
View File
@@ -239,6 +239,29 @@ class TestSkippedStepsAreFailures:
remover = object.__new__(WatermarkRemover) # the writer needs no model state
with pytest.raises(RuntimeError, match="survived"):
remover._write_output(Image.new("RGB", (16, 16)), tmp_path / "out.png")
assert list(tmp_path.iterdir()) == [], "an output that kept its AI metadata was published"
@pytest.mark.parametrize("command", ["erase", "remove_visible", "invisible_writer"])
def test_an_output_the_package_just_wrote_is_not_probed_again(self, tmp_path, clean_png, monkeypatch, command):
"""The readability probe decodes every pixel; on our own encode it is pure waste."""
from PIL import Image
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
from remove_ai_watermarks.api import remove_visible
probed: list[Path] = []
real = metadata.unreadable_reason
monkeypatch.setattr(metadata, "unreadable_reason", lambda path: probed.append(path) or real(path))
out = tmp_path / "out.png"
if command == "erase":
result = _invoke(["erase", "--region", "2,2,10,10", str(clean_png), "-o", str(out)])
assert result.exit_code == 0, result.output
elif command == "remove_visible":
remove_visible(clean_png, out, write_noop=True)
else:
object.__new__(WatermarkRemover)._write_output(Image.new("RGB", (16, 16)), out)
assert out.exists()
assert [p for p in probed if p.name != clean_png.name] == []
def test_metadata_remove_reports_a_missing_ffmpeg_without_a_traceback(self, tmp_path, monkeypatch):
monkeypatch.setattr(shutil, "which", lambda _name: None)
+95 -7
View File
@@ -29,6 +29,12 @@ from remove_ai_watermarks.metadata import (
xai_signature,
)
def _blanked(box: bytes) -> bytes:
"""The same-length ``free`` box ``strip_c2pa_boxes`` leaves in place of a 32-bit-size box."""
return box[:4] + b"free" + bytes(len(box) - 8)
# Real, committed C2PA sample images used to ground the SynthID-source tests.
SAMPLES_DIR = Path(__file__).resolve().parent.parent / "data" / "fixtures" / "provenance"
@@ -123,7 +129,7 @@ class TestHasAiMetadata:
assert has_ai_metadata(path)
def test_strip_c2pa_boxes_removes_uuid_box(self, tmp_path: Path):
"""ISOBMFF strip should drop the C2PA uuid box and keep everything else."""
"""ISOBMFF strip should blank the C2PA uuid box in place and keep everything else."""
from remove_ai_watermarks._internal.isobmff import strip_c2pa_boxes
from remove_ai_watermarks.metadata import C2PA_UUID
@@ -133,7 +139,7 @@ class TestHasAiMetadata:
mdat = b"\x00\x00\x00\x10mdat" + b"pixeldat"
cleaned, stripped = strip_c2pa_boxes(ftyp + uuid_box + mdat)
assert stripped == 1
assert cleaned == ftyp + mdat
assert cleaned == ftyp + _blanked(uuid_box) + mdat
def test_strip_c2pa_boxes_passthrough_for_non_isobmff(self):
"""Non-ISOBMFF input must be returned unchanged."""
@@ -145,7 +151,7 @@ class TestHasAiMetadata:
assert cleaned == data
def test_remove_ai_metadata_strips_c2pa_in_avif(self, tmp_path: Path):
"""End-to-end: ``remove_ai_metadata`` on a fake .avif drops the C2PA box."""
"""End-to-end: ``remove_ai_metadata`` on a fake .avif blanks the C2PA box."""
from remove_ai_watermarks.metadata import C2PA_UUID, remove_ai_metadata
src = tmp_path / "in.avif"
@@ -157,7 +163,7 @@ class TestHasAiMetadata:
out = tmp_path / "out.avif"
result = remove_ai_metadata(src, out)
assert result == out
assert out.read_bytes() == ftyp + mdat
assert out.read_bytes() == ftyp + _blanked(uuid_box) + mdat
# And after stripping, detection must no longer flag the cleaned file.
from remove_ai_watermarks.metadata import has_ai_metadata
@@ -2087,7 +2093,7 @@ class TestIsobmffMetadataRemoval:
"""Container-level AI-provenance stripping across ISOBMFF image/video/audio."""
def test_strips_ai_xmp_uuid_box(self):
# A uuid box carrying a TC260 AIGC label is dropped by content match,
# A uuid box carrying a TC260 AIGC label is blanked by content match,
# regardless of the (non-C2PA) XMP UUID's byte order.
from remove_ai_watermarks._internal.isobmff import strip_c2pa_boxes
@@ -2096,7 +2102,7 @@ class TestIsobmffMetadataRemoval:
box = (24 + len(payload)).to_bytes(4, "big") + b"uuid" + xmp_uuid + payload
cleaned, stripped = strip_c2pa_boxes(_MP4_FTYP + box + _MP4_MDAT)
assert stripped == 1
assert cleaned == _MP4_FTYP + _MP4_MDAT
assert cleaned == _MP4_FTYP + _blanked(box) + _MP4_MDAT
def test_keeps_plain_non_ai_xmp(self):
# A uuid box with ordinary (non-AI) XMP must be preserved.
@@ -2128,7 +2134,7 @@ class TestIsobmffMetadataRemoval:
src.write_bytes(_MP4_FTYP + uuid_box + _MP4_MDAT)
out = tmp_path / "out.bin"
remove_ai_metadata(src, out)
assert out.read_bytes() == _MP4_FTYP + _MP4_MDAT
assert out.read_bytes() == _MP4_FTYP + _blanked(uuid_box) + _MP4_MDAT
def test_unparseable_audio_raises(self, tmp_path: Path):
# Garbage that ffmpeg can't parse must raise a clear error, not crash in
@@ -2560,3 +2566,85 @@ class TestTc260AudioPlacements:
cleaned = tmp_path / f"clean{suffix}"
remove_ai_metadata(source, cleaned)
assert "aigc_label" not in get_ai_metadata(cleaned)
def _shift_heif_iloc(data: bytes, delta: int) -> bytes:
"""Add ``delta`` to every file-offset extent in the HEIF ``iloc`` box.
Lets a test insert a box ahead of ``mdat`` the way a C2PA signer does, which
moves the coded image by ``delta`` bytes and so needs ``iloc`` rewritten.
"""
out = bytearray(data)
meta = data.index(b"meta") - 4
iloc = data.index(b"iloc", meta) - 4
version = data[iloc + 8]
pos = iloc + 12
offset_size, length_size = data[pos] >> 4, data[pos] & 15
base_size = data[pos + 1] >> 4
index_size = data[pos + 1] & 15 if version in (1, 2) else 0
pos += 2
id_size = 2 if version < 2 else 4
count = int.from_bytes(data[pos : pos + id_size], "big")
pos += id_size
for _ in range(count):
pos += id_size
method = 0
if version in (1, 2):
method = data[pos + 1] & 15
pos += 2
pos += 2 # data_reference_index
base = int.from_bytes(data[pos : pos + base_size], "big") if base_size else 0
if method == 0 and base_size:
out[pos : pos + base_size] = (base + delta).to_bytes(base_size, "big")
pos += base_size
extents = int.from_bytes(data[pos : pos + 2], "big")
pos += 2
for _ in range(extents):
pos += index_size
if method == 0 and not base_size:
offset = int.from_bytes(data[pos : pos + offset_size], "big")
out[pos : pos + offset_size] = (offset + delta).to_bytes(offset_size, "big")
pos += offset_size + length_size
return bytes(out)
class TestIsobmffStripKeepsOffsets:
"""A HEIF locates its coded image through absolute ``iloc`` offsets, and an MP4
through ``stco``/``co64``. Dropping a C2PA ``uuid`` box that sits ahead of
``mdat`` moved the image without rewriting them, so the cleaned HEIC no longer
decoded (a phone HEIC, 2026-10-13)."""
def test_c2pa_box_before_mdat_keeps_heic_decodable(self, tmp_path: Path):
pillow_heif = pytest.importorskip("pillow_heif")
pillow_heif.register_heif_opener()
plain = tmp_path / "plain.heic"
Image.new("RGB", (64, 48), (200, 30, 30)).save(plain, format="HEIF")
data = plain.read_bytes()
c2pa_box = struct.pack(">I", 8 + 16 + 64) + b"uuid" + C2PA_UUID + b"\x07" * 64
mdat = data.index(b"mdat") - 4
signed = _shift_heif_iloc(data[:mdat], len(c2pa_box)) + c2pa_box + data[mdat:]
source = tmp_path / "signed.heic"
source.write_bytes(signed)
with Image.open(source) as im:
expected = im.convert("RGB").tobytes()
cleaned = remove_ai_metadata(source, tmp_path / "clean.heic")
assert C2PA_UUID not in cleaned.read_bytes()
with Image.open(cleaned) as im:
assert im.convert("RGB").tobytes() == expected
def test_blanked_box_keeps_every_later_offset(self):
from remove_ai_watermarks._internal.isobmff import strip_c2pa_boxes
ftyp = b"\x00\x00\x00\x18ftypheic\x00\x00\x00\x00mif1heic"
c2pa_box = struct.pack(">I", 8 + 16 + 8) + b"uuid" + C2PA_UUID + b"manifest"
mdat = b"\x00\x00\x00\x10mdat" + b"pixeldat"
data = ftyp + c2pa_box + mdat
cleaned, stripped = strip_c2pa_boxes(data)
assert stripped == 1
assert len(cleaned) == len(data)
assert cleaned.index(b"mdat") == data.index(b"mdat")
assert cleaned == ftyp + _blanked(c2pa_box) + mdat
+2 -2
View File
@@ -595,11 +595,11 @@ class TestISOBMFF:
assert not is_isobmff(b"abc")
def test_strips_jpegxl_jumb_box(self):
"""JPEG-XL stores JUMBF in a ``jumb`` box, always stripped."""
"""JPEG-XL stores JUMBF in a ``jumb`` box, always blanked to ``free``."""
jumb = struct.pack(">I", 8 + 5) + b"jumb" + b"hello"
cleaned, stripped = strip_c2pa_boxes(FTYP + jumb)
assert stripped == 1
assert cleaned == FTYP
assert cleaned == FTYP + struct.pack(">I", 8 + 5) + b"free" + bytes(5)
def test_keeps_non_c2pa_box_with_64bit_size(self):
"""size==1 means a 64-bit largesize follows; non-C2PA box is kept."""
+84 -19
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import json
import re
from typing import TYPE_CHECKING
import provider_oracle_web as web
@@ -76,6 +77,36 @@ def _thordata_slot() -> oracles.ExecutionSlot:
"not_detected",
"unavailable",
),
(
"synthid-web",
"Watermark detected",
"detected",
"unavailable",
),
(
"synthid-web",
"Watermark not detected",
"not_detected",
"unavailable",
),
(
"synthid-web",
"SynthID was not detected\nIt is unlikely that this media was made with AI from a SynthID partner",
"not_detected",
"unavailable",
),
(
"synthid-web",
"Please sign in before detection.",
"indeterminate",
"unavailable",
),
(
"synthid-web",
"Watermark detected\nWatermark not detected",
"indeterminate",
"unavailable",
),
],
)
def test_parse_provider_result(
@@ -122,7 +153,12 @@ def test_openai_provider_error_is_a_settled_indeterminate_result() -> None:
@pytest.mark.parametrize(
("surface", "ready_state"),
[("meta-web", "networkidle"), ("microsoft-web", "networkidle"), ("openai-web", "load")],
[
("meta-web", "networkidle"),
("microsoft-web", "networkidle"),
("openai-web", "load"),
("synthid-web", "networkidle"),
],
)
def test_react_uploads_wait_for_surface_readiness_before_assigning_the_file(
tmp_path: Path,
@@ -183,29 +219,44 @@ def test_proxy_settings_are_loaded_from_an_environment_reference() -> None:
}
def test_thordata_proxy_settings_pin_one_exit_for_the_browser_session() -> None:
def test_thordata_proxy_settings_take_a_fresh_europe_or_us_session() -> None:
proxy = web.proxy_settings(
_thordata_slot(),
{"THORDATA_ROUTE_US": "http://route:secret@gateway.pr.thordata.net:9999"},
sticky_session_id="oracle0123456789abcdef",
{
"THORDATA_ROUTE_US": "http://td-customer-user-country-JP-sessid-prepared-sesstime-15:secret@gateway.pr.thordata.net:9999"
},
country="DE",
session_id="freshsession01",
)
assert proxy == {
"server": "http://gateway.pr.thordata.net:9999",
"username": "route-sessid-oracle0123456789abcdef-sesstime-90",
"username": "td-customer-user-country-DE-sessid-freshsession01-sesstime-90",
"password": "secret",
}
def test_thordata_proxy_settings_preserve_an_explicit_session() -> None:
proxy = web.proxy_settings(
_thordata_slot(),
{"THORDATA_ROUTE_US": ("http://route-sessid-manual-sesstime-15:secret@gateway.pr.thordata.net:9999")},
sticky_session_id="oracle0123456789abcdef",
)
def test_thordata_proxy_settings_reject_a_country_outside_europe_or_the_us() -> None:
assert "US" in web.EUROPE_OR_US_COUNTRIES
assert "DE" in web.EUROPE_OR_US_COUNTRIES
assert "JP" not in web.EUROPE_OR_US_COUNTRIES
assert proxy is not None
assert proxy["username"] == "route-sessid-manual-sesstime-15"
with pytest.raises(ValueError, match="Europe or the US"):
web.proxy_settings(
_thordata_slot(),
{"THORDATA_ROUTE_US": "http://route:secret@gateway.pr.thordata.net:9999"},
country="JP",
session_id="freshsession01",
)
def test_new_proxy_session_changes_between_calls() -> None:
first = web.new_proxy_session()
second = web.new_proxy_session()
assert first != second
assert re.fullmatch(r"[A-Za-z0-9]{1,32}", first)
assert re.fullmatch(r"[A-Za-z0-9]{1,32}", second)
def test_proxy_settings_do_not_treat_a_missing_secret_as_direct_access() -> None:
@@ -356,6 +407,8 @@ def test_run_web_batch_loads_the_selected_proxy_from_dotenv(
proxies.append(proxy)
return [web.WebVerdict("not_detected", "unavailable", "No OpenAI signals detected")]
monkeypatch.setattr(web, "choose_proxy_country", lambda rng=None: "NL")
monkeypatch.setattr(web, "new_proxy_session", lambda: "freshsession01")
monkeypatch.setattr(web, "_submit_batch", submit)
report = web.run_web_batch(
manifest_path,
@@ -367,11 +420,10 @@ def test_run_web_batch_loads_the_selected_proxy_from_dotenv(
)
assert report["watermark_results"] == {"not_detected": 1}
manifest_sha256 = json.loads((manifest_path.parent / "results.json").read_text(encoding="utf-8"))["manifest_sha256"]
assert proxies == [
{
"server": "http://proxy.pr.thordata.net:9999",
"username": (f"td-customer-user-country-US-sessid-oracle{manifest_sha256[:16]}-sesstime-90"),
"username": "td-customer-user-country-NL-sessid-freshsession01-sesstime-90",
"password": "test-thordata-password",
}
]
@@ -392,14 +444,12 @@ def test_run_web_batch_requires_explicit_upload_acknowledgement(
web.run_web_batch(manifest_path, acknowledge_uploads=False)
@pytest.mark.parametrize("surface", ["gemini-web", "synthid-web"])
def test_run_web_batch_refuses_google_real_browser_workflow(
def test_run_web_batch_refuses_gemini_real_browser_workflow(
tmp_path: Path,
tmp_clean_png: Path,
surface: str,
) -> None:
manifest_path = oracles.prepare_batch(
surface,
"gemini-web",
[tmp_clean_png],
output_dir=tmp_path / "oracle-batch",
repository_root=oracles.REPOSITORY_ROOT,
@@ -407,3 +457,18 @@ def test_run_web_batch_refuses_google_real_browser_workflow(
with pytest.raises(ValueError, match="real Chrome"):
web.run_web_batch(manifest_path, acknowledge_uploads=True)
def test_run_web_batch_requires_a_slot_for_synthid(
tmp_path: Path,
tmp_clean_png: Path,
) -> None:
manifest_path = oracles.prepare_batch(
"synthid-web",
[tmp_clean_png],
output_dir=tmp_path / "oracle-batch",
repository_root=oracles.REPOSITORY_ROOT,
)
with pytest.raises(ValueError, match="explicit execution slot"):
web.run_web_batch(manifest_path, acknowledge_uploads=True)
+71 -10
View File
@@ -36,7 +36,8 @@ def test_catalog_covers_every_project_vendor_oracle_surface() -> None:
"do not infer from visible content, logos, text, or metadata."
)
synthid = oracles.SURFACES["synthid-web"]
assert synthid.automation == "real_browser"
assert synthid.automation == "playwright"
assert synthid.account_slots is False
assert synthid.provider_key == "google"
assert synthid.url == "https://synthid.com/"
assert synthid.media_types == ("image", "video", "audio")
@@ -84,9 +85,9 @@ def test_plan_json_keeps_every_google_web_surface_primary() -> None:
assert result.exit_code == 0, result.output
payload = json.loads(result.output)
assert [(row["key"], row["preference"]) for row in payload] == [
("gemini-web", "primary"),
("synthid-web", "primary"),
assert [(row["key"], row["preference"], row["url"]) for row in payload] == [
("gemini-web", "primary", "https://gemini.google.com/"),
("synthid-web", "primary", "https://synthid.com/"),
]
@@ -101,6 +102,16 @@ def test_plan_json_marks_web_as_an_explicit_fallback_when_api_exists() -> None:
]
def test_plan_text_tells_the_caller_to_run_the_next_surface() -> None:
result = CliRunner().invoke(oracles.cli, ["plan", "openai"])
assert result.exit_code == 0, result.output
assert "primary\topenai-api\tapi\tContent Provenance API" in result.output
assert "fallback\topenai-web\tplaywright\tContent Provenance web verifier" in result.output
assert "run the next surface as its own command" in result.output
assert "separate operator decision" not in result.output
def test_slots_support_multiple_accounts_and_networks(tmp_path: Path) -> None:
config = tmp_path / "slots.json"
config.write_text(
@@ -140,6 +151,14 @@ def test_slots_support_multiple_accounts_and_networks(tmp_path: Path) -> None:
"network_label": "home",
"api_key_env": "OPENAI_API_KEY_1",
},
{
"name": "synthid-proxy",
"surface": "synthid-web",
"account_label": None,
"browser_profile": None,
"network_label": "thordata",
"proxy_url_env": "THORDATA_PROXY",
},
],
}
),
@@ -153,7 +172,12 @@ def test_slots_support_multiple_accounts_and_networks(tmp_path: Path) -> None:
"gemini-research",
"openai-api-1",
"openai-west",
"synthid-proxy",
}
assert slots["synthid-proxy"]["browser_profile"] is None
assert slots["synthid-proxy"]["account_label"] is None
assert slots["synthid-proxy"]["proxy_url_env"] == "THORDATA_PROXY"
assert slots["synthid-proxy"]["google_account_index"] is None
assert slots["gemini-research"]["account_label"] == "research-b"
assert slots["gemini-research"]["google_account_index"] == 2
assert slots["openai-west"]["network_label"] == "west-egress"
@@ -192,6 +216,33 @@ def test_named_slot_uses_repository_local_config_by_default(
assert slot["api_key_env"] == "OPENAI_API_KEY_1"
def test_proxy_is_only_for_an_unauthenticated_oracle(tmp_path: Path) -> None:
for surface in oracles.SURFACES.values():
slot: dict[str, str | int | None] = {
"name": "probe",
"surface": surface.key,
"account_label": "account" if surface.account_slots else None,
"browser_profile": "Profile 1" if surface.automation == "real_browser" else None,
"network_label": "route",
"proxy_url_env": "THORDATA_ROUTE_US",
}
if surface.key == "gemini-web":
slot["google_account_index"] = 0
if surface.key == "openai-api":
slot["api_key_env"] = "OPENAI_API_KEY_1"
config = tmp_path / f"{surface.key}.json"
config.write_text(json.dumps({"format_version": 1, "slots": [slot]}), encoding="utf-8")
if surface.account_slots:
with pytest.raises(ValueError, match="does not use proxy_url_env"):
oracles.load_slots(config)
continue
loaded = oracles.load_slots(config)
assert loaded["probe"]["proxy_url_env"] == "THORDATA_ROUTE_US"
assert loaded["probe"]["account_label"] is None
def test_slots_override_requires_a_named_slot(tmp_path: Path) -> None:
with pytest.raises(ValueError, match="--slots requires --slot"):
oracles.resolve_slot("openai-api", slots_path=tmp_path / "slots.json", slot_name=None)
@@ -278,25 +329,35 @@ def test_slots_override_requires_a_named_slot(tmp_path: Path) -> None:
),
(
{
"name": "synthid-no-profile",
"name": "synthid-with-profile",
"surface": "synthid-web",
"account_label": "personal-a",
"browser_profile": None,
"account_label": None,
"browser_profile": "Profile 2",
"network_label": "home",
},
"requires a browser_profile",
"only a real-browser surface uses a browser_profile",
),
(
{
"name": "synthid-with-account-index",
"surface": "synthid-web",
"account_label": "personal-a",
"browser_profile": "Profile 2",
"account_label": None,
"browser_profile": None,
"google_account_index": 0,
"network_label": "home",
},
"only gemini-web uses google_account_index",
),
(
{
"name": "synthid-with-account",
"surface": "synthid-web",
"account_label": "personal-a",
"browser_profile": None,
"network_label": "home",
},
"does not use account slots",
),
],
)
def test_slots_require_the_identity_needed_by_the_surface(
+1 -1
View File
@@ -117,7 +117,7 @@ class TestIsobmffStripFailSafe:
cleaned, stripped = isobmff.strip_c2pa_boxes(data)
assert stripped == 1
assert len(cleaned) == len(data) - len(c2pa_box)
assert len(cleaned) == len(data)
assert isobmff.C2PA_UUID not in cleaned
def test_malformed_box_does_not_truncate_tail(self):
Generated
+1 -1
View File
@@ -3479,7 +3479,7 @@ wheels = [
[[package]]
name = "remove-ai-watermarks"
version = "0.46.0"
version = "0.47.0"
source = { editable = "." }
dependencies = [
{ name = "c2pa-python" },