mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-20 16:07:17 +02:00
Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
540d5648cc | ||
|
|
a752bcda33 | ||
|
|
04364efde2 | ||
|
|
303aba6b8a | ||
|
|
f6b7c366e8 | ||
|
|
30423aca5e | ||
|
|
de6fe3d549 | ||
|
|
7e3563b879 | ||
|
|
76462f9916 | ||
|
|
3818f40096 | ||
|
|
577d1bd953 | ||
|
|
7376d278fd |
No files matched your search
@@ -29,9 +29,15 @@ The unified maintainer entry point is `scripts/provider_oracles.py`; its workflo
|
||||
slot schema live in `docs/provider-oracles.md`. Multi-account and multi-network work
|
||||
selects each named slot explicitly from the gitignored repository-local
|
||||
`.oracle-slots.json` by default and snapshots it into a separate immutable batch
|
||||
manifest. A provider check may use the finite set of configured account slots and surfaces
|
||||
after a refusal. Do not add new logins, IP rotation, network-route failover,
|
||||
unbounded retries, or cross-session result merging. Never commit the local
|
||||
manifest. A provider check walks that provider's existing order: the official API
|
||||
surface, then its Web oracle, then the remaining configured account slots.
|
||||
After a refusal, quota response, transport error, or indeterminate verdict,
|
||||
record the response and continue with the next item as its own command. Do not
|
||||
ask for confirmation before the switch. Anyone doing the task may run the
|
||||
configured checks. Do not add new logins, IP rotation, or unbounded retries.
|
||||
Do not walk a prepared route list or rotate addresses inside one call; the next
|
||||
run is a new command and picks a new session. Do not merge results from
|
||||
different sessions into one response. Never commit the local
|
||||
slot file.
|
||||
Every manual result stays bound to the exact sanitized upload hash and preserves the
|
||||
provider response verbatim.
|
||||
@@ -39,38 +45,49 @@ provider response verbatim.
|
||||
Provider surface selection is API-first. OpenAI and Microsoft use their official API
|
||||
adapters before their Web surfaces when the requested media and local configuration are
|
||||
supported. Google and Meta currently use Web because this tooling has no usable API route
|
||||
for them. A request to use a provider covers every configured surface, including Web,
|
||||
without a second confirmation. An API refusal, quota response, transport error, or
|
||||
indeterminate verdict stays on that surface and never silently submits the file to Web.
|
||||
for them. Google's Web surfaces are `gemini-web` and `synthid-web`
|
||||
(`https://synthid.com/`). After the current surface is exhausted, record the
|
||||
response and continue with the next surface as its own command. One call does
|
||||
not upload the same file to the next surface, and the verdicts stay separate.
|
||||
|
||||
Google Web oracle automation uses only the user's existing authenticated real Chrome
|
||||
session. Gemini uses an explicit `/u/N/` account index. The SynthID Detector at
|
||||
`https://synthid.com/` (`synthid-web`) uses the same Chrome profile and does not take a
|
||||
`/u/N/` index; its terms forbid driving the page as an automated API. OpenAI, Microsoft, and Meta Web automation
|
||||
uses isolated Playwright contexts; it must never attach to or copy state from the user's
|
||||
real browser. Proxy credentials are read only through a slot's named environment variable
|
||||
A signed-in account is required only for `gemini-web`. That surface uses the user's
|
||||
existing authenticated real Chrome session and an explicit `/u/N/` account index.
|
||||
Every other Web surface, including `synthid-web`, uses an isolated Playwright
|
||||
context and may name a proxy. Those contexts must never attach to or copy state
|
||||
from the user's real browser. `synthid-web` has no `/u/N/` index. Its terms forbid
|
||||
bulk verification and quota evasion, so one run submits the prepared rows once
|
||||
and does not start a sign-in. If the page says `Please sign in before detection.`
|
||||
and returns no watermark label, keep that row and continue the same prepared
|
||||
upload in the user's existing real Chrome. That Chrome session stays on the
|
||||
direct route: no proxy, no new login, and no cookie copy into Playwright. If
|
||||
that Chrome also asks for sign-in, leave its row unsubmitted. API calls
|
||||
stay on the direct SDK transport and do not take `proxy_url_env`. Proxy credentials are read only through a slot's named environment variable
|
||||
and never enter a manifest or log. Dotenv interpolation may derive routed proxy URLs from
|
||||
one secret key without mutating the process environment. A selected proxy context may
|
||||
accept that proxy's TLS interception certificate; direct contexts keep normal TLS checks.
|
||||
ThorData rotating endpoints are pinned to one manifest-derived sticky session for the
|
||||
life of a Web batch so one browser context does not change exit identity between page
|
||||
requests. This is route continuity, not account or IP failover; a refusal still ends on
|
||||
the selected slot and never selects another route automatically.
|
||||
A ThorData exit is not prepared ahead of time. Each unauthenticated Web run picks
|
||||
a new session and a country in Europe or the United States, and keeps that one exit
|
||||
for the browser context so page requests do not change identity. A country or
|
||||
`sessid` already written into the proxy URL is discarded. The next run picks again.
|
||||
Upload readiness is surface-specific: OpenAI reaches it at page load but keeps background
|
||||
requests alive, while Microsoft and Meta require network idle. Preserve exact provider
|
||||
errors as indeterminate evidence rather than retrying or interpreting them as clean.
|
||||
requests alive, while Microsoft, Meta, and SynthID Detector require network idle. Preserve exact provider
|
||||
errors as indeterminate evidence, then continue with the next configured route or surface.
|
||||
|
||||
OpenAI API slots likewise carry only `api_key_env`. Store multiple key values in the
|
||||
gitignored `.env`, select one slot explicitly, and never rotate to another token after a
|
||||
refusal or rate limit. The process environment may override the matching `.env` value.
|
||||
Their SDK transport sets `trust_env=False`; environment proxy variables must not reroute
|
||||
an API check.
|
||||
gitignored `.env` and select one slot explicitly. The process environment may override
|
||||
the matching `.env` value. Their SDK transport sets `trust_env=False`, so ambient proxy
|
||||
variables do not reroute an API check, and the slot does not take `proxy_url_env`.
|
||||
After a refusal or rate limit, continue with the next configured slot in a new call,
|
||||
then with the Web surface.
|
||||
|
||||
Microsoft API slots use the Azure Content Safety and private Blob Storage resource
|
||||
identities from the gitignored `.env`. The adapter reads resource keys only into memory
|
||||
through authenticated Azure CLI, downloads and hashes the private blob before requiring
|
||||
equality with the named local source, keeps `trust_env=False`, never retries the submit,
|
||||
and reports Watermark independently from C2PA.
|
||||
equality with the named local source, keeps `trust_env=False`, and makes one submit
|
||||
inside that call. Ambient proxy settings do not apply, and the slot does not take
|
||||
`proxy_url_env`. The next slot or the Web surface is a new call and runs after a
|
||||
refusal, quota response, transport error, or indeterminate verdict. Report
|
||||
Watermark independently from C2PA.
|
||||
|
||||
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
|
||||
|
||||
|
||||
@@ -1,54 +1,65 @@
|
||||
---
|
||||
name: provider-oracles
|
||||
description: Run hash-bound Google, OpenAI, Microsoft, or Meta watermark-oracle checks for this repository. Use for provider verification, multi-account Gemini checks, isolated Playwright checks, or recording oracle evidence.
|
||||
description: Run hash-bound Google, OpenAI, Microsoft, or Meta watermark-oracle checks for this repository. Use for provider verification, multi-account Gemini checks, SynthID Detector checks at synthid.com, isolated Playwright checks, or recording oracle evidence. Walk the existing plan automatically when a task needs a provider verdict.
|
||||
metadata:
|
||||
internal: true
|
||||
---
|
||||
|
||||
# Provider oracles
|
||||
|
||||
Read `docs/provider-oracles.md` before acting. Use `scripts/provider_oracles.py`
|
||||
for surface discovery, slot validation, upload preparation, result recording,
|
||||
and integrity verification.
|
||||
Read `docs/provider-oracles.md` for surface keys, slot schema, preparation, and
|
||||
recording commands. Use `scripts/provider_oracles.py` for surface discovery, slot
|
||||
validation, upload preparation, result recording, and integrity verification.
|
||||
|
||||
Named slots live in the gitignored repository-local
|
||||
`.oracle-slots.json`. Never commit that file. Pass `--slot NAME` to use it;
|
||||
pass `--slots PATH` only to override the local path.
|
||||
|
||||
Never substitute one provider's negative verdict for another provider's signal.
|
||||
Do not optimize candidates adaptively against an external oracle.
|
||||
Do not optimize candidates adaptively against an external oracle. Keep each
|
||||
submitted artifact frozen inside its own batch and bind the result to that
|
||||
artifact's hash. Do not erase failures.
|
||||
|
||||
Anyone doing the task may run the configured checks. Do not ask who is allowed
|
||||
to use the oracle, and do not ask for confirmation before moving to the next
|
||||
surface, slot, or route.
|
||||
|
||||
## Order
|
||||
|
||||
Walk `plan PROVIDER`. That command only prints the order. The order itself stays:
|
||||
|
||||
1. The official API surface, when this tooling has one for that provider, media
|
||||
type, and configured slot. OpenAI is `openai-api`. Microsoft is
|
||||
`microsoft-api`.
|
||||
2. That provider's Web oracle. OpenAI is `openai-web` and Microsoft is
|
||||
`microsoft-web`, both through isolated Playwright. Meta is `meta-web`
|
||||
through isolated Playwright. Google is `gemini-web` on the user's existing
|
||||
real Chrome, then `synthid-web` at `https://synthid.com/` through isolated
|
||||
Playwright. Meta and Google have no API surface, so their first Web oracle is
|
||||
the first row.
|
||||
3. The remaining configured account slots of those surfaces. A signed-in
|
||||
account is required only for Gemini, and that slot stays on the direct
|
||||
route. Every other Web surface may use isolated Playwright and a proxy.
|
||||
That proxy exit is chosen when the run starts.
|
||||
|
||||
When the current surface, slot, or route is exhausted by a refusal, quota
|
||||
response, transport error, or indeterminate verdict, record the exact response
|
||||
and run the next item as its own command. One adapter call still makes one
|
||||
request.
|
||||
|
||||
## Account continuation
|
||||
|
||||
A request to verify a bounded set of files with a provider authorizes using that
|
||||
provider's configured account slots unless the user limits the account scope.
|
||||
Before starting, identify the finite set of available authenticated slots and
|
||||
prepare one immutable batch per slot. After a quota refusal or service failure,
|
||||
record the exact response, leave that slot's remaining rows unsubmitted, and
|
||||
continue automatically through the remaining authorized slots. Do not stop to
|
||||
request permission for each account or file already covered by the task.
|
||||
Before starting, identify the finite set of configured slots and prepare one
|
||||
immutable batch per slot. After a quota refusal or service failure, record the
|
||||
exact response, leave that slot's remaining rows unsubmitted, and continue
|
||||
through the remaining slots in the order above.
|
||||
|
||||
Submit each frozen artifact at most once per slot in that pass. Preserve
|
||||
inconclusive results and disagreements between accounts. Stop when the bounded
|
||||
plan is complete or no authorized usable slot remains; report remaining rows.
|
||||
New logins, another provider, or a new network route require authorization.
|
||||
The oracles and slots already configured for the requested provider may be used
|
||||
again, including their Web surfaces, without a second confirmation. Account
|
||||
continuation does not authorize changing candidate pixels or erasing failures.
|
||||
|
||||
## Default surface priority
|
||||
|
||||
Prefer an official API adapter whenever that provider, media type, and configured
|
||||
slot support it. For OpenAI and Microsoft, the default plans are API then Web.
|
||||
Google and Meta currently have only usable Web routes in this tooling. Google's
|
||||
routes are `gemini-web` and `synthid-web`. Inspect the order with `plan PROVIDER`.
|
||||
The same request may use every configured surface.
|
||||
|
||||
The priority is a selection policy, not automatic failover. An API refusal,
|
||||
quota response, transport error, or indeterminate verdict must be recorded and
|
||||
reported. The same request already covers every configured surface of that
|
||||
provider, including Web. Run Web as its own acknowledged check and keep its
|
||||
verdict separate. Do not replace the API result with the Web result.
|
||||
inconclusive results and disagreements between accounts. Stop when the plan is
|
||||
complete or no configured usable slot or route remains, and report remaining
|
||||
rows. Do not start a new login. Do not change candidate pixels. An API result
|
||||
is not replaced by the Web result. Run Web as its own acknowledged check and
|
||||
keep its verdict separate.
|
||||
|
||||
## Route by surface
|
||||
|
||||
@@ -69,10 +80,8 @@ Playwright CLI runner, copied cookies, or a new login.
|
||||
AI metadata. Inspect the prepared upload again and do not submit it if a visible
|
||||
provider mark remains. A model answer based on a visible logo is not a SynthID
|
||||
result.
|
||||
4. Confirm that the prepared files and provider are covered by the user's
|
||||
request or existing upload approval. Ask only if that scope is missing or
|
||||
changed; do not request confirmation again for each file or account in the
|
||||
authorized plan. Do not upload the source path directly.
|
||||
4. Upload the prepared files for the task. Pass the acknowledgement flag where
|
||||
the command requires one. Do not upload the source path directly.
|
||||
5. Attach one prepared upload and ask: `Does this image/video/audio contain a
|
||||
SynthID watermark? Use the SynthID verification tool only; do not infer from
|
||||
visible content, logos, text, or metadata.` Do not substitute a general
|
||||
@@ -84,34 +93,43 @@ Playwright CLI runner, copied cookies, or a new login.
|
||||
7. Record the verbatim response and verify the completed batch.
|
||||
|
||||
If a planned Chrome profile or `/u/N/` account is not already authenticated,
|
||||
record that it is unavailable and continue with another authorized existing
|
||||
session. Ask the user only when no usable authorized session remains. Do not
|
||||
start a new login.
|
||||
record that it is unavailable and continue with another configured existing
|
||||
session. When no usable Gemini session remains, record that and continue with
|
||||
`synthid-web`. Do not start a new login.
|
||||
|
||||
### SynthID Detector
|
||||
|
||||
Use the user's real authenticated Chrome. Do not use isolated Playwright: the
|
||||
portal's additional terms forbid using the page as an automated API, bulk
|
||||
verification, and quota evasion.
|
||||
Use the isolated Playwright runner at `https://synthid.com/`. The page does not
|
||||
require an account login, so the slot may name a proxy. Do not start a sign-in.
|
||||
The portal's terms forbid bulk verification and quota evasion, so one invocation
|
||||
submits the prepared rows once and does not retry.
|
||||
|
||||
1. Validate a `synthid-web` slot. It names `browser_profile` and `account_label`.
|
||||
It does not use `google_account_index` or a proxy variable.
|
||||
1. Validate a `synthid-web` slot. It does not name `account_label`,
|
||||
`browser_profile`, or `google_account_index`. It may name `proxy_url_env`.
|
||||
2. Prepare an immutable batch outside the repository. Inspect images the same
|
||||
way as for Gemini, and do not submit a file that still shows a provider logo.
|
||||
3. Open `https://synthid.com/` in the named Chrome profile. Accept the terms
|
||||
dialog when it is shown. Do not start a new login. If the page requires
|
||||
sign-in and that profile has no session, leave the row unsubmitted.
|
||||
4. Attach one prepared upload through the page's own file control.
|
||||
5. Read the settled label. `Watermark detected` is `detected`. `Watermark not
|
||||
detected` is `not_detected`. Quota, sign-in, and unclear wording stay
|
||||
`refused` or `indeterminate`.
|
||||
3. Run `run-web` with the upload acknowledgement. The runner attaches one
|
||||
prepared upload through the page's file input.
|
||||
4. Read the settled label. `Watermark detected` is `detected`. `Watermark not
|
||||
detected` and `SynthID was not detected` are `not_detected`. Quota and
|
||||
unclear wording stay `refused` or `indeterminate`. Do not click a sign-in
|
||||
button.
|
||||
5. If the anonymous page says `Please sign in before detection.` and returns no
|
||||
watermark label, keep that row. Open the same prepared upload in the user's
|
||||
existing real Chrome, where a session is already signed in. Read
|
||||
`~/Documents/GitHub/claude-config/docs/browser-automation.md` and use that
|
||||
authenticated Chrome. Do not start a new login, do not copy cookies into
|
||||
Playwright, and do not send that Chrome session through a proxy. If that
|
||||
Chrome also asks for sign-in, leave its row unsubmitted.
|
||||
6. Record the verbatim response. The result covers partner SynthID from Google
|
||||
DeepMind, Nvidia, OpenAI, and Kakao, and it does not name which partner. Do
|
||||
not record it as a Gemini-only or OpenAI-only verdict.
|
||||
|
||||
### OpenAI, Microsoft, and Meta Web
|
||||
### OpenAI, Microsoft, Meta, and SynthID Detector Web
|
||||
|
||||
Use the isolated Playwright runner, never the user's real Chrome:
|
||||
Use the isolated Playwright runner. Do not attach that runner to the user's
|
||||
real Chrome. The sign-in fallback in the SynthID Detector section is a separate
|
||||
session in the existing Chrome, not a proxy through this runner:
|
||||
|
||||
```bash
|
||||
uv run python scripts/provider_oracles.py run-web /outside/repo/batch/manifest.json \
|
||||
@@ -122,41 +140,43 @@ The runner is headed by default because OpenAI's Cloudflare gate blocks the
|
||||
same isolated Chromium in headless mode. Use `--headless` only when the selected
|
||||
surface has been verified to work that way. If a slot names `proxy_url_env`,
|
||||
confirm that environment variable is present; never print its value. The runner
|
||||
uses one context, one route, fresh page navigation per upload, and no retry.
|
||||
CAPTCHA, bot checks, throttling, transport failure, and unknown page wording
|
||||
remain distinct recorded outcomes.
|
||||
uses one context, one route, and fresh page navigation per upload. One
|
||||
invocation does not retry internally. CAPTCHA, bot checks, throttling, transport
|
||||
failure, and unknown page wording remain distinct recorded outcomes. After any
|
||||
of those, or after a refusal, run the next configured route or surface.
|
||||
|
||||
Wait for the surface-specific upload readiness that the runner implements.
|
||||
OpenAI completes at page load but keeps background requests alive, while
|
||||
Microsoft and Meta require network idle. Do not replace those conditions with
|
||||
Microsoft, Meta, and SynthID Detector require network idle. Do not replace those conditions with
|
||||
one shared wait. Treat OpenAI's exact `Something went wrong` response as a
|
||||
settled `indeterminate` result, not as a clean result or a reason to retry.
|
||||
settled `indeterminate` result, not as a clean result. Do not resubmit that
|
||||
same page in the same batch; continue with the next configured route or surface.
|
||||
|
||||
For ThorData residential proxy mode, keep `THORDATA_ROUTE_US`,
|
||||
`THORDATA_ROUTE_DE`, `THORDATA_ROUTE_GB`, `THORDATA_ROUTE_NL`, and
|
||||
`THORDATA_ROUTE_JP` URLs in the gitignored `.env`;
|
||||
OpenAI, Microsoft, and Meta Web slots may each name one of them through
|
||||
`proxy_url_env`. Encode an explicit two-letter ISO country code in each route,
|
||||
verify egress before relying on it, and never automatically change route after
|
||||
a refusal. Bound a batch before starting.
|
||||
A signed-in account is required only for Gemini. Every other Web surface,
|
||||
including SynthID Detector, starts in isolated Playwright and may use a proxy.
|
||||
When SynthID Detector asks for a sign-in, continue that one file in the user's
|
||||
existing real Chrome, on the direct route. An
|
||||
API call stays on the direct SDK transport and does not take a proxy.
|
||||
|
||||
ThorData user-and-password endpoints without `sessid` rotate the exit on each
|
||||
request, which breaks a browser page whose resources must share one network
|
||||
identity. The runner therefore derives one sticky `sessid` from the immutable
|
||||
manifest hash and keeps it for that batch. A new prepared batch gets a new
|
||||
session; rerunning or resuming the same batch keeps its session while the proxy
|
||||
retains it. An explicit `sessid` in the configured route is preserved. This is
|
||||
session continuity, not permission to retry a refused batch on another route.
|
||||
A ThorData credential may live in one gitignored environment variable. Do not
|
||||
prepare a country or a session in that URL or in a slot. Each run of an
|
||||
unauthenticated Web oracle discards a country or `sessid` already present and
|
||||
picks a new session in a country in Europe or the United States. That one
|
||||
session stays for the browser context, because a ThorData endpoint without
|
||||
`sessid` rotates the exit on each request and breaks a page whose resources
|
||||
must share one network identity. The next run picks again. After a refusal,
|
||||
start that next run; do not walk a list of prepared country routes.
|
||||
|
||||
### OpenAI API
|
||||
|
||||
Use `check-openai` with one explicit upload acknowledgement. Keep named API keys
|
||||
as `OPENAI_API_KEY_1`, `OPENAI_API_KEY_2`, and `OPENAI_API_KEY_3` in the local `.env`, and select
|
||||
only their variable names through an `openai-api` slot's `api_key_env`. Never
|
||||
print a key or copy one into a slot or manifest. Continue through authorized
|
||||
named API slots according to the bounded account plan above; keep a separate
|
||||
batch for each. API slots use a direct SDK transport that ignores environment proxy
|
||||
settings. The current adapter is image-only.
|
||||
print a key or copy one into a slot or manifest. Continue through the configured
|
||||
named API slots, with a separate batch for each, and then continue to
|
||||
`openai-web`. API slots use a direct SDK transport that ignores environment
|
||||
proxy settings. The current adapter is image-only. One adapter call does not
|
||||
switch tokens; the next slot is a new call.
|
||||
|
||||
### Microsoft API
|
||||
|
||||
@@ -167,8 +187,10 @@ the gitignored `.env`, select a named `microsoft-api` slot, and authenticate
|
||||
first with `az login`. Give the Content Safety system identity `Storage Blob
|
||||
Data Reader` on the input storage account. The adapter reads resource keys only
|
||||
into memory through Azure CLI, never logs them, ignores environment proxy
|
||||
settings, makes one submit request with no retry, and polls only the returned
|
||||
operation. Keep Microsoft's Watermark and C2PA markers as separate verdicts.
|
||||
settings, makes one submit request with no retry inside that call, and polls
|
||||
only the returned operation. After that call is refused, hits quota, fails, or
|
||||
returns indeterminate, continue with another `microsoft-api` slot or with
|
||||
`microsoft-web`. Keep Microsoft's Watermark and C2PA markers as separate verdicts.
|
||||
|
||||
## Evidence rules
|
||||
|
||||
|
||||
@@ -84,7 +84,13 @@ finding.
|
||||
9. Run `bash maintain.sh`. Report evidence limits, incomplete rows, crashes,
|
||||
unsupported formats, and oracle-unavailable cohorts explicitly.
|
||||
|
||||
External oracle uploads are separate mutations. Use the `provider-oracles`
|
||||
skill only for a bounded, already-selected candidate, and obtain the required
|
||||
action-time upload approval. Do not adapt candidates against an oracle, retry
|
||||
automatically, or substitute one provider's verdict for another's.
|
||||
External oracle uploads use the `provider-oracles` skill. When a cohort needs a
|
||||
provider verdict, walk that skill's plan automatically: API first, then the Web
|
||||
oracle, including `synthid-web` at synthid.com for Google, then the remaining
|
||||
configured account slots. A signed-in account is required only for Gemini.
|
||||
Every other Web surface, including `synthid-web`, may use isolated Playwright
|
||||
and a proxy, and each proxied run picks a new session in Europe or the United
|
||||
States. If SynthID Detector asks for a sign-in, continue that file in the user's
|
||||
existing real Chrome on the direct route, and do not start a new login. Do not ask for
|
||||
confirmation before switching, and do not adapt candidates against an oracle.
|
||||
Do not substitute one provider's verdict for another's.
|
||||
@@ -32,6 +32,7 @@ removal.
|
||||
[](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml)
|
||||
[](https://github.com/sponsors/wiltodelta)
|
||||
[](https://skills.sh/wiltodelta/remove-ai-watermarks)
|
||||
[](https://glama.ai/mcp/servers/wiltodelta/remove-ai-watermarks)
|
||||
|
||||
> This is a research project on AI provenance signals, for lawful use on content
|
||||
> you own. It is provided as is, without warranty or liability, and each user is
|
||||
|
||||
+32
-23
@@ -52,35 +52,44 @@ Do these in order after the skill is on the default branch.
|
||||
people run `npx skills add`. The listing is live and has recorded real
|
||||
installs; the README install line remains the seed, not a registration.
|
||||
4. **Awesome-list PRs.** Hand-curated GitHub lists are how people browse skills
|
||||
today. One accurate PR per list, with the scope sentence intact.
|
||||
- `ComposioHQ/awesome-claude-skills`: submitted 2026-09-05 as PR 1841
|
||||
(Creative & Media); no maturity bar in its contribution rules. Rechecked
|
||||
2026-09-11: still open, CI green, no review comments, waiting on the
|
||||
list maintainers.
|
||||
- `heilcheng/awesome-agent-skills`: submitted 2026-09-09 as PR 483 under
|
||||
today. One accurate PR per list, with the scope sentence intact. This list
|
||||
keeps each list's entry rules and the submission; read live review state
|
||||
from GitHub (`gh search prs --author @me --state open remove-ai-watermarks`
|
||||
lists every open one), not from here.
|
||||
- `ComposioHQ/awesome-claude-skills`: submitted 2026-09-05 as
|
||||
[PR 1841](https://github.com/ComposioHQ/awesome-claude-skills/pull/1841)
|
||||
(Creative & Media); no maturity bar in its contribution rules.
|
||||
- `heilcheng/awesome-agent-skills`: submitted 2026-09-09 as
|
||||
[PR 483](https://github.com/heilcheng/awesome-agent-skills/pull/483) under
|
||||
Community Skills > Media. The skill's three concrete input/output examples
|
||||
satisfy the list's working-example requirement. Rechecked 2026-09-11:
|
||||
still open; the only check failure is their Vercel deploy authorization,
|
||||
satisfy the list's working-example requirement. The maintainer put it
|
||||
`on hold` on 2026-10-15: "Tool strips AI provenance (SynthID/C2PA); policy
|
||||
call for maintainer". Its Vercel check fails on deploy authorization,
|
||||
which a contributor cannot grant.
|
||||
- `VoltAgent/awesome-agent-skills`: DEFERRED. Its CONTRIBUTING rejects
|
||||
brand-new skills ("give your skill time to mature and gain users before
|
||||
submitting"). The skill now has real installs and shipped releases, but
|
||||
adoption remains too small to make the list's maturity claim comfortably.
|
||||
- `VoltAgent/awesome-agent-skills`: submitted 2026-10-17 as
|
||||
[PR 1225](https://github.com/VoltAgent/awesome-agent-skills/pull/1225), a
|
||||
one-line entry. Its CONTRIBUTING requires "real community usage" ("Give
|
||||
your skill time to mature and gain users before submitting").
|
||||
- `VoltAgent/awesome-openclaw-skills`: DEFERRED on adoption only. Entries
|
||||
are indexed via clawskills.sh, and the ClawHub prerequisite in step 6 is
|
||||
now met. The list still requires proven real-world usage. Re-attempt once
|
||||
the ClawHub listing shows organic installs.
|
||||
- `hashgraph-online/awesome-codex-plugins`: READY, not submitted. It lists
|
||||
only Codex plugins, so it needs `.codex-plugin/plugin.json` and the icon in
|
||||
`assets/icon.svg`. Its contribution CI runs `scripts/validate-plugin-pr.py`
|
||||
against the repository's `HEAD.zip`, then the HOL plugin scanner with a
|
||||
minimum score of 80 and no high finding; that scan job is
|
||||
`continue-on-error`, so it reports rather than blocks. The scanner's `\beval\s*\(`
|
||||
rule also matches PyTorch's `.eval()`, which is why the code calls
|
||||
`.train(False)`. On 2026-10-17 `plugin-scanner` 3.49.0 scored the tree 86
|
||||
with no high finding. The remaining loss is a tracked video over the
|
||||
scanner's 16 MiB file budget. Recheck with
|
||||
`uvx --from plugin-scanner==3.49.0 plugin-scanner scan . --min-score 80 --fail-on-severity high`.
|
||||
- `hashgraph-online/awesome-codex-plugins`: merged 2026-10-19 as
|
||||
[PR 541](https://github.com/hashgraph-online/awesome-codex-plugins/pull/541).
|
||||
It lists only Codex plugins, so the entry depends on
|
||||
`.codex-plugin/plugin.json` and the icon in `assets/icon.svg`. Its
|
||||
contribution CI runs `scripts/validate-plugin-pr.py` against the
|
||||
repository's `HEAD.zip`, then the HOL plugin scanner with a minimum score
|
||||
of 80 and no high finding, as `continue-on-error`. The scanner's
|
||||
`\beval\s*\(` rule also matches PyTorch's `.eval()`, which is why the code
|
||||
calls `.train(False)`. Before changing the plugin, rescan with
|
||||
`uvx --from plugin-scanner==3.49.0 plugin-scanner scan . --min-score 80 --fail-on-severity high`
|
||||
(86 with no high finding on 2026-10-17; the loss is a tracked video over
|
||||
the scanner's 16 MiB file budget).
|
||||
- `punkpeye/awesome-mcp-servers`: submitted 2026-10-18 as
|
||||
[PR 16924](https://github.com/punkpeye/awesome-mcp-servers/pull/16924) for
|
||||
the MCP server. The list requires a Glama listing and the Glama score
|
||||
badge; its bot checks both and labels the PR `has-glama`.
|
||||
5. **Claude community marketplace.** Form at
|
||||
[platform.claude.com/plugins/submit](https://platform.claude.com/plugins/submit)
|
||||
(individuals) or the claude.ai directory form (Team/Enterprise). Validate
|
||||
|
||||
@@ -68,10 +68,14 @@ Important contracts:
|
||||
- `visible` and `erase` strip and verify AI metadata on a staged sibling and exit
|
||||
`1` with nothing published when a marker survives; they used to swallow a failed
|
||||
strip unless `--verbose` was set. The library `remove_visible` does the same and
|
||||
raises `MetadataStripIncomplete`, as `remove_all` already did, and the diffusion
|
||||
writer behind `invisible` raises when its own strip leaves a marker.
|
||||
raises `MetadataStripIncomplete`, as `remove_all` already did. The diffusion
|
||||
writer behind `invisible` stages the same way and raises with nothing published
|
||||
(fixed 2026-10-19; it used to leave the marked file at the output path).
|
||||
`_internal.utils.staged_sibling` is the shared stage: it never pre-creates the
|
||||
name, so the published file keeps the default permissions.
|
||||
name, so the published file keeps the default permissions. These writers strip
|
||||
through `metadata.strip_written_output`, which skips the readability probe:
|
||||
the file is the package's own encode, and the probe's full decode doubled the
|
||||
cost of the strip (decided 2026-10-19).
|
||||
- A quiet `metadata --check` and a successful `metadata --remove` (same for
|
||||
`video metadata`) end by repeating the `identify` limit: the pixel channel is
|
||||
untouched and a watermark such as SynthID has no local decoder once its
|
||||
@@ -266,6 +270,13 @@ Publication is atomic, and a malformed top-level walk is copied unchanged,
|
||||
although `strip_and_verify` refuses such a source before the stripper runs. A
|
||||
generic `AIGC` key whose value has no TC260 field is ignored.
|
||||
|
||||
The in-memory image path (AVIF, HEIF, JPEG-XL and unknown suffixes, through
|
||||
`strip_c2pa_boxes`) blanks the same top-level boxes to same-size `free` boxes.
|
||||
It used to drop them, which moved `mdat` under an unchanged `iloc`: a phone HEIC
|
||||
whose C2PA `uuid` box preceded `mdat` came out undecodable (2026-10-13).
|
||||
`tests/test_metadata.py::TestIsobmffStripKeepsOffsets` decodes a cleaned HEIC to
|
||||
guard it.
|
||||
|
||||
App-export JSON in a keyed or keyless metadata item is blanked the same way.
|
||||
Dreamina's MP4 export writes it twice, under `LvMetaInfo` and
|
||||
`com.apple.quicktime.artwork`. The file-level scan read both from the raw head
|
||||
|
||||
+89
-102
@@ -1,9 +1,9 @@
|
||||
# Provider oracle tooling
|
||||
|
||||
`scripts/provider_oracles.py` is the development-only entry point for the four
|
||||
remote watermark-oracle families used by this project. It drives Google through
|
||||
the user's real authenticated Chrome and drives the other public Web surfaces
|
||||
through isolated Playwright. It keeps provider scope, execution identity,
|
||||
remote watermark-oracle families used by this project. It drives Gemini through
|
||||
the user's real authenticated Chrome and drives every other public Web surface,
|
||||
including SynthID Detector, through isolated Playwright. It keeps provider scope, execution identity,
|
||||
immutable media, and verbatim results in one format without exposing any oracle
|
||||
through the installed CLI or public Python API.
|
||||
|
||||
@@ -13,8 +13,8 @@ through the installed CLI or public Python API.
|
||||
| --- | --- | --- | --- |
|
||||
| `openai-api` | OpenAI SynthID | Official Content Provenance API | Images |
|
||||
| `microsoft-api` | Microsoft InvisMark | Azure Content Provenance Detection API | Images |
|
||||
| `gemini-web` | Google SynthID | Signed-in Gemini web account | Images and videos |
|
||||
| `synthid-web` | Partner SynthID | SynthID Detector at synthid.com, real Chrome | None recorded |
|
||||
| `gemini-web` | Google SynthID | User's authenticated Chrome | Images and videos |
|
||||
| `synthid-web` | Partner SynthID | Anonymous Playwright page at synthid.com | None recorded |
|
||||
| `meta-web` | Meta Content Seal | Anonymous Meta identification page | Images |
|
||||
| `microsoft-web` | Microsoft InvisMark | Public Azure AI validation page | Images |
|
||||
| `openai-web` | OpenAI SynthID | Public web verifier | Images |
|
||||
@@ -25,7 +25,7 @@ The default selection order is API first, then Web. OpenAI therefore plans
|
||||
`openai-api` before `openai-web`, and Microsoft plans `microsoft-api` before
|
||||
`microsoft-web`. Google and Meta currently have no usable API adapter in this
|
||||
tooling, so each of their Web surfaces is primary. Google has two: `gemini-web`
|
||||
and `synthid-web`.
|
||||
and `synthid-web` at `https://synthid.com/`.
|
||||
|
||||
Inspect the machine-readable order before preparing a batch:
|
||||
|
||||
@@ -35,12 +35,15 @@ uv run python scripts/provider_oracles.py plan microsoft
|
||||
uv run python scripts/provider_oracles.py plan google
|
||||
```
|
||||
|
||||
A request to use a provider covers every configured surface of that provider,
|
||||
including Web, without a second confirmation. API-first is still the order to
|
||||
try, not a lock on the other surfaces. An API refusal, quota response, transport
|
||||
failure, or indeterminate verdict is recorded as-is. It does not silently
|
||||
upload the same file to Web; run that surface as its own acknowledged check and
|
||||
keep the two verdicts separate.
|
||||
API-first is the default order. Run the API surface when this tooling has a
|
||||
usable route for the provider, media, and local configuration. Google and Meta
|
||||
have no API surface, so each of their Web oracles is primary. For Google that
|
||||
is `gemini-web`, then `synthid-web`. After a refusal, quota response, transport
|
||||
failure, or indeterminate verdict, record that result and continue with the next
|
||||
surface as its own acknowledged command, then the remaining configured account
|
||||
slots of those surfaces. One call does not upload the same file to the next
|
||||
surface. Keep the verdicts separate. Do not ask for confirmation before the
|
||||
switch. Anyone doing the task may run the configured checks.
|
||||
|
||||
The old Google SynthID Detector portal at `labs.google/synthid` does not work.
|
||||
On 2026-09-07 that entry point redirected a signed-in browser to a Google 404;
|
||||
@@ -60,10 +63,14 @@ and Kakao, and it did not list Apple. It defined two settled labels:
|
||||
`.tif`, `.gif`, `.wav`, `.mp3`, `.ogg`, `.flac`, `.aac`, `.m4a`, `.mp4`,
|
||||
`.mov`, and `.webm`. This tooling still prepares only the project's existing
|
||||
JPEG, PNG, WebP, MP4, MP3, and WAV suffixes. The portal's additional terms
|
||||
forbid using the page as an automated API, bulk verification, and quota evasion,
|
||||
so `run-web` does not drive it. Use the real Chrome workflow below. A
|
||||
`Watermark detected` result is partner SynthID, not a Gemini-only or OpenAI-only
|
||||
verdict.
|
||||
forbid bulk verification and quota evasion. `run-web` may submit one prepared
|
||||
upload through isolated Playwright, with a proxy when the slot names one, and
|
||||
it does not start a sign-in. If the page says `Please sign in before detection.`
|
||||
and returns no watermark label, keep that row and open the same prepared upload
|
||||
in the user's existing real Chrome. That session stays on the direct route, with
|
||||
no proxy and no new login. If that Chrome also asks for sign-in, leave its row
|
||||
unsubmitted. A `Watermark detected` result is partner SynthID, not a Gemini-only
|
||||
or OpenAI-only verdict.
|
||||
Google's newer Search verification entry point did not provide an independent
|
||||
oracle allowance in the session tested on 2026-09-13. Lens accepted the
|
||||
metadata-stripped, previously Gemini-positive image
|
||||
@@ -218,43 +225,20 @@ default; `--slots PATH` explicitly overrides it.
|
||||
"network_label": "office"
|
||||
},
|
||||
{
|
||||
"name": "synthid-personal",
|
||||
"name": "synthid-proxy",
|
||||
"surface": "synthid-web",
|
||||
"account_label": "google-account-a",
|
||||
"browser_profile": "Your Chrome",
|
||||
"network_label": "home"
|
||||
"account_label": null,
|
||||
"browser_profile": null,
|
||||
"network_label": "thordata",
|
||||
"proxy_url_env": "THORDATA_PROXY"
|
||||
},
|
||||
{
|
||||
"name": "openai-web-thordata-us",
|
||||
"name": "openai-web-proxy",
|
||||
"surface": "openai-web",
|
||||
"account_label": null,
|
||||
"browser_profile": null,
|
||||
"network_label": "thordata-us-residential",
|
||||
"proxy_url_env": "THORDATA_ROUTE_US"
|
||||
},
|
||||
{
|
||||
"name": "microsoft-web-thordata-de",
|
||||
"surface": "microsoft-web",
|
||||
"account_label": null,
|
||||
"browser_profile": null,
|
||||
"network_label": "thordata-de-residential",
|
||||
"proxy_url_env": "THORDATA_ROUTE_DE"
|
||||
},
|
||||
{
|
||||
"name": "meta-web-thordata-us",
|
||||
"surface": "meta-web",
|
||||
"account_label": null,
|
||||
"browser_profile": null,
|
||||
"network_label": "thordata-us-residential",
|
||||
"proxy_url_env": "THORDATA_ROUTE_US"
|
||||
},
|
||||
{
|
||||
"name": "openai-web-thordata-de",
|
||||
"surface": "openai-web",
|
||||
"account_label": null,
|
||||
"browser_profile": null,
|
||||
"network_label": "thordata-de-residential",
|
||||
"proxy_url_env": "THORDATA_ROUTE_DE"
|
||||
"network_label": "thordata",
|
||||
"proxy_url_env": "THORDATA_PROXY"
|
||||
},
|
||||
{
|
||||
"name": "openai-api-1",
|
||||
@@ -292,30 +276,34 @@ default; `--slots PATH` explicitly overrides it.
|
||||
```
|
||||
|
||||
A routed batch selects one slot explicitly. A Gemini slot must name the Google
|
||||
account, its real Chrome profile, and its nonnegative `/u/N/` account index. A
|
||||
SynthID Detector slot names the account and the same real Chrome profile, and it
|
||||
does not use `google_account_index`. An
|
||||
API slot must name its account or project and cannot name a browser profile. The
|
||||
account, its real Chrome profile, and its nonnegative `/u/N/` account index. An
|
||||
API slot must name its account or project and cannot name a browser profile. A
|
||||
Playwright slot for `openai-web`, `microsoft-web`, `meta-web`, or `synthid-web`
|
||||
does not name an account or a browser profile. The
|
||||
tool records the network route that the operator has already established.
|
||||
Non-Google Web slots may name `proxy_url_env`; Playwright reads the secret proxy
|
||||
URL from that environment variable without writing or logging it. The tool does
|
||||
not merge sessions or automatically change network routes. Account continuation
|
||||
is orchestrated by the skill, not by an individual adapter call. A request to
|
||||
verify a bounded set of files with a provider covers its configured account slots
|
||||
unless the operator restricts the account scope. Identify the finite set of
|
||||
available authenticated slots before starting and prepare one immutable batch
|
||||
per slot. Record quota refusals as `refused` and service failures according to
|
||||
their actual response, leave remaining rows in that slot unsubmitted, and
|
||||
continue automatically with the next authorized slot. Do not request permission
|
||||
again for files or accounts already covered by the task.
|
||||
Those four pages may name `proxy_url_env`; Playwright reads the
|
||||
secret proxy URL from that environment variable without writing or logging it.
|
||||
The country and session are not part of that preparation. Each run replaces
|
||||
them with a new session in a country in Europe or the United States.
|
||||
A signed-in account is required only for `gemini-web`. API calls stay on the
|
||||
direct SDK transport and do not take `proxy_url_env`. Every other Web surface
|
||||
uses isolated Playwright and may use a proxy. The tool does
|
||||
not merge sessions inside one adapter call. Account continuation is orchestrated
|
||||
by the skill, not by an individual adapter call. A request to verify files with
|
||||
a provider covers its configured account slots, surfaces, and network routes.
|
||||
Anyone doing the task may run those checks. Identify the finite set of
|
||||
configured slots before starting and prepare one immutable batch per slot.
|
||||
Record quota refusals as `refused` and service failures according to their
|
||||
actual response, leave remaining rows in that slot unsubmitted, and continue
|
||||
with the next slot, surface, or configured route. Do not ask for confirmation
|
||||
before that switch.
|
||||
|
||||
Submit each frozen artifact at most once per slot in that pass. Keep every
|
||||
Submit each frozen artifact at most once per slot in one pass. Keep every
|
||||
verbatim refusal, inconclusive reply, and disagreement between accounts. Stop
|
||||
when the planned checks finish or no authorized usable slot remains, and report
|
||||
unsubmitted rows. A new login, another provider, or a new network route needs
|
||||
authorization. The oracles and slots already configured for the requested
|
||||
provider may be used again, including their Web surfaces, without a second
|
||||
confirmation. Candidate pixels remain frozen throughout account continuation.
|
||||
when the plan is complete or no configured usable slot, surface, or route
|
||||
remains, and report anything still open. Do not start a new login. Each
|
||||
submitted artifact stays frozen inside its own batch. Do not change candidate
|
||||
pixels during the continuation.
|
||||
|
||||
Each `openai-api` slot must name exactly one `api_key_env`. Put the corresponding
|
||||
values in the gitignored local `.env`, never in the slot JSON:
|
||||
@@ -324,29 +312,27 @@ values in the gitignored local `.env`, never in the slot JSON:
|
||||
OPENAI_API_KEY_1=replace-with-first-key
|
||||
OPENAI_API_KEY_2=replace-with-second-key
|
||||
OPENAI_API_KEY_3=replace-with-third-key
|
||||
THORDATA_ROUTE_US=http://td-customer-USERNAME-country-US:PASSWORD@HOST.pr.thordata.net:9999
|
||||
THORDATA_ROUTE_DE=http://td-customer-USERNAME-country-DE:PASSWORD@HOST.pr.thordata.net:9999
|
||||
THORDATA_ROUTE_GB=http://td-customer-USERNAME-country-GB:PASSWORD@HOST.pr.thordata.net:9999
|
||||
THORDATA_ROUTE_NL=http://td-customer-USERNAME-country-NL:PASSWORD@HOST.pr.thordata.net:9999
|
||||
THORDATA_ROUTE_JP=http://td-customer-USERNAME-country-JP:PASSWORD@HOST.pr.thordata.net:9999
|
||||
THORDATA_PROXY=http://td-customer-USERNAME:PASSWORD@HOST.pr.thordata.net:9999
|
||||
```
|
||||
|
||||
The tool reads only the variable selected by the requested slot. A nonempty
|
||||
process environment value overrides the matching `.env` entry. It never prints
|
||||
the value or falls through to another token after authentication, entitlement,
|
||||
quota, or transport failure. The no-slot compatibility path reads
|
||||
the value. One call reads only its selected slot. After authentication,
|
||||
entitlement, quota, or transport failure, continue with the next configured
|
||||
slot in a new call. The no-slot compatibility path reads
|
||||
`OPENAI_API_KEY`.
|
||||
|
||||
Several Google accounts may share one real Chrome profile: give them distinct
|
||||
slot names and `google_account_index` values while keeping `browser_profile` the
|
||||
same.
|
||||
|
||||
The ThorData examples use its [residential proxy country parameter](https://doc.thordata.com/doc/proxies/residential-proxies/location-settings/country-region).
|
||||
The proxy username encodes one explicit two-letter ISO country code. Route
|
||||
selection remains explicit and never changes after a refusal. OpenAI,
|
||||
Microsoft, and Meta Web slots may reuse any route variable; the surface remains
|
||||
part of each distinct slot. OpenAI API slots never accept `proxy_url_env` and
|
||||
use the current direct network.
|
||||
ThorData encodes the exit in the username with its [country parameter](https://doc.thordata.com/doc/proxies/residential-proxies/location-settings/country-region)
|
||||
and a [session id](https://doc.thordata.com/doc/proxies/residential-proxies/session-control).
|
||||
A stored `country-XX` or `sessid` is not the exit this run will use. The runner
|
||||
discards both and appends one new session and one country from Europe or the
|
||||
United States. That pair stays for the browser context of the run. The next run
|
||||
chooses again, including after a refusal. Gemini and API slots never accept
|
||||
`proxy_url_env`. Anonymous Playwright slots may.
|
||||
|
||||
Validate and inspect the slots:
|
||||
|
||||
@@ -419,16 +405,18 @@ uv sync --extra dev
|
||||
uv run playwright install chromium
|
||||
```
|
||||
|
||||
Google is intentionally excluded from the standalone runner. Both `gemini-web`
|
||||
and `synthid-web` use the user's existing real Chrome profile, and the SynthID
|
||||
Detector terms forbid driving that page as an automated API. Invoke the project
|
||||
`provider-oracles` skill to prepare the batch and record the settled outcome.
|
||||
For Gemini, select the named profile and `/u/N/` account. For SynthID Detector,
|
||||
open `https://synthid.com/` in that profile. The portal has no `/u/N/` index.
|
||||
Accept the terms dialog when it is shown, attach one prepared upload, and read
|
||||
the explicit `Watermark detected` or `Watermark not detected` label. If the page
|
||||
requires sign-in and that profile has no session, record the row unsubmitted
|
||||
and do not start a new login.
|
||||
Gemini is the only surface excluded from the standalone runner, because it is
|
||||
the only one that requires a signed-in account. Invoke the project
|
||||
`provider-oracles` skill to drive that existing Chrome profile at the named
|
||||
`/u/N/` account. `synthid-web` is an anonymous page: `run-web` opens
|
||||
`https://synthid.com/` in isolated Playwright, attaches one prepared upload
|
||||
through the page's file input, and reads the explicit `Watermark detected`,
|
||||
`Watermark not detected`, or `SynthID was not detected` label. The portal has
|
||||
no `/u/N/` index. If the page says `Please sign in before detection.` and
|
||||
returns no watermark label, keep that row and continue the same prepared upload
|
||||
in the user's existing real Chrome. That session is direct: no proxy, no new
|
||||
login, and no cookie copy into the isolated browser. If that Chrome also asks
|
||||
for sign-in, leave its row unsubmitted.
|
||||
|
||||
For Gemini, ask the direct question stored in the batch manifest: `Does this
|
||||
image/video/audio contain a SynthID watermark? Use the SynthID verification
|
||||
@@ -437,8 +425,8 @@ general question about whether Google AI created or edited the media is not
|
||||
equivalent: it may elicit origin or visual-content reasoning without a SynthID
|
||||
verdict. SynthID Detector has no prompt; its settled labels are the verdict.
|
||||
|
||||
OpenAI Web, Microsoft Web, and Meta run entirely through a fresh isolated
|
||||
Playwright browser. A direct-route example:
|
||||
OpenAI Web, Microsoft Web, Meta, and SynthID Detector run through a fresh
|
||||
isolated Playwright browser. A direct-route example:
|
||||
|
||||
```bash
|
||||
uv run python scripts/provider_oracles.py prepare meta-web input.png \
|
||||
@@ -449,8 +437,9 @@ uv run python scripts/provider_oracles.py run-web \
|
||||
/tmp/meta-check-001/manifest.json --acknowledge-uploads
|
||||
```
|
||||
|
||||
For a ThorData-routed OpenAI, Microsoft, or Meta Web slot, keep the generated
|
||||
proxy URLs in the gitignored `.env`, then select one route explicitly:
|
||||
For an OpenAI, Microsoft, Meta, or SynthID Detector slot that names a ThorData
|
||||
credential, keep that URL in the gitignored `.env`. The run chooses the country
|
||||
and the session:
|
||||
|
||||
```bash
|
||||
uv run python scripts/provider_oracles.py run-web \
|
||||
@@ -467,12 +456,10 @@ ThorData [user-and-password endpoints without a `sessid` rotate the exit on each
|
||||
request](https://doc.thordata.com/doc/proxies/residential-proxies/session-control).
|
||||
That is unsuitable inside one browser context: page resources can
|
||||
arrive through different exits and trigger the provider's bot checks. For a
|
||||
ThorData hostname, the runner derives a non-secret sticky session id from the
|
||||
immutable manifest hash and appends a 90-minute session window when the
|
||||
configured username has no explicit session. The same batch therefore keeps
|
||||
one route while the provider retains it, a new prepared batch gets a distinct
|
||||
route, and an operator-supplied `sessid` remains unchanged. This pins a selected
|
||||
route; it does not retry or fail over after a refusal.
|
||||
ThorData hostname, each run therefore discards a prepared country and session,
|
||||
then keeps one new session and one country in Europe or the United States for
|
||||
that browser context. The next invocation, including a retry after a refusal,
|
||||
chooses again.
|
||||
For a selected proxy route it enables certificate handling in that isolated
|
||||
Chromium launch and context; direct launches keep Playwright's normal TLS
|
||||
verification. A 2026-09-07 preflight sent one read-only request through each
|
||||
@@ -489,7 +476,7 @@ quota refusal is `refused`, a transport failure is `unreachable`, and unknown or
|
||||
changed page wording is `indeterminate`.
|
||||
|
||||
Each provider mounts its file input before the upload handler is necessarily
|
||||
ready. Microsoft and Meta reach a usable handler at network idle. OpenAI keeps
|
||||
ready. Microsoft, Meta, and SynthID Detector reach a usable handler at network idle. OpenAI keeps
|
||||
background requests alive and never reaches network idle, so its observable
|
||||
readiness seam is the page `load` state instead. The adapter waits for the
|
||||
surface-specific state before assigning the file. OpenAI's static FAQ contains
|
||||
|
||||
+3
-1
@@ -579,7 +579,9 @@ provides verification in Gemini and, since the public SynthID Detector opened,
|
||||
at [synthid.com](https://synthid.com/). That site is the `synthid-web` oracle
|
||||
surface in [`provider-oracles.md`](provider-oracles.md). On 2026-10-17 its FAQ
|
||||
named Google DeepMind, Nvidia, OpenAI, and Kakao, and its settled labels were
|
||||
`Watermark detected` and `Watermark not detected`. The old limited portal does
|
||||
`Watermark detected` and `Watermark not detected`. On 2026-10-19 UTC the
|
||||
signed-in page also used the status line `SynthID was not detected`, which this
|
||||
tooling records as `not_detected`. The old limited portal does
|
||||
not work: on 2026-09-07, its official `labs.google/synthid` entry point
|
||||
redirected a signed-in browser to a Google 404. On 2026-09-23 the same entry
|
||||
point redirected to `deepmind.google.com/frontiers/synthid`, which also returned
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@
|
||||
# on it, including the ComfyUI node package. The console script below carries
|
||||
# the same weight, since users have it on PATH.
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.46.0"
|
||||
version = "0.47.0"
|
||||
description = "AI watermark remover for visible, invisible, and provenance marks in images and video"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11,<3.15"
|
||||
|
||||
@@ -5,7 +5,9 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
import secrets
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from time import monotonic
|
||||
@@ -22,7 +24,35 @@ if TYPE_CHECKING:
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
PLAYWRIGHT_SURFACES = frozenset({"openai-web", "microsoft-web", "meta-web"})
|
||||
PLAYWRIGHT_SURFACES = frozenset({"openai-web", "microsoft-web", "meta-web", "synthid-web"})
|
||||
# A proxy exit is chosen when the run starts. These are the countries that
|
||||
# choice may use: the United States and European ISO codes.
|
||||
EUROPE_OR_US_COUNTRIES = (
|
||||
"AT",
|
||||
"BE",
|
||||
"CH",
|
||||
"CZ",
|
||||
"DE",
|
||||
"DK",
|
||||
"ES",
|
||||
"FI",
|
||||
"FR",
|
||||
"GB",
|
||||
"GR",
|
||||
"HU",
|
||||
"IE",
|
||||
"IT",
|
||||
"NL",
|
||||
"NO",
|
||||
"PL",
|
||||
"PT",
|
||||
"RO",
|
||||
"SE",
|
||||
"US",
|
||||
)
|
||||
_THORDATA_COUNTRY = re.compile(r"-country-[A-Za-z]{2}", re.IGNORECASE)
|
||||
_THORDATA_SESSION = re.compile(r"-sessid-[A-Za-z0-9]+", re.IGNORECASE)
|
||||
_THORDATA_SESSION_TIME = re.compile(r"-sesstime-\d+", re.IGNORECASE)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -51,6 +81,11 @@ _RESULT_MARKERS = {
|
||||
"No AI signatures from Meta were found",
|
||||
"No AI signatures were found",
|
||||
),
|
||||
"synthid-web": (
|
||||
"Watermark detected",
|
||||
"Watermark not detected",
|
||||
"SynthID was not detected",
|
||||
),
|
||||
}
|
||||
|
||||
_REFUSAL_MARKERS = (
|
||||
@@ -63,7 +98,10 @@ _REFUSAL_MARKERS = (
|
||||
"access denied",
|
||||
)
|
||||
|
||||
_INDETERMINATE_MARKERS = ("Something went wrong. Please try again. If the issue persists, contact support.",)
|
||||
_INDETERMINATE_MARKERS = (
|
||||
"Something went wrong. Please try again. If the issue persists, contact support.",
|
||||
"Please sign in before detection.",
|
||||
)
|
||||
|
||||
|
||||
def _contains_any(text: str, markers: tuple[str, ...]) -> bool:
|
||||
@@ -111,6 +149,16 @@ def parse_provider_result(surface: str, page_text: str) -> WebVerdict:
|
||||
return WebVerdict("detected", provenance, response)
|
||||
if "inconclusive" in lowered or "no provenance detected" in lowered:
|
||||
return WebVerdict("not_detected", provenance, response)
|
||||
elif surface == "synthid-web":
|
||||
# The exact labels are the verdict. A page that shows both, or that
|
||||
# asks for a sign-in, has no single watermark result. The sign-in
|
||||
# sentence is kept; the skill then continues in the user's existing Chrome.
|
||||
detected = "Watermark detected" in response
|
||||
not_detected = "Watermark not detected" in response or "SynthID was not detected" in response
|
||||
if not_detected and not detected:
|
||||
return WebVerdict("not_detected", "unavailable", response)
|
||||
if detected and not not_detected:
|
||||
return WebVerdict("detected", "unavailable", response)
|
||||
else:
|
||||
if "no ai signatures from meta were found" in lowered or "no ai signatures were found" in lowered:
|
||||
return WebVerdict("not_detected", "unavailable", response)
|
||||
@@ -123,13 +171,37 @@ def parse_provider_result(surface: str, page_text: str) -> WebVerdict:
|
||||
return WebVerdict("indeterminate", provenance, response or "Provider page returned no readable result")
|
||||
|
||||
|
||||
def choose_proxy_country(rng: random.Random | None = None) -> str:
|
||||
"""Pick one exit country from Europe or the United States."""
|
||||
picker = random.SystemRandom() if rng is None else rng
|
||||
return picker.choice(EUROPE_OR_US_COUNTRIES)
|
||||
|
||||
|
||||
def new_proxy_session() -> str:
|
||||
"""Return a new ThorData session id. A later call does not reuse it."""
|
||||
return "s" + secrets.token_hex(8)
|
||||
|
||||
|
||||
def fresh_thordata_username(username: str, *, country: str, session_id: str) -> str:
|
||||
"""Drop a prepared country and session, then attach the ones for this run."""
|
||||
if country not in EUROPE_OR_US_COUNTRIES:
|
||||
raise ValueError("proxy country must be in Europe or the US")
|
||||
if re.fullmatch(r"[A-Za-z0-9]{1,32}", session_id) is None:
|
||||
raise ValueError("proxy session id must be 1-32 ASCII letters or digits")
|
||||
base = _THORDATA_SESSION_TIME.sub("", username)
|
||||
base = _THORDATA_SESSION.sub("", base)
|
||||
base = _THORDATA_COUNTRY.sub("", base).strip("-")
|
||||
return f"{base}-country-{country}-sessid-{session_id}-sesstime-90"
|
||||
|
||||
|
||||
def proxy_settings(
|
||||
slot: oracles.ExecutionSlot,
|
||||
environ: Mapping[str, str] = os.environ,
|
||||
*,
|
||||
sticky_session_id: str | None = None,
|
||||
country: str | None = None,
|
||||
session_id: str | None = None,
|
||||
) -> ProxySettings | None:
|
||||
"""Resolve a secret proxy URL and pin rotating ThorData routes per batch."""
|
||||
"""Resolve a secret proxy URL. A ThorData exit is chosen for this call."""
|
||||
variable = slot.get("proxy_url_env")
|
||||
if variable is None:
|
||||
return None
|
||||
@@ -147,10 +219,11 @@ def proxy_settings(
|
||||
if parsed.username is not None:
|
||||
username = unquote(parsed.username)
|
||||
is_thordata = parsed.hostname.endswith((".thordata.net", ".thordata.online"))
|
||||
if is_thordata and sticky_session_id is not None and "-sessid-" not in username.casefold():
|
||||
if re.fullmatch(r"[A-Za-z0-9]{1,32}", sticky_session_id) is None:
|
||||
raise ValueError("sticky_session_id must be 1-32 ASCII letters or digits")
|
||||
username = f"{username}-sessid-{sticky_session_id}-sesstime-90"
|
||||
if is_thordata:
|
||||
chosen_country = choose_proxy_country() if country is None else country
|
||||
chosen_session = new_proxy_session() if session_id is None else session_id
|
||||
username = fresh_thordata_username(username, country=chosen_country, session_id=chosen_session)
|
||||
log.info("ThorData proxy exit country=%s session=%s", chosen_country, chosen_session)
|
||||
settings["username"] = username
|
||||
if parsed.password is not None:
|
||||
settings["password"] = unquote(parsed.password)
|
||||
@@ -198,7 +271,7 @@ def _set_upload_file(page: Page, surface: str, upload: Path, timeout_seconds: fl
|
||||
# OpenAI keeps background requests alive, so network-idle is
|
||||
# unreachable. Its visible input becomes handler-ready at page load.
|
||||
page.wait_for_load_state("load", timeout=timeout_ms)
|
||||
elif surface in {"meta-web", "microsoft-web"}:
|
||||
elif surface in {"meta-web", "microsoft-web", "synthid-web"}:
|
||||
# These pages mount hidden inputs before their selection handlers are
|
||||
# active. Network-idle is the observable handler-readiness seam.
|
||||
page.wait_for_load_state("networkidle", timeout=timeout_ms)
|
||||
@@ -296,11 +369,7 @@ def run_web_batch(
|
||||
if slot is None:
|
||||
raise ValueError("Playwright batches require an explicit execution slot")
|
||||
secret_values = dict(environ) if env_file is None else oracles.secret_environment(env_file, environ=environ)
|
||||
proxy = proxy_settings(
|
||||
slot,
|
||||
secret_values,
|
||||
sticky_session_id=f"oracle{results['manifest_sha256'][:16]}",
|
||||
)
|
||||
proxy = proxy_settings(slot, secret_values)
|
||||
pending = [
|
||||
(manifest_row, result_row)
|
||||
for manifest_row, result_row in zip(manifest["rows"], results["rows"], strict=True)
|
||||
|
||||
@@ -175,18 +175,18 @@ SURFACES: dict[str, OracleSurface] = {
|
||||
provider="Google",
|
||||
signal_family="synthid",
|
||||
surface="SynthID Detector",
|
||||
automation="real_browser",
|
||||
automation="playwright",
|
||||
url="https://synthid.com/",
|
||||
media_types=("image", "video", "audio"),
|
||||
project_evidence=(),
|
||||
prompt=None,
|
||||
clean_result_note=(
|
||||
"Record detected only for the explicit label Watermark detected, and not_detected only for "
|
||||
"Watermark not detected. The portal reports partner SynthID from Google DeepMind, Nvidia, "
|
||||
"OpenAI, and Kakao without naming which partner. A not_detected result is not proof of "
|
||||
"human authorship. Preserve quota, sign-in, and unclear wording."
|
||||
"Record detected only for the explicit label Watermark detected, and not_detected for "
|
||||
"Watermark not detected or SynthID was not detected. The portal reports partner SynthID from "
|
||||
"Google DeepMind, Nvidia, OpenAI, and Kakao without naming which partner. A not_detected "
|
||||
"result is not proof of human authorship. Preserve quota, sign-in, and unclear wording."
|
||||
),
|
||||
account_slots=True,
|
||||
account_slots=False,
|
||||
network_slots=True,
|
||||
),
|
||||
"meta-web": OracleSurface(
|
||||
@@ -444,6 +444,7 @@ def load_slots(path: Path) -> dict[str, ExecutionSlot]:
|
||||
raise ValueError(f"API surface {surface_key} does not use a browser_profile")
|
||||
if surface_key not in real_browser and browser_profile is not None:
|
||||
raise ValueError("only a real-browser surface uses a browser_profile")
|
||||
# A proxy belongs to Playwright pages. Gemini and API calls stay direct.
|
||||
if proxy_url_env is not None and surface_key not in playwright:
|
||||
raise ValueError(f"surface {surface_key} does not use proxy_url_env")
|
||||
if surface_key == "openai-api" and api_key_env is None:
|
||||
@@ -894,6 +895,8 @@ def plan(provider: str, as_json: bool) -> None:
|
||||
return
|
||||
for row in rows:
|
||||
click.echo(f"{row['preference']}\t{row['key']}\t{row['automation']}\t{row['surface']}")
|
||||
if len(rows) > 1:
|
||||
click.echo("This command only prints the order. After a failure, run the next surface as its own command.")
|
||||
|
||||
|
||||
@cli.command()
|
||||
@@ -1035,7 +1038,7 @@ def run_web(
|
||||
timeout_seconds: float,
|
||||
env_file: Path,
|
||||
) -> None:
|
||||
"""Submit a prepared non-Google Web batch through isolated Playwright."""
|
||||
"""Submit a prepared anonymous Web batch through isolated Playwright."""
|
||||
from provider_oracle_web import run_web_batch
|
||||
|
||||
report = run_web_batch(
|
||||
|
||||
+2
-2
@@ -3,7 +3,7 @@
|
||||
"name": "io.github.wiltodelta/remove-ai-watermarks",
|
||||
"title": "Remove AI Watermarks",
|
||||
"description": "Identify and remove AI watermarks and provenance metadata from your own images and video.",
|
||||
"version": "0.46.0",
|
||||
"version": "0.47.0",
|
||||
"repository": {
|
||||
"url": "https://github.com/wiltodelta/remove-ai-watermarks",
|
||||
"source": "github"
|
||||
@@ -14,7 +14,7 @@
|
||||
"registryType": "pypi",
|
||||
"registryBaseUrl": "https://pypi.org",
|
||||
"identifier": "remove-ai-watermarks",
|
||||
"version": "0.46.0",
|
||||
"version": "0.47.0",
|
||||
"runtimeHint": "uvx",
|
||||
"transport": {
|
||||
"type": "stdio"
|
||||
|
||||
@@ -32,7 +32,7 @@ _GPU_MARKERS = ("GPU ", "gpu:")
|
||||
# told an agent its CLI was current right before it typed a value that build
|
||||
# rejects. tests/test_agent_skill.py pins it against the package version and
|
||||
# against every "(since CLI X.Y.Z)" annotation in the skill.
|
||||
MIN_CLI_VERSION = (0, 46, 0)
|
||||
MIN_CLI_VERSION = (0, 47, 0)
|
||||
# Markers a build without the pixel stack prints. The first is the current CLI's own
|
||||
# install hint; the rest are what older builds emit before the guard existed, and the
|
||||
# Homebrew formula ships exactly such a build.
|
||||
|
||||
@@ -35,7 +35,7 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
|
||||
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
|
||||
|
||||
|
||||
__version__ = "0.46.0"
|
||||
__version__ = "0.47.0"
|
||||
|
||||
__all__ = [
|
||||
"BatchItemResult",
|
||||
|
||||
@@ -736,10 +736,17 @@ def strip_isobmff_media_file(
|
||||
return len(targets), len(tc260_key_spans | ai_metadata_key_spans | app_key_spans) + bool(kling_sei_offsets)
|
||||
|
||||
|
||||
def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
|
||||
"""Return ``(cleaned_bytes, stripped_count)`` with AI-provenance boxes removed.
|
||||
def _blank_box(data: bytes, start: int, end: int, payload_off: int) -> bytes:
|
||||
"""Return ``data[start:end]`` as a ``free`` box of the same length and size
|
||||
encoding, with its payload zeroed, so no later file offset moves."""
|
||||
return data[start : start + 4] + b"free" + data[start + 8 : payload_off] + bytes(end - payload_off)
|
||||
|
||||
Walks top-level boxes and drops:
|
||||
|
||||
def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
|
||||
"""Return ``(cleaned_bytes, stripped_count)`` with AI-provenance boxes blanked.
|
||||
|
||||
Walks top-level boxes and turns each of these into a same-length ``free`` box
|
||||
with a zeroed payload:
|
||||
- any ``uuid`` box whose UUID equals ``C2PA_UUID`` (a C2PA manifest);
|
||||
- any ``uuid`` box whose payload carries an AI-label marker (an XMP packet
|
||||
with a TC260 / IPTC / IPTC-2025.1 AI field -- caught by content, not by the
|
||||
@@ -747,10 +754,15 @@ def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
|
||||
non-AI XMP intact);
|
||||
- any ``jumb`` box (JPEG-XL JUMBF container).
|
||||
|
||||
All other boxes (incl. ``mdat`` / codestream) are emitted verbatim, so pixel
|
||||
and audio data is preserved bit-for-bit. Non-ISOBMFF input is returned
|
||||
The box is blanked rather than dropped because HEIF ``iloc`` and MP4
|
||||
``stco``/``co64`` locate the coded data by absolute file offset: dropping a
|
||||
box that precedes ``mdat`` moved the image without rewriting them, and a
|
||||
phone HEIC cleaned that way no longer decoded (2026-10-13). This matches
|
||||
:func:`strip_isobmff_media_file`. All other boxes (incl. ``mdat`` /
|
||||
codestream) are emitted verbatim, so pixel and audio data is preserved
|
||||
bit-for-bit. Non-ISOBMFF input is returned
|
||||
unchanged. Despite the name this also covers MP4/MOV/M4A video and audio
|
||||
(all ISOBMFF). NOTE: this drops only top-level boxes. AI metadata stored as an
|
||||
(all ISOBMFF). NOTE: this blanks only top-level boxes. AI metadata stored as an
|
||||
*item inside the ``meta`` box* (typical for AVIF/HEIF) is handled separately and
|
||||
in place (same length, no offset rewrite): AI-label XMP by
|
||||
:func:`blank_ai_xmp_packets`, and AI-generator tokens in an ``Exif`` item by
|
||||
@@ -764,17 +776,16 @@ def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
|
||||
consumed = 0
|
||||
for start, end, box_type, payload_off in _iter_top_level_boxes(data):
|
||||
consumed = end
|
||||
provenance = box_type == b"jumb"
|
||||
if box_type == b"uuid":
|
||||
# uuid boxes carry the 16-byte UUID immediately after the type.
|
||||
is_c2pa = payload_off + 16 <= end and data[payload_off : payload_off + 16] == C2PA_UUID
|
||||
has_ai_label = any(marker in data[payload_off:end] for marker in _AI_LABEL_MARKERS)
|
||||
if is_c2pa or has_ai_label:
|
||||
stripped += 1
|
||||
continue
|
||||
elif box_type == b"jumb":
|
||||
provenance = is_c2pa or any(marker in data[payload_off:end] for marker in _AI_LABEL_MARKERS)
|
||||
if provenance:
|
||||
stripped += 1
|
||||
continue
|
||||
out.extend(data[start:end])
|
||||
out.extend(_blank_box(data, start, end, payload_off))
|
||||
else:
|
||||
out.extend(data[start:end])
|
||||
|
||||
# Fail-safe: the walker returns early on a malformed box (bad size, or a box
|
||||
# that runs past EOF), so anything after it was never visited. Emitting `out`
|
||||
|
||||
@@ -212,17 +212,18 @@ class WatermarkRemover:
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks import image_io
|
||||
from remove_ai_watermarks._internal.utils import staged_sibling
|
||||
from remove_ai_watermarks.metadata import strip_written_output
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
bgr = np.ascontiguousarray(np.asarray(image.convert("RGB"))[:, :, ::-1])
|
||||
if not image_io.imwrite(str(output_path), bgr, display_tags_from=display_tags_from):
|
||||
image.save(output_path)
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
|
||||
# Verified, so a marker that survives fails the run instead of shipping with it.
|
||||
_, leftover = strip_and_verify(output_path, output_path, keep_standard=True)
|
||||
if leftover:
|
||||
raise RuntimeError(f"AI metadata survived the strip: {', '.join(sorted(leftover))}")
|
||||
# Staged and verified, so a marker that survives fails the run with nothing
|
||||
# published at output_path.
|
||||
with staged_sibling(output_path) as staged:
|
||||
if not image_io.imwrite(str(staged), bgr, display_tags_from=display_tags_from):
|
||||
image.save(staged)
|
||||
if leftover := strip_written_output(staged):
|
||||
raise RuntimeError(f"AI metadata survived the strip: {', '.join(sorted(leftover))}")
|
||||
|
||||
def remove_watermark(
|
||||
self,
|
||||
|
||||
@@ -157,10 +157,9 @@ def _write_visible_result(
|
||||
raise OSError(f"failed to write output (is the destination writable?): {out_path}")
|
||||
|
||||
if strip_metadata:
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
from remove_ai_watermarks.metadata import strip_written_output
|
||||
|
||||
_, leftover = strip_and_verify(staged, staged)
|
||||
if leftover:
|
||||
if leftover := strip_written_output(staged):
|
||||
raise MetadataStripIncomplete(set(leftover))
|
||||
|
||||
|
||||
@@ -464,7 +463,7 @@ def remove_all(
|
||||
import tempfile
|
||||
|
||||
from remove_ai_watermarks import image_io, watermark_registry
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
from remove_ai_watermarks.metadata import strip_written_output
|
||||
|
||||
def say(stage: str, detail: str) -> None:
|
||||
if progress is not None:
|
||||
@@ -519,8 +518,7 @@ def remove_all(
|
||||
# ── 3. AI metadata ──
|
||||
# Read the pristine ORIGINAL for provenance above and the STAGED file here:
|
||||
# the visible pass has already dropped this file's C2PA.
|
||||
_, leftover = strip_and_verify(staged, staged)
|
||||
if leftover:
|
||||
if leftover := strip_written_output(staged):
|
||||
# Before the write, on purpose -- see MetadataStripIncomplete.
|
||||
raise MetadataStripIncomplete(set(leftover))
|
||||
say("metadata", "stripped")
|
||||
|
||||
@@ -467,7 +467,7 @@ def _write_output_or_exit(
|
||||
published, so a marker that survives exits 1 with nothing written. Both commands
|
||||
using this used to swallow a failed strip unless ``--verbose`` was set."""
|
||||
from remove_ai_watermarks._internal.utils import staged_sibling
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
from remove_ai_watermarks.metadata import strip_written_output
|
||||
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
with staged_sibling(output) as staged:
|
||||
@@ -477,7 +477,7 @@ def _write_output_or_exit(
|
||||
if not strip_metadata:
|
||||
return
|
||||
try:
|
||||
_, leftover = strip_and_verify(staged, staged)
|
||||
leftover = strip_written_output(staged)
|
||||
except (OSError, RuntimeError, ValueError) as e:
|
||||
console.print(f" Error: failed to strip AI metadata from {output}: {e}")
|
||||
raise SystemExit(1) from e
|
||||
|
||||
@@ -1968,6 +1968,25 @@ def strip_and_verify(
|
||||
"""
|
||||
if (reason := unreadable_reason(source_path)) is not None:
|
||||
raise ValueError(f"cannot read {source_path.name}: {reason}")
|
||||
return _strip_and_rescan(source_path, output_path, keep_standard=keep_standard)
|
||||
|
||||
|
||||
def strip_written_output(path: Path, *, keep_standard: bool = True) -> dict[str, str]:
|
||||
"""Strip ``path`` in place and return the markers that survived.
|
||||
|
||||
For a file whose pixels this run has already decoded or written itself, so the
|
||||
readability probe that :func:`strip_and_verify` runs first would only decode
|
||||
them a second time.
|
||||
"""
|
||||
return _strip_and_rescan(path, path, keep_standard=keep_standard)[1]
|
||||
|
||||
|
||||
def _strip_and_rescan(
|
||||
source_path: Path,
|
||||
output_path: Path | None,
|
||||
*,
|
||||
keep_standard: bool,
|
||||
) -> tuple[Path, dict[str, str]]:
|
||||
out = remove_ai_metadata(source_path, output_path, keep_standard=keep_standard)
|
||||
remaining = get_ai_metadata(out)
|
||||
if not remaining:
|
||||
|
||||
+1
-1
@@ -430,7 +430,7 @@ class TestRemoveAllLibrary:
|
||||
from remove_ai_watermarks import api, invisible_engine, metadata
|
||||
|
||||
monkeypatch.setattr(invisible_engine, "is_available", lambda: False)
|
||||
monkeypatch.setattr(metadata, "strip_and_verify", lambda src, dst: (dst, {"c2pa"}))
|
||||
monkeypatch.setattr(metadata, "strip_written_output", lambda _path: {"c2pa": "survived"})
|
||||
out = tmp_path / "clean.png"
|
||||
|
||||
with pytest.raises(api.MetadataStripIncomplete, match="c2pa"):
|
||||
|
||||
+1
-4
@@ -675,10 +675,7 @@ class TestAllCommand:
|
||||
# The no-signal gate lives in the library now; patch it there, not on the
|
||||
# CLI module, or the diffusion stage runs for real.
|
||||
patch("remove_ai_watermarks.api._SourceEvidence.has_invisible_target", return_value=False),
|
||||
patch(
|
||||
"remove_ai_watermarks.metadata.strip_and_verify",
|
||||
return_value=(tmp_path / "intermediate.png", {"c2pa": True}),
|
||||
),
|
||||
patch("remove_ai_watermarks.metadata.strip_written_output", return_value={"c2pa": "survived"}),
|
||||
):
|
||||
result = runner.invoke(main, ["all", str(sample_png), "-o", str(output)])
|
||||
|
||||
|
||||
@@ -239,6 +239,29 @@ class TestSkippedStepsAreFailures:
|
||||
remover = object.__new__(WatermarkRemover) # the writer needs no model state
|
||||
with pytest.raises(RuntimeError, match="survived"):
|
||||
remover._write_output(Image.new("RGB", (16, 16)), tmp_path / "out.png")
|
||||
assert list(tmp_path.iterdir()) == [], "an output that kept its AI metadata was published"
|
||||
|
||||
@pytest.mark.parametrize("command", ["erase", "remove_visible", "invisible_writer"])
|
||||
def test_an_output_the_package_just_wrote_is_not_probed_again(self, tmp_path, clean_png, monkeypatch, command):
|
||||
"""The readability probe decodes every pixel; on our own encode it is pure waste."""
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
|
||||
from remove_ai_watermarks.api import remove_visible
|
||||
|
||||
probed: list[Path] = []
|
||||
real = metadata.unreadable_reason
|
||||
monkeypatch.setattr(metadata, "unreadable_reason", lambda path: probed.append(path) or real(path))
|
||||
out = tmp_path / "out.png"
|
||||
if command == "erase":
|
||||
result = _invoke(["erase", "--region", "2,2,10,10", str(clean_png), "-o", str(out)])
|
||||
assert result.exit_code == 0, result.output
|
||||
elif command == "remove_visible":
|
||||
remove_visible(clean_png, out, write_noop=True)
|
||||
else:
|
||||
object.__new__(WatermarkRemover)._write_output(Image.new("RGB", (16, 16)), out)
|
||||
assert out.exists()
|
||||
assert [p for p in probed if p.name != clean_png.name] == []
|
||||
|
||||
def test_metadata_remove_reports_a_missing_ffmpeg_without_a_traceback(self, tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(shutil, "which", lambda _name: None)
|
||||
|
||||
+95
-7
@@ -29,6 +29,12 @@ from remove_ai_watermarks.metadata import (
|
||||
xai_signature,
|
||||
)
|
||||
|
||||
|
||||
def _blanked(box: bytes) -> bytes:
|
||||
"""The same-length ``free`` box ``strip_c2pa_boxes`` leaves in place of a 32-bit-size box."""
|
||||
return box[:4] + b"free" + bytes(len(box) - 8)
|
||||
|
||||
|
||||
# Real, committed C2PA sample images used to ground the SynthID-source tests.
|
||||
SAMPLES_DIR = Path(__file__).resolve().parent.parent / "data" / "fixtures" / "provenance"
|
||||
|
||||
@@ -123,7 +129,7 @@ class TestHasAiMetadata:
|
||||
assert has_ai_metadata(path)
|
||||
|
||||
def test_strip_c2pa_boxes_removes_uuid_box(self, tmp_path: Path):
|
||||
"""ISOBMFF strip should drop the C2PA uuid box and keep everything else."""
|
||||
"""ISOBMFF strip should blank the C2PA uuid box in place and keep everything else."""
|
||||
from remove_ai_watermarks._internal.isobmff import strip_c2pa_boxes
|
||||
from remove_ai_watermarks.metadata import C2PA_UUID
|
||||
|
||||
@@ -133,7 +139,7 @@ class TestHasAiMetadata:
|
||||
mdat = b"\x00\x00\x00\x10mdat" + b"pixeldat"
|
||||
cleaned, stripped = strip_c2pa_boxes(ftyp + uuid_box + mdat)
|
||||
assert stripped == 1
|
||||
assert cleaned == ftyp + mdat
|
||||
assert cleaned == ftyp + _blanked(uuid_box) + mdat
|
||||
|
||||
def test_strip_c2pa_boxes_passthrough_for_non_isobmff(self):
|
||||
"""Non-ISOBMFF input must be returned unchanged."""
|
||||
@@ -145,7 +151,7 @@ class TestHasAiMetadata:
|
||||
assert cleaned == data
|
||||
|
||||
def test_remove_ai_metadata_strips_c2pa_in_avif(self, tmp_path: Path):
|
||||
"""End-to-end: ``remove_ai_metadata`` on a fake .avif drops the C2PA box."""
|
||||
"""End-to-end: ``remove_ai_metadata`` on a fake .avif blanks the C2PA box."""
|
||||
from remove_ai_watermarks.metadata import C2PA_UUID, remove_ai_metadata
|
||||
|
||||
src = tmp_path / "in.avif"
|
||||
@@ -157,7 +163,7 @@ class TestHasAiMetadata:
|
||||
out = tmp_path / "out.avif"
|
||||
result = remove_ai_metadata(src, out)
|
||||
assert result == out
|
||||
assert out.read_bytes() == ftyp + mdat
|
||||
assert out.read_bytes() == ftyp + _blanked(uuid_box) + mdat
|
||||
# And after stripping, detection must no longer flag the cleaned file.
|
||||
from remove_ai_watermarks.metadata import has_ai_metadata
|
||||
|
||||
@@ -2087,7 +2093,7 @@ class TestIsobmffMetadataRemoval:
|
||||
"""Container-level AI-provenance stripping across ISOBMFF image/video/audio."""
|
||||
|
||||
def test_strips_ai_xmp_uuid_box(self):
|
||||
# A uuid box carrying a TC260 AIGC label is dropped by content match,
|
||||
# A uuid box carrying a TC260 AIGC label is blanked by content match,
|
||||
# regardless of the (non-C2PA) XMP UUID's byte order.
|
||||
from remove_ai_watermarks._internal.isobmff import strip_c2pa_boxes
|
||||
|
||||
@@ -2096,7 +2102,7 @@ class TestIsobmffMetadataRemoval:
|
||||
box = (24 + len(payload)).to_bytes(4, "big") + b"uuid" + xmp_uuid + payload
|
||||
cleaned, stripped = strip_c2pa_boxes(_MP4_FTYP + box + _MP4_MDAT)
|
||||
assert stripped == 1
|
||||
assert cleaned == _MP4_FTYP + _MP4_MDAT
|
||||
assert cleaned == _MP4_FTYP + _blanked(box) + _MP4_MDAT
|
||||
|
||||
def test_keeps_plain_non_ai_xmp(self):
|
||||
# A uuid box with ordinary (non-AI) XMP must be preserved.
|
||||
@@ -2128,7 +2134,7 @@ class TestIsobmffMetadataRemoval:
|
||||
src.write_bytes(_MP4_FTYP + uuid_box + _MP4_MDAT)
|
||||
out = tmp_path / "out.bin"
|
||||
remove_ai_metadata(src, out)
|
||||
assert out.read_bytes() == _MP4_FTYP + _MP4_MDAT
|
||||
assert out.read_bytes() == _MP4_FTYP + _blanked(uuid_box) + _MP4_MDAT
|
||||
|
||||
def test_unparseable_audio_raises(self, tmp_path: Path):
|
||||
# Garbage that ffmpeg can't parse must raise a clear error, not crash in
|
||||
@@ -2560,3 +2566,85 @@ class TestTc260AudioPlacements:
|
||||
cleaned = tmp_path / f"clean{suffix}"
|
||||
remove_ai_metadata(source, cleaned)
|
||||
assert "aigc_label" not in get_ai_metadata(cleaned)
|
||||
|
||||
|
||||
def _shift_heif_iloc(data: bytes, delta: int) -> bytes:
|
||||
"""Add ``delta`` to every file-offset extent in the HEIF ``iloc`` box.
|
||||
|
||||
Lets a test insert a box ahead of ``mdat`` the way a C2PA signer does, which
|
||||
moves the coded image by ``delta`` bytes and so needs ``iloc`` rewritten.
|
||||
"""
|
||||
out = bytearray(data)
|
||||
meta = data.index(b"meta") - 4
|
||||
iloc = data.index(b"iloc", meta) - 4
|
||||
version = data[iloc + 8]
|
||||
pos = iloc + 12
|
||||
offset_size, length_size = data[pos] >> 4, data[pos] & 15
|
||||
base_size = data[pos + 1] >> 4
|
||||
index_size = data[pos + 1] & 15 if version in (1, 2) else 0
|
||||
pos += 2
|
||||
id_size = 2 if version < 2 else 4
|
||||
count = int.from_bytes(data[pos : pos + id_size], "big")
|
||||
pos += id_size
|
||||
for _ in range(count):
|
||||
pos += id_size
|
||||
method = 0
|
||||
if version in (1, 2):
|
||||
method = data[pos + 1] & 15
|
||||
pos += 2
|
||||
pos += 2 # data_reference_index
|
||||
base = int.from_bytes(data[pos : pos + base_size], "big") if base_size else 0
|
||||
if method == 0 and base_size:
|
||||
out[pos : pos + base_size] = (base + delta).to_bytes(base_size, "big")
|
||||
pos += base_size
|
||||
extents = int.from_bytes(data[pos : pos + 2], "big")
|
||||
pos += 2
|
||||
for _ in range(extents):
|
||||
pos += index_size
|
||||
if method == 0 and not base_size:
|
||||
offset = int.from_bytes(data[pos : pos + offset_size], "big")
|
||||
out[pos : pos + offset_size] = (offset + delta).to_bytes(offset_size, "big")
|
||||
pos += offset_size + length_size
|
||||
return bytes(out)
|
||||
|
||||
|
||||
class TestIsobmffStripKeepsOffsets:
|
||||
"""A HEIF locates its coded image through absolute ``iloc`` offsets, and an MP4
|
||||
through ``stco``/``co64``. Dropping a C2PA ``uuid`` box that sits ahead of
|
||||
``mdat`` moved the image without rewriting them, so the cleaned HEIC no longer
|
||||
decoded (a phone HEIC, 2026-10-13)."""
|
||||
|
||||
def test_c2pa_box_before_mdat_keeps_heic_decodable(self, tmp_path: Path):
|
||||
pillow_heif = pytest.importorskip("pillow_heif")
|
||||
pillow_heif.register_heif_opener()
|
||||
plain = tmp_path / "plain.heic"
|
||||
Image.new("RGB", (64, 48), (200, 30, 30)).save(plain, format="HEIF")
|
||||
data = plain.read_bytes()
|
||||
c2pa_box = struct.pack(">I", 8 + 16 + 64) + b"uuid" + C2PA_UUID + b"\x07" * 64
|
||||
mdat = data.index(b"mdat") - 4
|
||||
signed = _shift_heif_iloc(data[:mdat], len(c2pa_box)) + c2pa_box + data[mdat:]
|
||||
source = tmp_path / "signed.heic"
|
||||
source.write_bytes(signed)
|
||||
with Image.open(source) as im:
|
||||
expected = im.convert("RGB").tobytes()
|
||||
|
||||
cleaned = remove_ai_metadata(source, tmp_path / "clean.heic")
|
||||
|
||||
assert C2PA_UUID not in cleaned.read_bytes()
|
||||
with Image.open(cleaned) as im:
|
||||
assert im.convert("RGB").tobytes() == expected
|
||||
|
||||
def test_blanked_box_keeps_every_later_offset(self):
|
||||
from remove_ai_watermarks._internal.isobmff import strip_c2pa_boxes
|
||||
|
||||
ftyp = b"\x00\x00\x00\x18ftypheic\x00\x00\x00\x00mif1heic"
|
||||
c2pa_box = struct.pack(">I", 8 + 16 + 8) + b"uuid" + C2PA_UUID + b"manifest"
|
||||
mdat = b"\x00\x00\x00\x10mdat" + b"pixeldat"
|
||||
data = ftyp + c2pa_box + mdat
|
||||
|
||||
cleaned, stripped = strip_c2pa_boxes(data)
|
||||
|
||||
assert stripped == 1
|
||||
assert len(cleaned) == len(data)
|
||||
assert cleaned.index(b"mdat") == data.index(b"mdat")
|
||||
assert cleaned == ftyp + _blanked(c2pa_box) + mdat
|
||||
@@ -595,11 +595,11 @@ class TestISOBMFF:
|
||||
assert not is_isobmff(b"abc")
|
||||
|
||||
def test_strips_jpegxl_jumb_box(self):
|
||||
"""JPEG-XL stores JUMBF in a ``jumb`` box, always stripped."""
|
||||
"""JPEG-XL stores JUMBF in a ``jumb`` box, always blanked to ``free``."""
|
||||
jumb = struct.pack(">I", 8 + 5) + b"jumb" + b"hello"
|
||||
cleaned, stripped = strip_c2pa_boxes(FTYP + jumb)
|
||||
assert stripped == 1
|
||||
assert cleaned == FTYP
|
||||
assert cleaned == FTYP + struct.pack(">I", 8 + 5) + b"free" + bytes(5)
|
||||
|
||||
def test_keeps_non_c2pa_box_with_64bit_size(self):
|
||||
"""size==1 means a 64-bit largesize follows; non-C2PA box is kept."""
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import provider_oracle_web as web
|
||||
@@ -76,6 +77,36 @@ def _thordata_slot() -> oracles.ExecutionSlot:
|
||||
"not_detected",
|
||||
"unavailable",
|
||||
),
|
||||
(
|
||||
"synthid-web",
|
||||
"Watermark detected",
|
||||
"detected",
|
||||
"unavailable",
|
||||
),
|
||||
(
|
||||
"synthid-web",
|
||||
"Watermark not detected",
|
||||
"not_detected",
|
||||
"unavailable",
|
||||
),
|
||||
(
|
||||
"synthid-web",
|
||||
"SynthID was not detected\nIt is unlikely that this media was made with AI from a SynthID partner",
|
||||
"not_detected",
|
||||
"unavailable",
|
||||
),
|
||||
(
|
||||
"synthid-web",
|
||||
"Please sign in before detection.",
|
||||
"indeterminate",
|
||||
"unavailable",
|
||||
),
|
||||
(
|
||||
"synthid-web",
|
||||
"Watermark detected\nWatermark not detected",
|
||||
"indeterminate",
|
||||
"unavailable",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_parse_provider_result(
|
||||
@@ -122,7 +153,12 @@ def test_openai_provider_error_is_a_settled_indeterminate_result() -> None:
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("surface", "ready_state"),
|
||||
[("meta-web", "networkidle"), ("microsoft-web", "networkidle"), ("openai-web", "load")],
|
||||
[
|
||||
("meta-web", "networkidle"),
|
||||
("microsoft-web", "networkidle"),
|
||||
("openai-web", "load"),
|
||||
("synthid-web", "networkidle"),
|
||||
],
|
||||
)
|
||||
def test_react_uploads_wait_for_surface_readiness_before_assigning_the_file(
|
||||
tmp_path: Path,
|
||||
@@ -183,29 +219,44 @@ def test_proxy_settings_are_loaded_from_an_environment_reference() -> None:
|
||||
}
|
||||
|
||||
|
||||
def test_thordata_proxy_settings_pin_one_exit_for_the_browser_session() -> None:
|
||||
def test_thordata_proxy_settings_take_a_fresh_europe_or_us_session() -> None:
|
||||
proxy = web.proxy_settings(
|
||||
_thordata_slot(),
|
||||
{"THORDATA_ROUTE_US": "http://route:secret@gateway.pr.thordata.net:9999"},
|
||||
sticky_session_id="oracle0123456789abcdef",
|
||||
{
|
||||
"THORDATA_ROUTE_US": "http://td-customer-user-country-JP-sessid-prepared-sesstime-15:secret@gateway.pr.thordata.net:9999"
|
||||
},
|
||||
country="DE",
|
||||
session_id="freshsession01",
|
||||
)
|
||||
|
||||
assert proxy == {
|
||||
"server": "http://gateway.pr.thordata.net:9999",
|
||||
"username": "route-sessid-oracle0123456789abcdef-sesstime-90",
|
||||
"username": "td-customer-user-country-DE-sessid-freshsession01-sesstime-90",
|
||||
"password": "secret",
|
||||
}
|
||||
|
||||
|
||||
def test_thordata_proxy_settings_preserve_an_explicit_session() -> None:
|
||||
proxy = web.proxy_settings(
|
||||
_thordata_slot(),
|
||||
{"THORDATA_ROUTE_US": ("http://route-sessid-manual-sesstime-15:secret@gateway.pr.thordata.net:9999")},
|
||||
sticky_session_id="oracle0123456789abcdef",
|
||||
)
|
||||
def test_thordata_proxy_settings_reject_a_country_outside_europe_or_the_us() -> None:
|
||||
assert "US" in web.EUROPE_OR_US_COUNTRIES
|
||||
assert "DE" in web.EUROPE_OR_US_COUNTRIES
|
||||
assert "JP" not in web.EUROPE_OR_US_COUNTRIES
|
||||
|
||||
assert proxy is not None
|
||||
assert proxy["username"] == "route-sessid-manual-sesstime-15"
|
||||
with pytest.raises(ValueError, match="Europe or the US"):
|
||||
web.proxy_settings(
|
||||
_thordata_slot(),
|
||||
{"THORDATA_ROUTE_US": "http://route:secret@gateway.pr.thordata.net:9999"},
|
||||
country="JP",
|
||||
session_id="freshsession01",
|
||||
)
|
||||
|
||||
|
||||
def test_new_proxy_session_changes_between_calls() -> None:
|
||||
first = web.new_proxy_session()
|
||||
second = web.new_proxy_session()
|
||||
|
||||
assert first != second
|
||||
assert re.fullmatch(r"[A-Za-z0-9]{1,32}", first)
|
||||
assert re.fullmatch(r"[A-Za-z0-9]{1,32}", second)
|
||||
|
||||
|
||||
def test_proxy_settings_do_not_treat_a_missing_secret_as_direct_access() -> None:
|
||||
@@ -356,6 +407,8 @@ def test_run_web_batch_loads_the_selected_proxy_from_dotenv(
|
||||
proxies.append(proxy)
|
||||
return [web.WebVerdict("not_detected", "unavailable", "No OpenAI signals detected")]
|
||||
|
||||
monkeypatch.setattr(web, "choose_proxy_country", lambda rng=None: "NL")
|
||||
monkeypatch.setattr(web, "new_proxy_session", lambda: "freshsession01")
|
||||
monkeypatch.setattr(web, "_submit_batch", submit)
|
||||
report = web.run_web_batch(
|
||||
manifest_path,
|
||||
@@ -367,11 +420,10 @@ def test_run_web_batch_loads_the_selected_proxy_from_dotenv(
|
||||
)
|
||||
|
||||
assert report["watermark_results"] == {"not_detected": 1}
|
||||
manifest_sha256 = json.loads((manifest_path.parent / "results.json").read_text(encoding="utf-8"))["manifest_sha256"]
|
||||
assert proxies == [
|
||||
{
|
||||
"server": "http://proxy.pr.thordata.net:9999",
|
||||
"username": (f"td-customer-user-country-US-sessid-oracle{manifest_sha256[:16]}-sesstime-90"),
|
||||
"username": "td-customer-user-country-NL-sessid-freshsession01-sesstime-90",
|
||||
"password": "test-thordata-password",
|
||||
}
|
||||
]
|
||||
@@ -392,14 +444,12 @@ def test_run_web_batch_requires_explicit_upload_acknowledgement(
|
||||
web.run_web_batch(manifest_path, acknowledge_uploads=False)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("surface", ["gemini-web", "synthid-web"])
|
||||
def test_run_web_batch_refuses_google_real_browser_workflow(
|
||||
def test_run_web_batch_refuses_gemini_real_browser_workflow(
|
||||
tmp_path: Path,
|
||||
tmp_clean_png: Path,
|
||||
surface: str,
|
||||
) -> None:
|
||||
manifest_path = oracles.prepare_batch(
|
||||
surface,
|
||||
"gemini-web",
|
||||
[tmp_clean_png],
|
||||
output_dir=tmp_path / "oracle-batch",
|
||||
repository_root=oracles.REPOSITORY_ROOT,
|
||||
@@ -407,3 +457,18 @@ def test_run_web_batch_refuses_google_real_browser_workflow(
|
||||
|
||||
with pytest.raises(ValueError, match="real Chrome"):
|
||||
web.run_web_batch(manifest_path, acknowledge_uploads=True)
|
||||
|
||||
|
||||
def test_run_web_batch_requires_a_slot_for_synthid(
|
||||
tmp_path: Path,
|
||||
tmp_clean_png: Path,
|
||||
) -> None:
|
||||
manifest_path = oracles.prepare_batch(
|
||||
"synthid-web",
|
||||
[tmp_clean_png],
|
||||
output_dir=tmp_path / "oracle-batch",
|
||||
repository_root=oracles.REPOSITORY_ROOT,
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="explicit execution slot"):
|
||||
web.run_web_batch(manifest_path, acknowledge_uploads=True)
|
||||
@@ -36,7 +36,8 @@ def test_catalog_covers_every_project_vendor_oracle_surface() -> None:
|
||||
"do not infer from visible content, logos, text, or metadata."
|
||||
)
|
||||
synthid = oracles.SURFACES["synthid-web"]
|
||||
assert synthid.automation == "real_browser"
|
||||
assert synthid.automation == "playwright"
|
||||
assert synthid.account_slots is False
|
||||
assert synthid.provider_key == "google"
|
||||
assert synthid.url == "https://synthid.com/"
|
||||
assert synthid.media_types == ("image", "video", "audio")
|
||||
@@ -84,9 +85,9 @@ def test_plan_json_keeps_every_google_web_surface_primary() -> None:
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
payload = json.loads(result.output)
|
||||
assert [(row["key"], row["preference"]) for row in payload] == [
|
||||
("gemini-web", "primary"),
|
||||
("synthid-web", "primary"),
|
||||
assert [(row["key"], row["preference"], row["url"]) for row in payload] == [
|
||||
("gemini-web", "primary", "https://gemini.google.com/"),
|
||||
("synthid-web", "primary", "https://synthid.com/"),
|
||||
]
|
||||
|
||||
|
||||
@@ -101,6 +102,16 @@ def test_plan_json_marks_web_as_an_explicit_fallback_when_api_exists() -> None:
|
||||
]
|
||||
|
||||
|
||||
def test_plan_text_tells_the_caller_to_run_the_next_surface() -> None:
|
||||
result = CliRunner().invoke(oracles.cli, ["plan", "openai"])
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "primary\topenai-api\tapi\tContent Provenance API" in result.output
|
||||
assert "fallback\topenai-web\tplaywright\tContent Provenance web verifier" in result.output
|
||||
assert "run the next surface as its own command" in result.output
|
||||
assert "separate operator decision" not in result.output
|
||||
|
||||
|
||||
def test_slots_support_multiple_accounts_and_networks(tmp_path: Path) -> None:
|
||||
config = tmp_path / "slots.json"
|
||||
config.write_text(
|
||||
@@ -140,6 +151,14 @@ def test_slots_support_multiple_accounts_and_networks(tmp_path: Path) -> None:
|
||||
"network_label": "home",
|
||||
"api_key_env": "OPENAI_API_KEY_1",
|
||||
},
|
||||
{
|
||||
"name": "synthid-proxy",
|
||||
"surface": "synthid-web",
|
||||
"account_label": None,
|
||||
"browser_profile": None,
|
||||
"network_label": "thordata",
|
||||
"proxy_url_env": "THORDATA_PROXY",
|
||||
},
|
||||
],
|
||||
}
|
||||
),
|
||||
@@ -153,7 +172,12 @@ def test_slots_support_multiple_accounts_and_networks(tmp_path: Path) -> None:
|
||||
"gemini-research",
|
||||
"openai-api-1",
|
||||
"openai-west",
|
||||
"synthid-proxy",
|
||||
}
|
||||
assert slots["synthid-proxy"]["browser_profile"] is None
|
||||
assert slots["synthid-proxy"]["account_label"] is None
|
||||
assert slots["synthid-proxy"]["proxy_url_env"] == "THORDATA_PROXY"
|
||||
assert slots["synthid-proxy"]["google_account_index"] is None
|
||||
assert slots["gemini-research"]["account_label"] == "research-b"
|
||||
assert slots["gemini-research"]["google_account_index"] == 2
|
||||
assert slots["openai-west"]["network_label"] == "west-egress"
|
||||
@@ -192,6 +216,33 @@ def test_named_slot_uses_repository_local_config_by_default(
|
||||
assert slot["api_key_env"] == "OPENAI_API_KEY_1"
|
||||
|
||||
|
||||
def test_proxy_is_only_for_an_unauthenticated_oracle(tmp_path: Path) -> None:
|
||||
for surface in oracles.SURFACES.values():
|
||||
slot: dict[str, str | int | None] = {
|
||||
"name": "probe",
|
||||
"surface": surface.key,
|
||||
"account_label": "account" if surface.account_slots else None,
|
||||
"browser_profile": "Profile 1" if surface.automation == "real_browser" else None,
|
||||
"network_label": "route",
|
||||
"proxy_url_env": "THORDATA_ROUTE_US",
|
||||
}
|
||||
if surface.key == "gemini-web":
|
||||
slot["google_account_index"] = 0
|
||||
if surface.key == "openai-api":
|
||||
slot["api_key_env"] = "OPENAI_API_KEY_1"
|
||||
config = tmp_path / f"{surface.key}.json"
|
||||
config.write_text(json.dumps({"format_version": 1, "slots": [slot]}), encoding="utf-8")
|
||||
|
||||
if surface.account_slots:
|
||||
with pytest.raises(ValueError, match="does not use proxy_url_env"):
|
||||
oracles.load_slots(config)
|
||||
continue
|
||||
|
||||
loaded = oracles.load_slots(config)
|
||||
assert loaded["probe"]["proxy_url_env"] == "THORDATA_ROUTE_US"
|
||||
assert loaded["probe"]["account_label"] is None
|
||||
|
||||
|
||||
def test_slots_override_requires_a_named_slot(tmp_path: Path) -> None:
|
||||
with pytest.raises(ValueError, match="--slots requires --slot"):
|
||||
oracles.resolve_slot("openai-api", slots_path=tmp_path / "slots.json", slot_name=None)
|
||||
@@ -278,25 +329,35 @@ def test_slots_override_requires_a_named_slot(tmp_path: Path) -> None:
|
||||
),
|
||||
(
|
||||
{
|
||||
"name": "synthid-no-profile",
|
||||
"name": "synthid-with-profile",
|
||||
"surface": "synthid-web",
|
||||
"account_label": "personal-a",
|
||||
"browser_profile": None,
|
||||
"account_label": None,
|
||||
"browser_profile": "Profile 2",
|
||||
"network_label": "home",
|
||||
},
|
||||
"requires a browser_profile",
|
||||
"only a real-browser surface uses a browser_profile",
|
||||
),
|
||||
(
|
||||
{
|
||||
"name": "synthid-with-account-index",
|
||||
"surface": "synthid-web",
|
||||
"account_label": "personal-a",
|
||||
"browser_profile": "Profile 2",
|
||||
"account_label": None,
|
||||
"browser_profile": None,
|
||||
"google_account_index": 0,
|
||||
"network_label": "home",
|
||||
},
|
||||
"only gemini-web uses google_account_index",
|
||||
),
|
||||
(
|
||||
{
|
||||
"name": "synthid-with-account",
|
||||
"surface": "synthid-web",
|
||||
"account_label": "personal-a",
|
||||
"browser_profile": None,
|
||||
"network_label": "home",
|
||||
},
|
||||
"does not use account slots",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_slots_require_the_identity_needed_by_the_surface(
|
||||
|
||||
@@ -117,7 +117,7 @@ class TestIsobmffStripFailSafe:
|
||||
|
||||
cleaned, stripped = isobmff.strip_c2pa_boxes(data)
|
||||
assert stripped == 1
|
||||
assert len(cleaned) == len(data) - len(c2pa_box)
|
||||
assert len(cleaned) == len(data)
|
||||
assert isobmff.C2PA_UUID not in cleaned
|
||||
|
||||
def test_malformed_box_does_not_truncate_tail(self):
|
||||
|
||||
Reference in new issue
Block a user