mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-22 00:31:15 +02:00
Add SynthID Detector surface, drop extra web confirmation
This commit is contained in:
9 files changed
+195
-58
No files matched your search
@@ -29,9 +29,9 @@ The unified maintainer entry point is `scripts/provider_oracles.py`; its workflo
|
||||
slot schema live in `docs/provider-oracles.md`. Multi-account and multi-network work
|
||||
selects each named slot explicitly from the gitignored repository-local
|
||||
`.oracle-slots.json` by default and snapshots it into a separate immutable batch
|
||||
manifest. A bounded provider check may continue through its finite set of authorized
|
||||
account slots after a refusal; do not add new logins, IP rotation, network-route
|
||||
failover, unbounded retries, or cross-session result merging. Never commit the local
|
||||
manifest. A provider check may use the finite set of configured account slots and surfaces
|
||||
after a refusal. Do not add new logins, IP rotation, network-route failover,
|
||||
unbounded retries, or cross-session result merging. Never commit the local
|
||||
slot file.
|
||||
Every manual result stays bound to the exact sanitized upload hash and preserves the
|
||||
provider response verbatim.
|
||||
@@ -39,11 +39,14 @@ provider response verbatim.
|
||||
Provider surface selection is API-first. OpenAI and Microsoft use their official API
|
||||
adapters before their Web surfaces when the requested media and local configuration are
|
||||
supported. Google and Meta currently use Web because this tooling has no usable API route
|
||||
for them. Web remains an explicit second surface, never an automatic retry after an API
|
||||
refusal, quota response, transport error, or indeterminate verdict.
|
||||
for them. A request to use a provider covers every configured surface, including Web,
|
||||
without a second confirmation. An API refusal, quota response, transport error, or
|
||||
indeterminate verdict stays on that surface and never silently submits the file to Web.
|
||||
|
||||
Google Web oracle automation uses only the user's existing authenticated real Chrome
|
||||
session and explicit `/u/N/` account index. OpenAI, Microsoft, and Meta Web automation
|
||||
session. Gemini uses an explicit `/u/N/` account index. The SynthID Detector at
|
||||
`https://synthid.com/` (`synthid-web`) uses the same Chrome profile and does not take a
|
||||
`/u/N/` index; its terms forbid driving the page as an automated API. OpenAI, Microsoft, and Meta Web automation
|
||||
uses isolated Playwright contexts; it must never attach to or copy state from the user's
|
||||
real browser. Proxy credentials are read only through a slot's named environment variable
|
||||
and never enter a manifest or log. Dotenv interpolation may derive routed proxy URLs from
|
||||
|
||||
@@ -31,22 +31,24 @@ request permission for each account or file already covered by the task.
|
||||
Submit each frozen artifact at most once per slot in that pass. Preserve
|
||||
inconclusive results and disagreements between accounts. Stop when the bounded
|
||||
plan is complete or no authorized usable slot remains; report remaining rows.
|
||||
New logins, another provider, a new network route, or another pass after the plan
|
||||
is exhausted require authorization unless already requested. Account
|
||||
New logins, another provider, or a new network route require authorization.
|
||||
The oracles and slots already configured for the requested provider may be used
|
||||
again, including their Web surfaces, without a second confirmation. Account
|
||||
continuation does not authorize changing candidate pixels or erasing failures.
|
||||
|
||||
## Default surface priority
|
||||
|
||||
Prefer an official API adapter whenever that provider, media type, and configured
|
||||
slot support it. Use Web only when no usable API route exists or the operator
|
||||
explicitly requests the Web surface as a second check. For OpenAI and Microsoft,
|
||||
the default plans are API then Web. Google and Meta currently have only usable
|
||||
Web routes in this tooling. Inspect the order with `plan PROVIDER`.
|
||||
slot support it. For OpenAI and Microsoft, the default plans are API then Web.
|
||||
Google and Meta currently have only usable Web routes in this tooling. Google's
|
||||
routes are `gemini-web` and `synthid-web`. Inspect the order with `plan PROVIDER`.
|
||||
The same request may use every configured surface.
|
||||
|
||||
The priority is a selection policy, not automatic failover. An API refusal,
|
||||
quota response, transport error, or indeterminate verdict must be recorded and
|
||||
reported. Do not submit the same artifact to Web unless the operator separately
|
||||
authorizes that next surface.
|
||||
reported. The same request already covers every configured surface of that
|
||||
provider, including Web. Run Web as its own acknowledged check and keep its
|
||||
verdict separate. Do not replace the API result with the Web result.
|
||||
|
||||
## Route by surface
|
||||
|
||||
@@ -86,6 +88,27 @@ record that it is unavailable and continue with another authorized existing
|
||||
session. Ask the user only when no usable authorized session remains. Do not
|
||||
start a new login.
|
||||
|
||||
### SynthID Detector
|
||||
|
||||
Use the user's real authenticated Chrome. Do not use isolated Playwright: the
|
||||
portal's additional terms forbid using the page as an automated API, bulk
|
||||
verification, and quota evasion.
|
||||
|
||||
1. Validate a `synthid-web` slot. It names `browser_profile` and `account_label`.
|
||||
It does not use `google_account_index` or a proxy variable.
|
||||
2. Prepare an immutable batch outside the repository. Inspect images the same
|
||||
way as for Gemini, and do not submit a file that still shows a provider logo.
|
||||
3. Open `https://synthid.com/` in the named Chrome profile. Accept the terms
|
||||
dialog when it is shown. Do not start a new login. If the page requires
|
||||
sign-in and that profile has no session, leave the row unsubmitted.
|
||||
4. Attach one prepared upload through the page's own file control.
|
||||
5. Read the settled label. `Watermark detected` is `detected`. `Watermark not
|
||||
detected` is `not_detected`. Quota, sign-in, and unclear wording stay
|
||||
`refused` or `indeterminate`.
|
||||
6. Record the verbatim response. The result covers partner SynthID from Google
|
||||
DeepMind, Nvidia, OpenAI, and Kakao, and it does not name which partner. Do
|
||||
not record it as a Gemini-only or OpenAI-only verdict.
|
||||
|
||||
### OpenAI, Microsoft, and Meta Web
|
||||
|
||||
Use the isolated Playwright runner, never the user's real Chrome:
|
||||
|
||||
+64
-25
@@ -14,6 +14,7 @@ through the installed CLI or public Python API.
|
||||
| `openai-api` | OpenAI SynthID | Official Content Provenance API | Images |
|
||||
| `microsoft-api` | Microsoft InvisMark | Azure Content Provenance Detection API | Images |
|
||||
| `gemini-web` | Google SynthID | Signed-in Gemini web account | Images and videos |
|
||||
| `synthid-web` | Partner SynthID | SynthID Detector at synthid.com, real Chrome | None recorded |
|
||||
| `meta-web` | Meta Content Seal | Anonymous Meta identification page | Images |
|
||||
| `microsoft-web` | Microsoft InvisMark | Public Azure AI validation page | Images |
|
||||
| `openai-web` | OpenAI SynthID | Public web verifier | Images |
|
||||
@@ -23,26 +24,46 @@ through the installed CLI or public Python API.
|
||||
The default selection order is API first, then Web. OpenAI therefore plans
|
||||
`openai-api` before `openai-web`, and Microsoft plans `microsoft-api` before
|
||||
`microsoft-web`. Google and Meta currently have no usable API adapter in this
|
||||
tooling, so their Web surfaces are primary rather than fallback routes.
|
||||
tooling, so each of their Web surfaces is primary. Google has two: `gemini-web`
|
||||
and `synthid-web`.
|
||||
|
||||
Inspect the machine-readable order before preparing a batch:
|
||||
|
||||
```bash
|
||||
uv run python scripts/provider_oracles.py plan openai --json
|
||||
uv run python scripts/provider_oracles.py plan microsoft
|
||||
uv run python scripts/provider_oracles.py plan google
|
||||
```
|
||||
|
||||
API-first is a selection policy, not automatic failover. Use Web only when the
|
||||
provider, requested media, or local configuration has no usable API route, or
|
||||
when the operator explicitly requests a second surface. An API refusal, quota
|
||||
response, transport failure, or indeterminate verdict is recorded as-is and
|
||||
never silently triggers a Web upload.
|
||||
A request to use a provider covers every configured surface of that provider,
|
||||
including Web, without a second confirmation. API-first is still the order to
|
||||
try, not a lock on the other surfaces. An API refusal, quota response, transport
|
||||
failure, or indeterminate verdict is recorded as-is. It does not silently
|
||||
upload the same file to Web; run that surface as its own acknowledged check and
|
||||
keep the two verdicts separate.
|
||||
|
||||
The old Google SynthID Detector portal does not work. On 2026-09-07, the
|
||||
official `labs.google/synthid` entry point redirected a signed-in browser to a
|
||||
Google 404; on 2026-09-23 it redirected to
|
||||
`deepmind.google.com/frontiers/synthid`, which also returned 404. Historical corpus rows may still name `synthid-portal`; that value
|
||||
describes how an old result was obtained and is not an operational surface.
|
||||
The old Google SynthID Detector portal at `labs.google/synthid` does not work.
|
||||
On 2026-09-07 that entry point redirected a signed-in browser to a Google 404;
|
||||
on 2026-09-23 it redirected to `deepmind.google.com/frontiers/synthid`, which
|
||||
also returned 404. Historical corpus rows may still name `synthid-portal`; that
|
||||
value describes how an old result was obtained and is not an operational surface.
|
||||
|
||||
The public SynthID Detector is a different site, [synthid.com](https://synthid.com/),
|
||||
and it is the `synthid-web` surface. On 2026-10-17 the page title was
|
||||
`SynthID Detector`. After the terms dialog, the uploader was present without a
|
||||
completed sign-in in that session. The Account menu still offered Sign in, and
|
||||
no file was submitted, so this check does not show whether a result requires an
|
||||
account. The live FAQ named the partners as Google DeepMind, Nvidia, OpenAI,
|
||||
and Kakao, and it did not list Apple. It defined two settled labels:
|
||||
`Watermark detected` and `Watermark not detected`. The file input accepted
|
||||
`.jpg`, `.jpeg`, `.png`, `.bmp`, `.webp`, `.avif`, `.heic`, `.heif`, `.tiff`,
|
||||
`.tif`, `.gif`, `.wav`, `.mp3`, `.ogg`, `.flac`, `.aac`, `.m4a`, `.mp4`,
|
||||
`.mov`, and `.webm`. This tooling still prepares only the project's existing
|
||||
JPEG, PNG, WebP, MP4, MP3, and WAV suffixes. The portal's additional terms
|
||||
forbid using the page as an automated API, bulk verification, and quota evasion,
|
||||
so `run-web` does not drive it. Use the real Chrome workflow below. A
|
||||
`Watermark detected` result is partner SynthID, not a Gemini-only or OpenAI-only
|
||||
verdict.
|
||||
Google's newer Search verification entry point did not provide an independent
|
||||
oracle allowance in the session tested on 2026-09-13. Lens accepted the
|
||||
metadata-stripped, previously Gemini-positive image
|
||||
@@ -161,9 +182,12 @@ Adobe Inspect and local C2PA, DWT-DCT, TrustMark, and research detectors are
|
||||
useful evidence readers, not interchangeable provider pixel-watermark oracles.
|
||||
|
||||
The surface capability and this project's evidence coverage are separate. The
|
||||
current Google, Microsoft, and Meta pages accept image, video, and audio inputs;
|
||||
the OpenAI API accepts image and audio. A media type does not become certified
|
||||
merely because a surface accepts it. Run `list --json` to read both fields.
|
||||
current Google, Microsoft, and Meta pages, including `https://synthid.com/`,
|
||||
accept image, video, and audio inputs; the OpenAI API accepts image and audio.
|
||||
A media type does not become certified merely because a surface accepts it.
|
||||
`synthid-web` therefore lists image, video, and audio as accepted media and
|
||||
records no project evidence until a batch is verified. Run `list --json` to read
|
||||
both fields.
|
||||
|
||||
## Execution slots
|
||||
|
||||
@@ -193,6 +217,13 @@ default; `--slots PATH` explicitly overrides it.
|
||||
"google_account_index": 2,
|
||||
"network_label": "office"
|
||||
},
|
||||
{
|
||||
"name": "synthid-personal",
|
||||
"surface": "synthid-web",
|
||||
"account_label": "google-account-a",
|
||||
"browser_profile": "Your Chrome",
|
||||
"network_label": "home"
|
||||
},
|
||||
{
|
||||
"name": "openai-web-thordata-us",
|
||||
"surface": "openai-web",
|
||||
@@ -261,7 +292,9 @@ default; `--slots PATH` explicitly overrides it.
|
||||
```
|
||||
|
||||
A routed batch selects one slot explicitly. A Gemini slot must name the Google
|
||||
account, its real Chrome profile, and its nonnegative `/u/N/` account index; an
|
||||
account, its real Chrome profile, and its nonnegative `/u/N/` account index. A
|
||||
SynthID Detector slot names the account and the same real Chrome profile, and it
|
||||
does not use `google_account_index`. An
|
||||
API slot must name its account or project and cannot name a browser profile. The
|
||||
tool records the network route that the operator has already established.
|
||||
Non-Google Web slots may name `proxy_url_env`; Playwright reads the secret proxy
|
||||
@@ -279,9 +312,10 @@ again for files or accounts already covered by the task.
|
||||
Submit each frozen artifact at most once per slot in that pass. Keep every
|
||||
verbatim refusal, inconclusive reply, and disagreement between accounts. Stop
|
||||
when the planned checks finish or no authorized usable slot remains, and report
|
||||
unsubmitted rows. A new login, another provider, a new network route, or a new
|
||||
pass after the bounded plan is exhausted needs authorization unless already
|
||||
requested. Candidate pixels remain frozen throughout account continuation.
|
||||
unsubmitted rows. A new login, another provider, or a new network route needs
|
||||
authorization. The oracles and slots already configured for the requested
|
||||
provider may be used again, including their Web surfaces, without a second
|
||||
confirmation. Candidate pixels remain frozen throughout account continuation.
|
||||
|
||||
Each `openai-api` slot must name exactly one `api_key_env`. Put the corresponding
|
||||
values in the gitignored local `.env`, never in the slot JSON:
|
||||
@@ -385,18 +419,23 @@ uv sync --extra dev
|
||||
uv run playwright install chromium
|
||||
```
|
||||
|
||||
Google is intentionally excluded from the standalone runner because its oracle
|
||||
requires the user's existing authenticated Chrome session. Invoke the project
|
||||
`provider-oracles` skill to prepare the batch, select the named real Chrome
|
||||
profile and `/u/N/` Google account, upload each prepared artifact, capture the
|
||||
settled SynthID tool outcome, and record it.
|
||||
Google is intentionally excluded from the standalone runner. Both `gemini-web`
|
||||
and `synthid-web` use the user's existing real Chrome profile, and the SynthID
|
||||
Detector terms forbid driving that page as an automated API. Invoke the project
|
||||
`provider-oracles` skill to prepare the batch and record the settled outcome.
|
||||
For Gemini, select the named profile and `/u/N/` account. For SynthID Detector,
|
||||
open `https://synthid.com/` in that profile. The portal has no `/u/N/` index.
|
||||
Accept the terms dialog when it is shown, attach one prepared upload, and read
|
||||
the explicit `Watermark detected` or `Watermark not detected` label. If the page
|
||||
requires sign-in and that profile has no session, record the row unsubmitted
|
||||
and do not start a new login.
|
||||
|
||||
Ask the direct question stored in the batch manifest: `Does this
|
||||
For Gemini, ask the direct question stored in the batch manifest: `Does this
|
||||
image/video/audio contain a SynthID watermark? Use the SynthID verification
|
||||
tool only; do not infer from visible content, logos, text, or metadata.` A
|
||||
general question about whether Google AI created or edited the media is not
|
||||
equivalent: it may elicit origin or visual-content reasoning without a SynthID
|
||||
verdict.
|
||||
verdict. SynthID Detector has no prompt; its settled labels are the verdict.
|
||||
|
||||
OpenAI Web, Microsoft Web, and Meta run entirely through a fresh isolated
|
||||
Playwright browser. A direct-route example:
|
||||
|
||||
+5
-1
@@ -575,7 +575,11 @@ The SynthID decoder is proprietary and not released:
|
||||
> -- Gowal et al., arXiv:2510.09263
|
||||
|
||||
There are no released payload-decoder weights or public algorithm. Google
|
||||
provides verification in Gemini. The old limited SynthID Detector portal does
|
||||
provides verification in Gemini and, since the public SynthID Detector opened,
|
||||
at [synthid.com](https://synthid.com/). That site is the `synthid-web` oracle
|
||||
surface in [`provider-oracles.md`](provider-oracles.md). On 2026-10-17 its FAQ
|
||||
named Google DeepMind, Nvidia, OpenAI, and Kakao, and its settled labels were
|
||||
`Watermark detected` and `Watermark not detected`. The old limited portal does
|
||||
not work: on 2026-09-07, its official `labs.google/synthid` entry point
|
||||
redirected a signed-in browser to a Google 404. On 2026-09-23 the same entry
|
||||
point redirected to `deepmind.google.com/frontiers/synthid`, which also returned
|
||||
|
||||
@@ -262,8 +262,8 @@ and implemented by `scripts/provider_oracles.py`.
|
||||
|
||||
Proprietary watermark removal cannot be verified locally by design -- no public decoder
|
||||
exists. Each vendor has its own oracle and it covers only that vendor's content:
|
||||
the Content Provenance API first and `openai.com/verify` as an explicit fallback for
|
||||
OpenAI, the Gemini app for Google, and Microsoft's
|
||||
the Content Provenance API first and `openai.com/verify` for
|
||||
OpenAI, the Gemini app and the SynthID Detector at `synthid.com` for Google, and Microsoft's
|
||||
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
|
||||
for InvisMark. The Microsoft API reports `Watermark` and `C2PA` separately. It therefore
|
||||
needs a pixel-identical metadata-stripped control: the control must lose `C2PA` while
|
||||
|
||||
@@ -288,8 +288,8 @@ def run_web_batch(
|
||||
oracles.verify_batch(manifest_path)
|
||||
manifest, results = oracles.load_batch(manifest_path)
|
||||
surface = manifest["surface"]
|
||||
if surface == "gemini-web":
|
||||
raise ValueError("gemini-web requires the real Chrome workflow, not isolated Playwright")
|
||||
if oracles.SURFACES[surface].automation == "real_browser":
|
||||
raise ValueError(f"{surface} requires the real Chrome workflow, not isolated Playwright")
|
||||
if surface not in PLAYWRIGHT_SURFACES:
|
||||
raise ValueError(f"surface {surface} is not a Playwright Web oracle")
|
||||
slot = manifest["slot"]
|
||||
|
||||
+35
-11
@@ -169,6 +169,26 @@ SURFACES: dict[str, OracleSurface] = {
|
||||
account_slots=True,
|
||||
network_slots=True,
|
||||
),
|
||||
"synthid-web": OracleSurface(
|
||||
key="synthid-web",
|
||||
provider_key="google",
|
||||
provider="Google",
|
||||
signal_family="synthid",
|
||||
surface="SynthID Detector",
|
||||
automation="real_browser",
|
||||
url="https://synthid.com/",
|
||||
media_types=("image", "video", "audio"),
|
||||
project_evidence=(),
|
||||
prompt=None,
|
||||
clean_result_note=(
|
||||
"Record detected only for the explicit label Watermark detected, and not_detected only for "
|
||||
"Watermark not detected. The portal reports partner SynthID from Google DeepMind, Nvidia, "
|
||||
"OpenAI, and Kakao without naming which partner. A not_detected result is not proof of "
|
||||
"human authorship. Preserve quota, sign-in, and unclear wording."
|
||||
),
|
||||
account_slots=True,
|
||||
network_slots=True,
|
||||
),
|
||||
"meta-web": OracleSurface(
|
||||
key="meta-web",
|
||||
provider_key="meta",
|
||||
@@ -278,6 +298,11 @@ def ordered_surfaces() -> list[OracleSurface]:
|
||||
return sorted(SURFACES.values(), key=_surface_order)
|
||||
|
||||
|
||||
def _automation_keys(automation: str) -> frozenset[str]:
|
||||
"""Return the surface keys that use one automation kind."""
|
||||
return frozenset(key for key, surface in SURFACES.items() if surface.automation == automation)
|
||||
|
||||
|
||||
def _inside(path: Path, parent: Path) -> bool:
|
||||
"""Return whether resolved PATH is inside resolved PARENT."""
|
||||
try:
|
||||
@@ -364,6 +389,8 @@ def load_slots(path: Path) -> dict[str, ExecutionSlot]:
|
||||
"api_key_env",
|
||||
}
|
||||
slots: dict[str, ExecutionSlot] = {}
|
||||
real_browser = _automation_keys("real_browser")
|
||||
playwright = _automation_keys("playwright")
|
||||
for index, raw_slot in enumerate(slot_items):
|
||||
if not isinstance(raw_slot, dict):
|
||||
raise ValueError(f"slot {index} must be a JSON object")
|
||||
@@ -407,19 +434,17 @@ def load_slots(path: Path) -> dict[str, ExecutionSlot]:
|
||||
raise ValueError(f"surface {surface_key} requires an account_label")
|
||||
if account_label is not None and not SURFACES[surface_key].account_slots:
|
||||
raise ValueError(f"surface {surface_key} does not use account slots")
|
||||
if surface_key == "gemini-web" and browser_profile is None:
|
||||
raise ValueError("surface gemini-web requires a browser_profile")
|
||||
if surface_key in real_browser and browser_profile is None:
|
||||
raise ValueError(f"surface {surface_key} requires a browser_profile")
|
||||
if surface_key == "gemini-web" and google_account_index is None:
|
||||
raise ValueError("surface gemini-web requires a google_account_index")
|
||||
if surface_key != "gemini-web" and google_account_index is not None:
|
||||
raise ValueError("only gemini-web uses google_account_index")
|
||||
if SURFACES[surface_key].automation == "api" and browser_profile is not None:
|
||||
raise ValueError(f"API surface {surface_key} does not use a browser_profile")
|
||||
if surface_key != "gemini-web" and browser_profile is not None:
|
||||
raise ValueError("only gemini-web uses a real browser_profile")
|
||||
if surface_key == "gemini-web" and proxy_url_env is not None:
|
||||
raise ValueError("surface gemini-web does not use proxy_url_env")
|
||||
if proxy_url_env is not None and surface_key not in {"openai-web", "microsoft-web", "meta-web"}:
|
||||
if surface_key not in real_browser and browser_profile is not None:
|
||||
raise ValueError("only a real-browser surface uses a browser_profile")
|
||||
if proxy_url_env is not None and surface_key not in playwright:
|
||||
raise ValueError(f"surface {surface_key} does not use proxy_url_env")
|
||||
if surface_key == "openai-api" and api_key_env is None:
|
||||
raise ValueError("surface openai-api requires an api_key_env")
|
||||
@@ -858,18 +883,17 @@ def list_oracles(as_json: bool) -> None:
|
||||
def plan(provider: str, as_json: bool) -> None:
|
||||
"""Show the API-first surface order for one provider."""
|
||||
definitions = provider_plan(provider)
|
||||
has_api = any(definition.automation == "api" for definition in definitions)
|
||||
rows = []
|
||||
for index, definition in enumerate(definitions):
|
||||
for definition in definitions:
|
||||
row = definition.to_dict()
|
||||
row["preference"] = "primary" if index == 0 else "fallback"
|
||||
row["preference"] = "fallback" if has_api and definition.automation != "api" else "primary"
|
||||
rows.append(row)
|
||||
if as_json:
|
||||
click.echo(json.dumps(rows, indent=2, sort_keys=True))
|
||||
return
|
||||
for row in rows:
|
||||
click.echo(f"{row['preference']}\t{row['key']}\t{row['automation']}\t{row['surface']}")
|
||||
if len(rows) > 1:
|
||||
click.echo("Web fallback requires a separate operator decision; failures do not trigger it automatically.")
|
||||
|
||||
|
||||
@cli.command()
|
||||
|
||||
@@ -392,12 +392,14 @@ def test_run_web_batch_requires_explicit_upload_acknowledgement(
|
||||
web.run_web_batch(manifest_path, acknowledge_uploads=False)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("surface", ["gemini-web", "synthid-web"])
|
||||
def test_run_web_batch_refuses_google_real_browser_workflow(
|
||||
tmp_path: Path,
|
||||
tmp_clean_png: Path,
|
||||
surface: str,
|
||||
) -> None:
|
||||
manifest_path = oracles.prepare_batch(
|
||||
"gemini-web",
|
||||
surface,
|
||||
[tmp_clean_png],
|
||||
output_dir=tmp_path / "oracle-batch",
|
||||
repository_root=oracles.REPOSITORY_ROOT,
|
||||
|
||||
@@ -23,6 +23,7 @@ def test_catalog_covers_every_project_vendor_oracle_surface() -> None:
|
||||
"microsoft-web",
|
||||
"openai-api",
|
||||
"openai-web",
|
||||
"synthid-web",
|
||||
}
|
||||
assert {"google", "meta", "microsoft", "openai"} == oracles.PROVIDER_KEYS
|
||||
assert oracles.SURFACES["openai-api"].automation == "api"
|
||||
@@ -34,6 +35,14 @@ def test_catalog_covers_every_project_vendor_oracle_surface() -> None:
|
||||
"Does this image/video/audio contain a SynthID watermark? Use the SynthID verification tool only; "
|
||||
"do not infer from visible content, logos, text, or metadata."
|
||||
)
|
||||
synthid = oracles.SURFACES["synthid-web"]
|
||||
assert synthid.automation == "real_browser"
|
||||
assert synthid.provider_key == "google"
|
||||
assert synthid.url == "https://synthid.com/"
|
||||
assert synthid.media_types == ("image", "video", "audio")
|
||||
assert synthid.project_evidence == ()
|
||||
assert synthid.prompt is None
|
||||
assert "Google DeepMind, Nvidia, OpenAI, and Kakao" in synthid.clean_result_note
|
||||
|
||||
|
||||
def test_default_slot_config_is_repository_local() -> None:
|
||||
@@ -52,6 +61,7 @@ def test_list_json_exposes_scope_and_surface() -> None:
|
||||
"meta-web",
|
||||
"microsoft-web",
|
||||
"openai-web",
|
||||
"synthid-web",
|
||||
]
|
||||
assert next(row for row in payload if row["key"] == "meta-web")["signal_family"] == "content_seal"
|
||||
|
||||
@@ -59,7 +69,7 @@ def test_list_json_exposes_scope_and_surface() -> None:
|
||||
@pytest.mark.parametrize(
|
||||
("provider", "expected"),
|
||||
[
|
||||
("google", ["gemini-web"]),
|
||||
("google", ["gemini-web", "synthid-web"]),
|
||||
("meta", ["meta-web"]),
|
||||
("microsoft", ["microsoft-api", "microsoft-web"]),
|
||||
("openai", ["openai-api", "openai-web"]),
|
||||
@@ -69,6 +79,17 @@ def test_provider_plan_prefers_api_before_web(provider: str, expected: list[str]
|
||||
assert [surface.key for surface in oracles.provider_plan(provider)] == expected
|
||||
|
||||
|
||||
def test_plan_json_keeps_every_google_web_surface_primary() -> None:
|
||||
result = CliRunner().invoke(oracles.cli, ["plan", "google", "--json"])
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
payload = json.loads(result.output)
|
||||
assert [(row["key"], row["preference"]) for row in payload] == [
|
||||
("gemini-web", "primary"),
|
||||
("synthid-web", "primary"),
|
||||
]
|
||||
|
||||
|
||||
def test_plan_json_marks_web_as_an_explicit_fallback_when_api_exists() -> None:
|
||||
result = CliRunner().invoke(oracles.cli, ["plan", "openai", "--json"])
|
||||
|
||||
@@ -255,11 +276,32 @@ def test_slots_override_requires_a_named_slot(tmp_path: Path) -> None:
|
||||
},
|
||||
"only openai-api uses api_key_env",
|
||||
),
|
||||
(
|
||||
{
|
||||
"name": "synthid-no-profile",
|
||||
"surface": "synthid-web",
|
||||
"account_label": "personal-a",
|
||||
"browser_profile": None,
|
||||
"network_label": "home",
|
||||
},
|
||||
"requires a browser_profile",
|
||||
),
|
||||
(
|
||||
{
|
||||
"name": "synthid-with-account-index",
|
||||
"surface": "synthid-web",
|
||||
"account_label": "personal-a",
|
||||
"browser_profile": "Profile 2",
|
||||
"google_account_index": 0,
|
||||
"network_label": "home",
|
||||
},
|
||||
"only gemini-web uses google_account_index",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_slots_require_the_identity_needed_by_the_surface(
|
||||
tmp_path: Path,
|
||||
slot: dict[str, str | None],
|
||||
slot: dict[str, str | int | None],
|
||||
message: str,
|
||||
) -> None:
|
||||
config = tmp_path / "slots.json"
|
||||
|
||||
Reference in new issue
Block a user