Add SynthID Detector surface, drop extra web confirmation

This commit is contained in:
Victor Kuznetsov committed 2026-10-17 11:47:53 -07:00
1 parent f1d4d4afba
commit 21f4621eb7
9 files changed
+195 -58

No files matched your search

+9 -6
View File
@@ -29,9 +29,9 @@ The unified maintainer entry point is `scripts/provider_oracles.py`; its workflo
slot schema live in `docs/provider-oracles.md`. Multi-account and multi-network work
selects each named slot explicitly from the gitignored repository-local
`.oracle-slots.json` by default and snapshots it into a separate immutable batch
manifest. A bounded provider check may continue through its finite set of authorized
account slots after a refusal; do not add new logins, IP rotation, network-route
failover, unbounded retries, or cross-session result merging. Never commit the local
manifest. A provider check may use the finite set of configured account slots and surfaces
after a refusal. Do not add new logins, IP rotation, network-route failover,
unbounded retries, or cross-session result merging. Never commit the local
slot file.
Every manual result stays bound to the exact sanitized upload hash and preserves the
provider response verbatim.
@@ -39,11 +39,14 @@ provider response verbatim.
Provider surface selection is API-first. OpenAI and Microsoft use their official API
adapters before their Web surfaces when the requested media and local configuration are
supported. Google and Meta currently use Web because this tooling has no usable API route
for them. Web remains an explicit second surface, never an automatic retry after an API
refusal, quota response, transport error, or indeterminate verdict.
for them. A request to use a provider covers every configured surface, including Web,
without a second confirmation. An API refusal, quota response, transport error, or
indeterminate verdict stays on that surface and never silently submits the file to Web.
Google Web oracle automation uses only the user's existing authenticated real Chrome
session and explicit `/u/N/` account index. OpenAI, Microsoft, and Meta Web automation
session. Gemini uses an explicit `/u/N/` account index. The SynthID Detector at
`https://synthid.com/` (`synthid-web`) uses the same Chrome profile and does not take a
`/u/N/` index; its terms forbid driving the page as an automated API. OpenAI, Microsoft, and Meta Web automation
uses isolated Playwright contexts; it must never attach to or copy state from the user's
real browser. Proxy credentials are read only through a slot's named environment variable
and never enter a manifest or log. Dotenv interpolation may derive routed proxy URLs from
+31 -8
View File
@@ -31,22 +31,24 @@ request permission for each account or file already covered by the task.
Submit each frozen artifact at most once per slot in that pass. Preserve
inconclusive results and disagreements between accounts. Stop when the bounded
plan is complete or no authorized usable slot remains; report remaining rows.
New logins, another provider, a new network route, or another pass after the plan
is exhausted require authorization unless already requested. Account
New logins, another provider, or a new network route require authorization.
The oracles and slots already configured for the requested provider may be used
again, including their Web surfaces, without a second confirmation. Account
continuation does not authorize changing candidate pixels or erasing failures.
## Default surface priority
Prefer an official API adapter whenever that provider, media type, and configured
slot support it. Use Web only when no usable API route exists or the operator
explicitly requests the Web surface as a second check. For OpenAI and Microsoft,
the default plans are API then Web. Google and Meta currently have only usable
Web routes in this tooling. Inspect the order with `plan PROVIDER`.
slot support it. For OpenAI and Microsoft, the default plans are API then Web.
Google and Meta currently have only usable Web routes in this tooling. Google's
routes are `gemini-web` and `synthid-web`. Inspect the order with `plan PROVIDER`.
The same request may use every configured surface.
The priority is a selection policy, not automatic failover. An API refusal,
quota response, transport error, or indeterminate verdict must be recorded and
reported. Do not submit the same artifact to Web unless the operator separately
authorizes that next surface.
reported. The same request already covers every configured surface of that
provider, including Web. Run Web as its own acknowledged check and keep its
verdict separate. Do not replace the API result with the Web result.
## Route by surface
@@ -86,6 +88,27 @@ record that it is unavailable and continue with another authorized existing
session. Ask the user only when no usable authorized session remains. Do not
start a new login.
### SynthID Detector
Use the user's real authenticated Chrome. Do not use isolated Playwright: the
portal's additional terms forbid using the page as an automated API, bulk
verification, and quota evasion.
1. Validate a `synthid-web` slot. It names `browser_profile` and `account_label`.
It does not use `google_account_index` or a proxy variable.
2. Prepare an immutable batch outside the repository. Inspect images the same
way as for Gemini, and do not submit a file that still shows a provider logo.
3. Open `https://synthid.com/` in the named Chrome profile. Accept the terms
dialog when it is shown. Do not start a new login. If the page requires
sign-in and that profile has no session, leave the row unsubmitted.
4. Attach one prepared upload through the page's own file control.
5. Read the settled label. `Watermark detected` is `detected`. `Watermark not
detected` is `not_detected`. Quota, sign-in, and unclear wording stay
`refused` or `indeterminate`.
6. Record the verbatim response. The result covers partner SynthID from Google
DeepMind, Nvidia, OpenAI, and Kakao, and it does not name which partner. Do
not record it as a Gemini-only or OpenAI-only verdict.
### OpenAI, Microsoft, and Meta Web
Use the isolated Playwright runner, never the user's real Chrome:
+64 -25
View File
@@ -14,6 +14,7 @@ through the installed CLI or public Python API.
| `openai-api` | OpenAI SynthID | Official Content Provenance API | Images |
| `microsoft-api` | Microsoft InvisMark | Azure Content Provenance Detection API | Images |
| `gemini-web` | Google SynthID | Signed-in Gemini web account | Images and videos |
| `synthid-web` | Partner SynthID | SynthID Detector at synthid.com, real Chrome | None recorded |
| `meta-web` | Meta Content Seal | Anonymous Meta identification page | Images |
| `microsoft-web` | Microsoft InvisMark | Public Azure AI validation page | Images |
| `openai-web` | OpenAI SynthID | Public web verifier | Images |
@@ -23,26 +24,46 @@ through the installed CLI or public Python API.
The default selection order is API first, then Web. OpenAI therefore plans
`openai-api` before `openai-web`, and Microsoft plans `microsoft-api` before
`microsoft-web`. Google and Meta currently have no usable API adapter in this
tooling, so their Web surfaces are primary rather than fallback routes.
tooling, so each of their Web surfaces is primary. Google has two: `gemini-web`
and `synthid-web`.
Inspect the machine-readable order before preparing a batch:
```bash
uv run python scripts/provider_oracles.py plan openai --json
uv run python scripts/provider_oracles.py plan microsoft
uv run python scripts/provider_oracles.py plan google
```
API-first is a selection policy, not automatic failover. Use Web only when the
provider, requested media, or local configuration has no usable API route, or
when the operator explicitly requests a second surface. An API refusal, quota
response, transport failure, or indeterminate verdict is recorded as-is and
never silently triggers a Web upload.
A request to use a provider covers every configured surface of that provider,
including Web, without a second confirmation. API-first is still the order to
try, not a lock on the other surfaces. An API refusal, quota response, transport
failure, or indeterminate verdict is recorded as-is. It does not silently
upload the same file to Web; run that surface as its own acknowledged check and
keep the two verdicts separate.
The old Google SynthID Detector portal does not work. On 2026-09-07, the
official `labs.google/synthid` entry point redirected a signed-in browser to a
Google 404; on 2026-09-23 it redirected to
`deepmind.google.com/frontiers/synthid`, which also returned 404. Historical corpus rows may still name `synthid-portal`; that value
describes how an old result was obtained and is not an operational surface.
The old Google SynthID Detector portal at `labs.google/synthid` does not work.
On 2026-09-07 that entry point redirected a signed-in browser to a Google 404;
on 2026-09-23 it redirected to `deepmind.google.com/frontiers/synthid`, which
also returned 404. Historical corpus rows may still name `synthid-portal`; that
value describes how an old result was obtained and is not an operational surface.
The public SynthID Detector is a different site, [synthid.com](https://synthid.com/),
and it is the `synthid-web` surface. On 2026-10-17 the page title was
`SynthID Detector`. After the terms dialog, the uploader was present without a
completed sign-in in that session. The Account menu still offered Sign in, and
no file was submitted, so this check does not show whether a result requires an
account. The live FAQ named the partners as Google DeepMind, Nvidia, OpenAI,
and Kakao, and it did not list Apple. It defined two settled labels:
`Watermark detected` and `Watermark not detected`. The file input accepted
`.jpg`, `.jpeg`, `.png`, `.bmp`, `.webp`, `.avif`, `.heic`, `.heif`, `.tiff`,
`.tif`, `.gif`, `.wav`, `.mp3`, `.ogg`, `.flac`, `.aac`, `.m4a`, `.mp4`,
`.mov`, and `.webm`. This tooling still prepares only the project's existing
JPEG, PNG, WebP, MP4, MP3, and WAV suffixes. The portal's additional terms
forbid using the page as an automated API, bulk verification, and quota evasion,
so `run-web` does not drive it. Use the real Chrome workflow below. A
`Watermark detected` result is partner SynthID, not a Gemini-only or OpenAI-only
verdict.
Google's newer Search verification entry point did not provide an independent
oracle allowance in the session tested on 2026-09-13. Lens accepted the
metadata-stripped, previously Gemini-positive image
@@ -161,9 +182,12 @@ Adobe Inspect and local C2PA, DWT-DCT, TrustMark, and research detectors are
useful evidence readers, not interchangeable provider pixel-watermark oracles.
The surface capability and this project's evidence coverage are separate. The
current Google, Microsoft, and Meta pages accept image, video, and audio inputs;
the OpenAI API accepts image and audio. A media type does not become certified
merely because a surface accepts it. Run `list --json` to read both fields.
current Google, Microsoft, and Meta pages, including `https://synthid.com/`,
accept image, video, and audio inputs; the OpenAI API accepts image and audio.
A media type does not become certified merely because a surface accepts it.
`synthid-web` therefore lists image, video, and audio as accepted media and
records no project evidence until a batch is verified. Run `list --json` to read
both fields.
## Execution slots
@@ -193,6 +217,13 @@ default; `--slots PATH` explicitly overrides it.
"google_account_index": 2,
"network_label": "office"
},
{
"name": "synthid-personal",
"surface": "synthid-web",
"account_label": "google-account-a",
"browser_profile": "Your Chrome",
"network_label": "home"
},
{
"name": "openai-web-thordata-us",
"surface": "openai-web",
@@ -261,7 +292,9 @@ default; `--slots PATH` explicitly overrides it.
```
A routed batch selects one slot explicitly. A Gemini slot must name the Google
account, its real Chrome profile, and its nonnegative `/u/N/` account index; an
account, its real Chrome profile, and its nonnegative `/u/N/` account index. A
SynthID Detector slot names the account and the same real Chrome profile, and it
does not use `google_account_index`. An
API slot must name its account or project and cannot name a browser profile. The
tool records the network route that the operator has already established.
Non-Google Web slots may name `proxy_url_env`; Playwright reads the secret proxy
@@ -279,9 +312,10 @@ again for files or accounts already covered by the task.
Submit each frozen artifact at most once per slot in that pass. Keep every
verbatim refusal, inconclusive reply, and disagreement between accounts. Stop
when the planned checks finish or no authorized usable slot remains, and report
unsubmitted rows. A new login, another provider, a new network route, or a new
pass after the bounded plan is exhausted needs authorization unless already
requested. Candidate pixels remain frozen throughout account continuation.
unsubmitted rows. A new login, another provider, or a new network route needs
authorization. The oracles and slots already configured for the requested
provider may be used again, including their Web surfaces, without a second
confirmation. Candidate pixels remain frozen throughout account continuation.
Each `openai-api` slot must name exactly one `api_key_env`. Put the corresponding
values in the gitignored local `.env`, never in the slot JSON:
@@ -385,18 +419,23 @@ uv sync --extra dev
uv run playwright install chromium
```
Google is intentionally excluded from the standalone runner because its oracle
requires the user's existing authenticated Chrome session. Invoke the project
`provider-oracles` skill to prepare the batch, select the named real Chrome
profile and `/u/N/` Google account, upload each prepared artifact, capture the
settled SynthID tool outcome, and record it.
Google is intentionally excluded from the standalone runner. Both `gemini-web`
and `synthid-web` use the user's existing real Chrome profile, and the SynthID
Detector terms forbid driving that page as an automated API. Invoke the project
`provider-oracles` skill to prepare the batch and record the settled outcome.
For Gemini, select the named profile and `/u/N/` account. For SynthID Detector,
open `https://synthid.com/` in that profile. The portal has no `/u/N/` index.
Accept the terms dialog when it is shown, attach one prepared upload, and read
the explicit `Watermark detected` or `Watermark not detected` label. If the page
requires sign-in and that profile has no session, record the row unsubmitted
and do not start a new login.
Ask the direct question stored in the batch manifest: `Does this
For Gemini, ask the direct question stored in the batch manifest: `Does this
image/video/audio contain a SynthID watermark? Use the SynthID verification
tool only; do not infer from visible content, logos, text, or metadata.` A
general question about whether Google AI created or edited the media is not
equivalent: it may elicit origin or visual-content reasoning without a SynthID
verdict.
verdict. SynthID Detector has no prompt; its settled labels are the verdict.
OpenAI Web, Microsoft Web, and Meta run entirely through a fresh isolated
Playwright browser. A direct-route example:
+5 -1
View File
@@ -575,7 +575,11 @@ The SynthID decoder is proprietary and not released:
> -- Gowal et al., arXiv:2510.09263
There are no released payload-decoder weights or public algorithm. Google
provides verification in Gemini. The old limited SynthID Detector portal does
provides verification in Gemini and, since the public SynthID Detector opened,
at [synthid.com](https://synthid.com/). That site is the `synthid-web` oracle
surface in [`provider-oracles.md`](provider-oracles.md). On 2026-10-17 its FAQ
named Google DeepMind, Nvidia, OpenAI, and Kakao, and its settled labels were
`Watermark detected` and `Watermark not detected`. The old limited portal does
not work: on 2026-09-07, its official `labs.google/synthid` entry point
redirected a signed-in browser to a Google 404. On 2026-09-23 the same entry
point redirected to `deepmind.google.com/frontiers/synthid`, which also returned
+2 -2
View File
@@ -262,8 +262,8 @@ and implemented by `scripts/provider_oracles.py`.
Proprietary watermark removal cannot be verified locally by design -- no public decoder
exists. Each vendor has its own oracle and it covers only that vendor's content:
the Content Provenance API first and `openai.com/verify` as an explicit fallback for
OpenAI, the Gemini app for Google, and Microsoft's
the Content Provenance API first and `openai.com/verify` for
OpenAI, the Gemini app and the SynthID Detector at `synthid.com` for Google, and Microsoft's
[Content Provenance Detection API](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/how-to-provenance-detection)
for InvisMark. The Microsoft API reports `Watermark` and `C2PA` separately. It therefore
needs a pixel-identical metadata-stripped control: the control must lose `C2PA` while
+2 -2
View File
@@ -288,8 +288,8 @@ def run_web_batch(
oracles.verify_batch(manifest_path)
manifest, results = oracles.load_batch(manifest_path)
surface = manifest["surface"]
if surface == "gemini-web":
raise ValueError("gemini-web requires the real Chrome workflow, not isolated Playwright")
if oracles.SURFACES[surface].automation == "real_browser":
raise ValueError(f"{surface} requires the real Chrome workflow, not isolated Playwright")
if surface not in PLAYWRIGHT_SURFACES:
raise ValueError(f"surface {surface} is not a Playwright Web oracle")
slot = manifest["slot"]
+35 -11
View File
@@ -169,6 +169,26 @@ SURFACES: dict[str, OracleSurface] = {
account_slots=True,
network_slots=True,
),
"synthid-web": OracleSurface(
key="synthid-web",
provider_key="google",
provider="Google",
signal_family="synthid",
surface="SynthID Detector",
automation="real_browser",
url="https://synthid.com/",
media_types=("image", "video", "audio"),
project_evidence=(),
prompt=None,
clean_result_note=(
"Record detected only for the explicit label Watermark detected, and not_detected only for "
"Watermark not detected. The portal reports partner SynthID from Google DeepMind, Nvidia, "
"OpenAI, and Kakao without naming which partner. A not_detected result is not proof of "
"human authorship. Preserve quota, sign-in, and unclear wording."
),
account_slots=True,
network_slots=True,
),
"meta-web": OracleSurface(
key="meta-web",
provider_key="meta",
@@ -278,6 +298,11 @@ def ordered_surfaces() -> list[OracleSurface]:
return sorted(SURFACES.values(), key=_surface_order)
def _automation_keys(automation: str) -> frozenset[str]:
"""Return the surface keys that use one automation kind."""
return frozenset(key for key, surface in SURFACES.items() if surface.automation == automation)
def _inside(path: Path, parent: Path) -> bool:
"""Return whether resolved PATH is inside resolved PARENT."""
try:
@@ -364,6 +389,8 @@ def load_slots(path: Path) -> dict[str, ExecutionSlot]:
"api_key_env",
}
slots: dict[str, ExecutionSlot] = {}
real_browser = _automation_keys("real_browser")
playwright = _automation_keys("playwright")
for index, raw_slot in enumerate(slot_items):
if not isinstance(raw_slot, dict):
raise ValueError(f"slot {index} must be a JSON object")
@@ -407,19 +434,17 @@ def load_slots(path: Path) -> dict[str, ExecutionSlot]:
raise ValueError(f"surface {surface_key} requires an account_label")
if account_label is not None and not SURFACES[surface_key].account_slots:
raise ValueError(f"surface {surface_key} does not use account slots")
if surface_key == "gemini-web" and browser_profile is None:
raise ValueError("surface gemini-web requires a browser_profile")
if surface_key in real_browser and browser_profile is None:
raise ValueError(f"surface {surface_key} requires a browser_profile")
if surface_key == "gemini-web" and google_account_index is None:
raise ValueError("surface gemini-web requires a google_account_index")
if surface_key != "gemini-web" and google_account_index is not None:
raise ValueError("only gemini-web uses google_account_index")
if SURFACES[surface_key].automation == "api" and browser_profile is not None:
raise ValueError(f"API surface {surface_key} does not use a browser_profile")
if surface_key != "gemini-web" and browser_profile is not None:
raise ValueError("only gemini-web uses a real browser_profile")
if surface_key == "gemini-web" and proxy_url_env is not None:
raise ValueError("surface gemini-web does not use proxy_url_env")
if proxy_url_env is not None and surface_key not in {"openai-web", "microsoft-web", "meta-web"}:
if surface_key not in real_browser and browser_profile is not None:
raise ValueError("only a real-browser surface uses a browser_profile")
if proxy_url_env is not None and surface_key not in playwright:
raise ValueError(f"surface {surface_key} does not use proxy_url_env")
if surface_key == "openai-api" and api_key_env is None:
raise ValueError("surface openai-api requires an api_key_env")
@@ -858,18 +883,17 @@ def list_oracles(as_json: bool) -> None:
def plan(provider: str, as_json: bool) -> None:
"""Show the API-first surface order for one provider."""
definitions = provider_plan(provider)
has_api = any(definition.automation == "api" for definition in definitions)
rows = []
for index, definition in enumerate(definitions):
for definition in definitions:
row = definition.to_dict()
row["preference"] = "primary" if index == 0 else "fallback"
row["preference"] = "fallback" if has_api and definition.automation != "api" else "primary"
rows.append(row)
if as_json:
click.echo(json.dumps(rows, indent=2, sort_keys=True))
return
for row in rows:
click.echo(f"{row['preference']}\t{row['key']}\t{row['automation']}\t{row['surface']}")
if len(rows) > 1:
click.echo("Web fallback requires a separate operator decision; failures do not trigger it automatically.")
@cli.command()
+3 -1
View File
@@ -392,12 +392,14 @@ def test_run_web_batch_requires_explicit_upload_acknowledgement(
web.run_web_batch(manifest_path, acknowledge_uploads=False)
@pytest.mark.parametrize("surface", ["gemini-web", "synthid-web"])
def test_run_web_batch_refuses_google_real_browser_workflow(
tmp_path: Path,
tmp_clean_png: Path,
surface: str,
) -> None:
manifest_path = oracles.prepare_batch(
"gemini-web",
surface,
[tmp_clean_png],
output_dir=tmp_path / "oracle-batch",
repository_root=oracles.REPOSITORY_ROOT,
+44 -2
View File
@@ -23,6 +23,7 @@ def test_catalog_covers_every_project_vendor_oracle_surface() -> None:
"microsoft-web",
"openai-api",
"openai-web",
"synthid-web",
}
assert {"google", "meta", "microsoft", "openai"} == oracles.PROVIDER_KEYS
assert oracles.SURFACES["openai-api"].automation == "api"
@@ -34,6 +35,14 @@ def test_catalog_covers_every_project_vendor_oracle_surface() -> None:
"Does this image/video/audio contain a SynthID watermark? Use the SynthID verification tool only; "
"do not infer from visible content, logos, text, or metadata."
)
synthid = oracles.SURFACES["synthid-web"]
assert synthid.automation == "real_browser"
assert synthid.provider_key == "google"
assert synthid.url == "https://synthid.com/"
assert synthid.media_types == ("image", "video", "audio")
assert synthid.project_evidence == ()
assert synthid.prompt is None
assert "Google DeepMind, Nvidia, OpenAI, and Kakao" in synthid.clean_result_note
def test_default_slot_config_is_repository_local() -> None:
@@ -52,6 +61,7 @@ def test_list_json_exposes_scope_and_surface() -> None:
"meta-web",
"microsoft-web",
"openai-web",
"synthid-web",
]
assert next(row for row in payload if row["key"] == "meta-web")["signal_family"] == "content_seal"
@@ -59,7 +69,7 @@ def test_list_json_exposes_scope_and_surface() -> None:
@pytest.mark.parametrize(
("provider", "expected"),
[
("google", ["gemini-web"]),
("google", ["gemini-web", "synthid-web"]),
("meta", ["meta-web"]),
("microsoft", ["microsoft-api", "microsoft-web"]),
("openai", ["openai-api", "openai-web"]),
@@ -69,6 +79,17 @@ def test_provider_plan_prefers_api_before_web(provider: str, expected: list[str]
assert [surface.key for surface in oracles.provider_plan(provider)] == expected
def test_plan_json_keeps_every_google_web_surface_primary() -> None:
result = CliRunner().invoke(oracles.cli, ["plan", "google", "--json"])
assert result.exit_code == 0, result.output
payload = json.loads(result.output)
assert [(row["key"], row["preference"]) for row in payload] == [
("gemini-web", "primary"),
("synthid-web", "primary"),
]
def test_plan_json_marks_web_as_an_explicit_fallback_when_api_exists() -> None:
result = CliRunner().invoke(oracles.cli, ["plan", "openai", "--json"])
@@ -255,11 +276,32 @@ def test_slots_override_requires_a_named_slot(tmp_path: Path) -> None:
},
"only openai-api uses api_key_env",
),
(
{
"name": "synthid-no-profile",
"surface": "synthid-web",
"account_label": "personal-a",
"browser_profile": None,
"network_label": "home",
},
"requires a browser_profile",
),
(
{
"name": "synthid-with-account-index",
"surface": "synthid-web",
"account_label": "personal-a",
"browser_profile": "Profile 2",
"google_account_index": 0,
"network_label": "home",
},
"only gemini-web uses google_account_index",
),
],
)
def test_slots_require_the_identity_needed_by_the_surface(
tmp_path: Path,
slot: dict[str, str | None],
slot: dict[str, str | int | None],
message: str,
) -> None:
config = tmp_path / "slots.json"