Name classify provider classes by training scope. (#92)

openai, google, and tc260 stay provider classes. The Muse Image head
is public as muse-image, not meta. The freeze file still keys that
head meta_muse_image.
This commit is contained in:
Victor Kuznetsov authored and GitHub committed 2026-09-01 10:50:30 -07:00
1 parent 84ea8ede2e
commit 36c36212f0
9 files changed
+31 -22

No files matched your search

+1 -1
View File
@@ -84,7 +84,7 @@ remove-ai-watermarks classify image.png --json
The command runs the frozen photo detector (CLIP-L-ft ridge AND freeze MLP).
Only a DEFINITELY result is reported as `ai`. On that result only, the 124-d
provider head may return `openai`, `google`, `meta`, or `tc260`. POSSIBLY is
provider head may return `openai`, `google`, `muse-image`, or `tc260`. POSSIBLY is
`unknown`. Camera-like photographs are `human`. The call does not run cleanup
and does not set `is_ai_generated`.
+2 -1
View File
@@ -587,7 +587,8 @@ derived results influence detection.
[`classify.py`](../src/remove_ai_watermarks/classify.py) is the 2026-08-31
photo freeze: CLIP-L-ft ridge AND freeze MLP, then 124-d focal heads only on
DEFINITELY. The public label is `ai` / `human` / `unknown`. POSSIBLY is
`unknown`. Provider is `None` unless the label is `ai`. Tests in
`unknown`. Provider is `openai` / `google` / `muse-image` / `tc260` / `None`.
The freeze file keys the Muse Image head `meta_muse_image`. Tests in
`tests/test_classify.py` pin the gate without downloads and pin that
`identify` does not import this module.
+9 -7
View File
@@ -43,8 +43,8 @@ The training catalog is not in this Hub repository and is not on GitHub.
Use it when you have a photographic still and you want a pixel opinion after
metadata is gone:
- `label=ai` plus `provider=openai|google|meta|tc260` on a stripped ChatGPT,
Gemini, Meta Muse, or TC260 photograph
- `label=ai` plus `provider=openai|google|muse-image|tc260` on a stripped ChatGPT,
Gemini, Muse Image, or TC260 photograph
- `label=human` on camera photographs at a low false-positive rate
- `label=unknown` when the detector is only POSSIBLY AI, or when 124-d
features cannot be extracted
@@ -78,7 +78,7 @@ flowchart TD
m1 -->|likely_human| human[label human, provider none]
m1 -->|possibly| unk[label unknown, provider none]
m1 -->|definitely| m2[Model 2: 124-d one-vs-rest focal]
m2 -->|openai google meta tc260| named[label ai plus provider]
m2 -->|openai google muse-image tc260| named[label ai plus provider]
m2 -->|no_ai or extract fail| aiOnly[label ai, provider none]
```
@@ -89,8 +89,9 @@ remain open. The shipped heads do not include a receipt specialist.
Provider attribution tracks the **renderer**, not the front-end. Bing Image
Creator rows signed Microsoft, OpenAI score as `openai`. Designer rows signed
Microsoft, Google LLC score as `google`. Native Designer is mixed. There is
no Microsoft pixel class. Instagram `made_with_ai` is not a Meta pixel class.
Meta requires known-provenance Muse Image output.
no Microsoft pixel class. `openai`, `google`, and `tc260` are provider classes.
`muse-image` is Muse Image output, not a general Meta class. Instagram
`made_with_ai` is not that class.
## Architecture
@@ -115,8 +116,9 @@ Meta requires known-provenance Muse Image output.
(`openai`, `google`, `tc260`, `meta_muse_image`, `no_ai`)
- Decision: a class wins only if it beats `no_ai` by margin 0.30, then argmax
among those that passed
- Public name `meta` is the Muse Image class. `no_ai` is not reported as
`human`; it becomes `provider=None` on an `ai` label (for example FLUX)
- Public name `muse-image` is the Muse Image class. The freeze file still
keys that head `meta_muse_image`. `no_ai` is not reported as `human`; it
becomes `provider=None` on an `ai` label (for example FLUX)
## Training
+3 -3
View File
@@ -21,9 +21,9 @@
"model2": {
"feature_width": 124,
"margin": 0.3,
"classes": ["openai", "google", "tc260", "meta_muse_image", "no_ai"],
"public_names": {
"meta_muse_image": "meta"
"classes": ["openai", "google", "tc260", "muse-image", "no_ai"],
"checkpoint_keys": {
"muse-image": "meta_muse_image"
},
"runs_only_after": "definitely"
},
+2 -1
View File
@@ -78,7 +78,8 @@ construction and fits the same heads from the pack above.
Threshold is the 1.67% quantile on `detector_photo_dev_oi`.
2. AND that MLP with the frozen ridge in the probe file. DEFINITELY is both.
3. Load 124-d vectors. Fit one-vs-rest focal heads for openai, google, tc260,
meta_muse_image, no_ai. Margin 0.30.
meta_muse_image, no_ai. Margin 0.30. The public class for the Muse Image
head is `muse-image`.
4. Repeat with the same seeds. Compare bytes.
CLIP-L fine-tune of the last two vision blocks is earlier, seed 20260822, and
+5 -4
View File
@@ -54,7 +54,7 @@ flowchart TD
m1 -->|likely_human| human["label human, provider none"]
m1 -->|possibly| unk["label unknown, provider none"]
m1 -->|definitely| m2[Model 2 124-d focal heads]
m2 -->|openai google meta tc260| named["label ai plus provider"]
m2 -->|openai google muse-image tc260| named["label ai plus provider"]
m2 -->|no_ai or extract fail| aiOnly["label ai, provider none"]
```
@@ -63,7 +63,7 @@ flowchart TD
| `label` | `ai`, `human`, `unknown` | Public verdict. `ai` only on DEFINITELY |
| `domain` | `photo` | This freeze is photographic only |
| `detector` | `definitely`, `possibly`, `likely_human` | Raw Model 1 gate |
| `provider` | `openai`, `google`, `meta`, `tc260`, or `None` | Model 2, only if `label` is `ai` |
| `provider` | `openai`, `google`, `muse-image`, `tc260`, or `None` | Model 2, only if `label` is `ai` |
`human` is camera-like under this contract, not a proof of authorship.
`unknown` is POSSIBLY, or a domain this freeze does not support. `provider=None`
@@ -84,8 +84,9 @@ wrong job on that bank, which is why Model 2 is gated.
Provider names the renderer, not the product UI. Bing Image Creator signed
Microsoft, OpenAI scores `openai`. Designer signed Microsoft, Google LLC
scores `google`. There is no Microsoft pixel class. Meta is Muse Image, not
an Instagram `made_with_ai` tag.
scores `google`. There is no Microsoft pixel class. `openai`, `google`, and
`tc260` are provider classes. `muse-image` is Muse Image output, not a
general Meta class and not an Instagram `made_with_ai` tag.
## Evaluation
+1 -1
View File
@@ -160,7 +160,7 @@ print(result.label, result.detector, result.provider)
`label` is `ai` only on a DEFINITELY detector result (ridge AND freeze MLP).
POSSIBLY is `unknown`. Camera-like photographs are `human`. `provider` is
`openai`, `google`, `meta`, or `tc260` only when `label` is `ai` and the 124-d
`openai`, `google`, `muse-image`, or `tc260` only when `label` is `ai` and the 124-d
head beats `no_ai` by the freeze margin. Otherwise it is `None`, including
when 124-d extraction refuses the file.
+5 -3
View File
@@ -35,17 +35,19 @@ MLP_THRESHOLD = 5.9586493237495395
RIDGE_THRESHOLD = 0.3056212276800537
PROVIDER_MARGIN = 0.30
CLIP_WIDTH = 768
# Checkpoint keys in provider.pt. openai/google/tc260 are provider classes;
# meta_muse_image is Muse Image output. Public names follow that split.
PROVIDER_LABELS = ("openai", "google", "tc260", "meta_muse_image", "no_ai")
DetectorLevel = Literal["definitely", "possibly", "likely_human"]
PixelLabel = Literal["ai", "human", "unknown"]
PixelDomain = Literal["photo"]
PixelProvider = Literal["openai", "google", "meta", "tc260"]
PixelProvider = Literal["openai", "google", "muse-image", "tc260"]
PUBLIC_PROVIDER: dict[str, PixelProvider] = {
"openai": "openai",
"google": "google",
"tc260": "tc260",
"meta_muse_image": "meta",
"meta_muse_image": "muse-image",
}
_runtime: _Runtime | None = None
@@ -104,7 +106,7 @@ def label_for(level: DetectorLevel) -> PixelLabel:
def provider_from_scores(scores: dict[str, float], *, margin: float = PROVIDER_MARGIN) -> PixelProvider | None:
"""Argmax among openai/google/tc260/meta that beat ``no_ai`` by ``margin``."""
"""Argmax among openai/google/tc260/muse-image that beat ``no_ai`` by ``margin``."""
no_ai = scores["no_ai"]
ai_names = [name for name in PROVIDER_LABELS if name != "no_ai"]
passed = [name for name in ai_names if scores[name] > no_ai + margin]
+3 -1
View File
@@ -55,6 +55,8 @@ def test_shipped_operating_point_matches_the_runtime_defaults() -> None:
assert payload["model1"]["ridge_threshold"] == RIDGE_THRESHOLD
assert payload["model2"]["margin"] == PROVIDER_MARGIN
assert payload["model2"]["runs_only_after"] == "definitely"
assert payload["model2"]["classes"] == ["openai", "google", "tc260", "muse-image", "no_ai"]
assert payload["model2"]["checkpoint_keys"]["muse-image"] == "meta_muse_image"
def test_definitely_is_the_and_of_ridge_and_mlp() -> None:
@@ -74,7 +76,7 @@ def test_provider_requires_margin_over_no_ai() -> None:
assert provider_from_scores(_scores(openai=1.0, no_ai=0.5)) == "openai"
assert provider_from_scores(_scores(openai=0.5, no_ai=0.4)) is None
assert provider_from_scores(_scores(google=2.0, openai=1.5, no_ai=0.0)) == "google"
assert provider_from_scores(_scores(meta=1.0, no_ai=0.0)) == "meta"
assert provider_from_scores(_scores(meta=1.0, no_ai=0.0)) == "muse-image"
assert provider_from_scores(_scores(tc260=1.0, no_ai=0.0)) == "tc260"